Joint effects of illumination geometry and object shape in the perception of surface reflectance

Abstract

Surface properties provide useful information for identifying objects and interacting with them. Effective utilization of this information, however, requires that the perception of object surface properties be relatively constant across changes in illumination and changes in object shape. Such constancy has been studied separately for changes in these factors. Here we ask whether the separate study of the illumination and shape effects is sufficient, by testing whether joint effects of illumination and shape changes can be predicted from the individual effects in a straightforward manner. We found large interactions between illumination and object shape in their effects on perceived glossiness. In addition, analysis of luminance histogram statistics could not account for the interactions.

1. Introduction

Visual perception of object surface reflectance properties can aid object identification and guide tactile interaction with objects (Adelson 2001). For example, how glossy an object appears provides information about how tightly it should be gripped to avoid slippage. The percepts are useful, however, only to the extent that they are stable across variation in illumination and object shape. Because the light reflected from an object varies with these object-extrinsic factors, stable perception of surface properties requires visual processing of the raw retinal image (see Figure 1). A fundamental question is how and to what extent such stabilization is accomplished by the visual system.

Figure 1.

This scene contains objects with different surface geometries, shapes, and surface reflectance properties viewed under spatially varying illumination. The identification of surface material—porcelain, fabric, wood, glass—is important for object identification and action planning but is complicated by the fact that the signal coming to the eye depends not only on surface reflectance but also on the illumination and object shape. The insets on the right show close-ups of different parts in the image, with the numbers next to the insets corresponding to the numbers in the picture. Note, for example, the variation in the image corresponding to different locations on the vase (locations 1 and 2) and the similarity between a location on the vase (location 2) and on the wooden window sill (location 3). The photograph is courtesy of Toni Saarela.

The geometric aspects of a surface's reflectance can be approximated by its bidirectional reflectance distribution function (BRDF), which describes the amount of light reflected as a function of the directions of the incident and reflected light (Nicodemus et al 1977). Moreover, the BRDFs of many common surfaces can be described by simple parametric models. Here we will employ the isotropic Ward BRDF model (Ward 1992). At each wavelength, this model specifies reflectance as the sum of a non-directional diffuse component (ρ_d) and a directional specular component. The strength of the specular component is described by the parameter ρ_s and its spatial spread by a roughness parameter α. We assume ρ_s and α are constant as a function of wavelength, while ρ_d has a spectral dependence. Roughly speaking, the perceived color and lightness of an object correlate with the diffuse reflectance component, while the perceived glossiness correlates with the specular component.

The constancy of surface color and lightness perception for flat matte objects and simple illumination geometries has been extensively studied and is now fairly well understood (see reviews on lightness by Adelson (2000) and Kingdom (2010) and on color by Shevell and Kingdom (2008) and Brainard and Maloney (2011). Perception of color and lightness for non-matte and three-dimensional objects has historically received less attention but is being actively studied, perhaps because of the availability of computer graphic techniques for stimulus generation and display (see Maloney and Brainard 2010). Similarly, the study of the perceptual correlates of geometric aspects of object reflectance (aka material properties) is also of much current interest (again, see Maloney and Brainard 2010).

It is clear that human observers can judge the glossiness of three-dimensional objects and that this percept varies with the specular component of object reflectance (eg, Beck and Prazdny 1981, Blake and Bülthoff 1990, Pellacini et al 2000; Fleming et al 2003; Obein et al 2004; Ji et al 2006). Which aspects of the image drive this percept are less clear. Some authors have emphasized the causal role of low-level image cues derived from the luminance histogram of the object's image (Nishida and Shinya 1998; Motoyoshi et al 2007; Sharan et al 2008), while others have noted the importance of the relationship between luminance variation and perceived object shape (Beck and Prazdny 1981; Blake and Bülthoff 1990; Berzhanskaya et al 2005; Anderson and Kim 2009; Kim and Anderson 2010; Kim et al 2011; Marlow et al 2011). There have also been some initial studies of the stability of perceived glossiness under variations in scene illumination (Fleming et al 2003; te Pas and Pont 2005; Pont and te Pas 2006; Doerschner, Boyaci et al 2010; Olkkonen and Brainard 2010), variations in surface geometry (Nishida and Shinya 1998; Ho et al 2008; Wendt et al 2010; Wijntjes and Pont 2010), and variations in object shape (Vangorp et al 2007). Taken as a whole these studies indicate that the visual system does stabilize the percept to some degree, but by no means perfectly. It is not well-understood how the stabilization is accomplished, nor when it works well and when it fails.

Human observers can also judge the lightness of three-dimensional objects even when they are viewed in isolation (Beck 1964; Sharan et al 2008). This is possible because of the shadows and interreflections caused by the three-dimensional surface structure, which provide cues to surface reflectance. In addition, Motoyoshi et al (2007) and Sharan et al (2008) found that perceived lightness was relatively stable when the specular component of surface reflectance was varied (see also Xiao and Brainard 2008). There were some deviations from constancy, however, and these authors proposed a theory that based perceived lightness on luminance histogram statistics to account for their results. Data from other authors, however, suggest that such simple theories do not generalize well to a broader class of stimulus conditions (Anderson and Kim 2009; see also Beck 1964; Todd et al 2004). The discrimination of diffuse reflectance also depends on illumination geometry and object shape (Khang et al 2003), and it is better under some illuminants than others. As with perceived glossiness, there have not been studies that covary both illumination geometry and object shape.

The fact that perceived lightness and glossiness vary with both illumination geometry and object shape raises a fundamental question: Do these effects interact? This question is important, since the answer will provide guidance about the generality of conclusions that may be drawn from studies that vary object shape under a fixed illumination geometry (eg, Nishida and Shinya 1998; Vangorp et al 2007; Ho et al 2008; Wendt et al 2010) and studies that vary illumination geometry for a fixed object shape (eg, Fleming et al 2003; Pont and te Pas 2006; Doerschner, Boyaci et al 2010; Olkkonen and Brainard 2010). Indeed, if these two factors interact, understanding the perception of glossiness and lightness for natural images will either require a large database of factorial measurements or a theory of the interaction that would enable prediction of the joint effects from a smaller set of measurements. Since making a full set of factorial measurements is not tractable, we believe that exploring the question of interactions is a priority.

Here we report experiments that study the perception of both glossiness and lightness for three-dimensional objects and concentrate on the stability of these percepts across joint variations in illumination geometry and object shape. To minimize artifacts that might result from the limited dynamic range of conventional computer displays (Phillips et al 2009; Doerschner, Boyaci et al 2010), we used a custom high-dynamic range display so that our stimuli incorporated the large variations in luminance that can be generated by specular reflection under directional illumination. We tested whether the data for joint manipulations of shape and illumination geometry could be predicted by models that treat the effects of each variable as separable and found profound interactions for judgments of glossiness. We asked whether the interactions could be accounted for by image-based models that have been suggested by others but found that these also failed. Our data thus provide a challenge for future modeling efforts.

2. Methods

The purpose of the experiments was to measure how changing both illumination geometry and object shape affect perceived glossiness and lightness. On each trial of the experiment, observers judged which of two objects appeared glossier or lighter (depending on the trial type), and a staircase procedure was used to adjust the reflectance properties of one of the objects to seek the point of subjective equality. The objects could either be the same shape illuminated by different light fields, different shapes illuminated by the same light field, or different shapes illuminated by different light fields.

2.1. Observers

Four naive observers participated in the experiment. All four ran conditions with the Grove/Kitchen light field pair (see below for explanation of the conditions). Three of the four observers also ran conditions with the Grove/Galileo light field pair. All participants had normal visual acuity (20/20) as verified with a Snellen chart and normal color vision as verified with the Ishihara color plates. The experimental protocols were approved by the human subjects institutional review board (IRB) at the University of Pennsylvania and were in accordance with the World Medical Association Helsinki Declaration as revised in October 2008 (http://www.wma.net/en/30publications/10policies/b3/index.html).

2.2. Stimuli

We generated two shapes using the Maya 3D modeling software (Autodesk, Inc.) by modulating a sphere with three sinusoids of different frequencies, phases, and amplitudes (see Figure 2). For mnemonic convenience, we refer to the shapes as Blob and Pepper. Wireframe models of the shapes were exported from Maya and rendered with specified reflectance parameters under three real-world light fields captured by Debevec (1998). The rendering was performed using Radiance (Ward 1994) via the RenderToolbox (www.rendertoolbox.org) Matlab routines. The RenderToolbox provides wrappers that allow wavelength-by-wavelength specification of surface reflectance and illuminant spectral power.

Figure 2.

The reference stimulus set. The two levels of reference diffuse reflectance (ρ_d) are shown in the two major columns and the two levels of reference specular reflectance (ρ_s) are shown in rows. The three column images show the same stimulus rendered under three different light fields (Grove, Kitchen, and Galileo). The two shapes are shown in the two major rows. The checkered backgrounds have been cropped relative to their size in the experiment. The actual stimuli were high-dynamic range images. To render the images for publication at a low dynamic range, they were transformed from the LMS cone representation to the sRGB standard primaries, gamma corrected by taking the square root of each value, and scaled by eye with high luminance values clipped, so as to preserve as well as possible the relative appearances they had when displayed at high dynamic range. The same procedures were used to produce the other stimulus images shown in the paper.

The diffuse reflectance of the object surface was that of surface 14 in the GretagMacbeth color chart (Munsell notation 0.25G 5.4/8.65). To render the scenes spectrally within the RenderToolbox environment, the RGB light fields provided by Debevec were converted to a spectral representation. The procedure used is described in Olkkonen and Brainard (2010). The light fields were scaled so that the mean luminance reflected from a matte Blob with the diffuse reflectance of Munsell 0.25G 5.4/8.65 was the same for each light field. We chose a target mean luminance for the matte images of 4 cd/m², which was a compromise between obtaining a reasonably high mean luminance and minimizing the amount of tone mapping required to render the stimuli. The light fields varied in their mean chromaticities, a feature that was preserved in the experimental stimuli. We refer to the light fields as Kitchen, Grove, and Galileo, based on the locations where they were captured (a kitchen, a eucalyptus grove, and Galileo's tomb).

The objects varied in their geometric reflectance properties. As described in the introduction, we used the isotropic Ward (1992) BRDF model and specified geometric reflectance using its specular (ρ_s), diffuse (ρ_d), and roughness (α) parameters. A value of ρ_d = 1 corresponded to a diffuse spectral reflectance function equal to that of Munsell 0.25G 5.4/8.65, and other values scaled this reflectance function equally at all wavelengths. On any given trial, two stimuli were shown, a reference stimulus, which could take on any of four reflectances, and a comparison stimulus, whose reflectance was determined by an adaptive staircase algorithm (see Procedure). We chose two levels of reference specular reflectance (ρ_s = [0.04 0.16]) and two levels of diffuse reflectance (ρ_d = [.4.9]). The roughness parameter was kept constant at Ward α = 0.01.

The rendering procedure resulted in a hyperspectral image with 31 planes, with each plane corresponding to one wavelength band. Sample wavelengths were between 400 and 700 nm at 10 nm steps. The hyperspectral image was converted to an LMS cone image using the Stockman-Sharpe two-degree cone fundamentals (Stockman and Sharpe 2000). Where required, tone mapping was accomplished by setting pixels with luminances above the maximum displayable value (460 cd/m²) to that value while preserving their chromaticities.

During the experiment, we needed to vary the reflectance properties of the comparison stimuli. To do so rapidly, we adjusted and combined two pre-rendered basis images (Griffin 1999; Xiao and Brainard 2008; Olkkonen and Brainard 2010). One basis image contained a matte stimulus and one a glossy stimulus. By taking the difference between the glossy and matte basis images, we could extract a difference image that represented the effect of adding a specular component to the stimulus' reflectance. To generate stimuli with different levels of specularity, we combined the matte basis image with the difference image and varied the weight on the difference image. To simulate different levels of diffuse reflectance, we scaled the matte basis image prior to combination with the scaled difference image.

The stimuli were embedded in a checkerboard background, where the chromaticities and luminances of single checks were drawn from a three-dimensional Gaussian distribution with mean x = 0.34, y = 0.35, Y = 7.1 and standard deviation x = 0.014, y = 0.011, Y = 2.02 (Judd-Vos xyY). The values approximate the mean and standard deviation xyY values averaged across the bit maps of the three light fields used in the experiments.

Table 1 lists the CIE chromaticities and luminances of the reference stimuli, obtained by taking the spatial mean over each object.

Table 1. The Judd-Vos corrected CIE chromaticities and luminances of the reference stimuli, averaged over all stimulus pixels.

Reflectance	Shape	Grove	Kitchen	Galileo
ρd: 0.4 ρs: 0.04	Blob Pepper	0.30 0.43 3.5 0.30 0.42 4.2	0.30 0.41 3.7 0.30 0.43 2.7	0.34 0.43 3.6 0.34 0.42 3.8
ρd: 0.4 ρs: 0.16	Blob Pepper	0.31 0.40 4.4 0.30 0.38 6.1	0.30 0.36 5.1 0.30 0.41 2.3	0.35 0.40 4.9 0.34 0.38 5.9
ρd: 0.9 ρs: 0.04	Blob Pepper	0.30 0.44 7.5 0.30 0.43 8.7	0.3 0.43 7.7 0.30 0.43 6.2	0.30 0.44 7.6 0.34 0.44 7.7
ρd: 0.9 ρs: 0.16	Blob Pepper	0.31 0.42 8.4 0.30 0.40 10.6	0.30 0.39 9.1 0.30 0.43 5.8	0.34 0.42 8.9 0.34 0.41 9.7

2.3. Apparatus

The stimuli were displayed on a custom high-dynamic-range (HDR) display. Briefly, the display consisted of an LCD flat-panel display with the backlight replaced by a three-chip DLP video-projector that illuminated the LCD screen with a projected image. By controlling the input setting to both LCD screen and DLP projector, this arrangement allows large (> 10,000:1) modulation of the image intensity at each pixel. Both display devices were driven at a pixel resolution of 1280 by 1024 and at a refresh rate of 60 Hz through a dual-port video card (NVIDIA GeForce GT 120). The host computer was an Apple Macintosh with a quad-core Intel Xeon processor. The display is described in detail in an earlier report (Olkkonen and Brainard 2010). Since publication of that work, some improvements in the calibration and display control software were implemented. These are described in a technical report (Radonjić et al 2011). The maximum luminance of the display was 460 cd/m². The minimum luminance was below the measurement range of our radiometer, but at least a factor of 10,000 below the maximum luminance.

The experiment was programmed in Matlab (MathWorks, Inc) with display control functions based on the MGL toolbox (http://justingardner.net/mgl) and the Psychtoolbox (Brainard 1997; Pelli 1997). We used the Jacket software (http://www.accelereyes.com/) to offload some computations to a second GeForce GT 120 video card, which enabled us to perform rapidly the calculations needed to render the stimuli on each trial.

2.4. Procedure

On each trial of the experiment, two stimuli were presented side by side on the display. The checkerboard backgrounds subtended 9.3 degrees of visual angle on each side, with the shapes subtending approximately 6 × 6 degrees. The distance between the centers of the reference and comparison shapes was 9.3 degrees. The reflectance of the reference stimulus remained the same throughout one block. The left-right locations of the reference and comparison were randomized on each trial. The observer's task was to respond which stimulus appeared glossier/lighter depending on the trial. In 12 consecutive trials, the observer gave glossiness estimates; in the following 12 trials, lightness estimates, and so on. The observer was alerted when the judgment changed via synthesized speech emitted from the computer. In addition, the current judgment was indicated by white text at the top of the display. The exact instructions given to the observers are provided in the supplementary materials available at http://color.psych.upenn.edu/supplements/materialshape/.

For any given reference stimulus, there were four interleaved staircases: two for lightness judgments and two for the glossiness judgments. For each staircase, the starting reflectance parameters of the comparison were picked either below or above the reference values (two in each direction), and were consequently changed by a 1-up, 1-down procedure. In a given block, the reflectance of the reference stimulus was kept constant, and there were two reference shapes and two reference light fields. As each staircase had 15 trials, there were 240 trials in total in one block. Each block was repeated twice to get two estimates for each data point.

There were three blocked change conditions whose order was counterbalanced across observers. In one condition, the two stimuli in a given trial had the same shape, but differed in illumination geometry (change light field). This condition measured the effect of illumination geometry on perceived reflectance. In another condition, the two stimuli within a trial had different shapes, but the same illumination geometry (change shape). Here, the effect of object shape on perceived reflectance was measured. In the third condition, the stimuli had both different shapes and different illumination geometries (change both). This condition measured the joint effect of changing illumination and object shape on perceived reflectance.

The experiment was run for two light field comparisons: Grove/Kitchen and Grove/Galileo. The experiment was completed first for one comparison, after which the other comparison was run. The order of the two comparisons was counterbalanced across observers. Note that the shape change trials were the same for Grove in both comparisons and were run only once for one of the observers to minimize the length of the whole experiment. For the other observers the conditions were run in full.

2.5. Data analysis

We quantified perceived glossiness and lightness as follows. For a given reference stimulus, change condition, and judgment type, we averaged the reversals in each staircase after discarding the first two reversals. We then averaged over staircases to obtain the point of subjective equality (PSE). In 4% of the cases, the perceptual effects were so large that there were no reversals before the comparison reflectance parameters led to stimuli that reached the luminance limit of the display, in which case we took this most extreme possible stimulus value as an approximation of the PSE. The display limits are indicated in the data figures, so that such points may be easily identified.

We further summarized the data for each change condition and judgment type with linear regression lines fit to the data for all reference stimuli for that condition/judgment (Olkkonen and Brainard 2010). We assumed the offset of all lines to be zero and used just the slope parameter for the fits. The slopes were taken as a measure of the perceptual effect in each condition. For each judgment and change condition, we obtained four slopes. For example, for the glossiness judgments in the light field change condition there were two shapes and two levels of reference diffuse reflectance leading to four regression lines.

3. Results

3.1. Effects of illumination geometry

We found large effects of illumination geometry on perceived glossiness, confirming our previous findings for spheres. This is shown for a representative observer in Figure 3a. Each data point shows the specularity values for a pair of stimuli that matched across the Grove/Kitchen light field change; data for the Blob (circles) and Pepper (squares) are shown separately. The deviations of the data from the unity diagonal indicate the perceptual effect of the light change.

Figure 3.

Effect of light field on perceived reflectance, Grove/Kitchen. (a) Points of subjective equality, glossiness judgments, observer OCL. Each data point represents one PSE and shows the specularity value for the stimulus under the Grove light field on the x-axis and under the Kitchen light field on the y-axis. Circular symbols show data for the Blob, and square symbols show data for the Pepper. Black symbols denote data for stimuli with low diffuse reflectance; pink symbols denote stimuli with high diffuse reflectance. Error bars show the standard error of the mean over two repetitions of the condition. Symbols with horizontal error bars indicate pairs where the stimulus under Grove (x axis) was the comparison and the stimulus under Kitchen (y axis) was the reference. Symbols with the vertical error bars indicate the opposite case. The diagonal dashed line indicates unity; the horizontal and vertical dashed lines indicate the reflectance parameter space of the display. (b) Points of subjective equality, lightness judgments, observer OCL. Plotting conventions are as in (a) with the exception that here the black and pink symbols indicate stimuli with low and high specular reflectance, respectively. Corresponding plots for all observers for both light field changes are provided in the supplementary materials (Figures S1–S7).

Figure 4a summarizes the effects with the regression slopes for all of the observers. Here, the deviation of slopes from unity indicates the perceptual effect. The data are generally similar across the two levels of diffuse reflectance, indicating that to first order it is sufficient to characterize the effect of illumination geometry on perceived glossiness by studying one level of reference stimulus diffuse reflectance (see, also, Olkkonen and Brainard 2010). Note that the effect of light field is quite different for the two shapes, and for some observers (GLA, OCL) the effect is in the opposite direction relative to the unity line.

Figure 4.

Effect of light field on perceived reflectance, Grove/Kitchen. (a) Each panel shows the slopes of regression lines fitted to PSE data for the glossiness judgments for one observer. Each set of bars in a panel shows the slopes for one shape. Slopes for low and high diffuse reflectance are shown in black and pink, respectively. Error bars show the 67% confidence intervals of the regression line fit. (b) Regression slopes of the lightness judgments. Plotting conventions are as in (a) with the exception that here the black and pink symbols indicate stimuli with low and high specular reflectance, respectively.

Data for the Grove/Galileo light field change are plotted in the same format as Figure 4a in the supplementary materials (Figure S8a). The effects for that light field change were variable across observers, but in general the light field again affected perceived glossiness. For one observer, there was a clear effect of shape on the effect of the light field.

Figure 3b shows the effect of illumination geometry on perceived lightness for observer OCL. All data points lie above the diagonal, showing that for both shapes and both levels of specular reflectance the stimuli under the Kitchen light field had to have slightly higher diffuse reflectance to perceptually match the stimuli under the Grove light field. Notably, though, the effect of light field on perceived lightness was relatively small compared with the effect on perceived glossiness, and similar for both shapes and both levels of specular reflectance. The smaller effects may arise from the fact that our manipulations of illumination geometry were performed under conditions where the mean illuminant intensity was approximately equated (see Methods).

Figure 4b summarizes the effects on lightness for all observers. Note that the scale on the y-axis is expanded relative to Figure 4a. Again, the data were similar for all observers. There was a small effect of illumination geometry on perceived lightness, but no interaction between light field and shape. Also, the effects did not depend strongly on stimulus specular reflectance, although there is a tendency for a larger effect for the higher level of specular reflectance. The data for the Grove/Galileo light field change are plotted in the same format as Figure 4b in the supplementary materials (Figure S8b). As with the glossiness judgments, there was more observer variability for this light field change. Two observers, however, showed clear effects of the light field on perceived lightness.

3.2. Effects of shape

We found a large effect of shape on perceived glossiness when illumination geometry was kept constant. This is illustrated in Figure 5a for observer OCL. The data are shown in the same format as in Figure 3, but here the axes are the specularities for the two shapes instead of for the two light fields. The data fall below or above the diagonal depending on the light field, in other words, the effect of shape was opposite for the two light fields.

Figure 5.

Effect of shape on perceived reflectance, Grove/Kitchen. (a) Points of equal perceived glossiness across a change in shape for observer OCL. A given data point shows a pair of specularity values for two shapes that match in perceived glossiness. Diamonds show data for the Grove light field, and stars show data for the Kitchen light field. Black symbols denote data for stimuli with low diffuse reflectance; light red symbols denote stimuli with high diffuse reflectance. The error bars show the standard error of the mean over two repetitions. Symbols with horizontal error bars indicate pairs where the Blob (x-axis) was the comparison and the Pepper (y-axis) was the reference. Symbols with vertical error bars indicate the opposite case. The diagonal dashed line indicates unity; the horizontal and vertical dashed lines indicate the reflectance parameter space of the display. (b) Points of equal perceived lightness for the shape comparison. Plotting conventions are as in (a) with the exception that here the black and pink symbols indicate stimuli with low and high specular reflectance, respectively. Corresponding plots for all observers for both light field changes are provided in the supplementary materials (Figures S1–S7).

Figure 6a shows the slopes for glossiness estimates across a shape change for all observers. The two sets of bars in each panel show data for the two light fields. As in the light field comparison, there was an interaction between the effect of light field and that of shape for all observers. In addition, for all observers except for VHY the results were similar for the two levels of diffuse reflectance.

Figure 6.

Effect of shape on perceived reflectance, Grove/Kitchen. (a) Each panel shows the slopes of regression lines fitted to PSE data for the glossiness judgments for one observer. The two sets of bars in a panel show the slopes for the two light fields. Slopes for low and high diffuse reflectance are shown in black and pink, respectively. Error bars show the 67% confidence intervals of the line fit. (b) Slopes of linear regression lines fitted to the lightness PSE data for each observer. Plotting conventions are as in (a), with the exception that here the black and pink symbols indicate stimuli with low and high specular reflectance, respectively.

The effects of shape on perceived lightness, shown in Figure 5b for OCL and summarized in Figure 6b for all observers, were small, and there was not a large interaction between shape and light field.

Data for the Grove/Galileo light field change are shown in the supplementary materials (Figure S9). There was again a large effect of shape on perceived glossiness, with an interaction shown by one of three observers. The effects on lightness were small relative to the effects on glossiness.

3.3. Joint effects

The results from the change light field and change shape conditions show a strong interaction between object shape and illumination geometry for glossiness judgments. To probe this interaction further, we ran a third condition where observers made glossiness and lightness judgments across joint changes in light field and shape. That is, we directly measured matches across changes in both illuminant and shape (eg, Blob/Grove and Pepper/Kitchen).

Slopes of the linear regression fits for the Grove/Kitchen comparison are shown in Figure 7. The glossiness estimates were more variable across observers than in the single change conditions. The slopes for the two levels of diffuse reflectance were similar for three observers, while the fourth observer (CDI) showed a clear difference between the two levels of diffuse reflectance. The data for the lightness estimates were less variable across observers and were qualitatively, if not quantitatively, similar for the two levels of specular reflectance. Data for the Grove/Galileo light field change are shown in the supplementary materials (Figure S10), with large effects shown for perceived glossiness and smaller, less consistent effects for perceived lightness.

Figure 7.

Effect of a joint light field and shape change on perceived reflectance, Grove/Kitchen. (a) Each panel shows the slopes of regression lines fitted to the glossiness PSE data for each observer. Each set of bars shows the slopes for one shape/light field combination. Slopes for low and high diffuse reflectance are shown in black and pink, respectively. Error bars show the 67% confidence intervals of the line fit. (b) Slopes of linear regression lines fitted to the lightness PSE data for each observer. Plotting conventions are as in (a) with the exception that here the black and pink symbols indicate stimuli with low and high specular reflectance, respectively.

Our first analysis of the joint measurements checked the consistency of the measurement procedure by asking whether the measurements satisfied transitivity (see Doerschner, Boyaci et al 2010). What we mean by this may be understood with reference to Figure 8a. The measurements for shape and illuminant changes alone are characterized by slopes. We denote these effects by the red arrows in the figure. For example, the arrow labeled A indicates the effect of changing the illuminant from Grove to Kitchen for the Blob shape. Since each of the effects we measure are reasonably described by a multiplicative scaling (slope of the fit regression lines), the slope obtained for a joint change (indicated in the figure by the dashed blue arrow for Blob/Grove to Pepper/Kitchen) should be predicted by the product of the slopes obtained for the component changes. For the case illustrated by the figure, this product may be obtained from slopes for the two measurements indicated by the red arrows. We compared predictions made in this way against the joint measurement for shape and illuminant changes.

Figure 8.

(a) The transitivity of the effects of light field and shape on perceived reflectance can be understood as follows. The arrow labeled A shows changes from the Grove light field to the Kitchen light field for the Blob shape, whereas the arrow labeled B shows changes from the Blob to the Pepper for the Kitchen light field. If transitivity holds, the effect of changing from Grove/Blob to Kitchen/Pepper (arrow E) should be the combination of the two separate effects (arrow A x arrow B). Note that a second transitivity prediction for arrow E may also be obtained by multiplying the slope obtained for the change from Blob to Pepper under the Grove light field and with the slope for the change from Grove to Kitchen for the Pepper. (b) If separability holds, the joint change of light field and shape (arrow E) should be a combination of changing the light field (arrow A) and changing the shape (arrow C) for the same reference stimulus (arrow A x arrow C). Note that a second separability prediction for arrow E may be obtained by multiplying the slope for the change for Blob to Pepper under the Kitchen light field with the slope for the change from Grove to Kitchen for the Pepper.

Figure 9 summarizes the results of the transitivity check for glossiness (left panel) and lightness (right panel). For most of the measurements, transitivity held well. In the figure, the solid points are those where the prediction error for the slope of the joint measurement obtained from both pairs of individual measurements was less than 0.8 for both glossiness and lightness. This criterion was chosen by eye. For the glossiness measurements, this criterion was met for 47 out of 56 cases (84%, solid points). For lightness, all of the cases satisfied the criterion. As we viewed transitivity of the data as a prerequisite for asking questions about how well candidate models might predict joint effects, we restricted our subsequent analysis of the joint effects to conditions corresponding to the solid points in Figure 9a.

Figure 9.

(a) Transitivity analysis for glossiness (left) and lightness (right). The x-axis in each panel shows the slopes fitted to the data in the joint change condition. Each data point is one slope fitted to the data of one observer for a given level of reference diffuse (glossiness estimates) or specular (lightness estimates) reflectance. There were two separate predictions for each measurement, corresponding to the two independent ways that transitivity predictions could be made (see caption of Figure 9). Black symbols show the data for the Grove/Kitchen comparison, and green symbols show the data for the Grove/Galileo comparison. The data points adhering to our criterion for transitivity are shown with filled symbols, and the outliers are shown with open symbols (see text for details). The diagonal dashed line shows unity. The solid lines denote slopes of 1 on each axis, and the quadrants along the negative diagonal denote regions where the predictions and data go in different directions. Note that the two panels have different scales. (b) Separability analysis. The x-axes are the same as in (a). The y-axes show slopes predicted from a separability model (see text). As in (a), there were two independent predictions for each point on the x-axis. Predictions are plotted only for measurements satisfying transitivity (solid points in (a)). Other plotting conventions are as in (a).

Since both illumination geometry and shape affect lightness and glossiness, understanding these phenomena requires at minimum measurements of the effect of each variable when the other is held fixed. If the effects of the two variables do not interact, the joint effects can be characterized by measurements of each alone at one fixed level of the other. This idea is illustrated by Figure 8b. Here the question becomes whether the joint effect illustrated by the dashed blue line (E) is predicted by the product of the effects of the two red arrows, A and C. Note that when separability holds, the number of measurements required to jointly characterize effects of light field and shape increases linearly with the number of light fields and shapes studied, while in the general case, the number increases as the product of the two conditions. Thus if the data satisfy separability, the joint measurement enterprise is greatly simplified.

Separability does not hold in general for the glossiness estimates for either of the light field conditions (Grove/Kitchen and Grove/Galileo), as shown by Figure 9b (left panel; only predictions that met the transitivity criterion are plotted). The large failures shown are a consequence of the interactions shown in Figures 4 and 6 above. The analysis here confirms that such interactions are typical of the entire dataset.

Interestingly, the predictions for glossiness span a much wider range of values than the data. This indicates that for the stimuli employed here independence of the light field and shape effects would make the stimulus pairs in the joint condition to appear much more different (shown with slopes deviating widely from unity) than how the observers perceived them.

Separability holds better for the lightness estimates (Figure 9b, right panel), and indeed the quality of the separability predictions is similar to that of the transitivity predictions.

3.4. Image statistics

Our results show that both illumination geometry and object shape can have a large effect on perceived glossiness. In addition, the effect of a combined light field and shape change on glossiness cannot be easily predicted from the individual effects. Scrutinizing the stimulus images in Figure 8 provides some intuition to why this is so: It is hard to describe the result of combining the two changes as an addition or a product of the changes seen for each manipulation alone.

Some authors have suggested that simple luminance histogram statistics predict perceived reflectance and in particular perceived glossiness (Nishida and Shinya 1998; Motoyoshi et al 2007; Sharan et al 2008). Anderson and Kim (2009) have pointed out limitations of such models. In addition, models based on luminance histogram statistics did not work well in our hands when we tried to use them to account for effects of illumination geometry on the perceived glossiness of spheres (Olkkonen and Brainard 2010). Nonetheless, it remains of topical interest to understand the role of luminance histogram statistics, particularly for the perception of glossiness, and we thus asked how well our current data could be accounted for with models that predict perceptual equivalence on the basis of equating such statistics.

First, we calculated the statistics of the luminance histograms of the Blob and Pepper objects, across changes in light field, shape, or both for stimuli of constant reflectance. These calculations provide a useful benchmark, as they tell us the relation between the statistics that we would expect for a visual system that exhibited perfect glossiness constancy. Next, we calculated the same statistics for stimuli that matched in perceived (but not physical) reflectance. These stimuli matched in both perceived glossiness and perceived lightness. According to the image statistics hypothesis, perceptually important statistics should match for these stimuli. We chose to analyze the standard deviation and skewness, based on previous reports that considered perceived glossiness, and added a third statistic that measured the number of bright pixels in the image (defined as those with more than five times the mean luminance of the whole stimulus).

The results of these analyses are shown in Figure 10a–b. Figure 10a illustrates the effect of changing the light field, shape, or both light field and shape on the statistics of physically matching stimuli. Each data point shows a particular stimulus with fixed reflectance undergoing either a light field, shape, or a joint change. Figure 10b shows the statistics across perceptually matching stimuli for all three change types. If observers are sensitive to a given statistic, we would expect the data points to fall considerably closer to the diagonal than in Figure 10a. This was not the case.

Figure 10.

Analysis of luminance histogram statistics. (a) Each panel shows how changing the light field, shape, or both light field and shape changes image statistics for stimuli of fixed reflectance. For each change, the reference statistic is plotted on the x-axis and the matched statistic on the y-axis. Data are plotted only for stimuli for which the statistics could be matched within our stimulus parameter space. From left to right, the statistics analyzed are the standard deviation the luminance histogram, the skewness of luminance histogram, and the number of bright pixels. (b) Each panel shows the statistics for perceptually matching pairs across the three changes. Error bars show one standard error of the mean across two measurements of each data point. Data are plotted only for stimuli for which the statistics could be matched within our stimulus parameter space. Other plotting conventions are as in (a). (c) Statistics for pairs of stimuli matched by an optimization algorithm. See text for details. The closed symbols show data points where the difference between the optimized statistic and the statistic of the reference stimulus is less than 10% of the maximum value that the statistic took on. Open symbols show the remaining cases.

Figure 10c rules out a possible artifactual reason for the predictive failure of the luminance histogram statistics: if the stimulus manipulations available to our subjects did not allow equating of a particular statistic, it would not be possible for the model to work. We thus used numerical parameter search to find the best match for each experimental condition and statistic within our stimulus parameter space. The results are shown in Figure 10c. For most cases, perfect equation is possible. Open circles show cases where stimulus limitations prevented a match. In Figure 10a–b only points corresponding to the closed circles in Figure 10c are plotted. It is clear that many prediction failures occur for stimuli where the statistic under consideration could have been matched within our experiment.

It is possible that summary statistics fare badly as predictors for perceived reflectance because they are calculated over the whole image such that the spatial properties of the highlights and their relationship to shading are not taken into account. To probe this possibility, we analyzed the statistics in several additional ways: by selecting a few informative subbands of the luminance channel, by selecting two regions of interest (ROI) on predominantly convex or concave parts of the object surfaces, and by extracting the specular image and analyzing its statistics (Figure 11). The results are shown in the format of Figure 10 in the supplementary materials (Figures S11–S14), and lead to the same conclusion as the analysis of the full luminance histogram. Perception was not determined by any of these statistics.

Figure 11.

Image statistics analyses were run on different versions of the original images. (a) Two ROIs in the stimulus images with different surface geometries were chosen for luminance histogram analysis. The ROIs for the three light fields and the two shapes are shown to the right of the stimulus images. (b) Stimulus images for the kitchen light field are shown bandpass filtered with three filters centered on the spatial frequencies of 28, 64, and 128 cycles/stimulus. (c) The specular images (ie, the difference images between the glossy and the matte basis images) are shown for the three light fields and the two shapes.

3.5. Effect of stimulus movement

In an additional experiment with new observers, we tested whether stimulus movement would aid in estimating surface reflectance in the face of changes in illumination or object geometry, as Wendt et al (2010) have suggested. The results are presented in the Appendix and show a slight improvement in constancy when the stimulus is rotated.

4. Discussion

4.1. Transitivity and separability

We checked whether our data satisfied transitivity (see Doerschner, Boyaci et al 2010). This property held well for lightness but less well for glossiness. Indeed, there were large violations of transitivity for glossiness for some of our measurement conditions. We do not know the cause of these violations. It is possible that they arise because measurement error is amplified when two measured quantities are multiplied, as is done when making the transitivity predictions. This explanation, however, would predict similar violations for glossiness and lightness and is thus not entirely satisfactory. It is possible that in making pairwise comparisons, observers weight different image features differently depending on the comparison, an effect which could produce transitivity violations. In any case, we used the transitivity analysis to select only data points for which transitivity was satisfied in our analysis of separability.

For glossiness estimates, separability did not hold. This means that extreme caution must be used in generalizing results on the effect of object shape obtained under a single light field or results on the effect of illumination geometry obtained for a single shape. For perceived lightness, separability held reasonably well. It should be noted that in general the effects of illumination geometry and shape on lightness were not large, so that there was less room for violations of separability to arise.

4.2. Individual differences

There were some marked individual differences in how surface glossiness was perceived in some of the experimental conditions, particularly for the Grove/Galileo comparison. We do not know the cause of these differences. As with the transitivity failures, one possibility is that different observers attended to different aspects of the stimuli when making their judgments. Some observers noted that the glossiness comparison task was difficult, an observation consistent with this interpretation, at least if one takes it to mean that observers were not quite sure what aspect of the complex stimuli should be compared.

4.3. Relationship to previous research

The effect of illumination geometry on perceived reflectance has been shown in several studies for different types of stimuli, but always for fixed object geometry. Fleming et al (2003) showed that illumination geometry affected the perceived glossiness of spheres in conventional displays, and Doerschner, Boyaci et al (2010) extended the result by showing in binocularly viewed conventional displays that the effects were transitive across light field changes. In addition, te Pas and Pont (2005) showed that changes in reflectance were confused with changes in illumination geometry, especially for smooth rendered objects as opposed to objects with three-dimensional surface texture (see also Pont and te Pas 2006). Olkkonen and Brainard (2010) studied the perceived glossiness and lightness of spheres in HDR displays, showing large effects of complex illumination geometry on perceived glossiness. The present study extends the previous findings to more complex shapes in full-color, HDR scenes, suggesting that the failures of reflectance constancy found in previous studies were not due to missing color or luminance information or to the use of simple shapes.

The effect of object shape on reflectance perception was first shown by Nishida and Shinya (1998). They were able to account for their data on the assumption that observers were basing their estimates about surface reflectance largely on the luminance histograms of the images. More recently, Ho et al (2008) showed that surface mesotexture (”bumpiness”) affects the perceived glossiness of a surface. In effect, mesotexture was confounded with surface glossiness to some extent (see, also, Wijntjes and Pont 2010). An effect of surface geometry was also shown by Wendt et al (2010), who additionally found that surface rotation and binocular viewing decreased the perceptual effects of shape. Notably, illumination geometry was held constant in all of these experiments. Vangorp et al (2007) studied the effect of object shape, rather than surface mesotexture, on perceived reflectance and showed that reflectance estimation was difficult across shape changes, especially for objects with flat geometry (such as tessellated spheres). Vangorp et al (2007) repeated the experiment with several different illumination conditions, but in any given pair the illumination geometry was held constant. They did not focus on whether the effect of shape depended on illumination geometry, but did note that ability to judge glossiness differed for one of their illuminant conditions. Our experiments confirmed the effects of object shape on perceived glossiness and showed a strong dependence of the effects on illumination geometry.

To our knowledge, ours is the first study to investigate the joint effects of object shape and illumination geometry on perceived reflectance.

4.4. Luminance histogram statistics

We were not able to account for our data on the basis of any of the luminance histogram statistics we analyzed, nor with variants of these statistics computed from subband histograms, regions of interest, or the specular reflectance component. Indeed, observers appeared to be performing closer to constancy than to matching on the basis of luminance histogram statistics (see, also, Olkkonen and Brainard 2010). Following Anderson and Kim (2009), we speculate that previous studies that have been successful in accounting for the data on the basis of luminance histogram statistics have obtained this result because the stimulus manipulations were not sufficiently rich to reveal the type of failures we find. Indeed, an important point emphasized by Beck and Prazdny (1981) and more recently by Anderson and colleagues (Anderson and Kim 2009; Kim and Anderson 2010; Marlow et al 2011; Kim et al 2011) is that the perception of glossiness depends not only on the distribution of luminance values in the image but also on the spatial distribution of the luminances. In a seminal demonstration, Beck and Prazdny (1981) showed that objects appear glossier if the highlights are oriented in the direction of minimum curvature on the surface. Rotation of the highlights away from this point decreased apparent gloss (see, also, Berzhanskaya et al 2005). Similar effects were shown by Anderson and Kim (2009) and Kim et al (2011) for objects with more complex surface structure. In our stimuli, the illumination and shape manipulations induced changes in both the luminance histograms and the spatial relations between highlights and surface relief.

4.5. Summary

We studied the effects of illumination geometry and object shape on perceived glossiness and lightness. In general, both manipulations affect both judgments. Of central importance, we examined the joint effects of the two variables. For perceived glossiness, we found strong interactions. The effect of a joint manipulation of both variables cannot be easily predicted from measurements of each alone. In addition, we examined a number of luminance histogram statistics as possible predictors of the glossiness effects, and found that none explained the data. Our results present a challenge for future research on perceived glossiness, as the enterprise of measuring the joint effects of illumination geometry and object shape will not be tractable without the development of a theory that can account for the joint effects without requiring full factorial measurements. Our conclusion on perceived lightness is more optimistic. Although perceived lightness does vary with both object shape and illumination geometry, we did not find large interactions between the two variables for the lightness judgment.

Biography

Maria Olkkonen received her MA in psychology from the University of Helsinki in 2004, after which she did a PhD on color constancy with Professor Karl Gegenfurtner at Giessen University. After receiving her degree in 2009, she moved on to work on material perception with Professor David Brainard at the University of Pennsylvania. She is currently a postdoctoral associate at Rutgers University with Dr. Sarah Allred, studying the relationship between color constancy and color memory.

David Brainard received his AB in physics from Harvard University (1982) and MS (electrical engineering) and PhD (psychology) from Stanford University in 1989. He is currently Professor of Psychology at the University of Pennsylvania and his research focuses on human color vision and color image processing. He is a fellow of the Optical Society of America and the Association for Psychological Science.

Appendix: Rotation experiment

We ran an additional experiment to test whether stimulus movement would serve as a cue to object and illumination geometry and make the perception of reflectance more veridical. Four new observers, one author (KMO) and three naive to the purposes of the study, participated. The display was the same as in the main experiment, except for the fact that the stimuli were rotating around their vertical axes 5 degrees in each direction from the neutral position (perpendicular to the line of sight). The movement was linear with a sinusoidal ramp at each end to make the rotation appear smoother. Only glossiness judgments were collected.

The PSEs were collected with the method of constant stimuli instead of a staircase procedure due to computer memory limitations. To provide a direct comparison, data were collected for both stationary and rotating stimuli. We studied illumination changes and changes of object shape separately.

As in the main experiment, we quantified the perceptual effects of changing light field or shape by fitting one-parameter linear regression lines to each data set separately for each observer, and taking the slope as the size of the effect. We computed the perceptual bias as the deviation of the slopes from unity. In most cases (11 of 16), rotation reduced the bias (Figure A1, where most of the points lie below the diagonal). Our results are consistent with recent reports by Wendt et al (2010) and Sakano and Ando (2010) and suggest that in real-world viewing motion cues may help stabilize the perception of glossiness.

Figure A1.

The relationship of the perceptual bias in glossiness judgments for the stationary and rotating stimuli. Each data point shows the deviation of a slope from unity for a particular observer and condition for stationary stimuli (x-axis) and the deviation from unity of a slope for the same condition for rotating stimuli (y-axis). Symbols stand for different observers as indicated in the legend. The two experimental conditions, change light field and change shape field are shown in orange and blue, respectively.