Distinct Mechanisms Mediate Visual Detection and Identification

Summary

A core organizing principle for studies of the brain is that distinct neural pathways mediate distinct behavioral tasks [1, 2]. When two related tasks are mediated by a common pathway, studies of one are likely to generalize to the other. Here, we test whether performance on two laboratory tasks that model object detection and identification are mediated by common mechanisms of visual adaptation. Although both tasks rely on the luminance pattern in images, their demands on visual processing are quite different. Object detection requires discriminating image luminance differences associated with the light reflected from adjacent objects. To encode these differences reliably, neurons adapt their limited dynamic range to prevailing viewing conditions [3–6]. Object identification, on the other hand, demands a fixed response to light reflected from an object independent of illumination [7]. We compared performance in discrimination and identification tasks for simulated surfaces. In striking contrast to studies with less structured contexts, we found clear evidence that distinct processes mediate judgments in the two tasks. These results challenge models that account for perceived lightness entirely through the action of image-encoding mechanisms.

Results and Discussion

Although he was wrong about the brain’s basic function, Descartes’s idea that the central nervous system is structured with distinct pathways serving distinct behavioral tasks now forms a core organizing principle for studies of the brain. Although direct measurements of brain activity can delineate anatomical pathways involved in different tasks, they cannot establish whether different pathways perform redundant functions or, conversely, whether a single pathway performs multiple functions. We used psychophysical data and a quantitative model to test whether performance on two laboratory tasks that model object detection and identification are mediated by common mechanisms of visual adaptation.

We considered detection and identification in light of three well-established ideas: (1) Some visual processes are shared by most visual tasks (e.g., image encoding by the photoreceptors) [8], (2) visual information is sometimes segregated into parallel channels thought to subserve different functions [1, 2], and (3) the visual system adapts to prevailing viewing conditions [6]. Here, we use the term adaptation generally, to refer to any context-dependent change in the visual response to a focal stimulus. This definition of adaptation thus incorporates a wide range of possible effects, from gain changes in the photoreceptors to the action of complex cortical processes.

Adaptation can be revealed both by measures of image intensity discrimination [9, 10] and appearance judgments [11, 12]. These are laboratory methods used for measuring the processes underlying object detection and identification. Although both judgments rely on the luminance pattern in images, their demands on visual processing are quite different [13]. Object detection demands reliable discrimination of image luminance differences associated with surface reflectance differences. Object identification, on the other hand, demands a fixed response to light reflected from an object independent of context. Given these distinct demands on visual processing, adaptation measured by discrimination and identification judgments may reflect different processes.

Consider the shadowed checkerboard, adapted from Adelson’s checker-shadow illusion [14] (see also [15]), in the top panel of Figure 1. The two spots on the checkerboard, as well as the checker they sit on, are physically identical. Yet most people judge the spot on the right as lighter than the spot on the left. Two quite different classes of explanation have been offered for this and other related effects.

Figure 1. Images of the Shadowed and Painted Checkerboards.

The two images shown here are physically identical except that the penumbra of the shadow has been replaced by a sharp edge coinciding with the checkers. In the latter case, it tends to look like the checkers along the negative diagonal have been painted with a darker paint. Despite the physical similarity of the two images, most people see a greater difference in the appearance of the spots in the shadow than in the paint image. The fact that the appearance effect in the paint image is lesser than in the shadow image suggests that the apparent lightness of the spots is influenced, perhaps implicitly, by consideration of the causal structure of the images: In effect, the visual system infers that there is less light reaching the shadowed region of the checkerboard and compensates for this illumination difference to create a stable representation of surface reflectance.

One class of explanation attributes the effect to image-encoding regulation that serves to optimize the use of limited neural capacity [3, 4, 16]. The first step in vision is to encode the light pattern reaching the eyes. To maximize information transmission, neurons at the front end of the visual system adapt to image intensity and contrast. Here, the relevant stimulus feature is that the region surrounding the spot on the right is darker than the corresponding region for the spot on the left. Because sensitivity to a target image generally increases with increases in darkness [5, 9], the response of encoding mechanisms would be higher for the spot on the right. This in turn is postulated to produce that spot’s lighter appearance.

A key feature of this explanation is that effects of mechanisms that regulate image encoding propagate through the visual system and thereby influence both discrimination and appearance judgments. We therefore refer to this as the common-mechanism hypothesis. Under this hypothesis, changes in appearance are predicted by changes in sensitivity, and constancy of appearance and optimization for image encoding are accomplished by the same mechanisms. For a variety of stimulus configurations, context effects on discrimination sensitivity and appearance can be explained by common mechanisms of adaptation [17–21] (but see also [22]).

A second class of explanation [23] emphasizes the idea that appearance effects such as those shown in Figure 1 facilitate object identification across changes of illumination; this phenomenon is known as lightness or color constancy [7]. This view suggests that visual system, in effect, distinguishes the shadowed region in Figure 1 as such and compensates for the inferred dimmer illumination. Some theorists who take this general view model the visual processes that stabilize object appearance through computations that combine image data with information about possible world states to recover explicit descriptions of object physical properties [24–26]. Others downplay the role of explicit representations of the physics and instead focus on delineating heuristics whose action combines to produce stable percepts [22, 15, 27]. In either case, effects such as those shown in Figure 1 are not attributed to the action of mechanisms designed for optimizing sensitivity. Still, many complex appearance phenomena have been accounted for by models reminiscent of those that account for discrimination data [28–30], and there are little data that rule out the possibility that efficient image encoding and constancy are achieved by the same processes. Indeed, as noted above, most tests of the common-mechanism hypothesis fail to reject it [17–21]. Here, however, we show directly that the appearance effects of Figure 1 are not consistent with the common-mechanism hypothesis. An important implication of this result is that encoding regulation and appearance stabilization are controlled by distinct processes.

To test the common-mechanism hypothesis, we measured intensity discrimination thresholds at the two spot locations indicated in Figure 1 of the shadow image (top panel) and compared these with corresponding measurements in the paint image (bottom panel). Object detection requires discriminating the light reflected by the object from the light reflected by its surroundings. Because objects are found against backgrounds of varying complexity and color, measuring discrimination thresholds against different backgrounds in the laboratory provides information on how objects are detected in real scenes. As shown in Figure 2, there was no difference in sensitivity between the corresponding spot locations of the two images. Because of this equality, the common-mechanism hypothesis predicts that the appearance effects in the paint image should be the same as the shadow image. Inspection of Figure 1 readily reveals that this prediction does not hold. Experimentally, we had participants set lightness matches between the spots within each image and confirmed this observation (Figure 2): There is little if any appearance effect for the paint image and an easily measurable effect for the shadow image. The dissociation between the way the paint and shadow images affect sensitivity and appearance falsifies the common-mechanism hypothesis.

Figure 2. Discrimination and Matching Results for Eight Participants in the Paint and Shadow Conditions.

The top left and right panels show just-noticeable differences (JNDs) for tests located outside and inside the shadowed and painted regions, respectively (as indicated in the images in the lower right). The x axis is labeled as “Pedestal Intensity” and refers to the fixed spot intensity, I_p. Participants’ task was to discriminate I_p shown in one interval from I_p+ΔI shown in the other interval. For each participant the increments, ΔI, were normalized by detection thresholds (i.e., discrimination threshold for I_p = 0) measured at the test location outside the shadow. We pooled and fitted the data with cumulative Gaussians to estimate JNDs defined as the 75% correct point. The JNDs plotted in the top two panels were rescaled by the average absolute threshold. Diamonds and circles represent data from the shadow and paint conditions, respectively. Error bars are 95% confidence intervals. Data from the two participants who only observed in the paint condition are not included here. Asymmetric matches from the appearance task are plotted in the bottom-left panel. The mean intensity of participants’ settings is plotted against the fixed test intensities. Standard errors were smaller than the symbols used here. Matches were performed with the fixed test in both locations: Open and closed symbols represent, respectively, settings when the adjustable test was outside and inside the shadowed or painted region. In the paint condition, deviations from the physical identity of the test and matches are very small. In the shadow condition, matches occurred when the spot intensity in the shadow was lower than the spot intensity outside the shadow.

The results in Figure 2 provide a model-free falsification of the common-mechanism hypothesis: The sensitivity data are the same for the shadow and paint conditions, although the appearance data differ. What we do not learn directly from Figure 2 is whether the common-mechanism hypothesis breaks down for the shadow condition, the paint condition, or both. We can leverage the data to address this question by applying a quantitative model. The model provides an explicit link between discrimination and appearance data [20, 21] and allows us to examine the common-mechanism hypothesis within each condition.

Figure 3 illustrates the model. For any fixed context, visual responses to a local image region, represented in the central panel, accelerate at low intensities and saturate at high intensities [31, 32]. We, following others [32–34], used a modified version of the Naka-Rushton function to model this intensity-response function:

$$R=M\frac{{(gI+s)}^p}{{(gI+s)}^q+1}$$ (1)

where I is intensity and M, g, s, p, and q are free parameters controlling the shape of the response function.

Figure 3. Common-Mechanism Model.

The red and black curves in the central panel represent the intensity-response function of a typical mechanism in two states of adaptation. The black and red curves in the lower panel represent the just-noticeable differences (JNDs) associated with these two states of adaptation. Each point on each curve in the lower panel represents a predicted 75% correct discrimination threshold for the response curves in the middle panel. Note that JNDs are lowest where the response curves are steepest and become infinite when the response curves saturate. These JND curves were derived under the assumption that responses of the encoding mechanism are corrupted by additive, normally distributed noise of fixed variance. This is formally equivalent to Fechner’s proposal that discrimination thresholds correspond to a set difference in the neural response to the targets presented [34]. The blue and orange points in the central and lower panel demonstrate the logic for one test intensity. The intensity difference between the blue and orange points (x axis) in the central panel yields a response difference (y axis) that permits correct discrimination of intensity increment 75% of the time. In the lower panel, the test intensity is again plotted on the x axis, and the intensity difference (one JND) between the blue and orange points is represented on the y axis. Comparison of the red and black curves in the central and lower panels shows that low JNDs correspond to tests located at steep parts of the response functions, and high JNDs correspond to tests located at shallow parts. The top panel shows the matches predicted across the states of adaptation (i.e., two contexts) shown in the central panel. To predict apparent matches, we assume that the targets in the two contexts look the same when the response of the mechanism is the same in the two contexts. For a target presented in the first context with intensity indicated by the black point (central panel), a target with intensity indicated by the red point produces the same response the other context. These would therefore be considered a match. The target intensities that produce matches for the states of adaptation in the central panel are shown by the red line in the top panel. The faint gray line in this panel represents physical identity of the targets. In the case shown, the difference between the two intensity-response functions is described by a change in the gain parameter of Equation 1, so that the predicted matches plot along a line with unity slope in the log-log plot. Other shapes of the predicted matching function may be obtained when other parameters of Equation 1 vary with adaptation.

The two curves in the central panel represent the same encoding mechanism in two states of adaptation (i.e., in two different contexts). We modeled adaptation as a change in the value of one or more of the five parameters in Equation 1 with each change of context. In our formulation, there are no restrictions on the nature of this contextual dependence, and this makes Equation 1 general enough to account for the action of a wide variety of processes (e.g., local gain control, lateral inhibition, effects of contour-junction interactions, etc.) that might affect sensitivity or appearance.

To link discrimination and appearance data, we assume: (1) Discrimination sensitivity is proportional to the slope of the response function (i.e., $\mathit{JND}\propto \frac{1}{R^{\prime }}$) [35], and (2) appearance matches occur when the response of the mechanism is the same in the two contexts [36]. JNDs and appearance matches associated with the parametric change that produced the two response functions in the central panel are shown in the lower and upper panels, respectively. Can such parametric changes simultaneously account for effects of context on discrimination and appearance judgments?

In the context of Equation 1, there are 31 possible parametric models of adaptation: five models in which one adaptation parameter is allowed to vary with context, whereas the other four fixed parameters are held constant across contexts, ten models with two adaptation parameters and three fixed parameters, etc. Within any of these 31 models, we can address the question of whether the common-mechanism hypothesis holds by comparing three nested model variants [21]. In the common mechanism (CM) variant, all parameters are the same for discrimination and appearance judgments. In the independent adaptation (IA) variant, the fixed parameters are common to discrimination and appearance judgments, but the adaptation parameters can be different for each judgment. In the independent channels (IC) variant, both the fixed and adaptation parameters can vary with judgment. For a given parametric model of adaptation, the IC variant has more free parameters than the IA variant, and the IA variant has more free parameters than the CM variant (e.g., for a parametric adaptation model that allows g to vary with each of two contexts, the CM variant has a total of six free parameters, the IA variant has eight, and the IC variant has 12). Thus, for a given model of adaptation, the CM variant is nested within the IA variant that in turn is nested within the IC variant.

We fit the CM, IA, and IC variants of each of the parametric models to the data from the shadow and paint conditions separately. For each condition (shadow or paint), the data consisted of the discrimination data for the tests at the two locations and the appearance matches between the two locations. We then used Akaike’s an information criterion (AIC) [37, 38] and the Bayesian information criterion (BIC) [38, 39] to determine, again for the shadow and paint conditions separately, which variant of which model of adaptation provided the best account of the systematic effects observed in the data. By accounting for the number of free parameters as well as the goodness of fit to the data (i.e., by “punishing” for additional parameters), these information criteria discriminate when additional parameters bring the model closer to the systematic process that generated the data from cases in which additional parameters fit noise in the data. Analysis in terms of these model selection criteria allowed us to draw two key conclusions described below. The Supplemental Data (available online) provides details of the analysis, and figures showing the results of the fitting used in the analysis can be viewed at http://color.psych.upenn.edu/supplements/shadowpaint/mainwebshdpnt.html.

First, for the family of models we considered, the common-mechanism hypothesis is rejected for both the paint and shadow conditions. This failure is in striking contrast to our earlier results for simple image contexts that do not look like illuminated objects, i.e., blurry spots seen against uniform backgrounds or spatial noise [20, 21]. We speculate that the difference occurs because in relatively unstructured contexts, stimuli do not look like illuminated surfaces, and thus mechanisms controlling surface-appearance judgments are not invoked. In the present experiments, discrimination performance is unaffected by the processes that differentiate the paint and shadow interpretations. An important implication of this conclusion is that models of adaptation developed to account for discrimination data cannot by themselves provide a complete account of how context affects appearance.

Second, more complex parametric models of adaptation are required to account for the data from the shadow condition than for the paint condition (see Figure S3). A process interpretation that is consistent with this result (but not compelled by it) is that the mechanisms that generate appearance act on the output of early image encoding mechanisms and that the action of these appearance mechanisms is most strongly revealed by the shadow stimulus configuration. To be effective at stabilizing appearance, such a mechanism would need access to information about the adapted state of the earlier image encoding mechanisms (see [40]; this has been shown to be biologically plausible [41, 42]) as well as information about lighting variation in the scene. We cannot at this point make any definitive statement about how or at what stage in the visual pathways this information begins to affect lightness processing.

On a broader scale, our data are consistent with the speculation that evolutionary processes have favored the incorporation of distinct mechanisms specialized for detecting objects by their reflectance differences and for identifying objects by their surface reflectance.

Experimental Procedures

Eight paid volunteers who were unaware of the experimental hypotheses and the two authors participated in the experiments. Two participants did not complete observations for the paint condition, so there were only eight participants for this condition. All participants had normal or corrected to normal acuity and normal color vision as assessed by an Ishihara color-blindness test.

Discrimination performance was measured in a two-interval forced-choice task. The intensity in one interval was set at I_p. We call this the pedestal intensity. The intensity in the other interval was the pedestal plus an increment, I_p + ΔI. Participants indicated the interval they believed contained the increment. Trials alternated between the two possible locations in each (shadow/paint) condition. The value of ΔI was controlled by an adaptive staircase procedure.

The effect of context on appearance was measured with an asymmetric matching task. Participants matched the appearance of sequentially presented test spots located as shown in Figure 1. Spatial and temporal parameters of the targets were essentially identical to those used in the discrimination experiment. The only differences were (1) the intensity of the spots in the two target intervals at one location was the same, and (2) the participant responded only after the targets were presented in both target locations (as opposed once after each target location in the discrimination experiment). The response was a button press that changed the intensity of the spot at the second location. The selected intensity change was then used in the subsequent spot presentation. The participant indicated when a perceptual match had been obtained.

Specific instructions given to the participant in the matching experiment were: “Make the brightness of the adjustable spot the same as the fixed spot, disregarding, as much as possible, other areas of the display. That is, make it look like the amount of light coming from the adjustable spot is the same as that coming from the fixed spot.” We used these instructions to bias participants toward relying on a low-level percept.

Complete details of the experimental procedure and stimuli are available in the Supplemental Data.