Perceived luminance depends on temporal context

Abstract

Brightness—the perception of an object’s luminance—arises from complex and poorly understood interactions at several levels of processing¹. It is well known that the brightness of an object depends on its spatial context², which can include perceptual organization³, scene interpretation⁴, three-dimensional interpretation⁵, shadows⁶, and other high-level percepts. Here we present a new class of illusion in which temporal relations with spatially neighbouring objects can modulate a target object’s brightness. When compared with a nearby patch of constant luminance, a brief flash appears brighter with increasing onset asynchrony. Simultaneous contrast, retinal effects, masking, apparent motion and attentional effects cannot account for this illusory enhancement of brightness. This temporal context effect indicates that two parallel streams—one adapting and one non-adapting—encode brightness in the visual cortex.

We report here a novel illusion in which temporal relationships affect brightness perception. Two flashes appeared on either side of a fixation point: one was brief (56 ms), the other long (278 ms; Fig. 1a). Observers reported which flash appeared brighter. When flashes of identical luminance had simultaneous onset, subjects reported that the brief flash looked dimmer than the long flash (Fig. 1b). This was expected from the Broca–Sulzer effect, in which a flash of brief duration looks dimmer than a physically identical flash presented for a longer duration⁷.However, when the two flashes had simultaneous offset, the brief flash appeared brighter (Fig. 1b; t = −5.32, P = 0.0009). This surprising brightness enhancement averaged 30% and was seen by all observers tested. When the brief flash occurred at intermediate times (between onset and offset of the long flash), the brightness grew monotonically (Fig. 1c).We call this illusion the temporal context effect (TCE; see the online demonstration at http://nba.uth.tmc.edu/homepage/eagleman/TCE).

Figure 1.

The temporal context effect. a, Time slices (56 ms) of two stimulus conditions. b, In a two-alternative forced-choice task, observers reported whether the brief flash was brighter or dimmer than the long flash. Brightness on the ordinate is determined by the point of perceived equivalence on a psychometric function, derived from the method of constant stimuli. The long flash had a luminance of 5.91 cd m⁻², the brief flash ranged from 3.55 to 8.27 cd m⁻² and the background was less than 1 cd m⁻². n = 7. The average is shown with a hatched bar. c, Brightness of the brief flash at intermediate stimulus onset asynchronies.

It could be that the brief flash looks the same in all conditions but is reported as brighter because it is compared with a dimming long flash. To address this possibility we tested whether the long flash dims perceptually. Observers were asked whether they detected a change in flashes that either physically ramped up or down in luminance or stayed constant. Observers reported that a physically unchanging flash was perceived as unchanging; that is, the observers (at least in these conditions) were adaptation-blind (Fig. 2a). This suggests that relative timing actually changes the brightness of the brief flash. To test this, we presented the stimulus shown in Fig. 2b: a brief flash is onset-matched with two long flashes; a second brief flash is offset-matched with the long flashes. Observers directly compared the two brief flashes against each other. The offset-matched brief flash appeared brighter than the onset-matched brief flash. This effect depends on the presence of the long flashes: in a control experiment, in which only the brief flashes were presented, no brightness difference was seen between them (see Supplementary Information). In a second control, we found that in static displays of three patches, dimming the patches corresponding to the long ones in Fig. 2b did not induce simultaneous contrast effects that could account for the TCE (see Supplementary Information). Thus, the appearance of the brief flash itself changed, as a result of temporal context.

Figure 2.

Only the brief flash changes perceptually. a, Adaptation-blindness. The abscissa shows the total change of flash luminance during the 278-ms duration (linear ramping). Subjects reported whether the flash did or did not change in brightness during its appearance. Subjects are better at detecting decrements than increments¹⁵, but the emphasis for this study is on the zero point of the x-axis: a physically unchanging flash is perceived as unchanging. Stimulus properties and starting luminance are identical to those of the long flashes in Fig. 1. n = 6. b, Observers compared the brightness of the diagonal brief flashes in a two-alternative forced-choice task. c, Insensitivity to distance. Stimuli were enlarged according to a cortical magnification factor¹⁶ for easier judging. n = 3. d, Contrast polarity determines the direction of brightness change: the experiment shown in Fig. 1b was repeated with dark (less than 1 cd m⁻²), equiluminant (grey flashes against a 5.91 cd m⁻² green background), and light (11.82 cd m⁻²) backgrounds. The luminance of the long flash was 5.91 cd m⁻²; the luminance of the brief flash ranged between ± 40% of that of the long flash. The ordinate shows the percentage difference between the two conditions.

The TCE does not arise from low-level interactions in the retina or lateral geniculate nucleus, because presenting the brief flash to one eye and the long flash to the other produces the full effect (see Supplementary Information). This indicates that the comparison of luminance ratios takes place in primary visual cortex or later, where binocular information first converges. Further, the TCE appears unrelated to apparent motion or meta-contrast masking, because it is surprisingly insensitive to spatial distance (Fig. 2c). This insensitivity can easily be demonstrated by presenting the long and brief flashes at opposite edges of the computer screen (see Supplementary Information).

We also investigated the possibility that the offset-matched flash appears brighter (or perhaps more salient) because attention shifts to it. However, attention shifts are slow (~200–300 ms, ref. 8), whereas the brief flash appears and disappears within 56 ms. Nonetheless, we manipulated the contrast polarity of the background to test the attentional explanation. When the background was brighter than the flashes, the offset-matched brief flash appeared dimmer, not brighter, than the onset-matched flash (Fig. 2d; t = −2.2, P = 0.026). More importantly, when the background and flashes were made equiluminant, observers perceived no significant brightness illusion, arguing against a salience-enhancing attentional shift (Fig. 2d; t = −0.58, P = 0.28).

Spatial interactions in brightness perception are well known^1,2 but provide an incomplete description; the TCE shows that brightness is influenced by temporal context as well. Our results further show a surprising juxtaposition of facts: first, that information encoded about the brightness of stimuli changes over time, such that the appearance of physically identical brief flashes compared to a persisting long flash varies as a function of stimulus onset asynchrony (Fig. 1c); and yet, second, the perceived brightness of a long flash remains constant over time (Fig. 2a). This indicates that brightness encoding might involve at least two neural populations: one with an adapting response that diminishes over time, and the other with a downstream response that assigns brightness labels to objects and does not adapt. We propose that the TCE arises from an interaction between these non-adapting and adapting encodings. In our model, activity in the non-adapting population remains constant—thereby encoding an unchanging label—even while its input from the adapting population diminishes (Supplementary Fig. S1; for an example of such hysteresis, see ref. 9). The hypothesis that some neurons maintain a brightness label is consistent with the idea that the brain strives to monitor the external world undistracted by predictable changes in its own physiology.

The activity of some neurons in primary visual cortex is correlated with brightness; that is, the firing rate varies as a function of surrounding luminance, even as the luminance in the receptive field remains unchanged^10–14. The activity of these brightness-responsive neurons adapts with time: the firing rates peak after 100 ms and then drop off precipitously¹⁴. This decrease in firing is consistent with an adapting population, although in some circumstances the later part of the spike train may correspond to the response of a non-adapting population¹⁴. This raises the possibility that the two encodings could be multiplexed into the same population of cells in V1. The TCE separates the physical measure of luminance and the perceptual quality of brightness (which typically co-vary) and offers a new approach for examining the neural coding of brightness.