Attention alters appearance

Abstract

Does attention alter appearance? This critical issue, debated for over a century, remains unsettled. From psychophysical evidence that covert attention affects early vision—it enhances contrast sensitivity and spatial resolution—and from neurophysiological evidence that attention increases the neuronal contrast sensitivity (contrast gain), one could infer that attention changes stimulus appearance. Surprisingly, few studies have directly investigated this issue. Here we developed a psychophysical method to directly assess the phenomenological correlates of attention in humans. We show that attention alters appearance; it boosts the apparent stimulus contrast. These behavioral results are consistent with neurophysiological findings suggesting that attention changes the strength of a stimulus by increasing its ‘effective contrast’ or salience.

At any given moment, our visual system is confronted with far more information than it can process effectively. The high energy cost of neuronal activity involved in cortical computation severely limits our capacity to process this information¹. Visual attention serves as a mediating mechanism, enabling us to selectively grant priority of processing to certain aspects of the visual scene. One means of granting priority is to direct one’s gaze towards the relevant location. However, many situations call for one to attend to an area in the periphery without actually directing gaze toward it. For example, when driving it is generally best to keep your eyes on the road ahead while covertly monitoring the periphery for cars, pedestrians and potential road hazards. The impact of covert attention² on visual performance is well-documented across a range of perceptual tasks, such as visual search^3–6, letter identification^7,8, contrast sensitivity^9–12 and spatial resolution^13–16. Several studies that used single-cell recording^17–22, event-related potentials^23,24 and functional magnetic resonance imaging (fMRI)^25–27 indicate that attentional modulation occurs as early as striate and extrastriate visual cortex.

Transient attention, a type of covert attention, is the stimulus-driven, reflexive capture of attention by an abrupt, salient peripheral cue^{3–6,8–10,13–16,28–30}. For example, a ball rolling out into the street instantly grabs one’s transient attention, improving discriminability^8–16 and speeding information processing^3,4, enabling one to make a better and faster judgment of whether to swerve away. Explanations of how attention improves performance range from claims that the deployment of attention affects processing at the decisional level^9–11,31,32, to claims that attention actually enhances perceptual sensitivity^2–6,8–16. At the perceptual level, two prominent models have been proposed: signal enhancement (attention improves the quality of the stimulus representation^9,10,15) and external noise reduction (attention diminishes the impact of stimuli outside its focus^11,33).

Surprisingly, despite all the advancements in understanding visual attention, one long-standing debate remains unsettled: does attention alter appearance? Whether attention can actually affect the perceived intensity of a stimulus has been a matter of debate dating back to the founding fathers of experimental psychology and psychophysics. Whereas Helmholtz³⁴ and William James³⁵ believed that attention intensifies a sensory impression, Fechner³⁵ argued that attention does not alter sensory impressions.Well over a century of attention research has passed, and although recent studies characterizing the effects of attention on early visual processes suggest effects on appearance^2–30,36, very little direct empirical evidence has been brought to bear on the fundamental question regarding how attention might affect appearance^31,32,37. On the one hand, it has been reported that attention reduces perceived brightness contrast³⁷. On the other hand, it has been reported that attention does not change stimulus appearance in a number of perceptual domains, but rather reduces response variance, rendering a more veridical percept^31,32. However, a number of methodological concerns limit both findings. In one study³⁴, for example, observers were asked to make a comparison judgment between the target and one of four test patterns (held in memory), thus forcing observers to rely on a possibly biased categorical memory component for their responses³¹. In the other studies^31,32, a concurrent task paradigm was used; because in this paradigm attention allocation is not properly controlled^38–40, it is difficult to isolate the source of possible processing differences. In the same studies^31,32 observers were given an unlimited response time, which allowed them to make eye movements between the simultaneously presented target and the response palette, thus confounding results attributed to covert attention with overt eye movements, which could underlie the accuracy of their judgments.

Contrast, a basic dimension of vision, is a natural candidate for understanding the relationship between attention and appearance. The effects of covert attention on contrast sensitivity are documented across a wide range of psychophysical^8–12 and neurophysiological^17–22 tasks. Neurophysiological findings indicate increased contrast gain for attended relative to unattended stimuli^17–21. In this study, we addressed the effects of transient attention on perceived contrast. To do so, we implemented a method that enabled us to directly assess apparent contrast (Fig. 1), while circumventing the methodological limitations of previous studies. In addition, to preclude response bias, observers performed an orientation discrimination task contingent on the stimulus that appeared higher in contrast. This experimental design emphasized to observers the orientation judgment, when in fact we were interested in their contrast judgments. They were shown a pair of stimuli and asked to report the orientation of the stimulus that appeared higher in contrast: “Is the stimulus that looks higher in contrast tilted to the left or the right?” The two stimuli were Gabor patches (sinusoidal gratings of 2 or 4 cpd enveloped by a Gaussian, tilted 45° to the left or right), which appeared one on each side of a fixation point. In experiment 1, we kept the contrast of one Gabor fixed at a near-threshold level of 6% (standard patch), and varied the contrast of the other one from trial to trial (test patch), using a wider range of stimulus contrast (from 2.5 to 16%) than in previous studies^31,32,37.

Figure 1.

Sequence of events in a single trial. (a) Each trial began with a fixation point followed by a brief neutral or peripheral cue. The peripheral cue had equal probability of appearing on the left or right hand side, and was not predictive of the stimulus contrast or orientation. The timing of this sequence maximized the effect of transient attention and precluded eye movements. (b) Observers performed a two-by-two forced choice task: they were asked to indicate the orientation (left versus right) for the stimulus that appeared higher in contrast. In this trial, they would report the orientation for the stimulus on the right.

By assessing which stimulus observers perceived as being higher in contrast, we obtained psychometric functions describing the probability of choosing the test in reference to the standard, as a function of their contrast. The test contrast at which this function reaches 50% is the point of subjective equality (PSE). We measured these functions both when transient covert attention was directed to a particular location via a peripheral cue and when it was directed to the center of the display via a neutral cue. The peripheral cue was uninformative in terms of both stimulus orientation and contrast. This eliminated the possibility of observers giving more weight to the information at the cued location and ruled out a decisional explanation for an attentional effect^6,13. In the neutral condition, we expected subjective equality to occur at physical equality. However, if transient covert attention intensifies sensory impressions, when the test patch is cued, subjective equality should occur at lower test contrasts. Conversely, when the standard patch is cued, subjective equality should occur at higher test contrasts. We found that when observers’ transient attention was drawn to a stimulus location, observers reported that stimulus as being higher in contrast than it really was, thus indicating that attention changes appearance.

RESULTS

Experiment 1: low-contrast stimuli

The results for this experiment are summarized in the appearance psychometric functions (Fig. 2a). As expected in the case of the neutral cue, the PSE for both spatial frequency stimuli reflected the veridical percept. When they were both physically the same contrast (6%), observers were at chance for reporting which stimulus appeared higher in contrast. However, with the peripheral cue, the PSE corresponded to lower test contrasts. In other words, cueing the test stimulus reduced the test contrast required to match the standard. A nested hypothesis test⁴¹ (Weibull with separate fits for each condition vs. one fit for both conditions collapsed together) revealed highly significant differences between the two conditions (P < 0.0001). Similarly, cueing the standard stimulus increased the test contrast required to match the standard (P < 0.0001).

Figure 2.

Appearance psychometric functions for experiment 1 (low contrast). (a) Percentage of responses in which observers reported the contrast of the test patch as higher than the standard, plotted as a function of the test patch’s physical contrast. Data are shown for the neutral and peripheral conditions (test cued & standard cued). The standard was 6% contrast. The horizontal line intersecting both curves indicates the contrasts necessary for the test and standard stimuli to attain subjective equality (50%). (b) Psychometric functions for control experiment 1. When transient attention is extinguished via a longer timing interval, there are no differences between when the test is cued and the neutral cue. The standard was 8% contrast. Error bars correspond to the average ± standard error (s.e.) for each condition.

The data from individual observers were consistent with the averaged data. They are summarized in the appearance scatterplots in which each observer’s PSEs from the neutral and peripheral cue conditions are plotted against each other (Fig. 3a). The filled circles represent when the standard was cued, and empty circles represent when the test was cued. Had there been no effect of attention on appearance, the data would have fallen on a line of slope 1 (dashed line). When the test was cued, PSEs were consistently lower (below dashed line), and when the standard was cued, PSEs were consistently higher (above dashed line).

Figure 3.

Attentional effects on apparent contrast for individual observers. The PSEs for the peripheral condition (when either the standard or test was cued) versus the PSEs for the neutral condition (central cue). A slope of 1 (dashed line) represents similar PSEs for neutral and peripheral conditions; that is, no effect of attention on apparent contrast. However, when the test is cued, PSE’s are consistently lower, and when the standard is cued PSE’s are consistently higher, for both low-contrast (a) and high-contrast (b) stimuli.

There was a substantial change in perceived apparent contrast brought about by a peripheral cue (Fig. 4a). In this experiment, a 3.5% contrast, 4 cpd cued test stimulus appeared to the observers to be 6% contrast. Likewise, the cued stimulus at 6% contrast appeared as if it were 8.5% contrast. This difference in apparent contrast has the potential to boost performance in discrimination and localization tasks from near-chance levels to near-perfect performance¹⁰. Even in the case of this remarkably simple 45° orientation discrimination task, we found that attention improved performance. At contrasts within the dynamic range of the appearance psychometric function (above chance and below asymptote), for both spatial frequencies, performance was better with the test cue than with the neutral cue condition. Attention improved performance by 10% at PSE, and ranged from 18% at lower contrasts to 4% at higher contrasts where performance approached asymptote.

Figure 4.

Effect of covert attention on apparent contrast. (a) If you were looking at one of the four fixation points (black dot), and the grating to the left of that fixation point that was cued, the stimuli at both sides of fixation would appear to have the same contrast. With attention, a subthreshold, 3.5% test contrast stimulus appears as if it were at threshold (~6% contrast). Similarly, a cued 6% contrast standard at threshold appears as if it were a more clearly discriminable 8.5% contrast stimulus. (b) Likewise, with high-contrast stimuli when a 16% contrast grating is peripherally cued, it appears as if it were 22% contrast, and a cued 22% contrast grating appears as if it were 28%.

When observers’ transient covert attention was drawn to a stimulus via a peripheral cue, observers reported that stimulus as being higher in contrast than it really was, thus indicating a change in appearance with attention. To control for bias, observers were told prior to the experiment that the peripheral cue had equal probability of appearing either to the left or right of fixation and over the higher or lower contrast stimulus. To explicitly rule out the possibility that observers’ judgments were biased toward the stimulus location adjacent to the cue, we conducted a control experiment (control 1) that extended the interval between the cue and target onset to 500 ms. The time course for transient attention is short-lived: it peaks at ~120 ms and completely decays by ~250 ms^{3–6,8–10,13–16,28–30}. As a result of the ephemeral nature of transient attention, a long interval between the cue and target should eliminate any effect that it may have on perception, so any residual difference between the neutral and peripheral cues would be attributed to a cue bias. When the cue preceded the display by 500 ms, there were no systematic differences between the neutral and peripheral conditions (Fig. 2b). Thus, when transient attention was no longer active, the appearance of the stimulus was not altered.

Experiment 2: high-contrast stimuli

The observed decrement in the attentional effect on appearance at the higher range of stimulus contrast could be due to a ceiling effect: the upper bound does not leave room for improvement, thus restricting the possibility that observers’ responses manifest an increased apparent contrast. Therefore, in this experiment we investigated whether transient attention has the potential to alter the appearance of higher-contrast stimuli. This experiment was identical to the previous one, except that we kept the contrast of the standard patch fixed at 22%, and varied the contrast of the test patch from 6% to 79%.

The pattern of results is the same for both experiments (Fig. 5). With the neutral cue, the PSE for both spatial frequency stimuli reflected the veridical percept: Observers were at chance for reporting which stimulus appeared higher in contrast when they were both at 22% contrast. Again, cueing the test stimulus reduced the test contrast required to match the standard (P < 0.0001). Similarly, cueing the standard stimulus increased the test contrast required to match the standard (P < 0.0001). The distributions of PSEs for individual observers were consistent with the averaged PSEs (Fig. 3b).We show the magnitude of change in perceived apparent contrast brought about by a peripheral cue in Figure 4b. In this experiment, a 16% contrast, 4 cpd cued test stimulus appeared to the observers to be 22% contrast. Likewise, the cued stimulus at 22% contrast appeared as if it were 28% contrast. At contrasts within the dynamic range of the appearance psychometric function, performance for 4 cpd was better for the test cue condition than for the neutral cue condition. Attention improved performance by 10% at PSE, and ranged from 15% at the lower contrasts to 4% at higher contrasts where performance reaches a maximum. At 2 cpd, performance was at ceiling across contrast levels.

Figure 5.

Appearance psychometric functions for experiment 2 (high contrast). (a) Percentage of responses in which observers reported the contrast of the test patch as higher than the standard, plotted as a function of the test patch’s physical contrast. Data are shown for the neutral and peripheral conditions (test cued & standard cued). The standard was 22% contrast. The horizontal line intersecting both fits indicates the contrasts necessary for the test and standard stimuli to attain subjective equality (50%). (b) Psychometric functions for control experiment 2. When transient attention is extinguished via a longer timing interval, there are no differences between when the test is cued and the neutral cue. Error bars correspond to the mean ± 1 s.e. for each condition.

As in experiment 1, we conducted a control experiment (control 2) that extended the interval between the cue and target onset to 500 ms. Again, when the cue preceded the display by 500 ms, there were no systematic differences between the neutral and peripheral conditions (Fig. 5b). Thus, when transient attention was no longer active, the appearance of the stimulus was not altered.

Experiment 3: control experiment - lower contrast

We conducted an additional control experiment (control 3) to further rule out the possibility of cue bias. Observers were asked to indicate only which of the two gratings looked lower in contrast; that is, they were not asked to perform the orientation discrimination task. If the effects were due to cue bias, then observers would still have chosen the cued stimulus more often than the uncued one. Additionally, this experiment addressed the possibility that observers could have chosen the cued stimulus more often simply because the cue could have had a facilitatory effect on the orientation discrimination task. Were this the case, the cue would have no effect on appearance when observers were not asked to report orientation. The results are inconsistent with both of these hypotheses (Fig. 6). Not only did the PSEs for cued and uncued stimuli differ, but observers consistently reported the cued stimulus less often than the uncued one, indicating that they perceived cued stimuli to be higher in contrast. Thus, even when the task instructions were changed to request observers to report the lower contrast stimulus, they still perceived the apparent contrast of a cued stimulus to be enhanced.

Figure 6.

Appearance psychometric functions for when observers report the lower contrast grating (control experiment 3). Percentage of responses in which observers reported the contrast of the test patch as lower than the standard, plotted as a function of the test patch’s physical contrast. Data shown for the neutral and peripheral conditions (test cued & standrd cued). If observers’ responses resulted from cue bias, observers would have still chosen the cued stimulus a higher proportion of times. However, observers were less likely to choose the cued stimulus as appearing lower in contrast. These results confirm that attention increases the apparent contrast of a cued stimulus.

DISCUSSION

Does attention alter appearance? We developed a psychophysical procedure to directly address the phenomenological correlate of attention. The three experiments consistently demonstrated that transient attention increases apparent contrast for a wide range of stimulus contrasts, which in turn enabled observers to perform better in a discrimination task. Indeed, it is likely that this enhanced appearance underlies the increased contrast sensitivity observed in previous psychophysical studies^8–12, and possibly mediates the attentional benefits found in many other visual tasks.

What neurophysiological correlates underlie this change in contrast appearance? Two attentional effects are observed in neural responses: contrast gain and response gain. The signature of a contrast gain is a shift in the contrast response function to the left (decreased threshold). In the case of attention, this means that less contrast is necessary for an ‘attending’ neuron to attain the same response level as a ‘non-attending’ (neutral) neuron^17–21. On the other hand, the signature of response gain is an increase in firing with increasing contrast (steeper slope), reflecting an increase in strength of response for a neuron, particularly for higher contrasts^17–21. The present data provide evidence for a contrast gain model, in which attention allows for greater neuronal sensitivity (decreased threshold), suggesting that attention changes the strength of a stimulus by enhancing its ‘effective contrast’ or salience. It is as if attention boosted the actual stimulus contrast^17–21. The shallower slopes in the 4-cpd condition for the cued test stimulus are likely to result from the reduced range at the upper bounds of the psychometric function^10,18. Although comparisons between neurophysiological and psychophysical studies should be made with caution, the present psychophysical results showed a shift in the psychometric function with attention that is consistent with a contrast gain change. The present results were also consistent with another psychophysical study in which transient covert attention affected contrast gain along the psychometric function of contrast sensitivity¹⁰. Whereas that study assessed performance in an orientation discrimination task, in the present study we directly investigated the phenomenological correlate of attention, as well as its effect on orientation discrimination.

This finding builds on a growing body of psychophysical^3–16 and neurophysiological^17–27 literature characterizing the effects of covert attention on early visual processing. As remarkable as the human visual and cognitive systems may be, inevitably we are still limited by both bandwidth and processing power. Visual attention is crucial in optimizing the systems’ restricted capacity. In this study, we have addressed a fundamental issue in visual attention: Does attention alter appearance? By developing a method that allowed us to assess the effects of spatial cueing on apparent contrast, for the first time we can conclude that covert attention does intensify the sensory impression of a stimulus.

METHODS

Observers

A total of 69 naive observers participated in this study. In addition to 3 trained lab members, 13 observers participated in each of the following: experiment 1 (low contrast), its control experiment (control 1), experiment 2 (high contrast) and its control experiment (control 2). Ten observers, the same 3 lab members and 7 naive subjects, participated in control experiment 3. All observers were undergraduates from the New York University Subject Pool, were naive as to the purpose of the study, and had normal or corrected-to-normal vision. Participants signed an informed consent approved by the NYU Institutional Review Board. The pattern of results was the same for the trained and the naive observers.

Apparatus

The stimuli were created on a G4 Power Macintosh using Matlab and the Psychophysics Toolbox^42,43. Observers viewed the stimuli on a gamma–corrected monitor⁴⁴. A video attenuator was used to drive just the green gun of a 21” IBM P260 monitor (1024 × 768; 120 Hz), thus providing a larger possible set of distinct luminance levels (~12.6 bits). Background luminance was set to 12.2 cd/m².

Stimuli and design

A black square (0.1° × 0.1°) was presented in the center of a uniform gray background, serving as a fixation point. The fixation point was presented in the center of the screen throughout the entire experiment. There were two types of cues: peripheral and neutral. The peripheral cue was a small black dot (0.3° × 0.3°), which appeared 1.5° above the center of the location where a stimulus was about to appear. The neutral cue was the same black dot presented at fixation. The target display consisted of two Gabor patches (sinusoidal gratings enveloped by a Gaussian; 2° × 2°) presented to the left and right of fixation at 4° eccentricity along the horizontal meridian. From trial to trial, the stimuli randomly varied (with equal probability) along a number of dimensions: spatial frequency, orientation and contrast. To avoid any adaptation effects, in a given trial, the spatial frequency of both stimuli was either 2 or 4 cpd. The stimuli were independently tilted 45° either to the left or the right.

In experiment 1, one of the Gabor patches was always presented at a fixed contrast of 6% (hereafter referred to as the standard), whereas the contrast of the other Gabor (hereafter referred to as the test) was randomly sampled from a set of Michelson contrasts in 9 log increments from 2.5% to 16%. The stimuli and design were the same in control experiment 1 except that the standard’s contrast was 8%.

In experiment 2, the standard was always presented at a fixed contrast of 22%, whereas the test contrast was randomly sampled from a set of Michelson contrasts in 23 log increments from 6% to 79%. We sampled more intervals than in experiment 1 to get a more precise psychometric function. The stimuli and design were the same in control experiment 2 (standard = 22% contrast). In control experiment 3, in which observers were asked to report the lower contrast stimulus, the contrast range was the same (6–79%), but the test contrasts were sampled in 13 log increments. Note that the lower contrasts (6–16%) overlap with the higher contrasts of experiment 1.

Procedure

Each observer participated in a practice block of 75 trials and ten experimental blocks of 200 trials each, which lasted approximately 1 h. Observers viewed the display binocularly at a distance of 114 cm from the monitor with their heads stabilized by a chinrest. They were asked to fixate on the fixation point throughout the experiment. In each trial, observers were presented with a fixation point for 500 ms, after which either the peripheral or neutral cue was briefly flashed (67 ms; Fig. 1a). Following an interstimulus interval of 53 ms, the two Gabor stimuli were presented for 40 ms.

Observers performed a two-by-two alternative forced choice task: they indicated the orientation of the Gabor that appeared higher in contrast. They were instructed as follows: “Is the stimulus that looks higher in contrast tilted to the left or the right?” If the stimulus to the left of fixation appeared higher in contrast, observers indicated its orientation by pressing either ‘Z’ (leftward tilt) or ‘X’ (rightward tilt) on a keyboard with the middle or index finger of their left hand. If the stimulus to the right of fixation appeared higher in contrast, observers indicated its orientation by pressing either ‘․’ (leftward tilt) or ‘/’ (rightward tilt) on a keyboard with the middle or index finger of their right hand. Observers were informed that the peripheral cue was not informative either in terms of contrast or orientation and that it had equal probability of appearing on either the higher or lower contrast stimulus.

The 120-ms interval between the cue and target onset was chosen to maximize the effect of the peripheral cue in automatically eliciting transient attention^{3–6,8–10,13–16,28–30}. Furthermore, the 160-ms interval between cue onset and stimulus offset was chosen to preclude eye movements⁴⁵, thus ensuring that the observers performed the task under the conditions of covert attention.

In control experiments 1 and 2, the interval between the cue and target onset was increased to 500 ms. Given that this time allowed for possible eye movements, an infrared camera was used to monitor eye movements and ensure that observers did not break fixation.

In control experiment 3, observers were instructed in a different manner. Rather than asking them to indicate the orientation of the stimulus that looked higher in contrast, we asked them to report only which stimulus looked lower in contrast.