Spatial attention alters visual appearance

Abstract

It is well established that attention improves performance on many visual tasks. However, for more than 100 years, psychologists, philosophers, and neurophysiologists have debated its phenomenology-whether attention actually changes one’s subjective experience. Here, we show that it is possible to objectively and quantitatively investigate the effects of attention on subjective experience. First, we review evidence showing that attention alters the appearance of many static and dynamic basic visual dimensions, which mediate changes in appearance of higher-level perceptual aspects. Then, we summarize current views on how attention alters appearance. These findings have implications for our understanding of perception and attention, illustrating that attention affects not only how we perform in visual tasks, but actually alters our experience of the visual world.

We confront an overwhelming amount of sensory information at any given moment, yet we have the impression of effortlessly understanding what we see. Selective attention enables us to select a certain location or feature of a visual scene to prioritize its processing and guide behavior. Selection is necessary given that our capacity to process visual information is limited by the high energy cost of cortical computation and the fixed amount of energy consumption available to the brain [1]. By enhancing the representation of relevant information while diminishing the representation of irrelevant signals, attention optimizes our system’s limited resources [2–7].

We are usually unaware of how attention alters visual representations, much like we are unaware of how the representation quality differs across the visual field [8–12]. Although visual quality decreases with eccentricity, our subjective perception in the periphery is ‘inflated’ [13] and does not reflect the drop in image quality [14–16]. Visual quality and appearance also differ pronouncedly along isoeccentric locations [8,11,12,17,18]. These differences in processing quality shape our subjective perceptual experience.

COVERT ATTENTION ALTERS APPEARANCE

Visual attention can be overtly or covertly deployed, with or without eye movements, respectively [2–7,19–21]. We constantly use both types of attention everyday; e.g. searching for objects, crossing the street, driving, biking, playing sports, and during interpersonal interactions. There are two types of covert spatial attention: Voluntary attention refers to the endogenous and sustained allocation of attention to a specific visual field location. Involuntary attention is the exogenous, transient capture of attention to a location brought about by a sudden change in the environment [2–6,20,21]. To characterize their effects on perception observers are cued to attend to specific locations while keeping their gaze at a central fixation point.

The observers’ attentional state affects perceptual performance in detection, localization and discrimination tasks [2–6,20,21] and the activity of sensory neurons throughout the visual cortex [7,20,22]. Recent research has furthered our understanding of how attention modulates the spatiotemporal sensitivity of early perceptual filters, and the neural computations underlying such processes.

Does attention alter our subjective experience of the world? The phenomenology of attention has been a topic of interest for over a century, with the pioneers of experimental psychology—Wundt, Mach, Fechner, Helmholtz and James [19,23,24]. Introspective subjective methods led to conflicting conclusions; e.g., Helmholtz and James claimed that attention intensifies sensory impressions, but Fechner disagreed.

Last century, a few empirical studies reported that attention reduces response variance, but does not change stimulus appearance, rendering a more veridical percept [25–27]. However, given their methodology, the results were inconclusive [3,28,29]. Last decade, we implemented a novel psychophysical paradigm to evaluate attention’s phenomenological correlates [28], which enables the objective and rigorous study of subjective experience [29,30].

Exogenous attention alters appearance of spatial dimensions

Contrast is an ideal dimension for investigating attention and appearance. To assess perceived contrast, observers are presented with two equidistant Gabor patches and asked to report the orientation of the higher contrast stimulus (Figure 2). The instructions emphasize the orientation judgment, but the main interest is the contrast judgment. On each trial, one of the patches has a fixed contrast (standard stimuli), whereas the contrast of the other patch (test stimuli) is randomly chosen from a range of contrasts around the standard’s contrast. The point of subjective equality (PSE) is obtained from psychometric functions describing the probability of choosing the test stimulus over the standard as a function of test contrast. The PSE corresponds to the contrast at which the test stimulus appears similar to the standard stimulus. To assess the effects of exogenous attention on perceived contrast, these functions are measured when attention is either distributed across the display, via a neutral cue, or automatically captured at one stimulus location via a peripheral cue. Observers are told that the peripheral cue is uninformative and has equal probability of appearing at either side, independently of the stimulus contrast and orientation.

Figure 2.

Exogenous attention alters appearance. (a) Trial sequence: exogenous attention is manipulated using brief, non-informative precues at either both locations (distributed, neutral) or at one location (test cued or standard cued). Observers have to report the orientation of the stimulus with higher contrast. (b) When plotting the proportion of time the test stimulus is chosen as being higher contrast than the standard stimulus (e.g., 20% reference contrast), cueing the test or standard stimulus location shifts the curve leftward (lower PSE) and rightward (higher PSE), respectively. (c) Control conditions: No change is observed when postcues or longer inter-stimulus intervals (ISI) are used. (d) Exogenous attention changes how we subjectively perceive various visual dimensions, except for hue; upper row: stimuli match physical stimuli under neutral condition; bottom row: perceived stimuli change under attention condition.

Using this paradigm, we found that exogenous attention significantly enhances perceived contrast (Figure 2; Table 1). In the neutral condition the PSE matched physical equality. Cueing the test patch shifted the PSE to lower contrasts, whereas cueing the standard patch shifted PSE to higher test contrasts, indicating that the attended stimulus appeared higher in contrast. Moreover, attention improved performance at the cued location in the concurrent orientation discrimination task. Thus, when observers’ attention is drawn to a stimulus location, the stimulus is objectively better discriminated and subjectively perceived as being higher in contrast than it actually was, indicating that attention alters appearance. Several control experiments have ruled out alternative explanations (Figures 2,3; Table 1).

Table 1.

Summary of the effects of covert spatial attention on visual appearance.

Visual Dimension	Perceived Effect	References	Controls and Variants
Contrast	Enhanced Contrast	Exogenous Carrasco et al., 2004 [28] Hsieh et al., 2005 [33] Ling & Carrasco, 2007 [36] Carrasco et al., 2008 [31] Fuller et al., 2008 [12] Fuller et al., 2009 [35] Störmer et al., 2009 [38] Anton-Erxleben et al., 2010 [39] Anton-Erxleben et al., 2011 [40] Kim et al., 2014 [42] Cutrone et al., 2014 [32] Störmer & Alvarez, 2016 [43] Mishra & Srinivasan, 2017 [44] Barbot & Carrasco, 2018 [37] Zhou et al., 2018 [34] Endogenous Liu et al., 2009 [60]	• effects eliminated with:  - postcues [31;35]  - lengthened ISI [12,31,36,37,38,43,44] • effects present with:  - reverse instructions [28;36;39;43]  - both comparative and equality judgments [34,39,40]  - auditory precues [38]  - different cue polarity [36]  - a wide contrast range; attenuated for very high contrast [32;34] • effects increase with cue salience:  - cue contrast [12]  - cue emotional valence [37] • changes in performance and appearance correlate [37] • enhanced apparent contrast can alter higher-level features such as perceived facial attractiveness [43] and emotion [44] • effects present with reverse instructions [60]
Spatial Dimensions	Increased Spatial Frequency	Exogenous Gobell & Carrasco 2005 [46]	• effects eliminated with postcues [46] • effects present with:  - both lower and higher spatial frequencies [46]  - reverse instructions [46] • effects not due to changes in perceived contrast [46]
		Endogenous Abrams et al. 2010 [61]	• effects eliminated when timing allows voluntarily attention to be redeployed across both stimulus locations [61]
	Increased Gap Size	Exogenous Gobell & Carrasco 2005 [46]	• effects eliminated with postcues [46]
	Increased Positional Repulsion	Exogenous Pratt & Turk-Browne 2003 [49] Fortenbaugh et al. 2011 [52] Klein et al. 2016 [51]  Cutrone, et al. 2018 [48]	• effects eliminated with postcues [52] • apparent shape of objects altered based on precue position [52] • larger positional biases with eccentricity predicted by an attention field model [51]
		Endogenous Suzuki & Cavanagh 1997 [50] Cutrone, et al. 2018 [48]	• effects modulated by the attentional field size [48]
	Increased Object Size	Exogenous Anton-Erxleben et al. 2007 [47]	• effects eliminated with postcues [47] • effects present with reverse instructions [47]
	Enhanced Perceptual Organization	Endogenous Barbot, et al. 2007 [72]	• effects present with reverse instructions [72] • effects eliminated when timing allows voluntarily attention to be redeployed across both stimulus locations [72]
Color	Enhanced Saturation	Exogenous Fuller & Carrasco 2006 [53] Kim et al. 2014 [42]	• effects eliminated with postcues [53] • effects present:  - with reverse instructions [53]  - in ADHD observers [42]
	No Change In Hue	Exogenous Fuller & Carrasco 2006 [53]	• even though attention improved performance
Temporal Dimensions	Increased Flickering	Exogenous Montagna & Carrasco 2006 [56]	• effects present with reverse instructions [56]
	Enhanced Motion Coherence	Exogenous Liu et al. 2006 [55]	• effects eliminated with lengthened ISI [55]
	Increased Speed	Exogenous Turatto et al. 2007 [58] Fuller et al., 2009 [35] Anton-Erxleben et al., 2013 [59]	• effects eliminated with:  - postcues [58]  - lengthened ISI [35;58] • effects present with:  - reverse instructions [58;59]  - different adaptation states [59]

Figure 3.

Endogenous attention alters appearance. (a) Trial sequence: Observers are asked to report whether a target letter ‘X’ was present in one of two rapid serial visual presentation (RSVP) streams. Endogenous attention was manipulated using symbolic precues at fixation indicating one (cued) or both (distributed/neutral) RSVP streams in which the target letter could be presented. In trials in which observers did not see a RSVP target, they were asked to do an appearance task. By varying the delay between the two tasks, we can ensure that voluntary attention is still sustained at the cued location when the stimuli for the appearance task are presented. (b) Similar to exogenous attention, cueing the test or standard stimulus location shifts the curve leftward (lower PSE) and rightward (higher PSE), respectively. (c) Reverse instructions: in addition to other control conditions in which no effect is observed (e.g., lengthened delay), reversing the instructions in the appearance task usually shows changes consistent with an appearance change, rather than a response bias. (d) Endogenous attention changes how we subjectively perceive various visual dimensions; upper row: stimuli match physical stimuli under neutral condition; bottom row: perceived stimuli change under attention condition. For the perceived brightness illustration, maintain fixation on any of the fixation spots while shifting attention from one disk to another. You should notice that the attended disk appears to darken [69].

Attentional changes in appearance have been replicated while manipulating stimulus locations [12], stimulus contrast [31–34], cue contrast [35], cue polarity [36], cue type (fearful vs. neutral faces [37]; auditory cues [38], and equality judgments [39,40] (but see [41]), as well as with people with ADHD [42]. Furthermore, increased perceived contrast affects the appearance of higher-level objects like facial attractiveness [43] and facial emotion [44].

Attention also affects performance in spatial resolution tasks [21,45]. The paradigm described above has shown that exogenous attention also increases perceived spatial frequency and gap size [46] and size of moving patterns [47], and alters perceived position. A repulsion effect illustrates that attention can distort the encoding of nearby positions [48–51]. Stronger repulsion effects with attention are observed with eccentricity and could be due to attentional shifts in receptive size [9,21,51]. Similarly, drawing attention to the center of a circular stimulus increases its perceived size [47]. Attention also affects the perceived shape of ovals; depending on whether the cue is inside or outside the contour, the stimulus is perceived as longer or shorter [52].

The effects of attention on orientation discrimination and contrast appearance correlate [25]. However, an attention effect on performance does not necessarily lead to enhanced appearance. Despite improving orientation discrimination for both saturation- and hue- defined stimuli, exogenous attention changes apparent color saturation [42,53], but not hue [50]. This differential effect may be related to the nature of perceptual dimensions. Saturation, like contrast, is a directional prothetic dimension. Hue is a metathetic dimension; our percepts of red and blue are qualitatively different; red is neither “more” nor “less” than blue [54].

Exogenous Attention alters appearance of temporal dimensions

Attention also affects the appearance of fundamental dynamic properties (Figure 2). Observers report the motion direction of the stimulus aperture with higher coherence of moving dots out of two simultaneous apertures. Attention enhances perceived motion coherence making the motion direction more salient and improves performance in a concomitant direction discrimination task [55]. Similarly, when using two simultaneously flickering Gabor patches of equal contrast, exogenous attention increases perceived flicker rate [56], much like the effect of increasing stimulus contrast on perceived flicker [57]. Moreover, when observers report the direction of the faster-moving stimulus out of two stimuli, they perceive the stimuli at the attended location as moving faster than at unattended locations [35,58,59].

Voluntary attention alters spatial dimensions

We adapted the appearance paradigm [28] to investigate the effect of voluntary, endogenous attention on subjective experience [60]. We used a rapid serial visual presentation (RSVP) detection task to engage observers’ focal attention at a location. Observers are instructed to attend either to the cued RSVP stream (central cue) or to both streams (neutral cue) and to detect the presence of a target letter (X, present on 20% of the trials, equally likely at either location). When they detect the X, they press the space bar and ignore the subsequent Gabor patches. When they do not see the X, they report the orientation of the higher-contrast Gabor patch (Figure 3). Crucially, the attention manipulation conveys no information about orientation discrimination, the task of interest. The short inter-stimulus interval (100 ms) between the RSVP streams offset and the Gabor patches is quick enough to prevent attention redeployment, ensuring that attention is still at the peripheral location when the Gabor patches appear.

When observers allocate endogenous attention to a specific location, performance in both the RSVP detection task and the orientation discrimination task improves and perceived contrast increases [60]. Similarly, endogenous attention increases perceived frequency [61] and the repulsion effect [48].

The similar phenomenological consequences of endogenous and exogenous attention, notwithstanding their distinct time courses, control processes and some differential effects on performance [62–65], may result from similar cortical computations [66,67].

More evidence that attention influences the contents of visual experience comes from Peter Tse and colleagues, who have shown that voluntary attention increases the duration of afterimages and modulates perceived brightness, color and location [33,68–70].

Endogenous attention also alters mid-level vision. Given that attention and perceptual organization modulate each other and affect performance [71], we investigated whether attention alters the perceived perceptual organization of two multi-element arrays, organized by luminance similarity as either rows or columns. We found that endogenous attention intensified apparent perceived organization at the attended location; thus, attention also alters appearance of mid-level vision [72].

Ruling Out Alternative Explanations

Many control experiments (Figures 2,3; Table 1) have ruled out alternative accounts regarding all discussed static and dynamic dimensions:

Reversing the task direction:

When observers report the orientation of the stimulus of lower, rather than higher, apparent contrast, they choose the cued test stimulus less frequently. Were results due to cue bias, observers would have chosen the cued stimulus more often than the other stimulus regardless of the task direction.

Lengthening the cue-target interval:

Due to the ephemeral nature of exogenous attention (~120 ms [5,73], when the cue-target interval is lengthened to 500 ms, neutral and peripheral conditions do not differ.

Postcue:

When observers judge the stimulus followed by a postcue, rather than preceded by a precue, appearance is unaltered, notwithstanding the same spatio-temporal contiguity between cue and stimulus. Were signals integrated over time, the effect would be the same with precues and postcues.

Performance:

Appearance and performance in orientation discrimination or direction discrimination have been concurrently assessed and altered [12,28,36,37,47,53,58–61,70]. A cue-bias or a response-bias cannot explain the performance effect.

Visual dimension:

Precueing does not affect hue appearance, although it affects judgments of stimulus saturation [53], as well as of many other static and dynamic dimensions (Table 1). A cue-bias or a response-bias should affect all dimensions similarly.

Notwithstanding the converging results from all these control experiments, alternative explanations have been proposed for the effects of exogenous attention on perceived contrast, but these alternative explanations have been empirically invalidated [[74] but [36]; [75] but [31];[41] but [39,40]].

Peripheral cues were reported to increase perceived brightness only near detection threshold, and it was hypothesized that reversing the cue’s luminance polarity would lead to differential cueing effects on perceived contrast [74]. However, both black and white cues increase apparent contrast of suprathreshold stimuli to the same degree [36], thus ruling out sensory factors.

Location uncertainty has been invoked to explain increased perceived contrast [75]. Such explanation predicts the largest effect for the lowest stimulus contrast, but this is not the case [12,28,31–33,35–40,42]. Whereas location uncertainty is a relevant factor for near-threshold [75], it is irrelevant for suprathreshold [12,28,31,32,36–38,43] stimuli.

Lastly, a cue effect on appearance has been attributed to a decisional bias rather than to attention, and it was reported that there is no attention effect with an equality judgment [41,76]. However, comparing the sensitivity of equality and comparative judgments regarding physical contrast and attentional modulation revealed several methodological limitations that render the equality judgment less sensitive to shifts in perceived contrast [39,40]. Regardless, attention enhances apparent contrast with both comparative and equality judgments [34,39,40].

How does attention alter appearance?

An electrophysiological study confirmed that enhanced perceived contrast at the cued location is attributable to an attention effect on early visual processing. Cueing attention boosts early processing (100–140 ms) of the attended stimulus in the ventral occipito-temporal visual cortex; and the higher the boosting, the higher the perceived contrast [38]. The temporal dynamics and occipito-temporal location of this modulation are consistent with a boost in early sensory processing but not with decision-making processes [38,77]. To further explore the relation between perceived contrast and underlying neural responses, we varied stimulus contrast and modeled perceived contrast as a function of the underlying neural contrast-response functions. An increased input baseline in neural responses accounted for the attention effect on perceived contrast [32,34].

Changes in perceived spatial frequency may be due to heightened sensitivity of higher spatial frequency channels [9,46], which would change the overall pattern of activity across channels, resulting in the phenomenological experience of higher spatial frequency. Such preferential enhancement of high-spatial frequency neurons has been observed with arousal state in mice [78,79], as well as with covert [65,80] and presaccadic [81,82] attention in humans. A similar mechanism may explain increases in perceived flickering rate and apparent speed, which may be correlated with enhanced activity in neuronal populations tuned to higher speed or flickering rate, re-weighting the population response [56,59].

CONCLUSIONS

The appearance paradigms developed to assess the effects of exogenous and endogenous attention on appearance have revealed that attention alters our subjective impression of many dimensions of spatial [12,28,31,33,35–38,43,46,52,60,61,72,83] and temporal [9,35,47,56–58] vision, mediated by the ventral and dorsal streams, respectively. These findings have implications for models of visual attention, as well as for our understanding of the psychology and philosophy [84] of perception.

Attention effects on contrast appearance have been systematically investigated. Contrast appearance enhancement likely accompanies increased contrast sensitivity [11,66,85–90]. The finding that attention increases apparent contrast supports a linking hypothesis stating that the attentional enhancement of neural firing is interpreted as if the stimulus had a higher contrast. Consistent with the hypothesis that both changes reflect a common attentional mechanism modulating contrast sensitivity responses in early visual cortex [7,28–30,32,38], changes in performance and in perceived contrast correlate [37]. Converging evidence from neurophysiological, psychophysical, and neuroimaging studies supports this proposal [2,3,7,29,30]. Modeling the appearance task as a function of underlying neural contrast- response functions revealed that an increased input baseline in the neural responses accounted for the enhancement of perceived contrast with covert attention [32].

The visual system optimizes processing resources, often producing non-veridical percepts. Attention augments perception by altering stimulus representation and by emphasizing relevant information with the expense of a sketchy representation of less relevant information. The biophysical machinery of the brain engenders our phenomenological experience of the world: attention affects not only how we perform in a visual task but also how we see and experience our visual world.

Figure 1.

Visual appearance varies across the visual field. (a) An original photograph of the Brunnen der Lebensfreude in Rostock, Germany (left image). Using a pooling model, synthetic images (right image) can be generated to appear nearly identical to the original when viewed with fixation at the center (red dot), despite gross distortions in the periphery. Adapted from [14]. (b,c) Visual information appears different across isoeccentric locations; both perceived contrast and perceived spatial frequency are higher along the horizontal than the vertical meridian, and at lower than upper locations along the vertical meridian. Consequently, when viewed with fixation at the center cross, signals of different contrast (b) or different spatial frequency (c) appear to be identical [see 8,12,18]. Given that these effects depend on eccentricity, these figures illustrate qualitative differences.

Carrasco & Barbot - Highlights.

• covert attention improves sensory processing in many visual tasks

• covert attention also alters the appearance of visual information

• many spatial and temporal dimensions are perceived differently with attention

• attention effects in performance and appearance seem to share a common origin

• future studies should focus on understanding the underlying mechanisms of appearance effects