How Attention Affects Spatial Resolution

Abstract

We summarize and discuss a series of psychophysical studies on the effects of spatial covert attention on spatial resolution, our ability to discriminate fine patterns. Heightened resolution is beneficial in most, but not all, visual tasks. We show how endogenous attention (voluntary, goal driven) and exogenous attention (involuntary, stimulus driven) affect performance on a variety of tasks mediated by spatial resolution, such as visual search, crowding, acuity, and texture segmentation. Exogenous attention is an automatic mechanism that increases resolution regardless of whether it helps or hinders performance. In contrast, endogenous attention flexibly adjusts resolution to optimize performance according to task demands. We illustrate how psychophysical studies can reveal the underlying mechanisms of these effects and allow us to draw linking hypotheses with known neurophysiological effects of attention.

ATTENTION IS A SELECTIVE PROCESS

Each time we open our eyes we are confronted with an overwhelming amount of information. Yet, we seemingly understand our visual world effortlessly. To make sense of a scene, we need to detect, localize, and identify relevant information. By focusing on a certain location or aspect of the visual scene, attention allows us to selectively process information, prioritizing some aspects of information while ignoring others. Attention lies at the crossroads between perception and cognition, bringing together scientists using psychophysics, neurophysiology, neuroimaging, and computational neuroscience techniques. Significant advances in visual attention have been facilitated by fruitful cross talk among these fields and levels of analyses. The interest in visual attention has exponentially grown; a PubMed search yields more than 3500 articles dealing with visual attention since 1970 (“visual attention” in title or abstract), with half of them published since 2008 (Fig. 1). Changes in an observer’s attentional state while keeping the retinal image constant can affect perceptual performance and appearance, as well as the activity of “sensory” neurons throughout visual cortex.

Figure 1.

Cumulative numbers of publications reported by PubMed since 1970 containing the key word “visual attention” in either the title or the abstract.

Selective attention arises from the brain’s limited capacity to process information. The fixed amount of overall energy available to the brain and the high bioenergetic cost of the neuronal activity involved in cortical computation require the use of efficient representational codes that rely on a sparse collection of active neurons, as well as the flexible allocation of metabolic resources according to task demands. These energy limitations allow only a small fraction of the machinery to be engaged concurrently, and provide a neurophysiological basis for selective attention (Lennie 2003; Carrasco 2011). The notion that stimuli compete for limited resources has been long recognized (Broadbent 1958; Neisser 1967; Kinchla 1980, 1992) and supported by electrophysiological, neuroimaging, and behavioral studies (for reviews, see Desimone and Duncan 1995; Reynolds and Chelazzi 2004; Carrasco 2011, 2014; Beck and Kastner 2014; Posner 2014).

Attention optimizes the use of the system’s limited resources by enhancing representations of the relevant locations or features of our environment while diminishing the representations of less relevant locations or features. Attentional trade-offs emerge across different tasks and displays, including noncluttered displays, in which only two simple stimuli are competing for processing; the benefit at the attended location has a concomitant cost at unattended locations (Luck et al. 1994; Lu and Dosher 1998; Pestilli and Carrasco 2005; Anton-Erxleben et al. 2007; Pestilli et al. 2007; Giordano et al. 2009; Montagna et al. 2009; Herrmann et al. 2010; Yeshurun and Rashal 2010; Barbot et al. 2011, 2012a).

SPATIAL COVERT ATTENTION: ENDOGENOUS AND EXOGENOUS

Knowledge and assumptions about the world, the behavioral state of the organism, and the sudden appearance of possibly relevant information, facilitate the processing of sensory input. Attention can be allocated overtly, by moving one’s eyes toward a location, and covertly, by attending to a given location without directing one’s gaze toward it. Covert attention aids us monitor our crowded environment and inform subsequent eye movements to locations where relevant information is likely. We deploy covert attention in many everyday situations: searching for objects, walking, driving, dancing—as well as in social situations, to conceal intentions eye movements would reveal (e.g., in competitive sports). Attention affects performance and appearance in many tasks mediated by early visual dimensions—contrast sensitivity and spatial resolution—once considered to be preattentive (for reviews, see Carrasco and Yeshurun 2009; Carrasco 2011, 2014; Anton-Erxleben and Carrasco 2013).

Two types of covert attention facilitate selective processing of information: endogenous attention, which enables us to voluntarily monitor information at a given location, and exogenous attention, which allows us to involuntarily orient to a location where sudden stimulation has occurred. Observers deploy voluntary attention in ~300 msec and can sustain it as needed; the involuntary deployment of attention is transient, peaks at ~100–120 msec and decays quickly (Müller and Rabbitt 1989; Nakayama and Mackeben 1989; Remington et al. 1992; Ling and Carrasco 2006a,b; Liu et al. 2007; Koenig-Robert and Vanrullen 2011). Whereas observers can allocate resources according to cue validity when deploying endogenous attention, they cannot do so when deploying exogenous attention (Sperling and Melchner 1978; Kinchla 1980; Mangun and Hillyard 1990; Yantis and Jonides 1996; Giordano et al. 2009). Exogenous cues cannot be ignored: Involuntary transient shifts of attention occur even when the cues are known to be uninformative and irrelevant (Pestilli and Carrasco 2005; Pestilli et al. 2007; Montagna et al. 2009; Herrmann et al. 2010; Yeshurun and Rashal 2010; Barbot et al. 2011) and even when they impair performance (Yeshurun and Carrasco 1998, 2000, 2008; Talgar and Carrasco 2002; Yeshurun and Levy 2003; Yeshurun 2004; Carrasco et al. 2006b; Hein et al. 2006; Bocanegra and Zeelenberg 2011). The different temporal characteristics and degrees of automaticity suggest that exogenous attention may be phylogenetically older than endogenous attention, allowing us to respond automatically and quickly to stimuli that may provide behaviorally relevant information.

SPATIAL RESOLUTION

Vision is limited by many factors, including spatial resolution—our ability to discriminate fine patterns. The visual system’s spatial resolution is best at the fovea (the focus of our gaze) and decays toward the periphery. The systematic resolution differences across eccentricity are due to several factors, starting with the spacing of photoreceptors in the retina. The visual system is retinotopic and arranged hierarchically from the retina to early extrastriate areas. In the visual cortex, the neurons that receive projections from the fovea have the smallest RFs, and RF sizes increase both with eccentricity and along the visual pathway (Fig. 2A). Additionally, the number and density of RFs decreases and the spacing among them increases with eccentricity (Levi et al. 1985; Martin 1986; DeValois and DeValois 1988; Zhaoping 2014). Thus, as eccentricity increases, information is pooled over a larger area, diminishing sensitivity to fine patterns. The central part of the visual field is overrepresented. A greater proportion of cortex is devoted to processing input from the central part than the periphery (cortical magnification). In area V1, ~25% of cortex is devoted to processing the central 2.5° of visual angle (DeValois and DeValois 1988). Population receptive field (pRF) models provide a tool for quantitatively linking the signal evoked by a visual stimulus to the fMRI response at a point on the cortical surface. The pRF model accounts for both the most effective point in space that produces an fMRI response at a cortical site (pRF center) and the spatial extent of the visual field that a cortical site is sensitive to (pRF size). The pRF size is an important measure of visual encoding that varies systematically with eccentricity within a visual field map and between visual field maps (Fig. 2B) (Dumoulin and Wandell 2008; Wandell and Winawer 2015). Note the similarity between the two sets of measures.

Figure 2.

Schematic depiction of (A) receptive field (RF) size and (B) population receptive field (pRF) size as a function of eccentricity based on physiological measurements in macaque areas V1, V2, and V4, and fMRI measurements in human areas V1, V3, and hV4. The center of each array corresponds to the fovea. The size of each circle is proportional to its eccentricity, based on the corresponding scaling parameters. At a given eccentricity, a larger scaling parameter implies larger receptive fields. (A, Reprinted by permission from Macmillan Publishers Ltd. from Freeman and Simoncelli 2011; B, reproduced with the permission of Jonathan Winawer and Hiroshi Horiguchi; https://archive.nyu.edu/handle/2451/33887.)

RF size is correlated with spatial frequency tuning, which measures a neuron’s sensitivity to different spatial scales of variation in contrast. Thus, selectivity for spatial frequency also varies across the visual field. The visual system is most sensitive to higher spatial frequencies near the fovea, and its peak sensitivity shifts toward lower spatial frequencies with eccentricity (Rovamo et al. 1978; Wright and Johnston 1983; Kitterle 1986; Strasburger et al. 2011). Consequently, we can clearly see an object in the center of the field but cannot discriminate its fine details when it is in the peripheral field. Signals from the field’s center are processed more accurately and faster (Rijsdijk et al. 1980; Cannon 1985; Carrasco et al. 1995). In many tasks, these performance differences are eliminated when stimulus size is enlarged according to the cortical magnification factor, which equates the size of the cortical representation for stimuli at different eccentricities (Rovamo and Virsu 1979; Virsu and Rovamo 1979; Kitterle 1986; Carrasco and Frieder 1997; Carrasco et al. 1998a; Harvey and Dumoulin 2011), or when thresholds are expressed in distance along the V1 surface rather than in degrees of visual angle (Duncan and Boynton 2003; Pelli and Tillman 2008). However, compensating for cortical magnification does not eliminate qualitative differences in neural processing between central and peripheral vision (Kitterle 1986; Strasburger et al. 2011). Moreover, at isoeccentric locations performance is better along the horizontal than the vertical meridian, and in the lower than the upper region of the vertical meridian (Rovamo et al. 1982; Talgar and Carrasco 2002; Fuller et al. 2008; Montaser-Kouhsari and Carrasco 2009).

COVERT ATTENTION ALTERS SPATIAL RESOLUTION

To investigate covert attention, it is necessary to keep both the task and stimuli constant across conditions while manipulating attention. Psychophysical studies have shown that we can differentially engage endogenous and exogenous attention by using different spatial cues. For endogenous attention, a cue—typically a line or a symbolic cue (number or color) at fixation—indicates the most likely location of the upcoming target. For exogenous attention, a brief peripheral cue is presented adjacent to the upcoming target or distractor.

Research in peripheral vision has historically neglected the role of spatial attention (Strasburger et al. 2011). When spatial attention is directed to a given location, performance improves in tasks mediated by spatial resolution—visual search, crowding, acuity, and texture segmentation (unless resolution is already too high for the task; see below). These psychophysical studies provide evidence for the “resolution hypothesis” (Yeshurun and Carrasco 1998), which states that attention can enhance spatial resolution, and that the magnitude of such an effect increases with eccentricity. Moreover, they illustrate that systematic psychophysics studies can reveal the underlying attention mechanisms. In this section, we discuss studies that have contributed to our understanding of the effects of covert attention on performance in spatial resolution tasks. Figure 3 summarizes the findings reviewed below.

Figure 3.

Summary of the effects of exogenous and endogenous attention on spatial resolution tasks described in this paper.

Visual Search

To identify objects in a scene, the visual system first needs to isolate them. In search, we need to separate a target stimulus from distracting stimuli in the scene (e.g., when we are trying to find a friend in a crowd or a book on the bookshelf). The area over which the visual system integrates information limits performance in search tasks: The larger the integration area, the more likely it includes irrelevant information that interferes with target identification. In a search task, observers are typically required to detect the presence of a predefined target appearing among other irrelevant items; for instance, in a feature search task, a purple tilted line appearing among purple horizontal lines. In most real-life situations, however, we search for targets that differ from the distractors by a conjunction of different features—that is, the target shares a feature with a subset of distractors (e.g., its color) and another feature with another subset of distractors (e.g., its orientation), but the unique combination of the two features defines the target (e.g., a purple tilted line among purple horizontal and orange tilted lines; Treisman and Gelade 1980; Treisman 1985). Typically, for feature searches the number of distractors does not affect performance, but for conjunction searches performance decreases (longer reaction times and higher errors) with the number of distractors—the set-size effect. There are, however, feature searches yielding a set-size effect (Carrasco and Yeshurun 1998; Carrasco and McElree 2001; Carrasco et al. 2004, 2006a; Dosher et al. 2004) and conjunction searches that do not (Nakayama and Silverman 1986; McLeod et al. 1988; Duncan and Humphreys 1989; Carrasco et al. 1998b).

The set-size effect had traditionally been explained by a serial shift of attention to each item in a display (Treisman and Gelade 1980; Treisman 1985; Wolfe et al. 1989; Eckstein 2011). With more distractors, the target is more likely to appear at more eccentric positions. Consequently, the set-size effect can also be explained by the decrease of spatial resolution with eccentricity, as more eccentric RFs integrate over a larger area and include more distractors. Consistent with this spatial explanation, performance deteriorates with target eccentricity (Carrasco and Chang 1995; Carrasco et al. 1995, 1998a; Carrasco and Frieder 1997; Carrasco and Yeshurun 1998). Indeed, adjusting the stimulus size to its eccentricity according to the cortical magnification factor (to equate cortical surface activated by different stimuli) eliminates the eccentricity effect (Fig. 4A), indicating that resolution is a limiting factor in visual search (Carrasco and Frieder 1997; Carrasco et al. 1998a). Interestingly, the eccentricity effect is also practically eliminated for both features and conjunctions when, instead of magnifying the stimulus size, we manipulated exogenous attention (Fig. 4B) (Carrasco and Yeshurun 1998). These studies show that, similar to the effects of stimulus magnification, attention reduces the eccentricity effect by enhancing resolution, which in turn reduces the set-size effect.

Figure 4.

Effects of attention in visual search tasks. (A) Performance in conjunction visual search decreases with eccentricity, as depicted by slower reaction times and higher error rates. Adjusting stimulus size according to the cortical magnification factor at a given eccentricity eliminates the eccentricity effect in visual search, indicating that spatial resolution constrains performance. (B) Manipulating exogenous attention has a similar effect to cortical magnification, strongly reducing the eccentricity effect. These results support the idea that attention increases resolution at the attended location, restoring visual search performance in the periphery where low resolution is a limiting factor. (A, Adapted from Carrasco and Frieder 1997; B, adapted from Carrasco and Yeshurun 1998.)

Search studies assessing orientation thresholds also support the idea that attention enhances resolution. Orientation discrimination thresholds worsen with increasing number of distractors, but peripherally cueing the target location reduces orientation thresholds to the same level found when the target is alone (Morgan et al. 1998; Baldassi and Burr 2000). Together with these studies, findings that in discrimination, detection and localization tasks, both exogenous and endogenous attention not only reduce the set-size effect but also boost performance when the target is presented alone, support distractor exclusion and signal enhancement as mechanisms underlying attentional selection (Carrasco and McElree 2001; Cameron et al. 2004; Giordano et al. 2009).

Crowding

Another consequence of spatial resolution limits is crowding—a stimulus that is easily recognized when presented in isolation in peripheral vision can be unidentifiable when presented among nearby distractors. For example, you can identify a book on your shelf without looking at it directly, but it is a lot more difficult to recognize it when other books surround it. One of the dominant explanations for crowding posits that when we identify a target we pool information across several feature detectors over the “integration field” area (Pelli et al. 2004). Integration fields expand with eccentricity, increasing the probability that the integration process will include flankers into the target signal (Parkes et al. 2001). The critical distance in crowding increases with eccentricity, consistent with the increase of RF size. Accordingly, the crowding effect depends on a critical distance (center-to-center spacing between the target and the distractors) and its ratio with eccentricity (Levi 2008; Pelli and Tillman 2008). Exogenous attention affects the critical distance between the target and distractors: Valid cues decrease the critical distance whereas invalid cues increase it (Freeman and Pelli 2007; Yeshurun and Rashal 2010; Grubb et al. 2013). Similarly, endogenous attention decreases crowding (Montaser-Kouhsari and Rajimehr 2005). These results further support the idea that attention enhances spatial resolution.

Acuity

Investigating attention’s role on the processing of stimuli used to test visual acuity further supports the hypothesis that attention enhances resolution. The retinal mosaic limits detection of a small gap in a Landolt square, whereas cortical processes limit offset discrimination with Vernier targets, which is hyperacute (Levi et al. 1985; Olzak and Thomas 1986). Performance in acuity tasks decreases with eccentricity, consistent with decreased resolution in the periphery. Directing exogenous attention to the target location improves performance in both acuity (Yeshurun and Carrasco 1999; Carrasco et al. 2002; Golla et al. 2004; Montagna et al. 2009; Bocanegra and Zeelenberg 2011) and hyperacuity (Shiu and Pashler 1995; Yeshurun and Carrasco 1999) tasks, and the benefits intensify with eccentricity. These results were obtained even when the display contained no added sources of external noise (local masks, global masks, or distractors). Furthermore, effects of covert attention on Landolt acuity are very similar in humans and nonhuman primates (Fig. 5A) (Golla et al. 2004). Moreover, for both endogenous and exogenous attention, there is a resolution trade-off; increased spatial acuity at the attended location is coupled with decreased acuity at unattended locations, gap-size discrimination thresholds are lower in the cued and higher in the uncued condition, compared to the neutral baseline condition (Fig. 5B) (Montagna et al. 2009).

Figure 5.

Effects of attention in acuity tasks. (A) Attention improves performance in acuity tasks in both humans and non-human primates (macaques), resulting in lower acuity thresholds with attention. (B) For human observers, both exogenous and endogenous attention trade-off acuity, increasing acuity at the attended location at the cost of decreased acuity at unattended locations. (A, Adapted from Golla et al. 2004; B, adapted from Montagna et al. 2009.)

Given that attention cannot operate at the level of the retina, these findings imply that cortical or subcortical attentional processes can influence acuity by altering the spatial frequency tuning of the relevant spatial filters. Alternative hypotheses regarding attentional mechanisms, such as shifts in the decisional criterion and reduction of location uncertainty or reduction of external noise, could not account for these attention effects, because (1) the peripheral cue conveyed neither information regarding the target location; (2) observers could not rely on the cue for correct discrimination; (3) indistinguishable results emerge with and without a local postmask; and (4) one or two suprathreshold stimuli at fixed locations cannot be confused with the blanks at other locations. Thus, improved performance in acuity and hyperacuity tasks can only be accounted for by enhanced resolution.

Texture Segmentation

An attentional mechanism that increases spatial resolution by default can be useful because heightened resolution is advantageous in most everyday tasks, such as localizing and discriminating objects, searching for small objects, identifying fine details, and reading. In certain situations, however, resolution enhancement is not beneficial; for example, when navigating under poor atmospheric conditions (e.g., fog or haze) or when a global assessment of a scene is required (e.g., when seeing a whole tree rather than its individual leaves or viewing paintings using pointillism or photomosaic/pixellated [Fig. 6A] techniques).

Figure 6.

(A) Lincoln in Dalivision (1977) by Salvador Dalí. Depending on the dominance of high- or low-spatial frequency content of the image, the observer will perceive Gala’s body or Lincoln’s face (note small inserts on the lower left). (B) Texture segmentation task used in the studies described. Note that performance peaks at perifoveal locations and decreases at central locations (central performance drop; CPD), where resolution is too high for the task, as well as at peripheral locations, where resolution is too low.

The detrimental effect of heightened resolution enabled a crucial test of the resolution hypothesis: If attention enhanced resolution, performance at the attended location would be impaired rather than improved (Yeshurun and Carrasco 1998). The visual system performs a texture segmentation task whenever we process a scene, to assign foreground and background and to parse different objects. The area over which information is integrated, hence, also limits segmentation. We used a basic texture segmentation task in which the to-be-detected texture target is embedded in a background of an orthogonal orientation presented along the horizontal meridian (Fig. 6B). In this task, observers’ performance peaks at mid-peripheral locations, and drops when the target appears at more central or farther peripheral locations. This “central performance drop” (CPD) is attributed to the average size of spatial filters at the fovea being too small and spatial resolution being too high for the scale of the target patch. The filters’ average size increases gradually with eccentricity and is optimal around the performance peak. At farther locations, the filters are too big and the resolution is too low for the task and performance drops. Accordingly, enlarging or decreasing the texture scale shifts the performance peak to farther or more central locations, respectively (Joffe and Scialfa 1995; Gurnsey et al. 1996; Kehrer 1997; Yeshurun and Carrasco 1998; Kehrer and Meinecke 2003; Yeshurun et al. 2008).

Exogenous attention improves performance at the periphery where the resolution is too low, but impairs performance near the fovea where the resolution is already too high for the task, in a smaller or larger range of eccentricities depending on the texture scale (Fig. 7A) (Yeshurun and Carrasco 1998). Thus, the texture scale and the filters’ average size at a given eccentricity determine whether exogenous attention helps or hinders performance. Along the vertical meridian (VM), performance peaks at farther eccentricities in the lower than the upper VM (Fig. 8A), consistent with the higher resolution in the former than the latter (Rovamo et al. 1982; Talgar and Carrasco 2002; Montaser-Kouhsari and Carrasco 2009), but the attention benefit and cost are the same in both regions of the VM in relation to their crossover eccentricity (Fig. 8B) (Talgar and Carrasco 2002). These findings provide further evidence that the VM asymmetry is limited by visual rather than attentional factors (Carrasco et al. 2001) and that attention enhances spatial resolution. Note that the attentional impairment at central locations cannot be explained by shifts in the decisional criterion, reduction of location uncertainty, or reduction of external noise, which would predict a benefit throughout all eccentricities.

Figure 7.

Effects of attention in texture segmentation tasks. (A) Exogenous attention automatically increases spatial resolution, improving texture segmentation performance in the periphery where the resolution is too low and impairing performance at central locations where the resolution is already too high, for the scale of the texture. The vertical dashed lines indicate the eccentricity of the performance peak. The blue overlay shows a range of eccentricity in which exogenous attention has opposite effects improving performance (left) or impairing performance (right), depending on the scale of the texture. (B) Endogenous attention benefits performance across eccentricities, regardless of whether performance is limited by the resolution being too low or too high. The vertical dashed lines indicate the eccentricity of the performance peak. (A, Adapted from Yeshurun and Carrasco 1998; B, adapted from Yeshurun et al. 2008.)

Figure 8.

Changes in resolution with attention along the vertical meridian (VM). (A) Changes in texture segmentation along the vertical meridian and with exogenous attention. Spatial resolution is higher in the lower VM than the upper VM, resulting in higher performance in the periphery where resolution is too low and worse performance at central locations where resolution is too high for the scale of the texture. Similarly, by enhancing resolution, exogenous attention impairs and improves performance at central and peripheral locations, respectively. (B) Effects of exogenous attention on texture segmentation along the upper and lower VM. Exogenous attention impairs and improves performance across eccentricity consistent with enhanced resolution. Note that the attentional crossover (indicated by the colored area) occurs closer to the fovea along the upper (lower resolution) than the lower (higher resolution) VM, consistent with the idea that increasing resolution impairs or improves performance according to resolution constraints. (Adapted from Talgar and Carrasco 2002, with kind permission from Springer Science+Business Media.)

We investigated the adaptability of exogenous attention by examining whether the cue size modulates the attention effect on the resolution at the attended location. Employing endogenous cues of different sizes or dual tasks has revealed that the larger the attended region, the lower the resolution (Hock et al. 1998; Goto et al. 2001; Muller et al. 2003; Greenwood and Parasuraman 2004). We tested whether this would also be the case with exogenous attention, considered to be less flexible than endogenous attention. Were the gradual increase in cue size to result in a gradual resolution decrement, performance should gradually improve at central locations and deteriorate at peripheral locations, and the performance peak’s eccentricity and the CPD should continuously decrease. We replicated the attention enhancement of resolution with small cues, but there was no significant effect for larger cues. Thus, we found no evidence that exogenous attention can flexibly lower resolution when it is attracted to a broader spatial region by large cues, it either increases resolution or has no effect (Yeshurun and Carrasco 2008).

The effect of exogenous attention on texture segmentation also reveals an automatic resolution trade-off. Capitalizing on the fact that increased resolution can hinder performance in texture segmentation, we uncovered a seemingly counterintuitive situation in which decreasing resolution benefits performance at unattended locations. When the peripheral cues are invalid and divert attention away from the target location, performance decreases at peripheral locations but improves at central locations. The impairment at unattended peripheral locations where resolution is too low and the improvement at unattended central locations where resolution is too high can only be explained by decreased resolution at the unattended location (Barbot et al. 2013). All alternative hypotheses would predict a cost on performance at unattended locations throughout all eccentricities.

To assess the level of processing at which exogenous attention affects resolution, we used textures composed of narrow-band stimuli to ensure that first- or second-order filters of various specific scales would be differentially stimulated. Theoretically, texture segmentation involves passage of visual input through two layers of spatial linear filters, separated by a point-wise nonlinearity. The first-order linear filters perform a more local analysis of spatial frequency and orientation, and reflect the activity of simple cortical cells in area V1. The second-order linear filters are of a larger scale, perform a more global analysis on the output of the first-order filters plus the intermediate nonlinearity, and reflect the activity of cortical cells in striate and extrastriate areas (for reviews, see Landy and Graham 2004; Landy 2013). Accuracy is higher for cued than neutral trials at peripheral eccentricities, but is lower at central locations, for both first-order low- and high-frequency conditions. In contrast, for the second-order spatial frequency content attention impairs performance in a larger eccentricity range for the low- than the high-frequency condition, and an attention benefit emerges only for the high-frequency condition. These findings are consistent with the idea that attention operates by reducing the size of the second-order filters in texture segmentation tasks (Yeshurun and Carrasco 2000). Note, however, that a similar performance pattern with eccentricity has been observed using first-order, luminance-defined target patches, indicating that the CPD is not exclusive to the second-order stage of processing (Kehrer and Meinecke 2006; Meinecke and Kehrer 2007).

To test the hypothesis that exogenous attention enhances spatial resolution by increasing sensitivity to high-spatial frequencies, we used a selective adaptation procedure in conjunction with peripheral cueing. This procedure is used to assess the visual system’s spatiotemporal properties (Blakemore and Campbell 1969; Movshon and Lennie 1979; Graham 1989). While keeping the stimulus content identical, the availability of spatial frequency information is manipulated by reducing observers’ sensitivity to a range of frequencies. Selective adaptation to high-SF filters should reduce observers’ sensitivity to high-SF, shift sensitivity toward lower frequencies, displace the performance peak toward the fovea, and reduce the CPD. Conversely, adapting to low-SF should reduce observers’ sensitivity to low-SF, shift sensitivity toward higher frequencies, displace the performance peak toward the periphery, and exacerbate the CPD (Fig. 9A). Furthermore, adapting to high-spatial frequencies should eliminate the attentional impairment observed with exogenous attention at central locations (Fig. 9B). Experimental results show that the CPD is primarily due to the predominance of high-spatial frequencies at central locations, and that exogenous attention automatically enhances resolution by increasing sensitivity to higher spatial frequencies (Carrasco et al. 2006b). Consistent with this finding, a classification image study showed that with exogenous attention, the perceptual templates become sharper and are characterized by stronger high-spatial frequency components (Megna et al. 2012). In addition, both exogenous (Gobell and Carrasco 2005) and endogenous (Abrams et al. 2010) attention increased perceived spatial frequency at the attended area.

Figure 9.

Schematic depiction of the effects of spatial frequency adaptation on texture segmentation performance and visual attention effects. (A) Effects of selective adaptation to spatial frequencies (SF) at central locations on texture segmentation. Adapting to high-SF shifts sensitivity toward lower SF, reducing the CPD and shifting the peak closer to the fovea, similar to a decrease in resolution. Conversely, adapting to low-SF shifts sensitivity toward higher-SF, increasing the CPD and shifting the peak toward the periphery similar to an increase in resolution. (B) Adapting to high SF, but not low SF, reduces the effects of exogenous attention at central locations, suggesting that exogenous attention enhances resolution by increasing the contribution of the high-SF filters. (C,D) Endogenous attention may benefit performance at central locations either by (C) enhancing activity in the low-SF filters or (D) suppressing activity in the high-SF filters. Adapting to high SF, but not to low SF, reduced the attentional benefits supporting a mechanism that can either boost or suppress the high SF to optimize resolution to the task demands.

The texture segmentation studies above described show that exogenous attention increases resolution even when it is detrimental to the task. Given that endogenous attention is allocated more flexibly according to task demands, we compared its effects with those of exogenous attention. Conversely to exogenous attention, for two different texture scales, endogenous attention improves performance at all eccentricities (Fig. 7B) (Yeshurun et al. 2008). Furthermore, comparing the effects of precues and postcues revealed that the precue benefit goes well beyond the mere effect of location uncertainty at the decisional stage. The improvement at all eccentricities supports the idea that endogenous attention is more flexible than exogenous attention, and suggests that endogenous attention may increase resolution at peripheral locations but decrease resolution at central locations to match task demands (Yeshurun et al. 2008).

To characterize how endogenous attention affects resolution at central locations, we used a selective adaptation paradigm to modulate spatial frequency sensitivity at central locations (CPD). We hypothesized two attentional mechanisms that can decrease spatial resolution to benefit performance at central locations: by increasing the sensitivity of the low-spatial frequency filters or by decreasing the sensitivity of the high-spatial frequency filters. Accordingly, selectively adapting to either the low-spatial or high-spatial frequency filters at central locations should decrease their availability and suppress the attention effect (Fig. 9C,D). Experimental results show that adapting to high-spatial frequencies diminishes the benefit of endogenous attention at central locations, pointing to high-spatial frequency filters as responsible for the benefit of endogenous attention at central locations (Fig. 9D; Barbot et al. 2012b). Therefore, high-spatial frequency filters mediate both endogenous and exogenous attention effects on resolution, but whereas exogenous attention reflexively increases their contribution, endogenous can flexibly modulate their contribution to modify resolution according to task demands.

Linking Neurophysiology and Behavior

Physiologically, attention modulates neuronal responses and alters the profile and position of receptive fields near the attended location. Changes in RF size and position can qualitatively account for the aforementioned behavioral effects. Anton-Erxleben and Carrasco (2013) proposed a linking hypothesis based on two neural mechanisms. First, by concentrating processing resources at the attentional focus, RF shift toward the focus of attention (Connor et al. 1996, 1997; Ben Hamed et al. 2002; Womelsdorf et al. 2006; Anton-Erxleben et al. 2009) could improve performance in search, acuity, hyperacuity, and texture segmentation tasks. Second, RF shrinkage (Womelsdorf et al. 2006; Anton-Erxleben et al. 2009) could improve performance by reduction of filter size and integration area. The combination of RF shift and RF shrinkage leads to more and smaller RFs at the attended location, and thus to better resolution. The ability of smaller RFs to resolve finer details is correlated with a reduction of the area over which a single RF integrates information. Thus, attention could improve performance in search and crowding tasks, in which the integration area is critical to isolate a target from nearby distractors.

In texture segmentation tasks, the increase in spatial resolution can be explained by RF shrinkage, or by stronger weighting of small, high-spatial frequency-selective RFs (Yeshurun and Carrasco 2000; Carrasco et al. 2006b). The improved performance at central locations with endogenous attention seems to be mediated by decreased resolution (Figs. 7B and 9D), achieved by enlarging RFs or by decreased weighting of high-pass relative to low-pass filters (Yeshurun et al. 2008; Barbot et al. 2012b). RF expansion has been reported when attention is directed to a stimulus next to the RF (Anton-Erxleben et al. 2009) and in attentive tracking tasks (Niebergall et al. 2011). Additionally, attention’s modulation of center-surround interactions (Anton-Erxleben et al. 2009; Sundberg et al. 2009; Schwartz and Coen-Cagli 2013) could result in finer or coarser perceptual analysis.

Attention also affects resolution at the population level in human fMRI studies. Directing attention to a particular location decreases the spatial overlap for adjacent locations in BOLD responses, indicating a narrowing of the population’s integration area and increased resolution (Fischer and Whitney 2009). Conversely, withdrawing attention from the periphery results in larger pRFs and blurrier representations (de Haas et al. 2014), consistent with decreased resolution at unattended locations (Montagna et al. 2009; Barbot et al. 2013). Furthermore, attention attracts pRFs toward the focus of attention across the visual field and throughout the visual system (Klein et al. 2014). Inspired by psychophysical and neurophysiological findings, several computational models have implemented ways in which spatial attention can increase resolution (Lee et al. 1999; Deco and Zihl 2001; Cutzu and Tsotsos 2003; Compte and Wang 2006; Womelsdorf et al. 2008; Miconi and VanRullen 2011; Baruch and Yeshurun 2014). For example, some have proposed that the attentional effects on RFs can best be explained by a combination of attentional modulation of feed-forward connections with reciprocal modulatory feedback and local inhibition (Miconi and VanRullen 2011). But others have proposed a feed-forward model in which attention changes the gain of inputs to the RF, and a multiplicative interaction between the baseline RF and the attentional modulation results in a Gaussian profile that is narrower and shifted toward the attentional focus (Womelsdorf et al. 2008).

CONCLUSION

Covert attention enables us to better resolve fine details at attended parts of the visual scene, thus overcoming limitations in processing and partially restoring performance in the periphery. When attending to a particular location, observers’ performance improves in a variety of tasks mediated by spatial resolution, such as search, crowding, acuity, and texture segmentation. Endogenous attention is a flexible mechanism that adjusts its operation on resolution to meet task demands and optimize performance. Conversely, exogenous attention is automatic and increases resolution even when detrimental for the task. Attention enhances the visual system’s effective resolution by concentrating neuronal resources and reducing the area of spatial integration at the attended locations. Consistent with a selective representation of the world, there are processing trade-offs in resolution at the attended and unattended locations. As a selective process that optimizes relevant details, attention provides an organism with a heightened representation of the sensory input.