Attentional enhancement of spatial resolution: linking behavioural and neurophysiological evidence

Abstract

Attention allows us to select relevant sensory information for preferential processing. Behaviourally, it improves performance in various visual tasks. One prominent effect of attention is the modulation of performance in tasks that involve the visual system’s spatial resolution. Physiologically, attention modulates neuronal responses and alters the profile and position of receptive fields near the attended location. Here, we develop a hypothesis linking the behavioural and electrophysiological evidence. The proposed framework seeks to explain how these receptive field changes enhance the visual system’s effective spatial resolution and how the same mechanisms may also underlie attentional effects on the representation of spatial information.

The visual system constantly has to solve a variety of problems to make sense of a visual scene, and in order to do this, we need to detect, identify and localize relevant information. Vision is limited by many factors such as visibility and conspicuity. One of the most important of these factors is the visual system’s spatial resolution. Spatial resolution (also called acuity), is the ability to discriminate two nearby points in space. It is generally constrained by the spacing of photoreceptors in the retina (the retinal mosaic) as well as the size, spacing and number of receptive fields (RFs) along the visual pathway^1,2 (BOX 1).

Box 1. Receptive fields.

The receptive field (RF) of a neuron is defined as the part of stimulus space within which a stimulus elicits a response from the neuron¹³¹. In the visual system, the neuron responds to a stimulus presented in a region of space in the visual field (that is, its RF) but not to the same stimulus when it is presented outside this region.

A given stimulus typically elicits the strongest response from the centre of the RF, with the response gradually declining as the stimulus is presented further away from the centre of the RF. Thus, the RF can be well described by a two-dimensional Gaussian distribution. The left panel of part a of the figure shows an RF (orange) map derived from responses of a hypothetical neuron to a stimulus presented at different locations along x and y. The right panel of part a shows the RF profile along the black dashed line from the left panel. The RF can be viewed as a neuron’s spatial tuning curve, which is conceptually similar to any other tuning curve for a feature dimension that measures the variation of a neuron’s response along stimulus properties such as orientation or motion direction, with the highest response for the preferred and the lowest response for the non-preferred stimulus. Correspondingly, a spatial tuning curve for populations of neurons measures the response of the population to a stimulus at various spatial positions and is composed of the spatial tuning curves of the single neurons that are part of the population.

Each RF at one stage of the visual hierarchy is composed of input from many neurons of the previous stage; thus, RF size increases along the visual hierarchy¹³². Because a given neuron only ‘sees’ what is inside its RF and it cannot resolve variations of a stimulus within the RF, the size of RFs is an important factor that influences the spatial resolution achieved by a cortical area. Another factor is the overlap of RFs: a population of neurons can encode the location of a stimulus via different response levels of neurons with slightly offset RFs. Part b of the figure shows RF profiles of a set of five neurons with different RF locations. Horizontal dashed lines indicate the response of these five example neurons to two stimuli at nearby locations (vertical green and purple lines). Both stimuli fall into the same RF (middle grey curve), but they stimulate neurons with neighbouring RFs differently so that the population can resolve the two locations even though a single neuron cannot. In addition, the size of the RF determines the neuron’s spatial frequency tuning: the smaller the RF, the higher the spatial frequency it can resolve.

Spatial resolution varies systematically across the visual field, with the highest resolution at the focus of our gaze (that is, at the fovea). The visual system is arranged hierarchically from the retina up to early extrastriate areas, and mapping from one stage to the next is retinotopic, but the central part of the visual field is overrepresented (BOX 2). In the visual cortex, RF sizes increase with eccentricity, with the neurons that receive projections from the fovea having the smallest RFs. In addition, the number of RFs decreases and the spacing among them increases with eccentricity. Thus, the visual system’s spatial resolution and perceptual sensitivity is best at the fovea and decays towards the periphery. RF size is correlated with spatial frequency tuning, which measures a neuron’s sensitivity to different spatial scales of variation in contrast (that is, areas of relative light and dark in the visual scene). Thus, selectivity for spatial frequency also varies across the visual field, and the visual system is most sensitive to higher spatial frequencies closer to the fovea^3–5.

Box 2. Spatial resolution across the visual field.

Spatial resolution in the visual system monotonically declines with increasing distance from the fovea. A number of mechanisms at different levels of the visual hierarchy are responsible for this, and they are listed below.

Retinal receptor density

Whereas cone density is highest in the fovea and decreases sharply with eccentricity, rod density peaks at about 20 degrees eccentricity and decreases both towards the centre of the fovea, which contains no rods at all, and towards the periphery. Thus, under daylight levels of illumination, a stimulus presented in the centre of the visual field activates many more photoreceptors than the same stimulus presented in the periphery.

Ganglion cell density

The density of ganglion cells in the retina decreases logarithmically with eccentricity so that the same number of ganglion cells covers a strip between 1–2 degrees and 10–20 degrees eccentricity¹³³.

Mapping of photoreceptors

The mapping of photoreceptors to retinal ganglion cells has a finer resolution near the fovea than in the periphery; a smaller number of photoreceptors is connected to each ganglion cell in the central than in the peripheral visual field¹³³.

Receptive field size and density

Greater spatial pooling in the peripheral visual field is not restricted to the retina; in many cortical areas, receptive field (RF) size increases linearly with eccentricity, whereas RF density decreases. Because RF size determines spatial frequency selectivity with smaller RFs tuned to higher spatial frequencies, peak spatial frequency sensitivity changes from central to peripheral vision¹³³.

Cortical magnification

In the lateral geniculate nucleus¹³⁴ and many cortical visual areas^135–137, a greater proportion of neuronal resources is devoted to processing the input from the central visual field than from the periphery. For example, in area V1, the first stage in the cortical hierarchy to receive inputs from the retina, approximately 25% of cortex is devoted to processing the central 2.5 degrees of visual angle¹³³.

We can clearly see objects in the centre of the visual field but cannot discriminate fine details of the same object when it falls into the peripheral visual field. Therefore, we usually make eye movements (saccades) to the object (or objects) that have drawn our attention. Such an attention shift accompanied by an eye movement is called overt attention. Once we move our eyes to fixate the stimulus of interest, a different part of the retina — the fovea — and thus a different group of neurons with small RFs respond, enabling fine spatial resolution.

Sometimes, however, an eye movement is not possible or desirable, such as in certain social situations. For example, when you run into someone at a conference and you have forgotten their name, you might want to look at their name tag (FIG. 1a), but that would reveal that you forgot the person’s name. Surprisingly, we are able to shift attention covertly to the periphery of our visual field without making an eye movement. This covert shift in spatial attention results in enhanced visual processing, including enhanced spatial resolution at the attended location, thus allowing us to keep our gaze at the other person’s face while reading the name tag (FIG. 1b). In addition to improving spatial resolution^6–8, covert attention enhances various other aspects of visual performance^9–12, such as contrast sensitivity^13–15, speed of information accrual^16–18 and grouping¹⁹.

Figure 1. Allocation of spatial attention.

a | Overt attention. We shift our gaze to the location of interest (the name tag). b | Covert voluntary (endogenous) attention. We shift attention covertly to the location of interest while keeping our gaze somewhere else (at the person’s face). c | Covert involuntary (exogenous) attention. While we keep our gaze straight ahead, an external event (the falling coffee cup) leads to an involuntary shift of attention to the location of the event.

We use covert spatial attention in many other everyday situations — such as driving, crossing the street and playing sports — whenever we have to monitor the periphery of our visual field selectively. Often, we shift covert spatial attention first and then may make an eye movement to further inspect the object that caught our interest^9,20–22.

Covert spatial attention

At any given moment, our visual system receives an overwhelming amount of information. Owing to the high energy cost of neuronal activity, only a very small fraction of cortical neurons can be active (above base level) at a given time²³, and we cannot process all this information equally well. We use overt or covert spatial attention to select a relevant location of the visual field for enhanced processing with high resolution. Covert attention can be allocated voluntarily or captured involuntarily to selectively enhance perception at relevant locations in the periphery without moving the eyes. Reading a person’s name tag while keeping your gaze at their face is an example of a voluntary, endogenous shift of covert attention (FIG. 1b). If, for example, while you are talking, someone else in the room drops their coffee cup, your attention might be automatically drawn to the location of this event (FIG. 1c). Such sudden movements (and sounds) are typical events that draw involuntary, exogenous attention in daily life. Often, this is distracting, but it also serves the purpose of alerting us to locations at which important events might occur.

In most instances, the perceptual consequences of both types of attention are similar, but the temporal dynamics are different, with the exogenous system being faster than the endogenous one^9,20. In an endogenous attention paradigm, spatial attention is typically guided by a central or symbolic cue presented at fixation (at the fovea) indicating the most likely location of the subsequent target. Central cues are often small lines that point to particular locations in the periphery, whereas symbolic cues can be different numbers or coloured shapes that indicate different locations that the observer is asked to attend. In an exogenous attention paradigm, a peripheral cue — typically a dot or a small bar — is briefly presented adjacent to either the target or a distractor location just before display onset. Peripheral cues grab exogenous attention in an automatic, stimulus-driven manner and are typically effective within a time window of 80–130 ms after the cue onset, whereas the allocation of endogenous attention is slower and needs about 300 ms to be effective^6,9,24–26. For endogenous attention, the magnitude of the effect scales as a function of how predictive the cue is of the upcoming target location (cue validity); more informative cues result in stronger effects²⁷. By contrast, for exogenous attention, the cue need not be predictive of the subsequent target location. Because of the quick onset and transient nature of exogenous attention, the effects are similar regardless of how informative the cue is²⁷.

Covert spatial attention improves performance and accelerates the processing speed of various tasks, many of which are mediated by spatial resolution and contrast sensitivity, at the attentional focus^{6–17,28–33} at the cost of information from unattended locations^{9,12,27,34–38}. Physiologically, covert spatial attention selectively modulates firing rates of neurons with RFs overlapping the attended location in the visual field^39–45 and is also evident in spatially selective modulation of blood-oxygen-level-dependent (BOLD) responses^15,46–53. Covert attention can thus help overcome the limits of peripheral vision by enhancing visual processing in general, and by increasing spatial resolution in particular, at the attended location.

The hypothesis that attention modulates spatial selectivity and integration of sensory information by the visual system was proposed over 25 years ago³⁹, and recent neurophysiological and computational studies provide further data supporting this hypothesis^54–61. In parallel, behavioural studies have shown that attention modulates performance in numerous types of spatial resolution tasks^{7,8,29,62–69} and alters perception of spatial properties, such as spatial frequency^70,71, spatial separation between objects⁷², the size of objects⁷³ and the shape of objects⁷⁴. It has been suggested that these changes in spatial resolution and perception involve changes in spatial tuning — that is, changes in RFs (BOX 1) — at the neuronal level, but direct evidence for a link has remained elusive. In this Review, we provide an overview of recent behavioural evidence that demonstrates in detail the ways in which attention can modulate spatial resolution, discuss proposed neurophysiological mechanisms underlying changes in RF size and propose a hypothesis that explains how the latter underlies the former.

Attention alters spatial resolution: behaviour

As mentioned above, many of the tasks that the visual system has to solve are limited by spatial resolution. The effect of attention on spatial resolution has been systematically studied with human observers in these three types of spatial resolution tasks: visual search, acuity and texture segmentation.

Visual search and spatial interference

In order to identify objects embedded in a visual scene, the visual system first needs to isolate them. In visual search, we need to separate a target stimulus from distracting stimuli in the visual scene: for example, when we are trying to find a book on the bookshelf. Thus, the area over which the visual system integrates information limits performance in visual search tasks: the larger this integration area is, the more likely it is that it includes irrelevant information that interferes with target identification.

In the simplest case, we search for a target that differs from distractors by a single feature, such as its orientation (for example, a purple vertical line among purple tilted lines in the left panel of FIG. 2a). In most real-life situations, however, we search for targets that differ from the distractors by a conjunction of different features — that is, the target shares a feature with a subset of distractors (for example, its colour) and another feature with another subset of distractors (for example, its orientation), but the combination of the two features (orientation and colour) makes the target unique (for example, a purple vertical line among purple tilted lines and green vertical lines in the right panel of FIG. 2a)^75,76. For conjunction search, performance (as measured, for example, by reaction time or error rate) typically decreases with the number of distractors (the set-size effect (FIG. 2b))^75–78, whereas for feature search it does not, although there are exceptions to both^63,79–89.

Figure 2. Behavioural evidence of attention effects on spatial resolution.

a | Typical search task display for feature and conjunction search. In a feature search, observers report a target that is defined by a single feature (for example, a vertical line among tilted lines), whereas in a conjunction search, the target is defined by a combination of features (for example, a purple vertical line among purple tilted lines and green vertical lines). b | When the number of distractors (set-size) increases, there are more possible target locations at farther eccentricities. Performance deteriorates with eccentricity and thus with set-size. Attention improves performance (both speed and accuracy) and reduces both the eccentricity and the set-size effect. c | Possible mechanism for effects of attention in a visual search task: without attention, information from the target and distractors is integrated where spatial resolution is low. Attention effects could be mediated by a reduction of the integration area (orange) so that the influence of distractors is diminished. d | Attention improves acuity as a function of eccentricity. Attention could improve performance by reducing spatial integration, so that the visual system is better able to resolve the gap in the Landolt stimulus (insets). e | Texture segmentation performance typically peaks at mid-peripheral locations and drops at nearer as well as farther eccentricities (blue line). Attention modulates performance (red line) consistent with a change in optimal filter size; performance improves where resolution is too low (in the periphery) but declines where resolution is too high (centrally). f | Texture stimulus with filters (orange) that are too small, optimal or too large depending on target eccentricity (top panels). Attention may reduce the filter size so that the eccentricity at which the filter size is optimal shifts to farther eccentricities (bottom panels). Part b is modified, with permission, from REF. 63 © (1998) American Psychological Association. Part d is modified from REF. 65 © (2002) Association for Research in Vision and Ophthalmology. Part e is modified, with permission, from REF. 7 © (1998) Macmillan Publishers Ltd. All rights reserved.

The set-size effect had traditionally been explained by a serial shift of attention to each item in a display^75,76,78,89. However, with more distractors, the target is more likely to appear at more eccentric positions. Consequently, because of the lower spatial resolution of the visual system at farther eccentricities, the set-size effect could also be explained by the decrease of spatial resolution with eccentricity^83,84,86. Performance could decrease with target eccentricity because more eccentric RFs integrate over a larger area and therefore include more distractors. Consistent with this spatial resolution explanation, performance deteriorates with the eccentricity of the target^83,84,86 (FIG. 2b). Indeed, when the size of the items was adjusted to their eccentricity according to the cortical magnification factor (CMF; also called M-scaling), performance no longer depended on target eccentricity, suggesting that spatial resolution is a limiting factor in visual search^83,84.

When attention was directed to the target location in a visual search task, performance was improved⁶³ (FIG. 2b). Similar to adjusting stimulus size according to the CMF^83,84, attention reduces both the eccentricity effect and, accordingly, the set-size effect⁶³. Attention could improve performance, especially at farther eccentricities, by shrinking RFs and thereby reducing the integration area (FIG. 2c). This suggests that attention can reduce the performance difference between the fovea and the periphery by enhancing spatial resolution⁶³.

Along similar lines, another consequence of spatial resolution limits is that, in peripheral vision, a stimulus that is easily recognized when presented in isolation can be unidentifiable when presented among nearby distractors (crowding); for example, you can easily read the title of a book on your shelf without looking at it directly if it is the only book you have, but it is a lot more difficult if it is surrounded by other books. One explanation for crowding is that performance is impaired when the target and the distractor (or distactors) fall into the same ‘integration field’ and thus information is pooled across target and distractor locations. The crowding effect therefore depends on a critical distance (centre-to-centre spacing between the target and the distractors)⁹⁰. Similar to visual search performance, the extent of crowding is probably related to the size of RFs. Consistent with the increase of RF size, the critical distance in crowding increases with eccentricity.

Attention can improve performance in crowded displays to a similar level of performance attained when a stimulus is presented by itself^28,31,32. Most importantly, attention reduces the critical distance between the target and distractors⁶⁴. As in visual search tasks, attention might improve performance by reducing the integration area, possibly by shrinking RFs, which would increase the ability to isolate the target from the distractors. Thus, these results further support the idea that attention enhances spatial resolution.

Acuity tasks

To more directly test whether attention indeed enhances spatial resolution, a series of studies investigated the effects of attention on the processing of stimuli specifically designed to test visual acuity, such as a Landolt stimulus. Remarkably, attention improves detection of a small gap in a Landolt stimulus or a broken line^8,29,65–67 (FIG. 2d).

Consistent with the lower spatial resolution in the periphery, performance in acuity tasks decreases with eccentricity. The attentional benefit increases with eccentricity, suggesting that it operates by enhancing spatial resolution; in the periphery, where the resolution is lowest, there is more room for attentional improvement, therefore the effect is larger^8,65. In addition to a benefit at the attended location, there is a cost at unattended locations, which is consistent with a decrease of spatial resolution by the withdrawal of attention²⁹.

As attention cannot possibly operate at the level of the retina, these findings imply that cortical or subcortical processes can influence acuity by altering the spatial frequency tuning of the relevant spatial filters^8,29,65,67. For example, concentrating smaller RFs, which are tuned to higher spatial frequencies (BOX 1), would improve acuity at the attentional focus (compare top and bottom insets of FIG. 2d). Note that the same cortical processes could also be responsible for the improvement of performance by attention in Vernier tasks^6,8,91–93, in which spatial resolution is better than predicted by photoreceptor spacing (‘hyperacuity’) and therefore must be enhanced by cortical processes^1,94–96. The finding that attention enhances acuity is in line with a computational model postulating that spatial attention sharpens tuning of spatial filters for orientation and spatial frequency⁹⁷.

Texture segmentation

Perhaps the strongest evidence for a spatial resolution explanation of attentional effects on visual performance comes from studies using texture segmentation tasks. The visual system performs a texture segmentation task whenever we process a visual scene: for example, in order to assign fore- and background and to distinguish different objects. Texture segmentation, thus, is also limited by the area over which information is integrated — that is, by spatial resolution.

Experimentally, texture segmentation is tested using tasks in which observers are required to isolate a pattern within a larger texture: for example, a patch of oriented lines embedded in a larger array of differently oriented lines (FIG. 2f). Performance in this task peaks at mid-peripheral locations and drops at more peripheral locations, as well as at central locations. This is known as central performance drop (CPD), which is representred by the blue curve in FIG. 2e. This pattern of performance is attributed to the match between the spatial extent of the texture target and the average size of spatial filters at that eccentricity^98–101, which is related to the size and preferred spatial frequency of RFs (BOX 1). Performance peaks at eccentricities where spatial filters are of optimal size for the texture target and drops peripherally where filters are too large. It also drops centrally where filters are too small for the scale of the texture (see the top panel of FIG. 2f). Consistent with this explanation, the eccentricity of the performance peak moves outwards with a larger texture scale and towards the fovea with a smaller texture scale^7,98,102.

When exogenous attention is drawn to the texture target location, performance is improved at eccentricities farther than the performance peak, where filters were too large, but impaired near the fovea and at eccentricities closer than the performance peak, where filters were too small for the scale of the texture^7,103 (indicated by the red curve in FIG. 2e). The range of eccentricities with performance impairment depends on the scale of the texture⁷.

A selective loss of sensitivity of small, high-frequency selective filters should alleviate the CPD by shifting the weight of available filters to larger ones. Indeed, either removing high spatial frequency information from the display¹⁰⁴ or selectively adapting to high but not low spatial frequency⁶⁹ eliminates the CPD, indicating that filters selective for high spatial frequencies mediate the effects on performance at central locations. Eliminating the influence of these high-frequency-selective filters also eliminates the central attentional impairment, suggesting that it is mediated by high-frequency filters⁶⁹. The attentional modulation of performance could be explained by a shrinkage of RFs: when attention reduces RF size, the point of optimal filter size shifts outwards to farther eccentricities (see the bottom panel of FIG. 2f). Alternatively, it could be explained by stronger weighting of high-frequency filters over low-frequency filters by attention^68,69.

In these studies, exogenous attention seemed to default to enhancing spatial resolution even when it was detrimental to the task at hand. Therefore, one may wonder whether endogenous attention, as a more flexible means of attentional allocation^27,105,106, would do the same. It turns out that endogenous attention improves performance at all eccentricities, suggesting that it can flexibly adjust filter size¹⁰², thus increasing or decreasing spatial resolution depending on the current behavioural goal.

Attention alters appearance of spatial features

The effects of attention on performance in spatial resolution tasks are accompanied by changes in the subjective experience of spatial features. For example, attention increases perceived spatial frequency^70,71 (FIG. 3a). This effect can be explained by a selective increase in the sensitivity of the smallest RFs — that is, those that are tuned to high spatial frequencies (BOX 1) — at the attended location^70,71,92 (FIG. 3b). The same stimulus would then activate higher spatial frequency-selective neurons more strongly, shifting the weight of the population response. The population response would be as if a stimulus of higher spatial frequency had been presented, and the perceived spatial frequency would be increased.

Figure 3. Attention alters perception of spatial stimulus features.

a | The same physical stimulus appears to have higher spatial frequency with attention (right panel) than without attention (left panel) allocated to it. b | Attention may modulate perceived spatial frequency by a shift of sensitivity to higher spatial frequencies. Spatial frequency tuning curves of a set of neurons with different preferred spatial frequencies without attention are shown in blue. Horizontal dashed lines indicate the response of two example neurons to a stimulus of a particular spatial frequency (vertical black line). Spatial frequency tuning curves and response levels of the same set of neurons after attention has selectively up-modulated those neurons tuned for higher spatial frequencies are shown in red. The same stimulus now elicits a higher response from the example neurons (red dashed lines) and others tuned for the higher spatial frequency (shown as darker red curves on the right side of the graph) and the population response is biased towards higher spatial frequencies. c | Attention increases perceived offset between two lines of a Vernier stimulus after a pair of cues are flashed diagonally (for example, in the upper left and lower right quadrant of the middle panel), which is consistent with perceptual repulsion (indicated by arrows) of the lines away from the cues. d | The same physical stimulus appears larger (indicated by arrows) after attention has been drawn to its centre. e | The same oval stimulus appears either stretched out along the vertical or horizontal dimension after a pair of cues is either flashed inside or outside the contour, consistent with perceptual repulsion of the line away from the cues. f | Illustration of how RF shifts could mediate an increase in perceived size. Without attention, a regular grid of receptive fields (RFs) covers the visual scene (middle left panel). Attention shifts RFs towards the focus of attention, in this case the centre of a banana piece (middle right panel). The shift is accompanied by a shrinkage at the attentional focus and an expansion around it. Because RFs are still labelled with their original position, stimuli are perceptually pulled away from the centre of the attentional focus, so that the banana appears larger (far right panel). Note that the effect sizes depicted here are exaggerated for clarity. Part a is modified, with permission, from REF. 71 © (2005) Sage. Part b is modified from REF. 70 © (2010) Springer. Part c is modified, with permission, from REF. 72 © (1997) American Psychological Association. Part d is modified from REF. 73 © (2007) Association for Research in Vision and Ophthalmology. Part e is modified, with permission, from REF. 74 © (2011) Springer. The images in part f are courtesy of K.A.-E.

In line with enhanced spatial resolution, focused attention enables more precise position coding⁵⁶ and improves localization performance in some tasks^28,107,108. However, attention can also alter perceived position information: attention elicits perceptual repulsion of the two lines of a Vernier stimulus from the attended location^72,109 (FIG. 3c). Drawing attention to the centre of a circular stimulus increases the perceived size of the stimulus, which is consistent with a perceptual repulsion of the stimulus’ outline away from the cue⁷³ (FIG. 3d). Similarly, attention also affects the perceived shape of an oval: depending on cue placement inside or outside the contour, the aligned dimension (height or width) is perceived as longer or shorter, respectively⁷⁴ (FIG. 3e).

Several of the behavioural effects of attention that have been discussed so far suggest that attention might enhance spatial resolution by modulating visual RFs. Such effects of attention on RF profiles have indeed been found. In the following sections, we review these findings and then explain how they can account for the behavioural effects.

Attention alters visual cortical receptive fields

Several physiological studies have demonstrated that attention influences visual processing by recording activity from single neurons in the cortex of non-human primates while they perform visual tasks under different attentional conditions^{26,39–45,110–112}. The idea that attention operates by changing the spatial selectivity of single neurons was first suggested by Moran and Desimone³⁹. They recorded from single neurons in macaque visual area V4 and the inferior temporal cortex (IT) while the monkey attended to one of two stimuli presented inside the neuron’s classical RF (cRF) — the excitatory part of the RF (BOX 1; FIG. 4a). One of the two stimuli elicited a strong response from the neuron when presented alone (preferred stimulus), whereas the other stimulus elicited a weak response when presented alone (non-preferred stimulus). When presented simultaneously without attention on either stimulus, the neuron’s response approximately corresponded to the average of the response to either stimulus by itself. When attention was allocated to one of the two stimuli, the neuron’s response was primarily determined by the attended stimulus, meaning that its firing rate was enhanced when the preferred stimulus was attended but reduced when the non-preferred stimulus was attended. The neuron’s response was thus biased in favour of the attended stimulus, whereas the influence of the unattended stimulus on the response was attenuated, as if the neuron’s RF had contracted around the attended stimulus, so that the unattended stimulus now would fall outside this area. Such an effect has since been confirmed in numerous studies^44,111–113. It suggests that RFs shrink, but these results could also be explained by a shift of the RF profile towards the attended stimulus.

Figure 4. Attention alters receptive field profiles.

a | The left panel shows that responses to a preferred stimulus (represented by a purple vertical bar) and a non-preferred stimulus (represented by a green horizontal bar) presented simultaneously inside a neuron’s receptive field (RF; represented by a black line) reflect an average between the neuron’s response to the preferred stimulus presented alone (light purple line) and the response to the non-preferred stimulus presented alone (light green line). Attention biases this average response in favour of the attended stimulus. Attention on the preferred stimulus enhances the response (dark purple line), whereas attention on the non-preferred stimulus reduces the response (dark green line). The panels on the right show hypothetical changes in RF profile (indicated by orange shading). The results are consistent with a shrinkage of the RF around the attended stimulus as well as a shift towards the attended stimulus. b | Attention on a stimulus inside the RF (red dot) shifts and shrinks the RF towards and around the attended stimulus (right panel) compared with the same stimulus configuration when attention is allocated elsewhere (left panel). c | Attention on a stimulus near (but not inside) the RF shifts and expands the RF towards the attended stimulus (red dot in the right panel) compared with the same stimulus configuration when attention is allocated elsewhere (left panel). Part a is modified, with permission, from REF. 111 © (1999) Society for Neuroscience. Part b is modified, with permission, from REF. 58 © (2006) Macmillan Publishers Ltd. All rights reserved. Part c is modified from REF. 57.

Shifts of the RF profile have indeed been found in both ventral and dorsal visual pathways — for instance, in area V4 (REFS 54,55), the medial temporal area (MT)^57,58 and the lateral intraparietal area (LIP)⁵⁶ — and range from about 10% to 25% of the RF diameter or 1–3 degree visual angle. Some of these studies (in area MT) have also measured changes in the size of RFs with attention^57,58. While attention was directed to one of two stimuli inside or near the RF, another stimulus was flashed in a random sequence at different positions covering the RF so that RF maps could be created for each attentional condition from the responses of the neuron to the presentation of the stimulus at the different positions. These studies found that RFs are about 5% smaller when attention is directed to one of the stimuli inside the RF compared with a neutral condition without attention inside the RF^57,58 (FIG. 4b). When attention is directed to a stimulus next to the RF, the RF expands by about 14% compared with the neutral condition⁵⁷ (FIG. 4c). Consistent with this finding, an expansion of RFs has also been found with an attentive tracking task, in which stimuli travelled through a neuron’s RF¹¹⁴. Interestingly, these changes in RF size are accompanied by a shift of the centre of the RF towards the attended stimulus. The shift reaches far, so that it is still measurable when attentional focus and the RF lie in opposite visual hemifields, but declines with distance between the attended stimulus and the RF⁵⁸. Thus, both RF size and RF position are modulated by attention.

In addition, in area V4, attention differentially modulates responses depending on the relative position of the attended stimulus to the centre of the RF^54,55. A given neuron might, for example, respond strongly when attention is focused below the RF, but it might respond weaker when attention is directed above of the RF. It has been suggested that such tuning for the attentional focus location combined with partial RF shifts towards the attentional focus during intermediate processing stages, such as those that occur in area V4 and area MT, could enable a transformation from retinal-centred to attention-centred coordinates, in which positions are coded relative to the attentional focus^115,116.

In line with these electrophysiological studies, a functional MRI (fMRI) study has shown that attention can sharpen the population activity profile in early visual areas (specifically, areas V1–V4)¹¹⁷. This study measured the spatial spread of fMRI BOLD responses to stimuli at adjacent locations. Directing attention to one of the locations decreased the spatial overlap of the responses to each stimulus location, indicating a narrowing of spatial tuning. This narrowing of spatial tuning could be explained by the shrinking RFs of single cells. Alternatively, increasing or decreasing the activity of neurons with RFs inside or outside the attentional focus, respectively, could result in narrower population tuning.

These studies provide evidence for an attentional mechanism that brings the attended stimulus into the excitatory part of the RF and enhances the influence of the attended stimulus on the neuron’s firing rate.

Linking neurophysiology and behaviour

The goal of this review is to develop a unifying hypothesis between the behavioural and neurophysiological effects of attention on spatial resolution. Whereas the behavioural studies discussed before have sometimes postulated an attentional mechanism involving changes in RF profiles, such changes have actually been observed in the physiological studies described above. Note that quantitative comparisons between the behavioural and physiological effects are difficult; not only are the paradigms in both types of studies very different, it is also not clear how a particular magnitude of an RF shift or size change translates to measures of performance improvement typically used in behavioural studies, such as percent correct, d′ (an unbiased measure of sensitivity) or reaction time. In behavioural studies in which acuity thresholds or perceptual repulsion magnitudes are reported, effect sizes vary considerably. In addition, although several neurons and areas must be involved in each of the behavioural tasks reported here, physiological studies have measured effects in single neurons in only a few select cortical areas. Nevertheless, changes in RF size and position can qualitatively account for the aforementioned behavioural effects, as we explain here.

RF shifts improve performance by concentrating processing resources at the attentional focus but distort perception of space

RF shifts can explain enhanced performance at the attended location: RFs shift towards the focus of attention^54–58. Such a selective concentration of processing resources at the attentional focus should lead to an enhanced representation of the attended stimulus. RF shifts can thus explain improved performance in various psychophysical tasks, such as detection, discrimination, acuity and hyperacuity, visual search and texture segmentation tasks^{6–9,15–18,20,28–33,62–69}.

RF shifts can also explain the attentional repulsion effect^72,73 (FIG. 3c–f). The visual system can extract position information from the activity of spatially selective RFs in retinotopic maps¹¹⁸. Using a labelled-line code, the visual system could construct a map of objects in the visual field by evaluating the relative response strength of different neurons that represent different spatial locations. The shift of RFs towards the focus of attention changes the location at which a stimulus elicits the strongest response from a given neuron. In the case in which attention is directed to a location nearby the stimulus, as in the Vernier task⁷² (FIG. 3c), RFs originally centred beyond the stimulus shift towards the attentional focus so that they are activated by the stimulus, but because they still code for their original position, the perceived position of the stimulus is repulsed away from the attentional focus. Similarly, RFs centred outside the edges of an object would, when attracted towards the object’s centre, report the edge as lying within the RF, perceptually enlarging the object⁷³ (FIG. 3d, f). However, note that, alternatively, the population response profile could be skewed away from the attentional focus by selective suppression of RFs surrounding the attentional focus or by RF shrinkage at the attentional focus⁷². A recent model of the attentional effects on RF size and position predicts a ‘zoom’ effect of attention on the population level, which enlarges the representation of locations close to the attentional focus⁶¹ (BOX 3). This model might thus provide a direct link between RF changes and the attentional repulsion effect.

Box 3. Mechanisms of attention effects on receptive fields.

Several models^{59–61,97,110,138–142} have been proposed to explain attentional effects on receptive fields (RFs). Originally, the results of Moran and Desimone³⁹ had been interpreted as supporting the ‘biased competition’ model of attention^141,142. This model suggests that stimuli at neighbouring locations activate populations of neurons that engage in competitive interactions, which are possibly mediated by local, intracortical inhibitory connections. Attention shrinks RFs by selectively increasing the weight of those inputs representing the attended stimulus and thereby increases inhibition of the neighbouring inputs. Part a of the figure shows the circuitry for the biased competition model. Two input populations of neurons project to the neuron of interest (output neuron). The input population responding to the stimulus preferred by the output neuron (purple) has predominantly excitatory connections (thick black line), whereas the one responding to the non-preferred stimulus has predominantly inhibitory connections (thick grey line) to the output neuron. (Note that inhibitory input may operate via local inhibitory interneurons, which are not shown.) Attention strengthens the connections from the input population that responds to the attended stimulus, so that the response of the output neuron is dominated by the attended stimulus (right panels of part a).

Feedforward models in which attention selectively modulates inputs to the RF can explain how RFs contract and shift towards the attentional focus^54–58 without invoking lateral inhibitory connections^{59–61,110,138–140}. Part b of the figure shows how feedforward models can explain RF shift and shrinkage: attention changes the gain of inputs to the RF. The strength of the attentional influence (modulation amplitude (right axis), represented by red dashed line) follows a Gaussian distribution, with the strongest modulation at the attentional focus (centre of red dashed line) and weaker modulation elsewhere. Multiplicative interaction of the baseline RF (blue solid line, attention, left axis) and the attentional influence (red dashed line) results in a Gaussian RF profile (red solid line, attention, left axis) that is narrower and shifted towards the attentional focus. In these models, information from the attentional focus is selectively routed to higher cortical areas. This selective weighting would lead to a strengthened (or exclusive) representation of information from the attended location compared with information from the unattended locations. Such a simple two-layer feedforward model⁶⁰ also explains the switch from RF shrinkage to RF expansion with increasing distance between the centre of the RF and attentional focus⁵⁷. However, to account for RF shifts towards an attentional focus beyond the classical RF border^54,55,57, this model would need to invoke unrealistically strong attentional modulation.

The attentional signal that modulates feedforward input could involve feedback from high-level frontal or parietal cortical areas^143–146. A recent model also implements two layers of RFs but includes feedforward and feedback connections as well as local inhibition⁶¹ (part c of the figure). Attentional modulation propagates from top-layer RFs to bottom-layer RFs and interacts with local inhibitory connections: short-range inhibitory connections create competition scaled to the RF. A stimulus that is non-preferred for output layer neuron O1 but preferred for another, nearby output layer neuron (O2) activates O2, which in turn inhibits O1 when it falls into the RF of O2 (top left panel of part c) but not when it falls outside of the RF (top right panel of part c). Because neighbouring neurons O1 and O2 have overlapping RFs, the same stimulus is likely to fall inside or outside both RFs. The bottom panels illustrate how RFs expand towards an attentional focus outside the RF. Without attention, the connection between input layer neuron I2 and O2 is too weak to activate O2 and therefore I2 falls outside O2’s RF (bottom left panel of part c). When attention is directed to a location nearby, attentional modulation is redistributed (indicated by arrows) via feedback connections to those neurons (I1 and I2), which provide input to the output layer neuron that is modulated (O1). I2 now responds more strongly to the same stimulus and therefore activates O2. O2 thus responds to a stimulus closer to the attentional focus to which it did not respond without attention (shown in blue) — that is, its RF has expanded (shown in red) towards the attentional focus. The combination of shrinkage by competitive interactions (see top panels of part c) and expansion towards the attentional focus results in a net shift of the RF (not shown). This model can explain RF shifts, shrinkage and expansion.

These models provide insight into the neuronal mechanism by which attention modulates RFs and thus enhances spatial resolution. Some assume simple feedforward mechanisms, and others use feedback connections. Whereas for endogenous attention feedback from higher cortical areas has been established^143–146, no physiological studies have investigated either feedback or RF modulation with exogenous attention, which, conceivably being sensory driven, could operate in a purely feedforward manner. Part a of the figure is modified, with permission, from REF. 111 © (1999) Society of Neuroscience. Part b of the figure is modified, with permission, from REF. 59 © (2008) Society of Neuroscience. Part c of the figure is modified, with permission, from REF. 61 © (2011) IEEE Press.

RF shrinkage improves performance by reducing spatial integration

Whereas RF shifts may lead to enhanced performance with attention in general, RF shrinkage^57,58 additionally improves performance in specific tasks via a reduction of filter size and thus the integration area. The combination of RF shift and RF shrinkage at the attended location leads to more and smaller RFs at the focus of attention, which should lead to better spatial resolution. Thus, these effects can explain better performance in acuity tasks^8,29,65–67 that are designed to specifically measure the visual system’s spatial resolution (FIG. 2d).

The ability of smaller RFs to resolve finer details is correlated with a reduction of the area over which a single RF integrates information. As a psychophysical concept, the integration area is thought to be crucial in visual search^83,84,86 and crowding^64,90 tasks, in which it is essential to isolate a target from nearby distractors. The psychophysical integration area is probably related to the size of visual RFs: that is, the physiological integration area. Thus, attention might improve performance in these tasks by reducing the integration area via RF shrinkage (FIG. 2c).

Texture segmentation studies make a particularly strong case for a spatial resolution explanation of attention effects on performance. In these tasks, performance varies with target eccentricity and peaks at the eccentricity at which filter size is optimal for the scale of the texture. Exogenous attention by default shifts this point outwards and improves performance beyond the peak (where resolution is too low) but impairs it before the peak (where resolution is already too high)^7,68,69,103. Endogenous attention, however, always improves performance, which is consistent with a flexible adjustment of resolution¹⁰². The increase in spatial resolution can be explained by RF shrinkage, but, alternatively, it can also be explained by stronger weighting of small, high spatial frequency-selective RFs^68,69. It is conceivable that the improved performance at near eccentricities with endogenous attention could be mediated by decreased resolution, which might be achieved by enlarging RFs or stronger weighting of low-pass filters.

In addition to improving performance, RF changes alter the representation of visual information. If the attended location is inside the RF, the RF contracts around it, whereas if the attended location is outside the RF, the RF extends towards it^57,58. Both effects could serve to bring the attended stimulus into the excitatory part of the RF and enhance the attended stimulus’ influence on the neuron’s activity. Attention thus enhances the representation of the attended stimulus at the cost of stimuli outside the attentional focus. If the magnitude of these effects increases with visual hierarchy, representation of visual information would gradually change from a relatively unbiased image of the visual world to one that contains mostly attended information¹¹⁹.

Open questions

How does spatial resolution change around the attentional focus?

RFs overlapping the focus of attention shrink, whereas those nearby the focus of attention expand. Consequently, whereas spatial resolution is improved at the attended location, RF expansion around the attended location should make spatial resolution worse. Some studies of attention indeed support a ‘Mexican hat’ model, in which processing is enhanced at the attentional focus but impaired in its vicinity when compared with baseline performance^120–123. However, none of these studies has explicitly tested spatial resolution. Thus, behavioural studies that test spatial resolution not only at but also around the attended location are needed.

Can attention switch the analysis of a scene from local to global via modulating spatial integration?

By modulating spatial summation and surround suppression, attention might switch the analysis of a visual scene between a local and a global level of analysis (BOX 1). For example, by increasing surround suppression, attention could facilitate an analysis of local contrast and detection of a salient object. Alternatively, when a global analysis is needed, we would benefit from spatial integration over a larger area; in that case, attention could help by decreasing surround suppression. In the case of motion perception, for example, surround suppression and summation are important for analysing object versus background motion^124,125. Thus, modulating segmentation and integration could be an intriguing potential function of attention, but these effects are not yet well understood (BOX 4).

Box 4. Attentional modulation of surround suppression.

The part of the receptive field (RF) from which a stimulus elicits an excitatory response is the classical RF (cRF). RFs often have a surround (the non-classical RF (ncRF)), in which a stimulus does not elicit a response from the neuron by itself but modulates the neuron’s response to a cRF stimulus. These modulatory influences can be inhibitory or facilitatory.

RFs with inhibitory surrounds respond best to a stimulus with preferred features in the centre and non-preferred features in the surround. They support a local analysis of the visual scene by signalling the presence of a local variation in a particular stimulus feature. RFs with facilitatory surrounds respond best to stimuli with preferred features in both the centre and the surround; they integrate over a larger region of space (the same as larger RFs) and perform a more global analysis of a scene.

Whereas attentional effects on the cRF profile have been well characterized^54–58, attentional effects on the non-classical RF have seldom been investigated and are less conclusive^57,147–149. Neurons in area V1 are typically tuned for the length or size of a stimulus, which is related to surround inhibition. Responses increase with stimulus size as long as the stimulus overlaps only the excitatory part of the RF, whereas responses drop for larger stimuli that also stimulate the inhibitory surround. Attention modulates length tuning differentially depending on eccentricity; attention inside the cRF decreases preferred length of parafoveal RFs but increases preferred length of peripheral RFs¹⁴⁷.

In area V4, attention decreases surround suppression when a cRF stimulus is attended and increases surround suppression when a surround stimulus is attended¹⁴⁹, but the opposite pattern occurs in the medial temporal area (MT)⁵⁷. Additionally, in area MT, surround inhibition is stronger on the attended side of the RF, so that the inhibitory surround centre is shifted towards the attentional focus⁵⁷.

Thus, attention affects not only the feedforward connections that construct the cRF but also lateral and feedback connections that are considered to mediate spatial integration beyond the cRF. However, attentional effects on ncRF influences are complex and may differ depending on cortical area and eccentricity. By modulating RF surrounds, attention might switch the analysis of a visual scene from local to global or vice versa. Further research is needed to characterize and understand these effects.

How can we narrow the gap between behavioural and physiological effects?

The attentional effects on RFs can best be explained by a combination of attentional modulation of feedforward connections with reciprocal modulatory feedback and local inhibition⁶¹ (BOX 3), which may emerge from signals in the frontal and parietal cortex. However, this explanation seems intuitive for endogenous attention but less so for exogenous attention, which in principle could be purely feedforward-driven⁴⁸. This points to a current gap in the unifying framework: whereas all electrophysiological studies on attentional modulation of RFs have used endogenous attention manipulations, most evidence for spatial resolution effects at the behavioural level comes from exogenous attention studies. Future research would benefit from using paradigms that allow a direct comparison of physiological results with behavioural results with the same attentional manipulation. Particularly, physiological studies that use exogenous attention are needed.

Another important difference is that most physiological studies use relatively simple displays with one or two stimuli inside a neuron’s RF, whereas behavioural studies often use richer displays — for example, textures or crowded displays — and might thus lead to different predictions. For example, for such displays, it could be beneficial to sharpen feature tuning⁹⁷ in order to exclude information from distractors with slightly offset features, but for a single stimulus display, it would not be beneficial. Therefore, physiological studies using displays comparable with those used in behavioural studies (or vice versa) could facilitate the comparison of behavioural with physiological results.

Furthermore, with respect to spatial tuning, the finding that endogenous attention increases performance across eccentricities in texture segmentation tasks¹⁰² can be explained by expanding or shrinking RFs depending on whether lower or higher resolution is needed. This explanation could be tested in physiological experiments using a task in which lower spatial resolution would be desirable.

Do attentional changes in spatial tuning reflect a general principle?

Spatial attention alters RF profiles — that is, spatial tuning (BOX 1) — but does not change tuning for features such as motion direction or orientation^41,110. Although a theoretical model predicts that spatial attention sharpens orientation and spatial frequency tuning⁹⁷, such an effect has so far not been found on the physiological level. By contrast, attention to a feature^110,126,127 (feature-based attention) attracts tuning curves towards the attended feature¹²⁸. Thus, it seems that attention generally shifts tuning along the relevant dimension. In the case of feature-based attention, it might help to improve discriminability — that is, resolution in feature–space — around the relevant feature. Modulation of tuning is also used in different sensory systems; attention to sounds alters frequency tuning of spectrotemporal RFs in primary auditory cortex in ferrets, so that responses to frequencies near a target frequency are enhanced, whereas responses to other frequencies are suppressed^129,130. These effects are consistent with a shift of frequency tuning towards the attended frequency. Thus, tuning changes may be a general principle underlying different types of attention, but more research is needed to better understand these effects.

Conclusion

By covertly attending to a location, we can overcome the limitations of visual processing in the periphery and (partially) restore visual performance. Both endogenous attention and exogenous attention improve performance in tasks mediated by spatial resolution: visual search, acuity tasks and texture segmentation. However, exogenous attention can also impair performance when resolution is already too high. The same changes in RF profile that can account for these performance effects can also account for attentional effects on the perception of spatial stimulus features such as position and size. By concentrating neuronal resources at the attentional focus and reducing the area of spatial integration, attention enhances the visual system’s effective spatial resolution.

Glossary

Spatial resolution

The ability to discriminate two nearby points in space

Fovea

The central part of the retina with highest receptor density, finest receptor-to-retinal ganglion cell mapping and therefore best spatial resolution. It contains the highest density of cones and the highest cone-to-rod ratio

Extrastriate areas

All visual cortical areas that are higher in the processing hierarchy than primary visual cortex (V1; also known as striate cortex), including areas along the temporal (for example, area V4 and the inferior temporal cortex) as well as the dorsal pathway (for example, the medial temporal area and medial superior temporal area)

Eccentricity

The distance from the fovea. Visual field eccentricity corresponds to retinal eccentricity during fixation

Spatial frequency tuning

The variation of the neuronal response to variations in spatial frequency of a stimulus. Spatial frequency describes the scale over which local contrast varies in a visual scene

Saccades

Rapid eye movements that align gaze with a new location in visual space several times per second

Spatial attention

Selection of a particular region in space so that processing of information from that location is enhanced. Here, we focus on visual spatial attention; that is, the selection of visual information and its effects on the activity of visual neurons and visual performance. Spatial attention can be directed overtly — that is, by moving the eyes towards the location of interest — or covertly — that is, without eye movements (covert attention). Covert attention can be allocated voluntarily (endogenous attention) or captured involuntarily (exogenous attention)

Attentional focus

The location in the visual field at which attention is allocated

Cortical magnification factor (CMF)

The area of cortical surface to which a stimulus subtending 1 degree of visual angle on the retina projects. Often, the reciprocal of the CMF is used to determine the number of degrees visual angle a stimulus should subtend to activate 1mm of cortex. This number increases linearly with eccentricity

Landolt stimulus

A typical stimulus used to measure acuity. Observers have to detect a small gap in a circle or square or discriminate the location of the gap (for example, the left or right side of the circle or square)

Vernier tasks

Typical tasks that are used to measure hyperacuity. Observers have to report a small lateral offset between two lines

Selectively adapting

Prolonged exposure to a particular stimulus selectively decreases the responses of neurons involved in processing the stimulus and therefore decreases sensitivity to the subsequent presentation of the same or a similar stimulus

Visual hemifields

These are one half of the visual field

Retinotopic maps

Neighbouring locations on the retina also stimulate neighbouring locations in retinotopic cortical areas. Thus, even though distances may be distorted (cortical magnification), spatial relations between different locations are kept from the retina through higher levels of the visual processing hierarchy, including the visual areas V1, V2 and V4, and the medial temporal area

Labelled-line code

The idea that information about a stimulus (for example, its location) in the nervous system is transmitted by activity in specific connections — ‘labelled lines’. For example, receptive fields are labelled with their location in the visual field, and thus activity of a neuron with a certain receptive field creates the sensation of a stimulus at that particular location

Feature-based attention

The selection of a particular feature within a dimension, such as vertical orientation, red colour or upwards motion direction. Processing of the selected feature is enhanced independent of spatial location