Attention improves or impairs visual performance by enhancing spatial resolution

Abstract

Covert attention, the selective processing of visual information at a given location in the absence of eye movements, improves performance in several tasks, such as visual search and detection of luminance and vernier targets^1-6. An important unsettled issue is whether this improvement is due to a reduction in noise (internal or external)^6-9, a change in decisional criteria^10,11, or signal enhancement^3,5,12. Here we show that attention can affect performance by signal enhancement. For a texture segregation task in which performance is actually diminished when spatial resolution is too high, we observed that attention improved performance at peripheral locations where spatial resolution was too low, but impaired performance at central locations where spatial resolution was too high^4-12. The counterintuitive impairment of performance that we found at the central retinal locations appears to have only one possible explanation: attention enhances spatial resolution.

We previously demonstrated that when a spatial cue directs covert attention to an upcoming target location, observers’ performance improves for stimuli designed to measure spatial resolution (for example, the Landolt-square—a square with a small gap on one side)⁵. This ‘peripheral cue’ putatively draws covert attention to its location automatically^2,3,13. Because the display characteristics ensured that neither a reduction in noise nor a change in decisional criteria could explain this attentional facilitation, this finding indicated that attention could enhance spatial resolution at the cue location⁵. Similarly, in visual search tasks in which observers’ performance is slower and less accurate as target eccentricity increases, due to the lower spatial resolution of the periphery¹⁴, cueing the target location diminished this eccentricity effect¹. In another study⁶, when observers searched for a slightly tilted target, cueing the target location decreased the detrimental effect caused by vertical distracters; the authors suggested that focal attention changes the size of the ‘stimulus analyser’, permitting a finer-scale analysis of a target⁶.

We present a critical test of the ‘resolution hypothesis’, which predicts that attention can actually enhance spatial resolution, so that we can resolve finer details at the attended location. We explored the effects of spatial attention on a task in which performance would be diminished by heightened resolution. If attention indeed enhances resolution, performance at the attended location should be impaired, not improved. Such impairment could not be predicted by previous models of visual attention^7-11,15,16. The task involves the detection of a texture target of one orientation appearing at a large range of eccentricities in a background of an orthogonal orientation (Fig. 1a). Unlike in most visual tasks^1,5,9,14,17 observers’ performance does not peak when the target appears at foveal locations, where resolution is highest^9,17, but rather at mid-peripheral locations, and drops towards central or farther peripheral locations^18-21. Psychophysical and physiological evidence indicates that we process visual stimuli by means of parallel spatial filters, each tuned to a band of spatial frequencies^9,17. The central performance drop may be due to the fovea being more sensitive to high spatial frequencies for which neural processing is slower²⁰. Moreover, this drop could reflect a mismatch between the average size of spatial filters at the fovea and the scale of the texture. Spatial filters at the fovea may be too small for the scale of the texture¹⁸, such that the foveal resolution may be too high for this texture. As retinal eccentricity increases, the average size of the filters becomes larger and their resolution decreases^9,17; filter size is presumably optimal around the peak of performance. At farther eccentricities, the filters may be too big and their lower resolution would limit performance¹⁸.

Figure 1.

Texture segregation task. a, The display consisted of a 2 × 1:5 cm target-texture, composed of 3 × 3 lines (oriented at 45 or 135°), embedded in a background-texture composed of 287 lines (7 rows × 41 columns, subtending 5 × 28 cm) whose orientation was orthogonal to the target. The elements were jittered by 0.3 cm. From a viewing distance of 57 cm, the target subtended 2 × 1:5° of visual angle and the texture display subtended 5 × 14° of visual angle to each side of the centre of the display. The target appeared equally often in each interval (50% of the time) and was centred at any of 35 possible locations along the horizontal meridian in a random order. b, Each interval began with a fixation dot at the centre, followed by a brief cue. The cue was either ‘peripheral’ (a green horizontal bar of 0:3 × 0:6 cm appearing 0.3 cm above the target location) or neutral (two horizontal lines of 0:3 × 28 cm appearing above and below the display). After an interstimulus interval (ISI), the texture was displayed for an average of 40 ms. The ISI was set individually to keep overall performance level 75% correct; display duration ranged from 15 to 50 ms. A mask, with crosses as elements, followed the stimulus. Observers were asked to indicate the interval containing the target by pressing one of two keys. The order of the 100 practice trials, as well as that of the 288 experimental trials, was randomized.

We hypothesized that if attention enhances spatial resolution, attending to the target location will improve performance where the resolution is too low (periphery) but will impair performance where resolution is already too high for the task (central locations). We employed a two-interval forced-choice task. Observers indicated which of two intervals contained a texture-target whose orientation differed from that of the background (Fig. 1a). Half the trials were ‘cued trials’ in which a peripheral cue indicated the target location in the interval containing the texture target, and a non-target location in the interval without a target. The cue was a 100% valid cue because the target could only appear at the cued location; however, it did not signal which interval was more likely to contain the target because both displays were preceded by a cue¹. The rest were ‘neutral trials’ in which a pair of lines in both intervals indicated that the target had an equal probability of appearing at any location. The viewing distance was 57 cm for all 28 naive observers. Brief presentation times precluded eye movements between the onset of the cue and the offset of the texture display (Fig. 1b).

In neutral trials, performance peaked when the target appeared at 5° eccentricity, not at the fovea, replicating the central performance drop found in a similar texture task^18-21. Cueing condition and target eccentricity interacted significantly (Fig. 2a), in a manner consistent with the resolution hypothesis. Accuracy was higher for the cued than the neutral trials at all target eccentricities except at foveal locations (<1°), where it was lower. These results are inconsistent with previous models of attention which predict that attention always helps and never hinders performance, but they are consistent with our hypothesis. Given that performance is worse at the fovea because its spatial filters are too small and have too high a resolution for the scale of the texture^18,21, further increasing resolution at foveal locations led to a more pronounced drop in performance. Such impairment could result from an attentional mechanism that enhances resolution by effectively decreasing the average size of filters at the attended location (see below). The same mechanism should improve performance at regions where spatial filters are too large and have too low a resolution for the scale of the texture.

Figure 2.

Observers’ performance as a function of cueing condition and target eccentricity. a, Experiment 1; b, experiment 2; and c, experiment 3 (viewing distance of 57 cm, 28 cm, and 228 cm, respectively). Within-observers ANOVAs demonstrated significant interactions (P < 0:001); accuracy was higher for the cued (squares) than the neutral (circles) trials at more peripheral eccentricities but was lower at central locations (exp.1: 0-1°; exp, 2: 0-5°; exp. 3: 0-0.66°). Analysis of reaction time (RT) indicated that neither the main effects of cueing and eccentricity nor their interaction were significant (bottom panels). Thus, there were no speed-accuracy tradeoffs. In experiment 2, halving the viewing distance doubled the target eccentricity range; in experiment 3, increasing the viewing distance by a factor of 4 reduced the range accordingly (see abscissa scales). Note the shift in performance peak and the range in which performance was impaired by the cue. Error bars correspond to the average ±1 s.e. for each condition; symbols denote statistically significant differences at P < 0:05 (asterisks) or P < 0:01 (daggers) according to least significant differences (LSD) post hoc comparisons.

Performance peaks at different eccentricities when the scale of the texture is manipulated by changing the size of the textural elements, the space between them, or the viewing distance^18-20. Enlarging the scale of the texture shifts the peak of performance to farther eccentricities, supporting the idea that segregating larger textures requires larger filters, more abundant at farther eccentricities¹⁸. If by increasing the scale of the texture, its mismatch with the size of the filters extends farther towards the periphery, and attending to a location is similar to reducing the size of spatial filters, then cueing should impair performance for a wider range of eccentricities. In experiment 2, we tested this hypothesis by doubling the scale of the texture; the viewing distance for 20 naive observers was reduced to 28 cm.

Consistent with previous studies^18-20, doubling the scale shifted the peak of performance to a farther eccentricity (from 5 to 7.6°). Again, we observed a significant interaction of cueing and eccentricity (Fig. 2b). Performance in the cued trials improved in peripheral regions but diminished at the four central locations, which extend into the parafovea. Moreover, with the larger texture scale, the cue impaired performance for a larger range of eccentricities (0–5°) than in experiment 1. In experiment 3, in contrast, the scale of the texture was reduced for 23 naive observers who viewed the display from a distance of 228 cm, and the range of impaired performance shrank to only 0–0.66° (Fig. 2c). These results suggest that the ranges of attentional impairment and enhancement are mediated by the scale of the texture. Specifically, the finding that target detection at a given eccentricity was either impaired or improved by the cue, depending only on the scale of the texture, rules out the possibility that attention is unable to enhance processing of foveal and parafoveal stimuli in this task. Rather, this impairment results from the characteristics of a given texture and the effects of covert attention.

The mere fact that attentional deployment affected performance is noteworthy. Most models of visual processing portray texture segregation based on orientation differences as an early pre-attentive task²², in which information (for example, simple features) is extracted from the display effortlessly and in parallel. Such models cannot account for the effect found here. This effect is consistent, however, with psychophysical studies showing that attention can improve performance in tasks considered to be pre-attentive, including feature search^1,6, feature discrimination²³, spatial acuity⁵ and orientation acuity⁶ tasks. This effect also supports the finding that performance in a simple feature search task declines when additional attentional demands are imposed^24,25. Similarly, neurophysiological studies demonstrate that attention can modify the responses of early visual areas such as V1, V2, V4 and MT/MST neurons^26-30. This body of psychophysical and neurophysiological evidence indicates that attentional mechanisms can influence early vision.

Either decreasing the viewing distance (Fig. 3a) or directing attention to the target location (Fig. 3b) similarly shifted the peak of performance to farther eccentricities, suggesting that both manipulations increased the mismatch between texture scale and the size of the filters. The former may have done so by increasing the scale of the texture and the latter by decreasing the size of the filters responsible for texture segregation. Like moving closer to the display, attention allows us to resolve better the various details in front of us, which would almost always be advantageous. Yet when a more global inspection of the display is required, for example when one is appreciating an impressionist painting, moving closer is not the optimal strategy.

Figure 3.

Observers’ performance as a function of viewing distance and cueing condition. For both panels, performance is depicted as a function of per cent correct (left axis) and relative per cent correct, with respect to optimal performance (right axis). The data from experiments 1 and 2 were fitted to second-order polynomials. The fits’ R² ranged from 0.8 to 0.97. a, Comparing the neutral trials of the two experiments demonstrates the effect of viewing distance on performance. Viewing the display from half the distance shifted the performance peak from 5° to 7.6°. b, Comparing the cued and neutral trials of experiment 2 demonstrates the effect of cueing on performance. Directing attention shifted the performance peak from 7.6° to 13.3°.

Physiological studies support the hypothesis that attention can enhance spatial resolution by reducing the size of spatial filters at the attended area. When attended and unattended stimuli are both within a cell’s receptive field, the neuronal response is primarily determined by the attended stimuli; responses to the unattended stimuli are attenuated^26,27, as if the cell’s receptive field shrinks around the attended stimulus^26-28. These authors proposed that attentional modulation of sensory processing is accomplished in two stages: first, top-down signals bias activity in favour of the neurons representing the relevant location, and then these favoured neurons compete with other neurons, ultimately suppressing their response. This competition may result from mutual inhibition between cells or between the inputs to the cells^27,28, and its outcome could effectively reduce the cell’s receptive field, allowing finer spatial resolution. Alternatively, enhanced resolution at the attended location could result from increased sensitivity of the neurons with the smallest receptive fields at the attended area^4,5, which in turn may inhibit neurons with larger receptive fields there¹⁷. According to this hypothesis, the overall sensitivity of the attended region would shift towards higher spatial frequencies. However, the possibility that the central performance drop is mediated by interference between high- and low-frequency information is controversial^18,21.

We conclude that attentional facilitation in visual tasks may reflect a combination of mechanisms—such as noise reduction, decisional factors and signal enhancement^5,6,10,12. This study supports the last mechanism because directing attention improved performance in peripheral locations, but impaired it at foveal and parafoveal locations. These findings imply that attention increased the mismatch between the texture scale and the size of central spatial filters, and provide strong support for the hypothesis that attention can enhance spatial resolution.