The dynamics of the spread of selective visual attention

Acentral issue in the study of visual attention is what is selected when one's attention is directed to a location or an object. Location-based theories often employ a “spotlight” metaphor (1) in which attention enhances processing of items within a particular region of space. Any object falling within that region will receive a processing advantage. It is also possible to attend to specific visual features. For example, one might scan a crowd for a friend wearing a blue coat. As you search, your attention is likely to also be drawn to various other blue objects. Other object-based theories suggest that preattentive mechanisms segment visual scenes into discrete objects, perhaps on the basis of Gestalt grouping principles, and that attention then selects whole objects (e.g., ref. 2). Much of the evidence for object-based selection comes from behavioral studies in which inferences are made on the basis of subjects' reaction times or accuracy rates. However, recent physiological data have confirmed central predictions of object-based theories, finding that when attention is directed to a particular feature of a visual object, brain regions associated with other, irrelevant features of the attended object also show increased responses (3). In a recent issue of PNAS, Schoenfeld et al. (4) described new physiological data that support this notion and that extend our understanding by measuring the speed with which the selection of task-irrelevant features of attended objects occurs.

Different regions of the brain's visual system respond selectively to stimuli with specific characteristics. In various areas, the types of stimuli that produce the highest levels of activation may be relatively simple (e.g., color or motion) or quite complex (e.g., faces or houses). The extent to which regions responsive to complex stimuli, such as faces, are dedicated modules specifically for processing these types of stimuli remains hotly debated (5, 6), but evidence suggests that the processing of visual objects involves activity across a distributed representation of the objects' features (7). Brain imaging data have shown that when attention is directed toward a particular feature, e.g., color, brain regions that are selective for that feature show an enhanced response (e.g., ref. 8). This appears to be true not only for simple feature types but also for complex features such as faces. Furthermore, simply imagining these types of visual items leads to similar increases in associated brain regions (9).

Feature Binding Across Space and in Time

How the various features that are associated with any particular object are bound together to create a coherent percept is a fundamental issue in psychology and neuroscience and is commonly known as the “binding problem.” The specific mechanism by which visual feature binding takes place remains unclear. However, many researchers have proposed a link between binding and attention. One view (10) is that attention directed to a spatial location binds features in that location (which will tend to be from the same object). Another view is that, although space is important, attention selects whole objects (2). Biased competition theory (11) holds that, within a cortical region, different features compete with each other for representation, and this competition may be biased by a number of factors including behavioral relevance or attention. The integrated competition hypothesis (12) extends this model, suggesting that this competition is integrated between cortical regions in such a way that features that gain a competitive advantage in one region will tend to cause features in other regions to also have an advantage when they are from the same object.

Recent functional MRI (fMRI) data (3) provide important neurophysiological evidence of object-based attentional selection. A key prediction of objectbased attention theories is that, when attention is directed to one aspect of a visual stimulus, the other aspects of the same stimulus should also show increased levels of activation. O'Craven et al. (3) localized three brain regions, each of which shows a high degree of selectivity for a particular type of visual stimulus (faces, houses, and motion). By creating stimuli that combined these features in independent ways, O'Craven et al. then showed that, when subjects were directed to attend to one of two types of objects on the visual display, increased activation also occurred in the brain region associated with the other, irrelevant feature associated with the attended object. The data indicate that attention to one aspect of a particular visual object enhances activity in brain regions that are responsive to other features of the object that are not relevant to the behavior of the subject. Under object-based theories of attention, this spread of attentional enhancement must occur with a rapid time course if it is to be related to the mechanism by which different features of an object are bound together. By itself, fMRI is unable to measure the relative timing of activation between brain regions with this level of resolution.

Schoenfeld et al. (4) examine the time course of object-based attention effects by integrating functional imaging data with data from two high temporal resolution methods: electroencephalography (EEG) and magnetoencephalography (MEG). They found that, when subjects were instructed to attend to one feature of a visually presented object, activity was rapidly enhanced (within ≈50 ms) for another irrelevant feature of the same object. Subjects viewed displays of two sets of dots simultaneously moving in opposite directions and attended to those moving in one direction to detect trials in which they moved at a faster speed. The subjective experience of these displays is that of viewing two overlapping transparent surfaces. In some trials, either the attended or the unattended dots changed color; the authors compare trials in which there was a color change in the unattended dots to those in which there was no color change to identify the time course of a “sensory effect” corresponding to the color change. Importantly, color was never relevant to the task. They also compare trials when the attended vs. unattended dots changed color to determine the latency with which attention enhances the response to the task-irrelevant color change. Electromagnetic source modeling and fMRI each localized both the sensory and attention effects to bilateral regions of fusiform cortex, regions previously shown to be involved in color processing and to be enhanced by attention to color (8, 13). Critically, the attention effect was found to occur within ≈40–50 ms after the onset of the sensory effect. As with the study by O'Craven et al., both the attended and ignored objects occupied the same regions of visual space, so the results cannot be explained strictly by spatial attention.

Converging Methodologies for Space and Time

The Schoenfeld et al. study took advantage of several imaging methodologies to “converge” on a common solution. Although fMRI provides excellent spatial resolution down to the millimeter scale, as a measure of blood flow its temporal resolution is limited to the scale of seconds; it is therefore not a useful technique for resolving temporal issues in the brain, where events must be measured at the millisecond scale. The other noninvasive brain imaging methods used by Shoenfeld et al., EEG and MEG, are capable of measuring brain activity at the millisecond scale but are substantially more limited in their spatial resolution. Clearly, an ideal solution is to merge the spatial resolution of fMRI with the temporal resolution of the electrophysiological techniques, thereby providing a detailed spatio-temporal description of interactions across a given network of brain areas. Statistical methods for source analysis of the electrophysiological data can estimate the approximate intracranial generators of a given effect measured at the scalp surface and, when these sources are found to be overlapping with activations from fMRI, the investigator can derive a relatively high degree of confidence regarding their localization. The inclusion of specific constraints from one modality on localization in another modality strengthens the multimodal approach. Schoenfeld et al. took advantage of this by constraining their head model with data from anatomical MRI. Although the utility of the multimodal approach has been touted for nearly a decade now, surprisingly few studies have succeeded in applying it.

Mechanisms and Multiple Types of Attention

The study of Schoenfeld et al. provides important constraints for future work on the relationship between attention and feature binding, although many issues remain. It puts an upper limit on how fast features might be bound after attention is directed to an object, although it does not itself indicate the mechanism by which this binding takes place. Several possible mechanisms for binding have been proposed, e.g., synchronous oscillations in the 40-Hz range (14), but considerable controversy still exists (15). Future research will be needed to show whether the time course of spreading attentional enhancement is similar between other features, especially when the features are not completely overlapping in visual space. Also not clear yet is the mechanism by which modulation by attention spreads from one cortical region to another and how this relates to the source of the initial top-down attentional signal, which remains incompletely understood, though data implicate regions of parietal and frontal cortices (16, 17).

The different views of attentional selection are not necessarily completely mutually exclusive. Although there is substantial evidence for object-based selection, attention to locations may still play an important role in many cases. Attention has been shown to modulate retinotopically specific regions of visual cortex even at very early stages of cortical processing (e.g., ref. 18) and even in the absence of visual stimulation (19). Furthermore, visual attention also appears to have inhibitory or suppressive components and, although some of these components appear to work at the level of object-based selection (20), other suppressive components have been shown to operate in the spatial domain but object-based effects have not yet been shown (21).

An object-based attention system makes sense in that, typically, the various features of an object work together to create a stable representation of the item. However, in some cases, competition may occur among the properties of an object as well as between objects. An example of such a case is one of the classic findings in the psychology literature: the so-called Stroop effect (22). In this paradigm, subjects are presented with words that spell out the names of various colors (e.g., red, green, purple), but these words are also “printed in the ink” of another color, such that a given word stimulus (e.g., RED) might actually be displayed in green. Subjects are considerably slower to name the color of the ink when the word spells out an incongruous color name than when it does not spell a color name or when simple orthographic-like symbols are used. Here, then, is a good example of how irrelevant features of a stimulus can compete with the currently relevant task. Indeed, Stroop went on to show that even 8 days of training failed to eliminate this interference effect, which strongly suggests that processing of the irrelevant feature was automatically invoked. A number of clinical populations have particular difficulties with such stimuli, suggesting that they have significant disturbances in their abilities to suppress irrelevant features. These data indicate that another facet of this seemingly automatic binding is that some amount of suppressive activity can also be used when the demands of a given task require the subject to ignore the irrelevant features. Of interest is the fact that evidence for such suppressive effects has been seen in a number of recent studies in humans (23, 21). Future research will be needed to clarify the relative timing and interaction of suppressive mechanisms and automatic activation of irrelevant features by attention.

The Speed of Visual Processing

One of the central findings of the Schoenfeld et al. study concerns the relatively short interval between the emergence of the sensory color effect (≈190 ms) and the time at which attention effects were manifest (≈240 ms). Although 50 ms is certainly a short period, it is worthwhile to consider this timing in the context of known transmission rates through the visual system. It has been shown through intracranial recordings in awake behaving macaques (24) and also through scalp mapping in humans (25) that transmission rates through the visual system are remarkably rapid. For instance, Schroeder has shown that virtually the entire visual system, from the primary (V1) through secondary (V2) visual cortex and on into higher regions of the infero-temporal visual cortices (IT), becomes activated within just 30 ms of the initial afferent input to area V1. The dorsal visual stream, dominated by the fast magnocellular inputs, is even faster, with transmission through most dorsal structures occurring within just 10 ms of initial afference. Indeed, data in humans have shown (25, 26) that even regions of the frontal cortices are activated by visual stimuli within 30–40 ms of initial V1 activation, at a latency of just 70–90 ms after stimulus. The typical ERP waveform shows that the brain's response can last hundreds of milliseconds after initial afference. As yet, we have only the crudest knowledge of the nature of processing that is occurring over successive epochs within this timeframe or what an epoch of a given length of time actually signifies. The Schoenfeld et al. study is admirable in that it lays out some basic temporal information and suggests that automatic feature binding can occur relatively rapidly over a period of just 40–60 ms. What is clear from the studies mentioned above is that 50 ms is actually a remarkably long time in terms of neural transmission, and that many regions of cortex can be activated during such an epoch.

Another interesting aspect of timing dynamics reported in the Schoenfeld study is that the latency of color sensitivity to motion stimuli (180–190 ms) appears to be quite late in processing relative to the initial input. For instance, Givre et al. (27) have shown color sensitivity in early visual cortices (V1 and V2) with large differences in the response to equiluminant red and white stimuli seen at a latency <50 ms. Although this finding was in macaques, one would predict color effects in humans well within the first 100 ms of processing (e.g., ref. 28). Further studies using a variety of feature combinations may help to resolve this seeming disparity in results.