The Neural Basis of Selective Attention

Abstract

Selective attention is an intrinsic component of perceptual representation in a visual system that is hierarchically organized. Modulatory signals originate in brain regions that represent behavioral goals; these signals specify which perceptual objects are to be represented by sensory neurons that are subject to contextual modulation. Attention can be deployed to spatial locations, features, or objects, and corresponding modulatory signals must be targeted within these domains. Open questions include how nonspatial perceptual domains are modulated by attention and how abstract goals are transformed into targeted modulatory signals.

The human brain contains more than 10 billion neurons and more than 10 trillion synapses, making up networks and subnetworks of unimaginable complexity. These vast numbers seem as if they should be quite sufficient to support the performance of nearly any function the brain should be called upon to do—but this is apparently not the case. Evolution has supplied us with a brain in which each neuron can serve multiple functions, depending on what task is required at each moment. That is to say, neurons (and thus neural networks, and the brain as a whole) are subject to contextual modulation of their function.

Selective attention is an unsurpassed example of contextual neural modulation. Selective modulation of neural activity is made necessary by the hierarchical organization of the primate visual system. Neurons in the primary visual cortex (area V1) have small receptive fields (that is, they monitor a small patch of the retinal image) and are tuned to relatively simple visual features (e.g., edge orientation). Neurons at later levels (e.g., area V4) have larger receptive fields that are tuned to relatively more complex features (e.g., combinations of shape and color). The representation of an object is distributed over this network: Local metric details are represented in early areas, global properties in later areas.

Selective attention is required when the visual system is confronted with typically cluttered natural scenes. Neurons at later levels are likely to have multiple stimuli in their receptive fields, some of which would effectively drive the neuron if they were presented in isolation and others that would not. Not all of these stimuli can be represented simultaneously. Selective attention provides a means to specify what will be represented and what will not (Desimone & Duncan, 1995).

Reynolds, Chelazzi, and Desimone (1999) found that, in the absence of focused attention, neurons in V4 containing both effective and ineffective stimuli within their receptive fields produce responses that are roughly the average of the responses they would produce to either stimulus alone. When the task requires that one of the stimuli guide behavior, then the neuron's response more closely matches the response it would produce if that stimulus were present in isolation. In effect, when one of several competing stimuli is attended, the brain reconfigures itself in a way that modifies the response of the neuron (perhaps by changing synaptic efficacy) so that the attended, task-relevant object now determines the response of that neuron. The same selection occurs in a coordinated fashion throughout the visual hierarchy so that a coherent, distributed representation of the attended object is maintained.

Two categories of factors influence how perceptual selection is achieved. One is bottom-up, involuntary, and stimulus-driven and depends on physical salience—a property most often associated with contrast within one or more feature dimensions. Objects that are uniquely colored or that have unique motions express high contrast in those dimensions, and this permits them to compete more effectively with stimuli whose features are similar to those of other stimuli (Beck & Kastner, 2005). Itti and Koch (2000) proposed a model of stimulus-driven capture of attention that focuses on the role of local feature contrast in guiding attention.

The second kind of influence on selection is voluntary (often called top-down) and depends on the organism's behavioral goals. In the rest of this brief review, I will summarize recent efforts to understand two related aspects of voluntary selective attention: (a) the effects of selective attention on the magnitude and coherence of sensory representations in visually responsive regions of the brain; and (b) the sources of the top-down modulatory signals, including their relationship to neural systems subserving other domains of cognitive control, such as the control of eye movements (see Fig. 1).

Fig. 1.

Some of the regions in the human brain that are known to be modulated by attention (red) and regions that are thought to be sources of attentional control signals (blue). MT = middle temporal area; V1–V4 = primary and extrastriate visual areas.

It is important to keep in mind that the distinction between cortical sources and targets of attentional modulation is often far from clear-cut. Neurons in the earliest levels of the visual system (e.g., the lateral geniculate nucleus or LGN) are driven principally by properties of the visual scene and the retinal image, whereas neurons in the prefrontal and parietal cortex are more likely to be driven by behavioral goals and the reward value or meaning of stimuli in relation to those goals. Nevertheless, there is evidence for attentional modulation as early as the LGN (O'Connor, Fukui, Pinsk, & Kastner, 2002), and stimulus properties can influence neural responses in the parietal and prefrontal cortex. It is unlikely that a strict dichotomy between purely sensory neurons and purely goal-related neurons can be maintained, complicating any analysis of the control and effects of attention.

TARGETS OF ATTENTIONAL MODULATION

Behavioral studies of selective attention over the last 50 years have amply demonstrated attentional modulation within multiple perceptual domains including space, features, objects, and sensory modalities. Here I review a handful of recent studies of the brain systems that are thought to underlie these perceptual effects.

By far the most studied domain of selective attention is visual space, which was a focus of intense investigation following the early spatial-cuing studies in the 1970s. This focus is no accident: Vision is an inherently spatial sense, and the earliest cortical stages of visual representation are organized spatially. This provides a natural indexing system for targeting visual representations for attentional modulation. A large number of studies have documented that spatially focused attention modulates neural activity in the extrastriate cortex (e.g., Reynolds et al., 1999).

More recently, other domains of attentional selection that are not strictly spatial—including both feature-based and object-based attention—have been a focus of investigation. Feature-based attention refers to the selection of a stimulus based on the value it expresses within a feature dimension (e.g., red within the dimension of color, or upward within the dimension of motion). Treue and Martinez-Trujillo (1999), for example, recorded the activity of single neurons in the macaque monkey brain while the monkeys attended to the direction of motion of moving dots. They observed that the magnitude of the neural response in the motion-selective middle temporal area (termed area MT) depended on which direction of motion the monkey was attending to, despite the fact that the attended dots were spatially distant from the receptive field of the measured neuron. In effect, when the monkey was attending to, say, upward motion at one location in the visual field, then the responses of upward-preferring neurons to their preferred stimulus were enhanced for locations throughout the visual field. Saenz, Buracas, and Boynton (2002) made similar observations in humans using functional magnetic resonance imaging (fMRI). They suggested that this global feature-based enhancement may operate in conjunction with spatial-attention mechanisms to select task-relevant spatial and nonspatial sensory information.

Object-based attention, another nonspatial domain of attentional modulation, has been a focus of behavioral investigation for over 25 years. Recently several studies have begun to unravel how attention can be directed to one of two spatially superimposed objects. O'Craven, Downing, and Kanwisher (1999) showed observers spatially overlapping, semi-transparent house and face stimuli. At any given moment, they were to attend to either the house or to the face. One of the two stimuli was slowly oscillating at all times. The authors observed activity in face-selective and house-selective cortical regions that depended on which of the two stimuli was attended; furthermore, they found that the magnitude of the motion-driven signal in area MT also depended on whether the attended object was moving or not, suggesting that all the features of the attended object were selected, not just those required for the task.

There are, of course, many other examples of modulation of cortical activity based on spatial, feature-based, or object-based deployments of attention. The detailed mechanisms by which these modulations are achieved are a focus of intense scrutiny.

SOURCES OF ATTENTIONAL-MODULATION SIGNALS

That attention can modulate cortical activity in sensory regions is clearly documented. A consensus about the parts of the brain that are relevant to the control of attention is emerging. Early investigations of attentional control focused on the role of the posterior parietal cortex (PPC), principally because damage to right PPC (due to stroke, for example) often leads to unilateral visual neglect, which is thought to be a disorder in the ability to deploy spatial attention. More recently, several brain regions have been investigated for their contributions to attentional control; they include subregions of the PPC (including the lateral intraparietal area [LIP] within the intraparietal sulcus [IPS] and the superior parietal lobule) and subregions of the prefrontal cortex; including the frontal eye field [FEF] and the supplementary eye field), as well as the superior colliculus, a subcortical structure important for the control of voluntary eye movements.

Area LIP exhibits spatially specific increases in activity during tasks requiring sustained deployments of attention to spatial locations. Bisley and Goldberg (2003) found that neurons in the monkey LIP dynamically represent attended locations in the visual field, including both sustained, voluntary attention and transient, stimulus-driven attention to an abrupt onset. Several groups have shown (using human fMRI) that the IPS contains one or more spatial maps of attention. For example, Silver, Ress, and Heeger (2005) reported traveling waves of brain activity in the IPS during a task in which observers directed attention voluntarily to successive locations in the visual field. The frontal eye field also contains a spatial map of the locus of visuospatial attention (Hagler & Sereno, 2006).

Notably, FEF and LIP were first identified as critical for the control of eye movements. A longstanding and unresolved debate concerns the extent to which the control of visual attention is merely a side effect of preparing an eye movement that is not executed; this view is known as the “premotor theory of attention” (Rizzolatti, Riggio, Dascola, & Umilta, 1987). Evidence for this idea comes from experiments in which stimulation of neurons in FEF that is too weak to evoke an eye movement both enhances detectability of stimuli in the location to which that FEF neuron would drive an eye movement and potentiates the response of sensory neurons in V4 to an effective stimulus in its receptive field (Moore & Armstrong, 2003).

However, several recent studies have suggested that the deployment of attention can be dissociated from the preparation of an eye movement. For example, Zhou and Thompson (in press) trained monkeys to report (with an eye movement) which of two flashed dots was brighter. Before the targets appeared, a cue indicated where the two targets would appear. Neural activity was measured during the interval following the cue but before the targets appeared. FEF neurons exhibited an increase in activity when the cue indicated that a target was about to appear in the receptive field, but not elsewhere. Because eye movements were never made to the luminance targets, these changes in FEF activity could not be eye-movement preparation signals. This increase in activity was interpreted by the authors as providing an attentional control signal to early visual areas, enhancing the perceptual representations there and ultimately improving the accuracy of the required decision.

Paralleling the studies on the effects of attention in nonspatial domains reviewed earlier, several studies have begun to investigate the control of nonspatial attention. Liu, Slotnick, Serences, & Yantis (2003) required observers to shift attention between the color and the direction of motion of dots presented in a central aperture. These voluntary shifts of attention caused changes in the magnitude of activity in area MT, the locus of visual motion representation. Each shift of attention was associated with a transient increase in activity in the precuneus (medial superior parietal lobule or mSPL) that closely mirrored similar findings observed during spatial shifts of attention. Serences, Schwarzbach, Courtney, Golay, & Yantis (2004) observed similar transient increases in cortical activity in mSPL that were time-locked to shifts of object-based attention between spatially superimposed house and face stimuli. These transient signals in mSPL are thought to reflect a domain-independent signal to suppress the current stable state of attention and to specify a new state.

SYNERGIES BETWEEN COGNITIVE NEUROSCIENCE AND COGNITIVE PSYCHOLOGY

The scientific investigation of attention can be traced at least to Helmholtz in the mid-19th century, and until the 1970s, virtually all of the empirical and theoretical work in this area was based on behavioral evidence (with the notable exception of neuropsychological studies of visual neglect and related disorders). Single-unit recording (starting in the 1980s) and then functional neuroimaging (starting in the 1990s) brought new findings about the neural basis of attention and new ideas to the table. Donald Broadbent's influential notion of an all-or-none attentional filter (Broadbent, 1958) has been superseded by a better understanding of how the brain represents and processes sensory information (e.g., Desimone & Duncan, 1995). Distributed neural representations of objects, along with contextual modulation of neural activity and the need to select which of the objects and features in a receptive field will effectively drive neural activity, provide a framework for understanding well-characterized behavioral effects.

Behavioral studies of selective attention tend to precede investigations of its neurophysiological underpinnings. For example, behavioral studies of spatial attention began in the 1960s and 70s and continued through the 80s and 90s; neuroscientific studies of spatial attention followed some time later. Behavioral studies of object-based attention began in the 1980s and continued into the 90s; neurophysiological investigations began in the late 1990s.

Despite this typical pattern, certain kinds of questions appear to be addressable only through neuroscientific studies. For example, a deep understanding of why certain combinations of features can be detected efficiently in visual search while others cannot will, it seems to me, require a better understanding of how working memory representations of search targets are transformed into modulatory feedback signals that target sensory brain regions. Both behavioral and neuroscientific studies will be part of a complete framework for understanding selective attention.

OPEN QUESTIONS

I have reviewed evidence for a framework in which cortical representations of perceptual objects are modulated by attentional signals originating in brain regions representing perceptual goals. Many key questions remain.

First, how do the modulatory signals target just the locations, features, and objects that are currently task relevant? It is often assumed that there exist topographically organized “priority maps” in the prefrontal and/or parietal cortex (e.g., FEF and IPS) that send signals to cortical targets in spatially organized visual brain regions. The mechanisms of these feedback signals have not been specified in detail.

If the topographic cortical organization of spatial location (in vision) or frequency (in audition) is the basis for targeting to-be-attended attributes, then how do these modulatory signals target domains that are not topographically organized? For example, it is possible to attend to all the red items in a display, or to all the items moving upward. Either these attributes, too, have an as-yet not clearly delineated topographic organization in the cortex or there are additional principles to be discovered that will clarify how these domains are targeted.

At the highest level is the question of how abstract goals held in working memory are translated into targeted signals that can modulate cortical activity so that relevant sensory input is robustly represented and irrelevant input is not. An answer to this question will require a coordinated effort to account for the format of working-memory representations and their interaction with perceptual-attention mechanisms. Such efforts appear to be coming into focus now, but there remains much to learn.