Neuronal Mechanisms of Visual Attention

Abstract

Advances on several fronts have refined our understanding of the neuronal mechanisms of attention. This review focuses on recent progress in understanding visual attention through single neuron recordings made in behaving subjects. Simultaneous recordings from populations of individual cells have shown that attention is associated changes in the correlated firing of neurons that can enhance the quality of sensory representations. Other work has shown that sensory normalization mechanisms are important for explaining many aspects of how visual representations change with attention, and must be taken into account when evaluating attention-related neuronal modulations. Studies comparing different brain structures suggest that attention is composed of several cognitive processes, which might be controlled by different brain regions. Collectively, these and other recent findings provide a clearer picture of how representations in the visual system change when attention shifts from one target to another.

Introduction

The human brain processes the activity of millions of sensory receptors to create a representation of the sensory world that includes hundreds of millions of neurons in cerebral cortex. At any moment, however, we can act on only a tiny fraction of the available signals. Attention is the process of selecting a manageable subset of signals that is appropriate to guide action in the current behavioral context.

The neurophysiological mechanisms that underlie attention have long been a subject of study (e.g., Hubel et al. 1959). They have attracted increased effort in recent years partly because they hold the promise of providing insights into which components of neuronal activity are most critical for perception. Attention can profoundly impact behavioral sensitivity to sensory stimuli, and it can be studied under conditions where sensory and motor conditions are kept constant, making it possible to isolate changes in neuronal activity that correlate with the behavioral signatures of attention.

The neural correlates of attention are studied using a broad range of physiological approaches. In humans, they have been examined using EEG, ECoG, MEG and fMRI (see Carrasco 2011; Kastner & Ungerleider 2000; Posner & Rothbart 2007). It is only at the level of single neurons, however, where direct observations can be made about how attention relates to changes in neuronal receptive field size, selectivity and sensitivity. Because the attention literature is so extensive, here we will focus on single neuron data, concentrating on observations from the visual system.

Single neuron recordings from human brain are possible in rare situations (see Carrasco 2011; Kastner & Ungerleider 2000; Posner & Rothbart 2007), but animal studies provide the most precise, complete and well-controlled data on attention, and are consistent with the more limited recordings from humans. Signatures of visual attention in the activity of visual neurons have been studied in a range of species, including insects (Wiederman & O’Carroll 2013), but most work has been done in macaque monkeys because they are well suited to learning the challenging behavioral tasks that are needed for attention experiments. The organization of their brain is also similar to that of humans, and the mechanisms of attention are likely to be similar across primate species.

The visual system has been the favored sensory system for studying the neuronal correlates of attention. Sensory processing has been extensively studied in the visual system compared with other systems. This provides a large foundation of data about its functional organization, which can facilitate the design and interpretation of experiments on attention. In primates, far more neurons are devoted to vision than to the other sensory modalities (see Felleman & Van Essen 1991). This can mean that different visual structures have greater specialization and are more distinct from one another, making the interpretation of neuronal signals easier.

Several notable advances in understanding the neuronal basis of visual attention will be considered here. Recordings with microelectrode arrays that permit simultaneous recording from many neurons have revealed that attention is associated not only with changes in spike rates and spike rate variability in single cells, but also with changes in the correlated spiking of neuronal pairs or assemblies. These changes in correlated spiking can have important implications for the quality of sensory representations. Sensory response normalization is a phenomenon that scales the size of neuronal responses in many brain regions depending on visual context, and recent observations have linked the mechanisms of normalization and of attention. Additionally, new results raise questions about the precise relationship between attention and other cognitive phenomena examined in neurophysiological experiments, and suggest that attention is composed of several cognitive processes, which might be controlled by different brain regions.

Even with the scope limited to single neuron studies of visual attention, not all relevant findings can be considered here. Many excellent recent reviews provide other perspectives on the neuronal basis of attention (Bisley 2011; Carrasco 2011; Cohen & Maunsell 2014; Gottlieb 2014; Knudsen 2007; Treue 2014).

Attention and Changes in Visual Responses

It has long been known that shifts of attention toward or away from a visual stimulus are associated with changes in the way that neurons respond to that stimulus. These include modulations of the latency, magnitude and variability of neuronal responses. Such changes in neuronal responses are most easily described when a single stimulus is presented in a neuron’s receptive field and attention is directed toward or away from that stimulus. In that case, attention to a receptive field stimulus is usually associated with responses that are faster, stronger, and less variable compared to responses when attention is directed elsewhere.

In experiments with isolated stimuli, most neurons in visual cortex show relatively modest change in the magnitude of their responses (5-30%) when attention is directed toward or away from their receptive fields, but the magnitude of attention-related modulations of neuronal responses depends on several factors. One is the amount of attention that is directed to the stimuli. Attention is in some ways a limited resource, but behavioral studies have shown that demanding tasks can command more attention overall than less challenging tasks (Lavie & Tsal 1994; Swallow & Jiang 2013). Correspondingly, the response magnitude of individual neurons is more modulated when subjects perform more challenging attention tasks (Boudreau et al. 2006; Spitzer et al. 1988). Attentional effort likely accounts for why visual neurons respond more strongly when animals perform a visual task, compared to the presentation of the same retinal stimulus during periods when the subject is not working (Mountcastle et al. 1987).

The neuronal response modulation associated with attention also differs between structures in the visual system. When a subject performs a task with a given level of effort, attention-related changes are typically larger in later stages of visual cortex (see Maunsell & Cook 2002). Task difficulty and whether neurons are in earlier or later stages of the visual system presumably also influence the magnitude of attention-related modulation of latency and variability, but this question has not been examined in single neuron experiments.

In addition to response magnitude, response latencies also change with attention. Studies of V4 (Sundberg et al. 2012) and the middle temporal visual area (MT, Galashan et al. 2013) have shown that neuronal response latency becomes shorter as response magnitude grows with attention. The change in latency is small, however, amounting to only a few milliseconds. In particular, the latency change is much smaller than that which would occur if the firing rate were modulated by the same amount by adjusting the contrast of the stimulus with attention kept constant (Lee et al. 2007). Latency changes associated with changes in stimulus intensity arise in retinal mechanisms (Levick 1973; Maunsell et al. 1999). Those associated with attention arise from other mechanisms.

Attention is also associated with changes in response variability. When the same exact stimulus is presented repeatedly, sensory neurons characteristically respond to different stimulus presentations with different numbers of spikes. Typically, the variance of the spike count is approximately equal to the mean spike count (Tolhurst et al. 1983). This variability can be lower when attention is directed to a stimulus in a neuron’s receptive field. Although an early report failed to detect attention-related changes in the variance of neuronal responses in V4 (McAdams & Maunsell 1999b), a subsequent study of V4 using a more sensitive approach found that attention is associated with a small reduction in response variance (Mitchell et al. 2007). These authors separated V4 neurons with broad and narrow spikes and found that attention to a stimulus was associated with decreased variability in putative inhibitory (narrow-spiking) neurons, but not in the more common putative excitatory (broad-spiking) neurons, which maintained low variance in regardless of attention. Subsequent studies have also found attention-related decreases in variance in V4 (Cohen & Maunsell 2009; Mitchell et al. 2009), MT (Niebergall et al. 2011) and V1 (Herrero et al. 2013).

Correlations in neuronal responses also change with attention. As described above, individual neurons give variable responses to repeated presentations of the same stimuli. Such fluctuations in the response of pairs of nearby neurons are generally weakly correlated (see Cohen & Kohn 2011). Attention to a stimulus is generally associated with reduction in these pairwise correlations (Cohen & Maunsell 2009; Herrero et al. 2013; Mitchell et al. 2009; Zenon & Krauzlis 2012). This modulation can be proportionately larger than changes in rate of firing or individual variability, and can be the dominant factor in improving the quality of the population’s representation a visual stimulus (Cohen & Maunsell 2009). Reduced pairwise correlation in the spiking of neurons might be related to the long-standing observation that attention is associated with desynchronization of alpha band activity in the EEG (Ray & Cole 1985) and with reduced beta power in the local field potential (Fries et al. 2001).

While attention-related changes in latency, firing rate, individual variability and pairwise correlation might all stem from a common underlying mechanism, a recent study suggests that might not be the case. Although reducing pairwise correlations can be advantageous, in some cases stronger correlations can lead to better performance (Abbott & Dayan 1999; Averbeck et al. 2006). Ruff and Cohen (2014) trained monkeys to do a task where correlations between neurons representing different stimuli might be advantageous. They found that directing attention to such stimuli was associated with a small increase in correlations, rather than a decrease. This suggests that attention-related changes in pairwise correlations might be dissociated from other attention-related changes in neuronal activity and be adjusted in a flexible, strategic way when attention shifts.

Dynamics of Attention-Related Modulations

Most single neuron studies of attention compare average spike rates over extended periods when attention is directed to one stimulus or another. However, the essence of attention is that it can be rapidly and frequently deployed to different stimuli as task demands change. Relatively few single neuron studies have examined the dynamics of attention, but those studies have seen signatures of rapid changes in the allocation of attention.

An early study showed that V4 neurons modulate their rate of firing within 100-200 ms after the appearance of a cue to direct their attention to specific stimuli (Motter 1994). Rapid changes in modulation have subsequently been seen in V1 (Khayat et al. 2006) and MT (Busse et al. 2008) and the lateral intraparietal area (LIP, Herrington & Assad 2010; Bisley & Goldberg 2006; 2003). Notably, when the timing of the modulation of neuronal activity in different cortical visual areas is compared, changes appear 60 ms earlier in LIP than in MT (Herrington & Assad 2010), which lies at an earlier level of processing in visual cortex, consistent with top-down flow of signals related to the allocation of attention.

Other studies have shown that attention-related modulations of the firing of neurons in V4 (Ghose & Maunsell 2002) and LIP (Janssen & Shadlen 2005) vary over correspondingly brief intervals based on monkeys’ expectations about the probability that a target will appear. Presumably, the animals vary the amount of attention they devote to the task between periods of high and low target probability. Signals based on the analysis of populations of neurons suggest that the allocation of attention drifts considerably from trial to trial even when subjects are motivated to maintain relatively constant attention (Cohen & Maunsell 2010; 2011a; 2011b).

Collectively, these studies suggest that attentional changes can occur over a few hundred milliseconds or less, and that attention fluctuates over time, even when constant attention would be optimal for behavior.

Attention and Stimulus Tuning

When a single stimulus is placed in a neuron’s receptive field, attention is often associated with effects that are like an overall change in the gain the cell’s response. Gain changes lead to multiplicative scaling in neuronal responses, so the larger the initial response, the larger the change. Thus, changes in gain scale the maximum neuronal response amplitude while leaving the sharpness of neurons’ tuning curves unchanged. Examples of tuning curve invariance in different attention conditions have been shown for orientation tuning in V4 (McAdams & Maunsell 1999a), direction tuning in MT (Treue & Martinez-Trujillo 1999) and contrast response functions in MT (Lee & Maunsell 2010a). Multiplicative scaling has also been seen in cases where attention-related changes in the temporal integration window of neuronal responses have been examined. Attention is associated with a multiplicative scaling of the temporal integration window, and not changes in its shape (Cook & Maunsell 2004; McAdams & Reid 2005).

While the primary mode of action might be a simple gain change when neurons are presented with a single stimulus in their receptive field, attention is associated with more complicated changes in neuronal responses when multiple stimuli are presented in or near a cell’s receptive field. Moran and Desimone (Moran & Desimone 1985) were the first to describe large modulations of the responses of V4 and inferotemporal (IT) neurons that can occur when attention was shifted between a preferred stimulus and a non-preferred stimulus that were both placed within the receptive field of a V4 or IT neuron. Directing attention to the non-preferred stimulus typically was associated with reduced responses, even if the non-preferred stimulus was excitatory when presented alone. Modulations that depend on which receptive field stimulus the animal is attending to cannot be explained as a simple gain change. Subsequent studies have replicated this finding in these and other visual areas (Ghose & Harrison 2009; Ghose & Maunsell 2008; Lee & Maunsell 2010b; Luck et al. 1997; Ni et al. 2012; Reynolds et al. 1999; Treue & Maunsell 1996). Although this effect of a second stimulus was originally described as limited to situations where both are within the receptive field (Moran & Desimone 1985), it is now clear that similar effects are associated with shifting attention between one stimulus in a neuron’s receptive field and another lying in the surrounding region immediately adjacent the receptive field (Sundberg et al. 2009).

Another departure from a gain change has been seen in studies that have mapped the spatial distribution of receptive fields with attention directed to different locations in or around the receptive fields. Those measurements show that the center of gravity of a receptive field is often displaced toward the locus of attention (Connor et al. 1997; Womelsdorf et al. 2006), which cannot occur with a simple change in the overall sensitivity of the neuron. Attention has also been associated with shifts in the feature preferences of V4 neurons (David et al. 2008) and the temporal integration characteristics of MT neurons (Ghose & Bearl 2010), neither of which is consistent with a simple gain change.

Attention and Sensory Normalization

The complex effects associated with attention shifts when multiple stimuli are in a neuron’s receptive field have spurred the development of models that can accommodate those effects. Several groups have identified normalization as a critical factor in determining how the responses of cortical neurons vary between different states of attention (Boynton 2009; Ghose & Harrison 2009; Lee & Maunsell 2009; Reynolds & Heeger 2009). These normalization models show that a wide range of modulations associated with attention could be explained when sensory normalization is taken into account. This understanding has been an important step forward in making sense of how spike rates depend on the state of attention.

What is normalization? Normalization is not defined by a specific mathematical operation. Instead, it refers to a class of operations that relate values to some baseline. In the context of neuronal processing, normalization describes the way in which neurons integrate excitatory and suppressive inputs from multiple sources. In sensory systems, it generally applies to the way that neurons sum responses to individual stimuli when they are presented together, and in particular to address the non-linear summation of responses. Most neuroscience textbooks describe response summation in the visual system using the example of the linear summation of retinal ganglion cells or simple cells in V1 (e.g., Kandel et al. 2013). For these cells, the response to two stimuli presented together is typically the sum of the responses to those stimuli when they are presented in isolation. This emphasis on linear summation is unfortunate, because for the vast majority of visual neurons, responses to two stimuli presented together is sublinear (less than the sum of the individual responses). Instead, their response is described by a normalization in which the summed response is reduced in proportion to the number of stimuli contributing to the response. Because of this normalization, adding a weakly excitatory stimulus to a strongly excitatory stimulus frequently reduces rather than increases a neuron’s response. The response of a typical visual neuron to multiple stimuli is in general closer to the average of its responses to those stimuli presented individually than it is to the sum of those responses. Normalization also explains why contrast response functions saturate at high contrasts (Heeger 1993). It is likely that normalization dominates most input summation in the nervous system (Carandini & Heeger 2012).

Normalization Models and Attention

Normalization is described compactly by divisive normalization (Heeger 1993), and most models of normalization are based on this approach. Attention has been incorporated into normalization models by allowing attention to one stimulus to selectively modulate the inputs related to that stimulus. Normalization models that incorporate attention explain many aspects of the way that neuronal firing rates change with attention. In particular, they explain why responses change markedly when attention is shifted between preferred and non-preferred stimuli that both lie within a neuron’s receptive field. Although this effect was previously described as a shrinking of the receptive field around the attended stimulus (Moran & Desimone 1985) or a biased competition (Desimone & Duncan 1995), it is more comprehensively explained as a consequence of normalization. Normalization explains why attention to a non-preferred stimulus can reduce a neuron’s response (Boynton 2009; Ghose & Harrison 2009; Lee & Maunsell 2009; Reynolds & Heeger 2009). It also explains why response modulation associated with attention to a receptive field depends on the contrast of a second stimulus in the receptive field (Khayat et al. 2010; Lee & Maunsell 2010a). Importantly, normalization is also consistent with more modest, gain-like changes in neuronal responses when a receptive field is presented with a single stimulus.

The ability of normalization to explain the large response modulations seen when attention is shifted between preferred and non-preferred stimuli in a neuron’s receptive field was demonstrated directly for responses of neurons in MT (Lee & Maunsell 2009; Ni et al. 2012). This was done by first measuring how normalization affects responses to preferred and non-preferred stimuli presented together in the receptive fields of MT neurons while attention was directed to a distant location. Then attention-related modulation of responses to individual stimuli was measured. Each cell had a fixed amount of normalization, and a specific gain change associated with attention to individual stimuli. The responses of individual neurons when attention was directed to either the preferred or the non-preferred stimulus were well-described using the measured normalization and attention-related gain changes in the responses to individual stimuli.

One notable feature of normalization in MT is that it varies greatly between individual cells (Ni et al. 2012). For most MT neurons, the response to a preferred stimulus is reduced when a non-preferred stimulus is added to the receptive field, even if the non-preferred stimulus is excitatory by itself. This is expected from the way that normalization produces responses that approximate the average of the responses to the individual stimuli. For other MT neurons, however, the response to a preferred stimulus is unaffected by the addition of a non-preferred stimulus. For these cells, it is as if the non-preferred stimulus is unable to engage normalization. This difference between cells can be explained if the normalization mechanism for some MT neurons is tuned, and engaged only by stimuli moving in a direction close to the neuron’s preferred direction (Ni et al. 2012). Because normalization is engaged only weakly in neurons with tuned normalization, those neurons are expected to show less modulation when attention is shifted between preferred and non-preferred stimuli in their receptive fields. That relationship is found among MT neurons.

Because normalization can effectively amplify attention-related modulations, it might contribute to some of the difference in attention-related modulation in different visual areas. Attention-related changes are stronger in later stages of visual cortex in both monkey single neuron (see Maunsell & Cook 2002) and human fMRI (O’Connor et al. 2002) measurements. Later stages of visual cortex have neurons with larger receptive fields, for which it is more likely that edges of a display screen or other features of the environment will encroach on the receptive field or the adjacent receptive field surround. The presence of unintended secondary stimuli in or near a receptive field creates a situation in which normalization can enhance attention-related modulations. It is possible that some of the more pronounced modulations seen in later stages of visual cortex depend on the larger receptive fields in those stages allowing normalization to amplify modulations.

Normalization and Contrast and Response Gain

Normalization models have played a role in evaluating whether the changes in neuronal responses that occur with attention mimic changes in the contrast of a visual stimulus. To some extent the stronger neuronal responses and superior behavioral performance linked to greater attention are similar to what would be seen if the contrast of the stimulus were increased instead. However, the neuronal and behavioral changes associated with attention could arise from a variety of mechanisms, and it would be surprising if the correspondence between stimulus contrast and attention were precise. A strict equivalence between attention and stimulus contrast would require that attention did not engage cortical mechanisms directly, and instead was involved with modulations at the earliest stages of the visual system that were propagated to later stages. Feature-based attention would be difficult to implement in this way.

In fact, it is clear that contrast and attention are not equivalent, and produce several divergent effects on neuronal responses. As described above, changes in neuronal response latency associated with attention are much smaller than those associated with changes in stimulus contrast that produce the same difference in firing rates (Lee et al. 2007). Attention and stimulus contrast also have opposite effects on the amplitude of the early stimulus-evoked LFP in area V4 (Sundberg et al. 2012).

The idea that attention is equivalent to a change in stimulus contrast is captured by contrast gain, which proposes that the responses associated with attention are those that would occur if the contrast of an unattended stimulus had been multiplied by a fixed value. Because the contrast response functions of visual neurons typically saturate at high contrasts, contrast gain implies that attention will be associated with proportionally smaller increases in firing rate for high contrast stimuli because changes in contrast have little effect in that range. Consistent with this, Reynolds and colleagues (Reynolds et al. 2000) found proportionally greater effects in area V4 for stimuli of intermediate contrasts rather than high contrasts. Contrast gain was also found the responses of neurons in MT (Martinez-Trujillo & Treue 2002). However, a similar study of V4 neurons found the greatest modulation was at the highest contrasts, where responses were strongest (Williford & Maunsell 2006), as if responses to all contrasts were affected proportionately (termed response gain). A study of MT responses that was designed to produce large attention-related modulations and neuronal responses that were far into the saturating range of the contrast response function similarly found modulations that were strongest for the responses to the highest contrasts (Lee & Maunsell 2010a).

The normalization model of Reynolds and Heeger (Reynolds & Heeger 2009) can explain why neurons might show contrast gain in some attention experiments and response gain in others. Owing to the structure of this model, when attention is distributed over a region much larger than the stimulus, shifting attention can affect neuronal responses like a contrast gain. Alternatively, if attention is distributed over a region that is small relative to the stimulus, shifting attention can affect neuronal responses like a response gain. No single neuron study has tested this prediction by measuring the extent over which attention is distributed, and it is not clear that the stimuli used in the experiments that found differing results would have encouraged the relationships suggested by the model. Nevertheless, in support of the model, experiments with human subjects that have controlled the spatial extent of attention have shown that human performance followed the expected shifts between contrast gain and response gain (Herrmann et al. 2010).

The data described here suggest that neuronal response changes associated with attention are generally neither a pure contrast gain nor pure a response gain. Indeed, for reasons that are not understood, individual cells recorded using the same animal and conditions can show signs of either property (Williford & Maunsell 2006). It seems clear that attention can be associated with either contrast gain or response gain. Nevertheless, the two effects might not be equally prevalent or equally potent. Across the human and non-human studies, response gain appears to be a larger and more robust phenomenon, sometimes enhancing the response to a given stimuli by 50% or more (Herrmann et al. 2010; Lee & Maunsell 2010a). When contrast gain effects are described, they are more modest (e.g., Reynolds et al. 2000; Martinez-Trujillo & Treue 2002). Further experiments will be needed to resolve how task dependent the modulation is, and the relative dominance of contrast gain and response gain in typical viewing situations.

Biased Competition and Feature Similarity Models

Biased competition (Desimone & Duncan 1995) has been a popular framework for describing selective visual attention. Biased competition has not been associated with a specific model or set of equations, but is instead descriptive in characterizing attention as a competition between visual objects for a limited number of representations in cerebral cortex. Receptive fields of visual neurons in particular are viewed as the critical visual processing resource, with objects in the visual field competing for dominance of a neuron’s response. Which objects dominate a neuron’s response is determined in part by the visual properties of the stimulus (bottom-up bias) and an attentional template for behaviorally relevant stimuli (top-down bias).

Because biased competition described a range of phenomena related to attention, including responses when attention is shifted between two stimuli inside a receptive field (e.g., Moran & Desimone 1985), it is unsurprising that it is consistent with normalization models of attention. Normalization models capture bottom-up bias as stimulus intensity terms, and capture top-down bias as attention terms that adjust the weighting assigned to different stimuli. Indeed, an early model designed to capture biased competition used normalization (Ghose & Maunsell 2008). Thus, normalization models of attention can be viewed as explicit, comprehensive models of the phenomena laid out in descriptive terms as biased competition.

The feature-similarity model of attention (Martinez-Trujillo & Treue 2004; Treue & Martinez-Trujillo 1999) posits that attention to a feature is associated with modulation of the gain of a neuron’s responses according to how much the neuron prefers the attended feature. Feature similarity accounts for the observation that attention to a receptive field increases responses to either a preferred or a non-preferred stimulus by approximately the same factor. However, in its simplest form, feature-similarity has difficulty when preferred and non-preferred stimuli are presented simultaneously in the receptive field. The expectation of the same modulation as seen with single stimuli is not met: gain is increased if a preferred stimulus appears at the locus of attention, and decreased if a non-preferred stimulus appears there (Lee & Maunsell 2010b; Ni et al. 2012). Normalization models capture this situation with a single parameter modulating the gain of inputs according to which match the attended location or feature (Boynton 2009; Ghose & Harrison 2009; Lee & Maunsell 2009; Reynolds & Heeger 2009). If the gain in feature-similarity is taken to apply to the inputs to a neuron (Patzwahl & Treue 2009), it then provides a good account for how various inputs to a neuron are modulated with attention, whether that attention is to a spatial location, a visual feature, or both.

The Relationship between Attention and Sensory Normalization

Normalization models provide a compact and relatively complete explanation for changes in the rate of firing of a neuron when attention is directed toward or away from one or more stimuli in its receptive field. Satisfactory descriptions of attention-related modulations with multiple stimuli seem to require that sensory normalization be included. Sensory normalization clearly plays an important role in determining attention-related modulations, but what is the relationship between them? The answer remains to be determined, but it is important to recognize that the data described above are consistent with no interactions between the mechanisms underlying attention-related modulations and normalization.

All of the normalization models of attention keep the process of normalization fixed between different states of attention. In most implementations of these models, attention does not alter the normalization mechanism, but only alters the inputs to the normalization. Depending on the specific stimuli configuration, normalization might increase the modulation seen with attention, but attention does not change the nature of the normalization. While it is essential to factor in normalization to predict how a neuron will change its rate of firing in different attention conditions, normalization is a separate stage of processing that comes after attention-related modulation of the signals that are the inputs to normalization. It is possible that attention is associated with changes to the way normalization occurs, but current measurements lack the sensitivity to detect those changes. An understanding of the full relationship between attention and normalization is likely to require a much better grasp of the microcircuitry and computations that underlie each.

Before leaving the subject of normalization and attention, it is worth noting that the short timescale dynamics of attention-related neuronal activity are not well understood. When an intense visual stimulus is presented, attention-related modulations are often absent during the initial 50-75 ms of a neuron’s response, only emerging in the weaker, sustained portion of the response (e.g., Lee & Maunsell 2010b). This time-varying modulation is unlikely to be related to changes in top-down signals related to attention. In these experiments attention is directed toward the cued side before, during and after the stimulus presentation, and if the intensity of the stimulus is reduced enough to eliminate the initial overshooting transient of the response, attention-related modulations can be seen throughout (Lee et al. 2007). Instead, it seems likely that rapid increases in excitation affect local circuits in ways that mask differential inputs related to attention, perhaps owing to delays in normalization-related signals when an intense stimulus appears. Exploring the rapid dynamics of cortical circuits in attending subjects might provide valuable insights into the mechanisms by which signals are modified by attention (e.g., Louie et al. 2014).

Neuronal Origins of Spatial Attention

The question of which brain structures give rise to neuronal signals related to attention is frequently raised. Attention-related signals are widespread in the brain, and there is no reason that they must have a discrete or consistent source. They are usual thought to arrive in visual cortex from higher centers, a view that is supported by the relative timing of modulations in earlier and later visual areas (Herrington & Assad 2010).

Moore and his colleagues have shown that manipulating the activity of neurons in the frontal eye fields (FEF) can mimic some of the behavioral effects of attention. Weak electrical microstimulation or local blockage of dopamine D1 receptors in the FEF in monkeys performing a visual detection task can improve detection of targets in a position corresponding to the affected visual field representation (Moore & Fallah 2001; 2004; Noudoost & Moore 2011). FEF microstimulation also enhances a motion-induced position illusion (Schafer & Moore 2007), paralleling the effects of increased attention to the stimulated visual field representation. Additionally, the responses of V4 neurons with receptive fields in the appropriate location are modulated by either FEF microstimulation (Armstrong et al. 2006; Moore & Armstrong 2003) or local D1 receptor block (Noudoost & Moore 2011), much as might occur when attention is directed to that location. Corresponding activation of human visual cortex has been seen using fMRI in conjunction with transcranial magnetic stimulation of the FEF (Ruff et al. 2006).

While these results suggest that the FEF might serve as a source for attention-relates signals, equivalent microstimulation experiments in the superior colliculus (Cavanaugh et al. 2006; Cavanaugh & Wurtz 2004; Muller et al. 2005) and posterior parietal visual cortex (Cutrell & Marrocco 2002) have produced equivalent behavioral effects. Such diverse structures producing behavioral enhancement underscores the uncertainties inherent in attempting to identify sources of attention-related signals with this approach. It is possible that subthreshold microstimulation of V4 or other relatively early visual area could also enhance behavioral performance in similar ways.

There is also considerable interest in which circuit mechanisms and neurotransmitters are involved in attention-related modulations of neuronal activity. Attention to a visual field location has been shown to enhance the efficacy of thalamocortical synapses in V1 (Briggs et al. 2013). Cholinergic inputs to cerebral cortex have been implicated in attention (Sarter et al. 2005), and acetylcholine agonists increase responses and enhance the contrast sensitivity of neurons in V1 in a manner similar to the effects seen with attention (Disney et al. 2007). Pharmacological intervention in V1 of monkeys performing an attention task has shown that changes in firing rate depend on muscarinic cholinergic mechanisms (Herrero et al. 2008). In contrast, attention-related changes in response variance and pairwise correlations appear to depend on ionotropic glutamate receptors (Herrero et al. 2013).

Attention to Visual Features and to Other Sensory Modalities

Although most single neuron studies of attention in the visual system involve attention to visual field locations, attention can also be directed to specific visual features without reference to spatial location. For example, visual search tasks typically challenge a subject to find a target that has a specific feature such as a color, size, or shape. Such tasks require that the subject examine visual field locations with attention to the target feature, typically ignoring other features. Many human psychophysical studies have shown that feature-based attention enhances behavioral performance throughout the visual field (e.g., Rossi & Paradiso 1995; Saenz et al. 2003; but see Theeuwes 2013).

Feature-based attention has been shown to modulate the activity of individual neurons in many visual areas, including V4, MT and IT (see Maunsell & Treue 2006). The many similarities of spatial and feature-based attention led to the proposal that they were different aspects of the same neurobiological mechanism (Treue & Martinez-Trujillo 1999). In this view spatial location is a feature like any other, and the modulation of a neuron’s responses is proportional to the extent to which currently attended visual features match the stimulus preferences of that neuron.

Because most visual structures have some visuotopic organization, spatial attention necessarily involves neurons that have a localized distribution within visual areas. That is not true for feature attention, which affects responses across the entire visual field. However, except for that difference in the cortical distribution of effects, the mechanisms supporting spatial and feature-based attention might be the same. Experiments that have examined attention-related changes in pairwise correlations in the responses of neurons support this view. As described above, attention to a visual stimulus is associated with a reduction in pairwise correlations between neurons that represent that stimulus (Cohen & Maunsell 2009; Mitchell et al. 2009). When the relationship between changes in rate of firing and changes in pairwise correlation are examined separately for spatial and feature attention, the same quantitative relationship is seen for both types of attention (Cohen & Maunsell 2011a). This consistency supports the view that both forms of attention depend on similar mechanisms.

In everyday life, it is also common to shift attention between different sensory modalities, for example, attending in turn to the color, weight or temperature of a coffee cup. Many studies have shown that shifting attention between sensory modalities is associated with changes in neuronal activity consistent with those described here (Lakatos et al. 2009; see Lee et al. 2014). It is possible that the same mechanisms that divert attention from one visual feature to the next would serve to shift attention between visual and auditory features. Similarly, it seems plausible that the mechanisms that modulate visual responses related to spatial attention could equally underlie modulations of representations of auditory space. It is conceivable that attention to abstract concepts involves the same mechanisms to engage higher representations in regions like prefrontal cortex. Whether the modulations in visual cortex associated with attention are part of a mechanism that acts across all of cerebral cortex remains an important question.

Defining Attention

Considerable progress has been made on understanding the neurobiological mechanisms that underlie attention, but we are still far from a complete description of what produces the behavioral phenomena we associate with attention. A major obstacle to understanding is our limited grasp of what attention is. After more than 100 years of behavioral studies and more than 50 years of neurophysiological studies, we continue to rely on operational definitions of the mechanisms we wish to understand.

Anderson (2011) provided a thoughtful critique of problems related to experimental approaches to attention, addressing in particular the tendency for investigators to treat attention as a cause of brain processes, rather than an effect. For example, while it is natural to say that “attention increases the selectivity of visual neurons,” that thinking leaves attention as mysterious as a homunculus, and provides no insights beyond those that can be gleaned from the less loaded view that changes in selectivity of visual neurons effect the behavioral consequences that define attention.

The terminology we use to describe attention and related phenomena is vague. On the one hand, it is likely that we fail to appreciate distinct aspects of attention. On the other hand, we might be using different terms to describe a common mechanism. With regard to the latter, there are important issues regarding the relationship of attention to reward expectation and priority maps.

Reward Expectation

Single neuron experiments that examine attention typically involve animals working for a water or juice reward, with attention controlled by changing either the size or the probability of a reward associated with one stimulus. It has been noted that these experiments cannot dissociate attention from reward expectation (Maunsell 2004) and neurophysiological experiments that approach either reveal equivalent effects (Stanisor et al. 2013). Attention can be decoupled from external measures of reward expectation. For example, a completely unmotivated subject will not attend to a rewarded stimulus at all. But the situation is different if we consider the broadest definition of reward, one that includes not only physical rewards but also factors like preferences for novel stimuli, satisfaction from performing well and desire to complete a day’s work. When reward is defined to include all motivating factors, there is little basis for distinguishing between attention and reward expectation. It is difficult to see why attention would ever be distributed differently than the subject’s current view of what is expected to be associated with reward (broadly defined). Thus, the phenomena explored in some experiments on attention might involve the same neurobiology and behaviors explored in some experiments on reward expectation. This view is consistent with the observation that experiments showing that the neural correlates of reward (Hikosaka 2007; Serences 2008) and attention (Corbetta & Shulman 2002; Serences et al. 2005) both involve similar brain regions. A deeper understanding of which aspects of attention and reward expectation can be separated might lead to richer and more incisive experimental questions.

Priority and Salience Maps

Issues exist with other terminology related to attention. For example, it has been suggested that neurons in LIP create a “priority map” (see Bisley & Goldberg 2010), such that their neurons represent stimuli according to their behavioral priority, which would include both sensory and cognitive inputs. This priority map would then guide either visual attention or saccade planning. The concept of a priority map, or the closely related salience map (Itti & Koch 2000; Koch & Ullman 1985; Walther & Koch 2006), is well defined, but its distinction from the allocation of spatial attention remains unclear. It has been shown that behavioral performance at different visual field locations closely follows relative neuronal activity in the corresponding visual field representations in LIP (Bisley & Goldberg 2003; 2006), consistent with the idea that LIP provides a priority map that drives spatial attention. What has not been shown is whether spatial attention is ever dissociated from activity in LIP. It is not clear why the allocation of spatial attention would ever depart from the output of a priority map, or what would control visual attention when a priority map did not. Thus, the concepts of priority and attention are often used when describing the same phenomena.

A further problem with priority or salience maps is that virtually every visual structure would comprise a different priority map. The defining characteristic of these maps is that they combine bottom-up stimulus salience with top-down behavioral significance to produce a presentation that embodies both. However, neurons in almost every stage of the visual system are affected by both bottom-up and top-down influences to varying degrees (Cook & Maunsell 2002; Fecteau & Munoz 2006), including stations as early as the lateral geniculate nucleus (LGN, O’Connor et al. 2002). Correspondingly, many different areas have been proposed to constitute a priority or salience map (Balan & Gottlieb 2006; Bisley & Goldberg 2010; Chelazzi et al. 2014; Kusunoki et al. 2000; Li 2002; Mazer & Gallant 2003; Thompson et al. 2005). This multiplicity undercuts the hypothesis that either saccades or covert attention can be simply allocated to the peak of “the” map (Bisley 2011). At any given moment, different stimuli will be associated with the peak activity in different visual structures depending on the relative weighting of bottom-up and top-down influences in each map.

Conceivably one visual field map is the master. If so, that master has not been identified conclusively. Alternatively, the priority map can be viewed as distributed, with different areas serving to guide behaviors depending on the relative weighting assigned to bottom-up and top-down considerations at a given moment (Fecteau & Munoz 2006; Treue 2003). However, that would undermine the power of the priority map concept considerably. The notion that the peak of activity in any map governs behavior is itself questionable. There is little evidence for such “winner-take-all” operation in the nervous system. For example, the superior colliculus is intimately involved with the generation of saccades in primates, and has a well-organized map of saccade direction and magnitude (Schiller & Stryker 1972; Wurtz & Goldberg 1972). Nevertheless, saccades are not simply made to the location corresponding to the most active representation in the superior colliculus. Several experiments have shown that saccade direction and magnitude is a vector average of all the active representations across the colliculus at the time when the saccade is triggers (Lee et al. 1988; see Lee & Hall 2006).

Distinct Spatial Attention Mechanisms

The superior colliculus was one of the first structures where neuronal responses to visual stimuli were shown to vary with behavioral state (Goldberg & Wurtz 1972; Wurtz & Mohler 1976). Many studies have described attention-related modulation there (see Krauzlis et al. 2013). The central role of the colliculus in triggering and organizing saccades and other orienting movements (see Sparks 1999) suggests that it might operate at a level downstream from cerebral cortex and the attention-related modulations seen there. However, recent results from Krauzlis and his colleagues suggest that the colliculus is critical for visual spatial attention as well as orienting movements.

Using a clever design that cleanly separated covert visual attention from saccadic eye movements, Krauzlis and Lovejoy (Lovejoy & Krauzlis 2010) showed that pharmacological block of a portion of the colliculus in monkeys impairs the animals’ ability to selectively attend to the affected representation. Subsequent single neuron recordings showed that while the colliculus was blocked neurons in cortical areas MT and the medial superior temporal area (MST) continued to show normal attention-related modulations even when behavioral performance showed that control of attention was profoundly impaired (Zenon & Krauzlis 2012). This was a notable result, because it showed that visual spatial attention could be impaired even when cortical correlates of attention are intact.

These investigators have suggested several interpretations of this result (Krauzlis et al. 2013), including the possibility that distinct circuits mediate different aspects of visual spatial attention. Attention is usual treated as a unitary phenomenon, perhaps because its essence is often viewed as reducing a large set of possibilities down to a single commitment. However, intuition is often a poor guide to brain mechanisms. It has been argued that attention should not be viewed as a single process (Chun et al. 2011). Most people would describe their vision as unitary, although we know that different cortical pathways mediate vision for navigation and vision for recognition (Merigan & Maunsell 1993), and a function like visual recognition is supported by many distinct areas that contribute in different ways (Felleman & Van Essen 1991; Grill-Spector & Malach 2004). Visual attention might similarly be the product of distinct circuits and mechanisms that contribute components of attention that are distinct. If the colliculus and visual cortex make distinct contributions, attention-related modulations in the colliculus might persist while attention was perturbed by blocking neuronal activity in visual cortex. There is little reason to believe that all the attention-related signals seen throughout the visual system convey the same information. Whether visual spatial attention, or other forms of attention, can be divided into distinct components remains an important question for future research.

Concluding Comments

There is great potential for rapid progress toward an understanding of the neurobiological and conceptual underpinnings of attention. Advancement is likely to depend on collective effort from multiple fronts. In particular, genetic tools now make it possible to monitor the activity of all the neurons in small brain circuits in behaving animals, and to control the activity of nearby or distant subsets of cells that have been selected based on either the genes they express or the sites to which they send axonal projections. With such powerful techniques in hand, it becomes feasible to design experiments to conclusively reveal the structures and cells that are critical to attention-related modulations in visual cortex and elsewhere. It might also be possible to dissect the neuronal mechanisms subserving sensory normalization and establish precisely how they related to response modulations related to attention.

Another area ripe for progress is in refining the concepts we use in exploring higher brain functions. The vocabulary we use to discuss attention, decisions, reward and motivation is informal and driven by intuitions that have advanced little over the years. Study of memory was greatly facilitated by the recognition that memory comprises a set of different memory systems that are specialized for different types of memory and served by different neurobiological mechanisms (Squire 1986). While there have been calls for a taxonomy of attention (e.g., Chun et al. 2011), it remains elusive. Nevertheless, given the number of laboratories now focusing on attention, reward and decision-making, and the high quality of the work produced, we can hope for progress toward a refined understanding of higher brain mechanisms in the near future.