The mechanisms by which attentional feedback shapes selectivity

Abstract

The glut of information available for the brain to process at any given moment necessitates an efficient attentional system that can ‘pick and choose’ what information receives prioritized processing. A growing body of work, spanning numerous methodologies and species, reveals that one powerful way the act of attending to an item separates the wheat from the chaff is by altering a basic response property in the brain: neuronal selectivity. Selectivity is a cornerstone response property, largely dictating our ability to represent and interact with the environment. Although it is likely that selectivity is altered throughout many brain areas, here we focus on how directing attention to an item affects selectivity in the visual system, where selectivity is generally more well-characterized. First, we review the neural architecture supporting selectivity, and then discuss the various changes that could occur in selectivity for an attended item. In a survey of the literature, spanning neurophysiology, neuroimaging and psychophysics, we reveal that there is general convergence regarding the manner with which selectivity is shaped by attentional feedback. In a nutshell, the literature suggests that the type of changes in selectivity that manifest appears to depend on the type of attention being deployed: whereas directing spatial attention towards an item only alters spatial selectivity, directing feature-based attention can alter the selectivity of attended features.

Introduction

The act of attending allows us to selectively enhance relevant information and filter out irrelevant noise, which from a signal-processing perspective has ubiquitous functional utility. Although attentional modulation clearly plays a universal role in shaping cognition, the field is only beginning to understand the mechanisms by which the deployment of attention guides behavior. In recent years, however, exceptional empirical and theoretical gains have been made in characterizing the mechanisms by which attentional feedback alters vision, with a growing body of convergent evidence from a variety of methodological approaches. In this review, we lay out what we have learned about the mechanisms underlying attentional modulation, focusing on the early-level visual system. Why focus on vision? Our cognitive abilities hinge on the quality of neural representations, which are largely dictated by one key factor: selectivity. Neural selectivity largely determines our ability to represent and interact with the environment. Neuroscience is in the nascent stages of understanding the selective response properties for the vast majority of the brain, with the exception of a handful of brain regions, such as visual cortex. Decades of work in visual neuroscience has yielded an unsurpassed understanding of the response properties and neural circuitry within visual areas, thus providing a solid foundation for understanding the neural computations carried out by attention. Focusing on vision is valuable not only because it serves as a convenient test bed, but also because changes within early visual cortex can have a tremendous impact on virtually all downstream stages in the information processing hierarchy. The visual system serves as the ‘gatekeeper’ for information, and even subtle changes in the properties of visual processing can snowball into substantial changes in the quality with which we process information downstream in higher-level subsystems, such as object recognition and categorization.

This review will describe the current state of research in visual attention, encompassing work that spans neurophysiology, neuroimaging and psychophysics --all aimed towards understanding how selective attention contributes to our ability to perceive our environment. In particular, we focus on studies that have parametrically measured visual selectivity. Reviewing the literature with this framework in mind has numerous advantages. First and foremost, this approach allows us to simplify the problem. Because the selective properties of vision are relatively well understood, we can define the handful of ways these response properties can be augmented by attentional feedback. Consequently, the number of potential mechanisms at play with attentional modulation becomes constrained to a much smaller subset of biologically plausible mechanisms. An added advantage of this approach is that it facilitates building and testing of computational models. By parametrically testing how attentional feedback influences core response properties, these studies allow us to synthesize empirical results into testable computational models, with which we can pinpoint the neural mechanisms that underlie attentional modulation. To that end, we approach the review with the perspective that physiological constraints in the formation and malleability of selectivity may impose important theoretical limits on what changes in selectivity are plausible with attentional feedback, and what changes are not.

In our review of the literature we reveal that, generally speaking, there is consensus regarding how attentional feedback affects cortical selectivity. These findings often appear to go beyond methodology used (neuroimaging, electrophysiology and behavioral psychophysics), and even species (human and non-human primates). On the one hand, attentional feedback has been shown to cause a consistent change in selectivity when it is directed to a specific feature of the stimuli in the visual environment. On the other hand, attention directed to a spatial location shows scant modulatory effects, if any, on cortical selectivity for features, but does seem capable of changing the spatial layout of the receptive field. We theorize how these different effects might be seen as a natural development of the structure of the neuronal architecture supporting visual selectivity.

Selectivity

While the quality of feature processing is influenced by a handful of factors, including the responsivity and variability of the neural response, this review focuses on the impact attention has on visual selectivity. Almost every sensory neuron is selectively tuned to some assemblage of inputs, whether it be selectivity for simple visual features such as oriented lines (Hubel and Wiesel 1959; Blakemore and Campbell 1969; Sclar and Freeman 1982; Ferster and Miller 2000) or selectivity for objects (Malach et al. 1995; Kanwisher et al. 1997a; Kanwisher et al. 1997b; Grill-Spector 2003). Fully characterizing the selective properties of neurons throughout the brain, however, is a daunting task for neuroscientists because selectivity is incredibly multidimensional, involving complex combinations of inputs, particularly in higher-order processing stages within the brain. That being said, quite a bit is known about the selective properties within early visual cortex, where the inputs are more easily quantified as lower-order properties of the visual input. One of the most well-studied properties in vision is orientation selectivity (Figure 1a). Neurons in early visual cortex preferentially respond to edges or contours of a particular orientation (Hubel and Wiesel 1959; Blakemore and Campbell 1969; Sclar and Freeman 1982; Ferster and Miller 2000). As a population, neurons in visual cortex are selective for all manner of orientations, which taken together support the detection of contours and shapes in our environment --the cornerstone features of vision. The model for how orientation arises in the cortex, which was originally championed by (Hubel and Wiesel 1962), remains a gold standard for models of neural selectivity.

Figure 1. Orientation selectivity in visual cortex.

A) A model example of a typical orientation tuning function. This example neuron prefers vertical orientations. The bandwidth of orientation tuning in V1 has been shown to be ~30–40° (full width at half max). B) The excitatory convergence model for orientation selectivity. Here, we illustrate three LGN neurons and their circular receptive fields, which happen to lie vertically next to each other. The afferents from a set of LGN neurons converge onto a V1 neuron, granting its orientation selectivity -- in this example, for vertically oriented contours. C) As a population, sensory neurons in V1 are selective for all manner of orientations, and their combined activity forms the basis for our ability to analyze visual scenes. Take, for instance, this desert scene: channels selective for horizontal orientations respond to the clouds and the horizon, while the vertical detectors respond to the cacti. Their combined activity enables us to detect these elements in the desert scene.

Almost every major model of orientation selectivity is a variant of the ‘aligned convergence’ model (Hubel and Wiesel 1959; Hubel and Wiesel 1962; Chung and Ferster 1998; Ferster and Miller 2000; Ling, Pearson & Blake, 2009). Under this model, receptive fields of neurons in the LGN are assumed to have very little orientation selectivity, reflected in their circular ON-OFF receptive field structure. Importantly, the model assumes that the group of LGN neurons that share afferent connections into a V1 neuron possess receptive fields that are spatially co-aligned, with ON and OFF regions that tend to fall along a common axis (Jin et al. 2011). This coalignment, in turn, grants V1 neurons orientation selectivity (Figure 1b). While there remains some debate regarding whether orientation selectivity arises purely from the structure of feedforward afferents (Anderson et al. 2000a; Anderson et al. 2000b; Ferster and Miller 2000; Priebe and Ferster 2006; Finn et al. 2007), or whether intracortical activity plays a role in sharpening orientation processing (Ringach et al. 1997; Sompolinsky and Shapley 1997; Pugh et al. 2000; McLaughlin et al. 2000), almost all models begin with the notion of aligned convergence.

The specificity with which a neuron, or population of neurons, responds to a particular feature can have a substantial impact on the quality of a neural signal. For instance, if neurons shifted their selective properties, this could substantially change the neural representation of the signal. A shift in selectivity effectively changes the ‘channel’ to which the neuron is tuned. Shifting the preference of neurons towards a feature, such as a given orientation, would lead to an increased number of detectors sensitive to that feature. This would, in turn, bias the population representation towards that given orientation, potentially at the expense of the representation of others (Figure 2a).

Figure 2. Models of changes in selectivity with attention.

A) Hypothetical neurons responding to a vertical stimulus without directed attention. Dashed lines represent the neuron’s preferred orientation. Solid lines represent the hypothetical orientation tuning functions of neurons when an input signal is vertically oriented (tuning functions are scaled in response amplitude, relative to vertical). B) Top panel. Responses of a neuronal population to a vertical stimulus under a hypothetical shift in neural selectivity towards vertical orientations due to attention. Compare the center of the hypothetical neurons (dashed lines) in A and B. Bottom panel. As neurons shift their response preference towards the attended orientation (vertical) the net result is an increase in population response for the attended orientation, and a response decrease for the unattended orientations. C) Top panel. Responses of a neuronal population to a vertical stimulus under a hypothetical sharpening in neural selectivity for vertically-tuned neurons due to attention. Compare the width of the hypothetical neurons in A and C. Bottom panel. Sharpened orientation tuning at the detector level gives rise to a more sharply tuned population response with attention. Assuming no increase in response variability, such sharpened population responses would yield a higher-fidelity stimulus representation. If response variability does change, then the amount of information encoded by the network can go either up or down, depending on the nature of the neuronal noise as well as the particular readout scheme employed by downstream areas (Abbott and Dayan 1999; Shamir and Sompolinsky 2004; Averbeck et al. 2006).

Selectivity can also be altered through changes in the bandwidth of tuning, without changes in the preference of a neuron, resulting in a response profile that is more sharply or bluntly tuned around the preferred feature (Figure 2b). Sharpening the bandwidth would be akin to having the neuron be more specifically tuned to a particular feature. This increases the likelihood that the neuron responds to that preferred feature, rather than to some neighboring feature, rendering a higher-fidelity representation of that signal. Broadening the bandwidth would allow you to deduce the activity of a broader range of inputs, but at the cost of having a less precise estimate of the identity of the input.

Neurophysiological and selectivity

How does directing attention to an item affect the selective properties of individual neurons? As we will reveal in this section, it would appear that the nature of selective changes seem to depend largely on the type of attention being deployed. The visual attentional literature generally divides attentional deployment into two categories: spatial attention and feature-based attention. Spatial attention, as the name implies, refers to the prioritized processing of a specific retinotopic location in the visual field, with the classic metaphor for spatial attention being that of a ‘spotlight’. Feature-based attention refers to the prioritized processing of a given feature of an item, such as its color, direction of motion or orientation.

Although initial physiological reports suggested that directing spatial attention to an item sharpens the bandwidth of orientation selective cells in macaque visual area V4 (Spitzer et al. 1988), this was later shown not to necessarily be the case. Follow-up studies using a more sensitive measure for tuning bandwidth found no effect of spatial attention on the width of the orientation tuning function (McAdams and Maunsell 1999; David et al. 2008). Rather, these studies instead only found changes in the responsivity and baseline firing rate of neurons coding for the spatially attended location. Thus, the neurophysiological evidence appears to indicate that spatially attending to a location leaves a neuron’s feature tuning unaffected.

Interestingly, directing feature-based attention does seem to influence the feature selectivity of individual neurons. For example, David et al. (2008) found that when monkeys were trained to direct feature-based attention to a stimulus, neurons often exhibited a distinct shift in their selective properties, biasing the selective quality of the cells towards the orientation or spatial frequency of the attended feature. Thus, it appears that the selectivity of responses can, in some cases, be altered. This notion is intriguing even when considered outside of the context of attention, as it is inconsistent with the classical notion that neurons act as ‘labeled lines’, in which tuning properties are fixed regardless of behavioral state. Rather, these shifts in tuning imply that the selective properties of cells can be transiently modified to cater to a particular task (Dragoi et al. 2000; Kohn and Movshon 2004). This observation has interesting implications for downstream areas reading out the sensory response, which must somehow take into consideration the attention-based shift in lower-level tuning. If downstream areas have no ‘knowledge’ of the lower-level shifts in feature tuning, then this could ultimately result in both a loss of visual information and non-veridical perception (Figure 3).

Figure 3. Modified neural wiring would alter visual representation.

Reading out the lower-level representation requires knowledge of neuronal tuning properties. A) In the absence of attention, wiring between LGN and V1 allows for a representation of the orientation of line elements in V1. Downstream neurons receiving this information subsequently ‘decide’ what is being seen. In the example, the readout areas (correctly) infer that the presented line element is horizontal. B) Now, consider an extreme example in which attention shifts the preferred orientation of the V1 vertical detectors towards horizontal, which could come about through a reweighting of receptive field outputs in the LGN. In this scenario, the amount of evidence for horizontal may reduce in downstream areas. That is, if downstream areas have no ‘knowledge’ of the V1 change in feature preference, they will (incorrectly) infer that the lower-level evidence suggests the presence of both vertical and horizontal orientations (see also (Schwartz et al. 2007).

Although directing spatial attention does not seem capable of altering featural selectivity such as orientation or motion tuning, it does appear to alter neuronal selectivity across space. Selectivity is often considered in the context of tuning for features, such as orientation or motion direction, but neurons in the visual cortex also respond selectively to particular regions in visual space, commonly referred to as a neuron’s spatial ‘receptive field’. Given that feature-based attentional feedback affects neuronal tuning for attended visual features, does directing attention to a spatial location change the layout of the neuronal receptive field? Physiological evidence indicates that the preferred spatial location of neurons in the visual cortex shifts with spatial attention. For example, by measuring receptive field properties with and without the deployment of spatial attention, the spatial preference of a subset of cells within V4 and MT has been shown to shift towards the center of the attended location (Connor et al. 1996; Connor et al. 1997; Womelsdorf et al. 2008; Niebergall et al. 2011). A modest shrinkage in the bandwidth of spatial selectivity around an attended location has been found as well (Womelsdorf et al. 2008). These attention-based changes in receptive field structure could attenuate the influence of ignored stimuli at nearby locations, while increasing the number of detectors sensitive to the attended item, altogether biasing the input in downstream areas towards stimuli presented at the attended location. Much like attention-driven changes in the feature-selective properties of the visual cortex, these changes in spatial selectivity likely arise through a reweighting of synaptic inputs, demonstrating that visual properties that have ostensibly been viewed as rigid in their selectivity become malleable with spatial attention. Indeed, recent evidence suggests that directing attention does indeed selectively alter thalamocortical synaptic weights, enhancing the detection of attended signals in downstream areas (Briggs et al. 2013).

Why would feature-based and spatial attention exhibit differential effects on feature- and location-selectivity, with each capable of shaping selectivity exclusively in their respective domains? Consider the information that is available to someone when deploying these two forms of attention. Spatial attention is directed to a location, and on its own is completely agnostic as to what the features at that location are comprised of. Thus, when directing pure spatial attention, one has no prior knowledge of what features could be useful to recalibrate selectivity around, which is likely the reason why changes in feature-selectivity simply do not occur with spatial attention. Now consider the information available to someone when deploying feature-based attention. Feature-based attention provides prior knowledge, or expectation, of a given stimulus feature in the visual environment. This prior knowledge of what features might be relevant provides an incentive to alter the selectivity of visual responses, potentially optimizing their selective properties for that relevant, attended feature. Interestingly, pure feature-based attention, in which one is told ‘what’ to look for, but not ‘where’ it is, is agnostic in the spatial domain. As a consequence, while feature-based attention selectively enhances a given feature, this feature-based enhancement has been shown to spread non-selectively across retinotopic space (Treue and Martínez Trujillo 1999; Saenz et al. 2002; Sàenz et al. 2003; Busse et al. 2005; Maunsell and Treue 2006; Jehee et al. 2011; Liu and Mance 2011).

Mechanistically speaking, how might selectivity change at the level of single neurons? The aligned convergence model for orientation selectivity in V1 assumes that orientation selectivity arises due to the structure of afferent inputs from LGN. Thus, it is possible that selectivity can change through a reweighting of the strength and arrangement of afferent inputs, thus resulting in changes in the selectivity of the neuron that the inputs converge upon. Aligned convergence would then suggest that changes in selectivity in a given visual area result from selective changes in the strength of the neural response that transpire in the preceding visual area. For instance, any potential changes in the tuning of V1 neurons would, under this model, arise due to changes in the gain of afferent inputs within the LGN. Indeed, there is evidence to suggest that attentional modulation occurs as early as the thalamus (O’Connor et al. 2002; McAlonan et al. 2008). Alternatively, some have proposed that orientation selectivity is not only the result of excitatory convergence, but that intracortical activity plays an active role in sharpening the bandwidth of orientation tuning in V1 (Ringach et al. 1997; Sompolinsky and Shapley 1997; Pugh et al. 2000). Under this model, it is also possible that intracortical activity within an area alters the selectivity landscape, whether through horizontal connections or through feedback. Understanding precisely how attentional feedback changes selectivity is important because it can have profound consequences for neural information processing. For example, some connectivity schemes have been shown to increase the degree of correlated noise in the network, potentially resulting in a severe loss of information (Seriès et al. 2004; Schwartz et al. 2007; Bejjanki et al. 2011); see also Figure 4).

Figure 4. Potential mechanisms underlying attention-based changes in orientation selectivity.

A) Most models assume that orientation selectivity arises from the structure of LGN inputs. B) Attention may affect orientation tuning by changing the balance in this structure. For example, by selectively increasing the gain of some LGN neurons (orange), while leaving others untouched, attention could sharpen orientation selectivity in V1 (by virtue of a more elongated receptive field along the axis of the preferred orientation). C) Alternatively, attention may operate through a change in recurrent connectivity –that is, involving lateral or feedback connections. How do we decide between two models that capture physiological observations equally well? One possibility is to consider the amount of orientation information carried by either network, when implemented computationally. For example, computational modeling has shown that sharpening selectivity through lateral connections has the potential to substantially increase correlated activity in a neural network, resulting in a severe loss of orientation information. Gain changes, on the other hand, seem to suffer less from this issue (Seriès et al. 2004).

While thus far we have treated selectivity as a property of individual visual neurons, another way of conceptualizing selectivity is as a function of a large population of neurons. Indeed, while an individual detector selectively conveys information regarding a small subset of the available inputs, the assembly of these detectors as a population form a powerful signal detector. Strictly speaking, however, changes in the population response do not necessarily correspond to a change in the selectivity of individual neurons as defined above, as a selective change in the responsivity or gain of individual detectors would also increase the selectivity of the population response. For instance, imagine a boost in the response of detectors that prefer the attended feature, and suppression in the response of those that do not prefer that attended stimulus. From the perspective of an individual detector, its selectivity has not changed, but from the perspective of the neural population, the response profiles have been altered to more ‘selectively’ represent the relevant stimulus, while ‘tuning out’ the response of detectors that are tuned to the irrelevant noise. Evidence for such ‘population-based’ changes in selectivity have been observed in macaque area MT, which exhibits selectivity for the direction of a stimulus’ motion: directing feature-based attention to a particular motion direction has been shown to both enhance the response of cells whose direction preferences match the attended direction, and attenuate the response of cells tuned ‘away’ from the attended direction (Treue and Martínez Trujillo 1999). While such population-based changes in ‘selectivity’ do not necessarily fall in line with the traditional definition, we introduce this concept here as it serves as a segue to the coming discussion on neuroimaging and psychophysical studies on selectivity.

Neuroimaging and psychophysics are, of course, substantially more ‘coarse’ in spatial resolution than single-unit electrophysiology. Most current imaging techniques can, at best, give us a view of neural responses occurring on the scale of millimeters, and psychophysical measures result from the output of the entire cognitive system. Thus, while it is difficult to make conclusions from these methods regarding changes in selective properties of individual neurons, these measures provide their own unique perspective: they allow us to measure changes in the population response.

Neuroimaging & selectivity

While a number of physiological studies have examined the effects of attention in animal models, the last two decades has also seen a staggering number of neuroimaging studies exploring how attentional modulation affects the human brain. Neuroimaging techniques such as fMRI and optical imaging provide measurements that nicely complement the information that can be measured from studying the response properties of individual neurons in animal models. Two fundamental differences with neurophysiological measurements have made neuroimaging extremely valuable for investigating attention and cognition. First, neuroimaging allows for the noninvasive investigation of attention in the living human brain, allowing for far more complex tasks. Moreover, neuroimaging provides information on cortical activity gathered beyond the limits of a small number of neuronal cells in a restricted cortical patch: it allows one to measure the population-based activity across the network of cortical areas.

How does directing attention to an item impact the selectivity of a population of neurons? Although many studies have investigated the effects of attentional feedback on neural population activity, the majority of these have focused on an attention-related increase in overall response amplitude, which more likely results from a change in cortical responsivity rather than selectivity. Neuroimaging studies using adaptation techniques, however, present a notable exception. Adaptation refers to a change in the sensitivity of neurons after prolonged exposure to a particular stimulus (Kohn and Movshon 2003; Kohn and Movshon 2004; Webster et al. 2007). This change in neuronal sensitivity depends on the initial response of the neuron: the higher the initial response, the larger the subsequent change. Interestingly, the effects of adaptation can reliably be detected at fMRI’s coarse spatial resolution. For example, repeated exposure to a sinusoidal grating of a given orientation reduces the BOLD amplitude for that particular orientation in V1. Changing the stimulus to an orthogonal orientation, however, results in a recovery of the BOLD response from adaptation, with intermediary levels of recovery for smaller changes in orientation, consistent with the orientation-tuning properties of neurons in V1 (Tootell et al. 1998). Adaptation methods have been used by neuroimaging studies to infer the effects of attentional feedback on feature- and object-selectivity in cortex. For example, it has been proposed that directing feature-based attention to a visual feature specifically enhances the responses of neurons selective for that feature, and (Liu et al. 2007) reasoned that this should then lead to greater adaptation in these neurons as compared to neurons selective for ignored visual features. Consistent with the prediction, fMRI adaptation effects were found to be larger for attended than unattended orientations in early visual areas (Liu et al. 2007). Attention-based changes in selectivity have been observed in higher-level visual areas as well. (Murray and Wojciulik 2004) measured the effects of adaptation for attended versus unattended objects in the human lateral occipital complex, and found that attentional feedback not only enhanced object-selective responses in this region, but also shifted population selectivity towards the attended object, much like the shifts in feature-tuning that have been observed in physiology (David et al. 2008).

Although the adaptation technique has proven to be a valuable measure of the effects of attention on neural selectivity, it is rather indirect. Moreover, it has been proposed that there are a number of limitations in interpreting fMRI adaptation studies (Krekelberg et al. 2006; Hegdé 2009). Beyond fMRI adaptation methods, it was long believed that fMRI’s comparatively coarse spatial resolution made it impossible to directly measure neural tuning for basic visual features. However, recent developments in neuroimaging techniques have revealed that it is possible to obtain robust feature-selective responses from the visual cortex by pooling the information available across many fMRI voxels. For example, pattern-based decoding approaches have shown that population activity patterns in the human visual cortex contain reliable information about orientation, from which it is possible to ‘decode’ which orientation a person is viewing (Kamitani and Tong 2005). Decoding accuracy quantifies how well a given feature can be extracted from brain activity patterns, and depends on a combination of the sharpness of tuning and the amount of noise in the population activity patterns. Studies of attention have deployed decoding techniques to explore how population activity patterns change with attention. For example, consistent with the hypothesis that attending to a visual feature enhances neurons selective for that feature, the attended orientation or color can reliably be decoded from fMRI activity patterns, not just in early visual areas (Kamitani and Tong 2005) but also in frontal and parietal areas (Liu et al. 2011). In addition, this enhancement of feature-selective responses has been found to occur non-selectively across space (Saenz et al. 2002; Sàenz et al. 2003; Serences and Boynton 2007; Jehee et al. 2011), which is consistent with physiological studies showing effects of feature-based attention spreading across the visual field (Treue and Martínez Trujillo 1999). Using similar decoding techniques, others have found a sharpening of the population response with feature-based attention (Serences et al. 2009a), and changes in population activity consistent with the notion that attentional feedback targets the most informative neurons in the current behavioral task (Scolari and Serences 2009; Scolari and Serences 2010; Verghese et al. 2012). Consistent with this, when attentional modulation is spread thin across multiple items, the selectivity of population responses has been shown to be broader than when the focus is on just one item (Anderson et al. 2013).

How does directing spatial attention affect feature-selective activity in the human visual cortex? Cortical processing has been found to improve for spatially attended visual features (Saproo and Serences 2010) that are task-relevant (Jehee et al. 2011), but the lion’s share of the effects of spatial attention on cortical activity appear to be non-selective, additive shifts in the population response (Buracas and Boynton 2007; Murray 2008; Saproo and Serences 2010; Pestilli et al. 2011), which does not necessarily improve feature tuning at the population level (Jehee et al. 2011). In contrast, directing spatial attention to a location does seem to have profound effects on the spatial layout of the population receptive field. For example, the BOLD response is more sharply tuned to spatially-attended locations (Fischer and Whitney 2009), and also magnetoencephalographic (MEG) evidence suggests changes in location sensitivity: the MEG signal is enhanced for stimuli presented at the attended location (Mangun and Hillyard 1988; Hopf and Mangun 2000; Jha et al. 2000; Luck et al. 2000), but reduced for stimuli positioned in a small region surrounding the focus of attention (Boehler et al. 2006). As we will see in the next section, this ‘Mexican hat’ profile for attended items nicely parallels behavioral findings showing that perceptual sensitivity is attenuated around the attended location.

Psychophysics & selectivity

Psychophysicists have devised a variety of techniques to measure feature selectivity through behavior, many of which yield estimates of feature-tuning bandwidths that resemble estimates found within visual cortex. For example, psychophysical tuning curves can be estimated by measuring an observer’s visual sensitivity for a stimulus while it is presented in visual noise (noise masking technique; (Legge and Foley 1980; Blake and Holopigian 1985; Solomon and Pelli 1994; Majaj et al. 2002), or by inspecting the visual characteristics of an incorrectly-perceived stimulus (psychophysical reverse correlation; (Eckstein and Ahumada 2002). For orientation and some other visual features, such behavioral tuning functions yield shapes similar to what has been observed for individual neurons in physiology. How does attentional feedback affect this behavioral feature-tuning? By and large, studies using psychophysical techniques to assess selectivity have converged on results that square quite nicely with the neurophysiological results: feature-based attention to an item selectively changes psychophysical tuning curves (Lee et al. 1999; Baldassi and Verghese 2005; Ling et al. 2009; Paltoglou and Neri 2011), while directing spatial attention to that item leaves behavioral feature tuning untouched (Eckstein et al. 2004; Lu and Dosher 2004; Neri 2004; Talgar et al. 2004; Baldassi and Verghese 2005; Busse and Treue 2008; Ling et al. 2009; Paltoglou and Neri 2012; Wyart and Nobre 2012). Such sharper tuning at the population level could be achieved by a number of physiological mechanisms. For example, boosting the responses of neurons selective for the attended visual feature, while suppressing those preferring other features, would sharpen population activity consistent with these behavioral results.

As reviewed above, one potential mechanism of attentional modulation involves shifts in neural selectivity, which could increase the number of detectors sensitive to the attended feature. What changes in behavior would be consistent with such attention-based shifts in feature selectivity? As demonstrated in Figure 3, one predicted outcome of a shift would be a change in the subjective percept of an attended item, potentially perceiving things nonveridically. Indeed, some behavioral studies suggest that attentional modulation does indeed alters subjective appearance, a predicted outcome of changes in selectivity (Carrasco et al., 2004a; 2004b; Ling and Carrasco, 2007), although this remains a matter of controversy (Schneider and Komlos 2008; Valsecchi et al. 2010; Prinzmetal et al.).

As demonstrated in Figure 2, another potential consequence of a shift in selectivity of feature detectors could be that features neighboring the attended feature are underrepresented in cortex. Results consistent with such a neural mechanism have been observed psychophysically with feature-based attention, whereby sensitivity was enhanced for an attended orientation, but decreased for orientations immediately adjacent to the attended orientation (Tombu and Tsotsos 2008). This ‘Mexican hat’ function, in which sensitivity is attenuated around an attended feature, is the distinct behavioral signature of attention-driven shifts in selectivity. Other behavioral evidence consistent with a shift in feature-selectivity is offered by behavioral studies on visual acuity. Many of these studies report that the details of a stimulus can be better discerned when attention is directed towards it (Shiu and Pashler 1992; Shalev and Tsal 2002; Golla et al. 2004; Carrasco et al. 2006), and it is possible that this attention-based increase in spatial acuity is the result of shifts in spatial frequency selectivity towards higher spatial frequencies. Interestingly, the properties of these shifts seem to depend on the type of attention being deployed. Involuntary deployment of spatial attention to a location appears to automatically shift sensitivity at that location towards higher spatial frequencies (Yeshurun and Carrasco 1998; Carrasco et al. 2006). Voluntary spatial attention to a location, on the other hand, appears to optimize shifts in spatial frequency selectivity more flexibly, accommodating for shifts towards higher or lower spatial frequencies, depending on the nature of the task (Yeshurun et al. 2008).

A handful of behavioral studies suggest that directing attention to a location also alters spatial selectivity. Much like shifts in feature selectivity predicts a penumbra of inhibition around the attended feature, shifts in spatial selectivity towards an attended location would leave a suppressive region around this location. Indeed, it has been shown that spatially attending to a stimulus weakens visual sensitivity to the areas immediately surrounding the stimulus’ location (Cutzu and Tsotsos 2003; Müller et al. 2005), which suggests that attentional feedback causes a shift in spatial selectivity towards the attended location.

Conclusion

In this review, we considered the physiological and behavioral consequences of an attention-based change in neural selectivity. There are several conclusions that can be derived from this body of work. First, attentional feedback seems capable of altering some aspects of selectivity, but not all. Quite reasonably, the type of changes in selectivity seems to depend on the type of attention being deployed: spatial attention refines selectivity in space, and feature-based attention refines selectivity around the attended feature dimension. The second main conclusion that can be drawn from the literature is that the scale at which selectivity changes tends to be closer to the population-level. Whereas there is scant evidence that the selectivity bandwidth of individual neurons changes with feature-based attention, shifts in single-unit selectivity ‘towards’ the attended stimulus can occur. However, there seems to be a consensus across physiology, imaging, and psychophysics that population-wide responses can be selectively honed around an attended item or location, resulting in enhanced neural processing, and improved behavioral performance, for the attended feature or location.

Although we focused on attention-based changes in neural selectivity, other neural response properties have been shown to be affected by attentional feedback, as well. Most prominently, directing attention appears to modulate the responsivity (Buracas and Boynton 2007; Murray 2008; Pestilli et al. 2011) and variability of neural responses (Cohen and Maunsell 2009; Mitchell et al. 2009), thereby enhancing the signal-to-noise ratio in the visual system. More recently, it has been proposed that directed attention may also operate through an efficient selection mechanism that excludes irrelevant visual signals from the decision process (Pestilli et al. 2011; Chen and Seidemann 2012) -- a gating mechanism that would improve processing in downstream areas. In a way, this notion of attentional ‘selection’ can be thought of as a variant of refined spatial selectivity, whereby the representation of unattended locations is not only attenuated, but abolished altogether in higher-level areas. The challenge for future studies will be to model the degree to which each of these mechanisms contribute to the improvements in behavioral performance that occur with attentional feedback.

How can computational models unify these attentional effects, measured across different scales of measurement, and make links to behavior? Reynolds & Heeger recently proposed a normalization model of attention, which accounts for a large variety of attentional effects, including changes in the selectivity of responses described in this review, by hypothesizing that the relative size of the stimulus and the extent of attention allocation play a fundamental role in shaping neuronal responses with attention (Reynolds and Heeger 2009; Herrmann et al, 2010; Ling & Blake, 2012; Bhatt et al, 2007). The normalization model of attention nicely predicts the diversity of attentional effects reported in single-units such as the studies discussed in this review. However, many of these findings are studied in the context of neuronal populations that are well-tuned to the stimulus, whereby only cells that respond near-optimally to a stimulus are considered for experimental measurement. In order to bridge findings from single-unit recordings with larger-scale measurements (i.e., BOLD fMRI and behavioral psychophysics), important factors such as the heterogeneity of tuning within populations contributing to large-scale measurements need to be properly accounted for. It has been shown that by using the normalization model in combination with heterogeneous neuronal populations, it is possible to reconcile the potentially different effects of attention on individual neuronal responses with those measured from large-scales neuronal populations (Hara et al. 2014).

Figure 5. Effects of attention on neural selectivity in monkey visual cortex.

A) The peak spatial preference of a neuron’s spatial receptive field shifts towards the attended location in area MT (Womelsdorf et al. 2008). Left panel. Spatial response profile of a single neuron when the monkey attended to Stimulus 1 (blue diamond). Middle panel. Spatial response profile of the same neuron when the monkey attended to Stimulus 2 (red circle). Right panel. Effect of spatial attention on the structure of the neuron’s receptive field. The map shows the difference in response between the left and middle panels, i.e. between attention directed to locations 1 and 2. Shifting attention from S1 to S2 enhanced responsiveness around S2, and reduced it near S1. B) Changes in orientation and spatial-frequency tuning of a V4 neuron with spatial- and feature-based attention (David et al. 2008). While both spatial and feature-based attention increased the gain of the neural response, only feature-based attention acted to alter the tuning of individual neurons. C) Average population response in macaque area MT when patterns of moving dots were presented in its receptive field. While population responses increased for neurons that preferred the attended motion direction, they decreased for neurons that preferred other directions (Treue and Martínez Trujillo 1999).

Figure 6. Effects of attention on orientation processing in human visual cortex.

A) Attending to the orientation of a stimulus increases the strength of orientation-selective signals in human visual cortex, compared to when the stimulus is ignored. Orientation selectivity was indexed through orientation decoding accuracy of fMRI activity patterns (Jehee et al. 2011). B) Feature-based attention enhances fMRI activity for attended visual features in areas V1 and V4v. Participants attended either 45° (red) or 135° (green) oriented bars that were presented simultaneously. Attention increased the activity of those fMRI voxels whose preferred orientation matched the attended feature (Serences et al. 2009b). Solid lines in B are best fitting circular Gaussians. Error bars represent +/−1 SEM.

Figure 7. Behavioral estimates of changes in selectivity with attention, using various psychophysical measures.

Top row corresponds to results obtained across studies for spatial attention, whereas the bottom row contains results from studies of feature-based attention. A) Using an image classification approach, Paltoglou and colleagues found that sensory tuning was only affected by feature-based attention (Paltoglou and Neri 2012) B) Using a psychophysical paradigm known as the equivalent noise approach, in combination with a computational model, Ling, Liu & Carrasco (2009b) estimated hypothetical changes in underlying population tuning. The model estimates suggested that while spatial attention operates strictly by means of a multiplicative gain mechanism, the estimated population response was both increased and sharpened by feature-based attention. C). Using a psychophysical noise masking approach, Baldassi & Verghese (2005) found that spatial attention caused an overall improvement in sensitivity, across orientations, whereas feature-based attention only improved sensitivity around the attended orientation. This is consistent with idea that population tuning was sharpened with feature-based attention.