Mechanisms of attention

Abstract

Sensory physiologists and psychologists have recognized the importance of attention on human performance for more than 100 years. Since the 1970s, controlled and extensive experiments have examined effects of selective attention to a location in space or to an object. In addition to behavioral studies, cognitive neuroscientists have investigated the neural bases of attention. In this paper, I briefly review some classical attention paradigms, recent advances on the theory of attention, and some new insights from psychophysics and cognitive neuroscience. The focus is on the mechanisms of attention, that is, how attention improves human performance. Situations in which the perception of objects is unchanged, but performance may differ due to different decision structures, are distinguished from those in which attention changes the perceptual processes. The perceptual template model is introduced as a theoretical framework for analyzing mechanisms of attention. I also present empirical evidence for two attention mechanisms, stimulus enhancement and external noise exclusion, from psychophysics, neurophysiology and brain imaging.

Introduction

Attention solves the problem of information overload in cognitive processing systems by selecting some information for further processing, or by managing resources applied to several sources of information simultaneously (Broadbent, 1957; Posner, 1980; Treisman, 1969).

Scientific enquiry on attention originated with cognitive approaches to psychology at the end of the 1800s and early 1900s (James, 1890; Pillsbury, 1908; Titchener, 1908; Wundt, 1902). Recognizing the importance of attention for human performance, pioneering investigators debated whether attention affects the perceived quality of objects, such as the brightness of a light patch, the loudness of a musical tone, the clarity of a visual pattern, or the vividness of a certain color.

Since the 1970s, selective attention to a location in space, or to an object, has been the subject of extensive study in controlled experiments (Posner, 1980; Shiffrin, 1988; Sperling & Melchner, 1978; Treisman & Gelade, 1980). Numerous studies have demonstrated that attending to a target usually leads to improved performance in accuracy (Bashinski & Bacharach, 1980; Cheal, Lyon, & Gottlob, 1994; Downing, 1988; Enns & Di Lollo, 1997; Shiu & Pashler, 1994), and/or response time (Egly & Homa, 1991; Eriksen & Hoffman, 1972; Henderson & Macquistan, 1993; Posner, Nissen, & Ogden, 1978).

Currently, empirical investigation of attention has primarily focused on (1) identifying mechanisms of attention: how and why attention improves human performance, or how lack of attention hinders performance, and (2) defining the networks of attention control: how is attention allocated in space and time, contributions of top-down and bottom-up processes in attention allocation, and the relationship between overt and covert attention.

In this paper, I provide a brief review on mechanisms of attention. The review consists of three topics: (1) Major paradigms in attention reseach, (2) Separating decision and perceptual effects of attention, and (3) Mechanisms of attention. Because of space limitations, I cannot provide a detailed review of the literature. Instead I attempt to highlight some of the most important paradigms and results.

Major Paradigms

Introspection

The classical question in attention was: Does attention affect the quality or strength of perception (James, 1890; Pillsbury, 1908; Titchener, 1908; Wundt, 1902)? The answer to this question relied on subjective reports, and the views of the early theorists differed (James, 1890). Despite extensive subsequent research, we still have only the most rudimentary answer to the original question, because it is very difficult to quantify or test the subjective appearance of perceived objects (but see Prinzmetal, Amiri, Allen, & Edwards, 1998). A somewhat more modest but substantially more tractable question is: Does attention change human performance?

The Posner paradigm

In the Posner paradigm (Posner, 1980), subjects are asked to detect a target as quickly as possible. The target appears either to the left or right of the fixation shortly after a briefly flashed central cue. The cue could be valid (80% predictive of the target location), neutral (50%), or invalid (20%). Compared to the neural cues, valid cues led to faster response times; invalid cues led to slower response times. The Posner paradigm and its variants have been widely used in both basic and applied attention research.

Inhibition of return

Posner & Cohen (1984) found that, although valid peripheral cues reduced response times in the cued location when the target-cue delay was short (100–300 ms), they increased the response times in the cued location when the target-cue delay was long (500–3,000 ms). They named the phenomenon, i.e., reduced inclination to revisit a previously cued item, “inhibition of return”. Later studies examined the effect of inhibition of return on the accuracy of perceptual discriminations, finding a reduction in performance accuracy in long cue target delay conditions (Cheal, Chastain, & Lyon, 1998).

Visual search

In visual search, subjects are shown visual displays containing varying numbers of objects and are asked to determine whether a target is included in the display. Treisman & Gelade (1980) contrasted two types of visual search: (1) feature search in which subjects were asked to look for a blue T among brown T's and green X's (the target was defined by one feature), and (2) conjunction search in which subject were asked to look for a green T among brown T's and green X's (the target was defined by the conjunction of two features). They found that search times increased linearly with display size in conjunction search. In contrast, display size had no effect on search times in the feature search task. They concluded that basic visual features are processed pre-attentively, whereas processing feature conjunctions requires attention.

Attention operating characteristics

Sperling & Melchner (1978) developed the attention operating characteristics (AOC) to investigate how attentional resource is shared between tasks. A typical AOC study consists of two tasks. Baseline conditions measure how well each task is done alone. If two tasks can be done simultaneously without loss, the joint performance is the same as that in the baseline conditions. If the two tasks make competing demands on attention, the joint performance is worse than that in the baseline conditions. Systematic manipulations of attention instructions that require subjects to allocate differential amounts of attention to the two tasks provide quantitative measures of the overlap of the resources shared by the two tasks.

Attention reaction time

Reeves & Sperling (1986) developed the attention reaction time (ART) paradigm to measure the time course of an attention episode. Subjects were shown two adjacent rapid serial visual presentations (RSVP), a stream of letters on the left with a target letter embedded at a random position in the middle and a stream of numerals on the right. They were instructed to initially pay attention to the letter stream, and as soon as they detected the target, switch attention to the numeral stream and report the earliest possible numeral. They found that the subject nearly always reported the numerals that occurred 327 or 436 ms after target onset. They later elaborated the procedure to ask subjects to report not merely the earliest occurring numeral but the earliest four. The rich data from the ART procedure led to the gating model of attention (Reeves & Sperling, 1986).

Attention blink

Another interesting phenomenon uncovered in rapid serial visual presentation (RSVP) is attention blink (Chun & Potter, 1995). In a typical demonstration, subjects are asked to detect a target that is embedded at a random position in the middle of a letter stream. If a probe occurs right after the target, subjects could perceive it most of the time. If the probe is very far away from the target, subjects could also perceive it most of the time. But if the probe is placed anywhere from 200 ms to 500 ms after the target, subjects are unaware of the target with high probability. Attention blink is thought to reflect the refractory period of attention.

Object attention

Object attention, reflecting competition between objects for focal attention, is evident in dual-object report disadvantages (Duncan, 1984): Two judgments that concern the same object can be made simultaneously with little or no loss of accuracy compared to a single judgment about that object. However, two judgments about separate objects exhibit losses compared to single or dual judgments about a single object. Object attention has become a major organizational principle in some theories of attention (Desimone & Duncan, 1995).

Change blindness

In a typical demonstration of change blindness, subjects are presented with two images of scenes alternating repeatedly with a brief (80 ms) blank screen after each image. Surprisingly large changes could be made to the scene without subjects reliably noticing them (Rensink, O'Regan, & Clark, 1997; Simons & Levin, 1998). Change blindness can be particularly dramatic when changes occur unexpectedly, reflecting the limitations of visual awareness.

Separating Decision and Perceptual Effects of Attention

One relatively recent major insight in the field of attention is that some observed performance improvements in some classical attention paradigms may reflect decision factors that do not change the perception of objects: (1) decision uncertainty by which performance may be reduced by multiple sources of false alarms (Eckstein, Thomas, Palmer, & Shimozaki, 2000; Gould, Wolfgang, & Smith, 2007; Palmer, Ames, & Lindsey, 1993; Shaw & Shaw, 1977), and (2) altered weights on the elements in the display or biases toward responses (Bundesen, 1990; Eckstein, Shimozaki, & Abbey, 2002; Logan, 2002; Sperling & Dosher, 1986).

The Posner paradigm

Using the classification image technique, Eckstein et al. (2002) tested an alternative model of the cost/benefit effects in the Posner paradigm. The model predicts a cueing effect without a change in the quality of processing at the attended and unattended locations: Subjects monitor the responses of two equivalent perceptual filters at the cued and uncued locations. Each of the perceptual filters linearly weights the luminance at the cued and uncued locations. Using a Bayesian rule, the model computes and combines the likelihoods across the cued and uncued locations. Eckstein et al. (2002) found no significant difference between the shapes of the inferred perceptual filters in the attended and unattended locations, although there was a difference in the magnitude of the classification images, supporting the idea that visual attention changes the weighting of information without changing the quality of processing at the cued and uncued locations.

Visual search

Although set-size effects in visual search have traditionally been attributed to attention processes, Palmer, Ames, & Lindsey (1993) evaluated an alternative decision hypothesis of visual search: In visual search with a set size of one, subjects decide whether an ambiguous percept came from a distractor or a target by adopting a response criterion. With larger set sizes, subjects have to base their decision on internal responses to all the items in the display. The distractor eliciting the maximum internal response is most confusable with the target. As the number of distractors increases, the distribution of the maximum internal response to distractors become less separated from that of the target and therefore result in the set-size effect. Focusing on visual thresholds (an accuracy measure), Palmer et al. (1993) concluded that a decision effect alone was sufficient to predict the set-size effects without any attentional limitation on perception.

Dosher, Han, & Lu (2004) extended the accuracy analysis of visual search to include an analysis of the full time course of visual search. The central theoretical issue is whether visual search involves a serial or parallel information processing architecture. Increased average response time and error rates for larger displays are not sufficient to discriminate the two types of processes, nor are accuracy measures alone. Using the speed-accuracy trade-off paradigm, Dosher et al. (2004) measured the time courses of visual search for easy C-in-O searches and difficult O-in-C searches. They found that the time courses of the two tasks were similar and independent of display size. In the absence of eye movements, asymmetric visual search, long considered an example of serial deployment of covert attention, is qualitatively and quantitatively consistent with parallel search processes.

Perceptual Mechanisms of Attention

In signal processing, there are three ways to improve the signal to noise ratio: amplification, improved filtering, and modified gain control. Similar principles of contrast gain, re-tuning of cellular signal selectivity, and reduced contrast-gain have also been demonstrated in single unit neurophysiology (Reynolds, Pasternak, & Desimone, 2000). Motivated by the principles in signal processing and neurophysiology, Lu & Dosher (1998) developed the external noise plus attention paradigm and a theoretical framework based on the Perceptual Template Model (PTM) to distinguish perceptual mechanisms of attention [see Lu & Dosher, 2008, for a recent review].

The paradigm adds systematically increasing amounts of external noise—random visual noise (similar to random TV noise)—to the visual stimulus and observes its effects on a perceptual task in attended and unattended conditions. Threshold versus external noise contrast (TvC) functions are estimated from the observations. Three perceptual mechanisms of attention can be distinguished: Stimulus enhancement acts by multiplying the contrast of the input stimulus by a factor greater than one, mathematically equivalent to internal additive noise reduction. The behavioral signature for this mechanism is performance improvement (reduced thresholds) in the region of low or zero external noise (Figure 1b). Because it affects both the signal and external noise in the input stimulus in the same way, the mechanism does not affect performance in high external noise conditions. External noise exclusion improves performance by focusing perceptual analysis on the appropriate time, spatial region, and/or content characteristics of the signal stimulus. The focus serves to eliminate external noise from further processing. The behavioral signature for this mechanism is performance improvements in the region of high external noise (Figure 1c) where there is external noise to exclude. Internal multiplicative noise reduction reduces the noise whose amplitude is proportional to the contrast energy in the input stimulus. The mechanism produces a signature of improvements in both high and low levels of external noise (Figure 1d). Measuring TvC functions at two or more criterion performance levels resolves the individual contribution of each mechanism in a mixed mechanism situation (Dosher & Lu, 1999).

Figure 1.

(a) The perceptual template model. (b, c, d) Signatures of the three mechanisms of attention.

Although there are three possible mechanisms of attention, two of them have dominated the empirical results. In the absence of decision uncertainty, covert attention has been found to operate primarily via two independent mechanisms: (1) Enhancing stimulus in the target location (Carrasco, Penpeci-Talgar, & Eckstein, 2000; Li, Lu, Tjan, Dosher, & Chu, 2008; Lu & Dosher, 1998; Lu & Dosher, 2000; Lu, Liu, & Dosher, 2000; Luck & Hillyard, 1995; McAdams & Maunsell, 1999; Motter, 1993; Reynolds et al., 2000), and (2) Excluding distractor or external noise in the target region (Dosher & Lu, 2000a; Dosher & Lu, 2000b; Lu & Dosher, 2000; Lu, Lesmes, & Dosher, 2002; Moran & Desimone, 1985; Reynolds, Chelazzi, & Desimone, 1999; Smith, Ratcliff, & Wolgang, 2004; Treue & Andersen, 1996).

Stimulus enhancement

Behaviorally, stimulus enhancement (Figure 1b) is demonstrated by improvements due to attention in the absence of or presence of low external noise, but cannot improve performance when it is limited by external noise (Lu & Dosher, 1998). Lu, Liu, & Dosher (2000) applied the external noise plus attention paradigm to study attention mechanisms involved in concurrent first-order and second-order motion perception at two spatial locations. Contrast thresholds for motion direction discrimination were measured at three criterion performance levels in every attention and external noise condition. Observers could, without any loss, simultaneously extract first-order motion directions at two widely separated spatial locations across a broad range of external noise conditions. However, considerable loss occurred at the unattended location in the low external noise conditions in processing second-order motion direction at two separated spatial locations. In second-order motion perception, attending to a spatial location enhances stimulus contrast by a factor of about 1.37.

In physiology, stimulus enhancement is best demonstrated by a leftward horizontal shift of contrast response functions (CRF; mean firing rate versus signal stimulus contrast) in the attended condition relative to the unattended condition (Reynolds et al., 2000). Based on single-unit recording on monkeys, Reynolds et al. (2000) concluded that, on average, attention increased the effective contrast by a factor of 1.51 for neurons in primate V4. A similar conclusion was also reached by Martinez-Trujillo & Treue (2002) in monkey MT (but see Williford & Maunsell, 2006).

In functional imaging studies, Li et al. (2008) measured the BOLD fMRI contrast response functions in the retinotopically defined early visual areas (V1, V2, V3, V3A, and V4) in humans. They found that covert attention increased both the baseline activities and contrast gains in the five cortical areas. Attention enhanced stimulus contrast by a factor of 3.2 across the areas. Increase in contrast gain accounted for approximately 88.0, 28.5, 12.7, 35.9, and 25.2% of the trial-by-trial effects of attention, consistent with single-unit findings in V4 and MT. The results provide strong evidence for a stimulus enhancement mechanism of attention.

External noise exclusion

In behavioral studies, pure external noise exclusion (Figure 1c) occurs when attention improves performance only in high external noise, yet has little or no effect in conditions with low or no external noise (Dosher & Lu, 2000a; Dosher & Lu, 2000b; Lu & Dosher, 2000; Lu et al, 2002). Dosher & Lu (2000b) demonstrated that valid pre-cueing one of four widely-separated spatial locations dramatically improved Gabor orientation identification relative to invalid pre-cues only in high external noise conditions. Four Gabor stimuli appeared on an annulus, one in each quadrant. A 62.5% valid central arrow pre-cue indicated the likely report location. A simultaneous report cue indicated the actual target location. The primary mechanism of attention in this central cueing paradigm was external noise exclusion, implying a retuning of the perceptual template. Lu & Dosher (2000) found that thresholds in high external noise but not zero or low external noise were improved in central pre-cuing, whereas thresholds in both high and low external noise were improved in peripheral pre-cuing. The results suggest that the endogenous and exogenous attention systems invoke different mechanisms of attention: external noise exclusion for the endogenous system, and external noise exclusion plus stimulus enhancement for the exogenous system.

In physiology, evidence of external noise exclusion came from studies that investigated attentional modulation of the responses of single neurons when two stimuli are placed in their receptive fields (Moran & Desimone, 1985; Reynolds et al., 1999). In a classical study, Moran & Desimone (1985) presented two stimuli in the receptive field of a V4 neuron. One of the stimuli was preferred by the neuron and elicited a high firing rate when it was presented alone. The other was a non-preferred stimulus of the neuron and elicited a weak response when it was presented alone. They found that, when the monkey attended to the preferred stimulus, the cell gave a good response; when the animal attended to the non-preferred stimulus, the cell gave almost no response, even though the effective stimulus was presented in its receptive field. The receptive field of the neuron “shrank and wrapped” the attended stimulus, excluding the unattended stimulus.

Lu et al. (2007) investigated the effect of covert attention in high external noise on the BOLD contrast response functions in retinotopically defined early visual brain areas. Using both rapid event-related and mixed designs, BOLD responses to a brief (100 ms), spatially windowed (5°–7° annulus) sinusoidal grating embedded in a single high level of external noise were obtained in V1, V2, V3v/VP, V3 A, and V4v. Four grating contrasts were tested in both attended and unattended conditions. In V1 and V2, the BOLD responses without attention were largely independent of signal contrast. Covert attention reduced the BOLD responses to low contrast signals and increased the BOLD response to high contrast signals, increasing the impact of signal and decreasing that of external noise. In higher cortical areas, attention did not alter the magnitude of the BOLD responses to low contrast signals but increased the BOLD responses to high contrast signals. Attention reduces the impact of external noise in early visual areas, resulting in increased signal to noise ratio and therefore better performance. Attention also enhances stimulus, which does not affect signal to noise ratio in high external noise.

Conclusion

In this review, I have focused on some of the major paradigms and classical results on attention. These paradigms document various aspects of effects of attention on human performance. Some effects of attention can be attributed to decision factors that do not affect perceptual processes. Using paradigms without structural uncertainty, we found converging evidence from psychophysics, neurophysiology, and function imaging that attention improves performance via two major mechanisms: Stimulus enhancement and External noise exclusion. Future studies on mechanisms of attention will help us elucidate the taxonomy of the mechanisms of attention, and relate the various mechanisms to human performance in demanding operator environments.