Covert attention accelerates the rate of visual information processing

Abstract

Whenever we open our eyes, we are confronted with an overwhelming amount of visual information. Covert attention allows us to select visual information at a cued location, without eye movements, and to grant such information priority in processing. Covert attention can be voluntarily allocated, to a given location according to goals, or involuntarily allocated, in a reflexive manner, to a cue that appears suddenly in the visual field. Covert attention improves discriminability in a wide variety of visual tasks. An important unresolved issue is whether covert attention can also speed the rate at which information is processed. To address this issue, it is necessary to obtain conjoint measures of the effects of covert attention on discriminability and rate of information processing. We used the response-signal speed-accuracy tradeoff (SAT) procedure to derive measures of how cueing a target location affects speed and accuracy in a visual search task. Here, we show that covert attention not only improves discriminability but also accelerates the rate of information processing.

Covert attention can be voluntarily allocated to a given location according to goals (“sustained attention”), or involuntarily allocated, in a reflexive manner, to a cue that appears suddenly in the visual field (“transient attention”; refs. 1–25). Covert attention improves discriminability in many visual tasks, such as contrast sensitivity (1–3), texture segmentation (4, 5), and visual search (6, 7), by diminishing the impact of stimuli that are outside the focus of attention (external noise reduction; refs. 1–3 and 8–10), intensifying the signal (1, 2, 11, 12), and enhancing spatial resolution (4, 5, 13–15). Current cognitive and neurophysiological models typically capture these diverse effects by treating attention as a form of weighting of different sources of information (2, 8, 14–17). However, an unresolved issue is whether attention can affect the rate of information processing directly: Does attention enable the visual system to deal more efficiently with the exigencies of a rapidly changing visual world by, for example, speeding up the coding of fleeting stimuli? Some researchers have suggested that it does, but without having direct evidence (18, 19). Such evidence would require cognitive and neurophysiological models of visual processing to account for how attention jointly affects the quality of the representation and accelerates computational processes.

Studies showing that directing covert attention to a location reduces the time to respond to a stimulus (6, 7, 11, 13, 16, 20, 21, 24) do not speak directly to the issue of whether attention speeds information processing. Differences in response time (RT) can be due to changes in decision criteria or to differences in discriminability (26–31). This point is made explicit in generic models of RT in which a response is executed when the amount of accumulated evidence exceeds a criterion value (31). A response threshold can be reached at an earlier point in time if a stimulus is more discriminable, even if the speed of information processing does not differ. The same principle holds for related time measures.§ Because it has been established that attention improves discriminability (1–5, 8–15), a procedure is needed to measure the speed of information processing in circumstances in which discriminability varies.

We used the response-signal speed-accuracy tradeoff (SAT) procedure to obtain conjoint measures of discriminability and the speed of information processing (26–30) in two visual search tasks, the standard tasks for investigating covert attention (7, 8, 17, 33, 34). Feature search (e.g., searching for a target of one orientation among distracters of another orientation) examines how the visual system extracts basic features. Conjunction search (searching for a unique combination of two features—e.g., orientation and spatial frequency—among distracters that share only one of these features) examines how the system combines features into perceptual wholes (8, 17, 34). In most feature searches, RT and accuracy in detecting or discriminating a target are largely unaffected by the number of distracters (set size); in most conjunction searches, RT increases and accuracy decreases with the number of distracters. This difference in the search patterns has been considered to reflect “parallel” and “serial” mechanisms: whereas features can be registered in parallel across the display, it has been claimed that conjunctions require the serial deployment of attention to bind the features (8, 17, 34, 35). Recent research on visual search has challenged this dichotomy (7, 10, 30, 33). Crucially, measures of the temporal dynamics of visual search derived from the SAT procedure indicate that a parallel processing mechanism best explains both feature and conjunction searches (30).

Methods

Stimuli and Procedure.

Fig. 1 shows the stimuli used in the search tasks reported here: Gabor patches (suprathreshold sinusoidal gratings vignetted by a Gaussian envelope) that varied in two basic dimensions, orientation and spatial frequency. Gabor patches are well matched to early linear spatial filtering in the visual system. The stimuli were displayed on a high-resolution color monitor. Background luminance was set to the middle of the monitor's range (16 cd/m²). All Gabor patches subtended 2° of visual angle (full width at 1/e), on the basis of a fixed 114-cm viewing distance. The stimuli were randomly presented at eight equidistant locations from a central fixation point on an invisible polar grid at 4° eccentricity. The distance between the centers of two neighboring locations was 3.06°.

Figure 1.

Response-signal speed-accuracy trade-off procedure. Sequence of events in a single trial. In both tasks, the 2-cpd target was 30° tilted to the right or left. In the feature task, all of the distracters were vertical 2-cpd patches. In the conjunction task, some distracters shared the orientation of the target and others share its spatial frequency: the 2-cpd distracters were vertical patches; half the 8-cpd distracters were tilted to the left, and half to the right. A small fixation dot was always present at the center of the screen. To implement the SAT procedure, a response tone was presented after the display at varying time lags ranging from 40 to 2000 ms. Feedback was provided after each trial and block. Each of 5 naive observers performed 13,440 experimental trials over 10 sessions.

In our feature search, five naive observers (New York University students) responded to a unique visual feature (30° tilt) presented in isolation or with three or seven distracters of a different orientation (vertical). Observers indicated whether a 2 cycle per degree (cpd) Gabor target was tilted to the right or left, when all distracters were vertical 2-cpd Gabor patches. In our conjunction search, these five observers responded to a unique conjunction of two features (spatial frequency and orientation) presented in isolation or with three or seven distracters. The observers' task was again to indicate whether a 2-cpd target was tilted to the right or left. However, this search included 8-cpd distracters, half tilted to the left and half to the right, in addition to the 2-cpd vertically oriented distracters. Because some distracters shared the orientation of the target and others share its spatial frequency, observers had to process both dimensions to discriminate successfully the target's orientation.

Before testing, stimulus contrast was adjusted for each observer so that the performance across all intervals was at 80–85% correct level. The contrast for the 2-cpd suprathreshold stimuli ranged from 8 to 12% (across observers), and the contrast for the 8-cpd ranged from 70 to 80%. Note that the contrast necessary to discriminate the orientation of the stimuli when presented in isolation is 4% for the 2-cpd and 8% for the 8-cpd Gabor patches, which is much lower than the contrast levels used here. Even at these lower contrast levels, there is virtually no spatial uncertainty—observers' performance for localizing the Gabor patch is about 99%.

In contrast to most visual search studies, which have inferred the role of attention from the “serial” RT search pattern (8, 17, 34), in this study we directly manipulated attention in a search task (6, 7, 10, 36). We did so by presenting a peripheral, transient precue (a small circle adjacent to the target location; refs. 1, 4–7, 13, and 20) on a random half of the trials, which indicated the display onset and target location but did not signal the target orientation. The remaining trials started with a neutral cue (a small circle in the middle of the display), which indicated the display onset and that the target had equal probability of appearing at any location (Fig. 1). After an interstimulus interval (ISI), the Gabor patches, with zero, three, or seven distracters, appeared briefly. No local postmask followed the display.

The interval between the cue onset and the target onset was 120 ms. This timing maximizes the effect of the peripheral cue, which triggers “transient” attention to the target location in a reflexive, involuntary manner (6, 20). Furthermore, the interval between the cue onset and the stimulus offset (160 ms) was brief enough to prevent goal- or target-directed eye movements, because about 200–250 ms are needed for a saccade to occur.

To measure discriminability and the speed of information processing conjointly, a tone sounded at one of seven randomly chosen times ranging from 40 to 2000 ms after the display onset. Observers were required to respond within 300 ms of the tone. The range of response times enabled us to sample the full time course of processing, from early times when discrimination was near chance level to late times when it had reached its asymptotic level. Feedback was provided after each trial and block.

All variables (cue type: peripheral vs. neutral; set size: 1, 4, 8; and response tone: 40, 94, 200, 350, 600, 1000, and 2000 ms) were randomly presented within each block of feature and conjunction searches. Feature and conjunction sessions were alternated across days with the order counterbalanced across observers.

Predictions.

Fig. 2, which plots discrimination performance as a function of processing time, illustrates hypothetical SAT functions for peripherally cued and neutral conditions. Fig. 2A illustrates the expected pattern if attention simply improves target discriminability at the attended location. Enhanced discriminability yields stronger evidence, and thereby engenders higher asymptotic performance. Fig. 2B illustrates acceleration of information processing at the attended location. Compared with the neutral condition, the peripherally cued condition could be associated with an earlier intercept, a faster rate of approach to asymptote (Fig. 2B), or both. The intercept of the SAT function (d′ = 0) measures the minimal time needed for above-chance discrimination. The rate of rise of the SAT function indexes either the rate of information processing directly, if the underlying process is continuous, or the distribution of finishing times, if the underlying processing is discrete (26–30). A difference in either rate or intercept will result in disproportional SAT dynamics, in that the functions will reach a given proportion of their respective asymptotes at different times. The lines that intersect the ordinate and abscissa in Fig. 2 show the time when the functions reach the 1 − 1/e (63%) point. In Fig. 2A, where the underlying dynamics are identical, the functions reach this point at the same time as indicated by the vertical line. In Fig. 2B, where the functions are associated with a common asymptote and intercept but different rates, the 1 − 1/e point is reached at different times. The circles show the corresponding RT points in SAT coordinates, illustrating that RT differences can arise from differences in either discriminability (Fig. 2A) or dynamics (Fig. 2B).

Figure 2.

Hypothetical SAT functions. Illustrative SAT functions, plotted in d′ units ( of the standard normal deviate of the probability of correctly judging the target's orientation) versus processing time (time of the response cue plus observer's latency to respond) in seconds. (A) Expected pattern if cueing increases target discriminability only. The functions differ in asymptotic accuracy, but are associated with the same intercept (point when accuracy departs from chance) and proportional rate of information processing. (B) One expected pattern if cueing target location alters the rate of information processing only. The functions display disproportional dynamics; they reach a given proportion of their asymptotes at different times. Circles show hypothetical RT results plotted in SAT coordinates (abscissa = mean RT; ordinate = the accuracy level associated with mean RT), illustrating that RT differences can reflect differences in discriminability (A) or the speed of information processing (B). Approximately the same difference in mean RT and accuracy is consistent with either differences in SAT asymptote (A) or SAT dynamics (B). The position of the RT points on the corresponding SAT functions are determined by the decision criteria that an observer uses to balance speed and accuracy. Here, the hypothetical RT data are shown slightly higher than the 1 − 1/e (63%) point—a position often found in direct comparisons of RT and SAT procedures—illustrating that, in conventional RT tasks, observers often trade modest decrements in accuracy for substantial gains in speed (27–30).

Results

Fig. 3 shows the group average time course functions for both feature and conjunction searches. To estimate asymptotic discriminability and temporal dynamics jointly, the time course data were fit with an exponential approach to a limit:

1

where λ is the asymptotic parameter reflecting discriminability at maximal processing time; δ is the intercept parameter reflecting the discrete point in time when discriminability departs from chance (d′ = 0); and β is the rate parameter indexing the speed with which discriminability grows from chance to asymptote. Eq. 1 provides a precise quantitative summary of the shape of the SAT function (26–30). We used a hierarchical model-testing scheme to determine how the experimental factors affected the shape of the SAT functions; the 3 parameters of the exponential equation were fit to each observer's and the average data. These models ranged from a null model in which the 6 functions for each task (3 set sizes and 2 cue types) were fit with a single asymptote (λ), rate (β), and intercept (δ) to a fully saturated (18-parameter) model in which each function was fit with a unique set of parameters. The quality of fit was determined by three criteria: (i) the value of an adjusted-R² statistic (26), where the proportion of variance accounted for by a model was adjusted by the number of free parameters; (ii) the consistency of parameter estimates across observers; and (iii) an evaluation of whether any fit left systematic residuals that could be accounted for by additional parameters.

Figure 3.

Results. Average (over observers) discrimination accuracy (in d′units) as a function of processing time in feature (A) and conjunction (B) searches. Smooth functions show the best-fitting exponential model (Eq. 1) for the cued (solid lines) and neutral (dashed lines) conditions, based on fits of nested models that systematically varied the three parameters of Eq. 1. Quality of fit was determined by the value of an adjusted-R² statistic (26–30), the proportion of variance accounted for adjusted by the number of free parameters, and by the consistency of parameter estimates across observers. The simplest best-fitting model for feature searches allocated a separate asymptotic parameter (λ) to each of the six conditions, one rate (β) parameter to the cued conditions and another to the neutral conditions, and a single intercept (δ) parameter (adjusted-R² = 0.979 for the average data, ranging from 0.897 to 0.944 across observers). The best-fitting exponential model for conjunction searches allocated a separate asymptotic (λ) and rate (β) parameter to each of the six conditions, and a single intercept (δ) parameter (adjusted-R² = 0.984 for the average data, ranging from 0.889 to 0.961 across observers). Table 1 shows average parameter values.

First, consider how attention affected discriminability, measured by the levels of asymptotic accuracy (λ). In the neutral condition in Fig. 3 (open symbols), where target location was not cued, asymptotic discriminability decreased as set size increased from one to eight in both feature (0.33 d′ units; P < 0.005) and conjunction (0.41 d′ units; P < 0.005) searches. When target location was peripherally cued, asymptotic discriminability improved. The cue improved overall performance by 0.36 d′ units in feature (P < 0.01) and 0.35 d′ units in conjunction (P < 0.01) searches. This difference in discriminability was evident in all observers' data, with differences ranging from 0.21 to 0.66. Furthermore, the discriminability differences across set sizes in the neutral condition were attenuated by cueing the target location, from 0.33 to 0.13 in feature (P < 0.01) and from 0.41 to 0.26 in conjunction (P < 0.01) searches. Because both set size and location cueing affected asymptotic accuracy, exponential model fits that ignored the impact of either factor produced substantially lower adjusted-R² values than did the class of models that allocated a separate asymptotic parameter (λ) to each of six conditions. In fits of the average data, for example, a 6λ-1β-1δ model produced an adjusted-R² value of 0.970 for features searches and 0.966 for conjunction searches, as compared (respectively) to 0.852 and 0.845 for a (null) 1λ-1β-1δ model, 0.885 and 0.883 for a 3λ-1β-1δ model in which separate λs were allocated to each set size irrespective of cueing, or 0.954 and 0.917 for a 2λ-1β-1δ model in which separate λs were allocated to each cueing condition irrespective of set size. All subjects exhibited similar reductions in R² for the 1λ-1β-1δ, 2λ-1β-1δ, and 3λ-1β-1δ models when compared with a 6λ-1β-1δ model.

Next consider processing time, measured by intercept (δ) and rate (β). For the neutral feature search, processing time was unaffected by set size. Model fits that varied intercept or rate as a function of set size reduced the adjusted-R² for each observer and for the average data, indicating that the additional parameters were not accounting for systematic variance in the data. For the neutral conjunction search, there was clear evidence that set size affected processing speed. As with our earlier investigation of temporal dynamics of conjunction searches (without precueing; ref. 30), these differences in processing speed were adequately modeled as differences in rate (β). A 3λ-3β-1δ model, in which a separate rate parameter was allocated to each set size, yielded an adjusted-R² of 0.979, as compared with 0.965 for a 3λ-1β-1δ model that assumed differences in discriminability (λ) alone.¶ All observers showed an improvement in adjusted-R² for the 3λ-3β-1δ model over the 3λ-1β-1δ model, and the rate estimates monotonically decreased as set size increased for every observer. On average, processing speed slowed in conjunction searches by 108 ms as set size increased from one to eight. The results for both feature and conjunction search replicated our previous findings (30). The pattern in both studies is indicative of parallel processing, capacity-limited in the case of conjunction searches (30).

The crucial finding of this study is that peripheral cueing not only improved discriminability but also accelerated processing dynamics. In feature search, cueing the target location reduced processing time by an estimated 45 ms. For the average data, a 6λ-2β-1δ model, which allocated separate rates for peripherally cued and neutral conditions, yielded a slightly higher adjusted-R² than did a simple 6λ-1β-1δ model (0.978 versus 0.970). However, this model engendered higher R² values in fits of all observers' data. Crucially, the rate estimates from this model were faster for the peripherally cued conditions than the neutral conditions for every observer (P = 0.002).

In the conjunction search, where processing time slowed with increasing set size, peripheral cueing reduced processing time for each set size by 33, 84, and 106 ms for set sizes one, four, and eight, respectively. In fits of the average data, a 6λ-6β-1δ model produced an adjusted-R² value of 0.984, which was higher than a simple 6λ-1β-1δ model (0.966), a 6λ-3β-1δ model that assumed no effect of cueing on processing speed (0.970), and a 6λ-2β-1δ model that assumed that only cueing (but not set size) affected processing speed (0.976). Again, this advantage in adjusted-R² was evident in all observers, and the rate estimates for peripheral cued conditions were faster than the corresponding estimates for the neutral cueing conditions for every observer (P < 0.005). The differences in rate ranged from 18 to 240 ms. The rate estimates also showed that attention attenuated differences in processing time across set size, from 108 ms in the neutral condition (P < 0.005) to 35 ms in the peripherally cued condition (P > 0.1).

Table 1 shows the exponential parameter estimates from fits of the 6λ-2β-1δ model to the average data in the feature search and the 6λ-6β-1δ model to the average data in the conjunction search. The smooth functions in Fig. 2 show the fits of these models to the average data.

Table 1.

Exponential parameter estimates

Parameters	Peripheral cue	Neutral cue
Feature search discriminability (λ in d′ units)
Set size 1	1.88	1.78
Set size 4	2.01	1.46
Set size 8	1.87	1.45
Conjunction search discriminability (λ in d′ units)
Set size 1	1.98	1.76
Set size 4	1.90	1.42
Set size 8	1.72	1.35
Feature search rate (β in ms units)
All set sizes	69	114
Feature search intercept (δ in ms units)
All set sizes	293	293
Conjunction search rate (β in ms units)
Set size 1	68	101
Set size 4	81	165
Set size 8	103	209
Conjunction search intercept (δ in ms units)
All set sizes	296	296

Discussion

This study demonstrated that a peripheral cue increased asymptotic performance and sped processing dynamics in both feature and conjunction searches. These results indicate that transient covert attention not only improves discriminability, but also accelerates information processing.

Accuracy.

The attentional benefit in discriminability when the target appeared amidst distracters could be due to signal enhancement and/or distracter exclusion (1–5, 8–15, 19). However, the improved discriminability when a suprathreshold target was presented alone suggests that covert attention enhances the signal (1, 11–13). This interpretation is consistent with a previous study (1) in which a peripheral cue increased sensitivity along the contrast sensitivity function, even when all factors that affect external noise were removed (viz., distracters, global and local masks, and spatial uncertainty‖). The finding that the peripheral cue had a greater benefit when distracters were present than when they were not (7, 37) is to be expected given that, in the former, both signal enhancement and noise exclusion mechanisms may be at play.

That attention can affect the quality of a sensory representation, either by enhancing the signal or reducing the noise, finds additional support in a growing body of physiological evidence: Single-cell recordings have demonstrated that directing attention toward the stimulus can alter responses of V1 neurons and results in stronger and more selective responses in both V4 and MT/MST neurons (12, 14, 15), and functional-MRI studies have shown attentional modulation in striate and extra-striate visual cortical areas (22, 23).

Speed.

Some studies have demonstrated that time is required to shift attention from one location to another location (11, 17, 21, 24). The speed differences documented here, however, cannot be attributed to this type of effect. To do so requires an assumption of serial processing; specifically, that information processing at the relevant location starts only once attention has been deployed to that location. According to this view, the observed temporal differences between the peripheral and neutral cueing conditions would arise because, in the former, attention has already been deployed at the location when the display appears, whereas in the latter, attention must be shifted to the relevant location before processing of the target can start. Many (but not all) researchers have argued that conjunction search involves serial shifts of attention (8, 17, 34, 35), but the consensus is that feature search (with homogeneous distracters as used here) does not involve serial shifts of attention (7, 8, 10, 17, 30, 34). Crucially, our results show that location cueing yielded a temporal advantage in both types of search. Moreover, this advantage emerged even when the suprathreshold target occurred in isolation and hence there was only one relevant location.

The time course patterns observed in both types of searches are consistent with the idea that processing takes place at all locations in parallel (30), but that processing is faster when a peripheral cue enables attention to be focused in a restricted region than when attention is more diffusely allocated across the display. Indeed, the finding in this study and in others (7, 37), that the peripheral cue diminished the set size effect rather than eliminated it, provides further evidence for parallel processing: Even when the target location was precued, performance was still affected by irrelevant locations. Given the strong evidence for parallel processing, our findings provide clear evidence that covert attention, in addition to improving discriminability, accelerates the rate of information processing at a specific location.

Additional work is needed to specify the precise mechanism by which attention speeds the processing of visual information. Covert attention may serve to boost the rate at which target information is acquired directly or, alternatively, may speed overall processing by enabling the observer to exclude task-irrelevant information from noise sources. Of course, these alternative accounts are not mutually exclusive and may not even be separable notions. A capacity-limited parallel model, for example, posits that the number of locations that are concurrently processed determines the rate of processing for a target at a particular location (18, 30). If attention enables noise exclusion, this type of processing architecture predicts that diverting resources from the noise locations to the target location will speed the processing of the target. However, to the degree that these notions are theoretically separable, we believe that accounts assuming that attention can speed the uptake of target information hold the most promise. We observed increases in the rate of processing in features searches, where capacity limits are minimal (30) or nonexistent (10), and when the suprathreshold target was presented in isolation, so that noise sources were minimal.

Whatever account proves correct, we note that the advantage in speed brought about by covert attention—from 33 to 106 ms on average—is substantial on the time scale of visual information processing. The information needed to identify many visual forms is extracted within 100 ms. In reading, for example, the visual system extracts the information necessary to identify a word in the first 50 ms of fixating on a region (38). In general, one visual pattern—a mask—will disrupt the processing of another—a target—only if they are presented within 100 ms of each other (39). Thus, this speeding of information processing may determine the success of visual processing in a dynamic environment; attention may enable the visual system to extract information before the potentially interfering effects of upcoming stimuli occur. This speeding of information processing could explain the findings that precueing the target location attenuates masking effects (19, 25, 40).

Conclusion.

Attention affects both spatial and temporal aspects of visual processing. By enhancing the signal, attention improves discriminability and enables us to extract relevant information in a noisy environment; by accelerating information processing, attention enables us to extract information efficiently in a dynamic environment.

Abbreviations

RT

response time

SAT

speed-accuracy tradeoff

cpd

cycle per degree