Feature-based attention and feature-based expectation

Abstract

Foreknowledge of target stimulus features improves visual search performance, due to “feature-based attention” (FBA). Recent studies have reported that “feature-based expectation” (FBE) also heightens decision sensitivity. Superficially, it sounds like the latter work has simply rediscovered (and relabelled) the effects of FBA. However, this is not the case. Here, we explain why.

Attention can prioritize the processing of stimulus features (e.g. red) or dimensions (e.g. colour). This “feature-based attention” (FBA) has been most intensively investigated using visual search paradigms. Consider a search task in which observers view several dot motion patches, and are asked to detect which one is moving coherently (Fig. 1a). Feature-based cues providing valid foreknowledge of the target motion direction (e.g. 45°) facilitate detection performance, relative to neutral or invalid cues [1, 2].

Figure 1.

A. Visual search task. Observers detect which stimulus contains coherent motion (here, top left). The motion can occur in any direction. B. Discrimination task. Observers are cued to report the motion direction (CW vs. CCW) in one of the patches (here, rightmost). The motion direction varies independently between patches. C. Priors and evidence in the visual search task. Simulated motion energy for each stimulus (black lines). The peak at 45° shows motion energy in the target stimulus. Black bars show the sum of motion energy for each stimulus. On valid trials, a prior belief over motion energy is present (upper red trace). Red bars: motion energy after filtering by the prior, equivalent to marginalising over a posterior computed via Bayes’ rule. Evidence in the target stimulus is stronger. D. Priors and evidence in the discrimination task. Valid and invalid cues lead to opposing priors (upper blue lines). When filtered by the prior, the overall level of evidence for CW and CCW is different (blue bars), and so is the optimal criterion (blue dashed line; shift shown by the blue arrow). However, the difference in evidence between blue bars is the same, so an ideal observer should perform equivalently in the two conditions.

A distinct line of research has investigated how expectations about features influence behaviour and modulate brain activity [3]. Consider a discrimination task in which observers view two dot motion patches, and are asked to report whether the motion direction in one patch (e.g., right of fixation) is clockwise (+45°) or counterclockwise (−45°) of vertical (Fig. 1b). When cues signal the expected direction of dot motion (e.g. +45°), observers can combine this prior knowledge with visual feature information. This leads to an overall increase in accuracy.

This advantage for expected features on the discrimination task seems wholly consistent with FBA, just as facilitation in search tasks seems to follow naturally from expectations about the target feature. Superficially, it may thus appear that these two manipulations (which, in our example, both cue an expectation of +45° motion) simply index the same attentional process. Here, however, we argue that this is not the case. Rather, we draw a distinction between manipulations (i) that provide information about the relevance of perceptual signals for a decision (FBA), and (ii) that offer information about signal probability, but not relevance. We call the latter “feature-based expectation” (FBE). In recent years, the distinction between FBA and FBE has attracted growing interest but also provoked confusion. Here, we discuss a principled way of distinguishing these two sources of foreknowledge.

Consider these tasks from the perspective of a theoretical agent who makes the best possible decisions given uncertainty in sensory signals. This “ideal observer” will weight sources of information according to their relevance to the task at hand. Consider first the search task (Fig. 1a,c). In the absence of a motion direction cue, the natural strategy is to sum noisy motion energy across all possible directions, and compare the resulting maxima between patches. However, in the presence of a cue, one motion direction becomes more relevant than others, and thus can be given more weight in the decision. This should lead to more sensitive detection judgments, as described in past studies of FBA [1, 2, 4].

By contrast, an ideal observer discriminating motion direction at a single location should add up and compare noisy motion evidence in favour of the two categories (e.g. clockwise vs. counterclockwise of vertical). Here, both the expected (e.g. +45°) and unexpected (−45°) motion signals are equally relevant, and should be given equivalent weight, irrespective of whether an FBE cue is available or not (Fig. 1d). Thus, according to the ideal observer framework, knowledge of stimulus probabilities should not enhance discrimination sensitivity. Rather, performance increases because the ideal observer adjusts the decision criterion to respond consistently with the cue more often, leading to a response bias towards the expected category.

The predictions of this normative framework – that FBA will increase decision sensitivity, whereas FBE will increase bias but not sensitivity – are supported by a longstanding psychophysical literature that has computed sensitivity (d’) and bias (β) from relative proportions of correct and error trials. Interestingly, however, recent studies have cast doubt on the classical perspective, suggesting instead that FBE and FBA may both facilitate detection sensitivity. Here, we evaluate these claims in the light of the ideal observer framework outlined above.

In one recent study [5], observers detected a degraded visual object that could occur at one of four spatial locations. The authors report that valid expectations about the category membership (e.g. “car”) of the target object facilitated detection. Here, however, the cues offered observers the opportunity to weight visual information according to its relevance in the high-dimensional space of object classes (e.g. to weight “car” more heavily than “non-car” features). Thus, a facilitation in performance is expected under the ideal observer framework. A similar logic applies to demonstrations that degraded objects are identified more readily when preceded by a valid written name [6]. In the terms proposed above, these interesting findings may be attributable to FBA, rather than FBE.

In another recent study [7], observers judged the presence or absence of a vertical grating embedded in noise, whilst orthogonal FBE and FBA cues provided information about the probability (cue present vs. absence) and relevance of the visual signals respectively. Rather than using conventional approaches to estimating decision-theoretic statistics, the authors used psychophysical reverse correlation methods, a more sensitive approach that measures the influence that signal-like fluctuations in noise (‘noise energy’) in a psychophysical stimulus have on decisions [8]. Choices were better predicted by noise energy in the presence of valid expectation cues (Fig. 2a), and unlike the influence of FBA cues, this effect was strongest when sensory signals were weak (Fig. 2b). This finding was explained by a computational simulation in which FBE cues facilitate detection sensitivity via an early gain control mechanism, whereas FBA acts to reduce noise in the decision process. In another study [9], observers discriminated the tilt (clockwise vs. counterclockwise) of a noisy grating, and PRC was used to assess the influence of sensory signals on choices, under FBA and FBE cues. Here, FBE cues increased sensitivity to more diagnostic, “off-channel” features, whereas FBA had a more general multiplicative influence on decision sensitivity (Fig. 2c).

Figure 2.

A. Left panel: energy sensitivity (i.e. sensitivity to signal-like fluctuations in noise) for attended (green line) and unattended (red line) vertical gratings in a detection task similar to that described in Fig. 1b. Right panel: energy sensitivity for expected (blue line) and unexpected (pink line) gratings, where expectations are guided by FBE cues as defined in the text. B. Detection rate (i.e. false alarm rate, for low energy trials; hit rate, for high energy trials) for as a function of signal energy at vertical orientation for attended and unattended gratings (left panel) and expected and unexpected gratings (right panel). The relative sensitivity advantage for attended gratings (slope of green vs. red line) is strongest for high signal energy trials, whereas the advantage for expected gratings (slope of blue vs. pink line) is greatest for low signal energy. Based on ref. [7]. C. Energy sensitivity for attended (green line) and unattended (red line) gratings in a fine discrimination task (left panel) and for expected (blue line) and unexpected (pink line) gratings. Observers were more sensitive to expected information at more tilted orientations (lateral shift in blue sensitivity curve), where information is most diagnostic for choices (based on ref. [9]). In all panels, black bars indicated significant differences in energy sensitivty between conditions. Error bars and shading show S.E.M.

The latter studies suggest that human behaviour deviates from that of an ideal observer. FBE cues (that offer probabilistic information alone) can enhance decision sensitivity. However, they do so in a fashion that is distinct from classical manipulations of FBA. The precise computational mechanisms underlying FBE and FBA remain unclear, but one possibility is that FBE cues engage an adaptive gain control mechanism that allows limited resources to be allocated efficiently, i.e. to those sensory features that are most likely to occur [3]. Consistent with this view, gratings with expected orientation elicit lower-amplitude BOLD signals in early visual cortex, and they can be decoded more accurately from multivoxel activity, suggestive of sharper population tuning for expected orientation [10].

More generally, FBA has been studied using a range of experimental approaches. Some researchers have employed a variant of the search paradigm that asks observers to respond when a specific feature occurs in an ongoing stimulation stream, reporting a neural advantage on “match” trials. Others have manipulated the similarity of features in a primary and secondary decision task, and measured the benefits when they overlap, prompting the “feature-similarity gain hypothesis”, a prominent model of FBA [11]. Because FBA effects are strongest when targets defined by a common feature are repeated between or within trials, others have suggested that FBA may be related to bottom-up processes such as repetition priming [12]. Given this diversity of experimental approaches, it is perhaps unsurprising that the mechanisms underlying FBA remain controversial. We suggest that it will be crucial for future research to explicitly orthogonalise FBE and FBA – guided by the normative distinction we highlight here – and to measure their separate influence, at both the neural and the computational levels.