Shaping what we see: Pinning Down the Influence of Value on Perceptual Judgements

Despite a commonsense view that vision provides a veridical window onto the world around us, a growing body of scientific evidence suggests a more complicated account. Imagine walking down a dark alley and catching a glimpse of a shadowy object. In a sedate neighbourhood you might dismiss it as harmless and inanimate; in a neighbourhood of ill repute you might interpret it as a potential attacker. This shift in categorisation reflects both the impoverished nature of sensory input coupled with the greater danger associated with your altered surroundings. An emerging consensus holds that perceptual decision processes are dynamically shaped by both prior expectations and the salience of objects in the environment. Indeed, to solve the problem of perceptual inference, it has long been known that a top-down, or “generative” model of the world is computationally crucial (Helmholtz, 1856).

Since the application of signal detection theory (SDT) to psychology in the 1950s, it has been clear that categorical decisions, including simple sensory judgements, necessitate decision thresholds (Green and Swets, 1966). These thresholds provide a means to splice up noisy sensory input and recover the most likely causes in the environment. The relatively simple solution provided by SDT is that a decision criterion is applied to the fixed, unchanging sensory evidence on any given trial. Any changes to this criterion, for instance the influences of prior expectations and rewards, are said to occur downstream of the accumulation of sensory evidence. However, while this theoretical dissociation between the compilation of evidence and incorporation of utility is inherent to SDT (and several more complex models of perception), there is no a priori reason to expect that the neural implementation reflects such a division of labour. Thus, the issue of where stimulus value exerts its effects within the sensorimotor transform remains an unresolved empirical question.

In a recent report, Liston and Stone (2008) set out to resolve this theoretical debate with an elegant psychophysical analysis of human observers' behaviour. At the start of each trial, they asked participants to make a rapid eye movement (saccade) to the brighter of two disks presented on a noisy background. Changes in the probability of reward at each target location produced systematic biases in the number of saccades towards the leftward or rightward target, which could have arisen at any stage in the decision pathway. The crucial part of the experiment involved immediately asking for a second judgement, in which the brightness of a new test stimulus had to be compared with the recently chosen target.

The authors reasoned that if value biases the incoming sensory evidence, then this will also scale the subsequent basis against which the test stimulus is compared, leading to a polarisation of the psychometric curves for brightness (note, however, that the designation of the first decision as “motor” and the second “perceptual” is somewhat arbitrary: both required a brightness judgement followed by a motor response, albeit an unspeeded mouse click in the latter case). If sensory noise is also scaled along with the signal, then the slope of these psychometric functions (a measure of variability) and their shift will be correlated. Across all observers, the latter prediction was most consistent with the data. There was a strong correlation between the magnitude of brightness changes induced by the saccadic bias, and the variability in this judgement. In other words, at the more rewarded location, judgements were both brighter and noisier. Furthermore, these perceptual changes were correlated with the extent of the bias in the first decision phase. Together, this evidence indicates that the saccadic bias is not purely motor, but instead amplifies both the signal and noise carried over into a subsequent brightness comparison.

A possible neural basis for the results reported by Liston and Stone (2008) can be found in a recent functional magnetic resonance imaging study. Serences (2008) found that spatially selective activity in visual cortex (V1, V2, V4 and intraparietal sulcus) was systematically modulated by the reward probability associated with a particular target, possibly via attentional control mechanisms, despite the task not containing an overt perceptual component. Interestingly, this modulation was not driven by the self-reported value of the targets, suggesting the influences of spatial value on low-level visual processes are tracked unconsciously. Asymmetric changes in spatially selective visual cortex are consistent with the findings of Liston and Stone (2008), where stimulus value modulated both sensory signal and noise in a spatially dependent fashion. Indeed, if we assume that basic neural codes follow a Poisson form (Ma et al., 2008) then amplification by value will naturally lead to a scaling of both posterior mean and variance, matching the psychophysical results. It remains unclear at what stage in the perceptual decision pathway value-based changes would be observed when the value of a stimulus is defined by identity, and not by spatial location.

More generally, the results of Liston and Stone (2008) and Serences (2008) reviewed here point out the limitations of a simple SDT approach for understanding biases in decision processes. Despite its empirical successes, SDT incorporates the implicit assumption that representation of sensory evidence and the formation of the decision are discrete, serial processes. By contrast, by using thoughtful psychophysical logic, Liston and Stone (2008) show how the influence of value on the perceptual decision process is subtle, altering an intermediate representation in the flow from sensation to action. Combining such careful psychophysics with neural data such as that reported by Serences (2008) will prove crucial in understanding how the brain adjusts attention and perceptual categorisation to optimally sample the environment, and how these changes ultimately shape what we see.