Neural Signature of Value-Based Sensorimotor Prioritization in Humans

Abstract

In situations in which impending sensory events demand fast action choices, we must be ready to prioritize higher-value courses of action to avoid missed opportunities. When such a situation first presents itself, stimulus–action contingencies and their relative value must be encoded to establish a value-biased state of preparation for an impending sensorimotor decision. Here, we sought to identify neurophysiological signatures of such processes in the human brain (both female and male). We devised a task requiring fast action choices based on the discrimination of a simple visual cue in which the differently valued sensory alternatives were presented 750–800 ms before as peripheral “targets” that specified the stimulus–action mapping for the upcoming decision. In response to the targets, we identified a discrete, transient, spatially selective signal in the event-related potential (ERP), which scaled with relative value and strongly predicted the degree of behavioral bias in the upcoming decision both across and within subjects. This signal is not compatible with any hitherto known ERP signature of spatial selection and also bears novel distinctions with respect to characterizations of value-sensitive, spatially selective activity found in sensorimotor areas of nonhuman primates. Specifically, a series of follow-up experiments revealed that the signal was reliably invoked regardless of response laterality, response modality, sensory feature, and reward valence. It was absent, however, when the response deadline was relaxed and the strategic need for biasing removed. Therefore, more than passively representing value or salience, the signal appears to play a versatile and active role in adaptive sensorimotor prioritization.

SIGNIFICANCE STATEMENT In many situations such as fast-moving sports, we must be ready to act fast in response to sensory events and, in our preparation, prioritize courses of action that lead to greater rewards. Although behavioral effects of value biases in sensorimotor decision making have been widely studied, little is known about the neural processes that set these biases in place beforehand. Here, we report the discovery of a transient, spatially selective neural signal in humans that encodes the relative value of competing decision alternatives and strongly predicts behavioral value biases in decisions made ∼500 ms later. Follow-up manipulations of value differential, reward valence, response modality, sensory features, and time constraints establish that the signal reflects an active, feature- and effector-general preparatory mechanism for value-based prioritization.

Introduction

When fast sensory-guided actions are required to attain rewards under time constraints, animals are compelled to bias their decision processes to prioritize higher-value alternatives. Such biases are likely set in place in advance of the imperative sensory event, at the time when the relevant contingencies—what action is required for each sensory alternative and how much each is worth—become fully known to the subject. For example, when a rugby player finds herself in a situation in which the impending movement of an oncoming player will constrain her to go for a try or a drop goal, she will prioritize the higher-value option in preparing for that event.

Many studies have shown that the timing and accuracy of perceptual decision reports are biased in favor of more valuable alternatives, especially when sensory information is weak (Whiteley and Sahani, 2008; Feng et al., 2009) or when the time to view and/or respond to it is limited (Diederich and Busemeyer, 2006; Stanford et al., 2010; Noorbaloochi et al., 2015). Extensive research has focused on models and neural correlates of how such value-related biases are expressed in the decision process itself (Bogacz et al., 2006; Feng et al., 2009; Fleming et al., 2010; Rorie et al., 2010; Summerfield and Koechlin, 2010; Mulder et al., 2012). However, the neural processes involved in putting these bias settings in place in preparation for an impending sensorimotor decision have received far less attention.

One obvious prerequisite for establishing value-based bias is a representation of value itself. A vast literature has described value sensitivity of neural activity in several brain regions (Rangel et al., 2008; Bartra et al., 2013), including regions implicated in perceptual decision formation such as posterior parietal cortex (Platt and Glimcher, 1999; Serences, 2008). Recent studies have further examined the overlap in neural circuits and mechanisms underlying purely value-based and purely sensory-based decisions (Milosavljevic et al., 2010; Polania et al., 2014; Grueschow et al., 2015). However, no direct link has been made between neural value representations and the degree of bias exhibited in sensory-instructed choice behavior, largely because both of these things are rarely measured in the same experiment, let alone implicated in the same task. Most studies of value encoding have focused on either free economic choice (Sugrue et al., 2004; Dorris and Glimcher, 2004) or motivational modulations of precued action preparation due to asymmetric reward schedules (Watanabe, 1996; Roesch and Olson, 2003; Watanabe et al., 2003) rather than value-biased sensorimotor decisions.

Here, we devised a paradigm that combines the fundamental elements of tasks typically used in the separate lines of research on value-based and sensory-based decision making (Summerfield and Tsetsos, 2012). Our sensorimotor decision was a suprathreshold color discrimination reported via a saccadic eye movement within a tight deadline, in which each color alternative was associated with a different number of points. In advance of this imperative cue, we presented the two value-associated sensory alternatives (colored discs) as peripheral “targets” placed to the left and right, indicating which alternative is linked to which action in the current trial. This allowed identification of spatially selective signals scaling with relative value analogous to those found in the monkey oculomotor system (Platt and Glimcher, 1999; Sugrue et al., 2004), which, on the scalp, should reverse across hemispheres depending on the position of the higher value alternative. We thus uncovered a transient, spatially selective signal in the event-related potential (ERP) ∼300 ms after target presentation, which scaled with relative value and the variation of which both across and within subjects was strongly predictive of the choice and reaction time (RT) biases exhibite on the sensorimotor decision performed as much as 500 ms later. Follow-up experiments established that the signal bears key characteristics that reflect an active feature- and effector-general role in prioritization and set it apart from previously characterized value, salience, intention, attention, and priority signals in any species.

Materials and Methods

Main experimental task.

A total of 15 neurologically normal (seven male), paid volunteers aged 22–40 years (mean 29) participated in the main experimental task of the study (see Figs. 1, 2, and 3). All subjects provided written informed consent and the Institutional Review Board of The City College of New York approved all experimental procedures. All subjects reported normal or corrected-to-normal vision. Subjects performed a cued-saccade task toward targets presented on a 56 cm LCD monitor (refresh rate 85 Hz) placed 57 cm away. The visual stimuli were generated using PsychToolbox (Brainard, 1997; Pelli, 1997) implemented in MATLAB (version 7.11; MathWorks).

Figure 1.

Original value-biased, speeded discrimination task and behavioral signatures of prioritization. A, Trial timeline of the main task. Subjects were presented with two colored discs (“targets”), respectively, associated with five and 40 points. After a delay, the central fixation color changed to one of the two alternatives, instructing the subjects to make an immediate saccade to the corresponding target within a tight deadline to earn the corresponding number of points. B, Saccade RT distributions pooled across subjects showing higher error rates and longer correct RTs for lower-value cues. C, Individual error rate differences (low-value minus high-value cues) plotted against RT effects (ROC AUC for correct high-value versus low-value saccades) revealing a wide range across subjects in the degree of behavioral value-biases.

Figure 2.

Electrophysiological signature of relative valuation in advance of the imperative visual discrimination in the original task. A, Electrode × time map of t-values quantifying the effect of placement of the higher value target on the left versus right side of the display. To identify spatially selective, value-encoding activity specifically, electrodes were considered in symmetric pairs ordered from anterior to posterior and the minimum absolute t-value across the two hemispheres is marked only for time points/electrode pairs having a signed t-value of opposite polarity on the left versus right hemisphere. This analysis highlighted a PCP, which, as shown in the following sections, is replicated in several other experiments in the same subjects (Fig. 4) and also in a different set of subjects (Fig. 5). B, Topographic map of the differential potential for high-on-left minus high-on-right trials integrated over the time period 280–400 ms, along with the waveforms for the individual conditions from the electrodes at the left (PO7) and right (PO8) focus. C, Average contralateral signal for four subgroups of subjects indicated in Figure 1C.

Figure 3.

Behavior prediction by the PCP. A, Intersubject correlation between the amplitude of the PCP and the value-based, starting-point bias parameter of the diffusion model. B, Saccade RT distributions for trials with a larger (left) and smaller (right) single-trial PCP amplitude with respect to the median. The inset shows the individual starting point bias parameter fits for large versus small PCP, highlighting the consistency of the relative difference in bias. C, Individual ROC area values for the prediction of error versus correct action on low-value cued trials based on single-trial PCP amplitude. Single-trial measures of PCP amplitude were taken from the discriminating “component” derived from linear classification of high-on-left trials versus high-on-right trials, conditions that, critically, are independent of behavioral outcome. ROC analysis for prediction of correct/incorrect behavioral outcome was then performed within each of these trial types separately. Due to the manner in which PCP amplitude was derived, ROC area values come out >0.5 (chance) for high-on-left trials but <0.5 for high-on-right, indicating that, in both conditions separately, a greater PCP predicts a greater likelihood of erroneous saccade toward the higher value target on a low-value cue.

In the main experimental task, subjects made rapid saccade responses to a fixation color change cue, with the two differently valued color alternatives (equiluminant green/cyan) presented peripherally shortly beforehand to indicate the stimulus–action mapping for the current trial. Each trial began with the presentation of a central light gray square (0.5°, equiluminant with imperative to-be-discriminated colors) on a midgray background. After holding fixation for 400–450 ms, two peripheral “targets,” isoluminant (determined by photometer) green and cyan discs of 2° diameter, were presented 8° to the left and right of fixation (see Fig. 1A). After a further 750–800 ms, the central fixation changed color to match one of the targets, instructing the subject to make an immediate saccade to the cued target within a tight deadline of 250–300 ms and to maintain fixation on it for at least 200 ms. The deadline was set to 250 ms by default for all subjects during training, but was adjusted before EEG recording for subjects who struggled to meet that deadline on the majority of trials (275 ms for three subjects and 300 ms for five subjects).

The two colors were associated with 5 and 40 points, respectively, which translated into a monetary amount earned for a correct saccade made within the deadline. The initial color-value mapping was counterbalanced across subjects and was switched within each subject after three blocks. If subjects broke fixation at any time from the initial fixation to the cue onset, then the trial was aborted. At the end of the trial, visual feedback was presented on the points awarded in that trial. If the movement started before the cue (“too early”), if its RT was slower than the deadline (“too slow”), or even slower than 550 ms (“way too slow”), or if they captured the wrong target (“wrong target”), then no points were given and the corresponding message was printed on the screen. It should be noted that this task was specifically designed so that speed pressure compels subjects to prioritize high-value alternatives, not perceptual uncertainty, which has primarily been used to induce biases in previous work (Feng et al., 2009). This emulates situations in which preparatory, prioritizing adjustments need to be made as soon as the relevant stimulus–action and value mappings become known, in advance of what might be a perceptually obvious, but time-constrained imperative sensory cue.

Subjects performed six blocks of 120 trials each, with eye position recorded and monitored continuously throughout using a remote eye tracker (EyeLink 1000; SR Research, 1000 Hz). During online task performance, saccade RT, and thus assessment of whether the deadline was met, was determined on the basis of the gaze position exiting the fixation window (centered square with sides of 2° of visual angle). Evaluation of which target was chosen was based on gaze entering a target window (squares of 3° of visual angle).

Subjects' participation was remunerated with a fixed $12 per hour of set-up in addition to a variable amount depending on the subjects' performance during the task. At the end of the experiment, three of the six blocks were selected randomly and the total points earned by the subject across those three blocks was multiplied by $0.005 (a maximum additional compensation of ∼$30).

Behavioral data analysis.

Saccade timing and direction was assessed offline on a trial-by-trial basis using EyeLink's automatic saccade onset detection algorithms (velocity exceeding 30°/s) supplemented with manual checking and correction using a single-trial graphical user interface. Trials in which the subject's gaze left the fixation window before cue onset (2.5 ± 2.3% of trials) were detected online and aborted. Also excluded were rare trials with eye movements occurring after the 550 ms limit (0.4 ± 0.3% of trials). Trials with saccades made to the cued or uncued target were analyzed as correct and error trials in behavioral and ERP analyses, even if made after the reward deadline (but before the 550 ms limit). Differences in error rate were computed between trials on which the lower-value minus the higher-value target was cued. The RT differences were assessed using receiver operating characteristic (ROC) curves quantifying the discriminability of RT distributions for correct high-value versus correct low-value saccades (see Fig. 1C). Two of the 15 subjects recruited for the original task exhibited extremely outlying behavior (four interquartile ranges above the 75th percentile on error rate difference) whereby they made saccades always to the higher value color and never based on the cue color, thus reaching a near 100% error rate on low-value trials. These subjects, who evidently approached the task with a qualitatively different strategy and were not making sensory-instructed decisions at all, were excluded from further analyses.

EEG data analysis.

Subjects' EEG activity was recorded at a sample rate of 500 Hz from 97 scalp electrodes (ActiCap; Brain Products). All analyses were performed using in-house MATLAB scripts incorporating data-reading and filtering functions from the EEGlab toolbox (Delorme and Makeig, 2004). The EEG data were first low-pass filtered at 55 Hz. We linearly detrended channels if doing so reduced the SD of the channel by at least half. Channels with outlying overall variance or which saturated during the recording were interpolated with spherical splines implemented in the EEGlab toolbox. ERPs were epoched from 100 ms before to 800 ms after target onset and baseline corrected relative to −100 to 0 ms. The eye movement data were scanned carefully for blinks and microsaccades on a trial-by-trial basis using a purpose-made graphical user interface for plotting each trial. Trials with large artifacts were excluded from EEG analysis. An additional artifact rejection threshold of 50 μV was applied to the ERP in the time interval −100 ms to 700 ms from the target presentation. An additional low-pass fourth-order Butterworth filter with 10 Hz cutoff was applied to the ERP waveforms for display purposes.

In the data of our original task version, we sought to identify a spatially selective neural representation of the relative value of the two action targets of a similar nature to electrophysiological signals previously identified in the occulomotor system of nonhuman primates (Platt and Glimcher, 1999). We reasoned that, because of the dominant representation of contralateral space by visual and sensorimotor brain regions, such a signal should be revealed as a difference due to the position of the higher value target in space. At each electrode, we thus subtracted the ERP to target pairs in which the higher value appeared on the right from those in which the higher value appeared on the left and identified activity for which the polarity of this difference reversed across hemispheres. To set a conservative criterion on identification of such value-lateralized activity, we required that, within a symmetric pair of electrodes, t tests at a given time point revealed a difference significantly divergent from 0 in opposing directions (i.e., if the left hemisphere electrode C3 had significantly more negative amplitude for high-value targets placed on the right, then right hemisphere electrode C4 must be significantly more negative when high-value targets are on the left). Based on an electrode pair × time map of minimum absolute t-values marking only time points and electrodes for which differences were reversed across hemispheres (see Fig. 2A), we identified a relative positivity contralateral to the higher-value target over posterior scalp and selected PO7 and PO8 to measure it over a time range of 280–400 ms (see Fig. 2B). We termed this the posterior contralateral positivity (PCP). The electrodes and time frame for PCP measurement were fixed across all follow-up experiments based on this. The “average contralateral waveforms” highlighting the PCP in Figures 2C, 5C, and 8B and the rightmost columns of Figures 4 and 6 were derived at sites PO7 and PO8 by subtracting the ipsilateral trace from the contralateral trace with respect to the position of the higher value target and averaging across high-on-left and high-on-right conditions. The restrictive nature of this initial statistical mapping means that we may have missed other value-sensitive signals that are either weaker or not spatially selective; nevertheless, we focused the current study on this spatially selective signal that has not been reported previously and aimed to characterize it fully and replicate it in additional analyses and experiments, deferring a wider exploration of other signals to future studies. We verified that the presence of very small microsaccades in the target–cue interval could not be spuriously generating the PCP by confirming the PCP amplitude did not decrease appreciably when rejecting all trials with any microsaccades compared with when rejecting no such trials; in fact, it increased by 5%.

Figure 5.

Three-value task version. A, RT differences (area under the ROC) between saccades going to the higher- versus lower-value target in each pair. Averages are shown as colored crosses and individual subjects as dots of different shades of gray with no particular ordering. Values >0.5 indicate that saccades had faster RT for the higher value of the pair (low-medium t₍₁₃₎ = 3.82, p = 0.002; medium-high t₍₁₃₎ = 6.12, p = 0.00004; low-high t₍₁₃₎ = 6.48, p = 0.00002). B, Scalp topography of the differential activity (higher value on left minus higher on right) in the time range 280–400 ms, by value pairing. C, Average and confidence interval of the contralateral signal over electrodes PO7–PO8 illustrating the increase of PCP with increasing value difference of the target pair.

Figure 8.

Precue task version. A, Trial timeline of the precue task. Subjects were first presented with the color change cue at fixation to inform them in advance of the correct target color, then 450–500 ms later, the two targets (again green and cyan discs associated with five and 40 points) were presented. After a further delay of 750–800 ms, the central fixation color changed back to gray, cueing the subject to make a button press response with the hand corresponding to the location of the precued color within a tight (250 ms) deadline to earn the corresponding number of points. B, Average and confidence interval of the signal over electrodes PO7–PO8 contralateral to the cued color (as opposed to contralateral to the target with higher longer-term value association). A PCP is clearly observed contralateral to the cued target for both high-value cued trials (t₍₁₄₎ = 4.78, p = 0.0003) and low-value cued trials (t₍₁₄₎ = 3.59, p = 0.003), marginally greater for the former (t₍₁₄₎ = 2.13, p = 0.051). Because the PCP was seen to peak slightly earlier in this task version, the measurement window was shifted earlier by 40 ms to 240–360 ms.

Figure 4.

Additional experiments addressing feature and effector general nature of the PCP. Eight subjects who showed strong behavioral biases in the original task also participated in follow-up experiments. A, Data from the original task in just these eight subjects. B, Data from a unilateral button press version of the task. C, Data from a shape-value version of the task. D, Data from a losses (instead of gains) version of the task. Across all panels, the first column shows the trial protocol for the task version, with the manipulated aspect of the original task version highlighted in a red box; the second column shows RT distributions; the third column shows difference-topographies (high value on the left minus high value on the right) in the PCP time frame (280–400 ms); and the fourth column shows the average and confidence interval of the contralateral signal from electrodes PO7/PO8.

Figure 6.

Follow-up experiments manipulating imperative to prioritize. Three more follow-up experiments were run on the same eight subjects as in Figure 4. Again, the manipulated aspect of the task with respect to the original version is highlighted in red in the trial diagram. A, Data from a version of the task with little value difference but the same average value as the original task. B, Data from a long-deadline version of the task. C, Data from a version of the task with an extremely short deadline. The correct low-value (solid green) and correct high-value (solid red) RT distributions closely overlap when either the value difference (A) or speed pressure (B) is all but removed and, accordingly, the PCP is absent. In contrast, with an extremely tight deadline (C), the incorrect low-value (dashed green) RT distribution closely overlapped with that of the correct high-value (solid red) RT, indicating that subjects acted entirely on the basis of value association, not sensory (color-discriminating) information. A strong and transient PCP was observed in this case.

Behavioral model.

To combine value effects on error rate and RT into a single metric and thereby simplify the analysis of the PCP's predictive relationship with behavioral biases, we fitted a simple drift diffusion model to the behavioral data of each subject on the original task. In the model, a decision variable drifts, under the additive influence of color information (drift rate) and Gaussian noise, from a starting point to one of two opposing decision bounds (+1: correct; −1: error), which triggers the corresponding action when reached. Decision variable trajectories were simulated by cumulatively sampling, at each of a sequence of 1 ms time increments, from a Gaussian distribution with positive mean v (drift rate resulting from sensory evidence, toward the correct bound) and SD s. Following the most common approach in diffusion modeling work, we incorporated value bias into the process via an offset in starting point (Ratcliff and McKoon, 2008; Mulder et al., 2012). On high-value cued trials, the decision variable thus starts from a point z_b where −1 < z_b < 1, and starts from an equally biased but opposite start point −z_b on low-value cued trials. Therefore, a positive value for the starting point bias z_b indicates prioritization of the higher-value alternative. Time spent on processes other than the diffusion to bound were captured in an additive nondecision time t_nd. Fast errors were permitted in the model by adding variability in the starting point taken from a uniform distribution of half range s_z (Ratcliff and McKoon, 2008). Fits of the five free parameters (t_nd, v, s, s_z, and z_b) for each subject were obtained by minimizing a χ² statistic using RT quantiles 0.1, 0.3, 0.5, 0.7, and 0.9, and a bounded version of the SIMPLEX algorithm (Nelder and Mead 1965; fminsearchbnd in MATLAB). The average model parameters fit across subjects were as follows: t_nd = 187 ms, v = 14.2, s = 102, s_z = 0.57, and z_b = 0.18. This model construction and procedure was chosen simply to follow the most common practice, including how value biases are typically incorporated (Ratcliff and McKoon, 2008). We emphasize that our current purposes were not to validate this particular model or compare it against alternative ones, but rather to simply use it to capture value-related biasing of behavior in a single metric, thus enabling us to establish the behavior-predictive nature of the PCP in a straightforward way. The drift diffusion model is well known to be versatile in capturing behavior quantitatively across many tasks performed under many conditions (Ratcliff et al., 2016) regardless of whether particular tasks are in reality performed by integration of noisy evidence (Uchida et al., 2006).

Analysis of PCP relationship with behavior.

To link the PCP with behavior across subjects, we simply computed a Pearson's correlation coefficient between individual PCP amplitude and the degree of behavioral value bias measured in the area under the ROC curve for correct low versus high value RT distributions and also in the parameter z_b of the behavioral model fit (see Fig. 3A). To obtain a robust single-trial measure of the PCP for the purposes of linking the signal to behavior within subjects, we used a multivariate signal processing technique based on the linear projection of the multichannel EEG data onto a single dimension providing a functionally defined “component” (Parra et al., 2005). This technique greatly improves the signal-to-noise ratio by integrating amplitude across channels, with channels weighted so that classification accuracy is maximized between two appropriate experimental conditions. For the current purposes, we computed weights w through a Fisher linear discriminant classifier, discriminating trials in which the high value target was on the right versus on the left of the display as follows:

where Δx is the average difference in the multichannel EEG x(t) between these two conditions in the PCP time frame and R₁ and R₂ are the covariance matrices for the two conditions. Only channels posterior to Cz were used in this analysis to gain the benefit of increased spatially sampling while staying in the region of the PCP and raw data were first high-pass filtered with a 0.5 Hz cutoff. The mean ROC value achieved by this classifier across subjects was Az = 0.72. A permutation test revealed that the ROC value obtained using the true data labels was >100% of the ROC values obtained in the same way but with target side randomly shuffled in 500 iterations (real ROC lay 10 SDs above the mean of the null ROC distribution thus derived). We then computed the single-trial discriminating component y(t) by the following projection:

We sorted the trials as a function of the signal amplitude of y(t) in the PCP timeframe (280–400 ms) and split the data at the median amplitude. We then plotted the group average RT distributions for trials with a smaller and larger PCP, respectively (see Fig. 3B). We emphasize that classifier accuracy was not our end goal; the classifier was derived based on discriminating high-on-left from high-on-right trials independent of any behavioral measure for the sole purpose of providing greater single-trial reliability in our PCP measurements to relate them to behavior. Because we trained our classifier to discriminate two target configuration classes and then analyzed links between the classifier-derived PCP measurements and behavior within each of those classes, the outcome of the analysis relating PCP to behavior is guaranteed to be unbiased. For each subject, we fit the diffusion model to each half of the data, allowing only the value bias parameter z_b to vary (exhaustive search, χ² minimization) with all other parameters taken from the fit of the whole dataset (see Fig. 3B).

To determine the predictive power of the PCP amplitude on the correctness of trial outcome during low-value cued trials, we grouped these trials by high value target location (left, right) and performance (correct, incorrect) and, independently for each high-value target location, compared PCP amplitudes on correct and error trials via ROC analyses (see Fig. 3C). This quantifies the accuracy with which behavioral outcome can be classified on the basis of the PCP.

Additional experiments.

Eight of the subjects who exhibited behavioral biases and a large PCP in the original task were available to return to perform several alternative versions of our value-associated cued saccade task. To determine the factors influencing the invocation of the PCP, each follow-up experiment modified only one key parameter with respect to the original task (see Fig. 4A). All task versions retained the critical structure whereby value-associated sensory alternatives (“targets”) were initially presented on the left and right of the display 750–800 ms before the imperative sensory cue.

In the button press task (see Fig. 4B), instead of eye movements, subjects indicated their decision regarding the sensory cue using mouse button clicks with their right hand while holding central fixation. They were instructed to press the right mouse button with their middle finger if the cue matched the target on the right of fixation and the left button with their index finger when the target was on the left. Because RTs for button presses are longer than for saccades, the response deadline was increased from 250–300 to 350 ms in this condition.

In the shapes task (see Fig. 4C), to test whether the PCP was specific to the discrimination of color, different yellow shapes (square and circle) were used instead as sensory alternatives. Because the fixation point used so far was a small square, we replaced it with a black fixation cross. As in the original version, subjects made a rapid saccade to the target that matched the cue.

In the losses task (see Fig. 4D), instead of gaining points for correct saccades to the cued target, subjects lost points if they failed to do so. They started each block of trials with the maximum number of points (2700) and lost the amount of points corresponding to the value of the cued color if they were too early, too late, or made a saccade to the wrong target.

The small value difference task (see Fig. 6A) was identical to the original task except that we assigned values of 20 and 25 points to the alternatives.

The long-deadline task (see Fig. 6B) tested the importance of having a strict time limit on sensorimotor decision reports for invocation of the PCP. We thus rewarded subjects for correct saccades made up to 3 s after cue onset.

In the high urgency task (see Fig. 6C), we decreased the deadline to 175 ms for most subjects (200 ms for two subjects who had difficulty performing saccades with shorter latencies).

For the three-value task, 14 neurologically normal subjects (seven male; four of whom participated in the original task) aged 18–36 years (mean 26) participated in an additional experiment involving three value levels instead of two (low: 1 point, medium: 20 points, and high: 50 points) associated with three isoluminant colors (green, cyan, and yellow). The deadline for the movement execution was slightly longer than in the previous experiments (300 ms) and fixed for all subjects because the task was slightly more difficult with the three different colors. The color value associations were switched every five blocks of 126 trials, so that each color was associated with each value amount across 15 blocks (see Fig. 5).

In the precue task, 15 neurologically normal subjects (eight male; five of whom participated in the original task) aged 21–35 years (mean 25) participated in an additional experiment using a task version with order of presentation of cue and targets switched, consistent with typical task scenarios examined in target selection studies. After 400–450 ms of central fixation, the gray fixation changed color to green or cyan (the same isoluminant hues), indicating to the subject the correct target color. Then, 450–500 ms later, the green and cyan targets were displayed on the left and right as usual and, after a further delay of 750–800 ms, the fixation color changed back to gray, serving as a “go” cue for the subject to make their planned left or right hand button press response according to the side of the target color (see Fig. 8). Again, the colors were associated with five and 40 points counterbalanced within subjects. The deadline for the movement execution was slightly shorter in this task version (250 ms) because the precue allowed subjects to determine the correct action during the target presentation delay in advance of the “go” cue.

Experimental design and statistical analysis.

All experiments were within-subjects designs. All t tests reported are therefore paired. Correlations were computed across subjects using Pearson's r statistic.

Results

In the standard version of our task, two color “targets” were presented to the left and right of central fixation, representing the sensory alternatives in the upcoming color discrimination (Fig. 1A). After a delay of 750–800 ms, the central fixation abruptly changed color to one of the alternatives (the imperative sensory “cue”) and subjects had to make an immediate saccade to the corresponding target within a tight deadline of 250–300 ms. The two colors were associated with five and 40 points, which translated into a monetary amount earned for a correct saccade made within the deadline. No points were awarded for saccades made to the incorrect target or after the deadline.

Value-biased sensorimotor decision behavior

Due to the imposed deadline, subjects exhibited choice and RT biases consistent with prioritization of the higher-value alternative. Erroneous saccades were made toward the wrong target more often on low-value cues than high-value cues (19.6% versus 10%; t₍₁₂₎ = 4.06; p = 0.0016) and saccadic RT was significantly faster for correct high-value saccades than for low-value ones (Fig. 1B; t₍₁₂₎ = 3.98; p = 0.0018). Plotting the difference in error rate due to cue value against differences in RT distribution for correct high-value versus correct low-value cues (area under ROC curve; see Materials and Methods) revealed that value-based behavioral biases were quite variable across subjects (Fig. 1C), thus facilitating examination of relationships with neural signals.

Electrophysiological index of relative value before the sensorimotor decision

In the ERP to the initial presentation of value-associated, bilaterally placed targets, we sought to identify spatially specific, relative value-encoding activity of a similar nature to electrophysiological signals previously identified in the occulomotor system of nonhuman primates (Platt and Glimcher, 1999). We thus conducted an exploratory analysis to identify amplitude differences due to target placement (higher-value target on the left versus right of the display) that reversed in direction across hemispheres (see Materials and Methods). This analysis highlighted a transient, relative positivity over posterior scalp contralateral to the higher-value color (Fig. 2A), which we therefore called the posterior contralateral positivity (PCP). Plotting individual ERP waveforms for the electrode pair with the longest-lasting significant value effect (PO7 and PO8) confirmed the signal's transient nature, lasting from 280–400 ms, and topographic analysis confirmed its posterior distribution (Fig. 2B). The PCP was larger in subjects who showed a greater degree of behavioral prioritization of the higher-value alternative; this was reflected both in the plots of the PCP waveform for four subgroups of subjects that clustered in terms of biasing behavior (Fig. 2C, corresponding to colors in Fig. 1C) and in a significant correlation across all subjects between the biasing effect on RT (ROC AUC) and individual PCP (Pearson's r = 0.7, p = 0.008).

Across- and within-subject prediction of behavior by the PCP

To further test the relationship between the PCP and subsequent sensorimotor decision behavior, we reduced the error rate and RT effects to a single behavioral bias metric by fitting the RT distributions for correct and incorrect trials with a simple diffusion model with starting point bias (see Materials and Methods). We first confirmed that there was a significant correlation between the PCP and this behavioral bias metric across subjects (Fig. 3A; r = 0.88, p = 0.0001), even stronger than the correlation with the RT effect alone described above. We then sought to determine whether on the single-subject level, trials with a larger PCP exhibit a greater bias than trials with a smaller PCP. To make robust measurements of PCP amplitude on the single-trial level, we applied a linear classifier approach (Parra et al., 2005) to determine a multivariate projection of the ERP data that maximally discriminates between high-on-left and high-on-right trials in the time range of the PCP. Then, within each of these conditions, we divided the trials into two equal-sized bins containing the larger and smaller half of single-trial PCP values directed toward the location of higher value. The pooled RT distributions show that the trials with a larger PCP display a stronger value-based choice and RT bias compared with trials with a smaller PCP (Fig. 3B). Fitting the diffusion model to the trials in each of the two PCP bins within each subject, with only the starting point bias parameter allowed to vary and all other parameters constrained to be equal to their values in the overall fit, confirmed that trials with a large PCP consistently showed a larger value-based bias (t₍₁₂₎ = 4.98, p = 0.0003; Fig. 3B).

As a final test, we took the low-value cued trials alone, on which value bias goes against the cue instruction, and quantified the accuracy with which trial outcome (correct/error) could be predicted on the basis of single-trial PCP amplitude. Such advance error prediction could be made significantly for both trials with high value on the left (t₍₁₂₎ = 26.71, p < 0.00001) and high value on the right (t₍₁₂₎ = 14.81, p < 0.00001), with an average predictive performance of 64% (maximum 78%) (Fig. 3C). These behavior-predictive effects are remarkable given that the PCP transiently comes and goes ∼500 ms before the sensorimotor decision process is kicked off, highlighting its potential role in the advance establishment of decision bias.

Feature and effector general, value-dependent signal invoked only under time constraints

To characterize the signal more fully, we ran seven follow-up experiments that each manipulated a key aspect of the original task. We first present three new experimental conditions that establish the feature- and effector-selective nature of the PCP (Fig. 4), then turn to the factors of value difference and deadline, which more directly influence the need for value-based prioritization (Figs. 5, 6). Apart from one of the value difference manipulations (Fig. 5), these additional experiments were run on eight subjects who displayed clear behavioral value prioritization effects and a robust PCP in the original task (Fig. 4A) and were available to return for further testing.

Given the strong correlations between PCP amplitude and subsequent behavioral biases toward higher-value targets, it could be postulated that the PCP, an interhemispheric difference measure, reflects a form of spatially selective saccadic motor preparation or “response bias” similar to signals found in several areas of the monkey oculomotor system (Dorris and Munoz, 1998; Lauwereyns et al., 2002; Rorie et al., 2010). To examine this, we asked our eight subjects to press a button with the index or middle finger of their right hand while maintaining fixation to indicate whether the cued target was on the left or right, respectively. Note that, in this case, the peripherally presented sensory alternatives do not equate to “targets” in the same sense as in the saccade task, in which they were acquired directly by the imperative actions; here, the “targets” indicate in a more abstract way the actions corresponding to each sensory alternative. Nevertheless, the PCP was again observed in the target-locked event-related response and, again, there was a strong behavioral adjustment in favor of the higher-value target (Fig. 4B). This experimental condition rules out two specific properties for the PCP: first, that it is specific to saccades because it was also evoked by button presses and, second, that it reflects a lateralized, spatial motor plan because both alternative actions were performed by the same hand in this task version.

Our second follow-up experiment tested whether the PCP is specific to the physical feature defining the value-associated sensory alternatives. In this case, the peripheral targets, as well as the imperative cues at fixation, had different shapes (circle and square) associated with the two value levels rather than different colors. The behavioral preference for high-value targets was still present, as well as the PCP (Fig. 4C), indicating that the latter is not specific to the association of value with color in particular.

In the third follow-up experiment, we tested the PCP's sensitivity to reward valence. Here, we investigated whether the PCP encodes the location of the target associated with the better outcome or, alternatively, the location of the highest “priority” target; that is, the one most important not to miss. In this condition, rather than being awarded the number of points associated with the correctly selected target, subjects started out with the maximum number of points (2700) and lost five or 40 points from their total on every incorrect trial. Although losing five points is a preferable outcome to losing 40, it is of lower priority for fast action. Strong behavioral prioritization was again seen and was accompanied by a relative positivity contralateral to the target associated with the higher loss if missed, indicating that the PCP lateralizes according to relative priority (Fig. 4D).

To examine the dependence of the PCP on the size of the value differential between the two potential targets, a slightly different version of the task was run on a different group of 14 subjects, in which three possible colors (mapped to one, 20, and 50 points) were presented in random unequal pairs. Both the RT difference and PCP amplitude were greater for greater target value differences (Fig. 5). Interestingly, this function was not linear in terms of either the ratio or difference of values because, for both the behavior and PCP, the medium-high pairing lay much closer to the low-high than to the low-medium pairing, indicative of a utility function in which 50 points were prioritized disproportionately above the two lower amounts. This experiment also served the valuable purpose of demonstrating replication of the PCP effect in a larger group, of whom only three participated in the preceding task conditions.

In the first three follow-up experiments, the imperative to prioritize higher-value alternatives was comparable despite differences in response modality, sensory feature, or valence. We went on to test directly, in the same eight subjects, whether the invocation of the PCP is specifically contingent on the need for prioritization, which presumably stems from having both a large value difference between the two options and a tight deadline in which to acquire the reward.

The results of the three-value experiment suggest that value difference modulates the amplitude of the PCP as well as the behavioral prioritization. However, in that experiment, the average value (or expected trial value) varied across pairings as well as the relative value, which, in combination with the fact that the pairings were interleaved randomly, may have encouraged a general down-weighting of the low-medium trials, which elicited the smallest PCP. We thus ran a blocked task version with a small value difference but the same average value as the original task (20 vs 25 points; Fig. 6A). As predicted, our subjects did not prioritize the slightly higher value and the PCP was absent.

Second, we sought to test the importance of the response deadline for the invocation of the PCP. We removed the time pressure to make saccade responses by extending the deadline to 3 s and found that both the PCP and behavioral prioritization effect were absent (Fig. 6B). This demonstrates that the timing constraint is crucial to the invocation of the PCP and again underlines the clear link between the PCP and the behavioral adjustment. Note that increased temporal jitter in an evoked potential component can cause a lowering of amplitude in the trial average and, if the PCP were to have increased temporal variability commensurate with the increased RT variability in that task, then this would cause a reduction. However, such temporal variability could only cause a peak reduction and broadening, but could not cause it to disappear entirely. In particular, a 36% increase in temporal variability, which equals the increase in RT SD in the no deadline compared with original task version, would result in only a minor reduction.

A salient characteristic of the PCP that sets it apart from reward modulations previously found in oculomotor circuits (Lauwereyns et al., 2002; Sugrue et al., 2004) as well as any other pre-evidence signal modulation previously shown to predict perceptual choice or accuracy (Kelly et al., 2009; Kelly and O'Connell, 2013; de Lange et al., 2013), is its transience. It peaks at ∼300 ms but concludes by 500 ms in all conditions, well before the imperative sensory cue (750–800 ms). It is still possible, however, that the PCP reflects the beginnings of a process of sustained biasing that tapers off early in this particular task because the prioritization is relatively slight, in that ultimately subjects do tend to primarily follow the cue instruction. To examine this in an additional experiment, we imposed a more extreme deadline than the original task (175–200 ms), leading all subjects to adopt extremely value-biased behavior in which they selected the higher-value color regardless of the sensory cue. The RT distributions for incorrect low-value and correct high-value cues were extremely fast and overlapped very closely in this task, suggesting that subjects planned and executed actions based entirely on the value associations of the targets and not at all on the imperative sensory information in the cue (Fig. 6C). A strong PCP was again observed and was just as transient as seen previously (Fig. 4).

The initial presentation of value-associated targets in each trial of our task bears some similarity to the target selection scenarios studied in the context of visual search, which have been studied extensively using human ERPs among other techniques. The most widely observed spatially selective ERP signal in such scenarios is the so-called “N2pc,” a transient negativity appearing typically at 200–250 ms contralateral to targets or salient objects in a search display, which has been linked to attention orienting (Luck and Hillyard, 1994; Eimer, 1996) and has been found to be sensitive to reward (Kiss et al., 2009). In the contralateral signal waveforms of several of the experimental conditions displayed above, the PCP appears to be preceded by a negative component resembling the N2pc. Examining the topographies across all experiments run on the same eight subjects (Fig. 7), an N2pc is indeed apparent at posterior electrodes in the 200–250 ms time frame in some conditions. Across the task versions that evoked a clear PCP, however, the N2pc appeared relatively inconsistently. Further, in the original task, its amplitude did not correlate significantly with the degree of behavioral value bias (r = 0.41, p = 0.167). Therefore, whereas both the N2pc and PCP signals can show up in the same experiment, there is no obvious systematic relationship between them.

Figure 7.

Fuller illustration of the contralateral activity with respect to the location of the higher value target in three time frames, in the original (row 1) and six control experiments (rows 2–7) that were all run on the same group of eight subjects. Scalp topographies show the differential lateralized activity (high-on-left minus high-on-right) averaged over the following posttarget time windows: 200–250 ms (first column), 280–400 ms (second column), and 450–750 ms (third column). The first time window corresponds to the typical time for observing the classic N2pc component (Luck and Hillyard, 1994). The PCP exhibited an asymmetric topography when shapes were associated with value, which may result from a degree of temporal overlap with the preceding N2pc component, which is more pronounced in the left hemisphere for this condition. This is consistent with previous work showing that the N2pc is more pronounced when targets are defined by their shape rather than by their color (Eimer, 1996) and is generally stronger on the left than on the right hemisphere (Nobre et al., 2000).

The paradigm that we have used in the present study has elements in common with traditional paradigms used to study processes of target selection (Eimer, 1996; Schall, 2001; Hickey et al., 2010), the crucial difference being that, in traditional target selection tasks, the target to be localized to guide action selection is specified in advance of the presentation of the target array itself. Given that we had uncovered a novel neural signal in response to what is ostensibly a typical display of target alternatives, it was of interest to examine the PCP's behavior in a task version that accords with this typical target selection paradigm. We thus analyzed data from an additional 15 subjects who performed a task with identical stimuli, value associations, and contingencies, but where the fixation color change cue was presented 450–500 ms before the targets rather than afterward. Then, 750–800 ms after the targets appeared, the fixation color turned back to the initial gray color to instruct the subject to execute the appropriate action (left or right hand button press) within a strict deadline of 250 ms. With this task sequence, subjects are informed of the correct color at the outset of the trial, so the uncued color effectively takes on a value of zero for the current trial. If the PCP reflects the longer-term association of color to value regardless of the immediate requirements and contingencies of the current trial, then it should lateralize according to the location of the higher value-associated cue regardless of whether that color is the cued target on the current trial. Alternatively, if the PCP reflects prioritization of the more valuable alternative in the current trial during the process of linking colors to actions, then it should lateralize according to the precued target color even when that precued color has a lower long-term value association. As shown in Figure 8, the PCP was observed as a positivity contralateral to the precued color for lower-value cues as well as higher-value cues, demonstrating that it prioritizes the alternative that is immediately more valuable rather than the one associated with greater long-term value. This again underlines its active role in biasing the preparation for impending action choices rather than passively reflecting long-term value associations.

Discussion

Value associations form a central factor in all aspects of natural behavior; they not only provide a basis for free economic choice, but also adaptively shape our sensorimotor computations to maximize rewards in challenging environments. We have investigated the latter case, investigating specifically whether neural processes could be identified before a sensorimotor decision that play a demonstrable role in generating an adaptive bias in that decision based on the relative value associated with the alternatives. Through a series of experiments, we have identified and characterized a discrete neural signal that encodes relative value without regard to response modality, sensory feature, or reward valence, but contingent on the presence of task demands that compel value-based prioritization. Despite its transient expression hundreds of milliseconds before the imperative sensory cue, this PCP strongly predicts the degree of value-based behavioral prioritization exhibited in choice, in RT, and in the bias parameter of a well established computational decision model both across and within subjects.

By taking together the results of our nine experiments, we can set the PCP in the context of current conceptions of value, motor intention, salience, attention, and priority and their known neural correlates. Popular neuroeconomic frameworks place a discrete process of valuation at the center of models of reward-maximizing behaviors (Rangel et al., 2008; Kable and Glimcher, 2009; Padoa-Schioppa, 2011). In fact, the discovery of a spatially selective signal that scaled with relative value in monkey lateral intraparietal area (LIP) in a study that included a sensory-instructed task very much like ours (Platt and Glimcher, 1999) was taken as evidence for such an explicit value representation and played a major part in the development of the field of neuroeconomics (Glimcher and Fehr, 2013). However, although the PCP scales with relative value in a similar way, it only does so in the context of tight speed pressure. This suggests a more active, strategic role rather than a passive or automatic representation of value, such as has been found in recent human neuroimaging work (Lebreton et al., 2009). Further, our valence manipulation showed that it encodes the location that is most costly to miss, rather than the most preferable of the two possible outcomes, further speaking against an automatic valuation account.

The PCP is also markedly distinct from the sustained preparatory signals that have been found to modulate according to reward expectancy in many motor control circuits (Leon and Shadlen, 1999; Lauwereyns et al., 2002; Watanabe et al., 2003). The early and transient timeframe of the PCP, even under extreme urgency conditions in which there is presumably strong motor preparation, clearly distinguishes it from such motor preparatory states. Moreover, the PCP was manifest even when the sensory alternatives were mapped to button presses made with one hand, indicating that the PCP is more likely cast in sensory coordinates than action coordinates to the extent that hand representations are lateralized in cortex. This suggests the possibility that the PCP reflects processes that initially create value biases rather than the modulated motor preparatory states that express those biases, as have been found in the single-unit activity of certain areas of the nonhuman primate brain such as the LIP (Rorie et al., 2010) and the caudate nucleus (Lauwereyns et al., 2002).

Spatially selective, reward-modulated neural activity is famously difficult to dissociate from spatial attention (Maunsell, 2004). In the case of the PCP, for example, it could be argued that its signaling of greater losses as well as greater gains point to the encoding of motivational salience of the targets (Leathers and Olson, 2012), a bottom-up or exogenous form of spatial attention linked to value, which could also account for the observed behavioral biases. Many studies have now provided strong evidence that a history of reward association endows targets with such automatic forms of salience, attracting covert (and sometimes overt) spatial attention to their locations (Kiss et al., 2009; Anderson et al., 2011; Chelazzi et al., 2014; San Martín et al., 2016), sometimes even with maladaptive consequences (Peck et al., 2009; Hickey et al., 2010). However, the disappearance of the PCP when urgency is removed, despite the prevailing value difference, argues against such an automatic form of salience or attentional capture.

Spatial attention can also be directed voluntarily through top-down control. However, if the PCP reflects such a process, then it is an entirely novel one. One of the most well established ERP indices of spatial attention is the N2pc (Eimer, 1996). Although the N2pc is similarly transient and measured from similar electrodes (PO7/PO8), it is of opposite polarity to the PCP, rendering it impossible to misconstrue one as the other. A positive potential contralateral to the higher value target can be equally construed as a negative potential contralateral to the low-value cue, raising the possibility that the PCP reflects more effortful covert selection of that target (Luck and Hillyard, 1994). However, this is directly at odds with the fact that a greater PCP strongly predicts greater biases toward the high-value target, not the low-value target. Moreover, we did observe an N2pc of a typical latency in several conditions of the study (Fig. 7) and, as expected, it was contralateral to the higher-value target. In some attention studies, an additional contralateral positive component has been observed shortly after the N2pc time frame, and was termed the distractor positivity (Hickey et al., 2009; Sawaki et al., 2012). Although this component matches the PCP in polarity, it is wholly opposite in its reported function. It appears contralateral to a distractor rather than a target and has been solely interpreted as suppressing attentional orientation to that distractor. Conversely, the PCP appears contralateral to a higher-value target. Although one could then interpret the PCP as a suppression of the higher value target to prevent “too early” responses earning no points, again, this would be at odds with the fact that, when this process is greater, biases are more strongly expressed toward the higher-value target. That is, the PCP relates to a facilitation of the following movements toward the higher value target, not an inhibition. To our knowledge, there has been no report of a positivity in the PCP time frame that appears contralateral to a relevant, attended, or sought “target” in any ERP study of visual search or covert attention. Although this in itself cannot rule out a spatial attention account of the PCP, it suggests that, whether labeled as attention or not, the PCP must play a distinct role relative to hitherto known signatures of attention.

In describing our results, we used the terms “prioritization” and “priority” to refer to the adaptive biasing of choice behavior and the “importance not to miss” an urgent sensory-instructed action, respectively. However, more specific conceptions of “priority” have developed in the literature, mainly through research into monkey LIP (Gottlieb, 2007; Bisley and Goldberg, 2010). Two defining properties of a priority representation have been set out: (1) it combines bottom-up (including salience) with top-down (including value) aspects of relevance and (2) it serves to guide both motor intention and spatial attention (Fecteau and Munoz, 2006; Bisley and Goldberg, 2010). Therefore, one way to view priority is as the union of the four constructs that we have just addressed above, in which case all of the abovementioned distinctions relative to existing conceptions and correlates together can be regarded to apply to a “priority” account of the PCP. Critically, the oculomotor regions that are most widely thought to house priority representations encode bottom-up visual saliency as well as the building preparation for actions toward the salient objects. The PCP, in contrast, maps spatially to the sensory alternatives, not the actions (Fig. 4B), and concludes hundreds of milliseconds before the actions, even under conditions promoting maximal motor preparation (Fig. 6C). Therefore, although most characteristics of the PCP are consistent with a broader notion of priority, it exhibits marked distinctions with respect to the most well established neural incarnations of priority.

A unique aspect of our task is that both the value–action and stimulus–action mappings are revealed afresh on each trial upon presentation of the targets. The PCP might well be specific to this situation and could reflect a value-based weighting in the process of linking visual features to corresponding motor acts (Salinas, 2004). In general, well established models exist both for the construction of value representations based on reinforcement learning (Dayan and Daw, 2008; Lee et al., 2012) and for the value-based biasing of perceptual decision mechanisms (Diederich and Busemeyer, 2006; Ratcliff and McKoon, 2008), but how exactly the former is translated into the latter in sensory-guided contexts has remained unknown. The present results, which provide empirical evidence for the existence of active processes making this link, represent a valuable clue in bridging these theoretical frameworks. The precise form of these active processes is as yet unclear, but we can speculate that they involve implicit selection of the target matching the higher value sensory alternative, which then acts as a prior favoring the subsequent action linked to that alternative and/or opposing the other.

Although a standard model of value-biased decision formation was used here for the basic purpose of detecting a relationship between PCP amplitude and behavior, future work will need to examine thoroughly the mechanisms of value-biasing in the decision process itself through model comparisons and EEG correlates of evidence accumulation (O'Connell et al., 2012; Kelly and O'Connell, 2013; Polania et al., 2014). A functional imaging approach will be required to investigate the anatomical locus of the PCP.

In summary, using a novel paradigm to study relative value representations in advance of biased sensorimotor decisions, we have identified a novel neuroelectric signal that plays a versatile, active role in prioritizing the more valuable alternative in an upcoming, time-constrained sensorimotor decision. These human neurophysiology findings may seed the development of new assays and biomarkers applicable to disorders associated with, for example, impaired valuation processes (e.g., depression) or mismanagement of value information in instructed behavior contexts (e.g., drug addiction).