Population Response Profiles in Early Visual Cortex Are Biased in Favor of More Valuable Stimuli

Abstract

Voluntary and stimulus-driven shifts of attention can modulate the representation of behaviorally relevant stimuli in early areas of visual cortex. In turn, attended items are processed faster and more accurately, facilitating the selection of appropriate behavioral responses. Information processing is also strongly influenced by past experience and recent studies indicate that the learned value of a stimulus can influence relatively late stages of decision making such as the process of selecting a motor response. However, the learned value of a stimulus can also influence the magnitude of cortical responses in early sensory areas such as V1 and S1. These early effects of stimulus value are presumed to improve the quality of sensory representations; however, the nature of these modulations is not clear. They could reflect nonspecific changes in response amplitude associated with changes in general arousal or they could reflect a bias in population responses so that high-value features are represented more robustly. To examine this issue, subjects performed a two-alternative forced choice paradigm with a variable-interval payoff schedule to dynamically manipulate the relative value of two stimuli defined by their orientation (one was rotated clockwise from vertical, the other counterclockwise). Activation levels in visual cortex were monitored using functional MRI and feature-selective voxel tuning functions while subjects performed the behavioral task. The results suggest that value not only modulates the relative amplitude of responses in early areas of human visual cortex, but also sharpens the response profile across the populations of feature-selective neurons that encode the critical stimulus feature (orientation). Moreover, changes in space- or feature-based attention cannot easily explain the results because representations of both the selected and the unselected stimuli underwent a similar feature-selective modulation. This sharpening in the population response profile could theoretically improve the probability of correctly discriminating high-value stimuli from low-value alternatives.

INTRODUCTION

A host of extrasensory factors—particularly those related to current goals and motivational drives—play a critical role in shaping and refining information processing so that only the most relevant external stimuli dominate awareness. Traditionally, the influence of such top-down biases on perception has been examined by manipulating explicit cues that instruct voluntary shifts of space- or feature-based attention (Posner 1980; Yantis and Egeth 1997). In addition, particularly salient stimuli such as abruptly appearing objects can capture attention in a largely involuntary manner (Yantis and Jonides 1984). Although voluntary and involuntary shifts of attention are thought to be mediated via partially dissociable cortical circuits, the end result is similar: voluntarily attended or physically salient stimuli are processed faster and more accurately (Corbetta and Shulman 2002; Itti and Koch 2001; McAdams and Maunsell 1999; Pashler 1998).

However, in everyday perception the behavioral relevance of a stimulus is often determined by the positive or negative valence of past experiences (e.g., do you have the fruit cup or the cheesecake for dessert?; Pessoa 2009) rather than by an explicit cue or a salient event. Accordingly, it is becoming increasingly apparent that other factors such as estimates of stimulus value based on prior rewards can also shape information processing so that the most behaviorally meaningful objects compete more effectively for cortical representation (Dorris and Glimcher 2004; Krawczyk et al. 2007; Platt and Glimcher 1999; Pleger et al. 2008, 2009; Serences 2008; Sugrue et al. 2004). Most research has focused on the role of learned value in biasing the neural mechanisms that are involved in planning and executing eye movements, suggesting an influence of value on decision making at a relatively late stage of information processing (Dorris and Glimcher 2004; Platt and Glimcher 1999; Seo and Lee 2008; Seo et al. 2009; Sugrue et al. 2004, 2005). However, several recent studies have shown that value also modulates neural gain in early sensory areas that are thought to encode low-level stimulus features, such as primary visual and somatosensory cortex (V1 and S1, respectively; Pantoja et al. 2007; Pleger et al. 2008, 2009; Serences 2008; Shuler and Bear 2006). These later results imply that value biases decision making not only by differentially weighting motor responses, but also by biasing the quality of relevant sensory inputs (but see Liston and Stone 2008).

Although well documented, the influence of value-related modulations on information processing in early sensory areas is not well understood. Stable perceptual representations are thought to be based on the shape of the response profile across populations of sensory neurons that individually respond to different low-level visual features (such as directions of motion, colors, orientations, etc.). In turn, this vector of neural responses represents the probability distribution of a given feature being present in the visual field (Deneve et al. 1999; Jazayeri and Movshon 2006; Ma et al. 2006; Pouget et al. 2003). Thus understanding how value influences population response profiles is necessary to understand how value influences the perception of behaviorally relevant objects. On the one hand, a change in general arousal that increases neural gain in all sensory neurons, regardless of how selective they are for the target feature, will not generally increase the precision of population responses. On the other hand, a selective increase in the gain of those sensory neurons that are most responsive to the relevant stimulus feature should improve the precision of population response profiles (Martinez-Trujillo and Treue 2004; Pouget et al. 2001). Unfortunately, most investigations into the modulatory role of value on sensory processing have used functional magnetic resonance imaging (fMRI), which is not generally sensitive to feature-selective changes in the precision of population response profiles in early sensory areas. Instead, fMRI is typically limited to assessing nonspecific changes in response amplitude across large groups of neurons that are tuned to all possible feature values.

Here we used a forced choice paradigm with a variable-interval payoff schedule to dynamically manipulate the relative value of two spatially separated oriented grating stimuli (Lau and Glimcher 2005). A quantitative model was then used to estimate the value of each alternative on a trial-by-trial basis, and fMRI-based voxel tuning functions (or VTFs; Kay et al. 2008; Serences et al. 2009) were used to estimate the influence of value on the shape of population response profiles in early areas of visual cortex. The data suggest that response profiles in early visual cortex are selectively biased in favor of high-value stimulus features, particularly in V1, which contains a high proportion of orientation-tuned cells. In addition, value modulates feature-selective response profiles associated with both the selected and the unselected stimuli on each trial, suggesting that simple space- and feature-based attention explanations cannot account for the present results. These modulations in response selectivity may facilitate the accumulation of information in the downstream areas that compute decisions about how and when to interact with behaviorally relevant objects in the environment (Beck et al. 2008; Ditterich et al. 2003; Gold and Shadlen 2007).

METHODS

Subjects

Eight neurologically intact right-handed observers (five females), ranging in age from 26 to 34 yr old, participated in the study. All subjects gave written informed consent in accord with the human subjects Institutional Review Board at the University of California, San Diego. Each subject was trained for two 1 h sessions outside the scanner and then participated in two 2 h scanning sessions held on separate days. Compensation was $10/hour for training and $20/hour for scanning. Subjects could also earn a monetary bonus based on behavioral performance during the scanning session (see following text for details).

Main experimental task

Observers maintained gaze on a small circular fixation point (0.25° visual angle radius) that was continuously visible in the center of the screen throughout each scan. At the start of each trial, two full-contrast grayscale square-wave gratings were presented (2.5° radius, 1.2 cycles/°) for 1 s in circular apertures centered 8° to the left or right and 2.6° above the fixation point. Each grating alternated with the middle gray background at 5 Hz (100 ms on, 100 ms off) in one of nine possible orientations, evenly spaced across 180° (0, 20, 40, 60, 80, 100, 120, 140, and 160°, where 0° is defined as horizontal; see Fig. 1). The spatial phase of the grating was randomly selected from a set of four possible offsets on each 100 ms “flicker” of the stimulus. The order of orientations was randomized on each scan, with the constraint that the same orientation could not be presented on successive trials and that each of the nine orientations had to appear an equal number of times. On each trial, one stimulus was rotated clockwise (CW) from vertical (90°) and the other was rotated counterclockwise (CCW) from vertical; subjects held an MR-compatible keypad and had to press one of two buttons with their right hand to select either the CW or CCW stimulus. The probability of earning a reward for a particular choice was linked to the rotational offset of each stimulus (CW or CCW). Importantly, no exteroceptive cue indicated whether a reward was available on the current trial or, if a reward was available, which choice alternative would yield that reward. Thus initially, the subject had to choose randomly. After acquiring some experience with the task, the subject's choice was presumably guided by the history of prior rewards and prior choices. Subjects were required to respond within 1.45 s following the onset of the stimulus display and were told to emphasize accuracy in reporting the desired choice over speed. Following the response window, the central fixation point turned red for 0.25 s if the choice was rewarded or green for 0.25 s if the choice was not rewarded (this mapping of color to reward was reversed for half of the subjects). The next trial began 1.3 s later, yielding a trial duration of 3 s. There were 80 trials in each run, along with 20 “null” trials on which no stimulus was presented; these null trials were included to facilitate the estimation of the event-locked hemodynamic response function (Burock et al. 1998; Dale 1999). The sequence of real and null trials was pseudorandomized, with the constraint that a null trial could not occur at the start of a run.

Fig. 1.

Depiction of a single trial of the behavioral task used in the scanner. At the beginning of each trial, subjects viewed 2 stimuli, one in each hemifield: one was oriented clockwise (CW) from vertical, the other counterclockwise (CCW). CW and CCW stimuli switched location between hemifields randomly over the course of the experiment. Subjects selected either the CW or CCW stimulus via a button press response and shortly afterward the color of the fixation point changed to indicate whether the choice earned a reward. The probability of receiving a reward for each choice alternative varied unpredictably from block to block. See methods for details.

Reward structure and payment scheme

On average, a reward was assigned to a stimulus on 40% of the trials. On each of the eight runs that each subject completed in an experimental session, rewards were distributed in a CW/CCW ratio of 1:1, 1:1, 1:2, 1:3, 1:4, 2:1, 3:1, and 4:1, respectively. All subjects experienced the same reward ratios over the course of the experiment, but the order of presentation was randomly determined for each subject and experimental session.

Rewards were assigned to each stimulus on a given trial in an independent and probabilistic manner and once a reward was assigned to a particular orientation offset (CW or CCW), the reward remained available until that option was selected. As a result, the probability that selecting a given alternative would yield a reward increased as a function of the time since that option was last chosen. This “baiting” scheme was adopted to encourage exploration of both choices over the course of a run. At the end of the scanning session, subjects were given $0.10 for every reward they earned; the average reward was $55.23 ± 2.33 and subjects obtained 82 ± 1% (means ± SE) of all available rewards.

Model used to predict stimulus value

In the following text is a brief description of the model that was used to estimate the value of each choice alternative on a trial-by-trial basis based not only on the prior reward history of each option but also on the history of choices made by the subject; full details of the model can be found in Lau and Glimcher (2005). Both prior rewards and prior choices are considered because 1) information about prior rewards can be used to estimate the relative probability of earning a reward for selecting an option and 2) the baiting scheme described earlier should encourage subjects to choose a stimulus that has not been recently selected (thus the importance of accounting for prior choices as well). To estimate the influence of prior rewards on choice, a vector describing the history of rewards experienced by an observer on each trial r(t) was constructed such that

$$r\left(t\right)=\{\begin{array}{l}0\text{if a trial was not rewarded}\\ \text{+1}\text{if CW was rewarded}\\ -1\text{if CCW was rewarded}\end{array}$$

Next, a vector was constructed to represent the choices a subject made on each trial c(t), such that

$$c\left(t\right)=\{\begin{array}{l}+1\text{if CW was selected}\\ 0\text{if CCW was selected}\end{array}$$

Under the assumption that prior rewards and choices combine linearly to guide choice on each trial

$$\log \left[\frac{p\left(C{W}_i\right)}{p\left(CC{W}_i\right)}\right]={\displaystyle \sum _{j=1}^{N_r}{\alpha }_j\left(r_{cw,i-j}-r_{ccw,i-j}\right)}+{\displaystyle \sum _{j=1}^{N_C}{\beta }_j\left(c_{,i-j}-c_{ccw,i-j}\right)+\gamma }$$ (1)

where the log odds of choosing a CW stimulus is a function of 1) rewards obtained j trials in the past, which will increase the log odds by α_j if the reward was received for choosing CW, otherwise it will decrease the log odds by α_j; and 2) choices made j trials in the past, which will modulate the log odds by β_j in an analogous manner. The constant term γ captures bias toward one alternative not attributable to either past rewards or past choices. By exponentiating both sides of the equation, the probability of choosing CW [or CCW, which equals 1 − p(CW)] on each trial can be recovered. Since it is assumed that prior rewards and choices jointly determine the probability of reward on the current trial, this probability estimate is used as a trial-by-trial proxy for the value of each option.

For each subject, 900 models were evaluated, where each model accounted for prior rewards and choices over the last 1–30 trials (e.g., the 900 models encompass all possible combinations of letting N_r and N_c vary independently from 1 to 30). The model that produced the smallest Akaike Information Criterion (AIC) was then selected from the 900 models evaluated for each individual. The AIC is given by

$$AIC=-2\text{ln}\left(L\right)+2k$$ (2)

where L is the maximum log likelihood of the model fit and k is the number of parameters. The AIC enforces parsimony by trading off goodness of fit (which lowers the first term) with the number of parameters (which increases the second term). The number of prior rewards and choices included for each subject is shown graphically in Fig. 2 and also in Supplemental Table S1.¹ Although the use of the AIC was adopted as a principled method to constrain the number of free parameters in the model used for each subject, all qualitative patterns of fMRI modulation reported herein are essentially unchanged if a reasonable constant was used for N_r and N_c across all subjects (i.e., across a range of 5–30 past trials).

Fig. 2.

First two panels in first row: average model parameters across all subjects showing influence of prior rewards (solid circles) and choices (dashed triangles) on the current choice (if no data were present at a particular temporal lag for a given subject, those data were treated as missing values and ignored during averaging). Error bars ±1SE across observers. Remaining panels: model parameters for individual subjects. The percentage in the inset of each panel represents the mean leave-one-out accuracy of the model for each subject (see text). Error bars show ±1SE of the parameter estimates.

fMRI data acquisition and analysis

MRI scanning was performed on a General Electric 3T scanner equipped with an eight-channel head coil at the W. M. Keck Center for Functional MRI, UCSD. Anatomical images were acquired using a magnetization-prepared rapid gradient-echo T1-weighted sequence that yielded images with a 1 × 1 × 1 mm resolution. Echo planar functional images (EPIs) were acquired in 19 axial slices that were positioned to cover primary and ventral visual cortex (repetition time = 1,500 ms, time to echo = 30 ms, flip angle = 90°, image matrix = 96 × 96, field of view = 192 mm, slice thickness = 2.6 mm, no gap).

Data analysis was performed using BrainVoyager QX (v 1.91; Brain Innovation, Maastricht, The Netherlands) and custom analysis routines written in Matlab (version 7.7; The MathWorks, Natick, MA). Data from the main experiment were collected in eight scans per subject, with each scan lasting 369 s; each functional localizer scan lasted for 367.5 s (see following text). EPI images were slice-time corrected, motion-corrected (both within and between scans), high-pass filtered (three cycles/run) to remove low-frequency temporal components from the time series, and then z-transformed on a scan-by-scan basis. The estimated motion parameters were then used to remove motion induced artifacts in the time series of each voxel using a general linear model (GLM).

Functional localizers and region of interest selection in visual cortex

Retinotopic mapping to identify ventral visual areas V1, V2v, V3v, and hV4 was carried out using standard stimulation procedures (Engel et al. 1994; Sereno et al. 1995). These regions contain neurons that have receptive fields primarily in the contralateral visual field, so areas in right visual cortex respond more to stimuli presented in the left hemifield and areas in left visual cortex respond more to stimuli presented in the right hemifield. In each experimental session, two runs of an independent functional localizer task were used to identify voxels within each visual area that responded specifically to the region of external space occupied by the stimuli in the main experimental task. In this localizer task, subjects maintained central fixation while a flickering oriented grating (same temporal frequency, size, spatial position, and set of possible orientations as the main experiment) was presented for 8 s in either the left or the right stimulus apertures, pseudorandomly interleaved with nine 8 s epochs of passive fixation. Subjects were instructed to press a button whenever the contrast of the grating dimmed, which happened at three preselected times on each 8 s trial (the hit rate was maintained at ∼80% by titrating the contrast decrement level that defined a target). A GLM, implemented in Brain Voyager, was then used to identify voxels within each visual area that responded more during epochs of contralateral compared with ipsilateral stimulation. All voxels that passed a false discovery rate corrected threshold of P < 0.05 and that fell within one of the retinotopically defined visual areas were retained for subsequent analysis.

Analysis of mean BOLD response amplitudes in each region of interest

Before analyzing the blood oxygenation level dependent (BOLD) data, trials were sorted into two equinumerous bins based on the value of the selected stimulus (high-value bin, and low-value bin). The magnitude of the evoked BOLD response in each region of interest (ROI; see preceding section) was then computed using a GLM that estimated the event-locked BOLD response as the scalar multiplier of a delta function at each of 10 time points following event onset (0 to 13.5 s poststimulus, a so-called finite impulse response [FIR] model; Dale 1999). Separate regressors were created for stimuli in the left and right hemifields in each of the two value bins. Importantly, this analysis focused only on trials in which no reward was earned, to rule out feedback signals associated with the presentation of a reward as a causal factor in any observed modulations (and regressors of no interest were included in the GLM to account for variance in the BOLD time series associated with rewarded trials). The average response from 3 to 4.5 s following event onset was then used as a summary measure of response amplitude within each ROI because these time points encompassed the peak of the mean event-related time course computed across all subjects and ROIs; see Fig. 4, A and B.

Fig. 4.

A: mean peak blood oxygenation level dependent (BOLD) responses from all areas associated with selected (dark bars) and unselected stimuli (light bars) when the value of the selected stimulus was low and (B) when the value of the selected stimulus was high. C: differential response in each visual area to selected and unselected stimuli as a function of value of the selected stimuli. Responses evoked by selected stimuli were larger overall than responses evoked by unselected stimuli and this difference increased when the value of the selected stimulus was high. All error bars reflect ±1SE across subjects.

Analysis of feature-selective VTFs in each ROI

The analysis described earlier focuses on examining the mean response amplitude across all voxels in the ROI as a function of the value of the stimulus in the receptive field of the ROI. However, this traditional analysis of mean response amplitude cannot reveal information about how feature-selective responses within different populations of sensory neurons might change as a function of stimulus value. Thus VTFs were used to evaluate the influence of value (low vs. high) on the orientation selectivity of individual voxels within early regions of visual cortex. This analysis rests on the assumption that the distribution of feature-selective neurons—in this case the distribution of orientation-selective columns—is not uniform across a given visual area (Boynton 2005a; Kamitani and Tong 2005; Swisher et al. 2010). Due to this nonuniform distribution of neural selectivity, a given voxel may contain more neurons tuned to one particular orientation, giving rise to a Gaussian-shaped response profile across orientations, which we refer to as the VTF (Serences et al. 2009; see also Kamitani and Tong 2005; Kay et al. 2008; Miyawaki et al. 2008).

To generate VTFs, we used a GLM to estimate the trial-by-trial response within each voxel as a function of the stimulus that was presented in the receptive field of the ROI. Thus a separate regressor was specified for each trial and the regressor was labeled based on three factors: 1) whether the stimulus was selected or unselected, 2) the orientation of the stimulus, and 3) the value of the stimulus. The onset time of each trial was marked with a delta function and then convolved with a canonical model of the hemodynamic response function (standard double-gamma function as implemented in Brain Voyager: time to peak, 5 s; undershoot ratio, 6; time to undershoot peak, 15 s). Note that because we were estimating responses on a trial-by-trial basis, we did not use a FIR model to generate the entire event-locked hemodynamic response function; there were too many events and the FIR approach produced extremely noisy data because it requires estimating so many more parameters (i.e., one parameter per time point along the HRF on each trial, as opposed to just one parameter to estimate the scalar magnitude of response on each trial). All VTF analyses focused exclusively on trials in which no reward was earned, to rule out reward-related feedback signals as a cause of any observed modulations; trials on which a reward was earned were modeled separately as regressors of no interest. Finally, after estimating the magnitude of the response within each voxel on every trial, the mean response associated with each orientation across all voxels was removed to correct for main effects of stimulus orientation that had a common influence on the response of every voxel (Haxby et al. 2001; Serences et al. 2009).

After generating an estimate of response magnitude on each trial, we evaluated the orientation selectivity of responses in each voxel using a split-half cross-validation method (to avoid selection bias). First, data from the even-numbered scans were used to assign every voxel within a given visual area to one of nine orientation-preference bins based on the orientation that evoked the largest response (collapsed across value bin). Then data from the odd-numbered scans were used to estimate the mean response of voxels in each orientation-preference bin to low- and high-value stimuli rendered in all nine possible orientations. The result of this analysis was a feature-tuning function indexing the response of a voxel to each stimulus orientation. If a voxel had a consistent orientation preference across even and odd runs, the tuning function should peak at that preferred orientation. In contrast, if a voxel exhibited no consistent orientation preference, then a flat VTF should be observed. This split-half validation procedure was then repeated using the odd runs to define the orientation preference of each voxel and the even runs to evaluate responses in each orientation/value bin. Data from each visual area, hemisphere, and experimental session were analyzed separately and then collapsed across analogous conditions to form an average tuning function for each visual area of each subject. Finally, data from orientation bins that were an equivalent distance from the preferred orientation of each voxel (e.g., +20° and −20°, +40° and −40°, etc.) were averaged to increase the number of samples in each bin since no systematic asymmetry in the shape of the tuning functions was observed.

We used the VTF approach in this study, as opposed to more common multivariate pattern classification algorithms (Kamitani and Tong 2005), because an increase in pattern classification accuracy would not reveal information about exactly how a factor like stimulus value modulates population response profiles. For instance, an increase in classification accuracy with increasing value might be due to a multiplicative scaling of BOLD responses in the voxels that are most selective for the current stimulus or to an increase in the feature selectivity of responsive voxels (i.e., decrease in bandwidth). Since pattern classifiers pool information from all voxels within a given cortical area, they can only confirm the existence of some type of information about stimulus features and are not designed to differentiate these possibilities. In contrast, assuming that changes in the BOLD response are monotonically related to changes in underlying neural activity (Logothetis 2003; Logothetis et al. 2001), VTFs can provide information about how a particular experimental manipulation selectively influences population response profiles in early visual cortex (Serences et al. 2009).

Eye movement data acquisition and analysis

An Avotech SV-7021 (Stuart, FL) infrared eye tracker was used to track eye position during scanning in four of eight subjects to assess fixation during the task. The position of the right eye was sampled at 60 Hz and before each run the eye tracker was drift corrected. Preprocessing and eye position were analyzed using custom functions written in Matlab. The raw data were first binned into temporal epochs corresponding to each trial, then blinks (periods when the pupil disappeared), as well as one sample on either side of each blink, were marked and removed from the epoched data. The average eye position was then computed across the 1 s stimulus presentation interval and one-tailed t-tests were performed to evaluate differences in horizontal and vertical eye positions as a function of the spatial location of the selected stimulus (left vs. right) as well as the value of the selected stimulus (low vs. high). No significant biases in eye position were associated with either factor (all values of P > 0.15; Supplemental Fig. S1). Note that ANOVA would have been a more appropriate statistical approach to control the family-wise error rate; however, separate t-tests were used to maximize the power to detect any potential differences in eye position should they exist.

RESULTS

Behavioral data and quantitative model of choice probability

From the subject's point of view, the task on each trial was to maximize the amount of reward they earned by choosing one of two oriented stimuli that were rotated either clockwise (CW) or counterclockwise (CCW) from vertical (see Fig. 1). Figure 2 (top row) shows the average model parameters describing how previous rewards and previous choices predict choice on the current trial. These group data suggest that choices are biased toward stimuli that were recently rewarded (Fig. 2, left) and away from stimuli that were recently selected (Fig. 2, right); this pattern was also observed in two highly trained nonhuman primates in a similar experimental protocol (Lau and Glimcher 2005). The bias away from recently selected items indicates that the subjects were generally sensitive to the “baiting” scheme that was used to encourage exploration (see methods).

Model fits associated with individual subjects are shown in the subsequent panels. For seven of eight subjects (all but AD), rewards earned in the recent past tended to increase the probability of choosing the previously rewarded option on the current trial (see solid lines in each panel of Fig. 2). The influence of prior choices on the current choice was less consistent across subjects. For one subject, a previous choice to an alternative clearly increased the probability of selecting that alternative again (dashed line, Fig. 2, subject AA). For three subjects, there was little biasing effect of the immediately preceding choice (dashed lines, Fig. 2, subjects AB, AF, and AG). For the remaining four subjects, a prior choice to one alternative strongly biased them to switch to the other alternative (dashed lines, Fig. 2, subjects AC, AE, AH, and AD), most closely matching results obtained using nonhuman primates (Lau and Glimcher 2005). Clearly, human subjects who received a relatively modest amount of experience with the task (∼3,600 trials) exhibited more variability in their strategies. However, the strategy used by each subject tended to be consistent across testing sessions, suggesting that these individual differences are relatively stable over time.

A leave-one-run-out cross-validation procedure was run on data from each subject to evaluate the predictive power of the model. If the estimated probability of picking one alternative was >50% on a particular trial, then that option was “selected” by the model. Averaged across subjects, the model predicted actual choices on 73% (±3%, SE) of the trials (min: 59%, max: 90%; see Fig. 2, text insets in each panel), which is reasonably high given the stochastic nature of the task. In addition, subjects were quite sensitive to manipulations of the relative reward probability associated with the CW and CCW alternatives. Consistent with previous reports, each alternative was selected roughly in proportion to the reward earned from that alternative, as described by Herrnstein's matching law (Fig. 3; de Villiers and Herrnstein 1976; Glimcher et al. 2005; Herrnstein 1961). However, fewer choices were made to the more valuable option as the expected reward ratio deviated from 1:1. For example, when the CW/CCW reward ratio was 4:1 (rightmost data point in Fig. 3), approximately 3:1 choices were made toward the richer CW alternative. This undermatching was also observed in similar tasks that used highly trained nonhuman primates as subjects (see Corrado et al. 2005; Lau and Glimcher 2005).

Fig. 3.

Mean proportion of CW choices as a function of reward probability. Whereas choice ratios generally tracked changes in reward probability, subjects tended to “undermatch” by selecting the more valuable option less often than expected when the reward probabilities were most extreme. Error bars reflect ±1SE across subjects.

To estimate internal representations of value, the model parameters shown in Fig. 2 were used to compute the probability of choosing each alternative on a trial-by-trial basis for each subject. Given that the subjects' goal was to maximize their total monetary reward and that current choices are influenced by past choices and rewards, these model-based estimates of choice probability were used as a proxy for how much value a subject assigned to each alternative on a trial-by-trial basis (see methods). In turn, these trial-by-trial estimates were used to investigate the influence of value on the magnitude of BOLD responses evoked by selected and unselected stimuli from spatially selective areas of retinotopically organized visual cortex (V1, V2v, V3v, and hV4; see methods).

Trials were sorted into two equally populated bins based on value: response times were slightly longer on low- compared with high-value trials (704 vs. 693 ms), but this effect was not significant at the group level [t(7) = 1.7, P = 0.21]. However, speed was not emphasized in the present study because we wanted subjects to be certain of their choices so that they did not make button press errors. Next, the amplitude of the BOLD response was estimated for selected and unselected stimuli separately for each value bin. Responses assigned to analogous value bins were averaged across hemispheres since no significant differences were observed between corresponding left and right visual areas. In addition, responses on trials in which a monetary reward was earned were modeled separately and were thus factored out of the main analysis; therefore all subsequently reported modulations of the BOLD signal were not driven by reward signals that occurred after stimulus presentation and are instead attributable to the expected value of a stimulus given previous choices and rewards.

To evaluate the influence of value on the magnitude of the BOLD response, a three-way repeated measures ANOVA was performed with stimulus selection [selected vs. unselected], value of selected stimulus [high vs. low], and visual area [V1, V2v, V3v, hV4] as factors. Overall, selected stimuli evoked a larger response than unselected stimuli [F(1,7) = 15.24, P = 0.006, Fig. 4, A and B] and this difference increased as the value of the selected stimulus increased, giving rise to an interaction between selection and value [F(1,7) = 9.5, P = 0.018, Fig. 4C]. The three-way interaction between selection, value, and visual area was not significant [F(3,21) = 0.27, P = 0.85]. Finally, responses were slightly larger on low-value trials compared with high-value trials, collapsed across selected and unselected stimuli [F(1,7) = 5.9, P = 0.045]. This main effect of value may be related to the slightly longer reaction times (RTs) on low- compared with high-value trials (see Serences 2008 for a similar observation in a case where both RT and BOLD differences were comparatively larger). As shown in Fig. 4C, the influence of value on the relative magnitude of responses associated with selected and unselected stimuli replicates a recent report using a slightly different paradigm and a slightly different quantitative approach to estimating trial-by-trial fluctuations in stimulus value (see Figs. 1 and 2 in Serences 2008). The fact that the present results are consistent with previous observations serves to partially validate the current model and experimental approach.

In Fig. 4 (described earlier) we show that the relative amplitude of BOLD responses evoked by selected and unselected stimuli is biased by stimulus value. However, analyzing the mean amplitude of responses does not provide any information about how value might alter the response profile across different populations of feature-selective cells tuned to different orientations. Thus we next carried out an analysis of feature-selective VTFs to estimate how value influenced the shape of feature-selective population response profiles (Serences et al. 2009; see methods). A three-way ANOVA with stimulus selection [selected vs. unselected], value of selected stimulus [high vs. low], and orientation tuning [0–80°] as factors was performed to evaluate the VTF data (collapsed across all visual areas). As shown in Fig. 5, A and B, there was a significant main effect of orientation tuning preference such that responses were larger in response to stimuli rendered in the preferred orientation compared with stimuli rotated away from the preferred orientation [F(4,28) = 18.94, P < 0.001]. In addition, VTFs associated with high-value stimuli were more sharply tuned (selective) than VTFs associated with low-value stimuli [two-way interaction between value and orientation offset from preferred, F(4,28) = 8.66, P = 0.001]. Interestingly, the magnitude of this interaction did not depend on whether the stimulus was selected or unselected [three-way interaction between selection, value, and orientation offset, F(4,28) = 0.48, P = 0.75, n.s.; separate two-way ANOVA examining the influence of value on the VTFs associated with only the unselected stimulus: F(4,28) = 3.70, P = 0.015, Fig. 5B]. Most of this sharpening was due to a larger response associated with the preferred stimulus and a relatively attenuated response associated with stimuli rotated 20° from the preferred orientation (Fig. 5, A and B).

Fig. 5.

Mean voxel-based tuning functions (VTFs) after recentering each VTF based on its preferred orientation (e.g., shifting VTFs that peak at different orientations so that all VTFs peak at 0°). VTFs associated with low-value stimuli rendered with a solid line; VTFs associated with high-value stimuli rendered with a dashed line. A and B: average VTFs collapsed across all visual areas for selected stimuli (A) and unselected stimuli (B). High-value stimuli evoked larger responses in voxels tuned to the currently presented stimulus compared with low-value stimuli (compare solid and dashed lines at 0° offset). In addition, responses in voxels tuned 20° from a high-value stimulus tended to be relatively attenuated: this was particularly evident in V1. Note that the area under each of the VTFs is 1 because the mean activation level was subtracted during data processing to focus on the shape of the VTFs (see methods and Fig. 4 for data pertaining to value-related changes in mean response amplitude). Error bars reflect ±1SE across subjects.

Next, data from each visual area were examined separately, with an emphasis on evaluating the influence of value on the slope of VTFs across the 0 and 20° offsets. In V1, high-value stimuli were associated with sharper VTFs [Fig. 5, C and D, two-way interaction between value and orientation offset, F(1,7) = 12.6, P = 0.009]. Similar value-related sharpening was observed in both V3v and hV4 [interaction between value and orientation offset from preferred, F(1,7) = 10.9, P = 0.013 and F(1,7) = 6.38, P = 0.04, respectively; see Fig. 5, E and F and I and J]. However, this interaction was not significant in V2v [F(1,7) = 0.52, P = 0.49, Fig. 5, G and H]. In addition, the value-related sharpening of VTFs did not significantly vary as a function of whether the stimulus was selected or unselected in any area (three-way interactions in V1, V2v, V3v, and hV4 between selection, value, and orientation offset; all values of P > 0.12, with hV4 having the smallest P value).

The data presented in Fig. 5 were based solely on trials in which subjects did not earn a reward, ruling out the actual receipt of a reward as a contributing factor. A separate analysis focused on only the rewarded trials revealed a similar qualitative trend, although the sharpening of VTFs with increasing value did not approach significance (interaction between value and orientation offset: P = 0.32, collapsed across all visual areas). The absence of significant value-related VTF modulations with value may be the result of interference induced by the presentation of the reward or it may be related to the fact that there were far fewer rewarded than unrewarded trials and thus the power of this analysis was substantially diminished.

DISCUSSION

Here, a quantitative model of behavior and fMRI are used to show that a relative increase in the value of an object biases the shape of population response profiles within the regions of visual cortex that are thought to be important for encoding low-level sensory properties of objects in the environment (Fig. 5). This bias in the population response profile was most pronounced in V1, which is perhaps not surprising given this region's presumptive importance in representing information about stimulus orientation (Hubel and Wiesel 1962). However, a similar modulatory pattern was observed in V3v and hV4, although the overall feature selectivity in these regions was substantially attenuated (compare peak magnitude of VTFs in V1 with peak magnitude of VTFs in other areas, Fig. 5). Presumably, altering the critical target feature would alter the cortical region that was most sensitive to value-related modulations. For instance, if the two choices were distinguished based on different directions of motion, then we would predict motion-selective regions such as the middle temporal area would show the most robust pattern of feature selectivity and the greatest sensitivity to changes in stimulus value.

In the context of visual search or visual discrimination tasks, the probability of correctly discriminating one stimulus from another (e.g., 80° grating from a 100° grating) should be proportional to the separation between the distribution of responses evoked by neural populations tuned to each stimulus (leftmost two points on each curve shown in Fig. 5: Deneve et al. 1999; Green and Swets 1966; Jazayeri and Movshon 2006; Ma et al. 2006; Navalpakkam and Itti 2007; Pouget et al. 2003; Seung and Sompolinsky 1993). In the present experiment, in which subjects had to discriminate between orientations separated by as little as 20°, higher value was associated with larger responses in voxels tuned to the stimulus (compare leftmost data points in Fig. 5, A and B), as well as attenuated responses in voxels tuned 20° away from the stimulus. This crossover interaction suggests that value increases the distance between the response distributions in these two populations of neurons, which should increase the probability of correctly discriminating the orientation of the stimulus (Green and Swets 1966; see also Martinez-Trujillo and Treue 2004). In turn, this increase in discriminability should promote a more rapid accumulation of sensory evidence concerning the identity of valuable stimuli in downstream decision mechanisms, almost as if the physical clarity or distinctiveness of the stimulus was enhanced (Beck et al. 2008; Carrasco and McElree 2001; Carrasco et al. 2004; Ditterich et al. 2003; Gold and Shadlen 2002, 2007; Navalpakkam and Itti 2007; Newsome et al. 1989).

The source of the observed modulations in early visual areas cannot be directly addressed in the present study. However, reasonable candidates include dopaminergic (DA) neurons in the ventral tegemental area (VTA) and the substantia nigra pars compacta (SNc) that might influence activity in visual cortex via direct projections to early areas of visual cortex (Berger et al. 1988, 1991; Devoto and Flore 2006). However, these projections are generally thought to be sparse, so it is likely that indirect DA signals relayed through the striatum and then to frontal and parietal cortex play an important role in regulating value-related changes in early visual cortex (Barraclough et al. 2004; Ding and Hikosaka 2006; Dorris and Glimcher 2004; Gläscher et al. 2009; Glimcher 2003; Hikosaka et al. 2008; Hollerman and Schultz 1998; Ikeda and Hikosaka 2003; Lau and Glimcher 2007; Leon and Shadlen 1999; Luk and Wallis 2009; Platt and Glimcher 1999; Schultz and Dickinson 2000; Seo et al. 2007; Sugrue et al. 2004; Wallis and Miller 2003; Watanabe 1996). Indeed, many of the cortical targets of reward signals—such as oculomotor neurons in frontal and parietal cortex—are ideally situated to send modulatory feedback signals to earlier sensory areas so that the cortical representation of high-value stimulus features can be enhanced (Bisley and Goldberg 2003; Ding and Hikosaka 2006; Gold and Shadlen 2007; Moore and Armstrong 2003; Moore et al. 2003; Serences and Yantis 2006; Shadlen and Newsome 2001). In addition to DA neurons, the locus ceruleus sends long-range norepinephrine projections that have been implicated in modulating activity in sensory cortices in response to highly relevant stimuli (Aston-Jones and Cohen 2005; Berridge 2008; Berridge and Waterhouse 2003; Hurley et al. 2004; Manunta and Edeline 2004; Sara 2009; Schultz and Dickinson 2000). Thus although both DA and norepinephrine pathways are viable candidates for the induction of value-related modulations in early visual cortex, future research will be required to more precisely characterize the source of these modulations, perhaps using pharmacological interventions in combination with fMRI (e.g., Pleger et al. 2009; Silver et al. 2008).

Attention and value

The term “selective attention” typically refers to the prioritized processing of behaviorally relevant stimuli (Desimone and Duncan 1995; Hillyard et al. 1998; Kastner et al. 1998; Martinez-Trujillo and Treue 2004; McAdams and Maunsell 1999; Pashler 1998; Reynolds and Chelazzi 2004; Reynolds et al. 1999; Serences and Yantis 2006), thus the relationship between attention and value must be considered in all studies that use rewards to manipulate behavioral relevance (Maunsell 2004). In the present study, the amplitude of responses evoked by selected stimuli was larger than that of responses evoked by unselected stimuli (compare light and dark bars in Fig. 4, A and B), even when the value of the selected stimulus was relatively low (Fig. 4A). This increase in response amplitude is consistent with the hypothesis that spatial attention was directed to the selected stimulus and thus the sharpening of the response profile associated with selected stimuli might be attributable to the differential deployment of spatial attention. However, we observed a similar value-related change in the shape of VTFs associated with both selected and unselected stimuli (compare Fig. 5, A and B), even though unselected stimuli evoked a smaller overall BOLD response (Fig. 4). Thus the modulation of VTFs associated with unselected stimuli argues against an account of these data based solely on voluntary deployments of spatial attention.

We also considered the possibility that value-related modulations of VTFs might be related to feature-based attention because previous single-unit recording studies have shown that attending to a specific feature increases the sharpness of population response profiles (Martinez-Trujillo and Treue 2004). For instance, when subjects attend to one of two stimuli that are simultaneously present in the visual field and presented on opposite sides of fixation (as in the present study), neurons tuned to the attended feature become more active and neurons tuned away from the attended feature are suppressed. These feature-based modulations are spatially global, so neurons that respond to the ignored stimulus on the other side of the display also undergo a similar pattern of excitation/suppression depending on the relationship between the attended feature and each neuron's preferred feature (for reviews, see Boynton 2005b; Maunsell and Treue 2006). Thus a spatially global feature based mechanism is in principle capable of modulating the shape of the VTFs associated with both selected and unselected stimuli in the present study. On this account, more feature-based attention is allocated to the selected stimulus feature when the value of the selected stimulus is high and this leads to a sharpening of response profiles for both stimuli (due to the spatially global spread of feature-based attention). However, this account seems unlikely because the orientations of the selected and unselected stimuli varied randomly from trial to trial: sometimes the two orientations were separated by only 20° and sometimes they were separated by as much as 90°. On average, a spatially global spread of feature-based attention from the selected stimulus to the unselected stimulus would result in nearly equal amounts of response gain (when the orientations were similar) and response suppression (when the orientations were far apart), which runs counter to our observations. Thus the present results suggest that the visual system is sensitive to the value of a stimulus, even if that stimulus is presented outside the current focus of attention while another stimulus is being selected as the target of a behavioral response.

Beyond understanding the nature of the modulations in early visual cortex, it is also important to consider the origins of the top-down signals that mediate these effects. For instance, shifts of attention can be guided not only by top-down factors related to voluntary goals but also by bottom-up factors related to stimulus salience; each of these biasing routes is thought to be mediated by at least partially separable neural mechanisms (Corbetta and Shulman 2002). In contrast to studies that explicitly cue relevant stimuli, subjects in the present experiment had to continuously integrate and remember information about prior rewards and choices to maximize their total earnings. Thus any influence of these factors on activation in visual cortex is likely to have quite a different etiology than that of a similar modulatory effect that was induced by, say, a centrally presented arrow cue that instructed a shift of attention to one item or another (as in the commonly used Posner attention-cueing task; Corbetta and Shulman 2002; Hopfinger et al. 2000; Posner et al. 1980). Ultimately, however, the question here is not whether value induces a shift in attention because the term “attention” can be applied to a variety of modulatory factors. Instead, the goal is to understand how different factors—such as explicit cues, bottom-up salience, the learned value of a stimulus, or emotional valence—influence the manner in which potentially relevant information is selectively processed by sensory systems (Padmala and Pessoa 2008; Pessoa et al. 2002; Phelps et al. 2006; Vuilleumier and Driver 2007). Moreover, examining the influence of prior rewards on information processing is arguably a more naturalistic paradigm in which to understand how previous experiences shape how we represent and interpret external stimuli.

General conclusions

Although stimulus value has long been known to bias decisions and the activity of sensorimotor neurons that mediate behavioral responses, recent evidence suggests that value may also alter the early sensory mechanisms that provide the neural foundation for perception. Here we show that feature-selective population response profiles in early visual cortex are biased in favor of stimuli deemed more likely to yield a monetary reward based on prior experience. These modulations in the population response profile may in turn increase the quality of inputs into regions of parietal and frontal cortex that integrate sensory evidence to form perceptual decisions (Beck et al. 2008; Gold and Shadlen 2002, 2007; Mazurek et al. 2003; Roitman and Shadlen 2002; Shadlen and Newsome 2001). Future studies might investigate this relationship by systematically examining the influence of value on the ability of subjects to perform difficult visual discriminations under explicit speed pressure.

GRANTS

This work was supported by National Institute of Mental Health Grant R21-MH-083902 to J. T. Serences.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).