Nucleus accumbens neurons encode predicted and ongoing reward costs

Abstract

Efficient decision making requires that animals consider both the benefits and costs of potential actions, such as the amount of effort or temporal delay involved in reward seeking. The nucleus accumbens (NAc) has been implicated in the ability to choose between options with different costs and overcome high costs when necessary, but it is not clear how NAc processing contributes to this role. Here, neuronal activity in the NAc was monitored using multi-neuron electrophysiology during two cost-based decision tasks in which either reward effort or reward delay was manipulated. In each task, distinct visual cues predicted high value (low effort/immediate) and low value (high effort/delayed) rewards. After training, animals exhibited a behavioral preference for high value rewards, yet overcame high costs when necessary to obtain rewards. Electrophysiological analysis indicated that a subgroup of NAc neurons exhibited phasic increases in firing rate during cue presentations. In the effort-based decision task (but not the delay-based task), this population reflected the cost-discounted value of the future response. In contrast, other subgroups of cells were activated during response initiation or reward delivery, but activity did not differ on the basis of reward cost. Finally, another population of cells exhibited sustained changes in firing rate while animals completed high effort requirements or waited for delayed rewards. These findings are consistent with previous reports that implicate NAc function in reward prediction and behavioral allocation during reward-seeking behavior, and suggest a mechanism by which NAc activity contributes to both cost-based decisions and actual cost expenditure.

INTRODUCTION

Obtaining food and other rewards often requires organisms to invest considerable resources such as time and the expenditure of energy. Decisions that involve these variables are therefore fundamental to adaptive behavior, and require efficient evaluation of both the costs and benefits of potential actions. This ability is disrupted in multiple psychiatric disorders including addiction, ADHD, and schizophrenia (Green and Myerson, 2004; Heerey et al., 2007; Scheres et al., 2009). Recent evidence suggests that the nucleus accumbens (NAc) is part of a distributed neural circuit that regulates cost-based decision making and is essential for animals to overcome heavy cost burdens. Lesions to the NAc shift behavior towards actions that require less effort or shorter temporal delays (Cardinal et al., 2001; Hauber and Sommer, 2009). Likewise, dopamine antagonism or depletion in the NAc produces a similar deficit, inhibiting the completion of larger response costs (e.g., an FR16 schedule of reinforcement) without altering primary motivation (Aberman et al., 1998; Aberman and Salamone, 1999). These observations suggest that normal processing within the NAc is necessary for animals to overcome large response costs to obtain rewards and is involved in normal cost-based decision making (however, see (Winstanley et al., 2005; Walton et al., 2009) for conflicting reports).

NAc neurons encode operant responding for food and other reinforcers as well as cues that predict rewards (Carelli, 2002a; Nicola et al., 2004a, 2004b; Taha and Fields, 2005; Day et al., 2006; Taha and Fields, 2006; Taha et al., 2007). The NAc receives and integrates information from other brain nuclei (such as the basolateral amygdala and anterior cingulate cortex) that have also been implicated in effort-based decision making (Walton et al., 2002; Rudebeck et al., 2006; Floresco and Ghods-Sharifi, 2007; Hauber and Sommer, 2009), and projects directly to motor output structures (Zahm, 2000). Therefore, the NAc (and the activity of NAc neurons) represents a candidate site for the storage or application of cost-related information. Indeed, recent investigations into the neural correlates of reward value and cost have reported that NAc activity reflects the cost-discounted value of a chosen action and the subjective value of delayed rewards (Kable and Glimcher, 2007; Roesch et al., 2009). To continue to elucidate the role of the NAc in cost-based decision making, we trained rats to lever-press for sucrose rewards available at low or high costs (effort-based decision task) or given immediately or following a delay (delayed-based decision task). Electrophysiological data were collected during the performance of these tasks to assess whether NAc neurons encode the costs required to obtain a reward and the potential contribution of delay, and exhibit different responses to discriminative cues that specifically predict reward cost.

METHODS

Animals

Male, Sprague Dawley rats (n = 26, Harlan Sprague Dawley, Indianapolis, IN) aged 90-120 d and weighing 260-350 gm were used as subjects and individually housed with a 12:12 light:dark cycle. All experiments were conducted between 9:00 am and 5:00 pm. Bodyweights were maintained at no less than 85% of pre-experimental levels by food restriction (10-15 gm of Purina laboratory chow each day, in addition to approximately 1 gm of sucrose consumed during daily sessions). This regimen was in place for the duration of behavioral testing, except during the post-operative recovery period when food was given ad libitum. All procedures were approved by the UNC Institutional Animal Care and Use Committee.

Behavioral training

All training occurred in custom experimental chambers (Med Associates) equipped with two retractable response levers, two corresponding cue lights, one reward receptacle, a house light, and a white noise generator. Lever pressing behavior in all rats was initially reinforced on a continuous schedule of reinforcement (fixed ratio 1, FR1) on two levers and without delay, such that every response on either lever resulted in the immediate delivery of a 45mg sucrose pellet to a centrally located food receptacle. A maximum of 100 reinforcers (50 per lever) were available per session (with 1 session per day). After stable responding developed (at least 5 sessions), rats were transferred to a multiple schedule task in which reward delivery was contingent on operant responses in 90 discrete trials per session. Each trial was initiated randomly after a variable time interval, with an average of 20s between trials. Distinct cue lights (located above two response levers) were illuminated for 5s before lever extension to signal which lever was active (i.e., which lever produced reinforcement; see Fig. 1a,b). Response levers were available for 15s unless response requirements were completed, in which case the levers were retracted and the reward was delivered. On 60 forced-choice trials, one cue was presented alone and only a response on the corresponding lever was reinforced. On these trials, responses made on the uncued lever (termed “errors”) resulted in the termination of the houselight for the remainder of the trial period and the absence of sucrose delivery for that trial. On another 30 free-choice trials, both cues were presented simultaneously, allowing a choice between both options. Forced and free choice trials were interleaved throughout the session according to a random selection (without replacement, meaning that a trial type could not be repeated until the other two types were completed). During the acquisition of each task, the response cost and the reward delay of each option was identical (an FR1 schedule of reinforcement; no reward delay).

Figure 1.

Task design and behavioral results. (a,b) Schematic representing effort-based (a) or delay-based (b) decision tasks. On forced-choice low cost/immediate reward trials (left panels), a cue light was presented for 5s and was followed by extension of two response levers into the behavioral chamber. A single lever press (FR1) on the lever corresponding to the cue light led to immediate reward (45 mg sucrose) delivery in a centrally located food receptacle. Responding on the other lever did not produce reward delivery and terminated the trial. On forced choice high cost/delayed reward trials (middle panels), the other cue light was presented for 5s before lever extension. On these trials, a reward was delivered after either sixteen responses (FR16, effort based decision task) or a delay (FR1 + 4 or 8s delay, delay based decision task). Responses on the opposite lever terminated the trial and no reward was delivered. On free choice trials (left panels), both cues were presented simultaneously, and animals could select either response option. After training, NAc electrophysiological activity was monitored in vivo during a single 90-trial behavioral session. (c-e) Behavioral performance in the effort-based task. (c) Percentage of possible rewards obtained on forced-choice trials. Animals overcame high effort requirements to maximize rewards. (d) Percentage of errors on forced-choice trials were significantly below chance levels (50%; p < 0.0001 for both comparisons), demonstrating behavioral discrimination between cues. (e) Response allocation on free-choice trials, as percentage of choices. Dashed line indicates behavioral indifference point (i.e., the lack of a preference). Animals robustly preferred the low cost option (* p < 0.0001). (f-h) Behavioral performance in the delay-based task. (f) Percentage of possible rewards obtained on forced-choice trials. For both trial types, animals obtained nearly all available rewards. (g) Percentage of errors on forced-choice trials were significantly below chance levels (50%; p < 0.0001 for both comparisons), demonstrating behavioral discrimination between cues. (h) Response allocation on free-choice trials. Animals robustly preferred the immediate reward option (* p < 0.0001).

Training occurred over 25 initial sessions. In order to produce an effort disparity between response options for the effort-based decision task, the required FR on one lever (termed the “high cost” option) was gradually increased from 1 to 16 (FR1 to FR16) according to the following schedule: Sessions 1-11, FR1; Session 12, FR2; Session 13, FR4; Sessions 14-16, FR8; Sessions 17-20, FR12; Sessions 21-25, FR16. The fixed ratio on the other lever (termed the “low cost” option) remained the same (FR1) throughout training. Likewise, in order to produce a disparity in reward delay between response options in the delay-based decision task, reward delay (the time interval between the behavioral response and reward delivery) was gradually increased for one option (termed the “delayed reward” option) according to the following schedule: Sessions 1-11, 0s; Session 12, 0.25s; Session 13, 0.5s; Sessions 14-16, 1s; Sessions 17-20, 2s; Sessions 21-25, 4s. The reward delay on the other lever (the “immediate reward” option) was held constant at 0s throughout training and electrophysiological recording.

Following 25 training sessions, all rats were prepared for electrophysiological recording in the NAc as described below. After recovery, rats underwent additional training sessions until behavior was stable. For half of the animals in the delay-based decision task (n=6), the delayed reward option was extended to 8s during an additional five post-surgery training sessions. On the test day, the electrophysiological activity of NAc neurons was recorded in a single session during the decision making task.

This design was advantageous for three reasons. First, it allowed animals to learn the predictive associations fully before effort or delay requirements were altered, meaning that cost-based decisions and learning rates were not confounded. Second, it ensured that initial biases in response allocation did not contribute to electrophysiological results. Third, it allowed full characterization of how behavioral preference changed as a function of effort and reward delay. In each task, the ability to discriminate between cues was necessary for better-than-chance performance on forced-choice trials. Therefore, the number of errors served as a convenient behavioral measure of cue discrimination. As each cue predicted different effort or delay requirements and preceded the opportunity to respond, this design enabled direct comparison of both cue-related and response-related NAc activity. Response allocation on free-choice trials was used to evaluate the subjective value associated with each choice.

Surgery

Animals were anesthetized with ketamine hydrochloride (100 mg/kg) and xylazine hydrochloride (20 mg/kg) and microelectrode arrays were implanted with the NAc, using established procedures (Carelli et al., 2000). Electrodes were custom-designed and purchased from a commercial source (NB Labs, Dennison, TX). Each array consisted of eight microwires (50 μm diameter) arranged in a 2×4 bundle that measured ~1.5 mm anteroposterior and ~.75 mm mediolateral. Arrays were targeted for permanent, bilateral placement in the core and shell subregions of the NAc (AP, +1.3-1.8 mm; ML, ±0.8 or 1.3 mm; DV, -6.2 mm; all relative to bregma on a level skull, (Paxinos and Watson, 2005)). Ground wires for each array were coiled around skull screws and placed 3-4mm into the ipsilateral side of the brain, ~5mm caudal to bregma. After implantation, both arrays were secured on the skull using surgical screws and dental cement. All animals were allowed at least 5 post-operative recovery days before being reintroduced to the behavioral task.

Electrophysiological Recordings

Electrophysiological procedures have been described in detail previously (Carelli et al., 2000; Carelli, 2002b; Hollander and Carelli, 2005). Before the start of the recording session, the subject was connected to a flexible recording cable attached to a commutator (Crist Intstruments) that allowed virtually unrestrained movement within the chamber. The headstage of each recording cable contained 16 miniature unity-gain field effect transistors. NAc activity was recorded differentially between each active and the inactive (reference) electrode from the permanently implanted microwires. The inactive electrode was examined before the start of the session to verify the absence of neuronal spike activity and served as the differential electrode for other electrodes with cell activity. Online isolation and discrimination of neuronal activity was accomplished using a neurophysiological system commercially available (multichannel acquisition processor, MAP System, Plexon, Inc., Dallas, TX). Multiple window-discrimination modules and high-speed analog-to-digital (A/D) signal processing in conjunction with computer software enabled isolation of neuronal signals based on waveform analysis. The neurophysiological system incorporated an array of digital signal processors (DSPs) for continuous spike recognition. The DSPs provided a continuous parallel digital output of neuronal spike events to a Pentium computer. Another computer controlled behavioral events of the experiment (Med Associates Inc., St. Albans,VT) and sent digital outputs corresponding to each event to the MAP box to be time stamped along with the neural data. The neurophysiological system has the capability of recording up to four neurons per microwire using real-time discrimination of neuronal action potentials. However, in the present study 1-2 neurons were typically recorded per active microwire (Chang et al., 1994; Nicolelis et al., 1997; Roitman et al., 2005). Principle component analysis (PCA) of continuously recorded waveforms was performed prior to each session and aided in the separation of multiple neuronal signals from the same electrode. This analysis generates a projection of waveform clusters in a three-dimensional space, enabling manual selection of individual waveforms. Before the session, an individual template made up of many “sampled” waveforms was created for each cell isolated using PCA. During the behavioral session, waveforms that “matched” this template were collected as the same neuron. Cell recognition and sorting was finalized after the experiment using the Offline Sorter program (Plexon, Inc., Dallas, TX), when neuronal data were further assessed based on PCA of the waveforms, cell firing characteristics, autocorrelograms, cross-correlograms, and interspike interval distributions.

Determining phasic response patterns of NAc neurons

Statistical analysis of spike-train data collected during behavioral sessions had two main goals. First, we sought to identify neurons that exhibited increased or decreased activity relative to three behavioral events: cue presentation, lever press responses, and reward delivery. Secondly, we sought to determine whether such response patterns were sensitive to differences in effort or delay. Each analysis is described in detail below.

Changes in neuronal firing patterns relative to behavioral events were analyzed by constructing peri-event histograms (PEHs) and raster displays (bin width, 250ms) surrounding each event using commercially available software (Neuroexplorer for Windows version 4.034, Plexon, Inc). For this analysis, a cell could exhibit a change in activity relative to cue onset (0 to 2.5s following cue presentation), prior to the lever press on a given trial (-2.5 to 0s before the response; the initial lever press was used for the high cost trials in the effort task), following the lever press (0 to 2.5s after response completion; the initial lever press was used for the high cost trials in the effort task), preceding reward delivery (-2.5 to 0s before the reward), or following reward delivery (0 to 2.5s after reward delivery). Individual units were categorized as either excitatory or inhibitory during one of these epochs if the firing rate was greater than or less than the 99.9% confidence interval (CI) projected from the baseline period (10s before cue onset) for at least one 250ms time bin. This CI was selected such that only robust responses were categorized as excitatory or inhibitory. Some neurons in this analysis exhibited low baseline firing rates, and the 99.9% CI included zero. Where this was the case, inhibitions were assigned if e₀ > 2b₀ (where e₀ = the number of consecutive 0 spikes/s time bins during the event epoch and b₀ = the maximal number of consecutive 0 spikes/s time bins during the baseline period). Units that exhibited both excitations and inhibitions within the same epoch were classified by the response that was most proximal to the event in question, unless the most proximal response was ongoing when the event occurred. Importantly, the above analysis was completed separately for both low and high cost trial types to determine how many neurons responded to each cue, lever press, and reward. However, the resultant categories of neuronal response profiles were not mutually exclusive. Thus, a neuron could potentially exhibit an excitation to the low cost cue and an inhibition to the low cost reward, or an inhibition to both the immediate reward cue and the delayed reward cue. Differences in the frequency or proportion of neuronal responses across different trial types or subregions were examined using Fisher’s exact test. Differences in cell counts of phasic neurons between tasks was evaluated using McNemar’s test.

Effort and delay sensitive neurons were identified by comparing the firing rate of event-responsive neurons on low cost and high cost or immediate and delayed reward trials. For each event-responsive unit, the neuronal activity during the given epoch on high value trials (low cost or immediate reward) was compared from the same epoch on low value trials (high cost or delayed reward) using paired t-tests (excitations) or Wilcoxon signed-rank tests (inhibitions; used because many values approached zero and were therefore not normally distributed). For a population of responsive cells, the overall selectivity was determined by comparing the mean differential activity (high value minus low value) to the theoretical mean of a non-selective population (zero) using a Wilcoxon signed-rank test. For population activity graphs in Figures 6&7, the firing rate of each cell was normalized by a Z-score transformation (using baseline mean and standard deviation) to reduce the potential influence of baseline differences in this analysis. All analyses were considered significant at α= 0.05. Statistical and graphical analysis was conducted in Graphpad Prism 4 (Graphpad software, Inc.) and Matlab (Mathworks).

Figure 6.

A subset of NAc neurons is activated throughout reward seeking or delay period. (a-b) PEH and raster plots from a representative reward-seeking activated NAc neuron on low and high cost trials. For both, data are aligned to cue onset, and the black triangle denotes lever extension (at 5s). Trials in raster plots are sorted based on the latency between lever extension and reward delivery (red circles). (c) Mean (± SEM) Z-score of 22 neurons that were excited throughout execution of response requirements on high cost trials. Data are aligned to cue onset (left panel), the initial response (center panel), and reward delivery (right panel). On high cost trials, activity was maintained until reward delivery. (d-e) PEH and raster plots from a single representative delay-activated NAc neuron on immediate and delayed reward trials. Blue circles in raster indicate lever press responses. Other conventions follow from panel a. (f) Mean (± SEM) Z-score of 11 neurons that were activated during the delay period on delayed reward trials. Data are aligned to cue onset (left panel), the lever press response (center panel), and reward delivery (right panel). On delayed reward trials, activity was maintained until reward delivery.

Figure 7.

Neurons inhibited prior to lever press responses exhibit prolonged responses on high cost or delayed reward trials. All conventions follow from Fig. 6. (a-b) PEH and raster plots from a representative response-inhibited NAc neuron on low and high cost trials. (c) Mean (± SEM) Z-score of 58 neurons that were inhibited before the initial response in the effort task. On high cost trials, activity was significantly lower than baseline both following the initial response and prior to reward delivery (*p < 0.01). (d-e) PEH and raster plots from a representative response-inhibited NAc neuron on immediate and delayed reward trials. (f) Mean (± SEM) Z-score of 35 neurons that were inhibited before the lever press response in the delay task. On delayed reward trials, activity was significantly lower than baseline both following the lever press response and prior to reward delivery (*p < 0.01).

Histology

Upon completion of the experiment, rats were deeply anesthetized with a ketamine and xylazine mixture (100 mg/kg and 20 mg/kg, respectively). In order to mark the placement of electrode tips, a 13.5μA current was passed through each microwire electrode for 5 seconds. Transcardial perfusions were then performed using physiological saline and 10% formalin, and brains were removed. After post-fixing and freezing, 50 μm coronal brain sections were mounted and stained with thionin and potassium ferricyanide to reveal a blue reaction product corresponding with the location of an electrode tip. The specific position of individual electrodes was assessed by visual examination of successive coronal sections. Placement of an electrode tip within the NAc core or shell was determined by examining the relative position of observable reaction product to visual landmarks (including the anterior commissure and the lateral ventricles) and anatomical organization of the NAc represented in a stereotaxic atlas (Paxinos and Watson, 2005).

RESULTS

Behavior during effort and delay-based decision making

Animals received 25 training sessions on the effort-based (n=14) or delay-based (n=12) choice tasks before bilateral implantation with chronic microelectrode bundles in the NAc. Multiple behavioral measures indicated that animals successfully acquired the task and could discriminate between cues to guide behavior, overcome large response costs or delays when necessary, and allocate behavior appropriately on free-choice trials to avoid high costs (Fig. 1). On the electrophysiological recording day, animals obtained nearly all available rewards on forced-choice trials (Fig. 1c,f). Likewise, the error rate was significantly below chance levels for both trial types in both tasks (one-sample t-test, comparison with theoretical mean of 50% (chance) error rate, p < 0.0001 for all comparisons; Fig. 1d,g), demonstrating that the animals used the cues to guide ongoing behavior and select the response option that would be rewarded. Moreover, the total number of errors committed decreased with training (test for linear trend, p < 0.0001 for both tasks; data not shown), indicating that behavioral performance was enhanced by experience with each contingency.

On free-choice trials, when both cues were presented and animals were free to respond on either option, behavioral allocation changed as a function of imposed effort and delay (effort task, F_6,91 = 12.70, p < 0.0001; delay task, F_7,82 = 7.79, p < 0.0001 data not shown). Thus, early in training when the options presented no difference in cost (sessions 1-11), animals chose each option equally (Bonferroni post hoc test, p > .05 for each task). However, as the response cost or reward delay was gradually increased for one option, animals demonstrated a significant behavioral preference for the other option, choosing it more frequently (and thus, revealing that they valued the low-cost option more than the high-cost option). This preference was present at the FR1:FR8, FR1:FR12, and FR1:FR16 low cost:high cost effort disparities in the effort task and at the 0s:4s and 0s:8s immediate:delayed reward disparities in the delay task (p < .01 for all comparisons; data not shown). On the recording day, animals significantly preferred the low cost and immediate reward options on free choice trials (paired t-test, p < 0.0001 for each task; Fig. 1e,h). Thus, animals avoided paying high costs by selecting low cost and immediate reward options when possible.

Further analysis of responding during the electrophysiological recording session on the effort task revealed a significant main effect of trial type on median response latency, or the time between lever presentation and initial lever press (paired t-test, t = 3.124, df = 13, p = 0.008). This effect was attributable to shorter response latencies on low cost trials as compared to high cost trials (low cost, 0.33 ± 0.03s; high cost, 0.88 ± 0.17s mean ± SEM; data not shown). In comparison, there was no significant difference in response latency on immediate and delayed reward trials in the delay task during electrophysiological recording (paired t-test, t = 1.422, df = 11, p = 0.183). After the initial response on high cost trials in the effort task, animals required an additional 4.75 ± 0.45s (mean ± SEM) to complete the FR16 requirement. There was no significant difference between the reward delay for high cost trials in the effort task and delayed reward trials in the delay task (unpaired t-test, t = 1.696, df = 24, p = 0.1027).

Overview of NAc firing patterns during behavioral tasks

A total of 230 individual NAc neurons were recorded from 26 rats during behavioral performance (n = 135 neurons from 14 rats in the effort-based choice task and n = 95 neurons from 12 rats in the delay-based choice task). Of these, 204 (88.7%; 122 of 135 in the effort task and 82 of 95 in the delay task) exhibited significant modulation in firing rate during at least one task event. In the effort task, 83 neurons (61.5%) exhibited changes in firing rate during cue presentation, 91 (67.4%) exhibited changes preceding the lever press (initial press used for high cost trials in the delay task), and 104 (77%) exhibited changes during reward delivery. Moreover, 88 neurons (65.2%) exhibited changes in firing rate during the completion of response requirements on high cost trials (i.e., following the initial response or preceding reward delivery). In the delay task, 49 neurons (51.6%) exhibited changes in firing rate during cue presentation, 63 (66.3%) exhibited changes preceding the lever press, and 66 (69.5%) exhibited changes during reward delivery. Additionally, 63 neurons (66.3%) exhibited changes in firing rate during the delay period on delayed reward trials. Neurons with a phasic change in activity were further divided into subgroups on the basis of whether firing rate increased or decreased during a given epoch, and are considered separately below. A complete analysis of neuronal response counts by task, event, and response direction can be found in Table 1.

Table 1.

Neuronal responses to task-related stimuli

	Event
Effort Task (n=135)		Lever Press*		Reward
Low Cost	Cue	Pre	Post	Pre	Post
Increase	41 (30.4)	27 (20.0)	n/a	n/a	36 (26.7)
Decrease	30 (22.2)	47 (34.8)	n/a	n/a	52 (38.5)
Total	71 (52.6)	74 (54.8)	n/a	n/a	88 (65.2)
High Cost
Increase	31 (23.0)	26 (19.3)	30 (22.2)	26 (19.3)	42 (31.1)
Decrease	30 (22.2)	41 (30.4)	45 (33.3)	48 (35.6)	45 (33.3)
Total	61 (45.2)	67 (49.6)	75 (55.6)	74 (54.8)	87 (64.4)
Delay Task (n=95)
Immediate Reward
Increase	14 (14.7)	25 (26.3)	n/a	n/a	30 (31.6)
Decrease	28 (29.5)	28 (29.5)	n/a	n/a	28 (29.5)
Total	42 (44.2)	53 (55.8)	n/a	n/a	58 (61.1)
Delayed Reward
Increase	13 (13.7)	20 (21.1)	29 (30.5)	15 (15.8)	22 (23.2)
Decrease	18 (18.9)	22 (23.2)	31 (32.6)	22 (23.2)	30 (31.6)
Total	31 (32.6)	42 (44.2)	60 (63.2)	37 (38.9)	52 (54.7)

Numbers expressed as count (percentage of total)

^*For high cost trials where multiple presses occurred, the first press is used for calculations

Task-related excitations

Previous studies indicate that a substantial number of NAc neurons exhibit phasic increases in activity during presentation of reward-paired cues, operant responses for rewards, and/or reward delivery (Carelli and Deadwyler, 1994; Hollerman et al., 1998; Ghitza et al., 2004; Nicola et al., 2004a; Roitman et al., 2005; Day et al., 2006; Ambroggi et al., 2008). Consistent with these results, we observed that presentation of reward-paired discriminative stimuli evoked changes in firing rate in a large population of NAc neurons (see Fig. 2a,b for characteristic example for a single cell, and overall proportion of responses in each decision task). In the effort task, 47 of 135 (34.8%) neurons exhibited increases in activity during at least one cue presentation. In the delay task, 19 of 95 neurons (20%) showed similar activity. There was a significant difference in the number of neurons activated by cue presentation in the effort and delay tasks (Fisher’s exact test, p = 0.018), with fewer neurons exhibiting increased activity during cue presentation in the delay task. Another subset of neurons exhibited changes in firing rate that began preceding the operant lever press (Fig. 2c,d). In the effort task, 35 of 135 neurons (25.9%) exhibited increases in activity prior to the operant response, whereas 31 of 95 neurons (32.6%) exhibited increased activity prior to the lever press in the delay task. There was no difference in the proportion of neurons activated during this epoch on each task (Fisher’s exact test, p = 0.3). Finally, a third group of neurons were activated during reward delivery (Fig. 2e,f). This subset represented the largest group of neurons in each behavioral task, with 53 of 135 neurons (39.3%) in the effort task and 33 of 95 neurons (34.7%) in the delay task exhibiting reward-related increases in activity. There was no difference in the proportion of neurons activated during this epoch on each task (Fisher’s exact test, p = 0.49). Critically, there were no differences in the number of neurons that were specifically activated during high and low value trials in each task (McNemar’s test, p > 0.05 for all comparisons).

Figure 2.

NAc neurons are activated during different components of the task. (a) Peri-event histogram (PEH) and raster plots of a representative cue-activated NAc neuron. Data shown are from high cost trials in the effort-based task, and are aligned to cue onset (left panel), the initial lever press response (middle panel), and reward delivery (right panel). Red circles in raster indicate timing of reward delivery. (b) Venn diagrams illustrating proportion of neurons that exhibited excitations following cue onset for the effort- and delay-based tasks. Insets show proportion of total cells that were activated by cues. Lower diagrams show number of cells that responded to the high value cue (gold), low value cue (blue), or both cues (overlap). (c) PEH and raster plots of a representative press-activated NAc neuron. Data shown are from delayed reward trials in the delay-based task. (d) Venn diagrams illustrating proportion of neurons that exhibited excitations preceding the lever press for the effort- and delay-based tasks. (e) PEH and raster plots of a representative reward-activated NAc neuron. Data shown are from high cost trials in the effort-based task. (f) Venn diagrams illustrating proportion of neurons that exhibited excitations following reward delivery for the effort- and delay-based tasks.

Cue activated neurons exhibit population bias towards low effort cues but not immediate reward cues

For the entire population of cue-evoked excitations, we first compared mean and peak activity during the cue onset period (2.5s after cue presentation) on high value and low value trials in each decision task. In both cases, population-level comparisons did not yield significant differences in either average or maximal firing rate during this epoch (paired t-tests, p > 0.1 for all comparisons). However, further examination revealed that a substantial portion of these neurons exhibited cue-selective responses (i.e., increases in activity that were significantly greater during one cue than the other; Fig. 3a,d). In the effort task, 19 of 47 neurons (40.4%) exhibited cue-selective responses, whereas 9 of 19 cue-responsive neurons (47.4%) in the delay task were cue selective. Figure 3b & 3e show the mean activity of each cue-responsive neuron on low vs. high cost and immediate vs. delayed reward trials, respectively. Interestingly, in the effort task, the population of cue-responsive neurons exhibited a significant tendency towards low cost selectivity (Fig. 3c; Wilcoxon signed-rank test, p = 0.04), indicating that as a population these neurons may be sensitive to future reward cost. In comparison, cue selectivity in the delay task was equally distributed for immediate and delayed reward cues (Fig. 3f; Wilcoxon signed-rank test, p = 0.97), indicating that neuronal responses were not modulated by predicted reward delay. Thus, while the entire population of cue-evoked excitations in NAc neurons seemingly signaled reward prediction alone (and provided no information on the costs of future rewards), a unique subset of neurons appeared to exhibit activity that was preferential for low cost options when compared to high cost options but not immediately rewarding options when compared to delayed rewarding options.

Figure 3.

Cue-activated neurons exhibit selective responses that encode future reward cost but not reward delay. (a) PEHs of representative low cost selective (left panel), high cost selective (middle panel), and non-selective (right panel) cue-activated NAc neurons during the effort task. Data are aligned to cue onset (black bar). (b) Mean activity (±SEM) of cue-activated NAc neurons on low and high cost trials. Diagonal line represents purely non-selective activity. (c) Differential activity of all cue-activated NAc neurons in the effort task (low cost minus high cost). Each circle represents a single neuron, with low cost selective neurons shown in gold, non-selective neurons in black, and high cost selective neurons in blue. As a population, these neurons were significantly biased towards the low cost option (p = 0.04). (d) PEHs of representative immediate reward selective (left panel), delayed reward selective (middle panel), and non-selective (right panel) cue-activated NAc neurons during the delay task. (e) Mean activity (±SEM) of cue-activated NAc neurons on immediate and delayed reward trials. Diagonal line represents purely non-selective activity. (f) Differential activity of all cue-activated NAc neurons in the delay task (immediate minus delayed reward). Conventions follow from panel c. As a population, these neurons were not significantly selective for either option (p = 0.97).

We next compared the neuronal responses of low cost selective cells on forced and free choice trials (Fig. 4) to determine whether responses were encoding the potential choice value (i.e., the value of the best available choice) or the specific action value (i.e., the value of the option that was eventually chosen). On forced-choice trials, these neurons exhibited significantly greater activity during the presentation of low cost cues than high cost cues (Fig. 4a-c, left panels; paired t-test, t = 4.857, df = 12, p = 0.0004). On free-choice trials, when both cues were presented and the animal could select between low and high cost options, the activity of these neurons reflected the specific action value (Fig. 4 a-c, right panels). Thus, when animals subsequently selected the low cost option, cue-evoked increases in firing rate were greater than when the high cost option was subsequently selected (paired t-test, t = 3.101, df = 12, p = 0.009). Moreover, because animals selected the high-cost option on significantly fewer free choice trials (see Figure 1), we repeated this analysis after equating the number of trials using random selection of low-cost choice trials. Importantly, even when the number of trials used for analysis were equal for high and low cost choices, there remained a significant difference in cue-evoked activity preceding low cost and high cost choices, with greater activity preceding low-cost choices (paired t-test, t = 3.308, df = 12, p = 0.006). As with comparisons on forced-choice trials, there was no difference in mean or peak firing rate on free choice trials in the delay task on the basis of future choice (p > 0.7 for both comparisons), again indicating a lack of delay-selective neuronal responses.

Figure 4.

Low-cost selective neurons in the effort task reflect action-specific reward value. (a) PEHs of a representative low cost selective neuron on forced choice (left panel) and free choice (right panel) trials. Conventions follow from Fig. 3a. Free choice trials involve the presentation of both cues, but were subdivided into trials in which the animal selected the low cost option or the high cost option. On forced choice trials, the cue-evoked increase in activity was smaller when the high cost option was subsequently chosen. (b) Mean (±SEM) activity traces of all low cost selective neurons (from Fig. 3c) on free and forced choice trials. (c) Comparison of activity following cue onset on forced choice and free choice trials. In both cases, activity was greater when the low cost choice was selected than when the high cost choice was selected (p < 0.01 for both comparisons).

Response and reward activated neurons do not show a population bias for either effort or delay

As a population, neurons that were activated during lever press responses or reward delivery exhibited no differences in mean or peak firing rate on high and low value trials (paired t-tests, p > 0.1 for all comparisons; data not shown). However, similar to cue-evoked excitations, a number of these neurons exhibited activations that were selective for high or low value trials (Fig. 5). To determine whether this selectivity reflected effort or delay in the decision tasks, we aligned neuronal data in several different ways to examine activations that occurred before and after the lever press response and before and after reward delivery. In the effort task, the majority of neurons (25 of 35, 71.4%) that were activated preceding the operant response(s) on low or high cost trials were selective for one response direction. This high degree of selectivity was also observed after the response was initiated (38 of 51 cells, 74.5%) and prior to the reward delivery (28 of 37, 75.7%; Fig. 5a). In comparison, during reward delivery only a small minority of cells (9 of 53, 17%) exhibited trial-selective activity. In each case, differential activity plots revealed that population activity was not biased towards the low or high effort trials (Wilcoxon signed-rank tests, p > 0.13 for all comparisons). In the delay task, large percentages of response or reward-activated neurons were also found to be selective for immediate or delayed reward trials (Fig. 5b). Thus, 14 of 31 (45.2%) pre-response activations, 22 of 31 (71%) post-response activations, 13 of 37 (35.1%) pre-reward activations, and 11 of 33 (33.3%) reward activations were found to be selective. Consistent with results obtained in the effort-based decision task, none of these populations exhibited a significant preference for immediate reward trials over delay reward trials (Wilcoxon signed-rank tests, p > 0.59 for all comparisons), indicating that these neurons did not encode reward cost in the form of effort or delay. These findings are consistent with a previous study which reported a significant portion of response-related activations are selective for the direction of movement rather than the value of the reward (Taha et al., 2007).

Figure 5.

Response- and reward-related activations do not encode effort or delay. (a) Differential activity plots for neurons activated prior to the response, after the initial response, prior to response completion/reward delivery, and following response completion/reward delivery in the effort-based task. Gray area in top panels illustrate the 2.5s epoch under consideration, relative to lever press responses on low cost and high cost trials (gold and blue tick marks, respectively). Lower panels represent differential activity for each neuron, as in Fig. 3c. Overall activity was not significantly cue-selective for any epoch (all p’s > 0.1). (b) Differential activity plots for neurons activated prior to the response, after the response, prior to reward delivery, and following reward delivery in the delay-based task. Conventions follow from panel a, except that tick marks in top panels represent lever press responses on immediate and delayed reward trials. Again, activity was not significantly cue-selective for any epoch (all p’s > 0.5).

A subset of neurons remain excited during work and delay periods

Although response and reward-related activations did not directly reflect reward cost, we found that a number of neurons exhibited activity throughout the completion of response requirements on high cost trials or during the delay period on delayed reward trials (Fig. 6). In the effort-based task, 22 of 135 neurons (16.3%) were activated both following the initial response and prior to the reward delivery. In the delay-based task, 11 of 95 neurons (11.6%) were activated following the response completion on delayed reward trials and remained active until reward delivery. For each task, the overwhelming majority of these neurons (18 of 22 in the effort task and 9 of 11 in the delay task) also exhibited an increased firing rate immediately before response initiation. As shown in characteristic examples recorded during the effort (Fig. 6a, b) and delay (Fig. 6d,e) task, and population activity during the effort (Fig. 6c) and delay (Fig. 6f) tasks, these cells tended to maintain an increased firing rate until the reward was delivered. As a result, these neurons exhibited longer periods of activity on high cost and delayed reward trials than on low cost and immediate reward trials.

Response-related inhibitions are prolonged on high cost and delayed reward trials

Previous reports demonstrate that a subset of NAc neurons are inhibited during operant responses and reward delivery (Carelli et al., 1993; Day et al., 2006; Taha and Fields, 2006; Taha et al., 2007). Consistent with these results, a large number of neurons in both the effort and delay tasks exhibited decreases in activity during cue presentation, lever press responses, and reward delivery (see Table 1). Importantly, there were no differences in the degree of inhibition (minimum firing rates) or selectivity of these responses between high and low value trial types in either task (paired t-tests, p > 0.05 for all comparisons; Wilcoxon signed-rank test, p > 0.05 for all comparisons). However, in both tasks these responses were broadly tuned, responding during multiple events leading to reward delivery (Fig. 7). Thus, neurons that were inhibited during cue presentation tended to also exhibit decreased firing rate during the pre-response epoch. In the effort-based task, 30 of 44 neurons (68.2%) that displayed a decreased firing rate during the cue onset period also displayed a decreased firing rate immediately before the initial lever press response. Further, of the 58 neurons that were inhibited preceding the lever press response, 43 (74.1%) were also inhibited during reward delivery. In the delay-based task, 23 of 33 cue-inhibited neurons also exhibited an inhibited response preceding the lever press, and 26 of 35 (74.3%) press-inhibited neurons were also inhibited during reward delivery. Thus, in a mirror-image of the long-duration excitations shown in Fig. 6, these neurons displayed prolonged inhibitions that began prior to the lever press response and were maintained on high cost and delayed reward trials until the reward was delivered (example neurons shown in Fig. 7a, b & d,e). Population activity plots (Fig. 7c,f) revealed that as group, press-inhibited neurons remained significantly inhibited during the post-response and pre-reward periods on high cost and delayed reward trials (one-way repeated measures ANOVA, Dunnett’s multiple comparisons test vs. baseline, p < 0.01 for all comparisons), and returned to baseline following reward delivery.

Electrode placement

A total of 384 microelectrodes (16 per animal) were implanted bilaterally and aimed at the NAc. For recordings made during the effort-based decision task, 66 neurons were recorded from 49 wires located in the NAc core, and 69 neurons were recorded from 62 wires located in the NAc shell. For the delay task, 48 neurons were recorded from 39 wires located in the NAc core and 47 neurons were recorded from 38 wires located in the NAc shell. Across animals, electrode placements ranged from 0.84 - 2.96mm anterior to bregma, 0.6 - 2.05mm lateral to the midline, and 6.8 - 8.3mm ventral from the brain surface. The precise placement of marked electrode tips in the NAc are shown in Figure 8, and are separated by task. Neurons were equally distributed between hemispheres, with 69 right and 66 left hemisphere neurons in the effort task and 45 right and 50 left hemisphere neurons in the delay task. Data from electrodes located outside the NAc were excluded from analysis. There was no difference in the distribution of any response type between the core and shell of the NAc, nor were selective responses more concentrated in one region than another (Fisher’s exact test on response frequencies across region, p > 0.065 for all comparisons).

Figure 8.

Successive coronal diagrams illustrating anatomical distribution of electrode locations across core and shell of the NAc for the effort-based task (left) and delay-based task (right). Marked locations are limited to electrodes that contributed to data presented here. Filled circles indicate electrode location in the NAc core, open circles indicate electrode locations in the NAc shell. Numbers indicate anteroposterior coordinates rostral to bregma (in mm).

DISCUSSION

The NAc has been implicated in a wide range of reward-related functions, including responding to reward-paired incentive cues and decision making. Here, NAc activity was monitored during decision tasks in which animals demonstrated behavioral preferences for response options with lower effort and delay-related costs. NAc neurons exhibited phasic patterns of activity (both excitations and inhibitions) relative to all aspects of the task, including cue presentations, operant responses, and reward delivery. However, specific components of these responses were sensitive to the costs imposed on reward seeking. First, the population of cue-activated neurons in the effort task exhibited a skewed distribution towards greater activity on low cost trials than high cost trials, indicating that at least some neurons may encode the cost-discounted value of the available choice. Second, two classes of responses exhibited changes in activity that were maintained during the completion of response requirements in the effort task or as animals waited for rewards in the delay task. One class of neurons was activated during the response execution or delay periods. In contrast, another group of cells exhibited decreased firing rates leading up to the operant responses and maintained this activity until reward delivery. These response patterns reveal that the NAc encodes information about costs in three unique ways, and are consistent with the hypothesis that the NAc is involved in cost-based decision making or selection of appropriate actions after decision making processes have been engaged (Nowend et al., 2001; Salamone and Correa, 2002; Salamone et al., 2007; Hauber and Sommer, 2009).

Numerous electrophysiological investigations of NAc function indicate that a subset of NAc neurons are responsive to conditioned or discriminative stimuli (Ghitza et al., 2003; Ghitza et al., 2004; Nicola et al., 2004a; Roitman et al., 2005; Wilson and Bowman, 2005; Day et al., 2006; Wan and Peoples, 2006; Wheeler et al., 2008). This responsivity is determined by the relationship between such cues and the future rewards (Nicola et al., 2004a; Day et al., 2006), and encodes unique information about the motivational valence, identity, magnitude, and location of upcoming rewards (Hassani et al., 2001; Cromwell and Schultz, 2003; Setlow et al., 2003; Roitman et al., 2005; Taha et al., 2007). Here, cues signaled the opportunity to respond for an identical reward volume that came at different costs. As in previous studies, a large subset of NAc neurons recorded during the effort-based decision task was activated by the presentation of discriminative cues. Importantly, as a population, these neurons exhibited larger magnitude excitations on low cost trials than on high cost trials, even before the animal selected a response option. Such activity is consistent with the idea that NAc cue responses are modulated by the subjective value of future choices (Hassani et al., 2001; Cromwell and Schultz, 2003; Cromwell et al., 2005; Samejima et al., 2005; Wilson and Bowman, 2005; Kable and Glimcher, 2007; Roesch et al., 2009). These results are also in agreement with a recent fMRI report showing that cue-related activity in the human ventral striatum is modulated by the net value (benefit minus cost) of future actions (Croxson et al., 2009).

A potential caveat to the interpretation of these results is that a substantial portion of NAc neurons also encode the direction of future reward-related movements (Taha et al., 2007). In the present study, the available choices were always associated with one response lever in order to maintain performance levels during training and allow for a clear distinction in choice allocation to develop. Nevertheless, it is possible that direction-selective and cost-selective neurons overlap (Roesch et al., 2009), and that some of the neurons which are reported here as “cost selective” are simply “direction selective”. However, there is no population difference in the degree of direction-selectivity (i.e., the number of neurons that prefer a right response over a left response is not significantly greater than the number that prefer a left response over a right response) (Taha et al., 2007). In contrast, the present results reveal that there is a population bias towards low-cost choices in the effort-based decision task, indicating that these cue-related activations in this task encode more than response direction alone. This is also in agreement with a recent study which found that response direction and value were integrated into the cue-related activity among a population of ventral striatum neurons (Roesch et al., 2009).

Cue-selective responses among NAc neurons may potentially encode multiple representations, including the value associated with reward-paired cues or the action-specific value (Samejima et al., 2005; Roesch et al., 2009). Here, we examined the activity of low cost selective neurons on free choice trials to distinguish between these possibilities. On free choice trials, animals were presented with both low and high cost cues and could choose either course of action. This allowed us to dissociate action and stimulus value on the basis of the animal’s decision. For example, if low cost selective neurons exhibited similar changes in cue-evoked activity irrespective of the subsequent response, this would indicate that cue-evoked responses were signaling the value of the stimulus. In contrast, if the activity of these neurons differed on the basis of the future choice, this would suggest that cue-evoked responses were encoding the value of the action. The results of this analysis were consistent with the second alternative. Thus, when the low cost option was selected, cue-evoked increases in firing rate were greater than when the high cost option was selected. These findings therefore may reveal that at least some cue-evoked responses reflect the value of the future choice rather than the value associated with the reward-paired cues.

As noted in the introduction, lesions or pharmacological manipulations in the NAc produce a behavioral bias towards low-cost response options, even if those options lead to inferior rewards (Salamone et al., 1994; Hauber and Sommer, 2009). Given that cue-related responses in the NAc appear to preferentially encode low-cost cues, it is perhaps surprising that NAc manipulations would produce a bias for these options rather than against these options. However, it is worth noting that in the studies which manipulate NAc function via lesions or dopamine antagonism, animals typically receive larger rewards for selecting high-cost response options than low-cost options. Therefore, NAc manipulations could bias animals towards low-cost choices in a number of ways, including reduced reward sensitivity and/or diminished motivation to overcome particularly high costs. In the present experiments, animals received the same reward on low and high cost trials, which may have contributed to the enhanced cue-related signals on low-cost trials. Additionally, lesions or other NAc manipulations are likely to disrupt other types of activity in the NAc, such as the prolonged neuronal responses that occur while animals are working for or waiting for rewards. In fact, it may be this activity (rather than cue-related activity) that is necessary to overcome larger costs. If so, disruption of this activity by NAc manipulations would have the observed effect of biasing animals towards response options that come with smaller costs.

Interestingly, we found no differences in the overall activity of cue responsive neurons on immediate and delayed reward trials in the delay-based task. In contrast, a recent study revealed that neurons in the ventral striatum do encode differences in reward delay associated with specific actions (Roesch et al., 2009). The precise reasons for this difference are unclear. However, one potential explanation for these discrepant results is that the cues used here signaled different reward delays from the time of the lever press rather than from the time of cue onset. Thus, whereas the immediate reward cue was at least 5s removed from the reward, the delayed reward cue preceded reward delivery by at least 9-13s (for 4 and 8s delays, respectively). Therefore, both cues effectively signaled delayed rewards, although one was more delayed than another. In contrast, the previous study used cues that predicted reward availability at much shorter delays from cue onset (less than 1s for immediate rewards). Since temporal discounting of rewards follows a hyperbolic function (Green and Myerson, 2004; Kable and Glimcher, 2007), it is possible that the subjective differences between the temporal delays employed in the present study were simply not sufficient to induce substantial delay encoding among NAc neurons.

Neuronal activity in the NAc is also critical for overcoming large costs or long delays to obtain rewards (Cardinal et al., 2001; Bezzina et al., 2008; Hauber and Sommer, 2009). The present study found that two different response patterns reflected the cost required on each trial type. The first consisted of neurons that became excited during the period prior to responding and remained activated until requirements were complete. On low cost trials and immediate reward trials, this resulted in a relatively short duration of activity. However, on high cost and delayed reward trials, the same neurons remained active until reward delivery, even when animals were working or waiting for rewards. Such responses may have multiple behavioral functions, including the anticipation or performance of specific responses or the expectation that a given action sequence will be rewarded (Pennartz et al., 1994; Hollerman et al., 1998; Cromwell and Schultz, 2003; Nicola, 2007; Taha et al., 2007). Deficits in such processing, induced by manipulations in the NAc, may therefore lead to an impaired ability to maintain a representation of action values over time and across large workloads, making animals more likely to choose smaller rewards that come at lesser costs.

Neverthless, it is also possible that NAc function, and in particular dopamine release within the NAc, is less important for decisions involving reward delay than those involving reward effort. Although phasic NAc dopamine release encodes both predicted reward delay and predicted response effort, these signals are dissociable, yielding larger differences in dopamine signaling in the effort tasks than similar delay tasks with no difference in effort requirements (Day et al., 2010). Likewise, other reports indicate that dopamine transmission in the NAc is not critical for performance on delay discounting tasks or progressive interval schedules of reinforcement (Wakabayashi et al., 2004; Winstanley et al., 2005), but is required for normal performance on delay tasks that also involve an effort component (Mingote et al., 2005). In fact, even when the temporal delay between behavioral responses and reward delivery is controlled for in effort-based choice tasks, dopamine antagonism still biases animals away from high-effort choices (Floresco et al., 2008). Thus, unlike neuronal activity in the NAc, dopamine release does not appear to be especially relevant for decisions between rewards with different temporal delays.

A second group of NAc neurons exhibited inhibitions preceding responses which were maintained until reward delivery. Again, this led to relatively shorter duration inhibitions on low cost or immediate reward trials than on high cost or delayed reward trials. Previous studies have also reported inhibitions among a subset of NAc neurons during goal-directed behavior (Taha and Fields, 2006). Similar to the present results, that study found that such inhibitions typically preceded the onset of reward-seeking behavior and continued through reward consumption. Considering the cellular composition and circuitry of the NAc, these types of responses are proposed to have a role in permissively ‘gating’ actions that lead to rewards, irrespective of the specific action (Roitman et al., 2005; Taha and Fields, 2006; Taha et al., 2007; Krause et al., 2010). This type of activity may play an integral role in keeping motor systems engaged and ready for reward delivery across delays or high effort demands, instead of allowing the animal to become disengaged. As such, these results may help explain the deficits in both effort and delay-based decision making induced by NAc lesions (Cardinal et al., 2001; Bezzina et al., 2007; Hauber and Sommer, 2009).

Individual NAc neurons receive diverse cortical and subcortical inputs, and can carry a heavy information processing load (Kincaid et al., 1998; Zahm, 1999). A number structures that project to the NAc, including the anterior cingulate cortex (ACC), orbitofrontal cortex (OFC), and basolateral amygdala (BLA), are known to process reward-related information (Critchley and Rolls, 1996; Watanabe, 1996; Behrens et al., 2007; Belova et al., 2007; Doya, 2008; Tye et al., 2008). These inputs are critical regulators of decisions that involve different costs. Thus, the ACC (but not OFC) is required for optimal performance on decision tasks that involve effort-related choices, whereas, the OFC appears to play an especially critical role in guiding delay-related decisions (Kheramin et al., 2002; Cardinal, 2006; Rudebeck et al., 2006; Rushworth and Behrens, 2008). Finally, dopamine release in the NAc also encodes the value of future choices (Day et al., 2010; Gan et al., 2010), and is critical for cost-related decisions (Salamone and Correa, 2002). Understanding how this circuit interacts in the context of decision making will therefore provide new insights into the neural basis of valuation, as well as disorders characterized by abberant valuation processes such as addiction and impulsivity.