Nucleus accumbens neurons track behavioral preferences and reward outcomes during risky decision making

Abstract

Background

In order to make appropriate decisions, organisms must evaluate the risks and benefits of action selection. The nucleus accumbens (NAc) has been shown to be critical for this processing, and is necessary for appropriate risk-based decision making behavior. However, it is not clear how NAc neurons encode this information to promote appropriate behavioral responding.

Methods

Here, rats (n=17) were trained to perform a risky decision making task in which discrete visual cues predicted the availability to respond for a smaller certain (safer) or larger uncertain (riskier) reward. Electrophysiological recordings were made in the NAc core and shell to evaluate neural activity during task performance.

Results

At test, animals exhibited individual differences in risk-taking behavior; some displayed a preference for the risky option, some the safe option, and some did not have a preference. Electrophysiological analysis indicated that NAc neurons differentially encoded information related to risk versus safe outcomes. Further, during free choice trials, neural activity during reward-predictive cues reflected individual behavioral preferences. In addition, neural encoding of reward outcomes was correlated with risk taking behavior, with safe-preferring and risk-preferring rats showing differential activity in the NAc core and shell during reward omissions.

Conclusions

Consistent with previously demonstrated alterations in prospective reward value with effort and delay, NAc neurons encode information during reward-predictive cues and outcomes in a risk task that tracked the rats’ preferred responses. This processing appears to contribute to subjective encoding of anticipated outcomes, and thus may function to bias future risk-based decisions.

Introduction

Selecting appropriate behaviors to secure the necessary resources for survival requires a complex evaluation of the potential risks and benefits of different actions (1-5). Impairments in appropriate cost-benefit decision making is associated with several psychiatric disorders including drug and gambling addiction (6-10), and more complex disorders such as schizophrenia (7). As such, there is a growing interest in understanding how the brain encodes normal decision making, and how changes in this signaling may result in disordered decision making.

The nucleus accumbens (NAc) is part of a neural circuit that is essential for assessing the costs and benefits of appropriate behavioral choices. Decision making under conditions of uncertainty has been modeled in humans and animals by using modified gambling paradigms where organisms choose between smaller certain rewards (safe option) and larger more uncertain rewards (risky option). Similar to humans, animals evaluate both the size of the reward and the probability of delivery when making appropriate decisions, and decrease responding for larger rewards as the probability decreases (2, 11-19). Disruptions of NAc circuitry result in specific dysfunctions in risky decision making. For example, NAc-lesioned rats were more averse to taking risks, choosing smaller certain reinforcers more than controls and even avoiding riskier options when they were more advantageous (11). Further, inactivation of the NAc biased animals away from larger rewards particularly when they were more uncertain (17). These observations suggest that NAc activity is critical for appropriately evaluating risks and making appropriate choices, and aberrations in this circuitry result in dysfunctional behaviors.

Previous research examined how NAc neurons encode explicit reward value based on external factors such as reward size or the effort required obtaining it (20, 21). However, many decisions involve subjective evaluations of reward value based on individual factors such as risk tolerance and sensitivity to reward omission. Indeed, there is evidence that this type of subjective value is encoded in the human ventral striatum (22). Further, studies indicate the NAc is critical for subjective decision making and is linked to impulsivity, risk taking behavior, and drug addiction (11, 23-25). Here, we incorporated electrophysiological recording methods to examine how NAc neurons encode risk-taking behavior, and if NAc neural activity is related to risk predictive cues, choices, behavioral responses, outcomes, or individual risk attitudes.

Materials and Methods

Detailed methods are described in Supplemental Methods. Briefly, male Sprague Dawley rats (n=17) were implanted with microelectrode recording arrays into the NAc core and shell (20, 26-30). Histological verification of electrode placement is shown in Figure S1. Following recovery, rats were trained on a risk-based decision making task developed in our laboratory (19) and illustrated in Figure 1A. Briefly, on Forced Choice Risk trials (left) a cue light was presented for 5s followed by the extension of two response levers. A single lever press on the associated lever positioned below the illuminated cue light led to a 2 sucrose pellets presented on 50% of lever presses. Responding on the other lever did not produce reward and terminated the trial. On Forced Choice Safe trials (middle) the other cue light was presented for 5s before lever extension. Here, presses on the associated lever resulted in 1 sucrose pellet; presses on the other lever did not produce reward and terminated the trial. On Free Choice trials (right) both cue lights were presented for 5s and animals could select either option. At test, NAc cell firing was recorded using established procedures (20, 30) during the task in well trained rats.

Figure 1.

Risky decision making task and behavior. (A) Schematic representation of risky decision making task. See main text and Supplementary Materials for details. (B) Percentage of errors on Risk and Safe trials were not significantly different from 0 (P>0.1 for all comparisons), demonstrating behavioral discriminations between cues. (C) Reward pellets received during Risk and Safe trials. All groups of animals received maximum amounts of rewards (30 pellets for each trial type). (D) Response allocation on choice trials (as a percentage of choice) during recording sessions. Three groups of rats were observed: non-preferring rats chose both options equally (P=0.516), risk preferring rats chose the risk lever significantly more than chance (P<0.0001), safe preferring rats chose the safe lever significantly more than chance (P=0.0007).

Analysis of neural activity had several goals. First, we identified task-related neurons as those that exhibited either increases (excitations) or decreases (inhibitions) in firing rate relative to specific behavioral events (e.g., lever press, cues). Second, we evaluated if the response patterns of each neuron was sensitive to differences in risk versus safe options. Further, we examined how populations of NAc neurons encoded each of these options. Details on analysis procedures are described in the supplementary materials. Statistical and graphical analyses were conducted in Graphpad Prism 4 (Graphpad software, Inc.) and STATISTICA (StatSoft, Tulsa, OK).

Results

Individual Differences in Risky Decision Making Behavior

Several behavioral measures were used to verify task acquisition. Animals showed a significant decrease in the percentage of errors on risk and safe trials (F_(24,384)=8.08, P<0.0001), with a significant reduction in errors by session 5 versus session 1 (Tukey's HSD test, P<0.05 for sessions 5-25). During the recording session, the number of errors was not significantly different from 0 for any of the groups or trial types (Figure 1B). Further, across groups there was were no significant differences in the number of pellets received for both cues (F_(5,48)=0.68, P=0.643), and the number of earned pellets was not significantly different from the maximum possible, 30 (one sample t-test, comparison with a theoretical mean of 30, P>0.15 for all comparisons; Figure 1C).

On choice trials, both cues signaled that both levers were active, and the animal was rewarded based on the contingency of the lever chosen. During session 25 the majority of rats (n=9) displayed a clear preference for the risk or safe lever (Fig. 1D; defined as 60% or greater presses on the preferred lever). Rats that did not show a preference on session 25 (n=8; t(7)=0.68, P=0.52, comparison to a theoretical mean of 50%) were grouped together as the non-preferring group. These rats were then given additional training sessions until a behavioral preference developed (1-8 more sessions). Once preferences developed, of all rats, 59% displayed a risk preference (t(11)=9.30, P<0.001, compared to chance) while the remaining rats showed a safe preference (t(6)=6.37, P<0.001, compared to chance).

NAc activity during risky decision making

339 NAc neurons were recorded from 17 rats during the task from non-preferring rats (118 total; 44 cells in the core, 74 cells in the shell), risk preferring rats (142 total; 46 core, 96 shell) and safe preferring rats (79 total; 43 core, 36 shell) (Figure S1). Of these, 286 cells (84%) exhibited significant changes characterized by increases (excitations) or decreases (inhibitions) in firing rate relative to at least one task event (Figure 2A). Specifically, 130 neurons (38%) showed significant changes in activity during cue presentations; an example of a representative neuron showing an increase in firing to the cue is shown in Figure 2B (top left). Other cells (225 neurons, 66%) exhibited phasic increases or decreases in firing rate during the prepress period. An example of a neuron that exhibited an increase in firing during the prepress period is illustrated Figure 2B (middle right). Finally, 244 cells (72%) exhibited either increases or decreases in firing rate during the early or late reward periods. A representative neuron displaying a decrease in firing immediately following the lever press during the early reward period is shown in Figure 2B (bottom, right). The distribution of NAc activity by trial type (risk versus safe) is shown in Table 1. The distribution of cells relative to large reward, small reward and omission trials is in Table 2.

Figure 2.

NAc neurons display patterned discharges during distinct components of the task. (A) Schematic representation of epochs used to analyze and classify neural data. Neural activity during each time epoch was compared to a 10 s baseline period. Neurons that displayed significant increases or decreases in firing rate 2.5 s following cue onset were classified as cue-related cells. Neurons exhibiting significant increases or decreases in firing rate during 2.5 s prior to the lever press were classified as prepress cells. Neurons showing increases or decreases in activity during the initial 2.5 s period following the lever press (denoted by ▲) were classified as the reward early cells while neurons showing patterned discharges during the next 2.5s period were classified as reward late cells. (B) Peri-event histogram (PEH) and raster plots of representative NAc neurons. Each cell is aligned to cue onset (left panel) and lever press (denoted by ▲; right panel). Cell A (Top) is a cue-related cell showing increased activity immediately after cue onset. Cell B (Middle) is an example of a prepress cell, showing increased activity prior to the lever press. Cell C (Bottom) shows a neuron that exhibited a decrease (inhibition) in firing rate following the lever press during the reward period.

Table 1.

Distribution of NAc Cellular Activity By Trial Type

Total Cells (n = 339)	Event
	Cue	Prepress
Risk Trial	98/339 (29%)	142/339 (42%)
Excitation	38/98 (39%)	43/142(30%)
Inhibition	60/98 (61%)	99/142 (70%)
Safe Trial	96/339 (28%)	138/339 (41%)
Excitation	29/96 (30%)	37/138 (27%)
Inhibition	67/96 (70%)	101/138 (73%)

Table 2.

Distribution of NAc Cellular Activity During Reward Period

Total Cells (n = 339)	Reward
	Early	Late
Large Reward	99/339 (29%)	111/339 (33%)
Excitation	40/99 (40%)	39/111 (36%)
Inhibition	59/99 (60%)	72/111 (65%)
Small Reward	124/339 (37%)	128/339 (38%)
Excitation	47/124 (38%)	53/128 (41%)
Inhibition	77/124 (62%)	75/128 (59%)
Omission	80/339 (24%)	107/339 (32%)
Excitation	39/80 (49%)	74/107 (69%) *
Inhibition	41/80 (51%)	33/107 (31%) *

^*signifies p<0.05 compared to early omission period

Cue-modulated neurons selectively encode risk versus safe options and signal behavioral preferences

We examined how NAc cells encode information about risk versus safe options during forced trials, and if this encoding was related to each animals’ individual behavioral preference. No differences were observed in the type or distribution of responses between the core and shell during the cue period (Fisher's exact test P>0.05 for all comparisons), thus NAc core and shell data are presented together. Of the neurons that displayed phasic changes in firing rate during risk trials, 39% showed increases (excitations) while 61% showed decreases (inhibitions) in activity during the cue period. During safe trials, 30% of cells showed excitatory activity while 70% showed inhibitions in firing during cue presentation (see Table 1). However, no significant differences were observed in the percentages of cells that displayed phasic activity across groups, such that 36% of cells responded during cue presentations in the risk preferring group, 39% in the safe preferring group and 44% in the non preferring group (χ²(2)=1.44, P=0.49).

A substantial proportion of cue-modulated neurons during forced trials exhibited either cue-selective excitations (e.g., greater activity relative to baseline for one cue compared to the other) or cue-selective inhibitions (e.g., lower firing rate relative to baseline for one cue compared to the other). We classified these responses into three separate types: “risk selective”, “safe selective”, and “both risk/safe” (e.g., firing rates that were not significantly different from each other but were different from baseline). Examples of cue-selective excitations are shown for representative neurons in Figure 3A. The majority of neurons (78 of 133 cells, 59%) displayed cue-selective encoding (i.e., risk or safe selective); the population response for all NAc cue-modulated neurons (i.e. exhibiting either excitatory or inhibitory activity) is shown in Figure 3B. There were no significant differences in the percentages of neurons that were risk selective, safe selective, or both risk/safe across groups (χ²(4)=2.65, P=0.618) (Figure 3B). Further, there was no difference in the percentage of cells that were risk versus safe selective within each group (P>0.05, Fisher's exact test for all comparisons).

Figure 3.

NAc neurons display cue-selective encoding during safe versus risk forced trials. (A) PEHs of representative risk selective (left panel), safe selective (middle panel) and both (risk/safe) (right panel) cue activated NAc neurons during the risky decision making task. Data are aligned to cue onset (black bar/dashed line). Grey bar signifies 2.5s cue effect period. Time bins in which neural activity is significantly increased is signified by * (P<0.05). (B) Percentages of phasic cells that display the three different types of encoding in the risk preferring (left), safe preferring (middle), and non-preferring (right) rats. There were no differences in the population encoding of cue selectivity across the three groups of rats (P=0.653).

The previous analysis only evaluated forced trial situations in which the animal did not have to “make a decision” as there was only one response that would be reinforced. To better understand if the strength of encoding in the NAc was related to behavioral preferences, neural encoding for stimuli in the animals’ preferred option (i.e., risk-encoding cells in risk-preferring rats and safe-encoding cells in safe-preferring rats) was analyzed during forced and free choice trials. In the core, during forced trials, neurons showed nearly similar responses for both excitatory (Figure 4A) and inhibitory (Figure 4D) activity. In contrast, when rats were given the opportunity to select their preferred option during the free choice trials, significant changes in core encoding were seen. Specifically, there appeared to be greater excitations (Figure 4B) and inhibitions (Figure 4E) for the choice cue when the animal subsequently chose their preferred option compared to when they chose their less-preferred option. To quantify this, we compared the greatest change in firing rate to baseline (i.e., “peak” firing) for the preferred and non-preferred cue during forced and choice trials. For core forced trial excitations (Figure 4C), there were no differences between the preferred and non-preferred cue (Wilcoxon: W=−3, n.s.), though there was a significant difference in encoding during choice cues when the animal chose the preferred compared to the non-preferred option, (Wilcoxon: W=21, P=0.05). This pattern was also seen during inhibitions in the core (Figure 4F). A significant preference X trial type interaction, F_(1,38)=13.70, P<0.001, was due to changes between the preferred and non-preferred option chosen on the choice trial (P<0.0005), but no differences were seen between the preferred and non-preferred cues on forced trials (P=0.99). Further, changes in firing for the non-preferred choice cue was significantly less extreme than either of the forced cues (P<0.0005 versus force preferred; p<0.0005 versus force non-preferred).

Figure 4.

Averaged population responses for excitatory (A-C) and inhibitory (D-F) cue-related neural activity in the NAc core. Cue onset is indicated by “C” on x-axis; duration is indicated by shaded area; “L” indicates lever extension. Firing rate during the preferred cue was greater than that during the non-preferred cue during choice trials (B) compared to forced trials (A). Further, peak firing was significantly higher during the choice cue when the rat subsequently chose its preferred option than when it chose its less preferred option (C). For inhibitions, there were no differences in firing during the forced cues (D) but there was differential firing during the choice cue based on the rats’ subsequent behavioral choice (E). Peak inhibitions during the cue were not different during the force trials, but firing a larger inhibition was evident during the choice cue when rats chose their preferred option (F). ** Tukey, Preferred versus Non-Preferred, P<0.001; ‡ Wilcoxon, P=0.05.

In the shell, a different pattern of encoding emerged. On average, excitations for the preferred cue were slightly greater than for the non-preferred cue during both forced (Figure 5A) and choice (Figure 5B) trials, and a similar pattern of bias for the preferred cues was seen for inhibitions (Figures 5D,E). For analysis of peak cue excitations, there was no interaction between preference and trial type, F_(1,23)=0.02, P=0.89, though there was a trend towards a main effect of preference, F_(1,23)=3.43, P=0.07 such that firing was higher for the preferred compared to the non-preferred option (Figure 5C). This pattern of biased encoding for the preferred option, regardless of choice, was seen more dramatically during inhibitions (Figure 5F). Here there was no interaction of preference X trial type, F_(1,37)=0.54, P=0.47, but there was a strong main effect of preference, F_(1,37)=25.77, P<0.0001. Post-hoc comparisons indicated that there was a significant difference between the preferred and non-preferred cues during both forced trials (P<0.02) and free choice trials (P<0.002).

Figure 5.

Averaged population responses for excitatory (A-C) and inhibitory (D-F) cue-related neurons in the shell. Conventions follow from Figure 4. # trend main effect of Preference, P=0.07; *P<0.02, **P<0.002, Tukey, Preferred versus Non-Preferred.

Finally, we examined the population of neurons recruited to encode forced and free choice trials. Neurons that encoded information about the preferred cue during forced trials but not choice trials were “Forced Only” and, likewise, for choice trials (“Choice Only”). Cells that encoded information about the preferred cue in both forced and choice trials were “Both Forced/Choice”. Cells that failed to encode either were considered nonphasic. During cue presentations, we found 35/89 (39%) core cells and 39/132 (30%) of shell cells encoded some feature of forced or choice activity for the cues. Of these, the near-majority of phasic cells encoded information about both the forced and choice cues (17/35 [49%] in the core; 25/39 [64%] in the shell), while a smaller proportion of cells encoded Forced Only information (3/35 [9%] in core; 9/39 [23%] in shell). However, there were a greater proportion of cells in the core that encoded the Choice Only (15/35 [42%]) compared to the shell (5/39 [13%]), χ²(2)=9.33, P<0.01 (Figure 6). Indeed, there were no differences in cue encoding between core and shell for the Both Forced/Choice, χ²=1.23, P=0.26, or for the Forced Only, χ²=1.89, p=0.17, but there was a difference in the percentage of cells encoding exclusively Choice Only information, χ²=6.89, P<0.01).

Figure 6.

The percentage of cells in the core (left) and shell (right) exhibiting nonphasic activity or patterned (phasic) cell firing during the preferred cues on force versus choice trials. There were similar percentages of cells in the core and shell that were phasic during only the force trials or during both choice and force trials. However, a significantly greater percentage of cue-related neurons were observed during choice trials in the core than in the shell.

Response-modulated neurons selectively encode risk versus safe options

Previous research reported that the NAc is critical for appropriate action selection (1, 3, 11, 17, 31). Here, we examined NAc activity during the 2.5s prior to lever press during forced risk and safe trials. A significantly greater proportion of NAc cells were phasic during the prepress period compared to the cue period (180 cells 53% during the prepress period;133 cells 39% during the cue period); χ²=12.558, P<0.001), supporting the role of the NAc in the processing of action selection. During the prepress period, roughly 30% of responsive cells displayed phasic excitations while 70% displayed phasic inhibitions for each cue type (Table 1). Further, there were no significant differences between the three groups in the number of cells that were phasically modulated during the prepress period (49% in the risk preferring group, 54% in the safe preferring group 57% in the non-preferring group; χ²(2)=1.52, P=0..468), suggesting that the three groups are not differentially encoding response behaviors.

A substantial proportion of neurons that were modulated during the response period of forced trials exhibited either selective excitations or inhibitions. A subset of cells showed selective changes in activity prior to risk lever presses, another subset prior to safe lever presses, and a third group were similarly modulated prior to both risk and safe presses. Examples of response-selective activity are shown for representative neurons in Figure 7A. There were no differences in the distribution of responses between the core and shell during the prepress period, nor were selective responses more concentrated in one region than another so NAc core and shell data are presented together (Fisher's exact test P>0.05 for all comparisons). Of prepress modulated cells, the majority (101 cells, 56%) displayed prepress selective encoding (i.e., phasic activity specific to risk or safe forced trials). While this encoding suggests an important involvment of the NAc in action selection, there were no significant differences during forced trials in the percentages of cells that were risk selective, safe selective or both risk/safe in the three groups (Figure 7B, χ²(4)=0.93, P=0.920). Similar percentages of cells were risk selective versus safe selective within each group of rats suggesting that the NAc does not differentially encode preferred action selection based on the percentage of cells recruited during responses.

Figure 7.

NAc neurons display prepress selective encoding for safe versus risk cues. (A) PEHs of representative risk selective (left), safe selective (middle) and both (risk/safe, right) prepress modulated NAc neurons during the task. Data are aligned to lever press (▲/dashed line). Black bar signifies cue period; grey bar signifies 2.5 s prepress period. Time bins in which neural activity is significantly different during the effect period is signified by * (P<0.05). (B) Percentages of phasic cells that display the three different types of encoding in the risk preferring (left), safe preferring (middle), and non-preferring (right) rats. There were no differences in the population encoding of prepress selectivity across the three groups of rats (P=0.864).

We next evaluated if population encoding during the prepress period reflected information about subsequent response preferences. For prepress excitations in the NAc core, an increase was observed in excitatory responses for the preferred option during free choice trials, but not forced trials (Figure S2A-C). Unexpectedly, there was a slightly greater inhibition for the non-preferred trial compared to the preferred option during forced trials (Figure S2D) though during choice trials signaling was more inhibitory for the preferred option (Figure S2E-F). In the shell, we found a similar pattern of neural activity as during cue presentations, with greater excitations observed for the preferred option only during choice trials (Figure S3A-C). Similarly, neurons exhibited greater inhibitions to the preferred option during both forced and choice prepress periods (Figure S3D, E, F). There were no differences in the percentages of cells that encoded forced versus choice trials in the NAc core or shell during the prepress period (Figure S4, P>0.05 for all comparisons).

NAc neurons encode unexpected reward deliveries and omissions

For all neurons (regardless of preference), we first compared how the NAc encodes unexpected reward presentations and omissions during the early (0-2.5s following press) versus late (2.5-5s following press) reward periods. A similar percentage of neurons (roughly 30%) were responsive (exhibited phasic excitations or inhibitions in activity) during large reward presentations, small reward presentations, and reward omissions, respectively (Table 2). Further, a larger percentage of cells displayed inhibitions compared to excitations during reward presentations. During omissions, particularly during the late reward period, more excitations compared to inhibitions were observed. There was also a significant increase in the percentage of cells that encoded reward omissions from the early to late period (80 neurons (24%) to 107 neurons (33%), χ²=4.99, P=0.03). Further, there was a significant shift in the ratio of cells that displayed excitations versus inhibitions across the early and late reward periods. Specifically, excitations increased to 69% in the late phase from 49% in the early phase, while inhibitions decreased to 31% in the late phase from 51% in the early phase (χ²=7.14, P=0.008). The difference in the ratio of excitations versus inhibitions for reward versus omission trials suggests that the NAc is able to differentially encode reward outcomes.

Neural responses to reward omissions and deliveries encode risk preferences

Here, we evaluated the relationship between cell firing in the core and shell during reward omissions and how this activity was correlated with behavioral preferences. There were no significant differences in the percentages of cells that were phasically responsive during the omission period in either the core (χ²(2)=0.69, P=0.70) or shell (χ²(2)=1.07, P=0.59) across groups. Cells exhibited either an increase in activity during the reward period (excitation, Figure 8 left) or a decrease in activity during this period (inhibition, Figure 8 right).

Figure 8.

Reward omission processing in the NAc. PEH and raster plot from two representative neurons showing either increased firing (excitation, left) or decreased firing (inhibition, right) following the lever press. Lever press indicated by ▲ at dashed line. Grey circles in rasters denote cue onset.

However, a relationship was observed between the type of phasic activity in response to reward omissions and risk preferences. In the core, there was a significant difference in the ratio of excitations versus inhibitions between the groups (P=0.016, Fisher's exact test), with significantly greater proportions of excitations in the safe-preferring compared to the risk-preferring group (P<0.05, Fisher's exact test; Figure 9A left). We then analyzed each animal individually to determine the excitation/inhibition ratio in relation to risk preference. A negative correlation exists between the percentage of excitatory cells encoding omissions and preference for the large risky lever (r²=0.32, P=0.02; Figure 9B), suggesting that the NAc core differentially encodes reward omissions based on risk preference.

Figure 9.

Individual risk preferences related to reward omission processing. (A) Percentages of NAc core cells that exhibited excitations versus inhibitions during the reward omission period in each of the three groups. There was no significant difference in the percentage of cells that were phasic during reward omissions; however, safe preferring rats had a significantly greater percentage of cells that exhibited excitations versus inhibitions compared to risk preferring animals. (B) Percentage of NAc core phasic cells that displayed excitations plotted against individual risk preferences. The x-axis is a lever press ratio that indicates the risk preference of each animal during free choice trials. The ratio was determined by dividing the number of presses on the risk lever during free choice trials by the total number of presses on free choice trials (Risk/(Risk+Safe)). A ratio greater than 0.6 indicates that an animal is risk-prone while a ratio of less than 0.4 indicates an animal is risk-averse. Area in between the dotted lines indicates no preference. The y-axis is the percentage of phasic cells that showed significant excitations during the reward omission period. Only animals with at least 5 cells recorded in each region were included in the analysis as to not bias results due to low cell counts (C) Reward omission processing in the NAc shell. There was no significant difference in the amount of cells that were phasic during reward omissions; however, risk preferring rats showed a trend towards a lower percentage of cells that exhibited inhibitions versus excitations compared to safe preferring rats. (D) Percentage of phasic cells that display excitations in the NAc shell plotted against individual risk preferences. Conventions follow from B. *P<0.05, #P<0.1.

The opposite relationship was observed in the NAc shell. The safe preferring group had a trend toward fewer excitations compared to risk preferring rats (P=0.08, Fisher's exact test, Figure 9C). When the percentage of excitations were compared with each animals risk preference, a trend was observed towards a decrease in the percentage of excitatory cells and greater risk preferences in the NAc shell (r²=0.20, P=0.063; Figure 9D). Taken together, the ratio of excitatory versus inhibitory firing by core and shell neurons may reflect differential encoding of reward omissions based on risk preference across NAc subregions.

Discussion

Here, we examined NAc neural encoding associated with risky decision-making behavior. NAc neurons exhibited phasic patterns of activity relative to behavioral aspects of the task, which were additionally sensitive to trial type (i.e., risk versus safe options). During free choice trials, populations of cue-modulated neurons exhibited pronounced responses towards the preferred option, while NAc activity during reward evaluation periods reflected individual risk preferences. These observations are consistent with reports that the NAc is critical for risky decision making (1, 5).

NAc neurons are responsive to reward-related stimuli (26, 30, 32-35). Here, a large subset of NAc neurons displayed phasic responses to presentation of cues that signaled safe versus risk options. Surprisingly, during forced choice trials, there were no differences in the percentages of neurons that were risk- versus safe-selective across groups. Further, in the NAc core, the activity of phasic neurons during forced cue presentation was similar for preferred versus non-preferred options. However, during forced trials only one option was linked to reward; therefore it may not be necessary to differentially encode preferred versus non-preferred options in this situation.

Neural activity was examined during free choice trials to determine how the NAc encodes subjective value (perceived preference) as animals actively make decisions. Cue-related activity was significantly stronger (greater excitations, more pronounced inhibitions) for the preferred versus non-preferred option. Importantly, this differential encoding occurred prior to the time in which the animal completed a response. This suggests that subjective value encoding during cue presentation may be critical for guiding appropriate responding as animals are actively making decisions.

The NAc core and shell are differentially involved in aspects of decision making. Previously, we demonstrated that dopamine release in the NAc core but not shell tracks reward value based on effort, delay, and risk (19, 36). Others have shown that disruption of NAc core biases animals towards choosing smaller rewards when they are better predicted, immediate (11, 31, 37) or less effortful to obtain (38) even when these are less advantageous, although inactivation of the NAc shell largely affect risky choice (17). The present findings are consistent with a role of the NAc in encoding relative outcome evaluations necessary for guiding decisions. For example, rats that fail to differentiate the relative values of anticipated small and large rewards may be biased towards choosing safer (impulsive) options, while the negative aspects of delays and uncertainty are given undue weight. In contrast, forced choices are easier, as even poor estimations of value should support selection of the rewarded option. NAc core neural activity tracks this pattern; when choices are simple (forced trials), NAc core activity is similar for rewarded options, but when animals must weigh the cost-benefit of value and risk, core neural activity differentially scales to the animals’ preferred option. In contrast, shell neurons differentiated between the subjective value of the different outcomes regardless of choice or forced trials, suggesting a more general role for value encoding rather than differential recruitment during choice behavior.

NAc activity also encodes information about reward (26, 39-41), and is critical for appropriate responses to reward omissions (42, 43). Here, reward omission differentially elicited excitatory phasic responses, proportions of which were modulated by NAc region and risk preference. Specifically, in both the NAc core and shell there was a reliable correlation between excitatory encoding and risk aversion. Previous research has shown that reward omissions are aversive. For example, cues predictive of reward omission suppress appetitive behaviors in a similar manner as cues paired with electric shock (44). As such, the degree to which animals perceive reward omissions as aversive may influence the degree to which they develop a bias in subsequent choice behavior.

The NAc is embedded in a larger circuit linked with risk-taking behavior. The frontal cortex is critical for mediating appropriate impulse control and risk-taking behavior. Specifically, the orbitofrontal cortex (OFC) encodes aspects of risk or value, while the posterior cingulate cortex encodes information related to the subjective value of available options when animals are engaged in risk-based decision making (45, 46). Further, the OFC is critical for encoding information about the salience of cue-outcome associations (47) as well as decision confidence (48). Importantly, both the salience and confidence of decisions are related to the value of future options. As the OFC sends a direct projection to the NAc (49, 50), OFC activity may contribute to the subjective value signaling observed here. Importantly, understanding how the corticolimbic reward systems interact during risky decision making will provide insight into the mechanism of appropriate risk taking behaviors, as well as provide a candidate circuit of interest for disorders characterized by aberrant decisions such as drug and gambling addiction.