Dopaminergic Modulation of Risky Decision-Making

Abstract

Many psychiatric disorders are characterized by abnormal risky decision-making and dysregulated dopamine receptor expression. The current study was designed to determine how different dopamine receptor subtypes modulate risk-taking in young adult rats, using a “Risky Decision-making Task” that involves choices between small “safe” rewards and large “risky” rewards accompanied by adverse consequences. Rats showed considerable, stable individual differences in risk preference in the task, which were not related to multiple measures of reward motivation, anxiety, or pain sensitivity. Systemic activation of D2-like receptors robustly attenuated risk-taking, whereas drugs acting on D1-like receptors had no effect. Systemic amphetamine also reduced risk-taking, an effect which was attenuated by D2-like (but not D1-like) receptor blockade. Dopamine receptor mRNA expression was evaluated in a separate cohort of drug-naive rats characterized in the task. D1 mRNA expression in both nucleus accumbens shell and insular cortex was positively associated with risk-taking, while D2 mRNA expression in orbitofrontal and medial prefrontal cortex predicted risk preference in opposing nonlinear patterns. Additionally, lower levels of D2 mRNA in dorsal striatum were associated with greater risk-taking. These data strongly implicate dopamine signaling in prefrontal cortical-striatal circuitry in modulating decision-making processes involving integration of reward information with risks of adverse consequences.

Introduction

Individuals are regularly faced with situations requiring rapid decisions, which involve assessment of numerous possible outcomes before deciding which is the most beneficial. Often, subjectively favorable (rewarding) outcomes are accompanied by some degree of risk of adverse consequences. Hence, the ability to appropriately assess such risks (and weigh them against potential rewards) is a critical component of optimal decision-making. Development of animal models of risky decision-making that are both reliable and possess face validity is critical for understanding the pharmacological and neurobiological substrates underlying the integration of risk and reward. Additionally, such models could have utility for developing treatment strategies to combat the maladaptive risk-taking that frequently accompanies psychiatric conditions such as attention deficit-hyperactivity disorder (ADHD), schizophrenia, major depression, addiction, and Parkinson's disease (Bechara et al., 2001; Ernst et al., 2003; Ludewig et al., 2003; Taylor Tavares et al., 2007; Kobayakawa et al., 2008).

There are several animal models of risk-based decision-making which have been well characterized (Floresco et al., 2008; Zeeb et al., 2009; Jentsch et al., 2010), but fewer in which rewarding outcomes are combined with the risk of an explicitly punishing stimulus (Negus, 2005; van den Bos et al., 2006). This latter approach captures the ambiguous nature of “real-world” risky decision-making, in which choices are often associated with both opportunities for benefits and risks of punishment (e.g., speeding up at a yellow traffic signal results in faster arrival at one's destination, but bears the risk of a traffic ticket or accident). To model this type of decision-making, we developed a task in which rats choose between a small, safe food reward and a large, risky food reward associated with a systematically increasing probability of punishment (footshock). Rats performing this “Risky Decision-making Task” (RDT) demonstrate a significant shift in preference from the risky to safe reward as risk of punishment increases within test sessions, and performance remains stable over long periods of time (Simon et al., 2009; Mitchell et al., 2011). Importantly, as in humans, there is considerable individual variability in reward preference in this task, such that some rats can be characterized as “risk-taking” (showing strong preference for the risky reward), whereas others are “risk-averse” (showing a strong bias away from the risky reward).

A common substrate underlying most if not all disorders characterized by maladaptive risky decision-making is abnormal dopamine transmission. Indeed, elevated risk-taking in ADHD can be alleviated through augmented dopamine signaling (DeVito et al., 2008), and acute d-amphetamine administration causes a substantial attenuation of risky choice in the RDT (shift toward a risk-averse pattern of choices; Simon et al., 2009; Mitchell et al., 2011). To elucidate the role of different dopamine receptor subtypes in risky decision-making, we assessed the effects of drugs specific to either D1- or D2-like receptors. We then took advantage of the robust and reliable individual differences in performance in the RDT to determine whether differences in mRNA expression of different dopamine receptor subtypes are related to individual differences in risk-taking.

Materials and Methods

Experiment 1

Subjects

Male Long–Evans rats (n = 12, Charles River Laboratories, weighing 275–300 g upon arrival) were individually housed and kept on a 12 h light/dark cycle (lights on at 8:00 A.M.) with ad libitum access to food and water except as noted. During behavioral testing, rats were maintained at 85% of their free-feeding weight, with allowances for growth. All animal procedures were conducted during the light cycle (9:00–11:00 A.M.) and were approved by the University Laboratory Animal Care and Use Committee and followed NIH guidelines.

The risky decision-making task

In the RDT, rats made discrete trial choices between two response levers, one which delivered a small reward, and the other which delivered a large reward accompanied by varying probabilities of footshock punishment ranging from 0 to 100%. The term “risk” is used in the context of this task in the sense that it refers to “the possibility that something unpleasant or unwelcome will happen” (Oxford English Dictionary, Oxford University Press), rather than “uncertainty”. Hence, risky decision-making refers to decision-making under conditions in which there is a possibility of adverse consequences.

Testing in the RDT took place in standard rat behavioral test chambers (Coulbourn Instruments) housed within sound attenuating cubicles. Each chamber was equipped with a recessed food pellet delivery trough fitted with a photobeam to detect head entries and a 1.12 W lamp to illuminate the food trough, which was located 2 cm above the floor in the center of the front wall. Forty-five milligram grain-based food pellets (PJAI, Test Diet) could be delivered into the food trough. Two retractable levers were located to the left and right of the food trough, 11 cm above the floor. A 1.12 W house light was mounted on the rear wall of the isolation cubicle. The floor of the test chamber was composed of steel rods connected to a shock generator which could deliver scrambled footshocks. Locomotor activity was assessed throughout each session with an infrared activity monitor mounted on the ceiling of the test chamber. This monitor consisted of an array of infrared (body heat) detectors focused over the entire test chamber. Movement in the test chamber (in x, y, or z planes) was defined as a relative change in the infrared energy falling on the different detectors. Test chambers were interfaced with a computer running Graphic State software (Coulbourn Instruments), which controlled programmed task events and data collection.

Before testing in the RDT, rats were shaped to perform the various task components using procedures described previously (Cardinal et al., 2000; Simon et al., 2007, 2010). Following magazine training, rats were trained to press a single lever (either the left or the right, counterbalanced across groups; the other lever was retracted during this phase of training) to receive a single food pellet. After reaching a criterion of 50 lever presses in 30 min, rats were then trained on the opposite lever under the same criterion. This was followed by further shaping sessions in which both levers were retracted and rats were trained to nose poke into the food trough during simultaneous illumination of the trough and house lights. When a nose poke occurred, a single lever was extended (left or right, pseudorandomly determined, such that each lever was presented once in every two-trial block), and a lever press resulted in immediate delivery of a single food pellet. Immediately following the lever press, the trough light was extinguished and the lever was retracted. Rats were trained to a criterion of 30 presses on each lever within 60 min.

Test sessions in the RDT were 60 min in duration and consisted of five 18-trial blocks. Each 40 s trial began with a 10 s illumination of the food trough and house lights. A nose poke into the food trough during this time extinguished the trough light and triggered extension of either a single lever (forced choice trials) or of both levers simultaneously (free-choice trials). If rats failed to nosepoke within the 10 s time window, the lights were extinguished and the trial scored as an omission. A press on one lever (left or right, counterbalanced across rats) resulted in one food pellet (the small safe reward) delivered immediately following the lever press. A press on the other lever resulted in immediate delivery of three food pellets (the large risky reward); however, selection of this lever was also accompanied by a possible 1 s footshock which occurred immediately following food delivery, contingent on a preset probability specific to each trial block. The probability of footshock accompanying the large reward was set at 0% during the first 18-trial block. In subsequent 18-trial blocks, the probability of footshock increased to 25, 50, 75, and 100%. Each 18-trial block began with 8 forced choice trials (4 for each lever) used to establish the punishment contingencies in effect for that block, followed by 10 free-choice trials. Once either lever was pressed, both levers were immediately retracted. Failure to press either lever within 10 s of their extension resulted in the levers being retracted and lights extinguished, and the trial was scored as an omission. Food delivery was accompanied by reillumination of both the trough and house lights, which were extinguished upon entry to the trough to collect the food or after 10 s, whichever occurred sooner. On the forced choice trials (in which only one lever was present) the probability of shock following a press on the large reward lever was dependent across the four trials in each block. For example, in the 25% risk block, only one of the four forced choice trials (randomly selected) always resulted in shock, and in the 75% risk block, only three of the four forced choice trials always resulted in shock. In contrast, the probability of shock on the free-choice trials (in which both levers were present) was entirely independent, such that the probability of shock on each trial was the same, regardless of shock delivery on previous trials in that block.

Performance on the RDT is determined by both the magnitude of the large reward (our unpublished data) and the shock intensity (Simon et al., 2009). The values of these parameters were chosen to maintain mean baseline levels of performance as close to the center of the parametric space as possible, so as to avoid potential floor or ceiling effects. However, over the course of testing, rats displayed some degree of habituation to the shock accompanying the large reward, manifested as a gradually increasing preference for the large, risky reward across multiple sessions. To compensate for this effect (and to maintain performance at the center of the parametric space), the shock intensity was adjusted upward by increments of 0.05 mA between some tests with different drugs (Mitchell et al., 2011). During initial training and SKF81297 administration, the shock intensity was 0.35 mA. This was increased to 0.40 mA before SCH23390 and eticlopride, 0.45 mA before bromocriptine, 0.50 mA before amphetamine + eticlopride, and 0.55 mA before amphetamine + SCH23390. Shock intensity was increased for all rats at the same time (i.e., for each drug test, all rats received the same footshock intensity). After each increase, stable performance (as described below) was reestablished.

Rats were tested in the RDT until stable performance was achieved (a total of 25 sessions—see Data analysis below for definition of stable performance), at which point drug testing commenced.

Drugs

Systemic pharmacological manipulations were performed using a repeated measures design, with different doses of each drug administered on days 1, 3, 5, and 7 of an 8 d period. Days 2, 4, 6, and 8 were untreated baseline days. For each drug tested, rats were given one of three different doses of the drug or vehicle, with the order of doses (including vehicle) counterbalanced across the four administration days. Rats were given a minimum of 5 d of drug-free training in the RDT after each 8 d drug administration period.

SKF81297 (Tocris Bioscience; 0.1, 0.3, 1.0 mg/kg) and SCH23390 hydrobromide (Tocris Bioscience; 0.005, 0.01, 0.03 mg/kg) were used as a D1-like receptor agonist and antagonist, respectively. Bromocriptine mesylate (Tocris Bioscience; 1.0, 3.0, 5.0 mg/kg) and eticlopride hydrochloride (Tocris Bioscience; 0.01, 0.03, 0.05 mg/kg) were used as a D2-like receptor agonist and antagonist, respectively. Sterile 0.9% saline was used as the vehicle for all drugs except bromocriptine, which was dissolved in dimethyl sulfoxide and then diluted at a 50:50 ratio with 0.9% saline. SKF81297 was administered 10 min before testing, SCH23390 and eticlopride 20 min before testing, and bromocriptine 40 min before testing. For drug coadministration, d-amphetamine sulfate (Sigma; 1.5 mg/kg) was administered 10 min before testing, along with separate injections of either SCH23390 (0.03 mg/kg) or eticlopride (0.05 mg/kg) 20 min before testing. All drugs were administered intraperitoneally. Drug doses and the timing of injections were based on previous studies with these drugs in either the RDT or similar cost-benefit decision-making tasks (Simon et al., 2009; St Onge and Floresco, 2009). Drug conditions were tested in the following order: SKF81297, SCH23390, bromocriptine, eticlopride, amphetamine + eticlopride coadministration, amphetamine + SCH23390 coadministration.

Data analysis

A repeated-measures ANOVA (session × punishment probability) conducted on data collected from 5 consecutive sessions was used to determine stable performance in the RDT. Stable performance was defined as the absence of a main effect or interaction involving session (Cardinal et al., 2000; Simon et al., 2007). Drug effects were analyzed using repeated-measures ANOVAs (drug dose × punishment probability). The baseline days (days 2, 4, 6, and 8 of each injection schedule) were compared with each other using repeated-measures ANOVAs; a lack of repeated measures effects across these baseline days indicated stable behavior.

Experiment 2

Subjects

A cohort of male Long–Evans rats (n = 18, 275–300 g) was used for Experiment 2. Rats were trained in the RDT until stable performance was achieved (see Experiment 1 Data Analysis for details). Rats took 21 sessions to achieve stable performance (at 0.35 mA shock intensity), after which they were tested on additional behavioral assessments (see below).

Behavioral tests

Sucrose consumption.

Rats were given access to daily 30 min tests with separate concentrations of sucrose solutions (0, 2.5, 5, 10, 20% in tap water; counterbalanced order) while in their home cages. Sucrose consumption was measured by weighing the sucrose containers before and after each test session.

Fixed ratio and progressive ratio responding.

Rats were food restricted to 85% of their free-feeding weight. Motivation for food reward was assessed daily using 30 min sessions in the same behavioral test chambers used for RDT testing, in which the rats could press a single lever under fixed ratio (FR) schedules (FR1, 3, 10, 20, 40, one schedule per session). Each FR schedule was presented only once (one schedule per day), with the schedules presented in ascending order. Following FR testing, rats were tested in the same manner in a single session using a progressive ratio schedule of reinforcement (see Mendez et al., 2009, for details).

Tail flick test.

Briefly, an IITC Model 33A tail flick apparatus was used to measure pain sensitivity. This device focused a hot lamp on the rats' tails, and the latency to move their tails was recorded. An average value for tail flick latency was determined by taking the mean of three consecutive tests in a single session. See Mendez and Trujillo (2008) for procedural details.

Shock sensitivity testing.

This procedure was modified from the method of King et al. (1996). Rats were restrained in a Plexiglas tube and habituated for 15 min. Shock sensitivity was assessed using a manual shocker (BRS/LVE, Model SG-903) which allowed continuous variation of shock intensity between 0 and 2 mA. Test shocks were applied 7 cm from the base of the tail through electrodes constructed from lightweight fuse clips. Shock intensity was gradually increased at a rate of 0.05 mA every 3 s. Latency to both movement and vocalization were assessed, after which shock was terminated.

Elevated plus maze.

The elevated (73 cm from the floor) plus-maze consisted of two opposing closed arms and two opposing open arms (42.7 cm length × 15.2 cm width/arm; arm enclosure height: 22.9 cm) connected to a central platform. The 10 min test sessions began with the rat facing the left open arm, with behavior recorded using a camera suspended over the maze. The amount of time spent in the open arms and the number of open arm entries were scored manually for each rat.

Locomotion test.

Baseline locomotion and overall exploratory behavior were assessed in activity monitoring chambers (Versamax System, AccuScan Instruments). Each chamber (40 × 40 × 30 cm) contained an array of photobeams used to detect movement in the horizontal plane throughout a 1 h session.

Data analysis

Mean percentage choice of the large, risky reward averaged across the final five sessions of RDT testing was used as the index of performance. Pearson correlations were used to compare performance in the RDT with performance in the other behavioral tasks.

Experiment 3

Tissue preparation

Following the completion of the behavioral testing in Experiment 2 (∼14 weeks after the final RDT test session), rats were killed with 100 mg/kg sodium pentobarbital, then perfused with 0.9% saline followed by 4% paraformaldehyde in 0.1 m phosphate buffer. Brains were removed and stored in 4% paraformaldehyde solution overnight, then postfixed in 4% paraformaldehyde/20% sucrose for 24 h. Brains were then rapidly frozen on dry ice and stored at −80° until sectioning. Brains were sectioned in the coronal plane on a sliding microtome, and sections (30 μm) collected in a 1-in-6 series beginning at the anterior portion of prefrontal cortex (5.2 mm from bregma, according to the atlas of Paxinos and Watson (1998)), and ending posterior to the nucleus accumbens (NAC) (−0.26 mm from bregma).

Probe preparation

Both D1 and D2 receptor probes were targeted to transcript sequences with minimal homology to other dopamine receptor subtypes. The D1 probe spanned nucleotides 1625–1968 of the D1 mRNA reference sequence (GenBank accession NM_012546) while the D2 probe spanned nucleotides 1630–2016 of the corresponding reference sequence (GenBank accession MN_012547). PCR amplified products from rat striatal cDNA were cloned into pGEM7zf+ plasmids as described previously by Haberman et al. (2011). The specificity of both probes was assessed through blast homology searches and in situ hybridization competition assays in which the addition of 5× identical unlabeled (cold) competitor (but not other dopamine receptor probes) abolished hybridization. Hybridization with sense probes to either D1 or D2 receptors also produced no specific binding. The radiolabeled antisense cRNA probe was transcribed with T7 RNA polymerase in the presence of ³⁵S-labeled UTP as per the MAXIscript in vitro transcription kit (Ambion).

In situ hybridization

Free-floating tissue sections were washed in 0.75% glycine in 0.1 m phosphate buffer (PB), pH 7.2, and 0.1 m PB alone to remove excess fixative. Sections were treated for 30 min at 37°C with proteinase K (1 mg/ml in 0.1 m Tris buffer containing 0.05% SDS), acetylated in 0.25% acetic anhydride in 0.1 m triethanolamine, pH 8.0, and rinsed twice in 2× SSC (1× SSC = 0.15 m sodium chloride and 0.015 m sodium citrate, pH 7.0). Tissue was then hybridized for 42–44 h at 60°C in solution containing 50% formamide, 1× Denhardt's solution, 10% dextran sulfate, 4× SSC, 0.25 mg/ml yeast tRNA, 0.3 mg/ml herring sperm DNA, 100 mm DTT, and the ³⁵S-labeled cRNA at a final concentration of 1 × 10⁷ cpm/ml. Following hybridization, sections were washed at 30 min intervals, twice in 4× SSC, once in 50% formamide/2× SSC at 60°C and then treated with ribonuclease A (20 mg/ml in 10 mm Tris saline buffer containing 1 mm ethylene-diaminetetracetic acid) for 30 min at 37°C. Tissue sections were then washed further in descending concentrations of SSC buffer containing 100 μm DTT to a final wash of 0.1× SSC and mounted onto gelatin-coated slides for film autoradiography. Air-dried sections were exposed along with ¹⁴C standards to PhosphorImager screens (PerkinElmer). Because dopamine receptor mRNA is less abundant in prefrontal cortex than in striatum, brain sections containing prefrontal cortex were exposed for 72 h, while sections containing dorsal striatum (DS) or NAC were exposed for 24 h. Screens were scanned at high resolution using a Typhoon PhosphorImager (PerkinElmer).

Relative D1 and D2 mRNA abundance was quantified by densitometric analysis using Densita imaging software (MBF Biosciences). Hybridization densities were linearized and calibrated relative to the ¹⁴C-labeled standards that were exposed to each phosphor screen along with tissue sections. Multiple measures were obtained from 4–6 sections per brain region per rat. For each brain structure analyzed, these values were averaged to provide an individual mean hybridization density (μCi/g protein) per region in each rat. These means were used for correlations and group comparisons.

For regional analyses, prefrontal cortex was divided into three subregions based on the atlas of Paxinos and Watson (1998): orbitofrontal, insular, and medial prefrontal (including infralimbic, prelimbic, and anterior cingulate cortex). Dorsal striatum and nucleus accumbens core and shell regions were also analyzed separately. One rat was removed from prefrontal analyses due to tissue damage.

Data analysis

Relationships between regionally specific mRNA expression and RDT performance (mean percentage choice of the large, risky reward averaged across the final five sessions of testing in Experiment 2) were analyzed in three ways. First, rats were separated into three groups on the basis of their preference for the large risky reward, and labeled risk-taking (>80% choice of the large, risky reward; n = 5), moderate (between 80% and 30% choice of the large, risky reward; n = 7), and risk-averse (<30% choice of the large, risky reward; n = 6—see Fig. 3c). One-way ANOVAs were then used to test for differences between these groups in mRNA expression in each brain region, and for variables for which the ANOVA was significant, Tukey's HSD post hoc tests were used to compare means. Second, Pearson correlations were used to test for linear relationships between performance in the RDT and mRNA expression. Finally, quadratic trend analyses using ANOVA were used to identify any nonlinear relationships between RDT performance and mRNA expression.

Figure 3.

Performance in the Risky Decision-making Task of rats used in Experiments 2 and 3. a, Risky Decision-making Task group, mean ± SEM. b, Individual variability of risky decision-making. Each line represents data from a single rat. c, Rats were divided into three groups based on risky decision-making performance: risk-taking (n = 5), moderate (n = 7), and risk-averse (n = 6). Data points in each figure represent the means (±SEM) across the final 5 sessions of testing.

Results

Experiment 1: Dopamine receptor pharmacology and risky decision-making

Risky decision-making behavior was determined to be stable between drug schedules and during baseline days (days 2, 4, 6 and 8) within each drug schedule (F values <2.71, p values >0.06). Additionally, for all drug tests, the main effect of trial block was significant (p values <0.05) and will not be reported further. This indicates that rats continued to demonstrate sensitivity to risk of punishment during each drug administration regimen.

Different doses of agonists and antagonists selective for D1- and D2-like receptors were administered systemically before testing in the RDT, using a within-subjects design. There was no effect of acute systemic administration of SKF81297 (D1-like agonist) on reward choice (F_(3,33) = 1.73, p = 0.18; Fig. 1a), nor was there an interaction between dose and risk of punishment (F_(12,132) = 0.83, p = 0.63). There were also no effects of SKF81297 on trial omissions, baseline locomotion [during intertrial intervals (ITIs)], or locomotion during the one second shock periods (shock reactivity)—see Table 1 for detailed results. The D1-like receptor antagonist SCH23390 similarly had no effects on reward choice (main effect, F_(3,33) = 0.03, p = 0.99; interaction with risk of punishment, F_(12,132) = 0.92, p = 0.53; Fig. 1b). SCH23390 did increase free-choice trial omissions at the highest dose F_(1,11) = 5.34, p = 0.04) and reduced locomotion during the ITI (F_(3,33) = 4.14, p = 0.01), but had no effect on shock reactivity.

Figure 1.

D2- but not D1-like receptor activation modulates risky decision-making. During the Risky Decision-making Task, rats were given choices between a small, safe food reward and a large food reward associated with risk of punishment. Each session consisted of 5 blocks of 10 free-choice trials, with punishment probability increasing with each consecutive block. a, b, Neither a D1-like agonist (SKF81297, a) nor antagonist (SCH23390, b) had any effect on risk-taking. c, The D2-like agonist bromocriptine had a dose-dependent effect on risk-taking, shifting preference away from the large, risky reward. d, The D2-like antagonist eticlopride had no effect on risk-taking. For all graphs: n = 12, means ± SEM.

Table 1.

Effects of drug administration in Experiment 1 on trial omissions, locomotion, and shock reactivity

Drug	Omissions (forced choice trials)	Omissions (choice trials)	Locomotion (locomotor units/ITI)	Shock reactivity (locomotor units/shock)
SKF81297
sal	0.50	0.83	39.14	2.74
0.1 mg/kg	0.42	0.50	37.27	2.92
0.3 mg/kg	0.58	0.17	39.77	3.07
1.0 mg/kg	3.83	4.17	37.55	3.07
SCH23390
sal	0.58	0.5	29.12	3.00
0.005 mg/kg	0.42	0.42	26.97	2.85
0.01 mg/kg	0.58	0.33	24.78	2.59
0.03 mg/kg	3.67	7.50**	17.61*	2.60
Bromocriptine
sal	1.50	0.25	34.46	2.91
1.0 mg/kg	3.83	2.50	28.86	2.98
3.0 mg/kg	4.75	5.83	30.14	3.02
5.0 mg/kg	5.58*	6.08	26.57	2.68
Eticlopride
sal	3.42	0.17	28.75	2.75
0.01 mg/kg	2.58	2.67	25.15	2.67
0.03 mg/kg	3.75	2.75	24.80	2.59
0.05 mg/kg	4.92	3.58	24.58	2.68
Amphetamine/SCH
sal/sal	0.33	0.00	37.48	2.51
amph/sal	4.33*	3.67	65.06**	2.57
amph/SCH	4.92*	3.25	48.14	2.83
Amphetamine/eticlopride
sal/sal	5.42	0.33	33.44	2.34
amph/sal	11.25**	2.25	59.73**	2.57
amph/eticlopride	9.75*	2.75	46.31*	3.13

sal, saline; amph, amphetamine.

*p < 0.05 compared to saline;

**p < 0.01.

In contrast to the results with D1-like receptor-acting drugs, bromocriptine (D2-like agonist) produced a dose-dependent decrease in choice of the large, risky reward (F_(3,33) = 3.37, p = 0.03; Fig. 1c), as well as an interaction between drug dose and risk of punishment (F_(12,132) = 2.07, p = 0.02) such that rats given bromocriptine shifted to a greater extent away from the large, risky reward as the risk of punishment increased than did rats administered saline vehicle (i.e., bromocriptine caused rats to be more risk-averse). There were no effects of bromocriptine on free-choice trial omissions, ITI locomotion, or shock reactivity. In contrast to the effects of bromocriptine, the D2-like antagonist eticlopride did not affect reward choice (main effect, F_(3,33) = 0.59, p = 0.63; interaction with risk of punishment, F_(12,312) = 0.71, p = 0.74; Fig. 1d). There was also no effect of eticlopride on trial omissions, ITI locomotion, or shock reactivity.

To confirm the role of D2-like receptor signaling in reducing risk-taking, the effects of coadministration of either the D1- or D2-like antagonist with amphetamine were examined. Coadministration of SCH23390 (D1-like antagonist) did not block the reduction in risk-taking induced by amphetamine (main effect of drug condition, F_(2,22) = 3.89, p = 0.04; planned comparison between SCH23390 + amphetamine and amphetamine alone, F_(1,11) = 0.68, p = 0.43; Fig. 2a). Notably, SCH23390 did attenuate the locomotor stimulant effect of amphetamine (Vezina and Stewart, 1989), indicating the effectiveness of this dose of the drug (Table 1). In contrast, coadministration of the D2-like antagonist eticlopride did significantly attenuate the reduction in risk-taking induced by amphetamine (main effect of drug condition, F_(2,22) = 4.32, p = 0.03). Planned comparisons between drug conditions showed that amphetamine reduced risk-taking relative to saline (F_(1,11) = 5.90, p = 0.03), whereas eticlopride + amphetamine did not (F_(1,11) = 1.01, p = 0.34). Moreover, amphetamine + eticlopride produced a near-significant attenuation of the effects of amphetamine alone (F_(1,11) = 4.16, p = 0.07; Fig. 2b). There was no effect of drug condition on either free-choice trial omissions or shock reactivity; however, both amphetamine and amphetamine + eticlopride increased locomotion relative to saline conditions (i.e., blockade of D2-like receptors did not block the locomotor stimulant effects of amphetamine—Table 1). In addition to their effects on choice behavior and locomotion, both drug combinations (as well as bromocriptine) increased the number of omissions of forced choice trials. Although this may have reduced the number of shocks experienced by the rats during these trials, it would not have altered the probabilities of receiving shocks on the remaining (completed) trials, and hence, it is unlikely that these omissions significantly affected performance on the free-choice trials.

Figure 2.

D2- but not D1-like receptor blockade abolishes the effects of amphetamine on risky decision-making. We showed previously that amphetamine, in a dose-dependent manner, decreases preference for the large risky reward in a manner similar to bromocriptine (Simon et al., 2009). To verify the involvement of D2-like receptors in risky decision-making, amphetamine was coadministered with either SCH23390 or eticlopride. a, Amphetamine alone attenuated risky choice relative to control conditions, and this effect was not blocked by coadministration of SCH22390 (D1-like antagonist). b, Amphetamine's effects on risky choice were significantly attenuated by coadministration of eticlopride (D2-like antagonist). These data indicate that D2-like receptor activation is necessary for amphetamine to reduce risky choice. For both graphs: n = 12, means ± SEM.

Experiment 2: Behavioral basis of individual differences in risk-taking

A striking feature of RDT performance is the high degree of individual variability in risk preference, with rats showing different degrees of preference for the large risky reward, ranging from strongly risk-averse to strongly risk-taking (Fig. 3a,b). Previous work showed that individual rats' performance is stable over several months (Simon et al., 2009), and similar stability was observed across baseline (non-drug) test sessions in Experiment 1 (r values >0.78, p values <0.01). Such stability suggests that the observed individual differences in risk preference reflect a “trait-like” feature of behavior; however, they could also be reflective of individual differences in other, “simpler” aspects of behavior such as reward motivation, anxiety, or pain sensitivity. To address this issue, drug-naive rats (n = 18) were first characterized in the RDT, then tested on measures of reward motivation (fixed and progressive ratio instrumental responding and sucrose consumption tests), anxiety (elevated plus maze and open field activity), and pain tolerance (shock sensitivity and tail flick latency). There were no correlations between performance on any of these tasks and risky decision-making (r values <0.35, p values >0.15; Table 2). Therefore, at least to some degree, the integration of reward information with risk of physical punishment to guide choice behavior can be considered an independent behavioral construct.

Table 2.

Relationship between risky decision-making and other behavioral measures

Factor	Correlation coefficient (r) (with RDT)	p value (with RDT)
Appetitive motivation
Weight	0.18	0.49
FR1	0.03	0.90
FR3	0.23	0.36
FR10	0.18	0.49
FR20	0.23	0.37
PR	−0.14	0.41
Consummatory motivation
2.5%	0.21	0.41
5%	−0.06	0.21
10%	0.25	0.33
20%	0.33	0.18
Anxiety
EPM: % time open arms	−0.12	0.49
EPM: % open arm entries	−0.14	0.59
Activity: time in center	−0.36	0.15
Activity: horizontal activity	0.23	0.35
Activity: distance traveled	0.29	0.24
Pain tolerance
Tailflick latency	−0.02	0.94
Shock reactivity: locomotor	0.24	0.38
Shock reactivity: vocal	0.15	0.59

There were no significant correlations between risky decision-making and any measures of motivation, anxiety, or pain sensitivity.

Experiment 3: Dopamine receptor mRNA expression and individual differences in risk-taking

Experiment 1 suggested that D2-like but not D1-like dopamine receptors are involved in modulating risky decision-making, such that systemic D2-like receptor activation attenuates preference for large risky rewards. These data suggest that dopamine signaling specifically through D2-like receptors might play a role in mediating the individual differences in risk-taking described in Experiment 2. Given that dopamine plays a critical modulatory role in several forebrain regions implicated in decision-making processes, we sought to determine potential sites of action for these effects (Robbins, 2005; Floresco et al., 2008). Hence, following behavioral testing in Experiment 2, rats were killed and brains processed for in situ hybridization for analysis of D1 and D2 receptor mRNA in prefrontal cortical and striatal regions implicated in decision-making (Roesch et al., 2007; Clark et al., 2008; Morrison and Salzman, 2009; Naqvi and Bechara, 2009; St Onge and Floresco, 2010). Representative brain sections for both D1 and D2 hybridization are shown in Figure 4.

Figure 4.

Hybridization of radiolabeled D1 and D2 mRNA in prefrontal cortex and striatum. a–d, Images from film autoradiograms show D1 (a, c) and D2 (b, d) receptor mRNA expression in coronal sections through the prefrontal cortex and striatum. Note the preferential expression of D1 and D2 mRNA in deeper layers of prefrontal cortex (Santana et al., 2009). Data from anterior cingulate cortex (ACC), prelimbic cortex (PL), and infralimbic cortex (IL) were merged (as medial prefrontal cortex) for analysis. OFC, orbitofrontal cortex; INS, insular cortex; DS, dorsal striatum; NC, nucleus accumbens core; NS, nucleus accumbens shell.

D1 receptor mRNA expression

Prefrontal cortex.

A one-way ANOVA revealed a difference in D1 expression in insular cortex (INS) on the basis of risk group (risk-taking, moderate, and risk-averse; F_(2,14) = 4.06, p = 0.04; Fig. 5c). Tukey's HSD post hoc tests revealed lower D1 mRNA expression in the risk-averse compared with the moderate group (p = 0.05), and a trend toward lower D1 mRNA expression in the risk-averse compared with the risk-taking group (p = 0.08). In addition, there was a linear correlation between risk preference and D1 mRNA expression in INS (r = 0.49, p = 0.04), such that higher levels of D1 receptor mRNA were associated with higher levels of risk-taking (but no significant nonlinear relationship). No relationships were observed between D1 mRNA expression in medial prefrontal cortex (mPFC) or orbitofrontal cortex (OFC) and risk preference (Fig. 5a; see Table 3 for full statistical details).

Figure 5.

D1 receptor mRNA expression in prefrontal cortex and striatum. a–f, There were positive correlations between risk-taking and D1 mRNA expression in insular cortex and nucleus accumbens shell, such that high D1 expression predicted high risk-taking (c, f). There was no significant relationship between risky decision-making and D1 expression in orbitofrontal cortex (a), medial prefrontal cortex (b), dorsal striatum (d), or nucleus accumbens core (e). All bar graphs represent means + SEM. *p < 0.05, Tukey's HSD post hoc test.

Table 3.

Statistical relationships (linear correlations, quadratic trend analyses, and one-way ANOVAs) between mRNA expression and risky decision-making

	Linear correlation (Pearson's r)		Nonlinear correlation (quadratic trend analysis)		Group differences (one-way ANOVA)
	D1	D2	D1	D2	D1	D2
Dorsal striatum	r = 0.18	r = −0.44	F(2,15) = 0.20	F(2,15) = 5.96	F(2,15) = 0.63	F(2,15) = 8.35
	p = 0.49	p = 0.07	p = 0.83	p = 0.01*	p = 0.55	p = 0.00*
Nucleus accumbens core	r = 0.26	r = 0.11	F(2,15) = 1.50	F(2,15) = 0.65	F(2,15) = 2.20	F(2,15) = 0.77
	p = 0.29	p = 0.66	p = 0.26	p = 0.54	p = 0.15	p = 0.48
Nucleus accumbens shell	r = 0.50	r = 0.07	F(2,15) = 5.44	F(2,15) = 0.07	F(2,15) = 2.63	F(2,15) = 0.12
	p = 0.03*	p = 0.80	p = 0.02*	p = 0.93	p = 0.10	p = 0.89
Medial prefrontal cortex	r = 0.06	r = −0.30	F(2,14) = 1.21	F(2,14) = 5.60	F(2,14) = 1.35	F(2,14) = 8.46
	p = 0.83	p = 0.13	p = 0.33	p = 0.02*	p = 0.29	p = 0.00**
Orbitofrontal cortex	r = 0.38	r = −0.05	F(2,14) = 1.20	F(2,14) = 3.10	F(2,14) = 1.36	F(2,14) = 14.58
	p = 0.14	p = 0.84	p = 0.33	p = 0.08	p = 0.29	p = 0.00**
Insular cortex	r = 0.49	r = 0.10	F(2,14) = 2.53	F(2,14) = 1.62	F(2,14) = 4.06	F(2,14) = 1.7
	p = 0.04*	p = 0.71	p = 0.12	p = 0.23	p = 0.04*	p = 0.22

Striatum.

No differences among risk groups in D1 expression were observed in nucleus accumbens shell (NACs), although this effect did approach significance (F_(2,15) =2.63 p = 0.10). However, there was a significant linear correlation between risk preference and D1 mRNA abundance in NACs (r = 0.50, p = 0.03), such that greater D1 receptor mRNA expression in NACs was predictive of greater risk preference (Fig. 5f). There were no relationships of any shape between D1 mRNA expression in nucleus accumbens core (NACc) or DS and risk preference (Fig. 5d,e).

Multiple regression analyses were also used to determine whether D1 mRNA expression in multiple brain regions could individually predict risk-taking. In support of this idea, both INS and NACs D1 mRNA expression significantly predicted risky decision-making behavior over and above the other (INS ΔR² = 0.32, p = 0.005; NACs ΔR² = 0.37, p = 0.003). The model including both INS and NACs D1 mRNA as predictors accounted for more variance than either did separately (overall R² = 0.61).

D2 receptor mRNA expression

Prefrontal cortex.

A one-way ANOVA revealed a robust relationship between risk preference and D2 mRNA expression in OFC (F_(2,14) = 14.58, p = 0.0004), such that both the risk-averse and risk taking groups displayed greater mRNA expression than the moderate risk group (Tukey's HSD post hoc analyses: risk-taking vs moderate: p = 0.001; risk-averse vs moderate: p = 0.002, Fig. 6a). This pattern of data was also reflected in a near-significant quadratic relationship between OFC mRNA expression and risk preference, which resembled a U-shaped curve (F_(2,14) = 3.10, p = 0.08). A contrasting nonlinear relationship between D2 mRNA expression and risk preference was observed in mPFC (i.e., an inverted U-shaped curve, Fig. 6b). There was a significant effect of risk group on expression in mPFC (F_(2,14) = 8.46, p = 0.004) such that the moderate risk group demonstrated greater mRNA expression than the risk-averse or risk-taking groups. Tukey's HSD post hoc tests revealed that the moderate risk group had higher levels of mRNA expression in mPFC than the risk-taking group (p = 0.003), but that the difference between the moderate and risk-averse groups did not reach significance (p = 0.10). This pattern of data was also reflected in a significant quadratic relationship between mPFC mRNA expression and risk preference (F_(2,14) = 5.60, p = 0.02). In contrast to D1, there were no relationships between D2 mRNA expression in INS and risky decision-making (Fig. 6c).

Figure 6.

D2 receptor mRNA expression in prefrontal cortex and striatum. a–f, D2 mRNA expression in orbitofrontal cortex predicted risk-taking as a U function: high levels of D2 expression predicted either high or low risk-taking (a). The opposite relationship was evident in medial prefrontal cortex, with low D2 expression predicting high or low risk-taking (b). D2 mRNA expression in dorsal striatum negatively predicted risky decision-making: lower levels of hybridization predicted greater risk-taking (d). There were no relationships between D2 mRNA expression in insular cortex or either region of nucleus accumbens and risky decision-making (c, e, f). All bar graphs represent means + SEM. *p < 0.05, Tukey's HSD post hoc test.

Striatum.

A one-way ANOVA revealed a significant main effect of risk group in DS (F_(2,15) = 8.35, p = 0.004), and Tukey's HSD post hoc tests confirmed that D2 mRNA expression was significantly greater in the risk-averse group than in the moderate group (p = 0.003, Fig. 6d), although this was not significant between the risk-taking and risk-averse groups (p = 0.19). Combining the moderate and risk-taking groups revealed a significant statistical difference between risk-averse rats and this combined group (F_(1,17) = 10.85, p = 0.005), indicating that risk-averse rats had greater D2 mRNA expression in DS than all other rats. Quadratic trend analysis also revealed a significant nonlinear relationship between D2 mRNA expression in DS and risk preference resembling a modified U curve (F_(2,15) = 5.96, p = 0.01). In contrast to D1, there were no significant relationships between D2 mRNA expression and risk preference in either subregion of NAC (Fig. 6e,f).

Discussion

Few decisions in life are without some risk of adverse consequences, and the ability to accurately evaluate risks that accompany rewarding choices and decide accordingly can significantly influence medical, social, and financial well-being. Here we used a rat model to demonstrate that risky decision-making is attenuated by D2-like (but not D1-like) receptor activation. We also showed that the stable phenotype of preference for small safe versus large risky rewards is related to both D1 and D2 receptor mRNA expression in specific brain regions. The results implicate dopamine signaling through distinct receptor classes in a network of corticostriatal brain regions as a critical modulator of decision-making under conditions of risk of adverse consequences.

The validity of these data hinges on establishment of risky decision-making as a reliable and independent behavioral construct. We show here that performance in the RDT was unrelated to multiple measures of reward motivation, pain tolerance, or anxiety, suggesting that integration of rewards with risk of physical punishment to guide choice behavior can be measured as a distinct behavioral construct. The absence of a relationship between pain sensitivity and RDT performance is of particular importance because this task incorporates an aspect of physical punishment (footshock) that, to our knowledge, has heretofore not been integrated into rodent cost-benefit decision-making tasks. This indicates that choice was likely not mediated solely by sensitivity to pain, but instead by a separate reward discounting process that reflects willingness to risk punishment. Notably, although there were no correlations between risk preference and measures of anxiety, the RDT does bear some similarity to conflict tasks such as punished responding tasks (File et al., 2004). The RDT differs from such tasks, however, in that rats are given a safe option as an alternative to the punished reward, and also in that the probability of punishment changes systematically throughout the task. There are also divergences in the effects of pharmacological manipulations in these tasks (e.g., unlike in the RDT, a D2-like agonist enhances punished responding in a conflict task) (Millan et al., 2004; see Mitchell et al., 2011, for further discussion).

Dopamine receptor behavioral pharmacology

Systemic administration of the D2-like agonist bromocriptine reduced preference for risky rewards in a fashion similar to amphetamine (Simon et al., 2009; Mitchell et al., 2011), and coadministration of the D2-like antagonist eticlopride attenuated amphetamine's effects on choice behavior. Surprisingly, eticlopride alone had no effect on choice behavior, indicating that D2-like receptors do not influence risk-taking in an entirely linear fashion. These effects of D2-like receptor-acting drugs differ from their effects on a probability discounting task, which measures risk of reward omission rather than risk of punishment. St Onge and Floresco (2009) reported that both amphetamine and bromocriptine increased risky choice in this task, which appears contrary to the decrease in risky choice found here. This dissociation is likely a result of the difference in the discounting factors associated with the large reward. Dopamine neurotransmission appears to enhance the salience of punishing factors (e.g., footshock; Thiebot et al., 1991; Killcross et al., 1997; Evenden and Ko, 2005), thereby biasing behavior away from the large reward because of the greater relative salience of the discounting factor in the RDT (risk of footshock) compared with the large reward. In the probability discounting task, elevated dopamine neurotransmission may bias behavior in the opposite direction because of the greater relative salience of the large reward compared with the discounting factor (reward omission). Another possibility is that integration of distinct discounting factors (punishment vs reward omission) with rewards may use separate neural mechanisms which respond differently to augmented D2-like receptor activity.

D1 receptor mRNA expression and risky decision-making

The results of Experiment 1 suggested that D2-like receptors are involved in modulation of risky decision-making. Experiment 3 showed that both D1 and D2 receptor mRNA expression in several corticostriatal brain regions were predictive of risk preference. Although acute systemic D1-like receptor activation had no effects on risk-taking, D1 receptor mRNA expression in both INS and NACs were positively correlated with risk preference (i.e., greater D1 mRNA expression predicted greater risk-taking). These data suggest that D1 receptors within this system (Reynolds and Zahm, 2005) may be involved in valuation of risky versus safe outcomes (or perhaps valuation of differing reward magnitudes), consistent with a role for this system in cost-benefit decision-making (Cardinal, 2006; Naqvi and Bechara, 2009). Future experiments involving local pharmacological manipulations will be useful for verifying the specific role(s) of D1 receptors within this system. Importantly, relationships between D1 mRNA expression and RDT performance were likely not attributable to differences in shock experience in the task. Although there is some evidence that chronic uncontrollable stress can alter D1 receptors (Rasheed et al., 2010; Matrov et al., 2011) it seems unlikely that shock experience in the RDT was experienced as stressful in the same manner, as it was easily avoidable and freely chosen (Maier et al., 2006). Additionally, no relationships were observed between shock experience and several measures of anxiety (Experiment 2), indicating that even high levels of shock experience in the RDT caused no lasting changes in stress- or anxiety-related behavior.

D2 receptor mRNA expression and risky decision-making

The differences observed across rats in D2 receptor mRNA expression in prefrontal cortex were not manifested in linear fashion, but instead reflected U-shaped relationships with behavior, consistent with findings that prefrontal cortically mediated aspects of cognition can be influenced by dopamine in nonlinear fashion (Robbins, 2005). Specifically, rats that demonstrated strongly biased preference for or aversion to the large risky reward (both the risk-taking and risk-averse groups, termed “biased responders”) had similar dopamine receptor profiles, whereas rats that shifted reward preference as a function of punishment risk (the moderate group, termed “flexible responders”) displayed a comparatively unique profile. Interestingly, while D2 receptor mRNA expression in both OFC and mPFC adhered to this pattern, the relationship between biased and flexible responders differed: in OFC, biased responders showed greater D2 mRNA expression than flexible responders, whereas in mPFC, the opposite pattern was evident. It is interesting to consider that, in some situations, unwillingness to take risks may be equally as maladaptive as excessive risk taking. Indeed, the fact that D2-like receptor blockade within mPFC impairs cognitive flexibility is consistent with this idea (Floresco and Magyar, 2006; but see Calaminus and Hauber, 2008).

D2 receptor mRNA in DS was also predictive of risky decision-making. While this relationship was best fit by a nonlinear function, it was clearly distinct from the patterns observed in prefrontal cortex. In DS, rats characterized as risk-averse showed greater D2 mRNA expression compared with combined data from risk-taking and moderate rats, indicating that increased D2 expression in this region is related to decreased risk preference (risk aversion). Of the relationships observed between D2 mRNA and risk-taking, this most resembled the acute effects of D2-like receptor activation; thus, it is possible that while baseline levels of risk-taking are determined by a circuit that includes both prefrontal cortical and striatal regions, DS is the major target of amphetamine and bromocriptine's effects on risk-taking. Anatomically, DS serves as a relatively direct conduit to motor regions while receiving inputs from prefrontal cortex (Albin et al., 1989; Groenewegen et al., 1990). Therefore, it is possible that D2 receptor activity in DS can supersede input from these other regions and bias reward preference in a specific direction (in this case, away from risky options). Reduced striatal D2 receptor availability is associated with addiction, ADHD, and risk-taking in humans (Volkow et al., 2004, 2011; Goldstein et al., 2009), and low striatal D2 receptor levels in animal models are associated with abnormal levels of habit formation (Everitt et al., 2008), increased motoric impulsivity, propensity for “compulsive” cocaine self-administration (Dalley et al., 2007), and compulsive-like eating (Johnson and Kenny, 2010). Our findings associating lower D2 mRNA in DS with greater risk-taking are consistent with this body of work, and suggest that individual differences in risk-taking may predate (and even play a causal role in) the onset of pathology.

The distinct relationships between striatal D1 and D2 receptor mRNA and risky decision-making are also consistent with data from human subjects. Frank et al. (2007) showed that a genetic polymorphism associated with high levels of D1 receptors in striatum was predictive of greater reward-driven behavior during decision-making, while a polymorphism associated with high D2 receptor availability in striatum was associated with greater avoidance of punishment (see also Richard and Berridge, 2011). The current data provide evidence that D1 and D2 receptor expression predict risk bias in an animal model (toward large rewards and away from punishments, respectively), while expanding on previous data by identifying NACs and DS as potential sites of D1 and D2 receptor mediation of risk-taking within striatum, respectively. Given the relationships observed here, it will be of considerable interest in future studies to investigate the functional roles of these receptors (as well as those of other dopamine receptor subtypes) within distinct components of corticostriatal circuitry.