Impaired flexibility in decision making in rats after administration of the pharmacological stressor yohimbine

Abstract

Rationale

Stress-induced disruption of decision making has been hypothesized to contribute to drug-seeking behaviors and addiction. Noradrenergic signaling plays a central role in mediating stress responses. However, the effects of acute stress on decision making, and the role of noradrenergic signaling in regulating these effects, have not been well characterized.

Objective

To characterize changes in decision making caused by acute pharmacological stress, the effects of yohimbine (an α2-adrenergic antagonist) were examined in a delay discounting task. Noradrenergic contributions to decision making were further characterized by examining the effects of propranolol (a β antagonist), prazosin (an αl antagonist), and guanfacine (an α2 agonist).

Methods

Sprague-Dawley rats were administered drugs prior to performance on a delay discounting task, in which the delay preceding the large reward increased within each session (ascending delays). To dissociate drug-induced changes in delay sensitivity from behavioral inflexibility, drug effects were subsequently tested in a modified version of the discounting task, in which the delay preceding the large reward decreased within each session (descending delays).

Results

Yohimbine increased choice of the large reward when tested with ascending delays but decreased choice of the same large reward when tested with descending delays, suggesting that drug effects could be attributed to perseverative choice of the lever preferred at the beginning of the session. Propranolol increased choice of the large reward when tested with ascending delays. Prazosin and guanfacine had no effect on reward choice.

Conclusions

The stress-like effects of yohimbine administration may impair decision making by causing inflexible, perseverative behavior.

Introduction

Acute stress can profoundly impair cognitive functions necessary for optimal decision making. The effects of acute stress result in part from elevated locus coeruleus noradrenergic (NA) signaling (Berridge and Waterhouse 2003; Birnbaum et al. 1999), which importantly regulates attentional processing, working memory, and behavioral flexibility through action on forebrain targets (Bouret and Sara 2005; Chamberlain and Robbins 2013; Lapiz and Morilak 2006; McGaughy et al. 2008; Tait et al. 2007). High levels of NA signaling in target regions such as prefrontal cortical areas have been shown to diminish working memory capacity, decrease attentional focus, and impair behavioral flexibility (Arnsten 2009; Aston-Jones et al. 1999; Caetano 2013; Howells et al. 2012). Optimal decision-making behavior relies on each of these cognitive faculties and thus would also be expected to be sensitive to disruption by acute stressors.

Acute stressors have been hypothesized specifically to promote impulsivity (de Wit 2009). Impulsivity is well recognized as a multi-dimensional construct, and two distinct types of impulsivity include the inability to inhibit inappropriate or irrelevant preplanned movements (motor impulsivity) and delay aversion (or increased desire for immediate reward, termed cognitive impulsivity) (Pattij and Vanderschuren 2008). Cognitive impulsivity is characterized by increased delay discounting, or time-dependent devaluation of delayed rewards. In tasks requiring response inhibition for successful performance, the pharmacological stressor yohimbine increases motor impulsivity across species, including primates (Ma et al. 2003), rodents, (Sun et al. 2010) and human volunteers (Swann et al. 2005; Swann et al. 2013). The effects of environmental or pharmacological stressors on cognitive impulsivity have received less experimental attention. A recent study found restraint stress altered effort- but not delay-based decision making in rats (Shafiei et al. 2012). However, human studies suggest stress can broadly affect decision making, including delay-based reward choice. Anticipation stress interacts with trait perceived stress to alter delay discounting in human volunteers (Lempert et al. 2012), and acute psychosocial stress increases delay discounting in individuals who show enhanced cortisol reactivity (Kimura et al. 2013). Related studies of risk taking in gambling tasks suggest that acute social or physiological stress can alter risk aversion during decision making (Porcelli et al. 2012; Preston et al. 2007; van den Bos et al. 2009). Furthermore, acute cold pressor stress has been shown to attenuate neural responses to monetary rewards (Porcelli et al. 2012). Thus, substantial evidence suggests acute stress alters reward valuation and decision-making strategies.

Stress effects on cognitive impulsivity are of significant interest, given the robust association of drug addiction with this form of impulsivity (Winstanley et al. 2010). Both active and abstinent drug users discount delayed rewards at rates that exceed those of healthy controls, and this relationship holds for cocaine, opioid, alcohol, and nicotine dependence (Coffey et al. 2003; Madden et al. 1997; Mitchell et al. 2007; Mitchell 1999; Petry 2001; Vuchinich and Simpson 1998). Causality in this association is not understood, but some evidence suggests that impulsivity is a risk factor for drug taking (Dom et al. 2006; Oberlin and Grahame 2009). Acute stress, then, may elevate drug seeking and taking as a consequence of increasing cognitive impulsivity. Stress effects on drug seeking and using are well established: acute stress potently contributes to relapse to drug taking in addicts (Breese et al. 2011; Goeders 2002; Higley et al. 2011; Sinha et al. 2011), and the pharmacological stressor yohimbine increases opioid-seeking in heroin-dependent individuals (Greenwald et al. 2013) and reinstates drug seeking in animal models (e.g. (Banna et al. 2010; Ghitza et al. 2006; Le et al. 2005; Shepard et al. 2004). However, the degree to which acute stress can cause cognitive impulsivity, thus potentially contributing to addictive behavior, remains incompletely understood.

To address this gap in our current understanding, we tested the effects the α2-adrenergic receptor antagonist yohimbine on decision making in a delay discounting task. Yohimbine increases firing in the locus coeruleus, as well as NA release in terminal regions (Bremner et al. 1996a; b; Crespi 2009; Holmberg and Gershon 1961), and causes an acute stress-like response in humans, rodents, and monkeys (Bremner et al. 1996a; b; Crespi 2009; Holmberg and Gershon 1961). Importantly, stress circuits activated by yohimbine are similar to those activated by environmental stressors such as tail pinch and footshock (Funk et al. 2006). In addition, to more fully characterize the effects of noradrenergic signaling on decision making, the effects of guanfacine (an α2 agonist), prazosin (an αl antagonist), and propranolol (a β receptor antagonist) were also examined. Each of these drugs has been investigated for potential therapeutic use in impulsivity-related disorders (Seixas et al. 2012; Vaiva et al. 2003). We found that rather than affecting impulsivity, yohimbine promoted inflexible, perseverative responding characterized by relative insensitivity to changes in delays preceding reward delivery.

Materials and Methods

Animals

Male Sprague-Dawley rats (n=16, Charles River Laboratories, St. Louis) weighing 250–350 g at the start of the experiment were food restricted to ~90% ad libitum body weight for the duration of the experiments. Experimental procedures were approved by the University of Utah Institutional Animal Care and Use Committee.

Apparatus and overview of training

Daily behavioral training sessions took place in operant conditioning chambers (30.5 cm × 24.1 cm × 21.0 cm, Med-Associates, Georgia, VT) equipped with two retractable levers flanking a central reward receptacle, with stimulus lights located above each of these devices. Each chamber was enclosed in a sound-attenuating box containing a fan for ventilation. Experimental data was recorded and stored on a PC, which was interfaced with the operant conditioning chambers using MedPC software (Med Associates). Rats progressed through three training stages (described below) before being trained on the full delay discounting paradigm. Rats were trained in two separate cohorts of 8 rats each and were advanced through each phase of training and drug administration as an entire cohort.

FR1 Training

Rats were first trained to lever press under a fixed ratio 1 (FR1) schedule of reinforcement for Intralipid, a highly palatable soy emulsion (4% w/v; Fresenius Kabi, Schaumburg, IL), delivered by pump into the reward receptacle. Trials were initiated with the illumination of a house light, extension of a single lever, and illumination of the cue light located above that lever. A lever press response within 30 seconds of lever extension resulted in retraction of the lever, delivery of 200 µL of Intralipid, and illumination of a light located above the reward receptacle. The house light and reward receptacle light extinguished when rats entered the reward receptacle or 8 seconds after reward delivery, whichever occurred first. During early sessions of this training phase, a trial was initiated every 40 seconds, and rats were required to execute a response within 30 seconds in order to receive the reward. Once rats learned to lever press, the intertrial interval increased to 50 seconds and rats were required to complete a lever press within 20 seconds of the lever extension in order to receive the reward. If no response was made within the allotted time, then the lever was retracted and the house light was extinguished until initiation of the next trial. The active lever (left or right) alternated across 120-minute sessions. When rats responded on at least 80% of lever presentations on both sides, they were advanced to the next training stage, an FR1 paradigm with randomized lever presentation.

FR1, randomized lever presentation

Task design was similar to FR1 training, with the exception that either the left or the right lever was presented on each trial. Selection of the lever presented was semi-random, with no single lever presented on more than three consecutive trials. Sessions were 90 minutes in length, and rats were required to execute a lever press response within 10 seconds of lever presentation in order to receive a reward. Rats were advanced to the next training stage when they responded on at least 80% of trials, with approximately equal number of responses on the left and right levers.

Large/Small Choice

During this phase of training, rats chose between a small and a large reward. Each session consisted of 60 trials divided into 5 blocks of 12 trials each. Each block began with 2 forced trials, during which only one of the two levers was extended, followed by 10 choice trials, during which both levers were extended. As in the previous stage, the lever(s) retracted and the house light extinguished if no response was made within 10 seconds of lever extension. A new trial was initiated every 70 seconds, regardless of the rats’ response. For each rat, a response on one lever (left or right) always delivered 50 µL of Intralipid (small reward), while a response on the other lever always delivered 300 µL of Intralipid (large reward). Lever assignment was counterbalanced across rats. Rats advanced to the final task when preference for the large reward exceeded 75%.

Delay Discounting: Ascending Delays

A delay was introduced before the delivery of the large reward for the full delay discounting task (Fig. 1a). Blocks were divided as described for the previous phase of training, with the length of the delay period increasing across blocks within each session, such that delay periods were 0.5, 10, 20, 30 and 40 seconds for blocks 1–5. At the beginning of each choice trial, both levers were extended and cue lights were illuminated. When the large, delayed reward was chosen, the light above that lever remained illuminated until the reward was delivered. The small, immediate reward was always delivered 0.5 seconds after a rat made a lever press response. An omission was scored if no response was made on a given trial. All other task parameters were as described for the Large/Small Choice task.

Figure 1.

Behavioral paradigms. Delay discounting tasks with a ascending and b descending delay order

Drug testing (yohimbine, propranolol, guanfacine and prazosin; see below) began after all animals displayed stable choice behavior. Behavioral stability was assessed by analyzing rats’ reward choice from five consecutive sessions using a two-way repeated measures ANOVA, with day and delay as factors.

Delay discounting behavior was considered stable if time-dependent changes in reward choice were absent (i.e., no significant main effect of day).

Descending Delays Task

After drug testing in the ascending delays discounting task, rats underwent training in the descending delays discounting task. In this task, the length of the delay period preceding delivery of the large reward was reversed, such that the delay length decreased, rather than increased, within each session (Fig. 1b). All task parameters were identical to those described for the ascending delays task, with the exception that delay periods for blocks 1–5 were successively 40, 30, 20, 10, and 0.5 seconds. Drug testing was initiated when animals displayed stable patterns of behavior on five consecutive sessions. Table 1 summarizes the timeline of training and drug administration for both ascending and descending delays.

Table 1.

Experimental timeline. Numbered training/drug testing stages indicates the order in which experiments took place. For each stage of training and drug testing, values indicate the number of training days for each of two cohorts of rats. Training days include both drug testing and drug-free training sessions

	Training days
Task trained/drug tested	Cohort 1	Cohort 2
1. Ascending delays training	20	30
2. Yohimbine	14	27
3. Propranolol	10	10
4. Prazosin	10	10
5. Guanfacine	10	12
6. Descending delays training	65	68
7. Yohimbine	15	14
8. Propranolol	11	14
9. Prazosin	5	6
10. Guanfacine	4	5

This descending delays task was used in combination with the ascending delays task to dissociate apparent drug effects on impulsivity from effects on behavioral flexibility. Decreased impulsivity in the ascending delays task is evident as an upward shift in the discounting curve (i.e., increased choice of the large delayed reward across the session). In ascending delays paradigm, the large reward was strongly preferred by all rats in the first block of trials, when the delay length was only 0.5 seconds (equal to that for the immediate reward). Thus, persistently elevated choice of the large reward across the session could reflect drug-induced perseverative responding on the initially preferred lever, rather than increased tolerance to delay. The descending delays task paradigm allows these possibilities to be dissociated. Because the delay in the first block of trials is large, rats prefer the small immediate reward initially. Drug-induced perseverative behavior would thus be expected to elevate responding for the immediate, rather than the delayed, reward in the reversal task. Previous studies have used this strategy in both delay and probability discounting tasks to assess drug-induced changes in choice behavior (Slezak and Anderson 2009; St Onge et al. 2010; St Onge and Floresco 2010; Tanno et al. 2013).

Drugs

All drugs were purchased from Sigma-Aldrich (St. Louis, MO) and dissolved in distilled water. Doses and administration times were based on those shown to be behaviorally relevant in previous studies. Yohimbine (0, 0.6, 1.25, 2.5, and 5 mg/kg, i.p.) was injected 30 minutes prior to the task in a volume of 0.5 ml/kg (Le et al. 2005; Packard and Wingard 2004). Propranolol (0, 1, 3, and 10 mg/kg, i.p.) was injected 40 minutes prior to the task in a volume of 1 ml/kg (Chiamulera et al. 2010). Guanfacine (0, 0.06, 0.125, and 0.25 mg/kg, i.p.) was administered 1 hour prior to the task in a volume of 1 ml/kg (Le et al. 2011). Prazosin (0, 0.5, 1, or 2 mg/kg, i.p.) was administered 40 minutes prior to the task (Le et al. 2011). Due to low solubility of prazosin, the 2 mg/kg dose was administered in a volume of 2 ml/kg, and vehicle was administered in an equivalent volume. The lower doses of prazosin (0.5 and 1 mg/kg) were injected in a volume of 1 ml/kg. For all drugs, dose delivery was randomized using a Latin square design. Rats typically received two injections per week. At least two days (drug-free rest or drug-free training) were allowed between each drug administration, and a drug-free training session always occurred on the day immediately preceding drug administration in order to ensure that baseline performance had returned to pre-drug levels. If persistent (carryover) drug effects were apparent, additional drug free training sessions were included. Importantly, all rats were included in these sessions, ensuring that training schedules and the timing of drug testing were similar for all rats. Finally, at least a four-day washout period was allowed between administration of different drugs.

All drug effects were studied first in the ascending delays discounting task. Because a significant effect of drug was observed in the ascending delays task for both yohimbine and propranolol, all doses of these drugs were subsequently studied in the descending delays task. In order to ensure that guanfacine and prazosin had no effect in both ascending and descending delays tasks, one dose of each was examined in the descending delays task. Because the highest dose of guanfacine caused sedation, the second highest dose (0.125 mg/kg) was used. The highest dose of prazosin (2 mg/kg) was used.

Data analysis

Statistical analyses were carried out using Matlab (Mathworks, Inc., Natick, MA) and Prism 5 (GraphPad, San Diego, CA). The primary variable of interest was the percent preference for the large reward on choice trials. The effect of each drug on delay discounting behavior was examined using two-way repeated measures ANOVAs (factors of drug and delay). Dunnett’s test was used for post hoc comparisons between each drug dose and vehicle administration. Response omissions and latency to lever press were analyzed using one-way ANOVAs. For prazosin and guanfacine, t-tests were used to analyze omissions and latency to lever press in the descending delays task. Long-lasting yohimbine effects were analyzed using two-way repeated measures ANOVA (factors of delay and behavioral session).

Results

Effects of yohimbine on reward preference

Rats were first trained on a delay discounting task in which the delay period preceding the delivery of the large reward increased within the session (Fig. 1a, ascending delays task). Preference for the large reward was high when the delay was short (0.5 seconds) and decreased as the delay length increased (Fig. 2a, main effect of delay, F_(4,300) = 23.10, p < 0.0001). Yohimbine administration increased preference for the large reward in a delay-dependent fashion (Fig. 2a, delay × drug interaction, F_(16,300) = 1.73, p < 0.05; no main effect of drug). Post hoc tests comparing each drug dose to vehicle administration indicated that all doses of yohimbine increased preference for the large reward when it was delayed by 20 seconds (p < 0.001 for 0.6, 1.25, and 5 mg/kg doses; p < 0.01 for 2.5 mg/kg). In addition to effects on reward choice, yohimbine administration resulted in an inverted-U shaped curve of both trial omissions and response latency (Table 2; main effect of yohimbine on omissions, F_(4,60) = 2.75, p < 0.05; and response latency, F_(4,60) = 8.52, p < 0.0001). Post hoc tests showed that yohimbine significantly decreased omissions after administration of the 2.5 mg/kg dose relative to vehicle (p < 0.05) and decreased response latency after administration of the 1.25 mg/kg dose relative to vehicle (p < 0.05).

Figure 2.

a Effect of yohimbine on preference for the large reward in the ascending delays task, b Effect of yohimbine on preference for the large reward in the descending delays task. * denotes significance vs. vehicle administration (p < 0.01). Graph symbols show mean preference ± SEM

Table 2.

Drug effects on trial omissions and response latencies. Mean (± SEM) values are shown.

	Ascending Delays Task		Descending Delays Task
Drug, dose (mg/kg)	Omissions	Lever press latencies (sec)	Omissions	Lever press latencies (sec)
Yohimbine
Vehicle	4.6(1.2)	2.1(0.1)	8.6(1.6)	2.5 (0.1)
0.6	3.2 (0.8)	1.9 (0.1)	4.2 (0.7)	1.9(0.1)***
1.25	2.3 (0.8)	1.7(0.1)*	4.3 (0.7)	2.0(0.1)*
2.5	1.7(0.7)*	1.9 (0.2)	6.1(1.3)	2.3 (0.2)
5	4.5(1.0)	2.3(0.2)	12.9 (2.5)	3.0 (0.3)
Propranolol
Vehicle	2.9 (0.8)	1.7(0.1)	10.8(1.6)	2.5 (0.2)
1	2.7(0.7)	1.7(0.2)	12.0(2.2)	2.7 (0.2)
3	2.7(0.5)	1.8(0.1)	9.6(1.9)	2.6 (0.3)
10	5.0(1.3)	2.0(0.2)	15.2(3.5)	2.7 (0.3)
Prazosin
Vehicle	4.1(1.2)	1.9 (0.2)	10.2(1.7)	2.4 (0.1)
0.5	6.1(1.0)	2.1(0.1)
1	7.6(2.1)	2.1(0.2)
2	7.7(1.8)	2.3(0.1)	15.0(3.3)	3.1(0.3)*
Guanfacine
Vehicle	4.1(0.9)	1.8(0.1)	11.1(2.9)	2.4 (0.2)
0.06	6.0(1.6)	2.1(0.2)
0.125	11.1(3.5)	2.7 (0.2)*	12.8(3.2)	2.8 (0.2)
0.25	21.2(6.0)*	3.2(0.4)***

Asterisks indicate significance relative to vehicle:

^*p < 0.05,

^***p < 0.001

The upward shift in the delay discounting curve following yohimbine administration suggested that the drug decreased impulsivity. However, this shift could also result from perseverative choice of the large reward across the behavioral session, consistent with reports of yohimbine-induced perseverative responding (Caetano 2013). To address the latter possibility, we tested yohimbine effects in a descending delays task (Fig. 1b), in which delays preceding the large reward were longest in the first block and declined in subsequent blocks.

Following training in the descending delays task, rats’ preference for the large reward increased as the delay preceding delivery of the large reward decreased (Fig. 2b, main effect of delay, F_(4,300) = 13.98, p <0.0001), demonstrating choice behavior was sensitive to changes in the order of delay length in the descending delays task. Yohimbine decreased preference for the large reward (Fig. 2b, main effect of drug F_(4,300) = 42.54, p < 0.0001), and this effect was delay-dependent (delay × drug interaction, F_(16,300) = 2.35, p < 0.01). Post hoc tests indicated that compared to vehicle, 5 mg/kg of yohimbine decreased preference for the large reward when the reward was delayed by 0.5, 10, or 20 seconds (all p < 0.001), or 30 seconds (p < 0.05), whereas 2.5 mg/kg decreased preference for the large reward at delays of 0.5 seconds (p < 0.01) and 10 seconds (p < 0.001). Yohimbine significantly altered trial omissions (Table 2; main effect of drug, F_(4,60) = 7.34, p < 0.0001), but post hoc testing showed that neither the 2.5 nor the 5 mg/kg dose significantly increased omissions. Yohimbine also significantly affected lever press latency in the descending delays tasks (Table 2; main effect of drug, F_(4,60) = 8.86, p < 0.001). Post hoc tests indicated that yohimbine significantly decreased response latency following administration of the 0.6 mg/kg (p < 0.001) and 1.25 mg/kg (p < 0.05) doses relative to vehicle administration. Together, results from the ascending and descending delays tasks suggest that yohimbine caused persistent responding on the initially preferred lever, rather than changing decision-making behavior by altering discounting rates.

Interestingly, the first dose of yohimbine tested in the descending delays task produced unexpectedly long-lasting effects in a small subset of rats (Fig. 3; 4 rats of 16 tested). In these animals, baseline (drug-free) preference for the large reward was high when delays were short, as expected, when tested with both ascending (Fig. 3a) and descending delays (Fig 3b). Furthermore, in sessions occurring immediately after yohimbine administration (doses of 1.25 and 2.5 mg/kg in two rats each), this subset of 4 animals showed persistent choice of the initially preferred lever associated with the small reward, consistent with drug-induced perseverative behavior (Fig. 3c). These patterns of behavioral choice were similar to those shown by other rats tested in this paradigm (Fig 3a–c). However, on subsequent (drug-free) training days, this subset of rats showed greater preference for the large reward at the beginning of the session (when delays were longest) and decreased preference as the session progressed, despite decreasing delay lengths (Fig. 3d–f; main effect of behavioral session, F_(4,60) = 2.62, p < 0.05; delay × day interaction, F_(16,60) = 2.90, p < 0.01). Post hoc tests showed decreased preference for the large reward during the fifth block of trials on days 1 and 3 post-yohimbine (relative to pre-yohimbine, p < 0.01). With extended retraining, rats returned to the expected pattern of reward choice, with greatest preference for the large reward when delays were short (Fig. 3g). Thus, in this subset of rats, a single yohimbine injection during testing in the descending delays task resulted in a long-lasting reversion to the behavioral pattern of responding seen in the initially learned ascending delays task. By contrast, other rats tested in this paradigm showed no persistent effects of yohimbine administration (Fig. 3d–f).

Figure 3.

Long-lasting yohimbine effects, a Preference for the large reward in the ascending delays discounting task, b Preference for the large reward in the descending delays task on the day preceding yohimbine administration, c Effect of acute administration of yohimbine on reward preference in the descending delays task, d–f Yohimbine administration had long-lasting effects on choice behavior in a subset of rats when tested in subsequent drug-free sessions, g Delay discounting behavior after extensive retraining in the descending delays task. * denotes significance vs. pre-yohimbine preference within a specific block (p < 0.01). Symbols indicate mean preference ± SEM

Figure 4.

a Effect of propranolol on preference for large rewards in the ascending delays task, b Effect of propranolol on preference for large rewards in the descending delays task. * denotes significance for 10 mg/kg propranolol vs vehicle (p < 0.05). Symbols indicate mean preference ± SEM

Effects of propranolol on reward preference

Propranolol administration increased preference for large, delayed rewards in the ascending delays task (Fig. 4a, main effect of drug, F_(3,225) = 8.41, p < 0.0001). Post hoc tests indicated a significance difference between vehicle and the 10 mg/kg dose of propranolol at the 30-second delay (p < 0.05). In order to determine if propranolol effects could be ascribed to perseverative responding, drug effects were also examined during performance of the descending delays task. Propranolol had no effect on preference for the large reward, and no interaction was present (Fig. 4b). Propranolol did not alter trial omissions or response latency (Table 2).

Effects of prazosin and guanfacine on reward preference

Neither prazosin nor guanfacine altered reward preference in the ascending delays task (Fig. 5). Moreover, neither drug altered reward choice in the descending delays task (data not shown). Prazosin administration had no significant effects on omissions but significantly increased lever press latency in the descending delays task (Table 2; t(15) = 2.61, p < 0.05), but not in the ascending delays task. Guanfacine increased omissions (main effect of drug, F_(3,60) = 4.57, p < 0.01) and response latency in the ascending delays task (main effect of drug, F_(3,59) = 6.01) but had no effect on response measures in the descending delays task.

Figure 5.

a Effect of prazosin on preference for the large reward in the ascending delays task, b Effect of guanfacine on preference for the large reward in the ascending delay task. Symbols indicate mean preference ± SEM

Discussion

The primary goal of the present experiments was to test the effect of the pharmacological stressor yohimbine on delay discounting behavior. A secondary goal was to characterize the effects of other manipulations of noradrenergic signaling on this behavior. Our main finding is that while yohimbine administration substantially altered decision-making behavior, this change was likely caused by drug-induced increases in perseverative behavior, rather than changes in delay discounting. As yohimbine causes stress-like physiological and behavioral effects, these findings may have implications for understanding stress-induced relapse to drug seeking, which we discuss further below.

In both the ascending and descending delays tasks, rats exhibited expected patterns of delay discounting: rats normally preferred large rewards when delays preceding those rewards were short, but preferred small rewards when delays preceding the large rewards were long. The effects of yohimbine strongly depended on the order in which delays were presented during the session. In the ascending delays discounting task, in which delay lengths increased over the course of the session, yohimbine increased preference for large rewards delivered after a 20 second delay. In contrast, in the descending delays task, in which delay lengths were initially long and then decreased over the course of the session, yohimbine had the opposite effect, resulting in decreased preference for the large reward. In each case, drug effects served to increase perseverative choice of the lever associated with the initially preferred outcome (the large reward in the ascending delays task and the small reward in the descending delays task). These results show that while yohimbine had significant effects on decision making, they likely arose through drug-induced inflexible behavior, rather than changes in delay discounting, as initially hypothesized.

Interestingly, amphetamine effects on decision-making are also dependent on the order in which reward contingencies are experienced. Amphetamine administration increases choice of an initially preferred large reward in both risk- and delay-based decision-making when the relative value of that reward decreases over the course of a session (corresponding to decreased probability of reward receipt and increased delay, respectively), but has the opposite effect when relative reward value increases within the session (St Onge et al. 2010; Tanno et al. 2013), but see (Slezak and Anderson 2009). Similar effects on risk-based decision-making occur after pharmacological inactivation of the medial prefrontal cortex (St Onge and Floresco 2010). Importantly, Floresco and colleagues provide evidence that these behavioral changes are not easily explained by simple response perseveration (i.e., repetition of a specific lever-pressing behavior) but rather may arise from perseveration in reward valuation, or failure to appropriately update internal representations of value when reward contingencies change. Response perseveration after yohimbine administration may arise from similar disruptions in neural function subserving reward valuation, particularly attentional mechanisms. This possibility fits well with a proposed role for phasic noradrenergic signaling as an “interrupt” signal that occurs in response to unexpected changes in reward contingencies and promotes adaptation of reward-directed behaviors (Dayan and Yu 2006). Tonic elevations in noradrenergic signaling after yohimbine administration would be expected to disrupt this phasic signal, and through this mechanism, could contribute to perseverative behavioral responses.

Restraint stress has recently been reported to selectively alter risk- but not delay-based decision-making (Shafiei et al. 2012). It is not clear what accounts for differences between these findings and our own results, though the different stressors used may importantly contribute. Along these lines, yohimbine administration alters some reward-seeking behaviors that are unaffected by restraint stress (Shaham et al. 2000). Our findings are consistent with evidence that a variety of stressors affect measures of behavioral and cognitive flexibility. Acute footshock stress has been shown to cause perseverative responding in mice during performance in the Morris water maze (Francis et al. 1995). Moreover, acute psychological stress in humans impairs cognitive flexibility in tasks requiring selective attention and problem solving (Alexander et al. 2007; Plessow et al. 2011). Convergent evidence suggests that NA signaling in prefrontal cortices importantly contributes to these stress effects (Arnsten 2009). Infusion of yohimbine into the medial prefrontal cortex of rats increases error perseveration, resulting in impaired ability to flexibly adjust performance in a delayed response task (Caetano 2013). These effects may arise through activation of low affinity αl-adrenergic receptors, which causes perseverative responding in tasks requiring working memory performance (Arnsten et al. 1999; Birnbaum et al. 1999). Thus, elevated noradrenergic signaling through α1-adrenergic receptors in the prefrontal cortex could contribute to the yohimbine-induced perseverative behavior we report here.

Inflexible, perseverative behavioral responding may be caused by stress-induced recruitment of neural circuits underlying habit-driven behaviors, at the expense of those serving more flexible goal-directed responding. Goal-directed behavior is defined by behavioral sensitivity to changes in reinforcer value (Yin et al. 2008). Experimentally, goal-directed behavior is often tested by devaluation of a normally preferred reinforcer, either by feeding to satiety or (in rodent models) through pairing with an aversive agent. Goal-directed behavior is evident by reduced performance of the action associated with the devalued outcome. Recent studies in human volunteers have shown that cold pressor stress renders individuals insensitive to outcome devaluation, leading to continued responding for the devalued reward (Schwabe et al. 2011b; Schwabe and Wolf 2009; 2010). These findings suggest that acute stress switches neural control of behavior away from circuits promoting goal-directed performance and toward those controlling habit-driven behavior. We speculate that a similar shift toward habit-driven behavior may contribute to yohimbine-induced changes in decision making, given that yohimbine significantly attenuated rats’ sensitivity to changing delay lengths. This hypothesis requires experimental testing, particularly as recent human and rodent studies suggest that yohimbine administration alone may not be sufficient to promote habit-driven behavior (Braun and Hauber 2013; Schwabe et al. 2010; 2012).

Yohimbine altered decision-making in both the ascending and descending delays tasks, but the effects were more pronounced in the latter task. It is possible that previous drug experience may have increased rats’ sensitivity to yohimbine effects during testing in the descending delays task. Some investigators have reported locomotor sensitization after repeated yohimbine treatment (Schroeder et al. 2003), but this has not been a consistent finding (Jimenez-Rivera et al. 2006). Moreover, yohimbine effects on plasma corticosterone and reward-seeking behaviors are not sensitized by repeated administration (Johnston et al. 1988; Simms et al. 2011). An alternative possibility is that yohimbine differentially affects choice behavior when perceived reward value progressively decreases (in the ascending delays task) vs. increases (descending delays task). Framing effects (i.e., potential gains vs. losses) importantly influence human decision-making (Tversky and Kahneman 1981), and acute stress has been reported to have selective effects on decisions made in the gain vs. loss domains (Pabst et al. 2013; Porcelli and Delgado 2009). It is not clear if perceived gains vs. losses contribute to rat decision-making in the ascending vs. descending delays tasks. This idea merits further exploration, however, given evidence that affective state (anxiety) differentially affects choice strategies in rats responding to negative vs. positive outcomes (de Visser et al. 2011).

Yohimbine administration unexpectedly caused long-lasting changes in decision making in a small subset of rats in the descending delays task. In these rats, choice behavior in the days following yohimbine administration shifted toward a paradoxical preference for the large reward at long delays and subsequent preference for the small reward at short delays, a choice strategy that served to increase delays while decreasing the amount of reward earned. This pattern of behavior was striking both in its aberration from more typical delay discounting behavior, but also in that it resembled the response pattern previously learned in the ascending discounting task. In that task, initial preferences were higher for the large reward (when delays were short) and subsequently shifted toward the small reward (when delays were long). A possible explanation for these long-lasting yohimbine effects is that the drug impaired recall of more recently learned patterns of responding appropriate for the descending delays task and caused reversion to patterns of behavioral responding previously learned in the ascending discounting task. Mechanisms contributing to this yohimbine effect require further investigation to test this hypothesis; however, it raises the possibility that stress may promote habit-driven behavior in two ways – by acutely decreasing sensitivity to changes in outcome value (i.e., insensitivity to changing delay length) and by diminishing recall of recently learned action-outcome associations, causing a reversion to previously and perhaps better learned action-outcome contingencies. Notably, similar mechanisms may contribute to yohimbine-induced reinstatement of drug-seeking, in which animals revert to previously learned patterns of drug-seeking, rather than more recently learned extinction behaviors (e.g., Le et al. 2005).

Administration of the β-adrenergic receptor antagonist propranolol increased preference for large rewards in the ascending delay discounting task. No significant changes were observed in the descending delays task. Thus, there may be a modest effect of the drug in decreasing impulsivity; however, this interpretation remains somewhat equivocal. Propranolol has been shown to block motor impulsivity induced by psychostimulant administration in animal models (Milstein et al. 2010) and to reverse stress-induced decision making deficits in abstinent heroin addicts (Zhang et al. 2011). Thus, β-adrenergic signaling may affect decision making primarily under conditions in which elevated noradrenergic signaling is present. The absence of an explicit stressor may thus account for the modest effects of propranolol on decision making in our study.

We found no effects of the αl-adrenergic receptor antagonist prazosin or the α2-adrenergic receptor agonist guanfacine on performance in the delay discounting task. Consistent with our findings, previous studies suggest that prazosin has little effect on motor impulsivity (Koskinen et al. 2003; Liu et al. 2009; Milstein et al. 2010; Roychowdhury et al. 2012). While guanfacine has been reported to decrease impulsive choice in primates (Kim et al. 2012), our results are consistent with those from a recent study showing that guanfacine had no effect on decision making in rats (Pardey et al. 2013). As for propranolol, prazosin and guanfacine may have more impact on decision-making behaviors under conditions in which elevated noradrenergic signaling is present, as after exposure to acute stress (Fox et al. 2012; Le et al. 2011; Manion et al. 2007; Nikiforuk 2013).

Our results suggest that caution is warranted in interpreting drug-induced changes in task performance in delay discounting tasks. Drug-induced changes in impulsivity cannot be readily dissociated from changes in perseverative behavior if analysis is confined to performance in the ascending discounting task. For this reason, it may be prudent to employ both ascending and descending delays tasks in assessing changes in decision-making behavior.

The present results may be relevant to mechanisms underlying stress-induced drug seeking and consumption. Acute stress has long been recognized as a factor importantly contributing to relapse to drug taking (Brownell et al. 1986). In controlled clinical settings, acute stress triggers craving for cocaine, alcohol, heroin, and tobacco (Fox et al. 2007; Jobes et al. 2011; Preston and Epstein 2011; Sinha et al. 1999; Umhau et al. 2011), and high levels of stress-induced craving are associated with poor treatment outcomes (Higley et al. 2011; Sinha et al. 2011). Here, we report that the pharmacological stressor yohimbine decreased behavioral flexibility and promoted perseverative choice of an initially preferred reward, a finding suggestive of stress-induced promotion of habitual behaviors at the expense of reward-directed behaviors. Acute stress might thus contribute to relapse by promoting well-learned, habit-driven behaviors associated with drug seeking (Schwabe et al. 2011a).