Both Risk and Reward are Processed Differently in Decisions Made Under Stress

Abstract

Years of research show that stress influences cognition. Most of this research has focused on how stress affects memory and the hippocampus. However, stress impacts other regions involved in cognitive and emotional processing, including the prefrontal cortex, striatum, and insula. New research examining how stress affects decision processes reveals two consistent findings. First, acute stress enhances selection of previously rewarding outcomes but impairs selection of previously negative outcomes, possibly due to stress-induced changes in dopamine in reward-processing brain regions. Second, stress amplifies gender differences in strategies during risky decisions, with males taking more risk and females less risk under stress. These gender differences in behavior are associated with differences in activity in the insula and dorsal striatum, brain regions involved in computing risk and preparing to take action.

The word stress describes experiences that are emotionally or physiologically challenging (McEwen, 2007). Stressful experiences elicit sympathetic nervous system responses and stimulate the release of stress hormones (e.g., cortisol in humans; Sapolsky, 2004) that mobilize the body’s resources to respond to a challenge. The physiological effects of a stressful experience such as making a speech are evident not only during the event, but also often in the next hour or so (Dickerson & Kemeny, 2004). When stressors are constantly present or anxiety about potential stressors is high, stress levels may become chronically elevated. Beyond the physiological effects of stress, a substantial literature indicates that both acute and chronic stress impact cognitive function.

Most previous studies examining stress and cognition focused on stress effects on memory; effects on other aspects of cognition, including decision making, have received less attention. It is crucial to understand whether and how stress may alter decision making, as important decisions are often made under stress. For example, decisions about finances, healthcare, and social relationships are frequently accompanied by stress or cause stress. Early work on stress and decision making determined that stressors like time pressure and noise impaired decision making, resulting in decision making that is hurried, unsystematic, and lacks a full consideration of options (Janis & Mann, 1977).

Current stress and decision research builds on earlier findings by investigating the mechanisms driving stress effects on decision making, particularly for decisions involving potential incentives. Acute stress potentiates dopaminergic reward pathways in the brain (Ungless, Argilli, & Bonci, 2010), which may intensify the allure of potential gains associated with decision options. Furthermore, the core brain-body feedback loops involved in the stress response also are involved in assessing risk and reward (Bechara & Damasio, 2005). As part of this brain-body feedback system, the insula helps represent somatic states and signal the probability of aversive outcomes during risky decisions (Clark et al., 2008). Measures of cerebral blood flow indicate that both physical and psychological stress activate the insula, but differently for males and females (Naliboff et al., 2003; Wang et al., 2007).

In the following sections, we review recent evidence for two distinct effects of stress. First, stress enhances learning about positive choice outcomes and impairs learning about negative choice outcomes. This effect appears to be similar across genders, as well as age groups. Second, stress affects decision strategies differently for males and females, with behavior diverging under stress when decision making involves immediate risk taking.

Seeing STARS: Stress can make rewards gleam more brightly

What makes decisions difficult? Often, it is the challenge of weighing and integrating positive and negative aspects of decision options. Is a higher salary worth a longer commute? Is the pleasure of watching your favorite television show worth the sacrifice of staying up later to meet an assignment deadline? In addition to their immediate impact on choice, rewarding and aversive outcomes of a decision can influence future choices through learning. For instance, receiving a poor grade on the assignment might influence future time-allocation decisions.

We propose that stress alters decision value assignments because stress triggers additional reward salience (STARS). The STARS model is based on research examining how stress affects dopaminergic reward-processing brain regions. Dopaminergic regions and their target structures – such as the striatum (especially the nucleus accumbens) and orbitofrontal cortex – play key roles in representing reward value (Rangel, Camerer, & Montague, 2008). In rats, acute stress increases nucleus accumbens extracellular levels of dopamine (Abercrombie, Keefe, Difrischia, & Zigmond, 1989; Kalivas & Duffy, 1995), an effect that is mediated by cortisol (Rouge-Pont, Deroche, Le Moal, & Piazza, 1998). Stress also increases firing rates in rat midbrain dopamine neurons (Anstrom & Woodward, 2005) and long-term potentiation in dopamine neurons (Saal, Dong, Bonci, & Malenka, 2003).

In positron emission tomography (PET) studies, researchers “tag” dopamine, making it possible to determine whether stress increases dopamine in the human brain. Experiencing painful stressors increases measures of striatal dopamine among healthy young adults (Scott, Heitzeg, Koeppe, Stohler, & Zubieta, 2006; Wood et al., 2007). In addition, how much cortisol levels increase when exposed to a psychological stressor (mental arithmetic) correlates with measures of striatal dopamine (Pruessner, Champagne, Meaney, & Dagher, 2004). It appears that, similar to findings observed in rats, stress enhances striatal dopamine in humans. Consistent with this idea, stress appears to increase drug craving and often induces relapse in drug addicts (Sinha, 2009). Importantly, dopamine plays a role in desire for drugs, suggesting that stress may increase dopamine levels in drug addicts and thereby amplify the reward value attached to their drug of choice—an example of how the STARS effect can have negative consequences.

We propose that stress enhances reward salience via modulation of the dopamine system, resulting in reward-biased learning and decision making under stress, a pattern that may be beneficial or detrimental depending on the context. The following studies provide support for a STARS account of stress effects on option valuation in humans.

Influence of stress on learning about decision values

We often utilize past experiences in decision making, as previous choices may carry positive or negative associations. For this reason, it is important to consider the impact of stress on learning associations between decisions and their outcomes. Recent behavioral studies with humans suggest that stress enhances learning about positive outcomes while diminishing learning about negative outcomes (Fig. 1; Lighthall, Gorlick, Schoeke, Frank, & Mather, under review; Petzold, Plessow, Goschke, & Kirschbaum, 2010). In these studies, participants completed a probabilistic reinforcement-learning task either after experiencing an acute stressor or a no-stress control task. The reinforcement task involved learning probabilistic associations between visual cues and different types of feedback. Using trial and error, participants learned which cues were most likely to result in correct or incorrect feedback, and were asked to select the cues that gave positive feedback most often. In both studies, stress enhanced learning from positive feedback but impaired learning to avoid negative feedback.¹ The similar pattern in the two studies occurred despite different types of stress inductions. Lighthall et al. induced stress before the start of the reinforcement-learning task by having participants hold their hand in ice water (cold pressor stress), whereas Petzold et al. induced psychosocial stress by making participants anticipate, then give, a speech and also do difficult mental arithmetic in front of an audience. Of additional interest, both studies found that the effects of stress on reinforcement learning were similar for males and females. Lighthall et al. also found similar effects among older adults.

Figure 1.

Participants were asked to make repeated choices among options that probabilistically delivered positive or negative feedback (Lighthall et al., under review). If they did this learning task after experiencing an acute stressor, they were better able to later select the option that delivered the most positive feedback but were less effective at avoiding the option that delivered the most negative feedback.

The increased learning from positive outcomes under stress seen in these studies is consistent with the STARS model, which proposes increased dopamine from stress should facilitate learning reward-associated behaviors but not punishment-associated behaviors. Furthermore, cortisol levels appear to be associated with impaired avoidance learning as Lighthall and colleagues found that cortisol was related to higher rates of erroneously selecting negative feedback cues. Thus, stress responses may result in a bias toward potentially rewarding options while diminishing avoidance of negative options. Stress can also impair avoidance of previously rewarding but no longer rewarding stimuli. For example, in one study participants learned actions to obtain two food rewards, but after becoming satiated for one of the foods, only the non-stressed participants stopped performing the action to obtain the satiated food (Schwabe & Wolf, 2009), even when the stress occurred after satiation (Schwabe & Wolf, 2010). Another study also suggests that stress affects learning about rewards or losses, as participants who were anticipating giving a speech performed worse than control participants on a gambling task when feedback was given, but not when no feedback was given (Starcke, Wolf, Markowitsch, & Brand, 2008). Unfortunately, the task used did not distinguish between learning about rewards versus losses.

Gender divergence effect: Stress can amplify gender differences in risk taking

Although stress affects learning about positive versus negative outcomes similarly for men and women, research indicates a gender divergence under stress in decision strategies (Fig. 2) when people must choose between safer options (that offer lower potential gains but also lower losses) and riskier options (higher potential gains but also higher losses). In one study, participants received points for inflating a series of balloons shown on the computer screen (Lighthall, Mather, & Gorlick, 2009). The larger a balloon got, the more it was worth; but with each additional pump, there was increased risk of an explosion and loss of earnings for that balloon. To earn points, participants had to choose when to “cash out” each balloon. Half of the participants completed the cold pressor stress task about 20 minutes before playing the balloon game. Experiencing cold pressor stress before the task led males to increase risk taking (more pumps per balloon) in pursuit of greater reward, whereas stress effects were opposite for females (Fig. 3). As risk taking in stressed males did not reach a level of diminishing returns, they were able to earn more reward than their female counterparts. Similarly, gender differences in stress effects were observed by others using the Iowa Gambling Task (Preston, Buchanan, Stansfield, & Bechara, 2007; van den Bos, Harteveld, & Stoop, 2009), such that men exposedto a psychological stressor prior to the task selected card decks that offered greater reward at the cost of higher risk of losses. Selecting cards from these risky decks resulted in lower earnings overall. In a study with only males, administering cortisol increased choices of risky gambles, especially those in which there was a large probability of losing and a large possible gain (Putman, Antypa, Crysovergi, & van der Does, 2010). Similarly, formerly heroin-addicted male patients made more disadvantageous risky choices after stress than before stress, an effect that was blocked by the ß-adrenoceptor antagonist propranolol (Zhang et al., 2011). Across the various laboratory decision studies, stress enhanced males’ performance when increased risk taking was beneficial but impaired males’ performance when increased risk taking was detrimental, and vice versa for females. In addition, unlike the STARS effects outlined earlier, these gender divergence effects on decision strategies seem unrelated to learning processes. For instance, for studies conducted in our lab, gender-by-stress interactions were similar in initial and final blocks of the games.

Figure 2.

Under stress, decision behaviors diverge more for males and females than when not under stress.

Figure 3.

Experiencing an acute stressor before playing a risky decision game increased males’ risk seeking behavior but decreased females’ risk seeking behavior in Lighthall et al., (2009).

In a follow-up to Lighthall et al.’s (2009) balloon game study, participants completed either the cold pressor stress task or a warm water control task before entering a functional magnetic resonance imaging (fMRI) scanner and playing an fMRI-adapted version of the balloon game (Lighthall et al., 2011). In this adapted version, participants each played for the same total time, resulting in a variable number of balloons played. This change meant that playing more balloons more quickly was an alternate strategy to earn more money. In this version of the game, stress did not affect the number of pumps per balloon that participants made (risk taking), but instead affected their decision speed and number of balloons “cashed out.” Thus, the frequency of reward collections (“cash outs”) during the risky decision task was altered by stress in a gender-dependent manner (Fig. 4). Gender-by-stress interactions were also seen in brain activation in the insula, which has been implicated in signaling the likelihood of aversive outcomes and weighing differences in expected value in risky decisions (Clark et al., 2008; O’Doherty, Critchley, Deichmann, & Dolan, 2003; Weller, Levin, Shiv, & Bechara, 2009), and in the putamen (in the dorsal striatum), which is thought to integrate sensorimotor, cognitive, motivational, and emotional signals to help select and initiate actions and to help control habit-based behaviors (Balleine, Delgado, & Hikosaka, 2007; Balleine & O’Doherty, 2010). In both the insula and putamen, stress increased activity during the task (compared with a no-decision control) for males but decreased it for females (Fig. 5). Furthermore, activation of the dorsal striatum was strongly associated with increased reward collection rate in stressed males but not females.

Figure 4.

In an fMRI-adapted version of the balloon analogue risk task in Lighthall et al., (2011), participants could work through more balloons by pumping them faster. In the control condition, males and females completed a similar number of balloons, whereas under stress their strategies diverged.

Figure 5.

Under stress, several brain regions (A) showed significant gender by stress interactions while participants played a decision game to earn money (Lighthall et al., 2011). Using outlines (“masks”) of the structurally defined putamen and insula (B) activation was examined in these two regions individually (C). Average brain activation from these two clusters revealed that, under stress, males showed greater activity in the putamen (D) and insula (E) while playing the decision game while females showed the opposite pattern.

Conclusions

Making decisions involves interacting brain mechanisms that compute the potential value of options and adjust that value to account for uncertainty and risk. Such calculations need to be translated into action, often under time pressure. The effects of stress on these processes are beginning to be examined and initial research reveals some consistent patterns. First, stress enhances learning about positive outcomes but impairs learning about negative outcomes of choices (Lighthall et al., under review; Petzold et al., 2010), effects that may help explain how stress increases the likelihood of acquiring and maintaining drug addiction (Saal et al., 2003; Sinha, 2009). In the laboratory, these STARS effects are similar across gender and age groups, but when it comes to drug addiction, gender differences in how stress influences learning about rewards and losses may be more likely as drugs affect the stress system differently for men and women (Fox & Sinha, 2009). Second, when decisions must be made under risk and uncertainty, stress alters decision strategies – but in opposite ways for men versus women (Lighthall et al., 2009; Lighthall et al., 2011; Preston et al., 2007; van den Bos et al., 2009). In this review we focused on the effects of acute stress on decision processes; however initial findings also suggest that chronic stress or anxiety also predict individual differences in risky decision making (de Visser et al., 2010; Salo & Allwood, 2011) and baseline cortisol levels predict decision impulsivity differently for males and females (Takahashi et al., 2010).

Decisions often are made under stress. For instance, anticipating a hectic day at work may influence one’s willingness to risk speeding through a yellow light on the way to the office. Feeling stressed may also induce a bias in weighing positive over negative aspects of a job offer more heavily. The laboratory studies reviewed here provide evidence that stress affects decision making, highlighting the need for additional work to better understand the nature of these effects and their brain mechanisms.

Recommended Readings.

1. Cahill, L. (2006). Why sex matters for neuroscience. Nature Reviews Neuroscience, 7, 477-484. A broad and accessible overview of sex differences in the brain.

2. Lighthall et al. (2009). (See references.) In this study, sex differences in decision strategies become more pronounced under stress than in the control group.

3. Sinha, R. (2008). Modeling stress and drug craving in the laboratory: Implications for addiction treatment development. Addiction Biology, 14, 84-98. This article reviews work investigating the interplay between stress and addiction.

4. Ungless, M. A., Argilli, E., & Bonci, A. (2010). Effects of stress and aversion on dopamine neurons: Implications for addiction. Neuroscience and Biobehavioral Reviews, 35(2), 151-156. This is a comprehensive review of how stress affects dopamine neurons.

5. Wang et al. (2007). (See references.) This study reveals gender differences in cerebral blood flow during acute stress such that females showed more activity in ventral striatum, putamen, insula and cingulate cortex whereas men showed more activity in orbital and other regions in prefrontal cortex.