Acute Stress Improves Long-Term Reward Maximization in Decision-Making under Uncertainty

Abstract

Acute stress influences reward-seeking tendencies and risky decision-making. However, it is unclear how acute stress influences decision-making in situations in which individuals must learn to either maximize long-term or immediate rewards from experience. Consequently, this study sought to investigate whether acute stress enhances salience of small, immediate or large, delayed rewards on decision-making under uncertainty. The Socially Evaluated Cold Pressor Task (SECPT) was used to induce acute stress. Participants in Experiment 1 (N=50) were exposed to either the SECPT or a warm-water control condition and then completed a decisionmaking task in which participants needed to learn to forego immediate rewards in favor of larger delayed rewards. The results demonstrated that acute stress enhanced decisions that maximized long-term, large rewards over immediate, small rewards. Experiment 2 (N=50) included an assessment of salivary cortisol. Results replicated the behavioral findings in Experiment 1 and demonstrated that the acute stress manipulation increased salivary cortisol, thus providing a potential physiological mechanism for these results. This work suggests that moderate acute stress can improve decision-making under uncertainty that depends on learning to maximize long-term rewards from experience.

1. Introduction

In situations in which the outcome of one’s decision is uncertain and the environment changes dynamically, elucidating an optimal decision strategy may be quite difficult. People often draw upon their prior experience, comparing the outcomes of those previous choices to future possibilities and searching for ways to maximize the value or payoff of their choices. Such decisions can entail both immediate and long-term effects, such as a choice that appears more rewarding at the moment and another option that may be a better choice in the long-run. For instance, a college student must decide whether to take a job after graduating or attend graduate school. While taking a job immediately might result in a larger immediate payoff, attending graduate school may lead to more income earned across one’s lifetime. One critical factor that may influence how individuals learn and make decisions under uncertainty is stress (Lighthall et al., 2013; Petzold et al., 2010). Given the far-reaching consequences that one’s choices can engender, it is important to understand the effect that stress elicits on the quality of such decisions that need to be made in uncertain situations. Consequently, the purpose of the present study is to determine how acute stress influences decision-making under uncertainty that entails maximizing long-term versus immediate reward.

1.1. Decision-Making under Risk vs. Uncertainty

The way in which individuals glean information about choices can be classified as either decision-making under risk (i.e., description-based) or decision-making under uncertainty (i.e., experience-based; Hertwig et al., 2004). In decision-making under risk, choices have explicit, known probabilities associated with them, such as a choosing between an option that has a 100% chance of yielding $1 versus an option that offers a 50% chance of $3. In contrast, decision-making under uncertainty entails learning the values of options by experiencing the outcomes they offer. Thus, the consequences of these decisions must be learned through prior feedback from each option. The decision between receiving a small immediate reward over a larger, delayed reward can be represented in these terms as well. For example, in delay discounting (Richards et al., 1999), which is a type of decision-making under risk task, individuals make hypothetical decisions between receiving a large amount of money available after a time delay (i.e., receiving $10 after 30 days) or a smaller amount of money available immediately (i.e., receiving $2 now). Such decisions between immediately rewarding and long-term delayed rewards can also be made through experience in which individuals learn about possible rewards by sampling each option and learning the consequences of each option over time (Worthy et al., 2014). Thus, the decision to choose options that maximize immediate versus delayed rewards may vary depending on how the information about those choices is acquired—either under risk or under uncertainty.

Extensive previous research demonstrates the presence of the description-experience gap (Hertwig & Erev, 2009) in which individuals make drastically different choices depending on whether information is presented from description or learned through experience (e.g., Lejarraga, 2010; Rakow & Newell, 2010). One rationale for these differences is the nature of the decisions themselves. When engaging in decision-making under uncertainty, individuals must utilize a broader range of cognitive processes, including memory, learning, and updating (Rakow & Newell, 2010). Such decisions also entail an affective component as individuals have to learn from positive and negative outcomes under uncertain conditions (Loewenstein & Lerner, 2003; Hochman et al., 2010). Because of these distinctions, it is reasonable to expect that acute stress could affect decision-making under uncertainty differently from decision-making under risk.

1.2. Mechanisms of Stress on Decision-Making

The neurobiological mechanism by which acute stress can affect decision-making hinges on the activation by two systems: the sympathetic nervous system and the hypothalamus-pituitary-adrenal (HP A) axis (Starcke & Brand, 2012). Activation of the sympathetic nervous system occurs quickly after the onset of the stressor and leads to an increase in heart rate and blood pressure (e.g., Breier et al., 1987). In contrast, activation of the slower HPA axis stress pathway stimulates the release of cortisol, a glucocorticoid, from the adrenal cortex (Dickerson & Kemeny, 2004). Cortisol can then bind to glucocorticoid receptors in the prefrontal cortex (PFC), hippocampus, and amygdala (de Kloet et al., 2005). As the PFC is critical to decisionmaking, cortisol can influence decisions by altering processing in this brain region (Arnsten, 2009; Kalivas & Duffy, 1995; Starcke & Brand, 2012). Furthermore, acute stress may increase dopamine release in the PFC and striatum (Arnsten, 2009; Deutch & Roth, 1991), which may in turn increase reward salience (Mather & Lighthall, 2012). Thus, acute stress can influence decision-making by altering corticostriatal-dependent cognitive processing and reward sensitivity.

1.3. Stress and Decision-Making under Risk

Numerous studies have been conducted on decision-making in risk contexts. In particular, acute stress has been shown to affect risky decision-making on the Game of Dice Task (GDT), a decision-making under risk task that provides rules for gains and losses associated with the dice stimuli (Starcke et al., 2008). Following a stress induction, participants choose more “high-risk, high-reward” options that result in disadvantageous performance on the task compared to controls (Pabst et al., 2013; Starcke et al., 2008). Other work using lottery-based decision-making has found a similar effect in which stress enhances risk-taking for rewards (Buckert et al., 2014). On delay discounting tasks, acute stress had been shown to lead to greater discounting rates (Kimura et al., 2013). Following an acute stressor, individuals prefer small, immediate rewards (i.e., receive $2 now) over large, delayed rewards (i.e., receive $10 after 30 days) compared to controls. Moreover, a recent meta-analysis concluded that acute stress enhances risk taking during decision-making (Starcke & Brand, 2016). Overall, previous work suggests that elevations in stress can increase risky choices and preference for immediate rewards in decision-making under risk.

1.4. Stress and Decision-Making under Uncertainty

In addition to decision-making under risk contexts, previous research has also been aimed at how stress influences risk preference in decisions under uncertainty. Specifically, studies that have used the Balloon Analog Risk Task (BART) have observed gender differences in the way that acute stress influences risky choices. Males tend to make more risky decisions under uncertainty following acute stress, while females make fewer risky choices (Lighthall et al., 2009; Lighthall et al., 2011). Similarly, there are also gender differences on the Iowa Gambling Task, a widely used paradigm that assesses risk preference and sensitivity to gains and losses (Bechara et al., 1994). After undergoing an acute stressor, moderate increases in cortisol in males leads to more disadvantageous option selections in males, while females choose more advantageous options (Preston et al., 2007; van den Bos et al., 2009). Thus, previous research indicates that in decision-making under uncertainty, stress enhances risk-taking for high potential reward options in males (Putman et al., 2010; Starcke & Brand, 2012).

Beyond the risk domain, acute stress also impacts how individuals learn about the values associated with decision options. Previous research using probabilistic reinforcement learning tasks have shown that moderate stress improves learning from previously rewarding outcomes and positive feedback (Lighthall et al., 2013; Petzold et al., 2010). This effect is in line with the “stress triggers additional reward salience (STARS)” hypothesis, which proposes that stress-induced dopamine increases change values associated with decision options (Mather & Lighthall, 2012). According to the STARS hypothesis, acute stress increases reward salience which in turn enhances learning from positive outcomes. Individuals must frequently rely on past experiences and outcomes in decision-making, and these prior decisions can involve positive or negative associations. However, while learning to choose an option with the highest chance of yielding a positive outcome in probabilistic learning tasks represents one type of choice, decision-making does not always have an “all-or-nothing” outcome. While the STARS hypothesis has been tested in probabilistic decision contexts, it is unclear how this hypothesis affects other decision-making situations such as choosing between options that involve long-term versus immediate benefits.

In situations in which long-term experience and learning from positive feedback are beneficial to performance, moderate acute stress may improve decisions under uncertainty that maximize long-term rewards over immediate gains. In support of this notion, previous researching using a similar decision-making under uncertainty task as the one employed in the present study found that young adults made better decisions under social pressure than controls (Cooper et al., 2013). Thus, it is reasonable to predict that this effect may also be observed following a stress induction.

1.5. Current Study

As demonstrated by prior work on decisions under risk and uncertainty, whether acute stress leads to optimal or suboptimal performance depends on the task and situation (Starcke & Brand, 2012). In the present study, we seek to investigate how acute stress, stimulated using the SECPT, influences decision-making under uncertainty in situations in which one can learn to maximize long-term rewards over immediate rewards. The decision-making task utilized in the present study differs from previous paradigms used to study the effects on stress on decisionmaking (Starcke & Brand, 2016) in two distinctive ways. First, the decisions that participants learn to make in the present study either maximize immediate or long-term rewards. Previous research has only examined this reward distinction in contexts of decision-making under risk using delay discounting paradigms (Haushofer et al., 2013; Kimura et al., 2013; Lempert et al., 2012). In such paradigms, participants choose from receiving a smaller immediate reward over a larger reward provided after a given period of time ranging from a few minutes to several months or even years (e.g., Richards et al., 1999; Worthy et al., 2014). Secondly, this decision-making task encompasses a choice history-dependent reward structure. The payoffs on each trial of the task depend on the sequence of choices that an individual made on prior trials. This is in contrast with choice-history independent tasks, such as the Iowa Gambling Task, where the outcomes are determined by a set schedule and are independent of the choices made on previous trials.

The task in this study is designed to reflect real-life decision-making situations in which the consequences of future decisions depend on those made previously. Moreover, this decision-making paradigm assesses participants’ ability to discern the optimal decision strategy when each choice affects both the immediate and delayed rewards received from each option and reward values change based on participants’ prior decisions (Gureckis & Love, 2009; Worthy et al., 2011; Worthy et al., 2012). Given previous research suggesting that acute stress influences males’ and females’ decision-making performance in different ways (e.g., Preston et al., 2007; Putman et al., 2010; Starcke & Brand, 2012; van den Bos et al., 2009), we further examined how gender differences may moderate the relationship between acute stress and decision-making.

Furthermore, previous research indicates that perceived stress and working memory capacity may moderate the relationship between acute stress and decision-making. Specifically, high working memory capacity has been shown to modulate the effects of acute stress on sequential decision-making (Otto et al., 2013). Additionally, high levels of perceived life stress have been shown to predict preference for small immediate rewards over larger, delayed reward on an intertemporal choice task (Lempert et al., 2012). Based on this previous research, we also included a self-report stress questionnaire and a test of working memory capacity using the operation span (OSPAN).

1.6. Hypotheses

Using evidence from prior research, competing predictions were made. On the one hand, acute stress has been shown to lead to a preference for small, immediate rewards over larger, delayed rewards in decision-making under risk (Kimura et al., 2013). Based on this work, we predicted that acute stress may enhance the salience of immediate rewards, resulting maximization of immediate rewards and ultimately poorer decision-making performance in the present task.

In contrast, acute stress has been shown to increase reward salience and improve learning from positive feedback in decisions made under uncertainty (Cooper et al., 2013; Lighthall et al., 2013; Mather & Lighthall, 2012). Indeed, the STARS model postulates that attention towards high rewards is enhanced, but long-term negative consequences are simultaneously neglected. Because this study entails a gains-only paradigm and does not incorporate losses, there are no long-term negative consequences per se. Consequently, it is possible that acute stress may heighten attention towards high rewards; in this context, this should lead to maximization of long-term rewards. Thus, acute stress may result in better learning from positive feedback, leading to increased salience of the long-term rewards. The results of this study will elucidate whether acute stress enhances salience of small, immediate or large, delayed rewards in decisionmaking under uncertainty.

2. Experiment 1

2.1. Method

2.1.1. Participants

A group of fifty undergraduate participants (33 females, M_age = 19.20, SD_age = 1.01) completed the experiment for partial fulfillment of a course requirement. Sample size determination for this experiment was based on between-groups sample sizes used in previous studies that examined the effect of stress on cognitive performance (decision-making, risky choice, or working memory). Sample sizes in these previous studies ranged from 24 – 56 total participants (Duncko, et al., 2004; Lighthall et al., 2009; 2011; Otto et al., 2013; Pabst et al., 2013; Petzold et al., 2010). Individuals were excluded from study participation if they were under the age of 18, had major injuries or surgeries to either of their hands, had any current cuts or minor injuries to their hands, were currently using drugs or psychoactive medication, or had been diagnosed with diabetes, blood pressure problems, neurological problems, cardiovascular problems, or circulatory problems. Participants were randomly assigned to either the stress (n=25; 15 females) or control (n=25; 18 females) condition.

2.1.2 Measures

2.1.2.1. Perceived Stress Scale.

The 10-item Perceived Stress Scale (PSS-10) assesses the extent to which individuals perceive events in their life as stressful (Cohen et al., 1983; Cohen & Williamson, 1988). The response scale ranged from 0 (Never) to 4 (Very Often).

2.1.2.2. Operation Span Working Memory Task.

To measure working memory capacity, the operation span (OSPAN) task was employed (Turner & Engle, 1989). Participants viewed a math problem (i.e., (2 * 5) + 4)) for two seconds, and then a new screen with a number (i.e., 15) appeared. Participants must respond whether the number on the screen correctly or incorrectly answered the math problem they previously viewed by responding “true” or “false”. After making a response, participants received feedback and then a letter appeared. After 3 – 7 math problem and letter trials, participants were asked to recall the letters they viewed in order. Participants were instructed to both maximize accuracy in letter recall performance and maintain a math accuracy score of at least 85%. Thus, participants had to both correctly answer the math problems while also maintaining a string of letters in working memory. The OSPAN contained 75 total trials. In line with previous research, working memory capacity was computed as the total number of letters recalled in the correct position (Unsworth et al., 2005).

2.1.2.3. Socially Evaluated Cold Pressor Task.

The SECPT was utilized to elicit a stress response (Schwabe et al., 2008). This paradigm has been extensively used in previous stress studies (e.g., Duncko et al., 2009; Schwabe & Schächinger, 2018; Schwabe & Wolf, 2009; 2010a; 2010b; 2011). It has reliably been shown to substantially activate the sympathetic nervous system and moderately increase HPA axis reactivity and, consequently, cortisol levels (McRae et al., 2006; Schwabe et al., 2008). Individuals submerged their right hand in ice water (0°C – 2°C) while staring into a camera. Participants were informed that the video footage would later be analyzed for facial expressions. A female experimenter also sat in the room facing them. Participants were asked to try to keep their hand submerged in the water for as long as possible to a maximum of three minutes, but that they could remove their hand at their discretion (Schwabe et al., 2008). Individuals in the control condition were asked to place their right hand into lukewarm water (35°C – 37°C) for three minutes, and neither a camera nor an observing experimenter was in the room with them.

2.1.2.4. Uncertainty-Based Decision-Making Task.

Participants completed a two-choice history-dependent decision-making task that has been extensively utilized to assess reward-based decision-making that depends on learning from prior choices (Byrne et al., 2015; 2016; Byrne & Worthy, 2013, 2015; Otto & Love, 2010). One of the options, the Increasing option, offered small immediate rewards but the value of the rewards for both options increased as it was selected more frequently, making it the optimal choice in the task. In contrast, the Decreasing option offered more points on each immediate trial compared to the Increasing option, but point values decreased over time as the Decreasing option was selected more often. The Increasing option had a possible range of 30 - 80 points, while the points for the Decreasing option ranged from 40 – 90 points. Point values increased or decreased by five-point increments. Figure 1a shows the rewards given based on the number of times participants had selected the Increasing option over the past ten trials. Point values began at 55 points for the Increasing option, and 65 points for the Decreasing option. If the Increasing option was selected, then the reward value increased to 60 points, then 65 points and so forth until the deck value reached 80 points. Once 80 points were reached, the Increasing option repeatedly gave a value of 80 points until the Decreasing option was selected. Thus, if participants selected the Increasing option on all of the ten previous trials, then they would receive 80 points, the maximum value for that option. In contrast, if the Decreasing option was chosen, then the initial 65 points decreased to 60 points if selected again and continued to decrease to a minimum value of 40 points. Once 40 points were reached, participants repeatedly earned 40 points, the minimum value for that option, if they continued to select the immediately rewarding Decreasing option. Table 1 depicts the specific pattern of points earned for the Increasing and Decreasing options.

Figure 1.

(a) Decision-making task reward structure. Rewards were a function of the number of times participants selected the Increasing option over the previous ten trials. If participants selected the Increasing option on all ten of the previous trials, then they would be at the right-most point on the x-axis. If they selected the Decreasing option on all ten of the previous ten trials then they would be at the left-most point on the x-axis. There is a consistent 10-point difference in rewards given for the Increasing (dotted line) and Decreasing (solid line) options. (b) Sample screenshot of the dynamic decision-making task with foregone reward feedback in which participants were shown what they “would have got” had they selected the other option.

Table 1.

Pattern of points earned in the decision-making under uncertainty task for the Increasing and Decreasing options

Increasing Option Selections		Decreasing Option Selections
Increasing Option	Decreasing Option	Increasing Option	Decreasing Option
55	65	55	65
55	65	55	65
60	70	50	60
60	70	50	60
65	75	45	55
65	75	45	55
70	80	40	50
70	80	40	50
75	85	35	45
75	85	35	45
80	90	30	40
80	90	30	40
80	90	30	40

Note. Pattern of points earned in the decision-making under uncertainty for the Increasing and Decreasing options if the Increasing option is repeatedly selected (left) or if the Decreasing option is repeatedly selected (right). Participants begin with 55 points for the Increasing option and 65 points for the Decreasing option. If the Increasing option is repeatedly selected, individuals will earn 80 points on each trial after the first ten trials. If the Decreasing option is repeatedly selected, individuals will earn 40 points on each trial after the initial ten trials. Thus, repeatedly selecting the Increasing option leads to a 40 point advantage compared to the Decreasing option. Switching between decks follows the same pattern.

The optimal strategy was to learn that selecting the Increasing option led to larger cumulative reward, even though it offered fewer points on each immediate trial, and repeatedly select that option. In order to increase the difficulty of the task, participants were presented with foregone rewards such that participants were shown the points they would have received if they had chosen the alternative option (Figure 1b). Presenting the foregone reward values was designed to bias participants toward the sub-optimal Decreasing option by showing the Decreasing option’s larger immediate reward values compared to the Increasing option (Otto & Love, 2010). The outcome measures analyzed from this task was the average proportion of optimal Increasing option selections.

2.1.3. Procedure

All testing was performed in the afternoon between 12pm – 5pm to control for diurnal rhythms in cortisol levels. All participants completed the experiment on computers with a Windows operating system using Matlab Psychtoolbox version 2.54. Upon arrival to the lab, participants spent the first five minutes providing informed consent and reading instructions on the study procedure. This time period also allowed participants to acclimate to the study environment. Next, participants completed the OSPAN followed by either the SECPT or control task, depending on randomized condition assignment. Previous research demonstrates that cortisol levels begin to rise 10 minutes after an acute stress manipulation and peak approximately 25 minutes after SECPT onset (e.g., Brown et al., 2017; Schwabe et al., 2008; Schwabe & Schachinger, 2018; Schwabe & Wolf, 2009; 2010a; 2010b). Consistent with this timeline, participants waited 10 minutes before proceeding to the task. Following the stress or control manipulation, participants first sat quietly for five minutes. During the next five-minute period as cortisol levels continue to rise, participants completed the PSS-10 questionnaire, demographics information, and read instructions for the task. Once this five-minute period had ended, participants were prompted to complete the 100-trial decision-making under uncertainty task, which takes approximately 15 minutes to perform. They were given a goal of earning at least 7,200 points by the end of the task, which was equivalent to selecting the Increasing optimal option on 80% of trials. They were not informed about the reward structure of the options or the choice-history dependent nature of the task. Upon completion of the decisionmaking task, participants were debriefed about the nature of the study. A timeline of the experimental procedure is shown in Figure 2 (excluding cortisol collection time points, which only applied for Study 2).

Figure 2.

Experimental timeline. Representation of the experimental procedure and timeline for Experiment 2. Time is represented in minutes. Experiment 1 entailed the same procedure with the exception of the baseline and post-task cortisol assessments.

2.2. Results

2.2.1. Correlational Analyses.

Because previous work has found effects of working memory (Otto et al., 2013) and perceived life stress (Lempert et al., 2012) on decision-making, bivariate correlations (rs) were computed between these measures (OSPAN and PSS-10) and decision-making performance in each condition. However, OSPAN working memory capacity was not associated with decision-making performance in the SECPT (r = .061, p = .771) or control (r = −.047, p = .822) conditions. Similarly, perceived life stress was not linked to decision-making performance in the SECPT (r = −.152, p = .468) or control (r = .028, p = .896) condition. As working measure capacity and perceived life stress were not associated with decision-making performance in this experiment, neither measure was included in subsequent analyses.

2.2.2. Effect of Stress Manipulation on Decision-Making Performance.

A 2 (Condition: Stress vs. Control) × 2 (Gender: Female vs. Male) × 4 (25-Trial Block) mixed ANOVA was conducted for the proportion of Increasing option selections during the decision-making task. The results showed that the main effect of Condition significantly influenced decision-making performance (F(l, 46) = 6.539, p = .014, η²_p = .124) such that those in the stress condition (M = 0.284, SD = 0.257) selected the Increasing option significantly more than those in the control condition (M = 0.142, SD = 0.215).

Consistent with previous research (Byrne & Worthy, 2013; 2015), Gender significantly predicted decision-making performance (F(l, 46) = 5.927, p = .019, η²_p = .114). Males (M = 0.306, SD = 0.310) selected more Increasing optimal options than females (M = 0.164, SD = 0.193). A main effect of Trial Block was also observed (F(3, 138) = 15.938, p < .001, η²_p =.257) indicating learning of the optimal strategy over time. Specifically, follow-up pairwise comparisons indicated significant differences between all blocks (ps < .05) except between Block 1 and 2 (p = 1.00).

Furthermore, a significant Condition × Trial Block interaction emerged (F(3, 138) = 6.415, p = .002, η²_p = .122; Figure 3). Follow-up t-tests in each block revealed that individuals in the stress condition selected significantly more optimal choices in block 3 (p = .025) and block 4 (p = .031). There were no significant group differences in the first (p = .966) or second (p = .057) Trial Block of the task. Furthermore, there was a significant effect of block within the stress condition (F(3, 72) = 13.301, p < .001, η²_p = .357) while this effect was not significant in the control condition (F(3, 72) = 2.708, p =.051, η²_p = .101). Notably, the effect size for the effect of block in the stress condition was over three times larger than in the control condition. This result indicates that the stress condition showed a steady increase in learning the optimal decision strategy, while the control condition showed minimal learning. The total points earned on the decision-making task are a direct function of the average proportion of Increasing optimal selections and as such are perfectly correlated, r = 1.00, p < .001. Thus, those in the stress condition both selected the optimal strategy and earned more points in the latter portion of the task than those in the control condition. The Gender × Trial Block interaction (F(3, 138) = 2.960, p = .055, η²_p = .060), Gender × Condition interaction (F(l, 46) = 0.402, p = .529, η²_p = .009), and three-way interaction (F(3, 138) = 0.859, p = .464, η²_p = .018) did not significantly influence decision-making performance.

Figure 3.

Proportion of optimal Increasing option selections in each 25-trial block in Experiment 1 by condition. Asterisks indicate significant group differences between the stress and control conditions at a threshold of p < .05. Error bars represent standard error of mean.

2.3. Discussion

In Experiment 1, individuals learned the optimal decision-making strategy better over time and earned more cumulative rewards after acute stress compared to participants in the control condition. Consistent with our second prediction supported by the STARS hypothesis, acute stress enhances learning from positive feedback, beginning in the latter half of the task. Individuals in this experiment were able to view the positive feedback from both the chosen and unchosen options (i.e., the foregone rewards). Acute stress may have led to heightened attention to high-magnitude rewards; in this context in which the delayed rewards progressively increase, this appears to have enhanced the salience of the Increasing reward option and, in turn, led to advantageous decision-making. Thus, following acute stress individuals were better able to learn that the Increasing option provided higher cumulative rewards than those in the control condition. This finding is in line with prior research using this decision-making under uncertainty task that showed that young adults tend to make more advantageous decisions under performance pressure (Cooper et al., 2013). The results of this study demonstrate that acute stress improves decision-making under uncertainty that maximizes long-term rewards.

Although Experiment 1 provides behavioral evidence that acute stress enhances choices to maximize long-term rewards in decision-making under uncertainty, some limitations should be noted. First, an assessment of female contraception use was not included, which has been shown to influence stress responses (Kudielka & Kirschbaum, 2004). Secondly, although the sample size in this experiment was in line with that of previous studies (e.g., Duncko, et al., 2004; Lighthall et al., 2009; 2011; Otto et al., 2013; Pabst et al., 2013; Petzold et al., 2010), an a priori power analysis was not performed. Furthermore, the design did not include a manipulation check as a control of whether the stress induction truly elicited stress, such as a measure of cortisol. While previous research has demonstrated that the SECPT activates the HPA axis, this experiment does not offer physiological evidence to account for the effect of stress on decisionmaking under uncertainty.

Therefore, in Experiment 2, an assessment of cortisol was included in order to assess whether activation of the HPA axis provides a physiological mechanism to account for the results of Experiment 1. Additionally, in Experiment 2 we sought to determine whether the effects of Experiment 1 could be replicated, as reproducibility and replicability have emerged as important goals in psychological science (Zwaan et al., 2018). Moreover, this replication study allowed for combining the data from Experiments 1 and 2 so that they could be analyzed together, which increases statistical power.

3. Experiment 2

3.1. Method

Based on the effect sizes observed in Experiment 1, an a priori power analysis was conducted with an effect size of f = .369, an alpha = .05 and power = 0.80 using GPower 3.1 software. Using these parameters, at least 20 participants per condition would be needed to detect an effect. Based on this power analysis we decided to recruit 50 participants, as we did in Experiment 1, and randomly assign them to one of the two between-subjects conditions. This resulted in about 86% power to detect a difference between conditions at the α=.05 level.

3.1.1. Participants

Fifty undergraduate participants (33 females, M_age = 19.78, SD_age = 1.65) completed the experiment for partial fulfillment of a psychology course requirement. Exclusion criteria was the same as described in Experiment 1. Eleven females were taking oral contraceptives. Participants were randomly assigned to either the stress (n=25; 17 females) or control (n=25; 16 females) condition.

3.1.2. Measures

Experiment 2 used the same materials (e.g., PSS-10, OSPAN, SECPT, and decisionmaking task) as Experiment 1. However, an assessment of salivary cortisol (described below) was added in this study.

3.1.2.1. Salivary Cortisol.

Saliva samples were obtained using Salimetrics SalivaBio oral swab collections devices at baseline recording and 25 minutes after the end of the SECPT or control condition. These collection time points were based on previous research indicating that this time period is when peak cortisol levels should be observed (Schwabe et al., 2008; Schwabe & Schachinger, 2018; Schwabe & Wolf, 2010a; 2010b). Saliva samples were stored at −20°C until they were analyzed. Before analysis, samples were thawed at room temperature and then centrifuged at 3000 rpm for 15 min at room temperature. Salivary cortisol concentrations were then measured by enzyme immunoassay (Salimetrics, Suffolk, United Kingdom) according to the manufacturer’s instructions. The analytical sensitivity for the cortisol assay is 0.007 μg/dL with the standard curve ranging from 0.012 to 3.00 μg/dL. Five of the samples were either contaminated with food particles or did not have both a baseline and post-task sample collected due to experimenter collection. Thus, the final cortisol sample size included 45 total samples.

3.1.3. Procedure

Before partaking in the study, participants were instructed to avoid intense exercise, eating, and drinking beverages besides water in the hour before the study. Figure 2 presents a timeline of the procedure. Participants provided their first salivary cortisol sample five minutes after arriving to the study. Participants then provided informed consent and then gave a saliva sample using Salimetrics SalivaBio Oral Swabs. The swab was placed under the tongue for two minutes, and then deposited in a test tube. All participants completed the experiment on Windows-based computers using Matlab Psychtoolbox version 2.54. After providing the initial saliva sample, participants completed the OSPAN working memory assessment and then completed the SECPT stress or control manipulation. Participants were then instructed to wait for ten minutes before continuing with the decision-making task. During this time period, participants first sat quietly for five minutes and then completed the PSS-10 and demographics questions during the second five-minute period. Next, participants completed 100 trials of the decision-making task described in Experiment 1. Eipon completion of the decision-making paradigm, participants provided a second saliva sample and were then debriefed about the nature of the study.

3.2. Results

3.2.1. Cortisol Responses.

To determine whether individuals in the SECPT condition showed an increase in salivary cortisol compared to the control condition, a 2 (Condition: Stress vs. Control) × 2 (Time: Baseline vs. Post-task) mixed ANOVA was performed. Neither the main effect of Condition (F(l, 43) = .823, p = .369, η²_p = .019) nor Time (F(l, 43) = 2.474, p = .123, η²_p = .054) was significant. There were also no overall significant differences in cortisol concentrations at baseline (t(43) = 0.570, p = .572, d = .169) or post-task (t(43) = −1.723, p = .092, d = .524) between the SECPT and control condition. Importantly, results indicated a significant interaction effect such that cortisol concentrations increased at Time 2 in response to the SECPT but not in the control condition (F(l, 43) = 4.972, p = .031, η²_p = .104; Figure 5). In the stress condition, cortisol levels significantly increased between baseline (M = 4.553, SD = .762) and post-task (M = 5.134, SD = 1.306) time points (t(23) = −2.390, p = .025, d = .543). In contrast, there was no such change between baseline (M = 4.683, SD = .764) and post-task (M = 4.583, SD = .709) time points in the control condition (t(20) = 0.586, p = .565, d = .136).

Figure 5.

Salivary cortisol concentrations in the stress and control condition in Experiment 2. Asterisks indicate a significant interaction between condition (stress vs. control) and time point (baseline vs. control) at a threshold of p < .05. The increase in cortisol levels was significant for the stress condition from baseline to post-task time points, but not for the control condition. Error bars represent standard error of mean.

3.2.2. Correlational Analyses.

Bivariate correlations were performed between decisionmaking performance, OSPAN working memory, and PSS-10 perceived stress. In line with the results from Experiment 1, neither working memory capacity (ps > .60) nor perceived life stress (ps > .60) were associated with decision-making performance in either condition. Moreover, cortisol levels alone (both baseline and post-task levels) were not correlated with proportion of Increasing option selections in either condition (ps > .10). Thus, these measures were not included in further analyses.

3.2.3. Effect of Stress Manipulation on Decision-Making Performance.

As with Experiment 1, a 2 (Condition: Stress vs. Control) × 2 (Gender: Female vs. Male) × 4 (25-Trial Block) mixed ANOVA was performed for the proportion of Increasing optimal choice selections during the decision-making task. Unlike Experiment 1, the main effects of Condition (F(3, 138) = 1.912, p = .167, η²_p = .041) and Gender (F(3, 138) = 0.447, p = 507, η²_p = .010) did not significantly influence decision-making performance in Experiment 2. However, a main effect of Trial Block was found (F(3, 138) = 10.388, p < .001, η²_p = .184), indicating learning of the optimal decision strategy across time. Follow-up tests showed that there were significant differences between all blocks (ps < .05) except between Block 1 and 2 (p = 1.00) and between Block 1 and 3 (p = 0.417).

Critically, however, the results revealed a significant Condition × Trial Block interaction (F(3, 138) = 4.681, p = .019, η²_p = .092; Figure 4). Follow-up t-tests did not show significant differences in any specific trial block (ps > .10). However, as with Experiment 1, the effect of block was significant in the stress condition (F(3, 72) = 7.201, p = .006, η²_p = .231) but not in the control condition (F(3, 72) = 3.133, p = .058, η²_p = .115). Similar to Experiment 1, the Gender × Trial Block interaction (F(3, 138) = 2.398, p = .071, η²_p = .050), Gender × Condition interaction (F(l, 46) = 1.770, p = .190, η²_p = .037), and three-way interaction (F(3, 138) = 3.00, p = .069, η²_p = .061) did not significantly influence decision-making performance.

Figure 4.

Proportion of optimal Increasing option selections in each 25-trial block in Experiment 2 by condition. No significant group differences between the stress and control conditions were observed in specific trial blocks at a threshold of p < .05. Error bars represent standard error of mean.

3.2.4. Effect of Stress Manipulation on Decision-Making Performance Combining Across Experiments 1 and 2.

Given that the design and measures were the same across Experiments 1 and 2 with the exception of the cortisol assessment, we also report the combined effect of acute stress on decision-making performance across both experiments. The purpose of this analysis was to provide a consistent account of the behavioral findings observed in both studies. With 100 total participants in the combined analysis, there is over 99% power to detect an effect at an α = .05 level.

The results of the 2 (Condition: Stress vs. Control) × 2(Gender: Female vs. Male) × 4 (25-Trial Block) ANOVA revealed a significant main effect of Condition (F(1, 96) = 7.211, p = .009, η²_p = .070) such that those in the stress condition (M = 0.256, SD = 0.248) selected more optimal options than the control condition (M = 0,150, SD = 0.219). Moreover, Gender significantly influenced decision-making performance (F(l, 96) = 4.597, p = .035, η²_p = .046), indicating that males (M = 0.267, SD = 0.266) selected the Increasing option more than females (M = 0.170, SD = 0.219). The main effect of Trial Block showed learning of the optimal strategy overtime (F(3, 288 = 26.351 , p < .001, η²_p = .215) between all blocks (ps < .01) except Blocks 1 and 2 (p = 1.00).

The results also showed that the Condition × Trial Block interaction was significant (F(3, 288) = 10.731, p < .001, η²_p = .101) and showed that those in the stress condition selected more optimal options than the control condition in Blocks 2 – 4 (ps < .05). Additionally, the interaction between Gender and Trial Block (F(3, 288) = 3.805, p =.028, η²_p = .038) indicated that males selected the optimal option more than females in the last trial block (p =.017). The Condition × Gender interaction was non-significant (F(3, 288) = 1.662, p = .200, η²_p = .017). In this combined analysis, three-way interaction significantly predicted decision-making performance (F(3, 288) = 3.295, p =.044, η²_p = .033). Follow-up tests within each condition showed that males in the stress condition selected the optimal option significantly more than females in the third (p = .025) and fourth block (p = .004). No significant gender differences in the control condition emerged (ps > .10).

3.3. Discussion

The results of Experiment 2 demonstrate that individuals in the acute stress condition selected the Increasing optimal option more over time relative to those in the control condition. Thus, those in the stress condition progressively learned the optimal decision strategy to maximize long-term rewards, while the control condition showed minimal learning across the task. These behavioral findings replicate the results of Experiment 1 and provide strong evidence that acute stress enhances decision-making under uncertainty that maximizes long-term gains. We note, however, that the effect of stress on decision-making performance was stronger in Experiment 1 than in Experiment 2. The combined effect across both Experiments indicated that acute stress both improved learning of the optimal decision-making strategy and overall performance.

The combined analysis also revealed gender differences in learning of the optimal strategy over time in the stress condition. In particular, the results indicated that stressed males selected the optimal option more than stressed females in the latter portion of the task. This finding is consistent with other work using this task that observed that males choose the larger, delayed reward option more frequently relative to females (Byrne & Worthy, 2013; 2015). Previous work using an experience-based probabilistic intertemporal choice task has also demonstrated that males tend to focus more on reward magnitude and prefer to wait longer when there is potential to obtain larger rewards compared to females (Cornwall et al., 2018). In line with this finding, the gender results from the present study suggest that stress may heighten attention for reward magnitude in males. However, prior stress research demonstrates that males tend to make more disadvantageous choices on the IGT in response to acute stress compared to females (Preston et al., 2007; Smeets et al., 2009; van den Bos et al., 2009). The findings from the present study appear to contrast these gender differences in IGT performance. Although the task in the present study and the IGT both entail decision-making under uncertainty, there are notable differences between these decision-making contexts that may account for the contrast in findings. For example, the IGT entails both gains and losses, while the present study is gains-only. Females tend to be more loss averse compared to males (e.g., Gachter et al., 2010; van den Bos et al., 2013), and stress could potentially impact this relationship between stress and loss sensitivity, but in the present study, loss aversion would not impact decision-making outcomes. Additionally, the reward structure of each task is distinct, with the present study involving a history-dependent reward structure, and the IGT presenting a fixed pattern of rewards. Future studies should be aimed at examining how stress impacts gender differences in decision-making under uncertainty in gains compared to loss contexts to further elucidate these effects.

Moreover, cortisol analyses confirmed that the SECPT acute stress manipulation significantly increased salivary cortisol levels. Consistent with previous research, this finding suggests that the SECPT manipulation may have led to activation of the HPA axis in our sample. Experiment 2 therefore extends the results of the Experiment 1 by providing a potential physiological mechanism for the behavioral findings.

4. General Discussion

The purpose of this investigation was to assess how acute stress influenced decisionmaking that entails learning the immediate and long-term consequences of each option from experience. Consistent evidence from two studies demonstrated that acute stress improves learning of the optimal decision strategy to maximize long-term rewards. Across the task, participants that were subjected to the SECPT procedure exploited the immediately rewarding, but long-term disadvantageous option less frequently than controls. This finding supports our second hypothesis that better learning from positive feedback should result in increased salience of long-term rewards and, consequently, better decision-making performance. Results from Experiment 2 suggested that the stress manipulation led to moderate increases in cortisol levels, thereby providing a potential physiological explanation for the observed differences in decisionmaking.

The findings of this study advance prior work on stress and decision-making under uncertainty in several ways. A recent meta-analysis concluded that stress detrimentally affects decision-making under uncertainty in ways that lead to disadvantageous choices, reward seeking tendencies relative to non-stress conditions (Starcke & Brand, 2016). For example, previous research shows that acute stress tends to lead to disadvantageous choices on the IGT, a decisionmaking under uncertainty task that assesses how individuals make choices in response reward and loss feedback (Preston et al., 2007; van den Bos et al., 2009; Wemm & Wulfert, 2017). However, the decision-making situation in this present study differs from prior decision-making under uncertainty tasks in three key respects. One difference is that the IGT incorporates losses in addition to rewards, while the present task focuses on a reward-based context only. A second distinction is that possible rewards in this study hinged on the choices that an individual previously made. Thus, one’s previous actions evoked consequences by influencing one’s future payoffs. To our knowledge, this is the first study to examine how stress influences such decision-making, yet such decisions are common in real-life situations. For example, the choice to study for an exam, rather socialize with friends or watch television, is not immediately rewarding, but in the long-term, this decision may improve a student’s GPA and ultimately future career prospects. In such contexts, moderate acute stress may lead to advantageous decisions that maximize long-term benefits.

Most notably, however, this task was designed to gauge how individuals learn to maximize either small, immediate or large, delayed rewards from experience. Whether decisions are presented in risk or uncertainty contexts can profoundly affect one’s choices. For example, people tend to be risk-seeking for small probability gains and risk averse for small probability losses in decisions made under risk, yet the exact opposite decision preference (e.g., risk averse for gains, risk seeking for losses) is observed for decisions under uncertainty (Rakow & Newell, 2010). Prior stress research has only compared choices for immediate or delayed rewards on decision-making under risk contexts, such as delay discounting tasks, rather than decision-making under uncertainty contexts (e.g., Kimura et al., 2013; Starcke & Brand, 2016). While stress enhances preference for small, immediate rewards in decision-making under risk contexts, our results suggest that in experience-based contexts stress may influence decision-making in a distinct way—by choosing to maximize long-term rewards over immediate rewards. Learning from positive feedback is a fundamental difference between delay discounting choices and the decision-making under uncertainty paradigm used in this study. Because acute stress enhances this type of learning (Mather & Lighthall, 2012), decision-making performance is improved. Thus, acute stress does not simply enhance preference for long-term rewards over immediate rewards; rather, it enhances learning of the optimal long-term reward decision strategy from experience.

Previous research indicates that stress may increase dopamine release in reward-related corticostriatal regions (e.g., Arnsten, 2009; Kalivas & Duffy, 1995; Rouge-Pont et al., 1993). While stress has been shown to impair PFC function on complex cognitive tasks (Arnsten, 2009), we speculate that this association between stress and PFC function may potentially be altered in reward contexts in which attending to rewards leads to advantageous outcomes. According to the STARS hypothesis, stress can influence the value of decisions through regulation of the dopaminergic system (Mather & Lighthall, 2012). Stress-induced dopamine release subsequently enhances the reward salience of different choices and improves learning from reward feedback (Lighthall et al., 2013; Mather & Lighthall, 2012). The STARS hypothesis further proposes that acute stress can result in reward-biased learning, which can enhance or hinder decision-making depending on the situation. In the present study, the effect of acute stress led to better learning of the decision values each choice provided from the reward feedback. Increases in PFC dopamine may enhance maintenance and updating of action-reward contingencies (Cohen et al., 2002; D’Ardenne et al., 2012; O’Reilly, 2006). Stress-induced PFC dopamine may augment updating of information acquired from experience, allowing for better learning of long-term decision values. Thus, in situations in which one must use past experience to choose between options that lead to large long-term rewards over small immediate rewards, moderate levels of stress can be beneficial.

Consequently, our results support the STARS hypothesis and extend its application beyond history-independent decision contexts to situations in which the consequences of future decisions hinge on prior ones. Our findings diverge from previous work with risky decisionmaking that showed that stress enhances risk-taking and leads to poorer decision-making performance on such decision-making under risk tasks (Buckert et al., 2014; Pabst et al., 2013; Starcke et al., 2008). This difference could potentially be attributed to several factors, including the way information about choices is provided (decision-making under risk compared to decision-making under uncertainty in this study), the nature of the feedback provided (both positive and negative compared to positive only in this study), or the probabilistic reward structure of these prior studies that differs from the choice history-dependent reward structure in this task. Future research should be aimed at distinguishing how stress uniquely affects these distinct aspects of decision-making.

As previous work observed that the effect of stress on decision-making depended on individual differences in working memory capacity (Otto et al., 2013), perceived life stress (Lempert et al., 2012), and gender (Preston et al., 2007; van den Bos et al., 2009), we also assessed these factors in this study. However, we did not observe any moderating effects of working memory or perceived life stress in either Experiment 1 or 2. In contrast, however, gender differences were observed such that males in the stress condition selected the optimal decision option more than females in the latter half of the task. This finding suggests that acute stress may affect males’ ability to learn from decisions made under uncertainty so that they learn to maximize long-term rewards over immediate rewards. Taken together, the results of this study uniquely demonstrate that acute stress enhances learning of long-term rewards, thus improving decision-making under uncertainty in this context.

4.1. Limitations and Future Directions

One limitation to the current study is that changes in dopamine were not measured, despite evidence that acute stress may affect dopamine release and reward-related neural circuitry (e.g., Oswald et al., 2005; Porcelli et al., 2012). Thus, we cannot definitively conclude that our results are due to dopamine increases. Further research using pharmacological or positron emission tomography (PET) methods is needed to test for the specific effect of striatal and PFC dopamine on decision-making under uncertainty.

Additional caveats to our design should also be addressed. Subjective or autonomic responses to the SECPT were also not assessed. As the salivary cortisol reactions to the SECPT were rather small and stress is a multidimensional construct (Levine & Ursin, 1991), this is a limitation of the present study. Additionally, the observed change in cortisol from baseline to post-task in the stress condition was lower than reported in previous research (e.g., Schwabe & Schachinger, 2018), which could have been due to confounding factors of human salivary cortisol responses, such as smoking or intake of medication, that were not accounted for in this study. Relatedly, by using the SECPT stress induction manipulation, we are able to assess only one intensity level of acute stress. Based on prior research, this induction elicits a moderate stress response (McRae et al., 2006; Schwabe et al., 2008). However, it is unclear how lower or higher levels of stress impact decision-making under uncertainty, and thus our effects may not generalize to all stress intensities. Thus, we cannot conclude whether the relationship between acute stress and decision-making under uncertainty is linear or curvilinear, as the Yerkes-Dodson effect might predict (Yerkes & Dodson, 1908; Teigen, 1994). Future research should consider examining the intensity of the stressor in their design. Moreover, menstrual cycle was not documented in female participants, which may have influenced salivary cortisol levels. Additionally, personality traits such as the Big Five and impulsivity were not assessed as part of this study, although prior research has linked these trait variables to decision-making under uncertainty (Byrne et al., 2015; Byrne & Worthy, 2013) and delay discounting (Kimura et al., 2013). Thus, future work should be aimed as examining how personality may modulate the effect of acute stress on decision-making under uncertainty.

It is also reasonable to consider that our studies may have lacked sufficient (<80%) power. Although we did not observe moderating effects of working memory capacity, perceived stress, or gender in either experiment, it is possible that with more power, these effects may emerge. We note, however, that the sample size per condition in both our experiments exceeded that of numerous previous studies that have examined the effect of acute stress on decisionmaking (e.g., Duncko et al., 2009; Kimura et al., 2013; Lighthall et al., 2013; Otto et al., 2013; Preston et al., 2007; Schwabe & Wolf, 2010a), and in the combined analyses there was 99% power to detect an effect. Furthermore, unlike these aforementioned studies, our design included two separate experiments. Given that the results of Experiment 2 replicated the interaction between stress condition and block found in Experiment 1, the findings are consistent in demonstrating the effect of acute stress on decision-making under uncertainty.

Conclusion

This study is the first to demonstrate that moderate acute stress improves decisionmaking under uncertainty that maximizes long-term rewards over immediate gains. Our results show support for the “stress triggers additional reward salience (STARS)” hypothesis in which stress enhances learning from positive feedback. We conclude that in experience-dependent learning environments, acute stress may be beneficial in learning to maximize long-term rewards over immediate rewards. Stress can influence how people make decisions in different situations—from routine choices to life-altering decisions. In situations in which people must rely on prior experience to make decisions that have immediate or long-term effects, such as using prior experience about consequences associated with studying for an exam or socializing with friends, a moderate amount of stress could result in a larger payoff in the long-run.

Highlights:

• We tested the effect of acute stress on decision-making under uncertainty.

• Decisions entailed learning to forgo immediate reward for larger delayed reward.

• Acute stress led to better learning of the optimal decision strategy.

• Acute stress can improve decision-making that relies on learning from experience.