Stress attenuates the flexible updating of aversive value

Significance

A primary goal of translational affective science is to gain a more mechanistic understanding of how stress renders individuals vulnerable to maladaptive threat responses that lead to affective psychopathology. Here, we examined how exposure to acute stress influences the flexible modulation of threat response using reversal learning, a canonical assay of behavioral flexibility. We found that stress exposure led to marked deficits in updating affective responses to shifting sources of threat and that this deficit emerged from a failure to adjust learning rates to accurately reflect new patterns of aversive reinforcement. Our findings point to stress exposure as playing a causal role in reducing affective flexibility through selective learning mechanisms and have implications for healthy and clinical populations alike.

Abstract

In a dynamic environment, sources of threat or safety can unexpectedly change, requiring the flexible updating of stimulus−outcome associations that promote adaptive behavior. However, aversive contexts in which we are required to update predictions of threat are often marked by stress. Acute stress is thought to reduce behavioral flexibility, yet its influence on the modulation of aversive value has not been well characterized. Given that stress exposure is a prominent risk factor for anxiety and trauma-related disorders marked by persistent, inflexible responses to threat, here we examined how acute stress affects the flexible updating of threat responses. Participants completed an aversive learning task, in which one stimulus was probabilistically associated with an electric shock, while the other stimulus signaled safety. A day later, participants underwent an acute stress or control manipulation before completing a reversal learning task during which the original stimulus−outcome contingencies switched. Skin conductance and neuroendocrine responses provided indices of sympathetic arousal and stress responses, respectively. Despite equivalent initial learning, stressed participants showed marked impairments in reversal learning relative to controls. Additionally, reversal learning deficits across participants were related to heightened levels of alpha-amylase, a marker of noradrenergic activity. Finally, fitting arousal data to a computational reinforcement learning model revealed that stress-induced reversal learning deficits emerged from stress-specific changes in the weight assigned to prediction error signals, disrupting the adaptive adjustment of learning rates. Our findings provide insight into how stress renders individuals less sensitive to changes in aversive reinforcement and have implications for understanding clinical conditions marked by stress-related psychopathology.

To survive in a rapidly changing environment, it is not only critical to respond appropriately to signals that predict safety or threat, but also to adjust behavior accordingly as the value of these signals change. Such flexibility facilitates mental and physical health by promoting goal-directed behavior that aligns with the current state of the world. The failure to flexibly update affective responses to stimuli as their associated outcomes change can lead to maladaptive behavior such as a failure to mount preparatory responses to imminent threats, or persistent defensive responses to stimuli that now signal safety. Accordingly, diminished affective flexibility has been shown to confer vulnerability to an array of anxiety and trauma-related disorders (1, 2). An emerging body of work has shown that acute stress—and the neurophysiological changes that accompany it—can rapidly impair higher-level cognitive function and flexibility. However, the way in which transient exposure to stress affects the capacity to flexibly update aversive value as reinforcement contingencies change remains unknown. Given the ubiquitous nature of stress in everyday life and the fact that it is often in the presence of stress that we are required to track shifting sources of threat, here we provide a direct examination of how and when acute stress exposure affects the flexible modulation of aversive learning.

A canonical assay of cognitive flexibility is reversal learning (3), a paradigm that tests how stimulus−outcome representations are adaptively updated as reinforcement contingencies change. During aversive reversal learning, an innocuous cue that acquires aversive value through pairing with biologically salient events (i.e., threat) suddenly signals a safe or neutral outcome, while a previously neutral cue comes to predict threat. Importantly, the aversive reversal paradigm affords the opportunity to identify how acute stress affects the process of updating aversive value for cues as the source of threats concurrently changes, making it a particularly well-suited paradigm for examining how stress influences the learning processes that support the flexible modulation of aversive value.

Stress generates large-scale neurophysiological changes that activate the autonomic nervous system, triggering catecholamine release (e.g., noradrenaline, dopamine), and engaging the hypothalamic−pituitary−adrenal (HPA) axis, which leads to the systemic release of glucocorticoids (e.g., cortisol) (4). These stress-induced changes have regionally specific effects on the brain areas involved in aversive reversal learning, including the amygdala, striatum, and ventromedial prefrontal cortex (vmPFC). Specifically, the amygdala and striatum have been found to track threat-predictive stimuli across initial acquisition and reversal, a pattern also reflected in physiological arousal signals (5). In contrast, the vmPFC preferentially responds to the newly safe stimulus that previously signaled threat, consistent with its broader role in extinction learning (5). The neuromodulatory changes that accompany stress exposure reduce prefrontal-dependent cognitive control and flexibility (6–9), working memory capacity (10–13) and goal-directed behavioral control (14, 15). This occurs in tandem with marked changes in patterns of amygdala and striatal activation, as well as reduced functional connectivity between these regions and the vmPFC (7, 16). Collectively, these findings suggest that acute stress exposure might disrupt the learning signals that track shifting stimulus−outcome contingencies, leading to an inability to adaptively respond to new aversive reinforcement patterns.

Here, we tested how acute stress affects the flexible updating of learned threat and safety associations in the face of dynamic reinforcement contingencies. To this end, a cohort of healthy participants first underwent a standard Pavlovian threat-conditioning paradigm, during which two neutral visual stimuli each served as a conditioned stimulus (CS). An aversive electric wrist-shock was used as the unconditioned stimulus (US). Skin conductance responses (SCRs)—an assay of sympathetic nervous system arousal—were recorded throughout each learning session to capture trial-by-trial phasic changes in CS-specific threat arousal. CSs were probabilistically reinforced by the US, such that one image (CS+) coterminated with shock on a subset (33%) of trials, while the other (CS−) was never paired with shock. The next day, participants either underwent a well-validated acute stress induction (cold pressor task, CPT) or control task (Methods) before completing a reversal learning task that was identical to that of the initial learning session with the exception that the reinforcement contingencies were switched. Salivary concentrations of α-amylase, a proxy of noradrenergic response, as well as cortisol, a glucocorticoid that serves as a reliably index of HPA-axis activity, were both collected throughout each phase of the experiment (Fig. 1 for Experimental Timeline).

Fig. 1.

Experimental timeline and procedure for day 1 (acquisition) and day 2 (reversal).

We hypothesized that, while no differences in conditioned responses would emerge during initial threat learning on day 1, stress exposure on day 2 would lead to impairments in flexibly adjusting conditioned responses to new reinforcement contingencies. The extant research on stress and flexible affective control suggests two primary ways through which these impairments might occur. First, evidence suggests that stress exposure enhances affective responses and impairs emotional control when presented with stimuli such as emotional faces (17, 18), negative images (19, 20), and aversive stimuli (21–23). This suggests that stress could potentially enhance threat responses to the new CS+ that predicts shock, or prompt sustained threat responses to a newly safe cue, leading to a failure to extinguish threat responses to the new CS− (24). Second, stress has consistently been shown to impair higher cognition and flexible learning (6), suggesting that, rather than simply increased threat responses globally, stress might selectively impair the updating of aversive value. Accordingly, recent research has shown that higher trait anxiety is related to inflexible learning rates during an aversive reinforcement learning (RL) task in humans (25) and that chronic stress confers greater perceived environmental uncertainty during an aversive prediction task (26). These updating impairments could manifest as a deficit in transferring aversive value to a previously safe stimulus despite new patterns of aversive reinforcement.

To directly test these hypotheses, we examined how SCRs developed during reversal learning and whether any identified group differences were specific to the CS+ or CS–. Further, to quantify how stress affects the progressive updating of aversive value, we used an RL model and compared its fit to trial-by-trial SCRs across each learning phase (Methods). We adopted a hybrid RL model that assigns a dynamic learning rate to an otherwise standard Rescorla−Wagner model (27), in which aversive value is updated through prediction error (PE) signals that arise from discrepancies between expected and experienced outcomes (i.e., shock). The model allows value updating to be gated by the CS’s associability—a learning variable that incorporates PE signals as an index of surprise to dynamically adjust learning rates (28). Thus, for a dynamic learning rate, larger PEs increase associability and accelerate value updating, while smaller ones slow it. We reasoned that rapid stress-induced increases in dopamine and noradrenaline, which can work synergistically in the brain with later-released glucocorticoids, are well positioned to selectively disrupt aversive value updating.

Results

Physiological Threat Responses.

Conditioned threat responses (CRs) were calculated by subtracting mean SCR to the safety stimulus from that of the threat-predictive stimulus (CS+ minus CS−). To assess how learning developed over time, CRs for each phase (acquisition, reversal) were divided into four blocks (three trials per block). Fig. 2 depicts CRs for each group across each learning phase. We first confirmed that groups did not differ on day 1 in their initial acquisition of CRs. Given our exclusionary criterion (SI Methods), we would expect all participants to have successfully acquired a CR during the day 1 learning phase; however, this criteria does not preclude groups from differing in their CRs above and beyond this criteria. A repeated-measures analysis of variance (RM-ANOVA) using a between-subjects factor of group (stress, control) and within-subjects factor of time (blocks 1 to 4) confirmed a main effect of time [F_(3,219) = 24.31, P < 0.0001, η_p² = 0.25], but no effect of group and no interaction (all p > 0.05). Planned t tests confirmed that learning increased over time across groups [CRs at block 4 > block 1: M = 0.29, 95% CI [0.21, 0.36]; t₍₇₄₎ = −7.61, P < 0.000001], and that CRs did not differ between groups during any of the acquisition blocks (all p > 0.05).

Fig. 2.

Conditioned responses across sessions. Differential SCR (mean CS+ minus mean CS−) divided into four blocks during each learning stage (three trials × block). Both groups showed significantly stronger learning at block 4 of each session relative to block 1. Group differences emerged during the intermediate two blocks of day 2 reversal learning only. Error bars denote SEM.

Our primary goal was to examine whether inducing stress before reversal learning would alter the flexible learning of new cue contingencies. A group × time RM-ANOVA during the day 2 reversal learning phase revealed a main effect of time [F_(3,219) = 13.01, P < 0.000001, η_p² = 0.15], a trend toward an effect of group [F_(1,73) = 3.01, P = 0.08, η_p² = 0.04] and, critically, a group × time interaction [F_(3,219) = 2.86, P = 0.03, η_p² = 0.04]. Consistent with previous studies showing that reversal learning takes time to develop (5, 27, 29, 30), we found no group differences during block 1 when cue contingencies initially reversed [M = 0.05, 95% CI [−0.07, 0.18], t₍₇₃₎ =−0.85, P = 0.39]. We note that, since our task requires the progressive updating of CRs, we would not necessarily expect group differences to emerge immediately. Accordingly, even the control group did not show immediate updating of CRs, instead displaying increased learning over time [block 4 vs. block 1: control: M = 0.38, 95% CI [0.16, −0.59], t₍₃₆₎ = −3.65, P = 0.001; stress: M = 0.19, 95% CI [0.07, 0.31], t₍₃₇₎ = −3.30, P = 0.002].

We found marked group differences, however, in the magnitude of CRs during the two intermediate reversal blocks. Relative to controls, stressed participants showed significantly lower CRs to the newly reversed reinforcement contingencies during block 2 [M =−0.16, 95% CI [−0.33, −0.003], t₍₇₃₎ = 2.04, P = 0.04] and block 3 [M = −0.20, 95% CI [−0.34, −0.05], t₍₇₃₎ = 2.77, P = 0.007], suggesting that acute stress led to deficits in aversive reversal learning by slowing the rate at which participants flexibly adjusted threat responses. Finally, groups also did not differ during the final block of reversal learning [M = −0.12, 95% CI [−0.35, 0.10], t₍₇₃₎ = 1.07, P = 0.29], indicating that successful reversal learning was eventually achieved in both groups.

Specificity of Stress-Induced Reversal Deficits.

The CR metric used in these analyses represents differential responses between two distinct learning components (CS+, CS−). Therefore, the stress-induced CR reductions we observed during reversal learning could be driven by decreased responses to the new CS+, or increased responses to the new CS−. The former would suggest a deficit in acquiring new aversive value for a previously safe stimulus, while the latter would suggest a failure to extinguish threat responses to a newly safe stimulus. To identify which of these accounts explained our findings, we assessed whether mean CS+ or CS− responses differed between groups across the learning phases. An RM-ANOVA revealed a main effect of phase [F_(1,73) = 7.79, P = 0.007, η_p² = 0.10] and CS [F_(1,73) = 130.28, P < 0.00001, η_p² = 0.64] and a trend toward a phase × CS × group interaction [F_(1,73) = 3.06, P = 0.08, η_p² = 0.04]. Planned t tests revealed that, in the control group, the magnitude of CS+ responses did not differ across learning phases; thus we could not differentiate between the original CS+ and a (new) CS+ that previously signaled safety [M = −0.02, 95% CI [−0.17, 0.12], t₍₃₆₎ = 0.38, P = 0.71]. In contrast, the stress group exhibited CS+ responses that were significantly lower during reversal relative to initial acquisition [M =−0.14, 95% CI [−0.20, −0.07], t₍₃₇₎ = 4.41, P = 0.00009]. This dissociation only applied to the CS+, as CS− responses diminished in both groups during reversal [control: M = −0.07, 95% CI [−0.14,−0.003], t₍₃₆₎ = 2.13, P = 0.04; stress: M = −0.06, 95% CI [−0.11,−0.01], t₍₃₇₎ = 2.57, P = 0.01]. We further confirmed that a tonic shift in skin conductance levels (SCL) did not account for the pattern of results seen in the stress group during reversal learning (SI Results and Fig. S1). Collectively, this suggests that the reversal deficits induced by stress were selective to the new CS+ (Fig. 3).

Fig. S1.

Mean SCL for each group on day 1 (acquisition) and day 2 (reversal). Groups’ SCL did not differ during either learning phase but decreased overall across sessions (*P < .05).

Fig. 3.

Mean SCR for each CS across sessions. Mean SCR for each CS is depicted for day 1 (acquisition) and day 2 (reversal). The deficit in reversal learning on day 2 was primarily driven by reduced SCR to the CS+. Error bars denote SEM.

Neuroendocrine Responses to Stress.

We measured two neuroendocrine markers of stress response (α-amylase, cortisol) during each learning phase. As expected, on day 1, groups did not differ for either assay (SI Results). On day 2, we examined how these neuroendocrine concentrations (i.e., baseline and +10 and +20 min post-CPT/control task) differed between groups to confirm the efficacy of our stress manipulation (Fig. 4). An RM-ANOVA using factors of group and time of α-amylase sample revealed no main effect of time [F_(2,136) = 0.01, P = 0.99, η_p² = 0.0001], but a significant group effect [F_(1,68) = 4.91, P = 0.03, η_p² = 0.07] and a trend toward a significant time × group interaction [F_(2,136) = 2.59, P = 0.07, η_p² = 0.04]. Planned t tests confirmed that α-amylase levels did not differ between groups at baseline [M = 19.03, 95% CI [−19.35, 57.41], t₍₇₁₎ = −0.99, P = 0.33], but were higher in stressed participants before [+10 min: M = 52.22, 95% CI [16.06, 88.37], t₍₇₂₎ = 2.87, P = 0.005] and (marginally) after [+20 min: M = 29.19, 95% CI [−1.08, 59.46], t₍₆₉₎ = 1.92, P = 0.059] the reversal task.

Fig. 4.

Neuroendocrine data. Mean (A) α-amylase and (B) cortisol concentrations. Samples were attained at baseline and 10 and 20 min after the stress/control task. Error bars denote SEM.

A comparable analysis assessing cortisol concentrations revealed no main effect of time [F_(2,114) = 2.19, P = 0.11, η_p² = 0.035], but a main effect of group [F_(1,57) = 5.52, P = 0.02, η_p² = 0.09] and, critically, a time × group interaction [F_(2,114) = 3.38, P = 0.04, η_p² = 0.05]. As expected, group differences in cortisol were not present at baseline [M = −0.01, 95% CI [−0.06, 0.04], t₍₆₀₎ = 0.47, P = 0.64], but were higher in the stress group before reversal learning [+10 m: M = 0.08, 95% CI [0.009, 0.16], t₍₆₉₎ = −2.25, P = 0.03]. Paired samples t tests confirmed that only the stress group showed elevations in cortisol relative to baseline [+10 m: M = 0.08, 95% CI [0.01, 0.15], t₍₂₈₎ = 2.42, P = 0.02; +20 m: M = 0.12, 95% CI [0.02, 0.22], t₍₂₉₎ = 2.54, P = 0.02].

Neuroendocrine Effects on Reversal Learning.

We next calculated a “reversal index” reflecting the difference in the magnitude of CRs across each learning phase (mean reversal CR minus mean acquisition CR). A positive reversal index indicates greater CRs during reversal relative to acquisition, whereas a negative index indicates more robust learning during the initial learning phase. Consistent with our earlier analysis, the stress group showed a significantly lower reversal index than controls [M = −0.11, 95% CI [−0.002,−0.21], t₍₇₀₎ = −2.04, P = 0.04]. Further, the stress groups’ reversal index was negatively correlated with α-amylase change measured before (+10 m: r = −0.39, P = 0.02) and after (+20 m: r = −0.40, P = 0.02) the reversal task, suggesting that higher levels of noradrenergic activity were related to deficits in fully reversing CRs. We found a similar correlation in the control group before (+10 m: r = −0.36, P = 0.04) but not after (+20 m: r = −0.16, P = 0.38) the reversal task, suggesting that higher endogenous levels of α-amylase before the reversal task were similarly related to poor updating (Fig. S2). In contrast, changes in cortisol for each group were unrelated to individuals’ reversal index at both time points (all p > 0.05). These results suggest that deficits in adjusting CRs to new patterns of aversive reinforcement were related to greater tonic noradrenergic activity.

Fig. S2.

Scatterplots depicting the linear relationship between α-amylase change from baseline and the reversal index for each group. Stressed participants showed a negative correlation both 10 and 20 min after the stressor, suggesting higher α-amylase was related to deficits in aversive value updating relative to initial learning. Control participants showed a similar relation to α-amylase before reversal learning, but no relation after.

Computational Model Fitting.

To more precisely characterize the updating process that would give rise to our effects, we next fit SCRs during each learning phase to our RL model (Methods) to identify learning parameters that might be selectively altered by stress exposure. By introducing a scaling factor (ρ) during reversal learning, we were able to estimate changes specific to the stress manipulation regarding how individuals adaptively weighed PE signals during reversal. Since associability is informed by the magnitude of unsigned PE signals, this scaling factor allowed us to identify changes in associability as a function of stress exposure. We hypothesized that the ρ estimated individually for stressed subjects (ρ_stress) and for control subjects (ρ_control) would differ during reversal learning only. Consistent with this prediction, a multivariate ANOVA assessing group differences in the weighting of PE signals (η on day 1, and ρ on day 2) revealed a significant effect of group on the two parameters [F_(1,73) = 4.05, P = 0.02, η_p² = 0.10]. Independent samples t test confirmed that, while the weight assigned to recent stimulus−outcomes pairings did not differ between groups during acquisition [η; M = 0.03, 95% CI [−0.10, 0.16], t₍₇₃₎ = 0.42; P = 0.68], stress before reversal learning led to a lower scaling factor than control participants [ρ; M = −0.55, 95% CI [−1.00, −0.10], t₍₇₃₎ = −2.46; P = 0.01; Fig. 5A]. This suggests that stress exposure diminished the relative weight of PE signals (i.e., η), thereby making stressed participants less sensitive to changes in aversive reinforcement. This change in associability resulted in slowing learning rates that allow individuals to adaptively adjust to a dynamic environment. We also tested whether individual differences in neuroendocrine responses were associated with our model-estimated parameters (for each subject). Indeed, our modeling results showed that the weight assigned to the PE during the reversal learning phase negatively correlated with the change of α-amylase concentration 10 min and 20 min after the stress/control treatment (r = −0.36, P = 0.002 and r = −0.26, P = 0.03, respectively; Fig. 5B). We note that the significance of the associations reported here was also preserved when using nonparametric tests that are resistant to extreme scores (e.g., Spearman’s rho) at both time points (10 min: r = −0.26, P = 0.03; 20 min: r = −0.30, P = 0.01). No such correlation was observed with change of cortisol 10 min or 20 min after the treatment (all p >0.05).

Fig. 5.

Modeling results. (A) Mean estimated scaling factor (ρ) during the day 2 reversal phase of learning for each group. Stressed participants revealed significant reductions in the weight assigned to PE signals that dynamically adjust learning rates during value updating. (B) Changes in α-amylase relative to baseline were negatively associated with the overall weight assigned to PE signals during value updating.

SI Results

Neuroendocrine Responses (Day 1).

Since we did not differentially manipulate stress levels on day 1, we expected no group differences for either neuroendocrine assay. An RM-ANOVA confirmed that, although α-amylase levels increased over time [F_(1,69) = 4.66, P = 0.03, η_p² = 0.06], there was no effect of group [F_(1,69) = 2.94, P = 0.09, η_p² = 0.04], nor a time × group interaction [F_(1,69) = 0.09, P = 0.76, η_p² = 0.001]. A comparable analysis using mean cortisol concentration levels similarly revealed no effect group [F_(1,67) = 0.83, P = 0.36, η_p² = 0.01] and no time × group interaction [F_(1,67) = 0.55, P = 0.46, η_p² = 0.008], indicating that α-amylase and glucocorticoid levels did not differ between groups before or after the initial learning session.

Self-Reported Anxiety.

To assess whether self-reported levels of anxiety (49) at the start of each day differed between conditions, we compared state and trait anxiety levels between the stress and control conditions. This analysis revealed no group differences on day 1 [state: M = 0.39, 95% CI [−3.42, 4.22], t₍₇₃₎ = −0.21, P = 0.84; trait: M = −1.39, 95% CI [−6.11, 3.32], t₍₇₃₎ = 0.59, P = 0.56] or day 2 [state: M = 0.67, 95% CI [−4.07, 5.42], t₍₇₃₎ = −0.28, P = 0.77]. Additionally, state anxiety reports did not differ between sessions for either condition, suggesting these levels were comparable across days [control: M = −0.59, 95% CI [−2.66, 1.47], t₍₃₆₎ = −0.58, P = 0.56; stress: M = −0.87, 95% CI [−3.53, 1.79], t₍₃₇₎ = −0.66 P = 0.51]. Further, self-reported anxiety on either day was uncorrelated with conditioned responding during both the acquisition and reversal sessions (all p > 0.05).

Tonic SCL.

To ensure that diminished differential SCRs in the stress group during reversal learning were not the result of constrained phasic responses in the presence of higher tonic SCL (i.e., higher baseline arousal) after stress exposure, we compared mean SCL during each ITI for the stress and control group during each learning phase (Fig. S1). Entering these values into a repeated-measures ANOVA revealed a main effect of phase [F_(1,72) = 29.35, P < 0.0001, η_p² = 0.29]—suggesting SCL changed across sessions for all participants—but, critically, yielded no main effect of group [F_(1,72) = 0.36, P = 0.55, η_p² = 0.002] or group × phase interaction [F_(1,72) = 0.03, P = 0.85, η_p² = 0.004]. Planned comparisons confirmed that, while both groups exhibited a significant decrease in SCL across phases [control: M = 3.16, 95% CI [1.36, 4.96], t₍₃₅₎ = 3.57, P < 0.001; stress: M = 2.96, 95% CI [1.52, 4.40], t₍₃₇₎ = 4.16, P < 0.0001], SCL did not differ between groups on either day 1 [acquisition: M = −0.69, 95% CI [−3.75, 2.37], t₍₇₃₎ = −0.45, P = 0.65] or day 2 [reversal: M = −0.96, 95% CI [−3.67, 1.70], t₍₇₂₎ = −0.72, P = 0.47].

Discussion

Maintaining cognitive and behavioral flexibility in a changing environment is a hallmark of adaptive responding. However, the same situations that require us to flexibly update affective responses to shifting sources of threat are often marked by stress. The inherently stressful nature of these circumstances can thus compromise the capacity to adaptively adjust responses to changing reinforcement patterns. Here, we provide evidence that acute stress exposure renders individuals less capable of flexibly updating responses to stimuli that signal threat or safety as their predictive value changes. Using a validated stress induction technique, we independently manipulated and measured acute stress responses and complemented this with a phasic measure of sympathetic arousal to stimuli paired with dynamically changing aversive reinforcement. This enabled us to measure reversal learning under the neuroendocrine influence of acute stress exposure without conflating stress responses with that of CS-specific SCR (i.e., arousal). The efficacy of our stress manipulation was confirmed by observing significantly higher concentrations in the stress group of α-amylase and cortisol—two well-established markers of acute stress response. Despite intact learning of the original cue contingencies, stressed participants showed reductions in CRs during reversal relative to the control group. Finally, fitting a computational learning model to participants’ SCRs revealed that these acute stress deficits were the result of failing to weigh PE signals that drive adaptive adjustments to dynamic learning rates.

Reversal learning constitutes two distinct learning processes that occur simultaneously: learning that a threatening stimulus now predicts safety (i.e., extinction) as well as generating a threat response to a previously safe stimulus. Previous research has shown that stress disrupts prefrontal cortex function, which is centrally implicated in fear extinction, and potentiates activity of the amygdala, a region critical for the acquisition and expression of learned threat (1, 23, 24). These divergent effects of stress on brain function raised the possibility that stress might impair fear extinction, while sparing the capacity to assign aversive value to the newly threatening stimulus. If this were the case, we would have expected the stress group to fail to attenuate threat responses to the new CS− that previously signaled threat. In contrast, we found that the stress group successfully extinguished threat responses to the CS− during reversal. Thus, stress did not appear to disrupt the extinction learning component of our reversal task. Instead, we found that stressed participants’ impairment in reversal learning stemmed from a selective disruption in the acquisition of a CR to the new CS+ (old CS−), indicating a failure to flexibly assign aversive value to a previously safe stimulus under stress. Although impaired safety learning has been observed in traumatized individuals (31), our results suggest that extinction learning—at least as measured by passive arousal signals—may be more resistant to acute stress than chronic stress, which accompanies trauma-related disorders. Whether acute stress has similar effects on extinction learning when not accompanied by other learning processes (i.e., threat learning) or using alternate assays of extinction learning (e.g., instrumental responses) is a relevant question for future research.

Using a hybrid computational RL model, we showed that stress disrupts aversive reversal learning by attenuating associability. This learning variable utilizes PE signals to adaptively adjust the learning rate for associated environmental cues (28). Therefore, the lower weight placed on PEs during reversal learning in the stress group led to dampened sensitivity to the changing shock contingencies and slowed the updating of SCR to the new CS+ compared with the control group. While the scaling factor during reversal would influence the PE weight parameter for both CSs, it had a larger impact on SCRs that were increasing over time (to the new CS+) than for those that were constantly decaying due to shock omission (new CS−). This reduced sensitivity to reinforcement can lead to a failure to adopt preparatory strategies to avoid or cope with imminent threats, rendering stressed individuals vulnerable to danger in their immediate environment.

Our neuroendocrine analyses revealed a notable role for α-amylase, a proxy of noradrenaline, in modulating both initial aversive learning and reversal learning, albeit in distinct ways. During day 1, we observed increases in α-amylase across acquisition, but no change in participants’ cortisol levels. This suggests that the aversive nature of the fear-conditioning task induced transient autonomic nervous system arousal, but failed to activate the HPA axis. This is consistent with the notable role of the noradrenergic system in modulating emotional learning, whereas glucocorticoids have primarily been associated with consolidating such learning. For example, blocking noradrenergic activity (32, 33) impairs aversive conditioning in rodents, but interfering with glucocorticoid signaling leaves fear learning intact (34). A distinct neuroendocrine profile emerged on the second day, where we observed significantly higher levels of both α-amylase and cortisol in the stress group vs. controls, suggesting that the CPT evoked an HPA-axis response. Notably, increases in α-amylase relative to baseline were inversely related to our behavioral and computational indices of reversal learning. Consistent with findings of inverted U-shaped functional effects of noradrenergic signaling (6, 35), our data suggest that, whereas mild increases in noradrenergic arousal may facilitate aversive learning (23), higher levels may impair the flexible updating of aversive value.

Neuroscientific investigations have shown that, during aversive reversal learning, striatal activity correlates with trial-by-trial aversive PEs, while amygdala activity correlates with both the aversive value and associability of CSs (5, 27). As a speculative neural mechanism, it is possible that stress-induced dopamine release, which has been proposed to arise primarily from increases in tonic dopamine (36), may constrain negative prediction errors during learning (37). Accordingly, recent research using reinforcement models has reported diminished learning from negative feedback in humans after stress exposure (38, 39). Further, noradrenergic activity within the locus coeruleus (LC) increases following contingency reversals (40) and is thought to carry signals that facilitate uncertainty-driven learning (41). While reciprocal noradrenergic connections between the LC and amygdala may have facilitated initial learning (i.e., in the absence of stress), phasic LC signaling may have been dampened by the rise in tonic noradrenergic activity under stress (35), disrupting associability. This is consistent with a recent study that reported that phasic noradrenergic sensitivity to environmental volatility during an aversive reinforcement task was related to adaptive learning of stimulus−outcome contingencies, but was markedly impaired in highly anxious individuals (25). Nonetheless, future research using brain imaging will be critical to fully examine this relationship.

Consistent with converging research showing that acute stress impairs higher-level cognitive function and flexibility, our study offers evidence that exposure to acute stress is capable of disrupting the ability to track and respond appropriately to shifting sources of threat in the environment. Importantly, our findings suggest that stress-induced deficits in adaptive affective learning may be selectively linked to mechanisms that integrate relevant new information about changes in reinforcement patterns into one’s current aversive value representation. Our results highlight the need for future research assessing the effects of stress exposure on the neural circuitry that supports the capacity to adaptively learn in a dynamic environment. Given the pervasive nature of stress exposure in everyday life, such work is critical for understanding real-world motivated behavior.

Methods

Participants.

One hundred and forty healthy participants (75 females) were recruited through online and departmental advertisements within and around the New York University campus. Participants were ineligible to join the study if they were pregnant, had heart or blood pressure problems, or were diagnosed or being treated for any neurological or psychiatric disorders. All experimental procedures were approved by New York University’s Committee on Activities Involving Human Subjects. Participants provided informed consent and were compensated. Our final sample (SI Methods) included a total of 75 healthy participants (41 females) with a mean age of 23.01 y (SD = 8.07; range = 18 y to 63 y).

Threat Acquisition and Reversal Learning.

We used an aversive learning task with delay conditioning and partial reinforcement. The US was a mild wrist shock. Our index of fear arousal was SCR, a dynamic assay of sympathetic nervous system arousal, consistent with the autonomic arousal characteristic of threat (42). CSs were two square fractal images. One image (CS+) coterminated with the US on a subset (33%) of trials, while the other image (CS−) was never reinforced and served as a safety stimulus to measure baseline arousal. On each trial, a CS was presented for 4 s against a black screen, followed by a variable 8- to 12-s intertrial-interval (ITI), during which a white fixation cross was presented. The acquisition session comprised a total of 30 trials: 12 CS− trials, 12 unreinforced CS+ trials, and 6 reinforced CS+ trials. Participants were instructed to monitor the relationship between the stimuli and occurrence of shock. Trial order was pseudorandomized such that no more than two of the same trial type occurred consecutively. CSs were counterbalanced across participants. Partial reinforcement enabled unreinforced CS+ trials to be analyzed for CRs, unaffected by the distinct physiological response induced by the US. Participants who demonstrated adequate learning on day 1 returned the following day for reversal learning. Ten minutes after the stress or control manipulation, participants underwent an aversive reversal learning task using the same protocol as the previous day, with the exception that the original CS−US contingencies were reversed. No additional directions before reversal were given.

Stress Manipulation.

Participants were randomly assigned to the stress (n = 38; 22 females) or control (n = 37; 21 females) group on day 1, before the initial learning session. On day 2, participants in the stress group completed the CPT, which entailed submerging their right forearm in ice-cold water (0 °C to 4 °C) for 3 min; control participants followed the same procedure using tepid water (30 °C to 35 °C). The CPT is widely used to model the effects of mild to moderate stress and reliably induces sympathetic nervous system and HPA-axis arousal as measured by increased physiological, neuroendocrine, and subjective stress levels (43, 44). Participants rated how unpleasant they felt during the water task on a scale from 1 (not unpleasant) to 10 (highly unpleasant) to assess subjective stress.

Neuroendocrine Analysis.

Saliva samples were collected throughout each session to assess α-amylase and cortisol concentrations, which serve as neuroendocrine markers of stress response. Saliva was collected for 2 min using a high-quality synthetic polymer-based salivette placed under participants’ tongues. To control for diurnal rhythms of stress hormone levels, participants were run between 1200 hours and 1700 hours and at the same time each day. Participants were asked to abstain from eating or drinking for an hour before each session. Upon arrival, participants acclimated to the laboratory setting in a quiet experiment room for 10 min, during which time they drank 4 oz of water to clear residual saliva before baseline sample collection. On day 1, saliva samples were collected at baseline and after aversive learning, to confirm that stress hormones levels did not differ between groups. On day 2, saliva collection occurred at baseline, 10 min after the CPT/control task, and immediately following the reversal task (∼20 min after CPT/control task). Any samples that contained insufficient saliva could not be analyzed and were excluded from our analyses.

Computational Model Fitting.

We define x_n as the conditioned stimulus on trial n (CS+ or CS−) and define r_n as the US delivered (1 for US, 0 for no US). We defined value (i.e., shock) predictions V_n(x) for each stimulus and trial. The PE δ_n = r_n − V_n(x_n) measures the difference between the experienced and predicted shock on trial n. Note that our model replaced the constant learning rate in the classic Rescorla−Wagner model with a dynamic associability-gated learning rate (27, 45, 46). The resulting model, for day 1 (acquisition) is

$$V_{n+1}\left({\chi }_n\right)=V_n\left({\chi }_n\right)+\kappa {\alpha }_n\left({\chi }_n\right){\delta }_n$$

$${\alpha }_{n+1}\left({\chi }_n\right)=\eta \left|{\delta }_n\right|+\left(1-\eta \right){\alpha }_n\left({\chi }_n\right),$$

where η indicate the weight assigned to the most recent PE and κ is a normalization factor. Note that trial n’s associability α_n depends on (absolute) PEs from past trials, but not the current one. This makes δ_n and α_n(x_n) relatively uncorrelated to one another and also means that δ_n is not redundant in the value update.

On day 2, we hypothesized that the acute stress manipulation might alter reversal learning by changing the weight assigned to the PE (i.e., η). This can be achieved by postulating an additional scaling factor ρ that captures this change. That is, for day 2 (reversal),

$$V_{n+1}\left({\chi }_n\right)=V_n\left({\chi }_n\right)+\kappa {\alpha }_n\left({\chi }_n\right){\delta }_n$$

$${\alpha }_{n+1}\left({\chi }_n\right)={\rho }^{\ast }\eta \left|{\delta }_n\right|+\left(1\hspace{0.17em}-\hspace{0.17em}{\rho }^{\ast }\eta \right){\alpha }_n\left({\chi }_n\right).$$

For the hybrid model, we tested the fit of associability to the SCRs in both sessions across 2 d. The model fitting was also described in our previous paper (27); specifically, we optimized the free parameters of the model to maximize the posterior probability (maximum a posteriori; MAP) of observing the sequence of SCRs measured following each CS. This method has been shown to be successful in preventing unreasonable or extreme individual fits given that we are fitting each subject’s parameters separately. We modeled the likelihood of each trial’s SCR S_n as independent and identically distributed Gaussian-distribution around a mean determined by associability predicted by the model on that trial plus a constant term,

$$S_n\sim N\left[{\beta }_0+{\beta }_1\hspace{0.17em}{\alpha }_n\left(X_n\right),\sigma \right].$$

In MAP estimation, we set the initial expected value V₀ and associability α₀ to 0.3. PE weight (i.e., η) was constrained to the range [0 1] with a Beta (1.2, 1.2) prior distribution slightly favoring values in the middle of the range, Both κ (normalization factor) and ρ (scaling factor) were constrained to be positive values with a Gamma (1.2, 1) prior distribution that favored smaller values. Likelihoods were pooled over all trials for each subject, and priors were implemented by adding to the data log likelihood, hence giving the posterior probabilities of the model parameters given the prior distributions (47). Reinforced shock trials were omitted in model fitting to avoid possible contamination of the predictive response by shock-related responses.

In general, our model provided a good fit in terms of capturing SCR patterns for control and stress groups across 2 d. Specifically, the model-predicted associability tracked the difference of the SCR for CS+ only on day 2, both aggregately (Fig. S3) and on a trial-by-trial basis (Fig. S4). The κ, η, and ρ were estimated for each subject in both control and stress groups, and then ρ were pooled together according to group label (control vs. stress) to test group differences.

Fig. S3.

Model-predicted aggregate SCR for both CS+ and CS− using a single PE weight from each subject. In the control group (Left), subjects showed comparable SCRs for both CS+ and CS− related SCRs across learning phases. However, in the stress group (Right), subjects showed impaired learning to the new CS+ due to reduced scaling factors (*P < .05).

Fig. S4.

Model-predicted trial-by-trial SCR using parameters estimated from each subject. The SCR pattern for CS+ and CS− did not differ between groups on day 1 (acquisition: trials 1 to 12). However, on day 2 (reversal: trials 13 to 24), stressed participants’ lower scaling factor led to dampened sensitivity to the changing shock contingencies (more dependent on cached associability value), and thus slower updating of SCRs to the new CS+ compared with the control group. While the scaling factor during reversal would influence the PE weight parameter for both CS, it has a bigger impact on the associability variable (predicted SCR) that is on the rise (CS+ on the second day, gray curve after trial 12) than the one that’s constantly decaying (CS− on the second day, blue curve after trial 12).

SI Methods

Exclusionary Criteria.

Consistent with previous research, participants were excluded from taking part in day 2 if they failed to show measurable SCRs on day 1 (n = 14; 9 females), or if they failed to acquire differential conditioned fear responses (mean CS+ > CS− by at least 0.1 μS) (n = 38; 19 females). These criteria were necessary, as reversal learning cannot be assessed if a conditioned response is not initially acquired. Additionally, 12 participants were excluded due to experimenter or technical software error (6 females), and 1 was excluded due to having neuroendocrine responses greater than 4 SDs from their corresponding group mean. Our final sample included a total of 75 healthy participants (41 females) with a mean age of 23.01 (SD = 8.07; range = 18 to 63).

Psychophysiological Stimulation and Assessment.

SCRs were collected using shielded Ag−AgCl electrodes applied to the middle phalanges of the second and third fingers of the left hand. Mild electric shocks (200 ms; 50 pulses per second) were delivered through a stimulator bar electrode (Biopac Systems) attached with a Velcro strap to participants’ right inner wrists. Bar electrode wells were manually filled with NaCl electrolyte gel to enhance conductance and attached to a SD9 Square Pulse Stimulator and stimulator isolation unit (Grass Technologies). Individual shock levels were set each day using a work-up procedure, beginning at a mild level and increasing incrementally either until the shock was reported to be “uncomfortable, but not painful” or to a maximum of 60 V. SCRs were sampled at 200 Hz using an MP100 Data Acquisition module (BioPac System) connected to an Apple computer.

Physiological Analysis.

Skin conductance data were recorded using a sampling rate of 200 Hz and were subsequently analyzed offline using AcqKnowledge software (BIOPAC System). Each individual’s SCR data were preprocessed using AcqKnowledge software (BIOPAC systems) before analysis by low-pass filtering (cutoff frequency 25 Hz) and mean value smoothing using a three-sample window. Stimulus-evoked responses were measured by taking the largest amplitude base-to-peak waveform response (in microsiemens) beginning within the 0.5- to 4.5-s interval after CS onset. Responses lower than a predetermined criterion of 0.02 μS were recorded as zero. Participants lacking measurable SCR on >75% of unreinforced trials were classified as nonresponders. Raw SCR amplitudes were square root-transformed to reduce skew and were subsequently divided by individuals’ mean US response to account for individual differences in physiological reactivity (48). We calculated conditioned fear responses by subtracting mean SCR to the baseline stimulus from that of the threat-related stimulus (CS+ minus CS−).

Neuroendocrine Analysis.

All samples were frozen immediately after collection in a sterile tube and stored in a freezer set to −20 °C. Samples were shipped frozen to Salimetrics Testing Services, a Clinical Laboratory Improvement Amendments-certified analytical laboratory, where they were analyzed using high-sensitivity enzyme immunoassay kits. Samples were brought to room temperature and centrifuged at ∼1,500 × g for 15 min before testing. Any samples that contained insufficient saliva could not be analyzed and were excluded from our analyses.

Statistical Analysis.

ANOVA with repeated measures was used to analyze all SCR and neuroendocrine data. Separate analyses were conducted to assess differential SCR across each conditioning phase (acquisition, reversal) and mean α-amylase and cortisol levels across each day. To better characterize how learning developed across both sessions, each session (acquisition, reversal) was divided into four blocks (three unreinforced CS+ and three CS− trials per block). Post hoc comparisons were conducted using Student t tests. All tests were two-tailed and considered statistically significant when P < 0.05. All correlational analyses used Pearson correlations unless tests of normality (Shapiro−Wilk) indicated that values were not normally distributed. In these cases, Spearman correlations were conducted and reported. All analyses were conducted using SPSS (version 20.0, 2011; IBM Corporation).