Emotional arousal amplifies competitions across goal-relevant representation: A neurocomputational framework.

Abstract

Emotional arousal often facilitates memory for some aspects of an event while impairing memory for other aspects of the same event. Across three experiments, we found that emotional arousal amplifies competition among goal-relevant representations, such that arousal impairs memory for multiple goal-relevant representations while enhancing memory for solo goal-relevant information. We also present a computational model to explain the mechanisms by which emotional arousal can modulate memory in opposite ways via the local/synaptic-level noradrenergic system. The model is based on neurophysiological observations that norepinephrine (NE) released under emotional arousal is locally controlled by glutamate levels, resulting in different NE effects across regions, gating either long-term potentiation or long-term depression by activating different adrenergic receptors depending on NE concentration levels. This model successfully replicated behavioral findings from the three experiments. These findings suggest that the NE’s local effects, rather than broad effects, are key in determining the effects of emotion on memory.

While our memory system does not allow us to remember all the events we experience in life, we typically remember emotional events for a long time. Indeed, decades of research reveals that emotional events are remembered better than neutral events (LaBar & Cabeza, 2006; Talmi, 2013). However, there has been little consensus on how emotion modulates memory in a retrograde manner, with some studies showing retrograde enhancement effects of emotion on memory while others show retrograde impairment effects of emotion on memory (for reviews see Chiu, Dolcos, Gonsalves, & Cohen, 2013; Dolcos, Katsumi, Denkova, & Dolcos, 2017; Dolcos, Katsumi, Weymar, et al., 2017; Mather & Sutherland, 2011). In the current paper, we investigate the mechanisms by which emotion enhances some memories while impairing others.

A predominant view in the literature on emotion and memory is that encountering emotional events induces short-term, phasic arousal reactions and triggers amygdala activity, which facilitates memory for the emotional events (Dolcos, LaBar, & Cabeza, 2004; Kensinger & Corkin, 2004; LaBar & Cabeza, 2006). Consistent with this view, research indicates that norepinephrine (NE) released from the locus coeruleus (LC) under phasic arousal interacts with β-adrenergic receptors in the amygdala to enhance memory for emotional information (for reviews see Markovic, Anderson, & Todd, 2014; McIntyre, McGaugh, & Williams, 2012).

While these accounts primarily focus on how emotional events are remembered well, they also predict that emotion has retrograde impairing effects on memory for neutral preceding events: enhanced memory for emotional information would leave fewer resources available for other information, resulting in impairment for preceding events. Consistent with this prediction, when task-irrelevant emotional stimuli are presented during working memory tasks, these emotional stimuli are remembered well at the expense of impaired working memory performance for preceding information (Dolcos et al., 2013; Dolcos & McCarthy, 2006; Shafer & Dolcos, 2012). Previous studies extend these findings to long-term memory and demonstrate that exposure to emotional events leads to impaired memory for what has been learned before the emotional events (Hurlemann et al., 2005; Loftus & Burns, 1982; Miu, Heilman, Opre, & Miclea, 2005; Strange, Hurlemann, & Dolan, 2003). These retrograde impairing effects of arousal were diminished by β-adrenergic receptor blockers (Strange et al., 2003), suggesting that NE is involved not only in the memory enhancement effects for emotional information but also in the arousal-induced retrograde amnesia effects.

However, other studies show opposite results: exposure to emotionally arousing events after learning something neutral enhances memory of the previously learned experiences (A. K. Anderson, Wais, & Gabrieli, 2006; Finn & Roediger, 2011; Nielson & Powless, 2007; Nielson, Yee, & Erickson, 2005). These retrograde enhancement effects of arousal are also related to noradrenergic mechanisms (Roozendaal, Castello, Vedana, Barsegyan, & McGaugh, 2008). For example, arousal induced by squeezing a handgrip or vagus nerve stimulation enhances memory for previously learned neutral materials (Clark, Naritoku, Smith, Browning, & Jensen, 1999; Nielson, Radtke, & Jensen, 1996). Critically, the handgrip squeeze procedure increases pupil dilation (an indicator of LC activity) as well as peripheral NE levels (Nielsen & Mather, 2015) and its modulation effect on memory is attenuated by β-adrenergic receptor blockers (Nielson & Jensen, 1994). Likewise, vagus nerve stimulation increased NE levels in the basolateral amygdala (Hassert, Miyashita, & Williams, 2004). Thus, it appears that noradrenergic mechanisms are involved in the retrograde effects due to emotional arousal, irrespective of whether arousal leads to memory enhancement or retrograde amnesia effects.

In summary, previous studies provide a confusing set of results concerning the retrograde effects of emotional arousal on memory as well as the role of noradrenergic systems in the emotion-memory interaction. Recent theoretical frameworks address this puzzle by positing that stimulus priority serves as a boundary condition between the emotion-induced facilitative vs. impairing effects on memory (Levine & Edelstein, 2009; Mather & Sutherland, 2011). Priority is determined by information’s goal-relevance and bottom-up saliency. Recent research also provides evidence consistent with predictions from these theoretical perspectives. According to these studies, arousal induced by encountering emotional events enhances memory for what has happened earlier when the event is goal-relevant and therefore has high priority, but impairs memory for preceding events when the events are goal-irrelevant and have low priority (Knight & Mather, 2009; Lee, Greening, & Mather, 2015; Ponzio & Mather, 2014; Sakaki, Fryer, & Mather, 2014).

How can then arousal have opposing effects at the same time, with enhancing memory representations for high-priority information while suppressing everything else? To address this question, the Glutamate Amplifies Noradrenergic Effects (GANE) framework was recently proposed (Mather, Clewett, Sakaki, & Harley, 2016). GANE posits that glutamate is the brain’s primary excitatory neurotransmitter and signals priority, such that neurons that process high-priority information release a larger amount of glutamate than those that process low-priority information. When NE is released under phasic arousal, NE further stimulates glutamate release via β-adrenergic receptors and glutamate also stimulates NE release. This positive feedback loop between glutamate and NE results in enhanced neural representations for high-priority information under arousal. At the same time, high glutamate signals and high NE signals around high-priority representations lead to stronger GABAergic activity to suppress lower priority representations. In addition, high NE levels near high-priority representations enable to activate low-affinity β-adrenergic receptors to facilitate long-term potentiation (LTP) to enhance memory for high-priority information. In contrast, lower NE levels elsewhere leads to activation of high-affinity α1-adrenergic receptors that facilitate long-term depression (LTD). Thus, according to GANE, arousal’s opposite effects on memory are explained by a) the selective enhancement of high-priority signals due to the positive feedback loop between glutamate and NE and b) the NE’s local action to enable LTP only for high-priority signals.

Importantly, GANE also suggests that there is a boundary condition for the enhancement effects of arousal on memory for high-priority representations. According to GANE, the enhancement effect of arousal on memory for high-priority representations should be limited to situations where there is only one dominant high-priority item; when individuals encounter multiple similarly high-priority items, GANE predicts the impairing effects of arousal, rather than enhancement effects. Specifically, when there are many similarly high-priority items, neurons that process high-priority items should release a relatively large amount of glutamate, leading to higher NE levels and higher activity under arousal; but the higher activity level associated with these representations should result in increased GABAergic activity to suppress competing representations. Thus, these representations would mutually suppress each other and have lower levels of NE that activate only α1-adrenergic receptors, resulting in LTD and impaired memory under arousal. In summary, it is expected that arousal enhances memory for solo high-priority representations but impairs memory for multiple competing high-priority representations.

In the current paper, we first tested this prediction by behavioral experiments. To further examine whether our behavioral results are consistent with GANE, we extended a recently developed computational model based on GANE (Lee et al., 2018) and tested whether the same computational model can reproduce the behavioral results.

Studies 1 and 2

To test whether arousal impairs memory for multiple high-priority representations, Studies 1 and 2 used similar procedures. During the experiment, participants first completed a fear conditioning phase, where they learned associations between one tone and electric stimulation (a fear-conditioned tone: CS+) and associations between another tone and lack of stimulation (a neutral-conditioned tone; CS−). This conditioning phase was followed by a learning phase, where participants learned target face images, followed by CS+ or CS− (i.e., arousal manipulation). To address our main goal of the studies, we manipulated the number of target stimuli participants needed to remember. In one condition, participants were shown one face and three objects and asked to remember the face while ignoring the objects (i.e., solo face condition). In the other condition, they were shown four faces and asked to remember all of them (i.e., multiple face condition). The learning phase was followed by a final memory test on all face images. According to GANE, arousal should enhance memory for goal-relevant stimuli presented alone but should impair memory if those same goal-relevant stimuli were presented with competing equally goal-relevant stimuli. Thus, we expected that CS+ would enhance memory for faces in the solo face condition but impair memory for faces in the multiple face condition.

Method

Participants.

Forty-five participants (11 males; Mage = 20.47, SD = 1.62) took part in Study 1 and 50 participants (13 males; Mage = 21.62, SD = 3.31) took part in Study 2 for either course credit or $15/hour. The sample size of Study 1 was determined by the effect size estimated from a previous study (Sakaki et al., 2014). The sample size of Study 2 was determined based on the effect sizes in the previous study (Sakaki et al., 2014) and Study 1. Participants were not screened for mental health disorders.

Stimuli.

All images were grayscale without any background objects or scenes. Targets were images of 160 celebrity faces (80 males and 80 females) obtained from the Internet (see Supplemental materials for a list of cerebrities used in the study). For each of the 160 celebrities, two different images were included: these two images depicted the same person at a similar age, but were different in a range of characteristics, including orientation, hair styles, gaze directions, and mouth shapes (e.g., open mouth vs. close mouth). None of them had clear emotional expressions. One from each pair was presented as a memorandum during initial presentation and the other was used in a working memory test.¹ Images that were used as memoranda were counterbalanced across participants.

The 160 celebrities were pseudo-randomly assigned to one of the five conditions (CS+/solo, CS−/solo, CS+/multiple, CS−/multiple, and foils in a final memory test) so that each condition had an equal number of female and male faces. The assignment of each celebrity to the five conditions was counterbalanced across participants. Among the 32 celebrities assigned to each of the four learning conditions (CS+/solo, CS−/solo, CS+/multiple, and CS−/multiple), eight celebrities were randomly chosen and used as foils in the working memory test. The remaining 24 celebrities were randomly assigned to one of the three working memory (WM) test item type conditions (see Procedures for details) for each participant. An additional 12 celebrity faces were used in booster trials, where participants received an unconditioned stimulus (US) after the CS+ (CS+ with shock trials).

An additional 144 celebrity faces (72 males and 72 females) were used as distractors in the multiple face condition. Each target face assigned to the multiple face condition was pseudo-randomly paired with three other faces so that each trial included two females and two males; this allowed us to ensure that the priority of each face due to its sex was similar across all four faces presented in each trial. We did not control other features (e.g., age, race).

In the solo face condition, we used 144 objects as distractors. They were obtained from previous research (Brady, Konkle, Alvarez, & Oliva, 2008; Kensinger, Garoff-Eaton, & Schacter, 2006; Sakaki et al., 2014) as well as the Internet. They were chosen from a wide range of categories, such as animals, kitchen utensils, tools, instruments, clothes, plants, food, stationary, vehicle, furniture pieces, buildings and so on. Each target face assigned to the solo condition were randomly paired with three objects. We also used an additional 15 objects and 15 celebrity faces as distractors in booster trials.

Procedures.

After giving consent, participants completed questionnaires on their mood (Watson, Clark, & Tellegen, 1988), depression (Radloff, 1977) and anxiety (Spielberger, Gorsuch, & Lushene, 1970) and the Wechsler Test of Adult Reading (Wechsler, 2001); the results from these measures will not be reported in the paper.

Participants next completed the fear-conditioning phase whose procedure was developed based on previous studies (Lee, Baek, Lu, & Mather, 2014; Lee et al., 2015; Lee, Sakaki, Cheng, Velasco, & Mather, 2014). Conditioned stimuli (CS) were two neutral tones that were different in pitch; which tone was used as CS+ was counterbalanced across participants. The US was electric stimulation. The fear conditioning task included 36 trials in a randomized order (12 trials for CS+ with shock, 12 trials for CS+ without shock and 12 trials for CS−). On each trial (Figure 1A), participants were presented with CS+ or CS− for 1 sec. To ensure that participants paid attention to tones, they were asked to press a key to indicate whether the tone was high- or low- pitched. After a 1-sec blank screen, participants received a shock for 0.4 sec in the CS+ with shock trials. In contrast, participants did not receive a shock in other trials. Each trial ended with a jittered fixation cross (6, 8, or 10 sec). Prior to the task, participants were informed which tone was predictive of shock but were not informed the probability of shock. To confirm that participants had acquired fear-conditioned responses to CS+, we monitored participants’ skin conductance reactions (SCR) during the fear-conditioning task.

Figure 1.

Procedures in experiments. (A) In the conditioning phase, participants learned associations between a neutral tone and electric stimulation (CS+) and associations between another tone and lack of stimulation (CS−). (B) During the learning phase in Studies 1 and 2, participants learned four faces in the multiple condition or one face in the solo condition, followed by CS+ or CS−. Face images were taken from the TarrLab face database for illustration purpose only (stimulus images courtesy of Michael J. Tarr, Center for the Neural Basis of Cognition, Carnegie Mellon University, http://www.tarrlab.org).

After completing the conditioning phase, participants completed a questionnaire on their mood (Watson et al., 1988; which will not be reported in the paper), followed by a learning phase developed based on previous research (Figure 1B; Dolcos & McCarthy, 2006). On each trial, participants saw four images for 3.5 sec: one celebrity face with three objects in the solo face condition and four celebrity faces in the multiple face condition. In the solo face condition, participants were asked to remember the face and ignore three objects. In contrast, in the multiple face condition, they were asked to remember all four faces. After a 2-sec blank screen, participants were presented with CS− or CS+ for 1 sec, followed by a WM test image. The WM test image was identical to the one of the memoranda on one-third of the trials (same condition), was different but depicted the same person as one of the memoranda on another one-third of the trials (similar condition), and depicted a new person not shown as memoranda on the remaining trials (new condition). Participants were told to select the “similar” option when the test image was different in any feature from those in the memoranda but depicted the same person. Participants indicated whether the image was “same,” “similar” or “new” by key press within 3 sec; if they failed to press a key within 3 sec, the image automatically disappeared.

The learning phase included eight trials in each condition from a 2 (arousal: CS+, CS−) × 2 (study set type: solo, multiple) × 3 (WM test item type: same, similar, new) design. During these 96 trials, participants did not receive shock even when they were presented with CS+. To prevent extinction, we included an additional nine booster trials, where participants received a 0.4-sec shock 1 sec after CS+. Data from these booster trials were not included in our analyses. The toal 105 trials were divided into 3 blocks, each of which included 3 booster trials and 32 main trails in a randomized order.

The learning phase was followed by a demographic questionnaire and a surprise old-new recognition test for faces. This final memory test included 96 old faces (one face image from each encoding trial) and 32 new faces (foils). For each face, participants first indicated whether each image was old or new without any time limit. In Study 1, participants further indicated their confidence with a 4-point scale (1 = definitely new, 2 = maybe new, 3 = maybe old, and 4 = definitely old). The confidence rating was not included in Study 2. In this final memory test, for old stimuli, we used images used as memoranda during the learning phase (rather than those presented during the WM test). The final memory test did not include objects in either Study 1 or Study 2.

Electric shock and skin conductance.

Electric stimulation was delivered to the third and fourth fingers of the left hand as US via a shock stimulator (E13–22; Coulbourn Instruments, Allentown, PA) which was connected to a grounded RF filter. Prior to the experiment, we determined the intensity of “highly unpleasant but not painful” electric stimulation for each participant (Study 1: M = 1.31 mA; SD = 0.37; Study 2: M = 1.52 mA; SD = 0.23). The level determined was used throughout the experiment as the US. SCR data were recorded at 1k Hz sampling rates through the MP-150 system (BIOPAC system, Goleta, CA).

SCR data epochs were extracted from a time window between 0 and 8 sec after CS tone onset, and baseline-corrected between 0 and 1 sec. The peak SCR amplitude was taken between 1 and 8 sec from the trial-by-trial average SCR epoch as a function of CS tone. To examine the effects of conditioning, rather than shock itself, our SCR analysis focused on SCR between the CS+ without shock condition and the CS− condition. Due to technical problems, SCR data from one participant were not recorded; thus data from this participant were not included in the SCR analysis.

Results

Fear conditioning.

Participants showed high accuracy in discriminating the high- vs. low- pitched tones during the conditioning phase with no significant differences across the conditions (ps > .10; Table 1). In addition, participants had greater SCR to CS+ than to CS− (Figure 2A–B), t(43) = 3.19, p = .0027, d = .48, t(49) = 4.72, p < .0001, d = .67, for Studies 1 and 2, respectively, indicating that they acquired arousal responses to CS+.

Table 1.

Accuracy of participants’ response in the tone discrimination task during the fear conditioning phase.

	Studies
	1	2	3
CS+ with shock	0.99	1	0.96
CS+ without shock	0.99	1	0.97
CS−	0.99	0.99	0.96

Figure 2.

Results from Studies 1 and 2. (A–B) CS+ induced higher SCRs than did CS− during the conditioning phase. (C–D) During the final memory test, there were selective impairment effects of emotional arousal on memory in the multiple condition. Error bars represent standard errors of means.

Memory test.

We next tested the main prediction of the study that arousal should enhance memory for solo goal-relevant items but impair memory for multiple goal-relevant items. To examine the effects of these factors with initial working memory performance equated, we focused on faces from trials where participants correctly answered the WM test. Two participants in Study 1 and three participants in Study 2 were at chance for foils (M < .55) in the final memory test; data from these participants were excluded when analyzing results in the final memory test. The remaining participants were significantly better than chance in responding to foils (Study 1: M = .79, Study 2: M = .84), t(42) = 16.31, p < .0001, d = 2.49, t(46) = 21.70, p < .01, d = 3.16, for Studies 1 and 2 respectively.

Two 2 (arousal: CS−, CS+) × 2 (study set type: solo, multiple) × 3 (WM test item type: same, similar, new) ANOVAs were performed on the proportion of correctly recognized faces in the final memory test in Studies 1 and 2 respectively. In both studies, we found main effects of study set type, F(1, 42) = 33.60, p < .0001, ηp² = 45, F(1, 46) = 51.32, p < .0001, ηp² = 55, and of WM test item type, F(2, 83) = 31.65, p < .0001, ηp² = 42, F(2, 91) = 25.54, p < .0001, ηp² = 40, and an interaction between WM test item type and study set type, F(2, 83) = 8.89, p = .0003, ηp² = 16, F(2, 91) = 9.05, p = .0003, ηp² = 16, such that having multiple faces significantly impaired memory performance with test items that were similar (Study 1: M_solo =.81, M_mult = . 70; Study 2: M_solo = .84, M_mult = .72) or new (Study 1: M_solo = .77, M_mult = .60; Study 2: M_solo = .76, M_mult = .53), Fs(1, 83) = 16.20, 38.61, ps < .0001, ds = .49, 1.10, Fs(1, 91) = 14.97, 52.88, ps < .0001, ds = .48, 1.04, but not with those that were the same as the target (p = .60, p= > .20; Study 1: M_solo =.85, M_mult = .83; Study 2: M_solo = .77, M_mult = .81). This interaction is not surprising given that even in the multiple face condition, participants had a second chance to relearn the images during the WM test in the same condition, which could help their performance in the final test. More importantly, in both studies, we found a significant interaction between arousal and study set type (Figures 2C–D), F(1, 42) = 4.50, p = .04, ηp² = 08, F(1, 46) = 4.66, p = .04, ηp² = 12. Consistent with our prediction, in the multiple face condition, CS+ significantly impaired memory for faces compared with CS−, F(1, 42) = 5.97, p = .02, d = .32, F(1, 46) = 6.38, p = .02, d = .32. In contrast, there were no significant effects of arousal in the solo face condition in either study (Study 1: p = .48, d = .11; Study 2: p = .60, d = .09).

A similar 2 (arousal) × 2 (study set type) × 3 (WM test item type) ANOVA on the confidence ratings in Study 1 confirmed the effects of study set type, F(1, 42) = 39.22, p < .0001, ηp² = 48, and of WM test item type, F(2, 83) = 31.12, p < .0001, ηp² = 43, and an interaction between them, F(2, 83) = 18.35, p < .0001, ηp² = 29. But neither the interaction between arousal and study set type (p = .11), nor any other effects involving arousal were significant (ps > .19).

Working memory test.

While our main focus is on final memory performance, according to GANE, a similar interaction between arousal and the number of goal-relevant items can occur even for participants’ ability to briefly maintain presented stimuli due to the positive feedback loop between glutamate and NE. To address this issue, each response was coded as a correct response when the participants made “same” responses for exact same faces as the one shown as memoranda, “similar” responses for similar images that depicted the same person as memoranda and “new” responses for images of individuals that were not shown as memoranda.

This correct response rate during the WM task was analyzed by separate 3 (arousal) × 2 (study set type) × 3 (WM test item type) ANOVAs for each study (Figures 3A–B). F-, t- and effect size values are reported below for Studies 1 and 2 in order for each effect. Across both studies, there were significant main effects of study set type, F(1, 44) = 212.21, p < .00001, ηp² = .86, F(1, 49) = 238.51, p < .00001, ηp² = .86, and of WM test item type, F(2, 88) = 79.51, p < .00001, ηp² = .64, F(2, 98) = 78.90, p < .00001, ηp² = .62. We also found a significant interaction between study set type and WM test item type, F(2, 88) = 35.44, p < .00001, ηp² = .64, F (2, 98) = 45.16, p < .00001, ηp² = .46. But the interaction between study set type and arousal was not significant (p = .41).

Figure 3.

Results from the working memory test in Studies 1 (A) and 2 (B). Error bars represent standard errors of means.

In addition to these similar results across studies, in Study 1, we found a significant interaction between arousal and WM test item type, F(2, 88) = 3.90, p = .02, ηp² = .08. Subsequent analyses revealed that arousal induced by CS+ enhanced performance in the similar condition (M_cs+ = .75 vs. M_cs− = .70), F(1, 88) = 8.90, p = .004, d = .37, but not in the other two conditions (ps > .60). These results suggest that emotional arousal induced by CS+ enhances the ability to discriminate similar inputs, thus facilitating pattern separation. We also obtained a pattern separation index (Yassa et al., 2011): [p(“similar”|similar items) – p (“similar”|old items)] and confirmed better pattern separation index score in the CS+ than in the CS− condition in Study 1, F(1, 44) = 4.13, p = .048, ηp² = .09, but not in Study 2 (p = .82).

Discussion

According to GANE, arousal should impair memory for multiple goal-relevant items. Consistent with this prediction, in Studies 1 and 2, emotional arousal induced by CS+ impaired memory for faces when there were four high-priority faces that needed to be remembered. However, contrary to our prediction, CS+ did not significantly enhance memory for solo high-priority faces. Thus, our results from these two experiments are partially consistent with the prediction of GANE.

Neither Study 1 nor Study 2 showed a significant interaction between study set type and arousal during the WM test, suggesting that the effects of arousal proposed in GANE might be stronger after a delay due to the role of NE on the long-term potentiation (LTP) and the long-term depression (LTD; Salgado, Kohr, & Trevino, 2012). In addition, in Study 1, we found a significant interaction between arousal and WM test item type during WM test, such that arousal enhanced performance in discriminating similar inputs. Recent research suggests that emotional arousal enhances pattern separation performance (Leal, Tighe, Jones, & Yassa, 2014; Segal, Stark, Kattan, Stark, & Yassa, 2012). Our results support this notion that emotional arousal facilitates one’s ability to discriminate similar inputs, although the valence-by-WM test item type interaction was not significant in Study 2 (p = .27) nor the effects of arousal on the pattern separation index (p = .82).

Study 3

Results from Studies 1 and 2 are consistent with the prediction from GANE that arousal impairs memory for multiple goal-relevant information. However, the arousal- induced memory enhancement effects in the solo condition were not significant. In Study 3, we increased priority of faces in the solo face condition to increase the effect size of this enhancement effect. To increase the priority of faces in the solo face condition, we a) added a preceding spatial cue to indicate the location of goal-relevant images as done in a prior study (Lee et al., 2015), b) presented four objects in the multiple condition, rather than faces, to make faces in the solo face condition stand out within the experiment context, and c) highlighted goal-relevant images by a red box. We also tested participants’ memory for objects to see the effects of emotional arousal on those representations.

Method

Participants.

Forty participants (15 males; M_age = 21.05, SD = 2.12) took part in the experiment for either course credit or $15/hour. The sample size was determined based on the effect size in another previous study (Lee et al., 2015) which used similar procedures (i.e., presenting spatial cues to indicate the location of subsequent high priority stimuli). As in Studies 1−2, participants were not screened for mental health disorders.

Stimuli. Faces.

Images of 80 celebrities (40 females and 40 males) were chosen from those used in Studies 1 and 2. For each celebrity, we included two different images: one of them was used in the initial presentation as a memorandum and the other was used in a working memory test (the image used as a memorandum was counterbalanced across participants).

We created two stimulus sets (each included 20 females and 20 males). One of them was used in the CS+ condition, the other was used in the CS− condition and the assignment was counterbalanced across participants. Among the 40 faces assigned to each condition, 24 were used as targets in the learning phase, another eight were used as foils in the WM task and the remaining eight were used as foils in the final memory test; the assignment of each face to targets vs. foils was determined pseudo-randomly for each participant so that all conditions had the same number of females and males. Each of the 24 faces chosen as target were further pseudo-randomly assigned to one of the three WM test item type conditions for each participant while matching the number of male and female images across the conditions. Another eight faces (4 males and 4 females) were used in booster trials.

Objects.

We used 208 pairs of photo objects that shared the same verbal label but had different perceptual features (e.g., color, shape and orientation). They were obtained from previous research (Brady et al., 2008; Kensinger et al., 2006; Sakaki et al., 2014) and the Internet. As we did for faces, from each image pair, one of them was used in the initial presentation as a memorandum and the other was used in a working memory test; which image was used as a memorandum was counterbalanced across participants.

The objects were split into two stimulus sets (one for the CS+ and the other for the CS− condition; the assignment was counterbalanced across participants). The 104 objects in each set were randomly assigned to one of the following conditions for each participant: 24 objects were used as targets in the multiple object condition, 24 objects were used as distractors in the multiple condition, 24 were used as distractors in the solo condition, 8 foils in the WM test and 24 foils in the last memory test. Those assigned to targets were further randomly assigned to one of the three WM test item type conditions. In addition, we used an additional 192 objects as two more distractors presented in each trial and 8 object pairs in booster trials.

Procedures and design.

The procedures were based on Studies 1 and 2. After filling out the same set of questionnaires as Studies 1 and 2, participants completed the fear-conditioning procedure; the average shock intensity determined for each participant was similar to that of Studies 1 and 2 (M = 1.45 mA; SD = 0.52). Participants next completed the questionnaire on their mood (Watson et al., 1988), followed by the learning phase. The learning phase was similar to that in Studies 1 and 2 (Figure 4), except that a) participants were shown four objects rather than four faces in the multiple condition (multiple object condition) and b) each trial started with spatial cues to indicate the location of high-priority images: trials in the multiple object condition started with four red boxes for 500 ms, while trials in the solo face condition started with a red box for 500 ms which was shown at the location of a subsequent face.

Figure 4.

Procedures in the learning phase in Study 3. Participants learned four objects in the multiple condition and one face in the solo condition, while faces in the solo condition were preceded by cues and highlighted to enhance their priority. Face images were taken from the TarrLab face database for illustration purpose only (stimulus images courtesy of Michael J. Tarr, Center for the Neural Basis of Cognition, Carnegie Mellon University, http://www.tarrlab.org).

The learning phase included 96 trials: 8 for each of the 2 (arousal: CS+ vs. CS−) × 2 (study set type: multiple vs. solo) × 3 (WM test item type: same, similar vs. new) conditions. We also included an additional 12 booster trials, where participants received stimulation following CS+ tone. These 108 trials were divided into three blocks, each of which included 32 main trials and 4 booster trials in a randomized order.

After participants completed the learning phase, they filled out a demographic questionnaire and completed a recognition test. The recognition test included 48 faces (one from each solo trial), 48 objects (one from each solo trial), 32 objects that were tested during the WM test (one from each multiple trial with the same and similar WM test), 64 un-tested objects from the multiple condition (one from each multiple trial with the same and similar WM test and two from each multiple trial with the new WM test), 16 new faces and 48 new objects. Images were presented in a randomized order, irrespective of whether they were faces or objects. For each image, participants were asked to indicate whether the image was old or new by key press.

Results and Discussion

Fear conditioning.

Participants showed high accuracy in discriminating the two tones during the fear-conditioning phase (Table 1) with no significant differences across the conditions (p = .95). They also showed higher SCR in the CS+ without shock condition than in the CS− condition (Figure 5A), t(39) = 3.58, p = 0.0009, d = .57.

Figure 5.

Results from Study 3. (A) CS+ induced higher SCRs than did CS− during the conditioning phase. (B) During the final memory test, emotion enhanced memory for solo high-priority faces. (C) Performance in the working memory test. Error bars represent standard errors of means.

Final memory for faces.

We next examined performance during the final memory test to test our main prediction. One participant was at chance when responding to foils (M < .55); data from this participant were not included in analyses reported in this section. The remaining participants were significantly better than chance in responding to foils (M = .79), t(42) = 24.07, p < .0001, d = 3.85.

To address whether arousal enhances memory for faces in the solo condition, a 2 (arousal) × 3 (WM test item type) ANOVA was performed on the proportion of correctly identified faces. This ANOVA revealed a significant effect of arousal, F(1, 38) = 4.15, p = .049, d = 0.22, with no significant effects of WM test (p = .29) and no significant interactions (p = .98). Participants showed better memory for faces in the CS+ condition than in the CS− condition (Figure 5B), suggesting that emotional arousal enhances memory for solo high-priority items when they have sufficient priority.

Final memory for objects.

We next tested the effects of emotion on memory for objects. First, we examined memory performance for objects that were tested in the WM test in the multiple condition by a 2 (arousal: CS+ vs. CS−) × 2 (WM test item type: same vs. similar) ANOVA. Note that objects in the solo condition were never tested during the WM test and therefore not included in this analysis. This ANOVA revealed a significant effect of WM test item type, F(1, 38) = 8.01, p = .007, ηp² = 17, reflecting better memory for the same (M = 91) than the similar condition (M = .83). But neither the main effect of arousal (p = .69) nor the interaction (p = .87) was significant. Participants remembered objects equally well irrespective of arousal both in the same (M_CS+ = .92, M_CS− = .91) and in the similar condition (M_CS+ = .84, M_CS− = .83).

We also examined memory performance for objects that were not tested in the WM test by another 2 (arousal: CS+ vs. CS−) × 2 (study set type: solo vs. multiple) ANOVA. There was a significant effect of study set type, F(1, 38) = 225.74, p < .0001, ηp² = .85, suggesting better memory in the multiple condition (M = .61) than in the solo condition (M = .14). This is not surprising as participants did not expect a memory test for objects from the solo condition. More importantly, neither the main effect of arousal (p = .94), nor the arousal-by-study set type interaction (p = .88) was significant. Once again, participants showed similar performance irrespective of arousal both in the solo (M_CS+ = .15, M_CS− = .14) and multiple conditions (M_CS+ = .61, M_CS− = .61). In summary, memory for objects did not show any effects of arousal (see General Discussion for further discussion regarding this result).

Working memory.

Finally, a 2 (arousal) × 2 (study set type) × 3 (WM test item type) ANOVA was performed on the correct response rate during the WM test (Figure 5C) to examine the effects of arousal on participants’ ability to briefly maintain presented stimuli. This ANOVA revealed significant main effects of study set type, F(1, 39) = 132.09, p < .0001, ηp² = 82, and of WM test item type, F(2, 78) = 68.49, p < .0001, ηp² = 66. There was also a significant interaction between them, F(2, 78) = 29.45, p < .0001, ηp² = 39. But neither the arousal-by-study set type interaction (p = .86) nor any other effects including arousal was significant (ps > .30). We also calculated the pattern separation index but it did not show significant effects of arousal (p = 31).

General Discussion across Studies 1 through 3

Across Studies 1 through 3, participants’ performance in the final memory test was consistent with GANE. In Studies 1 and 2, we found that emotional arousal impaired memory for multiple high-priority faces. In Study 3, we found that emotional arousal enhanced memory for solo high-priority faces. These results suggest that emotional arousal amplifies competitions across goal-relevant items and impairs memory for goal-relevant items when there are multiple high-priority items. However, contrary to our prediction, the enhancement effects of arousal on high-priority stimuli were not significant in Studies 1 and 2, where high-priority faces were not cued in advance. Thus, the results were more nuanced than our initial expectation.

Simulation

To test whether our behavioral results are consistent with GANE, we extended a recently developed computational model based on GANE (Lee et al., 2018) which concerns the effects of arousal on perception. Given that the current study concerns the effects of arousal on memory, we added the Hebbian learning mechanisms and a few more assumptions to explain the effects of NE on memory processing.

In brief, the model has an input, hidden and output layers (Figure 6). Each unit in the input layer represents one stimulus in a localist manner. Likewise, each unit in the output layer represents one stimulus. The model included 80 units in each of the three layers. The input units influenced their corresponding output units via an intermediary hidden layer. All units were fully connected with each other within each layer as well as with units in the adjacent layer but with different strengths; connection strengths were determined based on back-propagation algorithm so that activation of each input unit would activate one unit in the hidden layer and this hidden unit in turn activated the corresponding output unit² (see Appendix 1 for further details of the pre-training procedures). The model is based on standard neural network models but has a few critical parameters to represent GANE’s assumptions (see Appendix 1 for further details): a) excitatory effects of NE on activation via glutamate release to enable the positive feedback loop between NE and glutamate (Glutamate_i,t in Equation 8), b) inhibitory effects of NE on competing representations via GABAergic signals (GABA_k,t in Equation 8), c) a learning_rate for LTP based on β receptors (LTP_learning_rate in Equation 9) and d) a learning_rate for LTD based on α1-adrenergic receptors (LTD_learning_rate in Equation 9).

Figure 6.

A schematic representation of our neural network model. The figure displays only two of the input and output units for simplicity.

We ran a simulation experiment and tested whether our computational model produces the same pattern as the psychological experiments. The procedures in the simulation mimicked those in the psychological experiments. Specifically, we had the model learn either multiple high-priority items (multiple condition) or one high-priority item (solo condition) followed by the manipulation of arousal. If arousal amplifies the competition across high-priority items, arousal should enhance memory in the solo condition but impair memory in the multiple condition.

Method

Design.

There were 80 input units in each layer of our model, which allowed us to have 80 items in simulation (one input unit represented one item in a localist manner). The 80 items were assigned to one of the four conditions: 2 (arousal: arousal, no-arousal) × 2 (study set type: solo, multiple).

Simulation procedure.

First, the model completed a learning phase, where it was presented with one high-priority stimulus and three to-be-ignored stimuli (solo condition) or four high-priority stimuli (multiple condition). The learning phase included 20 trials (five trials from each condition) in a randomized order and each trial included 50 time steps. To manipulate post-encoding arousal, we included two stages in each trial of the learning phase: a) an encoding stage (1–20 time steps), where the model encodes high-priority items and b) a maintenance stage (21–50 time steps), where the model no longer receives external inputs but holds the information whilst updating its status at every time step.

The procedures used in the encoding stage were different between the solo vs. multiple conditions. In the solo condition, the model received the external input value of 1.0 (hard-clamping) for one item (i.e., a high-priority item; hard-clamping), but received the external input value of 0.5 for the other three items (low-priority items; hard- clamping). In contrast, in the multiple condition, we used different procedures. One possible assumption for participants’ behaviors in the multiple condition is that they always processed all four items in parallel and paid equal attention to all of them. However, it is also possible that participants engaged in serial processing, such that they paid attention to a certain item at one point but moved their attention to another item in a next time point, followed by an attention to another item in a following time point. We incorporated such participants’ serial behavior (see Appendix 2 for the results from the model based on the parallel processing during the encoding stage). Specifically, at every time step, one of the four input units was randomly selected and received the external input value of 1.0, whereas the remaining three received the value of 0.5 at that time step; each of the four units had the equal chance to be selected at every time step and this random selection was repeated for 20 time steps in trials of the multiple condition.

This encoding stage was followed by the maintenance stage, where we used the same procedure across the condition. In the maintenance stage, all external input values were set to zero, while the model updated its internal status (i.e., activations in the hidden and output units). To manipulate arousal, we applied the arousal manipulation (Equation 6) after the 30^th time step in the arousal condition (31–50 time steps). In contrast, (Equation 7) was applied in the non-arousal condition, as well as 1–30^th timesteps in the arousal condition. After the 50^th time step, the weights were updated through the learning algorithms (Equations 9–10).

After the learning phase of all the 80 items, we tested the models’ memory for each item. The learning algorithms applied during the learning phase allowed the network to activate the output units of the studied items more strongly than before the learning phase. In the test phase, each item was presented one by one with external input value of 1.0 and the network updated its status for 20 time steps (i.e., the same time steps as the encoding phase during the learning phase). The resultant output value in the target output unit was measured and was taken as a measure of memory performance (an index of item’s ‘familiarity’). Similar approaches have been widely used in past simulation studies (e.g., Norman, Newman, & Detre, 2007; Norman & O’Reilly, 2003).

Results

The ‘familiarity’ value was enhanced by arousal in the solo condition, but impaired by arousal in the multiple condition (Figure 7A left panel). Thus, the model showed the expected interaction between arousal and the number of high-priority items. To understand the effects of arousal during learning, we next examined NE levels during the learning phase and found that NE levels play a critical role in the opposing effects of arousal between the solo vs. multiple conditions. In the solo condition (Figure 8A), the NE level reached the threshold to activate β-receptors under arousal and these units did not receive strong inhibitory signals as there were no other items that showed strong activity, which helped maintain activation of β-receptors and induce LTP. In contrast, in the multiple condition, when we examined the average NE levels across four goal-relevant items, the average NE level was below the threshold to activate low- affinity β-receptors (Figure 8B). However, a closer inspection of the model revealed that local NE levels differed across four high-priority items; some items were initially activated more strongly than other items³ and the local NE levels reached the threshold to activate β-receptors for these stronger items (Figure 8C). This resulted in increased GABAergic inhibitory signals from these items, leading to increased suppression of other weaker goal-relevant items. Moreover, the stronger items also mutually suppressed each other and as a result, local NE levels quickly decreased even for these items. This resulted in the activation of high-affinity α1-receptors to induce inhibitory LTD for all four items.

Figure 7.

Simulation results for the arousal-induced competition. (A) The model showed that arousal enhances memory in the solo condition but impairs memory in the multiple condition. The model also reproduced the weaker enhancement effects of arousal observed in Studies 1 and 2. Data were averaged across 20 simulations. Error bars represent standard errors. (B) The results from our three studies were summarized to make it easier to compare the results from our simulation and psychological data.

Figure 8.

Local NE values in the output unit during the learning phase in (A) the solo condition, (B) the multiple condition (here we present the averaged NE values over four items) and (C) the multiple condition (separately for strongly activated items and weakly activated items). To illustrate the threshold to activate β receptors (a solid line) and the threshold to activate α1 receptors (a gray dashed line), raw NE values were log-transformed after converting them into nMol and shown in the figures. Data from one of the 20 simulations.

Discussion

We found that the model’s ‘familiarity’ index was impaired by arousal when there were multiple goal-relevant items, but enhanced by arousal when there was only one goal-relevant item. These results suggest that arousal amplifies competition among high-priority representations and can impair memory for high-priority information when there are multiple high-priority items but facilitate memory for solo high-priority information.

Simulation of Studies 1 and 2

As mentioned above, the enhancing effects of arousal in the solo condition were not significant in Studies 1 and 2. Thus, one remaining question is whether the model can reproduce the weaker effects of arousal observed in the solo condition in the first two studies compared with Study 3. Although there were several differences in the procedures between the first two studies and Study 3, one critical difference is that high-priority items were cued in advance in Study 3 but not in the first two studies. In Study 3, faces were preceded by a red spatial cue, which helped participants focus on faces without looking at other images, which was particularly helpful in the solo condition. In contrast, in Studies 1 and 2, participants did not know the location of faces in advance and needed to look at other images briefly to find out high-priority images.

To test whether this face identification process is responsible for arousal’s weaker enhancement effects in the first two studies, we ran another simulation where we included an identification stage at the beginning of the trials: a) an identification stage (1–10 time steps), where the model receives the same inputs for all four items to identify high-priority items irrespective of the conditions, b) the encoding stage (11–20 time steps), where the model receives stronger input for the single high-priority item, and c) the maintenance stage (21–50 time steps), where the model no longer receives external inputs but holds the information whilst updating its status at every time step. During the identification stage, irrespective of whether it was the solo vs. multiple condition, we used the same procedure as the encoding stage of the multiple condition in the main simulation. All the other procedures were the same as the main simulation.

In this simulation, we found only a small enhancement effect of arousal in the solo condition (Figure 7A; right panel) as observed in Studies 1 and 2 (Figure 7B). Thus, our GANE-based model showed similar patterns to our behavioral results. The model’s ‘familiarity’ index was impaired by arousal when there were multiple goal -relevant items, but enhanced by arousal when there was only one goal-relevant item. In addition, when we changed the procedures to make them similar to those in Studies 1 and 2, the model demonstrated a smaller enhancement effect of arousal in the solo condition.

Model evaluation

Taken together, all the behavioral data through Studies 1 to 3 have been simulated by our model in a qualitative way. Assessing a quantitative fit was challenging because individual observations from the simulation and the human data are not directly comparable. Nevertheless, it was possible to assess the fit between the two by predicting the averaged human data presented in Studies 1 to 3 (Figure 7B) by the model data presented in Figure 7A. A linear regression showed a highly significant relationship between the model and the human data, with more than 98.40% of the available variance in the humans’ performance accurately predicted by the values derived from the simulation. These results suggest that while our behavioral results are not as clear as we expected, these results are consistent with GANE.

We also tested whether the model was able to offer a good qualitative/quantitative fit to the behavioral data if any of the critical parameter values was set to zero. This approach is helpful to evaluate whether the complexity of our model is necessary or not. Although the complexity of our model is due to the complexity of NE’s neurophysiological effects and we used the same model as the one used in the previous study (Lee et al., 2018), it is still crucial to evaluate the necessity of these mechanisms to replicate the behavioral pattern. Figure 9 shows the outcomes of a series of these control simulations where we removed parameters crucial for GANE. These parameters include one for the excitatory effects of NE on activation via glutamate release ( Glutamate_i,t in Equation 8), one for the inhibitory effects of NE on activation via GABAergic signals (GABA_K,t in Equation 8) and NE-based modulations on learning_rate for LTP and LTD (see Equation 9). The results indicated that removing any of these key parameters led to either a low quantitative fit (i.e., lower prediction accuracy shown in lower R² values) or a failure to simulate the enhancement effect on the solo condition or impairment effect on the multiple condition in Studies 1 and 2 (i.e., low qualitative fit). Taken together, our results suggest that all the parameters in the model were important.

Figure 9.

The model’s behavior and its fit with behavioral data when changing key parameters for GANE. Data were averaged across 20 simulations.

General Discussion

There is ample evidence that neuromodulators affect a range of our behaviors and cognitive processes (Aston-Jones & Cohen, 2005; K. C. Berridge & Robinson, 1998; Bouret & Sara, 2005; Lisman, Grace, & Duzel, 2011; Sara, 2009). The literature of emotion and memory is not an exception. Previous studies have revealed a role for NE, a neuromodulator released from LC when encountering emotional events, in explaining the effects of emotion on memory (for reviews see Markovic et al., 2014; McIntyre et al., 2012). Since noradrenergic innervation is widespread across the brain (C. W. Berridge & Waterhouse, 2003), past studies often take the perspective of the broad and diffuse effects of NE by positing that NE has similar effects across regions or focus on the NE action in the amygdala. However, these theories do not clearly explain how emotional arousal sometimes enhances and sometimes impair memory in a retrograde manner via the same noradrenergic mechanisms.

A recently proposed model, GANE, addresses this issue and focuses on the local and synaptic-level NE mechanisms (Mather et al., 2016). According to GANE, high- priority representations are enhanced by arousal via local synaptic NE regulation mechanisms. In the current study, we tested whether GANE is a plausible model to explain the effects of arousal on memory by instantiating it into a computational model. According to GANE, there is a boundary condition for the emotional arousal’s enhancing effects on memory for high-priority information: While emotional arousal should enhance memory for solo high-priority information, it should amplify competition among representations and impair memory for high-priority information when there are multiple high-priorty items. Across three behavioral studies, we found that the number of high- priority items plays role in determining the effects of arousal on memory as expected by GANE: arousal enhances memory for goal-relevant information when there is only one goal-relevant item but impairs memory for goal-relevant information when there are multiple goal-relevant items. We also developed our computational model based on previous physiological studies and found that our model reproduces the results from our psychological experiments. Together, the current results suggest the importance of the neurobiological mechanisms posited in GANE to explain the retrograde effects of arousal on memory.

One puzzling result from the current study is the lack of the effects of emotional arousal on memory for objects in Study 3. In the current study, we included objects from a wide range of categories and therefore objects were more distinctive than faces. But even if objects were distinctive, arousal presumably should still impair memory for objects in the multiple object condition in Study 3 based on performance for faces in the multiple face condition in Studies 1 and 2. But arousal did not significantly affect memory for objects.

These differences in the effects of arousal on inhibition due to competition between items may be due to the differences in how distinctive faces were from each other versus how distinctive objects were from each other. In the present model, each processing unit in the hidden layer sends negative inputs to other units via GABAergic mechanisms and the size of these lateral inhibitory signals are set to the same value across all the interacting units within the same layer for the sake of simplicity. But lateral inhibition is typically observed only in neighboring neurons (Markram et al., 2004) and the brain is organized for each semantic category (e.g., Downing, Chan, Peelen, Dodds, & Kanwisher, 2006). Thus, it is possible that processing units for similar items are connected with each other with highly negative weights that allow stronger inhibitory signals, whereas processing units for distinctive items from different categories are connected with each other with smaller negative weights that allow weaker inhibitory signals. Critically, according to our model, the arousal-induced impairment effects are largely due to the GABAergic mechanisms (see Equation 8 in Appendix 1). Thus, consistent with the behavioral findings, our model also suggests that arousal’s impairment effects on memory are weak when high-priority items are distinctive from each other or when low-priority items are distinctive from high-priority items. Indeed, when we reduced the value of GABA_i,t parameter from 0.15 (standard model) to 0.10, the model also showed a smaller effect of impairment in the multiple condition (M_Arousal = .636, M_NoArousal = .643) than the standard impairment effect in the main simulation (M_Arousal = .596, M_NoArousal = .643). In summary, our model reproduced behavioral results from the three experiments with the same set of assumptions and the same set of parameters as the one from a previous study (Lee et al., 2018) -- except for the additional parameters that are relevant to learning algorithm.

It should be noted that while across three studies, we observed results consistent with the model and GANE, none of the three studies showed a significant double dissociation. As we just described, in Study 3, we found emotion-induced enhancement effects for solo high-priority solo items but did not find a significant effect for emotion-induced impairment for multiple high-priority objects (though this lack of significant effect was consistent with our model). Likewise, in Studies 1 and 2, we observed the emotion-induced impairment effect for multiple high-priority items but we did not find a significant emotion-induced facilitation effect for solo high-priority items (although once again the lack of significant effects was reproduced by our model). Previous studies also sometimes demonstrated that arousal impairs memory for low-priority items but does not necessarily facilitate memory for high-priority items (Clewett, Sakaki, Nielsen, Petzinger, & Mather, 2017; Ponzio & Mather, 2014). Other research also documented that arousal enhances memory for high-priority items but does not necessarily impair memory for low-priority items (Lee et al., 2015).

These results suggest that there are thresholds for items’ signal strength to be enhanced by arousal or impaired by arousal. In other words, arousal would not enhance memory for items that are relatively high in priority in the context but do not exceed the threshold for the enhancement effects. Likewise, arousal would not impair memory for items that are relatively low in priority in the context but still have sufficiently high levels of signal strength beyond the threshold to induce the impairing effects of arousal. These thresholds may be related to the biological threshold to activate low-affinity β receptors that induce LTP and the threshold to activate high-affinity al receptors that induce LTD.

It is unclear how these biological thresholds translate to behavioral manipulations. However, according to our model and the current findings, there are several factors that should interact with initial input strength in determining the strength of item representations in the brain and thus affecting the effects of arousal (Figure 10). The first factor is the number of competitors. When there is no competitor, the target item representations would not receive any inhibitory effects from competitors and thus maintain strong signals. But as the number of competitors increases, they compete with the high-priority target via the GABAergic mechanisms, resulting in the lower signal strength for the high-priority target which would allow only for the impairing effects of arousal. The second factor is semantic distinctiveness of items as described above, such that high-priority items are more likely to be suppressed when other items are semantically similar than they are semantically distinctive. The other factors, which are not completely separable from the first factor, are the signal strength of competitors and the difference in input signal strength between high-priority targets and low-priority competitors. Specifically, when the signal strength of low-priority competitors is low and there is a large difference in the signal strength between high- and low- priority representations, high-priority representations would be able to release a larger amount of glutamate and NE to help them enhance signal strength, without receiving much inhibitory signals from low-priority representations, leading to the facilitation effects of arousal. But when there is only a small difference in the signal strength between high- and low- priority representations and when the signal strength of low- priority representaions is high, the high-priority representations would receive strong inhibitory signals, thus leading to the impairing effects of arousal. The spacial cues we used in Study 3 presumably helped to enhance the signal difference in high- vs. low- priority representations by ensuring that participants allocated most of their attention to a single high-priority item from the beginning of a trial. Importantly, these factors should not act in an isolation but interact with one another (for example, even when low-priority representations have high signal strength, its inhibitory effects on high-priority representations are the function of semantic similarity between them). Future research is required to fully test the interaction across these factors.

Figure 10.

The potential factors that determine signal strength of high-priority target representations and affect whether the item shows facilitative or impairing effects of arousal.

In addition, it is possible that there are additional parameters/mechanisms that play roles in determining how arousal affects memory beyond what we posit in our model. For example, one recent study shows that the enhancement effects of arousal for high-priority items were related to prolonged levels of salivary alpha amylase (Clewett, Sakaki, Nielsen, et al., 2017) which is known to be correlated to tonic pupil size and tonic activity of the LC-NE system (C. J. Anderson, Colombo, & Unruh, 2013). Thus, the arousal-induced enhancement effects might be due to the interaction between the phasic NE effects modelled in this study and the tonic NE effects. Future research needs to fully understand how tonic and phasic NE activity interact in modulating the effects of emotion on memory.

The notion that NE is important for emotion-memory interactions is not new. Previous studies also suggest that NE plays major roles in the emotional modulation of memory (Hurlemann et al., 2005; Markovic et al., 2014; McIntyre et al., 2012; Roozendaal et al., 2008; Strange et al., 2003). However, it has been unclear how the LC-NE system sometimes allows for retrograde enhancement effects of emotion on memory (Roozendaal et al., 2008) but sometimes leads to retrograde impairment effects of emotion on memory (Hurlemann et al., 2005; Strange et al., 2003). The current research suggests that local NE levels are key in determining whether emotional arousal leads to memory impairment or enhancement in a retrograde manner.

In another line of research, researchers have examined how encountering emotional stimuli interrupts ongoing executive control tasks. For example, when task-irrelevant emotional distractors capture attention, these emotional distractors are remembered well in long-term memory, but they diminish activity in the dorsal executive brain regions, resulting in the disrupted working memory performance for other neutral stimuli (Dolcos, Diaz-Granados, Wang, & McCarthy, 2008; Dolcos et al., 2013; Dolcos & McCarthy, 2006; Shafer & Dolcos, 2012). These impairing effects in the working memory have been explained by positing that the ventral affective system which is responsible for processing of emotional stimuli competes with the dorsal executive system, leading to poor performance in tasks that require the dorsal executive system (Bush, Luu, & Posner, 2000; Dolcos, Iordan, & Dolcos, 2011). However, emotional responses are often associated with both ventral and dorsal prefrontal cortical regions (e.g., Phan, Wager, Taylor, & Liberzon, 2002), raising the question of whether these two systems are always competing with each other. In contrast, according to our model, emotional arousal triggers the local self-regulation mechanisms due to the interaction between NE and glutamate, which results in impairing effects on signals from whatever stimuli that have lower priority. Critically, emotional stimuli tend to have higher priority than other neutral stimuli due to individuals’ goals (e.g., emotional stimuli may activate one’s goal to maintain their survival) and/or higher bottom-up saliency (e.g., blood is red and salient than other stimuli). Thus, our model can provide a parsimonious account (without positing separate neural systems) about how and why emotional stimuli often disrupt ongoing executive function performance for other non-emotional materials while emotional stimuli themselves are remembered well.

Other computational models have also focused on the effects of the LC-NE system on cognitive processing (Eldar, Cohen, & Niv, 2013; Eldar, Niv, & Cohen, 2016; Nieuwenhuis, Gilzenrat, Holmes, & Cohen, 2005; Usher, Cohen, Servan-Schreiber, Rajkowski, & Aston-Jones, 1999). Based on the observation that NE increases signal- to-noise ratios (SNR) in neurons in sensory cortices (Foote, Freedman, & Oliver, 1975; Sara, 2009), the previous models increase the magnitude of the gain parameter in Equation (1) under arousal. Thus, both our model and these previous models predict that arousal leads to higher SNR. However, our model is different from the previous models in two important ways.

First, our model focuses on the local NE regulation mechanisms via its interaction with glutamate and GABA. In contrast, according to the previous models, the LC-NE system changes the steepness of the sigmoid curve in the output function of all units in the model. Thus, the previous models posit global and diffuse effects of NE across all neurons in the brain. These models reproduce the enhancement effects of arousal on signals for a higher-priority input and the impairment effects of arousal on signals for a lower-priority input when there are two competing items with different priority levels (Eldar et al., 2013). However, without modifications, these models would not predict that arousal impairs memory for high-priority information when there are multiple to-be-processed items with equally strong priority levels. According to these models, arousal leads to globally increased gain across all processing units, increasing the signal level of all strongly activated items, thereby resulting in arousal-induced memory enhancement effects for at least one of the competing goal-relevant items. However, our study did not find arousal-induced enhancement in variance in accuracy for the multiple condition (Study 1: Marousal = .37; Mno-arousal = .36, p = .63; Study 2: Marousal = .20; Mno-arousal = .19, p = .30; Study 3: Marousal = .27; Mno-arousal = .28, p = 26).

Second, our model explains how NE affects not only the signal strength during the processing/learning phase but also long-term memory strength. Based on GANE, our model posits that higher NE levels lead to LTP, whereas lower NE levels result in LTD. To the best of our knowledge, this is the first time such NE-based learning algorithms have been incorporated into a computational model, allowing simulation of human memory performance with the resultant model. Thus, our model is unique in being based on neurophysiological mechanisms concerning how NE levels are modulated locally and how NE interacts with different noradrenergic receptors to produce different effects both in a short-term manner (i.e., signal strength during initial processing) and in a long-term manner (i.e., synaptic connection strength after Hebbian learning).

There are several important questions for future studies to be noted. First, in the current model, we determined NE baseline levels and thresholds to activate α−1/β receptors based on previous observations in physiological studies. Yet as described above, it is not clear how the parameters of the current model map to real biophysical quantities. Future research should address this issue by testing the robustness of the model’s behaviour on this as well as other psychological findings while varying these values.

Second, in our behavioral studies, while we observed patterns consistent with GANE in the final memory test, performance in the WM test did not show selective enhancement effects of arousal on solo high-priority items. This suggests that the effects of arousal proposed in GANE are stronger after a delay possibly due to the role of NE on LTP and LTD as modelled by Equations 9–10 (see Appendix 1). However, previous research shows that arousal has selective facilitation effects for high-priority information during a short-term memory task (Sutherland & Mather, 2012) and an attention task (Lee et al., 2018; Lee, Sakaki, et al., 2014), both of which did not have a long delay. Future research should systematically manipulate the length of delay and test the interaction between arousal and delay length.

Third, participants in our behavioral studies were not balanced on their gender and we had more females than males across all three studies. Recent research indicates that females show stronger interaction between arousal and priority in memory than do males (Clewett, Sakaki, Huang, Nielsen, & Mather, 2017). Previous studies also suggest that ovarian hormone levels interacte with arousal in affecting memory in women (Clewett, Sakaki, Huang, et al., 2017; Nielsen, Barber, Chai, Clewett, & Mather, 2015). Future research should take into account sex and the action of ovarian hormones in the GANE model.

Another remaining question concerns whether CS+ induced similar emotional reactions across the multiple and solo conditions. In fact, previous research indicates that the ability to focus attention without being distracted by distractors decreases under high working memory load (Lavie, Hirst, de Fockert, & Viding, 2004). Given that working memory load should be higher in the multiple condition than in the solo condition, participants may be more distracted by the task-irrelevant CS+ in the multiple condition than in the solo condition, which may have resulted in greater emotional reactions to CS+ in the multiple condition than in the solo condition. While this account may be able to explain the selective impairment effects due to CS+ in the multiple condition in the first two studies, it cannot easily explain better memory for solo faces in the CS+ condition than in the CS− condition in Study 3. Therefore, our results are more consistent with the GANE’s prediction that arousal impairs memory for multiple high- priority items via increased competitions among high-priority representations. Nevertheless, future research should measure emotional reactions during the learning phase to address this possibility.

In conclusion, in the current paper, we found that emotional arousal amplifies competitions across goal-relevant items. We also found that a computational model which was developed in a recent separate study (Lee et al., 2018) can reproduce these behavioral results. This model should help clarify how and why emotional arousal sometimes enhances memory and sometimes impairs memory. The current model also provides a critical step to bridge a gap between human psychological research on emotion and memory and neurophysiological research on NE and glutamate. In addition, NE has been implicated not only in emotion and memory interactions, but also in general cognitive processing such as attention and learning (Aston-Jones & Cohen, 2005; Sara, 2009). Thus, the current results should also provide insights into how the LC-NE system supports human mental functioning in general.

Appendix 1: Model details and pre-training

Model details

The output of i-th unit (a_i) is computed by a standard sigmoid function as follows:

$$a_i=\frac{1}{1+e^{\left(-s_i*gain\right)}}$$ (1)

where gain is set to 1.0; s_i is the net input for unit i determined by weights and output values of all units that send their outputs to the unit (see Equation 2):

$$s_i=\sum _j\left(w_{j,i}*a_j\right)+b_i$$ (2)

where w_j.i is a weight of the connection from the sending unit j to the receiving unit i, and b_i is a bias input set to be −5. The network is operated in a continuous manner across N time steps, during which the time-integrated output value of each unit gradually changes by Equation (3):

$$timeintegrated{a}_{i.t}=a_{i.t-1}+0.1*\left(a_{i.t}-a_{i.t-1}\right)$$ (3)

where the second subscript t denotes the time step.

In addition to the above functions used in standard neural network models, we implemented core assumptions of GANE in our model. We used the same set of assumptions and the same set of parameters as Lee et al. (2018) except for the additional parameters that are relevant to learning algorithm (see below). The specific values of key parameters were changed from Lee et al. (2018) in order to offer the best fit to the study-specific behavioural data in this article. More specifically, we explored the model’s predictions across a range of parameter values using a coarse grid and compared it with data from the behavioral experiments (Plaut, 1997) (see Supplemental materials for the model’s prediction when we used other values for these parameters).

a). Glutamate signals priority.

According to GANE, glutamate is the brain’s primary excitatory neurotransmitter and signals priority, such that neurons that process high-priority information release a larger amount of glutamate than those that process low-priority information. Based on this assumption, in our model, we posit that the local unit activation level corresponds to the amount of glutamate released from the unit.

b). Each unit is associated with different levels of NE.

The second assumption of GANE is that NE concentration levels are controlled locally. To implement this, we introduced a unit-specific parameter NE_i,t to represent NE level for each unit i for each time step t. The baseline of NE_i,t is set to 0.000000001 (1e-9) mol/liter based on the baseline NE level observed in previous physiological studies (approx 1nM in the cortex; Slaney, Mabrouk, Porter-Stransky, Aragona, & Kennedy, 2013).

c). Local NE levels are modulated depending on arousal and local glutamate levels.

According to GANE, the elevated glutamate associated with high-priority neural representations stimulates NMDA receptors on LC axonal varicosities to release more NE to create NE hotspots when LC neurons are activated (i.e., immediately after phasic arousal is induced) and when the local glutamate neurons release sufficient amount of glutamate (Fink, Göthert, Molderings, & Schlicker, 1989; Lüscher & Malenka, 2012; Pittaluga & Raiteri, 1990). To implement this assumption, we defined NE_i,t under arousal as a decreasing function of the elapsed time after an arousing event and as an increasing function of the activity of the local unit as follows.

$$N{E}_{i,t}underarousal=\tau *{(a_{i,t})}^2*{0.9}^s$$ (4)

where s denotes a time step elapsed after the onset of an arousing event; a_i,t is the output value of the unit i at t-th time step; τ is a constant parameter which is set to 0.0001 to scale the product of the right side of Equation (4) to become a realistic NE concentration level (in mol/liter) under arousal based on previous physiological observations (Harley, Lalies, & Nutt, 1996; Salgado et al., 2012). The last term (0.9) was determined in a conservative way: Specifically, this should be less than 1 in order to realize the decreasing NE as a function of the elapsed time after an arousing event.

d). Phasic NE release is returned to baseline.

Due to the NE reuptake process (Amara & Kuhar, 1993), the increased NE level will gradually return to the baseline value of 0.000000001 (1e-9) (see Equation 5).

$$N{E}_{i,t}=0.9*N{E}_{i,t-1}+0.0000000001$$ (5)

By combining Equations (4) and (5), the local NE level for each unit is defined as follows:

NE levels after phasic arousal induction:

$$N{E}_{i,t}=\left[\tau *{\left(a_{i,t-1}\right)}^2*{0.9}^{s-1}\right]*0.9+0.0000000001$$ (6)

NE levels without phasic arousal:

$$N{E}_{i,t}=N{E}_{i,t-1}*0.9+0.0000000001$$ (7)

e). NE modulates additional glutamate release and GABAergic activity.

Another critical assumption of GANE is that higher NE concentrations in NE hot spots lead to stimulation of low-affinity β-adrenergic receptors, resulting in more glutamate release (e.g., Ferrero et al., 2013), and that strong glutamate signals (e.g., Xue, Atallah, & Scanziani, 2014) and high NE concentrations (e.g., Nai, Dong, Hayar, Linster, & Ennis, 2009) in NE hot spots stimulate GABAergic activity to inhibit other neurons that process competing representations.

To implement the interactions across NE, glutamate and GABA, we included two additional parameters for each unit i and each time t: a) Glutamate_i,t which refers to the excitatory effects of NE on glutamate release from the unit i via β-adrenergic receptors and b) GABA_k,t which refers to the inhibitory effects from unit k (other units within a layer) on unit i due to GABAergic activity. These two parameters are added to Equation (2) as follows:

$$s_{i.t}=\sum _j\left(w_{j.i}*a_{j.t}\right)+Glutamat{e}_{i,t}-\sum _{k}GAB{A}_{k,t}$$ (8)

where k ≠ i. The value of Glutamate_i,t and GABA_k,t are determined based on the local NE_i,t value as follows:

if NE_i,t ≤ 0.000007;

$$Glutamat{e}_{i,t}=0$$

$$GAB{A}_{i,t}=0$$

if NE_i,t > 0.000007;

$$Glutamat{e}_{i,t}=0.15$$

$$GAB{A}_{i,t}=0.15$$

Due to the action of Glutamate_i,t the outcome of Equation (8) gets larger when NE levels near unit i are higher than 0.000007 (7e-6) – a threshold value to activate p receptors. In contrast, due to the action of GABA_k,t the output of unit i gets smaller when NE levels near other units within the same layer (unit k) are higher than 0.000007 (7e-6). The threshold value to induce the β-driven facilitative effects was estimated based on physiological findings (Harley et al., 1996; Salgado et al., 2012).

Note that even after NE levels reach the threshold to activate β-adrenergic receptors, they will gradually decline and fall below that threshold via the reuptake process (see Equation 6). We assume that on these occasions, the effects of NE on glutamate and GABA gradually decline as follows:

$$Glutamat{e}_{i,t}=0.15*{0.9}^u$$

$$GAB{A}_{i,t}=0.15*{0.9}^u$$

where u steps have elapsed since NE_i,t gets below 0.000007 (7e-6).

f). NE gates LTP or LTD depending on its concentration levels.

According to GANE, NE should also modulate memory processing in addition to glutamate release. Specifically, high NE levels should allow low-affinity β-receptors to initiate long-term potentiation (LTP), whereas moderate or lower NE levels should stimulate only high-affinity α1-adrenoreceptors, resulting in long-term depression (LTD; Salgado et al., 2012). These effects are implemented by the following learning algorithms for each unit i after the last time step (t = N) in each learning trial:

$$\begin{gathered} w_{i,.,afte{r}_{-}updating}= \\ {w}_{i.,befor{e}_{-}updating}*{\left(1+LT{P}_{-}learnin{g}_{-}rate\right)}^v*{\left(1+LT{D}_{-}learnin{g}_{-}rate\right)}^w \end{gathered}$$ (9)

where w_i,. is all of the weights that project from unit i. The threshold values to induce either LTP driven by β-receptors or LTD driven by α1-receptors were determined based on previous findings that β-receptors require 20~30-fold higher concentrations of NE to be active than α1-receptors (Harley et al., 1996; Salgado et al., 2012). Specifically, v in Equation (9) denotes the number of time steps in which NE_i,t > 0.000007, and the value of LTP_leaming_rate was set to 0.015. In contrast, w in Equation (9) denotes the number of time steps in which 0.000007 ≥ NE_i,t > 0.0000003, and the value of LTD_learning_rate was set to −0.015.

In addition, the weight matrix was updated by the standard Hebbian learning algorithm in each learning trial:

$$w_{i,j,t+1}=w_{i,j,t}*(1+hebbia{n}_{-}learnin{g}_{-}rate*a_{i,t}*a_{j,t})$$ (10)

where hebbian_learning_rate was 0.15. Again, this learning algorithm was conducted at the end of trial for the sake of simplicity. Specifically, we saved the time-specific values of Equation (10) in each time step during the trial, and then this learning algorithm was repeated 50 times (i.e., the number of time steps) at the end of each trial by retrieving the stored values.

Network pre-training

The weight matrix of the model was determined so that activation of each input unit would activate one unit in the hidden layer (hereafter denoted as target hidden unit), and this unit in turn activated the corresponding output unit (hereafter denoted as target output unit). This means that units that are responsible for the same item were positively connected. The remaining links were inhibitory except for the self-recurrent connectivities. The connection strength was determined by pre-training the model with a back-propagation learning algorithm.

The pre-training stage consisted of a) training of a working memory task (where the model was required to reproduce a presented stimulus in the target output unit even after the input stimulus disappeared; each item was repeated for 100 times), b) a test of the memory task, c) training of a perception task (where the model was required to recognize and reproduce a presented stimulus in the target output unit during the stimulus presentation; each item was repeated for 100 times) and d) a test of the perception task. Items for which the model produced the correct answer during the tests were excluded from the subsequent training stage of the same task, and this cycle was repeated until the model produced the correct response to all items in both the memory test and the perception test.

Working memory task.

Each trial during the memory task included 50 time steps. In the first 20 steps, the model received an external input for one item, an input unit which corresponded to the presented item was activated (i.e., a_i = 1.0), and this activation spread to other units in the same and other layers. In the next 30 time steps (21–50 time steps), the model no longer received an external input and therefore the output value for the input unit was changed to 0 (i.e., a_i = 0.0) but the model updated its internal status (i.e., activations in the hidden and output units). If the activity of the target output unit is greater than 0.9 either at the penultimate time step (49^th time step) or the final time step (50^th time step), we considered that the model correctly maintained the presented stimulus, and therefore the weight matrix was maintained. In contrast, if the activity of the target output unit is smaller than 0.9 both at the 49^th and the 50^th time steps, we regarded that the model missed the presented stimulus and back-propagation learning algorithm was applied to the weight matrix to improve the model’s behavior in a next trial. In addition, if the activities of the other output units (i.e., non-target output units) were bigger than 0.1 both at the 49^th and the 50^th time steps, we regarded that the model produced a false alarm and applied the back-propagation algorithm.

After we completed the working memory training for 100 times for each item, we tested the model’s behavior. The procedures of this test were identical to those in the working memory task, except that we did not update the weight matrix based on the output. As done in the working memory task, we considered that the model correctly maintained the presented stimulus if the activity of the target output unit is greater than 0.9 either at the penultimate time step (49^th time step) or the final time step (50^th time step) and if the activities of the non-target output units were smaller than 0.1 both at the 49^th and the 50^th time steps.

Perception task.

Procedures of the perception task and a subsequent test were similar to those in the working memory task, except that we maintained the external input (i.e., a_i = 1.0) for 21–50 time steps.

Appendix 2

The results from simulation where we posited that participants engaged in parallel processing in the multiple condition.