Prediction errors and valence: from single units to multidimensional encoding in the amygdala

Abstract

The amygdala—one of the primary structures of the limbic system—is comprised of interconnected nuclei situated within the temporal lobe. It has a well-established role in the modulation of negative affective states, as well as in fear processing. However, its vast projections with diverse brain regions—ranging from the cortex to the brainstem—are suggestive of its more complex involvement in affective or motivational aspects of cognitive processing. The amygdala can play an invaluable role in context-dependent associative learning, unsigned prediction error learning, influencing outcome selection, and multidimensional encoding. In this review, we delve into the amygdala’s role in associative learning and outcome selection, emphasizing its intrinsic involvement in the appropriate context-dependent modulation of motivated behavior.

Introduction

Before you read this, take a second to think about the decisions that led you to this point…and yes, we mean this point (i.e., this exact moment in time in which you are willingly deciding to forgo any number of other things you could be doing in order to read a review about the amygdala and prediction error).

If you have done what we asked, we have hopefully made you aware of the numerous decisions you make in a day. These decisions vary from person to person, but include everything from what time you decided to wake up, to whether you decided to stop for coffee or make your own. This small sampling of what could easily average out to be several thousands of decisions in a typical day illustrates the fundamental role decision-making processes play in shaping our behavior.

Despite individual differences in the types of decisions made, and the myriad of contexts in which decisions are made, all decisions share certain common features. Generally speaking, decisions exist on a continuum--from trivial, to effortful. Many decisions--like how you get to work, or whether to get dressed are likely automatic, hardly passing for decisions at all. Other decisions--such as whether to accept or reject a manuscript--(hopefully) require more thought. Our decisions also exist on a continuum--from hardly worth remembering to memorable. Where a decision falls on that spectrum is often illustrated when you ask someone what they did that day and they reply “uh, nothing really”. While spectrums for degree of decision effort and decision memorability often correlate, it is also possible for them to diverge. For example--if someone offers to pay you a million dollars, no strings attached, the decision to accept this offer likely amounts to an easy, but memorable verdict.

Furthermore, decisions can shape our emotional context and disposition, which in turn modulates future decision making (e.g., an unexpected raise or a manuscript being accepted might facilitate a decision to go out and celebrate, whereas being fired or having your manuscript rejected may cause you to reevaluate previously established plans).

The field of neuroscience has devoted several decades to exploring decisional space and the factors that influence decision making. Historically, the amygdala has been limited to discussions of decision-making within the context of fear. However, many recent accounts have expanded this story, suggesting a role for amygdala in other cognitive processes and forms of learning that implicate, and even necessitate, interactions with frontal lobe areas--such as the anterior cingulate cortex (ACC), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC) (for reference, see [1,2]).

In the spirit of presenting a more holistic view of the amygdala’s contributions to behavior, this review will focus on the role of the amygdala--a small, almond-shaped brain region in the temporal lobe—in the context-dependent modulation of motivated behavior. We hope to show how existing work on the role of amygdala in prediction errors and associative learning--along with the anatomical work highlighting the interconnectedness of the amygdala with almost all prefrontal brain areas—suggest that the amygdala is responsible for tracking positively and negatively valenced outcomes in the service of influencing future actions.

Anatomy

The amygdala is comprised of a collection of small interconnected nuclei situated deep within the temporal lobe [1]. Although some interspecies variation exists, across evolution the anatomical profile of the amygdala is highly conserved, and largely shared across species--from lizards to humans [3]. Moreover, in species capable of more complex behaviors, such as rodents, non-human primates or humans, the relative size of the primary input region of the amygdala has increased, which is thought to be concordant with the substantial increase in the size of primate frontal areas [4].

Generally speaking, the amygdala is divided into two complexes--the basolateral amygdala (BLA)--which is comprised of lateral (LA), the basal (BA), and basomedial (BM) cell groups--and the central nucleus of the amygdala (CeA)--this nucleus is composed of the lateral (CeL) and medial (CeM) cell groups. While there is growing debate about the exact path of information flow through the amygdala [3], a simplified view of the flow of information is one in which information about the external environment is passed from the thalamus, hippocampus and sensory cortex to LA, which in turn sends projections to BA and BM, as well as unidirectional projections to CeA. Although highly interconnected, BLA and CeA differentially contribute to some behaviors, suggesting each complex of nuclei has unique input and output targets as well [5,6]. Importantly, the BLA is also highly interconnected with most frontal brain regions--sending and receiving input from ACC, mPFC, and OFC--as well as maintaining strong unidirectional outputs to striatum and the bed nucleus of the stria terminalis (BNST) [7–23]. In this light, the amygdala appears as a kind of junction, sending and receiving inputs from almost all brain regions associated with decision-making and action selection.

The Amygdala and Fear

Despite anatomical evidence linking the amygdala to brain areas important for learning and decision-making, early work focusing on the function of the amygdala tended to emphasize its role in emotional regulation, and--in particular--fear [24–29]. This focus on emotion was influenced by early human case studies such as that of Phineas Gage, who famously suffered an incident during which a tamping rod blasted through his skull, effectively and permanently severing connections between frontal and limbic brain regions [30,31]. Importantly, one of these severed connections was between the frontal cortex and amygdala, via the uncinated fasciculi [30,31]. As a result of the trauma, Gage was reported to exhibit fits of uncontrollable rage and dysregulated mood, leading many to speculate a role for the frontal areas modulating activity in more emotional limbic centers such as the amygdala [30,31].

In addition to Gage’s case, more controlled experimental work showed that bilateral ablation of the temporal lobe in monkeys resulted in reduced responding to emotionally significant stimuli--such as food, or fear-inducing visuals--as well as attenuated aggression [32,33]. These studies identified a condition, later known as Klüver-Bucy Syndrome, that results from bilateral lesion of the amygdala [32–34]. In addition to supporting much of the emotional dysregulation findings described in the Gage case study, this work in monkeys also revealed altered motivational function--as well as hypersexuality and hyperorality--clues that would become critical for future theoretical work on amygdala function [34]. Later work showed that amygdala lesions specifically impaired and delayed acquisition of learning shock-predictive cues, exemplified by attenuated behavioral responses to positive and negative cues [35]. Similar findings in response to shock/fear producing stimuli were reported in rodents [26,36] as well as humans [37,38]. Collectively, these early findings promulgated publications implicating the amygdala in fear, or--more specifically--fear learning.

The main thesis of this work is that the amygdala functions as a rapid detector of aversive events and is responsible for producing adaptive behaviors to mitigate the potential threat. In rodents, this idea was tested using conditioned freezing paradigms; in this experimental setup, a footshock (unconditioned stimulus; US) is paired with a previously neutral stimulus, such as a tone (conditioned stimulus; CS), and the degree to which a rodent exhibits freezing behavior (i.e., stops moving, a common reaction in rodents towards stimuli that elicit a defense response) is measured. With repeated pairing, rodents will exhibit freezing behavior in response to the CS alone, suggesting that a relationship between the CS and US has been learned.

Generally, conditioning studies have shown that the amygdala serves to establish connections between the neutral CS and a more affectively charged stimulus (either positively or negatively valenced) in a way that transfers this affective salience to effectively induce subsequent changes in motivated behavior. The amygdala’s established involvement in fear conditioning stems from experiments showing that LA neurons develop and maintain excitatory neural responses to an auditory cue (CS) that has been repeatedly paired with a footshock (US) [39–42], and from experiments showing that lesions to or disruption of LA specifically blocks the acquisition of freezing behavior [26,36,43]. The importance of this circuit has been further verified with optogenetic experiments demonstrating that when optical excitation of LA in rodents is paired with a neutral stimulus, that stimulation alone can produce freezing behavior independently of shock [44]. Similarly, it was shown that previously acquired auditory-cued conditioned fear memories could be reactivated by high-frequency optogenetic stimulation of auditory thalamus fibers to LA, even in the absence of the previously conditioned auditory stimulus [45].

Later, other work demonstrated that CeA lesions also blocked conditioned fear responses, suggesting a transfer of conditioned fear learning from LA to CeA [36,46,47]. Interestingly, the primary output of CeA—CeM--does not directly receive input from LA [28,48]. In order to understand how LA projections to CeA facilitate freezing behavior, researchers combined transgenic and cell labeling approaches to identify two subpopulations of cells that comprise the CeL nuclei. These two populations are either inhibited (CeL_OFF) or excited (CeL_ON) by auditory cues, and can be genetically identified by their expression of PKCδ--a protein expressed by CeL_OFF, but not CeL_ON, cells [49,50]. Studies have shown these cells engage in reciprocal inhibition; upon exposure to a CS, CeL_ON cells respond faster, inhibiting CeL_OFF cells; this leads to subsequent disinhibition of CeM, which facilitates conditioned freezing [49,50]. Additional work has revealed that certain long-range projection cells within CeL can bypass CeM altogether, which may also facilitate conditioned freezing [51] during a negatively valenced CS.

Despite these rich datasets implicating the amygdala in the acquisition, processing, and modulation of fear and aversion, other evidence suggests that this same circuitry can be co-opted to respond to a reduction of negatively valenced emotional states as well [52]. Original work quantified anxiety-like behavior in rodents using a highly validated behavioral assay known as the elevated plus maze (EPM). This maze is comprised of four arms—two of which are exposed, and two of which are enclosed. As rodents are adaptively more averse to exposed, open spaces, more time spent exploring the open arms of the EPM reflects attenuated anxiety-like behavior. Optogenetic stimulation of BLA cell bodies decreased time spent in the open arms; however, targeted stimulation of BLA-CeL projections produced an anxiolytic effect, increasing the amount of time mice spent in the open, less protected arms [53]. Amygdala involvement in the promotion of less anxious (i.e., less aversive) states is further supported by recent evidence suggesting that BLA terminal excitation activates CeL:PKCδ⁺ neurons, consequently promoting a less anxious state [54]. Other work has shown that optogenetic activation of BLA projections to BNST is also anxiolytic [55].

While these results clearly demonstrate a highly specific cellular architecture for fear learning in the amygdala, they also highlight that the amygdala is particularly well suited for high resolution tracking of behavioral events more generally. In particular, the data implicating the amygdala in feeding behavior and anxiolytic behavioral states suggests that a predominant focus on fear as the primary function of the amygdala is far too narrow.

The Amygdala and Reward

While an emphasis on fear and fear learning predominated early amygdala work, a separate area of research began to emerge attempting to explain some of the other behavioral differences associated with amygdala damage [32,33,35]. These findings include the observation that, in addition to emotion/affective changes in subjects’ behavior, subjects with amygdala lesions also failed to respond to predictive cues and sounds more generally [32,33,35]. This led some to speculate that the role of amygdala likely extended beyond fear to include the integration of multiple behavioral signals, including appetitive ones such as reward [56–62].

Evidence supporting this intuition came from work showing that lesions to CeA blocked conditioned reward-predictive orienting responses in rats [58]. This experiment measured orienting behavior (i.e., looking) toward either a light or auditory cue that had been paired with a small food reward. Although both groups of rats were physically able to orient toward the cue, rats with CeA lesions were unable to learn the associative relationship [58]. Several studies have shown that optogenetic activation of CeA or BLA causes rats to engage in compulsive reward-seeking behavior [63–65], even when faced with adverse consequences [63,64]. Conversely, optogenetic inhibition of BLA projections to the OFC or to the nucleus accumbens blocked cue-driven drug-seeking, highlighting the BLA’s role in associative drug-seeking [66]. Other work demonstrated that lesions to LA disrupted amphetamine-induced conditioned place preference in rats (i.e., favoring one side of a test arena due to its association with previous exposure to a drug), further highlighting the role that amygdala plays in outcome learning beyond fear [60].

Much like in the fear literature, neurons in BLA exhibit excitatory responses to auditory, visual, and olfactory cues that are paired with reward [67–71]. This excitatory response profile has been shown to develop through long-lasting enhancement of glutamatergic inputs from sensory thalamus onto LA [70]. In general, BLA appears to encode changes in the value of specific reward outcomes, which in turn influences behavior [5,59,61,62,72]. Evidence for this comes from experiments where animals are trained to associate two or more distinct actions with two or more unique food outcomes [5,59,61,62]. One of the food rewards is then devalued by allowing the animal to consume as much of that particular reward as possible, before again testing them on the original task. In control animals, the action associated with the selectively satiated reward is performed less, reflecting this devaluation. However, responding for the devalued reward persists in animals with BLA lesions, suggesting that BLA lesions impair the ability of animals to initially learn reward value information [61]. CeA lesions appear to disrupt instrumental responding more subtly--preserving the capacity to learn and track associations, but attenuating the ability to maintain representations of the motivational salience of an outcome [5].

Unsigned Prediction Errors and Valence

Data supporting a role for amygdala in reward clearly expands our understanding of functions of the amygdala and emphasizes its importance in responding to both fear and reward predictive cues. However, we must integrate these findings into a cohesive theory of amygdala function. By treating the amygdala as a center for stimulus outcome learning and applying principles of reinforcement learning, studies looking at prediction error (i.e., neural signals related to whether an outcome meets or fails to meet expectations) in the context of both reward and aversive outcomes have helped to partially unify theories of fear and reward in relation to amygdala function.

Early work using an olfactory-based Go/No-Go task revealed that large numbers of BLA neurons fire differentially, depending on whether an outcome is rewarding or aversive [67,68,73]. On this task, rats were trained to nose poke into an odor port, in which one odor (i.e., Odor 1) instructs rats to make a response to a nearby fluid well in order to receive a liquid sucrose reward (i.e., GO response) and the other (i.e., Odor 2) instructed the rat to withhold responding (i.e., No-Go), as failure to do so would result in the delivery of quinine, an aversive outcome (i.e., Negative-Go response) [67]. Rats were able to learn to discriminate between the two odors. Single unit recordings from OFC and BLA revealed that 22% of OFC and 36% BLA neurons differentially encoded the identity of the subsequent outcome (i.e., positive or negative) [67]. Moreover, some BLA neurons were particularly active on negative GO trials; this heightened activity preceded response outcome and thus seemed reflective of the impending aversive outcome [67].

Other neurons developed selectivity during presentation of odor cues (CS) reflecting the associated valence of the predicted outcome. Critically, discrimination at the level of single neurons was observed just prior to behavioral change (i.e., the point at which error rates dropped and remained low) indicative of learning, suggesting that BLA is important for discriminating between positive and negative outcomes [67]. Moreover, BLA is able to actively track these contingencies. Upon reversal of previously learned odor associations (e.g., Odor 1 now signaling No-Go and Odor 2 now signaling Go), BLA neurons that exhibited preference for the original association began showing preference for the newly rewarded odor [68]. In other words, BLA neurons mark when positive and negative events occur, and are able to track changes in value. This is illustrated in the single neuron example in Figure 1, which develops selectivity for cues that predict reward during discrimination learning, and subsequently tracks that outcome when contingencies are reversed. Importantly, these reward predictions preceded the onset of behavioral change, implicating BLA in the updating of behavior.

Figure 1.

BLA firing activity aligned to odor offset during an olfactory discrimination learning task (top panel is a schematic of the task). BLA single unit recordings revealed differential outcome encoding, and were able to track changes in value following reversals of previously learned odor associations. During early pre-criterion trials (left panel), there is no selective BLA activity. During post-criterion performance—after learning has occurred—neuronal activity is highly selective for the odor associated with reward (Fig. 1; “Post-criterion Selectivity”). Raster displays adapted with permission from Schoenbaum et al., 1999 [68].

This work also sheds light on the interaction between BLA and OFC--a brain region thought to be important for representing the value of behavioral strategies [74]. Lesions to BLA impair the OFC’s ability to adapt to reversals in odor associations [74]. This deficit appears to be due to a failure to develop cue-selective neural signals about the outcome of events in BLA, which in turn impairs OFC’s ability to adapt to changing task contingencies [74,75].

Other work employing a size-delay task [76] has demonstrated that BLA and CeA play different roles in supporting online contingency updating [77–79]. The size-delay task consists of four sixty-trial blocks, and allows for independent manipulations of reward value (10% liquid sucrose) through variations of delay (Blocks 1 and 2; short delay: 0.5s, long delay: 1–7 seconds) and size (Blocks 3 and 4; small: 1 bolus, large: 2 boli). On each trial, rats nose-poke into a central odor port to receive one of three odor cues, and then respond in the corresponding fluid well (left, right, or either well) to receive reward (Figures 2a and 2c). One odor signals reward in the left well (forced-choice), another indicates reward in the right well (forced-choice), and a third odor signals reward at either well (free choice). Optimal task performance requires rats to detect unexpected changes in reward value and update behavior accordingly to select the more favorable reward outcome on free-choice trials, while maintaining accurate responding on forced-choice trials. Critically, this paradigm allows for independent manipulation of both temporal and size properties of the reward, while also still allowing for assessment of prediction errors (Figure 2a) [77,79]. Block switches where an odor previously paired with a big reward was now paired with a small reward lead to negative prediction errors, whereas the converse would lead to positive prediction errors. Similarly, block switches where rewards were delivered earlier or later than expected respectively produced positive and negative prediction errors, which guide free-choice performance during optimal decision-making.

Figure 2.

a. The size-delay task consists of four sixty-trial blocks, and allows for independent manipulations of reward value (10% liquid sucrose) through variations of delay (Blocks 1 and 2; short delay: 0.5s, long delay: 1–7 seconds) and size (Blocks 3 and 4; small: 1 bolus, large: 2 boli). Blocks 1–4 are shown in the order performed (top to bottom—each row delineates a different block). Thus, during block 1 (row 1), rats responded after a ‘long’ delay or a ‘short’ delay to receive reward (actual starting direction – left/right - was counterbalanced in each block and is collapsed here). In block 2 (row 2), the locations of the ‘short’ delay and ‘long’ delay were reversed. In blocks 3–4 (rows 3–4), delays were held constant but the size of the reward (‘big’ or ‘small’) varied. Red rectangles denote when there was an unexpected upshift in reward value (start of blocks 2^sh, 3^bg, and 4^bg). b. Neural activity in BLA is increased in response to unexpected reward delivery and omission. Heat plots showing average activity over 58 BLA neurons. Red rectangles denote when there was an unexpected upshift in reward value (start of blocks 2^sh, 3^bg, and 4^bg). Activity throughout the course of the trials is plotted during the first and last ten trials in each training block (Blocks 1–4). Activity is shown, aligned to odor onset (‘align odor’—first and third columns of heat plots) and reward delivery (‘align reward’—second and fourth columns of heat plots). Line display (middle column, between heat plots) shows the rats’ behavior on free-choice trials (that occurred on 35% of trials per block). Value of 50% means that rats responded the same to both wells. c. Illustration of task; on each trial, rats nose-poke into a central odor port to receive one of three odor cues, and then respond in the corresponding fluid well (left, right, or either well) to receive reward. d. Distribution of indices [early-late/early+late] representing the difference in firing to reward delivery (1 s) and omission (1 s) during trials early and late trials after (i) up-shifts (in figure: x-axis; 2^sh, 3^bg, and 4^bg) and (ii) down-shifts (in figure: y-axis; 2^lo and 4^sm). Filled points in scatter plot indicate the number of cells that showed a main effect (p < 0.05) of learning (early vs late). Black diamonds indicate those neurons that also showed an interaction with shift-type (up vs. down-shift). e. Speed to orient to the odor port (i.e., measure of attention) increased following a block change, and decreased after learning occurred. f. BLA firing was correlated with faster odor port orienting (i.e., attention) at the beginning of the trial once shifts in reward value had occurred. Correlation between changes in firing in BLA on (trial n) and the orienting response on (trial n+2). sh = short; lo = long; bg = big; sm = small. Analysis and figures shown here include both free- and forced-choice trials. Error bars indicate SEM’s.

Rats were quickly able to learn to adapt to new associations. Single unit analysis of 284 BLA neurons revealed that at the time of reward, 70 neurons responded with increased firing 1s following reward delivery [79]. Moreover, 58 of the 70 increasing cells exhibited differential firing based on either the timing (Figure 2a, first two rows) or size (Figure 2a, last two rows) of the reward [79]. Interestingly, these outcome-selective neurons also exhibited changes in reward-related firing between the beginning and end of trial blocks (Figure 2b). Figure 2b shows that BLA activity in outcome-selective neurons was higher at the beginning of trial blocks--immediately following unexpected upshifts or downshifts in reward value--and declined as rats learned the new contingencies. On upshifts (Figure 2b, trials denoted by red rectangle; i.e., trials in which the odor associated with either the delayed or small reward respectively switched to being associated with immediate or big reward), BLA neurons exhibited strong firing 2–3 trials subsequent to the block change (Figure 2b), and diminished over the course of the trial block (Figure 2b) [79]. Similarly, on downshifts (i.e., trials where the odor associated with the immediate or big reward switched to being associated with delayed or small reward) also exhibited an initial increase in firing—when choice performance was poor (Figure 2b, middle column)--that diminished throughout the block (Figure 2b, heat plots) [79]. The effect of learning on BLA neuron firing was seen for both upshifts and downshifts in reward value (Figure 2d). Paralleling BLA signaling, the speed at which rats oriented to the odor port increased gradually following a shift in reward contingencies (Figure 2e). Further, this accelerated orientation to the odor port was correlated with increased BLA firing on subsequent trials (Figure 2f). Thus, these unsigned prediction errors were interpreted to reflect a redirection of attention to the associations needing to be updated (Figure 2f) [79]. Importantly, these signals seem to contribute to the attentional changes proposed by the amended Pearce-Hall classical conditioning model, in which changes in firing are influenced by a history of prediction errors over several trials [78,80,81]. Interestingly, however, subsequent results illustrated that these unsigned signals in BLA were partially dependent on the midbrain dopamine system, which classically signals signed reward prediction errors, suggesting that the two signals are intertwined [78]. Both of these signals are in stark contrast to firing observed in CeA in rats performing the same size-delay task, where single neuron firing in CeA contributed to signaling negative—but not positive—prediction errors. Omission responsive neurons comprised approximately 9% of the population of recorded cells. Moreover, 64% of these cells did fire for reward, however, firing in these cells in response to unexpected reward omission was always significantly higher [68].

The biological importance of this teaching signal is further illustrated when BLA function is examined across the lifespan [82]. Older rats (22–26 months of age) are slower and less accurate on the size delay task, but do ultimately modulate behavior in manner consistent with younger rats (3–6 months of age) [82]. Furthermore, neurons in the BLA of both old and young populations exhibit significant differential firing based on either the timing or size of reward; however, in older rats there are significantly fewer of these reward-sensitive neurons when compared to younger animals [82]. Moreover, the modulation in firing typically observed during upshifts and downshifts is absent in aged rats, whose neurons maintained a relatively constant level of firing from the beginning to the end of trial blocks [82]. This deficit was partially reflected behaviorally by aged rats choosing the high-valued reward less often than younger animals [82].

The presence of unsigned prediction errors, coupled with a clear sensitivity to discrepancies in expected outcome, led us to suspect that BLA may be an important contributor to inhibitory control [83]. Inhibitory control is the process by which the brain mitigates the simultaneous activation of opposing responses. Much research has focused on frontal and traditionally motor areas in this process; however, comparatively fewer studies have investigated the role of the amygdala. In order to assess how this signal arises, we recorded from the BLA of rats performing an odor discrimination task that pitted competing responses against one another. Two unique odors were associated with forced choice trials, and a third odor was indicative of a free choice trial. Responses on forced choice trials were identified as either congruent or incongruent, wherein congruent trials directed the rat to make a response in the direction rats were biased towards during free-choice trials, while an incongruent odor directed the rat to make a response in the opposite direction in order to receive reward. Failure to respond in the appropriate direction resulted in no reward. Consistent with our prediction, rats responded faster when the odor was congruent with their preferred response direction, and slower on incongruent trials. However, contrary to our prediction, neurons in BLA preferentially fired only on incongruent trials when an error was about to be committed [83]. This highly specific and predictive firing suggests that--in addition to tracking changes in reward--BLA also signals when an error is about to be committed. This preemptive error signaling may reflect a teaching signal in that knowledge of an impending error will likely slow responding on subsequent trials in an attempt to regain accuracy (i.e., conflict adaptation).

The amygdala is clearly able to encode changes in reward contingencies, as well as when rewards are omitted or--in the case of inhibitory control--when error leading to reward omission is likely. These teaching signals manifest at the single unit level, just prior to being behaviorally observed. This tracking of positive and negative outcomes is likely reflective of a kind of read-out about the overall state of the animal [84] where the valence and saliency of an event is encoded and in turn may influence future behavior, or specifically action selection [85]. Thus, within the immediate confines of a trial, the amygdala provides a trial-specific history of the outcomes of various past decisions. Like in the example at the beginning of this review, a collection of positive outcomes may influence a future decision to go out (e.g., because your manuscript was accepted, you may then decide to go and celebrate), just as a series of negative events may factor in (e.g., your paper may have been accepted, but feeling under the weather compounded with car troubles might diminish your enthusiasm for going out to celebrate). This kind of affective “memory” reflects a barometer for overall valence--rather than recalling a specific event or procedure, the amygdala seems to track the overall emotional state of the animal. How this state information in the amygdala then goes on to influence future action selection is less clear. Recent work in mice either trained to avoid shock or unable to escape shock (i.e. learned helplessness), suggests that connections between posterior aspects of BLA and the ventral CA1 of the hippocampus may encode this type of memory for emotional states [86]. Optogenetic disruption of BLA-CA1 connectivity abolished the responses associated with improved spatial memory performance in mice that learned to avoid shock, and stimulation of BLA-CA1 inputs potentiated spatial memory [86]. While it is unclear whether the BLA participates in the formation of emotional memory directly, it does appear to gate emotion-facilitated memory information and use it to modulate behavior, suggesting that that a core function of the amygdala might be to ensure that the overall valence of a collection of recent decisions/outcomes is factored into the decision-making process [87].

Multidimensional selectivity and the amygdala

While the intention of each of these accounts of amygdala function (e.g., fear, reward, and valence) is to more accurately characterize the role of the amygdala in behavior, these unidimensional explanations often fail to integrate findings in order to generate multidimensional theories about amygdala function. This failure is due, in part, to the tendency of unidimensional explanations to oversimplify function, combined with fact that these unidimensional accounts are often derived from experimental paradigms designed to tightly control and monitor all aspects of behavior--so as to better correlate neurophysiological measures with behavioral outcomes. While an emphasis on precise control of independent variables and operationalization is imperative, these efforts can also have the unintended consequence of limiting the scope and diminishing the generalizability of findings.

Single unit recordings from neurons in amygdala—as well as many other, particularly frontal, brain regions—have often revealed highly complex selectivity [88–94]. Recently, this has led to the proposal that a fundamental characteristic of individual amygdala neurons is multiselectivity (i.e., selectivity to multiple task features or behavioral variables) [92]. Indeed, a cursory read of this review alone would lead one to suspect that the amygdala is either functionally/ structurally divided into fear, reward, and valence encoding cell populations or--more likely--that the same cells across the amygdala encode multiple task features. In reality, this multiselectivity--or multidimensional encoding--of behavioral variables has been readily observable in the literature for many years [88,92].

One of the best recent examples of multidimensional coding by amygdala neurons in rodents--as well as a great example of the benefits of implementing fluid task structures to study brain function--comes from Kyriazi and colleagues [93]. Here, Kyriazi and colleagues sought to parse CS- and conditioned response (CR)-related activity to appetitive and aversive stimuli in BLA neurons to determine whether individual cells in the amygdala encode the CS, the CR, or both. Conventional models predict that specific subsets of valence-encoding neurons map unique behavioral responses onto appetitive and aversive stimuli. The authors correctly point out that these findings may be due to the fact that behavioral tasks, particularly those used in rodent research, often only allow for only one type of CR [93]. To account for this, they developed the Risk-Reward Interaction (RRI) task, which required rats to respond to both reward predicting and shock predicting cues. Importantly, this task allowed for the evaluation of contextualized risk and reward encoding. Unlike most tasks, shocks were restricted to only one section of the three-section behavioral apparatus, such that the authors could discriminate between active avoidance CRs (i.e., being in the signaled shock section and exiting before shock), passive avoidance CRs (i.e., refraining from entering/staying away from the signaled section), and reward seeking. The authors found that single BLA neurons concurrently and independently encode CSs (signaling both appetitive and aversive outcomes) and learned CRs (both approach and avoidance behaviors). These signals could potentially contribute to the aforementioned memory traces that influence conditioned emotional behaviors. Importantly, decoder analysis revealed distributed multiplexed ensemble activity, suggesting that most BLA neurons heterogeneously encode multiple task and stimulus features (Figure 3) [93]. Future research should look to evince which downstream neurons then receive inputs from different BLA ensemble populations in order to select subsequent behavioral adaptations.

Figure 3.

Multidimensional encoding of task-relevant variables. Decoder analysis of neurons recorded from BLA of rats performing the risk-reward interaction (RRI) task reveal individual cells (circles) encode various task-relevant variables including stimulus presentation (reward - blue, shock - pink) as well as the intended response (anticipation - green, active avoidance of the shock - orange). These signals appear at different time points within a trial suggesting that individual amygdala neurons represent multiple task dimensions. Figure adapted with permission from Kyriazi, Headley, & Pare, Neuron, 2018 [93].

Another example of multidimensional encoding in the amygdala was found in a task in which primates learned to associate reward or punishment with two different behavioral contexts [95]. Throughout the task, unsigned reversals of contingency-context associations occurred. This study found neurons in the amygdala that maintained context representations--in addition to encoding stimulus identity and reinforcement expectations--thus allowing primates to flexibly update their behavior throughout the task [95].

The encoding of multiple task-relevant features is thought to be highly adaptive and responsible for greater behavioral flexibility [89,90,92–95]. Specifically, the potential of the amygdala to encode linear or orthogonal representations of different task features may underlie the nuance or variability we see in behavior. These findings may also help explain why behavioral economic models of human behavior fail to fully account for the range—and, at times, inconsistency--of our responses. Understanding how higher dimensional encoding of task-related variables maps on to the execution of motor responses is imperative for our understanding of complex behavior.

Connectivity and Influence over Action Selection

Understanding how the amygdala influences future decision making/ action selection likely requires an examination of the interplay between frontal regions and the amygdala. We have already seen that amygdala neurons encode numerous decisional variables both individually, as well as holistically. We have also seen that cue-outcome associations formed by the BLA are important for updating value information in the OFC [74,75]. BLA lesions result in an impairment of reversal learning, in part due to failure of OFC to accurately map value onto actions [74,75].

The BLA is also highly interconnected to the rodent mPFC (i.e., prelimbic [PrL] and infralimbic [IL] regions), and projections from the BLA to the mPFC, and from the mPFC back to BLA, are anatomically dissociable [96]. The importance of this circuitry has mostly been studied in the context of fear, where PrL and IL subregions of mPFC are shown to often exert opposite roles in the facilitation of fear acquisition as well as extinction [97]. Using an auditory fear conditioning paradigm, inactivation of IL with a GABAA agonist--muscimol--prior to extinction training, impaired both acquisition of extinction and extinction memory without altering fear expression [97]. Conversely, PrL inactivation impaired fear expression without altering extinction memory [97]. Inactivation of BLA impaired both fear expression and extinction memory, suggesting that interaction between these two brain regions likely drives adaptive responding, although future research needs to examine specifically what this interaction is.

One possible clue is recent work demonstrating that populations of BLA neurons representing positive and negative valence are discrete and target different sub-regions in mPFC [98]. Negative valence neurons from BLA have been observed to project to superficial layers of PrL, while positive valence populations have been described as projecting to deep layers of IL [98]. This work indicates a potential means by which emotionally valenced information can interact with and guide decision making processes [98].

The amygdala is also highly interconnected with the ACC [99–101]. In primates, unsigned prediction error signals from BLA have been shown to be transmitted to ACC via synchronous theta phase coupling between the two regions [99]. This coupling has been shown to guide aversive learning, and is strongest when the association is initially forming. Studies have shown ACC involvement in attentional modulation following unexpected changes in reward value, as well as how disruption of this attentional signal impairs behavioral adaptation and results in impaired decision-making [84, 85, 87,88]. Interestingly, findings suggest that the BLA generates unsigned reward prediction errors that may be necessary for this attentional modulation to occur [101,103]. Importantly, the BLA’s representation of unsigned prediction errors develops over trials and thus appears contextual, taking into account reward contingencies from previous trials [70, 88]. Given the strong bidirectional connections between BLA and ACC—and taking into account the role of ACC in inhibitory control, attention, and the detection of conflict between two competing responses [104,105]--these studies provide a strong starting point for investigating the role of the amygdala in decision making and cognitive control.

Connectivity in Relation to Neuropsychiatric Disorders

The BLA’s dense innervations to the OFC make it an important candidate for investigating addiction and neuropsychiatric disorders. Using a Pavlovian-to-instrumental transfer paradigm, Lichtenberg et al. found that inactivation of BLA projections to the OFC disrupted the formation of cue-triggered reward expectancies [106]. As deficits in reward valuation are common symptoms of neuropsychiatric disorders, these findings emphasize the importance of studying the role of BLA and OFC connectivity during decision-making.

Additionally, studying reward-based decision-making using a rodent model of schizophrenia (neonatal ventral hippocampus lesion--NVHL) has revealed an aberrant overrepresentation of cues by the BLA [107] which results in inappropriate contextualization during task performance. Similarly dysregulated cue-selective firing was also seen in the BLA of cocaine-treated rats performing a reversal learning task [108]. Further, previous cocaine exposure led to altered delay-dependent activity in the BLA, and increased impulsivity during a delay-discounting task [109].

A study manipulating BLA-NAc connectivity during a cued risk/reward decision-making task found this circuitry implicated in optimal valuation of risk and reward [110]. Gambling disorder has also been associated with maladaptive OFC-amygdala interactions [111,112], resulting in aberrant valuation of rewards—specifically, the overvaluation of positive outcomes and the minimization of aversive ones, as well as alterations in emotional responses towards winning or losing [110,113–116]. Furthermore, in humans, the CeM of schizophrenic patients shows significantly reduced activity in response to positively valenced faces, as well as lower connectivity to the PFC, which may be related to the abnormal context processing that occurs in schizophrenia [117–121]. Collectively, these studies implicate the amygdala in the behavioral symptoms of neuropsychiatric disorders, as well as in the long-term decision-making impairments that follow drug use.

Conclusions

The amygdala is a highly sophisticated brain region involved in the integration of cue and outcome associations. The amygdala is comprised of 13 different nuclei, each of which have distinct and heterogeneous connections to diverse brain regions—thus giving rise to its functional complexity. From its strong historical links to fear and emotion, to its role in encoding the valence of decision outcomes, understanding how the amygdala contributes to decision making and outcome selection is important for our overall understanding of decision-making.

Countless models have attempted to explain human behavior generally, and the factors that influence decision making in terms of cognitive factors—such as how to handle simultaneously active competing responses--as well as psychological and economic factors associated with forecasting outcomes. While on some level these processes likely inform and shape the overall direction of decision making both at the biological and behavioral levels, they often fail to account for--or oversimplify--the emotional or affective factors at play. The amygdala represents a biological substrate well suited for the multidimensional encoding and integration of this valenced information. By examining how the amygdala interacts with frontal regions to influence decision-making, we will be working towards a more accurate understanding of the ways in which we behave.

Figure 4.

Model of amygdala function--information from various sensory modalities (grey box) reaches the amygdala (yellow) via thalamus and other pathways. Multiselective neurons in BLA then filter information representing different components of relevant decisional variables (i.e., task relevant/strategy information, attentional information, behavioral adaptation information, reward information, etc.). The relative weighting of this information can be influenced by activation of memories in the ventral hippocampus (red circle; blue arrow). Aspects of this information are then transmitted via reciprocal connections between the BLA and frontal brain areas (green circle), with the ultimate goal of selecting the appropriate action or modifying an ongoing action. Additionally, aspects of sensory information are passed to the CeA, which is thought to influence systems responsible for motivation and behavioral vigor. This model proposes accounts for how valenced information, via the BLA, is made available to other brain regions to aid in the adaptive selection of actions, and decision making.

Abbreviations

ACC

Anterior cingulate cortex

BLA

Basolateral amygdala

BA

Basal BLA

BM

Basomedial BLA

BNST

Bed nucleus of the stria terminalis

CeA

Central nucleus of the amygdala

CS

Conditioned stimulus

EPM

Elevated plus maze

IL

Infralimbic region

LA

Lateral BLA

CeL

Lateral CeA

CeM

Medial CeA

mPFC

Medial prefrontal cortex

OFC

Orbitofrontal cortex

PrL

Prelimbic region

US

Unconditioned stimulus