A surprised amygdala looks to the cortex for meaning

Abstract

Learning models propose a role for both signed and unsigned prediction errors in updating associations between cues and aversive outcomes. Here, Klavir et al. show how these errors arise from the interplay between the amygdala and anterior cingulate cortex.

Learning is a life-long endeavor, demanding that we constantly track and update the outcomes we associate with specific stimuli, so that we quickly learn not to respond to stimuli we encounter today with reactions that were appropriate yesterday. Indeed, a lack of cognitive flexibility can be a setback in daily life, and if serious enough, a sign of neurological distress. A critical driving force behind learning is prediction error, the discrepancy between expected and actual outcome. Influential models from learning theory propose different means for updating stimulus-outcome associations to correct for prediction error, bringing predictions in line with current conditions. For example, in the Rescorla-Wagner model, associations are updated when the unconditioned stimulus (US) violates previously established expectations, increasing in associational strength with the cue if the US is bigger than expected, and decreasing in strength if the US is smaller than expected (Rescorla and Wagner, 1972). Thus, in this model the prediction error is signed, driving the strength of cue-outcome associations upward (positive) if expectations are exceeded, and downward (or negative) if expectations are unmet. Conversely, in the Pearce-Hall and Mackintosh models, the prediction error is unsigned. If a cue produces an outcome that is different (whether more or less) than expected, more attention is given to the conditioned stimulus (CS), thereby strengthening the cue-outcome association regardless of the sign of the prediction error (Pearce and Hall, 1980; Mackintosh, 1975). Appetitive learning studies, where reward is unexpectedly delivered or omitted, support the co-existence of both models, and demonstrate that signed and unsigned errors are represented across a distributed network of brain regions (Shultz et al., 1997; Roesch et al., 2010). However, the neural substrates of prediction error during aversive learning, where a “positive” error – the surprising presence of a stimulus – results in the delivery of an aversive experience, have not been as well explored.

To investigate the temporal development of neural coding associated with aversive learning, Klavir et al. (in this issue of Neuron), simultaneously recorded from two areas known to support this task: the amygdala and the anterior cingulate cortex. Critically, the authors identify differential sequences of information processing in this circuit depending on whether neurons respond to signed or unsigned prediction errors. The group recorded units in the dorsal anterior cingulate (dACC) and in distributed locations of the basolateral nuclei of the amygdala (BLA) while monkeys performed a trace conditioning task where an auditory or a visual stimulus predicted either an aversive air puff to the eye (CS+) or no air puff (CS−). Once the association was learned, as evidenced by selective preparatory blinking to the CS+, the contingencies reversed. The authors show that successfully learning the reversal increases the functional connectivity between the dACC and BLA and imposes an error sign-dependent temporal order on information processing (Figure). Amygdala cells that increase their firing rate in response to both positive and negative prediction errors (as in the unsigned errors of the Pearce-Hall model) precede activity in the dACC. Conversely, amygdala cells that change their firing rate in opposite directions to the two error types (and therefore encode signed errors, as proposed by the Rescorla-Wagner model), fire after those in the dACC. These findings suggest that unsigned errors first propagate from the amygdala to the dACC, where they are given a sign and returned to the amygdala. Thus, the dense reciprocal connectivity of this circuit allows for processing of both signed and unsigned prediction errors by differentially controlling the directionality of information transfer. Given data from depressed patients indicating significantly compromised prediction error updating (Gradin et al., 2011), characterizing the strengths and vulnerabilities of this circuit would be of potential clinical relevance.

Figure.

Amygdala neurons representing unsigned prediction errors (top) increase their firing rates to both positive (CS− to CS+) and negative (CS+ to CS−) prediction errors during reversal learning. Activity in these “unsigned error” neurons preceeds that in dACC neurons. Firing rates in amygdala neurons representing signed prediction errors (bottom) change rates in opposite directions for the two types of error. Activity in these “signed error” neurons follows that in dACC neurons. These findings suggest that unsigned errors arise in the amygdala and are sent to the dACC, where the error signal is given a sign before being returned to the amygdala.

Early amygdala firing to surprising presence or absence of the aversive US, as shown in the Klavir et al. study, is similar to amygdala activity in processing unsigned prediction errors in appetitive tasks that use reward as a US (Roesch et al., 2010). Collectively, these data suggest that cells in the amygdala are highly tuned to any changes in the environment, constituting an ideal neural substrate for sign-free, attention-related error processing as described by the Pearce-Hall model. Given that the amygdala is an evolutionarily conserved structure with a central role in processing threat and orchestrating defensive reactions, it is appropriate that the amygdala is an effective detector of unexpected changes related to aversive stimuli. In support of this idea, the amygdala has been identified as a critical site for processing unpredictable stimuli that are not associated with a US of any value (Herry et al., 2007). Moreover, amygdala activity induced by surprising stimuli in humans was followed by enhanced attention toward upcoming threatening stimuli (Herry et al., 2007). An amygdala-mediated increase in attention to and salience of the CS, as suggested by the Pearce-Hall model, is consistent with the known role of this structure in emotional arousal and enhanced memory consolidation (McGaugh, 2013). Indeed, substantial evidence indicates that emotional arousal is accompanied by the release of glucocorticoids and activation of β-adrenergic receptors in the amygdala, processes that are essential for long term memory consolidation in humans and in rodents (McGaugh, 2013). Simply put, emotional events are remembered better. Given that memory consolidation is a time dependent process that occurs after training, it would be interesting to see if, after the surprise of the reversal in aversive contingencies of a given CS, the learning rate of the new stimulus-outcome associations is dependent on memory consolidation in the amygdala. If so, it would suggest that unsigned error-related increases in amygdala firing can lead to longer lasting changes in intracellular signaling cascades that ultimately alter the state of amygdala cells during learning.

Seminal work has shown that signed errors in reward processing, as predicted by the Rescorla-Wagner model, are mediated by dopaminergic neurons (Shultz et al, 1997). Furthermore, inhibitory neurons in the ventral tegmental area are active throughout the CS-US delay and in contrast to the dopaminergic cells, fire during the delivery of aversive as well as the rewarding stimuli (Cohen et al, 2012). Thus, interaction between the excitatory drive onto dopaminergic cells and local inhibition along their dendrites and soma likely participates in the updating of CS-US associational strength during prediction error learning and during dopaminergic silencing during aversive outcomes. Notably, it has also been demonstrated that normal appetitive error signaling in the striatum depends on input from the orbitofrontal cortex (OFC, Takahashi et al., 2011). In this work, the orbitofrontal signal was proposed to send contextual information to striatal cells, sculpting the striatal response as a function of the external context. Similarly, in the present study, Klavir et al., found that activity of signed dACC cells precedes activity of signed amygdala cells, suggesting that once the unsigned element of surprise is processed in the amygdala, dACC cells become active and could then in turn exert a top-down influence on the amygdala in signed prediction errors. An important goal for the future is to understand how amygdala and dACC inputs use the available local networks to shape the sign of an error by regulating the excitatory-inhibitory balance.

These intriguing similarities between frontal-striatal and frontal-amygdala circuits in updating prediction errors of rewarding and aversive conditioned stimuli, respectively, suggest that these networks rely on similar processing strategies. What may unite the two circuits is amygdala activity. The very nature of early amygdala activation to unsigned errors as shown by Klavir et al. and others (Herry et al., 2007, Roesch et al., 2010,), should influence those systems that are learning about both rewarding and aversive stimuli. In line with this idea, presentation of fearful faces to human subjects activated the amygdala and sped up the rate of cue-reward association learning and amygdala-striatal interactions (Watanabe et al., 2013). Thus, arousal via amygdala activation has the hallmark attributes of the Pearce-Hall model and can influence prediction error learning rates in cue-reward (Watanabe et al., 2013) and cue-aversive (Herry et al., 2007) associations, where signature elements of the Rescora-Wagner model are enacted in parallel systems.

One important task remaining is to integrate these findings into the growing body of data linking amygdala activity to value, particularly through interactions with the OFC. Recordings from similar reversal tasks demonstrate that amygdala neurons represent the value of learned stimuli, including the sign of that value (Paton et al, 2006). Simultaneous amygdala and OFC recordings have demonstrated a complex dance between the two structures during the reversal learning process, one where amygdala neurons more quickly acquire negative value representations, while OFC neurons more quickly acquire positive value representations (Morrison et al., 2011). It is not immediately clear how these signed value representations compare to the signed and unsigned prediction error representations described by Klavir et al. Also remaining to be puzzled out is how the dance between the amygdala and OFC compares to that between the amygdala and the dACC, and whether these circuits are activated serially or in parallel. Presumably, prediction errors are used to update value representations, and these updates require signed errors. It is therefore intriguing to imagine that inputs from the dACC and OFC converge within the amgydala, where signed errors from the dACC update stimulus-value encoding at OFC-amygdala synapses. Such computations occurring at multiple stages of top-down processing would create a pliable network capable of flexibly assigning meaning to the ever-surprising world around us.