Fear conditioning and extinction: emotional states encoded by distinct signaling pathways

Abstract

Conditioning and extinction of fear have traditionally been viewed as two independent learning processes for encoding representations of contexts or cues (conditioned stimuli, CS), aversive events (unconditioned stimuli, US), and their relationship. Based on the analysis of protein kinase signaling patterns in neurons of the fear circuit, we propose that fear and extinction are best conceptualized as emotional states triggered by a single CS representation with two opposing values: aversive and non-aversive. These values are conferred by the presence or absence of the US and encoded by distinct sets of kinase signaling pathways and their downstream targets. Modulating specific protein kinases thus has the potential to modify emotional states, and hence, may emerge as a promising treatment for anxiety disorders.

Introduction

The increase in the prevalence of anxiety disorders has stimulated extensive interdisciplinary research toward the understanding of their etiology, pathophysiology and treatment. Findings from recent molecular and computational approaches are changing our view on the fundamental brain mechanisms that turn adaptive fear responses to real or expected threat into anxiety disorders. Understanding conditioning and extinction of fear is particularly relevant because both enhanced encoding of traumatic memories and resistance to fear extinction have been causally linked to these disorders.

Humans and animals develop robust fear of environmental contexts or cues paired with aversive events. Fear conditioning is typically described as excitatory associative learning about a positive relationship between two environmental events: a neutral conditioned stimulus (CS) and an aversive unconditioned stimulus (US)(Glossary). In rodent experiments, the context CS is typically a chamber, CS cues are auditory or light stimuli, and the US is a brief footshock. Fear extinction is viewed as learning about a negative relationship between a CS and a US; hence it has been termed “inhibitory conditioning” [1]. This learning process leaves the conditioning memory intact, as revealed by spontaneous recovery, renewal, and reinstatement of fear in response to specific reminders [2, 3] or unrelated stress or anxiety [4]. Thus, after fear conditioning and extinction, the brain seems to store opposing information about the same CS. How this information is processed to result in fear - or lack of fear - is a question to which various theories provide divergent answers.

It is well established that the context CS representation is formed within hippocampal-cortical networks, whereas cue CS and US are processed within the basolateral amygdala [5–8], whose output is also critical for the elicitation of conditioned fear (Figure 1A). During extinction, the infralimbic medial prefrontal cortex (PFC) is recruited along with the hippocampal-amygdala circuit (Figure 1B), and exerts a key role in fear reduction by inhibiting the amygdala [9, 10]. In the present article, we analyze the patterns of neuronal signal transduction within the fear circuit to test some of the key predictions of classical and novel theories of fear conditioning and extinction. We first introduce theoretical and computational models that dissect the memory components of conditioned fear and extinction. We then analyze protein kinase signaling in neurons of the fear circuit in response to individual or paired CS and US presentations. Finally, we discuss these patterns of kinase signaling in the framework of attractor state concepts of memory and emotion. We propose that unique protein kinase responses most likely reflect the encoding of different values of a single CS representation, triggering fear or extinction states upon retrieval. These states can thus be alternately and instantaneously induced in response to the CS, with the dominance of one over the other critically depending on which CS value is retrieved.

Figure 1.

Fear conditioning and extinction in rodents. (A) The context CS representation is formed within hippocampal-cortical networks, whereas cue CS and US representations are thought to be processed within the basolateral amygdala [5–8], whose output is also critical for the expression of conditioned fear. Before CS-US presentation, animals normally explore the context until the US elicits activity burst (jump) and vocalization as URs. Following training, the CS triggers a central fear state resulting in freezing as a CR. Other CRs, such as avoidance, increased heart rate, etc, are also observed. (B) The two most prominent views in the field are based on either bond or convergence models. In the first model, the CS and US are associated based on the formation of a bond between encoded CS representations and US representations or values. The second model posits that there is a convergence of CS representations and US representations or values. A third view in the field incorporates computational-based TDL models, such as RLSC. These models posit that during fear conditioning the CS gains an aversive value (represented with an orange glow). (C) During extinction, the infralimbic medial PFC is recruited in concert with the hippocampal-amygdala circuit and exerts a key role in fear reduction by inhibiting the amygdala [10, 126]. This results in loss of CR (dashed line) (D) Nonassociative learning posits that repeated exposure to the CS reduces the CS or US representation (dotted line) and thus fails to trigger the CR. Traditional theories of extinction propose three associative mechanisms: unlearning (degradation) of the original association, formation of an direct inhibition (red arrow) between the CS and US, or formation of an excitatory association between the CS and noUS that triggers circuit inhibition (red arrow)via the infralimbic PFC. The RLSC model posits that during extinction, the CS representation splits in two identical copies and the new one gains a non-aversive, value (depicted by a purple glow).

Models of fear conditioning and extinction

The main rules underlying conditioning and extinction have been described by the Rescorla-Wagner [11] and temporal difference learning (TDL) [12] models, which propose that learning is triggered by surprise. The novelty of unexpected but delivered stimuli (conditioning), and prediction errors generated by expected but omitted stimuli (extinction), are typical examples of surprise. How these stimuli and their relationships are subsequently encoded is an ongoing focus of studies of fear conditioning and extinction.

Fear Conditioning

Based on robust changes of behavior in response to a CS, it is agreed that during conditioning the CS becomes associated with something [1, 13], but the nature of that thing remains controversial. It is predominantly thought that an association occurs between CS and US representations, which are defined by their sensory content and isomorphism with environmental stimuli [2, 8, 14–16]. In some models, the term association describes a bond connecting CS and US representations in the brain [17]. Thus, a CS activates the CS representation, which, via excitatory associations, triggers the US representation. In turn, the US representation elicits a conditioned response (CR). Instead of bonds, the convergence model considers associations as the sites of overlap between CS and US representations within the basolateral amygdala [10] (Figure 1C).

These traditional theories have dominated the field despite a lack of direct evidence for the encoding of a US representation. The notion of a US representation is additionally complicated, particularly in one-trial fear conditioning, by the fact that: (i) conditioning commonly involves only a brief US, which may not be sufficient to form a lasting US representation; (ii) the CR is freezing behavior, even though direct activation of a US representation would be expected to trigger a CR that is similar to the unconditioned response (UR) (e.g. activity burst); and (iii) fear can be acquired in the absence of a US, either by second-order conditioning [18], instruction, or observation [19, 20]. Because an aspect of a US may be a condition for learning without being itself involved in that learning [1], these observations strongly suggest that the formation of a US representation is not critical for fear conditioning, leaving open the question of what is being associated with the CS.

This problem has been circumvented in other theories of fear conditioning, which posit that the affective value, rather than sensory features, of the US is encoded and associated with the CS [20–23]. The notion of the US value is also incorporated in influential computational models, such as TDL that explain the mechanisms of conditioning and extinction in appetitive [12, 24, 25] and aversive paradigms [25]. These models propose that the US value is not separately encoded but instead transfers to the CS [24], which then becomes a predictor of the US.

In broader terms, TDL models consider the combined memory of sensory, value, and other aspects of a stimulus or situation as a state. For clarity, we will use the terms representation and value as memories of the sensory and affective features of a stimulus, respectively. The term state will be mainly used to describe the emotional condition triggered by the retrieval of these memories. In this framework, fear conditioning entails the formation of a CS representation associated with an aversive value, which, upon retrieval, triggers a central fear state (S0). Based on its marked adverse properties, it is likely that the fear state further increases the aversive value, but not representation, of the CS [26] and thereby contributes to post-retrieval memory strengthening processes such as reconsolidation. Nevertheless, after a sufficient number of CS exposures without a US, the fear state declines.

Fear Extinction

Different theories have considered several processes as the mechanism underlying extinction (Figure 1D). Habituation is a nonassociative learning process in which repeated presentation of a CS reduces the activation of the CS representation and, consequently, the CR [27]. Other processes either involve unlearning, a degradation of the CS-US bond resulting in a reduced CR [11], or learning new associations. An inhibitory CS-US bond that prevents the CR has dominated the theories of fear extinction [2, 28]. However, there is also evidence for circuit inhibition whereby the formation of an excitatory CS-no unconditioned stimulus (noUS) association activates the infralimbic PFC, which in turn provides excitatory input to amygdala interneurons and inhibits the CR [10, 17] (Figure 1D). More recently, based on TDL rules, the reinforcement learning and state classification (RLSC) model was developed to account for phenomena such as rapid renewal and reinstatement of conditioned behavior after extinction. RLSC describes extinction as a value change and reinterpretation of the CS [25, 29]. That is, when the situation is less aversive than expected, the original CS representation is “split” in two and the twin becomes associated with a second value that induces an extinction state (S1) upon retrieval. Although a state of relief is a likely initial alternative to fear, ongoing trials without the US are more likely to induce a state of safety, similar to habituation. Ultimately, the CS is encoded with two opposing values, aversive and non-aversive. Which of the parallel fear (S0) and extinction (S1) states determines behavior depends on a number of factors, such as specific reminders of the original events or overall feelings of safety, threat, or arousal.

Until recently, the validity and predictions of individual models could be examined only theoretically. However, extensive molecular approaches with conditioning and extinction of fear have identified some of the major mechanisms that parallel or underlie these processes. The key predictions of individual models can thus be tested experimentally at a molecular level by analyzing the activation and distribution of signal transduction pathways within neurons of the fear circuit.

Signal Transduction and Fear Regulation

Exposure to unexpected environmental stimuli triggers strong activation of different parts of the brain, leading to the formation of memories of those events. While sensory and affective properties of the stimuli determine which brain regions are activated [30], their processing is coordinated by intricate networks of signaling complexes [31, 32]. Ultimately, this cascade of events alters the cellular content and distribution of synaptic molecules along with neuronal excitability and firing patterns, which are viewed as a basis of memory [33–35]. Despite the complexity of regional and cellular regulation of signal transduction, principal protein kinases mediating learning processes have been identified. These kinases regulate molecular remodeling and gene expression [36, 37] and are not specialized for learning processes, as revealed by highly conserved and ubiquitous distribution across cell types. However, in the brain, protein kinases specifically modify key synaptic properties of neurons involved in memory formation [35]. Thus, kinase signaling can influence the connectivity, plasticity, or synchronized firing of neurons of the fear circuit at critical connections between the hippocampus, amygdala and PFC. An extensive body of pharmacological and genetic approaches has established a causal link between protein kinase signaling and memory formation (reviewed in [38]). However, the role of protein kinases and their downstream targets in processing specific memory components - such as sensory features and affective values of the CS and US - is not known.

Signal Transduction and Fear Conditioning

The best demonstrations of molecular processing of stimuli come from studies using one-trial contextual fear conditioning, where exposure to the CS, US, or both, can be carefully controlled. The use of an immediately delivered shock US, which does not lead to the formation of associative memory [39], provides a critical control for assessing responses to the US. Although most studies using immediate shock also include a subsequent contextual exposure, the latter component can be controlled by exposure to context alone. Molecular studies with cue-dependent fear conditioning typically compare CS/US paired to unpaired groups and rarely include naïve and context controls. This complicates the interpretation of molecular responses to an individual CS or US. We therefore focus on the principal protein kinases, phosphatases and transcription factors, including immediate early genes (IEGs), activated during contextual fear conditioning (Figure 2).

Figure 2.

Hippocampal and amygdala molecular mechanisms responding to CS, US, and CS + US presentations after fear conditioning in rats. Changes of activity (*), phosphorylation (**) or level (***) of individual signal transduction molecules and IEGs in response to the context CS, immediate shock US or paired presentation of CS and US when the US follows the CS exposure. Horizontal lines indicate lack of response whereas vertical arrows indicate increase. Each molecule is color-coded and the overall signaling pattern is shown as a distinct combination of colored squares. Lack of activity is marked with a gray square. The signaling response is distinct when the CS and US are presented together. Protein kinases: Fyn, Cdk5, ERK; Transcription factors: CREB and IEGs (c-Fos, Egr-1, Arc).

The traditional models of fear conditioning predict that CS or US alone would trigger molecular responses in specific brain areas, and that kinase activities to a paired CS/US presentation would either be equal to (bond) or less than (convergence) the sum of kinase activities to CS and US alone. According to TDL models, however, the US is expected to quantitatively or qualitatively modify signal transduction triggered by the CS without affecting signal transduction on its own. Thus, CS/US pairing would either enhance the molecular responses to the CS or trigger a completely different signaling pattern than CS.

During fear conditioning, the presentation of CS and US triggers strong activation of protein kinase/phosphatase pathways and transcription factors in hippocampal and amygdala neurons (Figure 3). The activities, phosphorylation, or levels of the kinases Fyn [40], cyclin-dependent kinase 5 (CDK5) [41], and extracellular signal-regulated kinase (ERK) [42] are significantly higher when the CS and US are presented in a paired fashion than when the CS is presented alone – a finding that has been consistently observed across studies. The same applies to their downstream targets: transcriptional regulator cAMP- response element binding protein (CREB) [43], the IEGs, activity-regulated cytoskeleton-associated protein (Arc) [44, 45], c-Fos [46, 47], and early growth response protein 1 (Egr-1, formerly known as Zif268) [45, 48]; as well as the whole family of CRE-regulated genes [49], Finally, key regulators of the calcium/calmodulin kinase II (CaMKII) pathway are not activated by context alone but are strongly up-regulated by context and shock [50, 51]. These CS/US-specific signaling patterns provide convincing evidence for differential processing of the CS in the presence or absence of US. Some of these molecular responses are directly regulated by projections from the basolateral amygdala to the hippocampus [46].

Figure 3.

Signaling molecules activated by fear conditioning versus extinction in the (A) hippocampus and (B) amgydala. Changes of activity (*), phosphorylation (**) or level (***) of individual signaling molecules in response to conditioning or extinction of fear. Horizontal lines indicate lack of response, vertical upward arrows indicate increase, and vertical downward arrows indicate decrease. Each molecule is color-coded and the overall signaling pattern is shown as a distinct combination of colored squares. Lack of activity is marked with a gray square. The signaling response patterns during fear conditioning and extinction show marked differences. Protein kinases: Fyn, Cdk5, ERK, PKA, PKC, CamKII, PI3K, Akt; Protein phosphatases: SHP1/2, calcineurin; Small GTPases Ras-related protein 2 (Rap2) and RIN1; Transcription factors: CREB and IEGs (c-Fos, c-Jun, JunB, JunD, Egr-1, Arc and CARP).

In the rodent hippocampus, the US was found to rarely trigger molecular responses above baseline control levels, whereas the CS alone triggered marked c-Fos and Arc responses [44, 45]. These findings do not provide support for the formation of US representations and, consequently, their bond or convergence with CS representations during one-trial fear conditioning. It cannot be ruled out, however, that sufficient US exposure would be encoded as a representation, particularly in conditioning paradigms where the US is repeatedly presented and therefore generate reliable predictions about US occurrence [8, 52]. In the basolateral amygdala, similar to the hippocampus, most molecules, such as phospho-CREB (pCREB), Egr-1, and c-Fos, are either triggered by the CS alone or by a combined of CS/US presentation, but not US alone [[43, 45, 47, 48]]. Findings with Arc were inconclusive because this protein showed inconsistent increases across shock control groups (e.g., lacking in immediate shock but present in latent inhibition groups) [44, 45].

Taken together, these findings show that the US significantly alters the kinase response to the co-occurring CS, and suggest that the effects of US alone on protein kinase activity are generally sub-threshold. We therefore propose that the US is a key modulator of the CS value, as predicted by TDL models. In line with this possibility, there is increasing evidence for separate neuronal populations within the hippocampus, amygdala, and cortex, which are specialized for encoding the values, rather than sensory features, of stimuli [53, 54].

Signal Transduction and Fear Extinction

In general, learning processes recruit highly conserved sets of signaling molecules [33]. Analysis of protein kinase/phosphatase and transcription factor activation patterns after fear extinction, when compared to conditioning and other types of learning, can therefore establish how putative nonassociative or associative mechanisms - namely, habituation, direct inhibition, circuit inhibition, or new values of the CS- contribute to fear extinction.

The habituation (nonassociative learning) model predicts that down-regulation of molecular responses in neurons involved in conditioning is required and sufficient for extinction. Behavioral studies with a mouse lacking the cannabinoid receptor 1 (CB1) gene suggest that the cannabinoid system may mediate extinction via a habituation process [55, 56]. However, CB1-activated kinases, such as ERK and Akt (protein kinase B) [57, 58] are up- rather than down-regulated during extinction, arguing against habituation. Alternatively, CB1 may independently regulate these processes via separate mechanisms.

According to the unlearning model, extinction should trigger degradation of key molecules that are induced by conditioning and required for ongoing fear. Presentation of a conditioned context CS without US triggers ubiquitin-mediated proteasomal degradation of selected proteins in the hippocampus (e.g., Shank, guanylate kinase–associated protein), a process required for fear extinction [59]. It was proposed that this reflects some degree of unlearning [59], consistent with observations that fear after reinstatement or renewal is usually less intense than after conditioning [2]. Alternatively, extinction may require a different subset of functional proteins so that some are degraded while others are increased.

Extinction based on direct inhibition predicts that preferential activation of inhibitory transmission is required to block the CS-US association. In support of inhibitory mechanisms, bidirectional changes in the amygdala were reported for gephyrin, a scaffolding protein within interneurons that regulates inhibitory neurotransmission [60]. This protein decreases and increases after conditioning and extinction of cue-dependent fear, respectively, suggesting corresponding changes in inhibitory transmission. However, these changes may be partly confounded by context exposure [61], so their causal role in conditioning and extinction remains to be established.

Extinction based on circuit inhibition postulates that an excitatory bond between new CS and noUS representations needs to be formed within the fear circuit to trigger extinction-specific inhibition of the amygdala [10]. At a molecular level, this process would share the conserved mechanisms for encoding new representations and their associations.

Several lines of evidence show that extinction does not trigger signal transduction typical of encoding new representations or associations (Figure 3), and are thus inconsistent with these possibilities:

1. In the mouse hippocampus, the IEGs c-Fos, JunB and Ca(2+)/calmodulin dependent protein kinase (CaMK)-related peptide (CARP, also known as Ania-4) exhibit marked increase during conditioning, but not extinction, of contextual fear. Both genes are up-regulated when animals are exposed to a novel context, and their expression decreases during subsequent exposures to the same context [47, 62, 63]. This suggests that once the context representation is established, the activity of c-Fos and JunB returns to a basal level and is not triggered by extinction. In contrast, fear extinction activates JunD, a transcription factor that is not affected by contextual processing during fear conditioning [64]. Thus, extinction does not seem to involve the formation of a new CS representation as a part of an excitatory CS-noUS association, but engages a different learning process.

2. Several signaling pathways exhibit opposite roles in fear conditioning and extinction. In the hippocampus, activation of protein kinase C (PKC), CDK5, cAMP-dependent protein kinase (PKA) and Fyn are required for contextual fear conditioning [40, 41, 65], whereas inhibition of any of these kinases enhances extinction [41, 66–68]. Different regulatory factors contribute to these opposite effects. First, Src homology 2-containing protein-tyrosine phosphatases 1 and 2 (SHP1/2) mediate extinction by dephosphorylating Fyn [69]. Second, the adenylyl cyclase/cyclic AMP (cAMP) pathway exerts opposing effects on contextual fear conditioning and extinction via different patterns of downstream effectors: conditioning is mediated by activation of both EPAC (exchange protein activated by cAMP) [70, 71] and PKA [72], whereas activation of PKA alone inhibits extinction [73–75]. Finally, PKC subtypes have differential effects on conditioning [76, 77], so their roles in extinction may also be isoform-specific. Similar to the hippocampus, in the basolateral amygdala, the levels of pCREB and Akt decrease after extinction of cue-dependent fear [78] but see [79, 80]. This parallels an increase in the activity of calcineurin [81, 82], a protein phosphatase known to inactivate PKA [83] and PKC [84]. Opposite regulation of conditioning and extinction is also mediated by the small GTPase protein Ras and Rab interactor 1 (RIN1) within the amygdala [85]. These findings demonstrate that the key signaling pathways expected to mediate excitatory associative learning are not only unnecessary, but actually prevent extinction.

3. A subset of hippocampal and amygdalar kinases- CaMKII, phosphoinositol triphosphate kinase (PI3K), and ERK- has been demonstrated to be required for both fear conditioning [86–89] and extinction [16, 57, 90, 91]. CaMKII autophosphorylation is also required for both processes, but extinction is more sensitive to its disruption [90]. In the case of ERK, the upstream regulation, cellular localization, and downstream targets significantly differ between fear conditioning and extinction [64, 92]. Thus, even when similar kinases are recruited, they play different roles in fear conditioning and extinction, leaving the processing of new excitatory associations in fear extinction questionable.

Given the lack of conclusive evidence for the formation of a new CS representation, inhibitory associations, or excitatory associations, protein kinase activity could mediate the encoding of a new, non-aversive value of the same CS representation (Figure 4). This possibility and several other features of extinction-induced signaling best conform to the RLSC model proposing value change and CS reinterpretation as mechanisms of extinction. The key features are:

Figure 4.

Encoding of conditioning and extinction of fear. (A) Phases of fear regulation. (a) During fear conditioning, a CS representation is encoded with an aversive value (orange glow) after pairing with a shock US (red flash), and triggers a transition from a baseline state to (b) a state of fear (S0) upon retrieval. After extinction, a non-aversive value (purple glow) is additionally encoded, so that the same CS representation can either trigger a fear (S0) or an extinction (S1) state. (c) Even when extinction has been established, (d) transitions back to S0 can rapidly occur after reminders, stress, or spontaneously. Freezing behavior (on a scale from 0–100% of the total observation time) shows increases, corresponding to S0 (orange; conditioning, spontaneous recovery, reinstatement) or decreases, corresponding to S1 (purple; extinction). E₁-E_n denotes the number of the extinction test. (B) Memory encoding during fear conditioning and extinction is triggered by neurotransmitters, and subsequently processed by protein kinase and phosphatase signaling pathways. These pathways, in turn, modulate transcription factor activity, which results in distinct patterns of cytoskeletal rearrangement, gene expression and chromatin remodeling during each of these two different states. B, baseline state.

1. Neurotransmitter-initiated detection of prediction errors is thought to trigger the splitting of the CS representation into two states with different values [25]. Therefore, some of the downstream signaling pathways should reflect the processing of prediction errors of expected but omitted US. Such a role has been demonstrated for ERK [87], which may thus be directly involved in encoding the non-aversive value of the CS and extinction state.

2. Contrary to the Rescorla-Wagner rule [11], which posits that learning is maximal when prediction error is maximal (i.e., the first extinction trial), RLSC postulates that repeated, tonic prediction error triggers extinction. Notably, in the hippocampus, ERK and Cdk5 pathways triggered by, and required for extinction are altered after repeated trials without a US [58, 93].

3. Partial reinforcement during contextual fear conditioning results in resistance to extinction [94]. As both presence and absence of reinforcement are encoded within the conditioning state, subsequent lack of reinforcement during extinction trials is not novel or unexpected, and prediction error does not occur [25]. Consistent with this idea, extinction trials following partial reinforcement do not trigger ERK activity [94]. As a result, extinction fails and conditioned responding persists, indicating that the fear state is maintained in response to the CS.

It should be mentioned, however, that some of the discussed behavioral and molecular findings do not support all assumptions of RLSC and other extinction theories. This primarily concerns the view that contextual stimuli only play a modulatory role in the conditioning and extinction of fear and are not the primary CS [17, 25]. Contrary to these views, the data summarized here demonstrate that contextual stimuli potently trigger protein kinase signaling, fear conditioning, and fear extinction. Furthermore, the partial reinforcement effect (resistance to extinction after intermittent US delivery) predicted by RLSC is readily observed after contextual [94] but not cue-dependent [95] fear conditioning paradigms. Analyses of contextual fear conditioning therefore warrant a much greater consideration in theoretical and computational studies of fear extinction.

In summary, there is molecular evidence for a limited role of habituation and unlearning in extinction. However, kinase signaling does not exhibit patterns typical of excitatory or inhibitory learning. Instead, extinction predominantly involves unique mechanisms and subsets of excitatory neurons in the hippocampus [92] and amygdala [96]. On this basis, we propose that these kinase responses most likely reflect the encoding of the new, non-aversive CS value that causes reduction of fear upon CS retrieval. Fear and extinction states can thus be alternately induced, with the dominance of one over the other critically depending on the circumstances surrounding memory retrieval.

Signal transduction in other processes of fear regulation

The preferential retrieval of the conditioning or extinction memory is considered to be a critical determinant of fear states [2, 3]. Retrieval is an extremely rapid process mediated by fast-acting ion channels [97]. Interestingly, a role of protein kinases in retrieval has also been provided [57] despite the slow kinase response to external stimuli. It is therefore likely that baseline, rather than stimulus-induced protein kinase activity contributes to memory retrieval. Kinases determine the steady state phosphorylation, ligand-sensitivity, and responsiveness of ion channels (reviewed in [98]), may thus regulate memory retrieval, and switch between different emotional states.

Another process contributing to fear regulation is memory reconsolidation. Here, instead of extinction, the CS retrieval triggers reconsolidation, a process for maintaining/updating the original memory. The mechanisms of reconsolidation partially overlap with mechanisms of conditioning [99], and are generally distinct from those of extinction [100–102]. It is not yet clear what information is encoded by reconsolidation mechanisms, given that the CS is already learned, and absence of US does not seem to be processed (otherwise one would expect extinction to occur). Possibly, the fear state (S0) induced by retrieval further strengthens the aversive value of the CS. This idea is consistent with the RLSC model and with observations of patients with anxiety disorders who not only dread specific contexts and stimuli, but also their own fear states [103].

Fear and Extinction as Protein Kinase-Regulated Attractor States

Memories and emotions have increasingly been conceptualized as attractor (stable) states in a dynamical central nervous system, providing a framework for integrating encoded information and coordinating its retrieval. Stimulus representations [104], values [25], and emotional states [105] are all viewed as attractor states, as inferred from analyses of behavior [106] and neuronal firing patterns, such as oscillations [107–109] or reverberating activity [105]. Notably, these neuronal properties are determined by ion channel receptors [110], whose levels, trafficking and function critically depend on protein kinase signaling and gene expression [111]. Thus, attractor states can be further characterized using molecular and genetic tools.

Recently, using cellular rather than systems models, attractor states have been defined as patterns of gene expression that lead to different cell fates [112]. In general, differentiating cells can be drawn to a limited number of stable states [113]. Protein kinases may play several key roles in these processes by functioning as multistable switches of gene expression patterns that control cellular states [112, 114]. Based on striking analogies of these processes to neuronal responses during learning, it is very likely that the same molecules play similar roles in neurons. For example, in models of cellular differentiation, fluctuations in neuronal gene expression and protein levels drive transitions between coexisting states and change the likelihood of otherwise homogenous cell populations to respond to various stimuli and regulatory factors [112, 115]. Consistent with this finding, increasing the level of CREB in a subset of amygdala neurons significantly increases the likelihood that this population will become integrated in a memory circuit [116]. In models of fear conditioning and extinction, such mechanisms could cause different excitatory neurons of the amygdala and hippocampus to up-regulate distinct signaling molecules [92], encode new CS representations and values, or add new values to an existing CS representation (Figure 5). This suggests the intriguing prospect that memories, including representations and values, are essentially differentiation stages of mature neurons manifested as distinct attractor states.

Figure 5.

Different values of a CS representation illustrated as attractor states. (a) Typically, the sum of all possible cellular states is described as a landscape. The valleys denote the limited number of stable states to which cells are drawn. In our model, orange and purple dashed arrows indicate protein kinase signaling pathways, which induce transitions to from baseline (B) to fear (S0), and fear to extinction (S1) states, respectively. (b) During fear conditioning, a CS representation is encoded with an aversive value (orange), (c) After extinction, a non-aversive value (purple) is added and the CS representation is encoded simultaneously with two different values. A fear (S0) or an extinction (S1) state will be induced depending on the retrieved CS value.

Conclusion and Perspective

It is challenging to convincingly demonstrate the existence of distinct fear and extinction states using molecular tools, but several predictions of such a model can be experimentally addressed (Box 1). If specific protein kinases are directly involved, modulation of their activity could serve as a powerful tool to trigger state changes and corresponding behavior. Consistent with this possibility, within the hippocampal-PFC circuit, brain-derived neurotrophic factor (BDNF), a potent activator of the MEK/ERK and PI3K pathways [117], completely substitutes for extinction trials and effectively reduces conditioned fear [118].

Box 1. Outstanding questions.

• Cellular and molecular mechanisms for encoding representations and values

  Ample evidence from animal electrophysiological recordings and human imaging studies supports the view that segregated neuronal populations of hippocampal-amygdalar-cortical neurons encode representations and values. What are the molecular mechanisms for encoding sensory versus affective features of stimuli? Are representations and values equally sensitive to modulation and disruption?

• US encoding after single or multiple presentations

  Fear and anxiety can be induced by acute or chronic stressors acting as US. Do these different types of US exposure induce differential processing within the fear circuit? How does this affect the encoding of the CS, the relationship between CS and US, and subsequent fear extinction?

• Relationship between values and predictions

  Human studies using a variable schedule of US presentation reveal that the CS may either gain an aversive value or serve as a predictor of an upcoming US [127]. What is the relationship between the mechanisms of predictions and values? Are predictions components of the CS memory or interpretations of the CS memory after retrieval?

• Mechanisms of resistance to fear extinction

  Controlled regulation of protein kinase signaling and IEG expression may prove particularly important in elucidating the types of learning mechanisms underlying resistance to extinction, such as reconsolidation [128] or partial reinforcement [94]. Do mechanisms triggered by reconsolidation strengthen the processing of aversive value within the neuronal population recruited during fear conditioning, or add a new neuronal population to the fear circuit, or both? What are the distinguishing mechanisms of fear triggered by partial versus continuous reinforcement?

• Mechanisms of memory retrieval

  What are the key mechanisms of memory retrieval? Does retrieval of CS with an aversive or non-aversive value require different molecular and neuroanatomical mechanisms? How do these mechanisms contribute to renewal, reinstatement, and spontaneous recovery of fear?

• Genetic and epigenetic mechanisms defining attractor states

  Which epigenetic and gene expression profiles triggered by specific kinase signaling patterns are critical for establishing and maintaining fear versus extinction states? How do epigenetic and genetic factors affect the connectivity, activity, and synchronization of neurons within the fear circuit?

• Kinases as targets for treatment of anxiety disorders

  Post-receptor signaling mediated by phosphorylation cascades of protein kinases provides a high degree of specificity in different cellular and tissue functions and is thus an important therapeutic target for disorders ranging from heart failure [129] to cancer [130]. Can these compounds effectively alleviate the symptoms of anxiety disorders? Is activation of kinases that normally contribute to extinction, or activation of a different set of protein kinases, required to overcome persistent fear?

While protein kinases and IEGs are expected to trigger encoding and transitions between fear and extinction states [66, 85], the key attributes of these states are most likely defined by their downstream targets, such as cytoskeletal proteins [119] and ion channels [98]. Furthermore, the stability of such states is likely maintained by epigenetic regulation [120–123]. Elucidation of the transcriptional mechanisms is now possible with large-scale analyses of gene expression in individual neuronal populations, and will advance our understanding not only of pathological fear responses, but also other cognitive and affective disorders. Namely, in addition to anxiety, animal and human findings implicate neuronal protein kinase signaling pathways in depression [124, 125]. By enabling a possible exit from these debilitating mental states, targeting specific protein kinase signaling mechanisms may emerge as a powerful approach for the treatment of psychiatric disorders.

Glossary

Affective

features of the stimulus generating emotion that are perceived and processed by the cortico-limbic system

Attractors

a limited number of stable neuronal states. Representations and values are attractors caused by learning and memory processes. Emotional states are attractors normally caused by memory retrieval, recognition and classification. In anxiety disorders, however, such states may become independent of retrieval

Conditioned stimulus (CS)

an originally neutral stimulus that after pairing with a US comes to trigger a behavioral response

Conditioned response (CR)

a behavioral response elicited by the CS after conditioning

Encoding

A process leading to memory formation and storage

Epigenetic mechanisms

molecular mechanisms that cause changes in the chromosome without alterations in the DNA sequence, thereby maintaining stable cellular phenotypes

Fear

an emotional state triggered by real, perceived, or remembered threat

Fear conditioning

a behavioral paradigm in which organisms learn to fear a CS associated with aversive events

Fear extinction

a behavioral paradigm in which organisms learn not to fear a CS that is no longer paired with a US

Fear and extinction states

emotional states caused by retrieval of CS representations with aversive or non-aversive values. According to some views, state-action pairs are co-encoded and inseparable. Thus, the terms fear and extinction can be used interchangeably to describe emotional states and behavioral responses

Immediate early genes (IEG)

genes responding with a rapid, inducible expression to various stimuli

Phenotype

The set of observable characteristics of a cell, circuit or organism resulting from the interaction of its genotype with the environment

Representations

perceptions or memories of the sensory attributes of a CS or US

Rescorla-Wagner learning rule

learning from the discrepancy between an expected and actual US

Sensory

physical features of the stimulus perceived and processed by the somatosensory brain areas

State

the information available to an organism at a given time. States may include internally available information as well as signals provided by the environment [12]. Here, we primarily discuss internal attractor states, i.e. memory and emotion

Temporal difference learning (TDL)

an extension of the Rescorla-Wagner rule in the temporal domain: a prediction method using sensory cues across successive moments in time to predict future rewards or punishments

Unconditioned stimulus (US)

a stimulus that unconditionally, naturally, and automatically triggers a behavioral response (UR)

Values

perceptions or memories of the affective attributes of the CS or US