Design of a Neurally Plausible Model of Fear Learning

Abstract

A neurally oriented conceptual and computational model of fear conditioning manifested by freezing behavior (FRAT), which accounts for many aspects of delay and context conditioning, has been constructed. Conditioning and extinction are the result of neuromodulation-controlled LTP at synapses of thalamic, cortical, and hippocampal afferents on principal cells and inhibitory interneurons of lateral and basal amygdala. The phenomena accounted for by the model (and simulated by the computational version) include conditioning, secondary reinforcement, blocking, the immediate shock deficit, extinction, renewal, and a range of empirically valid effects of pre- and post-training ablation or inactivation of hippocampus or amygdala nuclei.

Introduction

Cues that predict the imminent onset of pain or danger as well as the situational contexts in which such events occur come to evoke fear. In laboratory animals this has a variety of manifestations including freezing, cardiac responses, and analgesia (Bolles and Fanselow, 1980). Such fear conditioning provides one of neuroscience's most promising and active arenas for analyzing neural mechanisms of learning, generally. There is now considerable evidence that the acquisition of conditional fear can be substantially attributed to forms of long-term potentiation at synapses of cortical, thalamic, and hippocampal afferents on cells of the baso-lateral region of the amygdala (BLA; Blair et al., 2001). Synaptic plasticity within the central nucleus, which contains the amygdala's fear-producing output neurons and which receives input derived from BLA, also seems to contribute (Pare et al., 2004; Wilensky et al., 2006; Ciocchi et al., 2010). And although there is not yet consensus on the locus of extinction, a variety of lines of work, both behavioral and neurophysiological, have established that under most circumstances extinction is at least in part due to learned inhibition of activity in fear-producing pathways and not primarily to erasure of the changes that caused the original conditioning (Bouton, 1993, 2004). While these are major advances, there are many conspicuous properties of fear learning for which we neither know the mechanisms nor have hypotheses of a kind that invite physiological testing.

The well-known phenomenon of “renewal” provides an excellent example (Bouton, 1993, 2004). It is generally agreed that once fear is conditioned to a cue, fear will be expressed when that cue is presented in most any context. However, if fear is then extinguished, either in the context of original training or elsewhere, the extinction will be quite specific to the context in which it occurred, and the CS will still evoke fear in other contexts; i.e., fear will be “renewed.” Renewal provides some of the best evidence of the inhibitory nature of extinction, but hypotheses as to its mechanisms are not usually neural in character. Discussions of mechanism are generally in terms of “occasion setting,” “disambiguation,” “storage and retrieval,” “gating” of inhibition, etc. These are essentially psychological constructs and do not provide much guidance for implementation at a neural level.

The goal of the present work was therefore to construct a conceptual and computational neural model of fear conditioning that can account for what we consider some of the most salient aspects of fear conditioning and that can be justified by known neurophysiology. This model is referred to as “FRAT” (for “Fraidy Rat” after an earlier, simpler version of the model used for teaching). While for modeling purposes we focused on freezing, the model is expected to generalize to other commonly used measures such as fear-potentiated startle, autonomic arousal, conditioned suppression, and inhibitory avoidance, with the greatest differences being in response generation circuitry downstream from the BLA (Hitchcock and Davis, 1986; LeDoux et al., 1988; Tomaz et al., 1993; Amorapanth et al., 1999). The phenomena that the model was explicitly designed to account for are listed in Table 1. Models of this kind, which are constructed to provide possible explanations of certain phenomena, are sometimes referred to as “top-down” models. This is to be contrasted with so-called “bottom-up” models in which low-level properties of a system are comprehensively incorporated into a model and the emergent behavior of the running system then examined. A bottom-up model of fear conditioning that shares features in common with the present top-down model has recently been described and analyzed (Li et al., 2009). It incorporated much-more detailed information on synaptic and cellular properties than does the present model. It simulated both conditioning and extinction, but its scope did not permit examination of many of the phenomena of Table 1. The two types of models have complementary utility. Bottom-up models allow one to explore the completeness of one's understanding of the properties of a system that has been extensively analyzed; if one really understood it all and incorporated it into the model, the model ought to successfully emulate the functioning system. Top-down models allow one to explore what sorts of lower-level properties and inter-relations could explain the known behavior of a complex system; they can guide one in the search for actual mechanisms and the interpretation of existing observations.

Table 1.

Design targets.

CONDITIONING
A.	Conditioning contingent on CS onset preceding US onset.
B.	Conditional stimuli should reinforce new fear learning (higher-order conditioning).
C.	Cue conditioning and context conditioning should be stimulus-specific.
D.	CS fear should be well expressed outside the training context1.
E.	In an unfamiliar context the US must be delayed if context fear conditioning is to occur (“immediate shock deficit”)2.
F.	Blocking should occur and be PAG opiate receptor-dependent3.
EXTINCTION
G.	Extinction of CS fear and context fear should be stimulus-specific.
H.	Extinction of cued fear should not generalize to new contexts (renewal) and renewal should have “gating-like” properties4.
I.	Extinction should be PAG opiate receptor-dependent5.
AMYGDALA
J.	Conditioning should require BLA synaptic plasticity6,10.
K.	Cue conditioning should ordinarily be LA-dependent7 (but see Table 4).
L.	CEm should be required for expression of both cue and context fear16.
M.	Post-training ablation of BL should abolish previously learnt cued fear, but pre-training BL ablation should not prevent cue fear conditioning8.
N.	Context conditioning should ordinarily be BL-dependent9.
O.	Extinction should require BLA synaptic plasticity10.
HIPPOCAMPUS
P.	Newly acquired contextual fear should be lost if hippocampus-ablated (retrograde amnesia) but long established (“remote”) contextual fear should not11.
Q.	Pre-training hippocampal suppression (not ablation) or NMDAR blockage should preclude context fear learning (anterograde amnesia)12.
R.	Cued fear learning should ordinarily be hippocampus-independent 11.
S.	Context conditioning to long-familiar contexts is not hippocampus-dependent (no retrograde or anterograde amnesia)13.
T.	After pre-training hippocampal ablation, extinction should be context-specific as evaluated by ABA, ABC, and AAB renewal18.
U.	Pre-training hippocampal ablation should not prevent context fear learning but should accentuate the immediate shock deficit 14.
V.	After hippocampal ablation, acquisition of context conditioning should be PFC-dependent15.
W.	Expression of remote context fear should be PFC-dependent17.

References ¹Bouton and King (1983). ²Fanselow (1990). ³Cole and McNally (2007). ⁴Bouton (1993, 2004), Harris et al. (2000). ⁵McNally et al. (2004b), Parsons et al. (2010). ⁶Rodrigues et al. (2001). ⁷Goosens and Maren (2001), Nader et al. (2001), Calandreau et al. (2005). ⁸Anglada-Figueroa and Quirk (2005), Jimenez and Maren (2009). ⁹Helmstetter (1992), Fanselow et al. (1994), Yaniv et al. (2004), Calandreau et al. (2005), Onishi and Xavier (2010). ¹⁰Sotres-Bayon et al. (2007), Zimmerman and Maren (2010). ¹¹Kim and Fanselow (1992). ¹²Young et al. (1994), Stiedl et al. (2000), Bast et al. (2003), Quinn et al. (2005), Parsons and Otto (2008), Schenberg and Oliveira (2008), Raineki et al. (2010). ¹³Young et al. (1994), Anagnostaras et al. (2001). ¹⁴Wiltgen et al. (2006). ¹⁵Zelikowsky et al. (2010). ¹⁶See Wilensky et al. (2006), Ciocchi et al. (2010). ¹⁷Quinn et al. (2008a). ¹⁸Wilson et al. (1995), Frohardt et al. (2000), Zelikowsky et al. (2011) but see Ji and Maren (2005).

The present model can account for many properties of Pavlovian fear conditioning. Most of the ideas upon which it was based have been separately expressed by various authors. However, the inclusion of a computational implementation makes it possible to determine whether and how independently plausible mechanisms can work together in a comprehensive model. It is to be expected that the attempt to engineer a system that can successfully emulate the behavior produced by biological fear-learning circuitry will help us to make sensible interpretations of experimental findings on actual fear-learning circuitry and will aid efforts to determine the actual neural mechanisms of fear conditioning and extinction.

Materials and Methods

Scope

FRAT is an attempt to construct a relatively simple neurally oriented model of context conditioning and of “delay” cue conditioning, in which CS and US offset coincide or the US begins contiguously with CS offset. FRAT is not presently subject to trace conditioning.

The model has been made computational because we believe informal reasoning does not provide a reliable way of determining how the fairly large number of mechanisms needed to account for observed phenomenology will interact. Accurate qualitative predictions are the model's primary goal. Although some ordinal quantitative predictions of the model may be useful, available information is insufficient to guide production of a model that could be expected to make comprehensive quantitative predictions.

The phenomena that FRAT was constructed to emulate are those listed in Table 1. There are certainly some phenomena, such as latent inhibition and trace conditioning, that the model does not presently simulate, nor intend to. Although we believe that small additions would have allowed simulation of these and other phenomena, we have wished to limit the models complexity. However even as it stands, the model makes a substantial number of empirically valid predictions that go beyond the design-criteria of Table 1 and many predictions that remain to be tested.

At present FRAT deals with only two CSs (CS1 and CS2) and three contexts (A, B, and C), which are considered to have completely non-overlapping neural representations. This has allowed us to defer issues of storage capacity and related complexities that are better initially considered independently of the matters listed in Table 1 and that we hope to address in later work.

FRAT neurons

The resting potential of FRAT neurons is, for convenience, taken as zero. Membrane potential (V) is specified as deviation from rest. Active synapses open postsynaptic ion channels (cause conductances) for particular ions. Excitatory input causes ion movements that depolarize the membrane potential toward an excitatory reversal potential E (taken as 100 mV above rest). Inhibitory input opens ion channels that have an equilibrium potential near the resting level and thus result in current flows that move the membrane toward the resting level (as with GABA-mediated chloride conductance in real neurons in cases where the neurons are not persistently depolarized by tonic excitatory input). Therefore, inhibitory input, rather than working by hyperpolarizing the cell, is effective mainly because it allows excitatory currents to pass out of the cell instead of depolarizing it, thus attenuating EPSPs (so-called “divisive” inhibition).

When a FRAT neuron becomes depolarized beyond its firing threshold, its firing rate (also referred to as “activity” or “activation”) increases. Maximum activity, taken as unity, is reached at a depolarization level that varies according to neuron type. Individual spikes are not represented in FRAT, only firing rates.

Synaptic strength and synaptic conductance

Synapses can vary in strength or synaptic “weight” (W) according to their type and history of stimulation. The greater a synapse's weight, the more postsynaptic conductance G (conductances are expressed relative to leakage conductance) will be produced by a given level of presynaptic activity A according to the relationship,

$$G=A\cdot W$$ (1)

Summation of excitation and the interaction of excitation and inhibition

The neurons of this model treat the summation of excitatory inputs as though all excitatory input synapses were located close together on their postsynaptic targets; in computational terms, they all innervate a single “dendritic compartment.” Inhibition has quite different consequences depending on whether inhibitory synapses are located electrically close to excitatory synapses (i.e., in the same electrical compartment) or at a separate location more proximal to the soma (for modeling purposes, a second electrical compartment); we thus distinguish here between “distal” and “proximal” inhibition (Vu and Krasne, 1992). When inhibition is distal, it competes more or less symmetrically with excitation; an increase in one can always be offset by an increase in the other. However this is not the case for proximal inhibition, which for reasons explained below, was used here.

Denoting the sum of all the conductances generated by excitatory input as G_e, the equilibrium potential for excitation as E, and the sum of inhibitory conductances by G_i, the depolarization produced by summed inputs is of the form

$$V=\left(\frac{G_{\text{e}}\cdot E}{1+G_{\text{e}}}\right)\left(\frac{1}{1+G_{\text{i}}}\right)$$ (2)

In the absence of inhibition, membrane potential is given by the ratio in the left brackets, which is graphed in Figure 1 (bold solid line). When G_e is much smaller than unity, the depolarizations produced by each active, effective synapse sum approximately linearly. However, as G_e grows, each increment of input adds less to V, and the total depolarization becomes asymptotic to E. This non-linearity of summation is used to advantage in the model. As seen in the right bracket term of Eq. 2, proximal inhibition acts as a divisor for excitation. It is effect is shown in the thin solid lines of Figure 1 (for comparison the effect of distal inhibition is shown by the dashed lines).

Figure 1.

Relationship between depolarization (V) and excitatory and inhibitory conductances (G_e and G_i). Solid curves plot depolarization as a function of G_e for various values of G_i assuming proximal inhibition (V for distal inhibition is given by dashed curves).

Computational steps and time resolution

A detailed technical account of FRAT is provided in Section “Supplementary On-line Materials.” However a few basic features that will be needed for what follows must be mentioned here: The state of FRATs variables is updated every 1 s of real time, and all graph abscissas are in seconds. Values of variables should be thought of as averages or peak values for the interval in which they are determined. Thus phenomena that occur on millisecond time scales are not emulated as such. Events occurring in a given, 1 s time interval cause plastic changes that are expressed behaviorally in the next.

Parameters

Finding a single set of parameters (thresholds, change rates, etc.) that will allow FRAT to behave so as to meet the specifications of Table 1 required considerable experimentation. Search routines such as Simulated Annealing might have been useful, but we chose informal experimentation because it was helpful in providing insight into the interactions of various processes. The parameters that must be set include potentiation and depression rates for BLA principal cells and inhibitors in the presence of reinforcement and extinction modulator, the relative numbers of neurons carrying each elementary and configural representation of hippocampus and cortex, the thresholds and saturation depolarizations of all the neuron types indicated in Figure 5A, the rate of formation of hippocampal and cortical configural representations, and depolarization and “calcium” levels at which potentiation and depression began and ended in the curves of Figure 5C. The parameters eventually settled on (specified in section Supplementary On-line Materials) are certainly not claimed to be in any way a “best” set; however, they do establish that a model with the structure employed can generate the desired set of phenomena, which is the purpose of this paper.

Figure 5.

Full FRAT circuit and plasticity properties. (A) Circuitry controlling plasticity has been added to the circuit of Figure 3. The added elements are explained in the text. (B) Activity of R and X as a function of CEm activity at time of reinforcer onset. The curve labeled US indicates the effect of a full-strength footshock. The curves labeled CS refer to the secondary reinforcing effect of a CS that caused the amount of CEm activity indicated. (C) Neuromodulator-dependent factors determining change of synaptic strength of synapses on amygdala principal cells and inhibitory interneurons. Left: Factors R_P and X_P of Eq. 3 as a function of postsynaptic depolarization for amygdala principal (P) cells. Right: Factors X_I and R_I of Eq. 4 as a function of amount of “Ca” for inhibitory interneurons (I). Parameters α, β, η, ζ are max levels of the curves shown.

Notation

Neuron type names are in bold font; thus, “CEm” refers to neuron type CEm, which may be thought of as a single neuron or as a population of neurons all of which behave similarly. “CEm,” not in bold font, refers to the central nucleus CEm.

Supplementary on-line materials

A full mathematical description of the model and presentation of simulations for which there is not space here can be found at http://escholarship.org/uc/item/3hk369c4. A document providing Matlab code for FRAT and instructions for using the program is in preparation; when completed it will be placed at this URL.

Results

The model and rationale

In order to create a model, specific assumptions about fear learning and its circuitry had to be adopted. Many of the assumptions made are widely accepted as valid. However, some remain matters of debate. In these cases we selected particular possibilities as working hypotheses; as new facts come to light, revisions will surely be required.

Outline of major information flow pathways

Here we consider the broad BLA as composed of lateral amygdala (LA), the lateral basal area (BL), and the medial basal region (BM). These regions project via a variety of routes to medial central nucleus (CEm), which is believed to be the portal to fear-expression circuitry that lies downstream of the amygdala itself. Many of the synapses within the BLA and its pathways to CEm are plastic (starred in Figure 2A), and most have at one time or another been suggested to play some role in fear conditioning or extinction. Although it is generally agreed that changes mediating cue and context fear conditioning occur within BLA, the roles of the different sub-regions of the BLA are still being resolved, as is the route of information flow from LA and BL to CEm. Figure 2A, modeled after a figure of Pare et al. (2003), summarizes aspects of the known circuitry; Figure 2B portrays the circuit as implemented in FRAT.

Figure 2.

Organization of amygdala fear-learning circuitry. (A) Major inputs to amygdala are gray arrows (thicker lines ∼ richer projections). US pathways are indicated in red; dots indicate that PAG projections to indicated targets are at least in part indirect. Synapses known to be plastic are starred and numbered for reference. Based on a figure from Pare et al. (2003). (B) Organization of FRAT fear-learning circuitry. Dotted BM/ITC/CEl pathway to CEm neurons becomes enabled by training if BL is lesioned. The silhouette of the conductor is intended to characterize our view of the role of PFC in fear-learning processes: We suggest that it plays an “executive” role in complex information processing tasks such as establishing new configural representations and consolidation as well as acting as a modulator of fear learning, expression, and extinction without actually mediating these processes or being the final repository of contextual representations or fear or inhibitory engrams (cf. Rudy et al., 2005). References for (A): 1. – Chapman et al. (1990), Huang and Kandel (1998); 2. – Mahanty and Sah (1998), Bauer and LeDoux (2004); 3. – Bauer and LeDoux (2004); 4. – Rogan and LeDoux (1995), Bauer et al. (2002); 5. – Marsicano et al. (2002); 6. – Royer and Pare (2002), Amano et al. (2010); 7. – Amano et al. (2010); 8. – Pare et al. (2004); 9. – Marsicano et al. (2002); 10. – Bauer and LeDoux (2004).

Pathway for CS (cued) fear

Based on available data (see Figure 2A; McDonald, 1998; Goosens and Maren, 2001; Nader et al., 2001; Calandreau et al., 2005), the LA of FRAT receives input from cortical sensory and sensory association areas (hereafter referred to collectively as “stimulus representation” or “representation” cortex) and from thalamus (McDonald, 1998; Doron and Ledoux, 1999), and LA mediates a major portion of cued fear conditioning. Current belief is that LA output might reach CEm via BL or BM (Pare et al., 1995, 2003; Pitkanen et al., 1995; Savander et al., 1995; Royer et al., 1999) or by disinhibitory mechanisms via intercalated cells (ITC) or the lateral part of the central nucleus (CEl; Pare et al., 2004; Ciocchi et al., 2010; Haubensak et al., 2010), but since post-training lesions of BL appear to abolish conditional responses to discrete cues (Anglada-Figueroa and Quirk, 2005), FRAT LA was made to ordinarily access CEm via BL. However, to reflect the fact that CS fear can be acquired and extinguished normally after pre-training lesions of BL (Nader et al., 2001; Yaniv et al., 2004; Anglada-Figueroa and Quirk, 2005; Calandreau et al., 2005; Onishi and Xavier, 2010; but see Goosens and Maren, 2001), conditioning of FRAT in the absence of BL was made to cause development of an alternate pathway from LA-to-CEm.

Synaptic changes within LA are widely regarded as essential for development of cued fear. There is limited evidence that has been taken to suggest that most LA principal cells receive cortical and thalamic innervation that would allow conditioning to almost any CS (Repa et al., 2001; Rumpel et al., 2005; Han et al., 2007; Won and Silva, 2008; Zhou et al., 2009). Although far from established, we take this as justification for the initial, simplifying assumption that each LA principal cell is innervated by all afferents to LA and that after conditioning all LA principal cells respond similarly (referred to here as “uniform innervation”). Under this assumption, any synaptic changes downstream of LA would alter responses to all stimuli indiscriminately, which would lead to arbitrary and unwanted stimulus generalization. Therefore conditioning to a CS in FRAT has been made entirely due to plasticity at the level of BLA. However, in reality central nucleus plasticity appears to contribute to conditioning (Wilensky et al., 2006; Ciocchi et al., 2010). In fact, if one lifts the assumptions of uniform innervation at the level of LA, plasticity downstream of BLA becomes a possible asset, as addressed in the Section “Discussion.”

Pathway for conditioning to context and other configural stimuli

Since pre-training inactivation of BL as well as suppression of activity or plasticity of hippocampus, which projects richly to BL, substantially reduces context fear conditioning (Fanselow and Kim, 1994; Young et al., 1994; Stiedl et al., 2000; Bast et al., 2003; Yaniv et al., 2004; Calandreau et al., 2005; Quinn et al., 2005; Parsons and Otto, 2008; Schenberg and Oliveira, 2008; Onishi and Xavier, 2010; Raineki et al., 2010; but see Goosens and Maren, 2001; Matus-Amat et al., 2004), the hippocampus–BL pathway was taken to mediate fear conditioning in normal (unlesioned) FRAT, and BL was assumed to directly excite CEm. The actual pathway to BL in rodents may be dorsal hippocampus–ventral hippocampus–BL (see Fanselow and Dong, 2010), but dorsal and ventral hippocampus were not distinguished in the model.

The role commonly attributed to the hippocampus in context fear learning is the rapid exposure-dependent establishment of neural representations of elemental stimulus conjunctions, including contexts, independent of actual training with a US (Kiernan and Cranney, 1992; Fanselow, 2000; Rudy and O'Reilly, 2001; Rudy, 2009). It is presumed to be because of this that at least some tens of seconds of exposure to a context must precede US onset for context fear to be learned (the so-called “immediate shock deficit”; Blanchard et al., 1976; Fanselow, 1986, 1990; Landeira-Fernandez et al., 2006). This essential rapid learning process is prevented if the activity or plasticity of the dorsal hippocampus is suppressed (Matus-Amat et al., 2004; Stote and Fanselow, 2004). These characteristics were design-criteria for FRAT.

It is often postulated that hippocampus re-establishes in cortex representations of the full set of elemental cues that compose a context (e.g., Marr, 1971; McClelland et al., 1995). FRAT contains circuitry designed to implement such reinstatement. However, if reinstated representations of elementary contextual cues provide sufficient input to amygdala to support context fear conditioning, then according to the anatomy proposed in Figure 2B, context fear learning would remain possible despite BL lesions. Since we decided above to accept those data which suggest that such learning does not occur in rats, projection to amygdala of reinstated elementary cues has been made of negligible strength in the simulations of this paper. Nevertheless it must be acknowledged that BL lesion studies do typically find some residual context conditioning; therefore the consequences of increasing the effect of reinstated contextual cues should at some point be explored.

Alternative representations of configural stimuli- consolidation and compensation

It is well-known that whereas context fear is lost if hippocampus is ablated soon after conditioning (retrograde amnesia), this does not happen if sufficient time passes before ablation (Kim and Fanselow, 1992). It is believed that conjunctive representations established in the hippocampus are gradually shifted to cortex, the inferred process of “consolidation” (our references here will be only to this slow, “system” consolidation as opposed to the much faster, protein synthesis-dependent stabilization of synaptic change sometimes referred to as “cellular” consolidation). Moreover, if sufficient time for consolidation is allowed following simple familiarization, before context conditioning is attempted, no retrograde amnesia results from post-training hippocampal ablation (Anagnostaras et al., 2001). Thus, hippocampal representations, once formed by incidental learning, seem to become re-established in cortex even in the absence of conditioning. These characteristics were design requirements for FRAT (Table 1). Consolidation was assumed to cause replacement of hippocampal representations of familiarized conjunctive stimuli with representations located in sensory-representation cortex. Since hippocampus innervates BL, whereas representation cortex innervates LA, post-conditioning consolidation must not only transfer conjunctive representations from hippocampus to cortex but must also replace hippocampus–BL synaptic changes with cortex-LA changes that will produce similar behavior.

Whereas suppression of hippocampal function greatly impairs context conditioning, actual removal of hippocampus does not (Maren et al., 1997; Frankland et al., 1998; Cho et al., 1999; Richmond et al., 1999; Rudy et al., 2002; Wiltgen et al., 2006). One interpretation of this has been that intact hippocampus suppresses conditioning of fear to cortical representations of the elemental cues that compose contexts (Matus-Amat et al., 2004; Biedenkapp and Rudy, 2009; Rudy, 2009), and thus, once hippocampus is removed, conditioning to elemental cues of context becomes possible, whereas before it was not. However, if cortically represented elements of context that were present but suppressed before hippocampal removal, and operative after hippocampal removal, were capable of supporting context conditioning, then they should be able to support context conditioning even without any pre-exposure to context. However, the fact appears to be that even hippocampus-ablated rats require tens of seconds of experience with a context before they can be conditioned to it, just as do normal rats (Wiltgen et al., 2006). This suggests that conditioning in such animals is not to pre-existing elemental cues, but rather it is to conjunctive representations that, in the absence of hippocampus, form cortically over tens of seconds as they do within the hippocampus in normal animals (Fanselow, 2010). We refer to this hypothetical process as cortical “compensation.” In FRAT such compensation occurs and creates conjunctive representations in sensory-representation cortex; under the parameter values used for the simulations of this paper, elemental cues of context are never the basis for context conditioning.

It is not surprising that more signs of cortical compensation should be seen after hippocampal ablation than during disruptions of function, since ablation causes degeneration as well as dysfunction. However, whether some degree of compensation also occurs when only function is disrupted, is unclear. Limited context fear is often, though not always learned despite hippocampal APV or muscimol infusion (Young et al., 1994; Stiedl et al., 2000; Bast et al., 2003; Quinn et al., 2005; Parsons and Otto, 2008; Schenberg and Oliveira, 2008; Raineki et al., 2010), but it is not known whether such partial effects should be attributed to compensation or to manipulations that were incomplete. As a working hypothesis for the design of FRAT, we assumed the latter; in FRAT no compensation is produced by suppression of hippocampal activity or plasticity unless hippocampus is actually removed. This choice led to an unexpected ability to correctly predict the outcome of several experiments on hippocampal involvement in extinction described below.

Assumptions about the development and locus of conjunctive representations were important in the construction of FRAT not only because of the relevance of such representations to learning of context fear, but also because inhibition associated with such cues can contribute to the context-specificity of extinction that is revealed by renewal. Therefore, if as just discussed, hippocampus-ablated animals can form conjunctive representations cortically, it might be expected that they would also be subject to renewal. The literature on renewal in hippocampus-ablated rats is somewhat conflicting (Wilson et al., 1995; Frohardt et al., 2000; Ji and Maren, 2005; Zelikowsky et al., 2011); however we felt that there was sufficient evidence that such rats are subject to fairly normal renewal to make this capacity one of the design-criteria of Table 1.

Pre-frontal cortex

The pre-frontal cortex (PFC), which is reciprocally well connected to amygdala and hippocampus (Figure 2), affects fear conditioning in several ways:

1. Although, as discussed above, context conditioning occurs almost normally when hippocampus is ablated, this is not the case if PFC (infralimbic or prelimbic) has also been ablated (Zelikowsky et al., 2010).

2. Even with hippocampus intact, context-conditioned IL-PFC-damaged animals that have been trained to fear one context may also treat a neutral context as dangerous (Zelikowsky et al., 2010). Such over-generalization could be taken to suggest that PFC is involved in normal hippocampal pattern separation, and thus perhaps in the normal operation of the hippocampus, generally.

3. Consolidated context fear is reduced by post-consolidation PFC lesions (see Table 1, W; Frankland and Bontempi, 2005; Quinn et al., 2008a; Nieuwenhuis and Takashima, 2011). Given our assumption that a representation cortex-to-LA pathway replaces the hippocampus-to-BL pathway of original learning, this suggests that full activation of cortical contextual representations may require PFC.

4. In animals with certain PFC lesions, extinction is not well retained unless extra extinction training is given, though extinction itself seems to occur normally (Quirk et al., 2000; Lebron et al., 2004). It has also been found that pre-frontal neurons can recruit inhibitory activity within amygdala (Milad and Quirk, 2002), and that various drugs thought to affect learning or consolidation prevent or retard the stabilization of extinction when applied to PFC (Santini et al., 2001, 2004; Burgos-Robles et al., 2007). These and related observations are sometimes taken to suggest that extinction occurs because learning within PFC causes it to recruit inhibition of amygdala circuitry. However, extinction itself is normal in PFC-lesioned animals; it is only its persistence that is altered. Moreover, once extinction has occurred, inactivation of those parts of the PFC thought to drive inhibition of amygdala fear circuitry do not cause a return of fear responses (Sierra-Mercado et al., 2006); and drugs that are thought to affect learning or consolidation prevent extinction when applied to amygdala itself as well as to PFC (e.g., Lin et al., 2003; Herry et al., 2006; Zimmerman and Maren, 2010). Thus, while there is no doubt that PFC affects the stabilization of extinction, the exact nature of the effect is not yet understood.

The PFC is commonly postulated to play some sort of executive role that enables or facilitates complex information processing tasks that extend over time (Miller and Cohen, 2001). We think the potent, but somewhat complex and ill-defined effects of PFC on fear are consistent with its acting as a modulator of fear learning, expression, and extinction without it actually mediating these processes or being the final repository of contextual representations or fear engrams. Moreover, given the significance of context in extinction learning, we believe that the role of PFC in extinction may well be, at least in part, secondary to the role in context learning suggested by points 1–3 above (see Morgan et al., 2003; Zelikowsky et al., 2010). Indeed, if the normal functions of the hippocampus are in fact somewhat PFC-dependent, then one possible interpretation of the loss of extinction during the first 24 h after its establishment in PFC-lesioned rats is that hippocampal pattern-completion mechanisms are compromised. These rats may fail to compensate for small changes in context due to the passage of time, with the result that extinction, which is extremely context-specific, suffers more renewal due to the passage of time than would normally be the case. What appears as “forgetting” might merely be such renewal.

Given the above observations, and considering the presumed executive function of PFC, both the processes of compensation and consolidation and the activation of cortical conjunctive representations in FRAT have been made PFC-dependent. Some precedent for a role of PFC in the consolidation process is provided by recent evidence that off-line PFC activity is required for post-training stabilization of motor skill learning (Kantak et al., 2010).

Pathway for consolidated and compensated contextual fear

The destination within amygdala of cortical neurons that, through compensation or consolidation, have come to represent cue conjunctions have not been studied. Sensory association areas preferentially innervate LA, but very high level association cortex, and especially polymodal cortex, may also innervate BL (McDonald, 1998; Pitkanen et al., 2000; Lindquist et al., 2004; Suzuki, 2010). In FRAT we made these cortical representation neurons project exclusively to LA.

One can only guess whether the same or different cortical neurons contribute to the representations formed as the result of compensation and consolidation. It is commonly supposed that consolidation causes the development of cortical representations that are extensively integrated with other pre-existing cortical representations. Representations formed by compensation are perhaps more akin to temporary working memories or hippocampal “snapshots” of cortical activity patterns (as in Marr, 1971). Representations formed by consolidation and compensation might therefore be expected to be separate and have different properties. However, pending evidence of this, we provisionally opted for simplicity and used the same cortical neurons for both types of representation in FRAT.

Table 2 lists the main assumptions discussed above.

Table 2.

Summary of main assumptions.

1.	Information about CSs and contexts is provided by a thalamus-sensory-representation cortex (defined in text) pathway.
2.	Cued fear is normally learned in LA.
3.	Conjunctive representations (including representations of context and context/cue conjunctions) are rapidly developed in hippocampus during exposure and projected to BL where fear becomes conditioned to them.
4.	The normal route from LA to the output region CEm is via BL.
5.	Fear extinction occurs within LA and BL.
6.	Over time, hippocampal representations gradually shift to sensory-representation cortex, and fear conditioned to the hippocampal representations becomes conditioned to their cortical replacements. This process is PFC-dependent.
7.	In absence of hippocampus a PFC-dependent compensation process allows sensory-representation cortex to rapidly establish conjunctive representations.
8.	In absence of BL, training with a US rapidly establishes a functional pathway from LA-to-CEm via ITC, CEl, and/or BM.

US pathway

There are a great many anatomical routes via which information about footshock USs might reach the baso-lateral and central complexes of the amygdala. Nociceptive information reaches the brain at via spino-thalamic, spino-reticular, spino-mesencephalic, spino-parabrachial, and perhaps still other pathways, and each of these can then project nociceptive information to the amygdala via both direct and indirect routes (see, e.g., Willis and Westlund, 2004). Given this abundance, it has been difficult to define which pathways are actually necessary or sufficient to reinforce fear learning (Shi and Davis, 1999; Brunzell and Kim, 2001). However, a recent study (Johansen et al., 2010b) has reported that muscimol inactivation of periaqueductal gray (PAG) of the spino-mesencephalic pathway, electrical stimulation of which can act as a US for fear conditioning (Di Scala et al., 1987), abolishes at least cued fear conditioning. PAG output can itself reach amygdala via intralaminar thalamus, anterior cingulate, hypothalamus, locus coeruleus, and ventral tegmental area, and, again, which pathways are most important has not be determined (see, e.g., Herrero et al., 1991; Shi and Davis, 1999; Brunzell and Kim, 2001; Lanuza et al., 2004; Johansen et al., 2010b). For the purposes of designing FRAT, we accept the likely importance of portions of the PAG as an ascending route for the US, in addition to its being on the output pathway mediating freezing (see Carrive, 1993), but we do not try to specify further.

The fear-expression model

The operation of the model once learning has occurred is described first:

Conditioning is due to potentiation of thalamic, cortical, and hippocampal – BLA synapses

Conditioning is due to potentiation of synapses made by cortical and thalamic afferents on LA principal cells and hippocampal afferents on BL principal cells. LA input mostly codes simple cues used as CSs, but may also code for complex configurations (as in visual configuration-sensitive cells of inferotemporal cortex (e.g., Miyashita and Chang, 1988; Perrett et al., 1992) if these have developed as the result of consolidation or compensation (see previous section), or other processes. BL principal cells are innervated by LA output cells via non-plastic synapses as well as by conjunction-representing neurons of hippocampus.

Extinction is due to potentiation of afferent–LA and afferent–BL inhibitory interneuron synapses

The facts of renewal demand that extinction be at least in part due to active inhibition of responses whose ability to be excited is not erased. Given the assumption of uniform innervation (see above), extinction can only be stimulus-specific if inhibition is recruited via synapses made by LA and BL afferents. In FRAT these are synapses of LA and BL afferents on feed-forward inhibitory interneurons (Hestrin and Galarreta, 2005; Woodruff and Sah, 2007).

A recent paper by Herry et al. (2008) has reported a class of projection neurons in BL (referred to there as “extinction” neurons) that increase their responses during extinction, while activity of another class that previously became active during conditioning (“fear” neurons), declines. This suggests that in BL, afferents may at least in part recruit inhibitory interneurons via a special class of projection neurons. Computationally this would be equivalent to what was assumed here, though the downstream effects of the projecting extinction neuron activity, which are unknown, presumably have consequences of their own.

BL receives crucial input from context/CS conjunction-representing neurons

The phenomena of renewal require that the inhibition that mediates extinction of a CS be preferentially operative in the context where extinction occurred. In order to achieve this it was assumed that hippocampus forms representations of conjunctions of context and CSs that have been experienced in the context. Inhibition conditioned to such representations at the level of BL is the mechanism used in FRAT to produce context and CS-specific inhibition of fear responses. Blair and colleagues (Moita et al., 2003) have in fact seen hippocampal units that respond to a conditioned CS only when the rat is in a particular location.

Salience of contextual representations is reduced by habituation

Stimuli such as contextual cues that are continuously present would be expected to be less salient and have less impact than stimuli that have sudden onset and are present only briefly (Phillips and LeDoux, 1992). This is captured in FRAT by partially discounting the activations of contextual inputs to LA and BL by an “Habituation Factor” when calculating the activity of amygdala neurons; habituation in FRAT affects only performance, not synaptic weight alterations.

Output from LA and BL is truncated by downstream fear-expression-mediating neurons

Figure 4 shows the sort of output that a FRAT CEm cell (Figure 3) would produce in an experiment where training occurred in context A, extinction in B, and a test was then given in C (“ABC renewal”). After conditioning the CS in context A, potentiation would have developed at synapses of CS-representing neurons on LA principal cells, context A-representing neurons on BL principal cells, and context A/CS conjunction-representing neurons on BL principal cells. As a result, CEm neurons would respond to context A itself and to CS in context A as shown in the first two bars on the left of Figure 4. When moved to context B there would be no response either to the new context by itself or to the context B/CS conjunction. However, responding to the CS itself would remain, with the result that some CEm firing in the new context would still occur, as shown in the third bar on the left. After extinction in context B, inhibition would be conditioned in varying degrees to the CS itself, to context B itself, and to the context B/CS conjunction. This would result in greatly diminished CEm response to the CS when tested in context B, as shown in the fourth bar on the left of Figure 4. After the post-extinction move to context C the only inhibition would be to the CS itself, with the result that responding to the CS would be greater than it was in B as shown in the last bar on the left. If manifestations of fear were a direct reflection of this profile of activity, there would be two deviations from reality: (1). Although the fear produced by the CS would transfer substantially from context A to context B, it would be much stronger in A than B. (2). Extinction, though asymptotic, would be incomplete. The model deals with such difficulties by interposing between CEm and the activation of fear behavior a fear-expression neuron F whose firing threshold and dynamic range are such that F fails to fire when CEm activity is below level a and fires at its maximum rate when CEm activity is above b. The firing rate of F would then be as shown in the right panel; the result of the limited dynamic range of F is that expression of conditional fear prior to extinction is the same in B as in A, and behavioral extinction in B is complete even though some activity of CEm remains.

Figure 4.

Relation between CEm and F activity. Explanation in text.

Figure 3.

FRAT expression circuitry. Explanation in text. Reinf is input caused by US or other reinforcing events. The dashed direct pathway (ITC/BM/CEl) from LA principal cells to CEm neurons is enabled by training in the absence of BL.

Learning rules

Some forms of LTP have properties that seem to make them ideal as a mechanism for associative conditioning. Thus, it is natural to imagine that fear conditioning might be simply a matter of establishing LTP at CA1-like synapses of CS-representing cells on CR-producing cells by pairing presynaptic activity with strong US-caused postsynaptic depolarization. However, this simple Hebbian scenario by itself seems to us to be an insufficient basis for simulating many of the properties of real fear learning.

Determination of synaptic change at amygdala principal cell synapses

Three considerations have caused us to make potentiation of presynaptically active synapses on FRAT LA and BL principal cells dependent on something other than input which directly represents the US: (1) We do not want a CS that begins after the onset of the US to cause conditioning; but it would do so if LTP on principal cells were determined by direct US-caused depolarization. Although some procedures, demonstrate the possibility of simultaneous or backward pairing or even the superiority of the latter for producing associations (Mowrer and Aiken, 1954; Heth and Rescorla, 1973; Mahoney and Ayres, 1976; Rescorla, 1980; Ayres et al., 1987; Matzel et al., 1988; Albert and Ayres, 1997; Esmoris-Arranz et al., 2003; Leising et al., 2007), it appears to us that for the sorts of experiments modeled by FRAT, forward pairing produces considerably more substantial and reliable conditioned fear than do non-forward procedures when fear is measured by CS-produced suppression of an ongoing operant, aversion, or freezing (Mowrer and Aiken, 1954; Heth and Rescorla, 1973; Mahoney and Ayres, 1976; Ayres et al., 1987; Albert and Ayres, 1997; Esmoris-Arranz et al., 2003). We have chosen to approximate this situation by making the model produce freezing only after forward pairing. We have also chosen to make this rule apply to the formation of associative change per se, despite the possibility that the relative ineffectiveness of non-forward conditioning could be due to a performance rather than a learning deficit (Rescorla, 1980; Matzel et al., 1988). (2). A well-conditioned CS should block conditioning of a new stimulus presented along with it (Kamin, 1969). Furthermore, blocking should occur even if the representation for one of the stimuli innervates LA while that of the other innervates BL. (3) Second-order conditioning must be possible. Moreover, a CS that causes CEm activity and fear due to potentiation that has occurred in LA (e.g., a well-conditioned cue) should be able to reinforce new conditioning mediated by potentiation in BL (e.g., to a contextual stimulus), and conversely (Marlin, 1983; Helmstetter and Fanselow, 1989).

The first requirement, was met by having the depolarization needed for LTP induction in LA and BL principal cells produced, not by input derived directly from the US, but rather by input from a “reinforcing” neuron R that fires only transiently and only at US onset. R fires briefly when its input increases, which it does when the activity of the US-representing neuron U increases (Figure 5A). The transience of US-evoked depolarizing input to principal cells is consistent with the observation in rats that BLA principal cells fire only briefly at the onset of a US (Johansen et al., 2010b).

The second requirement was met by letting CEm inhibit the U neurons, which they do via inhibitory neuron Ω. Consistent with this, responses of rat BLA principal cells to shock are markedly attenuated when an established CS precedes them (Johansen et al., 2010b) and the apparent aversiveness of shock, as measured by post-shock freezing, is diminished if the shock comes during a previously well-conditioned CS (Fanselow and Bolles, 1979b). Moreover, opiate receptor antagonists reduce both blocking and the CS-caused diminution in post-shock freezing (Fanselow and Bolles, 1979b; Cole and McNally, 2007), and the locus of these effects appears to be the PAG (Hammer and Kapp, 1986; Helmstetter and Landeira-Fernandez, 1990; Cole and McNally, 2007). For this reason, the inhibition produced by FRAT's Ω neuron is taken to be mediated by an opioid transmitter. The quantitative effect of ongoing CEm activity on responses of R to USs is shown in Figure 5B₁. It should be noted that these mechanisms of blocking are very similar to those which appear to operate in the cerebellar circuit that mediates eye-blink conditioning (Fanselow, 1998; Kim et al., 1998).

Although introduction of the reinforcing neuron R and its inhibition by Ω may appear to allow the satisfaction of requirements (1) and (2), problems remain if R exerts its effect only by depolarizing BLA principal cells. Although, as described, onset of a new CS during a continuing unconditional stimulus would not cause conditioning to the CS, occurrence of a new CS during continued exposure to a previously well-conditioned CS would cause conditioning because the well-conditioned CS would cause a maintained depolarization of principal cells due its direct input; this would violate requirement (1). Other complexities would arise during blocking experiments, especially if the US were replaced by a previously conditioned CS. Such problems were avoided in FRAT by making potentiation depend not only on principal cell depolarization but also on a neuromodulatory input provided by R. Such an assumption is far from lacking empirical precedent. Development of virtually all forms of LTP and LTD as studied in slices appear to be sensitive to most neuromodulators. The VTA dopamine system provides an obvious precedent for neuromodulatory control of learning (Schultz and Dickinson, 2000). And there is increasing evidence in rats that reinforcement of fear conditioning is substantially dependent on dopaminergic or noradrenergic modulation (Johnston et al., 1999; Bissiere et al., 2003; Bush et al., 2010; Johansen et al., 2010a).

The third requirement was met by establishing a connection between CEm neurons and R-cells via a “secondary reinforcement” neuron S (Figure 5A). This connection allows sufficiently strong conditional responses to drive R at their onset and thereby both depolarize and provide reinforcing neuromodulator to LA and BL principal cells.

Blocking of second-order conditioning (Leyland and Mackintosh, 1978) could have been achieved by letting Ω inhibit S as well as U. However, if CEm is already very active due to the presence of a previously conditioned stimulus when a secondary reinforcement occurs, S will be able to increase its activity very little and thus will cause very little activity of R. Consequently, blocking of higher-order conditioning should occur even without input to S from Ω, and such a connection was not included in the model. The quantitative effect of prior CEm activity on responses of R to secondary reinforcers is shown in Figure 5B₁.

Determination of change at amygdala inhibitory interneuron synapses

Extinction in FRAT is due to LTP at synapses made on feed-forward inhibitory interneurons by the same afferents that feed LA and BL principal cells. Thus, LTP of synapses on inhibitors must develop when non-reinforced CSs occur. Inhibition should be sufficient to reduce CEm activity below the threshold a of the freezing neurons F, but it need not totally prevent CEm activation. Indeed, if inhibition at the level of LA is too great, renewal will not occur when shifts of context reduce inhibition at the level of BL. On the other hand, in order to simulate the observation that extinction remains approximately normal in BL-lesioned rats, sufficient inhibition must be able to develop at the level of LA to allow extinction. By making development of LTP at synapses on inhibitors contingent on a sufficient amount of activity of both CEm cells and the principal cell targets of the inhibitors, it can be assured that extinction will occur under appropriate conditions without inhibition at the level of LA growing too great to allow renewal.

Control of inhibitor LTP by CEm activity is mediated by “extinction” neurons X (Figure 5A), which distribute a neuromodulator that enables LTP development on inhibitory neurons and has other extinction-promoting effects. Although it would have been simplest to let CEm neurons directly excite X-cells, it is the case that in rats extinction is prevented by PAG opiate antagonists (McNally et al., 2004b; Parsons et al., 2010); therefore in FRAT CEm output was made to recruit X neurons through disinhibition driven by Ω neurons, as indicated in Figures 5A and B₂.

The control of LTP at synapses on inhibitors by principal cell targets is exerted via recurrent inputs to the inhibitors from their targets (Figure 5A). These inputs have ligand-gated synaptic effects that are somewhat parallel to those causing LTP and LTD of afferents to cerebellar Purkinje cells, which are by far the best studied of inhibitory neurons with plastic inputs. Recurrent input is taken to produce LTP by mechanisms similar to those which foster LTP at afferent synapses on Purkinje cells. There, moderate input is thought to initiate LTP by causing modest Ca²⁺ elevations (see Lev-Ram et al., 2003; Coesmans et al., 2004; Jorntell and Hansel, 2006). Since extinction in FRAT is mostly due to growth of inhibition rather than erasure of potentiation of principal cell synapses, reconditioning of a previously extinguished FRAT is achieved by erasing the inhibition that was established by extinction training. This is effected by strong input to inhibitors from R-cells. Such strong input causes large “Ca” elevations, which in the presence of reinforcing neuromodulator, initiates LTD of synapses made by active afferents on the inhibitors. This is analogous to the LTD of Purkinje cells synapses produced by climbing fiber input.

Quantification of postsynaptic factors affecting change

The graphs of Figure 5C quantify the effects and interactions of ordinary and neuromodulatory input to principal cell and inhibitory interneurons, as described in the preceding paragraphs. In the presence of R-cell neuromodulator, active synapses on principal cells potentiate in proportion to postsynaptic depolarization, above some threshold. In the presence of X-cell neuromodulator they may undergo slight LTD, but the parameter ζ (Figure 5C, left), which controls the extent of such depression is set to zero in the simulations of this paper. In the presence of X-cell neuromodulator (Figure 5C, right), active afferent synapses on inhibitory interneurons undergo LTP to an extent determined by the amount of “Ca” that enters the interneuron due to recurrent input or input from R. The increment of LTD increases with “Ca” concentration above a rather low threshold and then rapidly saturates with increasing concentrations. In the presence of R-cell modulator, LTD develops as a function of “Ca” concentration to an extent that mirrors the relationship between increments of LTP in principal cells and depolarization.

R-cell effects always dominate those of X-cells because R inhibits X absolutely (Figure 5A). Extinction never develops during a protracted US because US activity also inhibits X absolutely.

Presynaptic determinants of eligibility for change

The changes in synaptic strength just described occur only at active synapses. However, activity per se must be neither necessary nor sufficient, because learning fails if CS onset does not actually precede US onset, and USs that occur just as CSs terminate do cause conditioning. This caused Sutton and Barto (1981) to introduce the concept of eligibility for change. In FRAT eligibility is established by receipt of transmitter from active CS and context-representing neurons. However, as with NMDA receptors, which cannot open in response to postsynaptic depolarization until some milliseconds after glutamate binds to them and which continue to be responsive to depolarization for some milliseconds after unbinding (Magleby, 2004; Clarke and Johnson, 2006), eligibility slightly lags transmitter binding. In FRAT synapses become eligible for change one computational time step after the onset of presynaptic activity (i.e., the variable eligibility goes from zero toward unity) and retain eligibility for one time step after cessation of activity. Thus, onset of presynaptic activity must precede the occurrence of other change-promoting signals. Moreover, because eligibility is continued during the time step after CS offsets, cue conditioning is successful even if the US, instead of co-terminating with the CS, begins contiguously with CS offset. Eligibility in FRAT is a continuous variable that, above a threshold is proportional to activation of the input but that saturates at unity when activation reaches a certain level, which is a parameter of the model.

Calculation of change

The three factors of eligibility, R or X activity, and the effects of postsynaptic depolarization in principal (P) cells or calcium elevation in inhibitory interneurons (I) combine as the product of three corresponding numerical factors:

$$\begin{array}{l}\Delta StrengthP-cellsynapse\\ =\{\begin{array}{cc}{A}_R•R_P(Depol)•Eligibility& \text{if\hspace{0.17em}}{A}_R>0\\ A_X•X_P(Depol)•Eligibility& \text{if\hspace{0.17em}}{A}_X>0\end{array}\end{array}$$ (3)

$$\begin{array}{l}\Delta StrengthI-cellsynapse\\ =\{\begin{array}{cc}{A}_R•R_I("Ca")•Eligibility& \text{if\hspace{0.17em}}{A}_R>0\\ A_X•X_I("Ca")•Eligibility& \text{if\hspace{0.17em}}{A}_X>0\end{array}\end{array}$$ (4)

Eligibility is as defined above. A_R and A_X are the activations of R and X neurons, as discussed above. R_P, X_P, R_I, and X_I are the relative change factors for principal cells and inhibitors as given in the graphs of Figure 5C.

Plasticity of the non-BL LA-to-CEm pathway

If and only if BL is suppressed, coincident activity of LA principal cells and US-evoked activity of CEm neurons causes an effective pathway from BL to CEm to become established. This pathway is inhibited (but remains established) if suppression of BL is discontinued.

Forming representations of configural stimuli

Although there has been discussion in the literature as to how hippocampal circuitry might work to establish conjunctive representations (Marr, 1971; Skaggs and McNaughton, 1992; O'Reilly and McClelland, 1994), in FRAT this was simply assumed to occur. The numbers of hippocampal neurons (or in absence of hippocampus, of cortical conjunction-representing neurons) that can be activated by a stimulus and their levels of activation grow with negative acceleration whenever the stimulus is present (see Figures 6A and 10). About 20 s of exposure are needed before FRAT hippocampal representations develop to a point where they can have any impact on behavior, and it requires several minutes of exposure before activity is maximal. In the absence of hippocampus (Figure 6C), cortical representations grow similarly but a little slower (see simulations of Figure 10, which are roughly consistent with experimental observation Wiltgen et al., 2006). Once established, hippocampal representations are projected to BL, and cortical representations to LA, to provide bases for excitatory and inhibitory conditioning as described above.

Figure 6.

Pathways for context (configural) fear conditioning in normal and hippocampal FRAT. (A) An effective pathway (bold black arrow) from sensory-representation cortex to hippocampus is rapidly established by hippocampal NMDA-dependent LTP during exposure to novel configural stimuli (white block arrow indicates conditions for establishment of pathway). Once a hippocampal representation forms, conditioning potentiates afferent synapses on BL principal cells. (B) Consolidation. Over time (weeks) after context exposure or training, hippocampal representations of context get replaced by cortical ones, and the strength of cortical context-representing neuron synapses on LA cells get adjusted so that the new cortical pathway will produce behavior similar to that produced by the hippocampal pathway that it replaces. Open block arrows indicate these gradually developing replacements (all aspects of consolidation are PFC-dependent in FRAT). (C) In absence of hippocampus (ablated, not suppressed) stimulus exposure causes rapid cortical synaptic change resulting in establishment of cortical representations neurons for configural stimuli (Ctx, cnfg); this process is PFC-dependent. Once cortical representations form, conditioning potentiates their synapses on LA principal cells.

Figure 10.

Context conditioning in an unfamiliar context with pre-training hippocampal ablation or suppression. (A) Effect of shock delay in unsignaled context fear conditioning. A single shock at 12 s after introduction into context A caused no contextual freezing, but a shock at 100 s did. The slight increase of context fear after its onset is due to an increase of context representation activity with additional exposure to the context. (B) Amount of contextual freezing as a function of US delay in controls, hippocampus-ablated (Hx) FRAT, and hippocampus-suppressed (Hs) FRAT. (C) CS and context conditioning under three conditions. (1) Control FRAT; CS fear is learned rapidly and context fear begins after the first US. (2) Hippocampus-ablated FRAT (Hx); context fear develops a little more slowly than in controls. (3) Hippocampus-suppressed FRAT that has been extensively exposed to the context prior to conditioning; cued fear but no context fear becomes conditioned.

Consolidation requires the gradual and presumably experience-independent transfer of hippocampal conjunctive representations to representation cortex. Since representation cortex innervates LA rather than BL, it is also necessary to establish new synaptic weights that allow the newly functional circuitry to mimic that which it replaces (Figure 6B). Algorithms for doing this biologically have not been proposed, but the process is believed to depend on off-line neural activity, perhaps during sleep (Marr, 1971; Wilson and McNaughton, 1994; McClelland et al., 1995; Ji and Wilson, 2007). In FRAT the replacement was done one BL input channel (A, B, C, A1, A2, etc.) at a time: First the CEm output produced by a controlled BL input with inhibition disabled was determined and the synaptic weight on LA principal cells that would be needed in order for a similar LA input to produce the same output was calculated. Then BL principal cells were provided just enough excitation to bring CEm to its maximum firing rate, the reduction produced by BL input to the inhibitor was determined, and the synaptic weight on LA inhibitors needed to produce a similar reduction in CEm activity was calculated. Once the new weights were determined, the hippocampus–BL circuit was replaced by its cortex-LA correspondent (see Supplementary On-line Materials for details). We believe that plausible neural algorithms that involve off-line activity and synaptic plasticity could be invented.

Requirement for, and implications of, distal inhibition of P cells

Once representations of context and context-CS conjunctions have become cortical so that all input is to LA, AAB renewal cannot occur if excitation and inhibition are directed to a common electrical compartment, because the switch to context B removes comparable amounts of inhibition and excitation. However, if inhibition is proximal and G_e is large, the value of the left bracket of Eq. 2 decreases little, while that of the right bracket increases substantially, and renewal occurs. This is why we have used two-compartment models of principal cell dendrites and directed extinction-causing inhibition to the proximal compartment.

Implementation of experimental procedures

In the next section the freezing responses of FRAT to simulations of various ablations, inactivations, and pharmacological manipulations are discussed. The manipulations were done as follows:

Suppression of hippocampus (Hs) is emulated by reversibly setting the activations of all hippocampal neurons to zero and disabling exposure-dependent formation of hippocampal representations, thus disabling incidental learning.

Ablation of hippocampus (Hx) is emulated by additionally enabling the exposure-dependent development of cortical representations of configural stimuli, as described above. Hx is irreversible.

Suppression of LA (LAs) is emulated by reversibly setting all LA neuron responses (depolarizations, activity, etc.) to zero. Given the rules described above, this prevents all new synaptic change, but existing change is retained and can again be expressed if suppression is discontinued.

Suppression of BL (BLs) is emulated in the same manner as was LAs, but additionally the plasticity of the LA–CEm pathway (as described above) is enabled. LA and BL suppression disable the part of the consolidation algorithm that adjusts amygdala synaptic weights, but permit the establishment of cortical configural representations.

Suppression of CEm (CEMs) is emulated by reversibly setting the activity of CEm neurons to zero.

Suppression of PFC (PFCs) is emulated by reversibly disabling development of consolidated or compensatory cortical responsivity to configural cues. During the period of suppression any previously developed responses of cortical configuration-representing neurons is also attenuated. This last feature is considered especially provisional.

The effects of opiate receptor antagonism were simulated by attenuating the activity of Ω.

The effect of PAG GABA-agonist (antagonist) was simulated by attenuating (augmenting) activation of X by CEm.

Behavior of the model

Basic properties of conditioning and extinction

The behavior of FRAT is consistent with all of the design targets of Table 1. Some of these features are illustrated here; others are illustrated in Section “Supplementary On-line Materials.”

FRAT also has a number of behavioral characteristics that are consistent with available data but that were not considered in designing the model, and still others that constitute untested predictions. Some of these are listed in Tables 3 and 4, and a few are illustrated in this section. Unless otherwise indicated, all simulated experiments were preceded by a session in which FRAT was given sufficient exposure to all three contexts to establish full representations for them.

Table 3.

Effects of hippocampal ablation (Hx) or suppression (Hs) during extinction and renewal testing on extinction (Ext) and renewal (Rnl) in FRAT (suppression continues during renewal testing).

		Hx early	Hs early	Hx late	Hs late
AAB	Ext	Ok	Slow	Ok	Ok
	Rnl	Ok	No	Ok	Ok
ABA	Ext	Ok	Slow	Ok	Slow
	Rnl	Ok	No	Ok	No
ABC	Ext	Ok	Slow	Ok	Slow
	Rnl	Ok	No	Ok	No

Table 4.

Properties of FRAT that were not design explicit design targets.

CONSISTENT WITH AVAILABLE DATA
A.	Weak cue conditioning is training-context-specific1.
B.	Context-specificity of cue conditioning is unmasked following extinction in a novel context2.
C.	Extinction is enhanced when responses are strong and diminished when responses are weak3.
D.	Effectiveness of renewal is in the order (greatest to least) ABA, ABC, AAB4.
E.	Rate of extinction of new cued fear is substantially reduced by post-conditioning suppression of hippocampus but not by ablation5.
F.	Newly acquired CS fear extinguished with hippocampus-suppressed does not show ABC or AAB renewal6.
G.	Suppression or ablation of hippocampus soon after extinction of newly acquired CS fear causes partial loss of extinction and a loss of extinction's context-specificity7.
H.	Well-conditioned CSs suppress post-shock freezing, and this suppression is opiate-dependent8.
I.	Fear acquired normally cannot be extinguished if BL is suppressed during extinction training9.
SOME UNTESTED PREDICTIONS
J.	Extended post-training suppression of BLA prevents systems-type consolidation10.
K.	Pre- or post-training PFC lesions prevent systems-type consolidation.
L.	Extinction of both cue and context fear show systems-type consolidation. After such consolidation, context shifts cause renewal, independent of hippocampus.
M.	If hippocampus is suppressed (not ablated) after completion of systems-type consolidation, CS extinction is normal in the conditioning context but slow and context-independent in novel contexts.
N.	After ablation of LA, cue conditioning remains possible, but it is conditioning context-specific.
O.	When cues predict shock onset, LA ablation increases context conditioning, but not as much as does removal of predictive cues.
P.	Inhibition of GABA in PAG promotes development of extinction.
Q.	BL lesions do not prevent context fear if hippocampus has been ablated.
R.	Joint pre-exposure to CS and conditioning context will enhance strength of one-trial CS conditioning.
S.	Inactivation of CEm during unreinforced responding prevents extinction.

References and Notes: ¹Hall and Honey (1990). ²Harris et al. (2000). ³Rescorla (2002, 2003). ⁴Bouton (2002), Thomas et al. (2003). ⁵Corcoran et al. (2005). ⁶Corcoran et al. (2005), See text ⁷Corcoran and Maren (2001). ⁸Fanselow and Bolles (1979a). ⁹Herry et al. (2008). ¹⁰Several hours of tetrodotoxin-induced silencing of BLA at any time during the first 2 days after contextual fear conditioning, but not thereafter, reduces long-term retention of contextual freezing (Sacchetti et al., 1999). While consistent with prediction J, this appears to us to be more akin to an effect on cellular- than on systems-type consolidation, because CS conditioning is affected in the same way.

Conditioning, extinction, and context

Figure 7 shows some of the basic properties of the behavior produced by FRAT.

Figure 7.

Basic fear conditioning, extinction, and renewal in FRAT. Unless otherwise specified, simulation of extensive exposure to all three contexts preceded all simulations shown. (A) Conditioning and extinction of CS1 in context A. Event lines are labeled and have the same meaning throughout paper. Context is color-coded. Ordinate is freezing (complete stillness = 1; unsuppressed exploration or activity produced by shock = 0); abscissa in seconds. (B) Extinction in A following conditioning as above plus 50 min of exposure to context A-alone to extinguish context fear. (C) Conditioning in A followed by extinction inf B. (D) Different types of renewal. (D₁) ABA renewal. The end of extinction in B and the switch to A are shown. (D₂) As in (D₁) except that context A-alone extinction preceded testing renewal in A; the end of this extinction period and the renewal test are shown. The first few renewal test CSs re-established fear of A due to second-order conditioning. (D₃) ABC renewal. The end of extinction in B and the switch to C are shown. (D₄) AAB renewal. The end of extinction in A and the switch to B are shown. (E) Pure context conditioning.

Figures 7A–C illustrate the effects of five CS conditioning trials in context A followed by extinction of the CS in either context A or in new context B. Conditioning of fear to both the CS and the context develop rapidly and extinguish relatively slowly. As with rats, CS fear acquired in context A is also well expressed in context B.

Also, as with rats a precise evaluation of CS fear in the conditioning context is complicated by the presence of contextual fear. To provide a less contaminated evaluation of CS fear in A, Figure 7B shows a simulation in which extinction of the CS in context A was preceded by 50 min of contextual fear extinction without any CS occurrences. The Figure shows that even in the absence of background contextual fear, freezing to the CS was strong, but it then itself extinguished about as it did in Figure 7A. Also note in Figure 7B that occurrence of the CS at the start of extinction caused some second-order conditioning of fear to the context.

After extinction in any context, FRAT shows renewal of fear when the extinction context is changed. This is seen in Figure 7D for AAB, ABA, and ABC renewal. Evaluation of ABA renewal is complicated by the return of context fear when FRAT is returned to its training context (A); thus a test of renewal after extinguishing fear of A was also done. Renewal occurs with all three experimental paradigms, but it tends to be greatest for ABA, next for ABC, and least for AAB. This order seems to be consistent with available data for the behavior of rats (Bouton, 2002; Thomas et al., 2003). In the model, responsiveness during the renewal test is less for AAB than for ABC renewal because during tests in a new context (here, B or C) the only thing suppressing fear is inhibition conditioned to the CS itself. And more inhibition becomes associated to the CS during extinction in A than elsewhere because FRAT is more fearful during extinction in A (where CS, A, and the A/CS compound all promote CEm, and hence X, activity) than elsewhere (where only the CS itself promotes CEm and X activity). Responsiveness during a renewal test is greater for ABA than ABC paradigms because after extinction in B, return to A reinstates fear of both A and the A/CS compound whereas a shift to C is movement to a neutral context. Note, however, that the picture is complicated somewhat by the occurrence of secondary reinforcement during extinction, which is in fact a biological reality (Helmstetter and Fanselow, 1989).

Figure 7E illustrates the effect of contextual fear training when the US is presented without any predictive CS. A much greater level of context fear develops than was seen in Figures 7A–C, because there is no response to the CS to drive strong opioid inhibition of US-representing neurons. Animals show a similar enhancement of context fear when there are no predictive CSs (Fanselow, 1980; Fanselow et al., 1994).

Evidence of contextual dependence of conditioning

As discussed above, cued fear established in one context seems to be fully expressed in other contexts. However, underlying fear, as reflected in the activity of CEm neurons, is considerably greater in the conditioning context than elsewhere because fear conditioned to CS/context conjunctive representations, as well as possibly behaviorally subthreshold contextual fear, are absent outside the training environment. Fear-expression in FRAT is not usually diminished away from the training context, because the dynamic range of downstream fear-expression-producing neurons is limited; very strong CEm activity in the conditioning context and less strong activity elsewhere both commonly produce maximal freezing scores. However differences can become evident if conditioning is weak. This is shown in Figure 8, in which training consisted of a single conditioning trial with a very weak US. A similar result is found in real rats (Hall and Honey, 1990).

Figure 8.

Evidence of partial context-specificity of cue conditioning. CS evokes greater fear in the training context than a novel context if conditioning was established by a single weak US. (A) Testing in training context. (B) Testing in a different context. Summation tests would have disclosed subthreshold contextual fear in the training context, but not enough to account for the greater freezing seen in the training context.

Evidence of context-specificity of conditioning can also be seen in experiments where rats that had been conditioned to several CSs, each in different context, have all CSs extinguished in a new context. In such experiments (Harris et al., 2000), and in FRAT, renewal is much greater when a CS is subsequently tested in the context where it, rather than some other cue was originally conditioned (see simulations in on-line materials). Context-dependence of cued fear is also suggested by the observation that hippocampal ablation, which would be expected only to cause retrograde losses of responses conditioned to conjunctive cues, in fact causes losses in cue conditioning if the conditioning was weak (Quinn et al., 2008b); this is what would be predicted if some of the conditioning were to cue/context stimulus compounds.

Aspects of FRAT hippocampal function

As discussed above, at least at initial learning, the hippocampus is the FRAT amygdala's primary source of information about configural stimuli, and in particular of contextual information. The hippocampal representations of configural stimuli that project to amygdala are formed fairly rapidly during exposure to the stimuli, and once these are formed, they provide input to BL where conditioning of fear to them, as well as conditioning of inhibitory activity that contributes to context-specific extinction, become possible. From this alone it would be expected that (a). Contextual fear conditioning would be entirely due to potentiation of synapses within BL, which is where FRAT hippocampus projects. (b). Contextual fear could not be established until time has been allowed for the formation of a hippocampal representation of the context (c). Context fear conditioning should be prevented by hippocampal ablation (anterograde amnesia). (d). Post-training hippocampal removal should abolish previously learned contextual fear (retrograde amnesia). (e). The context-specificity of extinction and possibly extinction itself should be dependent on hippocampus.

However, there are several (empirically motivated) aspects of FRATs construction that complicate this picture: (1). If hippocampus is ablated, a PFC-dependent compensation process of unspecified mechanism allows sensory-representation cortex to form configural representations, though the formation of configural representations by this process is a little slower than normal. (2). If hippocampal plasticity or activity are suppressed pharmacologically but hippocampus remains intact, the PFC-dependent compensation process does not occur. (3). Over extended periods of time, systems-type consolidation occurs: hippocampal representations are replaced by ones located in sensory-representation cortex, and if hippocampus-to-BL principal cell or inhibitory interneuron synapses had become potentiated, potentiation that has similar consequences for downstream activity becomes established at the LA targets of the new cortical representations. In the following sections we discuss some of the behavioral consequences of these considerations.

Post-training hippocampal ablation (Hx) or suppression (Hs)

Ablation of hippocampus soon after conditioning in FRAT has little effect on CS-evoked freezing but abolishes freezing to context (compare Figures 9A,B). If time is allowed for consolidation before ablation, context fear survives (Figure 9C). This phenomenology replicates standard empirical findings.

Figure 9.

Effects of post-training hippocampal ablation or suppression. (A) AAB conditioning, extinction, and renewal shown as a control for other procedures. (B) Hippocampus was ablated right after conditioning. Context fear is lost, but responses to the CS remain and extinguish fairly normally. A switch to context B produces clear renewal. (C) If consolidation was allowed to occur and hippocampus then ablated prior to extinction, context fear is not lost. (D) If extinction is carried out soon after conditioning with the activity of the intact hippocampus-suppressed (no ablation), extinction occurs slowly [compare to (A)], and no renewal occurs when context is changed.

Extinction occurs in a normal manner after early post-training ablation (Figure 9B), but it is slow if hippocampus is suppressed rather than removed (Figure 9D). The slowing of extinction in hippocampus-suppressed FRAT was an unanticipated consequence of programming FRAT to meet the conditions of Table 1. It occurs because hippocampal representations of the context and context/CS conjunctions are missing with hippocampus-suppressed, so the only representations present during extinction are those of cortex. Consequently, less total inhibition per trial is acquired than with hippocampus intact (with ablation, as opposed to suppression, cortical representations of the extinction context and of context/CS conjunctions are rapidly established at the start of extinction). In fact, Corcoran and Maren (Corcoran et al., 2005) have observed slow extinction in rats under just these circumstances.

After extinction of hippocampectomized FRAT, renewal also occurs normally (Figures 9B,C). This is the case because cortical conjunctive representations to which inhibition gets conditioned develop rapidly at the start of extinction. However, if hippocampus is suppressed rather than ablated, AAB and ABC fail though ABA renewal persists. In rats all three types of renewal have been reported to fail under similar circumstances (Corcoran et al., 2005). This may represent a discrepancy between FRAT and reality; however, we consider it is possible that apparent lack of ABA renewal in the rats was due to ceiling effects imposed by the measurement of CS fear on a high contextual fear baseline in the experiments reported.

Table 3 summarizes a number of effects of post-conditioning, hippocampal ablations and suppressions in FRAT. It should be noted that if consolidation has been allowed to occur, extinction in the conditioning context during hippocampal suppression is normal and renewal occurs if FRAT is then tested in a different context (i.e., AAB extinction and renewal are normal). However, if extinction with hippocampus-suppressed occurs elsewhere, this is not the case. This is because consolidation establishes representations of the conditioning context and context/CS compound in cortex; but such representations for different contexts are lacking and cannot be established during extinction if hippocampus is suppressed. These would be interesting predictions to test.

If FRAT hippocampus is either ablated or suppressed soon after extinction is complete, there is partial recovery from extinction, and responsiveness becomes independent of both conditioning and extinction context (not illustrated). This occurs because all of the conjunctive representations to which inhibition has become conditioned become unavailable. Exactly this outcome was seen when rats were given hippocampal muscimol injections soon after extinction (Corcoran and Maren, 2001) although results of a later study gave somewhat different results (Corcoran and Maren, 2004); FRAT simulates the findings of both reports if one assumes that suppression of hippocampal activity by muscimol was not quite complete (see Supplementary On-line Materials). If post-extinction hippocampal ablation or suppression are performed after FRAT consolidation mechanisms have been allowed to operate, conjunctive representations have moved to cortex and synaptic potentiation has occurred that replaces the hippocampal pathway to principal cell inhibitors (as in Figure 6A). Thus loss of hippocampus causes no loss of extinction, and renewal remains normal. We are not aware of comparable animal experiments.

The immediate shock deficit

The immediate shock deficit as seen in FRAT is illustrated in Figure 10. In the normal case shocks given after less than about 13 s of exposure to an unfamiliar context cause development of no manifest context fear, but fear develops if there has been more cumulative exposure prior to the shock. Freezing has become nearly asymptotic by around 800 s of exposure.

Pre-training hippocampal ablation (Hx) or suppression (Hs)

If FRAT hippocampus is suppressed prior to exposure to an unfamiliar context, no fear can be conditioned to that context (Figure 10B). However, cue conditioning appears to occur normally (Figure 10C), though careful evaluation of strength of conditioning, context-dependence of conditioning, etc., discloses abnormalities (not illustrated). These characteristics follow from a total absence of any contextual input to amygdala if hippocampus has been suppressed prior to any stimulus exposure. If hippocampus is ablated rather than suppressed, compensation processes allow the formation of configural representations in cortex, so the properties of both cue and context conditioning and extinction appear fairly normal, though quantitative differences can be detected. In particular, a little more context exposure is needed prior to conditioning to achieve a given level of context fear (Figure 10B); in this, FRAT is very similar (by design) to real animals (Wiltgen et al., 2006).

Blocking, extinction, and PAG opiate receptors

As discussed above and as prescribed in the design-criteria of Table 1, PAG opiate receptor inhibitors attenuate both blocking (Figure 11A) and extinction (Figure 11B) in FRAT, as in rats (McNally et al., 2004a,b; Cole and McNally, 2007).

Figure 11.

Effects of PAG opiate antagonist on blocking and extinction. The circuit that controls the activity of globally distributing reinforcement R and extinction X neurons (from Figure 5) is shown above. (A) Blocking. Each graph shows two reinforced co-occurrences of CS1 and CS2 in context A followed by test presentations of CS2 in context B. Details of first reinforced co-presentation of CSs are shown in the blow-ups. (A₁) Control; no prior experience. (A₂) CS1 was previously conditioned in context C. Note that the response to CS2 is less than in A₁ because the well-conditioned CS1 partially blocked learning. (A₃) Same as A₁ except that opiate receptors were 70% blocked during training (simulated by attenuating input from O-cells input to their targets) during the two conditioning trials. Magenta arrows mark initial test responses. (B) Effect of PAG opiate receptor block on extinction. (B₁) Normal conditioning and extinction. (B₂) Same as B₁ except opiate receptors partially blocked during exctinction.

The discovery in rats of attenuation of extinction by opiate inhibitors and enhancement of extinction by an opiate enhancing agent (McNally et al., 2004b; McNally, 2005) would seem to imply that extinction is dependent on some signal that is itself PAG-dependent-in-effect an “extinction” signal that is extrinsic to the amygdala.

There are a number of minor variations of the circuitry controlling R and X in FRAT that would have the same ultimate effects on behavior. However, in the circuit actually employed (Figure 11) opiate antagonists, while attenuating blocking for primary reinforcement, would not do so for secondary reinforcement. Moreover, PAG GABA inhibitors would facilitate extinction and interfere with conditioning by causing continual firing of the X-cells. It would be interesting if such effects were found in animals.

Discussion

The goal of this project was to try to create a neurally plausible model that reproduces a number of known empirical features of fear learning (Table 1). If the neural plausibility of the model is accepted, the goal has been met. Moreover, the neural circuitry created to meet the criteria of Table 1 generated a number of additional behavioral properties which appear to be empirically valid (Table 4; simulations in Supplementary On-line Materials). For example, the model predicts that renewal should be only partial and that the procedure ABA should be most and AAB least effective (Table 4, D); this conforms with common findings (Bouton, 2002; Thomas et al., 2003). The model also correctly predicts that suppression of hippocampus during extinction should slow extinction and prevent what extinction occurs from being context-dependent, whereas suppression following extinction, should itself cause partial, context-independent recovery from extinction (Table 4, E–G; Corcoran et al., 2005).

The model additionally makes many untested predictions, some of which are listed at the bottom of Table 4. For example, it predicts that suppression of BLA, or CEm during the consolidation period over which contextual fear becomes hippocampus-independent should prevent such consolidation (Table 4, J). It also predicts that extinction should be prevented by CEm suppression during extinction training (Table 4, S).

Assumptions

Two aspects of the model especially deserve further discussion.

Globally distributing neuromodulators

Control, of plasticity by R and X-cell neuromodulators was used in two ways in FRAT: (i).To make synaptic change contingent on the ultimate output of the amygdala circuit (as assessed by CEm activity). This was crucial to the handling of both secondary reinforcement and blocking. Reliable renewal also depended on control of extinction by the circuit's output. In the absence of such control, inhibition at the level of LA could grow so strong during extinction training that not enough (context-independent) input from LA to BL would remain after extinction to renew fear when context shifts lead to reduced inhibition at the level of BL. (ii). To make conditioning contingent specifically on onset of USs (or in the case of secondary reinforcement, of well-trained CSs). This was crucial to prevent a new CS that comes on during a protracted US or established CS from becoming conditioned.

These matters could perhaps have been dealt with without the use of neuromodulators. However, changes in synaptic strength must in one way or another be made subject to control by the current output of the circuit, by the level of US activity, and by increments in each of these quantities. It is possible that increments could be computed by properties of the target cells themselves, but in FRAT the computation was implicit in the properties of the R-cells. In order for external plasticity-controlling information to be utilized, it must be represented within plastic cells in a way that does not interfere with other sorts of processing. It would seem to be difficult to mediate such control via target cell membrane potential without unwanted interactions between normal processing and plasticity control signals. Indeed, prevention of unwanted secondary reinforcement by CS-caused depolarization of principal cells was the immediate impetus to positing a neuromodulatory component to the R-cell signal in FRAT. It might have been possible to avoid this, but designing the circuit was made easier by assuming that external plasticity-controlling signal work via the cell's internal chemical signaling systems, i.e., that they are neuromodulatory.

Neuromodulatory control of conditioning of course has precedent in VTA dopamine system-dependent learning (Schultz and Dickinson, 2000). Fear learning or BLA plasticity also seems to be controlled, at least in some circumstances, by dopamine and/or norepinephrine (Selden et al., 1991; Johnston et al., 1999; Bissiere et al., 2003; Pezze and Feldon, 2004; Tully et al., 2007; Fadok et al., 2009, 2010; Bush et al., 2010; Iordanova, 2010; Johansen et al., 2010a).

The presumption that an external signal influences extinction has somewhat less precedent. However, discussions of the midbrain dopamine system often imagine a role for negative error signals in extinction, and there is evidence for the importance of such signals in extinction of cerebellar eye-blink conditioning (Medina et al., 2002). The discovery that extinction of fear conditioning in rats can be attenuated by opiate inhibitors and enhanced by an opiate enhancing agent (McNally et al., 2004b; McNally, 2005) would seem to imply that extinction is dependent on some signal that is itself PAG-dependent, and thus dependent on events external to the amygdala.

It is known that dopamine neurons, whose increased activity may mediate reinforcement, slow their ongoing rates of firing when expected reinforcements are omitted and that extinction of cerebellar eye-blink responses is at least in part due to slowed spontaneous firing of climbing fibers when unreinforced eye-blink responses occur (Medina et al., 2002). This suggests a single external signal whose increase promotes reinforcement and decrease promotes extinction. However, such bipolar control requires neurons with substantial baseline firing rates, and we elected instead to use separate reinforcing and extinction-promoting signals, carried by the R and the X-cells, respectively. One consequence of this, given the effects in rats of opiate transmitter system alterations, is that CEm had to recruit X by disinhibition, whereas if the global extinction signal had been reduction in spontaneous R firing, this complexity would have been avoided. The matter is probably subject to experimental test: If extinction-promoting signals are actually recruited via disinhibition, then it should be possible to prevent extinction with PAG GABA-agonists, and to augment extinction by use of antagonists (see Supplementary On-line Materials).

Heavy use of context/cue conjunctive representations

Learned representations of context/cue conjunctions are crucial to the return of extinguished CS fear when FRAT is moved from contexts in which extinction was carried out (i.e., renewal). The removal of inhibition conditioned to the context alone can cause some renewal, but in real rats renewal of responding to a CS is at least partially dependent on removing the animal from the particular context in which that particular CS was extinguished (e.g., Harris et al., 2000). Related observations are what led Bouton to suggest that the CS-driven inhibition postulated to mediate extinction of a CS is “gated” by context (Bouton and Ricker, 1994), and it is what caused us to introduce context/cue conjunctive representations in FRAT. In FRAT these representations are established in hippocampus. But as pointed out below, it might also be possible to have them arise by convergence on BL cells of CS information from LA and context information from hippocampus.

Given other assumptions, the existence of context/cue conjunctive representations and the requirement that they develop within hippocampus have several implications, some of which have been partially verified: (1). If cue conditioning is weak, cued fear should be greater in the context of conditioning (where conditioned context/cue representations are present) than elsewhere (see Hall and Honey, 1990; Harris et al., 2000; and Figure 8). (2). After LA lesions, cued fear learning to context/CS conjunctive representations, which are effective in BL, should still be possible, but such cued fear should be manifest only in the context of conditioning (Table 4, M). (3). Post-training hippocampal ablation should cause some retrograde amnesia for weak cued fear if the there is a degree of overlap between training and test contexts (see Quinn et al., 2008b). (4). Manifestations of conditioning to context/cue compounds should require hippocampus-dependent pre-conditioning experience with the to-be-conditioned CS in the context where conditioning will occur (Table 4, R).

Unfinished business

Necessity of model properties

Our goal in constructing FRAT was to create an “automaton” that would emulate properties of biological fear conditioning in a biologically plausible way, but biological plausibility and parsimony sometimes conflicted. For example, LTP on inhibitory interneurons must not be allowed to go too far, or renewal will be liable to fail. Two mechanisms served to limit such LTP: (1) X-cells were made to have a sufficiently high threshold so that the neuromodulator required for inhibitory neuron synapse LTP is not released once CEm activity falls below a certain point (see Figure 5B₂). (2) LTP at inhibitor synapses cannot occur if “Ca” level produced in inhibitors by recurrent input from principal cells falls too low (see Figure 5C, right). It may well be that the first mechanism, which is probably the more important, is sufficient to ensure failures of renewal. But the second mechanism conforms to the biological reality that LTP and LTD commonly depend on postsynaptic Ca²⁺ levels. Of course there is undoubtedly considerable redundancy in biological systems. Nevertheless, one important piece of unfinished business is to determine how important individual aspects of the model are to successful simulation of various attributes of fear conditioning.

Establishment and consolidation of configural representations and their effects

There were several points during the construction of FRAT where problems were solved by fiat without creating neural circuitry that could accomplish the needed computations. Thus: (i) Hippocampal representations of configural cues were assumed to form at defined rates during stimulus exposure. (ii) A slightly slower process with similar outcome was assumed to lead to creation of cortical conjunctive representations if hippocampus was ablated, and the formation of these cortical representations was PFC-dependent. No mechanisms for either process were postulated, nor was any mechanism for involving PFC in the latter process suggested. Nor was any mechanism postulated for detecting the absence, as opposed to the silence of the hippocampus after its ablation. (iii) Configural cue representations that had been formed in hippocampus were assumed to gradually shift to representation cortex, and their projections to LA were adjusted so that they could functionally replace the hippocampal–BL connections they replaced (consolidation). Although a detailed algorithm for accomplishing this was postulated, no neural mechanisms for implementing the algorithm was offered. (iv) Utilization of cortical representations of configural cues was made somewhat PFC-dependent, but no theory was offered as to why this might be so.

There has been considerable thinking about the formation of hippocampal representations (Marr, 1971; Skaggs and McNaughton, 1992; O'Reilly and McClelland, 1994), and this could provide guidance in creating a model of that process, but, as far as we know, the other matters have not previously been considered at a theoretical or computational level. Ever since Marr (1971) first presented his theory of hippocampal function, it has been conjectured that consolidation requires neural activity. Although experimental evidence for a role of off-line activity associated with consolidation has been sought (Marr, 1971; Wilson and McNaughton, 1994; McClelland et al., 1995; Ji and Wilson, 2007), we know of no concrete algorithms using such activity that have been proposed. In FRAT we found it necessary to set the weights of consolidated pathway synapses so that the output of the amygdala circuit in response to input under various conditions would be the same before and after consolidation. Thus, in-effect we found it necessary to use an activity-dependent mechanism. Neurally plausible algorithms for accomplishing this are needed.

The representation of time

Some kinds of conditional responses are well-timed events, with the CR beginning slightly before the expected onset time of the US, and there have been theoretical attempts to deal with this, such as the concept of the serial compound stimulus (Sutton and Barto, 1990) used in discussions of primate VTA dopamine neuron responses (Schultz et al., 1997). Time has been less a focus in fear conditioning studies, and the representation of time plays no role in FRAT. In models where CS duration is explicitly coded, it is possible to condition expectation of the US at a particular time after CS onset, and the failure of the US to occur at the expected time during extinction can generate an error signal that helps trigger changes responsible for extinction. In FRAT, which lacks such a signal, incremental changes responsible for extinction simply occur in each time interval during which there is fear but no reinforcement. A consequence of this is that extinction rate is affected rather strongly by CS duration during extinction; short CSs cause much less extinction per trial than longer ones (not illustrated). There is some evidence that this prediction is empirically valid (Shipley, 1974).

Heterogeneous responses of BLA neurons

We made the initial simplifying assumption that LA and BL cells of FRAT are uniform populations of neurons, each innervated by all the afferents to the region via plastic synapses, and that learning and extinction of fear responses do not involve plasticity of synapses downstream of BLA. However, if there were such uniform innervation of LA and BL neurons, and if, as is likely, stimuli are coded by groups of neurons each of which also contributes to the representation of other stimuli, then after conditioning to numerous stimuli, many synapses of the circuit would be potentiated, and over-generalization to novel stimuli would be likely.

This problem can probably be greatly reduced by abandoning the assumption of uniform innervation and assuming that each BLA principal cell is innervated by different, though partially overlapping populations of afferents, or that competitive mechanisms of the kind recently discussed by Silva and colleagues (Zhou et al., 2009) force different BLA cells to mediate responsiveness to different stimuli. Under such circumstances downstream plasticity should help to make conditioning stimulus-specific (see, e.g., Brindley, 1969). It is in fact the case that plasticity downstream of BLA has been found to be important for both learning and extinction (Wilensky et al., 2006; Amano et al., 2010).

If the cells of BLA are heterogeneously innervated, then different cells of BL will be innervated by different combinations of LA neurons and by different contextual representation neuron from hippocampus. This alone should lead to at least some degree of context-specificity of extinction even without the use of context/CS conjunctive input from hippocampus. A related approach was taken in a recent computational model of BL (Vlachos et al., 2011) aimed at explaining the “extinction” neurons found there (Herry et al., 2008). Whether context/CS conjunctive representations that develop within hippocampus would in this case still be needed to produce realistic renewal behavior will depend on quantitative considerations that cannot be pre-judged. However, whether essential or not, it seems plausible that context/CS conjunctive representations should in fact form in hippocampus and should be among the representations to which fear becomes conditioned.

Unsimulated phenomena

Although FRAT simulates many known attributes of fear conditioning (Tables 1 and 4), there are others that the model as it now stands does not simulate. Some of these could be incorporated by small elaborations of features already present. For example, spontaneous recovery would automatically emerge if passage of time were made to alter context (Bouton, 1993). Other phenomena would require somewhat greater additions. Of particular importance in this regard are a number of effects that are thought to be due to shifts of attention to or from particular CSs as a result of past experience (McNally et al., 2004a; Iordanova et al., 2006; McNally and Westbrook, 2006). Such phenomena have motivated or supported major psychological theories of conditioning such as the Mackintosh or Pearce–Hall models (Mackintosh, 1975; Pearce and Hall, 1980). The addition of attentional mechanisms is thus an important topic for development.

Conclusion

It should be clear from the present paper that computational models of the kind employed here can help to make sense of findings that previously lacked explanation and can also be a rich source of untested hypotheses. It is also clear from the preceding discussion that there is considerable scope for future work.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.