Dopamine and extinction: A convergence of theory with fear and reward circuitry

Abstract

Research on dopamine lies at the intersection of sophisticated theoretical and neurobiological approaches to learning and memory. Dopamine has been shown to be critical for many processes that drive learning and memory, including motivation, prediction error, incentive salience, memory consolidation, and response output. Theories of dopamine’s function in these processes have, for the most part, been developed from behavioral approaches that examine learning mechanisms in reward-related tasks. A parallel and growing literature indicates that dopamine is involved in fear conditioning and extinction. These studies are consistent with long-standing ideas about appetitive-aversive interactions in learning theory and they speak to the general nature of cellular and molecular processes that underlie behavior. We review the behavioral and neurobiological literature showing a role for dopamine in fear conditioning and extinction. At a cellular level, we review dopamine signaling and receptor pharmacology, cellular and molecular events that follow dopamine receptor activation, and brain systems in which dopamine functions. At a behavioral level, we describe theories of learning and dopamine function that could describe the fundamental rules underlying how dopamine modulates different aspects of learning and memory processes.

1. Introduction

A key insight from behavioral approaches to learning is that multiple theoretical processes contribute to the acquisition and extinction of learned behaviors. Over the years, theories have described effects on processing of the conditioned stimulus (CS), the unconditioned stimulus (US), the context in which learning occurs, and the associative connections that form among stimuli, responses, and outcomes. Neurobiological studies have revealed that dopamine signaling is involved in many of the different theoretical processes that underlie learning. Recent experiments examining the role of dopamine in acquisition and extinction of learned fear demonstrate that learning in aversive situations may be modulated by an engagement of appetitive systems, supporting long-held assumptions about appetitive-aversive interactions in learned behavior (Konorski, 1967; Dickinson & Dearing, 1979; Dickinson & Pearce, 1977).

In this review, we focus on recent evidence demonstrating the different ways in which dopamine signaling may modulate learning during extinction of fear. We first review some of the ways in which dopamine has been treated at a theoretical level. Dopamine is involved in the circumstances that produce learning (e.g., prediction error, coding of stimulus salience); it mediates the content of that learning (e.g., hedonic value of associations); and it modulates the expression of learning in performance (e.g., response vigor, memory retrieval). We consider the complexity of dopamine signaling at receptor, molecular, and neural systems levels and suggest that some of the reward-related processes that rely on dopamine may drive the learning of extinction contingencies.

2. The many roles of dopamine in learning

A central tenet of the neurobiology of reward is that dopamine plays a key role in different processes modulating reward-seeking behaviors. Due to the complexity of signaling that arises from the projections to and from dopamine neurons, there are several different theoretical explanations for dopamine’s role in reward-seeking behaviors. These theories parallel learning theories that have focused on different aspects of Pavlovian and operant processes in behavior. In particular, theories have focused on the nature of reinforcement and how motivational state alters the impact of an outcome on learning processes (e.g., Dayan & Balleine, 2002), how the conditioned reinforcing value of previously neutral stimuli may come to control behavior (e.g., Belin et al., 2013), and how the discrepancy between the expected and obtained outcomes in a given situation drives learning (e.g., Steinberg et al., 2013)

Theories that focus on dopamine’s role in reinforcement have described mechanisms through which dopamine increases stimulus-response associations when those responses lead to rewarding experiences (Wise, 2004). A consequence of reinforcement is that stimuli that have been reinforced will generate both proximal motivated behaviors (approach or avoidance), as well as distal motivated behaviors (allocating resources for effortful approach or avoidance of a goal stimulus; Salamone & Correa, 2012). Ettenberg (2004) separates motivation from reinforcement by defining motivation as the process that initiates goal-seeking behavior, while reinforcement is the consequence of operant behaviors that alters the probability that a behavior will be repeated under similar conditions.

Theories that focus on the way that dopamine alters the conditioned value of previously neutral stimuli suggest that reward alters how organisms process environmental cues. The incentive salience theory describes mechanisms by which dopamine transmission in the nucleus accumbens assigns value to reward-related stimuli and dissociates hedonic value from behaviors directed toward goal stimuli (Berridge & Robinson, 2003; Flagel et al., 2010). The incentive salience theory is useful for understanding the maintenance of drug addiction, as tolerance develops to the rewarding effects of drugs but drug-seeking behavior becomes sensitized to drug-paired stimuli.

Cues associated with reward come to evoke responding and further learning to the extent that they signal unexpected outcomes. The prediction error model of dopamine function describes dopamine neuron activation linked to predictive models generated from associative learning events. Violations of predicted outcomes alter dopamine neuron firing, leading to alterations in motivated behaviors (Schultz & Dickinson, 2000). Triggering a dopamine response can induce learning under conditions, such as blocking, in which learning normally fails (e.g., Steinberg et al., 2013). These behavioral theories are supported by cellular and molecular evidence that dopamine receptor activation induces signaling cascades that are involved in the long-term consolidation of appetitive or aversive memories (Lauzon et al., 2013).

Although these theories have been useful for understanding how dopamine is involved in reward learning, behavioral theories for the role of dopamine in aversive learning remain poorly defined. Fear conditioning occurs rapidly, often after few pairings of a CS with a shock US. During extinction, the fear response is suppressed as the organism learns that the CS no longer predicts the shock. Extinction does not erase the original fear memory, but creates a new representation that allows the animal to adapt behavioral responses to previously conditioned stimuli (Rescorla, 2001). Several researchers have proposed mechanisms by which dopamine may modulate aversive learning (Redgrave et al., 1999; Horvitz, 2000; Salamone, 1994) and technological advances such as optogenetics or fast-scan cyclic voltammetry could clarify the exact contribution of dopamine in particular aspects of fear-related behaviors. Characterizing the role of dopamine in fear is particularly interesting because dopamine release within reward circuitry may alter the subjective value assigned to fearful stimuli, in addition to directly affecting memory consolidation (Horvitz, 2000; Pezze & Feldon, 2004). Thus, there are multiple mechanisms through which dopamine may alter the establishment, maintenance, expression, and extinction of fear. These include altering the conditions through which fear occurs (e.g., prediction error), altering the content of the association (e.g., attaching rewarding hedonic value to previously fearful stimuli), and altering the expression of the memory (e.g., changing the conditions under which conditioned fear occurs in behavior).

The varied roles of dopamine in appetitive and aversive learning is not surprising given the distribution of dopamine receptors in the central nervous system (Mansour & Watson, 1995), where dopamine is found throughout regions important for aversive and reward memory. This creates a complicated pattern of regional activity that could have different behavioral outcomes depending on the behavioral experience. In addition to different regional activation patterns causing different theoretical processes to be engaged, analyses of dopamine function in learning are complicated by the different intra-cellular signaling cascades that are triggered by receptor binding of dopamine. Just as the competition between excitation and inhibition at behavioral and regional levels can change the long-term behavioral consequence of a learning experience, so too can competition between excitatory and inhibitory intra-cellular signals.

3. Dopamine Receptor Signaling and Pharmacology

The three questions that have motivated much of the behavioral research on learning (Rescorla & Holland, 1976) are recapitulated at a cellular level when it comes to assessing dopamine function. What are the conditions that cause activation of different dopamine receptor subtypes? How are those conditions translated into molecular changes that lead to learning? How are those subtypes involved in expression of learning in behavior? Although the specific answers to these questions are elusive, a great deal is now known about dopamine receptor signaling and pharmacology and how they relate to learning.

A major complexity in understanding the role of dopamine in learning processes is that there are multiple dopamine receptor subtypes that activate different second messenger signaling cascades (Figure 1). Although differences among receptor subtypes are widely documented and are often described in terms of different learning processes, it is important to consider that that the same receptor subtype can have very different molecular effects, which may result in different behavioral endpoints. These effects may be excitatory or inhibitory and may occur through different second messengers.

Figure 1. Dopamine receptor signaling pathways.

Activation of Gα_s/olf proteins on D1-like receptors stimulates adenylate cyclase (Beaulieu & Gainetdinov, 2011). Adenylate cyclase induces production of cyclic adenosine monophosphate (cAMP), leading to activation of protein kinase A, (PKA). PKA generates cellular signaling cascades necessary for long-term plasticity. PKA also induces phosphorylation of dopamine- and cAMP-regulated neuronal phosphoprotein (DARPP-32), which inhibits protein phosphatase 1 (PP1). Inhibitory interactions between DARPP-32 and PP1 regulate neural plasticity through extracellular signal-regulated kinase (ERK) pathways (Valjent et al., 2004). Activation of Gα_i/o proteins on D2-like receptors inhibits adenylyl cyclase, regulating cAMP activity. The phospholipase C pathway can be induced by G_β and G_γ proteins from D2 receptors, G proteins from D1-like receptors, or G proteins from D1-D2 heteromers. These convergent pathways regulate the PLC-mediated cleavage of the phospholipid phosphatidylinositol 4,5-bisphosphate (PIP2) into inositol 1,4,5-trisphosphate (IP3) and diacyl glycerol (DAG). DAG activity regulates protein kinase C, while IP3 binding to IP3 receptors in the endoplasmic reticulum increases intracellular calcium (Ca²⁺) levels. Increased Ca²⁺ levels leads to activation of protein phosphatase 2B (PP2B) and Ca²⁺/calmodulin-dependent protein kinase II (CaMKII) signaling cascades that have been identified as regulators of long-term plasticity. Expression of these different dopamine receptor types varies across neuronal populations innervated by dopamine neurons.

One of the challenges in research on cellular effects of dopamine receptors is determining the relation between inhibitory and excitatory processes at molecular and behavioral levels. Dopamine signaling involves clear interactions between these processes, revealed in effects of activating the two main subfamilies of dopamine receptors. D1-like receptors, comprised of D1 and D5 dopamine receptors, activate the stimulatory G proteins Gα_s and Gα_olf. D2-like receptors, which include the D2, D3, and D4 receptors, activate the inhibitory G proteins Gα_i and Gα_o (Beaulieu & Gainetdinov, 2011). Generally, activation of stimulatory G proteins stimulates adenylate cyclase activity and cyclic adenosine monophosphate (cAMP) production, while activation of inhibitory G proteins decreases adenylate cyclase activity. D1 receptors in the nucleus accumbens and neostriatum primarily activate Gα_olf stimulatory G proteins while D1 receptors in hippocampus and cerebral cortex activate Gα_s stimulatory G proteins. The classical view of dopamine receptor activity has focused on intracellular signaling through adenylate cyclase and cAMP activity, both of which are widely recognized to play central roles in learning. For example, inhibitory interactions between dopamine- and cAMP-regulated neuronal phosphoprotein (DARPP-32) and protein phosphatase-1 (PP1) are important for intracellular regulation of neural plasticity (Gould & Manji, 2005) and DARPP-32 activity has been identified as a critical component in detecting convergent dopamine and glutamate signals leading to long-term synaptic plasticity (Valjent et al., 2005).

There is considerable evidence, however, that in addition to modulating cAMP activity, D1-like receptors operate through activation of phospholipase C (PLC). PLC activity has been implicated in the formation of fearful memories (Ouyang et al., 2012; Buckley & Caldwell, 2004). There are conflicting results about the mechanisms of the PLC response (Neve, 2004); in some studies the response is lost in D5 receptor null mutant mice and spared in D1 receptor null mutant mice (Friedman et al., 1997; Sahu et al., 2009), implicating the D5 subtype in this response, whereas other studies implicate coupling of a D1/D2 receptor heteromer to Gα_q in the response, with activation of Gα_q lost in D1 or D2 receptor null mutant mice (Rashid et al., 2007). A recent study characterizing signaling by D1 and D2 receptors co-expressed in human embryonic kidney (HEK) 293 cells supports regulation of PLC by the heteromer, but more strongly implicates Gβγ subunits than Gα_q in the response (Chun et al., 2013). Pharmacologically targeting D1/D2 heteromers may have beneficial effects in the treatment of several neuropsychiatric diseases (Hasbi et al., 2011), and could modulate fear acquisition or extinction through PLC signaling pathways.

D2-like receptors also activate signaling pathways distinct from Gαi_/o-mediated inhibition of adenylate cyclase (Neve, 2004). In particular, in addition to pertussis toxin-sensitive pathways mediated by Gαi_/o or by Gβγ subunits liberated by activation of those G proteins, the D2 receptor modulates glycogen synthase kinase (GSK) 3α/β via a pathway that involves arrestin and inhibition of the protein kinase Akt (Beaulieu et al., 2005; Beaulieu et al., 2004). Examining the specific pathways activated by these different dopamine receptors is necessary for understanding the differential ligand-induced activation or inhibition of particular signaling cascades within the brain that underlie the behavioral effects observed with dopamine receptor agonists.

Agonists of dopamine receptors are widely used to assess the function of those receptors in different learning processes. The modulation of different pathways, as described above, demonstrates some of the challenges in interpretation at a molecular level of systemically administered agents that target different learning processes. There are other complexities, including selectivity within or between the dopamine receptor subfamilies, activity at receptors other than dopamine receptors, and efficacy. With regard to efficacy, some agonists are broadly less efficacious than others; SKF 38393, for example, the first D1-like receptor-selective agonist, is a partial agonist for both adenylate cyclase and PLC signaling pathways (Brewster et al., 1990; Mannoury la Cour et al., 2007), and is therefore likely to produce less robust behavioral responses than a full agonist. The existence of multiple signaling pathways for one receptor creates the possibility for another way in which efficacy can differ among agonists, often referred to as functional selectivity (Urban et al., 2007). A ligand may be a full agonist at one signaling pathway and a partial agonist or even an antagonist at another signaling pathway regulated by the same receptor (i.e., a biased agonist; Allen et al., 2011). For example, considerable data support the classification of the D1 receptor ligand SKF 83959 as a very weak partial agonist or antagonist for stimulation of adenylate cyclase but a full agonist for stimulation of PLC (Arnt et al., 1992; Jin et al., 2003) and SKF 83822 as a full-efficacy agonist for adenylate cyclase with little or no efficacy for PLC (Rashid et al., 2007; Undie et al., 1994). Further evidence that SKF 83959 and SKF 83822 are reciprocal biased agonists was provided by the report that SKF 83959 is capable of activating the D1/D2 heteromer to stimulate PLC, whereas SKF 83822 is inactive at the heteromer but activates the D1 receptor homomer to stimulate adenylate cyclase (Rashid et al., 2007). On the other hand, a recent study reported that SKF 83959 lacked agonist activity at the D1/D2 heteromer expressed in HEK 293 cells and that SKF 83822 was a partial agonist (Chun et al., 2013); for this reason and because of the likelihood that these drugs also have effects at the D5 receptor and perhaps other receptor subtypes, one must be cautious when interpreting results of behavioral studies with these ligands. The use of agonists with known bias may help to elucidate the specific intracellular signaling pathways involved in particular phases of fear memory formation or extinction.

Just as dopamine can alter learning through a variety of actions on different behavioral processes, it also can alter learning through a variety of actions on different molecular processes. This is important to keep in mind when thinking about dopamine, fear, and fear extinction, because it illustrates that the complexities and caveats that are needed when evaluating the function of dopamine in learning apply not just to behavioral processes, but also to cellular and intra-cellular mechanisms.

4. Dopamine signaling and fear

As described above, there are multiple pathways through which dopamine may alter signal transduction and plasticity. Much of the work on dopamine and fear has focused on the potential roles of D1 and D2 receptors in acquisition and extinction. Because of the excitatory and inhibitory effects of D1-like and D2-like receptors on cAMP, one might think that D1 receptors would be involved in fear acquisition and D2 receptors would be involved in fear extinction. This one-to-one correspondence between excitatory or inhibitory effects on cAMP does not capture the results of genetic and pharmacological approaches to studying these receptors in modulating fear.

There is evidence that the D1 receptor is involved in fear acquisition and extinction. One hypothesis for dopamine's role in acquisition is that dopamine neuron activation induces a counterconditioning or safety signal effect during aversive learning. This hypothesis would predict that blocking D1 receptors during fear acquisition would increase or maintain fear responding, based on observations that a rebound of dopamine release follows the offset of an aversive stimulus (Brischoux et al., 2009). However, D1 receptor antagonist and knockout studies have demonstrated impairments in acquisition and extinction of fear following loss of D1 receptor signaling, suggesting that dopamine signaling at the D1 receptor directly contributes to the acquisition of fear (El-Ghundi et al., 2001; Fadok et al., 2009; Greba & Kokkinidis, 2000; Inoue et al., 2000). These effects may be mediated by activation of dopamine neurons through NMDA receptors (Zweifel et al., 2009; Zweifel et al., 2011) and they are time-limited, with post-session injections of a D1 receptor antagonist 30-min following fear conditioning having no effect (Inoue et al., 2000). Together, these experiments reveal a critical role for D1/5 receptors in the acquisition of contextual fear.

In contrast, studies of fear with D2 receptor antagonists have had mixed effects. D2 receptor-lacking mice show normal fear-potentiated startle (Fadok et al., 2009) although D3 or D4 receptors may compensate for the loss of D2 receptors to support fear acquisition. Mueller et al. (2010) and Holtzmann-Assif et al. (2010) showed that a D2 receptor antagonist impairs fear extinction retention, but Ponnusamy et al. (2005) showed fear extinction enhancements with D2 receptor antagonism. The discrepancies between these studies are more thoroughly discussed by Mueller et al. (2010), but one contributing factor could be varying receptor affinity between different D2 antagonists, or differing CS presentation protocols used in the experiments. In spite of some mixed effects, it is clear that dopamine plays a modulatory role in aspects of fear acquisition and extinction. Although antagonist and genetic knockout studies suggest that dopamine neuron activation would likely enhance fear extinction, the effects are inconsistent due to the variety of behavioral and pharmacological manipulations that have been used.

In general, pre-session injections of any substance that alters dopamine release or dopamine binding are difficult to evaluate because of the activating effects that these agents have on locomotor activity. Post-session injections are not without their own complications (Cunningham et al., 1998), but they do allow the animal to learn the extinction contingencies without being affected by activating effects of the drugs. Differences in pre- and post-session effects are evident in studies that have examined the effects of psychostimulants on extinction. Methamphetamine administered prior to an extinction session impairs extinction of conditioned fear (Miczek & Luttinger, 1978; but see Carmack et al., 2010 and Mueller et al., 2009), and in agreement with these findings, Borowski and Kokkinidis (1998) demonstrated that a pre-extinction administration of cocaine, amphetamine or a dopamine D1/5 partial agonist impairs extinction of potentiated startle. However, other studies have found that methylphenidate (Ritalin; a dopamine and norepinephrine transport blocker; Abraham, et al., 2012) or L-dopa (a dopamine precursor; Haaker et al., 2013) administered after a fear extinction session enhances the retention of fear extinction.

In addition to differences between pre- and post-session drug administration, effects of dopamine on behavior will interact with brain systems and behavioral processes that are mediated by those systems. As can be seen in Figure 2, dopamine innervates brain regions that are critical for different aspects of learning. Dopaminergic innervation in the brain can be divided into four main pathways (Beaulieu & Gainetdinov, 2011). Generally, each of these projection pathways is regionally and functionally distinct, as dopamine plays different roles in guiding behavior in dopamine terminal regions. The mesocortical pathway, emanating from the ventral tegmental area, connects dopamine neurons to cortical regions. The mesolimbic pathway connects the ventral tegmental area to the nucleus accumbens, amygdala, and hippocampus. The mesocorticolimbic pathway has been well characterized as the region that guides associative learning in both instrumental and Pavlovian tasks. The nigrostriatal pathway connects the substantia nigra to the striatum and is important for guiding motivated motor responses. The tuberoinfundibular pathway connects dopamine neurons in the hypothalamus to the pituitary gland and induces hormone release (Reymond & Porter, 1985). Many of the regions innervated by the ventral tegmental area and substantia nigra have been implicated in aspects of fear learning, and suggest a critical role for dopamine in fear learning and extinction (Pezze & Feldon, 2004).

Figure 2. Dopamine circuitry in fear-related behaviors.

Dopamine neurons in the ventral tegmental area (VTA) generate signals to encode discrepancy between expected and unexpected outcomes (prediction error). These signals are relayed to several regions that have reciprocal connections with the VTA, such as the amygdala, nucleus accumbens, prefrontal cortex, and hippocampus. These reciprocal connections allow for modification of signals arising from the VTA, leading to precise control of dopamine release within dopamine terminal regions. Within the amygdala, the basolateral amygdala (BLA) encodes CS-US associations, allowing for fear acquisition and retrieval. The BLA projects to the central amygdala (CeA) to generate fear responses and motivated behavior through motor circuitry. Dopamine receptor activity in the amygdala modulates the formation and retrieval of fear associations. Dopamine transmission in the nucleus accumbens core (NAcc Core) is important for encoding the general salience of environmental stimuli, and activity in the nucleus accumbens shell (Nacc Shell) encodes outcome-specific predictions to guide motivated behavior. In the prefrontal cortex, dopamine activity is important for working memory and fear extinction. In the infralimbic region of the prefrontal cortex (IL), dopamine D1 receptors are required for consolidation of fear extinction (Hikind & Maroun, 2008). Activation of D1 receptors in the prelimbic region of the prefrontal cortex (PL) blocks the expression of conditioned fear (Lauzon et al., 2013). Interactions between the hippocampus and VTA are important for relaying contextual information in fear and reward learning. The VTA provides a coordinating signal to generate particular patterns of activity in dopamine terminal regions based on environmental stimuli and prior experience. Dopamine neurons in the substantia nigra are involved in motor responses and project to the dorsal striatum, which selects and alters appropriate behavioral responses based on inputs from a variety of different regions. Together, activity in the substantia nigra, ventral tegmental area, and dopamine terminal regions allow for the generation of appropriate behavioral responses (e.g. freezing, approach, or active avoidance) in response to shifting external stimuli.

5. Prediction Error and the Ventral Tegmental Area

The predominant neurobehavioral theory for dopamine neuron activity in the ventral tegmental area is a prediction error model (Schultz, 2006). Prediction error is a discrepancy between expected and actual outcomes and this discrepancy is a fundamental element of various models of associative learning (e.g., Mackintosh, 1974; Rescorla & Wagner, 1972). Dopamine neurons fire tonically under baseline conditions, but increase burst firing when a reward occurs that is greater than what is predicted or inhibit tonic firing when an expected reward is omitted. Over several pairings of a CS with a rewarding US, dopamine neurons begin to respond primarily to the CS. This suggests that instead of only encoding the hedonic value of a US, dopamine neuron firing signals expectations corresponding to previously neutral CSs. The ability to convey these predictions supports the hypothesis that dopamine conveys information to multiple target regions that are involved in different aspects of memory formation.

Theories to explain the nature of information encoded by the ventral tegmental area (VTA) have been restructured by observations that subpopulations of neurons in the VTA react differently to rewarding or aversive experiences. Dopamine neurons in the dorsal VTA are excited by reward or reward-associated stimuli, whereas dopamine neurons in the ventral VTA show high levels of bursting in response to footshock (Brischoux et al., 2009). The high levels of bursting could be important for the encoding of aversive stimuli and stimuli associated with the aversive experience, or it could signal the appropriate behavioral responses to variable contingencies. This suggests that the prediction error model may be applicable for both aversive and appetitive tasks, and the projections of dopamine neurons from the VTA provide a mechanism by which the subjective value of aversive or appetitive information is distinguished in dopamine terminal regions.

Lammel et al. (2011) demonstrated that dopamine neurons in the posterior ventromedial portion of the VTA that projects to the prefrontal cortex respond to aversive stimuli by significant increases in the AMPA receptor (AMPAR) mediated to NMDA receptor (NMDAR) mediated excitatory postsynaptic current (EPSC) ratio seen at the synapse between VTA and PFC. An injection of formalin into the hindpaw of a mouse (an aversive experience), leads to increases in the AMPAR/NMDAR EPSC ratio in neurons that project to the prefrontal cortex and the lateral shell of the nucleus accumbens. In contrast to neurons excited by aversive events, dopamine neurons that are inhibited by aversive events also encode the level of conditioned fear, as the duration of inhibitory pauses of tonic dopamine firing is related to the acute fear evoked by a CS that has been previously paired with shock (Mileykovskiy & Morales, 2011).

The offset of an aversive stimulus (Brischoux et al., 2009) has been shown to increase firing in particular dopamine neurons, suggesting that the lack of an expected aversive event may trigger dopamine release. Although this finding has not been thoroughly characterized, it has important implications for suggesting that the absence of a predicted aversive event may lead to increased firing in a subset of VTA neurons that have yet to be identified as belonging within the aversive/rewarding continuum. This observation suggests a mechanism by which the VTA encodes the prediction error present during an extinction session. Whereas appetitive stimuli lead to increased dorsal VTA dopamine neuron firing and the absence of a reward leads to decreased dopamine neuron firing, an aversive stimulus may reduce firing in the dorsal VTA and the lack of an aversive US may lead to the increased activity of these dopamine neurons. These effects may also be reversed in the ventral VTA, with increased activity to an aversive stimulus and reduced firing in response to the absence of an aversive stimulus. The activity of dopamine neurons in the VTA in response to the absence and presence of predicted appetitive and aversive events provides a neurobiological basis for theoretical models of aversive-appetitive interactions proposed by Konorski (1967) and Dickinson (Dickinson & Pearce, 1977; Dickinson & Dearing, 1979).

Following extinction, the presentation of the previously fearful CS no longer induces dopamine neuron inhibition (Mileykovskiy & Morales, 2011). This finding suggests that although VTA neurons encode acquisition of conditioned fear and can modulate the current value of the CS, they may not be involved in the long-term storage of a Pavlovian fear association (see de Oliveira et al., 2011). This idea is consistent with Lammel et al.’s (2011) finding that changes in the AMPAR/NMDAR EPSC ratio related to aversive stimuli in dopamine neurons that project to the prefrontal cortex were no longer increased after 10 days (but see Corral-Frias et al., 2012). In combination, these studies suggest that the initial signaling of aversive experiences requires coordinated action in subpopulations of VTA neurons, but other brain regions may maintain the long-term storage of these experiences.

The VTA shows neuronal firing in agreement with a prediction error hypothesis, but outcome expectancy involves the coordinated activity of different brain regions that mediate different aspects of learning. As a modulator of contextual information, the hippocampus may have an important role in facilitating VTA activity to discern novel stimuli or contexts from previously experienced events. Inactivation of the ventral hippocampus prevents increased firing in VTA neurons that are activated by footshock (Valenti et al., 2011). At the level of associative content, the hedonic value of an event (aversive or appetitive) may originate in the lateral habenula (Jhou et al., 2009). Activation of the lateral habenula promotes behavioral avoidance (Stamatakis & Stuber, 2012) and signals aversive events to the medial VTA through the rostromedial tegmental nucleus (RMTg), while the laterodorsal tegmentum (LDT) signals rewarding events to the lateral ventral tegmental area (Lammel et al., 2012). At the level of motivation and performance, the actions of dopamine in the substantia nigra are likely to be involved in the propagation of or inhibition of motor activities to approach or avoid a target stimulus, rather than to encoding the affective value or strength of an association (Bromberg-Martin et al., 2010).

In summary, these findings show that VTA dopamine neurons are involved in encoding a variety of signals, both rewarding and aversive. An important implication of these studies is related to the recent advance of using optogenetic techniques in exploring the role of dopamine neuron activation in examining aversive or rewarding events (Stuber et al., 2012). The use of optogenetics allows for fine-tuned control of dopamine neuron activity, and with careful consideration of the projections from dopamine neurons, optogenetic techniques may dissociate the neuronal populations involved in fear or reward learning and extinction. One projection site of the VTA that may be of interest for understanding fear extinction is the amygdala, as it encodes both aversive and rewarding events, and has been well characterized in and extinction of fear in many behavioral studies.

6. Associative Learning in the Amygdala

The amygdala is a key site for inducing, maintaining, and expressing associative learning. Neuronal activity in the amygdala can be induced by rewarding or aversive events, leading to plasticity changes there and elsewhere that are thought to underlie long-term memory (McGaugh, 2004). As with the cellular and molecular effects of dopamine, it is impossible to talk about the general role of the amygdala in learning. It is involved in acquisition, consolidation, and expression of learning; what aspect of the learning process gets encoded depends on where in the amygdala one looks. A review of all of the substructures and their roles in memory is beyond the scope of this paper (see Ehrlich et al., 2009). We focus on three general regions – the basolateral amygdala, which integrates sensory information about CSs and USs; the intercalated cell masses, which inhibit or excite activity in the BLA and central amygdala; and the central nucleus of the amygdala, which projects to output regions that translate these signals into behavioral responses to aversive or appetitive stimuli (Moustafa et al., 2013).

There are several models for how the amygdala modulates learning. Research on aversive learning has generally followed a serial processing model for the amygdala, with associative information being integrated in the basolateral amygdala and sent to the central amygdala and downstream targets. Other models suggest parallel processing, in which the different nuclei process different aspects of the association (Balleine & Killcross, 2006; Holland & Gallagher, 1999; Corbit & Balleine, 2005). In the context of aversive learning, the basolateral amygdala integrates particular sensory stimuli with nociceptive unconditioned stimuli and the central amygdala encodes general affective or attentional states that can alter motivated behavior. Acting together, these regions can generate appropriate fear responses based on specific sensory stimuli and general affective states. The wide range of associations that can be generated through increased activity in the amygdala indicates that examination of both terminal region and network activity from dopamine neurons is important to distinguish how appropriate and distinct behavioral responses are generated from common signaling mechanisms in appetitive and aversive situations.

The localization of dopamine receptors in the amygdala has important functional implications for the differences in information encoded by the basolateral and central amygdala (Figure 3). The basolateral amygdala expresses D1 receptors, while the central amygdala primarily expresses D2 receptors (Weiner et al., 1991). The intercalated cell mass, a group of cells associated with extinction (Busti et al., 2011) expresses D1 receptors highly (Mansour & Watson, 1995). It is possible that amygdalar dopamine receptors are involved in consolidation of fear conditioning and extinction, supported by findings that long term potentiation in the amygdala can occur from the coordinated activation of D1 receptors with TrkB receptors (Li et al., 2011). Based on the distribution of dopamine receptors, it would be tempting to speculate that D1 receptors in the amygdala maintain fear extinction while D2 receptors maintain fear elicitation. However, the complexity of D1 receptor distribution and transmission within the intercalated cell masses (Fuxe et al., 2003) in addition to agonist and antagonist studies suggest that D1 and D2 receptors are likely to be involved in both fear acquisition and extinction acquisition.

Figure 3. Distribution of dopamine receptors in the amygdala.

Neurons in the central amygdala primarily express dopamine D2 receptors (Weiner et al., 1991). Blockade of these receptors may lead to impaired fear conditioning (Guaracci et al., 2000) or prevent the animal from generating appropriate fear or reward-related behaviors. The central amygdala receives glutamatergic input (open triangles) from the basolateral amygdala (cell body represented by filled circles). The basolateral amygdala (BLA) coordinates activity from the nucleus accumbens, prefrontal cortex, and many other regions to generate and retrieve CS-US associations. Dopamine D1 receptors are primarily expressed in the BLA and intercalated cell masses of the amygdala (ITC). Antagonism of D1 receptors in the basolateral amygdala impairs acquisition of fear extinction (Hikind & Maroun, 2008). Glutamatergic neurons from the BLA project to ITC and GABAergic ITC neurons inhibit central amygdala activity (output represented by open squares). The activation of GABAergic ITC neurons mediated by the basolateral amygdala and the infralimbic region of the prefrontal cortex is critical to extinction of fear behaviors and inhibitory control of the central amygdala (Busti et al., 2011). In addition to high D1 receptor expression, ITC express D2 receptors at low levels (Weiner et al., 1991). In summary, the release of dopamine in the amygdala can have complex effects on behavior due to the distribution of dopamine receptors and the dopamine receptor subfamilies expressed in this region.

Pharmacological manipulations of the D1 receptor in the amygdala generally have been consistent with the idea that the D1 receptor is involved in aversive learning. Intra-amygdalar D1 agonism enhances fear conditioning and intra-amygdalar D1 antagonism impairs both first-and second-order fear conditioning (Guarraci et al., 1999; Lamont & Kokkinidis, 1998; Nader & LeDoux, 1999). Further, antagonism of the D1 receptor in the BLA immediately before, but not immediately after, an extinction session slows the rate of extinction over days (Hikind and Maroun, 2008), consistent with the idea that acquisition, but not consolidation of extinction depends on the D1 receptor in the BLA. A report that amygdalar D1 receptors are not linked to adenylate cyclase (Leonard et al., 2003) suggests that the PLC-dependent pathway (Figure 1) may be critical for D1 receptor-mediated changes in fear. The role of amygdalar D2 receptors in fear conditioning is less well characterized. In the amygdala, flupenthixol (a D1 & D2 receptor antagonist) blocks fear conditioning (LaLuimiere et al., 2004), and Guarraci et al. (2000) demonstrated that D2 receptor antagonism impairs fear conditioning and retrieval.

These studies suggest that D1 and D2 receptor activation in the BLA is necessary for the initial acquisition and extinction of fear. This conclusion is complicated by findings that mice lacking D1 receptors can acquire conditioned fear (El-Ghundi et al., 2001) and mice lacking D2 receptors can acquire fear-potentiated startle (Fadok et al., 2009). It is possible that compensatory mechanisms between dopamine receptor subtypes may support fear learning, or other neurotransmitter pathways may provide sufficient signaling to compensate for the loss of dopamine signaling, as suggested by redundant PLC signaling pathways activated by dopamine and noradrenergic receptors (Ouyang et al., 2012). Although the mix of findings creates interpretational difficulties for the role of amygdalar dopamine in fear conditioning, the acquisition of fear extinction appears to require amygdalar dopamine receptors. The interaction of the amygdala with other targets of dopamine innervation, such as the prefrontal cortex or the nucleus accumbens may reveal the underlying principles of dopamine-related mechanisms guiding fear extinction.

7. Salience and the Nucleus Accumbens

Dopamine signaling in the amygdala is important for acquisition and extinction of fear, but those signals need to be integrated with responses in other systems to result in memory. Although the nucleus accumbens is generally thought of as a region that mediates reward-related processes, there is growing evidence for its involvement in aversive memory regulation. Studies examining the combined action of the nucleus accumbens and the basolateral amygdala, for example, suggest that concurrent activation of dopamine signals in the nucleus accumbens and basolateral amygdala contributes to long-term memory formation (LaLumiere et al., 2005). This indicates that the nucleus accumbens and amygdala could be regions that integrate dopamine signaling to generate appropriate behavioral responses to new contingencies. In an inhibitory avoidance task, modulation of consolidation is directly influenced by activation of dopamine in both regions (LaLumiere et al., 2005). Similarly, restoration of dopamine in both the basolateral amygdala and nucleus accumbens in dopamine deficient mice is sufficient for long term memory of fear conditioning (Fadok et al. 2010). The effect is possibly mediated through glutamatergic projections in the nucleus accumbens shell (Floresco et al., 1998), and the interaction of glutamatergic signaling with dopamine in the nucleus accumbens may guide behaviors associated with aversive or appetitive events. In the presence of an AMPA receptor antagonist in the medial shell, blockade of D1 and D2 receptors disrupts fear behaviors (Richard & Berridge, 2011). The findings from Richard and Berridge (2011) illustrate the complexity of dopamine signaling and imply a balance between D1 and D2 receptors required for fear behaviors in the rodent. The balance of activation between the receptors is altered by environmental stress (Reynolds & Berridge, 2008) and prolonged stress can dysregulate the control that corticotropin-releasing factor exerts over dopamine release in the nucleus accumbens (Lemos et al., 2012). Although these studies indicate that dopamine release increases in portions of the nucleus accumbens in response to aversive conditions, McCutcheon et al. (2012) suggest that some findings of increased dopamine activity in the nucleus accumbens following aversive stimuli may be limited by technology that does not allow fine grain temporal analysis of dopamine alterations linked to stimulus presentation. However, fast-scan cyclic voltammetry studies have provided evidence that primary aversive stimuli increase dopamine release in the nucleus accumbens core and shell (Budygin et al. 2012).

Microinjection of the non-selective dopamine receptor antagonist haloperidol into the nucleus accumbens impairs consolidation and retention of fear extinction (Holtzmann-Assif et al., 2010). Pezze et al. (2001) suggested that the nucleus accumbens core and shell encode different aspects of fear conditioning by separately examining the effects of contextual and auditory conditioning. The authors demonstrate that during non-reinforced presentations of tone, there is increased extracellular dopamine in the nucleus accumbens shell and non-reinforced presentation of context leads to increased extracellular dopamine in the core. Bradfield and McNally (2010) expanded on these findings by showing that accumbens shell lesions impair learning about less predictive stimuli in a compound conditioning procedure, consistent with the idea that the nucleus accumbens shell is involved in gating learning that occurs with relation to stimuli that vary in predictive value for an expected aversive outcome.

These findings correspond with the incentive salience model (Berridge & Robinson, 2003) of dopamine function in the nucleus accumbens shell, as the stimuli that are least important to the aversive outcome are ignored when the nucleus accumbens shell is inactivated. Although the incentive salience theory has not thoroughly addressed aversive conditioning, it is possible that cues and contexts associated with aversive events acquire salience that leads to the animal avoiding or freezing in response to conditioned stimuli, and this salience may be encoded by dopamine receptors in the nucleus accumbens shell. When the shell is inactivated, the animal cannot assign negative value to poor predictors of an aversive unconditioned stimulus, so behavioral responses shift towards the most predictive stimuli.

Another way of thinking about the nucleus accumbens shell is that it is needed to generate outcome-specific responses to particular stimuli (i.e. individual prediction error computations). When it is lesioned, animals must rely on the nucleus accumbens core for general predictions about aversive cues or contexts to generate fear behaviors. Corbit and Balleine (2011) demonstrated the dissociation between the nucleus accumbens core and shell in a Pavlovian-instrumental transfer task, identifying the nucleus accumbens core as controlling general excitatory effects associated with reward seeking and the nucleus accumbens shell as generating outcome-specific reward predictions. Although this dissociation has not been explicitly demonstrated in aversive learning, similar processes are likely to occur in fear acquisition or extinction. Dopamine signaling alterations in the nucleus accumbens shell may be limited to certain aspects of aversive procedures, as some studies have shown no alterations of shell activity during fear conditioning (Levita et al., 2002). Levita et al. proposed that the variety of effects seen with dopamine manipulations may be due to differences between instrumental and Pavlovian procedures, because these conditioning paradigms do not produce similar behaviors and are unlikely to operate through identical neurobiological mechanisms.

The interactions of the nucleus accumbens with the hippocampus and the amygdala are critical for allowing the generation of appropriate behaviors in both fear and reward. Gill and Grace (2011) showed that activation of D2 receptors in the nucleus accumbens is required for the nucleus accumbens responsivity to the basolateral amygdala with a permissive role played by the ventral subiculum of the hippocampus. Gill and Grace offer a theoretical model of amygdala and hippocampal control of fear and appetitive behaviors, whereby theta burst stimulation of the basolateral amygdala and increased activity in the ventral subiculum of the hippocampus leads to activation of the rostral nucleus accumbens, tilting behavior towards fear and avoidance behaviors. Activation of the caudal nucleus accumbens leads to increased appetitive behaviors, suggesting that the integration of information in the nucleus accumbens can lead to altering behavioral output of the animal. Although these findings provide a framework by which the nucleus accumbens can mediate fear conditioning, fear extinction also requires nucleus accumbens activity. A recent study by Rodriguez-Romaguera et al. (2012) indicates that deep brain stimulation of the dorsal region of the ventral striatum, which includes portions of the nucleus accumbens, enhances fear extinction in rodents. Additionally, aversive memory retrieval and extinction decreases dopamine release in the nucleus accumbens core and increases dopamine transmission in the nucleus accumbens shell, with these alterations temporally locked with the offset of an aversive stimulus (Badrinarayan et al., 2012).

While the composition of neurons in the dorsal and ventral striatum appears very similar, with approximately 95 % of the neurons in these regions being medium spiny GABA-ergic projection neurons (Hyman and Malenka, 2001), dividing the whole striatum into dorsolateral and ventromedial portions allows identification of two separate, but overlapping pathways. The dorsolateral striatum primarily receives and modifies sensorimotor information, while the ventromedial striatum (including nucleus accumbens) receives input from the amygdala and other visceral afferents (Voorn et al., 2004). Although the whole striatum has been implicated in conditioned avoidance tasks (Darvas et al., 2011), the function of specific dopamine receptors in the dorsal striatum in Pavlovian fear conditioning or extinction remains unknown. Amphetamine in the dorsal striatum increases freezing in response to either a conditioned tone or context and suggests that interactions between the dorsal striatum and amygdala may maintain conditioned fear behavior (White & Salinas, 2003). The striatonigral (direct/ D1 receptor dense) and striatopallidal (indirect/ D2 receptor dense) pathways may have functions in learning and modifying appropriate motor responses to conditioned stimuli in a reward task (Yawata et al., 2012). The direct/D1 dense pathway is important for initial acquisition of a reward task, whereas the indirect/D2 dense pathway is important for learning flexibility (Yawata et al., 2012). This study suggests that the dorsal striatum may play an important role in modifying the motor responses that occur in response to a fearful stimulus, although the role of each receptor family (D1 and D2) in fear extinction has not yet been examined. Interactions between the dorsal and ventral striatum as well as other brain regions (e.g. prefrontal cortex, hippocampus) may allow for modifications of behavior following associative learning.

Overall, the role of the nucleus accumbens and dorsal striatum in fear extinction remains open for study, but evidence from reward literature suggests that the nucleus accumbens may be involved in encoding salience related to an aversive event, while dorsal striatum is important for motor behavior related to associative learning. Activity in the nucleus accumbens could allow for the assignment of an affective value to stimuli based on the distribution (D1-like vs. D2-like) and location (core vs. shell) of receptors that are activated by a burst of dopamine.

8. Contextual Learning and the Hippocampus

One of the most theoretically important findings in the behavioral study of extinction is that the learning that occurs during extinction is context specific. Changes in context between extinction and testing often lead to a renewal of conditioned fear (e.g., Bouton & Bolles, 1979) and there are many suggestions that this effect is mediated by the hippocampus (e.g., Corocoran & Maren, 2001). Because the hippocampus is thought to be important for developing representations of space and context, a disruption of hippocampal function at the time of extinction or testing may disrupt the development or retrieval of contextual associations with extinction.

Most of what is known about the role of hippocampal D1 receptors and fear comes from studies of acquisition. D1 receptor antagonists or agonists in the dorsal hippocampus impair or promote, respectively, initial inhibitory avoidance learning and long-term memory (Rossato et al., 2009). Interestingly, administration of a dopamine receptor agonist into the dorsal hippocampus within 12 hours of training enhances subsequent fear expression, suggesting a role for dopamine in a long-term consolidation mechanism that has not yet been thoroughly described (Rossato et al., 2009). Loss of hippocampal D1 receptors impairs trace eyeblink conditioning, similar to impairments observed in D1 knockout mice (Ortiz et al., 2010). Some evidence suggests that D1 receptors in the hippocampus contribute to long term potentiation, as shown by Gao et al. (2006) where D1 receptor stimulation in cultured hippocampal neurons increased AMPA receptor surface expression through a PKA-dependent pathway. These studies indicate that dopamine stimulation in the hippocampus may be important for consolidation of initial fear conditioning into long-term memory.

There is some evidence that D1 receptors in the hippocampus are involved in extinction. Fiorenza et al. (2012) showed that post-session intrahippocampal injection of a D1 receptor partial agonist can enhance extinction of contextual fear. Studies examining the interaction between the ventral tegmental area and the hippocampus have shown that the hippocampus is involved in integrating reward with contextual information through the lateral septum (Luo et al., 2011), so a similar mechanism may mediate contextual information in fear conditioning and extiction that is relayed to the ventral tegmental area. How this interaction may result in contextual renewal after fear extinction is not yet known.

9. Inhibitory Learning in the Prefrontal Cortex

Whereas the hippocampus is thought of as a modulator of contextual information that is encoded during extinction, the prefrontal cortex (PFC) is thought to be more directly involved in the inhibitory learning that may develop during extinction. Fear extinction could be mediated by dopamine activity, as the prefrontal cortex expresses D1 receptors in nonpyramidal neurons and D2 receptors in small pyramidal cells and large nonpyramidal neurons (Vincent et al. 1995) and cyclic AMP activity in the prefrontal cortex is important for expression of aversive and reward-related memories (Lauzon et al., 2013). Many studies have found that subregions of the prefrontal cortex are important for the consolidation and retrieval of fear extinction (Quirk & Mueller, 2008) and increased PTSD severity is correlated with decreased prefrontocortical activity (Shin et al. 2006). Patients with PTSD are able to acquire fear extinction normally in a laboratory setting, but recall of fear extinction is impaired (Milad et al., 2009). Thus, strengthening consolidation and retention of extinction in the PFC through pharmacological intervention may be sufficient for lessening PTSD symptoms. One method of strengthening the consolidation of extinction could be through increasing dopamine signaling in the PFC to activate second messenger pathways that lead to memory consolidation. Methylphenidate, a dopamine and norepinephrine transporter blocker, preferentially increases profrontocortical activity (Berridge et al., 2006), enhances the consolidation of fear extinction in C57BL/6 mice (Abraham et al., 2012), and converging evidence from human studies has shown that methylphenidate may alleviate some PTSD symptoms (Houlihan, 2011). These findings are supported by studies showing PFC dopamine is increased following extinction (Hugues et al., 2007) and PFC dopamine depletion decreases retention of fear extinction (Morrow et al., 1999, Fernandez-Espejo, 2003). The particular contribution of each dopamine receptor in the PFC has been more thoroughly explored through antagonist studies.

D1 antagonist studies show that the prefrontal cortical D1 receptors in the infralimbic region are critical to consolidation of fear extinction (Hikind & Maroun, 2008). When given following an extinction session, antagonist-treated animals maintained freezing during the following test day compared to vehicle-treated animals, demonstrating that infralimbic D1 receptors are required for consolidation of an extinction memory. In support of this finding, electrophysiological studies show that the D1 receptor is involved in the maintenance, but not induction of long-term potentiation (Huang et al., 2004). Evidence for the involvement of other dopamine subtypes comes from Mueller et al. (2010), who tested the role of PFC D2 receptors in fear extinction. Mueller et al. (2010) showed that a pre-extinction infralimbic administration of a D2 receptor antagonist led to no difference in acquisition of fear extinction, but increased fear on the following test day. These studies indicate the involvement of D1 and D2 receptors in fear extinction consolidation, although some behavioral effects may also be related to retrieval of the fear memory.

Prelimbic and dorsal infralimbic PFC D1 and D2 receptors are important for retrieval of a conditioned fear memory, as nonspecific blockade with cis-flupenthixol or activation with amphetamine led to decreased freezing during an extinction session (Pezze et al. 2003). Amphetamine treatment within the PFC during extinction led to decreased freezing on following test days, suggesting that activation of dopamine and noradrenergic receptors in the PFC may lead to enhanced consolidation of extinction. However, without a post-extinction manipulation, it is difficult to differentiate effects on retrieval of the original memory or encoding of the extinction memory from consolidation of the extinction memory. Some effects observed in D1 and D2 receptor antagonist studies may be related to the distribution of dopamine receptors within the PFC, as some PFC neurons express both D1 and D2 receptors, while some express only D1 or D2 (Vincent et al., 1995). Approximately 25% of neurons in the PFC with dopamine receptors express both D1- and D2-like receptors, suggesting that synergistic mechanisms between these receptors may also modulate PFC activity. One mechanism by which D1 and D2 receptors may interact has been provide by Xu and Yao (2010), by showing that D2 receptors suppress inhibitory GABA interneurons and D1 receptors increase excitatory activity, leading to coordinated activity between receptors that induces long-term potentiation. Other receptors may also interact through second messenger pathways with D1 receptors for the long-term storage of an extinction memory, as reported by Hu et al. (2010) in their demonstration that D1 receptor activation leads to increased expression of NR2B subunit of the NMDA receptor. An excellent review by Seamans and Yang (2004) provides a detailed examination of the modulatory role of dopamine in the prefrontal cortex and examines the role of the prefrontal cortex in working memory tasks.

In summary, the activity of dopamine in the prefrontal cortex appears to be critical for the consolidation and retrieval of fear extinction. The behavioral effects of fear extinction are likely to be mediated through prefrontocortical interactions with the hippocampus, striatum and amygdala. The PFC can fine-tune dopamine neuron activity in the ventral tegmental area through interactions with the basolateral amygdala and the ventral subiculum of the hippocampus (Patton et al., 2013). The interaction between the PFC and the amygdala is particularly relevant to fear conditioning and extinction, as the PFC modulates basolateral amygdala responses via inhibitory interneurons that results in decreased basolateral amygdala activity (Rosenkranz & Grace 2002). The prefrontocortical dopamine response to fear conditioning and extinction is likely related to memory consolidation and retrieval. Using a saccade based reward task in rhesus monkeys, Puig and Miller (2012) demonstrated that prefrontal cortical dopamine receptors are involved with increased cognitive flexibility and novel association formation, while leaving familiar associations intact. Dopamine signaling in the prefrontal cortex may be critical for extinction by gating cognitive and behavioral flexibility, as well as allowing the formation of associations that inhibit previously learned fear associations.

10. Conclusions and Future Directions

The evidence presented in this review shows that acquisition of fear extinction is mediated by dopamine in the amygdala, ventral tegmental area, dorsal and ventral striatum, and hippocampus, while consolidation of fear extinction requires dopamine in the prefrontal cortex. The literature is complicated by caveats that depend on dopamine receptor subtype, intracellular signaling cascades triggered by those receptors, and differential control of aspects of learning by subregions of the structures that have been investigated. Nonetheless, several broad conclusions can be made. First, amygdalar dopamine is involved in retrieval of the fear memory, as well as acquisition of the fear extinction contingency. Second, the firing of dopamine neurons in the ventral tegmental area may signal the prediction error present in an extinction session between the expected and actual outcome. Third, dopamine in the nucleus accumbens provides a mechanism by which the animal can assign salience to certain stimuli, in addition to mediating behavioral decisions (approach or avoidance) related to appetitive and aversive stimuli (Richard and Berridge, 2011). The role of the dorsal striatum in Pavlovian fear extinction remains open for study. Fourth, although the role of dopamine in the hippocampus is less clear, it may be important for the hippocampus to encode contextual information and modify ventral tegmental area output (Valenti et al., 2011). Fifth, the consolidation of fear extinction requires D1 and D2 receptor activation in the prefrontal cortex (Hikind & Maroun, 2008; Mueller et al., 2010), and the retrieval of fear extinction may involve coordinated dopamine activity in the prefrontal cortex, amygdala and nucleus accumbens guided by signaling from the ventral tegmental area. Although there are some mixed findings about the role of dopamine in the acquisition of fear, the evidence presented in this review strongly implicates dopamine signaling as a critical component in fear extinction.

The study of dopamine in fear extinction provides an ideal arena to examine interactions between appetitive and aversive learning. The ventral tegmental area provides aversive or appetitive information through separate circuits (Lammel et al., 2011) that alter behavioral responses to conditioned stimuli. It is possible that enhancing dopamine signaling during fear extinction can switch the ventral tegmental area signaling from aversive to appetitive circuits, essentially counter-conditioning an aversive stimulus to become a neutral or rewarding stimulus. Behavioral studies examining the effects of counterconditioning stimuli have suggested that there is a direct inhibitory interaction between appetitive and aversive tasks (Nasser & McNally, 2012; Bouton, 1993) that may mirror the development of inhibitory learning required for fear extinction (see Bouton & Peck, 1992; Peck & Bouton, 1990). Pan et al. (2008) showed evidence for the development of inhibitory activity on dopamine neurons responsive to rewarding stimuli, as well as the evidence for dopamine neurons excited by an extinction contingency, reinforcing theories that incorporate new learning, inhibitory activity, and forgetting as components of extinction.

Understanding these aversive-appetitive interactions may also allow for the development of pharmacotherapies for post-traumatic stress disorder and possibly substance dependence, as extinguishing the behaviors associated with both disorders may be interlinked through the dopamine system. In this respect, it may be valuable to evaluate the effects of putatively biased dopamine agonists on fear extinction, as biased agonism could theoretically enhance extinction with fewer side effects.

Future studies examining the role of dopamine in fear extinction should also consider the effect of manipulating the salience of stimuli associated with fear extinction. Although the concept of incentive salience has been primarily applied to reward learning (Berridge & Robinson, 2003), Bromberg-Martin et al. (2010) suggest expanding the role of dopamine in the mesocorticolimbic system to motivational salience, allowing both aversive and rewarding stimuli to be incorporated into behavioral theories of dopamine signaling in the nucleus accumbens. Additionally, Bromberg-Martin et al. (2010) suggest that the nigrocorticostriatal circuit encodes motivational value, and updating this circuit might be an important component of behavioral alterations leading to fear extinction.

The ventral tegmental area and substantia nigra may be involved in encoding associative predictions and sending information through the motor or limbic system (Schultz, 2006), but there also is a great deal of interaction between all regions expressing dopamine receptors that allows for continuous monitoring and updating of behavior that corresponds with environmental alterations. Although the striatonigral and mesocorticolimbic system are thought of as separate dopamine circuits, the integration of the information carried by both of these networks suggests a theoretical role for dopamine in encoding and altering predictions to match shifting environmental inputs, rather than simply propagating established behavior (Wickens et al., 2007). This can be accomplished through activity in the prefrontal cortex, and may be related to ‘behavioral flexibility’ (Ragozzino, 2002).

Many issues remain to be resolved in understanding the multi-faceted role of dopamine in fear extinction. Of particular interest is whether similar mechanisms control fear and reward extinction in the dopamine system, as this might provide a valuable pharmacological target for altering long established behaviors, such as those seen in substance abuse or post-traumatic stress disorder (Peters et al., 2009). Exciting technological innovations such as fast-scan cyclic voltammetry (McCutcheon et al., 2012), optogenetic control (Stuber et al., 2012) and genetic manipulations (Fadok et al., 2009; Ortiz et al., 2010) will provide a path to thoroughly explore and understand dopamine signaling in fear extinction.