A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP

Abstract

According to the Hebb rule, the change in the strength of a synapse depends only on the local interaction of presynaptic and postsynaptic events. Studies at many types of synapses indicate that the early phase of long-term potentiation (LTP) has Hebbian properties. However, it is now clear that the Hebb rule does not account for late LTP; this requires an additional signal that is non-local. For novel information and motivational events such as rewards, this signal at hippocampal CA1 synapses is mediated by the neuromodulator, dopamine. In this Review, we discuss recent experimental findings that support the view that this “neoHebbian” framework can account for memory behavior in a variety of learning situations.

Introduction

The idea that association underlies memory was already emphasized by the Greeks [1]. The discovery of neurons and synapses raised the question of whether these structures are modifiable by experience and could thereby mediate association. A potential answer to this question was proposed by Donald Hebb [2]. According to the Hebb rule, associations are encoded if synaptic plasticity obeys a simple rule: the synapse between cell A and cell B will be strengthened if two conditions are met: 1) the synapse from cell A onto cell B is active; and 2) cell B responds to this and other inputs by strong depolarization that triggers action potentials. Thus, if cell A represented object A and cell B represented object B, the co-occurrence of the two objects would, by the Hebb rule, strengthen the synaptic linkage between these cells. This link would subsequently be evident when only object A was presented because it would lead to the firing of cell B, thus bringing object B to mind by association.

Experimental support for the Hebb rule came through the study of long-term potentiation (LTP), an activity-dependent change in the strength of synapses. LTP has been found in many brain regions but has been most extensively studied in the CA1 region of the hippocampus. Early experiments showed that LTP in CA1 is governed by the Hebb rule: the induction of LTP requires both presynaptic input and strong postsynaptic depolarization (the role of Na⁺ spikes remains unclear) [3]. Furthermore, preventing strong depolarization by injecting negative current prevents the induction of LTP [4]. Implicit in Hebb’s rule is that LTP is specific to the synapses at which the rule is met (e.g., inactive synapses should be unaffected despite the strong postsynaptic depolarization). The ability to induce LTP at single visualizable synaptic connections by 2-photon uncaging of glutamate has directly confirmed the synapse specificity of LTP [5].

Other work has revealed some of the molecular mechanisms that underlie Hebbian plasiticity, allowing tests of the role of this plasticity in memory. Remarkably, the Hebbian computation in CA1 is done by a single type of molecule, a type of glutamate-activated channel termed the NMDA receptor (NMDAR). The opening of the NMDAR is Hebbian: the channel opens only if there is both presynaptic glutamate release and strong postsynaptic depolarization. When these channels open, the resulting influx of Ca2+ activates the enzyme, Ca2+/Calmodulin-Dependent Protein Kinase II (CaMKII), which then triggers the local biochemical changes that strengthen the synapse. Experiments show that genetic or pharmacological interference with NMDARs or CaMKII strongly interferes with memory formation [6]. There is thus little doubt that a Hebbian form of LTP is important for memory formation.

But is the simple form of association envisioned by Hebb the whole story? From our daily experiences, we know that items may co-occur but be only briefly registered in conscious memory if we don’t attach importance to them. This suggests that there are additional factors that determine whether information is stored persistently. Importantly, studies of LTP (mostly in CA1) show that there is an additional factor: the persistence of LTP depends not just on the two factors of the Hebbian condition (glutamate release and postsynaptic depolarization), but also on a third, the action of the neurotransmitter dopamine [7] (see Box 1). Importantly, the dopamine release depends on systems-level processes that include motivation and stimulus novelty [8] (Box 2, Box 2 Figure 1). We term this “neoHebbian” to indicate that in addition to the 2-factor Hebbian process, stable synaptic modification requires a third signal dependent on a systems level computation.

Box 1. Mechanisms by which dopamine stimulates the protein synthesis required for late LTP.

Dopamine enhances protein synthesis within dendrites of hippocampal neurons [85]. Other results point to a role of brain-derived neurotrophic factor (BDNF) in triggering protein synthesis [86]. Experimental results suggest that D1 receptor- and BDNF-mediated pathways interact in the activation of extracellular-signal regulated kinase (ERK) 1/2 [mitogen activated kinase (MAPK)] activity [87]. Active ERK1/2 can then induce nuclear transcription [via cyclic AMP response element-binding protein (CREB)], regulate translation [eg. by activating the eukaryotic translation initiation factor 4E (eIF4E)], and stimulate ribosomal function. Hippocampal late LTP is blocked when ERK1/2 activation is blocked by a dominant-negative mutation of MEK1, the kinase that directly phosphorylates ERK1/2 [88]. The pathway for D1-dependent control of ERK1/2 has been most thoroughly studied in the striatum. Activation of cAMP, protein kinase A (PKA) and dopamine releasing protein (DARP), lead to the stimulation of protein phosphatase 1 (PP1). ERK1/2 activation occurs because PP1 dephosphorylates (thereby inactivating) the striatal enriched protein tyrosine phosphatase (STEP) that dephosphorylates active ERK1/2 [89].

Box 2. What do dopamine neurons signal?

Novelty

In primates, novel stimuli can trigger burst firing of dopamine cells which habituates as stimuli become familiar [44]. Microdialysis [90, 91] and fast scan cyclic voltammetry [92] show that various types of novelty produce dopamine release in the rat striatum. These include social novelty [90], novel tastes [91], and novel environments [92]. Placing a rat in a novel environment leads to dopamine release in the hippocampus [19], and there is converging evidence from fMRI studies that the human SN/VTA is also activated by novelty (see main text and Figure I).

Reward prediction errors

In reinforcement learning, dopamine responses conform with a signed prediction error; they increase for better than expected (reward) outcomes and decrease for worse than expected (aversive) outcomes [43]. In rare cases, clinical requirements provide an opportunity to obtain direct recordings from the SN (e.g. in PD patients)[52]. SN neurons display higher spike rates to unexpected gains and lower rates to unexpected losses with a timing (ie. 150–375 ms) that is largely consistent with non-human primate studies. In the basal ganglia, dopamine release can reinforce action choices (‘go’ responses) within the so-called direct pathway to actively obtain rewards, whereas dips in dopamine firing can reinforce behavioral inhibition (‘no go’ responses) within the indirect pathway to passively avoid aversive outcomes [93] (note that an elevation of dopamine enhances LTP in the striatum, whereas a drop enhances LTD [94–96]). This functional architecture provides a plausible mechanism for instrumental learning of active responses through positive reinforcement and passive avoidance responses through punishment.

Aversive events

It is now well-established that dopamine neurons can also fire in response to unexpected aversive outcomes and cues predicting punishments [45]. In monkeys, a substantial fraction of dopamine neurons in the dorsolateral SN/VTA are responsive to conditioned cues predicting aversive events; in contrast, those in the more ventromedial portions of the SN/VTA are inhibited by aversive stimuli [97] (but see [98]). Similar findings have been demonstrated in anesthetized rats, where some VTA dopamine neurons are excited by noxious stimuli [99], whereas others are inhibited [100]. Recent studies in freely behaving mice have shown that VTA dopaminergic neurons can exhibit responses to different valences. Thus, although 89% of VTA dopaminergic neurons respond with activation to a conditioned reward, they also respond to fearful events, with the majority of the neurons showing suppression. Interestingly, 25% respond with excitation to both positive and fearful events [101]. It is important to note that the fearful stimuli are maintained over the time course of the fear-inducing event, in contrast to the transient nature of a brief noxious stimulus as mentioned above. Indeed, sustained aversive stimuli or stressors (eg. repetitive footshocks or physical restraint), increase the number of dopamine neurons firing over longer periods of time [102]. This response is consistent with the time course of the increase in dopamine release produced by sustained aversive events measured with microdialysis [103, 104].

Interactions between valence and action

The human lateral SN/VTA has been similarly shown to be activated during anticipation (i.e. before execution of motor responses) to acquire a reward or to avoid a punishment [53]. This suggests that SN/VTA responses to aversive events are observed primarily under conditions that require active, rather than passive, avoidance. This view is largely consistent with the suggestion that dopamine may promote a learning process that enhances the motivational salience, the ‘incentive salience’, of reward cues [105] and with linking dopamine to ‘wanting’ rather than ‘liking’ in the field of addiction [106].

Attention/Salience

An alternative interpretation favored in the attention and learning field is that both rewards and aversive events generally produce an increase in dopamine that enhances learning/attention [107, 108] and thereby enhance salience.

Box 2 Figure I.

Activation of SN/VTA by different types of stimuli. A Stimulus novelty: Activation was induced in the right medial SN/VTA by rarely appearing novel scenes (novel oddballs) compared with rarely appearing familiar scenes (familiar oddballs). The data were obtained from healthy older adults (Reprinted with permission from [109]). B Incidental stimulus encoding. FMRI responses in bilateral medial SN/VTA were elicited during the incidental encoding of images of scenes that correlate with scene recognition memory after a 24 hour retention interval. (Reprinted with permission from [110]). C Cues that predict novelty. fMRI response in right SN/VTA was elicited by symbolic cues that predict images of novel scenes as compared to cues predicting familiar scenes (Reprinted with permission from [64]. The SN/VTA is visible as a bilaterally bright area in the midbrain. Colour bars depict t-values.

In this review, we will first summarize the evidence for the dopaminergic involvement in LTP and learning. We will then briefly discuss the different types of stimuli that can lead to dopamine release. We will then turn to evaluate a simple idea: that all stimuli leading to dopamine release should enhance memory. This has now been examined in both animal models and in humans and the results have interesting implications for how learning occurs. Finally, we will discuss the tag and capture model [9], that makes further predictions about how stimuli interact to affect memory. Readers interested in additional perspectives may find other recent reviews useful (eg. [10–12]).

Neohebbian synaptic plasticity: role of dopamine in late LTP in CA1

Early experiments revealed that dopamine antagonists applied during the LTP induced by multiple tetani (generally 100Hz for 1s) had an effect that was strongly time dependent [7]. Whereas the first phase of LTP was not strongly affected, the potentiation measured an hour after induction was strongly attenuated. LTP at these later times (termed late LTP) is dependent on protein synthesis, whereas the LTP at early times is not [13]. It is therefore thought that dopamine antagonists prevent late LTP because they prevent the required protein synthesis (see Box 1).

Much additional research has strengthened the evidence that dopamine is required for late LTP in CA1 and identified the key role of the D1 class of dopamine receptors. Late LTP is blocked by D1 antagonists [14] (Fig. 1A) and by genetic deletion of D1 receptors (Fig. 1C) [15, 16]. The experiments described above were done in the slice preparation; similar effects of D1 antagonist have been observed when experiments are done in vivo [16, 17] (Fig. 1B). Furthermore, depletion of dopamine selectively blocks late LTP [18], a block that can be reversed by a D1 agonist (Fig. 1D). Together these results make a strong case for the involvement of dopamine in late LTP. Other experiments have directly measured event-dependent release of dopamine in the hippocampus [19]; less is known about the location of dopaminergic axons and the release/uptake mechanisms (Box 5).

Fig. 1.

Dopamine is required for late LTP in the CA1 region of the hippocampus in rodents. A. Under control conditions (closed black circles), robust LTP was induced after high frequency stimulation at time 0 min (as indicated by arrows) [14]. A significant reduction in LTP was observed in the presence of a D1 antagonist SCH23390 (open circles), and a complete absence of LTP induction in the presence of the NMDAR antagonist, APV (grey circles) [14]. Experiments were done in the acute slice preparation. B. Effect of SCH23390 on LTP measured in vivo [17]. C. Experiments in acute slice from D1 receptor knockout mouse (circles) and wild type controls (squares)[15]. LTP in the CA1 region of the hippocampus in D1-deficient mutant mice (circles) and wild-type (squares). Asterisks indicate statistically significant deviations (*, p< 0.05; ** p< 0.01) between the two groups at late stages of LTP. D. Block of late LTP by in vivo dopamine/noradrenaline depletion with 6-hyroxydopamine (open circles) is reversed by the addition of the D1 agonist, dihydrexidine (DHX) (black circles) [18]. The duration of potentiation in depleted animals is quite short, but consistent with the short potentiation after protein synthesis inhibition seen in some experiments [84]. Reproduced with permission from [14, 15, 17, 18], respectively.

Box 5. Outstanding questions.

How can the neoHebbian framework be tested pharmacologically in humans?

An important question to address further is whether dopamine is involved in the stabilization of human memory. Available dopaminergic drugs have substantial D2 receptor affinity, and selective D1/D5 compounds are not readily available for use in healthy humans (but see [123]). Another approach is to enhance dopamine levels with L-DOPA. This has been investigated mainly with respect to working memory [124, 125], but there is one report that L-DOPA improved long-term memory [126].

Does the neoHebbian framework apply to extrahippocampal MTL structures?

Protein synthesis-dependent forms of consolidation are not restricted to the hippocampus [127]. It would therefore be important to test whether dopamine plays a role in consolidation of non-episodic memory such as stimulus-familiarity.

How does dopamine affect encoding?

We have focused in this review on how dopamine affects consolidation. However, dopamine may be released in anticipation of information under various conditions [64](Box 2), and this could affect memory encoding. There are many ways this could occur: 1) Pathways can be turned off or on by dopamine [128]; 2) Network oscillations can be regulated [129, 130]; 3) Inhibition can be attenuated by a postsynaptic D3-dependent process [131]; 4) Encoding could be inhibited due to hyperpolarization by dopamine D2 receptors leading to an encoding of only strong stimuli (ie. increasing the signal to noise ratio [132]); 5) Explorative behavior could be energized (particularly in response to novelty) [133], [134], [135]; 6) Working memory capacity and attentional control [125] could be affected.

How can the neoHebbian framework be separated from encoding-related effects of dopamine?

Concern about encoding effects (see above) would be obviated if the “strong” stimulus that induces dopamine release enhances the memory for a ‘weak’ (e.g. ‘boring’ or pre-familiarized) stimulus given before the strong stimulus.

Do other neuromodulators contribute to neoHebbian plasticity?

LCholinergic [136] and noradrenergic [10] projections to MTL can also modulate LTP and long-term memory. Activation of the brainstem locus coeruleus, theorigin of noradrenergic projections to the CNS, can enhance protein synthesis in the dentate gyrus [137]and hippocampal stimulation can lead to increased hippocampal release of noradrenaline [138]. There is also crosstalk: the primary cholinergic drive of the hippocampus is strongly affected by dopamine [139], as is the locus coeruleus [140]. It is possible that noradrenaline is more important for consolidation of long-term memory for arousing and negative emotional events [141] than dopamine. Finally, serotonin may be necessary to consolidate memories about those types of aversive episodes that were associated with action inhibition (see [53] for a discussion).

How does the neoHebbian framework interact with off-line replay?

Whereas the Hebbian NMDAR-dependent processes in the slice occur only during the initial induction of LTP, events in vivo also involve NMDAR-dependent processes triggered by hippocampal replay processes [142]. In such replay, cells that were co-active during experience are reactivated during quiet wakefulness or sleep (reviewed in [143]). Such processes underlie consolidation events that both fix information at synapses and allow transfer of information from one region to another. Dopamine antagonists given only during such consolidation disrupt memory measured many hours later [144, 145].

Is there a tension between the penumbra and reward-related memory enhancement by dopamine?

It will be important to learn more about the time course of dopamine release, the duration of synaptic tags and the lifetime of plasticity related proteins, all of which will determine the properties of the penumbra. This may help to resolve the following puzzle. Reward-related SN/VTA activation improves memory for the rewarded stimulus, but not for the non-rewarded stimuli given in close temporal proximity [57], implying either a very short or very stimulus-specific penumbra. This is at odds with the observation that novelty-related activation of the SN/VTA has a long (ca. 30 minutes) penumbra that affects memory for unrelated information (e.g. exposure to novel scenes can affect memory for words) [74]. One possible resolution is that the duration or stimulus-specificity of the penumbra depends on the type of motivational event that triggers dopamine release.

Where is dopamine released in the hippocampus?

Older methods for labeling dopamine axons can give ambiguous results. Modern genetically-based labeling methods are needed to determine the location and density of these axons.

Two types of experiments demonstrate that dopamine can strengthen the synaptic potentiation produced by learning itself. In vivo recordings show that CA3/CA1 synapses are strengthened over time during trace eyeblink conditioning but that this strengthening is greatly reduced either in D1 receptor knockout mice or after reducing D1 receptors by virally delivered siRNA [16]. Another line of experiments examined the learning that stabilizes place fields in CA1 after the animal enters a new environment. It was found that D1 antagonists prevent this stabilization [20].

Late-phase LTP can be induced in the slice preparation without dopamine application; a result that seemed to indicate that dopamine was not required. However, the block of LTP by dopamine antagonists suggests that dopamine is indeed required. This apparent contradiction was resolved by the demonstration that the extracellular stimulation procedure used to induce LTP releases endogenous dopamine from axon terminals in the hippocampus [7]. Another quandary was posed by early work showing that dopamine agonists alone could produce potentiation [21]. However, it was subsequently shown that this potentiation depended on the action of test pulses given during agonist application and that activation of NMDARs by the test pulses was required [22]. Thus, it is now clear that dopamine by itself does not produce potentiation; rather, it acts as a permissive signal for the late-phase LTP induced by a NMDAR-dependent process.

Results such as those shown in Fig. 1 (top traces) suggest that early LTP is completely unaffected by dopamine antagonist. Subsequent work, however, showed that, with less strong induction protocols, there is a partial inhibition of early LTP [23]. Thus, both early and late LTP are dopamine-dependent, but late LTP has a stronger dependence. Based on these results, one would predict that dopamine antagonists would produce a weak reduction of memory at short times after learning but would produce a strong reduction at later times. In the next section we discuss experiments that tested this prediction.

D1 antagonists block 1 day old memory without affecting early memory

Plasticity in the hippocampus subserves the particular form of memory called episodic memory [24]. This form is rapidly acquired and is referenced to a specific time and place. The investigation of the role of dopamine in episodic memory has been facilitated by the development of a rodent model of episodic-like memory: after learning several reward-location associations in an environment, rats rapidly (i.e., in only one trial) update their memory if one of the reward-location associations is changed [25]. This one-trial memory for novel associations is critically dependent on the hippocampus and on glutamatergic neurotransmission during encoding [25, 26]. Intrahippocampal infusion of the D1/D5 dopaminergic antagonist SCH23390 just before encoding causes impaired memory for the new reward-location association measured 24 hours later, but has no effect on early memory measured at 30 minutes [26] (Fig. 2). A similar observation was made for one-trial encoding of the location of the escape platform in the Morris water maze [27]. These data demonstrate that D1/D5 receptor-mediated effects do not strongly affect encoding or early memory, but do affect memory at later times. Dopamine has no effect on the previously formed memories and is not required at the time of retrieval [27]. Further insight into the role of dopamine receptors in memory will come from the use of hippocampal-specific knockout of dopamine receptors (e.g. Sarinana and Tonegawa, Soc for Neurosci abstract 608.11, 2010).

Fig. 2.

A D1 antagonist blocks long-term memory but not early memory (30min) for novel episodic-like information in rats. A In this event-arena apparatus, six sand wells are available to which the rat can run to collect food. Animals are trained over several sessions to learn six flavor/sand-well associations. After acquisition, they are given one trial of a new flavor-location paired associate. Memory recall is measured as time spent digging (% Dig time) at the correct location for a given flavor (new cued). The primary measure of memory recall is calculated as the proportion of time spent at the cued location relative to the average time at the non-cued locations for the originally acquired flavor locations (original non-cued) and the non-cued location of the other novel flavors introduced during a training day (new non-cued). B The D1 antagonist SCH23390 given 20 minutes prior to the encoding of the new flavor/location paired associate has no effect on short-term memory (tested 30 minutes after the last training trial) but eliminates long-term memory (24 hours later), as compared to vehicle control (NaCl). Reprinted with permission from [26].

Firing properties of dopaminergic neurons

Given the importance of dopamine release for LTP and memory, it becomes crucial to understand the properties of dopaminergic neurons and the kind of stimuli that lead to dopamine release. Midbrain dopamine neurons consist of three largely contiguous cell groups: the retrorubral field (cell group A8 in the rat nomenclature), the substantia nigra pars compacta (SNc, A9), and the ventral tegmental area (VTA, A10) [28, 29]. In primates, the SNc is further divided into a dorsal and ventral tier [30]. Although the SN and VTA are known as dopaminergic regions, some cells in these regions are GABAergic or glutamatergic, and others release both glutamate and dopamine [31, 32]. Dopamine cells can be identified by their electrophysiological waveform signature and firing characteristics [33–35].

Dopamine cells can be silent, fire single spikes somewhat irregularly (tonic mode) or fire brief bursts of spikes. Studies performed both in vivo and in vitro show that the tonic mode is driven by intrinsic pacemaker conductances [34, 36]. This firing is modulated by GABAergic inputs from the ventral pallidum; a structure in which cells have high spontaneous activity [37]. As a result of this inhibition, up to 50% of dopamine neurons are in the silent mode at baseline. Activation of the ventral subiculum (an output structure of the hippocampus) excites the nucleus accumbens (Nac), which, in turn, inhibits the ventral pallidum. This reduces the inhibition of dopamine cells and thereby increases the number of tonically firing cells [38, 39]. Tonically firing neurons (but not silent neurons) can be driven into a bursting mode by stimuli such as those associated with reward (see below). These bursts last about 200–500 msec, have interspike intervals <100 msec, and generally contain about 3–5 spikes. Burst firing in dopaminergic neurons is potently driven by the brainstem pedunculopontine tegmentum (PPTg), which provides a glutamatergic input that acts via NMDARs ([39, 40] see Box 3). Recent work demonstrated the functional importance of these bursts: selective genetic inactivation of NMDARs in dopaminergic cells blocked burst production and prevented conditioned behavioral responses that were dependent on dopamine [41]; conversely, simulating burst firing using channelrhodopsin is sufficient to induce behavioral conditioning [42]. These results indicate that burst firing controls some downstream functions, but others may depend on how many cells are in the tonic mode (see Boxes 3 and 4).

Box 3. Circuits that control the firing of dopaminergic cells.

The novelty processing hierarchy of the MTL (Figure I) is part of the circuitry that can regulate dopamine responses to novelty. The perirhinal cortex [65], for instance, shows much stronger responses to novel items than familiar items and may form connects to the PPTg (Figure I), a structure that regulates the SN/VTA. Strong evidence for hippocampal involvement in novelty-dependent dopamine release comes from experiments showing that this release is blocked by inactivating the subiculum, an output structure of the hippocampus that projects, by a polysynaptic pathway, to the SN/VTA (Figure I). In humans, hippocampal event-related potentials signaling unexpected events precede those in the Nac [111], which is compatible with the possibility that information flows from the hippocampus to the Nac and from there to the SN/VTA (Figure I). Hippocampal novelty signals incorporate context information by signaling mismatches for learned objects-spatial location associations [112], temporal sequences [113] and during integrative encoding [49].

The PPTg regulates burst firing of dopamine cells rather than their tonic resting activity [[39]]. It is known to be driven by prefrontal afferents, and responds to single sensory events from different modalities with earlier burst firing than dopamine neurons [[114]]. It is unclear to what extent PPTg responses are modified by contextual factors and/or conditioning [[115]] or simply relay ‘accurately timed and attended sensory information’ [[114]]. The PPTg drive [116] depends on a permissive ‘gating’ input from the laterodorsal tegmentum (LDT) [[107, 116, 117]]. The LDT, in turn, receives substantial input from the medial prefrontal cortex (PFC) [[118]], allowing the PFC to indirectly affect mesolimbic dopaminergic neuron activity[[118]].

Another brain region which is known to participate in the modulation of dopamine neurons is the lateral habenula. The lateral habenula provides SN/VTA dopamine neurons with a negative reward-prediction signal through the fasciculus retroflexus fiber bundles [[119]]. The rostral medial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopaminergic neurons [120], may be the interface between the lateral habenula and the SN/VTA.

Box 3 Figure I.

Connectivity within the MTL and the hippocampus-VTA loop. Information about objects and their spatial location converges in the entorhinal cortex via the perirhinal and parahippocampal cortices. These inputs can be integrated in the hippocampus towards ‘object-in-context’ information. Contextual representations in the hippocampus are not limited to spatial information, but may also include the temporal sequence of occurrence and anticipatory states associated with current goals and expectations. Hippocampal contextual novelty signals generated on the basis of such representations can increase the pool of tonically active dopamine neurons in the SN/VTA. This could allow perirhinal cortical information about stimulus-novelty to generate phasic dopamine responses (italics in the figure indicates that this possibility needs experimental testing). The loop is completed by ascending dopaminergic fibers that innervate the hippocampus and other MTL structures.

Box 4. Phasic and tonic dopamine signals affect both memory and behavior.

The tonic and phasic dopamine signals have consequences for both memory and behavior. In a predictable, familiar (and safe) environment, the ventral subiculum would be in a low-activity state, causing the number of tonically firing dopaminergic neurons firing to be minimal. A phasic stimulus would produce burst firing only in the tonically firing dopamine neurons, leading to a moderate dopamine output. As a result, the attentional state of the animal would be little affected. However, if the animal is in an unpredictable, novel (and presumably less safe) context (e.g. hunting) a novelty signal from ventral subiculum would raise the fraction of dopamine cells that are tonically firing. Under these conditions, a behaviorally relevant stimulus would cause a stronger phasic dopamine signal [107] (referred to as ‘saliency capture’ in [54, 61]).

A crucial control point in this process is the ventral striatum, which receives input from both the subiculum and PFC. Electrophysiological and behavioral studies suggest that these two inputs compete for regulation of the ventral striatum and are regulated in an opposite manner by the dopamine system. Tonic dopamine, which works through the higher-affinity D2 receptors, attenuates medial PFC (mPFC) inputs. In contrast, higher-concentration phasic dopamine signals activate D1 receptors and thereby enhance subicular input [121]. In a condition in which the animal’s behavior leads to a better than expected outcome and phasic release (i.e. positive prediction error), the subicular input would be potentiated, causing the animal to maintain focus on the current task, thus allowing rewarded behaviors to be maintained. However, in the absence of an expected reward, or if a worse than expected outcome occurs (i.e., a negative prediction error), the consequent depression of dopaminergic neuron activity would attenuate dopamine-mediated potentiation of the ventral subiculum while at the same time removing inhibition of the mPFC input. Consequently the mPFC would drive the animal to switch response strategies [122].

Dopamine release by reward, novelty and aversive stimuli

The types of stimuli that increase burst firing of dopaminergic neurons have been extensively studied in monkeys. Classical work showed the importance of reward as a stimulus for the bursting of these cells [43], while subsequent studies have demonstrated that a novel stimulus (in the absence of reward) can also produce burst firing [44]. More recent work has demonstrated conditions under which dopamine cells can be driven by aversive stimuli (reviewed in [45]; see also Box 2).

An important recent advance has been the ability to use functional magnetic resonance imaging (fMRI) to determine whether similar classes of stimuli activate the dopamine system in humans [46–49] (reviewed in [50]). Currently, most fMRI studies do not have sufficient resolution to distinguish subregions of the SN/VTA (this will be achievable with ultrahighfield fMRI, which has greater than 1mm resolution). A further difficulty is that fMRI cannot distinguish tonic or bursting modes. Despite these difficulties, there is converging evidence for a close relationship between the fMRI response in the SN/VTA and dopamine release (reviewed in [50]). Such studies (e.g. [48, 51]) have demonstrated that the human SN/VTA is activated by rewards, a finding confirmed by a recent study using electrophysiological recordings [52]. Other work using fMRI has demonstrated that activation can also occur due to aversive events [53] as well as novel stimuli [54]. Thus, novelty, reward stimuli and aversive stimuli are all able to activate the dopamine system in both monkeys and humans.

Effect of Motivation (Reward) on Behavioral Memory

It is to an organism’s advantage to recollect circumstances which led to rewards. Indeed, according to the neoHebbian framework, activating the SN/VTA by a reward should enhance hippocampal-dependent episodic memory of information present at the time of reward. Furthermore, this improvement should be small with short delays when early LTP is present, but large at much longer delays when late LTP is present. A cognitive paradigm used to test these predictions in humans involved activating the SN/VTA at the time of encoding by giving reward. It was found that the prospect of receiving reward (e.g., monetary gain) improved long-term episodic memory for novel stimuli in incidental [47, 55–57] and intentional [46] encoding paradigms. This reward-related memory enhancement was associated with a co-activation of SN/VTA, striatum, and hippocampus, as detected by fMRI [46, 47]. Memory enhancement after long retention intervals (e.g. 24 hours) has been consistently found [56, 57]. Moreover, the enhancement was greater at late timepoints than at early intervals (ie. 3 weeks vs. 20 minutes) [47].

It is noteworthy that earlier cognitive psychology experiments on the effects of reward on memory used shorter retention intervals than the aforementioned studies (e.g. [58, 59]). Such studies did not show consistent memory enhancement and led to the idea that motivation was unimportant for learning. From the perspective of the neoHebbian framework, this negative result is understandable; i.e., the SN/VTA activity-related enhancement of memory would be more apparent if testing is conducted after long retention intervals. However, more studies are needed to clarify whether reward-related activation of the SN/VTA is associated with memory enhancement only after long retention intervals, compatible with the effects of intra-hippocampal injections of specific dopamine antagonists (e.g., [26]) (see Boxes 4 and 5).

Importance of the novelty and aversive signals for memory

In changing environments, it is vital to form long-lasting memories for novel events and, by definition, this requires encoding and consolidation after a single exposure. The role of hippocampal dopamine for consolidating novel associations after only one trial in rats [26] raises the question as to whether the human dopaminergic midbrain is activated by trial-unique novelty. Indeed, fMRI studies (for examples see Box 2 Figure 1) confirm SN/VTA activation by associative novelty [60] and stimulus novelty [61] but less by other salient events such as rareness or negative emotional content [61]. Interestingly, the personality trait of novelty seeking is correlated with the magnitude of the SN/VTA novelty fMRI response even when novel stimuli predict the absence of reward [56], suggesting that this effect of novelty is hard to change by learning. In further support of such hard-wiring, positron emission tomography (PET) studies in humans show that novelty-seeking traits are inversely associated with D2-like receptor availability in the SN/VTA region [62] and are positively correlated with white matter connectivity between the hippocampus and the ventral striatum [63]. The human SN/VTA is also activated by cues that predict novel images, or by unexpected novel images that followed familiarity-predictive cues (ie. an ‘unexpected novelty’ response) [64].

Some progress has been made in understanding the circuitry involved in novelty-dependent dopamine release (see Box 3). There is now converging evidence that different forms of novelty signals are computed along the medial temporal lobe (MTL) processing hierarchy from the parahippocampal and perirhinal cortices to the entorhinal cortex and from there to the hippocampus (see Box 3). In the perirhinal cortex, for instance, stimulus-related incoming information can be compared to stored information and a stimulus-related novelty signal is computed [65]. The hippocampus, on the other hand, integrates information about stimuli and their spatial location into a contextual representation and can compute various types of novel changes in these relationships.

It may seem evident that a general purpose system like episodic memory should also enable long -term memory consolidation for events associated with aversive outcomes. Surprisingly, there is relatively little work comparing the consolidation of rewarding and aversive events (e.g. losses) in humans (although there is extensive work on the consolidation of emotionally negative stimuli). However, the observation that there are dopaminergic responses to aversive events (see Box 2) suggests that dopaminergic consolidation mechanisms similar to those seen with novelty and reward may apply. Consistent with this, learning of responses to aversive stimuli in rodents (ie. during fear conditioning) is inhibited by dopaminergic antagonists [66–68]. Moreover, dopamine is required for association between stimuli and aversive events in the amygdala [69].

The synaptic tag and capture model: role of plasticity-related proteins

As noted before, the finding that late LTP is more strongly dependent on dopamine than early LTP has been influential because it inspired investigation into the effect of dopamine on early vs. late learning. This suggests that other physiological findings regarding the role of dopamine in LTP might also be relevant to behavior. There are indeed several additional results that fall into this category and these have been incorporated into the Synaptic Tagging and Capture model developed by Frey and Morris [9].

According to this model, weak stimulation induces only early LTP. In contrast, stronger stimulation produces the dopamine-dependent protein synthesis that allows late LTP. The proteins required for late LTP are termed plasticity-related proteins (PRPs). Given the synapse specificity of LTP, these proteins are presumed to interact only with synapses that have undergone Hebbian LTP. Frey and Morris propose that this specificity occurs because LTP immediately produces a synapse-specific “tag” and that PRPs can only produce late LTP at tagged synapses. However, the synapses that produce the PRPs can be different from those at which the tag is set. It appears that both the tag and the PRPs have considerable lifetimes. This follows from the finding that stimulation of protein synthesis and the induction of Hebbian plasticity do not need to be coincident; indeed, the trigger for protein synthesis can occur either before or after Hebbian plasticity (with intervening periods of many minutes). Taken together, these findings suggest that there is a “penumbra” surrounding events that cause dopamine release; the memory for events that occur before or after the dopamine release would depend not only on their own properties, but also on whether they fell within the penumbra of a dopamine-releasing stimulus.

Testing the synaptic tag and capture model at the behavioral level

A number of animal behavioral studies have used novelty exploration as a ‘strong’ event that is likely to induce dopamine release [70–72]. In agreement with the synaptic tagging and capture model, exploration of a novel environment can make a memory for a ‘weak’ event more persistent [70–72]. For instance, a single exposure to the location of a small reward is rapidly forgotten in rodents within a few hours, but persists for at least 24 hours if it is preceded or followed (within 30 minutes) by novelty exploration [72]. Also consistent with the synaptic tagging and capture hypothesis, intrahippocampal application of the D1/D5 receptor antagonist SCH23390 during novelty exploration diminishes memory retention [72].

Similar to the enhancement of long-term memory by reward in humans [47, 56, 57, 73], 24hr memory is enhanced in rats for rewarded locations (even after only a single exposure); moreover, this long-term memory can be blocked by a D1/D5 receptor antagonist at the time of encoding [72]. That this drug-induced memory impairment is not due to an effect of dopamine on encoding is nicely demonstrated by the rescue of long-term memory if there is a preceding exposure to a novel environment [72].

Studies of LTP have shown that strong stimuli can enhance the encoding of weak stimuli even if the strong stimulus is given many minutes before or after the weak one. We term this temporal smearing the “penumbra” of the strong stimulus. Experiments have been conducted in humans that specifically looked for this penumbra. Individuals were exposed to novel photographs of natural scenes (‘strong events’, like those one would expect to see in the magazine National Geographic), for 5 minutes before being presented with the ‘weak’ encoding events, single words that were pre-familiarized on the day before and hence were not by themselves novel. The result was a higher proportion of recollected (as opposed to those recognized on the basis of familiarity) words when compared to individuals who were exposed to familiar pictures prior to encoding. This effect was stronger after 24 hours as compared to 30 minutes after encoding [74]. Of note, pre-exposure to familiar, but emotionally very negative, photographs did not cause a comparable enhancement. It is therefore unlikely that affective arousal can account for the memory enhancement after novelty exposure. Hence, the behavioral penumbra seen after exposure to novelty in humans is compatible with the synaptic tag and capture model.

Although the concept of a penumbra is promising, further work to investigate its experimental basis is required. To understand factors that determine the duration of the penumbra, measurements need to be made on the duration of the dopamine signal in the hippocampus. The dopamine transporter (DAT), which rapidly clears dopamine from the synaptic cleft, is much less abundant in the hippocampus than in the striatum [75], so the duration of dopamine elevations may be relatively long. Recent elegant work is providing information about the spatial and temporal properties of protein-synthesis processes during LTP [76], which may also influence the time course of the penumbra. Behavioral experiments suggest that there are differences in the penumbra duration for novel as compared to rewarding events [57] (see Box 5). Perhaps a mechanistic understanding of the underlying events will be able to explain this difference. Additional experiments at the behavioral level are needed to examine the role of attention or arousal in the penumbra effect: if strong stimulation enhances the memory of a subsequent weak stimulus, perhaps this is due to stimulation of attention or arousal by the strong stimulus or to effects at encoding (see Box 5).

Concluding Comments

The results reviewed here indicate that the neoHebbian framework has explanatory power in the understanding of memory. A key insight to emerge from physiological experiments is that LTP at short times is relatively insensitive to dopamine, whereas LTP with longer delays is sensitive. Moreover, other properties of LTP (weak/strong; penumbra) appear also to have predictive power in explaining properties of memory.

The idea that long-term memory depends on more than the Hebbian condition makes sense, given the likelihood that too much information written into memory might lead to overwriting (and thus degradation) of pre-existing memories [77]. According to the neoHebbian model, the default condition is that information is stored automatically in the hippocampus by early LTP based on a local Hebbian process. Generally, this newly stored information will fade away as early LTP declines. Only if a systems-wide computation determines that there is a high level of novelty or motivational salience will dopamine be released, allowing the biochemical processes of late LTP to incorporate the information into long-term memory.

The conditions that lead to activation of the human dopamine system, as judged by fMRI signals, are similar to the conditions leading to activation in non-primate mammals. It therefore seems likely that these principles may apply across all mammals. Nevertheless, experiments that further test the neoHebbian ideas in humans are warranted. An important prediction is that long-term memory would be affected positively (but possibly with an inverted U-shape pattern) by drugs that increase dopamine availability, such as levodopa (L-DOPA), which might enhance the burst-evoked release of dopamine. There has been remarkably little work on the effect of dopaminergic manipulations on human long-term memory. The long-term safety of available dopaminergic drugs is an important problem that needs to be overcome. Alternatively, progress may be made by taking advantage of genetic diversity in the human population with respect to genes that strongly affect dopaminergic function, which has already proven valuable for studying the human reward system (eg. [78]).

A better understanding of the human dopaminergic system is relevant to several human neurological or psychiatric disorders and to aging. Parkinson’s disease (PD) usually begins with loss of nigrostriatal dopamine cells in the ventral tier SNc, leaving the dorsal tier projection neurons to the hippocampus, ventral striatum, and cortex intact [79]. It is thus understandable that severe episodic memory problems are not a symptom of early PD [80]. However, at late stages, when all types of dopaminergic projection neurons are affected, memory problems do become severe. In schizophrenia, the enhanced activation of the hippocampal system may increase dopamine release and skew behavioral choice towards repetitive behavior (see Box 3). In aging, there is now converging evidence from human imaging studies for a structural degeneration of the VTA/SN (for reviews, see [54, 81]). Consistent findings have also been recently reported in aged rats [82]. The possible negative effect of such a decline on the ability of information to enter into long-term memory has not yet been adequately considered.

Understanding the role of dopamine in memory may lead to methods that can be used to improve learning and teaching methods. First, based on the penumbra hypothesis, exposure to novel or salient information could be used to improve the long-term persistence of other information given in temporal proximity. Second, strategies based on the ability of external reward or reward anticipation to release dopamine might prove to be useful in increasing the retention of learned material. Finally, just as with external performance positive feedback, memory persistence might benefit from internally generated reward signals. For instance, the ability to recollect newly acquired information may be intrinsically rewarding. In fact, the study of human learning has revealed an interesting puzzle; long-term retention is not helped by simple re-exposure to recently learned material but is greatly helped by retesting even when subjects already know the answer [83]. One interesting possibility is that retesting provides an opportunity to generate intrinsic reward signals, thereby enhancing long-term persistence of newly learned material.