Abstract
This mini-symposium aims to integrate recent insights from anatomy, behavior, and neurophysiology, highlighting the anatomical organization, behavioral significance, and information-processing mechanisms of corticostriatal interactions. In this summary of topics, which is not meant to provide a comprehensive survey, we will first review the anatomy of corticostriatal circuits, comparing different ways by which “loops” of cortical–basal ganglia circuits communicate. Next, we will address the causal importance and systems-neurophysiological mechanisms of corticostriatal interactions for memory, emphasizing the communication between hippocampus and ventral striatum during contextual conditioning. Furthermore, ensemble recording techniques have been applied to compare information processing in the dorsal and ventral striatum to predictions from reinforcement learning theory. We will next discuss how neural activity develops in corticostriatal areas when habits are learned. Finally, we will evaluate the role of GABAergic interneurons in dynamically transforming cortical inputs into striatal output during learning and decision making.
Introduction
The striatum receives input from a vast expanse of areas distributed across the cortical mantle, yet it is still poorly understood how all of this input is transformed into outputs that regulate behavior and cognition. Cortical projections to the striatum are topographically ordered in a series of parallel anatomical “loops” running from neocortex to the striatum, pallidum, thalamus, and back to neocortex. These parallel macrocircuits have been linked to different global functions: whereas a “limbic,” ventromedial prefrontal–ventral striatal loop has been delineated to mediate motivational and reward processing, other loops engage in sensorimotor or cognitive processing (Alexander et al., 1990; Graybiel et al., 1994; Voorn et al., 2004; Yin and Knowlton, 2006). Information processed along these pathways is under the modulatory control of dopamine released from fibers originating in the ventral tegmental area (VTA) and substantia nigra pars compacta (SNpc). Given the parallel organization of corticostriatal circuits, the question arises how coherent behavior, requiring integration of sensorimotor, cognitive, and motivational information, is achieved. In this mini-symposium, we will first address the question of how parallel corticobasal ganglia circuits may communicate. Next we will ask how corticostriatal circuits are involved in learning and memory formation. One approach to investigate this topic is to zoom in on one corticostriatal subsystem in particular. Therefore we will examine the functioning of the hippocampal–ventral striatal subsystem in learning to associate places and contexts to outcomes, such as reward. We will subsequently focus on the “sensorimotor” subsystem, including the dorsolateral striatum (DLS), in mediating habit formation. Recent advances in associative learning and plasticity in corticostriatal circuits have been guided by theories of reinforcement learning (Sutton and Barto, 1998). Hence we will compare theoretical predictions about dorsal and ventral striatum with results from multineuron recordings in freely behaving rats. In addition to differences in information processing between loops, we will consider commonalities found across striatal subregions. One such common element is the GABAergic fast-spiking interneuron (FSI), which makes up only a tiny fraction of all striatal cells, but powerfully modulates activity of striatal projection neurons, called medium-sized spiny neurons (MSNs). We will review how these interneurons contribute to transforming cortical input into striatal output. Thus, by bringing together findings from neuroanatomy, in vivo electrophysiology, behavioral neuroscience, and computational modeling, this symposium will highlight key advances in our understanding of corticostriatal functions in decision making and memory formation.
Communication between corticostriatal systems via the ventral mesencephalon
For integrating sensorimotor, cognitive, and motivational information that is processed by different corticostriatal circuits, some form of communication between parallel circuits appears to be required. Such communication may take place via intrastriatal collaterals of projection neurons and striatal interneurons (van Dongen et al., 2005). Another route allowing information flow from limbic to sensorimotor-related corticostriatal pathways is via the projections from nucleus accumbens (NAc), which constitutes the main portion of the ventral striatum, to the SNpc and VTA and, subsequently, to dorsal, sensorimotor-related striatal regions via dopaminergic fibers originating from the SNpc/VTA complex (Nauta et al., 1978). Much attention has focused on communication via a cascade-like spiraling circuitry connecting ventromedial regions of striatum with ever more dorsolaterally located striatal areas by way of an open loop with the substantia nigra. This arrangement, described in the monkey (Haber et al., 2000), may hold true also for the rat and allows for a serial rather than parallel flow of information processing. The cellular population in rat SNpc/VTA projecting to the striatum has been shown to comprise at least two types of neurons. The first type shows a mediolateral topographical organization in its projection pattern, with medially located cells projecting to medial parts of the striatum and laterally located cells reaching lateral striatal regions, like a mirror image of the mediolateral topography in the striatonigral axonal projections. The second type does not obey such topography: (dorso)lateral striatal territories are reached from a medial and dorsal position in SNpc and VTA (Maurin et al., 1999).
The former population is probably involved in striatonigro striatal feedback loops (cf. Haber et al., 2000); the latter has nonreciprocal relationships with the striatum and may establish direct contacts between ventral and dorsal corticostriatal circuits. Whether such is indeed the case was studied using neuroanatomical tract tracing and neurophysiological techniques. Rats were simultaneously injected with an anterograde tracer (BDA MW 10,000) into the medial shell region of NAc and a retrograde tracer (Fluorogold) into DLS. A sensorimotor identity of the latter striatal area was confirmed by the presence of retrogradely labeled neurons in somatosensory and motor cortex. Importantly, the retrograde tracer labeled neurons along the entire medial to lateral extent of SNpc and dorsolaterally in VTA. Laterally in SNpc, anterograde fiber labeling from the shell remained separate from retrograde cell labeling. However, in the dorsal parts of medial SNpc and laterally in VTA, the distributional patterns of retrogradely labeled cells and the anterogradely filled fibers from the shell showed extensive overlap (Fig. 1). In these areas, fibers and cell bodies in close proximity to one another suggested the presence of synaptic contacts. Indeed, electrical stimulation of the shell induced an inhibitory response in electrophysiologically identified dopaminergic neurons that were antidromically driven from DLS and located in the very region of distributional overlap. These findings provide for a short and direct route via which limbic information can reach sensorimotor circuits.
Hippocampal–striatal communication in contextual conditioning
Traditionally, corticostriatal feedback loops have been associated with executing cognitive and motor functions in the mammalian brain (Alexander and Crutcher, 1990; Chudasama and Robbins, 2006). A potentially important role that is not so readily attributed to the corticostriatal system is that in learning and memory. Extensive evidence suggests that certain subregions of the neocortex and striatum receive convergent inputs from key component structures of learning and memory systems such as the hippocampus and amygdala, and are thus ideally placed to play a regulatory role. However, the functional significance of such a pattern of limbic innervations in the corticostriatal system is yet to be demonstrated. Data from lesion and drug infusion studies selectively targeting corticostriatal areas that receive inputs from the hippocampus and basolateral amygdala support the existence of separate streams of associative information processing in the ventral striatum. Indeed, the integrity of the NAc shell subregion and the activation of dopamine D2 receptors (Setlow and McGaugh, 1999; Ito et al., 2008) have been shown to be necessary for the processing of spatial information that is also dependent on the hippocampus. In contrast, the NAc core has been shown to be critical for the processing of associations between emotionally significant events and discrete pavlovian stimuli, which is also dependent on the basolateral amygdala (Cador et al., 1989; Parkinson et al., 2000; Ito et al., 2006, 2008).
There is some evidence for different subregions of the prefrontal cortex (PFC) supporting different domains of learning and memory processes, with a number of studies reporting marked impairment in spatial working memory (Ragozzino et al., 1998; Kesner and Ragozzino, 2003) as a consequence of selective lesions of the prelimbic cortex. However, somewhat surprisingly, this selective effect on spatial memory often fails to manifest itself when the lesions are not just restricted to the prelimbic cortex, but extend to other parts of medial PFC (de Bruin et al., 2001; Lacroix et al., 2002; Deacon et al., 2003), highlighting the difficulty of separating mnemonic deficits from concomitant non-mnemonic deficits such as attentional and behavioral flexibility. Nevertheless, studies using disconnection lesions of brain structures within the corticolimbic–striatal system to investigate the functional connectivity between two brain areas have established the importance of serial transfer of spatial information between the prelimbic cortex and hippocampal CA1 region (Floresco et al., 1997) and ventral hippocampus (Wang and Cai, 2006). For the first time, recent work has now demonstrated a functional pathway between the rat hippocampus and NAc shell in the control of conditioned appetitive behavior by spatial contextual information (Fig. 2) (Ito et al., 2008) that is clearly distinct from another circuit between the NAc core and basolateral amygdala, which is concerned with discrete stimulus control over Pavlovian behavior (Everitt et al., 1991). Importantly, interruption of neural processing between the hippocampus and NAc shell was found to cause disruption to the acquisition of context-dependent retrieval of discrete cue information, providing functional evidence that the shell subregion may provide a site at which contextual and discrete cue information may be integrated. In addition, new evidence has been obtained to indicate that the dopaminergic innervation of the rat NAc plays a key role in modulating learning and memory processes. Using a novel paradigm in which two types of sucrose-associated cues (spatial vs nonspatial) are concurrently made available to compete for control over conditioned appetitive behavior, direct infusions of d-amphetamine (which induces elevation in extracellular dopamine levels) into the NAc core and shell were found to exert differential and selective effects upon their conditioning rates. These results point to the important role of the dopaminergic innervations of the NAc shell and core in regulating limbic information flow, and illustrate differential, and even opposing, roles of the NAc core and shell regions in conditioned appetitive behavior under the control of spatial cues.
Cross-structural replay of contextual and reward information
Although it is unclear how the consolidation of composite place–reward memories is achieved, the strengthening of synapses involved in coding spatial and motivational information is thought to benefit from the spontaneous replay of recently acquired neural activity patterns that occurs in periods of quiet wakefulness and slow-wave sleep following a behavioral event (Marr, 1971; Buzsáki, 1989; McClelland et al., 1995; McNaughton et al., 2003). Theories on memory consolidation propose that the hippocampus initiates a state of information retrieval during postexperiential, “off-line” periods such as sleep, and transmits this state to its target structures (Marr, 1971; Buzsáki, 1989; McClelland et al., 1995; Pennartz et al., 2002; McNaughton et al., 2003). Neuronal reactivation, as observed during “off-line” periods in rats and monkeys, occurs in several brain areas including the hippocampus (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Lee and Wilson, 2002), neocortex (Qin et al., 1997; Hoffman and McNaughton, 2002; Euston et al., 2007; Ji and Wilson, 2007), and ventral striatum (Pennartz et al., 2004). These areas appear to replay a specific aspect of a composite memory trace corresponding to the information processed during the awake experience. Spatial–contextual information may be reactivated in the hippocampus as reactivation pertains to cells with neighboring place fields in a given environment (Wilson and McNaughton, 1994; Kudrimoti et al., 1999; Diba and Buzsáki, 2007). In contrast, the ventral striatum predominantly reactivates motivational information (Lansink et al., 2008). Neurons that first expressed a firing-rate change associated with expecting or consuming a liquid reward showed stronger reactivation than neurons not showing a modulation.
If replay across hippocampus and ventral striatum contributes to consolidation of place–reward associations, the different pieces of information belonging to the same event should be reprocessed coherently to prevent formation of erroneous associations with other events. Simultaneous recordings from hippocampal and ventral striatal neuronal ensembles in freely moving rats indeed demonstrated cross-structural reactivation of combined spatial and motivational information (Lansink et al., 2009). Spatial information carried by hippocampal cells is reactivated preferentially in advance of motivational information processed by ventral striatal neurons during postbehavioral rest (Fig. 3). This preferential order (“hippocampus first, ventral striatum second”) could not be explained from a similar preferred order in the behavioral phase. This is the only direct evidence to date of a temporally leading role of the hippocampus in cross-structural replay. Cross-structural replay appears compressed by a factor 10 compared with the behavioral time scale of neuronal activation. Hence, several seconds of “real-time” behavioral segments containing joint place–reward information are brought together in a time frame of hundreds of milliseconds during sleep. Thus, in joint replay activity of hippocampal cells that are directly or indirectly presynaptic to the ventral striatum advances that in striatal cells in a time window allowing long-term synaptic modifications to occur (Markram et al., 1997; Abbott and Nelson, 2000). This capture of spatial and motivational information in narrow time windows may offer a suitable mechanism for storage of place–reward associations.
Testing computational models of reward learning against corticostriatal physiology
Theories of motivated behavior have drawn much inspiration from the field of reinforcement learning (Sutton and Barto, 1998; Schultz, 2000; Daw and Doya, 2006). Temporal-difference reinforcement learning (TDRL) models, which learn the values of available actions in specific situations (states) to maximize reward, need to track actual reward receipt, state value, and prediction error. An influential view is that cortical areas provide a state signal, DLS associates states with actions, prediction errors are signaled by midbrain dopamine neurons, and ventral striatum encodes state value (Mishkin et al., 1984; Hikosaka et al., 1989; Houk et al., 1995; Schultz et al., 1997; O'Doherty et al., 2004). In such an “actor/critic” arrangement, the ventral striatum is not involved in on-line action selection, but rather serves to train action values located elsewhere, a suggestion supported by a recent lesion study in the rat (Atallah et al., 2007) and consistent with value-related signals found in human fMRI studies (Knutson and Cooper, 2005; Preuschoff et al., 2006) (but see Hare et al., 2008) as well as monkey recording work (Schultz et al., 1992; Cromwell and Schultz, 2003). In the rat literature, reports of reward-responsive neurons in ventral striatum are abundant, but these have generally not yet been related to value [but see Taha and Fields (2005) and Ito and Doya (2009)].
While a “critic” role for the ventral striatum is theoretically attractive, work in rats has long established that the ventral striatum is more directly involved in action selection (Mogenson et al., 1980). A wide range of pharmacological, lesion, and recording studies broadly supports the view that the ventral striatum plays a key role in mediating certain motivated actions, such as responding to cues in autoshaping and Pavlovian-instrumental transfer (Ikemoto and Panksepp, 1999; Cardinal et al., 2002; Nicola, 2007). These and other behaviors, such as outcome-dependent, deliberative decision making (Balleine and Dickinson, 1998) cannot be accommodated in the basic actor/critic TDRL model (Dayan, 2002). More powerful “model-based” approaches rely instead on the representation and evaluation of action outcomes before they are chosen (Daw et al., 2005; Niv et al., 2006; Johnson et al., 2007). Signals related to action outcomes have been identified in orbitofrontal cortex and hippocampus (Schoenbaum et al., 1998; Johnson and Redish, 2007;Ramus et al., 2007), and lesions of these areas impair flexible decision making requiring such nonlocal information (Morris, 2007; Murray et al., 2007). Both orbitofrontal cortex and hippocampus project to ventral striatum, suggesting that the latter may be involved in on-the-fly, internal evaluation (Dehaene and Changeux, 2000; Niv et al., 2006; Johnson et al., 2007) of afferent outcome representations. Such a scheme could reconcile experimental evidence for value representations in the ventral striatum with its direct involvement in action selection via a role for evaluation in such settings.
Is there any evidence for the activation of ventral striatal reward signals during decision making? Building on earlier work (Peoples et al., 2004; Roitman et al., 2004; German and Fields, 2007), van der Meer and Redish (2009a) showed that neurons in rat ventral striatum that responded to reward tended to fire during deliberative decision making as well. Because this effect disappeared with repeated experience and was also apparent during “reversals” at different points on the maze, it cannot be straightforwardly explained by a cue-based account. A similarly dynamic reward signal was present in ventral striatal local field potentials: 80 Hz oscillations not only ramped up to reward receipt, but were also increased at the high-cost choice point (van der Meer and Redish, 2009b). Further work is needed to determine whether these representations of reward during decision making are related to value, and how their timing relates to cortical and hippocampal outcome signals. Thus, ventral striatal gamma oscillations may provide a mechanism through which integration of action outcome and reward information could take place.
Corticostriatal interactions in habit learning
Components of the neocortical–basal ganglia loops are essential for learned actions to become habitual, and abnormal activity within these loops is implicated in a range of clinical disorders related to action compulsion (as in obsessive-compulsive spectrum disorders and drug addiction) and action disability (as in Parkinson's disease and Huntington's disease) (Graybiel, 2008).
Corticobasal ganglia pathways associated with the sensorimotor striatum are thought to be necessary for the transition of instrumental behavior into habits, a process suggested to involve the “chunking” of complex action sequences into units that are rapidly executable, fluid, and robust to changes in outcome contingency (i.e., stimulus–response based) (Dickinson, 1985; Kimura, 1995; Voorn et al., 2004; Poldrack et al., 2005; Yin and Knowlton, 2006; Graybiel, 2008; Balleine et al., 2009; Belin et al., 2009). Graybiel and colleagues have identified neuronal firing patterns in the rat sensorimotor striatum that may be a signature of activity related to this learning and chunking process (Jog et al., 1999; Barnes et al., 2005). Tetrode recordings of MSNs were made in the DLS as rats were trained and then overtrained on a conditional T-maze task. Initially, the average activity of task-related DLS neurons was distributed throughout the maze-running period as rats explored and learned task contingencies. As performance improved and reached asymptote, DLS ensemble spike activity was restructured and continued to evolve during overtraining: the firing of task-related neurons became predominant at the beginning and end of the maze runs and lessened during midrun (relating perhaps to a decrease in firing to the action sequence itself) (Tang et al., 2007), as though the behavior were becoming chunked. In parallel, there was a decline in the average firing of neurons not exhibiting phasic task-related responses. These patterns have been suggested as neural correlates of the explore–exploit behavior of the animals undergoing procedural learning and habit formation. Subsequent extinction of the overtrained behavior was slow, and there was a striking return of firing to resemble patterns recorded during initial learning. Reacquisition rapidly reinstated the task- and non-task-related activity patterns recorded during overtraining. These results indicate that habit formation and modification do not involve turning on and off a striatal “habit system,” but rather a dynamic repatterning of neural activity (Barnes et al., 2005).
Other regions of the brain have also been implicated in mediating action chunking or habit formation, including cerebellar circuits, sensorimotor and limbic cortices, and the substantia nigra (Hikosaka et al., 2002; Aldridge et al., 2004; Faure et al., 2005; Poldrack et al., 2005; Graybiel, 2008; Doyon et al., 2009). The ventral part of medial PFC (termed infralimbic in the rat; IL) has been shown in lesion/microinjection studies to be required for stimulus–response habit formation and maintenance, and neurons there to fire in relation to learned actions or associative signals (Coutureau and Killcross, 2003; Killcross and Coutureau, 2003; Mulder et al., 2003; Peters et al., 2005; Hitchcott et al., 2007). It is not clear how this IL region interacts with the DLS in promoting habitual behavior, as they are thought not to be directly connected, nor whether the IL and DLS serve redundant, parallel, or complementary functions during habit learning. One possibility currently being examined is that limbic–cognitive corticostriatal pathways (involving IL) support repetition of learned behavior or override cognitive goal-directed signals, while sensorimotor pathways (involving DLS) directly represent action chunks and stimulus–response rules (Fig. 4) (Coutureau and Killcross, 2003; Graybiel, 2008; Balleine et al., 2009).
Role of interneurons in transforming cortical input to striatal output
The striatum receives inputs from a very broad range of cortical areas, but has far fewer neurons. How are the rich patterns of cortical information transformed as they are compressed and manipulated within striatal microcircuits? While some features of cortical activity patterns are preserved during intrastriatal calculations, the resulting representations by single striatal output neurons appear to be quite different. For example, many medial/ventral striatal neurons show strong entrainment to the hippocampal theta rhythm (Berke et al., 2004). These cells are likely components of a PFC–hippocampal–striatal macrocircuit important for the use of contextual information in decision making; participation in theta rhythms may regulate communication within this macrocircuit. However, even theta-entrained medial striatal cells do not show the spatially linked firing rate change of hippocampal “place cells”; rather, their firing rate is modulated by nonspatial behavioral sequences (Berke et al., 2009).
One approach to deconstructing information processing within striatal microcircuits involves examining the activity patterns of distinct neuronal components. One such component, the FSIs, comprises only ∼1% of striatal neurons (Luk and Sadikot, 2001). However, these cells have been argued to have a critical role filtering out unwanted actions, and postmortem studies have found a deficit in striatal FSIs in Tourette syndrome (Kalanithi et al., 2005). Several groups studied putative FSIs in awake, behaving animals, identifying them on the basis of characteristic brief waveforms, tonic activity, and high-frequency discharges (Berke et al., 2004; Berke, 2008; Gage et al., 2008; Schmitzer-Torbert and Redish, 2008). In rats, FSIs show complex temporal sequences of activity during choice tasks, and do not normally act as a coordinated cell population with closely linked firing rates, even within striatal microregions (Berke, 2008). Thus FSIs are likely involved in the details of intrastriatal calculations, rather than providing a relatively uniform control signal (Fig. 5). This was surprising, not least because adjoining FSIs are electrically coupled together by dendritic gap junctions (e.g., Koós and Tepper, 1999; Fukuda, 2009)—although recent simulations suggest that these gap junctions do not normally synchronize the striatal FSI network (Hjorth et al., 2009). While individual FSIs can show phase locking to a range of cortical–striatal rhythms, including ∼80 Hz LFP oscillations provoked by rewards (Berke, 2005, 2009), such entrainment also appears to reflect particular combinations of cortical inputs rather than a general feature of striatal organization.
Despite such idiosyncratic firing, there are some situations in which FSIs do show more coordinated activity. Gage et al. (2008) recorded from multiple basal ganglia and cortical regions simultaneously in a striatum-dependent choice task. Rats had to maintain their position in a nose-poke hole, listen for an instruction tone, hold on for a further “go” signal, then make a rapid left or right movement to receive reward. As they initiated their chosen movement, there was a brief pulse of enhanced FSI activity, that was not seen in other neuronal populations or in conjunction with other movements. This FSI pulse may occur when there is a transient need to suppress alternative highly practiced movements, an interesting possibility in light of the Tourette syndrome findings. FSIs receive a specific but poorly understood backprojection from the globus pallidus (Fig. 5) (Bevan et al., 1998), and the population firing rates of FSIs and pallidal cells were markedly anticorrelated. Thus it appears that FSI are receiving unique combinations of inputs from cortex that may influence the fine timing of MSN spiking, while receiving and broadcasting a simpler, more coordinated feedback signal from globus pallidus.
Summary
Altogether, this mini-symposium will highlight a few of the key advances in research on corticostriatal circuits. Whereas these circuits were previously viewed as primarily operating in parallel, recent findings emphasize the interactions between corticobasal ganglia pathways via several modes of cross talk. The striatum does not appear to operate merely as an “active filter” that converts cortical cognitive, motor, or emotional information into actions, but is deeply involved in learning and memory processes itself, as illustrated by contextual conditioning paradigms, joint hippocampal–ventral striatal replay during sleep and repatterning of neural activity in DLS during overtraining. Activity of fast-spiking interneurons supports the generation of the striatum's own, differentiated dynamics, which is nonetheless coordinated with that in the cortex. Computationally, the dorsal and ventral striatum have been cast into an actor–critic architecture, but current systems neurophysiology suggests a more complicated scheme that emphasizes action value representations.
Footnotes
The work described in this review was supported by Human Frontier Science Program Grant RGP0127/2001, Netherlands Organisation for Scientific Research-VICI Grant 918.46.609, and European Union Grant FP7 #217148 (to C.M.A.P.); the Tourette Syndrome Association and National Institutes of Health (NIH) Grants DA14318 and MH12908 (to J.B.); NIH Grant MH60379 (to A.M.G.); Wellcome Trust Grant WT 078197 (to R.I.); NIH Grant R01 MH080318 (to A.D.R.); and NIH Grant F32-MH085454 (to K.S.S.).
References
- Abbott LF, Nelson SB. Synaptic plasticity: taming the beast. Nat Neurosci. 2000;3(Suppl):1178–1183. doi: 10.1038/81453. [DOI] [PubMed] [Google Scholar]
- Aldridge JW, Berridge KC, Rosen AR. Basal ganglia neural mechanisms of natural movement sequences. Can J Physiol Pharmacol. 2004;82:732–739. doi: 10.1139/y04-061. [DOI] [PubMed] [Google Scholar]
- Alexander GE, Crutcher MD. Functional architecture of basal ganglia circuits: neural substrates of parallel processing. Trends Neurosci. 1990;13:266–271. doi: 10.1016/0166-2236(90)90107-l. [DOI] [PubMed] [Google Scholar]
- Alexander GE, Crutcher MD, DeLong MR. Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, “prefrontal” and “limbic” functions. Prog Brain Res. 1990;85:119–146. [PubMed] [Google Scholar]
- Atallah HE, Lopez-Paniagua D, Rudy JW, O'Reilly RC. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci. 2007;10:126–131. doi: 10.1038/nn1817. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 1998;37:407–419. doi: 10.1016/s0028-3908(98)00033-1. [DOI] [PubMed] [Google Scholar]
- Balleine BW, Liljeholm M, Ostlund SB. The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res. 2009;199:43–52. doi: 10.1016/j.bbr.2008.10.034. [DOI] [PubMed] [Google Scholar]
- Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature. 2005;437:1158–1161. doi: 10.1038/nature04053. [DOI] [PubMed] [Google Scholar]
- Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res. 2009;199:89–102. doi: 10.1016/j.bbr.2008.09.027. [DOI] [PubMed] [Google Scholar]
- Berke JD. Participation of striatal neurons in large-scale oscillatory networks. In: Bolam JP, Ingham CA, Magill PJ, editors. The basal ganglia VIII. New York: Springer; 2005. pp. 25–35. [Google Scholar]
- Berke JD. Uncoordinated firing rate changes of striatal fast-spiking interneurons during behavioral task performance. J Neurosci. 2008;28:10075–10080. doi: 10.1523/JNEUROSCI.2192-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berke JD. Fast oscillations in cortical-striatal networks switch frequency following rewarding events and stimulant drugs. Eur J Neurosci. 2009;30:848–859. doi: 10.1111/j.1460-9568.2009.06843.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Berke JD, Okatan M, Skurski J, Eichenbaum HB. Oscillatory entrainment of striatal neurons in freely moving rats. Neuron. 2004;43:883–896. doi: 10.1016/j.neuron.2004.08.035. [DOI] [PubMed] [Google Scholar]
- Berke JD, Breck JT, Eichenbaum H. Striatal versus hippocampal representations during win-stay maze performance. J Neurophysiol. 2009;101:1575–1587. doi: 10.1152/jn.91106.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bevan MD, Booth PA, Eaton SA, Bolam JP. Selective innervation of neostriatal interneurons by a subclass of neuron in the globus pallidus of the rat. J Neurosci. 1998;18:9438–9452. doi: 10.1523/JNEUROSCI.18-22-09438.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Buzsáki G. Two-stage model of memory trace formation: a role for “noisy” brain states. Neuroscience. 1989;31:551–570. doi: 10.1016/0306-4522(89)90423-5. [DOI] [PubMed] [Google Scholar]
- Cador M, Robbins TW, Everitt BJ. Involvement of the amygdala in stimulus-reward associations: interaction with the ventral striatum. Neuroscience. 1989;30:77–86. doi: 10.1016/0306-4522(89)90354-0. [DOI] [PubMed] [Google Scholar]
- Cardinal RN, Parkinson JA, Hall J, Everitt BJ. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 2002;26:321–352. doi: 10.1016/s0149-7634(02)00007-6. [DOI] [PubMed] [Google Scholar]
- Chudasama Y, Robbins TW. Functions of frontostriatal systems in cognition: comparative neuropsychopharmacological studies in rats, monkeys and humans. Biol Psychol. 2006;73:19–38. doi: 10.1016/j.biopsycho.2006.01.005. [DOI] [PubMed] [Google Scholar]
- Coutureau E, Killcross S. Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats. Behav Brain Res. 2003;146:167–174. doi: 10.1016/j.bbr.2003.09.025. [DOI] [PubMed] [Google Scholar]
- Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol. 2003;89:2823–2838. doi: 10.1152/jn.01014.2002. [DOI] [PubMed] [Google Scholar]
- Daw ND, Doya K. The computational neurobiology of learning and reward. Curr Opin Neurobiol. 2006;16:199–204. doi: 10.1016/j.conb.2006.03.006. [DOI] [PubMed] [Google Scholar]
- Daw ND, Niv Y, Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci. 2005;8:1704–1711. doi: 10.1038/nn1560. [DOI] [PubMed] [Google Scholar]
- Dayan P. Motivated reinforcement learning. In: Dietterich TG, Becker S, Ghahramani Z, editors. Advances in neural information processing systems 14. Cambridge, MA: MIT; 2002. pp. 11–18. [Google Scholar]
- Deacon RM, Penny C, Rawlins JN. Effects of medial prefrontal cortex cytotoxic lesions in mice. Behav Brain Res. 2003;139:139–155. doi: 10.1016/s0166-4328(02)00225-5. [DOI] [PubMed] [Google Scholar]
- de Bruin JP, Moita MP, de Brabander HM, Joosten RN. Place and response learning of rats in a Morris water maze: differential effects of fimbria fornix and medial prefrontal cortex lesions. Neurobiol Learn Mem. 2001;75:164–178. doi: 10.1006/nlme.2000.3962. [DOI] [PubMed] [Google Scholar]
- Dehaene S, Changeux JP. Reward-dependent learning in neuronal networks for planning and decision making. Prog Brain Res. 2000;126:217–229. doi: 10.1016/S0079-6123(00)26016-0. [DOI] [PubMed] [Google Scholar]
- Diba K, Buzsáki G. Forward and reverse hippocampal place-cell sequences during ripples. Nat Neurosci. 2007;10:1241–1242. doi: 10.1038/nn1961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dickinson A. Actions and habits: the development of behavioral autonomy. Philos Trans R Soc Lond B Biol Sci. 1985;308:67–78. [Google Scholar]
- Doyon J, Bellec P, Amsel R, Penhune V, Monchi O, Carrier J, Lehéricy S, Benali H. Contributions of the basal ganglia and functionally related brain structures to motor learning. Behav Brain Res. 2009;199:61–75. doi: 10.1016/j.bbr.2008.11.012. [DOI] [PubMed] [Google Scholar]
- Euston DR, Tatsuno M, McNaughton BL. Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science. 2007;318:1147–1150. doi: 10.1126/science.1148979. [DOI] [PubMed] [Google Scholar]
- Everitt BJ, Morris KA, O'Brien A, Robbins TW. The basolateral amygdala-ventral striatal system and conditioned place preference: further evidence of limbic-striatal interactions underlying reward-related processes. Neuroscience. 1991;42:1–18. doi: 10.1016/0306-4522(91)90145-e. [DOI] [PubMed] [Google Scholar]
- Faure A, Haberland U, Condé F, El Massioui N. Lesion to the nigrostriatal dopamine system disrupts stimulus–response habit formation. J Neurosci. 2005;25:2771–2780. doi: 10.1523/JNEUROSCI.3894-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Floresco SB, Seamans JK, Phillips AG. Selective roles for hippocampal, prefrontal cortical, and ventral striatal circuits in radial-arm maze tasks with or without a delay. J Neurosci. 1997;17:1880–1890. doi: 10.1523/JNEUROSCI.17-05-01880.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fukuda T. Network architecture of gap junction-coupled neuronal linkage in the striatum. J Neurosci. 2009;29:1235–1243. doi: 10.1523/JNEUROSCI.4418-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gage GJ, Churchill MJ, Berke JD. Selective involvement of striatal fast-spiking interneurons during the choice between two learned actions. Soc Neurosci Abstr. 2008;34:578–15. [Google Scholar]
- German PW, Fields HL. Rat nucleus accumbens neurons persistently encode locations associated with morphine reward. J Neurophysiol. 2007;97:2094–2106. doi: 10.1152/jn.00304.2006. [DOI] [PubMed] [Google Scholar]
- Graybiel AM. Habits, rituals, and the evaluative brain. Annu Rev Neurosci. 2008;31:359–387. doi: 10.1146/annurev.neuro.29.051605.112851. [DOI] [PubMed] [Google Scholar]
- Graybiel AM, Aosaki T, Flaherty AW, Kimura M. The basal ganglia and adaptive motor control. Science. 1994;265:1826–1831. doi: 10.1126/science.8091209. [DOI] [PubMed] [Google Scholar]
- Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci. 2000;20:2369–2382. doi: 10.1523/JNEUROSCI.20-06-02369.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hare TA, O'Doherty J, Camerer CF, Schultz W, Rangel A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci. 2008;28:5623–5630. doi: 10.1523/JNEUROSCI.1309-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hikosaka O, Sakamoto M, Usui S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol. 1989;61:814–832. doi: 10.1152/jn.1989.61.4.814. [DOI] [PubMed] [Google Scholar]
- Hikosaka O, Nakamura K, Sakai K, Nakahara H. Central mechanisms of motor skill learning. Curr Opin Neurobiol. 2002;12:217–222. doi: 10.1016/s0959-4388(02)00307-0. [DOI] [PubMed] [Google Scholar]
- Hitchcott PK, Quinn JJ, Taylor JR. Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. Cereb Cortex. 2007;17:2820–2827. doi: 10.1093/cercor/bhm010. [DOI] [PubMed] [Google Scholar]
- Hjorth J, Blackwell KT, Kotaleski JH. Gap junctions between striatal fast-spiking interneurons regulate spiking activity and synchronization as a function of cortical activity. J Neurosci. 2009;29:5276–5286. doi: 10.1523/JNEUROSCI.6031-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hoffman KL, McNaughton BL. Coordinated reactivation of distributed memory traces in primate neocortex. Science. 2002;297:2070–2073. doi: 10.1126/science.1073538. [DOI] [PubMed] [Google Scholar]
- Houk J, Adams J, Barto A. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk JC, Davis JL, Beiser DG, editors. Models of information processing in the basal ganglia. Cambridge, MA: MIT; 1995. pp. 249–270. [Google Scholar]
- Ikemoto S, Panksepp J. The role of nucleus accumbens dopamine in motivated behavior: a unifying interpretation with special reference to reward-seeking. Brain Res Rev. 1999;31:6–41. doi: 10.1016/s0165-0173(99)00023-5. [DOI] [PubMed] [Google Scholar]
- Ito M, Doya K. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci. 2009;29:9861–9874. doi: 10.1523/JNEUROSCI.6157-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ito R, Robbins TW, McNaughton BL, Everitt BJ. Selective excitotoxic lesions of the hippocampus and basolateral amygdala have dissociable effects on appetitive cue and place conditioning based on path integration in a novel Y-maze procedure. Eur J Neurosci. 2006;23:3071–3080. doi: 10.1111/j.1460-9568.2006.04883.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ito R, Robbins TW, Pennartz CM, Everitt BJ. Functional interaction between the hippocampus and nucleus accumbens shell is necessary for the acquisition of appetitive spatial context conditioning. J Neurosci. 2008;28:6950–6959. doi: 10.1523/JNEUROSCI.1615-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ji D, Wilson MA. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat Neurosci. 2007;10:100–107. doi: 10.1038/nn1825. [DOI] [PubMed] [Google Scholar]
- Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM. Building neural representations of habits. Science. 1999;286:1745–1749. doi: 10.1126/science.286.5445.1745. [DOI] [PubMed] [Google Scholar]
- Johnson A, Redish AD. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J Neurosci. 2007;27:12176–12189. doi: 10.1523/JNEUROSCI.3761-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Johnson A, van der Meer MAA, Redish AD. Integrating hippocampus and striatum in decision-making. Curr Opin Neurobiol. 2007;17:692–697. doi: 10.1016/j.conb.2008.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kalanithi PS, Zheng W, Kataoka Y, DiFiglia M, Grantz H, Saper CB, Schwartz ML, Leckman JF, Vaccarino FM. Altered parvalbumin-positive neuron distribution in basal ganglia of individuals with Tourette syndrome. Proc Natl Acad Sci U S A. 2005;102:13307–13312. doi: 10.1073/pnas.0502624102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kesner RP, Ragozzino ME. The role of the prefrontal cortex in object-place learning: a test of the attribute specificity model. Behav Brain Res. 2003;146:159–165. doi: 10.1016/j.bbr.2003.09.024. [DOI] [PubMed] [Google Scholar]
- Killcross S, Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex. 2003;13:400–408. doi: 10.1093/cercor/13.4.400. [DOI] [PubMed] [Google Scholar]
- Kimura M. Role of basal ganglia in behavioral learning. Neurosci Res. 1995;22:353–358. doi: 10.1016/0168-0102(95)00914-f. [DOI] [PubMed] [Google Scholar]
- Knutson B, Cooper JC. Functional magnetic resonance imaging of reward prediction. Curr Opin Neurol. 2005;18:411–417. doi: 10.1097/01.wco.0000173463.24758.f6. [DOI] [PubMed] [Google Scholar]
- Koós T, Tepper JM. Inhibitory control of neostriatal projection neurons by GABAergic interneurons. Nat Neurosci. 1999;2:467–472. doi: 10.1038/8138. [DOI] [PubMed] [Google Scholar]
- Kudrimoti HS, Barnes CA, McNaughton BL. Reactivation of hippocampal cell assemblies: effects of behavioral state, experience, and EEG dynamics. J Neurosci. 1999;19:4090–4101. doi: 10.1523/JNEUROSCI.19-10-04090.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lacroix L, White I, Feldon J. Effect of excitotoxic lesions of rat medial prefrontal cortex on spatial memory. Behav Brain Res. 2002;133:69–81. doi: 10.1016/s0166-4328(01)00442-9. [DOI] [PubMed] [Google Scholar]
- Lansink CS, Goltstein PM, Lankelma JV, Joosten RN, McNaughton BL, Pennartz CM. Preferential reactivation of motivationally relevant information in the ventral striatum. J Neurosci. 2008;28:6372–6382. doi: 10.1523/JNEUROSCI.1054-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lansink CS, Goltstein PM, Lankelma JV, McNaughton BL, Pennartz CM. Hippocampus leads ventral striatum in replay of place–reward information. PLoS Biol. 2009;7:e1000173. doi: 10.1371/journal.pbio.1000173. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee AK, Wilson MA. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron. 2002;36:1183–1194. doi: 10.1016/s0896-6273(02)01096-6. [DOI] [PubMed] [Google Scholar]
- Luk KC, Sadikot AF. GABA promotes survival but not proliferation of parvalbumin-immunoreactive interneurons in rodent neostriatum: an in vivo study with stereology. Neuroscience. 2001;104:93–103. doi: 10.1016/s0306-4522(01)00038-0. [DOI] [PubMed] [Google Scholar]
- Markram H, Lübke J, Frotscher M, Sakmann B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science. 1997;275:213–215. doi: 10.1126/science.275.5297.213. [DOI] [PubMed] [Google Scholar]
- Marr D. Simple memory: a theory for archicortex. Philos Trans R Soc Lond B Biol Sci. 1971;262:23–81. doi: 10.1098/rstb.1971.0078. [DOI] [PubMed] [Google Scholar]
- Maurin Y, Banrezes B, Menetrey A, Mailly P, Deniau JM. Three-dimensional distribution of nigrostriatal neurons in the rat: relation to the topography of striatonigral projections. Neuroscience. 1999;91:891–909. doi: 10.1016/s0306-4522(98)00681-2. [DOI] [PubMed] [Google Scholar]
- McClelland JL, McNaughton BL, O'Reilly RC. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol Rev. 1995;102:419–457. doi: 10.1037/0033-295X.102.3.419. [DOI] [PubMed] [Google Scholar]
- McNaughton BL, Barnes CA, Battaglia FP, Bower MR, Cowen SL, Ekstrom AD, Gerrard JL, Hoffman KL, Houston FP, Karten Y, Lipa P, Pennartz CMA, Sutherland GR. Off-line reprocessing of recent memory and its role in memory consolidation: a progress report. In: Maquet P, Smith C, Stickgold R, editors. Sleep and brain plasticity. New York: Oxford UP; 2003. pp. 225–246. [Google Scholar]
- Mishkin M, Malamut B, Bachevalier J. Memories and habits: two neural systems. In: Lynch G, McGaugh JL, Weinberger NM, editors. Neurobiology of learning and memory. New York: Guilford; 1984. pp. 65–77. [Google Scholar]
- Mogenson GJ, Jones DL, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol. 1980;14:69–97. doi: 10.1016/0301-0082(80)90018-0. [DOI] [PubMed] [Google Scholar]
- Morris RGM. Theories of hippocampal function. In: Andersen P, Morris RGM, Amaral D, Bliss TVP, O'Keefe J, editors. The hippocampus book. Oxford UP: 2007. pp. 81–713. [Google Scholar]
- Mulder AB, Nordquist RE, Orgüt O, Pennartz CM. Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning. Behav Brain Res. 2003;146:77–88. doi: 10.1016/j.bbr.2003.09.016. [DOI] [PubMed] [Google Scholar]
- Murray EA, O'Doherty JP, Schoenbaum G. What we know and do not know about the functions of the orbitofrontal cortex after 20 years of cross-species studies. J Neurosci. 2007;27:8166–8169. doi: 10.1523/JNEUROSCI.1556-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nauta WJ, Smith GP, Faull RL, Domesick VB. Efferent connections and nigral afferents of the nucleus accumbens septi in the rat. Neuroscience. 1978;3:385–401. doi: 10.1016/0306-4522(78)90041-6. [DOI] [PubMed] [Google Scholar]
- Nicola SM. The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology (Berl) 2007;191:521–550. doi: 10.1007/s00213-006-0510-4. [DOI] [PubMed] [Google Scholar]
- Niv Y, Joel D, Dayan P. A normative perspective on motivation. Trends Cogn Sci. 2006;10:375–381. doi: 10.1016/j.tics.2006.06.010. [DOI] [PubMed] [Google Scholar]
- O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–454. doi: 10.1126/science.1094285. [DOI] [PubMed] [Google Scholar]
- O'Donnell P, Grace AA. Synaptic interactions among excitatory afferents to nucleus accumbens neurons: hippocampal gating of prefrontal cortical input. J Neurosci. 1995;15:3622–3639. doi: 10.1523/JNEUROSCI.15-05-03622.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parkinson JA, Willoughby PJ, Robbins TW, Everitt BJ. Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. Behav Neurosci. 2000;114:42–63. [PubMed] [Google Scholar]
- Pavlides C, Winson J. Influences of hippocampal place cell firing in the awake state on the activity of these cells during subsequent sleep episodes. J Neurosci. 1989;9:2907–2918. doi: 10.1523/JNEUROSCI.09-08-02907.1989. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pennartz CMA, Uylings HBM, Barnes CA, McNaughton BL. Memory reactivation and consolidation during sleep: from cellular mechanisms to human performance. Prog Brain Res. 2002;138:143–166. doi: 10.1016/S0079-6123(02)38076-2. [DOI] [PubMed] [Google Scholar]
- Pennartz CM, Lee E, Verheul J, Lipa P, Barnes CA, McNaughton BL. The ventral striatum in off-line processing: ensemble reactivation during sleep and modulation by hippocampal ripples. J Neurosci. 2004;24:6446–6456. doi: 10.1523/JNEUROSCI.0575-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Peoples LL, Lynch KG, Lesnock J, Gangadhar N. Accumbal neural responses during the initiation and maintenance of intravenous cocaine self-administration. J Neurophysiol. 2004;91:314–323. doi: 10.1152/jn.00638.2003. [DOI] [PubMed] [Google Scholar]
- Peters YM, O'Donnell P, Carelli RM. Prefrontal cortical cell firing during maintenance, extinction, and reinstatement of goal-directed behavior for natural reward. Synapse. 2005;56:74–83. doi: 10.1002/syn.20129. [DOI] [PubMed] [Google Scholar]
- Poldrack RA, Sabb FW, Foerde K, Tom SM, Asarnow RF, Bookheimer SY, Knowlton BJ. The neural correlates of motor skill automaticity. J Neurosci. 2005;25:5356–5364. doi: 10.1523/JNEUROSCI.3880-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Preuschoff K, Bossaerts P, Quartz SR. Neural differentiation of expected reward and risk in human subcortical structures. Neuron. 2006;51:381–390. doi: 10.1016/j.neuron.2006.06.024. [DOI] [PubMed] [Google Scholar]
- Qin YL, McNaughton BL, Skaggs WE, Barnes CA. Memory reprocessing in corticocortical and hippocampocortical neuronal ensembles. Philos Trans R Soc Lond B Biol Sci. 1997;352:1525–1533. doi: 10.1098/rstb.1997.0139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ragozzino ME, Adams S, Kesner RP. Differential involvement of the dorsal anterior cingulate and prelimbic-infralimbic areas of the rodent prefrontal cortex in spatial working memory. Behav Neurosci. 1998;112:293–303. doi: 10.1037//0735-7044.112.2.293. [DOI] [PubMed] [Google Scholar]
- Ramus SJ, Davis JB, Donahue RJ, Discenza CB, Waite AA. Interactions between the orbitofrontal cortex and the hippocampal memory system during the storage of long-term memory. Ann N Y Acad Sci. 2007;1121:216–231. doi: 10.1196/annals.1401.038. [DOI] [PubMed] [Google Scholar]
- Roitman MF, Stuber GD, Phillips PEM, Wightman RM, Carelli RM. Dopamine operates as a subsecond modulator of food seeking. J Neurosci. 2004;24:1265–1271. doi: 10.1523/JNEUROSCI.3823-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schmitzer-Torbert NC, Redish AD. Task-dependent encoding of space and events by striatal neurons is dependent on neural subtype. Neuroscience. 2008;153:349–360. doi: 10.1016/j.neuroscience.2008.01.081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenbaum G, Chiba AA, Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci. 1998;1:155–159. doi: 10.1038/407. [DOI] [PubMed] [Google Scholar]
- Schultz W. Multiple reward signals in the brain. Nat Rev Neurosci. 2000;1:199–207. doi: 10.1038/35044563. [DOI] [PubMed] [Google Scholar]
- Schultz W, Apicella P, Scarnati E, Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci. 1992;12:4595–4610. doi: 10.1523/JNEUROSCI.12-12-04595.1992. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
- Setlow B, McGaugh JL. Differential effects of immediate posttraining sulpiride microinfusions into the nucleus accumbens shell and core on Morris water maze retention. Psychobiology. 1999;27:248–255. [Google Scholar]
- Sutton RS, Barto AG. Reinforcement learning: an introduction. Cambridge, MA: MIT; 1998. [Google Scholar]
- Taha SA, Fields HL. Encoding of palatability and appetitive behaviors by distinct neuronal populations in the nucleus accumbens. J Neurosci. 2005;25:1193–1202. doi: 10.1523/JNEUROSCI.3975-04.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tang C, Pawlak AP, Prokopenko V, West MO. Changes in activity of the striatum during formation of a motor habit. Eur J Neurosci. 2007;25:1212–1227. doi: 10.1111/j.1460-9568.2007.05353.x. [DOI] [PubMed] [Google Scholar]
- van der Meer MAA, Redish AD. Covert expectation-of-reward in rat ventral striatum at decision points. Front Integr Neurosci. 2009a;3:1. doi: 10.3389/neuro.07.001.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- van der Meer MAA, Redish AD. Low and high gamma oscillations in rat ventral striatum have distinct relationships to behavior, reward, and spiking activity on a learned spatial decision task. Front Integr Neurosci. 2009b;3:9. doi: 10.3389/neuro.07.009.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- van Dongen YC, Deniau JM, Pennartz CM, Galis-de Graaf Y, Voorn P, Thierry AM, Groenewegen HJ. Anatomical evidence for direct connections between the shell and core subregions of the rat nucleus accumbens. Neuroscience. 2005;136:1049–1071. doi: 10.1016/j.neuroscience.2005.08.050. [DOI] [PubMed] [Google Scholar]
- Voorn P, Vanderschuren LJ, Groenewegen HJ, Robbins TW, Pennartz CM. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 2004;27:468–474. doi: 10.1016/j.tins.2004.06.006. [DOI] [PubMed] [Google Scholar]
- Wang GW, Cai JX. Disconnection of the hippocampal-prefrontal cortical circuits impairs spatial working memory performance in rats. Behav Brain Res. 2006;175:329–336. doi: 10.1016/j.bbr.2006.09.002. [DOI] [PubMed] [Google Scholar]
- Wilson MA, McNaughton BL. Reactivation of hippocampal ensemble memories during sleep. Science. 1994;265:676–679. doi: 10.1126/science.8036517. [DOI] [PubMed] [Google Scholar]
- Yin HH, Knowlton BJ. The role of the basal ganglia in habit formation. Nat Rev Neurosci. 2006;7:464–476. doi: 10.1038/nrn1919. [DOI] [PubMed] [Google Scholar]