Cortico-Basal Ganglia Reward Network: Microcircuitry

Abstract

Many of the brain's reward systems converge on the nucleus accumbens, a region richly innervated by excitatory, inhibitory, and modulatory afferents representing the circuitry necessary for selecting adaptive motivated behaviors. The ventral subiculum of the hippocampus provides contextual and spatial information, the basolateral amygdala conveys affective influence, and the prefrontal cortex provides an integrative impact on goal-directed behavior. The balance of these afferents is under the modulatory influence of dopamine neurons in the ventral tegmental area. This midbrain region receives its own complex mix of excitatory and inhibitory inputs, some of which have only recently been identified. Such afferent regulation positions the dopamine system to bias goal-directed behavior based on internal drives and environmental contingencies. Conditions that result in reward promote phasic dopamine release, which serves to maintain ongoing behavior by selectively potentiating ventral subicular drive to the accumbens. Behaviors that fail to produce an expected reward decrease dopamine transmission, which favors prefrontal cortical-driven switching to new behavioral strategies. As such, the limbic reward system is designed to optimize action plans for maximizing reward outcomes. This system can be commandeered by drugs of abuse or psychiatric disorders, resulting in inappropriate behaviors that sustain failed reward strategies. A fuller appreciation of the circuitry interconnecting the nucleus accumbens and ventral tegmental area should serve to advance discovery of new treatment options for these conditions.

INTRODUCTION

The neurotransmitter dopamine (DA) is released from neurons in the midbrain ventral tegmental area (VTA) that have widespread projections to regions known to be involved in reward processes and in guiding goal-directed behavior (Wise, 2004; Grace et al, 2007; Ikemoto, 2007). One area of the brain in which many of these systems converge is the nucleus accumbens (NAc). The NAc has a central role in the integration of cortical afferent systems under the modulatory influence of DA. In turn, the NAc and many of its inputs are also involved in directly or indirectly regulating DA neuron activity states. By examining the afferent drive of the NAc, its modulation by DA, and the afferent regulation of VTA DA cells, this article attempts to draw a functional circuit that illustrates the function of these two major structures in modulating behavioral responses that serve reward acquisition.

NUCLEUS ACCUMBENS

Connectivity

The NAc is part of the ventral striatal complex and serves as a critical region where motivations derived from limbic regions interface with motor control circuitry to regulate appropriate goal-directed behavior (Mogenson et al, 1980; Groenewegen et al, 1996; Nicola et al, 2000; Zahm, 2000; Wise, 2004). Like other parts of the striatal complex, the NAc receives extensive excitatory afferents from the cerebral cortex and thalamus. It projects to the ventral pallidum (VP), which innervates the mediodorsal and other thalamic divisions, thus completing cortico–striato–pallidal–thalamocortical loops (Zahm and Brog, 1992; O'Donnell et al, 1997). Together these structures form essential components of the circuitry that serves to optimize the behavioral response to rewards and conditioned associations. Alterations of synaptic transmission within various elements of this circuitry are strongly implicated in the development of addictive disorders (Kalivas et al, 2005; Robbins et al, 2008; Carlezon and Thomas, 2009).

Divisions

The NAc is divided into two major territories: the core is the central portion directly beneath and continuous with the dorsal striatum and surrounding the anterior commissure, and the shell occupies the most ventral and medial portions of the NAc. A third rostral pole division has also been identified (Zahm and Brog, 1992; Zahm and Heimer, 1993; Jongen-Rêlo et al, 1994). The NAc core and shell districts share striatal characteristics, in that approximately 90% of the cells are typical medium spiny projection neurons (Meredith, 1999). The remainder are local circuit interneurons, including cholinergic and parvalbumin cells (Kawaguchi et al, 1995). The NAc core and shell differ in their precise cellular morphology, neurochemistry, projection patterns, and functions (Heimer et al, 1991; Meredith et al, 1992; Zahm and Brog, 1992; Zahm and Heimer, 1993; Jongen-Rêlo et al, 1994; Meredith et al, 1996; Usuda et al, 1998; Meredith, 1999). The shell division, and particularly its medial aspect, is often more prominently associated with drug reward (Carlezon et al, 1995; Rodd-Henricks et al, 2002; Sellings and Clarke, 2003; Ikemoto, 2007), although the core also contributes to motivated behaviors that are cue-conditioned, including drug-seeking (Kalivas and McFarland, 2003; Robbins et al, 2008).

Superimposed on the core and shell subterritories of the NAc are compartments that at least partly resemble the patch and matrix organization of the dorsal striatum, the latter being based on the laminar patterns of cortical afferents and multiple specific biochemical markers (Gerfen, 1992). For the NAc, a simple patch–matrix organization has been difficult to define, and most authors agree that the compartmental segregation of cells and input–output channels in this region is highly complex (Voorn et al, 1989; Martin et al, 1991; Zahm and Brog, 1992; Jongen-Rêlo et al, 1993; Meredith et al, 1996; van Dongen et al, 2008).

Afferents: excitatory

Multiple limbic associated areas provide the excitatory cortical innervation to the NAc (Figure 1), including medial and lateral divisions of the prefrontal cortex (PFC), entorhinal cortex and ventral subiculum of the hippocampus (vSub), and basolateral amygdala (BLA) (Kelley and Domesick, 1982; Kelley et al, 1982; Groenewegen et al, 1987; Kita and Kitai, 1990; McDonald, 1991; Berendse et al, 1992; Brog et al, 1993; Totterdell and Meredith, 1997; Reynolds and Zahm, 2005). The NAc shell is innervated primarily by ventral portions of the prelimbic, infralimbic, medial orbital, and ventral agranular insular cortices, whereas the core receives input mainly from dorsal parts of the prelimbic cortex and dorsal agranular insular areas (Berendse et al, 1992; Brog et al, 1993). The vSub projects caudomedially with a preference for the NAc shell, whereas the dorsal subiculum projects to more rostrolateral regions including the core (Groenewegen et al, 1987; Brog et al, 1993). The BLA generates a complex rostral to core and caudal to shell topography that also varies according to patch–matrix compartments in the NAc (Wright et al, 1996).

Figure 1.

Principal afferents linking brain centers for goal-directed behavior with the NAc and VTA. For clarity, only some of the projections are shown, and the principal efferent pathways from the NAc are illustrated in Figure 2. Red indicates inhibitory structures and pathways, green excitatory connections, and yellow the modulatory influence of DA. Please refer to the text for detailed explanation. BLA, basolateral amygdala; LHA/LPOA, lateral hypothalamic and lateral preoptic areas; LHb, lateral habenula; Mid/Intral Thal, midline and intralaminar thalamic nuclei; NAc, nucleus accumbens; PAG/RF, periaqueductal gray and reticular formation; PFC, prefrontal cortex; PPTg/LDT, pedunculopontine and laterodorsal tegmentum; RMTg, mesopontine rostromedial tegmental nucleus; VP, ventral pallidum; vSub/Hipp, ventral subiculum of the hippocampus; VTA, ventral tegmental area.

Cortical neurons are the likely promoters of goal-directed behaviors, with the vSub providing spatial and contextual information, the PFC supplying executive control, including task switching and response inhibition, and the BLA communicating information regarding conditioned associations as well as affective drive (Moore et al, 1999; Wolf, 2002; Kalivas et al, 2005; Ambroggi et al, 2008; Ishikawa et al, 2008; Ito et al, 2008; Gruber et al, 2009a; Simmons and Neill, 2009). The NAc provides a crucial site for convergence of these various behavioral drives, although the relevant cortical structures also maintain interconnections with each other (Figure 1; Swanson and Köhler, 1986; Sesack et al, 1989; Jay et al, 1992; Brinley-Reed et al, 1995; Bacon et al, 1996; Pitkänen et al, 2000).

Thalamic afferents to the ventral striatum arise from midline and intralaminar nuclei (Figure 1), including the paraventricular, paratenial, intermediodorsal, central medial, rhomboid, reunions, and rostral parafascicular nuclei (Kelley and Stinus, 1984; Berendse and Groenewegen, 1990; Smith et al, 2004). In the rat and primate, the NAc core is innervated primarily by the intermediodorsal, the shell by the paraventricular, and the rostral pole by the paratenial nucleus (Berendse and Groenewegen, 1990; Smith et al, 2004). Some thalamic neurons innervating the NAc send collateral projections to the PFC (Otake and Nakamura, 1998). The functions of thalamostriatal projections are less well studied compared to corticostriatal pathways. Nevertheless, the former are likely to operate in arousal and directing attention to behaviorally significant events (Smith et al, 2004).

Afferents: inhibitory/modulatory

There are few strong inhibitory afferents to the NAc, although there are reciprocal GABA projections from the VP, other parts of the basal forebrain, and the VTA (Brog et al, 1993; Groenewegen et al, 1993; Churchill and Kalivas, 1994; Van Bockstaele and Pickel, 1995; Wu et al, 1996). The shell of the NAc also receives a projection from orexin (hypocretin) neurons in the lateral hypothalamus (Peyron et al, 1998). Although this peptide is often reported to be excitatory, it appears to have inhibitory actions on NAc neurons (Martin et al, 2002). Additional peptide-containing projections from the lateral hypothalamus express melanin-concentrating hormone (Bittencourt et al, 1992).

The NAc also receives modulatory afferents from the brainstem, including DA and GABA projections from the medial substantia nigra zona compacta (SNc) and VTA (Figure 1; see ‘Efferents' in section Ventral tegmental area) (Voorn et al, 1986; Van Bockstaele and Pickel, 1995; Ikemoto, 2007). The DA innervation forms an essential component of reward circuitry and is recruited by both natural rewards and psychostimulants (Koob, 1992; Wise, 2004; Ikemoto, 2007). The NAc also receives serotonin and non-serotonin inputs from the dorsal raphe nucleus (Van Bockstaele and Pickel, 1993; Brown and Molliver, 2000). There is a small norepinephrine projection from the locus coeruleus (LC) and nucleus of the solitary tract directed mainly to the NAc shell (Swanson and Hartman, 1975; Brog et al, 1993; Delfs et al, 1998) and additional sparse afferents from other brainstem regions, including the pedunculopontine tegmentum (PPTg), parabrachial nucleus, and periaqueductal gray (Brog et al, 1993).

Microcircuitry

Excitatory cortical afferents to the NAc typically synapse onto the spines of medium spiny neurons. Fewer synapse onto the dendrites of local circuit interneurons with preference for parvalbumin-containing GABA cells vs cholinergic neurons (Totterdell and Smith, 1989; Kita and Kitai, 1990; Meredith and Wouterlood, 1990; Meredith et al, 1990; Sesack and Pickel, 1990; Lapper and Bolam, 1992; Lapper et al, 1992; Sesack and Pickel, 1992b; Bennett and Bolam, 1994; Johnson et al, 1994; Totterdell and Meredith, 1997; Thomas et al, 2000; French and Totterdell, 2004; Smith et al, 2004; French et al, 2005). An important series of studies by French and Totterdell established that multiple sources of cortical innervation converge onto individual medium spiny neurons in the NAc. This was shown for PFC and vSub inputs as well as for BLA and vSub projections (French and Totterdell, 2002, 2003). The fact that both PFC and BLA afferents converge with vSub projections suggests that convergence is also likely to occur for PFC and BLA inputs to at least some medium spiny neurons, given the high degree of co-convergence reported. Physiological evidence also supports convergence of cortical inputs to medium spiny neurons, permitting temporal integration of excitatory drive (O'Donnell and Grace, 1995; Finch, 1996; McGinty and Grace, 2009) (see section Interaction between hippocampal and prefrontal inputs). It is possible that varying degrees of afferent convergence within the ventral striatum give rise to relatively segregated input–output channels that form functional ensembles (Pennartz et al, 1994; Groenewegen et al, 1999).

Accumulating evidence suggests that midline and rostral intralaminar thalamic structures synapse mainly onto dendritic spines in a manner similar to corticostriatal inputs, whereas caudal intralaminar thalamic nuclei more commonly contact the dendritic shafts of striatal and NAc neurons, including interneurons (Dubé et al, 1988; Meredith and Wouterlood, 1990; Lapper and Bolam, 1992; Sidibé and Smith, 1999; Smith et al, 2004).

Dopamine afferents to the NAc synapse onto GABA neurons (Pickel et al, 1988) with medium spiny morphology (Pickel and Chan, 1990; Smith et al, 1999). Whether DA axons also synapse onto local circuit neurons in the NAc has not been thoroughly investigated. There is one report of DA synapses onto the class of interneurons containing nitric oxide synthase (Hidaka and Totterdell, 2001). Careful ultrastructural analysis in the dorsal striatum has failed to reveal DA synaptic input to cholinergic cells (Pickel and Chan, 1990), which nevertheless express high levels of D2 receptors (Alcantara et al, 2003) and therefore respond to tonic DA levels in the extrasynaptic space (Wang et al, 2006).

For medium spiny neurons, the dendritic spines that receive excitatory synapses from cortical axon terminals sometimes also display inhibitory or modulatory-type synapses from DA axons. This has been demonstrated in the NAc for all three cortical afferent sources (Totterdell and Smith, 1989; Sesack and Pickel, 1990, 1992b; Johnson et al, 1994) in a manner similar to cortical projections to more dorsal striatal regions (Bouyer et al, 1984; Smith et al, 1994). The extent of this convergence is likely to be greater in the core than in the shell division (Zahm, 1992), given the less extensive dendritic trees of shell neurons (Meredith et al, 1992).

In the rat, convergence of DA and thalamostriatal projections has also been reported for the midline paraventricular innervation to the NAc shell (Pinto et al, 2003) and for presumed thalamostriatal projections labeled for the vesicular glutamate transporter type 2 (VGlut2) (Moss and Bolam, 2008). In the dorsal striatum of monkeys, caudal intralaminar thalamic afferents reportedly do not converge synaptically with DA axons onto common dendritic spines. However, this is likely to reflect the more proximal placement of synapses from this particular thalamic division (Smith et al, 1994, 2004).

The so-called triad of elements: spine, glutamate synapse, and DA synapse, creates the potential for DA to modulate discretely specific sources of glutamate transmission onto distal dendritic compartments as opposed to a more generalized effect on overall cell excitability. This structural configuration also enables presynaptic interactions between DA and glutamate by limiting the diffusion distance necessary for each transmitter to reach extrasynaptic receptors on the apposing nerve terminal (Moss and Bolam, 2008; Yao et al, 2008; Sesack, 2009).

On the other hand, dual synaptic convergence onto common spines is probably a relatively infrequent occurrence in the NAc, based on estimates of the dorsal striatum where they account for less than 10% of spines (Wilson et al, 1983). Moreover, not all of the spines that receive dual input may be innervated by DA axons. These observations suggest that the synapses of DA axons onto distal dendritic shafts, as opposed to spines (Pickel and Chan, 1990; Zahm, 1992), are also important for modulating discrete sources of glutamate transmission.

In contradiction to arguments favoring selective modulation of particular glutamate afferents, recent quantitative analyses suggest that DA axons in the striatum (and possibly by extension the NAc) are arranged to form a lattice network such that all parts of this region are within one micron of a DA synapse (Moss and Bolam, 2008). The importance of this suggestion is highlighted by reports that (1) DA receptors are predominantly extrasynaptic (Dumartin et al, 1998; Yao et al, 2008; Sesack, 2009), (2) DA communicates through volume transmission in addition to a synaptic mode (Descarries et al, 1996; Moss and Bolam, 2008), and (3) DA modulates the general excitability of striatal and NAc neurons (O'Donnell and Grace, 1996; Nicola et al, 2000; Surmeier et al, 2007).

Physiological data strongly support DA alterations of responses evoked by cortical afferents to NAc medium spiny neurons (Yang and Mogenson, 1984; O'Donnell and Grace, 1994; Nicola et al, 2000; Charara and Grace, 2003; O'Donnell, 2003; Brady and O'Donnell, 2004; Goto and Grace, 2005b) (see section Regulation of NAc activity and its role in reward). As discussed above, such modulatory actions may reflect specific synaptic or more generalized extrasynaptic effects. Nevertheless, the close convergence of DA and glutamate synapses onto spines or distal dendrites provides a potential substrate for enabling local plasticity of glutamate transmission based on synaptic experience (Flores et al, 2005; Day et al, 2006; Surmeier et al, 2007) or chronic exposure to psychostimulants that enhance DA levels (Robinson and Kolb, 2004; Wolf et al, 2004; Lee et al, 2006).

Based on information from studies of either dorsal or ventral striatum, medium spiny neurons appear to provide only weak inhibition of each other (Taverna et al, 2004; Tepper et al, 2008). However, a potential excitatory influence has been reported in the dorsal striatum based on peptide-induced facilitation of glutamatergic drive (Blomeley et al, 2009). Medium spiny neurons are more strongly and reciprocally connected to local circuit neurons (Izzo and Bolam, 1988; Pickel and Chan, 1990; Martone et al, 1992; Bennett and Bolam, 1994; Kawaguchi et al, 1995; Hussain et al, 1996; Taverna et al, 2007; Tepper et al, 2008), which are also interconnected with each other in the NAc (Hussain et al, 1996) and dorsal striatum (Kawaguchi et al, 1995). The innervation of local circuit neurons by cortical afferents to the striatum and NAc (see above) provides circuitry for feedforward inhibition of medium spiny cells. As shown in vitro or in anesthetized rats, this inhibition is powerful and influences multiple medium spiny neurons (Mallet et al, 2005; Tepper et al, 2008; Gruber et al, 2009b). However, during behavioral tasks in awake animals, the activity patterns of presumed striatal interneurons are highly variable and independent, suggesting that they contribute mainly to the specific details of striatal processing rather than the global coordination of firing (Berke, 2008).

Efferents

The major projections of the NAc are to the VP, substantia nigra, VTA, hypothalamus, and brainstem (Figure 2; Haber et al, 1990; Zahm and Heimer, 1990; Heimer et al, 1991; Usuda et al, 1998; Nicola et al, 2000; Zahm, 2000; Dallvechia-Adams et al, 2001). The NAc core projects primarily to the dorsolateral portion of the VP, the entopeduncular nucleus, and the substantia nigra zona reticulata (SNr). The shell mainly innervates the ventromedial VP division, substantia innominata, lateral hypothalamic area, lateral preoptic area, SNc, VTA, periaqueductal gray, parabrachial nucleus, and PPTg (Haber et al, 1990; Zahm and Heimer, 1990; Heimer et al, 1991; Usuda et al, 1998). The VP territories also project to some of the same targets, with the dorsolateral VP innervating mainly the SNr and subthalamic nucleus and the ventromedial VP projecting to the VTA, basal forebrain, and preoptic areas (Zahm, 1989; Zahm and Heimer, 1990). It should also be noted that projections of the NAc shell to the VTA influence DA cells that in turn project to the NAc core, creating a medial to lateral series of spiraling projections that allow limbic associated structures to influence transmission in successively more motor-related parts of basal ganglia circuitry. Evidence for this looped medial to lateral organization was first described in rats by Nauta in 1978 (Nauta et al, 1978) and later verified by others in rats and cats (Somogyi et al, 1981; Groenewegen and Russchen, 1984; Heimer et al, 1991; Zahm and Heimer, 1993). In the primate, where the functional subdivisions of the striatum are most discrete, the spiraling organization of striatonigral–striatal projections appears most refined and has been most thoroughly characterized (Haber et al, 2000).

Figure 2.

Hypothetical direct and indirect output pathways whereby the NAc core and shell may disinhibit or inhibit, respectively, adaptive motor pathways for maximizing reward acquisition. Only major projections are shown. Red indicates inhibitory structures and pathways, whereas green indicates excitatory connections. Please refer ‘Efferents' in section Nucleus Accumbens for detailed explanation. BF Hypoth, basal forebrain and hypothalamus; MD Thal, mediodorsal thalamic nucleus; NAc, nucleus accumbens; PFC, prefrontal cortex; SNr, substantia nigra zona reticulata; STN, subthalamic nucleus; VP dl/vm, ventral pallidum, dorsolateral, and ventromedial; VTA, ventral tegmental area.

Among the various outputs of the NAc and VP, a subset can be viewed as functionally analogous to the direct and indirect pathways that are involved in behavioral activation and response inhibition (Figure 2; Alexander et al, 1990). This organization is more striatal-like for the core than the shell division (Zahm, 1989; Zahm and Brog, 1992; Nicola et al, 2000). The direct pathway from the NAc core involves mainly projections to the SNr (Montaron et al, 1996) and from there to the mediodorsal thalamus. The dorsolateral VP, which is also targeted by the NAc core, appears to have only minor projections to the mediodorsal thalamus (Zahm et al, 1996; O'Donnell et al, 1997) but nevertheless mediates some direct actions on thalamic activity (Lavin and Grace, 1994). By the direct route, cortical activation of NAc neurons leads ultimately to disinhibition of appropriate action plans that facilitate reward acquisition. The indirect circuit travels through the dorsolateral VP and subthalamic nucleus before reaching the SNr (Figure 2). Cortical activation of this circuit is likely to inhibit motor plans that are maladaptive, either for obtaining reward or for avoiding punishment (Mink, 1996; Redgrave et al, 1999).

A simple division of NAc shell neurons into direct and indirect pathways is complicated by the fact that the shell is really a hybrid structure: part basal ganglia and part limbic region (Zahm, 1989; Zahm and Heimer, 1990; Heimer et al, 1991; Zahm and Brog, 1992). In addition to being a ventral extension of the striatum, with striatal cell types and input–output connections, the shell is also part of the extended amygdala complex with projections to hypothalamic and brainstem structures important for visceral motor control and affect (Alheid and Heimer, 1988; Waraczynski, 2006).

Despite these difficulties, some theories regarding direct and indirect pathways involving the NAc shell have been put forward (Figure 2). For example, it has been suggested that both direct and indirect projections might involve the ventromedial VP (Nicola et al, 2000), with the direct circuit contacting cells that project to the mediodorsal thalamus (O'Donnell et al, 1997) and the indirect projections involving VP neurons that subsequently project to the subthalamic nucleus. Alternatively, parts of the basal forebrain and hypothalamus may serve the role of output structures for visceral motor functions, with projections to them arising directly from the NAc (and producing inhibition) or indirectly through the VP (and ultimately producing disinhibition) (Nicola et al, 2000). However, the fact that these targets have only minor projections to primarily nonspecific thalamic nuclei strains comparisons to more dorsal parts of basal ganglia circuitry (Heimer et al, 1991; O'Donnell et al, 1997; Zahm, 2006).

A third possibility is that the direct and indirect pathways from the NAc shell converge on the VTA, which may act as a basal ganglia output structure via projections to the mediodorsal thalamus. The direct pathway would proceed from the NAc to the VTA, whereas the indirect route would first involve the connection to the ventromedial VP and then its projections to the VTA. Although VTA DA neurons project only weakly to the thalamus in the rat (Groenewegen, 1988), they provide extensive innervation of midline thalamic structures in the monkey (Sánchez-González et al, 2005; Melchitzky et al, 2006). Moreover, non-DA cells appear to participate in these projections in both rats and primates (Sánchez-González et al, 2005; Melchitzky et al, 2006; Del-Fava et al, 2007). Although not yet directly tested, it is likely that many of these are GABA VTA neurons serving as traditional basal ganglia output cells.

In the dorsal striatum, the direct and indirect output pathways are also distinguished by the expression of different DA receptor subtypes, with D1 receptors being the dominant subclass in direct pathway striatal neurons and D2 receptors expressed principally by indirect pathway cells (Gerfen et al, 1990; Surmeier et al, 2007; Sesack, 2009). This distinction is most evident in anatomical studies (Hersch et al, 1995; Le Moine and Bloch, 1995; Deng et al, 2006), whereas electrophysiological recordings tend to report cells responding to selective agonists for both receptors (Uchimura et al, 1986; Surmeier et al, 1992; Cepeda et al, 1993). Aspects of this controversy have been resolved by the finding that many striatal medium spiny neurons have the capacity to express mixed receptor subtypes from the extended D1 (D1 or D5) and D2 (D2, D3, or D4) families (Surmeier et al, 1996) and by the discovery that complex indirect mechanisms can explain some instances of apparent physiological coexpression of D1 and D2 receptors (Wang et al, 2006; Surmeier et al, 2007).

Different populations of NAc medium spiny neurons also appear to express D1 or D2 receptors selectively (Le Moine and Bloch, 1996; Lee et al, 2006), although this segregation is less complete as compared to the dorsal striatum. Moreover, the greater overall expression of DA D3 receptors in NAc neurons (Le Moine and Bloch, 1996) indicates a greater likelihood of mixed physiological response patterns (Uchimura et al, 1986) in this region. In general, D2 receptors are expressed mainly in NAc neurons that project to the VP and rarely in those that innervate the midbrain, whereas D1 receptors are expressed in both cell populations (Robertson and Jian, 1995; Lu et al, 1997, 1998).

Regulation of NAc Activity and its Role in Reward

Modulation by DA

Dopamine exerts multiple and complex effects on neurons within the striatal complex. DA acting on D2 receptors potently inhibits NAc neurons (White and Wang, 1986; Lin et al, 1996; O'Donnell and Grace, 1996). In contrast, D1 receptor stimulation potentiates glutamatergic drive (Cepeda et al, 1998; Chergui and Lacey, 1999; West and Grace, 2002). Confirmatory data derive from examining the effects of locally applied antagonists in vivo, such that D2 antagonists increase NAc neuron firing and D1 antagonists decrease cell excitability (West and Grace, 2002). Moreover, studies have shown that DA potently modulates gap junction interactions among NAc neurons by increasing synchrony among neurons (Onn and Grace, 1994; Onn et al, 2000). Such an effect is likely to be particularly effective in the lateral transmission of slow membrane voltage changes, such as those occurring during NAc neuron ‘up' states (O'Donnell and Grace, 1995). Therefore, DA has multifaceted effects in both altering NAc neuronal activity and modulating the balance of afferent inputs and their integration, presumably in a manner that most effectively shapes goal-directed behavior.

Ventral subiculum inputs

Neurons within the NAc when recorded in vivo are known to exhibit up–down states (O'Donnell and Grace, 1995). The up states appear to function as a gating mechanism, in that neurons only discharge action potentials from the depolarized up state. The up states are driven by afferent input from the vSub of the hippocampus (O'Donnell and Grace, 1995). The vSub is well positioned to provide such a modulatory gating influence. The vSub receives afferent inputs from a number of regions related to (1) affect, eg the amygdala and LC (Oleskevich et al, 1989; Schroeter et al, 2000; French et al, 2003); (2) spatial location, eg dorsal hippocampus/CA1 (Amaral et al, 1991); and (3) higher cognitive functions, eg indirect inputs from the PFC (O'Mara, 2005). The vSub itself is involved in the central regulation of stress (Herman and Mueller, 2006) and in context-dependent behaviors (Jarrard, 1995; Maren, 1999; Sharp, 1999; Fanselow, 2000). Thus, by integrating spatial and affective information, the vSub is positioned to provide information regarding the affective valence of locations in space, which would be critical in evaluating context-dependent processes. Indeed, several events in which context is important, such as context-dependent fear conditioning (Fanselow, 2000; Maren and Quirk, 2004), the behavioral responses to stress (Bouton and Bolles, 1979; Bouton and King, 1983), or amphetamine sensitization (Vezina et al, 1989; Badiani et al, 2000; Crombag et al, 2000), are disrupted by inactivation of the vSub (Lodge and Grace, 2008; Valenti and Grace, 2008).

The vSub drive of NAc neurons is potently modulated by the DA system. In particular, D1 agonists increase vSub drive of NAc neurons. This is likely because of an effect on the NAc neuron itself rather than a presynaptic action, given the results of paired-pulse experiments (Goto and Grace, 2005b) and the lack of presynaptic D1 receptors within the striatum (Hersch et al, 1995). This afferent modulation is affected primarily by phasic DA release (Grace, 1991; Goto and Grace, 2005b) driven by DA neuron burst firing (Grace, 1991). Given that DA neurons emit phasic bursts of spikes when exposed to stimuli signaling a rewarding event (Schultz, 1998b), the ability of bursts to potentiate vSub–NAc transmission is expected to be involved in selecting reward-related behavior. The DA input does indeed affect the vSub–NAc projection in a behaviorally salient manner. Thus, when the vSub is disconnected from the NAc by unilaterally inactivating the vSub and injecting a D1 antagonist into the contralateral NAc, there is a disruption in the acquisition of learned behavior in the rat (Goto and Grace, 2005b). In addition to DA modulation, the vSub input is also disrupted by psychotomimetic drugs such as phencyclidine. Administration of behaviorally effective doses of phencyclidine potently attenuates vSub-driven up states in NAc neurons (O'Donnell and Grace, 1998).

The drive of the NAc by the vSub also exhibits plasticity in response to repeated activation. Thus, tetanic stimulation of the vSub leads to long-term potentiation (LTP) within the vSub–NAc pathway. This is also dependent on D1 receptor stimulation, given that blockade of D1 receptors prevents the induction of LTP (Goto and Grace, 2005a). Moreover, LTP induction is NMDA dependent (Goto and Grace, 2005a).

Prefrontal cortical inputs

The medial prefrontal cortex (mPFC) also has glutamatergic inputs to the NAc. However, its impact is strongly dependent on the timing of its activation. Brief stimulation of the mPFC produces an excitatory potential within the NAc (O'Donnell and Grace, 1993, 1994); moreover, this mPFC input is potently and selectively attenuated by D2 receptor stimulation that acts presynaptically on mPFC terminals (O'Donnell and Grace, 1994; West et al, 2002). This D2 receptor stimulation is promoted primarily by tonic DA levels within the NAc that in turn are dependent on DA neuron population activity (Floresco et al, 2003; Goto and Grace, 2005b). In contrast to the vSub input, disconnection of the mPFC from the NAc (by unilateral inactivation of the mPFC and stimulation of D2 receptors in the contralateral NAc) does not interfere with learning a task, which is presumably more dependent on the vSub–NAc pathway. However, such disconnection does interfere with switching strategies (Goto and Grace, 2005b). In contrast, using paired-pulse stimulation, it is clear that activation of the mPFC also induces a subsequent inhibitory potential that decreases NAc neuronal excitability (O'Donnell and Grace, 1993).

Tetanic stimulation of the mPFC also induces LTP within the mPFC–NAc pathway; although the characteristics of the LTP are different from those evoked by vSub stimulation. Specifically, in addition to being attenuated by D2 stimulation, the induction of LTP in the mPFC–NAc pathway is not dependent on NMDA receptors (Goto and Grace, 2005a).

Amygdala inputs

The BLA comprises the third major input to the NAc. The amygdala is a region involved in expression of emotion and in learned emotional behaviors (LeDoux, 2000). This afferent is also glutamatergic in nature (O'Donnell and Grace, 1995; Charara and Grace, 2003; French and Totterdell, 2003) and produces a long latency, long duration excitation within the NAc (O'Donnell and Grace, 1995) that is modulated by D1 receptors (Charara and Grace, 2003). The BLA also has potent interactions with other components of the limbic system. For example, it provides strong excitatory drive to the vSub (Lipski and Grace, 2008) and to the mPFC. The BLA–mPFC projection is important in affective conditioning processes (Laviolette and Grace, 2006). Thus, neurons in the mPFC that are excited by the BLA exhibit potent excitation by stimuli associated with aversive events (Laviolette et al, 2005; McGinty and Grace, 2008). Moreover, this response depends on an intact DA input to the mPFC (Laviolette et al, 2005). In turn, the mPFC provides a powerful attenuation of BLA activation by sensory stimuli as shown both electrophysiologically (Rosenkranz and Grace, 2001, 2002) and in human imaging studies (Hariri et al, 2003). Thus, both the recognition of salience and the learned response to affective stimuli depend on the BLA–mPFC interaction.

Interaction between hippocampal and prefrontal inputs

The synaptic convergence of vSub, BLA, and mPFC inputs onto the same sets of NAc neurons (O'Donnell and Grace, 1995; French and Totterdell, 2002, 2003), and their common modulation by DA, provides strong evidence that the NAc serves as a crossroads for integration of information about environmental context and affect with higher cognitive processes. Moreover, the vSub and the mPFC exhibit complex interactions within the NAc that impact goal-directed behavior. The nature of these interactions is strongly dependent on the timing of the inputs. Thus, vSub stimulation potently promotes NAc neuron firing both by evoking EPSPs and by inducing up states (O'Donnell and Grace, 1995). However, the ability of the vSub to promote the NAc is apparently dependent on more than the direct vSub–NAc projection. The vSub also projects to the mPFC, which, in turn, projects to the NAc. If the mPFC is inactivated, there is a strong attenuation of the ability of the vSub to drive the NAc (Belujon and Grace, 2008). On the other hand, if the vSub–NAc pathway is stimulated at high frequency, facilitation by the mPFC is no longer required. Thus, the mPFC provides a ‘permissive' role in vSub–NAc drive and synaptic plasticity. Alternately, if the mPFC is stimulated first, it will attenuate the vSub drive by activation of local inhibitory circuits (O'Donnell and Grace, 1993; Goto and O'Donnell, 2002). Therefore, if the input from the vSub arrives first, the mPFC will facilitate this drive; however, if the mPFC is first activated, the vSub afferent input is attenuated.

The vSub and the NAc also exhibit dynamic interactions with respect to activation history. As reviewed above, high-frequency stimulation of either the vSub or the mPFC will induce LTP in the respective pathways. However, the vSub and mPFC also exhibit competition between these afferent systems. Thus, high-frequency stimulation of the vSub will not only induce LTP in the vSub–NAc pathway but will also induce long-term depression (LTD) in the mPFC–NAc pathway. Subsequent high-frequency stimulation of the mPFC reverses this condition, causing induction of LTP in the mPFC–NAc pathway while producing LTD in the vSub–NAc pathway. Therefore, activation of one afferent system will attenuate afferent drive from the alternate system (Goto and Grace, 2005a). This balance is further modulated by DA, with increases in DA favoring the vSub–NAc pathway and decreases in DA favoring the mPFC–NAc pathway. Such a condition could have important implications with respect to reward-related behaviors.

As reviewed above, the vSub–NAc pathway is proposed to maintain responding on a learned task, whereas the mPFC–NAc pathway facilitates switching to novel response strategies. It has been shown that behaviors that lead to reinforcement are associated with activation of DA neuron firing (Schultz, 1998b). Thus, a reinforced behavior would lead to DA release, followed by D1 receptor-mediated potentiation of vSub–NAc drive to reinforce ongoing behavior. At the same time, DA release would produce a D2 receptor-mediated attenuation of mPFC–NAc drive and so reduce mPFC-mediated task switching. Conversely, when the response strategy becomes ineffective, there would be a drop in DA neuron activity (Hollerman and Schultz, 1998; Schultz and Dickinson, 2000). Such a decrease in DA transmission would then be predicted to attenuate vSub-mediated drive of ongoing behavior while disinhibiting mPFC-mediated behavioral flexibility. This would be expected to cause the animal to switch from their current, ineffective behavioral strategy and test new strategies. Once a new strategy is found to be effective, the subsequent reinforcement-driven activation of the DA system would strengthen the new behavior by attenuating the mPFC input and facilitating vSub maintenance of activity (Goto and Grace, 2008).

Role of the dorsal striatum in reward learning

Studies have demonstrated a role for DA in the ventral striatum in the acquisition and in the expression of appetitive responses and motivation (Montague et al, 2004). There is increasing evidence that the dorsal striatum is important in reward-related processes. In particular, studies have suggested that the dorsal striatum is involved in instrumental behavior and in habit formation. Thus, the initial reinforcement of appetitive and drug stimuli activates ventral striatal structures (Bonson et al, 2002; Yin et al, 2008); however, with repetitive exposure, activation of more dorsal striatal structures will predominate (Robbins and Everitt, 2002; Yin et al, 2008). This transition from reinforcement to habit formation is believed to be under frontal cortical control (Berke, 2003) and enables an animal to exert cognitive influence over adaptive decision-making. Thus, with repeated exposure to drugs of abuse, there is progressive activation of more dorsal striatal areas (Porrino et al, 2004; Saka et al, 2004), and this transition is accompanied by a similar shift in DA release (Ito et al, 2002; Wong et al, 2006). Such a transition can be facilitated by the interconnected loops of the DA–striatal system, in which limbic activation affects progressively more cognitive and motor regions of the striatal loop (see ‘Efferents' in section Nucleus accumbens).

VENTRAL TEGMENTAL AREA

Connectivity

Dopamine and particularly its projections to the ventral striatal complex are strongly implicated in the facilitation of approach behaviors and incentive learning (Horvitz, 2000; Wise, 2004; Fields et al, 2007; Ikemoto, 2007; Schultz, 2007; Redgrave et al, 2008). The above-cited reports indicate that activity of DA neurons is influenced by a host of novel stimuli that are initially unpaired with behavioral outcomes but are potentially salient by virtue of their high intensity and fast onset. DA neurons also respond to unexpected natural rewards and to conditioned cues that predict reward. DA release in forebrain regions may be involved in both the response to reward, and the facilitation of motivated actions that lead to reward in the future. Consequently, DA has a greater impact on instrumental behavior than on actual consumption (Wise, 2004). DA is particularly important for learning how certain behaviors lead to reward, and animals with DA depletion either cannot learn such associations or fail to maintain them (Wise and Rompre, 1989; Wise, 2004). The DA projection to the NAc also contributes to the rewards associated with drugs of abuse (Koob, 1992; Wise, 2004; Ikemoto, 2007), and plasticity in this system is strongly implicated in addictive disorders that involve compulsive drug-seeking (Wolf et al, 2004; Zweifel et al, 2008).

Neurons

Dopamine neurons make up about 60–65% of the cells in the VTA (Swanson, 1982; Nair-Roberts et al, 2008). They are highly heterogeneous and vary by location, morphological characteristics, forebrain targets, afferent influences, firing properties, and content of calcium-binding proteins, ion channels, autoreceptors, DA transporter, and other molecular features (Smith et al, 1996; Sesack and Carr, 2002; Björklund and Dunnett, 2007; Lammel et al, 2008; Margolis et al, 2008). Non-DA neurons in the ventral midbrain are primarily GABAergic and make up approximately 30–35% of the cells in the VTA (Swanson, 1982; Mugnaini and Oertel, 1985; Steffensen et al, 1998; Nair-Roberts et al, 2008). Although they are often referred to as interneurons, the predominant evidence indicates that these cells issue long-range projections that parallel those of DA neurons (Figure 1; Swanson, 1982; Van Bockstaele and Pickel, 1995; Steffensen et al, 1998; Carr and Sesack, 2000a). The functions of these GABA projections from the VTA have not yet been fully explored. Electrophysiological and anatomical evidence indicates that VTA GABA neurons also have local axon collaterals that innervate neighboring cells (Johnson and North, 1992; Nugent and Kauer, 2008; Omelchenko and Sesack, 2009).

Recently, a population of glutamate neurons also has been discovered in the VTA but not the SNc (Hur and Zaborszky, 2005; Kawano et al, 2006; Yamaguchi et al, 2007; Descarries et al, 2008). These appear to comprise approximately 2–3% of VTA neurons (Nair-Roberts et al, 2008). The detailed connectivity of these cells will take time to decipher, given their low numbers and the fact that they can only be detected by in situ hybridization for VGlut2 mRNA, a selective marker of subcortical glutamate neurons (Herzog et al, 2001). Nevertheless, glutamatergic VTA cells have been shown to project at least to the PFC (Hur and Zaborszky, 2005) as well as locally (Dobi and Morales, 2007). A portion of VTA glutamate neurons also contains DA, and although some electrophysiological studies are interpreted as providing evidence of extensive colocalization of these transmitters (Chuhma et al, 2004; Lavin et al, 2005), this is not supported by anatomical studies. Rather, estimates regarding the degree of colocalization of DA and glutamate markers vary from as much as 20–50% in some VTA subdivisions (Kawano et al, 2006) to as little as 2% of all DA cells in the adult rat VTA (Yamaguchi et al, 2007). The extent of colocalization also appears to be developmentally regulated (Descarries et al, 2008), being more extensive in perinatal animals and considerably diminished in adults. A clear delineation of the extent to which DA and glutamate are colocalized in the various projections of the VTA and the functional significance of such colocalization continue to be important topics for investigation.

Efferents

Dopamine and GABA cells in the ventral midbrain form a lateral to medial continuum and, in the rat at least, project in a roughly topographic manner to multiple forebrain regions with minimal branching but considerable overlap in the terminal fields (Fallon and Moore, 1978; Nauta et al, 1978; Beckstead et al, 1979; Swanson, 1982; Loughlin and Fallon, 1983; Deutch et al, 1988; Van Bockstaele and Pickel, 1995; Gaykema and Záborszky, 1996; Carr and Sesack, 2000a; Hasue and Shammah-Lagnado, 2002; Björklund and Dunnett, 2007; Del-Fava et al, 2007; Ikemoto, 2007; Lammel et al, 2008). There are also considerable interconnections between subdivisions of the nigra–VTA complex, which have recently been elegantly described by Shammah-Lagnado and co-workers (Ferreira et al, 2008); the majority of these intra-areal connections are likely to be non-dopaminergic (Dobi and Morales, 2007; Ferreira et al, 2008; Omelchenko and Sesack, 2009).

Cells in the SNc project primarily to the striatal complex, although the most extreme lateral portion of the SNc projects to the amygdala (Loughlin and Fallon, 1983). More medially positioned neurons at the border between the SNc and VTA project more ventrally within the basal ganglia (ie to the NAc), and cells in this region also project to the septum and other parts of the basal forebrain, olfactory tubercle, and amygdala (Swanson, 1982; Loughlin and Fallon, 1983; Gaykema and Záborszky, 1996; Hasue and Shammah-Lagnado, 2002; Björklund and Dunnett, 2007; Ikemoto, 2007; Lammel et al, 2008). Projections from the SNc and VTA also reach the pallidum and subthalamic nucleus (Klitenick et al, 1992; Gaykema and Záborszky, 1996; Hasue and Shammah-Lagnado, 2002; Björklund and Dunnett, 2007; Smith and Villalba, 2008). Within the VTA proper are DA and GABA neurons that project to the prefrontal, cingulated, and perirhinal cortices; some cortically projecting cells are also localized within the SNc (Carr and Sesack, 2000a; Björklund and Dunnett, 2007).

The medial most rostral linear VTA subdivision projects extensively to the olfactory tubercle, VP, preoptic and lateral hypothalamic areas, lateral habenular complex, mediodorsal thalamus, and supraoculomotor region; minor projections include the PFC, BLA, and dorsal raphe (Klitenick et al, 1992; Gaykema and Záborszky, 1996; Del-Fava et al, 2007). The majority of projections from the rostral linear nucleus appear to be non-dopaminergic (Swanson, 1982; Del-Fava et al, 2007). The caudomedial (ie caudolinear) and ventromedial VTA regions innervate the bed nucleus of the stria terminalis, the pallidum and basal forebrain, the central amygdaloid nucleus, and the BLA (Hasue and Shammah-Lagnado, 2002; Del-Fava et al, 2007).

In the primate, the relative segregation of cortically projecting neurons as arising from the VTA is not observed, and DA cells in the SNc have been shown to have cortical projections (Williams and Goldman-Rakic, 1998; Björklund and Dunnett, 2007). Moreover, the DA innervation to the cortex is more extensive in monkeys and humans, particular in primary motor areas (Lewis and Sesack, 1997). Regarding striatal pathways, a medial to lateral spiraling topography that also involves reciprocal feedback projections has been reported (see ‘Efferents' in section Nucleus accumbens). Interestingly, as one shifts from medial to lateral cell groups in the rodent, the proportion of GABA neurons that project in parallel to the DA neurons declines, from as much as 60% in the mesoprefrontal projection to 15–35% in the mesoaccumbens projection and 5–15% in the nigrostriatal pathway (Swanson, 1982; Van Bockstaele and Pickel, 1995; Rodríguez and González-Hernández, 1999; Carr and Sesack, 2000a). Such contributions of GABA neurons to the ascending projections of the ventral midbrain have not been well studied in the monkey.

Recent tracing studies in the rat suggest that the projections of the VTA can be parceled in various ways depending on anatomical, physiological, and molecular features. An overall mediolateral topography in the efferent projections to the forebrain has long been recognized (Fallon and Moore, 1978; Beckstead et al, 1979). More recently, Ikemoto (2007) has put forward a model in which mesostriatal projections originating from the VTA consist of two major divisions: (1) a posterior division with projections to the medial, striatal portion of the olfactory tubercle and the medial NAc shell; and (2) a lateral VTA region projecting to the NAc core, lateral shell, and lateral olfactory tubercle. These observations can be interpreted within a broader literature detailing the stronger drug reward associations of the posteromedial vs anterior VTA as well as the medial NAc shell and olfactory tubercle as compared to the other striatal regions (Ikemoto, 2007).

Historically, the ascending DA projections have also been divided into two portions along a dorsoventral dimension: (1) a dorsal tier of cells expressing low DA transporter and substantial calbindin projects to the cortex, ventral striatum (especially the NAc shell), limbic structures, and the striatal matrix; and (2) a ventral tier of neurons projects mainly to the striatal patch compartment and comprises neurons with higher DA transporter and lower calbindin levels (Gerfen, 1992; Haber et al, 1995; Björklund and Dunnett, 2007).

Finally, Lammel (Lammel et al, 2008) working in mouse brain has divided midbrain DA cells according to their forebrain targets and physiological characteristics, with (1) fast-spiking DA cells also expressing low ratios of mRNA for DA transporter vs TH and projecting to the PFC, NAc core, medial NAc shell, and BLA; and (2) slow-firing DA neurons projecting to the lateral NAc shell and dorsolateral striatum. It will be important in future studies to determine exactly how each of these different population groupings contributes to the functions of the broader DA system.

Afferents: excitatory

The VTA receives input from widely distributed brain areas that have been described as forming a continuous band of afferent neurons not organized into discrete nuclei (Geisler and Zahm, 2005). This band stretches from the PFC to the medullary brainstem and follows the path of the medial forebrain bundle through the lateral hypothalamus. Cells in these areas have the morphological features and connectivity characteristic of the ‘isodendritic core' originally attributed to the brainstem reticular formation (Ramón-Moliner and Nauta, 1966; Geisler and Zahm, 2005). Moreover, many of these structures provide only a modest input to the VTA, but they innervate other regions that are also afferent to the VTA. These observations imply that VTA neuronal activity is unlikely to be influenced by a discrete set of brain structures, and rather that DA neurons are regulated by an integrated network of inputs (Geisler and Zahm, 2005).

For years, the VTA was thought to have excitatory afferents from only a few sources. The bulk of the cortical mantle does not project to brainstem structures. Moreover, the hippocampus also has no direct projection to the brainstem, despite mediating an important physiological influence on VTA DA neurons (see ‘Limbic modulation of VTA DA neuron activity' in section Ventral tegmental area). Hence, the only major cortical projection to the VTA originates from the PFC (Figure 1), including mainly the prelimbic and infralimbic cortices and less robustly the cingulate and orbital divisions (Beckstead, 1979; Phillipson, 1979a; Sesack et al, 1989; Sesack and Pickel, 1992b; Geisler and Zahm, 2005; Frankle et al, 2006; Geisler et al, 2007). The function of the PFC to VTA pathway in reward circuitry is unclear, although it appears to mediate an essential regulation of plasticity in DA neurons that can be altered by repeated exposure to drugs of abuse or stress (Wolf, 1998; Wolf et al, 2004).

Ultrastructural tract-tracing indicates that PFC axons synapse onto DA neurons that project back to the PFC, creating a circuit that allows the PFC to regulate the extent of its modulatory feedback by DA (Carr and Sesack, 2000b). Evidence has not been obtained for synapses from the prelimbic and infralimbic PFC onto mesoaccumbens DA neurons (Carr and Sesack, 2000b), although these cells might receive cortical input from structures outside the medial regions that were examined by tract-tracing (Geisler et al, 2007; Omelchenko and Sesack, 2007). GABA VTA neurons are also innervated by PFC synapses, and these appear to project mainly to the NAc as opposed to the PFC (Carr and Sesack, 2000b). Other populations of VTA DA or GABA cells defined by target projection have not yet been examined with regard to PFC synaptic input.

The PPTg and laterodorsal tegmentum (LDT) also provide an important input to the SNc and VTA (Figure 1; Lavoie and Parent, 1994; Oakman et al, 1995; Charara et al, 1996; Mena-Segovia et al, 2008). Within the ventral midbrain, the VTA is innervated by the LDT and caudal PPTg, whereas the SNc is innervated primarily by the rostral PPTg (Mena-Segovia et al, 2008). The SNr receives only minimal input. Synapses from the PPTg/LDT contact both DA and non-DA GABAergic neurons within the VTA of both rat and monkey (Charara et al, 1996; Omelchenko and Sesack, 2005). Ultrastructural evidence suggests that these projections originate from cholinergic, glutamatergic, and GABAergic neurons (Charara et al, 1996; Garzón et al, 1999; Omelchenko and Sesack, 2005, 2006). The use of tract-tracing in combination with immunocytochemistry reveals that probable glutamatergic and cholinergic LDT neurons synapse onto DA cells that project to the NAc (Omelchenko and Sesack, 2005, 2006). These findings are consistent with neurochemical observations that blockade of cholinergic and glutamatergic receptors in the VTA alters the ability of PPTg/LDT stimulation to evoke DA release in the NAc (Blaha et al, 1996; Forster and Blaha, 2000).

The results of anatomical studies are also consistent with electrophysiological evidence that the PPTg–LDT complex excites DA cells and promotes burst firing (Futami et al, 1995; Lokwan et al, 1999; Floresco et al, 2003; Lodge and Grace, 2006b) (see ‘Limbic modulation of VTA DA neuron activity' in section Ventral tegmental area). The fact that inhibitory responses are less often recorded, even though GABA cells make up 30–40% of PPTg/LDT neurons (Wang and Morales, 2009), may relate to observations that inhibitory-type synapses from this region tend to innervate VTA GABA neurons more often than DA cells (Omelchenko and Sesack, 2005). Such a disinhibitory organization might be expected to facilitate the recruitment of burst firing in DA neurons.

Recently, the seminal work of Geisler et al (2007) using retrograde tract-tracing in combination with in situ hybridization for VGlut subtypes has revealed multiple sources of glutamate afferents to the VTA, many of which had not been previously appreciated. Afferents expressing VGlut1 derive primarily from the medial and lateral PFC, including the prelimbic, infralimbic, dorsal peduncular, cingulate, and orbital cortices. VGlut2-containing afferents derive from multiple subcortical sites, including in relative order of predominance: the lateral hypothalamus, lateral preoptic area, periaqueductal gray, medial hypothalamus, VP, mesopontine reticular formation, lateral habenula, PPTg/LDT, and other regions (Figure 1). The presumed monosynaptic excitatory glutamate influence of the bed nucleus of the stria terminalis (Georges and Aston-Jones, 2002) has only scant confirmation by anatomical analysis (Geisler et al, 2007). Afferents to the VTA expressing VGlut3, a marker that has not yet been proven to correlate with glutamate transmission, arise primarily from raphe nuclei (Geisler et al, 2007). The revelation of so many new sources of glutamate input to the VTA has important implications for understanding how information related to reward behavior reaches this brain region. Nevertheless it will take time to delineate the functional role that each of these new projections mediates.

The findings of Geisler et al are consistent with ultrastructural data indicating that the dominant sources of glutamate afferents to the VTA are VGlut2 containing and therefore from non-cortical structures (Omelchenko and Sesack, 2007). Axons containing VGlut2 synapse extensively onto mesoaccumbens DA neurons, suggesting that many different brain regions contribute to the activation of one of the main pathways implicated in control of motivated behaviors. Mesoprefrontal DA cells also receive VGlut2 afferents, but a significant portion of their synapses is from VGlut1-containing axons, consistent with their more selective innervation from the PFC (Carr and Sesack, 2000b).

Some excitatory influences of the VTA are driven by peptides as opposed to classical neurotransmitters. For example, orexin afferents from the hypothalamus (Fadel et al, 2002) mediate an importance influence on reward behaviors (Harris et al, 2005) and synaptic plasticity (Borgland et al, 2006) presumably by excitatory actions on DA cells (Korotkova et al, 2003). The anatomical substrates for this influence are not yet clear, given that few orexin axons actually synapse within the VTA, and only half of these contact DA cells (Balcita-Pedicino and Sesack, 2007). Neurotensin- and corticotropin-releasing factors from multiple sources also mediate important excitatory influences on VTA DA cells (Geisler and Zahm, 2006; Reynolds et al, 2006; Rodaros et al, 2007; Tagliaferro and Morales, 2008; Wanat et al, 2008).

Afferents: inhibitory/modulatory

A complete list of sources of inhibitory GABA signals to the VTA has not yet been delineated in the same thorough manner as glutamate inputs. Nevertheless, a major inhibitory feedback from the basal ganglia is well known and likely to constitute the bulk of the inhibitory synapses in the VTA (Geisler and Zahm, 2005) as in the SNc (Somogyi et al, 1981; Smith and Bolam, 1990). These projections arise from the NAc shell and VP (Zahm and Heimer, 1990; Heimer et al, 1991; Zahm et al, 1996; Usuda et al, 1998). Additional inhibitory afferents to the VTA are likely to arise from the lateral hypothalamus and other hypothalamic regions, diagonal band, bed nucleus, lateral septum, periaqueductal gray, PPTg/LDT, parabrachial, and raphe nuclei (Geisler and Zahm, 2005). Many of these projections also contain neuroactive peptides and mediate complex actions on midbrain neurons (Sesack and Pickel, 1992a; Pickel et al, 1993; Dallvechia-Adams et al, 2002; Ford et al, 2006). A substantial projection exists from the central nucleus of the amygdala to the lateral SNc (ie reciprocal to the source of DA input to the amygdala) (Gonzales and Chesselet, 1990; Zahm, 2006), but only occasional fibers from this or any other amygdala division reach the medial SNc or VTA in the rat (Zahm et al, 2001; Geisler and Zahm, 2005; Zahm, 2006). In the primate, one study reported a robust projection from the central amygdaloid nucleus to the VTA (Fudge and Haber, 2000), although another paper described this connection as modest (Price and Amaral, 1981). These findings might represent an interesting and important species difference; nevertheless a substantial projection from the central amygdala to the VTA remains to be confirmed (for important technical considerations see Zahm, 2006).

In addition to the well-known afferents listed above, a new major ascending source of inhibition to the SNc and VTA has only recently been discovered and named. The mesopontine rostromedial tegmental nucleus (RMTg) lies just caudal to the VTA, dorsomedial to the medial lemniscus, dorsolateral to the interpeduncular nucleus, and lateral to the median raphe (Jhou et al, 2009b; Kaufling et al, 2009). It receives afferents from many forebrain and brainstem structures (Jhou et al, 2009b), consists primarily of GABA cells (Perrotti et al, 2005; Olson and Nestler, 2007; Kaufling et al, 2009), and has extensive projections to the entire SNc–VTA complex (Figure 1; Colussi-Mas et al, 2007; Ferreira et al, 2008; Geisler et al, 2008; Jhou et al, 2009b). It is therefore in a critical position to inhibit DA cell firing in response to aversive stimuli (Grace and Bunney, 1979; Ungless et al, 2004; Jhou et al, 2009a) or when expected rewards are not delivered (Schultz, 1998b). The latter influence is likely to arise first in the lateral habenula, which is activated by the absence of reward (Matsumoto and Hikosaka, 2007), has projections to the VTA and RMTg (Herkenham and Nauta, 1979; Araki et al, 1988; Bell et al, 2007; Jhou et al, 2009b; Kaufling et al, 2009), and mediates a nearly ubiquitous inhibitory influence on DA cell activity (Ji and Shepard, 2007; Matsumoto and Hikosaka, 2007; Hikosaka et al, 2008). Cells in the RMTg are activated by stress and by psychostimulant exposure (Perrotti et al, 2005; Colussi-Mas et al, 2007; Jhou and Gallagher, 2007; Geisler et al, 2008; Jhou et al, 2009a, 2009b; Kaufling et al, 2009), indicating that the RMTg may be a critical structure regulating the responses of DA cells to natural and drug rewards as well as their converse events.

In addition to the various extrinsic sources of inhibition, VTA DA neurons also receive inhibitory synapses from neighboring GABA cells (Figure 1). Such inputs have been reported in light microscopic and physiology studies (Grace and Bunney, 1979; Phillipson, 1979b; Grace and Onn, 1989; Johnson and North, 1992; Nugent and Kauer, 2008) but only recently confirmed by ultrastructural analysis (Omelchenko and Sesack, 2009). The local collaterals of GABA neurons also synapse onto GABA cells (Omelchenko and Sesack, 2009), creating the potential circuitry for disinhibitory actions on DA neurons (Celada et al, 1999; Fields et al, 2007).

The VTA also receives afferents from other brainstem monoamine groups that produce variable actions on target neurons depending on receptor type. Serotonin neurons in the dorsal raphe nucleus synapse onto DA cells (Hervé et al, 1987; Van Bockstaele et al, 1994) and mediate primarily inhibition (Gervais and Rouillard, 2000), although excitatory actions are also reported (Pessia et al, 1994). The ventral midbrain also receives inputs from the LC and other medullary norepinephrine cell groups (Liprando et al, 2004; Geisler and Zahm, 2005; Mejías-Aponte et al, 2009). Either excitatory or inhibitory actions of norepinephrine are produced on DA cells, mediated by α-1 and α-2 receptors, respectively, as well as more complex indirect actions (Grenhoff et al, 1995; Arencibia-Albite et al, 2007; Guiard et al, 2008). These inputs provide a pathway for visceral and homeostatic information to reach DA and non-DA cells in the VTA.

In summary, the VTA receives a rich assortment of influences from multiple ascending and descending and even intrinsic sources. The functional significance of each afferent in relation to reward has yet to be determined. For example, it is not known how the sensory information regarding receipt of an unpredicted reward reaches DA neurons. It is also unclear by what route visual and auditory information influence DA cell firing when these serve as conditioning cues that predict reward. Certainly VTA DA cells fire in response to visual cues in a manner that correlates with activity in neurons of the superior colliculus (Coizet et al, 2003; Dommett et al, 2005). However, the projection from the superior colliculus to the VTA is considerably weaker than its input to the SNc, and it is also not entirely glutamatergic (Comoli et al, 2003; Geisler and Zahm, 2005; Geisler et al, 2007). This raises the possibility that there are alternative pathways for sensory information to reach the VTA that remain to be elucidated.

Regulation of VTA Activity and its Role in Reward

Limbic modulation of VTA DA neuron activity

Dopamine neurons are known to exhibit different states of activity that depend on their intrinsic properties and afferent drive. The baseline activity of DA neurons is driven by a pacemaker conductance that brings the neuronal membrane potential from a very hyperpolarized state to its relatively depolarized spike threshold (Grace and Bunney, 1983, 1984b; Grace and Onn, 1989). This pacemaker conductance is responsible for the baseline activity of the neurons, which is then modulated up or down from this state. Although this pacemaker conductance causes the DA neurons to fire in a highly regular pacemaker pattern in vitro (Grace and Onn, 1989), this pattern is replaced by an irregular pattern when it is distorted by the constant bombardment of GABA IPSPs (Grace and Bunney, 1985). However, studies have shown that not all of the DA neurons in the SNc/VTA are firing spontaneously. Thus, evidence shows that a majority of DA neurons in anesthetized (Bunney and Grace, 1978; Grace and Bunney, 1984b) or awake (Freeman et al, 1985) animals are in a hyperpolarized, non-firing state. This is apparently because of a powerful inhibitory input arising from the VP. The VP, in turn, is under the inhibitory control of the NAc. The proportion of DA neurons firing spontaneously, which is termed the ‘population activity', depends primarily on the vSub inputs to the NAc; thus, the vSub will drive NAc inhibition of the VP, and thereby disinhibit DA neurons (Floresco et al, 2001, 2003). The role of the vSub in controlling the number of DA neurons firing spontaneously is consistent with its overall function in context-dependent processing, in that the state of activation of DA neurons can potently modulate the attentional state of the organism.

In addition to being modulated between a silent, non-firing state, and a condition of irregular activity, DA neurons can also exhibit burst firing. Burst firing is induced in DA neurons whenever behaving animals encounter a behaviorally salient stimulus such as one predicting reward (Schultz, 1998a). Burst firing is dependent on a glutamatergic drive of DA neurons acting on NMDA receptors (Grace and Bunney, 1984a; Chergui et al, 1993). The most potent driver of mesolimbic DA neuron burst firing appears to derive from glutamatergic afferents arising from the PPTg (Floresco and Grace, 2003; Lodge and Grace, 2006a). Moreover, the LDT provides a permissive gate over the ability of the PPTg to induce burst firing (Lodge and Grace, 2006b). Thus, the PPTg/LDT drives the behaviorally salient burst discharge of DA neurons. However, in order for this NMDA-mediated burst firing to take place, the DA neuron must be in a spontaneously firing condition (Floresco et al, 2003). The spontaneous firing state is dependent on input from the vSub–NAc–VP–VTA pathway (Figure 3). Thus, only neurons placed in a spontaneously firing state by the vSub system can respond to the PPTg with a burst of spikes. In this situation, the PPTg provides the behaviorally salient ‘signal,' whereas the vSub provides the amplification factor, or ‘gain,' of this signal (Lodge and Grace, 2006a; Figure 3). The higher the activity of the vSub, the larger the number of DA neurons that can be driven into a burst firing mode.

Figure 3.

DA neurons in the VTA can exist in several activity states. In the basal, unstimulated state, DA neurons fire spontaneously at a slow, irregular rate. The VP provides a potent GABAergic input to DA neurons, causing a proportion of them to be tonically inhibited and non-firing. The VP in turn is controlled by afferents from the vSub and the NAc. When the vSub is activated, it provides a glutamatergic drive to the NAc, which in turn inhibits the VP and releases DA neurons from inhibition, allowing them to fire spontaneously. In contrast, the PPTg provides a potent direct glutamatergic input to DA neurons; when the PPTg is activated, it causes DA neurons to fire in bursts, which is believed to be the behaviorally salient pattern signaling a rewarding event. The impact of the PPTg, however, is gated by the LDT; only when the LDT is active can the PPTg initiate burst firing. In order for a DA neuron to burst fire, it must first be firing spontaneously. Given that the vSub controls the proportion of DA neurons firing spontaneously, it also sets the number of DA neurons that can be made to burst fire by the PPTg. As such, the PPTg drives the behaviorally salient burst firing, whereas the vSub provides the ‘gain' or amplification of the signal. The greater the vSub-driven gain, the larger the DA response produced by a stimulus that activates the PPTg.

This organization would therefore permit the vSub to control the amplitude of the phasic burst firing response of the DA neurons. This is consistent with the role of the vSub in regulating context-dependent responses (Jarrard, 1995; Maren, 1999; Sharp, 1999; Fanselow, 2000). In conditions in which expectation would powerfully affect the magnitude of response to a stimulus, the vSub would be critical in controlling the amplitude of the DA neuron activation. Thus, if one were in a condition in which stimuli would have a high reward value (eg a casino), the ringing of a bell would be much more highly reinforcing than in other contexts (eg a church). Thus, the vSub provides a context-dependent modulation of the amplitude of the DA response to stimuli (Grace et al, 2007).

Alteration of DA neuron signaling

The state of the DA system can powerfully influence the response to stimuli occurring naturally and also pharmacologically. For example, the population activity of the DA neurons will affect the manner in which the DA system responds to drugs such as amphetamine. In cases in which the DA neuron population activity is high, there is an increase in the locomotor response to amphetamine injection; this can be reversed by inactivation of the vSub (Lodge and Grace, 2008). This is particularly true for manipulations in which the behavioral response has a contextual component. Thus, with repeated amphetamine administration, a behavioral sensitization to subsequent doses of amphetamine is produced, in which the same dose of drug will produce an exaggerated response when the animal is withdrawn from a repeated amphetamine treatment regimen (Segal and Mandell, 1974; Post and Rose, 1976). Moreover, the amplitude of the response is greatest if the test dose of amphetamine is given in the same environmental context as the original treatment (Vezina et al, 1989; Badiani et al, 2000; Crombag et al, 2000). During withdrawal from amphetamine sensitization, the increased behavioral response occurs in parallel with an increase in vSub firing and in the population activity of DA neurons (Lodge and Grace, 2008). Moreover, both the behavioral sensitization and the DA neuron population activity can be restored to baseline by inactivation of the vSub. A unique type of LTP due to AMPA receptor alteration (Bellone and Luscher, 2006) in VTA DA neurons following single or multiple doses of stimulants (Vezina and Queen, 2000; Ungless et al, 2001; Faleiro et al, 2003; Borgland et al, 2004; Faleiro et al, 2004; Schilstrom et al, 2006) may also have a function in the establishment of sensitization, particularly as this may potentiate the phasic DA responsiveness of the system. However, the induction with single drug doses and the short-lived (ie <10 days) nature of the response makes it insufficient in itself to account for the long-term sensitization process. Nonetheless, the necessary yet transient (Zhang et al, 1997) NMDA stimulation-dependent LTP (Kalivas, 1995; Vezina and Queen, 2000; Suto et al, 2003; Borgland et al, 2004) in the VTA that is required for sensitization may be necessary to supply NAc DA that will potentiate vSub–NAc inputs (Goto and Grace, 2005b). This in turn will allow the D1-dependent LTP that occurs in the vSub–NAC pathway in response to cocaine sensitization (Goto and Grace, 2005a). These data are also consistent with findings that, whereas glutamatergic mechanisms in the VTA are required for the induction of sensitization, the expression of sensitization is mediated by processes within the VTA (Kalivas and Stewart, 1991).

In contrast to sensitization, drug-seeking behavior such as that induced by drug self-administration appears to be dependent on a different process that reflects drug–behavior associations (Everitt and Robbins, 2005; Hyman et al, 2006). Interestingly, the induction of LTP in VTA DA neurons that is driven by cocaine self-administration appears to be uniquely persistent, lasting up to 3 months and persisting even after behavioral extinction of drug-seeking behavior has taken place (Chen et al, 2008). Thus, these longer-term changes appear to contribute to modifications that are better associated with drug-seeking behavior than with drug sensitization. In the case of drug sensitization, both experimenter injection-induced and self-administration-induced sensitization appear to exhibit similar actions with respect to the behavioral profile.

Amphetamine sensitization also is present with other types of context-dependent responses such as stress. Stress is known to be a context-dependent phenomenon, in that animals exhibit heightened responses to stressors when tested in an environment in which they had been previously exposed to stressors (Bouton and Bolles, 1979; Bouton and King, 1983). Moreover, stressors such as restraint are known to also increase the behavioral response to amphetamine (Pacchioni et al, 2002). In concert with this observation, a similar 2 h restraint stress will also increase the population activity of DA neurons (Valenti and Grace, 2008), and both the augmented behavioral response and the stress-induced increase in DA neuron population activity can be reversed by vSub inactivation.

CLINICAL IMPLICATIONS

The reward circuits that drive motivated behaviors are implicated in a broad array of disease states. Deficits in reward-related activity are central to the anhedonia of depression (Hyman et al, 2006), and altered stimulus valuation is also a known component of attention-deficit hyperactivity disorder and obsessive-compulsive disorder (Cardinal et al, 2004; Everitt et al, 2008; Huey et al, 2008). The integration of affective and cognitive processes that support optimal goal-directed behavior is critically regulated by the frontal cortex, and inadequate output from this region contributes to mental disorders ranging from schizophrenia to depression to drug abuse. Such a commonality of pathology may have its expression in the increasing convergence of treatment strategies, such as second-generation antipsychotic drugs now being used to treat depression and bipolar disorder (Ketter, 2008; Mathew, 2008). A greater understanding of systems integration on a basic neuroscience level may provide a neurobiological basis for interpreting novel findings from human imaging studies and a focus on disease endophenotypes that leads to a more individualized approach to the treatment of psychiatric disorders.

FUTURE RESEARCH DIRECTIONS

Limbic circuitry and its interactions with DA neurons provide the means for altering reward-related responding based on experience. The responsivity of the DA system is potently regulated by context and behaviorally salient stimuli. In turn, the output of DA neurons provides a critical modulation of the systems that regulate goal-directed behaviors, in particular the NAc. Such interconnected loops not only regulate behavioral responses but also select which salient stimuli are established into memory (Lisman and Grace, 2005). It is through such malleable, experience-dependent plasticity, subject to multiple nodes of influence that an organism can most successfully adapt to its environment. Conversely, disruption of these systems through developmental dysregulation, pharmacological intervention, or pathological stressors can lead to severely maladaptive responses in the form of mental and addictive disorders. Such concepts are most effectively derived from an integration of systems neuroscience with cellular and molecular analyses in normal and diseased states. By understanding the dynamics of these systems, the ability to treat or even prevent these conditions may be realized.