Drive and Reinforcement Circuitry in the Brain: Origins, Neurotransmitters, and Projection Fields

Abstract

Brain stimulation has identified two central subsets of stimulation sites with motivational relevance. First, there is a large and disperse set of sites where stimulation is reinforcing, increasing the frequency of the responses it follows, and second, a much more restricted set of sites where—along with reinforcement—stimulation also has drive-like effects, instigating feeding, copulation, predation, and other motivated acts in otherwise sated or peaceful animals. From this work a dispersed but synaptically interconnected network of reinforcement circuitry is emerging: it includes afferents to the ventral tegmental area and substantia nigra; the dopamine systems themselves; glutamatergic afferents to the striatum; and one of two dopamine-receptor-expressing efferent pathways of the striatum. Stimulation of a limited subset of these sites, including descending inhibitory medial forebrain bundle fibers, induces both feeding and reinforcement, and suggests the possibility of a subset of fibers where stimulation has both drive-like and reinforcing effects. This review stresses the common findings of sites and connectivity between electrical and optogenetic studies of core drive and reinforcement sites. By doing so, it suggests the biological importance of optogenetic follow-up of less-publicized electrical stimulation findings. Such studies promise not only information about origins, neurotransmitters, and connectivity of related networks, by covering more sensory and at least one putative motor component they also promote a much deeper understanding of the breadth of motivational function.

Introduction

Early studies of the effects of electrical brain stimulation in freely moving animals identified two motivational effects. First, diencephalic stimulation at many sites can be reinforcing, controlling the acts that it reliably follows (Olds, 1956; Olds and Milner, 1954). Second, stimulation can have drive-like effects, energizing a variety of species-typical, biologically primitive acts, such as eating, drinking, copulation, or attack, in otherwise sated or quiescent animals (Andersson and Wyrwicka, 1957; Delgado and Anand, 1953; Hess, 1957; Roberts and Carey, 1965; Wasman and Flynn, 1962). Stimulation at the same sites within the medial forebrain bundle (MFB) often has both drive-like and reinforcing effects (Caggiula and Hoebel, 1966; Margules and Olds, 1962; Mogenson and Stevenson, 1966; Roberts and Carey, 1965), whereas stimulation at a larger number of extraneous sites is reinforcing without inducing goal-directed behaviors. That rats should find stimulation reinforcing when it also makes them hungry or thirsty seemed paradoxical (Wise, 2013) and led to a range of experiments designed to explore whether the two effects were mediated by the same or by independent substrates (Coons and Cruce, 1968; Deutsch et al, 1962; Gratton and Wise, 1988a, 1988b,; Huston, 1971; Mendelson, 1970). These studies have not yet identified differences between reinforcement and drive fibers. However, because electrical stimulation preferentially activates fibers of passage (Ranck, 1975), electrical stimulation studies had limited success in identifying the directly activated fibers or connections of either effect.

In the present paper we integrate these electrical stimulation findings with recent optogenetic studies that can now identify cells of origin, neurotransmitters, and synaptic targets of the stimulated fibers. These studies, taken together, begin to sketch a core structure for an integrated mesencephalic, diencephalic, and telencephalic circuitry subserving motivational function. The optogenetic study of additional structures—currently identified only by electrical stimulation—is hoped to broaden the list of anatomical substrates and to deepen our understanding of motivational function.

TERMINOLOGY

Here we separate two motivational effects of stimulation, one associated with the animal’s state of mind before it earns a reward and one associated with the state of mind after the reward has been earned and is being experienced. We will use drive-like to refer to the former and reinforcing to refer to the latter. We have avoided, elsewhere in this paper, the use of the more familiar term ‘reward’ because it confuses the two meanings. The noun (‘a’ reward) refers to the object or effect to be sought, whereas the verb (‘to’ reward) refers to the action of obtaining and experiencing its effects.

Reinforcing effects are defined as the consequences of required behavior. This is measured by allowing an animal to control the stimulation. The gold standard is operant self-stimulation, where a specific response (ie, a lever-press or a nose-poke) results in a reinforcing event. In this case there is a fixed amount of reinforcement for each action. In the traditional situation, there are fixed quanta of reinforcement, and reinforcements can be counted. In the case of unsensed incentives such as earned brain stimulation, where there is no external stimulus object and continuous reinforcement can be offered (Wise, 2002), an alternative paradigm allows an animal to control both onset and offset of stimulation. This paradigm was originally called ‘shuttlebox stimulation’ and is referred to in the optogenetics literature as ‘real-time place preference.’ Care must be taken in this paradigm to differentiate between stimulus-onset effects and stimulus-offset effects; rats, having initiated the stimulation, will often work to discontinue it as if it had become aversive (Bower and Miller, 1958; Mendelson and Freed, 1973; Roberts, 1958a; Steiner et al, 1969). Furthermore, because continuous stimulation may impair the ability of the animal to remain in or return to the non-stimulation zone, locomotor artifacts can greatly influence measures in this situation. One workaround is using optogenetic stimulation where an animal is given fixed periods of stimulation or fixed periods between stimulations.

SITES AND SUBSTRATES

Ventral Midbrain to Striatum

Mesolimbic and nigrostriatal dopamine systems

Pharmacological challenge in early electrical stimulation studies suggested that the midbrain dopamine systems were a common substrate of MFB reinforcement and motivation. The reinforcing (Fouriezos and Wise, 1976; Lippa et al, 1973) and feeding-inducing (Phillips and Nikaido, 1975) effects of stimulation were each blocked by dopamine antagonists; the reinforcing effects were blocked in ways that could not be attributed to simple motor impairment (Fouriezos et al, 1978; Franklin and McCoy, 1979; Gallistel et al, 1982; Wise et al, 1978). Moreover, electrical stimulation in the region of the dopamine cell bodies was reinforcing (Crow, 1972; Routtenberg and Malsbury, 1969), and the dorsal, ventral, and lateral boundaries of the reinforcement system corresponded closely to the boundaries of the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) dopaminergic cell groups (Corbett and Wise, 1980; Wise, 1981). Subsequent paired-pulse, dual-electrode studies have revealed, however, that the directly activated fibers for both reinforcement and stimulation-induced feeding have refractory periods too short (Gratton and Wise, 1988b; Yeomans, 1979) and conduction velocities too fast (Bielajew and Shizgal, 1982; Gratton and Wise, 1988a) to be mediated directly by depolarization of dopaminergic fibers. Because the threshold for activating dopaminergic fibers within the MFB is orders of magnitude higher than what is required for brain stimulation reinforcement or stimulation-induced feeding, stimulation is now understood to activate other fibers preferentially (Gallistel et al, 1981); only very few and very local, dopaminergic fibers are activated at traditional stimulation parameters (Yeomans, 1989; Yeomans et al, 1988). Rather, the major contributions to the directly activated reinforcing effects of MFB electrical stimulation are now believed to be low-threshold, small, and descending myelinated fibers (Gallistel et al, 1981; Gratton and Wise, 1988a) one or more synapses upstream from the midbrain dopamine neurons (Wise, 1980; Yeomans, 1982).

With optogenetic methods, however, dopaminergic cell bodies of VTA or SNc can be selectively activated, and activation of these neurons is reinforcing (Ilango et al, 2014a; Ilango et al, 2014b; Kim et al, 2012; McDevitt et al, 2014; Pascoli et al, 2015; Rossi et al, 2013; Stauffer et al, 2016; Tsai et al, 2009; Wang et al, 2017; Witten et al, 2011). Two subpopulations of dopaminergic ventral midbrain neurons have been targeted—one that expresses the dopamine synthesis enzyme tyrosine hydroxylase (TH) but includes some midline GABAergic neurons that do not release dopamine (Lammel et al, 2015; Stamatakis et al, 2013), and another that expresses the dopamine transporter but excludes some dopaminergic neurons near the midline (Darvas et al, 2014; Li et al, 2013). Stimulation of each is reinforcing (for DAT-targeted animals see Kim et al (2012) and Wang et al (2017)). The reinforcing effects of optogenetic activation of TH-expressing midbrain neurons are attenuated by ventral striatal microinjections of D1-type or D2-type dopamine antagonists (Steinberg et al, 2014). However, genetic disruption of glutamate co-release from DAT-expressing neurons does not impact their reinforcing abilities (Wang et al, 2017). Consistent with this finding, rats will lever-press for direct microinjections of D1 and D2 agonists into this brain region (Ikemoto et al, 1997). The dorsal striatum is richly innervated by dopaminergic fibers from the SNc, and stimulation of the SNc cell bodies produces reward measures equivalent to those produced by ventral tegmental stimulation (Ilango et al, 2014b). The other projections from these systems remain to be studied, but it has been confirmed that stimulation of dopaminergic fibers projecting to the shell of the ventral striatum is reinforcing (Steinberg et al, 2014).

VTA glutamatergic cell bodies

Glutamatergic (and GABAergic) neurons in the VTA also project to mesencephalic and cortical terminal fields. Studies testing optogenetic stimulation of VTA cell bodies in the mouse generally support a role for glutamatergic neurons in reinforcement. Stimulation of VTA cell bodies is clearly reinforcing as measured in wheel-turning, and nose-poke tasks allowing animals to earn short trains of 20–40 Hz stimulation (Wang et al, 2015; Yoo et al, 2016a); constant stimulation at low frequencies is aversive (Yoo et al, 2016a). Here stimulation of glutamatergic neurons resulted in many brief entrances to a stimulation chamber, whereas mice with access to dopaminergic stimulation preferred long, constant, stimulation (Yoo et al, 2016a). These findings are consistent with electrical stimulation studies suggesting that stimulation at some sites is initially reinforcing but can become aversive if left on too long (Bower and Miller, 1958; Mendelson and Freed, 1973; Roberts, 1958a; Steiner et al, 1969), further suggesting that new experiments using optogenetic stimulation should consider the importance of train length on a circuit-by-circuit basis. The glutamatergic neurons make local synaptic contacts onto neighboring neurons (Dobi et al, 2010) and the reinforcing effects of stimulation were blocked by local infusion of glutamate antagonists, suggesting that the reinforcing effects were mediated by those excitatory contacts.

Terminals from VTA glutamatergic neurons

Three efferent projections of VTA glutamate neurons—to ventral striatum, ventral pallidum, and lateral habenula—have been tested (Qi et al, 2016b; Root et al, 2014a; Yoo et al, 2016b). Each of these projections involves distinct, non-overlapping, populations of ventral tegmental neurons (Yoo et al, 2016a). Of these, stimulation of the VTA glutamatergic projection to the habenula is aversive. This is clearly shown in real-time and conditioned place preference paradigms, where animals avoid the position where self-controlled stimulation was given (Root et al, 2014a; Yoo et al, 2016a), reverse their preference for chambers when reinforcement contingencies are reversed, and show residual avoidance for the chamber where they most recently received stimulation (Root et al, 2014a). The aversive effect was attenuated by habenula infusions of glutamate antagonists (Root et al, 2014a). The fibers in this projection co-release glutamate and GABA but not dopamine (Root et al, 2014b; Yoo et al, 2016a) and the net effect on postsynaptic neurons, as measured electrophysiologically in vivo and in slice, is inhibitory (Root et al, 2014b; Yoo et al, 2016a).

In the case of the projection to the pallidum, the stimulation appears to be reinforcing. In this case, the stimulation was not reinforcing when given to sated animals, but food-deprived animals nose-poked for brief, high-frequency stimulations (Yoo et al, 2016a). With some electrode placements (Fulton et al, 2006; Olds, 1958a), restricted food intake augments responding for brain stimulation reward (Carr, 1996; Fulton et al, 2000). In this optogenetic study, prolonged stimulation was aversive when the animals controlled stimulation duration by entrances and exits of the chamber; the animals clearly favored short trains of stimulation (Yoo et al, 2016a).

Stimulation of projections to the ventral striatum—tested in several studies using long, constant trains of high-frequency stimulation—is aversive (Qi et al, 2016a; Yoo et al, 2016a). The aversive effects are blocked by infusion of glutamatergic or GABAergic antagonists into the ventral striatum (Yoo et al, 2016a), suggesting involvement of local GABA release. Because VTA glutamate neurons projecting to the ventral striatum show minimal coexpression of GABA synthesis enzymes (Qi et al, 2016a) and do not synaptically release GABA (Yoo et al, 2016a), this projection appears to activate a population of GABAergic neurons projecting to or within the ventral striatum. Indeed, stimulation of this pathway preferentially induces c-fos expression in parvalbumin-expressing interneurons, which project locally to medium spiny neurons and produce aversion when directly stimulated (Qi et al, 2016a). At least two populations of VTA glutamatergic neurons project to the ventral striatum: glutamate–dopamine and pure glutamate (Kawano et al, 2006; Yamaguchi et al, 2011). The glutamate–dopamine neurons synaptically target ventral striatal cholinergic interneurons (Chuhma et al, 2014), a population of neurons that is neither rewarding nor aversive when directly stimulated (Witten et al, 2010). Accordingly, chemical lesion of dopamine terminals in the ventral striatum does not alter the aversive properties of glutamatergic pathway stimulation (Qi et al, 2016a). These results suggest that the aversive effects are selectively mediated by the pure glutamate subpopulation.

There is evidence for additional projections of VTA glutamate neurons to the prefrontal cortex and amygdala (Hnasko et al, 2012; Yamaguchi et al, 2011). Glutamatergic neurons have recently been identified within the boundaries of the SN and retrorubral field (Root et al, 2016; Yamaguchi et al, 2013). The roles of these additional glutamatergic neurons remain to be determined.

Afferents to the VTA

Lateral hypothalamus to VTA

Among the most studied reinforcement loci in the brain is the lateral hypothalamic area, which includes a bed nucleus and ascending and descending fibers of the MFB. Rats will lever-press several thousand times per hour for brief (0.5 s or less) trains of electrical stimulation at this site (Olds et al, 1960) and will do so almost continuously for tens of hours without signs of satiety (Annau et al, 1974; Olds, 1958b). At low levels the stimulation is not aversive (Hodos, 1964; Olds, 1960; Olds and Olds, 1963), but it becomes aversive when it is prolonged (Mendelson and Freed, 1973; Muenzinger and Baxter, 1957; Roberts, 1958b). The reinforcing effects are inversely related to stimulation frequency and intensity; this is a property of brain stimulation reinforcement not yet reflected in studies of optogenetic stimulation (Hodos, 1965). Nonetheless, when animals are allowed to lever-press for onset of short stimulation trains they lever-press almost continuously.

Paired-pulse electrical stimulation studies indicate that the majority of the directly activated MFB reinforcement fibers have short refractory periods (Yeomans, 1979), fast conduction velocities (Bielajew and Shizgal, 1982), project to or through the VTA (Shizgal et al, 1980), and carry reinforcement messages primarily in the rostral to caudal direction (Bielajew and Shizgal, 1986). Minority contributions appear to be made by unidentified cholinergic fibers (Gratton and Wise, 1985) and by a small number of ascending dopaminergic fibers passing very close to the electrode tip (Yeomans, 1989). One early hypothesis was that descending MFB fibers synapse on and excite VTA dopaminergic neurons (Wise, 1980); another is that the descending MFB fibers pass through the VTA, synapsing on cholinergic neurons that, in turn, relay back to the dopamine system (Yeomans, 1982). Projections to both targets remain possible.

Electrical stimulation at the same MFB reinforcement sites also induces species-typical behaviors such as eating (Delgado and Anand, 1953; Margules and Olds, 1962; Wise, 1971), drinking (Greer, 1955; Mogenson and Stevenson, 1966), gnawing (Cox and Valenstein, 1969; Roberts and Carey, 1965), nest building (Roberts and Carey, 1965), copulation (Caggiula, 1970; Caggiula and Hoebel, 1966), or predatory attack (Hutchinson and Renfrew, 1966; Wasman and Flynn, 1962). While dozens of fiber systems are intermingled in the MFB (Nieuwenhuys et al, 1982), several lines of evidence suggest that only a small subset of MFB fibers mediates the drive-like effects, on the one hand, and the reinforcing effects on the other. First, feeding and copulation are not restricted to lateral hypothalamic stimulation sites; in each case they can be induced by ventral tegmental stimulation (Eibergen and Caggiula, 1973; Gratton and Wise, 1988a) and, for feeding at least, by stimulation at sites more caudal in the brainstem (Ball et al, 1974; Berntson, 1973a). Paired-pulse studies show that the fibers mediating lateral hypothalamic and VTA feeding and reinforcement have the same distributions of refractory periods, including the small contribution from ultrafast cholinergic elements (Gratton and Wise, 1985, Gratton and Wise, 1988b); have similar conduction velocities (Gratton and Wise, 1988a); and have common anatomical alignment between the lateral hypothalamic and ventral tegmental regions (Gratton and Wise, 1988a). These findings suggest, though they do not confirm, a common substrate.

Recent optogenetic studies extend this suggestion, identifying the lateral hypothalamic cells of origin and the neurochemical phenotype of both the feeding and reinforcement induced by MFB stimulation (Figure 1a). Photostimulation of lateral hypothalamic GABA cell bodies (Jennings et al, 2015) or their terminals in the VTA (Barbano et al, 2016; Gigante et al, 2016; Nieh et al, 2015) is reinforcing and causes feeding, apparently by disinhibiting the dopamine system through hyperpolarizing VTA GABA interneurons (Jennings et al, 2013; Nieh et al, 2016). While glutamatergic fibers also project from the lateral hypothalamus to the VTA, activating these fibers reduces dopamine release in the ventral striatum and is aversive (Jennings et al, 2013; Nieh et al, 2016). The fact that activation of descending fibers from the same origin, and with the same transmitter induce both feeding and reinforcement still does not confirm a common substrate, however. There appear to be at least two subpopulations of LH GABAergic neurons that participate in aspects of feeding and food-seeking (Jennings et al, 2015; Nieh et al, 2015) and they need not project to the same target.

Figure 1.

Reinforcing and drive-like input pathways to the ventral tegmental area (a) and striatum (b), confirmed by optogenetic studies. Amyg: amygdala; BST: bed nucleus of the stria terminalis; DR: dorsal raphe; DS: dorsal striatum; LDTg: latero-dorsal tegmental nucleus; LHA: lateral hypothalamic area; mPFC: medial prefrontal cortex; mPOA: medial preoptic area; PPTg: pedunculopontine tegmental area; SNc: substantia nigra, zona compacta; vHippo: ventral hippocampus; VS: ventral striatum; VTA: ventral tegmental area.

Stimulation of glutamatergic lateral hypothalamic cell bodies or their projection to the lateral habenula is similarly aversive, and also inhibits feeding behavior in food-deprived animals (Jennings et al, 2015; Stamatakis et al, 2016). GABAergic fibers from the bed nucleus of the stria terminalis project to and inhibit lateral hypothalamic glutamatergic neurons; stimulation of these neurons also induces feeding (Jennings et al, 2013; Kim et al, 2013). Similarly, a real-time place preference is produced using nonspecific stimulation of bed nucleus neurons projecting to the VTA, but not to the lateral hypothalamus or parabrachial nucleus (Kim et al, 2013).

Similar organizations have been recently reported in systems linking hypothalamic sites with copulation and social attachment. In the case of social attachment, a subset of preoptic area neurotensin-expressing neurons projects to dopaminergic neurons in the VTA; optogenetic stimulation of this system is reinforcing (Kempadoo et al, 2013; McHenry et al, 2017) and potentiates social approach (McHenry et al, 2017). Electrical stimulation studies also implicate the posterior hypothalamic area and the VTA in stimulation-induced reinforcement and copulation (Caggiula, 1970; Caggiula and Hoebel, 1966; Eibergen and Caggiula, 1973).

Dorsal raphe to VTA

Electrical stimulation of the dorsal (Simon et al, 1976) and median (Miliaressis et al, 1975; Simon et al, 1973) raphe nuclei and the region between them (Rompré and Miliaressis, 1985) can be strongly reinforcing, establishing responding at high rates for low levels of current (despite, in some cases, strong motor artifacts) (Rompré and Miliaressis, 1985). Dual-electrode paired-pulse experiments implicate fibers connecting these sites with reinforcement sites in the VTA, but the direction of conduction has not been determined (Boye and Rompre, 1996). Recent optogenetic stimulation studies now confirm that glutamatergic fibers projecting to the VTA originate in the dorsal raphe (Geisler et al, 2007), project to and make asymmetric synapses on dopaminergic neurons in the VTA (Qi et al, 2014), cause ventral tegmental dopamine release (Qi et al, 2014), and provide the substrate for the reinforcing effect of stimulation (Liu et al, 2014; McDevitt et al, 2014; Qi et al, 2014). The dorsal raphe nucleus contains additional GABAergic and dopaminergic cell types; stimulation of each cell type fails to reinforce behavior (Matthews et al, 2016; McDevitt et al, 2014). Serotonin appears not to contribute to reinforcement function itself (Fonseca et al, 2015; McDevitt et al, 2014; Miyazaki et al, 2014).

Laterodorsal and pedunculopontine tegmental nucleus projections to VTA

An ultrafast subpopulation of cholinergic fibers contributes to the reinforcing effects of MFB stimulation (Gratton and Wise, 1985). It is thought to do so by activating descending MFB fibers that continue through the VTA and synapse on cholinergic fibers projecting back to the dopamine system (Lester et al, 2010; Yeomans et al, 1985; Yeomans et al, 1993).

Nonselective optogenetic activation of VTA projections from laterodorsal tegmental nucleus (LDTg) is reinforcing (Lammel et al, 2012; Steidl and Veverka, 2015), as is selective activation of either cholinergic (Steidl et al, 2016; Xiao et al, 2016) or glutamatergic (Steidl et al, 2016; Yoo et al, 2016a) fibers from this region. Selective activation of cholinergic fibers caused real-time and conditioned preference for the chamber where stimulation had been experienced, whereas selective activation of glutamatergic fibers was reinforcing when fixed 1.5 s trains were earned by each entry, but not when self-controlled stimulation train durations were offered. Thus, cholinergic and glutamatergic activations were significantly different, with glutamatergic activation—like most forms of electrical stimulation—becoming aversive if allowed to persist. Stimulation of the cholinergic and glutamatergic pathways from LDTg also have different effects on dopamine cell firing and on locomotion, indicating that the two transmitters have some functions in common and some that differ.

Nose-poke activation of VTA projections from pedunculopontine tegmental nucleus is also reinforcing (Yoo et al, 2017); such stimulation is most effective at 30 or 40 Hz, a frequency range not usually tested in the early optical self-stimulation tests. Stimulation at 20 Hz caused real-time but not conditioned place preferences (Xiao et al, 2016).

Lateral habenula projections to VTA and rostromedial tegmental nucleus

Nonselective optogenetic activation of lateral habenula projections that target neurons in the VTA (Lammel et al, 2012) or target the adjacent rostromedial tegmental nucleus (Stamatakis and Stuber, 2012) are aversive.

Afferents to the Striatum

The striatum receives input from several structures, including cortex, amygdala, thalamus, and hippocampus. The striatum is divided into dorsal and ventral compartments; electrical stimulation is reinforcing in each region (Figure 1b). A notable difference in afferent inputs between these two regions is that the dorsal striatum receives input from prefrontal, motor, and sensory cortices, whereas cortical inputs to the ventral striatum are limited primarily to the prefrontal cortex. Paired-pulse, dual-electrode studies have confirmed linkage between medial prefrontal cortical and dorsal striatal reinforcement sites (Trzcińska and Bielajew, 1998). Inputs to dorsal striatum from this and other regions have not been tested for reinforcement in optogenetic studies.

Optogenetic activation of glutamatergic inputs from amygdala, hippocampus, and prefrontal cortex is reinforcing (Britt et al, 2012; Stuber et al, 2011). Only one thalamic site has been tested with optogenetic stimulation; activation of glutamatergic input from the paraventricular nucleus to the medial shell of nucleus accumbens is aversive (Zhu et al, 2016).

Striatal Output Neurons

The reinforcing effects of stimulation of the output neurons of the striatum are of particular interest. This is the only system implicated in the reinforcing function that is efferent to the dopamine system; the dopamine system is the primary system involved in reinforcement of synaptic connections (the stamping in of memory traces (Wise, 2004)). Electrical stimulation is moderately reinforcing in both the dorsal and ventral striatum, with relatively equal regional specificity across these regions (Prado-Alcala and Wise, 1984). The reinforcing effects are also of particular interest because the two types of striatal output neurons—nearly identical and approximately equal in number—perform seemingly opposite functions. Classically viewed, the D₁R-mediated responses are thought to activate behavior and the D₂R-mediated responses are thought to inhibit behavior (Kravitz et al, 2010).

There are two output pathways in the striatum. In the dorsal striatum, optogenetic stimulation of the direct output pathway (projects directly to substantia nigra, pars reticulata) is clearly reinforcing, establishing lever-pressing or touch-plate contacts relative to unstimulated control contacts (Kravitz et al, 2012; Vicente et al, 2016), and increasing the amount of time an animal spends in a stimulation-associated test area (Kravitz et al, 2012). Activation of direct pathway neurons also established a conditioned preference for the environment where it had been previously received (Kravitz et al, 2012). Response habits were learned quickly, within a few minutes with an easy access task (Kravitz et al, 2012) and within a few days on a difficult task (Vicente et al, 2016).

Stimulation of the indirect pathway was initially avoided (Kravitz et al, 2012) or neutral (Vicente et al, 2016), but this effect was short-lived; over the course of 4 weeks of testing, mice developed a significant, though weak, preference for activation of the D₂R system (Vicente et al, 2016). This indirect pathway response was unusual in that the animals learned to press the inactive lever to almost the same extent as they pressed the active lever, showing marginal discrimination between the physically identical levers. The behaviors activated by indirect pathway stimulation were different from those caused by direct pathway stimulation, although simple tests like forward vs backward locomotion were not reported. The current assumption is that indirect pathway stimulation activates behaviors like postural stability, necessary for discrete direct pathway actions (Cui et al, 2013; Tecuapetla et al, 2016; Vicente et al, 2016).

In the ventral striatum, the effects of optogenetic stimulation reveal greater complications. When the ventral section of the structure was globally stimulated, the effect was reinforcing in both real-time preference and nose-poke tasks (Britt et al, 2012). However, in a follow-up experiment targeting direct pathway (dynorphin-expressing neurons), it was found that stimulation in the dorsal portion was reinforcing, whereas stimulation in the ventral portion was aversive (Al-Hasani et al, 2015). Direct pathway neurons coexpress the peptide co-transmitter dynorphin, a kappa opioid agonist, and both the rewarding or the aversive effects of stimulating this pathway were antagonized by a local infusion of a kappa-opioid antagonist; in separate trials, rewarding and aversive effects could be demonstrated in each animal (Al-Hasani et al, 2015).

The effect of kappa receptor antagonism raises several questions. First, because the kappa opioid receptor is expressed by several local neuronal subtypes—local neurons, incoming terminals, and recurrent collaterals (Carlezon and Krystal, 2016; Svingos et al, 1999; Tejeda et al, 2017)—it is not clear what kappa population or populations are crucial for the optogenetic effects. That the motivational effects of D₁R neurons can be blocked by the accumbens release of a peptide co-transmitter raises the important issue of which functions are controlled locally and which are controlled by the release of neurotransmitters from the distant terminals of these long-axon projections. Moreover, while dorsal–ventral differences have been identified, rostral–caudal differences are also suspected to have different motivational functions (Reynolds and Berridge, 2002). Finally, the findings with ventral striatal stimulation must raise the question of localized specialization within the dorsal striatum.

MOTIVATION AND REINFORCEMENT: CORE CIRCUIT ELEMENTS

The reviewed studies suggest an expanded core framework for reinforcement circuitry, a portion of which is also implicated in drive-like effects. In addition, the core elements include varieties of input-to and output-from the regions of the dopaminergic cell groups and their terminal fields in the striatum, regions where addictive drug exposure is known to cause neuroadaptations associated with compulsive drug habits. One of the inputs to the region of the dopamine cell bodies—the GABAergic input from the LHA—is implicated in both reinforcement and food-associated drive function.

SITES FOR FUTURE STUDY

Of obvious interest for further study are sites identified as motivational sites by early electrical stimulations studies. The variety of such sites—some clearly in presumed sensory areas and some in arguably motor areas—suggests a broader variety of reinforcement substrates and perhaps a deeper understanding of motivation itself. This includes sites in the brainstem where feeding and reinforcement can each be induced as well as sites all over the brain where stimulation is reinforcing (Figure 2). The study of a wider range of sites should improve our understanding of reinforcement in general.

Figure 2.

A variety of reinforcement-related regions identified in electrical stimulation studies but not yet tested in optogenetic studies.

Dorsal Pons and Deep Cerebellar Nuclei

Feeding and reinforcement can each be induced by electrical stimulation of sites caudal to the VTA. An important possibility is that these sites may be along caudal extensions of the MFB. The dorsal midbrain sites have been most extensively documented and fall along the path of (but not directly in) the caudal portions of the superior cerebellar peduncle, extending to the deep cerebellar nuclei (Ball et al, 1974; Berntson, 1973b; Corbett et al, 1982; Micco, 1974). Biting attack can also be induced by stimulation near these sites (Berntson, 1973a). Optogenetic methods would be ideal for testing the hypothesis that these additional sites are a caudal extension of the MFB (Berntson and Micco, 1976); these dorsal tegmental sites are the only brainstem sites that, like the diencephalic MFB, seem involved in both drive and reinforcement. On the other hand, these sites may identify rostrally projecting fibers, perhaps synapsing in the VTA.

Other Brainstem Sites for Feeding or Reinforcement

Feeding can also be induced by stimulation of sites in the lateral and ventrolateral medulla of the cat (Berntson and Hughes, 1974), near the region of the motor nucleus of the trigeminal where stimulation is reinforcing in the rat (Van Der Kooy and Phillips, 1977; van der Kooy and Phillips, 1979). Electrical stimulation of the dorsal medulla, in the region of the nucleus of the solitary tract is reinforcing (Carter and Phillips, 1975), and this nucleus is a relay for sweet taste (Blomquist and Antem, 1965), a primary reinforcing sensory stimulus (Pfaffmann, 1960). Cells of origin or projections have not been identified.

Other Forebrain Sites

Feeding can be induced by stimulation of a variety of forebrain sites in the monkey, including the medial thalamus, anterior cingulate, septal area, amygdala, internal capsule, putamen, and stria medullaris (Milgram et al, 1977; Robinson and Mishkin, 1962, 1968). Numerous thalamic sites have been found reinforcing in rats, with particular reinforcement sensitivity identified in the stria medullaris and the junction of the paratenial and centromedial nuclei (Sutherland and Nakajima, 1981; Vachon and Miliaressis, 1992). Electrical stimulation of the dorsal hippocampus is reinforcing (Campbell et al, 1978) and this is facilitated by food deprivation (Milgram et al, 1977); feeding is induced by dorsal hippocampal stimulation but follows offset rather than onset of stimulation (Milgram et al, 1977). Electrical stimulation of the olfactory bulb is reinforcing in the rat (Phillips and Mogenson, 1969), and this effect is enhanced by reinforcing odor stimuli (Phillips, 1970).

Summary and conclusions

Converging evidence from electrical and optogenetic brain stimulation studies implicates the long-axon forebrain dopamine systems and some of their afferents and one of their efferent targets (along with some of their inputs) in motivational function. Evidence from electrical stimulation studies also implicates fibers from several seemingly unrelated structures that remain to be studied with optogenetic techniques that can be used to identify cells of origin, neurotransmitters, and projection targets. Further study with optogenetic methods offers a method for elaborating and limiting the motivational circuitry first suggested by electrical stimulation. Of particular interest will be anatomical linkages between these electrical stimulation sites and the core motivational systems that optogenetics is establishing. These stimulation studies, linked to recording studies and loss-of-function studies, promise a broader understanding of the complex circuitry linking sensory and motor functions in behavior.

Funding and disclosure

The authors were supported by the Intramural Research Program, National Institute on Drug Abuse. The authors declare no conflict of interest.