New Insights into the Specificity and Plasticity of Reward and Aversion Encoding in the Mesolimbic System

Abstract

The mesocorticolimbic system, consisting, at its core, of the ventral tegmental area, the nucleus accumbens, and medial prefrontal cortex, has historically been investigated primarily for its role in positively motivated behaviors and reinforcement learning, and its dysfunction in addiction, schizophrenia, depression, and other mood disorders. Recently, researchers have undertaken a more comprehensive analysis of this system, including its role in not only reward but also punishment, as well as in both positive and negative reinforcement. This focus has been facilitated by new anatomical, physiological, and behavioral approaches to delineate functional circuits underlying behaviors and to determine how this system flexibly encodes and responds to positive and negative states and events, beyond simple associative learning. This review is a summary of topics covered in a mini-symposium at the 2013 Society for Neuroscience annual meeting.

Introduction

The dopamine neurons in the ventral tegmental area (VTA) and their targets in the nucleus accumbens (NAc) and the medial prefrontal cortex (mPFC) are often considered the nexus of the brain's mesocorticolimbic “reward circuit,” although it has long been known that neurons in these brain areas are also influenced by aversive stimuli and events (e.g., Bromberg-Martin et al., 2010). With the advent of new anatomical, physiological, and behavioral approaches, and critical attention to psychological constructs, a more sophisticated understanding is emerging of the role of the mesocorticolimbic system in motivated behavior.

One area of current investigation is the delineation of functional heterogeneity and specificity of subcircuits in the VTA (reviewed by Ungless and Grace, 2012; Lammel et al., 2013; Roeper, 2013). It is now apparent that the dopamine (DA) neurons in the VTA comprise several subpopulations distinguished by their afferent and efferent projections, gene expression profiles, electrophysiological properties, and participation in reward and aversion. Similarly, the GABAergic neurons in the VTA display diversity (Margolis et al., 2012). Tract-tracing experiments in both mice and rats have revealed physiological and pharmacological heterogeneity among DA projections to a variety of targets (Ford et al., 2006; Margolis et al., 2006a, 2008b; Lammel et al., 2008, 2011). In the mouse, the analysis of DA neuron heterogeneity has been greatly advanced by the use of optogenetic approaches that depend on molecular genetic tools, which have only recently become available for other species. Thus, although DA neuron heterogeneity has been found in other species, it is not yet known exactly how or whether the pattern of subcircuits differs among species (Margolis et al., 2008b; Ungless and Grace, 2012).

It is well established from studies in rodents and monkeys that putative DA neurons in the VTA increase their firing in response to the presentation of primary rewards and to conditioned stimuli that predict reward. However, the nature of the response of these neurons to aversive events and associated conditioned stimuli remains controversial (Ungless et al., 2010; Ungless and Grace, 2012). A full discussion of this issue is beyond the scope of this review, but briefly, although many DA neurons are inhibited by aversive stimuli, a subset of confirmed DA neurons are excited (Brischoux et al., 2009; Cohen et al., 2012). In addition, studies using fast scan cyclic voltammetry (FSCV) and microdialysis have shown elevated DA levels in the NAc and mPFC in response to aversive stimuli (Bassareo et al., 2002; Budygin et al., 2012). However, questions remain about differences between responses in anesthetized versus awake animals, whether some aversive stimuli used in these studies are truly aversive, whether recorded responses coincide with the onset of an aversive event or its offset, which may be perceived as rewarding, and other interpretations of DA release, such as in response to stimuli that predict successful punishment avoidance (Oleson and Cheer, 2013). A recent study, described in more detail below, took a more global approach to analyzing how value-related information is encoded in VTA (Kim et al., 2012).

There is a long tradition of single-unit electrophysiological recording of NAc responses to natural and drug rewards and associated stimuli (Carelli, 2002; Carelli and West, 2013). Based on these data and other evidence, Carlezon and Thomas (2009) proposed that reward is encoded by reduced activity of medium spiny neurons (MSNs) in NAc, whereas aversion is encoded by excitation of MSNs. More recently, the understanding of encoding of reward and aversion in NAc has been advanced by the use of FSCV to measure DA transients released from VTA terminals and by the design of behavioral experiments to differentiate hedonic value, motivation, and learning as separate constructs (Richard et al., 2012; reviewed by McCutcheon et al., 2012; Berridge and Kringelbach, 2013). These studies have revealed regional differences in the NAc, such as the “hedonic hotspot” in the rostrodorsal portion of the medial shell (Peciña et al., 2006), other differences along the rostrocaudal dimension (Richard et al., 2013), and modulation of the valence of NAc neuronal function and release of DA in the NAc by behavioral state and learned associations (Reynolds and Berridge, 2008; Roitman et al., 2010; Loriaux et al., 2011; McCutcheon et al., 2012; Robinson and Berridge, 2013). Finally, new molecular approaches are beginning to reveal the functional organization of the direct and indirect output pathways of the NAc (Hikida et al., 2010, 2013; Lobo and Nestler, 2011).

New approaches to understanding neural circuits in the VTA

Dopaminergic, GABAergic, and glutamatergic neurons are intermingled in the VTA (Fields et al., 2007). Approximately 60% of the neurons in the VTA are dopaminergic (Swanson, 1982; Margolis et al., 2006b; Nair-Roberts et al., 2008), 25% are GABAergic (Margolis et al., 2012), and data suggest that the remaining 15% are glutamatergic neurons (Yamaguchi et al., 2007, 2011). Over 30 years ago, it was established that, whereas substantia nigra pars compacta DA neurons send axon collaterals to multiple brain regions with broad terminal arbors, individual VTA neurons project to single target brain regions (Fallon, 1981; Swanson, 1982; Margolis et al., 2006a; Matsuda et al., 2009). These projections include the PFC and limbic structures, such as the NAc as well as the basolateral amygdala (Björklund and Dunnett, 2007). There is also growing evidence for selective synapse formation on subpopulations of VTA neurons. For instance, combinations of optogenetics, electrophysiology, and immunocytochemistry have been used to demonstrate that functional GABAergic inputs to the VTA from the NAc synapse specifically on non-DA neurons, whereas GABAergic inputs arising from the ventral pallidum synapse onto both DA and non-DA neurons (Xia et al., 2011; Hjelmstad et al., 2013). Anatomical studies indicate even more specific organization, such that afferent inputs sort by both the projection target and neurotransmitter content of the postsynaptic cell (e.g., Carr and Sesack, 2000; Omelchenko and Sesack, 2005). For example, glutamatergic inputs from the mPFC synapse onto VTA DA neurons that project back to the mPFC, but not DA neurons that project to the NAc (Carr and Sesack, 2000). This anatomical organization of the VTA raises the possibility of differential activity coursing through the multiple circuits of the VTA.

Given the heterogeneity of neurons in the VTA, one persistent challenge has been accurate identification of the type of neuron recorded in vivo and ex vivo. The anatomical evidence for parallel, potentially independent, projections from the VTA to its targets raises the possibility that VTA DA neural activity is indeed more heterogeneous than previously postulated. Recent studies suggesting that established electrophysiological criteria may be insufficient for distinguishing VTA neurons of different neurotransmitter content have generated significant attention in the field (Margolis et al., 2006b, 2012; Ungless and Grace, 2012). However, neuronal markers that clearly differentiate these cell types in vivo and ex vivo have not yet emerged. Established methods, such as immunocytochemical identification of biocytin-labeled cells or the use of transgenic mice that express fluorescent proteins in different cell populations (e.g., TH-GFP mice), are currently the most reliable approaches for distinguishing VTA neurons (Margolis et al., 2010). In addition, novel methods, such as optical tagging (i.e., the identification of cell types based on their response to optogenetic stimulation), should be used to unambiguously identify VTA cell populations (Cohen et al., 2012).

Although many electrophysiological studies have overlooked heterogeneity among VTA DA neurons, recent work has begun to assign intrinsic and pharmacological properties to VTA DA neurons according to their projection targets. By combining retrograde tracing of dopaminergic projection targets with ex vivo electrophysiology, profound differences in the anatomical, molecular, physiological, and pharmacological properties of VTA DA subpopulations were shown (Ford et al., 2006; Margolis et al., 2006a, 2008b; Lammel et al., 2008, 2011; Roeper, 2013). Moreover, the prediction of the dopaminergic phenotype seems to be more reliable when the projection target is known (Margolis et al., 2008b; Lammel et al., 2011). VTA GABA neurons also exhibit a broad set of physiological and pharmacological properties, often similar to those of VTA DA neurons (Margolis et al., 2012). It remains to be determined how VTA GABA neuron properties are distributed in different efferent subpopulations.

Recently, researchers have begun to uncover the behavioral relevance of the diversity of afferent-specific inputs to VTA neurons using optogenetic stimulation of selected presynaptic terminals. Their studies showed that rewarding and aversive behaviors in mice can be controlled by optogenetic stimulation (Lammel et al., 2012; Stamatakis and Stuber, 2012; Jennings et al., 2013). Specifically, by expressing light-activated channel channelrhodopsin-2 in the laterodorsal tegmentum, it was shown that optical stimulation of laterodorsal tegmentum terminals in the VTA induced conditioned place preference (CPP) (Lammel et al., 2012). In contrast, light stimulation of terminals from the lateral habenula in the VTA and rostromedial tegmentum (RMTg, a GABAergic nucleus that extends posteriorly and dorsal from the VTA and sends a large projection to midbrain DA neurons) produced conditioned place aversion (Lammel et al., 2012). Brain slice recordings of functional synaptic connectivity revealed that laterodorsal tegmentum neurons synapse on VTA DA neurons projecting to NAc lateral shell. In contrast, lateral habenula neurons make synaptic connections on both GABAergic neurons in the RMTg as well as DA neurons projecting to PFC (Omelchenko and Sesack, 2005; Balcita-Pedicino et al., 2011; Lammel et al., 2012). Importantly, optical stimulation of lateral habenula terminals in the VTA induced inhibitory postsynaptic currents in VTA DA neurons projecting to NAc lateral shell (Lammel et al., 2012), which supports the idea that VTA DA neurons are tonically inhibited by RMTg GABAergic neurons (Bourdy and Barrot, 2012). These findings show that VTA neurons are part of segregated circuits that control reward- and aversion-associated behaviors.

Layered on top of the different circuits passing through the VTA contributing to behavior is the differential control of these circuits by peptides and other neuromodulators. For instance, μ opioid receptor (MOR) modulation of circuit elements in the VTA has been intensely studied because of the robust behavioral effects of microinjecting MOR agents into the VTA. For example, microinjecting a MOR agonist into the VTA produces CPP (Phillips and LePiane, 1980; Bals-Kubik et al., 1993; Nader and van der Kooy, 1997; Zangen et al., 2002). This manipulation also increases DA release in the ventral striatum (Spanagel et al., 1992; Devine et al., 1993), and it is thought that this occurs through disinhibition: MORs on GABA terminals inhibit GABA release onto DA neurons, permitting them to fire more (Kelley et al., 1980; Johnson and North, 1992). Although MOR function in the VTA is required for systemic opioid CPP (Olmstead and Franklin, 1997; Zhang et al., 2009), mounting evidence suggests that the mechanism of MOR reward in the VTA is significantly more complicated than increasing DA in the NAc. That is, CPP to either systemic or intra-VTA morphine can persist in animals trained while DA signaling was either pharmacologically blocked or genetically diminished (Bechara et al., 1992; Nader and van der Kooy, 1997; Hnasko et al., 2005). When animals are opiate dependent, however, systemic or intra-NAc blockade of DA signaling does eliminate acquisition of intra-VTA morphine CPP (Bechara et al., 1992; Nader and van der Kooy, 1997). This suggests that there are both dopaminergic and non-dopaminergic pathways through the VTA that can signal reward.

In addition to disinhibition, there are a variety of other effects of MOR activation on VTA neurons that could contribute to either DA-dependent or DA-independent reward. For instance, MOR activation not only inhibits GABA release onto VTA DA neurons, but also onto non-DA neurons (Margolis et al., 2008a). It also inhibits glutamate release onto many VTA neurons (Bonci and Malenka, 1999; Margolis et al., 2005). Importantly, over half of VTA neurons, including DA neurons, are postsynaptically inhibited by MOR agonists (Cameron et al., 1997; Margolis et al., 2003; Ford et al., 2006). Further, about half of the VTA GABA neurons are inhibited by MOR activation, and many of these are likely projection neurons (Margolis et al., 2012). Previous studies demonstrating that postsynaptic κ opioid receptor effects sort by projection target among VTA DA neurons (Ford et al., 2006; Margolis et al., 2006a, 2008b) raise the possibility that MOR effects on DA and GABA neurons in the VTA may also sort by projection target. Together, these observations indicate that MOR activation in the VTA produces reward through at least two circuits, possibly harnessing those revealed as separable in the aforementioned optogenetic studies.

Coordinated activity of VTA neurons during appetitive and aversive learning

As described above, recent studies indicate that the midbrain VTA DA system is more heterogeneous than previously assumed (Carr and Sesack, 2000; Margolis et al., 2006b, 2012; Fields et al., 2007; Yamaguchi et al., 2007; Luo et al., 2008; Lammel et al., 2012; Roeper, 2013). Given that this system is implicated in multiple forms of learning and memory, motivation, valuation, and executive function (Goldman-Rakic, 1996; Gibbs et al., 2007; Schultz, 2007; Salamone and Correa, 2012), it is not surprising that the conceptualization of it as a highly homogeneous network of cells is beginning to fall away. The heterogeneity of VTA neurons raises new questions, however, such as the degree to which they encode information through their joint activity during behaviorally activated states, which can be addressed by investigating their population dynamics to quantify the extent to which they interact. Understanding the nature of this interaction is also relevant to the pathophysiology of neurological and psychiatric disorders. For example, impairment of coordinated activity and joint processing of information in the VTA may be one mechanism that produces the behavioral symptoms of disorders, such as schizophrenia, for which there is no consistent marker of dopamine neuron pathology (Moghaddam and Wood, 2013).

In cortical regions, it has been well recognized that dynamic coordination between groups of neurons is critical for orchestration of behavior (Cohen and Kohn, 2011). Few studies, however, have focused on investigating the nature of interaction of simultaneously recorded VTA neurons. Kim et al. (2012) observed modulations in the correlation structure between VTA neurons, according to the conditioned value of a stimulus. After associative learning, stimuli that had been paired with a rewarding outcome caused the firing of simultaneously recorded VTA neurons to become more correlated. Similar findings have also been observed in substantia nigra DA neurons (Joshua et al., 2009). Correlations in VTA were context specific, were decreased by presentation of aversive stimuli paired with a mild electrical shock, and also tracked reversals of appetitive and aversive associations (Kim et al., 2012). Other studies have reported synchronization (and modulations of synchrony) among VTA neurons with drug exposure (Hyland et al., 2002; Li et al., 2011). Although the role of these correlations in VTA activity in cognition is not well understood, it is hypothesized that they are essential for encoding appetitive information (Joshua et al., 2009).

In addition to correlations in activity between neurons, local field potential oscillations, which mainly reflect the summation of synaptic inputs to a region, provide information about the correlation structure of VTA networks. Kim et al., (2012) found that the responses of many VTA neurons to visual and auditory stimuli are phase-locked to the θ rhythm; after associative conditioning, the proportion phase-locked to stimuli predicting a rewarding outcome increased, whereas the proportion phase-locked to stimuli predicting an aversive outcome decreased. Interactions between neuronal firing and θ oscillations have also been observed in the VTA during appetitive association and working memory tasks (Fujisawa and Buzsáki, 2011; Kim et al., 2012).

Collectively, these data suggest that VTA neurons flexibly organize as functional networks to process behaviorally relevant information. Although there currently is little understanding of the role of these neuronal interactions in cognition, evidence is mounting that they are essential to influencing how the VTA processes information related to both positive and negative motivation.

Encoding and modulation of positive and negative motivation in the NAc

Although the difference between VTA neurons that encode reward versus aversion may be based in part on their projection targets, even within a given target region, overlapping neurotransmitters and neuromodulators, including dopamine, may contribute to both positive and negative motivation. The NAc is most well known for its role in reward and is associated with the generation of appetitive motivational and affective states, including reward seeking, ingestion, and pleasure (Kelley et al., 2005), but it has also been implicated in aversive motivational and affective states (Levita et al., 2009). The particular role of the NAc in encoding and causing behaviors related to aversion remains the subject of ongoing investigation and may differ depending on both the subregion of the NAc and the nature of the “aversion” in question.

The rostrocaudal gradient of the NAc medial shell

Although inhibition of NAc neurons appears important for appetitive motivation and reward seeking (Carlezon and Thomas, 2009), inhibition of NAc neurons is also capable of generating certain negative affective and motivational states, particularly when more caudal areas of the medial shell region are inhibited (Richard et al., 2013). Inhibition of medial shell by either blockade of AMPA glutamate receptors or GABA receptor agonism generates intense hyperphagia, causing rats to eat up to 10 times what they would eat under control conditions (Maldonado-Irizarry et al., 1995; Stratford and Kelley, 1997). Yet, inhibitions of caudal NAc shell elicit less eating, and instead generate a suite of defensive reactions (Reynolds and Berridge, 2001). When the experimenter attempts to touch a rat that has received caudal shell inhibition, the rat is likely to emit audible distress vocalizations, attempt to escape, and attempt to (and sometimes successfully) bite the experimenter. Caudal shell inhibitions also generate a behavior known as defensive treading or burying, in which rodents use rapid forepaw movements to throw dirt or debris at a threatening stimulus or predator (Coss and Owings, 1978; Treit et al., 1981).

Yet, the rostrocaudal localization of eating and fear is not rigid, and varies depending on the nature of the current environmental context (Reynolds and Berridge, 2008; Richard et al., 2013). When rats are tested in the familiar, comfortable environment, the AMPA receptor antagonist DNQX generates purely appetitive behaviors (eating without fear) at >80% of NAc shell microinjections sites. In contrast, when rats are tested in a stressful, bright, and loud environment, DNQX generates fearful behaviors at 80% of sites tested. Therefore, the valence of motivated behavior generated from the same microinjection, especially at intermediate rostrocaudal locations, can switch from appetitive to defensive, and vice versa.

What are the neural mechanisms of these changes in motivational valence? Importantly, whereas the valence of behavior elicited by AMPA glutamate receptor blockade is flexible and modulated by the environment, the valence of behavior generated by GABA agonism is resistant to changes in the environment (Richard et al., 2013). Furthermore, although eating and defensive behaviors generated by glutamate antagonism have been shown to depend on local endogenous DA (Faure et al., 2008), GABA agonism robustly generates eating and defensive behaviors even in the face of local DA blockade (Richard et al., 2013). Given that eating and fear generated by AMPA antagonism depend on different DA receptor subtypes (eating requires D1 alone, and fear requires D1 and D2 stimulation), perhaps subtle changes in DA transmission can account for the appetitive versus fearful bias of DNQX-induced behaviors in different environments (Richard and Berridge, 2011). Additionally, AMPA blockade in NAc shell may also differentially generate eating versus fear depending on the current states of its many corticolimbic glutamate inputs, including from PFC. For instance, when infralimbic cortex is activated (via pharmacological disinhibition), NAc shell DNQX generates both less eating and fewer defensive reactions (Richard and Berridge, 2013). In contrast, when medial orbitofrontal cortex is activated, DNQX generates more eating at intermediate and caudal sites, whereas fear is unaffected (Richard and Berridge, 2013), suggesting that some glutamate inputs may selectively potentiate or suppress eating versus fear. It remains to be seen whether changes in the state of other corticolimbic glutamate inputs can account for the dramatic flips from appetitive to defensive behavior induced by the environment.

Mesolimbic encoding of rewarding and aversive stimuli

A major challenge for determining the mesolimbic encoding of aversions is that, although rodents will voluntarily and avidly consume rewards, they avoid aversive stimuli when possible. One approach to directly compare reward and aversion responses within the mesolimbic system has been to use the intraoral delivery of taste stimuli (Grill and Norgren, 1978), which permits complete experimenter control. Intraorally delivered solutions evoke stereotypical orofacial responses that can be classified as either appetitive (rewarding) or aversive (Berridge, 2000). A combination of this approach and recording techniques that sample mesolimbic activity during intraoral infusions can serve to delineate mesolimbic encoding of these opposing hedonic evaluations. FSCV captures fluctuations in DA concentration from DA terminal regions with excellent spatial and temporal specificity (Robinson et al., 2003; Wightman et al., 2007; Owesson-White et al., 2012). Measurements in awake and behaving rats have revealed that brief (∼1–2 s) DA release events (∼25–100 nm), commonly referred to as dopamine transients, occur spontaneously with low probability (Owesson-White et al., 2012). Rewarding stimuli, including food (Roitman et al., 2004; Brown et al., 2011; McCutcheon et al., 2012), drugs of abuse (Stuber et al., 2005; Aragona et al., 2009), and cues that predict their delivery (Phillips et al., 2003; Daberkow et al., 2013), increase the probability of dopamine transients at their onset. Intraoral infusions of a rewarding sucrose solution also increase dopamine transients within the dorsomedial shell of the NAc that are time-locked to infusion onset, as well as throughout and after the intraoral infusion (Roitman et al., 2008). An aversive quinine solution, delivered via intraoral infusion during the same recording sessions, evoked the opposite response (i.e., dopamine transients were suppressed during and after intraoral infusions). This latter finding likely represents the neurochemical consequence of pauses in the firing of DA neurons, which have been recorded in response to aversive stimuli (Mirenowicz and Schultz, 1996; Matsumoto and Hikosaka, 2009; Cohen et al., 2012).

Electrophysiological recordings in the NAc during intraoral delivery of rewarding and aversive taste stimuli reveal that, like NAc dopamine transients, NAc neurons are also acutely responsive to both classes of hedonic stimuli but encode them differentially (Roitman et al., 2005). Sucrose consumption evokes a reduction in activity in the majority of responsive NAc neurons (Roitman et al., 2005; Taha and Fields, 2005, 2006). Quinine, instead, predominantly evokes increases in neuronal activity during intraoral infusions (Roitman et al., 2005). Thus, rewarding taste stimuli evoke an immediate increase in DA release in the NAc and a decrease in the population firing rate of NAc neurons. Aversive taste stimuli evoke the opposite pattern: a suppression of dopamine transients and an increase in the population firing rate of NAc neurons.

The differences in DA release and neural activity responses to rewarding and aversive taste stimuli could be the result of their different sensory qualities, rather than the hedonic responses they evoke. With the use of a conditioned taste aversion paradigm, the rewarding taste of sucrose is rendered aversive after it is paired with an injection of the malaise-producing agent lithium chloride. Intraoral infusions of sucrose in rats with a conditioned taste aversion evoke aversive orofacial responses and patterns of DA release and NAc activity similar to that observed in response to quinine (Roitman et al., 2010; McCutcheon et al., 2012). Rats that experience sucrose and lithium chloride on separate days, and therefore do not learn a taste aversion, show appetitive orofacial responses and patterns of DA release and neural activity similar to that observed in response to sucrose given to naive rats. These findings have been used, along with other approaches, to support the “activity hypothesis” of hedonic encoding by the NAc (Carlezon and Thomas, 2009).

Differential activity of NAc neurons is likely tied to motor plans to either approach and consume or avoid and reject stimuli. Indeed, decreases in the firing rate of NAc neurons during sucrose consumption, the predominant response, are linked to appetitive and consummatory behavior (Nicola et al., 2004; Roitman et al., 2005; Gutierrez et al., 2010), and preventing these decreases halts sucrose consumption (Krause et al., 2010). Interestingly, different subtypes of NAc neurons (i.e., DA D1- vs DA D2-receptor expressing) exert opposing influences over reward-directed behavior (Lobo et al., 2010). An important future course will be to identify the phenotype of NAc neurons that differentially respond to reward and aversion, and to determine whether there is a causal link between differential patterns of dopamine transients and NAc neural activity and appetitive versus aversive behavioral responses.

Differential motivational control by NAc direct and indirect pathways

Until recently, the complex cytoarchitecture of the NAc has impeded understanding of the specific contributions of different NAc neuron subtypes in motivational behaviors. The two types of NAc and dorsal striatum projection neurons, the MSNs, are differentiated by enrichment of many genes, including the DA D1 and DA D2 receptors (Gerfen et al., 1990; Lobo, 2009). They are further differentiated by their projections in the basal ganglia circuit: D1-MSNs in the dorsal striatum form the direct pathway and project the globus pallidus internal segment (GPi) and midbrain regions, including substantia nigra (SN); D2-MSNs in the dorsal striatum comprise the indirect pathway, projecting to globus pallidus external segment (GPe) (Gerfen, 1992; Smith et al., 2013). The NAc projections are not quite synonymous with the dorsal striatal direct versus indirect pathways. Similar to dorsal striatum, NAc D2-MSNs exclusively project to a pallidal region, ventral pallidum (VP), whereas NAc D1-MSNs project to both VP and midbrain regions, including SN and VTA, as demonstrated in rodent studies and more recently in D1-tdTomato/D2-GFP double transgenic lines (Nicola, 2007; Tripathi et al., 2010; Smith et al., 2013).

Early studies using inducible transgenic lines that serendipitously express ΔFosB, a transcription factor critical for plasticity changes to motivational stimuli, in D1-MSNs or D2-MSNs demonstrated a role for D1-MSNs in mediating positive motivational responses to drugs of abuse and natural reward, whereas D2-MSNs mediated negative responses to natural reward (McClung et al., 2004; Robison and Nestler, 2011). Recent use of D1-Cre and D2-Cre transgenic lines combined with Cre-inducible mouse lines or viral vectors has led to converging evidence that the two NAc projection MSNs, as well as the MSNs in dorsal striatum, have opposing actions on motor, reward, reinforcement, and drug seeking (Kravitz et al., 2010, 2012; Lobo et al., 2010; e.g., Hikida et al., 2010; Ferguson et al., 2011; Tai et al., 2012). Studies using optogenetic or pharmacogenetic tools to selectively activate or inhibit D1-MSNs and D2-MSNs demonstrate that recruitment of the D1-MSNs results in positive responses to rewarding stimuli. In contrast, activation of D2-MSNs produces punishment and suppression of behavior (reviewed by Kravitz and Kreitzer, 2012; Lenz and Lobo, 2013). Similarly, studies using cell-type selective profiling or manipulation of genes or signaling pathways, which are critical for behavioral and synaptic plasticity to drugs of abuse, demonstrate a role for D1-MSNs in molecular regulation of these plasticity events (Valjent et al., 2009; Lobo and Nestler, 2011; Chandra et al., 2013; Smith et al., 2013). However, this is not the case for all molecular signaling pathways as a recent study demonstrated that the positive behavioral responses to cocaine, mediated through VTA-NAc BDNF signaling, occur through BDNF-TrkB signaling in D2-MSNs (Lobo et al., 2010). Furthermore, studies examining gene expression and post-translational histone modifications in D1-MSNs and D2-MSNs after repeated exposure to cocaine reveal that each MSN subtype displays distinct transcriptional regulation after chronic exposure to drugs of abuse (Heiman et al., 2008; Jordi et al., 2013). Together, these studies conducted over the past decade demonstrate a complex and opposing functional and molecular regulation through the MSN subtypes to mediate motivational and molecular plasticity responses to rewarding and aversive stimuli.

Conclusion

In conclusion, the topics reviewed above provide a broad perspective on the emerging view that the neural circuits underlying reward and punishment and negative and positive affect and motivation are intertwined, but that they are being unraveled with the use of new technical and theoretical approaches. We expect that much more progress will be made over the next decade in understanding these circuits, their role in adaptive behavior, and their dysfunction in mental and addiction disorders.