Stress responses and the mesolimbic dopamine system: social contexts and sex differences

Abstract

Organisms react to threats with a variety of behavioral, hormonal, and neurobiological responses. The study of biological responses to stress has historically focused on the hypothalamic-pituitary-adrenal axis, but other systems such as the mesolimbic dopamine system are involved. Behavioral neuroendocrinologists have long recognized the importance of the mesolimbic dopamine system in mediating the effects of hormones on species specific behavior, especially aspects of reproductive behavior. There has been less focus on the role of this system in the context of stress, perhaps due to extensive data outlining its importance in reward or approach-based contexts. However, there is steadily growing evidence that the mesolimbic dopamine neurons have critical effects on behavioral responses to stress. Most of these data have been collected from experiments using a small number of animal model species under a limited set of contexts. This approach has led to important discoveries, but evidence is accumulating that mesolimbic dopamine responses are context dependent. Thus, focusing on a limited number of species under a narrow set of controlled conditions constrains our understanding of how the mesolimbic dopamine system regulates behavior in response to stress. Both affiliative and antagonistic social interactions have important effects on mesolimbic dopamine function, and there is preliminary evidence for sex differences as well. This review will highlight the benefits of expanding this approach, and focus on how social contexts and sex differences can impact mesolimbic dopamine stress responses.

Introduction

Stressful experiences induce a powerful set of behavioral, hormonal, cellular and molecular responses that assist organisms in adapting to the physical and social environment. Studies of physiological stress responses have historically focused on catecholamine responses and the hypothalamic-pituitary adrenal (HPA) axis (Herman et al. 2003; McEwen and Wingfield 2003). However physiological responses to stress are diverse, and over the past 30 years evidence has accumulated that dopamine neurons in the ventral tegmental area (VTA), (which project to limbic regions including the nucleus accumbens, amygdala, hippocampus, and frontal cortex) react strongly to stressful situations (Thierry et al. 1976; Herman et al. 1982; Tidey and Miczek 1996). Reports that the mesolimbic dopamine system reacts to stress were initially slow to attract wide interest, perhaps because they run counter to prevailing views that the pathway is primarily activated by contexts associated with rewards. Despite these headwinds, interest in mesolimbic dopamine responses to stress and aversive contexts is growing. Recent discoveries suggest that there may be distinct populations of VTA neurons that are preferentially activated by rewards or stress. Furthermore individual variation in VTA stress responses has been linked to individual differences in coping responses to stress (Krishnan and Nestler 2010). These discoveries are contributing to our still evolving understanding of the functions of mesolimbic dopamine neurons in behavior (Ikemoto and Panksepp 1999; Wise 2004; Hyman et al. 2006; Berridge 2007; Bromberg-Martin et al. 2010).

How an individual responds behaviorally and physiologically to challenges is influenced by an array of factors including early life experience (Seckl and Meaney 2004), seasonal cues (Nelson and Martin 2007), social environment (DeVries et al. 2007), and sex (Goel and Bale 2009). These factors are known to modulate how the HPA axis responds to stress. However, less is known about how these factors mediate mesolimbic dopamine responses to stress. Indeed, the majority of studies investigating dopaminergic responses to stress have focused on a few species of male rodents under relatively controlled laboratory conditions. Here, I will argue that there will be many benefits to diversifying the contexts in which the activity of the mesolimbic dopamine system is studied. The literature focusing on appetitive aspects of mesolimbic dopamine function has already started this process. A foundation of knowledge was formed by focusing on dopaminergic function in a few model species, under a limited set of controlled conditions (e.g. responses to food rewards or drugs of abuse) (Hyman et al. 2006; Wise 2006). This set the stage for interpreting how the mesolimbic dopamine system functions in more complex social situations. For example, the formation of a pair bond between a male and a female prairie vole induces a dramatic upregulation of dopamine D1 receptors in the nucleus accumbens (NAc), which causes males to attack unfamiliar females (Aragona et al. 2006). These results contrast starkly from observations in rats. Unfamiliar females rats typically induce increased male sexual arousal (Wilson et al. 1963) and increase dopamine release in the NAc (Fiorino et al. 1997). Several lines of evidence suggest that dopamine receptors (including D1 receptors specifically) in the NAc facilitate male sexual motivation in rats (Everitt et al. 1989; Pfaus and Phillips 1991; Liu et al. 1998; Bialy et al. 2010) but see (Moses et al. 1995). Thus, species differences in social organization are associated with divergent effects of dopamine receptors on male behavioral responses to novel females. This example highlights the value of comparative approaches.

This review will examine mesolimbic dopamine responses to aversive contexts (focusing on the VTA and NAc), and then focus on how social context and sex differences modulate these responses. Although most examples will come from a few widely used rodent model systems, this review will highlight the unique insights that can be gained from examining mesolimbic dopamine function under different social contexts, and in species with different social systems.

Mesolimbic dopamine system

Appetitive responses

The mesolimbic dopamine pathway consists of dopaminergic cell bodies in the VTA and its projections to striatal, limbic, and cortical regions. The NAc can be divided into two subregions: the core which has stronger connections to other nuclei within the basal ganglia (Zahm and Heimer 1990) and the shell which has stronger connections to the amygdala and bed nucleus of the stria terminalis (Alheid and Heimer 1988). Both natural rewards such as palatable foods (Baldo and Kelley 2007) and artificial stimuli such as drugs of abuse (Di Chiara et al. 2004) stimulate dopamine release in the NAc. However, it appears the consequences of dopamine release in the NAc are more complex than indicating the presence of a reward. For example, several findings suggest that NAc dopamine release does not signal a hedonic (pleasurable) state. Inhibiting dopaminergic responses in the NAc, either via pharmacological lesion (Berridge et al. 1989) or through genetic inactivation of dopamine synthesis (Cannon and Palmiter 2003), does not block behavioral preferences for rewards. Genetic deletion of dopamine synthesis in the NAc inhibits animals from seeking or working for rewards (Robinson et al. 2005), suggesting that dopamine released in the NAc may be more important for reinforcement (Berridge 2007; Salamone et al. 2007). Electrical recordings of dopamine neurons in the VTA during learning tasks suggest these neurons also play a role in learning. The activity of dopamine neurons increases following unexpected rewards, and over time “burst” responses are stimulated by cues that predict the onset of the reward rather than the reward itself (Schultz et al. 1997). With these discoveries in male rats, mice, and rhesus monkeys, hypotheses on the role of dopamine function in the context of natural and artificial rewards have shifted. Focusing on a few species has facilitated these major advances, but as outlined below, considering species with different social systems can facilitate important insights. This is because the behavioral responses of dopamine neurons are context dependent, and examining species with different social systems expands the range of contexts that can be studied.

There is substantial evidence from a wide range of vertebrates that sexual behavior is perceived as a rewarding experience (Pfaus and Phillips 1991; Domjan 1992; Burns-Cusato et al. 2005; Tenk et al. 2009). However, there are some interesting species differences in how dopamine is released during mating. Rats live in complex social groups, and sexual behavior typically involving intense competition among males for mating opportunities (Calhoun 1962). Under naturalistic conditions or cages in which females can avoid males, females control the pacing of sexual interactions by approaching and withdrawing from males (McClintock and Anisko 1982; Erskine 1989). This behavior has also been observed in Mus musculus (Garey et al. 2002). Intriguingly, dopamine release in the NAc of female rats increases during paced mating but does not increase when males have unrestricted access to females (Mermelstein and Becker 1995; Becker et al. 2001; Jenkins and Becker 2003). In contrast, female golden hamsters (Mesocricetus auratus) are solitary (Gattermann et al. 2001; Gattermann et al. 2008) and have increased NAc dopamine release during mating tests in which male hamsters have unrestricted access to females (Meisel et al. 1993; Kohlert et al. 1997). On the one had, female hamster sexual behavior would appear to be a less dynamic process, as females often adopt the lordosis posture for several minutes at a time (Lisk et al. 1983). However, a closer examination showed that female hamsters can indeed pace mating bouts through perineal movements (Noble 1980). This apparent pacing behavior in hamsters appears to increase the number of successful matings (Noble 1980). Currently, it is unclear whether this more subtle pacing behavior in hamsters is critical for facilitating dopamine release, although there is some indirect evidence for a connection. Cytotoxic lesions of dopamine neurons of the basal forebrain (including nucleus accumbens) prevents increases in mating efficiency that occur with sexual experience in females (Bradley et al. 2005). It would be interesting to determine how female dopamine responses in the NAc vary across species with different levels of mating competition. One possibility is that in species with fewer mating partners, dopamine release might not be as tightly linked to mating behavior compared to other courtship cues. In monogamous zebra finches, male courtship behavior is positively correlated with the number of c-fos positive tyrosine hydroxylase (TH) neurons in the caudal VTA (Goodson et al. 2009), and electrophysiological recording suggest that VTA neurons are more active during male courtship behavior (Huang and Hessler 2008). It’s not yet clear how these neurons respond to courtship behavior in females, and dopamine release following sexual behavior has not yet been examined. An intriguing, yet relatively unexplored question is whether VTA dopamine neurons play a role in the selection of mating partners. Increasing dopaminergic activity systemically appears to decrease the selectivity of mate searching behavior in female starlings (Pawlisch and Riters 2010). These data resemble findings that methamphetamine increases the probability that female rats will mate with less preferred males (Winland et al. 2011) and reports that methamphetamine use in humans is associated with increased numbers of sexual partners (Molitor et al. 1998).

Aversive responses

As hypotheses related to the function of mesolimbic dopamine neurons in appetitive situations have evolved, there has been a growing appreciation that VTA dopamine neurons are important in aversive situations. Several studies using electrophysiological recordings demonstrated that aversive stimuli can alter the activity of VTA neurons, either increasing or decreasing activity (Maeda and Mogenson 1982; Mantz et al. 1989; Guarraci and Kapp 1999; Wang and Tsien 2011). In these studies dopamine neurons were identified by electrophysiological properties, an approach that became controversial after the discovery that some nondopaminergic VTA neurons have similar electrophysiological profiles as dopaminergic neurons (Margolis et al. 2006). It has been suggested that immunostaining TH may be a more accurate approach for identifying dopamine neurons in the VTA (Margolis et al. 2010). Using this approach Ungless and colleagues determined that the majority of VTA neurons in rats that respond to a toe pinch were TH negative and therefore unlikely to be dopamine neurons (Ungless et al. 2004). However, further investigation revealed a more complex relationship between VTA dopamine neurons and aversive stimuli. In rats, it appears that there are different populations of dopamine neurons that may differentially respond to reward related stimuli or aversive stimuli (Fig. 1). Dopamine neurons in the dorsal VTA were observed to be inhibited by footshocks whereas dopamine neurons in the ventral VTA were excited by footshocks (Brischoux et al. 2009). The results provide a potential explanation for why dopamine release in the NAc can be observed both after rewards and aversive experiences. It is important to note that recordings were conducted on anesthetized animals, and the shocks used were much more intense than typically used in aversive conditioning paradigms (Gross et al. 2000; Darvas et al. 2011; Spannuth et al. 2011). It will be important to examine whether similar results are observed in awake behaving animals, and whether different types of stressors induce different profiles of neuronal responses in the VTA.

Figure 1.

Responses of ventral tegmental area dopamine neurons to aversive stimuli. Electrophysiological studies show that there is variability in the responses of VTA dopamine neurons. Work by Brischoux et al. showed that more ventral VTA neurons responded to footshocks with increased activity whereas more dorsal VTA neurons were inhibited by footshock. These changes in activity would be expected to correspond to changes in dopamine (and possibly BDNF) release in axon terminals, although this has not yet been measured directly. The effects of predator odors (which induce delayed dopamine release) on VTA neurons has not been examined.

In addition to changing the activity of VTA neurons, stressful experiences such as restraint (Abercrombie et al. 1989; Copeland et al. 2005; Ling et al. 2009) and footshock (Thierry et al. 1976; Herman et al. 1982; Kalivas and Duffy 1995), induce dopamine release or dopamine turnover in the NAc shell. Interestingly, this rapid dopamine response quickly habituates when restraint is applied on consecutive days (Imperato et al. 1992). The mechanism mediating this plasticity has not yet been identified. In contrast, a surge of dopamine release occurs immediately after rats are removed from restraint tubes, and this response does not diminish over repeated episodes of restraint (Imperato et al. 1992). This dopamine response may serve to reinforce behaviors that were associated with escape, although this hypothesis has not been tested directly. Studies investigating the role of the NAc in the formation of fear memories have reported mixed results (Josselyn et al. 2004). One trend that has emerged is that the NAc appears to be less important for learning that specific cues (a sound or light) predict aversive events like footshock (Riedel et al. 1997; Jongen-Relo et al. 2003; Bradfield and McNally 2010). In contrast, the NAc appears to be more important for learning contextual cues (smells, or the general area in which training occurred) that predict aversive events (Levita et al. 2002). For the most part the literature investigating the electrophysiological responses of VTA neurons to aversive stimuli has not been directly paired with measurements of dopamine release in the NAc (Ungless et al. 2010).

To date the majority of studies examining mesolimbic dopamine responses to aversive experiences have used artificial stimuli such foot shock. A few studies have examined dopaminergic responses to more natural stressors such as predator odor in male rats. Rats exposed to fox odor for 20 min increased dopamine turnover in the amygdala and frontal cortex but not NAc (Morrow et al. 2000). These data agreed with a subsequent study which showed that NAc dopamine responses to predator odors are delayed. A microdialysis study showed that 5 min of fox odor exposure did not increase NAc shell dopamine release until 40 min later (Fig. 1, Bassareo et al. 2002). Dopamine remained elevated for about 20 min and then returned to baseline levels. This study also showed that predator odors induced a rapid increase in frontal cortex dopamine release. Interestingly, although dopamine release in the NAc is not induced rapidly by fox odor, changes in other neurotransmitters do occur. Fox odor rapidly induces an increase in both GABA and glutamate release, but these responses are only observed in individuals with strong behavioral responses to fox odor (Hotsenpiller and Wolf 2003; Venton et al. 2006).

In general, less is known about mesolimbic dopamine responses under aversive contexts compared to appetitive contexts. Our knowledge base is growing, but most data come from studies under controlled conditions and are usually based on artificial stimuli such as restraint or footshock. There are hints that there are important insights to be gained by examining natural stressors. Restraint and footshock induce a rapid increase in NAc dopamine release whereas the response to predator odors is delayed. As discussed below, there is considerable evidence that social factors have important effects on how the mesolimbic dopamine system responds to aversive contexts.

Effects of Social Context

Social factors are known to have important effects on behavioral responses to stress, sometimes exerting a buffering effect. Studies examining the effects of social isolation highlight the importance of positive social interactions on brain and behavior. However, social interactions can also be a powerful source of stress, especially aggressive interactions. Indeed, many species engage in elaborate signals and threats to settle disputes without fighting (Altmann 1962; Reby et al. 2005). When an aggressive contest does occur, the loser usually adopts one of several behavioral coping responses (Veenema et al. 2005; Koolhaas et al. 2007). One widely observed behavioral responses to losing aggressive contests is withdrawal from social contact. This effect has been observed in several species of rodents (Kudryavtseva et al. 1991; Meerlo et al. 1996; Trainor et al. 2011) and birds (Carere et al. 2001). Social withdrawal following aggressive contests has also been observed in primates (Bernstein and Sharpe 1966), although reconciliation between winners and losers is also common (Silk et al. 1996). An additional intriguing observation is that there is a striking degree of individual variation in behavioral responses to social stressors (Wommack and Delville 2003; Schmidt et al. 2009). This variability is also observed within mouse strains such as C57Bl6 (Krishnan et al. 2007; Elliott et al. 2010), which are often assumed to be relatively homogenous as the mice are inbred and housed in relatively constant conditions. There is also growing interest in how the social environment during development impacts the brain and behavior (Marler et al. 2003; Roth et al. 2009; Bales et al. 2011). As reviewed below, social interactions during development and in the mature adult can have long lasting effects on the function of the mesolimbic dopamine system.

Social Instability During Development

Adolescence is a critical time period in which a great deal of neurobiological development occurs (Sisk and Foster 2004), including within the mesolimbic dopamine system (Leslie et al. 1991; Coulter et al. 1996; Tarazi et al. 1998). Developmental studies in rats show that males greatly increase D1-like and D2-like receptors in the NAc at the time of puberty whereas changes in dopamine receptor expression are less pronounced in females (Andersen et al. 1997; Andersen et al. 2002). Most studies examining the effects of stress during development on mesolimbic dopamine function have focused on responses to drugs of abuse. In humans exposure to stress during adolescence increases risk of drug abuse (King and Chassin 2008), a process that is linked to changes in mesolimbic dopamine function (Dietz et al. 2009). It has been hypothesized that social stress during adolescence alters the “programming” of the mesolimibic dopamine system, potentially making it more sensitive to drugs of abuse (McCormick 2010). However, there is also evidence that stress during adolescence can lead to desensitization of the mesolimbic dopamine system (Gatzke-Kopp 2011). Further study is needed to titrate how different forms and durations of stress during development can lead to alternate behavioral phenotypes.

Social instability is one form of stress that can impact behaviors regulated by the mesolimbic dopamine system. One approach to inducing social instability in rodents is to routinely switch cage partners. Two studies in rats reported that this form of social instability, when conducted during adolescence, increases sensitivity to amphetamine (McCormick et al. 2005; Mathews et al. 2008). Amphetamine induces the release of dopamine in the striatum and also blocks dopamine reuptake. This increased dopaminergic activity increases locomotor behavior in rodents and this effect becomes exaggerated with repeated exposure to amphetamine (sensitization). Social instability during adolescence facilitated the sensitization of locomotor responses to amphetamine and this effect was larger in females than males. Social instability also strengthened the formation of conditioned place preferences (which is an indirect measure of reward value) for higher doses of amphetamine in females but not males. Interestingly, social instability during adulthood had no effect on behavioral responses to amphetamine. However, social instability in adulthood is known to affect other aspects of behavior and brain function (Schmidt 2010). When social instability was combined with other forms of stress such as crowding and novel environments, behavioral responses to amphetamine where actually reduced (Kabbaj et al. 2002). The authors hypothesized that the complex set of experiences used may have resembled environmental enrichment, because juvenile rats housed under enriched environmental conditions also are less behaviorally reactive to repeated amphetamine (Bardo et al. 1995).

Effects of Social Isolation

Social interactions are an important factor modulating behavioral and neurobiological responses to stress. Social interactions in rodents have been found to be reinforcing (Stewart and Grupp 1985; Van den Berg et al. 1999), so it is not surprising that social isolation exerts powerful effects on the brain and behavior. A complicating factor in rodent studies of isolation is that there are important sex differences. Results from studies of female rodents are more intuitive, with most evidence pointing towards social isolation exerting an anxiogenic effect (Westenbroek et al. 2003; Grippo et al. 2008) In contrast socially housed male rodents often engage in aggressive interactions in the their home cage (Van Loo et al. 2000; Howerton et al. 2008), which may explain why most data suggest that social isolation has anxiolytic effects in males (Westenbroek et al. 2003; Thorsell et al. 2006; Workman et al. 2008, but see Ruis et al. 1999; Von Frijtag et al. 2000). Most of what is known about the effects of social isolation on brain function comes from studies on male rodents. The dramatic sex differences in the effects of isolation on behavior illustrate the need for a greater emphasis on sex differences in neuroscience research (Beery and Zucker 2011).

In general most studies report that long term social isolation increases male presynaptic dopamine function in the NAc. Socially isolated male rats have higher levels of tyrosine hydroxylase in the NAc shell and have exaggerated dopamine responses to drugs of abuse (Jones et al. 1990; Robbins et al. 1996; Hall et al. 1998). The effects of isolation on dopamine responses in the NAc to stressors may depend on context. Dopamine turnover following footshock was exaggerated in isolated males (Ahmed et al. 1995; Fulford and Marsden 1998) but dopamine turnover was reduced following exploration of a novel environment (Miura et al. 2002). A series of studies suggest reduced dopamine turnover in the NAc shell could influence behavior via reduced activation of D1 receptors. Activation of D1 receptors induces activation (phosphorylation) of the transcription factor cyclic AMP response element binding protein (CREB), which is known to modulate exploratory behavior in novel environments (Fig. 2). In male C57Bl6 mice, social isolation decreases the expression of CREB in the NAc shell (Barrot et al. 2005), which is consistent with a decrease in D1 receptor signaling. Furthermore, adult male C57Bl6 mice and Sprague-Dawley rats that are socially isolated show decreased exploratory behavior in a novel environment (Pinna et al. 2006), a behavior that is regulated by the NAc shell. Overexpression of CREB in the NAc shell increased exploratory behavior, reversing the effects of social isolation (Barrot et al. 2005; Wallace et al. 2009). Activation of CREB also occurs in response to other stressful contexts such as footshock, restraint, and social defeat (Barrot et al. 2002; Muschamp et al. 2011). Although overexpression of CREB in the NAc does not alter the development of conditioned fear to footshock, it does impair the extinction of contextual fear-based memories (Muschamp et al. 2011). This finding is consistent with data from male C57Bl6 mice, in which isolation inhibits the extinction but not formation of fear memories (Pibiri et al. 2008). Intriguingly, housing under semi-natural, enriched conditions appears to mediate at least some of the developmental effects of social isolation on the NAc (Lesting et al. 2005).

Figure 2.

Molecular pathways leading to the activation of CREB (From Purves et al; Neuroscience, 2^nd edition, used with permission). Activation of certain G-protein coupled receptors (such as D1 dopamine receptors) increases the production of cyclic AMP, leading to the phosphorylation of CREB. However, other G-protein receptors (such as D2 dopamine receptors) can have the opposite effect on cyclic AMP production with would inhibit phosphorylation of CREB. Activation of tyrosine kinase receptors (such as when BDNF binds to TrkB) increases phosphorylation of MAP kinase (also known as ERK) which then facilitates phosphorylation of CREB. Neuronal activity can also increase phosphorylation via opening of voltage gated calcium channels.

In summary, the effects of social isolation on mesolimbic dopamine function are context dependent. When exploring novel environments, social isolation reduces dopamine turnover, and this would presumably reduce the activation of CREB by D1 receptors (Fig. 2). Decreased CREB activation would be expected to reduce exploratory behavior, which matches empirical observations in male rats and mice. In contrast, social isolation enhances mesolimbic dopamine responses to footshock, and mesolimbic dopamine signaling has been found to modulate the extinction of fear memories (Pezze and Feldon 2004). These observations may help explain why the extinction of fear memories is inhibited by social isolation. As mentioned previously, the effects of social isolation on neurobiological mechanisms of behavior have been understudied in females. Addressing this deficit should be a priority for future studies.

Social Defeat

As mentioned previously, aggressive interactions can be a powerful source of stress, especially for individuals that lose aggressive contests. Aggressive interactions in general increase either dopamine release or indirect markers of dopamine activity in a wide range of vertebrates (Winberg et al. 1991; Filipenko et al. 2001; Bharati and Goodson 2006; Bondar et al. 2009). Studies of social defeat focus on animals that have been randomly assigned to be exposed to aggressive interactions. Postmortem analyses of dopamine turnover (Louilot et al. 1986; Puglisi-Allegra and Cabib 1990) as well as microdialysis studies (Tidey and Miczek 1996) show that exposure to social defeat in particular increases dopamine release in the NAc but not dorsal striatum. Social isolation exaggerates both behavioral (Ruis et al. 1999) and neurobiological (Isovich et al. 2001) responses to defeat in rats. In addition to these short term responses, social defeat induces long lasting changes in the activity of both the VTA and NAc (Miczek et al. 2008; Krishnan and Nestler 2010). Many of these changes are known to contribute to the effect of defeat on social withdrawal.

In rats, social defeat induces long lasting increases in brain derived neurotrophic factor (BDNF) gene expression and protein within the VTA (Fanous et al. 2010). The effects of growth factors such as BDNF on brain function are not limited to development. Neurotrophins such as BDNF are known to have many important effects on the adult brain (McAllister et al. 1999) and are sensitive to stress (Duman and Monteggia 2006). The receptor for BDNF is the TrkB receptor, and its activation leads to phosphorylation of extracellular signal regulated kinase (ERK) and thymoma viral proto-oncogene (AKT), which are important signaling kinases (Fig. 2). In C57Bl/6 mice ten days of social defeat increases expression of phosphorylated (activated) ERK in the VTA (Iñiguez et al. 2010). Social defeat also decreases the activation (phosphorylation) of AKT, also known as protein kinase B, within the VTA (Krishnan et al. 2008). Overexpression of a constitutively active AKT in the VTA blocks the effect of defeat on social withdrawal. In contrast overexpression of an inactive form of AKT (a dominant negative protein) induces social withdrawal after only three brief episodes of social defeat on a single day. Increased ERK signaling and decreased AKT signaling in the VTA appear to exert their effects on behavior by altering the activity of dopamine neurons. Overexpression of ERK in the VTA increases the excitability of VTA dopamine neurons in slice culture (Iñiguez et al. 2010) whereas drugs that inhibit AKT activity have a similar effect (Krishnan et al. 2008). Social defeat may also increase activity of VTA neurons by increasing sensitivity to glutamate (Covington et al. 2008).

Social defeat increases the firing rate of VTA neurons in in vitro slice preparations (Krishnan et al. 2007). Chronic antidepressant treatment, which blocks social withdrawal responses (Berton et al. 2006; Tsankova et al. 2006), also blocked the effects of social defeat on elevated VTA neuronal activity (Krishnan et al. 2007). These studies measured tonic firing (or single spike) activity, which is more closely associated with steady state dopamine release. However VTA neurons are also capable of burst firing, which associated with larger increases in dopamine release that can last several seconds (Grace 1991). Restraint stress has been observed to increase burst firing in about 80% of VTA neurons (Anstrom and Woodward 2005). As measured via in vivo recordings, increased burst firing in VTA neurons was observed during social defeat in rats (Anstrom et al. 2009). This observation is consistent with observations of increased dopamine release (as measured via microdialysis) in the NAc during social defeat (Tidey and Miczek 1996). Social defeat was also observed to decrease burst firing three weeks after social defeat in mice (Razzoli et al. 2011). These recordings in mice were made under isoflurane anesthesia, indicating that the increased activity in the VTA was a long term change in activity as opposed to a transient response to social threat. Intriguingly, a subpopulation of C57Bl6 mice that are exposed to defeat develop an “unsusceptible” phenotype, characterized by the absence of social withdrawal that is observed in “susceptible” mice (Fig. 3, (Krishnan et al. 2007). When in vivo observations were conducted on susceptible and unsusceptible mice, burst firing of VTA neurons was increased in susceptible mice and negatively correlated with social interaction behavior (Cao et al. 2010). It should be noted that all studies examining the effects of social defeat on VTA neuronal activity used electrophysiological criteria to classify dopamine neurons. While it is highly likely that dopamine neurons were actually examined in these studies, future studies should use tyrosine hydroxylase immunohistochemistry to confirm dopaminergic status.

Figure 3.

Susceptible and unsusceptible phenotypes are associated with different activity profiles in the VTA. In male mice, two phenotypes are induced by social defeat. The susceptible phenotype is associated with increased burst activity by VTA neurons and increased BDNF release in the NAc. Increased dopamine release in the NAc has been observed during social defeat, but has not yet been measured during social interaction tests following social defeat. Unsuceptible mice increase the expression of K+ channels, which prevents increased burst activity and subsequent changes in BDNF release.

Surprisingly, despite numerous observations that social defeat alters the activity of VTA neurons, how this activity translates into altered patterns of dopamine release or dopamine receptor activation within the NAc has not been extensively examined. However, changes in other cellular pathways in the NAc have been reported. Social defeat in C57Bl6 mice increases expression of BDNF protein in the NAc (Berton et al. 2006). Since BDNF mRNA is undetectable in NAc, it is assumed that the source for this BDNF is the VTA. Supporting this observation, selective deletion of BDNF in the VTA blocks the effects of defeat on social withdrawal (Berton et al. 2006). Increased BDNF signaling at the TrkB receptor should increase activation of ERK, and indeed social defeat increases phosphorylation (or activation) of ERK in the NAc (Krishnan et al. 2007). When ERK function is inhibited via overexpression of a dominant negative ERK protein in NAc, the effects of defeat on social avoidance behavior were blocked. Interestingly, overexpression of a constitutively active form of ERK2 in the NAc of mice exposed to only 3 episodes of social defeat on a single day induced social avoidance, but had no effect in control mice. This is important because it demonstrates that increased activation of the VTA-NAc circuit alone is not sufficient to induce the social avoidance phenotype, it must occur in the context of an adverse experience.

Social Subordination

The effects of aggressive interactions on the mesolimbic dopamine system have also been observed under more natural conditions. In contrast to social defeat, in which animals are randomly assigned, studies of social subordination focus on subordinate individuals within a social group. The visible burrow system (VBS) was designed to observe the effects of social status in a more naturalistic setting with more complex social groups (usually with males and females) than typically used in laboratory experiments (Blanchard and Blanchard 1989). The apparatus is about 1 m² and consists of numerous chambers connected with tubes, roughly mimicking burrow systems created by wild rats. After introduction into the VBS, subordinate males lose weight and develop increased basal corticosterone levels compared to dominant males (Blanchard et al. 1995). Several observations also suggest that these subordinate males show changes in dopaminergic tone in the NAc. Subordinate males have decreased dopamine transporter binding and increased D2 receptor binding as well as reduced enkephalin gene expression compared to dominants (Lucas et al. 2004).

Dominance hierarchies can also be observed in social groups of cynomolgus monkeys. Subordinate female monkeys are more vigilant and more likely to spend time alone (Shively et al. 1997). This behavioral phenotype is paired with abnormal hypothalamic-pituitary-adrenal function as well as altered dopamine receptor. Dopaminergic function is sometimes estimated clinically through a haloperidol (a D2 antagonist) challenge (Asnis et al. 1988), as antagonizing D2 receptors facilitates the release of prolactin (Gudelsky and Porter 1980). Subordinate female monkeys had lower prolactin responses to haloperidol than dominant monkeys (Shively 1998), suggesting that D2 function was inhibited. Although these data are not a direct measure of brain D2 function, it is interesting that both subordinate rats and monkeys show evidence of altered D2 receptor function.

Rodent social defeat studies are more controlled than studies focusing on subordinate individuals (because individuals are randomly assigned to stress or control conditions), which makes it easier to establish cause-effect relationships between stress and behavioral/neurobiological responses. In general it appears that stress-induced increases in the activity of VTA dopamine neurons increases the probability of long term changes in behavior following social stress. However, there is growing evidence across a wide range of species that not every individual responds the same way to social stressors. Studies of subordinate individuals could be helpful for providing more insights to individual variation. Studies of subordinates could be considered less controlled because subordinates are not randomly assigned. However, this type of approach may be more closely resemble natural social groups. In addition studying subordinates in more natural social groups may produce more individual variation in behavior, which could be valuable for studying susceptible and resilient phenotypes.

Humans

For many individuals with social anxiety disorders, engaging in social interactions can be an unusually stressful activity. Most studies examining brain function in the context of social anxiety disorders have focused on the amygdala and frontal cortex (Freitas-Ferrari et al. 2010), but some findings suggest that there may be differences in the mesolimbic dopamine system. Male patients with social anxiety disorder were studied using functional magnetic resonance imaging (fMRI) 1 minute before making a public speech. Patients with social anxiety disorder had higher levels of activity in ventral striatum (which includes the NAc and caudate putamen) compared to matched controls (Lorberbaum et al. 2004). Adolescents with social anxiety disorder have also been observed to have heightened NAc responses when anticipating a reward in a monetary incentive delay task (Guyer et al. 2011), which involves reacting to cues that signal monetary reward (Knutson et al. 2001). Efforts to link altered activity patterns in ventral striatum to altered dopamine function have yielded inconsistent results, with one report suggesting decreased striatal dopamine reuptake sites (Tiihonen et al. 1997) and a second report suggesting increased striatal dopamine reuptake sites (van der Wee et al. 2008) in patients with social anxiety disorder. Similarly inconsistent results were found in studies examining D2 receptor binding in adult populations with social anxiety phenotypes (Schneier et al. 2000; Schneier et al. 2009). Although it is possible that altered striatal responsiveness observed with fMRI in social anxiety studies is independent of altered dopamine function, it is also possible that current PET scanning techniques are too limited to detect changes in dopamine function (e.g. changes in dopamine release). Most likely it will take a combination of clever behavioral experimentation and pharmacological manipulations to determine whether social anxiety phenotypes are associated with altered dopamine signaling.

In addition to imaging data, there are some limited measurements of molecular signaling pathways in human postmortem brain samples. Most datasets have compared patients with depression using antidepressants to control populations. For example, patients with depression were observed to have higher BDNF protein levels in NAc (Krishnan et al. 2007) and increased phospho-AKT in the VTA (Krishnan et al. 2008) compared to controls. These results illustrate the complexity of the human brain, as increased BDNF in the NAc corresponds with a susceptible phenotype observed in mice while increased phospho-AKT in the VTA resembles the unsusceptible phenotype. Future studies with larger sample sizes and more detailed life history data will be extremely valuable for determining whether stressful experiences induce the same types of changes in mesolimbic dopamine function that has been observed in animal models.

Sex Differences in Mesolimbic Dopamine Responses to Stress

Many (but certainly not all) sex differences in brain function and behavior are mediated by gonadal steroids. Compared to other hypothalamic and limbic pathways, the mesolimbic dopamine system is not usually considered a steroid-sensitive pathway. However, a variety of anatomical and functional data suggest that steroid hormones act on both cell bodies and terminal fields to modulate dopaminergic function. In rats low levels of androgen receptor mRNA were detected in VTA but not NAc (Simerly et al. 1990) which is consistent with observations that testosterone infusions increase c-fos expression in the VTA but not NAc (DiMeo and Wood 2006). However, testosterone infusions into the NAc of male rats can induce conditioned place (Packard et al. 1997). One possible explanation for these contrasting results is that AR expression in the NAc is plastic. In male California mice, winning aggressive encounters increases AR immunoreactivity in the NAc to amounts similar to the medial amygdala (Fuxjager et al. 2010). AR has also been observed in homologous regions of NAc and VTA in several fish (Forlano et al. 2010; Munchrath and Hofmann 2010) and birds (Maney et al. 2001). Estrogen concentrating cells are also present in the NAc and VTA (Pfaff and Keiner 1973). In situ hybridization and immunohistochemistry studies suggest that these cells express ERβ but not ERα receptors (Shughrue and Merchenthaler 2001; Creutz and Kritzer 2002). However in other species ERα is also reported in homologous regions to NAc and VTA (Munchrath and Hofmann 2010). Despite relatively low levels of estrogen receptor protein expression in the mammalian mesolimbic dopamine system, there is substantial evidence that behaviors dependent on this system are potently regulated by estradiol (Becker 1999). These effects of estrogens could be indirect, or could possibly be mediated by membrane bound estrogen receptors such as GPR30 which has been identified in the NAc (Hazell et al. 2009).

Sex Differences in Response to Nonsocial Stressors

Many studies have outlined strong sex differences in how the NAc responds to reinforcing stimuli such as drugs of abuse (Wood 2011), primarily in rodents. However, much less is known about whether there are sex differences in how stressful contexts affect NAc function. Here the focus will shift to two behavioral responses that are known to be regulated by the NAc. The chronic mild stress method was developed as an animal model of mood disorders (Wilner 2005) and affects several behaviors that are modulated by the mesolimbic dopamine system. Sucrose preferences are commonly measured as an estimate of anhedonia. Intriguingly, even rodents display hedonic reactions immediately after consuming sucrose that closely resemble responses made by humans and primates (Grill and Norgren 1978; Steiner et al. 2001), and decreased consumption of sucrose following stress is mediated in part by the NAc (Muschamp et al. 2011). However, the effects of stress on sucrose preferences are sometimes inconsistent, and this is reflected in results from studies of sex differences. Chronic mild stress was found to reduce sucrose intake in both male and female Sprague-Dawley rats (Grippo et al. 2005) whereas a different study reported that female but not male Sprague-Dawley and Long Evans rats reduced sucrose intake (Konkle et al. 2003). Studies from a third research group have reported that chronic mild stress reduced sucrose intake in both male and female Wistar rats but that the effect was stronger in females (Dalla et al. 2005; Dalla et al. 2008). Social isolation in prairie voles also decreased sucrose preferences in females and males, but the effect was again stronger in females (Grippo et al. 2007). The general pattern that has been reported across lab groups is that both males and female sucrose preferences are sensitive to stress but that on average stronger changes are observed in females.

Stress has also been observed to affect behavior in the forced swim test. In this test, the amount of time an individual spends swimming or floating in a small water-filled cylinder is recorded (Cryan and Mombereau 2004), with floating behavior often interpreted as a behavior related to “despair” (Porsolt et al. 1978). Although it could be argued that floating behavior is a more efficient behavioral response (Nishimura et al. 1988), the forced swim test has been useful for identifying drugs with antidepressant properties. Several molecular pathways in the NAc are known to modulate behavioral responses in the force swim (Pliakas et al. 2001; Shirayama and Chaki 2006). Forced swim testing increases CREB activation in the NAc shell of male rats and inhibition of CREB activity inhibits immobility behavior (Pliakas et al. 2001). Chronic mild stress increased floating behavior in the forced swim test in females but not males, and this sex difference was found to be mediated by ovarian hormones (LaPlant et al. 2009). This study used microarrays to examine the effect of chronic mild stress on gene expression in the NAc and identified several subunits of the NfKB pathway as potential targets that appear to mediate behavioral changes induced by stress. In summary, while there is compelling evidence for sex differences in sucrose preferences and behavior in forced swim tests, less is known about how much sex differences in mesolimbic dopamine function contribute to these behavioral effects.

Sex Differences in Effects of Social Stressors

As mentioned previously, the social defeat model has proven to be extremely robust across laboratories. However, a major obstacle has been the inability to study females due to low aggression levels in female rats and mice. However, some rodent species have different social systems which are associated with much higher levels of female-female aggression. The California mouse (Peromyscus californicus) is a monogamous species in which both males and female aggressively defend territories (Ribble and Salvioni 1990). Laboratory studies show that female residents are very aggressive in resident-intruder tests (Davis and Marler 2003; Silva et al. 2010), and show stronger glucocorticoid responses than males after aggressive encounters (Trainor et al. 2010). Three episodes of social defeat across a three day period reduced social interaction behavior in female but not male California mice (Trainor et al. 2011). Defeat also reduced investigation of social odors in habituation-dishabituation tests conducted in the home cage, again with a stronger effect in females compared to males. Interestingly, the effect of defeat on social interaction behavior in California mouse females becomes stronger over a 4 week period. This suggests that a long term change in gene expression or synaptic remodeling may contribute to the observed behavioral changes. Analyses of brain activity immediately after social interaction testing suggested that some of these changes may occur within the NAc.

Brain activity was assessed indirectly using immunohistochemistry for phosphorylated CREB. The activation of CREB can occur through multiple molecular pathways (Fig. 4), such as BDNF signaling or the activation of D1 dopamine receptors, making it a broad marker of cellular activity. In both the NAc shell and core, females exposed to defeat had more pCREB positive cells than control females whereas this difference was absent in males (Fig. 4). In the NAc shell, the number of pCREB positive cells was negatively correlated with social interaction behavior in both males and females. There was no effect of defeat on the number of pERK positive cells in males or females, and the number of pERK positive cells was not correlated with the number of pCREB cells. A similar dissociation between ERK and CREB phosphorylation in the NAc has been observed in several studies examining the effects of cocaine on brain function (Edwards et al. 2007) and behavior (Tropea et al. 2008). Since the binding of BDNF to its receptor leads to phosphorylation of ERK, this result suggests that an increase in BDNF signaling may not be the causal mechanism driving social withdrawal behavior in females. Thus, social defeat in female California mice induces a reduction in social interaction behavior and this behavioral change is associated with changes in the activity of the NAc. These results parallel previous observations described above in male mice. Preliminary data showed that bilateral infusion of 500 ng of the D1 agonist SKF38393 into the NAc shell reduced social interaction behavior in control females (Campi & Trainor unpublished), thus inducing a phenotype similar to that observed in females exposed to defeat. As discussed above, social stress has been observed to increase the activity of VTA neurons, which would be expected to increase dopamine release in the NAc. However, it is not known whether D1 receptors mediate social withdrawal in male mice exposed to defeat, or whether this effect is solely mediated by increased BDNF signaling (Fig. 3).

Figure 4.

Social interaction behavior in male and female California mice after social defeat. In the apparatus used for testing (A), the interaction zone is indicated by a blue box. Females, but not males exposed to social defeat showed reduced social interaction behavior with a novel same sex mouse (B). *** planned comparison control group versus stress group. The number of pCREB positive cells in the NAc shell was negatively correlated with social interaction time with a novel same sex mouse (C). Filled circles (control males), open circles (stressed males), filled triangles (control females), open triangles (stressed females). Spearman r = −0.37, p < 0.05.

The effect of social stress on mesolimbic dopamine function has also been studied in primates. Male and female cynomolgus monkeys (Macaca fascicularis) form dominance hierarchies with dominant individuals showing higher levels of aggressive behaviors and subordinate monkeys receiving more aggressive behaviors (Adams et al. 1985; Deruiter et al. 1994; van Noordwijk and van Schark 1999). One study examined the relationship between behavioral responses to a low dose of cocaine (which increases dopamine release in the NAc) and dominance status. Subordinate monkeys had stronger locomotor responses to cocaine than dominant monkeys (Morgan et al. 2000), which suggests either an increased dopamine response or increased sensitivity to dopamine. A positron emission tomography (PET) study showed that subordinate males had lower D2 receptor binding in basal ganglia than dominants, and that this difference was absent before social hierarchies were formed (Morgan et al. 2002). Decreased D2 receptor binding in basal ganglia as measured via PET was reported in subordinate females compared to dominant females (Grant et al. 1998). Thus it appears that social subordination has similar effects on striatal D2 binding in both male and female cynomolgus monkeys. However, since males and females have not been compared directly, it is not clear whether the effects of subordination are stronger in males versus females. Another question is whether the observed changes in D2 binding occur broadly across the basal ganglia or whether changes in D2 binding are limited to discrete nuclei (resolution of the instruments in these studies could not distinguish subregions such as ventral or dorsal striatum). Interestingly, decreased D2 binding in subordinate monkeys is consistent with observations of elevated pCREB signaling in female California mice exposed to defeat. Activation of post-synaptic D2 receptors reduces expression of cyclic AMP, and would thus be expected to decrease activation of CREB. However, D2 receptors are also expressed on pre-synaptic dopamine neurons and act as autoreceptors inhibiting dopamine release (Wolf and Roth 1990). Based on the methodology used to measure D2 receptor expression in studies of subordinate individuals, it is not possible to determine whether changes in D2 binding affect pre-synaptic neurons, post-synaptic neurons, or both.

Conclusions

The idea that the mesolimbic dopamine system is more than an approach-based system has gained traction as more experimental data document responses to aversive contexts. Observations that certain types of stressors can induce NAc dopamine release would appear to support hypotheses that NAc dopamine signals the salience of events in addition to participating in reward-based learning. The discovery that different populations of VTA dopamine neurons respond differently to appetitive and aversive contexts suggests the possibility that different VTA neurons may modulate either salience or reward learning (or possibly both). As so often occurs in intense debates, (like the nature-nurture debates of the 20^th century) there may be more than correct explanation for the function of VTA dopamine neurons.

Data contributing to this debate, such electrophysiological studies of VTA dopamine neurons, come primarily from studies using domesticated male rats and mice. Going forward, it will be essential to study the responses of VTA neurons in females, as this review has highlighted that neurobiological and behavioral responses to stress are frequently sex-dependent. It will also be interesting to confirm whether the altered activity of VTA neurons following social defeat occur in tyrosine hydroxylase positive cells. The effects of easily controlled stressors such as footshock or restraint stress have provided important discoveries. It is also clear that social contexts, especially social defeat, can dramatically alter signaling pathways in the mesolimbic dopamine system. Studying species with different sensory systems or social organizations has the potential to provide novel insights into these relationships, and will be a critical component to future research on mesolimbic dopamine stress responses.

Highlights.

• The mesolimbic dopamine system is activated during stress

• This review examines how social factors influence dopaminergic stress responses

• The ventral tegmental area can induce social withdrawal in rodents

• More work is needed in females and in species with different social systems