The effects of early life stress on motivated behaviors: A role for gonadal hormones

Abstract

Motivated behaviors are controlled by the mesocorticolimbic dopamine (DA) system, consisting of projections from the ventral tegmental area (VTA) to the nucleus accumbens (NAc) and prefrontal cortex (PFC), with input from structures including the medial preoptic area (mPOA). Sex differences are present in this circuit, and gonadal hormones (e.g., estradiol and testosterone) are important for regulating DA transmission. Early life stress (ELS) also regulates the mesocorticolimbic DA system. ELS modifies motivated behaviors and the underlying DA circuitry, increasing risk for disorders such as substance use disorder, major depression, and schizophrenia. ELS has been shown to change gonadal hormone signaling in both sexes. Thus, one way that ELS could impact mesocorticolimbic DA is by altering the efficacy of gonadal hormones. This review provides evidence for this idea by integrating the gonadal hormone, motivation, and ELS literature to argue that ELS alters gonadal hormone signaling to impact motivated behavior. We also discuss the importance of these effects in the context of understanding risk and treatments for psychiatric disorders in men and women.

Introduction

Motivation is crucial for survival. It encompasses the processes that allow an individual to regulate its interactions with stimuli in the environment; for example, directing organisms toward stimuli that are rewarding and away from those that are aversive, as well as activating behaviors to seek or avoid different stimuli (Salamone et al. 2016). Eating, reproducing, and providing parental care can all be considered motivated behaviors as they direct an individual toward a specific stimulus (e.g., food or a conspecific) and animals will demonstrate a high level of speed, vigor, or persistence to initiate or maintain these behaviors (Salamone et al. 2016). This can be contrasted with reflexive behaviors, for example, which do not require a high level of behavioral activation. Expression of these behaviors depends on the appropriate function of the neural reward system, including the mesocorticolimbic pathway, which consists primarily of dopaminergic neurons that project from the ventral tegmental area (VTA) to the nucleus accumbens (NAc) and prefrontal cortex (PFC) (Kelley and Berridge 2002, Salamone et al. 2016). The mesocorticolimbic pathway does not work in isolation and is highly influenced by hypothalamic and limbic regions involved in social processing (O’Connell and Hofmann 2011). In addition to dopamine (DA), other signaling molecules, such as endogenous opioids, are important for motivated behavior (Spanagel, Herz and Shippenberg 1992, Devine et al. 1993). Dysregulation of any of these interacting networks and the signaling molecules involved can therefore disrupt motivation for natural rewards and contribute to pathology. Either an excess or a lack of motivation in different contexts can impair an individual’s quality of life and ability to survive and is therefore thought to be maladaptive. As an example, continued seeking of a certain stimulus, such as a drug of abuse, despite negative consequences (e.g., overdose, loss of job, loss of relationships) is often characterized as maladaptive. On the other end of the spectrum, a lack of motivation for life-sustaining behaviors, such as eating or socializing, is also considered maladaptive. These types of dysregulated motivated behavior are linked to the etiology and symptomatology of psychiatric disorders, such as substance use disorder (SUD), major depressive disorder (MDD), and schizophrenia (Dailly et al. 2004, Nestler 1993, Laruelle et al. 1999, Seeman and Seeman 2014).

The link between maladaptive motivated behaviors and psychiatric disorders has prompted much research into factors that contribute to disruptions in reward circuitry. One such factor is early life stress (ELS). For the purposes of this review, ELS is broadly defined as any adverse experiences spanning from prenatal development through the onset of puberty and ranging from mild to severe. In humans, this can take a wide variety of forms including poverty, abuse, neglect, and more. Clinical studies have repeatedly shown that ELS increases risk for motivation-related disorders, including SUD, MDD, and schizophrenia (Enoch 2011, Jumper 1995, Molnar, Buka and Kessler 2001). Much research has focused on how this type of stress directly or indirectly via epigenetic processes affects brain development and there is evidence that it can alter the maturation of brain regions, including those involved in motivated behavior (Bath, Manzano-Nieves and Goodwill 2016, Manzano Nieves et al. 2019, Goodwill et al. 2018, Reincke and Hanganu-Opatz 2017, Honeycutt et al. 2019). However, ELS can also affect gonadal hormones that regulate brain function. This review will argue that an important mechanism by which ELS can alter motivated behavior is via regulating gonadal hormones. Support for this idea will come from integrating data demonstrating that gonadal hormones regulate motivated behavior, with data revealing how ELS affects motivated behavior. Given that gonadal hormones occur at different concentrations in males and females, this review will also highlight sex differences in hormonal mechanisms by which ELS can impact motivated behavior.

Gonadal Hormones and Motivated Behavior

Sexual differentiation of the brain

Sexual differentiation of both the male and female brain is dependent on gonadal hormone signaling, as well as chromosomal sex and environmental factors (McCarthy and Arnold 2011). While emerging data is highlighting the important role of sex chromosome effects in contributing to sex differences in behavior (Arnold, Chen and Itoh 2012, Arnold and Chen 2009), here we focus on gonadal hormonals, as this review argues that these hormones mediate some effects of ELS on motivated behavior. There are several periods in development where gonadal hormones are particularly influential on the brain. The perinatal period represents a sensitive window during which testosterone (through its conversion to estradiol in rodents) triggers masculinization and defeminization processes in the male-typical brain (Amateau and McCarthy 2004, Wright, Burks and McCarthy 2008). Early estradiol is also critical for feminization processes in the female-typical brain (Bakker et al. 2002, Bakker and Brock 2010, Brock, Baum and Bakker 2011). These processes lead to permanent, or organizational, effects on brain development and function. The organization by gonadal hormones, in turn, determines the activational effects of sex hormones on sex-specific behaviors later in life (Lenz et al. 2011). For example, in adult female mammals, circulating ovarian hormones can enact transient effects on the female-organized brain to activate sex-specific adult behavioral responses, such as lordosis in rodents (Kow and Pfaff 1975). Most adult males, on the other hand, will not display lordosis behavior even when primed with the appropriate ovarian hormones (Yamanouchi and Arai 1976). Thus, although both male and female adults have detectable levels of both estrogens and androgens, these hormones do not have the same activational effects in males and females because of the organizational sex differences already present in the brain by adulthood (Sisk and Zehr 2005). The levels of androgens and estrogens available both during development to exert organizational effects, as well as during adulthood to exert activational effects are critical components of male-typical and female-typical brain development and behavior.

Most studies investigating the effects of gonadal hormones on regions involved in motivated behavior have explored the effect of circulating levels of these hormones in adult animals on the mesolimbic portion of the mesocorticolimbic dopaminergic pathway, which involves the release of DA in the NAc (Steinberg et al. 2014). There is also evidence for organizational and activational effects of testosterone and estradiol in other brain regions involved in motivational behaviors and reward processing, including the PFC and the medial preoptic area (mPOA). An in-depth review of the roles of estradiol and testosterone in mediating motivational brain circuitry has been covered elsewhere (Becker and Chartoff 2019, Becker 1999, Becker 2016, Becker and Hu 2008), so here we overview key findings.

Clinical data on estrogens and disorders characterized by changes in motivated behavior

SUD, MDD, and schizophrenia are all characterized by changes in motivated behavior and regulated by gonadal hormones. In general, high levels of estrogens are linked to the dysregulated increase in motivated behavior found in SUD (Bobzean, DeNobrega and Perrotti 2014). In contrast, low levels or dropping estrogen levels are associated with the amotivation that characterizes some patients with depression and schizophrenia (Young et al. 2000, Seeman 1996). Understanding the nuances of how estrogens affect these disorders is critical to elucidating their etiology and finding better treatments.

Although historically men have suffered from SUD at higher rates than women, this sex difference is likely cultural and as more women have the opportunity to access drugs of abuse this disparity is getting smaller (Van Etten and Anthony 2001, Etten, Neumark and Anthony 1999, Becker and Hu 2008). In fact, when women do access some drugs, including MDMA and nicotine, they are more likely to report subjective drugs effects and they escalate their intake more quickly than men (Liechti, Gamma and Vollenweider 2001, Sofuoglu and Mooney 2009), although men have reported greater subjective effects of cocaine than women (Lukas et al. 1996, Sofuoglu et al. 1999). Women also take drugs for different reasons than men, and stress triggers more use in women (Hudson and Stamp 2011). Some of these sex differences may be driven by the higher levels of estrogens in women. For example, estradiol increases the subjective drug effects of amphetamine. Women in the luteal phase of the menstrual cycle, which is characterized by relatively high levels of estrogens, report “feeling” amphetamine more than women in the follicular phase of the menstrual cycle, which is characterized by relatively low levels of estrogens (Justice and de Wit 1999). The same researchers also reported that subjective amphetamine effects were greater during the late follicular phase, when estrogen levels are rising, than the early follicular phase, when estrogen levels are low (Justice and De Wit 2000). To our knowledge, no studies have been conducted to directly investigate the role of estrogens in the escalation of drug taking in humans, which is an important gap in the literature. However, women compared to men exhibit a shorter latency from the onset of drug taking to seeking treatment for abuse of alcohol (Randall et al. 1999) and heroin (Hser, Anglin and Booth 1987). Given that women have higher levels of ovarian hormones, this finding suggests that these hormones potentiate escalation to problematic drug use.

In contrast to SUD, decreased motivation is observed in MDD and schizophrenia. Reduced estrogens may contribute to this amotivation. The sex difference in MDD emerges during puberty (Angold and Worthman 1993, Cyranowski et al. 2000), which could suggest that higher levels of estrogens are a risk factor for women with MDD. However, it seems instead that a portion of women after puberty become sensitive to dropping estradiol levels and this sensitivity contributes to conditions, such as premenstrual dysphoric disorder, postpartum depression, and depressive symptoms in some perimenopausal women (Payne 2003). For example, depressive symptoms increase during the perimenopausal window, when estrogen levels begin to drop (Freeman et al. 2004), and estrogen treatment seems to improve mood in the peri- and postmenopausal periods (Grigoriadis 2002, Schmidt et al. 2015). Similarly, estradiol treatment alleviates depressive symptoms in postpartum women (Ahokas et al. 2001, Gregoire et al. 1996), however its widespread use as a treatment is limited because it antagonizes lactation (Kelsey 1996).

While in MDD dropping levels of estrogens precipitate symptoms in some women, in schizophrenia the higher levels of estrogens in women relative to men may confer resilience. Women tend to have a later schizophrenia onset than men and clinical symptoms worsen during phases of the menstrual cycle when estrogen levels are low (Grigoriadis and Seeman 2002). The idea of estrogens having protective effects against schizophrenia is further supported by a study showing that co-administration of antipsychotics and estrogen treatment reduced symptoms more than antipsychotic treatment alone in female patients (Akhondzadeh et al. 2003). Variations in the ESR1 gene, which codes for ERα, have also been associated with schizophrenia risk in women (Miranda-Angulo et al. 2008).

In sum, MDD and schizophrenia are very complex disorders and there are many contributing factors. Collectively though, these studies suggest that low or dropping estrogens may increase symptoms. One possibility is that they do so by decreasing motivation. More studies are needed to directly test this hypothesis. Also, lacking are neuroimaging studies linking the symptom-reducing effect of estradiol treatment to changes in the mesocorticolimbic system. Given the inadequate treatments for MDD and schizophrenia, more studies in patient populations exploring whether estradiol alleviates amotivation and alters reward circuitry are warranted. That said, as reviewed below, much preclinical work supports the idea that estrogens alter the function of the mesocorticolimbic system.

Clinical data on androgens and disorders characterized by changes in motivated behavior

Another gonadal hormone associated with symptoms of SUD, MDD, and schizophrenia is testosterone. In fact, high levels of testosterone are linked to SUD. For example, young men with high testosterone consume more alcohol, engage in more binge drinking, are more frequently intoxicated, and have a greater incidence of alcohol dependence than men with low testosterone levels (Eriksson et al. 2005, La Grange et al. 1995). Anabolic androgenic steroids (AAS) are synthetic derivatives of testosterone and their use is associated with SUD. Specifically, AAS users are at higher risk for opioid and alcohol abuse than non-users (Arvary and Pope 2000, DuRant et al. 1993, Wines Jr et al. 1999). The role of testosterone in potentiating SUD is consistent with testosterone-increased responsitivy to reward in healthy subjects. Interestingly, women admistered testosterone increase risk taking (van Honk et al. 2004) and show activation of the ventral striatium during reward anticipation (Hermans et al. 2010) compared to controls. Together, these data suggest that testosterone can enhance reward process, which may contribute to SUD.

Lower levels of testosterone have also been linked to MDD. A meta-analysis of studies of testosterone therapy showed that testosterone administration improved Hamilton ratings of depression in men across seven studies (Zarrouf et al. 2009). Testosterone may be especially effective in treating depressive symptoms in aging men (Carnahan and Perry 2004), as there is evidence that lower levels of testosterone is associated with increased risk for depression in older men in particular (Almeida et al. 2008, Seidman and Walsh 1999). Interestingly, the protective effect of testosterone on depression may only be true for men, because depressed women have elevated testosterone levels compared to controls (Baischer et al. 1995). More research is needed to elucidate the role of testosterone and sex differences therein in MDD.

Reduced testosterone levels have also been linked to schizophrenia symptoms. Free and plasma testosterone levels are lower in male schizophrenia patients than in controls (Akhondzadeh et al. 2006). Several lines of evidence suggest that lower testosterone may drive negative symptoms (e.g., flattened affect) rather than positive symptoms (e.g., hallucinations). For example, testosterone levels are significantly lower in male patients with predominantly negative symptoms compared to healthy controls, but no such effect was found for patients with predominantly positive symptoms (Goyal et al. 2004). There is also a negative correlation between testosterone levels and the severity of negative symptoms (Ko et al. 2007). Additionally, testosterone levels also predict cognitive performance in men with schizophrenia, but not in healthy men (Moore et al. 2013).

In sum, alterations in testosterone are associated with psychiatric disorders. High testosterone contributes to SUD, while low testosterone is linked to MDD and schizophrenia symptoms. It is noteworthy that these effects parallel the effects of estrogens in these disorders, and testosterone can be converted to estradiol via aromatase. Given the challenges of dissociating the effects of testosterone from estradiol in humans, it is possible that mechanistically changes in testosterone may affect brain circuits via their conversion to estradiol. Future studies are needed to assess this possibility.

Preclinical data on estrogens and the mesocorticolimbic DA system

Preclinically, the role of estrogens in motivated behavior has been studied most in animal models of SUD. Interest in estrogenic effects was initially sparked by studies revealing sex differences in drug taking behavior. Recall that clinically, women more rapidly progress from the onset of drug taking to seeking treatment (Randall et al. 1999, Hser, Anglin and Booth 1987), which suggests they escalate drug taking more quickly than men. Preclinical studies in rodents are consistent with this finding and reveal that females acquire drug self-administration training procedures faster than males (Hu et al. 2003, Lynch 2006, Roth and Carroll 2004). Compared to male rats, female rats also show more rapid escalation of drug taking when given prolonged access to drugs (Reichel et al. 2012), greater motivation to seek drugs (Cummings et al. 2011, Westenbroek, Perry and Becker 2013), and more relapse-like behaviors (Hudson and Stamp 2011). There is evidence that some of these sex differences are driven by circulating estrogen levels, because animals exhibit faster acquisition of drug self-administration procedures following estradiol administration (Hu et al. 2003, Jackson, Robinson and Becker 2005, Lynch et al. 2001, Perry, Westenbroek and Becker 2013, Roth, Casimir and Carroll 2002). Additionally, female mice exhibit greater conditioned place preference to cocaine during proestrus/estrus, when circulating estradiol levels are high, suggesting that estradiol enhances cocaine reward in females (Calipari et al. 2017).

The link between ovarian hormones and drug self-administration suggests a role for estrogenic regulation of reward circuits. Such regulation is possible because estrogen receptors (ERs) are present throughout the mesocorticolimbic DA system (Creutz and Kritzer 2002, Creutz and Kritzer 2004). For example, the VTA, which is the major source of DA mesocorticolimbic system, is regulated by estradiol. Ovariectomy (the surgical removal of the ovaries) in mice and rats reduces the number of neurons in the VTA that are immunopositive for tyrosine hydroxylase (TH), the rate-limiting enzyme of DA synthesis (Johnson et al. 2010). Hormone replacement in ovariectomized females with 17β-estradiol, the selective ERα agonist propyl-pyrazole-triol (PPT), or the selective ERβ agonist diarylpropionitrile (DPN) attenuates the effects of ovariectomy (Johnson et al. 2010), suggesting that both types of receptors play a role in controlling the population of dopaminergic neurons in the VTA. It is worth noting that ER agonists such as DPN have relatively low selectivity and can bind to both ERα and ERβ at high doses (Carroll et al. 2012). However, complementary work in ER knockout mouse models further support the roles that ERs play in modulating VTA DA. ERα knockout mice that have lacked the receptor throughout development show diminished numbers of TH-positive neurons in the VTA, while ERβ knockout mice show cell counts comparable to control animals (Johnson et al. 2010), suggesting that ERα is more crucial for this effect than ERβ. Collectively, these data indicate that estradiol via ERα activation increases TH, an effect that would enhance the capacity of VTA neurons to synthesize DA.

In addition to increasing the number of TH positive neurons in the VTA, estradiol also alters their physiology. Although basal DA firing in the VTA is highest during estrus (relatively low estrogens) and lowest during proestrus (relatively high estrogens) (Zhang et al. 2008), rats with high estradiol levels show increased sensitivity of VTA DA neurons to both the excitatory effects of ethanol and the inhibitory effects of administered DA acting locally on autoreceptors in the VTA (Vandegrift et al. 2017). These findings suggest that estradiol can enhance both excitatory and inhibitory effects on DA transmission. Ovariectomized rats, on the other hand, show no VTA fMRI response to amphetamine, an effect that was rescued by administration of either selective ERα (16α-lactone-estradiol) or ERβ (DPN) agonists (Sárvári et al. 2014). Overall, these findings point to a key role for estradiol in modulating the effects of drugs on the physiological action of VTA DA neurons.

One target of VTA DA release is the NAc. Therefore, regulation of the VTA by estradiol can impact DA release in this brain area. Consistent with this idea, circulating estradiol in adulthood triggers DA release in the NAc (Alderson and Baum 1981, Becker 1999, Di Paolo, Rouillard and Bédard 1985, Hernandez et al. 1994, Lammers et al. 1999, Landry, Lévesque and Di Paolo 2002). In fact, DA release in the NAc fluctuates throughout the estrous cycle, with stimulated DA release increasing during phases characterized by high levels of estrogens (Thompson and Moss 1997). Extracellular DA in the dorsolateral striatum is higher in female rats in cycle stages with high estradiol than ovariectomized females and males (Xiao and Becker 1994). Increased DA in the NAc can increase reward sensitivity (Drevets et al. 2001, Volkow et al. 1996, Volkow et al. 1999), so it may not be surprising that estradiol alone in the NAc produces conditioned place preference (Walf et al. 2006). There is evidence that the reward-enhancing effects of estrogens in the NAc are exerted particularly through ERβ rather than ERα signaling. For instance, in female mice, estradiol and the ERβ agonist DPN both enhanced cocaine-induced place preference and induced cFOS expression in the NAc, whereas the ERα agonist PPT had neither effect (Satta et al. 2018). Similarly, estradiol and DPN, but not PPT, increase cocaine-induced DA release in the NAc (Yoest, Cummings and Becker 2019). Furthermore, RNA interference-mediated knockdown of ERβ, but not ERα, impaired cocaine-induced place preference in female mice (Satta et al. 2018). These studies point to a specific role for ERβ in mediating the effects of estradiol on DA action in the NAc of female rodents.

As noted, estradiol can affect DA release in the NAc via direct regulation of DA neurons in the VTA. However, estradiol can also regulate DA transmission in the NAc via the modulation of opioid function. Endogenous opioids and their receptors (μ, δ, and κ) are present throughout the mesolimbic DA system (Dilts and Kalivas 1989, Unterwald et al. 1989, Dilts and Kalivas 1990, Mansour et al. 1987). Activation of μ and δ opioid receptors in this circuit can increase DA release, whereas activation of κ opioid receptors reduces DA release (Chartoff et al. 2016, Spanagel, Herz and Shippenberg 1990, Spanagel, Herz and Shippenberg 1992, Ebner et al. 2010). There is not much research on the role of estradiol in regulating opioid receptors in the VTA. However, a recent paper found that a supraphysiological dose of estradiol valerate, known to produce a β-endorphin neuronal deficit, reduced δ opioid receptors in the VTA (Molina-Martínez and Juárez 2020). Yet, how physiological levels of estrogens or the estrous cycle impact opioid receptor expression or function in the VTA is unknown. More research has been conducted in the NAc. Ovariectomy decreases mRNA levels of the endogenous opioid preproenkephalin in the core and shell of the NAc, an effect that was prevented by treatment with estradiol, the ERα agonist PPT, and the ERβ agonist DPN (Le Saux and Di Paolo 2005). Estradiol treatment also increases μ opioid receptors in the NAc (Di Chiara and Imperato 1988, Le Saux and Di Paolo 2005). In contrast, estradiol blunts κ opioid receptor signaling in the NAc (Abraham et al. 2018). A combined estradiol-induced increase in μ opioid receptor signaling and suppression of κ opioid receptor signaling would increase DA transmission in the NAc, providing another mechanism by which estrogens can potentiate DA function.

Motivated behavior is also regulated through DA reuptake, the process by which DA is taken via DA transporters (DAT) into the presynaptic neuron for degradation and resynthesis. Blocking DATs pharmacologically causes DA to stay longer in the synapse and is linked to increased motivation. For example, administration of the DAT inhibitor GBR12909 heightened food reward seeking in a progressive ratio breakpoint task in mice (Young and Geyer 2010, Milienne-Petiot et al. 2017b), as well as increased motivation for rewarding intracranial self-stimulation (Esumi et al. 2013). Similarly, genetic knockdown of DAT in mice results in a heightened progressive ratio breakpoint for seeking a food reward (Cagniard et al. 2006, Milienne-Petiot et al. 2017a). There is evidence that estradiol can regulate DA reuptake, but the direction of the effects on reuptake vary. One study found that 48 hours of estradiol treatment in ovariectomized rats decreased the rate of DA uptake and increased DA clearance time following an injection of DA into the NAc, an effect that would allow for more DA action in the synapse for a longer period (Thompson 1999). In contrast, DA reuptake in the broadly dissected striatum was shown to peak during proestrus in naturally cycling rats, when estradiol levels are highest (Morissette and Di Paolo 1993, Morissette and Paolo 1993), which could be a mechanism designed to compensate for excess DA release, as DA levels in the striatum also peaked during proestrus (Morissette and Di Paolo 1993). Similarly, other studies have demonstrated that estradiol treatment of ovariectomized rats increases binding density at DA uptake sites in the striatum, which would suggest a faster uptake of DA from the synapse (Morissette, Biron and Di Paolo 1990), however, this appears to be true only for nigrostriatal sites, such as the substantia nigra pars compacta, and not true for the NAc (Morissette and Paolo 1993). While the exact mechanism of estradiol effects on DA reuptake in the NAc remains unclear, one possibility is that estrogens directly impact DAT expression. Treatment of ovariectomized females with the selective ERα agonist, 16α-lactone-estradiol, increased DAT mRNA in VTA neurons (Sárvári et al. 2014). However, this effect is not observed following estradiol or DPN treatment. Collectively, these studies suggest that changing levels of estradiol do impact the rate of DA uptake and the density of DA binding sites in certain dopaminergic nuclei. Further research on the effects of estradiol on DA reuptake in specific nuclei of the striatum and the mechanisms that regulate uptake are warranted.

DA signals through two families of receptors: D1-like (which includes D1 and D5 receptors) and D2-like (which includes D2, D3, and D4 receptors). These receptors generally have opposite effects at the cellular level, with D1-like activation stimulating and D2-like activation inhibiting postsynaptic adenylyl cyclase activity (Sibley and Monsma 1992, Hopf et al. 2003, Gingrich and Caron 1993). Additionally, D2-like receptor activation has been linked to increased K+ channel activity and decreased Ca2+ channel activity, effects that can contribute to postsynaptic inhibition, and these types of receptors can also be located presynaptically as inhibitory autoreceptors (Dominguez and Hull 2005, Sibley and Monsma 1992, Gingrich and Caron 1993). Estradiol treatment can increase striatal D1 receptor density in both male (Hruska and Nowak 1988) and ovariectomized female rats (Lévesque and Di Paolo 1989). Similalry, estradiol treatment increases striatal D2 receptor density in male (Hruska, Ludmer and Silbergeld 1980) and female rats (Di Paolo, Poyet and Labrie 1982), including specifically the NAc (Di Paolo et al. 1979) in females. In female rats, estradiol and ERβ agonist (DPN) treatment, but not ERα agonist (PPT) treatment, also prevented an ovariectomy-induced decrease in D2 binding in the NAc core (Le Saux, Morissette and Di Paolo 2006). In many cases, a simultaneous increase in D1 and D2 receptors may not alter the net effect of DA on NAc function as these receptors generally oppose each other in their effects. However, the two receptors have been shown to work together for certain outcomes, in a model known as D1/D2 receptor synergism (Gershanik, Heikkila and Duvoisin 1983, Dziedzicka-Wasylewska 2004). Concurrent activation of D1 and D2 receptors is necessary for DA-mediated stereotyped behaviors (White et al. 1988) and simultaneous D1 and D2 receptor activation in the NAc has rewarding effects in rats (Ikemoto et al. 1997). This synergism between D1 and D2 receptors in the NAc suggests that estradiol-induced increases in D1 and D2 receptor expression may be able to enhance reward processing, despite their apparent opposite actions. On the other hand, ovariectomized female rats show increased D2 receptor expression in the NAc (Gordon and Fields 1989). This may lead to an imbalance in D1 and D2 receptors which could cause hyper- or hypo-activity of the mesolimbic reward system. Furthermore, estradiol treatment of ovariectomized rats has been shown to increase both D1 and D2 receptor density in striatum in response to irreversible peripheral DA receptor blockade (Lévesque and Di Paolo 1991). These studies point to an important role for estradiol not only in potentiating DA signaling in the mesolimbic system, but also in regulation and maintenance of the appropriate balance of D1 to D2 receptor levels.

A focus of the field has been on estrogenic regulation of DA in the NAc, however, DA release into other brain regions is also regulated by estrogens. Specifically, dopaminergic projections from the VTA also regulate the PFC to mediate cognitive control (Floresco and Magyar 2006, Cools 2008) and estrogens can also increase mesocortical DA release (Aubele and Kritzer 2010). Female rats have a higher proportion of PFC-projecting DA neurons in the VTA than males at baseline (Kritzer and Creutz 2008). Estradiol administration increased spinophilin immunoreactivity (suggesting greater spine density) in PFC of nonhuman primates and rats (Khan et al. 2013, Lasley et al. 2004) and increased dendritic spine density in the PFC of aged ovariectomized females (Bailey et al. 2011, Khan et al. 2013, Velázquez-Zamora, Garcia-Segura and González-Burgos 2012). When synapses are specifically assessed, estradiol treatment increases their number in the PFC of rats (Chisholm and Juraska 2012) and restores multisynaptic bouton levels in the aged female primate PFC (Hara et al. 2016). Collectively, this estradiol-induced plasticity could make the PFC more responsive to DA afferents. Moreover, estradiol alters DA function in the PFC. Specifically, estradiol-treated female rats show higher levels of DA and its metabolites in the PFC compared to ovariectomized controls (Sárvári et al. 2014). Additionally, estradiol-treated animals showed an upregulation of DA receptors in the PFC and a more robust blood-oxygen-level dependent signal (BOLD) response to amphetamine in the PFC (Sárvári et al. 2014). Correspondingly, ovariectomy significantly lowers spine density in the PFC (Wallace et al. 2006) and increases the density of DA β-hydroxylase axons in the dorsolateral PFC, suggesting higher rates of DA conversion to norepinephrine (Kritzer and Kohama 1999). Together, these findings demonstrate an enhancing effect of estradiol on DA transmission in the PFC at a variety of endpoints.

The effects of estrogens on the mesocorticolimbic system are summarized in Table 1. When considered together, there is evidence for estrogenic regulation of the mesocorticolimbic system at every part of the circuit. Thus, environmental insults, such as ELS, that can alter estrogen levels would be expected to alter motivated behavior.

Table 1.

Effects of estrogens in the mesocorticolimbic dopamine system.

Brain Region	Effects of Estrogens	Species	Sex	Citat
VTA	increased number of TH- positive cells following ovariectomy	rat	female	Johnson et al. 2010
	enhanced fMRI response to amphetamine	rat	female	Sarvari et al. 2014
	increased DAT	rat	female	Sarvari et al. 2014
NAc	restored DA levels after castration	rat	male	Alderson and Baum 1981
	increased DA turnover	rat	female	Di Paolo, Rouillard and Bédard 1985
	induced c-fos expression	mouse	female	Satta et al. 2018
	enhanced cocaine-induced place preference	mouse	female	Satta et al. 2018
	decreased DA uptake rate	rat	female	Thompson 1999
	increased DA clearance time	rat	female	Thompson 1999
	increased D1 receptor density	rat	male	Hruska and Nowak 1988
	increased D1 receptor density	rat	female	Levesque and Di Paolo 1989
	increased D2 receptor density	rat	male	Hruska, Ludmer and Silbergeld 1980
	increased D2 receptor density	rat	male and female	Di Paolo, Poyet and Labrie 1982
	increased D2 receptor density	rat	female	Di Paolo et al. 1979
	increased preproenkephalin expression	rat	female	Le Saux and Di Paolo 2005
PFC	increased spinophilin expression and number of dendritic spines	rat	female	Khan et al. 2013
	increased number of spines	rhesus monkey	female	Lasley et al. 2004
	increased spine density	rhesus monkey	female	Bailey et al. 2011
	increased number of synaptophysin-positive boutons	rat	female	Chisholm and Juraska 2012
	increased multisynaptic bouton levels	rhesus monkey	female	Hara et al. 2016
	increased levels of DA and its metabolites	rat	female	Sarvari et al 2014
	upregulated DA receptors	rat	female	Sarvari et al. 2014

Preclinical data on androgens and the mesocorticolimbic DA system

As noted, testosterone can be converted to estradiol and estrogens can regulate the mesocorticolimbic system. However, testosterone is also poised to directly regulate structures in the mesocorticolimbic system because there are androgen receptors (AR) present in the VTA (Creutz and Kritzer 2002, Creutz and Kritzer 2004). Testosterone has organizational effects of dopaminergic neurons in the VTA. Specifically, male sheep have more TH-positive cells in the VTA than females, and prenatal testosterone exposure increased TH-positive cell numbers in females (Brown et al. 2015). Furthermore, TH-positive cells in males and testosterone-treated females showed greater density of staining, suggesting increased levels of TH within the neurons (Brown et al. 2015). Testosterone can also have activational effects on the VTA. Central administration of testosterone increased Fos expression in the VTA of the male Syrian hamster without altering AR or ER expression in the brain (DiMeo and Wood 2006). There is one study that reported that DA levels in the VTA of male rats did not change after castration (Mitchell and Stewart 1989). However, there is evidence that testosterone is synthesized locally within the corticolimbic system, as gonadectomy does not eliminate testosterone from the mesocorticolimbic system (Tobiansky 2018), suggesting testosterone plays a crucial role in the normal regulation of this system.

As male rats age and testosterone levels drop (Coquelin and Desjardins 1982), DA metabolism in the VTA also decreases (Goudsmit, Feenstra and Swaab 1990). The AR antagonist flutamide administered prenatally during late gestation decreased production of MAP2, a protein found primarily in dendrites, in the VTA and PFC at both prepubertal and adult timepoints, suggesting a reduction in dendritic arborization (Pallarés et al. 2014). Animals treated with flutamide also exhibited a reduction in the number of TH-positive cells in the VTA at a prepubertal time point, but reached normal cell numbers by adulthood (Pallarés et al. 2014), suggesting testosterone is especially important in mesocorticolimbic transmission during adolescence. Blocking conversion of testosterone into dihydrotestosterone (DHT), a non-aromatizable androgen, by finasteride reduced TH expression in the VTA in adolescent male rats (Li et al. 2018). Together, these findings show that testosterone plays an important role in the modulation of DA transmission at the level of the VTA.

Androgens can also regulate opioids in the mesocorticolimbic system. β-endorphin binds to μ opioid receptors in the VTA to trigger DA release in the NAc (Spanagel et al. 1991). VTA levels of β endorphins are increased after exposure to drugs of abuse, including alcohol (Jarjour, Bai and Gianoulakis 2009) and cannabis (Solinas et al. 2004). Similarly, AAS can increase β-endorphin levels in the male rat VTA (Johansson et al. 1997). This effect may explain some of the rewarding effects of AAS and points to another route by which androgen signaling can enhance mesolimbic DA transmission. More work is needed to assess how variations in physiological levels of testosterone affect the opioid system to impact DA transmission.

Testosterone has similar DA-enhancing effects in the NAc. For example, intranasal testosterone was found to increase DA in the NAc of male rats (de Souza Silva et al. 2009) and there is also evidence that intra-NAc testosterone infusion itself has rewarding properties, explaining the addictive properties of AAS (Packard, Cornell and Alexander 1997). This rewarding effect can be blocked by administration of the DA receptor antagonist α-flupenthixol (Packard, Schroeder and Alexander 1998), demonstrating that testosterone induces its rewarding effects in the NAc by leading to activation of mesolimbic DA receptors. Similarly, castration of male rats reduced DA levels in the NAc, while testosterone, estradiol, and DHT all reversed this effect (Alderson and Baum 1981), showing further support for the idea that both androgens and estrogens can enhance NAc DA transmission. In addition to these activational effects of testosterone in the NAc, there are organizational effects of testosterone. Specifically, brain masculinization by neonatal administration of testosterone in female rats decreased NAc DAT expression compared to both controls and estradiol-treated females (Dib et al. 2018). A decrease in DAT expression in the NAc would lead to a longer period of action for DA in the synapse, suggesting that testosterone exposure early in development causes organizational alterations to the brain that enhance the capacity of DA in the NAc to exert its synaptic effects. Together, these studies point to a role for testosterone in potentiating dopaminergic transmission in the mesolimbic reward system.

Another key region in the mesocorticolimbic circuit, the PFC, is regulated by testosterone. Gonadectomy of adult male rats decreased extracellular DA levels in the mPFC compared to intact controls four days after surgery but increased extracellular DA in the long-term (Aubele and Kritzer 2010). Both effects were rescued by testosterone replacement (Aubele and Kritzer 2010). Aged male rats show a reduction in TH fibers in the PFC, an effect that is not mirrored in aged females (Chisholm, Kim and Juraska 2013). Because testosterone can be converted into estrogens via aromatization, it is not always clear whether these effects on mesolimbic DA functioning are due to testosterone signaling or estrogen signaling. However, some studies have demonstrated that DHT can impact DA transmission. For instance, one study showed that DHT treatment of castrated males restored DA turnover in the NAc to normal levels (Yang and Shieh 2007). The effects of androgens on the mesocorticolimbic system are summarized in Table 2. In summary, while both estrogens and testosterone enhance DA function in the mesocorticolimbic system, there is evidence that some effects directly result from testosterone rather than indirectly from its conversion to estrogens.

Table 2.

Effects of androgens in the mesocorticolimbic dopamine system.

Brain Region	Effects of Androgens	Species	Sex	Citation
VTA	organizational increase in TH-positive neurons	sheep	male an female	Brown et al. 2015
	greater density of TH staining	sheep	male an female	Brown et al. 2015
	increased FOS expression	Syrian hamster	male	DiMeo and Wood 2006
	enhanced dendritic arborization	rat	male	Pallares et al. 2014
	increased beta endorphin levels	rat	male	Johansson et al. 1997
NAc	increased DA release	rat	male	de Souza Silva et al. 2009
	organizational decrease in DAT expression	rat	female	Dib et al. 2018
PFC	increased extracellular DA	rat	male	Aubele and Kritzer 2010

Preclinical data on gonadal hormone regulation of DA in the mPOA

In addition to acting directly on the mesocorticolimbic pathway, estrogens and androgens can also modulate reward processing for reproductive behaviors by acting on brain areas in the social behavior network that have outputs to the mesocorticolimbic system. The mPOA is critical for male sexual behavior, as it integrates relevant sensory input and, in turn, projects to motor regions critical for the copulation and the mesocorticolimbic system to mediate reward (Simerly and Swanson 1986, Simerly and Swanson 1988). Given its integratory role, the mPOA is important for both the consummatory aspects (i.e., execution) of male sexual behavior, as well as some appetitive (i.e., motivational) aspects of male sexual behavior (Will, Hull and Dominguez 2014, Paredes, Highland and Karam 1993, Dominguez and Hull 2005). Early lesion studies found that ablation of the mPOA reduced or eliminated male sexual behavior in rats (Heimer and Larsson 1967, Paredes and Agmo 1992, Paredes et al. 1993, Paredes, Tzschentke and Nakach 1998, Lupo et al. 1983, Hansen et al. 1982, Paredes and Baum 1995), as well as decreased several measures of sexual motivation, including preference for a receptive female over a non-receptive female (Edwards and Einhorn 1986) and precopulatory pursuit behavior (Paredes et al. 1993). The role of the mPOA in male sexual behavior and motivation is modulated by gonadal hormone action. The sexually dimorphic nucleus of the mPOA is larger in males than in females, and this sex difference is organized estradiol converted from perinatal testosterone surge (Bloch and Gorski 1988, Döhler et al. 1984). However, it is not this specific nucleus, but rather the larger mPOA that is necessary for the full expression of male sexual behavior (Arendash and Gorski 1983). When considering hormone effects on the mPOA more broadly, activational effects of gonadal hormones can regulate reproductive behavior. Implants of either testosterone propionate (Davidson 1966) or estradiol (Christensen and Clemens 1975, Davis and Barfield 1979) into the mPOA of male rats castrated in adulthood restored sexual behavior. Furthermore, blockade of aromatization of testosterone into estradiol in the mPOA prevented testosterone from restoring sexual behavior in castrated male rats, revealing that estradiol (converted from testosterone) drives this behavior (Christensen and Clemens 1975, Clancy, Zumpe and Michael 1995, Vagell and McGinnis 1997). Collectively, these studies demonstrate a critical effect of estradiol in controlling male reproductive behaviors.

DA mediates both the appetitive and consummatory behaviors required for male reproduction, in part via regulation of the mPOA (Will et al. 2014, Dominguez and Hull 2005). DA levels in the mPOA increase in response to a sexual stimulus (e.g., female rat in estrus) and during copulation (Hull et al. 1995, Hull et al. 1993). An increase in DA in the mPOA facilitates motivation, genital reflexes, and motor patterns required for copulation (Dominguez and Hull 2005, Hull et al. 1992, Markowski et al. 1994, Hull et al. 1995). Gonadal hormones regulate DA in the mPOA, because gonadectomy reduces DA release in this region (Du, Lorrain and Hull 1998). Testosterone treatment restores DA in the mPOA and normal copulatory behavior (Putnam et al. 2001). Further studies determined the role of specific testosterone metabolites on mPOA DA release: estradiol maintains basal levels of DA in the mPOA, while DHT is required for DA release caused by a sexual stimulus (Putnam, Sato and Hull 2003). One way by which testosterone can facilitate mPOA DA release is via the upregulation of nitric oxide synthesis (Putnam et al. 2005). However, there is much we do not know about how gonadal hormones regulate DA in the mPOA, because few studies have investigated how gonadal hormones regulate the incertohypothalamic DA system. The incertohypothalamic DA system consists of DA-producing cells in the periventricular hypothalamus (A14 group) and the rostral zona incerta (A13), which release DA into the mPOA (Moore 1995, Björklund, Lindvall and Nobin 1975, Lookingland and Moore 1984). Gonadectomy in male and female rats reduces TH immunoreactivity in the incertohypothalamic DA system, an effect reversed by hormone replacement (Sanghera et al. 1991). The similarities between the effect of gonadal hormones on TH in incertohypothalamic system and mesocorticolimbic system suggest that these hormones affect dopaminergic systems similarly throughout the brain. However, TH is just one aspect of DA function and much more research is needed on the effects of estradiol and testosterone on the hypothalamic DA system.

The source of DA for the mPOA is hypothalamic, but the mPOA projects to the mesocorticolimbic dopaminergic system to further regulate motivated behavior (Hull and Dominguez 2007). Specifically, there is an important afferent from the mPOA to the VTA (Tobiansky et al. 2016, Simerly and Swanson 1988), and there is emerging evidence that this projection can be regulated by gonadal hormones. There is an organizational impact of gonadal hormone signaling on this projection, as neonatal overexpression of ERα in mPOA increases the number of TH-positive neurons in the VTA (Peña and Champagne 2015). In adulthood, projections from the mPOA to the VTA continue to modulate DA signaling, with estrogen-sensitive neurons in the mPOA synapsing onto both GABAergic and dopaminergic neurons in the VTA. Indeed, microinjections of estradiol in the mPOA of adult rats enhances DA release in the NAc in response to cocaine administration (Tobiansky et al. 2016), demonstrating that hormonal signaling can also impact this circuit indirectly via the mPOA.

Collectively these studies reveal gonadal hormone regulation of the DA in the mPOA. A focus of this work is on male reproductive behavior. However, given the role of the mPOA in regulating appetitive aspects of natural reward and its ability to modulate the mesocorticolimbic dopaminergic system, it is possible the gonadal hormone regulation of the mPOA also contributes to aspects of SUD, or other disorders where reward circuitry is dysregulated.

Summary: Gonadal hormones and motivated behavior

In summary, estradiol and testosterone facilitate DA synthesis and release. Estradiol can also induce structural changes in the mesocorticolimbic system. A schematic depicting the effects of estrogens and androgens at the VTA-NAc synapse is shown in Figure 1. The consequence of these hormonal effects is altered motivated behavior. As noted, ELS can also alter motivated behavior. Next, we will discuss specific ways by which this can happen, with a focus on ELS-induced changes in gonadal hormones as an important mediating factor. Given that ELS is, by definition, early in life, we will discuss evidence suggesting this stress can induce organizational changes. We will also review data indicating that ELS causes lasting alterations in plasma gonadal hormone levels, which could induce activational effects on motivated behavior.

Figure 1.

This schematic shows how estrogens and androgens regulate the mesolimbic DA system at the VTA-NAc synapse. Estrogens enhance DA activity by increasing the number of VTA DA neurons (blue), increasing DA release, decreasing the rate of DA reuptake, and increasing D1 and D2 receptor density at the postsynaptic neuron (green). Androgens can also enhance DA activity by increasing TH expression, increasing DA release and decreasing the expression of DAT, however, some of the effects of androgens may be mediated by aromatization to estrogens. Different forms of ELS can increase or decrease estrogen and androgen levels in males and females, altering gonadal hormone regulation of the mesolimbic DA system. DA, dopamine; TH, tyrosine hydroxylase; DOPA, L-dihydroxyphenylalanine; DAT, dopamine transporter; VTA, ventral tegmental area.

ELS and Motivated Behavior

Typical methods for modeling ELS in rodents

Much of the research exploring the neurobiological mechanisms by which ELS affects reward circuitry comes from rodent studies using rats and mice. In these species, several animal models of ELS, including both prenatal and postnatal stressors, have been developed (Liu et al. 1997, Pryce and Feldon 2003, Surakul, Weerachatyanukul and Chutabhakdikul 2011, Liu et al. 2000, Ivy et al. 2008, Rice et al. 2008). Some studies employ prenatal stressors aimed at stressing pregnant dams before parturition. The most common approach is restraint stress from days 14 through 21 of pregnancy (Ward and Weisz 1984), but other gestational timepoints and stressors (e.g., chronic variable stress, swim stress, corticosterone administration) are also used (Mueller and Bale 2008, Leuner et al. 2014, Brummelte and Galea 2010). A common postnatal stressor is the maternal separation model. In this manipulation, pre-weaning rat or mouse pups are separated from the dam and removed from the home cage for a period of time ranging from 15 min to 24 h per day (Schmidt, Wang and Meijer 2011). Variations of this model that utilize a shorter period of separation result in increased levels of maternal care behaviors, which is associated with higher stress resiliency and lower anxiety in offspring, whereas variations that utilize a longer period of separation induce impaired maternal care behaviors and are used as models of ELS (Nishi et al. 2013). Another common ELS model is the limited bedding and nesting manipulation, which is typically implemented from postnatal days 2 through 9. Dams in the limited resource environment lack accessing bedding and proper nesting materials, which results in fragmented maternal care that stresses the offspring (Ivy et al. 2008, Rice et al. 2008). Given the variability in ELS manipulations and implementation, it is not surprising that differing ELS protocols can result in different outcomes for the offspring. However, there are many similarities in the way ELS manipulations affect motivated behavior and gonadal hormones.

A role for estradiol in mediating the effects of ELS on motivated behavior

Studies on the effects of ELS on mesocorticolimbic signaling reveal a fairly consistent pattern of downregulation of dopaminergic activity in females. For instance, in the NAc specifically, prenatal restraint stress decreases DA levels in adult female rats (Reynaert et al. 2016) and maternal separation decreased D1 receptor expression in adult female mice (Sasagawa et al. 2017). Further, maternal separation stress also decreased acute stress-induced DA release in the NAc of juvenile (post-weaning but pre-puberty; postnatal day 26–30) male and female rats (McCormick et al. 2002), suggesting DA suppression following ELS. There are several theories as to why this as occurs. One theory is that ELS disturbs the development of the HPA axis, leading to changes in circulating glucocorticoid levels which can impact sex-specific organization of DA circuitry in the early postnatal period (Chocyk et al. 2015, Majcher-Maślanka et al. 2017, Gillies and McArthur 2010, de Kloet et al. 2005). ELS has also been proposed to interfere with the normal maturation of DA circuitry by altering the processes of overproduction and subsequent pruning of dopaminergic synapses (Majcher-Maślanka et al. 2017, Manzano Nieves et al. 2019). Still another theory suggests that ELS may enact its changes in brain function by affecting the timing normal maturational processes in females, with evidence for delayed sexual development (Manzano Nieves et al. 2019), as well as altered maturation of parvalbumin systems in the orbitofrontal cortex (Goodwill et al. 2018). However, one idea that has received less attention is that ELS induces changes in estradiol, which in turn alters DA signaling in the NAc. A summary of the effects of ELS on estradiol levels in males and females can be found in Figures 2 and 3. Recall that estradiol enhances DA signaling in the NAc by multiple mechanisms, including increased DA release via ERβ activation, increased density of D1 and D2 receptors, and increased number of TH-positive neurons in the VTA that project to the NAc (Satta et al. 2018, Le Saux et al. 2006, Lévesque and Di Paolo 1989, Di Paolo et al. 1979, Johnson et al. 2010). Interestingly, both prenatal restraint stress (Ordyan, Fedotova and Pivina 2013, Reynaert et al. 2016) and prenatal exposure to lipopolysaccharide-induced immunological stress (Izvolskaia et al. 2016) lower adult estradiol levels in females relative to their unstressed counterparts (Reynaert et al. 2016). Although more studies are needed, this result suggests that reduced DA signaling in reward circuity in prenatally stressed females may result from changes in estrogens.

Figure 2.

The effects of prenatal stress on plasma levels of androgens and estrogens in male and female rodents.

Figure 3.

The effects of early postnatal stress on plasma levels of androgens and estrogens in male and female rodents.

In unstressed females, estradiol potentiates reward, which could contribute to aspects of SUD (Justice and de Wit 1999, Justice and De Wit 2000). However, abnormally low levels of DA signaling are also linked to addiction risk (Melis, Spiga and Diana 2005), and as noted earlier, low estradiol is associated with MDD and schizophrenia (Young et al. 2000, Seeman 1996). One shared aspect of these disorders, which may be driven by low estradiol is anhedonia. Lower baseline levels of mesolimbic DA can lead to anhedonia in response to natural rewards like food (Melis et al. 2005). This anhedonia can increase risk for substance abuse because drugs of abuse become the only stimulus potent enough to induce a rewarding effect (Garfield, Lubman and Yücel 2013, Hatzigiakoumis et al. 2011). Anhedonia is also reported in patients with MDD and schizophrenia (Gard et al. 2007, Pelizza and Ferrari 2009). There is evidence from rodent studies that prenatal stress increases anhedonia in adult females (Reynaert et al. 2016). This effect was reversed by estradiol treatment (Reynaert et al. 2016). Taken together these studies suggest that anhedonia resulting from prenatal stress in females is mediated by low estradiol. This anhedonic state can contribute to disorders characterized by maladaptive motivated behaviors.

In contrast to the effect of prenatal stress, postnatal stress seems to impact estradiol differently and can do so in a sex-specific manner. Exposure to the limited bedding and nesting manipulation from postnatal day 2–9 significantly increases plasma estradiol in adult male rats, but not in females (Eck et al. 2019). Consistent with the data revealing that high estradiol increases DA function in the NAc, studies have demonstrated enhanced mesolimbic DA signaling following ELS in males. For instance, in the NAc, parental separation in the biparental mandarin vole has been linked to an increase in DA levels and D1 receptor expression in adult males and females, as well as an increase in D2 receptor expression in males only (Yu et al. 2013). In the biparental Octodon degus, parental separation actually increases DAT density in the NAc in both male and female adults (Kunzler et al. 2015). This increased DAT would result in faster DA clearance from the synapse, but perhaps this helps compensate for a stress-induced increase in DA release. Notably, estradiol and ELS causes the same changes in DA release, receptors, and uptake, suggesting that effects of postnatal ELS on NAc DA function in males may be mediated by high estradiol. This elevated estradiol and subsequent enhancement of DA transmission in the mesolimbic system of stressed males could increase the risk of developing later substance abuse issues, as elevated DA transmission in the NAc has been linked to increased reward sensitivity to drugs of abuse, including amphetamine (Bradberry et al. 1991, Piazza et al. 1991) and cocaine (Stacy Hooks et al. 1992, Hooks et al. 1991).

Rodent models of ELS typically alter maternal care, such as increasing time apart from pups in the maternal separation model or causing fragmented maternal care in the limited bedding and nesting model (Walker et al. 2017). However, there is variability in maternal care, even in unstressed dams. There is a large literature on how individual differences in maternal care affect estradiol signaling in hypothalamic brain regions that regulate reward circuitry. The paraventricular nucleus (PVN) of the hypothalamus and mPOA also interact with the mesolimbic pathway to increase DA release in the NAc (Numan and Stolzenberg 2009, Melis et al. 2007, Succu et al. 2007). Adult female offspring of rat dams that engage in low levels of licking and grooming (LG) behaviors exhibit increased ERα expression in the PVN (Cameron et al. 2008). In contrast, ERα expression in the mPOA is increased in high LG adult female offspring compared to low LG adult offspring (Champagne et al. 2003). It is unclear exactly how the altered maternal care in postnatal stress models would regulate ERs in these regions, but, based on these data, their regulation is likely. Future studies investigating estrogenic mediation of ELS effects on motivated behavior will also need to consider changes in ERs in afferents to mesocorticolimbic regions.

A role for androgens in mediating the effects of ELS on motivated behavior

As noted, testosterone has organizational and activational effects that masculinize the brain. ELS-induced changes in either of these surges could alter motivated behaviors. Recall that testosterone increases DA release in the NAc and PFC, likely through its ability to increase TH, and thus DA synthesis, in the VTA (Brown et al. 2015, Pallarés et al. 2014, Li et al. 2018). Moreover, DA release in the NAc can further be potentiated by androgen-induced increases in β-endorphin (Johansson et al. 1997). Therefore, increases in testosterone facilitate DA function in the mesocorticolimbic circuit.

There is some evidence that ELS causes lasting changes in testosterone that persist into adulthood, although the degree and consistency of these changes seems to depend on time of exposure and type of stressor used. A summary of the effects of different types of ELS on androgen levels in males and females can be found in Figures 2 and 3. Prenatal stress reliably lowers adult plasma testosterone in male rats (Anderson et al. 1986, García-Vargas et al. 2019, Reynaert et al. 2016), although this effect was not observed in one mouse study, perhaps indicating a species difference (Crump and Chevins 1989). Lower testosterone would be expected to reduce DA function in reward circuits, an effect that could contribute to symptoms of amotivation that can occur in MDD and schizophrenia. Interestingly, one study by Reynaert and colleagues tested the effect of prenatal restraint stress on levels of both testosterone and DHT in adult male rats. The authors found the expected decrease in testosterone, but also an increase in DHT (Reynaert et al. 2016). Because testosterone is synthesized into DHT via the enzyme 5α-reductase, this result is consistent with increased 5α-reductase activity observed in 10-day old rats following prenatal stress (Reznikov and Tarasenko 2007). Reynaert and colleagues (Reynaert et al. 2016) also found that prenatal restraint stress increased preference for natural rewards in adult male rats, an effect that was mimicked by DHT in unstressed males and abolished by castration in stressed males. Taken together, these studies suggest that typically prenatal stress reduces T, but that change in testosterone can be accompanied by an increase in DHT. Dissociating the effects of these changes in testosterone and DHT on reward circuitry will be critical to understand how gonadal hormones mediate effects of ELS on motivated behavior.

Stress manipulations that occur postnatally do not always result in changes in testosterone, but when testosterone is altered, it is also decreased. One study found that maternal separation in male mice led to a reduction in testosterone levels in adulthood (Tsuda, Yamaguchi and Ogawa 2011). Yet, another study that used maternal separation in rats demonstrated a more nuanced effect on testosterone: there were no differences in plasma testosterone levels in adult male rats, but swim stress caused a smaller in increase in testosterone levels in adult male rats exposed to maternal separation vs. the control housing condition (Veenema et al. 2006). In contrast, the limited bedding and nesting model did not alter adult plasma testosterone levels in males (Eck et al. 2019), indicating different postnatal stressors have different effects on adult testosterone. Collectively, these studies indicate that prenatal stress is a more potent regulator of adult testosterone levels than postnatal stress, and thus prenatal stress is more likely to influence motivated behavior via changes in testosterone levels.

Not only can ELS change adult testosterone levels, but there is evidence that ELS alters the early androgen exposure that masculinizes the brain. There are sex differences in dopaminergic mesocorticolimbic circuits that are observed throughout development and may be critical for driving aspects of sex-specific reproductive behavior (Gillies et al. 2014). Although some these early sex differences are driven by sex chromosome complement (Sibug et al. 1996, Seney et al. 2013, Arnold 2014), testosterone treatment also contributes to sexual differentiation of this system. In fact, prenatal testosterone exposure increases TH-positive neurons in the VTA (Brown et al. 2015) and catecholamine activity in the frontal cortex of rats (Stewart and Rajabi 1994). Similarly, a sex difference in impulsivity, which is mediated by the mesocortical system, is organized by perinatal testosterone exposure (Bayless, Darling and Daniel 2013). Many studies have linked prenatal stress to altered androgen exposure and subsequent demasculinization in males, and in some cases, masculinization in females (Ward 1972, Dahlöf, Hård and Larsson 1978, Sachser, Hennessy and Kaiser 2011). One proxy for androgen exposure is anogenital distance (AGD), such that a reduction in AGD is associated with lower prenatal androgen exposure in both sexes (Clemens, Gladue and Coniglio 1978, van den Driesche et al. 2011, Hotchkiss et al. 2007). Prenatal stress in rodents and humans is typically associated with shorter AGDs in males and longer AGDs in females (Barrett et al. 2013, Morgan and Bale 2011, Desaulniers, Lamberson and Safranski 2016, Vom Saal et al. 1990). Interestingly, postnatal stress decreased AGD in both male and female juvenile rats, suggesting that stress during the postnatal critical period when androgens can still affect the brain may also alter masculinization (Eck et al. 2019). These changes in early androgen exposure have been linked to reduced reproductive behaviors, including decreased preferences for sexual odors, which can be influenced by reward circuitry (Ward 1972, Vom Saal and Bronson 1978, Zehr, Gans and McClintock 2001, Hotchkiss et al. 2002, Rhees et al. 1997, Freeman, Sheehan and Ophir 2019). However, how stress-induced changes in prenatal androgen exposure regulate behaviors more directly related to those observed in pathological conditions (e.g., self-administration, anhedonia) is unclear, and much more research is needed.

Conclusions and Implications

It is well established that ELS contributes to disorders characterized by changes in motivation. There are multiple mechanisms by which this can occur. However, given the link between gonadal hormones and reward circuitry, as well as the evidence that ELS can alter gonadal hormone levels, we propose that ELS-induced changes in gonadal hormones are one important way by which stress can increase risk for certain psychiatric disorders. More research is needed to fully characterize hormonal changes induced by ELS in rodents and humans, particularly in patient populations. Such research is worthy of investment because manipulating gonadal hormones is relatively easy to accomplish. However, it is important to note that enthusiasm for hormone replacement therapies dwindled after initial findings from the Women’s Health Initiative reported increased risk of invasive breast cancer and coronary heart disease after the treatment of postmenopausal women with estrogen plus progestin (Rossouw et al. 2002). These surprisingly negative outcomes have been attributed to the timing of replacement, and contrast to the beneficial effects observed if replacement is started within 10 years of menopause (Kling and Manson 2017, Rocca, Grossardt and Shuster 2010). The Women’s Health Initiative did not focus on psychiatric disorders, but limited evidence suggests a beneficial effect of estrogen therapy. As noted, estradiol replacement reduces the incidence depression in vulnerable women as they go through menopause (Schmidt et al. 2015), and perhaps this occurs via estrogenic regulation of reward circuitry. Moreover, estrogen administration as an adjunctive therapy to antipsychotic medication is more effective at reducing schizophrenia symptoms than antipsychotic medication alone (Akhondzadeh et al. 2003). Much more research is needed. However, it is possible that in the future, such hormone treatments can be extended to other patient populations who have experienced ELS-induced endocrine changes to better alleviate symptoms and improve patient outcomes.

Highlights.

• Mesocorticolimbic dopamine signaling underlies motivated behaviors

• Mesocorticolimbic dopamine is modulated by gonadal hormones

• Gonadal hormones may mediate some early life stress-induced changes in dopamine

• Gonadal hormones are a potential therapeutic target for motivation-related deficits