Estradiol, Dopamine and Motivation

Abstract

The gonadal hormone estradiol modulates mesolimbic dopamine systems in the female rat. This modulatory effect is thought to be responsible for the observed effects of estradiol on motivated behaviors. Dopamine acting in the nucleus accumbens is thought to be important for the attribution of incentive motivational properties to cues that predict reward delivery, while dopamine in the striatum is associated with the expression of repetitive or stereotyped behaviors. Elevated concentrations of estradiol are associated with increased motivation for sex or cues associated with access to a mate, while simultaneously attenuating motivation for food. This shift in motivational salience is important for adaptive choice behavior in the natural environment. Additionally, estradiol’s adaptive effects on motivation can be maladaptive when increasing motivation for non-natural reinforcers, such as drugs of abuse. Here we discuss the effect of estradiol on mesotelencephalic dopamine transmission and subsequent effects on motivated behaviors.

Graphical Abstract

Estradiol modulates mesolimbic dopamine systems in the female rat, enhancing motivation for sex while attenuating motivation for food. This shift in motivational salience is adaptive in the natural environment.

1. Introduction

The gonadal hormone estradiol has extensive effects on multiple neural systems. In addition to its primary role in reproduction, estradiol plays a role in sexual differentiation of the nervous system, adult neurogenesis and neuroprotection, and synaptic plasticity [1–3]. In the striatum, estradiol rapidly modulates dopamine (DA) release by binding to membrane estradiol receptors (mER) [3–5]. As DA release in this area is crucial for the expression of motivated behaviors, modulation of striatal DA by brain estrogens may underlie observed sex differences in motivation for both natural and learned rewards, as well as estrous cycle dependent variation in motivation in female rodents.

1.1 Neural basis of motivation

The mesolimbic dopamine circuitry has been strongly implicated in motivational processes. Broadly defined, motivation refers to an internal drive to engage in a specific behavior, typically in pursuit of a reward or reinforcer. This can refer to natural reward seeking behaviors, such as those directed towards obtaining food and water, sex, or social interaction, as well as learned behaviors, including behaviors involved in drug seeking and taking. However, this broad conceptualization of motivation does not fully reflect the complexity of motivated behaviors or the underlying neural systems. A full appreciation of the diverse constellation of behaviors that fall under the umbrella of motivation requires recognition not only of the differences between the behaviors themselves and various aspects of a behavior (i.e., appetitive vs. consummatory), but also the conditions under which those behaviors are expressed.

While the specific neural pathways controlling a particular behavior may be distinct, they also overlap in several important ways. One neural system that plays a crucial role in the expression of all motivated behaviors is the striatum. The striatum is commonly divided into two separate but interacting pathways: the ventral striatum, which is largely innervated by the ventral tegmental area (VTA) and includes the nucleus accumbens (NAc) and ventromedial caudate-putamen, and the dorsal striatum, which receives dopaminergic projections from the substantia nigra and incorporates the remaining majority of the caudate-putamen [6]. Within these diverse structures there is significant heterogeneity in the types of cells that are expressed, the functional targets of those cells, as well as differences in the inputs and receptors that control their activity [7].

Unsurprisingly, the diversity of brain structures involved in motivation parallels the numerous processes that contribute to the expression of motivated behaviors. The NAc has been shown to play a crucial role in generating rewarding or reinforcing aspects of motivation, as well as learning about associations between primary reinforcers and the cues that predict them [8–10]. DA acting in the NAc is implicated in the attribution of motivational or reinforcing properties – often referred to as “incentive salience” – to cues that predict reward [11]. The NAc may be uniquely poised to evaluate and predict rewards, as it receives glutamatergic inputs from the hippocampus, prefrontal cortex, and amygdala as well as GABAergic connections that participate in action selection and coordination of motor outputs [12, 7]. Additionally, NAc dopamine systems may act to energize responding for rewards, as Pavlovian approach behavior is significantly correlated with the magnitude of DA release in the NAc core [13].

The dorsal striatum is largely implicated in establishing and expressing habitual behaviors and is particularly important in the sensorimotor aspects of motivation. Specific areas of the dorsal striatum are required for the progression from instrumental responding for a reward to reflexive or compulsive responding [9]. The diverse areas involved in motivation reflect diversity in the processes underlying these psychological processes. Motivational circuitry, then, requires not only the sensorimotor systems that are crucial for the evaluation of stimuli and subsequent expression of targeted behaviors, but also relies on an energizing component that modulates motivation for various rewards as a function of the organism’s adaptive needs.

DA systems play a dynamic role in reinforcement learning. During Pavlovian conditioning, DA neurons initially fire in response to a primary reinforcer or novel stimulus, and these increases in firing rate are associated with orienting and approach behaviors [14, 15]. After continued training, however, DA neurons stop firing at reward delivery, and instead fire at the presentation of the predictive cue [14, 15]. It is hypothesized that these increases in DA neurons, and the observed shift in firing from reward to cue, are the neurobiological correlates of “wanting,” a specific form of incentive motivation often seen in Pavlovian conditioning that promotes approach toward and consumption of rewards [16]. During initial trials the presentation of the reward elicits increased DA signaling and therefore increased motivation, as measured by increases in appetitive behaviors. Then, after training, the cue gains motivational salience, and the rise in DA to the cue leads to transient increases in “wanting” that are associated with behaviors that prepare the animal for reward delivery [16].

Additional observations that, after Pavlovian condition, reward omission results in a decrease in DA firing rates, have also indicated that DA functions as a learning signal that continually updates information about rewards and the cues that predict them [14, 15]. As evidenced by the considerable controversy over the function of DA, to say that DA is equivalent to motivation is far too simplistic, but it is clear that striatal DA is associated with anticipation for reward, both in motivating goal directed behaviors and in value calculations during reinforcement learning.

1.2 Sex differences in motivation

Much of what is known about the effect of gonadal hormones on DA comes from research on sex differences in motivation and the underlying neural systems. There are significant sex differences in both the organization of the central nervous system and in the effects of gonadal steroids in activating specific circuits. This sexual dimorphism likely reflects differences in the environmental pressures imposed on males and females over the course of evolution. While successful reproduction for males, particularly males who do not contribute to parental care, depends on the ability to mate with and inseminate as many females as possible, female reproductive success requires considerably more effort [17]. Like males, females should be motivated to find a mate; however, they must also dedicate considerable resources to gestation, parturition, and maternal care [18, 19]. Additionally, the inherent dangers associated with reproduction, due not only to the risk of disease but also predation and general injury from the rigorous copulatory act, prevent females from being highly motivated for sex at times when conception is unlikely to occur, at least in species where males do not offer continued protection [17]. Therefore, female sexual motivation is closely timed with ovulation in most species, and gonadal hormones that indicate whether the female is fertile or not can serve as physiological signals that modulate the neural systems responsible for changes in motivation [20].

A considerable body of work has demonstrated that DA functional activity varies with reproductive state. Ovariectomized (OVX) females have significantly lower basal levels of extracellular DA than castrated (CAST) males, and although estradiol treatment increases basal DA concentrations in OVX females, levels remain significantly lower than is seen in both intact and CAST males [21]. Expression of striatal D1 receptors is 10% higher in males than in females, and although there are no sex differences in D2 receptor densities, there is a sexually dimorphic effect of estradiol on D2 receptors, where estradiol rapidly down regulates D2 binding in females but not males [22]. There are also differences in the behavioral response to dopaminergic agonists, where females display faster escalation of cocaine taking behavior and work harder for cocaine reward on a progressive ratio schedule [23]. It is hypothesized that exposure to gonadal hormones during sensitive periods of development first results in differential organization of neural substrates for motivation in males and females, which then contribute to differences in the effect of estradiol on DA release [18, 19, 24]. It appears that perinatal testosterone release, which is then aromatized to estradiol in specific brain regions, serves not only to masculinize the brain, but also to defeminize brain areas involved in the expression of female specific behaviors, such as the lordosis posture [1]. The feminized brain, then, results from the absence of testosterone and estradiol during this sensitive period of development [1, 19]. Additionally, sex differences in the hormonal profiles of pubertal development influence patterns of neuronal and dendritic pruning, resulting in further differences in the organization of brain areas crucial for the expression of sex specific behaviors [19].

2. Estradiol and DA

Perhaps largely due to the sexual dimorphic organization of the nervous system, males and females have very different circulating levels of gonadal hormones in adulthood, and the ability of these hormones to act on brain systems is also sexually dimorphic. Importantly, estradiol has been demonstrated to have significant modulatory effects on DA systems in females but has not been shown to enhance striatal DA release in males [4, 26]. The mechanism by which estradiol is able to act on DA systems has not yet been fully elucidated, though significant strides have been made in recent years. Estradiol has both acute and chronic effects on functional activity affecting DA release, reuptake, and some of the downstream targets of DA receptor activation.

Specifically, exogenously administered estradiol has biphasic effects on D2 receptor binding. In addition to its ability to acutely down regulate D2 binding in OVX females, chronic estradiol has been shown to increase D2 binding sites in both striatum and NAc core, likely through an effect on ERβ [22, 25]. Chronic treatment with estradiol or the selective ERβ agonist diarylpropionitrile (DPN) also prevents an OVX induced decrease of DA transporter (DAT) expression in the medial striatum, and the effect on both D2 receptors and DAT is seen with no effect on mRNA expression, although these later changes may be consequences of increased DA release, rather than a direct effect of estradiol on receptor expression [3]. Acute treatment with estradiol has also be shown to enhance the AMPH stimulated DA response in striatum of females but not males, both in vivo and in vitro [26, 27]. K+ stimulated DA release in both striatum and NAc is also enhanced by acute estradiol treatment [26, 28].

Estradiol may also have downstream effects on mesolimbic DA neurons. In cell culture, pre-treatment with 17-β-estradiol reduces D2 receptor inhibition of adenylate cyclase activity, and enhances D1 receptor activation of adenylate cyclase [29]. Additionally, estradiol treatment decreases expression of Regulator of Gprotein Signaling 9-2 (RGS9-2), a GTPase activating protein (GAP) associated with D2 receptor signaling [30].

The work of Mermelstein et al. suggests that estradiol may be influencing DA release via GABA medium spiny neurons [31]. Acute application of estradiol rapidly decreases Ca²⁺ currents in GABAergic striatal neurons in tissue from females but not males. The same effect is seen in estradiol conjugated with bovine serum albumin (BSA), and is irreversible when GTP inactivation is prevented, indicating that estradiol is acting on a G-protein coupled receptor on the cell membrane [31]. Estradiol conjugated with BSA also mimics the effect of estradiol in potentiated amphetamine induced striatal dopamine release, further supporting the existence of mER in striatum [32].

Subsequent work has implicated metabotropic glutamate receptors (mGluRs), functionally coupled to the classic estradiol receptors ERα and ERβ via caveolin proteins, in coordinating the membrane effects of estradiol on striatal neurons [5, 33]. Specifically, estradiol binding to ERα activates mGluR5 (a group I mGluR), resulting in activation of a Gq subunit and increased CREB phosphorylation, while estradiol binding to both ERα and ERβ associated mGluR3 (a group II mGluR) inhibits CREB phosphorylation via activation of a Gi/o subunit [5]. mGluR3 may be of particular importance, as inhibition of Ca²⁺ current-dependent CREB phosphorylation by estradiol requires activation of group II mGluRs [5]. The functional association of these receptors appears to be organized by specific caveolin proteins, CAV1 for mGluR5 and CAV3 for mGluR3 [34, 5]. Importantly, estradiol’s enhancement of cocaine induced locomotor sensitization requires mGluR5, suggesting that ERα may play a crucial role in the behavioral effects of estradiol on midbrain DA systems [33].

2.1 Estradiol and motivation for sex

As briefly mentioned previously, aspects of motivated behaviors can be described as being either appetitive or consummatory. Consummatory behaviors include those involved with the actual consumption of a reward, eating in the case of food reward or copulation in the case of sexual reward. Appetitive behaviors, on the other hand, are more variable, and serve to prepare the animal to engage in consummatory behaviors [6]. This can include reward seeking, approach, and other behaviors targeted at locating and procuring rewards.

Research in male rats has implicated DA in both appetitive and consummatory aspects of sexual behavior. DA levels in both the NAc and striatum rise during the presentation of a sexually receptive female and subsequent copulation in both experienced and sexually naive rats [35–37]. Additionally, pharmacological inactivation of DA receptors in the ventral striatum increases, while administration of a DA agonist decreases, latency to mount and intromit in male rats [6]. However, reproductive behavior in the female differs from that of males in several important ways. While intact male rats are continuously capable of engaging in sexual behavior, a surge of estradiol at proestrus and the subsequent rise in progesterone is required for the onset of sexual receptivity in females [38]. In addition, the conditions used to assay male sexual behavior – where the male has free access to the female and the female is not allowed to escape - are not rewarding for female rats, and do not allow for expression of the full complement of female sexual behaviors. In a semi-natural environment, females will actively pace sexual behavior by running away from the male at a regular interval [39–41]. This pattern of behavior increases the latency between bouts of intromissions and allows for activation of a neuroendocrine reflex that increases the probability of conception [39, 18].

Under these conditions, females will express a complex pattern of behaviors that solicit the male to approach and then mount [41, 39]. These behaviors include ear wiggling, hops and darts, and other general approach behaviors, and serve not only to attract attention from conspecifics, but also to hold the male’s attention between intromissions [39–41]. Female sexual behaviors are crucial for pacing, particularly in natural group mating settings, and serve as evidence that females play an active role in mating [40].

Receptive females develop a conditioned place preference following paced mating behavior, and will readily work for access to a mate when they are able to pace the rate of copulation, indicating that the female will find copulation to be rewarding under her preferred conditions [42, 43, 20]. Importantly, extracellular DA levels in striatum and NAc rise during sexual behavior only when copulation occurs at the female’s preferred interval [44, 45]. Additionally, the greatest increase in DA release is typically seen prior to the male’s intromission, rather than during intromission, and the increase in DA in not seen during the first few intromissions, indicating a possible role for anticipatory learning in these dynamic changes in DA concentration during rewarding sexual behavior [46].

Observations of the effect of DA manipulations on sexual behavior have highlighted the distinction between sexual arousal or motivation and sexual performance. In male rats, alteration of DA signaling in the ventral striatum has little effect on the consummatory aspects of sexual behavior, while significantly reducing measures of sexual motivation and arousal. Conversely, depletion of DA in the dorsal striatum disrupts normal consummatory sequences, likely through an impairment of motor behavior [6].

In females, observations that extracellular DA levels in the NAc rise in anticipation of mating when it occurs at the female’s preferred interval, and not due to coital stimulation or general social interaction, indicates that NAc DA is associated with anticipatory motivational processes, rather than the sensorimotor aspects of sexual behavior [46, 47]. Unfortunately, behavioral tests that operationally measure female sexual motivation have only recently come into use [20]. Previously, lordosis quotient, ear wiggling, and the number of hops and darts have been used as measures of female sexual arousal and proceptivity [48]. However, these reflexive behaviors may be neuroanatomically dissociable from sexual motivation, as was seen in the dissociation between male sexual performance and rates of operant responding for access to a mate. Future work using operant tasks that objectively measure motivation for sex that is rewarding to the female without confounds of reflexive behaviors is necessary to illuminate the precise effects of estradiol and DA on female sexual motivation [20].

2.2 Estradiol and Motivation for food

In the natural environment, organisms are constantly faced with alternative opportunities to engage in different behaviors. Decisions about which behavior will best meet the animal’s needs require information about the amount of reward that the animal will receive by performing the behavior, the odds that the behavior will end up being rewarded, as well as information about the animal’s physiological or adaptive needs [14, 16]. Circulating levels of gonadal hormones provide information about the female’s reproductive systems, and so may influence decision-making in favor of copulatory behaviors at times when conception is likely to occur. These changes are independent of the learned value of rewards and the cues that predict them, and so may dynamically alter choice behavior over the female estrous cycle. Interestingly, the ability of estradiol to induce sexual receptivity is accompanied by a complementary decrease in eating behavior. This may seem counter-intuitive, as copulatory behavior is energetically demanding, and one might expect females to increase their food intake in order to compensate for the increased metabolic demands associated with reproduction [49, 50]. An alternative variable that may better explain the observed decrease in food consumption is time. Although copulation itself is not necessarily time consuming, mate seeking requires a significant time investment [51]. Importantly, many of the behavioral changes seen during periods of fertility, such as increased locomotor behavior in rats and expansion of geographic ranges in many primates, increase the probability of coming across a mature male and subsequently copulating [52, 51]. Following this logic, females who spend less time eating during periods when probability of conception is high are able to spend comparatively more time looking for a mate and engaging in other reproductive behaviors, and are therefore more likely to successfully reproduce. This explanation highlights the ecological significance of estradiol’s dual role in inducing receptivity while attenuating feeding.

As with its effect on sexual behavior, it is likely that estradiol modulates feeding through multiple pathways. Estradiol treatment decreases meal size without an effect on meal frequency via activation of ERα [53, 54]. Much of the work on the estrous linked reduction in feeding has focused on energy balance and satiety signals such as cholecystokinin (CCK) [54, 50]. However, this does not rule out an indirect effect of estradiol on DA systems, as a significant population of DA neurons projecting from the VTA to the posterior-medial NAc co-release mature forms of CCK [55]. CCK has been shown to inhibit K+ induced DA release [56] in the rostral NAc while potentiating release in more caudal regions, potentially by either stimulating or inhibiting activation of cAMP [55, 56]. Interestingly, CCK microinjections to the rostral NAc have been shown to diminish self-stimulation of the medial forebrain bundle and block amphetamine-induced locomotor activity [57]. NAc DA has been shown to have a significant role in control of feeding behavior, as a low dose of amphetamine infused directly into the NAc increases, while a high dose decreases, feeding in male rats [58].

In addition to the observed effect of estradiol on meal size, ovariectomized rats shift their preference toward a high cost/high reward food option, and estradiol replacement subsequently reduces choice of the high effort reward. Surprisingly, although treatment with the selective ERα (propyl pyrazole triol/PPT) or ERβ (DPN) agonists alone increase choice of the high effort/high reward food, combination of these agonists mimics the effect of estradiol [59]. Similar to its role in normal feeding behavior discussed above, DA has also been shown to strongly impact choice of high effort/high reward over low effort/low reward options in a biphasic manner [60]. While low doses of the DA agonist amphetamine bias choice toward a high effort/high reward option, higher doses have the inverse effect. It is possible that ERα and ERβ coordinatively mediate estradiol dependent changes in DA systems of food motivation, leading to a decrease in motivation for food during periods of sexual receptivity.

2.3 Sex differences in motivation for drugs of abuse

The impact of gonadal hormones on neural systems involved in motivation and reward are of great relevance to clinical observations of sex differences in drug taking behavior in humans. Although rates of drug use today are lower in women than in men, this has not always been the case, and a large body of work suggests that women may be more susceptible to addiction [61, 24, 62].

Women escalate to compulsive drug use more rapidly than males and there is some indication that female addicts are more responsive to cocaine cues and experience greater levels of subjective craving [63, 64]. Although there are undoubtedly important social and environmental influences on individual susceptibility for drug use and abuse, substantial evidence supports the idea that observed sex differences in human drug use are due at least in part to the effects of gonadal hormones [24]. The subjective effects of stimulant drugs vary across the menstrual cycle, with subjective reports from women in the follicular phase, when estradiol is low, being more similar to reports from men, while rising estradiol during the luteal phase, as well as the administration of exogenous estradiol, enhances the subjective effects of stimulants like cocaine and amphetamine [65, 66].

It is possible that the underlying flexibility of mesolimbic dopamine systems in females that allows for adaptive changes in motivated behaviors across the female reproductive cycle may contribute to these clinical outcomes. Alternatively, it may be possible that there exists a specific subset of women that are particularly susceptible to developing substance dependence, but further research is needed to determine what underlying population differences may be responsible for the observed sex differences in potential for addiction. Understanding the role of estradiol in modulating motivation for natural reinforcers like food and sex will greatly assist in the development of sex specific treatments for addiction.

3. Conclusion

Estradiol has complex and widespread effects on the nervous system. In the striatum and NAc, estradiol is able to acutely enhance DA signaling in females but not males. The rapid flexibility of mesolimbic DA transmission in females likely mediates adaptive changes in motivated behaviors by increasing motivation for sex and decreasing motivation for food at times when conception is most likely to occur. Future work that emphasizes the importance of dissociating various aspects of motivated behaviors is necessary for a full understanding of estradiol’s complex relationship with midbrain DA systems.

Figure 1.

Estradiol binding directly to GABA medium spiny neurons decreases GABA transmission and disinhibits DA neurons in striatum.