Regulation of sex differences in risk-based decision making by gonadal hormones: Insights from rodent models

Abstract

Men and women differ in their ability to evaluate options that vary in their rewards and the risks that are associated with these outcomes. Most studies have shown that women are more risk averse than men and that gonadal hormones significantly contribute to this sex difference. Gonadal hormones can influence risk-based decision making (i.e., risk taking) by modulating the neurobiological substrates underlying this cognitive process. Indeed, estradiol, progesterone and testosterone modulate activity in the prefrontal cortex, amygdala and nucleus accumbens associated with reward and risk-related information. The use of animal models of decision making has advanced our understanding of the intersection between the behavioral, neural and hormonal mechanisms underlying sex differences in risk taking. This review will outline the current state of this literature, identify the current gaps in knowledge and suggest the neurobiological mechanisms by which hormones regulate risky decision making. Collectively, this knowledge can be used to understand the potential consequences of significant hormonal changes, whether endogenously or exogenously induced, on risk-based decision making as well as the neuroendocrinological basis of neuropsychiatric diseases that are characterized by impaired risk taking, such as substance use disorder and schizophrenia.

1. Introduction

To successfully navigate the world, humans are constantly making value-based decisions, ranging from the infinitesimal, such as choosing an outfit to wear for the day, to the significant, such as deciding how to invest money. Most of these choices involve weighing the benefits of each option against their associated costs, and, based on this calculus, selecting the option that, under the current condition, is the most adaptive, or that maximizes the benefits while minimizing potential costs to the individual. Although the majority of individuals are capable of making effective value-based decisions, individuals with certain neuropsychiatric conditions exhibit impairments in cost/benefit decision making. For example, individuals with substance use disorder (SUD) tend to overweight rewards and underweight the risk of adverse consequences that might accompany those rewards (Chen et al., 2020; Gowin et al., 2013). Eating disorders, such as anorexia nervosa, are also characterized by maladaptive choice behavior, with individuals displaying pathological risk aversion (Kaye et al., 2013). It is therefore not surprising that significant resources have been invested in understanding how risk-based decision-making processes become compromised in these (and other) neuropsychiatric diseases.

Toward this endeavor, researchers have leveraged the ability of rats to perform complex cognitive tasks in order to model cost/benefit decision making. Findings from studies using such tasks have recapitulated various aspects of human risk-taking behavior (exaggerated preference for risky options), including the increased risk taking following drug exposure (Blaes et al., 2022; Cocker et al., 2020; Ferland and Winstanley, 2017; Mitchell et al., 2014). Moreover, by combining these animal models of decision making with the ability to monitor or manipulate neural activity, a fairly comprehensive (although in no way conclusive) understanding of the neural substrates underlying decision making has emerged. A significant limitation of these studies, however, is that they predominantly use male subjects. This is noteworthy as there are reports of sex differences in risk-based decision making in humans and rodents alike (Islas-Preciado et al., 2020; Orsini and Setlow, 2017; van den Bos et al., 2013a, 2013b) as well as sex differences in the structure and function of several brain regions (Cahill and Aswad, 2015; McCarthy and Arnold, 2011; McCarthy et al., 2012) that govern this form of decision making (based on studies in males). Recent work shows that sex differences in choice behavior arise from hormonal modulation of cognitive processes involved in decision making (Ambrase et al., 2021; Hernandez et al., 2020; Orsini et al., 2021; Uban et al., 2012), such as the ability to evaluate outcomes in order to guide subsequent choices. Given the rich literature documenting the mechanisms by which gonadal hormones interact with neural substrates to contribute to sex-specific reward-seeking behavior (Bagley et al., 2017; Kerstetter et al., 2012; Larson et al., 2007), it is likely that a similar relationship accounts for sex differences in risk-based decision making. Few studies, however, exist to support this claim.

This review will provide a comprehensive discussion of the current state of knowledge regarding hormonal modulation of risky decision making, and then, based on this information, propose mechanisms by which hormones could influence neural systems involved in this behavior to give rise to sex differences in risky decision making. The review will primarily survey the preclinical literature to complement excellent reviews of the influence of hormones on value-based decision making, emotional regulation and cognition in humans (Ambrase et al., 2021; Toffoletto et al., 2014; van Wingen et al., 2011). The first section of the review will provide a broad overview of the key brain regions that have established roles in risky decision making. This will be followed by a discussion of animal studies that have unequivocally demonstrated the existence of sex differences in risky decision making and a necessary role for hormones in mediating such sex differences. Because of a lack of preclinical studies investigating the neurobiological basis of hormonal regulation of decision making, we will then briefly review findings from studies with human subjects that have advanced our knowledge on this topic, and where appropriate, suggest ways in which the use of animal models may address gaps that remain. Finally, we will conclude by considering the implications this knowledge has for understanding the impact of significant changes in hormonal profiles (e.g., pregnancy, menopause) on risk-based decision making. We will also discuss how such knowledge can guide our understanding of neuropsychiatric diseases that differentially manifest between sexes and are characterized by risk-based decision-making deficits.

2. A brief overview of the neurobiological substrates of risky decision making

Decision making is a complex cognitive process that requires the recruitment of multiple different brain regions, which work in concert to guide choice behavior (Fobbs and Mizumori, 2017; Orsini et al., 2019). Such widespread engagement of the brain supports the numerous cognitive operations that must occur, at times within milliseconds, to engender an adaptive decision. Broadly, brain regions that govern various forms of decision making lie within the mesocorticolimbic system, including the prefrontal cortex, basolateral amygdala, and nucleus accumbens (Orsini et al., 2015a, 2015b; Piantadosi et al., 2021; Winstanley and Floresco, 2016). As illustrated in Fig. 1, all of these regions receive dopaminergic projections from the ventral tegmental area. It is therefore not surprising that dopamine neurotransmission and receptor signaling within each of these brain structures contributes to aspects of decision making (e.g., outcome evaluation). This section will provide an overview of the contributions of several regions in risk-based decision making. This discussion, however, is by no means an exhaustive review of the neurobiology of decision making, and we instead recommend several comprehensive reviews on such a topic (Orsini et al., 2015a; Piantadosi et al., 2021; Wassum and Izquierdo, 2015; Winstanley and Floresco, 2016). Rather, we will focus on brain regions that are modulated by gonadal hormones and may therefore contribute to the neurobiological mechanisms underlying hormonal regulation of decision making, a topic that will be discussed in further detail in Section 5.

Fig. 1.

Neural circuits underlying risky decision making. Risk-based decision making is mediated by brain regions within the mesocorticolimbic circuit, including the prefrontal cortex (PFC), basolateral amygdala (BLA), and nucleus accumbens (NAc). All of these regions receive dopaminergic input from the ventral tegmental area (VTA). Dopaminergic, glutamatergic and GABAergic projections between these structures are depicted in this figure. Nuclear receptor sites (subtypes for estrogen and progesterone receptors are combined) for each of the different hormones are also shown in this diagram. Although membrane estrogen and progesterone receptors are located in the NAc, nuclear progesterone receptors are not expressed in the NAc (Almey et al., 2015; Laflamme et al., 1998; Meffre et al., 2013; Simerly et al., 1990; Zhang et al., 2002) and only one study has provided evidence for nuclear estrogen receptors in the NAc (Maher et al., 2021). ARs, androgen receptors; ERs, estrogen receptors; PRs, progesterone receptors.

2.1. Prefrontal cortex

The prefrontal cortex (PFC) is involved in higher order cognitive processing (Funahashi and Andreau, 2013), including decision making (Bechara et al., 1994; Manes et al., 2002). Early evidence for the role of the PFC in risk-based decision making came from studies in humans with damage to this brain region (Bechara et al., 1994; Clark et al., 2003; Fellows and Farah, 2005; Manes et al., 2002). In these studies, patients were tested in the Iowa Gambling Task (IGT), wherein they had to choose between several decks of cards, each of which was associated with different payoffs and losses. Patients with PFC damage consistently selected “riskier” options associated with short-term payoffs but long-term losses. Animal studies have corroborated these findings and extended them to other measures of risk-based decision making. The medial prefrontal cortex (mPFC), which consists of the prelimbic (PL) and infralimbic (IL) areas, is considered the rodent homolog to the human dorsolateral PFC (Uylings et al., 2003). Combined inactivation of the IL and PL causes deficits in a rodent analog of the IGT (rGT) similar to those observed in humans with PFC damage in the IGT (Zeeb et al., 2015). Providing additional support for the involvement of the mPFC in risk-based decision making, St. Onge and Floresco et al. (2010a) examined the effects of pharmacological inactivation of the PL of the mPFC on performance in a probability discounting task in which rats chose between a small reward delivered 100% of the time and a large reward delivered with systematically varying probabilities. Silencing the PL increased choice of the large, uncertain reward (i.e., increased risk taking) when probabilities of reward delivery were initially high and decreased over the session. When probabilities of reward delivery were low at the beginning and increased over the session, PL inactivation caused a decrease in risk taking. These findings suggested that rather than regulating risk taking per se, the PL area of the mPFC may instead have an important role in promoting flexible behavior in the face of changing choice contingencies during decision making. Recent work has revealed that the mPFC is similarly engaged in other forms of decision-making processes that involve different costs associated with the rewarding outcomes. Using the “Risky Decision-making Task” (RDT), wherein rats choose between a small, “safe” food reward and a large, “risky” food reward accompanied by varying probabilities of footshock punishment, Orsini et al. (2018) demonstrated dissociable effects of mPFC inactivation on risk-based decision making that were dependent on the order in which the risk probabilities were presented. Similar to the findings of St. Onge et al., combined inactivation of the PL and IL increased choice of the large, risky reward (i.e., increased risk taking) when probabilities of footshock increased over the session but decreased risk taking when probabilities of footshock decreased over the session. These results again demonstrate the importance of the mPFC in adjusting decision making in the face of changing risk contingencies. This aspect of decision making appears to be mediated via dopamine (DA) activation of D2 dopamine receptors (D2Rs) in the PL of the mPFC, as blockade of these receptors induces disruptions in risk taking in the probability discounting task similar to those observed with pharmacological inactivation (St Onge et al., 2011). Intriguingly, blockade of D1 dopamine receptors (D1Rs) in the PL of the mPFC had the opposite effect on risk taking in this task, causing a decrease in risk taking. Based on this data, it has been proposed that DA within the PL of the mPFC must achieve a delicate balance between activation of D1Rs and D2Rs, particularly under conditions of changing risk contingencies, to promote optimal decision making.

In addition to the mPFC, the orbitofrontal cortex (OFC) is another subregion of the PFC that plays an important role in risk-based decision making. The effects of OFC manipulations on risk-based decision making, however, vary depending on several factors, including the type of risk associated with the larger reward, the subregion of the OFC targeted by the manipulation, and the manipulation itself. For example, pharmacological inactivation of the medial, but not lateral, portion of the OFC increases risk taking in the probability discounting task, in which the risk is that of reward omission (Mar et al., 2011; St Onge and Floresco, 2010; Stopper et al., 2014). Contrary to this, lesions of the lateral OFC decrease risk taking in the RDT, in which the risk is that of punishment delivery (Orsini et al., 2015b). In addition to differences in risk valence and OFC subregion, further reconciliation of these findings may be possible with more specific manipulations of neurotransmitter systems within the OFC. Indeed, Jenni et al. (2021) found that, similar to the dissociable effects of DA receptor manipulations in the mPFC, blockade of D1Rs versus D2Rs in the medial OFC had contrasting effects on risk taking in the probability discounting task. Whereas infusions of a D1R antagonist into the medial OFC decreased risk taking, infusions of a D2R antagonist into this same region increased risk taking (similar to the effects of pharmacological inactivation of this region in the same task; Jenni et al., 2021). Additional analyses revealed that the effect of D1R blockade was specifically driven by enhanced sensitivity to negative feedback (i.e., increased likelihood to bias choice away from risky options after reward omission). Hence, as in the mPFC, DA neurotransmission in the medial OFC influences optimal decision making as risk probabilities change by striking a balance between activation of D1Rs and D2Rs. Caution should be taken, however, when using these studies to construct a comprehensive neurobiological framework for the roles of both the mPFC and OFC in risk-based decision making as only male subjects were used, despite evidence that the prefrontal cortical DA system is sexually dimorphic and sensitive to gonadal hormones (Kritzer and Creutz, 2008). Future studies are therefore warranted to extend the investigation of the role of the prefrontal cortex in risk-based decision making to the female brain.

2.2. Basolateral amygdala

A growing body of literature has begun to show that regions of the amygdala, such as the basolateral amygdala (BLA), are implicated in risk-based decision making (Orsini et al., 2017; Wassum and Izquierdo, 2015; Winstanley and Floresco, 2016). Similar to the PFC, initial evidence for the role of this limbic structure in risky decision making originated from studies showing that patients with damage to their amygdala made riskier choices than controls in the IGT (Bechara et al., 1999; Brand et al., 2007). Studies in rodents have corroborated these findings and further refined our understanding of how the BLA contributes to risky decision making. Similar to the OFC, effects of BLA manipulations on risk taking appear to depend on the cost that is associated with the more rewarding of the available options. For example, excitotoxic lesions of the BLA increase risk taking in the RDT (Orsini et al., 2015b) and the rGT (Zeeb and Winstanley, 2011), both of which involve risk of adverse consequences, either in the form of footshock or a timeout penalty, respectively. In contrast, inactivation of the BLA in the probability discounting task, in which the cost is not inherently aversive, decreases risk taking (Ghods-Sharifi et al., 2009). Considered together, these data suggest that in the face of a potential aversive outcome, the BLA promotes choice behavior toward safer options, whereas in the face of ambiguous appetitive-based outcomes, the BLA is responsible for biasing behavior towards actions that will maximize the rewarding outcomes. More recent studies have further shown that activity in the BLA is differentially engaged during risky decision making. In the RDT, optogenetic inhibition of the BLA during the pre-choice phase (i.e., deliberation) decreased risk taking, but optogenetic inhibition of the BLA during the delivery of the large reward accompanied by an aversive outcome increased subsequent choice of that risky option (Orsini et al., 2017). Neural activity in the BLA exerts such dynamic control over behavior during decision making through its projections to the nucleus accumbens (NAc; Orsini et al., 2019; St Onge et al., 2012). Indeed, Bercovici et al. (2018) demonstrated that inhibition of BLA projections to the NAc during the deliberation phase of the probability discounting task decreased risk taking, whereas inhibition of this circuit during reward omissions increased risk taking. Hence, the BLA, and its connections with the NAc, play a pivotal role in shaping cognitive processes during decision making to guide optimal and adaptive choice behavior.

The BLA is modulated by DA via activation of DA receptors expressed throughout this brain region (Chu et al., 2012; Pinard et al., 2008; Rosenkranz and Grace, 1999; Wei et al., 2018). Surprisingly, there have been very few studies on the role of BLA dopamine receptors in decision making in general, and only one specific to their contribution in risk-based decision making. Using the probability discounting task, Larkin et al. (2016) found that blockade of D1Rs, but not D2Rs, in the BLA affected risk taking, causing a reduction in choice of the large, uncertain reward. Conversely, stimulation of D1Rs and D2Rs with agonists had distinctive effects on choice behavior based on individual differences in baseline risk preference. Whereas a D1R agonist increased risk taking in risk-averse rats (or those that predominantly preferred the small, certain reward over the large, uncertain reward) when the probability of reward delivery was high, a D2R agonist decreased risk taking in risk-seeking rats (or those that predominantly preferred the large, uncertain reward over the small, certain reward) when the probability of reward delivery was low. On the basis of these data, it appears that activation of DA receptors in the BLA optimizes decision making as risk contingencies change. Similar to the body of work that has focused on the role of the PFC in decision making, this study only used male subjects. Recent work has shown that there are sex differences in D2R mRNA levels in the BLA, with greater expression in females compared with males (Georgiou et al., 2018). Hence, it is possible that the sensitivity of D2Rs to DA modulation of risk taking may be augmented in females, which could contribute to the well-established sex differences in risk taking (see Section 3).

2.3. Nucleus accumbens

The nucleus accumbens (NAc), a region located in the ventral striatum, receives input from the PFC and BLA and is densely innervated by DA input from the ventral tegmental area (VTA; Koob and Volkow, 2010; Fig. 1). Although this region is recognized for its reward processing capabilities, the NAc is also involved in complex behavior, such as decision making and behavioral flexibility (Floresco, 2014). Over a decade of work has established a role for the NAc in risk-based decision making using behavioral pharmacology, optogenetics, in vivo electrophysiology and methods of measuring DA efflux (e.g., amperometry, microdialysis, voltammetry) during behavior. For instance, pharmacological inactivation of the shell subregion of the NAc during the probability discounting task decreased risk taking, an effect that was driven by a decrease in sensitivity to the rewarding properties of the larger, uncertain outcome (Stopper and Floresco, 2011). Notably, pharmacological inactivation of the core subregion had no effect on risk taking in this task. The core subregion, however, is necessary for risk-based decision making when discrete external cues, such as visual or auditory stimuli, are incorporated into the decision-making task to guide choice behavior (Floresco et al., 2018). Studies using in vivo electrophysiology show that activity in the NAc core dynamically encodes the value of rewards associated with risk during active decision making. In particular, neural activity before making a choice discriminated between risky versus safe outcomes and signaled the rats’ preferred choice (Sugam et al., 2014). During reward deliveries, the occurrence of reward omission (i.e., a loss) was associated with greater excitatory activity in the NAc core but only in rats with a strong risk-averse phenotype at baseline. These findings suggest that neural activity in the NAc core is particularly important for representing the subjective value of options during risky decision making and could therefore account for individual differences in risk taking that serve as predisposing factors to the development of certain psychiatric conditions.

Such neural activity is tightly regulated by DA release from the VTA (Nicola et al., 2000; Tritsch and Sabatini, 2012). Indeed, several studies have used various methods of measuring DA release in the NAc to assess predictive relationships between DA levels and risk taking or to monitor changes in DA release in the NAc during ongoing decision making. Freels et al. (2020) reported that greater evoked DA release, as measured with amperometry, in the NAc shell predicted greater risk taking in the RDT and was augmented in rats with a risk-seeking phenotype relative to those with a risk-averse phenotype (Freels et al., 2020). Consistent with this, Sugam et al. (2012) monitored DA release in the NAc core during active decision making in a probability discounting task and found that DA release during cues predictive of choice outcomes tracked the subject’s preferred outcome (rather than representing reward value alone). In a more recent study, the role of DA release in the NAc was directly probed by chemogenetically stimulating DA projections from the VTA to the NAc (core and shell combined) during a risk-based decision making task (Verharen et al., 2020). Stimulating this projection, and consequently enhancing DA release in the NAc, rendered rats insensitive to loss and punishment in a probability discounting task and therefore caused them to continue to choose the risky outcome even when risk of reward omission was high. Collectively, these data demonstrate that DA in the NAc scales with greater risk preference and may do this by blunting sensitivity to potential negative consequences associated with the more rewarding, yet riskier, option.

Given the importance of DA neurotransmission in the NAc, it is not surprising that pharmacological or optogenetic manipulation of DA receptors in the NAc affect risk-based decision making. Results of these studies show that when the penalty associated with the large reward is that of uncertain reward delivery (i.e., as in a probability discounting task), D1Rs in the NAc (core and shell combined) reduce the impact of reward loss on choice behavior, promoting continued choice of the risky, uncertain option, particularly when it is optimal to do so (Stopper et al., 2013). In contrast, D2Rs seem to have no discernable role in this context. Excessive stimulation of D3 dopamine receptors (D3Rs), however, biases choice away from rewarding outcomes, even when risk of their omission is low. When the penalty associated with the large reward is that of punishment, stimulation of D2Rs in the NAc shell does alter risk taking by promoting risk aversion (Mitchell et al., 2014). These divergent findings allude to a dissociable role for NAc D2Rs, at least within the shell subregion, in decision making depending on the valence of the cost. Although the role of NAc D1Rs in decision making involving risk of punishment has yet to be directly investigated, risk preference in the RDT is not significantly correlated with D1R mRNA levels in the NAc core or shell (Simon et al., 2011), suggesting that D1Rs in the NAc may not contribute to this form of risk-based decision making. Finally, the use of optogenetics to manipulate NAc neurons (no distinction between subregions) that selectively express D2Rs has revealed that these cells, and therefore presumably D2R activation, is critical during the deliberation phase of risk-based decision making for promoting risk-averse behavior (Zalocusky et al., 2016). Reminiscent of the findings from behavioral pharmacology studies, this effect depended on baseline risk preference as manipulation of these D2R-expressing neurons was only effective in shifting choice behavior in rats that inherently preferred the risky option.

3. Sex differences in rodent models of decision making

The last several years has seen a surge in the publication of preclinical studies reporting sex differences in different forms of value-based decision making, including risky decision making. This section will provide a brief overview of this body work as it has been reviewed in extensive detail elsewhere (Orsini and Setlow, 2017). A more in depth discussion will instead be spent on hormonal mechanisms that may contribute to such sex differences in risky decision making.

In one of the first studies to examine sex differences in risky decision making in rodents, van den Bos et al. (2012) trained male and female rats in another rodent model of the Iowa Gambling Task (rIGT; not to be confused with the rGT, which is distinct from the rIGT), in which they choose between a long-term advantageous reward and a long-term disadvantageous reward. Whereas the former consists of frequent delivery of several small unadulterated sugar pellets (e.g., reward) and the occasional delivery of quinine-laced sugar pellets (i.e., punishment), the latter consists of frequent deliveries of large rewards and infrequent deliveries of punishment (van den Bos et al., 2012). Over the 10 days of training, males developed a preference for the advantageous choice more quickly than females, although both sexes eventually chose the advantageous option over the disadvantageous option to the same degree by the end of the learning period. A closer inspection of behavior during each of the test sessions, however, revealed that each sex used distinct strategies to learn how to best execute their decision, with males gathering information about the options on a more global scale and females investigating each option in detail to determine the most adaptive and advantageous choice. In the rGT, in which penalties consist of timeout periods rather than quinine-laced rewards, Georgiou et al. (2018) also reported sex-dependent differences in decision making. In contrast to the findings of van den Bos, however, optimal performance (i.e., selection of the optimal choice) in females did not improve in parallel with male performance, which could possibly reflect differences in processing information related to the distinct penalties in each task. Interestingly, using a similarly structured rodent gambling task, Peak et al. (2015) reported that females developed a preference for the most advantageous choice more quickly than males. Such differences across studies are likely due to slight, although nonetheless significant, task parameters. Although the findings from van den Bos et al. (2012) mimic those reported in humans (van den Bos et al., 2013a, 2007, 2013b), lending translational validity to this study, the lack of consistency with other preclinical studies (e.g., Georgiou et al., 2018; Peak et al., 2015) together with the inconsistency between these preclinical studies and sex differences in risk taking in humans highlights the need for additional studies in both species to reconcile the differences in findings and to fully understand male and female risk-based decision-making strategies.

In contrast, there are robust sex differences in decision making involving risk of explicit punishment in the form of footshock. Orsini et al. (2016) trained males and females in the RDT and found that males display a greater preference for the large reward associated with risk of punishment relative to females. Such risk aversion in females could not be accounted for by differences in motivation for food or disparities in body weight between sexes, which could influence their perception of footshock (Orsini et al., 2016). Increased risk aversion in females was accompanied by longer latencies to select the larger, riskier reward and a greater omission rate compared with males. Given the lack of sex differences in food motivation, an increase in omissions in females could be considered another risk-averse behavioral strategy to evade punishment (rather than explicitly choosing the safe option). These findings are consistent with more recent work showing that females take longer to select punished rewards and acquire instrumental avoidance learning faster than males (Chowdhury et al., 2019). Similar sex differences in decision making manifest when the punishment associated with the large reward is gradually delayed (Liley et al., 2019). Using the Delayed Punishment Decision-making Task, wherein rats make discrete choices between a small food reward and a large reward accompanied by an increasingly delayed footshock punishment, Liley et al. (2019) demonstrated that females were more sensitive to the effects of punishment on choice behavior, even when there were significant delays between reward and punishment delivery. Collectively, these data demonstrate that female choice behavior is more sensitive to risk of punishment. This phenotypical female behavior is consistent with decision-making strategies displayed in a more naturalistic setting. Pellman et al. (2017) examined sex differences in foraging behavior in an environment associated with unpredictable footshocks and observed that males and females exhibited different strategies to mitigate the threat of punishment on food seeking. Specifically, males consumed larger amounts of food in order to reduce the overall time spent foraging in the risky environment while still maintaining their body weight (i.e., preserving their metabolic needs) whereas females chose to reduce their food intake, and therefore sacrifice their metabolic needs, rather than risk potential adverse consequences (Pellman et al., 2017). The confluence of findings from these different animal models of risk-based decision making that incorporate explicit punishment demonstrate that sex differences in risky decision making are not merely due to differences in reward motivation or sensitivity to aversive stimuli, but rather appear to arise from unique sex-dependent strategies that are used to guide adaptive behavior.

Finally, sex differences in risky decision making have recently been extended to decision making in which the risk associated with the large reward is that of reward uncertainty. Using the probability discounting task, Islas-Preciado et al. (2020) reported that, compared with males, females displayed a greater preference for the small, certain reward over the larger, uncertain reward, particularly when probabilities of reward delivery were low. This risk aversion in females was primarily driven by an enhanced sensitivity to losses, as females were more likely to shift to the small, certain reward after experiencing reward omission (Islas-Preciado et al., 2020). Greater sensitivity to loss in females has also been observed in another variant of a rodent gambling task (Ishii et al., 2018), lending credence to the hypothesis that males and females process and use salient information (e.g., relative risks and outcomes) about decisions differently.

4. Hormonal regulation of decision making

Hormonal modulation of cognitive processes involved in decision making can give rise to the aforementioned sex differences in risky decision making. The majority of the studies that have investigated the effects of hormones on decision making have addressed the question from an activational standpoint and, consequently, little is known about how hormones during development shape decision making in adulthood (i.e., organizational effects; Arnold, 2009b; McCarthy and Arnold, 2011; McCarthy et al., 2012; Phoenix et al., 1959). To this end, this section will review recent work demonstrating the role of gonadal hormones in risk-based decision making in adults of both sexes, and, when possible, highlight findings implicating organizational influences of hormones on risk taking. We will begin this review with a brief discussion of each of the major gonadal hormones before providing behavioral evidence for hormonal regulation of decision making. Before reviewing each class of gonadal hormones, however, it is important to acknowledge the fact that biological mechanisms other than gonadal hormones could also potentially contribute to sex differences in risk-based decision making. In particular, the complement of genes on sex chromosomes can affect brain development, function and behavior independently of the actions of gonadal hormones (Arnold, 2020; McCarthy and Arnold, 2011). Genetically modified mouse models have been developed to dissociate between gonadal hormone and sex chromosome effects on physiological and behavioral outcomes in males and females (Arnold, 2009a, 2020; Arnold and Chen, 2009). These animal models will therefore be useful in determining if sex differences in risk-based decision making can be explained by sex chromosome complement differences (particularly in instances in which there is no evidence for gonadal hormone contributions). In this review, we have constrained our discussion to the role of gonadal hormones in sex differences in risk-based decision making, but we recognize the importance of other equally influential factors, such as the complement of genes on sex chromosomes, in modulating this cognitive process.

4.1. Overview of major gonadal hormones

Although gonadal hormones are characteristically thought to be synthesized and released from the gonads, they are also produced in the brain and can act locally in neural tissue as neurosteroids (Gillies and McArthur, 2010). Irrespective of where they are synthesized, they all share cholesterol as their precursor (Cui et al., 2013; Gillies and McArthur, 2010; Rossetti et al., 2016; Tobiansky et al., 2018). Gonadal hormones act by binding to receptors that are either membrane-bound or localized within the cell cytoplasm or nucleus (Almey et al., 2015; Cover et al., 2014; Cui et al., 2013; Tobiansky et al., 2018). Depending on the location of the receptor, hormones can have rapid and transient non-genomic actions (membrane-bound), or they can exert slow and long-lasting genomic effects (nuclear) on cellular physiology and cognition (Balthazart and Ball, 2006; Cover et al., 2014; Gillies and McArthur, 2010). Because the influence of hormones on risky decision making seems to occur via genomic mechanisms (Orsini et al., 2021), our discussion of hormone receptor subtypes will focus on those that are located in the cytoplasm and nucleus.

4.1.1. Estradiol

Of the four major forms of physiological estrogens (estrone, estradiol, estriol and estetrol), estradiol (17β-estradiol) is the most active during the premenopausal period of adulthood (Cover et al., 2014; Gillies and McArthur, 2010). Aromatase is the main enzyme responsible for converting testosterone to estradiol (Blakemore and Naftolin, 2016; Lephart, 1996; Shay et al., 2018) and, as it can be found in a region-specific manner in neurons and astrocytes, serves as a marker for areas of the brain in which local estradiol synthesis and function are high (Roselli, 2007). In fact, brain regions known to contain aromatase, such as the BLA (Blakemore and Naftolin, 2016), have detectable levels of estradiol in the absence of detectable levels of estradiol in peripheral circulation (e.g., ovariectomized females; Li and Gibbs, 2019). Further, exogenous estradiol treatment can enhance estradiol levels above those in circulation in ovariectomized females, suggesting that estradiol administration may enhance local production of the hormone in the brain (Li and Gibbs, 2019). Not only do these findings demonstrate that estradiol can be synthesized in the brain, but they also indicate that local synthesis can compensate for loss of circulating estradiol. Although estradiol is canonically thought of as a female gonadal hormone, males also synthesize estradiol and certain areas of the brain in male rats, such as the BLA, contain estradiol levels equivalent to that of females rats (Barker and Galea, 2009).

The two isoforms of the nuclear estrogen receptor family are estrogen receptor α (ERα) and estrogen receptor β (ERβ). Both ERs are expressed throughout the brain, with an overlap of expression in regions including the prefrontal cortex, the amygdala, hippocampus and hypothalamus (Almey et al., 2015; Creutz and Kritzer, 2002; Milner et al., 2008; Simerly et al., 1990). Although both ERs are expressed in the many of the same structures, one subtype may predominate expression over the other (Osterlund et al., 1998). For example, ERα is the dominant ER subtype in hypothalamic areas (Shughrue et al., 1997), such as the medial preoptic area, whereas ERβ is the dominant ER subtype in the hippocampus (Milner et al., 2005; Zhang et al., 2002) and cortex (Kritzer, 2002). When unbound to estradiol, these ERs exist as monomers in the cytoplasm; upon estradiol binding to ERs in the cytoplasm, ERs transform into homodimers or heterodimers and translocate into the nucleus where they bind to estrogen-response elements (EREs) of gene promoters on DNA, thereby regulating gene transcription (Cui et al., 2013). Activated nuclear ERs can also influence transcription processes independent of EREs by binding to other transcription factors, such as activator-protein 1, the interaction of which has been shown to influence cell growth and proliferation (Cui et al., 2013). Although both ERs are expressed in the male and female brain to a similar extent, there is evidence for sex differences in relative expression in certain brain regions (Laflamme et al., 1998; Zhang et al., 2002), as well as sex differences in ER-dependent signaling mechanisms (Gillies and McArthur, 2010), all of which could contribute to hormone-mediated sex differences in risky decision making.

4.1.2. Progesterone

In addition to its involvement in the reproductive system, progesterone is necessary for other non-reproductive functions in the nervous system, including cognition. Notably, although classically discussed in the context of female gonadal hormones, progesterone secretion and serum progesterone levels do not differ between males and females (Oettel and Mukhopadhyay, 2004). In addition to its synthesis in the reproductive organs, progesterone is also locally produced in the brain via activity of the enzyme 3β-hydroxysteroid on pregnenolone in the smooth endoplasmic reticulum of neurons, astrocytes and oligodendrocytes (Brinton et al., 2008; Rossetti et al., 2016). In the brain, progesterone selves to promote myelination, synaptogenesis, neuronal survival and dendritic growth, all of which point to a role in neurodevelopment and neuroprotection (Rossetti et al., 2016; Schumacher et al., 2004). There is also significant evidence that progesterone is an important regulator of estradiol function at both the physiological and behavioral level. For example, progesterone occludes estradiol-induced increases in spine density in the hippocampus (Woolley and McEwen, 1993) and prevents the ability of estradiol to reverse spatial memory deficits in ovariectomized females (Bimonte-Nelson et al., 2006). This antagonistic relationship between estradiol and progesterone has also been reported in their effects on dopamine-related behavior, such as drug-seeking, with estradiol potentiating and progesterone suppressing this behavior in females (Becker and Hu, 2008; Carroll and Anker, 2010).

Like estradiol, there are two main isoforms of nuclear progesterone receptors (PRs) that can contribute to the genomic actions of progesterone (Brinton et al., 2008; Rossetti et al., 2016). When activated by progesterone, progesterone receptor A (PRA) and/or progesterone receptor B (PRB) undergo a conformational change, dimerize and translocate to the nucleus where they bind to progesterone response elements of gene promoters on DNA. In this way, these receptors can directly regulate gene transcription and expression. These receptors are also widely distributed throughout the brain, particularly within the hippocampus, hypothalamus, bed nucleus of the stria terminalis, central nucleus of the amygdala and cerebellum (Brinton et al., 2008). Although there is evidence for sex differences in progesterone expression in some brain regions (e.g., cerebellum; Guerra-Araiza et al., 2003), there do not seem to be sex differences in PR expression in many of the brain regions in which PRs are densely expressed (e.g., BNST; Auger and De Vries, 2002).

4.1.3. Testosterone

Originally identified for its masculinizing effects (Berthold, 1849), testosterone is one of the most potent androgens that not only has a role in reproductive function (Hull and Dominguez, 2007), but also has more recently been shown to regulate behavioral and cognitive processes (Tobiansky et al., 2018). As with other steroid hormones, androgens are synthesized in the brain through enzymatic action of Cyp17a1 on progesterone, sourced either de novo from cholesterol or from circulation (i.e., gonadally-derived; Celec et al., 2015; Tobiansky et al., 2018). In support of local synthesis of testosterone, Tobiansky et al. (2018) recently reported that testosterone levels were 2–3 times higher in brain structures in the mesolimbic circuit (e.g., VTA, NAc) than in blood serum in intact male rats. Moreover, although gonadectomy decreased testosterone levels in some rats, there were still detectable testosterone levels in approximately 50% of these rats, and, in those rats with detectable levels, testosterone levels in the VTA were comparable to those in intact males. Notably, depending on enzymatic capabilities of the cells, locally synthesized testosterone can be metabolized into estradiol by aromatase or to 5α-dihydrotestosterone (DHT) by 5α-reductase (Celec et al., 2015); testosterone can therefore affect brain function and cognition indirectly through action of these metabolites.

Direct effects of testosterone on neurons occur through activation of androgen receptors (ARs), which are intracellularly located in the cytoplasm. When androgens bind to ARs, the receptor complex dimerizes and moves into the nucleus, binding to the androgen response elements of gene promoters on DNA (Gao et al., 2005). Like estradiol and progesterone, it is through this mechanism that testosterone can regulate gene transcription and expression. In addition to being densely expressed in the hypothalamus, ARs are located in the hippocampus, amygdala, VTA, NAc, PFC, and OFC (Lu et al., 1998; Simerly et al., 1990; Wood and Newman, 1999), and anatomical studies have shown that ARs are specifically expressed on VTA neurons that project to the amygdala, NAc, and PFC (Aubele and Kritzer, 2012; Creutz and Kritzer, 2004). Because AR-expressing neurons in the VTA express tyrosine hydroxylase (Kritzer, 1997), the rate-limiting enzyme in the DA synthesis pathway, it is possible that androgen activation of ARs in the VTA can selectively modulate dopaminergic activity in the mPFC and NAc. Further, Kritzer and Creutz (2008) found that the number of PFC-projecting neurons in the VTA that co-express ARs and tyrosine hydroxylase are greater in males than females. Hence, androgen regulation of DA-dependent function in mPFC may contribute to phenotypic male risky decision making (see Section 5 for further discussion).

4.2. Behavioral evidence for hormonal regulation of decision making

Given the ability of gonadal steroids to access the brain, the capacity of the brain to locally synthesize hormones, and the widespread distribution of hormone receptors in the brain (Fig. 1), it is not surprising that hormones modulate risky decision-making processes. To date, there have been two major approaches to determine the influence of gonadal hormones on decision making in rodents. The first approach, which has been exclusively conducted in females, is to assess whether choice behavior changes as a function of differing levels of circulating hormones across the estrous cycle. As represented in Fig. 2A, the rodent estrous cycle consists of four phases (proestrus, estrus, metestrus and diestrus), each of which is associated with different levels of estradiol and progesterone (Smith et al., 1975). Proestrus is characterized by a peak in estradiol, which subsequently declines, and rising levels of progesterone. Although levels of progesterone remain higher than levels of estradiol during estrus, both hormones gradually decline during this second phase of the cycle. During metestrus, levels of progesterone are moderate, but are higher than estradiol levels, and during the last phase of the cycle (diestrus), levels of progesterone decline slightly while levels of estradiol increase. Fig. 2B represents the different phases of the human menstrual cycle and highlights the similarity of hormone profiles between the rodent estrous and human menstrual cycles (i.e., estrus phase and luteal phase; diestrus phase and late follicular phase). In rodent studies, it is often the case that the estrous cycle is dichotomized into periods of high ovarian hormones (proestrus/estrus) and low ovarian hormones (metestrus/diestrus), which is then used as a factor to analyze hormonal influences on behavior. Although this approach may yield initial insight into whether hormones modulate behavior, it diminishes the importance of the natural cyclicity inherent in female physiology, which may be one of the most critical factors in truly understanding sex differences in behavior and underlying neural substrates. Consequently, in the studies reviewed herein, we will discuss findings from previous studies within the context of natural hormonal fluctuations across the hormonal cycle.

Fig. 2.

Representation of the rodent estrous cycle and human menstrual cycle. Estradiol (solid green line) and progesterone (dashed orange line) naturally fluctuate across the (A) rodent estrous cycle and (B) human menstrual cycle.

Although not usually considered explicitly in behavioral studies, levels of androgens also fluctuate in males. In contrast to the cyclicity of ovarian hormones across days in the female hormonal cycle, androgens (in both humans and rodents alike) fluctuate across the day, with higher concentrations in the morning that gradually decline over the course of the day (Cooke et al., 1993; Diver and Clinical Scince Reviews Committee of the Association for Clinical, 2006; Heywood, 1980; Tcholakian and Keating, 1978; Wilson et al., 1976). To control for the variability that may arise from these circadian fluctuations, behavioral studies are usually conducted at the same time every day. To our knowledge, no studies have examined whether decision making differs as a function of androgen level during the day; hence, in contrast to discussions of behavior in the context of hormonal cycling in females, the studies reviewed herein will focus on the role of testosterone in risk-based decision making in general (i.e., without consideration of natural circadian fluctuations of the hormone).

Another approach to determine the influence of gonadal hormones on decision making in rodents is to examine the effects of removing the endogenous source of the hormones on decision making and subsequently determine if exogenous replacement of the hormones ameliorates gonadectomy-induced changes. To identify the receptor mechanisms underlying hormonal regulation of behavior, selective receptor agonists or antagonists can be administered to attempt to mimic or block, respectively, effects of the hormone on behavior. As will be reviewed below, it is not always the case that these approaches lead to the same conclusions, providing some evidence for differences in organizational vs. activational effects of hormones on risk-based decision making.

4.2.1. Ovarian hormones

Although females display divergent risk-taking strategies from males in the rGT and rIGT, their behavior in this task is not affected by phase of the estrous cycle (Georgiou et al., 2018; van den Bos et al., 2012). This is the extent of our knowledge with respect to the influence of fluctuations in endogenous ovarian hormones on performance in this task in rodents. In contrast, two recent studies have directly investigated the role of ovarian hormones in modulating risk taking in the probability discounting task. In one study, Islas-Preciado et al. (2020) found no effects of ovariectomies on risk taking, nor did they observe changes in risk taking after acute administration of estradiol benzoate (EB). In a second study, chronic administration of EB to ovariectomized females also did not significantly affect risk taking in the probability discounting task (Wallin-Miller et al., 2017). Although these studies initially suggest that estradiol does not modulate this form of risk-based decision making, future studies with larger sample sizes and additional control groups (i.e., sham females) are necessary to thoroughly examine the role of estradiol in probability discounting (as well as in the rGT and rIGT).

The ability of estradiol to modulate risk-based decision making has recently been extended to decision making involving risk of punishment. Despite the fact that performance in the RDT does not change as a function of phase of the estrous cycle (Orsini et al., 2016), female risk taking is significantly impacted by the removal of the ovaries. Specifically, Orsini et al. (2021) reported that ovariectomized females displayed greater risk taking relative to their pre-surgical performance and intact females. The administration of EB mitigated the impact of ovariectomies but had no effect in intact females. The ability of EB to reduce risk taking in ovariectomized females seemed to be driven by a reduction in sensitivity to the rewarding properties of the large, risky outcome, as these females were less likely to continue to choose this option, even if they had just received a large reward without punishment. Neither the ovariectomy-induced increase in risk taking nor the EB-induced decrease in risk taking in ovariectomized females could be accounted for by changes in food motivation or sensitivity to shock as these manipulations had no effect on assays directly measuring willingness to work for food and shock intensity thresholds. These findings provide an unequivocal role for the activational influence of ovarian hormones, and estradiol in particular, in decision making involving risk of explicit punishment. A preliminary study from our laboratory suggests that estradiol may promote female risk aversion through the activation of ERβ. Intact female rats (n = 8) were trained in the RDT and then received subcutaneous injections of a selective ERβ antagonist (PHTPP; 4-[2-phenyl-5,7-bis(trifluoromethyl) pyrazolo[1,5-a]pyrimidin-3-yl]phenol; 25 μg/0.1 mL) or vehicle (sesame oil) for 10 days using a randomized within-subjects design. If estradiol promotes female risk aversion through activation of this receptor, we expected that blocking this receptor would increase risk taking, similar to, but not to the same extent as, ovariectomies. Consistent with this prediction, the ERβ antagonist caused a near-significant increase in choice of the large, risky reward relative to vehicle conditions [dose X risk block, F (4, 28) = 2.36, p = 0.07; Fig. 3]. It remains to be determined whether ERα is similarly involved in female risk aversion as there are currently no ERα-targeting drugs that exclusively act as antagonists (Santollo et al., 2010). Future experiments, however, are using selective agonists for each ER isoform in ovariectomized females to disentangle the receptor mechanisms by which estradiol exerts its effects on risk taking in females.

Fig. 3.

Antagonism of estrogen receptor β increases choice of the large, risky reward in intact females. The administration of PHTPP (4-[2-phenyl-5,7-bis (trifluoromethyl) pyrazolo[1,5-a]pyrimidin-3-yl]phenol), an estrogen receptor β antagonist, caused a near-significant increase in choice of the large, risky reward in intact females (n = 8) in the Risky Decision-making Task. Data are represented by mean ( ± standard error of the mean) percent choice of the large, risky reward.

Ovarian hormones may also modulate risk-taking behavior in a more naturalistic setting for rodents. Pellman et al. (2017) not only tracked the estrous cycle in females during assessment of foraging behavior in the face of potential threat, but also evaluated the impact of ovariectomies on female behavior. Relative to other phases of the estrous cycle, females spent more time in the foraging arena, an area associated with the delivery of unpredictable shocks, during the proestrus phase, suggesting that risky foraging behavior is influenced by fluctuations in hormone levels (Pellman et al., 2017). The authors note, however, that although females spent more time in this area during proestrus, there was not the expected increase in foraged food that would accompany more time spent an area with these available resources; they therefore suggest that their observations likely reflect changes in behavior related to reproduction rather than alterations in risky behavior per se. This interpretation would be consistent with the fact that there were no effects of ovariectomies on risky foraging behavior in females. Based on these findings, Pellman et al. conclude that the fluctuations in risky foraging behavior in females, whether specific to risk taking or reproductive behavior, arise from the organizational effects of hormones early in development. These findings contrast with those described above in the study by Orsini et al. (2021), but this discrepancy can easily be reconciled when considering the numerous differences in the tasks, such as the environment in which choice was assessed (home environment vs. operant chamber distinct from home environment) and whether explicit choice between a safe and harmful option was involved.

Finally, it should be noted that there has yet to be a focused investigation of the role of progesterone in risk-based decision making. Although such information can be inferred through monitoring the estrous phase as estradiol and progesterone levels differ across the cycle (Fig. 2), it is still difficult to completely dissociate their respective roles with confidence. In addition, ovariectomies not only remove circulating estradiol, but also eliminate circulating progesterone. Unless each ovarian hormone is exogenously administered separately, it is not possible to attribute behavioral changes to just one of these gonadal hormones. Although this approach has been used with estradiol, such a systematic evaluation of progesterone’s contribution to risk-based decision making has yet to be conducted.

4.2.2. Testosterone

In the last several years, studies have repeatedly shown that testosterone is responsible for male choice behavior in risky decision-making tasks (Tobiansky et al., 2018). Although orchiectomies do not alter risk taking in a probability discounting task, exogenous administration of testosterone does alter risk taking in males, albeit in different directions depending on the duration of testosterone administration, dose of testosterone and gonadal status of the rat. Acute administration of testosterone decreased risk taking in orchiectomized males when the probability of reward delivery was low (Islas-Preciado et al., 2020). In contrast, chronic administration of testosterone at a dose known to produce physiological levels similar to those in an adult male rat increased risk taking in orchiectomized males (Wallin-Miller et al., 2017). Chronic administration of testosterone at a dose that results in supraphysiological levels of testosterone, purported to model anabolic-androgenic steroid use, also increased risk taking in the probability discounting task in intact male rats (Wallin et al., 2015). This same testosterone regimen also increased disadvantageous choice behavior in intact males in the rGT (Wallin-Miller et al., 2018). Collectively, these data provide strong support for testosterone regulation of male risk-taking behavior. In further support of this role for testosterone in male choice behavior, Tan and Vyas (2016) reported that chronic administration of testosterone to orchiectomized males increased willingness to withstand risk of reward loss in another probabilistic decision-making task modeled after the human Balloon Analog Risk Task.

Similar to ovarian hormones, testosterone is also involved in male risk taking in decision making involving risk of explicit punishment (Orsini et al., 2021). In contrast to the absence of effects of orchiectomies on risk taking in the probability discounting task, orchiectomies significantly decreased risk taking in males in the RDT. This increased risk aversion seemed to be driven by a decrease in sensitivity to rewarding outcomes, as males were less likely to continue to choose the large, risky option even after receipt of the large reward without delivery of punishment. There were, however, no differences between orchiectomized and intact males in responding for a food reward in a control assay used to measure willingness to work for food, therefore excluding the possibility that alterations in food motivation accounted for the effects of orchiectomies. The decrease in risk taking was also not due to changes in baseline shock reactivity, as there were no differences between orchiectomized and intact males in a control task used to directly assess shock intensity thresholds. Surprisingly, exogenous administration of testosterone did not reverse the effects of orchiectomies on risk taking, nor did it increase risk taking in intact males, as has been previously shown in the RDT by others (Cooper et al., 2014). This study clearly demonstrates that endogenous testosterone promotes male-typical risk-taking behavior, but it also suggests that testosterone may not be the primary androgen responsible for regulation of this behavior. Indeed, dihydrotestosterone (DHT) is a more potent androgen than testosterone (Celec et al., 2015; Tobiansky et al., 2018) and is capable of attenuating the impact of orchiectomies on punishment-related behavior (Edinger et al., 2004). It will therefore be important for future experiments to determine if other behaviorally relevant androgens, such as DHT, contribute to male-typical risk taking.

In contrast to testosterone administration, chronic estradiol administration does influence male risk taking in the RDT (Orsini et al., 2021). Using a dose and regimen effective in decreasing risk taking in ovariectomized females, estradiol reduced risk taking in males, irrespective of their gonadal status. Similar to the effects of orchiectomies, this reduction in risk taking was accompanied by a shift in behavioral strategy. Rats treated with EB were less likely to continue to choose the large, risky reward after receiving a large reward without accompanying punishment and instead preferred the smaller, safer option. Such effects of EB mirror those observed in ovariectomized females and therefore suggest that, regardless of sex, estradiol has a critical role in promoting risk aversion. Interestingly, serum concentrations of estradiol are equivalent between males and females (Walker et al., 2012), suggesting that male-typical risk taking is due to greater testosterone levels (Walker et al., 2012). Alternatively, one could argue that it is the ratio of estradiol to testosterone that contributes to phenotypical male choice behavior. Although the removal of testes deprived males of testosterone and estradiol and consequently decreased risk taking in the RDT (Orsini et al., 2021), the exogenous administration of testosterone alone did not have a restorative effect on choice behavior. Moreover, exogenous administration of estradiol alone caused a further decrease in orchiectomized males and reduced risk taking in intact males, possibly by disrupting the balance between circulating estradiol and testosterone. This hypothetical model has yet to be fully examined, but experiments designed to assess the effects of testosterone on risk taking in females may provide additional support for its validity. Further, experiments using androgen receptor antagonists or an aromatase inhibitor (to block the conversion of testosterone to estradiol) will also be helpful in disentangling the roles of testosterone versus estradiol in risk taking in males and females.

5. Potential neurobiological mechanisms underlying hormonal regulation of decision making

Hormones likely influence decision making through modulation of brain regions that contribute to this cognitive process (see Section 2). Indeed, many of the aforementioned brain regions densely express steroid receptors (Kritzer, 2002; Lu et al., 1998; Osterlund et al., 1998; Shughrue et al., 1997; Simerly et al., 1990; Fig. 1). This section will review what has been learned from studies investigating the neurobiological basis of hormonal regulation of decision making. Because of the dearth of animal studies on this topic, this discussion will draw mostly from human imaging studies, but, when relevant, will include findings from animal studies that have provided information about hormonal effects on neural activity (outside the context of decision making). Using this knowledge, we will propose hypothetical mechanisms that could be tested using animal models of risk-based decision making.

5.1. Prefrontal cortex

5.1.1. Ovarian hormones

Neuroimaging studies investigating the influence of ovarian hormones on decision-making related activity in the PFC commonly use probabilistic monetary decision-making tasks. Similar to the probability discounting task used in rodents, subjects can choose between monetary rewards of different magnitudes and that differ in the probability of their delivery. Several studies have shown that activity specifically in the OFC during decision making in this task fluctuates across the menstrual cycle (Fig. 2B). For example, Dreher et al. (2007) reported greater OFC activity during the anticipation of uncertain rewards during the follicular phase of the menstrual cycle (low progesterone and rising estradiol). Bayer et al. (2013) similarly showed heightened OFC activation in anticipation of delayed (but certain) rewards during the follicular phase relative to the luteal phase (peak in progesterone and low estradiol). Other groups have assessed the influence of ovarian hormones on reward-related processing in the PFC by directly administering estradiol and progesterone (i.e., hormone therapy) to postmenopausal women. Using this approach, Thomas et al. (2014) found that, relative to placebo treatment, hormone therapy increased activity in the ventromedial region of the PFC during the delivery of a probabilistically delivered monetary reward in a slot machine task. Further, levels of estradiol were positively correlated with activity in the lateral subregion of the PFC during anticipation of rewards (Thomas et al., 2014). In a more recent study, Girard et al. (2017) investigated the effects of hormone therapy in postmenopausal women on activity in the dorsolateral region of the PFC (dlPFC) during a task assessing cognitive flexibility. Not only did hormone therapy increase activity in the dlPFC compared to placebo administration, but in women with the greatest activation of the dlPFC, it also improved performance in the task (Girard et al., 2017). Although risk-based decision making was not explicitly assessed in this study, the ability to efficiently and effectively adjust behavior is a critical cognitive component of the decision process. Consequently, this study suggests that ovarian hormones may enhance PFC-dependent cognitive control required to make adaptive decisions. These studies, however, have not directly assessed how ovarian hormones influence PFC activity related to risk or loss during decision making and, as a result, our understanding of how hormones regulate decision making through modulation of the PFC is still limited.

Previous research has shown that ovarian hormones influence DA neurotransmission within the PFC and contribute to DA-dependent PFC cognitive function. Baseline levels of DA in the mPFC of female rats fluctuate across the estrous cycle, with the lowest levels of DA occurring during proestrus when circulating estradiol is the greatest (Dazzi et al., 2007). Others have shown, however, that ovariectomies decrease DA in the mPFC (Kokras et al., 2018) and treatment with estradiol or agonists for ERs increase DA and upregulate DA receptors in the mPFC in ovariectomized females (Jacome et al., 2010; Sarvari et al., 2014). There is also evidence that locally synthesized estradiol affects DA neurotransmission, as inhibition of aromatase with letrozole decrease rates of DA turnover in the PFC of both male and female rats (Kokras et al., 2018). Notably, these studies did not distinguish between subregions of the mPFC and therefore it is not known whether estradiol regulates DA neurotransmission in a subregion-dependent manner. In behavioral studies, others have shown that infusions of estradiol directly into the mPFC shifts rats’ use of strategies to navigate spatial environments (Almey et al., 2014). Because the ability to flexibly switch between different behavioral strategies depends on DA function in the PFC (Floresco, 2013; Floresco and Magyar, 2006), these behavioral findings suggest that the cognitive flexibility required for decision making may arise from estradiol-dependent regulation of DA neurotransmission in the PFC. This hypothesis has yet to be directly tested using animal models of risk-based decision making, but provides a very compelling (and empirically testable) account for how ovarian hormones can influence risky decision making.

5.1.2. Testosterone

Because the developmental period of adolescence is characterized by exaggerated risk taking and drastic changes in hormone levels (Blakemore and Naftolin, 2016; Blakemore et al., 2010; Burnett et al., 2010; Crone et al., 2008; Forbes and Dahl, 2010), many of the studies that have investigated the neural substrates underlying testosterone’s influence on risk taking have used adolescent subjects. These studies have shown that testosterone levels are correlated with risk-based decision making in both sexes, and this relationship seems to be mediated by structural and/or functional differences in the OFC. For example, Peper et al. (2013) reported that greater testosterone levels were associated with greater risk taking in both adolescent males and females in the Balloon Analog Risk Task, another risk-based gambling task commonly used in the laboratory. The strength of this relationship in males and females differed based on the size of the medial OFC (mOFC): whereas the association was the strongest in males with a small mOFC, it was the weakest in females with a small mOFC (Peper et al., 2013). Somewhat incongruous with these findings, Op de Macks et al. (2016) found that the relationship between testosterone levels and risk taking in adolescent females was mediated by increased activation of the OFC. Such discrepancies, however, may be due to the use of different decision-making tasks in each of these studies. It is also still unclear how structural differences, such as those observed by Peper et al., relate to functional differences, such as those observed by Op de Macks et al., in the brain.

In rodents, gonadectomies impair performance in cognitive tasks that are dependent on the mPFC and the administration of testosterone reverses these deficits (Kritzer et al., 2007, 2001). Such effects have not been as elegantly demonstrated for risk-based decision making, which also depends on the integrity of the mPFC (Orsini et al., 2018; St Onge and Floresco, 2010; Zeeb et al., 2015; Zeeb and Winstanley, 2013). This lack of information is largely due to the fact that many of these studies investigating the role of testosterone in risk-based decision making did not include pre-gonadectomy baselines (to which to compare behavior after gonadectomy) and/or were using doses of testosterone that produced supraphysiological levels of testosterone (Cooper et al., 2014; Wallin-Miller et al., 2017). The one exception is the study by Orsini et al. (2021), wherein male risk taking in the RDT, which requires a functional mPFC (both PL and IL; Orsini et al., 2018), decreased after gonadectomy. Although testosterone administration did not reverse these effects, it is possible that other androgens, such as DHT, may be responsible for phenotypical male behavior in this decision-making task. Consistent with this possibility, DHT can bind to ARs in the brain and is capable of altering structural plasticity in the mPFC of gonadectomized male mice (Hajszan et al., 2008).

Testosterone may influence PFC-dependent aspects of decision making through modulation of DA release in the PFC. Evidence for such a mechanism stems from a body of work showing that DA cells in the VTA that project to the mPFC (no distinction between subregions) express ARs, with a greater proportion of these cells in males compared with females (Kritzer and Creutz, 2008). Further, the density of these projections to the mPFC decreases after gonadectomy in males (Kritzer, 2003). This reduction can be can be reversed with treatment of testosterone, but not estradiol (Kritzer, 2003). Although this work demonstrates that testosterone affects DA function in the mPFC via binding to ARs on mPFC-projecting neurons in the VTA, testosterone can also indirectly influence DA activity by binding to ARs on glutamatergic neurons in the mPFC that project to the VTA (Aubele and Kritzer, 2012). In fact, this projection is more heavily enriched with ARs than dopaminergic VTA to mPFC projections and is sensitive to testosterone, but not estradiol, manipulations. Together, these findings suggest that testosterone influences DA function in the mPFC by regulating mPFC afferents to DA neurons in the VTA that in turn project to the mPFC. Finally, it is worth noting that due to the presence of aromatase in the mPFC of both males and females, testosterone may indirectly influence DA function and overall mPFC activity through its conversion to estradiol, which, as described above, can moderate such neural mechanisms and regulate decision making. Indeed, as mentioned earlier, chronic inhibition of aromatase decreases DA turnover rate in the mPFC of male and female mice (Kokras et al., 2018). Future studies are needed to determine the functional relevance of these anatomical findings as they relate to the ability of testosterone to influence PFC-dependent aspects of risk-based decision making.

5.2. Basolateral amygdala

5.2.1. Ovarian hormones

As reviewed in Section 2, the amygdala is a critical hub in the neural circuitry that underlies various forms of decision making (Orsini et al., 2015a, 2015b; Wassum and Izquierdo, 2015). Given the fact that this brain region is sexually dimorphic in both structure and function (Blume et al., 2017; Cahill et al., 2001, 2004; Kilpatrick et al., 2006; Rowniak et al., 2015), it is not surprising that amygdala activity is potently modulated by hormones, likely through activation of hormone receptors that are densely expressed in this structure. It is therefore conceivable that hormones influence cognitive processing involved in decision making by regulating neural activity in the amygdala related to risk and reward information. Several neuroimaging studies have provided support for this hypothesis. For example, using a probabilistic decision-making task, Dreher et al. (2007) reported greater amygdala activation during anticipation of an uncertain reward in women in the follicular phase of their menstrual cycle, similar to the activation pattern observed in the OFC. In a more recent study, Macoveanu et al. (2016) showed that greater activation of the amygdala in response to greater monetary rewards (compared with smaller monetary rewards) was positively correlated with ovarian hormone levels during a monetary gambling task. Relative to administration of placebo, the administration of a gonadotropin-releasing hormone agonist, which effectively decreases hormone synthesis, attenuated the differential pattern of amygdala activation to rewards of different magnitudes (Macoveanu et al., 2016). These studies clearly demonstrate that ovarian hormones contribute to the amygdala’s ability to process and evaluate reward-related information during decision making. Although not in the context of decision making, there is also evidence that ovarian hormones influence amygdala activation in response to negative or aversive stimuli (Hwang et al., 2015; Lebron-Milad et al., 2012; Merz et al., 2013; Petersen and Cahill, 2015). For example, greater amygdala activation during extinction of fear-inducing stimuli is positively correlated with elevated levels of estradiol (Zeidan et al., 2011). Hence, it is possible that, in addition to reward-related information, ovarian hormones may also influence the encoding and representation of the costs or risks that accompany rewarding outcomes.

Independent of behavior, ovarian hormones heavily influence cellular and physiological properties in the BLA. In a recent study, Blume et al. (2017) extensively characterized the cellular composition and physiological properties across subdivisions of the BLA during the rodent estrous cycle. Relative to proestrus, pyramidal neurons in the lateral nucleus (LA) exhibited greater sensitivity to glutamate and had higher intrinsic firing rates during diestrus. Furthermore, diestrus was associated with greater frequency of spontaneous inhibitory post-synaptic currents and increased expression of parvalbumin, a calcium-binding protein indicative of GABAergic neurons. Considered together, these findings suggest that activity in the LA is greater during phases of the estrous cycle when levels of estradiol and progesterone are low. Intriguingly, cycle-dependent effects on neural activity were also observed in the basal nucleus of the amygdala (BA) but in the opposite direction. Specifically, intrinsic firing rates of pyramidal neurons in the BA were lower during diestrus relative to proestrus, as was the sensitivity to exogenously applied glutamate. Neurons in the BA were also more sensitive to GABA-induced inhibition during diestrus compared with proestrus. Thus, in contrast to the LA, activity in the BA seems to be greater during phases of the cycle in which estradiol and progesterone are high and lower during phases of the cycle in which levels of estradiol and progesterone are low. Although these findings are divorced from behavioral measures of decision making, they provide indirect support for the hypothesis that ovarian hormones influence risk-based decision making through modulation of the BLA, a critical node in the decision-making circuit.

5.2.2. Testosterone

In contrast to ovarian hormones, few, if any, studies have directly examined whether testosterone influences risk-based decision through regulation of the BLA. There are, however, several studies that have repeatedly reported positive correlations between testosterone levels and amygdala reactivity in response to threatening or fearful faces in men and women (Derntl et al., 2009; Hermans et al., 2008; Manuck et al., 2010). Moreover, acute administration of testosterone to women increases amygdala reactivity to these aversive stimuli (Bos et al., 2013; Hermans et al., 2008; van Wingen et al., 2009) and simultaneously decreases connectivity between the amygdala and the OFC (van Wingen et al., 2010). Based on these studies, testosterone is thought to significantly contribute to emotional processing via its regulation of brain regions like the amygdala; it is unknown, however, whether such modulatory influence extends to processing positively or negatively valenced stimuli and/or outcomes during risk-based decision making. Given the presence of aromatase in the BLA, it is also possible testosterone indirectly modulates BLA activity during decision making through its conversion to estradiol.

5.3. Nucleus accumbens

5.3.1. Ovarian hormones

Neuroimaging studies have also have revealed that activity in the NAc during probabilistic monetary decision-making tasks is influenced by circulating ovarian hormones. As with the studies investigating the influence of hormones on PFC and BLA activity, these studies monitored NAc activity in women in different phases of their menstrual cycle while they were tested in this decision-making task. Although the findings do confirm that the NAc is differentially recruited during decision making across the cycle, the results across studies are somewhat inconsistent. For example, whereas Dreher et al. (2007) reported no changes in NAc activity in anticipation of uncertain probabilistic rewards across the menstrual cycle, Bayer et al. (2013) and Ossewaarde et al. (2011) reported that the NAc was recruited during this anticipation period specifically in the early follicular phase of the menstrual cycle, in which ovarian hormone levels are low. To more directly assess the causal role of ovarian hormones on NAc activity during decision making, other studies have manipulated hormone levels either through hormone replacement or interference in the synthesis of these hormones (Macoveanu et al., 2016; Thomas et al., 2014). Surprisingly, such manipulations have no discernable effect on NAc activity. In a recent study, Bayer et al. (2020) attempted to reconcile these incongruous findings by examining the effects of estradiol administration on DA-related NAc activity (as opposed to non-specific widespread NAc activity) during a probabilistic decision-making task. Oral administration of estradiol dose-dependently increased NAc activity in a manner consistent with the DA-dependent prediction error hypothesis: neural activity in the NAc increased when the outcomes of the subjects’ choices were better than expected. Based on these findings, the authors concluded that estradiol can directly stimulate reward-related NAc activity and that the inconsistencies observed across prior studies may be due to the fact that they were not specifically tapping into DA-dependent processes in the NAc (Bayer et al., 2020). This conclusion is supported by another study in which estradiol regulation of DA function in the NAc was indirectly measured by assessing the interaction between fluctuations in hormones over the menstrual cycle and genetic polymorphisms for the DA transporter (DAT; Jakob et al., 2018), which is responsible for the reuptake of DA from the synapse into the presynaptic terminal. Previous work in animals has shown that estradiol can reverse the flow of DA through DAT on the presynaptic membrane, leading to DA efflux (Alyea et al., 2008; Alyea and Watson, 2009). The authors of the study predicted that when levels of estradiol are the greatest, women with the DAT polymorphism would have elevated DA in the striatum and that this increased DA would subsequently alter sensitivity to outcomes during a probabilistic reversal task (Jakob et al., 2018). Consistent with their predictions, Jakob et al. (2018) found that during the late follicular phase, in which estradiol levels are high and progesterone levels are low, carriers of this polymorphism exhibited an attenuated ability to adjust ongoing behavior following a punished outcome. Although the task used in this study did not assess decision making per se, it does offer insight into the neural mechanisms by which estradiol (but not progesterone) may modulate sensitivity to costs, such as risk of punishment, that must be evaluated during the decision making process to guide future choice. Collectively, the work by Bayer et al. (2020) and Jakob et al. (2018) provide convincing evidence that estradiol strongly influences DA-dependent processes in the NAc responsible for the representation and evaluation of both reward- and risk-related information necessary to guide decision making.

The regulation of DA-related processes in the NAc by ovarian hormones is consistent with decades of animal studies that have examined the specific mechanisms by which estradiol and progesterone interact with the DA system to influence reward-based behavior. In particular, estradiol can affect DA function by impacting its synaptic availability and receptor affinity and density within the striatum (Almey et al., 2015). For example, estradiol administration in female rats augments phasic and tonic striatal DA release and increases D2 dopamine receptor binding (Becker and Hu, 2008; Thompson and Moss, 1994; Yoest et al., 2018). Administration of estradiol and progesterone together further enhances DA release in the striatum beyond that elicited by estradiol alone (Becker et al., 1984; Becker and Ramirez, 1981; Becker and Rudick, 1999). It is this facilitating effect of estradiol on DA function that is thought to contribute to DA-dependent cognitive processes. Although several anatomical studies have reported very few nuclear ERs in the NAc (Almey et al., 2015), a recent study provided evidence for the presence of nuclear ERs in the NAc (Maher et al., 2021), suggesting that the effects of estradiol on DA-mediated cognition could occur through both membrane and nuclear-bound ERs. An alternate and not mutually exclusive possibility is that estradiol modulates activity of DA projections from the VTA, which can then indirectly alter NAc neural firing and DA receptor signaling. Indeed, Calipari et al. (2017) recently demonstrated that DA projections from the VTA to the NAc core are greater in female mice when physiological levels are estradiol are high. It is important to consider, however, that the majority of these studies focused on the faster non-genomic, effects of estradiol on DA transmission, and, although such mechanisms may also underlie estradiol’s influence on decision making, there is evidence that at least some forms of decision making rely on the slower, genomic actions of estradiol (Hernandez et al., 2020; Orsini et al., 2021; Uban et al., 2012). The use of animal models will therefore provide the opportunity to dissect non-genomic versus genomic actions of estradiol on NAc activity, and DA signaling therein, during risk-based decision making.

5.3.2. Testosterone

Similar to studies examining the influence of testosterone on PFC activity during decision making, many of the studies that have investigated the influence of testosterone on NAc activity during decision making have used adolescent subjects. Across these studies, testosterone has been shown to potentiate reward-related activity in the NAc during probabilistic monetary decision-making tasks. For example, Op de Macks et al. (2011) examined the relationship between testosterone levels and NAc activation to rewards in a gambling task in adolescent boys and girls. Testosterone levels were positively correlated with NAc activity specifically during receipt of the reward (Op de Macks et al., 2011). Similar findings have been reported with other risk-based gambling tasks, such as the Balloon Analog Risk Task (Braams et al., 2015). Only one study has reported an inverse relationship between testosterone levels and NAc activity during a gambling task (Forbes et al., 2010), but these discrepancies have mostly been attributed to significant procedural differences across the different decision-making assays. To test the causal role of testosterone on NAc activity related to rewards, Hermans et al. (2010) administered testosterone to women and monitored NAc activity while subjects performed a gambling task. Testosterone administration increased NAc activity in response to cues predictive of probabilistically delivered monetary rewards but did not alter activity in response to cues with no predictive value (Hermans et al., 2010). It is worth noting that across all of these studies, testosterone’s potentiating effect on NAc activity was linked to the potential for rewards, rather than the risk of lost opportunity, which is consistent with effects of testosterone manipulations on risk-based decision making observed in animal studies (e.g., selective effects on reward sensitivity; Orsini et al., 2021; but see Wallin et al., 2015).

Although no study has directly tested the modulatory influence of testosterone on DA function in the NAc on risk-based decision making, indirect evidence suggests that it may be one mechanism by which testosterone exerts its effects on choice behavior. Similar to estradiol, testosterone regulates DA signaling and receptor function in the NAc, processes that have been shown to mediate risk-based decision making (see Section 2). For example, high doses of androgens decrease D1R and D2R density in the NAc of male rats (Kindlundh et al., 2001). A recent animal study explored whether these effects of testosterone on DA receptors could account for testosterone’s ability to decrease risk taking in the probability discounting task. Rats undergoing chronic treatment with a high dose of testosterone received acute systemic injections of a D1R or D2R agonist and were subsequently tested in the probability discounting task. Replicating their previous work (Wallin et al., 2015), Wallin-Miller et al. (2018) reported that testosterone administration decreased choice of the large, uncertain reward relative to vehicle administration. The administration of either a D1R or D2R agonist, however, reversed this effect of testosterone on risk taking. These findings suggest that high doses of testosterone decrease risk taking by reducing sensitivity of D1R and D2Rs. Although this study did not manipulate DA receptors in the NAc, it is conceivable that this region is the locus of testosterone’s action on these receptors given their role in decision making (Mitchell et al., 2014; Stopper et al., 2013) and the specific effects androgens have on their expression (Kindlundh et al., 2001). Other studies have shown that orchiectomies increase DA metabolites in the NAc (Bitar et al., 1991; Mitchell and Stewart, 1989) and alter other synaptic properties related to DA neurotransmission, including vesicular storage and re-uptake (Bitar et al., 1991; Meyers and Kritzer, 2009), indicating that androgens have an important role in regulating DA turnover and availability in the NAc. Through these mechanisms, testosterone may also contribute to the ability of DA in the NAc to track risk preference during risk-based decision making (Freels et al., 2020; Sugam et al., 2012). Finally, testosterone could influence NAc activity during risky decision making through its local metabolism to either estradiol or DHT, both of which can influence DA activity in the NAc (Alderson and Baum, 1981; Becker, 1999). The use of animal models of risk-based decision making, in combination with hormone and pharmacological manipulations, will provide opportunities to test these hypothetical mechanisms.

6. Translational implications

6.1. Understanding how natural changes in hormone fluctuations influence decision making

In addition to the hormonal fluctuations that occur across the menstrual cycle, women also experience several significant events associated with dramatic changes in hormone levels across the lifespan. Although gonadal hormones are critical for organizational development of the brain early in life (Arnold, 2009b; McCarthy et al., 2012), significant hormonal exposure in adulthood can also have long-lasting effects on the brain and, consequently, on cognition. For example, there are potent changes in hormonal exposure in women during pregnancy and the postpartum period (Buckwalter et al., 1999; Henry and Sherwin, 2012; Tulchinsky et al., 1972). A wealth of preclinical studies has suggested that these hormone changes, although transient relative to the entire lifespan, may impart long-lasting alterations in hippocampal structure and physiology (see the following for excellent reviews on this topic: Koebele and Bimonte-Nelson, 2015; Macbeth and Luine, 2010). These changes in hippocampal function are thought to contribute to improvements in spatial memory observed after pregnancy. To date, there are no studies in humans or in animals that have examined how pregnancy and the postpartum period affect risk-based decision making (either during or after those periods). Intriguingly, lactating rats during the postpartum period are faster at finding and catching prey than female rats that have never been pregnant (Kinsley and Lambert, 2006; Lambert and Kinsley, 2007). Although there was no risk to the rats while foraging in these studies, they suggest that reproductive experience, and potentially the distinct hormonal environment that accompanies it, may result in mothers engaging in more naturalistic risk-taking behavior for the benefit of their offspring. Because hormonal changes during pregnancy and the postpartum period can have re-organizing effects on the brain, they can potentially alter decision-making abilities of the mother for the rest of her life. This has important translational implications as it suggests that our understanding of the cognitive, hormonal and neural mechanisms underlying female decision making, much of which was generated from preclinical studies using virgin or nulliparous female rats, may not be as translationally relevant as we thought to understanding decision making in women, a large proportion of whom have had children. It is therefore critical that future preclinical studies examine whether reproductive experience and its associated hormonal changes alter risk-based decision making later in life.

Another major hormonal change that women inevitably experience in middle age is menopause, in which both estradiol and progesterone decline and then become undetectable during the postmenopausal period (Burger et al., 2007). Such a decline in gonadal hormones may contribute to cognitive impairments associated with aging (Greendale et al., 2009; McCarrey and Resnick, 2015; Mitchell and Woods, 2011), possibly as a result of another re-organization of neural circuits that underlie cognitive function. Numerous studies have shown that initiation of hormone therapy shortly after menopause (or estropause in rodents) can prevent cognitive decline in aging females; if, however, hormone therapy occurs outside of a “critical window” of time, hormone therapy is ineffective in improving cognitive function (Bimonte-Nelson et al., 2006; Daniel and Bohacek, 2010; Luine, 2014; Mitsiades et al., 2008). Such effects of menopause and hormone therapy on cognition have predominantly been limited to studies assessing memory, with little consideration for other cognitive domains like decision making. Aging itself is associated with a change in risk taking, with some studies reporting increased risk aversion and others reporting increased risk taking (Gilbert et al., 2011; Goh et al., 2016; Mata et al., 2016, 2011). Similar to other cognitive domains, these alterations in decision making could in fact be related to the significant changes in a female’s hormonal environment. In the absence of hormonal therapy, permanent hormonal deprivation during menopause may irrevocably alter activity in brain regions and circuits that subserve risk-based decision making. If, however, hormone therapy is initiated within the critical window of responsivity, the trajectory of these neural changes (and consequential changes in decision making) may be averted. Work from Dreher and colleagues have shown that when initiated at the beginning of menopause, hormone therapy in menopausal women increases activity in the NAc and PFC in response to lower probability rewards relative to placebo treatment (Thomas et al., 2014). In a different study, the same group demonstrated that hormone therapy administered early in menopause also increases activity in the PFC during performance in a task assessing cognitive flexibility (Girard et al., 2017). Moreover, women with greater activity in this brain region were the subjects that exhibited the greatest cognitive flexibility. Although these studies did not assess risk-based decision making per se, they quantified changes in cognitive mechanisms that are involved in the decision-making process (e.g., motivation for probabilistic rewarding outcomes, the ability to alter behavior in the face of a dynamic environment), and therefore suggest that hormone therapy, when initiated early in menopause, may offer neuroprotection against changes in risk-based decision making (i.e., increased risk aversion) that may have a negative impact on quality of life (e.g., excessive avoidance of necessary medical procedures). Table 1.

Table 1.

Summary of findings from preclinical studies assessing sex differences in and hormonal regulation of risky decision making.

Reference	Decision-making Task	Abbreviation	Task Measure	Sex Differences	Hormonal Modulation
Cooper et al. (2014)	Risky Decision-making Task	RDT	Choice of small reward vs. large reward associated with probabilistic footshock	N/A, only male subjects	Chronic T increases risky choice
Orsini et al. (2016)				Risky choice ♀ > ♂	No relationship with estrous cycle
Orsini et al. (2021)				Risky choice ♀ > ♂	OVX increases risky choice; EB decreases risky choice; ORX decreases risky choice; T does not alter risky choice, but EB decreases risky choice in males
Wallin et al. (2015)	Probability Discounting	N/A	Choice of small, guaranteed vs. large, probabilistic rewards	N/A, only male subjects	Chronic T decreases risky choice
Wallin-Miller et al. (2017)				N/A	Chronic T increases risky choice in ORX males
Islas-Preciado et al. (2020)				Risky choice ♀ > ♂	Acute T decreases risky choice in ORX males
van den Bos et al. (2012)	Rat Iowa Gambling Task	rIGT	Learning about probabilities of different reward outcomes	♂ develop preference for the advantageous option more quickly than ♀	Unknown
Wallin Miller et al., 2018	Rat Gambling Task	rGT	Choice of optimal (more reward, less timeout, punishment) vs. suboptimal options	N/A, only male subjects	Chronic T increases disadvantageous choice
Georgiou et al. (2018)				Advantageous choice ♀ > ♂	No relationship with estrous cycle
Peak et al. (2015)				♀ develop preference for the advantageous option more quickly than ♂	Unknown
Tan and Vyas (2016)	Probabilistic Decision-making Task (rodent model of the Balloon Analog RIsk Task)	N/A	Choice of ‘add lever’ that increases the size of a potential food reward (but also associated with a risk that the trial would fail and no reward would be delivered) vs. ‘cash-out lever’ that leads to delivery of accrued reward	N/A, only male subjects	Acute T increases tolerance for risk of reward forfeiture
Liley et al. (2019)	Delayed Punishment Decision-making Task	DPDT	Choice of small reward vs. large reward associated with delayed footshock	♀ discounted delayed punishment less than ♂	No relationship with estrous cycle
Pellman et al. (2017)	Risky “Closed Economy” System	N/A	Time spent in a risky foraging area, where footshocks can be delivered unpredictably	Risky choice ♀ > ♂	Increase in risky choice during proestrus phase; No effects of OVX on risky choice

Legend: T, Testosterone; OVX, ovariectomy; EB, estradiol benzoate; ORX, orchiectomy; ♀, female; ♂, male.

Although discussed less frequently, males also experience a gradual decline in gonadal hormone reproduction, often referred to as andropause (Matsumoto, 2002). The decline in gonadal hormones in males is much more gradual compared to that experienced by women during menopause, but several longitudinal studies suggest that, although not as drastic as in women, these hormonal changes in men may also be associated with cognitive decline (Mitsiades et al., 2008; Moffat et al., 2002; Yaffe et al., 2002). Similar cognitive deficits have been reported in men with prostate cancer who undergo androgen deprivation therapy as part of cancer treatment (Mitsiades et al., 2008; Wibowo, 2017). Further, several studies have shown that androgen therapy (e.g., testosterone administration) can improve certain cognitive functions, particularly those involving spatial memory, in elderly or hypogonadal men (Cherrier et al., 2001, 2005; Janowsky et al., 1994). Notably, however, for all those studies that have reported altered cognitive function in men with androgen deficiency (either during andropause or after androgen deprivation) or in men undergoing hormonal therapy, there are a similar number of studies that have shown no changes in cognitive function in the same conditions (Emmelot-Vonk et al., 2008; Fonda et al., 2005; Sih et al., 1997; Vaughan et al., 2007). Moreover, because the majority of these studies mainly focused on changes in the spatial and verbal domains, it is unknown whether changes in androgen levels, either natural or medically-induced, lead to long-term effects on decision making (let alone risk-based decision making). Nevertheless, preclinical studies reporting effects of orchiectomies on decision making involving risk of punishment (Orsini et al., 2021) suggest that similar changes (e.g., reduction in risk taking) would occur in men undergoing androgen deprivation therapy, as both manipulations lead to fairly dramatic reductions in androgens. The fact that testosterone administration does not restore risk taking in orchiectomized rats (Orsini et al., 2021) also suggests that androgen therapy would not be beneficial in attenuating deficits in risk taking in this patient population. In this same rodent model of risk-based decision making (i.e., RDT), aged male rats also display reduced risk-taking behavior relative to young male rats (Dragone et al., 2019), consistent with increased risk aversion reported in a proportion of older adults (Mata et al., 2011). This change in risk-based decision making could therefore in part be due to moderate, but still behaviorally relevant, reductions in circulating androgens. Additional studies at the preclinical and clinical levels are necessary to test this hypothesis and fully understand the causal relationship between age-related decreases in androgens and changes in risk-based decision making.

6.2. Potential effects of chronic exposure to exogenous hormones during reproductive years on decision making

More than 100 women globally are on some form of hormonal contraceptive (HC) during their prime reproductive years (Christin-Maitre, 2013). In addition to using these compounds for contraceptive purposes, many HC users take these compounds, particularly oral contraceptives (OC), to combat negative affect associated with hormonal fluctuations of the menstrual cycle or for non-contraceptive medical purposes, such as polycystic ovary syndrome (Porcu et al., 2019). Consisting of synthetic estradiol and progestins (although some contain only progestins; Christin-Maitre, 2013), OCs work by binding to ERs and PRs and prevent release of estradiol and progesterone from the ovaries, which ultimately inhibits ovulation (Lobo and Stanczyk, 1994; Porcu et al., 2019; Stanczyk et al., 2013). Use of OCs therefore results in a hormonal milieu consisting of high levels of synthetic hormones and low levels of endogenous hormones. Like pregnancy and menopause, chronic exposure to this distinct hormonal environment may lead to a reorganization of cellular and circuit activity in the brain, which may consequently alter cognition (Pletzer and Kerschbaum, 2014). Given the large proportion of women who use HCs, there is significant interest in understanding how this hormonal environment, particularly during the premenopausal period, affects cognition and underlying brain mechanisms.

Evidence for modulation of cognition by OCs in women is inconclusive, with effects depending on the cognitive domain being assessed and the type of OC used in the subjects (e.g., progestins alone vs. synthetic estradiol and progestin combined). Most of these studies have focused on assessments of spatial, numerical and verbal ability (Bronnick et al., 2020; Porcu et al., 2019). There are far fewer studies that have examined the effects of OCs on cognitive processes related to decision making. Of those that have been conducted, the findings suggest that OCs may in fact be beneficial. For example, Egan and Gleason (2012) found that women who use HCs during the premenopausal period display better cognitive flexibility during the postmenopausal period and this effect is dependent on the duration of HC use (better flexibility later in life with longer use duration earlier in life). Similarly, relative to women given placebo control, women who are administered OCs display improvements in the ability to regulate and restrain impulses (Gingnell et al., 2016). Finally, and more directly related to decision making, a recent study reported that HC use (not restricted to OCs) improves female participants’ understanding of choice contingencies in the Iowa Gambling Task (although HCs did not affect overall choice performance; Nobile et al., 2021). The subjects in this study, however, were women diagnosed with bulimia nervosa (BN), which itself is associated with hormonal dysregulation (Crow et al., 2002; Pirke et al., 1987). Hence, it is possible that HCs are effective in ameliorating slight decision-making deficits in women with existing hormonal dysfunction, but may not confer the same benefits to women who have relatively normal hormonal function.

Hypotheses about the effects of HCs on decision making can be drawn from studies that have explored the impact of HCs on activity in brain regions known to contribute to decision making. Cahill and colleagues have shown that women using OCs have decreased amygdala activity in response to negative emotional stimuli (Petersen and Cahill, 2015). Further, relative to non-OC users in the late follicular phase (i.e., temporarily high estradiol levels), OC users have reduced amygdala activity in response to a stimulus paired with an aversive electric shock (Hwang et al., 2015). Interestingly, non-OC users in the luteal phase (i.e., temporarily low estradiol levels) also have reduced amygdala activity to the stimulus predicting the shock. The findings of this study suggest that the effects of estradiol on amygdala activity may depend on its cyclicity and whether it is naturally or synthetically derived. Previous work has also provided evidence that chronic HC use may lead to structural reorganization in the brain. For example, there is reduced gray matter volume in the amygdala (Lisofsky et al., 2016) and changes in connectivity between the amygdala and prefrontal cortical regions (Engman et al., 2018; Lisofsky et al., 2016) in healthy HC users. Hence, chronic exposure to synthetic hormones (as opposed to cyclical exposure) may lead to a gradual reduction in amygdala function, which could result in alterations in amygdala-dependent behavior like risk-based decision making. Future studies are clearly needed to directly examine the long-term effects of HCs on activity in brain regions like the amygdala while concomitantly assessing their impact on decision making. Such knowledge is critical to determine whether long-term use of HCs influences vulnerability to the development of psychiatric diseases associated with altered risk-based decision making (e.g., substance use disorders, anorexia nervosa; Chen et al., 2020; Kaye et al., 2013).

6.3. Implications for understanding neuropsychiatric diseases

There are sex differences in the prevalence, onset, and severity of many psychiatric diseases that are associated with impairments in risk-based decision making, such as substance use disorder, depression, and schizophrenia. Because hormones influence risk-based decision making, it is conceivable that impairments in this cognitive process arise from disease-related dysfunction in hormonal regulation of decision making (and the underlying neural substrates). Although there is little research demonstrating a direct link between dysregulated hormonal modulation of risk-based decision and neuropsychiatric diseases, there is evidence for hormonal influence on several pathological conditions, providing indirect support for this hypothesis. Several aspects of substance use disorders (SUDs), for example, that disproportionately affect women relative to men are due to the interactions between ovarian hormones and the neural mechanisms that contribute to drug-related behavior. For instance, women report greater drug responses during phases of their menstrual cycle when levels of estradiol are greatest (Evans, 2007; Evans et al., 2002; Terner and de Wit, 2006). Similar effects are also observed in animal studies, with the greatest stimulant-induced locomotor effects occurring during proestrus (Becker et al., 1984, 1982). Moreover, acquisition and escalation of cocaine self-administration is accelerated by administration of estradiol in ovariectomized females (Hu et al., 2004; Larson et al., 2007; Lynch et al., 2001). Such facilitating effects of estradiol on drug-related behavior occurs through estradiol’s ability to enhance dopamine release in the dorsal striatum and ventral striatum (i.e., NAc; for excellent reviews on this topic, see Kokane and Perrotti, 2020; Yoest et al., 2018). Although the fact that estradiol facilitates drug-seeking behavior seems to contradict its ability to promote risk aversion in females in some rodent models of risk-based decision making (i.e., RDT; Orsini et al., 2021), there is evidence to suggest that chronic exposure to cocaine leads to long-term changes in the menstrual cycle in women and non-human primates and in the estrous cycle in rodents (King et al., 1990; Mello and Mendelson, 1997; Mello et al., 1997). Hence, irrespective of estradiol’s influence on drug-seeking behavior, chronic exposure to drugs could disrupt hormonal regulation of cognitive processes, such as risk-based decision making, leading to the cognitive impairments that can contribute to continued drug use and/or relapse.

In contrast to its role in drug-related behavior associated with SUDs, estradiol seems to protect against or mitigate positive symptoms of schizophrenia. Indeed, hormone therapy is often used as an adjunct treatment with antipsychotics for schizophrenia (Seeman, 1996). This approach stemmed from the “estrogen protection hypothesis”, which states that greater estradiol levels in women at an early age protect them from the development of schizophrenia (relative to males) and that declining levels of estradiol at menopause may lead to onset of schizophrenia in predisposed women (Kulkarni, 1997; Seeman, 1997). In support of this hypothesis, psychotic symptoms in females fluctuate across the menstrual cycle, with the most severe occurring during phases associated with low levels of circulating estradiol (Bergemann et al., 2007; Rubin et al., 2010). Other studies have shown that male and female schizophrenics have low levels of serum estradiol (i.e., hypoestrogenism; Belvederi Murri et al., 2016; Bergemann et al., 2002, 2007; Dogan Bulut et al., 2016; Gogos et al., 2019). Similar to SUDs, it is the interaction between estradiol and dopamine which seems to underlie symptoms of schizophrenia (Kulkarni et al., 2012). Animal studies have shown that common behavioral endophenotypes of schizophrenia, such as disrupted pre-pulse inhibition, can be induced by dopamine receptor agonists (Gogos et al., 2010). The administration of estradiol, however, prevents these effects. Notably, alterations in other neurotransmitter systems also contribute to the pathology of schizophrenia and are regulated by estradiol (e.g., serotonin; Kulkarni et al., 2012). Regardless of specific mechanism, studies have consistently shown that estradiol has a protective effect against the development of schizophrenia. Schizophrenia is associated with impaired risk-based decision making (Brown et al., 2015), which could contribute to other risky behavior that commonly manifest with schizophrenia, such as pathological gambling (Desai and Potenza, 2009) and substance use (Hartz et al., 2018, 2017). Neuroimaging studies have shown that such cognitive deficits are associated with increased activation in brain regions involved in evaluating outcomes of decisions (i.e., striatum; Tikasz et al., 2019). It is therefore conceivable that, in addition to contributing to other positive symptoms of schizophrenia, low levels of estradiol, or a general dysfunction in the neuroendocrine system, may moderate deficits in decision making in males and females with schizophrenia. Future studies are needed to test this hypothesis and determine whether hormone therapy can ameliorate these specific cognitive deficits associated with schizophrenia.

7. Conclusion

Risk-based decision making is a cognitive process that we engage in on a daily basis to allow us to navigate our environment and maintain a healthy lifestyle. Our ability to make effective decisions is governed by a network of brain regions, many of which are susceptible to the influence of circulating hormones. The modulation of neural activity by hormones can give rise to sex differences in the ability to evaluate different choices and their outcomes to guide future decisions. Despite progress in recent years in identifying the role of hormones in risk-based decision making, we still have a limited understanding of the mechanisms by which this process occurs. The use of animal models of risky decision making, in combination with techniques to monitor and/or manipulate brain activity and hormone levels simultaneously, can address this gap in knowledge and advance our understanding of the interactions between the neurobiological and neuroendocrinological substrates of decision making.