Sex differences in learning processes of classical and operant conditioning

Abstract

Males and females learn and remember differently at different times in their lives. These differences occur in most species, from invertebrates to humans. We review here sex differences as they occur in laboratory rodent species. We focus on classical and operant conditioning paradigms, including classical eyeblink conditioning, fear conditioning, active avoidance and conditioned taste aversion. Sex differences have been reported during acquisition, retention and extinction in most of these paradigms. In general, females perform better than males in the classical eyeblink conditioning, in fear-potentiated startle and in most operant conditioning tasks, such as the active avoidance test. However, in the classical fear conditioning paradigm, in certain lever-pressing paradigms and in the conditioned taste aversion males outperform females or are more resistant to extinction. Most sex differences in conditioning are dependent on organizational effects of gonadal hormones during early development of the brain, in addition to modulation by activational effects during puberty and adulthood. Critically, sex differences in performance account for some of the reported effects on learning and these are discussed throughout the review. Because so many mental disorders are more prevalent on one sex than the other, it is important to consider sex differences in learning when applying animal models of learning for these disorders. Finally, we discuss how sex differences in learning continue to alter the brain throughout the lifespan. Thus, sex differences in learning are not only mediated by sex differences in the brain, but also contribute to them.

Why study sex differences in learning and memory?

It is well established that men and women perform differently on some types of learning tasks, but the reasons for those differences are not well understood. For example, women tend to outperform men on tasks that involve verbal or semantic rules and memories related to personal experience, whereas men perform better on tasks that require the manipulation of complex spatial information, such as the manipulation of shapes in space [1-4]. Numerous studies report sex differences in humans [5, 6]. These differences are attributed to a variety and a combination of factors including those of genetic, hormonal and social origin [7-14]. That said, many of the reported sex differences are relatively small in magnitude and their emergence may only arise as a function of the task, its difficulty or the specific experimental design [14]. For these reasons, sex differences in human cognition are difficult to be observed and once observed, even more difficult to interpret. Despite these difficulties and the controversies that arise, there is much interest in the topic. In part, this information may be useful for educational purposes, because girls and boys may be better suited to different types of instruction. Although men and women often have capacities, they may excel with different teaching or environmental challenges [15]. For education purposes as well as general knowledge, it is important to know whether sex differences that do exist are genetic, hormonal or social basis in nature and as a consequence, whether they are immutable or not.

Perhaps more importantly, our knowledge on sex differences in learning can help us prevent and treat more effectively a wide range of sex-differentiated psychiatric and neurological disorders in which cognitive alterations are associated with their aetiology or symptomatology. For example, alcoholism, autism and attention deficit hyperactivity disorder (ADHD) are much more common in men, whereas Alzheimer's disease, social phobias, eating disorders, post-traumatic stress disorder (PTSD) and major depression are more common in women [16-24]. Many of these disorders are treated with cognitive-behavioral therapy (CBT) and other models of learning. For example, PTSD and anxiety disorders are often treated with exposure therapy, extinction and systematic desensitization, all processes of associative learning. Similarly, drug addictions, which are more common in men than women, are interpreted as a function of reinforcement based on principles of classical and operant conditioning theory [25, 26].

In all animal species, including humans, sex differences in learning vary according to the demands of the task and what the animal is required to perform and presumably learn [27-29]. Thus, a greater appreciation for these differences will likely enhance the validity of the animal model. Minimally, the presentation of sex differences in learning underscores the need to include females in basic research studies [30]. They are often not included either because of the problems inherent to monitoring the estrous cycle, but also because it is assumed that their behaviours will be similar to males, albeit more variable. Oddly enough, in human studies, men tend to express more variance on test scores of cognitive and intellectual performance [31].

In this review, we present evidence for sex differences in classical and operant conditioning paradigms, with a focus on classical eyeblink conditioning, fear conditioning, escape and active avoidance, and conditioned taste aversion. We will also discuss whether and if so, how these sex differences are inherited or minimally organized during development by the presence of gonadal hormones [32]. Some of the sex differences depend on the presence of reproductive hormones, because they can be either eliminated or in some cases reversed by hormonal manipulations in adulthood [33]. When available, we will present data showing differences in performance as a function of stages of the estrous cycle and/or in response to reproductive hormone manipulations. Sex differences in spatial learning, although important, will not be discussed in this review, because they have been reviewed elsewhere [28, 29, 34-38].

Sex differences in classical eyeblink conditioning

The classical eyeblink conditioning paradigm is an associative learning task. In this paradigm, the animal is presented in each trial with a white noise (conditioned stimulus, CS), which is followed closely (within a second) by an aversive stimulation to the eyelid, which causes the animal to blink (unconditioned response, UR). The unconditioned response becomes conditioned as the animal learns that the CS predicts the occurrence of the eyelid stimulation, the unconditioned stimulus (US) (Figure 1a). This is a useful response to study because the animal quickly learns that the CS and the US are associated (i.e. emits a conditioned response, CR), but requires many trials to learn to time the eyeblink so that it occurs precisely (within milliseconds) before the US. The slow rate of acquisition allows the experimenter the opportunity to assess learning and asymptotic performance with a certain degree of accuracy. Moreover, this task is useful because its anatomical circuitry has been described, and it can be assessed in most species, including humans [5, 39, 40]. Finally, different versions of this task exist that allow a more sophisticated approach in identifying specific learning processes and anatomical substrates. For many of our studies, we rely on a procedure known as trace conditioning. This task depends on the hippocampus and requires multiple trials to learn because the CS is separated with the US with a trace interval of 500 ms. We also observe sex differences during delay conditioning. Delay conditioning is much easier to learn than trace conditioning, because the CS and the US overlap in time; learning this association does not depend on the hippocampus (Figure 1) [41, 42].

Figure 1. Females outperform males in eyeblink conditioning.

a. During eyeblink conditioning, a conditioned stimulus (CS) of white noise is paired with an unconditioned stimulus (US; periorbital stimulation), which elicits an eyeblink response. In the non-hippocampal dependent task of delay conditioning the two conditioning stimuli overlap in time. In the hippocampal-dependent task of trace conditioning the two conditioning stimuli are separated by a trace interval or “temporal gap” of 500 ms. As an animal learns that the two stimuli are associated, it blinks in anticipation of the US. Eyeblinks are detected by an increase in the magnitude of the electromyographical (EMG) response of the muscle of the upper eyelid. Those blinks that occur close to the US are considered conditioned responses (CRs). The electrophysiological record shows an example of a CR that occurred just prior to the onset of the US.

b. Intact females emit more conditioned responses than intact males during trace eyeblink conditioning (p<0.05). In this experiment, all females were in diestrus on the first day of training. Rats were trained with 800 trials of trace eyeblink conditioning over a period of 4 days (200 trials/day). The sex difference is evident during the first two days of training [47].

During both trace and delay conditioning, females outperform males [33, 43-47] (Figure 1b). In particular, females begin to learn to time the CR (i.e. eyeblink) sooner than do males and the sex difference is most evident during the first day of training. After a few days of training, males and females eventually usually reach similar levels of performance (i.e. asymptote), but the sex differences are prevalent and evident across hundreds of trials of training. When the timing of the learned responses is analyzed, we observe that females emit well-timed learned responses sooner during the trace interval than do males. Thus, for whatever reason, they are learning to anticipate the onset of the US and as a consequence can emit a more well-timed response. Also, in experiments with a limited number of trials, more females than males reach a learning criterion of 60% responding [43, 47, 48].

Importantly, classical eyeblink conditioning is less influenced by performance factors that can inadvertently affect learning. That said, one can never rule out performance effects entirely. In terms of general responsivity, we have never seen any sex difference in spontaneous blinking before training. Neither have we observed any sex differences in eyeblink response to the white noise alone (the eventual CS) before training or in the magnitude of the UR. Also, both males and females do not respond to the first trial, before any US has occurred [47]. Most convincing is our recent data on memory. In this study, females that are trained and weeks later they are re-exposed to the CS, respond with many more CRs than do males. These data indicate that female rats actually learn the CR better than males and thus they retain it better [49].

The magnitude of the sex difference in eyeblink conditioning appears to be sensitive to sex hormone concentrations, because it is most evident when females begin training in proestrus, when estrogens levels are high [43-45, 50]. Also, removal of ovarian hormones prevents the sex difference in performance [44]. On the other hand, estrogens replacement at physiological doses does not enhance conditioning in ovariectomized females. Only very high non-physiological doses of estradiol are effective. In this case, the hormone enhances conditioning and extinguishes responding [33]. As said, under normal cycling levels, we observe that females outperform males. This sex difference is very evident in females that are trained in proestrus, when estrogens levels are at their peak. However, the sex difference is apparent even in females that are trained during diestrus, a phase of the estrous cycle characterized by lower levels of estrogens and progesterone (Figure 1b) [47]. Thus, sex differences are evident across the estrous cycle and are not solely dependent on the presence of estrogens in females. Other factors that might regulate these sex differences are not known, but progesterone and luteinizing hormone levels in adult cycling females may also contribute to sex differences in learning. It is noted that one would not expect a total dependence of basic learning processes on one or even a few hormones, because the adaptive value of learning is too great. Males and females of all species undergo massive changes in sex hormone availability across their lifespan, so learning cannot be massively influenced by their levels.

In addition to estrogens, sex differences in learning are also mediated or minimally modulated by the presence of testosterone in both males and females, at least under some limited circumstances. For example, classical conditioning in adult male rats that were treated with a testosterone antagonist in utero or were castrated at birth, is not affected [46]. Neither does castration in adulthood has any effect on performance during trace eyeblink conditioning [51]. Therefore, testosterone during brain development or in adulthood does not influence trace eyeblink conditioning in males. On the other hand, classical conditioning is decremented in adult females if they are treated with testosterone on the day of their birth. In fact, masculinized females then perform similarly to males [46]. These results indicate that testosterone exerts some organizational effects in the brain of these females. It is noted that the adult masculinized females do not cycle and have lower plasma levels of estrogens than intact females, suggesting that low levels of estrogens in adulthood may be linked with their delayed acquisition during eyeblink conditioning [46]. Again, this relationship is not absolute because, as mentioned above, females with relatively low levels of estrogens still outperform males in adulthood [47]. Therefore, it seems that the manipulation of hormones during development is inducing a more general and probably pervasive effect on conditioning than that observed across the estrous cycle.

Interestingly, the sex differences in learning emerge after puberty, whereas no sex differences are observed before or during puberty [45]. Again, these data indicate that sex differences in eyeblink conditioning cannot be fully explained by differences in plasma estrogens levels, because pubescent and adult females have similar estrogens levels, but only adult females exhibit enhanced performance in comparison to males. The same conclusion holds for progesterone levels, which are higher in females than in males both during puberty and in adulthood [45].

Thus, sex differences in learning are influenced by both organizational and activational effects of sex hormones, but in a complex and probably interactive way. High estrogens levels in the plasma of adult female rats are definitely an important factor for the expression of enhanced learning in females, but they do not fully account for the sex differences. Among other potential candidates, it is possible that local estrogens production in the brain, which is also influenced by organizational effects of sex hormones and estrous cyclicity, might contribute to the enhanced learning in females [52, 53].

The neuroanatomical substrates that might account for sex differences have not been identified, but one does run in parallel – that of changes in the number of dendritic spines on pyramidal cells of the CA1 area of the hippocampus [54-56] (Figure 2a). Spine densities are enhanced during proestrus and after exogenous estrogens administration [57-59]. Additionally, spine densities are higher in proestrus females than in males [55] (Figure 2b). These differences, along with other neurobiological differences, might be linked with sex differences in learning. This is further supported by the fact that both trace and delay eyeblink conditioning increase spine densities in the CA1 area of the hippocampus, at least in male rats that this effect has been studied [60]. Further evidence comes from studies related to stress [61]. In particular, exposure to an uncontrollable stressor severely impairs classical conditioning in females, which also possess significantly fewer dendritic spines in their hippocampus after the stressor. In contrast, exposure to the same uncontrollable stressor greatly enhances classical conditioning in males, which they possess significantly more spines after the stressor [43, 54, 55]. Finally, we find that stress enhances learning in females that are masculinized and they possess more spines after the stressor [32, 62]. Thus, under all these various circumstances, performance of the learned response relates in a positive way to the presence of dendritic spines in the CA1 area of the hippocampus [61].

Figure 2. Dendritic spines in the hippocampus might represent an anatomical background for learning.

a. Representative pictures of pyramidal neurons in the CA1 area of the hippocampus stained with the Golgi impregnation technique (magnification 200×). Density of dendritic spines is measured in the basal and the apical dendrites of the pyramidal cells. A closer image (magnification 1000×) of dendritic spines in the apical dendrites of a female rat is shown [56].

b. Female rats in proestrus have higher densities of dendritic spines in the CA1 area of the hippocampus, in comparison to females in diestrus and in estrus and to intact males [55].

Another very interesting effect of successful learning during trace eyeblink conditioning is the enhancement of the survival of new cells that were born in the hippocampal formation one week before the start of training [63-66]. In a recent study we found that female rats learn better than males in trace eyeblink conditioning and as a consequence a greater proportion of new neurons are rescued in their hippocampus than in males (Figure 3) [47]. This finding underscores the importance of sex differences in learning and is an example of how learning can influence the anatomy of the adult brain in a sex-differentiated way.

Figure 3. Associative learning increases neurogenesis in the male and female hippocampus.

a. The density of adult generated cells (BrdU-labeled cells) in the dentate gyrus of the hippocampus is enhanced in rats trained with trace eyeblink conditioning, in comparison to untrained animals. The percent increase is greater in trained females (34%) than in trained males (17%). The density of new cells in untrained females is lower than that in untrained males (significant differences are noted with an *) [47].

b. A representative picture of the dentate gyrus is shown (magnified 20×). In the smaller panel (magnified 1000×), the new cells in the dentate gyrus of the adult hippocampus are stained in brown (BrdU-labeled cells). The hippocampal slice was counterstained with cresyl violet (cells shown in purple).

Sex differences in fear conditioning

Sex differences have been reported for a different type of classical conditioning often referred to as fear conditioning. In the typical procedure, animals are trained to associate a cue or a context (CS) with an aversive stimulus, usually a footshock (US). As a result, when they are re-exposed to the same context or cue (e.g. tone), they learn to freeze or express an enhanced startle reflex (CR) in anticipation of the footshock. Females tend to express less conditioned freezing behaviour than males during acquisition and testing for contextual fear conditioning [67, 68]. As during classical eyeblink conditioning, castration of male rats in adulthood does not alter the conditioned responses, while ovariectomy is effective [33, 51, 69, 70]. In particular, females that are ovariectomized in adulthood express similar levels of fear to males with an enhanced resistance to extinction. Some studies suggest that estrogens replacement normalizes these effects, implicating activational effects of estrogens [70]. Thus, it seems that fear conditioning is not mediated by testosterone levels in adulthood, while estrogens have an effect. Sex differences in fear conditioning and effects of ovariectomy do not appear to be directly related to less general freezing behavior in females than in male rats (see also below), because gonadally-intact females and ovariectomized females exhibit the same levels of freezing during acquisition [70]. However, it is still possible that the tendency of male rats to respond to shock with enhanced freezing might influence performance on this task.

In cue fear-conditioning, where a tone is used as the CS, again male rats acquire the association faster than females by exhibiting higher freezing in fewer trials [67, 68]. This effect has also been shown with a different measurement of fear, such as the ultrasonic vocalizations, suggesting that males exhibit more conditioned fear than females [71]. However, conflicting results have been obtained on whether sex differences exist during testing (retention) and extinction of cue fear conditioning, when rats are placed in a different context (CR: time spend freezing) [67, 68, 72]. When ultrasonic vocalizations are used as measurement of the CR, males show a higher responding than females during retention tests [71].

Thus, in general, males outperform females in fear conditioning paradigms. The sex differences in fear conditioning are in contrast to those observed during training with the eyeblink conditioned response. The differences are probably due to the inherent differences of the two tasks and the variable used as the conditioned response (CR). In the fear conditioning paradigm non-associative factors, such as fear might interfere with learning. It is possible that males perform better than females, because they acquire and retain the conditioned fear in a higher degree. In the classical eyeblink conditioning paradigm, in which fear may be minimized, females outperform males. However, when males are previously exposed to a traumatic fearful experience, such as exposure to tailshocks, then associative learning is enhanced and this is facilitated when the context of stress and training is the same [73].

As in other learning paradigms, various stressors, such as neonatal isolation and maternal separation influence fear-conditioning during adulthood, in a sex-dependent way [71, 74]. In particular, the early life stress of neonatal isolation results to enhanced context-induced fear in adult females, while it tends to impair context-induced fear in adult males. On the other hand, prolonged maternal separation impairs context-induced fear in adult females, while brief maternal separation impairs both context and cue fear conditioning in both sexes [74]. Again, these effects are seen only with the sensitive measure of ultrasonic vocalizations and not with measurement of the freezing behavior [71, 74]. Overall, stress either enhances or impairs associative learning depending on the stressor, the task, the age and the sex of the animal.

In another paradigm, fear is assessed by the startle response of the rats. In the fear-potentiated startle paradigm, the rats learn that the presence of a cue, such as a light is paired with a footshock. Therefore, when they are later exposed to the light without the footshock, they show enhanced startle reflex, in response to a sudden noise [75]. In this paradigm intact females show a greater potentiation of startle than males [76]. However, females also have a higher baseline acoustic startle reflex (non-associative measure) than males and this is influenced by the phase of the estrous cycle, lactation and previous stress exposure [77-79]. Estrogens and progesterone treatment of ovariectomized females induce opposite effects on fear-potentiated startle, with estrogens enhancing performance in retention tests and addition of progesterone to counteract these effects [80]. However, when acquisition is delayed in the fear-potentiated startle paradigm by presentation of two light-shock pairings a day over a period of 8 days, no sex differences are observed, at least with respect to acquisition of the startle response; as in other studies, castration of adult males has no effect on performance [79, 81]. Thus, it seems that testosterone levels in males do not influence their performance on the fear potentiated startle, while estrogens and progesterone levels mediate performance on this task.

Exposure to a bright light for 5-20 min leads to enhanced amplitude of the acoustic startle response and thus this manipulation has been used for the development of the light-enhanced startle paradigm, which measures unconditioned anxiety [81, 82]. In this paradigm females exhibit an enhanced response than males, an effect that is reversed by castration of adult males. In retention tests, treatment of castrated male rats with testosterone reduces light-enhanced startle, and this has been attributed to the anxiolytic effects of testosterone [79, 81]. In this instance, ovariectomy has no effect, although pregnancy is associated with an enhancement of the light-enhanced startle response [79]. These scientists went on to conduct a rather ingenious set of studies using a discrimination task which afforded measurement of the excitatory (fear-provoking) and the inhibitory (fear-reducing) aspects of learning. In these studies, estrogens treatment of ovariectomized females disrupts the inhibition of fear [83]. These data along with other indicate that females with high estrogens levels may not be able to suppress the fear associated with aversive experience.

Sex differences in operant conditioning

Females also outperform males during operant conditioning, but probably for very different reasons than their facilitated acquisition during classical conditioning [27, 84-88]. During an operant task, an animal must make an overt response in order to learn. Often, they learn to escape an aversive stimulus, such as a mild foot shock. Recently, as part of our learned helplessness studies, we trained male and female rats on the one-way avoidance task (FR1) [84, 85]. In this task, the rats must pass through the door way of a shuttle box once, in order to avoid a mild footshock (1 mA) [89]. This is an easy task and most rats learn to avoid the footshock very fast on the first day of training. However, females escape sooner than males during the first trials of the one-way avoidance task [84, 85]. More striking sex differences emerge when males and females are tested on a more difficult task, the two-way avoidance task (FR2) (Figure 4a) [84, 85]. In this task the rats need to pass twice through the doorway of the shuttle-box, in order to terminate the footshock. Although the task is more difficult, most females readily learn to escape the stimulus on the first trials of testing, whereas males require more trials. In some cases, male rats never learn to escape the shock, even after 30 trials of training [84]. It seems that males avoid the side of the box where they just got shocked and thus they do not pass through the doorway twice. In contrast, females do not avoid it and thus they learn to pass twice. This has been shown before, in passive avoidance studies, in which animals must leave the light compartment and enter the dark compartment of the testing box, where they had received a footshock the previous day. In this test, again females are more likely than males to re-enter the dark compartment, in which they previously have been shocked (for review see: [27]). Also, extinction studies have shown that when females are placed again in the shuttle-box without any shock presentation, they escape sooner than males, suggesting that extinction of the avoidance behaviour is slower to occur in females [27]. Finally, females also outperform males in another type of operant conditioning, in which they need to avoid the shock by pressing a lever [90, 91].

Figure 4. Females outperform males in the active avoidance task.

a. Gonadally-intact females escape sooner than gonadally-intact males in active avoidance (p<0.05). Females were in various stages of the estrous cycle at the time of testing. Rats were trained to avoid a mild footshock by passing though the doorway of the shuttle box twice (FR2), for 30 trials in one day [84].

b. Ovariectomy of adult females in adulthood has no effect on active avoidance. Gonadally-intact and ovariectomized females exhibit the same escape latencies when they are trained to avoid a mild footshock by passing twice though the doorway of the shuttle box (FR2), for 30 trials in one day [84].

c. Castrated adult males learn to escape sooner the mild footshock in the FR2 task, in comparison to gonadally-intact adult males (p<0.05). Rats were trained to avoid a mild footshock by passing though the doorway of the shuttle box twice (FR2), for 30 trials in one day [84].

Sex differences in operant conditioning are at least in part attributable to sex differences in performance [92]. Female rats respond actively to aversive stimulation, such as the footshock, whereas male rats exhibit behavioural inhibition by reacting passively and freeze [27, 86, 87, 93]. Several performance issues have been raised in the explanation of sex differences in avoidance behaviour. Most evident are the sex differences in activity; female rats are in general more active than males, as demonstrated with the open-field test and with running wheels [94-97]. Also, sex differences in electrical resistance and reactivity likely have an impact on their performance in operant conditioning [27, 87, 98]. In general, females are more sensitive to shock than males, as measured by fear responses [27, 71], although it has been suggested that in most studies the shock intensitivity is sufficient to eliminate these differences [87]. Perhaps, sex differences in nociception or stress-induced analgesia phenomena might play an undetectable role in these phenomena [99-102]. That said, hormonal or environmental manipulations that alter activity, shock and pain sensitivity do not always result to parallel changes in avoidance behaviour [84, 87, 97, 99, 103]. Thus, it seems that males and females have different strategies in response to aversive stimuli and probably account for some sex differences in the expression of fear (see above). Nevertheless, in order to control for performances issues, it is recommended to measure activity levels before and/or after the test and also to assess more behaviours other than freezing, such us jumps and vocalization, during shock exposure.

Removal of the ovaries does not prevent the sex difference in operant conditioning, as it has no effect on general activity [84, 87, 88, 91, 97] (Figure 4b). However, cycling estrogens and progesterone seem to influence conditioned avoidance behaviour and escape latencies, because females perform better during proestrus, when estrogens levels are high [104]. Studies with exogenous estrogens administration showed that low doses of estrogens disrupt, while high doses facilitate acquisition of the active avoidance behaviour [103]. Finally, estrogens replacement delays extinction in the one-way avoidance task in female rats, suggesting that estrogens play a role in the maintenance of the escape behaviour [105]. In contrast, castration of adult males facilitates learning in operant conditioning (Figure 3c) [84, 91]. It seems that high testosterone levels in adulthood mediate the behavioural inhibition that male rats develop when they are exposed to the footshock. Testosterone has been proposed to have anxiolytic properties and this might contribute to the passive strategy that male rats choose in response to the footshock [106]. Additionally, organizational effects of testosterone seem to play some role in the emergence of sex differences in avoidance behaviour [27]. When females are masculinized by exposure to testosterone on the day of their birth, they do not perform like males as adults [84, 87]. However, if they are also treated with testosterone in adulthood, then they exhibit male-like escape avoidance latencies [87]. Neonatal castration of male rats does not have an effect on this kind of behaviour [88, 107], but prenatal exposure to an anti-androgen combined with neonatal castration produces escape latencies in adulthood, which are indistinguishable from females [88].

In another type of operant-conditioning, rats learn to press a lever, in order to receive a positive stimulus, such as food. In this task, males exhibit higher contact with the lever, irrespective of whether they will receive the reinforcer (i.e. food) or not. So, male rats perform better than females in this task (for detailed review see [27]). Sex differences in lever-pressing have been attributed to higher general activity of females than males and to the fact that during testing females engage to different activities than pressing the lever [27]. On the other hand, females perform better in a variation of the task, in which they need to systematically increase the number of times that they press the lever, in order to receive the reinforcer (differential reinforcement of low rate responding) [108, 109]. Interestingly, when other objects are placed in the experimental environment, the acquisition of the differential reinforcement of low rate responding is facilitated in males, but not in females [109]. Also, sex differences in differential reinforcement of low rate responding seem to depend on activational effects of hormones, because they are not observed during puberty, they are abolished by ovariectomy and neonatal castration of males has no effect [110]. Also, bar-press duration correlates with estrous cyclicity [111]. Thus, sex differences exist in lever-pressing paradigms with males outperforming females in simple tasks, while females performing better in more difficult tasks, such as the differential reinforcement of low rate responding.

In a recent study, no sex differences in pre-pubertal and adult rats are observed in compulsive lever-pressing, which is observed during extinction training. In this task, rats are trained to associate the pressing of a lever with a tone, a light and a food reward, while during extinction the pressing of the lever results to the emergence of the tone and the light, but not to food reinforcement [112]. In the initial phases of training, adult females acquire the association slower than males, while pre-pubertal rats are faster than adults. Interestingly, compulsive responding during extinction is highest during late diestrus, when estrogens levels are very low and lowest during estrous phase of the estrous cycle, when estrogens levels start to rise. Acute administration of estrogens to pre-pubertal females attenuates lever-pressing, while withdrawal from chronic administration of estrogens facilitates this behaviour, suggesting that estrogens protect from compulsive lever-pressing [112].

Again, in operant conditioning, as in classical eyeblink conditioning it seems that sex differences in learning are organized during development of the brain, mainly by prenatal exposure to androgens in males. These effects are combined with exposure of the male brain to testosterone during the postnatal critical period of the development and with other effects of testosterone later in puberty and adulthood. It seems that in active avoidance tasks testosterone plays a more important role than estrogens in the emergence of sex differences, while in eyeblink and fear conditioning estrogens and progesterone in females seem to be more important. These tasks differ in the fact that during operant conditioning, the animal must emit a voluntary motor response in order to learn the contingency, whereas during classical (i.e. Pavlovian) conditioning, the animal emits an obligatory unconditioned response to the unconditioned stimulus, irrespective of volition. Thus, the two types of training procedures are vastly different in terms of their dependence on volitional activity and the learning processes involved. Therefore, different hormonal factors seem to mediate each task.

Again, previous stress exposure has a profound effect on these behaviours. The active avoidance task has been extensively used after exposure to uncontrollable stress, in order to assess learned helplessness [113, 114]. When male rats are exposed to uncontrollable stress, they do not learn to escape the shock in the active avoidance task and thus they become helpless. On the contrary, females exposed to the same regime of uncontrollable stress learn to escape and thus they do not express learned helplessness behaviour [47, 85, 86, 93]. The sex differences in helplessness are certainly mediated by the aforementioned sex differences in operant conditioning. However, gonadectomy of either sex in adulthood or masculinization of the female brain with testosterone on the day of birth does not prevent sex differences in learned helplessness behaviour [84].

Sex differences in conditioned taste aversion

During conditioned taste aversion, animals learn to associate the consumption of a sweet solution with an aversive compound that makes them physically ill. In the laboratory, the compound is usually quinine or lithium chloride administered after the animal drinks sucrose or saccharin. As a consequence of the illness, rats learn to avoid the sweet solution when they are re-exposed to it. Sex differences are not readily apparent during acquisition, which is conserved and persistent, but sex differences do occur during extinction [115-117]. Females typically extinguish sooner than males; they will again consume the sweet solution in less time and do so in greater quantities [116, 118, 119]. It is reported that castration of males accelerates extinction, whereas ovariectomy has no consequence. Interestingly, administration of testosterone exogenously increases the duration of extinction in both sexes, suggesting an activational role for testosterone in this task [118, 119]. On the other hand, when estradiol is administered in ovariectomized female rats before extinction studies, it accelerates extinction of the aversive association [118, 120, 121]. Thus, activational effects of sex hormones seem to influence extinction of the learned response during the conditioned taste aversion paradigm. To our knowledge, there is no report in the literature concerning developmental effects of sex hormones. Given the time course, such studies would be difficult to do, although not impossible.

As with most findings, the emergence of sex differences depends on the parameters of the task, compounds and doses used and the status of the animal, such as whether they were water deprived or not [27, 116, 122]. Again, performance on this task is most likely influenced by sex differences in the preference for sweet solutions and in the aversive/rewarding properties of the compounds used [115, 123-125]. This is also the case, when a drug of abuse is used as the aversive compound [116, 117, 124, 126]. For example, when female rats are given i.p. injections of amphetamine as the aversive stimulus, they acquire significantly stronger aversions than males at low doses and show slower rates of extinction [127]. Although these data are very interesting, it is difficult to evaluate them from a learning perspective, because of the potent rewarding properties of the different classes of drugs of abuse. Accordingly, in stimulant self- administration paradigms, it has been consistently reported that female rats acquire stimulant self-administration of low doses of cocaine, methamphetamine, opioids and nicotine at higher percentages and at a faster rate than males [128]. Also, in the conditioned place preference paradigm, they require fewer trials than males, in order to exhibit place preference for cocaine [129, 130].

Thus, sex differences in conditioned taste aversion exist and they are influenced by activational effects of sex hormones [116, 118, 119]. However, taste reactivity, as well as rewarding and aversive responses are also influenced by sex hormones [125]. To make the issue more complicated, exogenous sex hormone's administration also has illness-inducing properties, which can in turn influence conditioned taste aversion [121]. As a result, it is difficult to study sex differences in conditioned learning in this paradigm, although they exist. However, this paradigm is very useful because it studies a natural mechanism that allows the animal to learn to select an appropriate diet available in its environment [131].

Conclusions

Under the associative learning paradigms described in this review, females tend to acquire and retain the associations more effectively and extinguish them more slowly than males [27, 47, 84, 106]. However, under some limited conditions, males outperform females [67, 68] (Table 1). These effects seem to depend on the amount of fear that is generated and expressed during the learning experience [27]. Also, some sex differences probably do not reflect sex differences in learning, but rather sex differences in strategies. This has been suggested before for spatial learning paradigms, in which male rats usually perform better than females by using a more direct strategy [29, 36]. In most paradigms, high estrogens levels in females seem to facilitate acquisition and delay extinction, while high testosterone levels in adult males seem to disrupt certain types of learning [33, 70, 81, 83, 84, 103, 105]. Also, developmental effects of testosterone prenatally and postnatally seem to play an important role in the emergence of sex differences in many learning paradigms [32, 87, 88].

Table 1.

summarizes differences between female (F) and male (M) rats in certain conditioning paradigms. It is noted whether the difference has been shown in acquisition of the learned response or during retention and/or extinction studies (slower extinction is indicated with the symbol >). The arrows depict the effect of ovariectomy of adult females or castration of adult males during acquisition or extinction of the learned responses (slower extinction is indicated with the arrow pointing down). Wherever data are available, the effects of hormones during development of the brain are also summarized. Please see text for different variations of tasks, for controversial findings between studies and for relevant references.

Task	Sex difference	Ovariectomy	Castration	Organizational effects
Eyeblink conditioning	F > M acquisition - retention	Acquisition	No effect	Acquisition in masculinized females
Contextual fear conditioning	M > F acquisition - retention	Extinction	No effect	-
Fear potentiated startle	F > M acquisition	Opposite effects of estrogens and progesterone	No effect	-
Light enhanced startle	F > M acquisition	No effect	Acquisition	-
Active avoidance	F > M acquisition - extinction	No effect Estrogens effects	Acquisition	Acquisition in masculinized females treated with testosterone in adulthood
Differential reinforcement of low rate responding	F > M acquisition	Acquisition	-	No effect
Conditioned taste aversion	M > F extinction	No effect Estrogens effects	Extinction	-

As it is discussed above, performance effects and non-associative factors, such as higher activity levels, higher acoustic startle reflex, higher response to shock, higher reinforcement and lower freezing behavior in female rats, interfere with the study of sex differences in most paradigms. It is recommended to measure relevant behaviors when males and females are tested in these paradigms, in order to control for performance issues. However, sex differences in performance cannot fully explain sex differences in learning and therefore we conclude that sex differences in associative learning exist. We recommend that in the study of the effects of a manipulation, such as stress, rearing conditions, pharmacological innervations or hormonal changes, we should test male and female subjects in a variety of learning paradigms. In most of our studies, we have opted to use the classical conditioning because performance issues are more easily evaluated than in other procedures. We have chosen the eyeblink as the response because it is unambiguous.

Sex differences in learning are influenced by environmental and social factors in humans but also by genes, hormones and sex differences in the structure of the brain [6, 9, 11, 132, 133]. It has been suggested that sex differences in neuronal structure allow the male and female brains to respond differently from each other at different times in their lives. However, experience and learning also exert profound effects in the brain. It seems that sex differences in learning are not only mediated by sex differences in brain anatomy, but they are also able to induce sex differences in that anatomy. In the end, sex differences in learning and their effects on the brain must be taken seriously if we are to elucidate the mechanisms of the major psychiatric and neurological disorders, as well as the mechanisms of change based on processes of learning.