Abstract
Pavlovian conditioning is a ubiquitous form of associative learning that enables animals to remember appetitive and aversive experiences. Animals possess appetitive and aversive conditioning systems that memorize and retrieve appetitive and aversive experiences. Here, we addressed a question of whether integration of competing appetitive and aversive information takes place during the encoding of the experience or during memory retrieval. We developed novel experimental procedures to address this question using crickets (Gryllus bimaculatus), which allowed selective blockade of the expression of appetitive and aversive memories by injecting octopamine and dopamine receptor antagonists. We conditioned an odour (conditioned stimulus 1, CS1) with water and then with sodium chloride solution. At 24 h after conditioning, crickets retained both appetitive and aversive memories, and the memories were integrated to produce a conditioned response (CR). Importantly, when a visual pattern (CS2) was conditioned with CS1, appetitive and aversive memories formed simultaneously. This indicates that appetitive and aversive second-order conditionings are achieved at the same time. The memories were integrated for producing a conditioned response. We conclude that appetitive and aversive conditioning systems operate independently to form parallel appetitive and aversive memories, which compete to produce learned behaviour in crickets.
Keywords: Pavlovian conditioning, conflicting memories, second-order conditioning, cricket, octopamine, dopamine
1. Introduction
Pavlovian conditioning is a basic form of associative learning that allows animals to remember appetitive and aversive experiences for achieving behavioural adaptation to a changing environment. Many animals possess appetitive and aversive conditioning systems, in which different types of dopamine (DA) neurons or other aminergic neurons convey rewarding and aversive reinforcement signals [1–7]. In the natural environment, animals are exposed to stimuli that predict appetitive and aversive consequences at the same time, and animals must integrate the conflicting information for producing appropriate behaviour. However, it has not yet been fully clarified whether integration of competitive appetitive and aversive information takes place during the encoding of the experience or during memory retrieval. More specifically, for the memory encoding process, it remains unclear whether appetitive and aversive conditioning systems operate independently and form conflicting appetitive and aversive memories when a conditioned stimulus (CS) is paired with an appetitive or aversive stimulus (unconditioned stimulus, US), either sequentially or simultaneously, or alternatively, whether they interact to produce a unified memory.
Some physiological studies in primates provided evidence supporting the interaction of appetitive and aversive conditioning systems. It was shown that activities of neurons in some brain regions are modulated by both appetite and aversive conditioning, suggesting that effects of appetitive and aversive learning converge to single neurons [8–11]. This is in accordance with the hypothesis that the effects of appetitive and aversive learning converge to form unified memories.
On the other hand, some studies in arthropods showed that animals form appetitive and aversive memories in parallel and independently when a CS is presented with a mixture of specific appetitive and aversive stimuli [12,13]. A study of Pavlovian learning in the fruit fly Drosophila melanogaster demonstrated successful formation of parallel short-lived aversive memory and longer lived appetitive memory when an odour was paired with a mixture of DEET (diethyltoluamide, an insect repellent with a bitter taste) and sucrose [12]. It was also shown that separate classes of DA neurons mediate formation of appetitive and aversive memories by modification of synaptic connections of output neurons of the mushroom body that control the appetitive or aversive response to the CS in fruit flies [14]. A study of larval fruit flies also showed that a mixture of 20 amino acids (AAs) has both a rewarding effect and an aversive effect, i.e. conditioning of an odour with the AA mixture established appetitive and aversive memories at the same time [13].
Appetitive and aversive memories formed in those studies were expressed at different time courses after the training [12] or in different testing situations [13] and, hence, one memory or both memories were expressed without being integrated to the other memory in some cases. For example, the memories competed to produce an appetitive or aversive response to the CS immediately after training, but longer lived sugar memory determined the conditioned response (CR) thereafter [12]. A remaining question is whether conflicting appetitive and aversive memories can be produced at the same time if they always compete during retrieval to produce a given level of CR. Addressing this question is not trivial since it can be argued that simultaneous formation of conflicting memories is redundant if the memories always compete to produce a fixed level of CR and, instead, formation of a single unified memory might be more economical.
In this study, we established new procedures to address this question in the cricket Gryllus bimaculatus. At first, we paired an odour with water and then with sodium chloride solution, or vice versa, to associate an odour (CS) with both an appetitive US and an aversive US. This procedure is called counterconditioning [15,16]. Bouton [15] performed pairing of a sound CS with food and then with electric shock in rats. The rats exhibited an aversive response to the CS (freezing) but not an appetitive response (head jerk) when tested soon after the aversive conditioning. On the other hand, the rats exhibited both appetitive and aversive responses when tested 28 days after conditioning, indicating retention of both appetitive and aversive memories. We investigated whether similar retention of appetitive and aversive memories occurs in crickets. Next, we investigated whether simultaneous formation of appetitive and aversive memories occurs by pairing of a visual pattern (CS2) with an odour (CS1) that had been paired with appetitive and aversive stimuli. With this pairing, we tested whether appetitive second-order conditioning (SOC) and aversive SOC can be achieved at the same time.
An advantage of using the cricket G. bimaculatus is that it is feasible to selectively impair formation and retrieval of appetitive and aversive memories in this species by injection of epinastine (an octopamine (OA) receptor antagonist [17]) or flupentixol (a DA receptor antagonist [18]) into the haemolymph [6,7,19–21]. For example, injection of epinastine impaired retrieval of appetitive memory but had no effect on retrieval of aversive memory, whereas injection of flupentixol impaired retrieval of aversive memory but had no effect on retrieval of appetitive memory [19–21]. Accordingly, we showed that OA1 (a type of OA receptor) and Dop1 (a type of DA receptor) play roles in appetitive and aversive conditioning, respectively, by gene knockout with the Clustered regularly interspaced palindromic repeats (CRISPR)/Cas9 system [22] and gene-silencing through RNA interference (RNAi) [23]. We found no evidence suggesting that DA neurons participate in appetitive conditioning in crickets, in contrast to the findings that different classes of DA neurons play roles in appetitive and aversive conditioning in fruit flies [1,24]. To account for these findings, we proposed a model (electronic supplementary material, figure S1a) in which different classes of OA neurons participate in the formation and retrieval of appetitive memories and in which different classes of DA neurons participate in the formation and retrieval of aversive memories [25,26]. We used the model to help interpretation of the results of pharmacological studies using OA and DA receptor antagonists.
To complement the pharmacological analysis, we investigated the effect of US devaluation on execution of a CR (or on memory retrieval). We showed that crickets that were given water until satiation after conditioning of an odour CS with water US exhibited no response to the CS in the test [26]. In contrast, crickets that received conditioning of an odour CS with sucrose and then given water until satiation exhibited a normal CR [26]. We thus concluded that the CR is governed by the current value of the US as has been demonstrated in some forms of Pavlovian conditioning in mammals [27]. In our model, we assume that water satiation suppresses a class of OA neurons whose activations are needed for production of an appetitive CR (or expression of appetitive memory), and hence water-satiated crickets exhibit no appetitive CR (electronic supplementary material, figure S1a) [26,28]. By integrating the results of various experiments, we conclude that appetitive and aversive conditioning systems operate independently and produce conflicting memories that compete to produce a CR.
2. Material and methods
(a). Insects
A wild-type strain of crickets (G. bimaculatus) has been inbred for several decades in our laboratory (Hokudai WT strain). The crickets were reared in 12 h light/dark cycles at 29°C ± 2°C and were fed a diet of insect pellets and water ad libitum. Three days after the imaginal moult, adult male crickets were individually isolated in 200 ml or 100 ml glass beakers: they were isolated in 200 ml beakers in the case of SOC experiments and in 100 ml beakers for all other experiments. They were given insect pellets ad libitum but were deprived of water for 3 days to enhance their motivation to search for water.
(b). Olfactory conditioning
We used classical conditioning and operant testing procedures (figure 1a ) described previously [6,7,29]. A syringe containing water and a syringe containing 2 M sodium chloride solution were used for appetitive conditioning and aversive conditioning, respectively. A filter paper soaked with apple or banana essence was attached to the needle of the syringe. The filter paper was placed above the cricket’s head for 3 s to present an odour (CS), and then water or 20% sodium chloride solution (US) was presented for 2 s to the mouth for appetitive or aversive conditioning, respectively. The appetitive trial was repeated six times (in experiments for which the results are shown in figure 4a and b) or four times (in all other experiments) with an intertrial interval (ITI) of 5 min. The aversive trial was repeated five times with an ITI of 5 min. After each trial, the air in the beaker was ventilated.
Figure 1.

Experimental procedure. (a) Odour conditioning procedure. Crickets were placed individually in a beaker. A syringe containing water or sodium chloride solution was used for conditioning. A filter paper soaked with an odour essence was attached to the needle of the syringe. For conditioning, the filter paper was placed within 1 cm of the cricket’s head to present a particular odour, and water or sodium chloride was then presented to the mouth. (b) Apparatus used for testing the preference of odours. (c) Procedure for pairing a visual pattern with an odour. (d) Apparatus used for testing the preference of visual patterns.
(c). Odour preference test
All groups of crickets were subjected to odour preference tests before and after conditioning. The procedure for the preference test was described previously [6]. The floor of the test apparatus contained two holes that connected the chamber with two odour sources (figure 1b ). Each odour source consisted of a plastic container containing filter papers soaked with essences of apple or banana, covered with a fine gauze net. Three containers were mounted on a rotative holder, and two of the three odour sources could be located just below the holes of the test chamber. Before the odour preference test, a cricket was transferred to the waiting chamber and left for about 5 min to become accustomed to the surroundings. Then the cricket was allowed to enter the test chamber, and the test started. Two minutes later, the relative positions of the odour sources were changed by rotating the container holder. The preference test lasted for 4 min. The time that the cricket probed the top net with its mouth or palpi was measured cumulatively.
(d). Conditioning of a visual pattern (CS2) with odour (CS1) that was paired with water and sodium chloride solution
A visual pattern (CS2) was paired with an odour (CS1) after pairing the CS1 with water and then with sodium chloride solution or vice versa. This was an attempt to achieve a form of SOC, and the procedure was modified from an SOC procedure described previously [25]. At first, an odour (CS1) was paired with an appetitive US (water), and the same odour (CS1) was paired with an aversive US (sodium chloride solution) 1 h later. At 24 h after conditioning, a white-centre visual pattern CS2 and odour CS1 were presented simultaneously for 5 s (figure 1c ). Seven pairing trials were performed with an ITI of 5 min.
(e). Pattern preference test
All groups of crickets used for the CS2–CS1 pairing (or SOC training) were subjected to a visual pattern preference test before and after conditioning. The procedure for the pattern preference test was described previously [7]. Two white-centre patterns (paired pattern) and one black-centre pattern (control pattern) were attached on a grey sliding wall at the end of the test chamber, and two of the three patterns could be presented at the same time (figure 1d ). A cricket was transferred to the waiting chamber and left for 5 min. Then the cricket was allowed to enter the test chamber, and the test started. Two minutes later, the relative positions of the white-centre pattern and black-centre pattern were changed by sliding the wall. The test lasted for 4 min. The time that the cricket probed the patterns with its mouth or palpi was measured cumulatively.
(f). Water satiation procedure
The procedure to provide water until satiation for devaluating water reward was described previously [26]. One day after completion of conditioning, a syringe was used to give drops of water to the mouths of the crickets until they stopped consuming. Typically, crickets consumed ca 0.45 ml of water. At 1 h after providing water until satiation, crickets were subjected to the odour or pattern preference test.
(g). Drug injection
By using a 10 μl syringe, 3 μl of physiological saline [30] or saline containing 1 μM of epinastine (OA receptor antagonist) or 100 μM of flupentixol (DA receptor antagonist) was injected into the head haemolymph. Thirty minutes later, odour or pattern preference was tested.
(h). Quantification and statistical analysis
In the odour or pattern preference test, we measured the time that the cricket visited odour sources or patterns. If the total visiting time was less than 10 s we considered that the cricket was less motivated to visit odours or patterns, and the data were rejected. The crickets for which data were rejected were ≤2% of all crickets used. The relative preference of each animal was evaluated by using the preference index (PI) for the conditioned odour or pattern, defined as T c/ (T c + T n), where T c was the time spent exploring the odour or pattern used in conditioning and T n was the time spent exploring the odour or pattern not used in conditioning. Wilcoxon’s test (WCX test) was used to compare preferences before and after training. In the case of multiple comparisons, p-values were adjusted by Holm’s method. For statistical analysis, we used GraphPad Prism (v. 9.5.0).
3. Results
(a). Formation and expression of parallel memories after sequential appetitive and aversive conditioning of a CS
At first, we studied memory expression at 24 h after sequential conditioning of a CS with appetitive and aversive stimuli. Two groups of crickets were subjected to pairings of either apple or banana odour with water (appetitive conditioning) and then subjected 4 h later to pairings of the same odour with sodium chloride solution (aversive conditioning) (figure 2a,b ). This procedure is called appetitive-to-aversive counterconditioning [14]. Then one group was injected with 10 μl saline containing 1 μM epinastine (figure 2a ) and the other group was injected with 10 μl saline containing 100 μM flupentixol (figure 2b ). The choice of drugs and the doses of drugs were based on our previous studies [6,7,19,25]. Relative preference for the conditioned odour compared with the control odour was tested before training (Pre-test), 30 min after appetitive conditioning (Post-test 1), 30 min after aversive conditioning (Post-test 2), 24 h after aversive conditioning (Post-test 3) and 30 min after epinastine injection (Post-test 4).
Figure 2.
Retention of appetitive and aversive memories after conditioning of an odour with water and then with sodium chloride solution as evaluated by injection of epinastine (OA receptor antagonist) or flupentixol (DA receptor antagonist). The time schedules of the experiments are shown above the graphs. (a,b) Two groups of crickets received appetitive and aversive conditioning (Ape and Ave training) for the same odour (CS). Odour preference was tested before (Pre-test), 30 min after appetitive conditioning (Post-test 1) and 30 min and 24 h after aversive conditioning (Post-tests 2 and 3). Each test took 85–90 min. Crickets in each group were then injected with epinastine (a) or flupentixol (b) and received the final test 30 min later (Post-test 4). (c,d) Two groups of crickets received appetitive and aversive conditioning. Odour preference was tested before (Pre-test), 30 min after appetitive conditioning (Post-test 1) and 30 min after aversive conditioning (Post-test 2). Then the crickets in each group were injected with epinastine (c) or flupentixol (d) and received the final test 30 min later (Post-test 3). Relative preferences for the conditioned odour compared with a control odour, measured as the preference index (PI), are shown as box plots. The horizontal line in the box is the median and the box represents the 25−75 percentiles in this and in all following figures. Whiskers extend to extreme values as long as they are within a range of 1.5 x box length. The outliers are shown as dots. The results of statistical comparisons are shown as asterisks, with p-values adjusted by Holm’s method for comparison of multiple groups (WCX test, ***p < 0.001; n.s. p > 0.05, n = 45 in a and b; n = 20 in c and d).
At 30 min after appetitive conditioning, crickets in both groups exhibited significantly increased preference for the CS odour compared with that before conditioning (figure 2a,b : Pre-test versus Post-test 1, detailed statistical results are shown in electronic supplementary material, table S1). At 30 min after the aversive conditioning of the same odour, crickets in both groups exhibited significantly reduced preference for the CS compared with that before conditioning (figure 2a,b : Pre-test versus Post-test 2). When tested at 24 h after aversive conditioning, however, the preference for the CS did not significantly differ from that before conditioning (figure 2a,b , Pre-test versus Post-test 3).
A possible explanation for no response to the CS at 24 h after aversive conditioning is that crickets retained no memory. Alternatively, crickets may have retained both appetitive and aversive memories and these memories competed with each other and were cancelled out, resulting in no response to the CS. To discriminate these possibilities, we tested the effect of injection of epinastine or flupentixol into the head haemolymph of crickets 24 h after completion of counterconditioning, and preferences for the CS before injection and 30 min after injection were compared. Crickets injected with epinastine or flupentixol exhibited significantly reduced (figure 2a : Post-test 3 versus 4) or increased (figure 2b : Post-test 3 versus 4) preference for the CS compared with that before drug injection. This supports the latter possibility since epinastine and flupentixol inhibit the expression of appetitive and aversive memories, respectively, but not the other way around [6,7,19,25]. These results indicate that crickets retain both appetitive and aversive memories 24 h after completion of counterconditioning. We suggest that presentation of the CS in the test activated both appetitive and aversive memories, and they were cancelled out for production of a CR and hence there was no response to the CS.
We performed multiple odour preference tests in this experiment, and we checked if this produced a reduction of memory retention. Appetitive memory after flupentixol injection (figure 2b , Post-test 4) did not significantly differ from that after appetitive conditioning (Post-test 1), indicating that three-time repetition of odour preference tests did not significantly reduce appetitive memory.
In experiments for which results are shown in figure 2a,b , either apple or banana odour was used as the CS, and the other odour was used as the control odour. Since the conditioning scores of the apple-CS subgroup and banana-CS subgroup did not significantly differ as we have reported [24], data from the two subgroups were pooled. In subsequent experiments, the apple odour was used as the CS and banana odour was used as the control odour.
We also performed conditioning of a CS with sodium chloride solution and then with water (aversive-to-appetitive counterconditioning) and then injected the crickets with epinastine or flupentixol (electronic supplementary material, figure S2a and b). The results reproduced the findings in experiments shown in figure 2a,b , namely, crickets retained both appetitive and aversive memories 24 h after conditioning, whereas they exhibited suppression of aversive memory expression 30 min after appetitive conditioning.
(b). Expression of previously formed appetitive memory is transiently inhibited after subsequent aversive conditioning of the same CS
Next, we performed appetitive-to-aversive counterconditioning and tested the effect of suppression of the expression of appetitive or aversive memory after aversive conditioning. Two groups of crickets received appetitive and aversive conditioning and were then injected with epinastine or flupentixol (figure 2c,d ). Crickets in both groups exhibited significantly higher preference for the CS after the appetitive conditioning (figures 2c,d , Pre-test versus Post-test 1) and significantly lower preference after subsequent aversive conditioning compared with that before conditioning (figures 2c,d , Pre-test versus Post-test 2). Crickets injected with epinastine after aversive conditioning exhibited a slight but significant decrease of preference for the CS compared with that before injection (figure 2c : Post-test 2 versus 3). This indicates that appetitive memory contributed to the response to the CS. Crickets injected with flupentixol after aversive conditioning exhibited significantly increased preference for the CS compared with that before injection (figure 2d : Post-test 2 versus 3).
The preference for the CS after flupentixol injection did not significantly differ from that 30 min after appetitive conditioning (figure 2d : Post-test 1 versus 3). This indicates that appetitive memory formed by the appetitive conditioning is retained without reduction after subsequent aversive training. On the other hand, aversive memory dominated the response to the CS in the test 30 min after aversive conditioning (Post-test 2), indicating that expression of previously formed appetitive memory is inhibited by expression of newly formed aversive memory. The inhibition of expression of previously formed conflicting memory is transient since appetitive and aversive memories competed to produce no response to the CS in the test 24 h after conditioning (Post-test 3 in figures 2a,b ).
(c). Simultaneous formation of appetitive and aversive memories after pairing a stimulus (CS2) with a counterconditioned stimulus (CS1)
We next performed pairing of a visual pattern (CS2) with an odour (CS1) that was paired with water and then with sodium chloride solution and investigated if this pairing leads to acquisition of appetitive and aversive memories. This pairing was an attempt to achieve appetitive SOC and aversive SOC at the same time. In one experiment, two groups of crickets were subjected to SOC training followed by testing of relative preference for the conditioned pattern and control pattern, and then the crickets were injected with saline or epinastine and tested again 30 min later (figure 3a ). In another experiment, another two groups of crickets were subjected to the same training and testing and were injected with saline or flupentixol (figure 3b ).
Figure 3.
Formation of appetitive and aversive memories by pairing a visual pattern (CS2) with an odour (CS1) that was paired with water and then with sodium chloride solution, evaluated by injection of epinastine (a) or flupentixol (b) The time schedules of the experiments are shown above the graphs. Four groups of crickets received pairing of an odour (CS1) with water and then sodium chloride solution, and they received pairing of a pattern (CS2) with the odour (CS1) 24 h later. Then the crickets in each group were injected with either saline (a,b) or saline containing epinastine (a) or flupentixol (b) Pattern preference was tested before training (Pre-test), 1 h after CS2–CS1 pairing (Post-test 1) and 30 min after injection (Post-test 2). Relative preferences for the conditioned pattern compared with a control pattern are shown as box plots. The results of statistical comparison are shown as asterisks, with the p-values adjusted by Holm’s method for multiple comparisons (WCX test, ***p < 0.001; n.s. p > 0.05, n = 23 in a and b).
The preference for CS2 1 h after SOC training did not significantly differ from that before training (figure 3a,b : Pre-test versus Post-test 1). Crickets injected with epinastine (figure 3a ) and crickets injected with flupentixol (figure 3b ) exhibited significantly lower preference and significantly higher preference for the CS2, respectively, than that before injection (Post-test 1 versus 2) or before training (Pre-test versus Post-test 2). In contrast, saline-injected crickets exhibited no significantly different preference for the CS compared with that before injection (Post-test 1 versus 2) or before training (Pre-test versus Post-test 2).
These results indicate that appetitive and aversive memories were formed simultaneously for CS2 by pairing CS2 with CS1. In other words, appetitive and aversive SOCs are achieved at the same time by this pairing. Following our model of SOC (electronic supplementary material, figure S1b), presentation of CS1 activated a class of OA and DA neurons (OA2 and DA2 neurons), which are assumed to mediate appetitive and aversive memories [28], and activation of these neurons led to the formation of appetitive and aversive memories, by changing the efficacy of synaptic transmission to neurons that produce appetitive and aversive responses to CS2 (see legend of electronic supplementary material, figure S1b) [25,26].
(d). Effect of devaluation of appetitive US on expression of memories
We previously reported suppression of the expression of appetitive memory by devaluation of the appetitive US (water) after conditioning of a CS with water [26]. Next, we investigated the effect of devaluation of water after conditioning of a CS with water and sodium chloride solution, to see if the results of pharmacological dissection of appetitive memory by epinastine injection can be reproduced. Four groups of crickets received appetitive-to-aversive counterconditioning, and 24 h later they were given water until satiation before testing odour preference. Then two groups were each injected with saline or epinastine (figure 4a ) and the other two groups were each injected with saline or flupentixol (figure 4b ). In all groups, water-satiated crickets exhibited significantly reduced preference for the CS compared with that before conditioning (figure 4a,b , Pre-test versus Post-test 1). Groups injected with flupentixol exhibited significantly higher preference for the CS, whereas crickets injected with saline or epinastine exhibited no significantly different preference for the CS compared with that before injection (figure 4a,b , Post-test 1 versus 2). The results indicate that expression of appetitive memory was inhibited in water-satiated crickets and hence aversive memory governed the response to CS.
Figure 4.
Reduction of the expression of appetitive memory by US (water) devaluation after pairing of an odour (CS1) with water and then with sodium chloride solution (a,b) and after pairing of a visual pattern (CS2) with the odour (CS1) (c), evaluated by injection of epinastine or flupentixol. The time schedules of the experiments are shown above the graphs. (a,b) Crickets in four groups received an odour preference test 2 h before training (Pre-test), 30 min after providing water until satiation (Post-test 1) and 30 min after injection of saline (a,b) or saline containing epinastine (a) or flupentixol (b) (Post-test 2). (c) Crickets in another two groups received a pattern preference test before training (Pre-test), 1 h after pairing of a pattern (CS2) with the odour (CS1) paired with water and sodium chloride solution (Post-test 1), 1 h after providing water until satiation (Post-test 2) and 30 min after injection of saline or flupentixol (Post-test 3). Relative preferences for the conditioned odour or pattern are shown as box plots. The results of statistical comparison are shown as asterisks, with the p-values adjusted by Holm’s method for multiple comparisons (WCX test, ***p < 0.001; **p < 0.01; n.s. p > 0.05, n = 20 in a, b and c).
In the final experiment, we investigated the effect of devaluation of appetitive US (water) after pairings for achieving appetitive and aversive SOCs. Two groups of crickets received a pairing of CS2 with CS1 that was paired with water and sodium chloride solution and then the preference for CS2 was tested before and after providing water until satiation. Crickets exhibited no significantly different preference for CS2 after the SOC training compared with that before training (figure 4c , Pre-test versus Post-test 1), in accordance with results shown in figure 3a,b (Pre-test versus Post-test 1). Water-satiated crickets exhibited significantly reduced preference for CS2 compared with that before providing water (figure 4c , Post-test 1 versus 2). Then two groups of crickets were each injected with flupentixol or saline. Crickets injected with saline exhibited no significant difference in the preference for CS2 compared with that before injection, whereas crickets injected with flupentixol exhibited significantly increased preference for CS2 (figure 4c , Post-test 2 versus 3), the level of which did not significantly differ from that before providing water (Post-test 1 versus 3). The results indicate that expression of appetitive memory is inhibited in water-satiated crickets and hence aversive memory governs the response to the CS2. In short, US devaluation experiments provided results that were consistent with those obtained in pharmacological experiments.
4. Discussion
(a). Independent operations of appetitive and aversive conditioning systems
In this study, we addressed the question of how appetitive and aversive Pavlovian conditioning systems operate to form appetitive and aversive memories, either sequentially or simultaneously, in crickets. At first, we found that when an odour (CS1) was sequentially paired with water and then sodium chloride solution and tested at 24 h after conditioning, both appetitive and aversive memories were retained and the response to CS1 was determined by competition of appetitive and aversive memories. A similar finding has been reported in fruit flies, in which when flies received aversive conditioning trials and then extinction trials that omitted punishment, they formed aversive and appetitive memories in parallel, and the memories were integrated in mushroom body output neurons that direct avoidance behaviour [31]. Next, we showed that when crickets were subjected to pairings of a visual stimulus (CS2) with the odour (CS1) following an SOC procedure, they formed appetitive and aversive memories that compete to determine the response to CS2. We conclude that appetitive and aversive conditioning systems operate independently and produce conflicting memories, either sequentially or simultaneously, and the memories compete with each other to produce a CR in crickets.
In this study, competitive appetitive and aversive memories were produced when crickets received a 5 s presentation of CS2 and CS1 seven times. We suggest that each 5 s presentation of CS2 and CS1 activated cellular and biochemical processes to produce appetitive and aversive memories, and the memories are consolidated into a long-lasting form by repetition of paired presentations. We thus consider that the conflicting memories are formed ‘simultaneously’ by pairings of CS2 and CS1.
Previous studies in fruit flies demonstrated the simultaneous formation of appetitive and aversive memories by conditioning of a CS with a mixture of appetitive and aversive taste stimuli [12,13]. This finding does not rule out the possibility that inhibitory interactions occur between appetitive and aversive conditioning systems during conditioning in these studies, but such inhibitory interactions, if they exist, are not strong enough to fully prevent the parallel formation of appetitive and aversive memories. In those studies in fruit flies, appetitive and aversive memories were expressed at different time courses [12] or in different testing situations [13], and hence one or both memories produced learned behaviour without integration to the other memory in some situations (see §1). In crickets, in contrast, appetitive and aversive memories competed to produce learned behaviour in any testing situations. We deduce that a possible reason for the difference of the nature of the memories is the difference of the taste stimuli used, i.e. DEET (toxin) and sucrose or a mixture of 20 amino acids in fruit flies and sodium chloride solution and water in crickets, but this remains to be studied. To the best of our knowledge, the present study is the first study to demonstrate a case of simultaneous formation of conflicting memories by Pavlovian conditioning of any animals, in which the memories compete to produce a given level of learned behaviour. In this case, the level of learned behaviour is unchanged as long as the motivation is unchanged as is discussed later.
Apart from the studies on Pavlovian conditioning, formation of appetitive and aversive memories of the training context by sequential presentation of appetitive and aversive stimuli in the same training context has been reported in crabs [32]. The memories compete to produce an appetitive or aversive response during presentation of the training context. In fruit flies, conditioning of an odour with ethanol produces both appetitive memory and aversive memory [33], indicating that ethanol serves as aversive and appetitive USs at the same time. It is likely that formation of competitive appetitive and aversive memories is widespread among various forms of learning in arthropods.
We demonstrated the simultaneous formation of appetitive and aversive memories using an SOC procedure, not by presenting a mixture of appetitive and aversive taste stimuli. An advantage of the use of an SOC procedure is that it allowed the use of water and sodium chloride solution as appetitive and aversive stimuli. Water and sodium chloride solution are established as effective USs for achieving appetitive and aversive Pavlovian conditioning [6,7,34,35] and SOC [25] in crickets, and hence the results of this study are easily integrated to findings in our previous studies. In the future, however, it would be interesting to investigate the effect of conditioning an odour with a mixture of appetitive and aversive stimuli, such as sucrose and DEET, to test if simultaneous formation of conflicting memories is achieved as in the case of fruit flies. Such experiments will help to clarify whether independent operations of appetitive and aversive conditioning systems are ubiquitous features of Pavlovian conditioning in crickets.
(b). OA and DA systems compete for expression of appetitive and aversive memories
We concluded that integration of appetitive and aversive information takes place not during the memory formation process but during the memory retrieval process in crickets. To account for this finding, we revised our model of Pavlovian conditioning (electronic supplementary material, figure S1a), which we proposed to account for the roles of OA and DA neurons in the formation and retrieval of appetitive and aversive memories [19,20,26,28,35]. Our model shown in electronic supplementary material, figure S1a assumes that paired presentation of a CS and an appetitive US (or aversive US) strengthens synaptic connections from ‘CS’ neurons to two classes of OA neurons (or DA neurons), one class participating in memory acquisition, ‘OA1’ neurons and the other class participating in memory retrieval, ‘OA2’ neurons (or ‘DA1’ and ‘DA2’ neurons). For memory formation, the simultaneous activation of ‘CS’ neurons and ‘OA1’ neurons (or ‘DA1’ neurons) strengthens synaptic connections from ‘CS’ neurons to ‘CRap’ neurons (or ‘CRav’ neurons) that produce an appetitive (or aversive) CR. For memory retrieval after conditioning, presentation of the CS activates ‘CS’ neurons and ‘OA2’ neurons (or ‘DA2’ neurons), and their simultaneous activation activates ‘CRap’ neurons (or ‘CRav’ neurons) and then produces an appetitive (or aversive) CR (for details, see legends of electronic supplementary material, figure S1).
In the model to account for parallel formation and retrieval of appetitive and aversive memories (electronic supplementary material, figure S3), we combined the circuit models of appetitive and aversive conditionings in electronic supplementary material, figure S1a and added a mutual inhibition between ‘CRap’ neurons and ‘CRav’ neurons. Presentation of a CS after appetitive and aversive conditioning activates ‘CS’ neurons, ‘OA2’ neurons and ‘DA2’ neurons. This activates ‘CRap’ neurons and ‘CRav’ neurons that compete to determine the CR (for details, see legends of electronic supplementary material, figure S3).
(c). Simultaneous formation of appetitive and aversive memories by an SOC procedure
We showed that pairing of a stimulus (CS2) with another stimulus (CS1) that was conditioned with appetitive and aversive stimuli results in the formation of both appetitive and aversive memories for CS2 at the same time (figure 3), suggesting the simultaneous occurrence of appetitive and aversive SOCs. To our knowledge, this study is the first study to demonstrate simultaneous occurrence of appetitive and aversive SOCs, which we refer to as dual SOC. We account for the dual SOC as follows. According to our models of SOC (electronic supplementary material, figure S1b) [25,36], presentation of CS1 after pairing the CS1 with appetitive US and aversive US activates ‘OA2’ and ‘DA2’ neurons. Thus, during pairing of CS2 with CS1 in the second stage of dual SOC training, ‘CS2’ neurons, ‘OA2’ neurons and ‘DA2’ neurons are activated simultaneously, and this leads to changes in the strength of synaptic connections from ‘CS2’ neurons to ‘CRAP’ neurons and ‘CRAV’ neurons that produce appetitive CR and aversive CR (see legend of electronic supplementary material, figure S1b). Hence, dual SOC is achieved.
(d). Reward devaluation suggests functional significance of formation of conflicting memories
We showed that crickets that received training to associate a CS with water and then with sodium chloride solution retained appetitive and aversive memories, and the crickets exhibited no response to the CS since the memories compete with each other during retrieval. On the other hand, when crickets were given water until satiation before the test, expression of appetitive memory was suppressed and hence crickets exhibited an aversive response to the CS. Reduction of response to a CS by devaluating the appetitive US has been reported in Pavlovian learning in many animals including mammals [27], insects [26,37–39] and crabs [32]. In food learning in fruit flies, a subset of DA neurons that suppress appetitive CR is inhibited when the flies are starved. Hence, fruit flies exhibit appetitive CR when they are starved but not when they are satiated, thereby achieving control of the CR in accordance with the motivational state [40].
The effect of US devaluation to alter the response to a CS provides a hint for understanding the functional roles of formation of appetitive and aversive memories that always conflict for production of a CR. If competition of memories always produces a consistent response to a given CS, it would be more economical to form a single unified memory by integrating appetitive and aversive experiences. However, since the response to the CS alters in accordance with the current value of the US, retention of conflicting memories allows flexible changes of learned behaviour following the changes of the motivation to require (or avoid) the USs.
(e). Two-dimensional valuation of stimuli by Pavlovian conditioning of animals
We conclude that appetitive conditioning and aversive conditioning take place in parallel and independently in crickets. Notably, this conclusion is analogous to that in some studies of DA neurons participating in Pavlovian conditioning in primates [41–44]. Fiorillo [42] argued that appetitiveness and aversiveness are separately coded in different classes of DA neurons in the midbrain of primates, and hence appetitiveness and aversiveness should represent two distinct dimensions for value evaluation. More studies are needed to clarify to what extent parallel and independent formation of appetitive and aversive memories is ubiquitous among Pavlovian conditioning systems in animals. Experimental procedures we developed in this study may help to facilitate such studies.
Contributor Information
Sadniman Rahman, Email: rahmansadniman@gmail.com.
Kanta Terao, Email: kanta_terao@cis.shimane-u.ac.jp.
Kohei Hashimoto, Email: khashimoto56714@gmail.com.
Makoto Mizunami, Email: mizunami@es.hokudai.ac.jp.
Ethics
This work did not require ethical approval from a human subject or animal welfare committee.
Data accessibility
Our data can be accessed in the Repository [45].
Supplementary material is available online [46].
Declaration of AI use
We have not used AI-assisted technologies in creating this article.
Authors’ contributions
S.R.: data curation, formal analysis, investigation, visualization, writing—original draft, writing—review and editing, conceptualization, methodology, validation; K.T.: supervision, validation, writing—review and editing, investigation; K.H.: investigation, writing—review and editing, supervision, validation; M.M.: conceptualization, funding acquisition, methodology, project administration, resources, software, supervision, writing—original draft, writing—review and editing, validation, investigation, visualization.
All authors gave final approval for publication and agreed to be held accountable for the work performed therein.
Conflict of interest declaration
We declare we have no competing interests.
Funding
This study was supported by Grants-in-Aid for Scientific Research from the Ministryof Education, Science, Culture, Sports and Technology of Japan (No. 21K19245 to M.M.).
References
- 1. Burke CJ, et al. 2012. Layered reward signalling through octopamine and dopamine in Drosophila. Nature 492 , 433–437. ( 10.1038/nature11614) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Hammer M, Menzel R. 1998. Multiple sites of associative odor learning as revealed by local brain microinjections of octopamine in honeybees. Learn. Mem. 5 , 146–156. [PMC free article] [PubMed] [Google Scholar]
- 3. Schultz W. 2015. Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95 , 853–951. ( 10.1152/physrev.00023.2014) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Starkweather CK, Uchida N. 2021. Dopamine signals as temporal difference errors: recent advances. Curr. Opin. Neurobiol. 67 , 95–105. ( 10.1016/j.conb.2020.08.014) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Modi MN, Shuai Y, Turner GC. 2020. The Drosophila mushroom body: from architecture to algorithm in a learning circuit. Annu. Rev. Neurosci. 43 , 465–484. ( 10.1146/annurev-neuro-080317-0621333) [DOI] [PubMed] [Google Scholar]
- 6. Unoki S, Matsumoto Y, Mizunami M. 2005. Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study. Eur. J. Neurosci. 22 , 1409–1416. ( 10.1111/j.1460-9568.2005.04318.x) [DOI] [PubMed] [Google Scholar]
- 7. Unoki S, Matsumoto Y, Mizunami M. 2006. Roles of octopaminergic and dopaminergic neurons in mediating reward and punishment signals in insect visual learning. Eur. J. Neurosci. 24 , 2031–2038. ( 10.1111/j.1460-9568.2006.05099.x) [DOI] [PubMed] [Google Scholar]
- 8. Matsumoto M, Hikosaka O. 2009. Representation of negative motivational value in the primate lateral habenula. Nat. Neurosci. 12 , 77–84. ( 10.1038/nn.2233) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Morrison SE, Salzman CD. 2009. The convergence of information about rewarding and aversive stimuli in single neurons. J. Neurosci. 29 , 11471–11483. ( 10.1523/JNEUROSCI.1815-09.2009) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Paton JJ, Belova MA, Morrison SE, Salzman CD. 2006. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439 , 865–870. ( 10.1038/nature04490) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Li J, Fan R, Liu X, Shen X, Liu X, Zhao H. 2021. The convergence of aversion and reward signals in individual neurons of the mice lateral habenula. Exp. Neurol. 339 , 113637. ( 10.1016/j.expneurol.2021.113637) [DOI] [PubMed] [Google Scholar]
- 12. Das G, Klappenbach M, Vrontou E, Perisse E, Clark CM, Burke CJ, Waddell S. 2014. Drosophila learn opposing components of a compound food stimulus. Curr. Biol. 24 , 1723–1730. ( 10.1016/j.cub.2014.05.078) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Toshima N, Weigelt MK, Weiglein A, Boetzl FA, Gerber B.. 2019. An amino-acid mixture can be both rewarding and punishing to larval Drosophila melanogaster. J. Exp. Biol. 222, jeb209486. ( 10.1242/jeb.209486) [DOI] [PubMed] [Google Scholar]
- 14. Waddell S. 2013. Reinforcement signalling in Drosophila; dopamine does it all after all. Curr. Opin. Neurobiol. 23, 324-329. ( 10.1016/j.conb.2013.01.005) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Bouton ME, Peck CA. 1992. Spontaneous recovery in cross-motivational transfer (counterconditioning). Anim. Learn. Behav. 20 , 313–321. ( 10.3758/BF03197954) [DOI] [Google Scholar]
- 16. Keller NE, Hennings AC, Dunsmoor JE. 2020. Behavioral and neural processes in counterconditioning: past and future directions. Behav. Res. Ther. 125 , 103532. ( 10.1016/j.brat.2019.103532) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Roeder T, Degen J, Gewecke M. 1998. Epinastine, a highly specific antagonist of insect neuronal octopamine receptors. Eur. J. Pharmacol. 349 , 171–177. ( 10.1016/s0014-2999(98)00192-7) [DOI] [PubMed] [Google Scholar]
- 18. Mustard JA, Blenau W, Hamilton IS, Ward VK, Ebert PR, Mercer AR. 2003. Analysis of two D1-like dopamine receptors from the honey bee Apis mellifera reveals agonist-independent activity. Mol. Brain Res. 113 , 67–77. ( 10.1016/s0169-328x(03)00091-3) [DOI] [PubMed] [Google Scholar]
- 19. Nakatani Y, Matsumoto Y, Mori Y, Hirashima D, Nishino H, Arikawa K, Mizunami M. 2009. Why the carrot is more effective than the stick: different dynamics of punishment memory and reward memory and its possible biological basis. Neurobiol. Learn. Mem. 92 , 370–380. ( 10.1016/j.nlm.2009.05.003) [DOI] [PubMed] [Google Scholar]
- 20. Terao K, Matsumoto Y, Mizunami M. 2015. Critical evidence for the prediction error theory in associative learning. Sci. Rep. 5 , 8929. ( 10.1038/srep08929) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Terao K, Mizunami M. 2017. Roles of dopamine neurons in mediating the prediction error in aversive learning in insects. Sci. Rep. 7 , 14694. ( 10.1038/s41598-017-14473-y) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Awata H, et al. 2016. Roles of OA1 octopamine receptor and dop1 dopamine receptor in mediating appetitive and aversive reinforcement revealed by RNAi studies. Sci. Rep. 6 , 29696. ( 10.1038/srep29696) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Awata H, Watanabe T, Hamanaka Y, Mito T, Noji S, Mizunami M. 2015. Knockout crickets for the study of learning and memory: dopamine receptor dop1 mediates aversive but not appetitive reinforcement in crickets. Sci. Rep. 5 , 15885. ( 10.1038/srep15885) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Liu C, et al. 2012. A subset of dopamine neurons signals reward for odour memory in Drosophila. Nature 488 , 512–516. ( 10.1038/nature11304) [DOI] [PubMed] [Google Scholar]
- 25. Mizunami M, Unoki S, Mori Y, Hirashima D, Hatano A, Matsumoto Y. 2009. Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect. BMC Biol. 7 , 46. ( 10.1186/1741-7007-7-46) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Mizunami M, Hirohata S, Sato A, Arai R, Terao K, Sato M, Matsumoto Y. 2019. Development of behavioural automaticity by extended Pavlovian training in an insect. Proc. R. Soc. B 286 , 20182132. ( 10.1098/rspb.2018.2132) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Holland PC, Lasseter H, Agarwal I. 2008. Amount of training and cue-evoked taste-reactivity responding in reinforcer devaluation. J. Exp. Psychol. 34 , 119–132. ( 10.1037/0097-7403.34.1.119) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Mizunami M. 2021. What is learned in Pavlovian conditioning in crickets? Revisiting the S-S and S-R learning theories. Front. Behav. Neurosci. 15 , 661225. ( 10.3389/fnbeh.2021.661225) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Matsumoto Y, Mizunami M. 2002. Temporal determinants of long-term retention of olfactory memory in the cricket Gryllus bimaculatus. J. Exp. Biol. 205 , 1429–1437. ( 10.1242/jeb.205.10.1429) [DOI] [PubMed] [Google Scholar]
- 30. Matsumoto Y, Noji S, Mizunami M. 2003. Time course of protein synthesis-dependent phase of olfactory memory in the cricket Gryllus bimaculatus. Zool. Sci. 20 , 409–416. ( 10.2108/zsj.20.409) [DOI] [PubMed] [Google Scholar]
- 31. Felsenberg J, et al. 2018. Integration of parallel opposing memories underlies memory extinction. Cell 175 , 709–722.( 10.1016/j.cell.2018.08.021) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Klappenbach M, Nally A, Locatelli FF. 2017. Parallel memory traces are built after an experience containing aversive and appetitive components in the crab Neohelice. Proc. Natl. Acad. Sci. USA 114 , E4666–E4675. ( 10.1073/pnas.1701927114) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Kaun KR, Azanchi R, Maung Z, Hirsh J, Heberlein U. 2011. A Drosophila model for alcohol reward. Nat. Neurosci. 14 , 612–619. ( 10.1038/nn.2805) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Mizunami M, Matsumoto Y. 2017. Roles of octopamine and dopamine neurons for mediating appetitive and aversive signals in Pavlovian conditioning in crickets. Front. Physiol. 8 , 1027. ( 10.3389/fphys.2017.01027) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Mizunami M, Terao K, Alvarez B. 2018. Application of a prediction error theory to Pavlovian conditioning in an insect. Front. Psychol. 9 , 1272. ( 10.3389/fpsyg.2018.01272) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Segi Y, Hashimoto K, Mizunami M. 2023. Octopamine neurons mediate reward signals in social learning in an insect. iScience 26 , 106612. ( 10.1016/j.isci.2023.106612) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Krashes MJ, Waddell S. 2008. Rapid consolidation to a radish and protein synthesis-dependent long-term memory after single-session appetitive olfactory conditioning in Drosophila. J. Neurosci. 28 , 3103–3113. ( 10.1523/JNEUROSCI.5333-07.2008) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Lai Y, Despouy E, Sandoz JC, Su S, de Brito Sanchez MG, Giurfa M. 2020. Degradation of an appetitive olfactory memory via devaluation of sugar reward is mediated by 5-HT signaling in the honey bee. Neurobiol. Learn. Mem. 173 , 107278. ( 10.1016/j.nlm.2020.107278) [DOI] [PubMed] [Google Scholar]
- 39. Klappenbach M, Lara AE, Locatelli FF. 2022. Honey bees can store and retrieve independent memory traces after complex experiences that combine appetitive and aversive associations. J. Exp. Biol. 225 , jeb244229. ( 10.1242/jeb.244229) [DOI] [PubMed] [Google Scholar]
- 40. Krashes MJ, DasGupta S, Vreede A, White B, Armstrong JD, Waddell S. 2009. A neural circuit mechanism integrating motivational state with memory expression in Drosophila. Cell 139 , 416–427. ( 10.1016/j.cell.2009.08.035) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Matsumoto M, Hikosaka O. 2009. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459 , 837–841. ( 10.1038/nature08028) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Fiorillo CD. 2013. Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341 , 546–549. ( 10.1126/science.1238699) [DOI] [PubMed] [Google Scholar]
- 43. Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC. 2012. Input-specific control of reward and aversion in the ventral tegmental area. Nature 491 , 212–217. ( 10.1038/nature11527) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44. Verharen JPH, Zhu Y, Lammel S. 2020. Aversion hot spots in the dopamine system. Curr. Opin. Neurobiol. 64 , 46–52. ( 10.1016/j.conb.2020.02.002) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45. Rahman S, Terao K, Hashimoto K, Mizunami M. 2024. Data for: Independent operations of appetitive and aversive conditioning systems lead to simultaneous production of conflicting memories in an insect. Dryad Digital Repository. ( 10.5061/dryad.tb2rbp098) [DOI] [PubMed]
- 46. Rahman S, Terao K, Hashimoto K, Mizunami M. 2024. Data from: Independent operations of appetitive and aversive conditioning systems lead to simultaneous production of conflicting memories in an insect. Figshare. ( 10.6084/m9.figshare.c.7456293) [DOI] [PubMed]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Our data can be accessed in the Repository [45].
Supplementary material is available online [46].



