Reduction of contextual control of conditioned responses by extended Pavlovian training in an insect

Abstract

The effect of repetitive training on learned behavior has been an important subject in neuroscience. In instrumental conditioning in mammals, learned action early in training is often goal-driven and controlled by outcome expectancy, but as training progresses, it becomes more habitual and insensitive to outcome devaluation. Similarly, we recently showed in Pavlovian conditioning in crickets (Gryllus bimaculatus) that a conditioned response (CR) is initially sensitive to devaluation of the unconditioned stimulus but becomes insensitive to it after extended training. It is known that habitual responses after extended instrumental training are characterized by a higher context specificity than are initial goal-directed actions in mammals. In this study, we investigated whether this is applicable to Pavlovian conditioning in crickets. In crickets that received a standard amount of training to associate an odor with water reward under illumination, CR under illumination was stronger than that in the dark. In crickets that received extended training under illumination, on the other hand, the level of CR did not differ in different light conditions. Further experiments confirmed that context specificity decreases with the development of behavioral automaticity by extended training, as opposed to findings in instrumental training in mammals. We conclude that the nature of habitual behaviors after extended training differs in different learning systems of animals.

In humans, habitual behavior often underlies dysfunctional behaviors such as drug addiction (Everitt et al. 2001) and obsessive-compulsive disorder (Gillan et al. 2011). It is for this reason that much research conducted on behavioral neuroscience and psychology has aimed to understand the effects of repetitive training on the performance of learned actions or, in other words, the development of habits (Gardner 2015; Wood and Rünger 2016). One of the most common findings in instrumental conditioning in mammals is that actions in an early stage of training are more goal-driven; i.e., they are controlled by the specific outcome (reinforcer) of their actions. However, as training proceeds, behaviors often become more automatic and independent of the actual value of the outcome, although both the goal-driven and habitual behavioral components often coexist in both early and later stages of training (Dickinson 1985; Yin and Knowlton 2006; Kosaki and Dickenson 2010; Smith and Graybiel 2014). Goal-directed actions and habitual behavior can be distinguished by using an outcome devaluation procedure in which the value of the outcome (reward) is reduced by aversive conditioning (Adams 1982; Dickinson 1985) or by outcome-specific satiation (Balleine and Dickinson 1998).

We have investigated associative processes underlying Pavlovian conditioning and its possible neural substrates in the cricket Gryllus bimaculatus (Unoki et al. 2005; Mizunami et al. 2009; Mizunami and Matsumoto 2017; Mizunami et al. 2018), and we recently found that the conditioned response (CR) to an odor (conditioned stimulus, CS) diminishes when the value of water reward (unconditioned stimulus [US]) is reduced after satiation early in training but that it becomes insensitive to US devaluation after extended training in crickets (Mizunami et al. 2019). To our knowledge, these results provide the first evidence of increased automaticity of learned behavior by extended Pavlovian training in any animals. Crickets that received four-trial training to associate an odor CS with water US, which we refer to as standard training, exhibited no CR after water had been given until satiation before the test. On the other hand, crickets that received extended training (i.e., four conditioning trials per day on three consecutive days) exhibited the same level of CR regardless of whether they had been provided with water until satiation or not before the test.

Notably, in instrumental conditioning in mammals, it has been documented that habitual behavior that is insensitive to outcome devaluation has features that are distinct from those of goal-directed behavior typically seen after limited training. The most notable one is its higher context specificity: Habitual responses are more likely to occur in the context in which training occurred than outside that context (in rats) (Thrailkill and Bouton 2015). Similarly, in humans, such context specificity is considered a hallmark of the habitual behavior formed by repetition of the same learned responses and it is considered a critical feature to which special attention must be paid in the therapy of mal-adaptive habits (Tricomi et al. 2009; Gardner 2015; Wood and Rünger 2016).

The aim of this study was to determine whether the same context specificity that is observed in the habitual response after extended training in mammals is applicable to Pavlovian conditioning in insects. To do so, we performed standard or extended training for crickets under illumination, and their CRs were tested under illumination or in the dark. We used light and darkness as contexts since we observed that a change of background light level between training and testing influences the level of CRs in one paradigm of Pavlovian conditioning in crickets (Matsumoto and Mizunami 2004).

Results

Effects of background light on odor preference of crickets

Crickets prefer vanilla odor over peppermint odor (Matsumoto and Mizunami 2002), and we used peppermint odor as a conditioned odor and vanilla odor as a control odor (see the Materials and Methods). We first studied whether the relative preference between peppermint and vanilla odors differs when tested under illumination and in the dark, as a necessary control experiment for subsequent studies. Two groups of naïve (untrained) crickets were subjected to an odor preference test under illumination or in the dark in a test arena (Fig. 1B; see the Materials and Methods). Odor preferences of crickets tested under illumination and those in the dark did not differ (Fig. 2A; statistical results shown in Table 1), indicating that the presence or absence of light does not alter odor preference of crickets. We also investigated whether illumination or darkness could affect a cricket's motivation to search for odors by investigating the total search time for the two odors in both contexts. We found no significant difference between the groups (Fig. 2B), suggesting that their motivation to search for odors is not affected by the presence or absence of light.

Figure 1.

Procedures used for olfactory conditioning in crickets. (A) Procedure for associating an odor (CS) with water (US). A filter paper soaked with peppermint essence was approached to the antennae of a cricket for 2 sec and then a drop of water was presented to the mouth. For paired training, the CS was paired with the US four times with 5 min intervals. For unpaired training, the CS and the US were presented four times each with an interval of 2.5 min. (B) Apparatus used for the odor preference test. A cricket was placed in a test arena on the floor of which there were two holes that connected the chamber with containers that contained a filter paper soaked with peppermint odor (conditioned odor) or vanilla odor (control odor).

Figure 2.

Relative preference and total search time for odors of two groups of naïve (untrained) crickets tested under illumination or in the dark, shown as box plots. Outliers are shown as open circles. (A) Relative preference between peppermint and vanilla odors are shown as preference index (PI) for peppermint odor. (B) Total time to visit odors. Statistically significant differences are indicated by asterisks. M–W test, (NS) P > 0.05. The sample size is shown at each graph.

Table 1.

Statistical results

Effect of background light condition on execution of the CR after standard training and extended training

Next, we investigated whether crickets that received four-trial training (i.e., standard training or four-trial × 1 d of training) under illumination exhibit preference for the conditioned odor under illumination and in the dark. Two groups of crickets were subjected to four-trial training in which peppermint odor (CS) was paired with water (US), by presenting the odor to the antennae for 2 sec and then presenting a drop of water to the mouth. This training was conducted in a beaker under illumination and with an intertrial interval (ITI) of 5 min (Fig. 1A; see the Materials and Methods). At ∼24 h after completion of training, one group was subjected to a preference test between peppermint odor (conditioned odor) and vanilla odor (control odor) under illumination (referred to as the standard training and the same context testing group or standard-same group) and the other group was tested in the dark (standard-different group). Another group was subjected to unpaired presentations of peppermint odor to the antennae and water to the mouth four times each under illumination. The ITI for the control group was 2.5 min. At ∼24 h after training, the group was subjected to odor preference test under illumination (standard unpaired training and same context testing group or standard-unpaired group). The preferences for the conditioned odor (peppermint odor) of the standard-same group and standard-different groups were significantly greater than that of the standard-unpaired group, indicating conditioning to the peppermint odor (Fig. 3A). The preference for the conditioned odor of the standard-different group was also significantly greater than that of the standard-unpaired group, although the P-value was close to the significance level (P = 0.048) (Table 1). The preference for the conditioned odor of the standard-same group was significantly greater than that of the standard-different group. Therefore, after standard training under illumination, the CR was greater in the context in which the training was performed. We did not perform training in the dark, since it was not feasible to present an odor and water to the cricket in precise timing under dim red light.

Figure 3.

Effect of change of the ambient light condition on relative preference for the conditioned odor after standard training and after extended training. (A) Relative preferences for peppermint odor shown by two groups of crickets subjected to standard training (four-trial × 1 d of training) under illumination and tested under illumination (standard-same group) or in the dark (standard-different group). Relative preference for peppermint odor shown by another group that received four unpaired presentations of the CS and US (four-trial × 1 d of unpaired training) and was tested under illumination (standard-unpaired group). (B) Relative preferences for peppermint odor shown by two groups of crickets that were subjected to extended training (four-trial × 3 d of training) under illumination and were tested under illumination or in the dark (extended-same group and extended-different group). Relative preference for peppermint odor shown by another group that received four trials × 3 d of unpaired training and was tested under illumination (extended-unpaired group). The odor preferences were determined as PIs for the conditioned odor and are shown as box plots. The top trace in each figure indicates the schedule of the training and the test. Dark or pale bar at the top indicates a day. Statistically significant differences are indicated by asterisks. M–W test, Holm's method, (NS) P > 0.05, (*) P < 0.05, (**) P < 0.01, (***) P < 0.001. The sample size is shown at each graph.

Next, we examined the effect of context switch in groups that were subjected to an extended amount of training under illumination. Two groups of crickets were subjected to four conditioning trials per day during three consecutive days (four trials × 3 d of training). At ∼24 h after training, one group's odor preference was tested under illumination (extended-same group) and the other group was tested in the dark (extended-different group). Another group received unpaired presentations of the odor and water also four times per day for three consecutive days under illumination. At ∼24 h later, the group received a test under illumination (extended-unpaired group). The preference for the conditioned odor (peppermint odor) of the extended-same group was significantly greater than that of the extended-unpaired group (Fig. 3B), indicating conditioning to the peppermint odor. The preference for the conditioned odor of the extended-different group was also significantly greater than that of the extended-unpaired group, but it was not significantly different from that of the extended-same group (Fig. 3B). Thus, crickets that received extended training under illumination exhibited the same levels of CR under illumination and in the dark. We thus conclude that the CR is less context-dependent after extended training, in contrast to what occurs after standard training, where context specificity is observed.

Is context-independent CR after extended training insensitive to US devaluation?

As already mentioned, we previously reported that crickets exhibited no CR to the conditioned odor if, after standard conditioning, they had been given water until satiation before the test and that, on the contrary, they did exhibit CR if they had been subjected to extended training and water satiation before the test. The level of conditioning to the CS in the last situation did not differ from that shown by crickets that had not been given water before the test (Mizunami et al. 2019). Since the effect of water satiation on the CR could not be accounted for by nonspecific effects of water satiation on the sensory, motivational or motor functions that are necessary for performing the CR, we concluded that the execution of the CR was governed by the actual value of the US during early training but not after extended training (Mizunami et al. 2019).

Here we performed experiments to test whether the CR is less sensitive to context switch when crickets were subjected to extended training and then water satiation before the test. Two groups of crickets received extended training under illumination. About 24 h later, these groups were given water until satiation and then tested under illumination (extended-same group) or in the dark (extended-different group). Another group received unpaired extended training under illumination, and ∼24 h later, this group received water until satiation followed by a test in the same context (extended-unpaired group). In a parallel experiment, another three groups received the same training but were not subjected to water satiation before the test (extended-same, extended-different, and extended-unpaired groups). When crickets did not receive water until satiation before the test, the preference for the conditioned odor shown by the extended-same group and that shown by the extended-different group were significantly greater than that shown by the extended-unpaired group (Fig. 4A). Importantly, the preference for the CS shown by the extended-different group did not differ from that of the extended-same group. These results replicate the findings in the previous experiment: Extended training results in high levels of conditioned response to the CS regardless of the context in which the test takes place (Fig. 3B). The same was observed when crickets were water-sated before the test was conducted (Fig. 4B). Namely, the preferences for the conditioned odor of the extended-same group and the extended-different group were significantly greater than that of the extended-unpaired group, and no differences were observed in terms of CS preference between the extended-different and extended-same groups. We thus conclude that US value-insensitive CR after extended training is indeed less context-specific.

Figure 4.

Effect of context switch on relative odor preference of crickets that received extended training and were not given water (A) or were given water until satiation (B) before being tested. (A) Relative preference for peppermint odor shown by two groups of crickets that received extended (four-trial × 3 d) training under illumination and that were tested under illumination (extended-same group) or in the dark (extended-different group). Relative preference for peppermint odor shown by another group that received both extended (four-trial × 3 d) unpaired training and testing under illumination (extended-unpaired group). None of the crickets were given water before the test was conducted. (B) Relative preference for peppermint odor shown by two groups that received extended training under illumination and were given water until satiation before the test, which was conducted under illumination (extended-same group) or in the dark (extended-different group). Relative preference for peppermint odor shown by another group that received extended unpaired training under illumination and water satiation before the test was conducted also under illumination (extended-unpaired group). The relative odor preferences are shown as PIs for the conditioned odor. The top trace in each figure indicates the schedule of the training and the test. Statistically significant differences are indicated by asterisks. M–W test, Holm's method, (NS) P > 0.05, (*) P < 0.05, (***) P < 0.001. The sample size is shown at each graph.

Is the number of conditioning trials a critical determinant for a decrease in context specificity of the CR?

In a previous study, we concluded that the number of conditioning trials does not necessarily determine whether responding to a CS is still observed after US satiation (Mizunami et al. 2019). Namely, we showed that a group of crickets that received 12 trials in one single day (12-trial × 1 d of training) followed by water satiation before the test exhibited no CRs, indicating that the CRs are governed by the actual value of the water US at the particular moment of the test. This was in contrast to the results observed in a group that received four trials of training per day during three consecutive days. We thus concluded that it is not the number of training trials but repetition of multiple trials with sufficiently long intervals that is needed to make the CRs independent of the value of the US at the moment of the test (Mizunami et al. 2019). In this study, we investigated whether the same is true for achieving decreased sensitivity to context switch with progress of training. We conducted an experiment in which two groups received 12 trials × 1 d under illumination. One group was tested under illumination (12 trials-same group) and the other group was tested in the dark (12 trials-different group). A control group received 12 unpaired presentations of peppermint odor and water under illumination (12 trials-unpaired group). The results showed that the 12 trials-same group, but not the 12 trials-different group, exhibited a significantly greater preference for the conditioned odor than that of the 12 trials-unpaired group (Fig. 5) and that the level of preference of the 12 trials-same group was significantly greater than that of the 12 trials-different group, indicating that the CRs are context-specific. The results indicate that 12 trials × 1 d of training is not sufficient to make the CR less context-specific.

Figure 5.

Effect of context switch on odor preference of groups that received 12 trials × 1 d of training. Relative preference for peppermint odor shown by two groups that received 12 trials × 1 d of training under illumination and that were tested under illumination (12 trials-same group) or in the dark (12 trials-different group). Relative preference for peppermint odor shown by another group that received both 12-trials × 1 d of unpaired training and testing under illumination (12 trials-unpaired group). The odor preference is shown as the PI for the conditioned odor. The top trace indicates the schedule of training and test. Statistically significant differences are indicated by asterisks. M–W test, Holm's method, (NS) P > 0.05, (*) P < 0.05. The sample size is shown at each graph.

Discussion

We showed that crickets that had received standard training (four trials × 1 d of training) under illumination exhibit a greater level of CR under illumination than that in the dark, whereas crickets that had received extended training (four trials × 3 d of training) under illumination exhibited the same level of CR under illumination and in the dark. The results indicate that the CR is initially context-dependent but loses context dependency after extended training (Fig. 3). We previously showed that the CR is sensitive to US devaluation after standard training but not after extended training (Mizunami et al. 2019). Here we replicated this result and confirmed that the CR that is not sensitive to US devaluation, acquired after extended training, is less context-specific (Fig. 4). Notably, our finding in Pavlovian conditioning in crickets is the opposite to findings in instrumental conditioning in mammals: Habitual behaviors that are insensitive to outcome devaluation, typically seen after extended training, are more context-specific compared with the goal-directed actions, which are typically seen early in training and are sensitive to outcome devaluation (Thrailkill and Bouton 2015). Thus, we conclude that the widespread notion that habitual behavior after repetitive training is more context-specific, which has been observed in studies on instrumental learning in mammals including humans (Gardner 2015; Wood and Rünger 2016), is not applicable to Pavlovian conditioning in crickets.

We were not able to perform training in the dark condition, since presenting an odor and water in precise timing under dim red light was difficult. To overcome this difficulty, we plan to perform training under red light of slightly increased brightness.

The exact nature of visual stimuli that crickets use for discriminating lit and dark conditions remains to be clarified. Because the training was conducted in a transparent beaker and the test was conducted in a box made of white Lucite plates, the visual scenes that crickets experienced under illumination in the training and testing situations greatly differed (Fig. 1). Thus, it is not likely that crickets used specific visual objects to define the lit condition. Either the lightness or the presence of a visual scene or both may have been used to discriminate the lit condition and the dark condition.

Crickets experience different ambient light in the daytime and at night, and it is not surprising that crickets that had learned to associate an odor with water or food in the daytime initially exhibit only a low level of CR in the night. After extended training, their CR in the night is as high as that in the daytime, which would help them to increase opportunities to obtain useful resources.

Models to account for decreased context specificity of the CR after extended training

To account for the observed decrease in context specificity with progress of Pavlovian training, we propose neural circuit models shown in Figure 6, which are modifications of the model that we proposed to account for the loss of sensitivity of the CR to US devaluation with extension of training (Mizunami et al. 2019). The model is assumed to represent neural circuits of the mushroom body (MB), which is known to play critical roles in Pavlovian conditioning of odors in insects (Waddell 2013; Hige 2018). The basic assumptions of the model are that (1) paired presentations of the CS and US strengthen the efficacy of two different types of synapses: synapses from neurons that code CS signals (CS neurons, which are assumed to represent Kenyon cells of the MB) to neurons that govern the execution of conditioned responses (CR neurons, assumed to be output neurons of the MB) and synapses from CS neurons to octopamine neurons that mediate appetitive US signals (OA2, assumed to be octopamine neurons projecting to the MB); (2) after standard training, simultaneous activation of CS neurons and OA2 neurons is needed for activation of CR neurons and hence for the CR to occur (shown as AND gate); but (3) with extended training, the synapses from CS neurons to CR neurons become strengthened so that the CS-induced activation of OA2 neurons that in turn activates CR neurons is no longer required for the CR to be executed (For additional description of the model, see the legend for Fig. 6).

Figure 6.

Models to account for a decrease in the context specificity with progression of training. The model is based on one of the two models we proposed to account for a decrease in the sensitivity of the CR to US devaluation with extension of training (Mizunami et al. 2019). The other model is not described here for simplicity. The basic assumptions of the model are that (1) it represents neural circuits in the lobe of the mushroom body (MB), consisting of “CS” neurons that encode CS signals (Kenyon cells), “CR” neurons for which activation leads to the CR (output neurons of the MB lobe) and “OA2” neurons that code US value signals (octopamine neurons projecting to the MB); (2) two kinds of memory traces are formed by standard training, one being the strengthening of synaptic connection from “CS” to “OA2” neurons, which represents the CS-US (S-S) connection, and the other being that of synaptic connection from “CS” neurons to “CR” neurons, which represents the CS-CR (S-R) connection; (3) simultaneous activation of “CS” neurons and “OA2” neurons is needed for activating “CR” neurons (AND gate) and hence for producing the CR after standard training; and (4) activation of “OA2” neurons is inhibited when the animal has been satiated with water US. With extended training the CR becomes insensitive to US devaluation due to strengthening of the CS-CR connection. This implies that the activation of “OA2” neurons is no longer required for the CR to be executed. In A, it is assumed that the context signals control or gate (AND gate) activation of “OA2” neurons mediated by activation of “CS” neurons. In B, it is assumed that the context signals control or gate activation of “CS” neurons by presentation of the CS after standard training, but the control is lost after extended training. Notice that the models are hypothetical and await future validation in physiological studies.

We suggest that there are two different ways in which the background light (i.e., the context) governs the execution of the CR in response to the presentation of the CS. One possibility is that the context could be modifying (or gating) the CS-induced activation of OA2 neurons (i.e., US-processing neurons). In this situation, the background light stimulus “sets the occasion” for activating OA2 neurons and hence for producing a CR. After extended training, activation of the OA2 is not required to activate CR neurons. Thus, context dependency of the CR is lost (Fig. 6A). In this case, the loss of context dependency and the loss of sensitivity to US devaluation of the CR would be achieved by changes in the efficacy of the same synapses. The other possibility is that the ambient light could be modifying (or gating) the activation of CS neurons induced by presentation of the CS itself (Fig. 6B). In this case, on the other hand, the loss of context dependency and loss of control by the US value would be achieved by changes in the efficacy of different synapses. Comparison of conditioning parameters that lead to the loss of context dependency and that lead to the loss of sensitivity to US devaluation will enable determination of whether the ambient light modifies the activation of OA2 or CS neurons.

Comparisons with changes of context dependency by extended instrumental training in mammals

Notably, the loss of context dependency by extended training found in Pavlovian training in crickets is in sharp contrast to findings in instrumental conditioning in mammals. It was found that goal-directed actions, which are sensitive to outcome devaluation and more prominent early in training, are less context-dependent than habitual responses that are insensitive to an outcome devaluation and more prominent after extended training. In humans, context dependency is considered a hallmark of habitual behavior that is formed by repetition of learned actions: Once a habit is formed, context cues come to automatically activate the habit representation in memory (Gardner 2015; Wood and Rünger 2016).

One possible explanation for the differences reported in terms of context dependency as training progresses between Pavlovian conditioning in insects and instrumental conditioning in mammals may be the use of different training protocols. In the study of instrumental conditioning in rats, for example, rats received food pellets after lever pressing in one chamber (context A), but they received no food pellets in contingency with lever pressing in another chamber (context B) (Thrailkill and Bouton 2015). With this training protocol, it is not surprising that the response becomes more context-specific with the progress of training. In the present study, on the other hand, crickets were given Pavlovian training to associate an odor with water in one context but had not experienced another context during training, and thus there is no reason to expect an increase of context specificity with extended training. However, such a difference may not fully account for the differences in context dependency as training progresses that have been reported in Pavlovian conditioning in crickets and instrumental learning in mammals, since habitual responses with a high context specificity appear to develop without experiencing another context in humans (Gardner 2015).

Another possibility is that different changes of context dependency with extended training in Pavlovian conditioning in insects and instrumental conditioning in mammals reflected differences in the nature of associative processes underlying learned behavior. In instrumental conditioning in mammals, it has been suggested that goal-directed actions early in training are guided by action-outcome (A-O) associations and habitual responses after extended training are based on stimulus-response (S-R) associations and that the S-R association is more context-specific than is the A-O association (Dickinson 1985; Balleine and Dickinson 1998; Thrailkill and Bouton 2015). In Pavlovian conditioning in crickets, our model shown in Figure 6 suggests that initial CRs are guided by both stimulus-stimulus (S-S) associations (CS-US connections; see the figure legends.) and stimulus-response (S-R) associations (CS-CR connections), whereas CRs are solely dependent on S-R associations after extended training (For detailed discussion, see Mizunami et al. 2019.). Thus, the question to be addressed is whether such differences in the associative processes indeed lead to different changes in context dependency.

In conclusion, change of context dependency associated with an increase in behavioral automaticity by extended training differs in different learning systems of animals. This study provides a starting point for elucidation of cellular and molecular mechanisms for producing such differences in different learning systems of animals.

Materials and Methods

Insects

Adult male crickets (Gryllus bimaculatus) at 1 wk after the imaginal molt were used. They were reared in 12-h light/dark cycles (light period: 08:00–20:00) at 29°C ± 2°C and were fed a diet of insect pellets and water ad libitum. Four days before the start of the experiments, crickets were placed in 100-mL glass beakers and deprived of drinking water to enhance their motivation to search for water. All experiments were carried out in the daytime.

Pavlovian olfactory conditioning

We used an appetitive conditioning procedure in which an odor (CS) was paired with water (US) as described previously (Matsumoto and Mizunami 2002; Unoki et al. 2005). A syringe (1 mL) containing water was used for conditioning. A small piece of filter paper soaked with peppermint essence was attached to the needle of the syringe. For paired trials, the filter paper was placed within 1 cm of the cricket's antennae for ∼2 sec, and then a drop of water was delivered to the mouth (Fig. 1A). One training session consisted of four paired trials conducted with an intertrial interval (ITI) of 5 min. For the control (unpaired) condition, the CS and the US were presented four times each with an interval of 2.5 min. The air in the beaker was ventilated after each trial. The cricket was kept in a beaker until the next training or test. Crickets were subjected to one training session on 1 d (four trials × 1 d of training), one training session every day on three consecutive days (four trials × 3 d of training) or three consecutive training sessions on 1 d (12 trials × 1 d of training).

Odor preference test

At ∼24 h after completion of training, all groups were subjected to relative preference tests between peppermint odor (CS) and vanilla odor (control), following procedures described previously (Matsumoto and Mizunami 2002; Matsumoto et al. 2003; Unoki et al. 2005). This odor pair is one of two odor pairs used in our preceding study, which produced a higher conditioning score and allowed statistically significant results to be obtained with a smaller sample size (Mizunami et al. 2019). Hence, we used this pair in the present study. The test apparatus consisted of waiting chambers and a test chamber (Fig. 1B). On the floor of the test chamber, there were two holes that connected the chamber with containers containing a filter paper soaked with 3 µL of a solution of peppermint essence or vanilla essence, covered with a fine gauze net. Three containers were mounted on a rotatable holder and two of three odor sources could be located simultaneously just below the holes of the test chamber. For testing, a cricket was transferred from the beaker to the waiting chamber, where it was left undisturbed for 4 min to become accustomed to the surroundings, and then the door to the test chamber was opened. The relative position of the odor source was changed 2 min after the start of the test. The time that the cricket spent exploring the top net with its palpi was recorded cumulatively during a period of 4 min. If the total searching time of a cricket to odor sources was <10 sec, we considered that the animal was less motivated to visit odor sources, possibly due to a poor physical condition, and the data were rejected. Such individuals were ∼10% of all animals tested.

Background light condition

All of the training sessions were performed under white fluorescent light (∼1000 l×), and tests were performed under the same illumination or under dim red light provided by an array of LEDs. Since the photoreceptors of compound eyes or ocelli of crickets are not sensitive to red light (Lall and Trough 1989; Zufall et al. 1989), we refer to the dim red light condition as “dark.” In most experiments, crickets were divided into three groups (Fig. 3B). Two groups received training to associate a CS with a US and then tested under illumination (same group) or in the dark (different group). The third group received unpaired presentations of the CSs and the USs under illumination and was then tested under illumination (unpaired group). Prior to the start of training or testing in the light or dark condition, crickets were acclimated to the lit or dark condition for 4 min.

Water satiation procedure

To investigate the effects of US devaluation on the CR, crickets were given water until satiation, that is, until they stopped drinking. A 1-mL syringe was used to give water, and 30 min later, the test was conducted (see Mizunami et al. 2019 for a more detailed description of the procedure used).

Data analysis

The relative preference between peppermint odor (CS) and vanilla odor (control) was determined by using a preference index (PI), which is the search time for peppermint odor divided by the total search time, multiplied by 100 (%). Since distribution of the data often deviated from normality, comparisons between two groups were performed using the Mann–Whitney test (M–W test). Holm's sequential Bonferroni procedure (Holm's method) was used to adjust the P-values for multiple comparisons. The differences were considered statistically significant when P < 0.05. All statistical analyses were conducted using R 3.5.2.