Roles of Aminergic Neurons in Formation and Recall of Associative Memory in Crickets

Abstract

We review recent progress in the study of roles of octopaminergic (OA-ergic) and dopaminergic (DA-ergic) signaling in insect classical conditioning, focusing on our studies on crickets. Studies on olfactory learning in honey bees and fruit-flies have suggested that OA-ergic and DA-ergic neurons convey reinforcing signals of appetitive unconditioned stimulus (US) and aversive US, respectively. Our work suggested that this is applicable to olfactory, visual pattern, and color learning in crickets, indicating that this feature is ubiquitous in learning of various sensory stimuli. We also showed that aversive memory decayed much faster than did appetitive memory, and we proposed that this feature is common in insects and humans. Our study also suggested that activation of OA- or DA-ergic neurons is needed for appetitive or aversive memory recall, respectively. To account for this finding, we proposed a model in which it is assumed that two types of synaptic connections are strengthened by conditioning and are activated during memory recall, one type being connections from neurons representing conditioned stimulus (CS) to neurons inducing conditioned response and the other being connections from neurons representing CS to OA- or DA-ergic neurons representing appetitive or aversive US, respectively. The former is called stimulus–response (S–R) connection and the latter is called stimulus–stimulus (S–S) connection by theorists studying classical conditioning in vertebrates. Results of our studies using a second-order conditioning procedure supported our model. We propose that insect classical conditioning involves the formation of S–S connection and its activation for memory recall, which are often called cognitive processes.

Introduction

Biogenic amines regulate various functions of central nervous systems in vertebrates and invertebrates (Blenau and Baumann, 2001). In vertebrates, dopamine (DA) pathways are involved in the coordination of motor behavior, motivation, addiction, and reward-based learning of a wide range of sensory stimuli (Schultz, 1998, 2006). In insects, DA appears to play roles in regulating motor behavior (Blenau and Baumann, 2001) and arousal (Andretic et al., 2005), and octopamine (OA), the invertebrate counterpart of noradrenaline, plays roles in desensitizing sensory inputs and regulating various forms of behavior (Roeder, 1999), including aggression (Stevenson et al., 2005; Hoyer et al., 2008; Zhou et al., 2008) and sleep (Crocker et al., 2010). In this article, we review recent advances in studies on the roles of OA-ergic and DA-ergic neurons in classical conditioning in insects, focusing on findings from our behavioral and pharmacological studies on crickets Gryllus bimaculatus.

Roles of Aminergic Neurons in Formation of Olfactory Memory in Honey Bees and Fruit-Flies

Insects are useful animal models for the study of cellular and molecular mechanisms underlying learning and memory (Giurfa, 2003; Heisenberg, 2003; Davis, 2005; Menzel and Giurfa, 2006; Menzel et al., 2006; Keene and Waddell, 2007). This is mainly because insect brains consist of a relatively small number (<10⁶) of neurons and the organization of “microbrains” (Mizunami et al., 1999, 2004) is relatively simple, and thus their functions are highly susceptive to genetic, pharmacological, and other means of experimental manipulation. One of the pioneering studies on the roles of aminergic neurons in insect learning was performed in honey bees by Hammer (1993). A hungry honey bee extends its proboscis in response to sucrose stimulation applied to its antennae, proboscis, or tarsi. The proboscis extension response can be conditioned by pairing an odor applied to the antennae (conditioned stimulus, CS) with sucrose stimulation (unconditioned stimulus, US) (Kuwabara, 1957; Erber et al., 1980). Hammer (1993) observed that pairing of an odor with intracellular stimulation of the VUMmx1 neuron, a putative octopamine immunoreactive neuron (Kreissl et al., 1994) that exhibited responses to sucrose stimulation, induced a conditioning effect. Hence, he concluded that this neuron mediates the reinforcing property of sucrose reward in olfactory conditioning. Later, Hammer and Menzel (1998) showed that local injection of octopamine into the antennal lobes and the calyces of the mushroom bodies, termination areas of the VUMmx1 neuron, substituted the sucrose US in olfactory conditioning; the antennal lobes are primary olfactory centers and the mushroom bodies are higher-order olfactory and multi-sensory association centers (Erber et al., 1980; Heisenberg et al., 1985; Mizunami et al., 1998a,b; Okada et al., 1999; Heisenberg, 2003; Davis, 2005; Menzel and Giurfa, 2006). In addition, Farooqui et al. (2003) showed that RNA interference of OA receptors or pharmacological blockade of OA receptors by mianserin in the antennal lobe impaired olfactory conditioning.

Another pioneering study was performed in the fruit-fly Drosophila by Schwaerzel et al. (2003). Fruit-flies can be conditioned to choose an odor associated with sucrose, or avoid an odor associated with electric shock (Dudai et al., 1976; Tully and Quinn, 1985). Schwaerzel et al. (2003) demonstrated that transgenic flies defective in OA or DA synthesis exhibited defects in appetitive olfactory learning with sucrose reward or aversive olfactory learning with electric shock, respectively. Subsequent studies in fruit-flies confirmed that OA- or DA-ergic neurons convey signals for sucrose reward or electric shock punishment, respectively, in olfactory conditioning in larval (Schroll et al., 2006; Honjo and Furukubo-Tokunaga, 2009; Selcho et al., 2009) and adult flies (Riemensperger et al., 2005; Tomchik and Davis, 2009; Gervasi et al., 2010), although a few exceptions have also been found (Kim et al., 2007, Sitaraman et al., 2008). Notable findings in these studies are that photoactivation of OA-ergic or DA-ergic neurons paired with an odor stimulation successfully induced an appetitive or aversive conditioning effect, respectively, in larval flies (Schroll et al., 2006) and that photoactivation of a class of DA-ergic brain neurons also induced an aversive conditioning effect in adult flies (Claridge-Chang et al., 2009). Roles of DA in conveying aversive US have also been demonstrated in honey bees in olfactory conditioning of the sting extension reflex, in which an odor was paired with electric shock punishment (Vergoz et al., 2007).

Procedures for Olfactory and Visual Pattern Conditioning in Crickets

We studied the roles of OA-ergic and DA-ergic signaling in classical conditioning in crickets Gryllus bimaculatus. We have demonstrated that crickets have excellent olfactory learning capabilities: for example, they can learn (1) to associate an odor with reward by a single operant or classical conditioning trial (Matsumoto and Mizunami, 2000, 2002a), (2) to form a lifetime olfactory memory (Matsumoto and Mizunami, 2002b), (3) to memorize seven pairs of odors at the same time (Matsumoto and Mizunami, 2006), and (4) to associate one odor with reward and another odor with punishment in one visual context and to associate the opposing in another visual context (Matsumoto and Mizunami, 2004). Their high learning capabilities may reflect their omnivorous foraging habit, i.e., they test many potential food items to assess whether they are edible or not. Capacity for forming visual place memory has also been demonstrated in crickets (Wessnitzer et al., 2008). Moreover, we have shown that crickets are suitable materials for the study of molecular mechanisms underlying learning and memory by using pharmacological manipulation (Matsumoto et al., 2006, 2009) and RNA interference (Takahashi et al., 2009).

We used a “classical conditioning and operant testing” procedure, which is based on a high capability of crickets to transfer memory formed in a classical conditioning situation to an operant testing situation (Matsumoto and Mizunami, 2002a; Unoki et al., 2005, 2006). For appetitive olfactory conditioning, crickets were individually placed in a beaker, and one of two odors (e.g., banana and apple odors) was presented to their antennae and then water reward was presented to their mouth (Figure 1A). For aversive olfactory conditioning, one of these two odors was presented to the antennae before presenting 2 M sodium chloride solution to the mouth. In the odor preference test, animals were allowed to freely choose between two odor sources. Each odor source consisted of a container containing a filter paper soaked with a solution of odor essence. The time that the animals touched the gauze net covering the top of each container with their mouths or palpi was measured for evaluating relative odor preference of the animals (Figure 1B). For visual pattern conditioning, either of a white-center and black-surround pattern or a black-center and white-surround pattern was paired with water reward or sodium chloride punishment (Figure 1C). In the preference test, the time that the animals touched each of the two patterns was measured for evaluating relative preference (Figure 1D).

Figure 1.

Procedures for olfactory and visual pattern conditioning in crickets. (A) Procedures for olfactory conditioning. One of two odors (e.g., banana and apple odors) was used as CS, and water or 20% sodium chloride solution was used as US. A syringe containing water or sodium chloride solution was used for conditioning. A filter paper soaked with banana or apple essence was attached to the needle of the syringe. The filter paper was approached to the cricket's antennae so as to present an odor, and then water or sodium chloride was presented to the mouth for appetitive or aversive conditioning, respectively. (B) Apparatus for the odor preference test. On the floor of the test chamber (TCH), there were two holes (H) connecting the chamber with odor sources (OS). Each odor source consisted of a container containing a filter paper soaked with 3 μl solution of banana or apple essence, covered with fine gauze net (N). Three containers were mounted on a rotative container holder (CH) and two of three odor sources could be presented at the same time. A cricket was placed in the waiting chamber (WCH) for 4 min for acclimation and then allowed to enter the test chamber to visit odor sources, by opening a sliding door (SD). Two minutes later, the relative positions of the banana and apple sources were changed. The preference test lasted for 4 min. RA: rotating axle. (C) Visual patterns used for conditioning. A black-center and white-surround pattern (black-center pattern) or a white-center and black-surround pattern (white-center pattern) was used as CS and water or sodium chloride solution was used as US. A pattern was attached to the needle of the syringe. The pattern was presented above the cricket's head and then water or sodium chloride was presented to the mouth for appetitive or aversive conditioning, respectively. (D) Apparatus for the pattern preference test. Two white-center patterns and one black-center pattern (P) were presented on a gray sliding wall (SW) at the end of the test chamber, and two of the three patterns could be presented at the same time. After 4-min acclimation in the waiting chamber, the cricket was allowed to enter the test chamber and to visit visual patterns. Two minutes later, the relative positions of the patterns were changed by sliding the wall, and the choices of the cricket are noted during the next 2 min. Modified from Matsumoto and Mizunami (2002a) and Unoki et al. (2006).

These procedures were highly effective for achieving conditioning. In the case of appetitive olfactory conditioning, for example, one conditioning trial was sufficient to establish conditioning, with its memory lasting for several hours (mid-term memory; see Figure 4; Matsumoto and Mizunami, 2002a; Unoki et al., 2005). Two appetitive conditioning trials with a 5-min interval induce memory that lasts for at least 1 day (see Figure 4), which matches protein-synthesis-dependent long-term memory because it is blocked by injection of a protein-synthesis inhibitor, cycloheximide, into the hemolymph before conditioning (Matsumoto et al., 2003).

Figure 4.

Time course of memory retention after appetitive (A) and aversive (B) conditioning. Seventeen groups of crickets were subjected to 1-trial (gray circles), 2-trial (black triangles), or 6-trial (open squares) appetitive or aversive conditioning trials with an ITI of 5 min. Preference indexes (PIs) for rewarded odor (A) or those for unpunished control odor (B) before (data points at the left) and at various times after conditioning are shown with mean ± SEM. The number of animals is shown at each data point. The results of statistical comparisons before and after conditioning are shown as asterisks (WCX test; p < 0.05; p < 0.01; p < 0.001; NS p > 0.05) and those at different times after conditioning are shown as letters (M–W test, different letter indicating at least p < 0.05). Modified from Unoki et al. (2005), with data on one-trial aversive conditioning provided by A. Hatano.

Roles of Aminergic Neurons in Formation of Olfactory Memory in Crickets

We studied the effect of OA and DA receptor antagonists on appetitive and aversive olfactory conditioning in crickets (Unoki et al., 2005). Crickets injected with epinastine or mianserin, antagonists of insect OA receptors (Roeder et al., 1998; Degen et al., 2000b), into the hemolymph before conditioning exhibited a complete impairment of appetitive conditioning to associate an odor with water reward (Figure 2A); the preference for the rewarded odor after conditioning did not significantly differ from that before conditioning. On the other hand, these animals exhibited no impairment of aversive learning with saline punishment (Figure 2B). The latter observation shows that OA receptor antagonists do not impair sensory function, motor function, or motivation necessary for learning, and we thus conclude that OA is specifically involved in conveying water reward. We also found that fluphenazine, chlorpromazine, or spiperone, antagonists of insect DA receptors (Degen et al., 2000a; Mustard et al., 2003), completely impaired aversive learning with sodium chloride punishment (Figure 2C) but did not affect appetitive learning with water reward (Figure 2D). The latter finding indicates that DA receptor antagonists do not impair the sensory function, motor function, or motivation necessary for learning, and thus we conclude that DA is specifically involved in conveying sodium chloride punishment. It should be cautioned that the specificity of antagonists used in our studies is not necessarily perfect (see Discussion in Unoki et al., 2005). However, two different kinds of OA receptor antagonists impaired appetitive learning but not aversive learning and three different kinds of DA receptor antagonists impaired aversive learning but not appetitive learning, suggesting that the impairments were due to the blockade of OA or DA receptors. Thus, we concluded that OA- or DA-ergic neurons convey information about appetitive or aversive US, respectively, for olfactory conditioning in crickets (Unoki et al., 2005).

Figure 2.

Effects of OA or DA receptor antagonists on appetitive and aversive olfactory conditioning. (A) Dose-dependent effects of OA receptor antagonists on appetitive olfactory conditioning. Six groups of crickets were injected with 3 μl saline (white squares) or saline containing 0.04, 0.1, or 1 μM epinastine (black triangles) or 0.1 or 1 μM mianserin (gray circles). (B) Effects of OA receptor antagonists on aversive olfactory conditioning. Three groups of crickets were injected with 3 μl saline or saline containing 1 μM epinastine or 1 μM mianserin 30 min before 6-trial aversive conditioning. (C) Dose-dependent effects of DA receptor antagonists on aversive olfactory conditioning. Eight groups of crickets were injected with 3 μl saline (white squares) or saline containing 50 or 500 μM fluphenazine (black triangles), 50 or 500 μM chlorpromazine (gray circles) or 20, 50, or 500 μM spiperone (white diamonds). (D) Effects of DA receptor antagonists on appetitive olfactory conditioning. Four groups of crickets were injected with 3 μl saline or saline containing 500 μM fluphenazine, 500 μM chlorpromazine or 500 μM spiperone 30 min before 2-trial aversive conditioning. Relative odor preferences were measured as preference indexes for rewarded odor (A,D) or unpunished control odor (B,C) before (data points at the left in A, C; white bars in B, D) and at 30 min after conditioning (data points at the right in A, C; black bars in B, D) and are shown with mean ± SEM. The number of animals is shown at each data point. The results of statistical comparison before and after conditioning are shown as asterisks (Wilcoxon, WCX test, p < 0.05;*p < 0.01;*p < 0.001, NS p > 0.05). Modified from Unoki et al. (2005).

Roles of Aminergic Neurons in Formation of Visual Pattern Memory and Color Memory

We next studied the effect of OA and DA receptor antagonists on appetitive and aversive conditioning of visual pattern (Unoki et al., 2006) and color (Nakatani et al., 2009). Crickets injected with epinastine or mianserin, OA receptor antagonists, into the hemolymph exhibited a complete impairment of appetitive learning to associate a visual pattern with water reward, but aversive learning to associate a visual pattern with sodium chloride punishment was unaffected (Figure 3A). In contrast, fluphenazine, chlorpromazine, or spiperone, DA receptor antagonists, completely impaired aversive learning but not appetitive learning (Figure 3B). We also found the same for color learning: OA receptor antagonists impaired appetitive color learning with water reward without affecting aversive color learning with sodium chloride punishment. In contrast, DA receptor antagonists impaired aversive color learning without affecting appetitive color learning (Nakatani et al., 2009). These findings indicate that the roles of OA-ergic and DA-ergic neurons in conveying information about appetitive and aversive US, respectively, are ubiquitous in learning of odor, visual pattern and color stimuli, suggesting that these neurons serve as the general reward or punishment system for insect learning.

Figure 3.

Effects of OA or DA receptor antagonists on appetitive and aversive visual pattern conditioning. (A) Effects of OA receptor antagonists. Six groups of crickets were each injected with 3 μl saline or saline containing 1 μM epinastine or 1 μM mianserin at 30 min before 8-trial appetitive (left) or before 12-trial aversive conditioning (right). (B) Effects of DA receptor antagonists. Six groups of crickets were each injected with 3 μl saline containing 500 μM fluphenazine, 500 μM chlorpromazine, or 500 μM spiperone at 30 min before 8-trial appetitive (left) or before 12-trial aversive conditioning (right). Preference indexes for rewarded visual pattern (in the case of appetitive conditioning) and those of unpunished visual pattern (in the case of aversive conditioning) before (white bars) and at 30 min after conditioning (black bars) are shown with mean ± SEM. The number of animals is shown at each data point. The results of statistical comparison before and after conditioning (WCX test) and between experimental and saline-injected control groups (M–W test) are shown as asterisks (p < 0.05; p < 0.01; p < 0.001, NS p > 0.05). Modified from Unoki et al. (2006).

In mammals, midbrain DA-ergic neurons play major roles in a wide range of visual, auditory, and somatosensory stimuli and thus are considered to serve as a general reward system (Schultz, 1998, 2006). It appears that the roles of aminergic neurons in conveying reinforcement signals are conserved across different phyla, but the kind of reinforcement signal that each biogenic amine mediates is different: DA mediates appetitive reinforcement in mammals but mediates aversive reinforcement in insects. Future studies on the roles of DA in learning in phylogenetically ancient species may clarify how different roles of DA in positive or negative reinforcement have emerged during the course of evolution. Neurotransmitters meditating negative reinforcement in mammals are less known, but roles of noradrenaline, serotonin, or DA in some forms of aversive learning have been suggested (Daw et al., 2002; Harley, 2004; Wise, 2004; Schultz, 2006).

Different Dynamics of Appetitive Memory and Aversive Memory

In the work described above, we noticed that the time courses of appetitive memory and aversive memory fundamentally differ (Unoki et al., 2005, 2006; Nakatani et al., 2009). In Figure 4, the time course of memory after 1-trial, 2-trial, and 6-trial appetitive olfactory conditioning and that after 2-trial and 6-trial aversive olfactory conditioning are shown. A group subjected to 1-trial appetitive conditioning exhibited a significant level of retention at 30-min after conditioning: the preference for rewarded odor was significantly greater than that before conditioning. The memory did not significantly decay from 30 min to 1 h after conditioning, but it was completely diminished at 1 day after conditioning. Two-trial and 6-trial groups exhibited a significant level of 30-min retention, with no significant decay of retention from 30 min to 1 day after conditioning. Two-trial aversive conditioning group exhibited a significant level of 30-min retention. However, the memory decayed to a non-significant level at 1 h after conditioning. Six-trial aversive conditioning group exhibited a significant decay of retention from 30 min to 1 h after training. Thereafter, a significant level of retention was maintained with no significant decay from 1 h to 1 day after conditioning. In short, aversive olfactory memory exhibited a prominent decay from 30 min to 1 h after conditioning, whereas appetitive olfactory memory exhibited little decay from 30 min to 1 day after conditioning.

We found similar distinction of dynamics between appetitive memory and aversive memory for visual pattern conditioning (Unoki et al., 2006) and color conditioning (Nakatani et al., 2009) in crickets. It was obvious that the number of conditioning trials, and hence the levels of initial acquisition, also influenced memory dynamics, but effects of these factors did not account for the difference in memory dynamics observed after appetitive and aversive learning: reward memory was sustained even when the level of 30-min retention was low, and punishment memory exhibited a characteristic decay even when the level of 30-min retention was high.

Comparisons with studies in other species of insects showed that our finding that aversive memory is less durable than appetitive memory is not specific to the type of US we used (water as appetitive US and sodium chloride as aversive US) or the species used (crickets). In fruit-flies, it has been reported that punishment memory after conditioning of an odor with electric shock punishment decays much faster than reward memory after conditioning of an odor with sucrose reward, regardless of intensity of electric shock and the concentration of the sucrose solution (Tempel et al., 1983). A recent study also suggested that aversive olfactory memory with quinine or saline punishment is less durable than appetitive olfactory memory with sucrose reward in fruit-fly larvae (Honjo and Furukubo-Tokunaga, 2009). Thus, dynamics of punishment memory and reward memory differ for different intensities or kinds of unconditioned stimulus (US; water or sugar as reward and saline, quinine, or electric shock as punishment), for different species of insects (crickets and fruit-flies), and for different conditioning paradigms (individual conditioning in crickets and group conditioning in fruit-flies).

By examining the literature in human psychology, we have proposed that these findings in insects are comparable to findings in humans (Nakatani et al., 2009). Many studies in educational psychology have proposed that punishment is highly effective for immediately suppressing behavior of children at school or home, but the effect tends to be short-lived compared to the effect of reward (Peine and Howarth, 1975; Gershoff, 2002; Driscoll, 2005). However, to our knowledge, no convincing evidence supporting the argument that the dynamics of reward and punishment memory fundamentally differ has been obtained in rigorously controlled animal experiments (Walters and Crusec, 1977), except for several studies on insects discussed above. Obviously, our proposal that different dynamics after punishment and reward learning is conserved across phyla is highly speculative, and more studies on vertebrates are clearly needed to evaluate the validity of this idea.

If different dynamics of punishment and reward memory is conserved across different phyla, what is the possible adaptive significance for it? We have proposed that it is related to a different significance of reward and punishment leaning for survival in a changing environment (Nakatani et al., 2009). Since the environment is constantly changing, stimuli that once served as predictors of punishment may change to predict reward or vice versa. Consider that an inedible food item in one season may become profitable in the next season. In this case, long-term retention of avoidance of stimuli that once predicted aversive stimuli is not necessarily beneficial, because it reduces the opportunity to obtain useful resources in the future. Stimuli that once predicted reward may also be changed to predict punishment, but long-term retention of preference for once-rewarded stimuli has no such cost, because animals can re-learn to avoid such stimuli when they encounter the stimuli again. Thus, we have proposed that different adaptive significance of durability of memory between reward and punishment learning is the basis of different dynamics of reward and punishment memories (Nakatani et al., 2009).

Participation of Octopaminergic and Dopaminergic Neurons in Appetitive and Aversive Memory Recall

We then studied the effect of an OA or DA receptor antagonist on appetitive or aversive memory recall (retrieval). The results suggested that intact OA-ergic or DA-ergic signaling is necessary for recall of appetitive or aversive memory, respectively, after olfactory learning and visual pattern learning (Mizunami et al., 2009). Crickets were subjected to appetitive or aversive olfactory conditioning and were injected with an OA or DA receptor antagonist before retention test. Injection of epinastine or mianserin, OA receptor antagonists, completely impaired appetitive olfactory memory recall but had no effect on aversive olfactory memory recall (Figure 5A). On the other hand, injection of fluphenazine, chlorpromazine, or spiperone, DA receptor antagonists, completely impaired aversive memory recall but had no effect on appetitive memory recall (Figure 5B). This is in accordance with observations in honey bees that disruption of OA-ergic transmission in the antennal lobe, the primary olfactory center, by an OA receptor antagonist (mianserin) or by RNA interference of the OA receptor gene disrupted appetitive olfactory memory recall (Farooqui et al., 2003), although the results of the study by Farooqui et al. were not conclusive as we have discussed before (Mizunami et al., 2009). We also found that OA and DA receptor antagonists impaired appetitive and aversive memory recall, respectively, in visual pattern conditioning (Mizunami et al., 2009). Therefore, we concluded that intact synaptic transmission from OA- and DA-ergic neurons is needed for the recall of appetitive memory and aversive memory, respectively, in both olfactory and visual pattern learning.

Figure 5.

Octopamine and dopamine receptor antagonists impair appetitive and aversive olfactory memory recall, respectively. Effects of OA (A) or DA (B) receptor antagonists on olfactory memory recall. Twelve groups of crickets were each subjected to 2-trial appetitive (left) or 6-trial aversive (right) olfactory conditioning trials. On the next day, each group was injected with 3 μl of saline or saline containing 1 μM epinastine, 1 μM mianserin, 500 μM fluphenazine, 500 μM chlorpromazine or 500 μM spiperone at 30 min before the final test (upper diagram). Preference indexes for rewarded odor (in the case of appetitive conditioning) or unpunished control odor (in the case of aversive conditioning) before (white bars) and 1 day after (black bars) conditioning are shown with mean + SEM. The number of crickets is shown at each data point. The results of statistical comparison before and after conditioning (WCX test) and between experimental and saline-injected control groups (M–W test) are shown as asterisks (p < 0.05; p < 0.01; p < 0.001, NS p > 0.05). Modified from Mizunami et al. (2009).

We noticed that our findings are not consistent with conventional neural models of insect classical conditioning. Figure 6A depicts perhaps the best model proposed to account for the roles of extrinsic and intrinsic neurons of mushroom bodies in olfactory conditioning in the fruit-fly Drosophila (Schwaerzel et al., 2003). This model assumes that (1) “CS” neurons (intrinsic neurons of the mushroom body, called Kenyon cells) that convey signals about a CS make synaptic connections with dendrites of “CR” neurons (efferent (output) neurons of the mushroom body lobe), activation of which leads to a CR (conditioned response) that mimics UR (unconditioned response), but these synaptic connections are silent or very weak before conditioning, (2) OA- or DA-ergic efferent neurons projecting to the lobes (“OA/DA” neurons), which convey signals for appetitive or aversive US, respectively, make synaptic connections with axon terminals of “CS” neurons, and (3) the efficacy of the synaptic transmission from “CS” neurons to “CR” neurons that induces a conditioned response (CS–CR or S–R connection) is strengthened by coincident activation of “CS” neurons and “OA/DA” neurons during conditioning (assuming Kandelian synaptic plasticity; see Abrams and Kandel, 1988). In short, this model assumes that presentation of a CS after conditioning activates the CS–CR or S–R connection to induce a CR. Thus, this model is characterized as an S–R model (Figure 7A), following terminology in studies on classical conditioning in higher vertebrates (Rescorla, 1988; Pickens and Holland, 2004; Holland, 2008). It can be pointed out that the S–R model accounts for most forms of classical conditioning in invertebrates, including classical conditioning of gill withdrawal reflex in the mollusk Aplysia, where pairing of a gentle tactile stimulus to the siphon (CS) and a strong tactile stimulus to the gill (US) results in an enhancement of the efficacy of synaptic transmission from siphon sensory neuron to gill motor neuron (Abrams and Kandel, 1988; Kandel, 2001; Roberts and Glanzman, 2003), which is characterized as an S–R connection. The model by Schwaerzel et al. (2003), however, is inconsistent with our findings because it predicts that activation of OA- or DA-ergic neurons is not required for appetitive or aversive memory recall, respectively.

Figure 6.

Conventional and new models of classical conditioning in insects. (A) A model proposed to account for the roles of intrinsic and extrinsic neurons of the mushroom body in olfactory conditioning in fruit-flies (Schwaerzel et al., 2003). OA-ergic or DA-ergic neurons (“OA/DA” neurons) convey signals for appetitive or aversive US, respectively. “CS” neurons, which convey signals for CS, make synaptic connections with “CR” neurons that induce the conditioned response (CR), the efficacy of the connection being strengthened by conditioning. “OA/DA” neurons make synaptic connections with axon terminals of “CS” neurons. (B) A new model of classical conditioning, termed Mizunami–Unoki model. The model assumes that efficacy of synaptic transmission from “CS” neurons to “OA/DA” neurons is strengthened by conditioning and that coincident activation of “OA/DA” neurons and “CS” neurons is needed to activate “CR” neurons to lead to a CR (AND gate). (C) Mizunami–Unoki model to account for second-order conditioning, in which an odor (CS1) is paired with water or sodium chloride solution and a visual pattern (CS2) is paired with the odor (CS1), as indicated in the inset. The model predicts that pairing of CS1 and US at the first conditioning stage results in enhancement of synapses from “CS1” neurons to “OA/DA” neurons, and activation of the synapses (by CS1) at the second conditioning stage leads to simultaneous activation of “OA/DA” and “CS2” neurons, and this leads to enhancement of synaptic transmission from “CS2” neurons to “OA/DA” neurons and to “CR” neurons. Modified from Mizunami et al. (2009).

Figure 7.

S–R and S–S theories to account of classical conditioning. Two theories, i.e., the stimulus–response (S–R) association theory and the stimulus–stimulus (S–S) association theory, have been proposed to account for classical conditioning in higher vertebrates including humans (Rescorla, 1988). In the S–R theory (A), classical conditioning is viewed as the strengthening of a new reflex pathway for the CS to evoke a conditioned response (CR) (i.e., a pathway from neurons that code for the CS to neurons whose activities lead to behavioral response), as a result of pairing of the CS with a US (Rescorla, 1988; Pickens and Holland, 2004; Holland, 2008). According to this view, an initially insignificant event, CS, is incorporated into the reflex system under the control of a more biologically significant stimulus, US, whenever those two events occur in close temporal contiguity. This view accounts for some forms of classical conditioning in higher vertebrates (Rescorla, 1988; Pickens and Holland, 2004). Many other forms of classical conditioning in higher vertebrates, however, have been suggested to involve the strengthening of S–S connection, i.e., connection from neurons representing CS to those representing US (B). According to this view (S–S theory), an association is formed between internal representation of the CS and that of the US are strengthened (i.e., a connection from neurons that code for the CS to neurons that code for the US is strengthened), and the growth of this association permits the CS to activate a representation of the US in the absence of the US itself. This anticipatory activation of the US representation produces the CR. This view is referred to as cognitive account of classical conditioning, since it assumes the formation of internal representation of the relationship between external sensory events (i.e., contingent occurrence of the CS and US) (Rescorla, 1988; Pickens and Holland, 2004; Holland, 2008). Notice that these theories address the question of what kinds of connections are strengthened by learning, not the question of how or by which mechanism such connections are formed.

We have proposed a new model (Figure 6B), with minimal modifications of the model proposed by Schwaerzel et al. (2003). We have assumed that (1) activation of “OA/DA” neurons and resulting release of OA or DA are needed to “gate” the sensori-motor pathway from the “CS” neurons to “CR” neurons after conditioning and (2) synaptic connection from “CS” neurons to “OA/DA” neurons representing US is strengthened by coincident activation of “CS” neurons and “OA/DA” neurons by pairing of a CS with a US (assuming Hebbian synaptic plasticity). The latter connection is termed a CS–US or S–S connection (Figure 7B; Rescorla, 1988; Pickens and Holland, 2004), In short, our model assumes that two kinds of memory traces are formed by conditioning and that activation of both memory traces is needed for memory recall. This model corresponds to a hybrid of the S–R and S–S models (for explanation of the S–S model, see legends of Figure 7).

An alternative possibility to explain our findings is that different sets of “OA/DA” neurons govern reinforcement and memory retrieval processes, respectively. This is achieved by modifying the model shown in Figure 6A by assuming other “OA/DA” neurons in neural pathways downstream of the “CR” neurons. This model, however, failed to account for our results with second-order conditioning described below.

Evaluation of Our Model by Using a Second-Order Conditioning Procedure

The critical assumption of our model is that the pathway from neurons representing CS to OA/DA neurons representing appetitive or aversive US (S–S connection) is strengthened by conditioning. We evaluated this assumption by using a second-order conditioning procedure (Mizunami et al., 2009). Second-order conditioning (Figure 6C) is a procedure for testing whether a CS can acquire the reinforcing property of a US, by pairing a CS (CS1) with a US and then pairing another CS (CS2) with CS1 (Rescorla, 1988). Our model predicts that blockade of OA or DA receptors (on axon terminals of “CS” neurons) during the initial CS1–US pairing stage does not impair the enhancement of synapses from “CS” neurons to “OA/DA” neurons. This is because blockade of OA or DA receptors should not affect normal activities of “CS” neurons and “OA/DA” neurons (Figure 6C). Hence, CS1 should act as a reinforcer in the second CS1–CS2 pairing stage, although CS1 per se does not induce a CR (notice that impairments of first-order conditioning by DA/DA receptor antagonists are shown in Figures 2 and 3). Therefore, our model predicts that blockade of OA or DA receptors during the first CS1-US pairing stage does not impair second-order conditioning, but the same treatment during the second CS2–CS1 pairing stage or during the final retention test impairs second-order conditioning.

We first studied whether second-order conditioning can be achieved in crickets (Figure 8; Mizunami et al., 2009). We used an olfactory stimulus as CS1 and a visual pattern as CS2. For appetitive or aversive conditioning, an odor (CS1) was paired with water or sodium chloride solution, respectively, and then a visual pattern (CS2) was paired with an odor (CS1). A group of animals that was subjected to appetitive second-order conditioning trials exhibited significantly increased preference for the CS2 (Figure 8A). In contrast, control groups that were each subjected to unpaired presentations of stimuli at the first or second conditioning stage exhibited no significantly increased preference for the CS2 (Figure 8A), thus indicating that the increased preference for the CS2 in the experimental group is truly the result of second-order conditioning. Similarly, we showed that second-order aversive conditioning could be achieved (Figure 8B).

Figure 8.

Appetitive (A) and aversive (B) second-order conditioning. Two groups of animals were each subjected to appetitive (A) or aversive (B) second-order conditioning trials (P/P groups). Four control groups were each subjected to unpaired presentations in the first (UP/P groups) or second (P/UP groups) stage in appetitive (A) or aversive (B) second-order conditioning. Animals received 4 first-stage trials and then 4 second-stage trials for appetitive second-order conditioning and 6 first-stage trials and then 4 second-stage trials for aversive second-order conditioning. Preference indexes for the CS2 (in the case of appetitive second-order conditioning) or control pattern (in the case of aversive second-order conditioning) before (white bars) and after (black bars) conditioning are shown with mean + SEM. The results of statistical comparison before and after conditioning (WCX test) and between experimental and saline-injected control groups (M–W test) are shown as asterisks (p < 0.05; p < 0.01; p < 0.001, NS p > 0.05). Modified from Mizunami et al. (2009).

We then studied the effect of OA or DA receptor antagonist on appetitive or aversive second-order conditioning (Figure 9). A group of animals injected with epinastine before the first conditioning stage of appetitive second-order conditioning exhibited a significantly increased preference for the CS2, thus indicating that blockade of OA receptor during the first stage does not impair appetitive second-order conditioning (Figure 9A). In contrast, the group injected with epinastine before the second stage (Figure 9B) or before final test (Figure 9C) for appetitive second-order conditioning exhibited no significantly increased preference for the CS2, indicating a complete impairment of second-order conditioning. Similarly, blockade of DA receptor antagonist at the first stage did not impair aversive second-order conditioning, but that at the second stage or final test impaired aversive second-order conditioning (Figures 9D–F).

Figure 9.

Octopamine and dopamine receptor antagonists impair appetitive and aversive second-order conditioning. (A–C) Three groups of animals were each injected with 3 μl of saline containing 1 μM epinastine at 30 min before the first conditioning stage (A), before the second conditioning stage (B) or before the final test (C) in appetitive second-order conditioning. One control group received no injection (B, intact), and two other groups were each injected with saline at 30 min before the second conditioning stage (B, saline) or before the final test (C, saline). (D–F) Three groups were each injected with 3 μl of saline containing 500 μM fluphenazine at 30 min before the first conditioning stage (D), before the second conditioning stage (E) or before the final test (F) in aversive second-order conditioning. Animals received 4 first-stage trials and then 4 second-stage trials for appetitive second-order conditioning and 6 first-stage trials and then 4 second-stage trials for aversive second-order conditioning. One control group received no injection (E, intact), and two other groups were each injected with saline at 30 min before the second conditioning stage (E, saline) or before the final test (F, saline). Preference indexes for the CS2 (in the case of appetitive second-order conditioning) or the control pattern (in the case of aversive second-order conditioning) before (white bars) and after (black bars) conditioning are shown with mean + SEM. The results of statistical comparison before and after conditioning (WCX test) and between experimental and saline-injected control groups (M–W test) are shown as asterisks (p < 0.05; p < 0.01; p < 0.001, NS p > 0.05). Modified from Mizunami et al. (2009).

The results were in full accordance with our model. In addition, the results were inconsistent with an alternative model in which different sets of OA/DA neurons participate in reinforcement and memory recall, because the model predicts that blockade of OA/DA receptors at any stage of conditioning impairs second-order conditioning.

Our finding that OA-ergic signaling and DA-ergic signaling are needed for appetitive and aversive memory recall, respectively, is in accordance with some previous findings in honey bees and fruit-flies but not with other findings, as we have discussed elsewhere (Mizunami et al., 2009). Thus, we proposed that activations of OA- or DA-ergic neurons are needed for memory recall in some forms of classical conditioning in insects but not in other forms (Mizunami et al., 2009). The critical factors for determining the requirement of OA- or DA-ergic signaling in appetitive/aversive memory recall remain to be clarified.

Future Perspectives

Many questions concerning the roles of OA- or DA-ergic neurons in insect learning remain to be addressed. Firstly, although there has been a substantial progress in identifying DA- and OA-ergic neurons involved in appetitive and aversive learning in honey bees (Hammer, 1993) and fruit-flies (Schroll et al., 2006; Claridge-Chang et al., 2009), more studies need to be performed to clarify the morphology and physiology of DA or OA-ergic neurons participating in insect learning. In crickets, it has been suggested that the mushroom bodies are involved in olfactory learning (in Acheta domesticus: Scotto-Lomassese et al., 2003), and thus OA- and DA-ergic neurons projecting to the mushroom bodies need to be anatomically and physiologically characterized. Detailed immunohistochemical studies for OA and DA will provide the first step for this. Secondly, we have proposed that different dynamics of appetitive and aversive memory are the result of different dynamics of biochemical processes after activation of OA and DA receptors, respectively (Nakatani et al., 2009). To examine this hypothesis, physiology, and biochemistry of neurons participating in association of CS and US, most probably Kenyon cells of the mushroom bodies, need to be characterized. Thirdly, our successful demonstration of cross-modal second-order conditioning suggests that OA- or DA-ergic neurons participating in olfactory learning also participate in visual pattern learning (Mizunami et al., 2009), and this suggestion needs to be examined by electrophysiologically characterizing responses of OA- and DA-ergic neurons during olfactory learning and visual learning. Fourthly, mammalian midbrain DA-ergic neurons have been suggested to convey prediction error for rewarding outcomes, namely, they encode the error between expected and actual rewarding outcomes (Schultz, 1998, 2006), and it is important to clarify whether or not OA-ergic (resp. DA-ergic) neurons in insects work in the same way or not in appetitive (resp. aversive) learning, respectively. Further examination of these questions should contribute to clarification of phylogenetically conserved principles of brain systems underlying learning and memory among vertebrates and invertebrates, as well as each specific functional specialization in each group.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.