Skip to main content
NCBI home page
Learning & Memory logoLink to Learning & Memory
. 2017 May;24(5):191–198. doi: 10.1101/lm.045062.117

Common microbehavioral “footprint” of two distinct classes of conditioned aversion

Emmanouil Paisios 1, Annabell Rjosk 1, Evren Pamir 1, Michael Schleyer 1
PMCID: PMC5397685  PMID: 28416630

Abstract

Avoiding unfavorable situations is a vital skill and a constant task for any animal. Situations can be unfavorable because they feature something that the animal wants to escape from, or because they do not feature something that it seeks to obtain. We investigate whether the microbehavioral mechanisms by which these two classes of aversion come about are shared or distinct. We find that larval Drosophila avoid odors either previously associated with a punishment, or previously associated with the lack of a reward. These two classes of conditioned aversion are found to be strikingly alike at the microbehavioral level. In both cases larvae show more head casts when oriented toward the odor source than when oriented away, and direct fewer of their head casts toward the odor than away when oriented obliquely to it. Thus, conditioned aversion serving two qualitatively different functions—escape from a punishment or search for a reward—is implemented by the modulation of the same microbehavioral features. These features also underlie conditioned approach, albeit with opposite sign. That is, the larvae show conditioned approach toward odors previously associated with a reward, or with the lack of a punishment. In order to accomplish both these classes of conditioned approach the larvae show fewer head casts when oriented toward an odor, and direct more of their head casts toward it when they are headed obliquely. Given that the Drosophila larva is a genetically tractable model organism that is well suited to study simple circuits at the single-cell level, these analyses can guide future research into the neuronal circuits underlying conditioned approach and aversion, and the computational principles of conditioned search and escape.

Approaching desirable situations and avoiding undesirable situations are fundamental tasks for any animal, and learning the cues that predict such situations gives an edge in the struggle of life. Animals can learn to avoid situations when these situations feature either something bad or the lack of something good, and to approach situations when they feature either something good or the lack of something bad (for review, see Seymour et al. 2007; Gerber et al. 2014; Kong et al. 2014; see also Discussion). Understanding the processes that bring about conditioned approach or conditioned avoidance requires a model organism that on the one hand provides convenient experimental access for detailed analyses of sensory systems, brain networks, and specific motor actions, and on the other hand is capable of learning-modulated navigation. The larva of the fruit fly Drosophila melanogaster arguably is such a model organism. It is a well-established study case in particular for navigation tasks with respect to olfactory cues (Cobb 1999; Louis et al. 2008; Gomez-Marin et al. 2011; Lahiri et al. 2011; Gershow et al. 2012; Gomez-Marin and Louis 2012, 2014; Schulze et al. 2015; Wystrach et al. 2016) as well as for odor-tastant learning (Scherer et al. 2003; Gerber et al. 2009; Diegelmann et al. 2013; Schleyer et al. 2013).

After odor–tastant training, both conditioned aversion and conditioned approach can be observed. Larvae avoid an odor after it has been paired with, for example, the bitter tastant quinine (Gerber and Hendel 2006; Schleyer et al. 2011, 2015a; El-Keredy et al. 2012), or after it has been paired with the lack of, for example, sugar (Gerber and Hendel 2006; Saumweber et al. 2011; Schleyer et al. 2011, 2015a). Likewise, larvae approach an odor after it has been paired with the lack of quinine, or with sugar (Gerber and Hendel 2006; Saumweber et al. 2011; Schleyer et al. 2011, 2015a; El-Keredy et al. 2012). We note that according to Pavlovian terminology, sugar and quinine in such an experiment would be called “appetitive unconditioned stimulus” and “aversive unconditioned stimulus,” respectively. In the Drosophila literature, however, “reward” and “punishment” are commonly used instead for many decades (e.g., Tempel et al. 1983). In the following, we will stick to this established terminology of the Drosophila community and use “reward” for an appetitive unconditioned stimulus, and “punishment” for an aversive unconditioned stimulus.

Importantly, learned behavior after appetitive and aversive training serves two different functions. Learned behavior after aversive training ceases in the absence of the trained punishment and therefore is best grasped as an escape from something bad, being pointless in the absence of anything to escape from. Learned behavior after appetitive training, in turn, ceases in the presence of the trained reward and therefore may best be viewed as a learned search for the reward that is obsolete in the presence of a sought-for item (for more detailed discussions on this account, see Gerber and Hendel 2006; Schleyer et al. 2011, 2015a). Thus, one can observe two classes of conditioned aversion, namely after paired aversive training and unpaired appetitive training, serving two different functions, namely escape from punishment and search for reward, respectively. Likewise, two classes of conditioned approach can be observed.

Conditioned aversion and approach are commonly measured by counting how the animals distribute themselves in the test arena at the end of a test phase. Only recently, efforts were made to understand the actual modulations of locomotion that underlie conditioned aversion and approach in high resolution (Schleyer et al. 2015b). In this study, we refer to these behavioral modifications as “microbehavior.” While the microbehavioral impact of conditioned approach is relatively well understood (Schleyer et al. 2015b), the microbehavioral impact of conditioned aversion remains unknown. This information gap we seek to close.

We ask whether avoiding in order to escape from something bad and in order to search for something good result from the same modulations of locomotion, and whether these modulations concern the same aspects of locomotion as those found for odor approach (Fig. 1). The results can add to the conceptually interesting comparison of learning about rewards/punishments versus the lack of rewards/punishments. Furthermore, this study can serve as a behavioral framework for the on-going analyses of the underlying circuits (e.g., Berck et al. 2016; Fushiki et al. 2016; Rohwedder et al. 2016). Together, these efforts can provide a detailed picture of how associative memory helps to organize conditioned approach and aversion in a simple nervous system, and how both are implemented in specific motor actions. This study case may then guide future research on how more complex brains organize these processes.

Figure 1.

Figure 1.

Open in a new tab

Analysis of larval microbehavior. (A) Example of a larva track. Larvae orient through odor gradients by a sequence of relatively straight runs and lateral head movements (head casts, HC). An HC can be either accepted and followed by a change in orientation (e.g., the first two HCs), or rejected and followed by a second HC (e.g., the last few HCs). (B–B″) HC detection. The larval body is segmented into 12 spine points, from the tail (spine point 1) to the head (spine point 12). Two vectors are defined: the tail vector (spine points 2–6) and the head vector (spine points 9–11). (B′–B″) The angular speed of the head vector and tail vector, respectively, for the example track displayed in A. The stippled lines represent thresholds used to define HCs. Red and green zones correspond to the left and right HCs in the example track. An HC is detected whenever the angular speed of the head vector exceeds ±35°/sec. Usually, the angular speed of the tail vector increases after an HC because the larval body is following the head's new direction (often called “turn”). (C–C″) Orientation of the animal. The heading angle measures the orientation of the animal's head relative to the odor source, with positive and negative angles indicating the odor being to the right or left, respectively. The heading angle changes quickly during HCs and stays relatively stable during runs. The bearing angle measures the orientation of the animal's body relative to the odor source. After each HC or group of HCs, the bearing angle changes with some delay. (D–D′) Analysis of orientation changes. During an HC, the bending angle increases and reaches a maximum shortly after the end of the HC. When the larva moves straight again, the bending angle drops to zero. Thus, in order to turn right, in this example (A), a larva first moves its head to the right (B′), causing a strong bending of the animal (D′) and a quick change in the head's orientation toward the negative (C′). This is followed by a lateral movement of the back part of the body (B″) to align it with the head's new direction (D′). This results in a change of orientation and a more negative bearing angle (C″).

Results

We performed classical conditioning experiments with larval Drosophila either using quinine or sugar as punishment or reward, respectively (these are unconditioned stimuli in Pavlovian terminology). First, we determined the group-level preference. We found conditioned odor aversion after paired aversive training (i.e., when odor and quinine punishment were presented simultaneously) (Fig. 2A–A″) as well as after unpaired appetitive training (i.e., when odor and fructose reward were presented separately, in consecutive trials) (Fig. 2B–B″). After unpaired aversive and paired appetitive training, respectively, we found conditioned odor approach (Fig. 2A–A″,B–B″). The difference between paired and unpaired aversive training reflects learned escape from quinine as it was abolished when there was no quinine to escape from; likewise, the difference between paired and unpaired appetitive training reflects learned search for sugar as it was abolished in the presence of the sought-for sugar (Supplemental Fig. S1; Gerber and Hendel 2006; Saumweber et al. 2011; Schleyer et al. 2015a; for a discussion concerning the use of “search” and “escape” in this context, see Schleyer et al. 2011). This offered the opportunity to measure baseline levels of olfactory behavior without the behavioral influence of associative memory (Supplemental Fig. S1; see Schleyer et al. 2015b for further discussion). Given that the measurement of baseline behavior involved testing on different substrates, we asked whether the presence of either the sugar or the quinine substrate also had an influence on innate olfactory preference. We found this not to be the case (Supplemental Fig. S2A), confirming earlier results (Hendel et al. 2005; Schleyer et al. 2011, 2015a,b).

Figure 2.

Figure 2.

Open in a new tab

Olfactory preferences after (A–A″) aversive training and (B–B″) appetitive training. (A,B) show example tracks, (A′–B′) the animals’ preferences over time as detected by the software; (A″–B″) preferences as based on manual counting after 3 min. (A) Representative example tracks of individual animals after paired (left) and unpaired (right) aversive training. Black dots represent the starting positions of the animals; the small red circle shows the position of the odor source. (A′–A″) After paired aversive training, larvae avoid the odor whereas after unpaired aversive training (i.e., presenting odor and quinine separately, in consecutive trials) they approach it. When tested in the absence of quinine, learned behavior is not observed; instead, the larvae display an intermediate baseline preference. (B) Representative example tracks of individual animals after paired (left) and unpaired (right) appetitive training. (B′–B″) After paired appetitive training, larvae approach the odor, whereas after unpaired appetitive training they avoid it. When tested in the presence of fructose, learned behavior is not observed; instead, the larvae display an intermediate baseline preference. Both after appetitive and aversive memory, preference differs across groups (P < 0.05, df = 2, N = 27, 56, 27, KW). Significant between-group differences (MWU, P < 0.05 corrected according to Bonferroni–Holm) are indicated with lower case letters above the boxes, significant differences to chance level (OSS, P < 0.05 corrected according to Bonferroni–Holm) are indicated by asterisks below the boxes. For a display of median and 25%–75% quantiles of the time-resolved preferences, see Supplemental Figure S8.

Our results provide in total four experimental groups displaying learned behavior: two groups showing conditioned aversion and two groups showing conditioned approach. Importantly, conditioned aversion can serve two functions: either to search for sugar or to escape from quinine. Hence, by comparing the two groups of animals that show conditioned aversion, we can uncover whether both classes of conditioned aversion are brought about by the same or different microbehavioral features. Likewise, larvae may approach the odor either in search of sugar or to escape from quinine. In the following, we compared the microbehavioral “footprint” (1) of the two classes of conditioned aversion, and (2) of conditioned aversion and conditioned approach. As larvae orient through odor gradients by a sequence of relatively straight runs and lateral head movements (head casts, HC), which are followed by changes in orientation (Fig. 1A), we focused on three features of chemotaxis: how fast larvae run, under which circumstances they initiate a HC, and where-to they direct the HC.

We found no effect of associative training experience on run speed (Fig. 3A,B). Thus, neither conditioned aversion nor conditioned approach comes about by general increases or decreases of run speed. On the contrary, run speed was decreased in the presence of quinine (left and right box in Fig. 3A). Also in presence of sugar run speed is slightly yet statistically nonsignificantly reduced (middle box in 3B). A reduction in presence of either substrate was also found in experimentally naïve larvae (Supplemental Fig. S2B). With respect to sugar, this observation could be a hint that gustatory behavior may be best viewed as a form of kinesis (Fraenkel and Gunn 1961): in a gustatory “good” situation, slowing down helps the larva not to drift away from a food source. However, following this logic one would expect that larvae would speed up in the presence of quinine in order to escape—but in fact they slow down. The observed effects on speed may rather be based on changes in the physical properties of the substrate caused by adding sugar or quinine (stiffness, structure of the surface, etc.; notably, agarose concentration has also been found to affect run speed [Apostolopoulou et al. 2014]), or may reflect a strategy of the larvae for discerning tastant gradients.

Figure 3.

Figure 3.

Open in a new tab

Analysis of run speed. (A,B) show run speed, (C,D) show run speed-modulation. (A) After aversive training, both reciprocally trained groups tested in the presence of quinine display lower run speed than the baseline, but do not differ from each other. (B) After appetitive training, the reciprocally trained groups tested on pure agarose display the same run speed, which is tendencially higher than under baseline conditions, i.e., in the presence of fructose. (C) After aversive training, there is no significant effect of training experience on run speed-modulation, and the pooled values of run speed-modulation are not different from zero (OSS, P > 0.05). That is, larvae do not modulate their run speed depending on their orientation to the odor. (D) After aversive training, there is no significant effect of training experience on run speed-modulation, and the pooled values of run speed-modulation are not different from zero (OSS, P > 0.05). Only with respect to run speed after aversive training did we find a significant difference across groups (P < 0.05, df = 2, N = 27, 56, 27, KW), which was, however, not associative in kind as both the reciprocally trained groups were affected. In all other cases, no significant difference across groups was detected (P > 0.05, df = 2, N = 27, 56, 27, KW). Significant between-group differences (MWU, P < 0.05 corrected according to Bonferroni–Holm) are indicated by lower case letters above the boxes.

Next, we tested whether larvae may modulate their run speed depending on their orientation to the odor source, and whether such modulation may differ due to training experience. Although after paired aversive and unpaired appetitive training, larvae tended to run faster while heading away from the odor than while heading toward it, whereas after unpaired aversive and paired appetitive training the opposite trend was observed (Fig. 3C,D), these trends were small and not significant. We conclude that modulations of run speed are no major determinants of conditioned aversion or conditioned approach.

In contrast to run speed, HC initiation was strongly modulated affected by associative experience. In both experimental groups displaying conditioned aversion (after paired aversive and unpaired appetitive training), the larvae tend to perform more HCs when heading toward the odor than when heading away from it, a behavior that upon the used convention shows as negative HC rate-modulation (Fig. 4A,B). Conversely, in both experimental groups displaying odor approach (after unpaired aversive and paired appetitive training), the larvae performed fewer HCs when heading toward the odor than when heading away, yielding a positive HC rate-modulation (Fig. 4A,B).

Figure 4.

Figure 4.

Open in a new tab

Analysis of head-casting. (A,B) show the HC rate-modulation and (C,D) the Reorientation per HC. (A) After paired aversive training, larvae tend to perform more HCs when heading toward the odor than away from it, yielding a slightly negative HC rate-modulation. After unpaired aversive training, in contrast, more HCs are performed when larvae head away from the odor, resulting in a positive HC rate-modulation. When tested in the absence of quinine, the larvae display an intermediate baseline HC rate-modulation. (B) HC rate-modulation is positive after paired appetitive training, and slightly negative after unpaired appetitive training. When tested in the presence of fructose, the larvae display an intermediate baseline HC rate-modulation. (C) After paired aversive training, larvae after an HC reorient themselves more away from the odor than toward it, indicated by negative reorientation values. After unpaired aversive training, in turn, larvae reorient themselves more toward the odor. When tested in the absence of quinine, the larvae display an intermediate baseline Reorientation per HC. (D) After paired appetitive training, larvae after an HC reorient themselves more toward the odor, whereas after unpaired appetitive training, they reorient themselves more away from the odor. When tested in the presence of fructose, the larvae display an intermediate baseline Reorientation per HC. All displayed features of chemotaxis differ across groups (P < 0.05, df = 2, N = 27, 56, 27, KW). Significant between-group differences (MWU, P < 0.05 corrected according to Bonferroni–Holm) are indicated with lower case letters above the boxes, significant differences to chance level (OSS, P < 0.05 corrected according to Bonferroni–Holm) are indicated by asterisks below the boxes.

A similar picture emerges regarding the direction of the HCs. In both groups displaying conditioned aversion (after paired aversive and unpaired appetitive training), the larvae reoriented themselves after a HC more likely away from the odor than toward (Fig. 4C,D). In both groups displaying odor approach (after unpaired aversive and paired appetitive training), in contrast, the larvae reoriented themselves after a HC more likely toward the odor than away from it (Fig. 4C,D).

Thus, avoiding an odor in order to escape from quinine and in order to search for sugar affected HC initiation and HC direction in the same way. These modulations were of opposite sign compared with the effects of conditioned approach.

Discussion

In this study, we investigated how two classes of odor aversion are microbehaviorally implemented in specific motor actions. In our experiments, larvae avoided an odor either because it predicted quinine that the larvae sought to escape from, or because it predicted the lack of sugar that the larvae were searching for (Fig. 2). Also in adult flies, vertebrates and humans, conditioned aversion and conditioned approach can have different causes: a cue can be avoided because it predicts a punishment, or the end, or a lack of a reward (sometimes referred to as “frustration learning”), and it can be approached because it predicts a reward, or the end of a punishment (called “relief learning”), or the lack of a punishment (called “safety learning”) (Rescorla and Wagner 1972; Dickinson and Dearing 1979; Gray 1987; Tanimoto et al. 2004; Rogan et al. 2005; Kim et al. 2006; Andreatta et al. 2010; Leknes et al. 2011; Navratilova et al. 2013; Marshall et al. 2014; Mohammadi et al. 2014; for review, see Seymour et al. 2007; Gerber et al. 2014; Kong et al. 2014). Although these different types of memories can be discriminated conceptually, the question arises to which extent the lack of a reward can be seen as just another kind of punishment, and likewise the lack of punishment as just another kind of reward. Our results show that the two classes of conditioned aversion observed in our experiments are strikingly alike—at the level of specific motor actions. In both cases larvae tend to perform more HCs when heading toward the odor than when heading away (Fig. 4A, left; Fig. 4B, right) and direct their HCs away from the odor rather than toward it (Fig. 4C, left; Fig. 4D, right). Likewise, the two classes of odor approach we observed share the same effects on chemotaxis, and are of opposite sign as compared with aversion (Fig. 4). From this perspective, the two classes of aversion are very similar.

On the other hand, as discussed earlier, the two classes of conditioned aversion serve different functions: either to escape from quinine or to search for sugar (Gerber and Hendel 2006; Schleyer et al. 2011). Importantly, they can be discriminated experimentally: larvae will avoid a quinine-predicting odor only in the presence of quinine (Fig. 2A). In contrast, larvae will avoid an odor that predicts the lack of sugar only in the absence of sugar (Fig. 2B). Thus, on a tasteless substrate that contains neither quinine nor sugar, larvae avoid an odor that predicts the lack of sugar, but they do not avoid an odor that predicts quinine (compare Fig. 2A″, middle box, and Fig. 2B″, right box; see also Schleyer et al. 2011). From this perspective, the two classes of aversion are clearly distinct.

To which extent are the neuronal substrates of aversive and appetitive memories in Drosophila shared or distinct? The current working hypothesis for associative olfactory learning in fruit flies and larvae locates the memory trace in the output synapses of the mushroom body Kenyon cells (for review, see Heisenberg 2003; Gerber et al. 2009; Diegelmann et al. 2013; Schleyer et al. 2013). The Kenyon cells receive on the one hand odor information, on the other hand reward and punishment signals. Thus, to this extent aversive and appetitive memories in Drosophila share the same circuit. Rewards (appetitive unconditioned stimuli in Pavlovian terminology) are signaled by a subset of dopaminergic neurons (Rohwedder et al. 2016), similar to the case in vertebrates (Schultz 2015). Whether dopaminergic neurons in Drosophila convey a prediction error signal rather than the reward signal per se as implicated by learning theory (Rescorla and Wagner 1972) and demonstrated in vertebrates (Schultz 2015) remains to be shown in Drosophila.

Punishments (aversive unconditioned stimuli in Pavlovian terminology) are signaled by another subset of dopaminergic neurons in the fruit fly (Schroll et al. 2006; Liu et al. 2012; Aso et al. 2014a,b; Rohwedder et al. 2016), which is controversial in vertebrates (Lammel et al. 2012; Schultz 2013). Thus, different sets of dopaminergic neurons were found to be necessary in Drosophila appetitive or aversive learning, respectively (e.g., Schroll et al. 2006; Liu et al. 2012; Aso et al. 2014a,b; Rohwedder et al. 2016). These different sets of dopaminergic neurons innervate clearly separated compartments of the mushroom body (Pauls et al. 2010; Aso et al. 2014a). From the mushroom body, appetitive and aversive memory information is signaled by different mushroom body output neurons, gathering input from different mushroom body compartments, toward motor control (Aso et al. 2014b). Thus, appetitive and aversive memories rely on different neuronal pathways on the level of dopaminergic neurons and mushroom body output neurons, and are likely to be summed up only downstream from the mushroom body output neurons (Fig. 5). Whether the same or different dopaminergic neurons, and the same or different mushroom body output neurons are involved in paired and unpaired aversive training (which could be interpreted as punishment and safety learning, respectively), remains unknown (the same applies to paired and unpaired appetitive training).

Figure 5.

Figure 5.

Open in a new tab

Working hypothesis for how conditioned odor aversion and approach come about. Both sweet and bitter substrates reduce run speed. Odor valence is formed by the sum of the odor's innate valence (which is usually positive in the case of larval Drosophila) and learned valence resulting from associative memory (see also Wystrach et al. 2016). Learned valence can be negative after paired aversive or unpaired appetitive training, and positive after unpaired aversive or paired appetitive training. Notably, signaling of learned valence after appetitive training (be it positive or negative) is blocked by the presence of the reward. Learned valence after aversive training, in contrast, is signaled only in the presence of the punishment. Signaling of innate valence is not affected by the presence of reward or punishment (see also Supplemental Fig S2A). After paired aversive or unpaired appetitive training, a negative learned valence is added to the innate valence of the odor, which may in sum lead to conditioned aversion. Consequently, more HCs are performed while heading toward the odor than while heading away, and HCs are directed away from rather than toward the odor. Notably, if the positive innate valence of an odor happens to be higher than the learned negative valence, larvae may not show net aversion (Schleyer et al. 2015b). After unpaired aversive or paired appetitive training, the summed innate and learned valences may lead to approach through a bias toward performing more HCs while heading away from the odor than while heading toward it. Moreover, HCs are directed toward rather than away from the odor.

The output neurons of the mushroom body thus are thought to code the learned valence of an odor. This learned valence then can be added to the odor's innate valence and accordingly shift behavior toward more approach or more aversion—depending on the kind of odor and the kind of memory associated with it (Fig. 5; Aso et al. 2014b; Owald et al. 2015; Schleyer et al. 2015b). After paired aversive and unpaired appetitive training, the learned valence is negative, shifting the behavior toward aversion. After paired appetitive and unpaired aversive training, in turn, the learned valence is positive, shifting the behavior toward approach.

Which individual mushroom body output neurons signal which type of learned valence, and how the integration of innate and learned valence is organized downstream from the mushroom body output neurons is the subject of on-going research. Such circuit-level understanding will give us insights into how a relatively simple nervous system tackles the tasks of approach and aversion, and hopefully also help us to understand how these processes are organized in more complex systems. Given that autonomous agents and robots also face the same crucial tasks of avoiding unfavorable situations and approaching desirable ones, the Drosophila larva could act as a model for tackling these issues.

Materials and Methods

Animals, reinforcement, and odors

Feeding-stage, third instar Drosophila melanogaster larvae five days after egg laying from Canton-S wild-type were used. The larvae were maintained on standard fly food at 25°C, 60% relative humidity, in a 12 h light–dark cycle. The experiments used Petri dishes of an inner diameter of 9 and 15 cm for training and test, respectively, which were prefilled with 1% agarose (electrophoresis grade; Roth) and stored at 4°C until used. 2 mol/L fructose (FRU; CAS: 57-48-7; purity 99%; Roth) or 5 mmol/L quinine hemisulfate (QUI; CAS: 6119-70-6; Sigma-Aldrich) were added to the agarose as reward and punishment, respectively. Please note that the terms “reward” and “punishment” are frequently used in Drosophila learning and memory literature instead of the Pavlovian terminology of “appetitive unconditioned stimulus” and “aversive unconditioned stimulus,” respectively.

As odor, n-amyl acetate (AM; CAS: 628-63-7; Merck) was diluted 1:20 in paraffin oil (CAS: 8012-95-1; Sigma-Aldrich) and presented in custom-made Teflon containers (5 mm diameter), covered by perforated lids.

Learning experiments

Associative learning experiments followed established protocols (Gerber and Hendel 2006; Saumweber et al. 2011; Schleyer et al. 2015a,b). For aversive training, one group of larvae was trained such that n-amyl acetate (AM) was presented simultaneously with the quinine substrate for 2.5 min, immediately followed by a blank trial on a Petri dish with empty Teflon containers (EM) and without quinine for another 2.5 min (paired training). In a second experimental group, the larvae were trained such that AM and quinine were presented on separate Petri dishes (unpaired training). This training cycle was performed three times. Across repetitions of the experiment, in half of the cases the odor-containing trial came first (AM + Quinine/EM and AM/EM + Quinine, respectively), in the other half of the cases the sequence was reversed (EM/AM + Quinine and EM + Quinine/AM, respectively). For the subsequent test, the animals were placed in the middle of a 15-cm-diameter Petri dish with an AM-loaded container on one side and an empty container on the other side in order to create a choice situation. Larval behavior was recorded using a camera and analyzed offline as described below.

For appetitive learning experiments, fructose was used instead of quinine. Analogous to the aversive protocol described above, odor and fructose were presented either paired or unpaired, and testing and recording were performed as mentioned.

The test Petri dishes may or may not contain a taste reinforcer as mentioned in the results. For all experiments, experimenters were blind with respect to the test conditions.

Data analysis

After the 3 min of test, we determined the number of animals on the odor side (#AM), the number on the no-odor side (#EM), and the number of larvae on a 1-cm-wide middle stripe (#middle). From this, we calculated the odor preference [−1; 1] as:

Preference=#AM−#EM#Total. (1)

During the test, we recorded larval behavior using a camera (Basler acA2040-90um). These videos were analyzed using custom-written analysis software based on the approach described in Schleyer et al. (2015b). The most salient change relative to that study was the use of head casts (HC) instead of turns to characterize chemotaxis. Larvae orient in odor gradients by a sequence of rather straight runs, HCs and turns (Fig. 1A). HCs are used to scan for local differences in odor concentration by laterally moving the head. An HC can either be rejected, resulting in another HC (usually in the opposite direction), or accepted, resulting in the body starting to run straight again. The body hence follows the new direction of the head; such an accepted HC is observed as a turn (Fig. 1). As such a turn is by definition preceded by an HC and the majority of HCs are in the same direction as the subsequent turn (Gomez-Marin et al. 2011), we decided to use exclusively HCs for the present analysis.

An HC was detected whenever the angular speed of the animal's head vector (Fig. 1B′) exceeded a threshold of 35°/sec and ended as soon as it dropped below that threshold again. If the angular speed of the tail vector (Fig. 1B″) at the same time exceeded a threshold of 45°/sec, this event was not regarded as an HC, but rather taken to indicate a rotation of the larval body. In accordance with previous work (Schleyer et al. 2015b), we took into account only HCs with an HC angle >20° (for a definition of the HC angle see below, for data on HCs with an HC angle <20° see Supplemental Fig. S3). The time when an animal was not head-casting was regarded as a run, discarding 1.5 sec before and after an HC to exclude decelerating and accelerating phases that usually happen before and after an HC, respectively.

We calculated the following variables:

  • Bearing angle toward odor: the orientation an animal's body relative to the odor source. 0° indicates that the odor is in front of the larva; positive and negative angles indicate that the odor is to the right or left, respectively; ±180° indicates that the odor is to the rear (Fig. 1C).

  • Heading angle toward odor: as bearing angle, but measuring the orientation of the animal's head relative to the odor source (Fig. 1C).

  • Run speed: the average speed (mm/sec) of the larval midpoint during runs. To quantify a potential modulatory effect of the bearing on run speed, we introduced “run speed-modulation” as the run speed while heading toward the odor minus the run speed while heading away from the odor, divided by the sum of both:
    Run speed-modulation=Run speed toward−Run speed awayRun speed toward+Run speed away. (2)

    Thus, if animals would modify their run speed such that they would speed up whenever they head away from the odor (absolute bearing angle >90°) and slow down whenever they head toward an odor (absolute bearing angle <90°) we would obtain a negative Run speed-modulation.

  • HC initiation: we measured a HC rate as the number of HCs (#HC) divided by the duration of time the larvae were tracked (T):
    HC rate(HC/s)=sum(#HC)sum(T). (3)
    For a display of the HC rate, see Supplemental Figure S4A,B. Animals that showed odor aversion had a high HC rate when they were heading toward the odor (absolute bearing angle <90°) and a lower HC rate when they were heading away from the odor (absolute bearing angle >90°). To quantify this modulatory effect of the bearing, we introduced “HC rate-modulation” as the HC rate while heading away from the odor source minus the HC rate while heading toward the source, divided by the sum of both:
    HC rate-modulation=HC rate away−HC rate towardHC rate away+HC rate toward (4)

    Thus, in the described example case of odor aversion (HC rate away < HC rate toward) we would obtain a negative HC rate-modulation.

  • HC direction: for each time point we determined an animal's bending angle (BA) as the angle between the tail vector and a vector from spine points 6–11 (Fig. 1D). Then, an HC angle was calculated as the difference in the bending angle before and after an HC:
    HC angle(°)=BA after HC−BA before HC. (5)

    For a display of the HC angle, see Supplemental Figure S4C,D.

In order to statistically compare the direction of HCs with respect to the odor source across groups, we determined the Reorientation per HC with respect to the odor. That is, we subtracted the angular deviation of the animal's head vector from the odor (as measured by the absolute value of the heading angle, HA, see Fig. 1C) after an HC from its angular deviation before the HC:

Reorientation per HC=abs(HA before HC)−abs(HA after HC). (6)

If an HC was directed away from the odor source, the heading angle after the HC would be higher than before, thus resulting in a negative reorientation value. The use of the heading angle instead of the bearing angle is warranted because the bearing angle after an HC changes only with some delay, see Fig. 1C″.

All these calculations were performed once per Petri dish. That is, in the data presented as box plots the sample size N equals the number of test Petri dishes, each containing approximately 20 animals.

To visualize how the different aspects of locomotion vary with both distance to the odor source and bearing toward odor we pooled the data from all Petri dishes for each experimental condition, applied a sliding box filter of ±60° and ±23 mm at each step (step width of 6° and 2.3 mm), and plotted the aggregated values of run speed (Supplemental Fig. S5), HC rate (Supplemental Fig. S6), and HC angle (Supplemental Fig. S7). It turned out that larval behavior differed with respect to the distance from the odor and that in particular aversion is mainly observed when the animals are relatively close to the odor (Supplemental Figs. S6, S7). Therefore we restricted all analyses (except Supplemental Figs. S5–S7) to animals near the odor source (distance <59 mm, which is half the maximal distance from the odor source).

Statistics and graphs

Nonparametric statistics (one-sample sign test, Kruskal–Wallis test, Mann–Whitney U-test; OSS, KW, MWU) were applied throughout the study, using Statistica (StatSoft) for the PC (for one-sample sign-tests we use a custom-written function for Excel [Microsoft]). When multiple comparisons were performed within one analysis, a Bonferroni–Holm correction was applied to keep the experiment-wide error rate below 5% (Holm 1979). When data are displayed as box plots, the middle line shows the median, the box boundaries the 25% and 75% quantiles, and the whiskers the 10% and 90% quantiles.

Ethics

Procedures comply with applicable law.

Data accessibility

Data sets were uploaded on a repository and will be made publicly available upon publication.

Competing interest statement

The authors declare no competing interests.

Supplementary Material

Supplemental Material
supp_24_5_191__index.html (1.6KB, html)

Acknowledgments

Experimental contributions of J. Saumweber and N. Schieting, as well as discussions with C. Eschbach, B. Gerber, S. Gläß, R. Glasgow, M. Louis, T. Saumweber, B. Webb, A. Yarali, and M. Zlatic, are gratefully acknowledged. This study received institutional support from the Leibniz Institute for Neurobiology (LIN) Magdeburg, the Leibniz Society, the Centre for Behavioural and Brain Sciences (CBBS) Magdeburg and the University of Magdeburg. We received project support from the European Commission (FP7-ICT project Miniature Insect Model for Active Learning [MINIMAL]) and the German Science Foundation (DFG) (GE 1091/4-1, SFB 779 Motivated behaviour).

Authors’ contributions: A.R. and M.S. performed the experiments, M.S. and Em.P. designed the study, interpreted the results, and wrote the paper, Em.P. and Ev.P. wrote the tracking and analysis software. All authors gave final approval for publication.

Footnotes

[Supplemental material is available for this article.]

Freely available online through the Learning & Memory Open Access option.

References

  1. Andreatta M, Muhlberger A, Yarali A, Gerber B, Pauli P. 2010. A rift between implicit and explicit conditioned valence in human pain relief learning. Proc Biol Sci 277: 2411–2416. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Apostolopoulou AA, Hersperger F, Mazija L, Widmann A, Wüst A, Thum AS. 2014. Composition of agarose substrate affects behavioral output of Drosophila larvae. Front Behav Neurosci 8: 11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aso Y, Hattori D, Yu Y, Johnston RM, Iyer NA, Ngo TT, Dionne H, Abbott LF, Axel R, Tanimoto H, et al. 2014a. The neuronal architecture of the mushroom body provides a logic for associative learning. Elife 3: e04577. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aso Y, Sitaraman D, Ichinose T, Kaun KR, Vogt K, Belliart-Guérin G, Plaçais PY, Robie AA, Yamagata N, Schnaitmann C, et al. 2014b. Mushroom body output neurons encode valence and guide memory-based action selection in Drosophila. Elife 3: e04580. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Berck ME, Khandelwal A, Claus L, Hernandez-Nunez L, Si G, Tabone CJ, Li F, Truman JW, Fetter RD, Louis M, et al. 2016. The wiring diagram of a glomerular olfactory system. Elife 5: e14859. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Cobb M. 1999. What and how do maggots smell? Biol Rev 74: 425–459. [Google Scholar]
  7. Dickinson A, Dearing MF. 1979. Appetitive-aversive interactions and inhibitory processes In Mechanisms of learning and motivation (ed. Dickinson A, Boakes RA), pp. 203–231. Erlbaum, Hillsdale, NJ. [Google Scholar]
  8. Diegelmann S, Klagges B, Michels B, Schleyer M, Gerber B. 2013. Maggot learning and synapsin function. J Exp Biol 216: 939–951. [DOI] [PubMed] [Google Scholar]
  9. El-Keredy A, Schleyer M, König C, Ekim A, Gerber B. 2012. Behavioural analyses of quinine processing in choice, feeding and learning of larval Drosophila. PLoS One 7: e40525. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Fraenkel GS, Gunn DL. 1961. The orientation of animals. Dover Publications, New York. [Google Scholar]
  11. Fushiki A, Zwart MF, Kohsaka H, Fetter RD, Cardona A, Nose A. 2016. A circuit mechanism for the propagation of waves of muscle contraction in Drosophila. Elife 5: e13253. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gerber B, Hendel T. 2006. Outcome expectations drive learned behaviour in larval Drosophila. Proc Biol Sci 273: 2965–2968. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Gerber B, Stocker RF, Tanimura T, Thum AS. 2009. Smelling, tasting, learning: Drosophila as a study case. Results Probl Cell Differ 47: 139–185. [DOI] [PubMed] [Google Scholar]
  14. Gerber B, Yarali A, Diegelmann S, Wotjak CT, Pauli P, Fendt M. 2014. Pain-relief learning in flies, rats, and man: basic research and applied perspectives. Learn Mem 21: 232–252. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gershow M, Berck M, Mathew D, Luo L, Kane EA, Carlson JR, Samuel AD. 2012. Controlling airborne cues to study small animal navigation. Nat Methods 9: 290–296. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Gomez-Marin A, Louis M. 2012. Active sensation during orientation behavior in the Drosophila larva: more sense than luck. Curr Opin Neurobiol 22: 208–215. [DOI] [PubMed] [Google Scholar]
  17. Gomez-Marin A, Louis M. 2014. Multilevel control of run orientation in Drosophila larval chemotaxis. Front Behav Neurosci 8: 38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gomez-Marin A, Stephens GJ, Louis M. 2011. Active sampling and decision making in Drosophila chemotaxis. Nat Commun 2: 441. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gray JA. 1987. The psychology of fear and stress. Cambridge University Press, Cambridge. [Google Scholar]
  20. Heisenberg M. 2003. Mushroom body memoir: from maps to models. Nat Rev Neurosci 4: 266–275. [DOI] [PubMed] [Google Scholar]
  21. Hendel T, Michels B, Neuser K, Schipanski A, Kaun K, Sokolowski MB, Marohn F, Michel R, Heisenberg M, Gerber B. 2005. The carrot, not the stick: appetitive rather than aversive gustatory stimuli support associative olfactory learning in individually assayed Drosophila larvae. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 191: 265–279. [DOI] [PubMed] [Google Scholar]
  22. Holm S. 1979. A simple sequentially rejective multiple test procedure. Scand J Stat 6: 65–70. [Google Scholar]
  23. Kim H, Shimojo S, O'Doherty JP. 2006. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol 4: e233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Kong E, Monje FJ, Hirsch J, Pollak DD. 2014. Learning not to fear: neural correlates of learned safety. Neuropsychopharmacology 39: 515–527. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lahiri S, Shen K, Klein M, Tang A, Kane E, Gershow M, Garrity P, Samuel AD. 2011. Two alternating motor programs drive navigation in Drosophila larva. PLoS One 6: e23180. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC. 2012. Input-specific control of reward and aversion in the ventral tegmental area. Nature 491: 212–217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Leknes S, Lee M, Berna C, Andersson J, Tracey I. 2011. Relief as a reward: hedonic and neural responses to safety from pain. PLoS One 6: e17870. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Liu C, Placais PY, Yamagata N, Pfeiffer BD, Aso Y, Friedrich AB, Siwanowicz I, Rubin GM, Preat T, Tanimoto H. 2012. A subset of dopamine neurons signals reward for odour memory in Drosophila. Nature 488: 512–516. [DOI] [PubMed] [Google Scholar]
  29. Louis M, Piccinotti S, Vosshall LB. 2008. High-resolution measurement of odor-driven behavior in Drosophila larvae. J Vis Exp. doi: 10.3791/638. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Marshall AJ, Acheson DT, Risbrough VB, Straus LD, Drummond SP. 2014. Fear conditioning, safety learning, and sleep in humans. J Neurosci 34: 11754–11760. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mohammadi M, Bergado-Acosta JR, Fendt M. 2014. Relief learning is distinguished from safety learning by the requirement of the nucleus accumbens. Behav Brain Res 272: 40–45. [DOI] [PubMed] [Google Scholar]
  32. Navratilova E, Xie JY, King T, Porreca F. 2013. Evaluation of reward from pain relief. Ann N Y Acad Sci 1282: 1–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Owald D, Felsenberg J, Talbot CB, Das G, Perisse E, Huetteroth W, Waddell S. 2015. Activity of defined mushroom body output neurons underlies learned olfactory behavior in Drosophila. Neuron 86: 417–427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Pauls D, Selcho M, Gendre N, Stocker RF, Thum AS. 2010. Drosophila larvae establish appetitive olfactory memories via mushroom body neurons of embryonic origin. J Neurosci 30: 10655–10666. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Rescorla RA, Wagner AR. 1972. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement In Classical conditioning II: current research and theory (ed. Black AH, Prokasy WF), pp. 64–99. Appleton-Century-Crofts, New York. [Google Scholar]
  36. Rogan MT, Leon KS, Perez DL, Kandel ER. 2005. Distinct neural signatures for safety and danger in the amygdala and striatum of the mouse. Neuron 46: 309–320. [DOI] [PubMed] [Google Scholar]
  37. Rohwedder A, Wenz NL, Stehle B, Huser A, Yamagata N, Zlatic M, Truman JW, Tanimoto H, Saumweber T, Gerber B, et al. 2016. Four individually identified paired dopamine neurons signal reward in larval Drosophila. Curr Biol 26: 661–669. [DOI] [PubMed] [Google Scholar]
  38. Saumweber T, Husse J, Gerber B. 2011. Innate attractiveness and associative learnability of odors can be dissociated in larval Drosophila. Chem Senses 36: 223–235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Scherer S, Stocker RF, Gerber B. 2003. Olfactory learning in individually assayed Drosophila larvae. Learn Mem 10: 217–225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Schleyer M, Saumweber T, Nahrendorf W, Fischer B, von Alpen D, Pauls D, Thum A, Gerber B. 2011. A behavior-based circuit model of how outcome expectations organize learned behavior in larval Drosophila. Learn Mem 18: 639–653. [DOI] [PubMed] [Google Scholar]
  41. Schleyer M, Diegelmann S, Michels B, Saumweber T, Gerber B. 2013. ‘Decision-making’ in larval Drosophila In Invertebrate learning and memory (ed. Menzel R, Benjamin P), pp. 41–55. Elsevier, München. [Google Scholar]
  42. Schleyer M, Miura D, Tanimura T, Gerber B. 2015a. Learning the specific quality of taste reinforcement in larval Drosophila. Elife 4: e04711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Schleyer M, Reid SF, Pamir E, Saumweber T, Paisios E, Davies A, Gerber B, Louis M. 2015b. The impact of odor-reward memory on chemotaxis in larval Drosophila. Learn Mem 22: 267–277. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Schroll C, Riemensperger T, Bucher D, Ehmer J, Voller T, Erbguth K, Gerber B, Hendel T, Nagel G, Buchner E, et al. 2006. Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae. Curr Biol 16: 1741–1747. [DOI] [PubMed] [Google Scholar]
  45. Schultz W. 2013. Updating dopamine reward signals. Curr Opin Neurobiol 23: 229–238. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Schultz W. 2015. Neuronal reward and decision signals: from theories to data. Physiol Rev 95: 853–951. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Schulze A, Gomez-Marin A, Rajendran VG, Lott G, Musy M, Ahammad P, Deogade A, Sharpe J, Riedl J, Jarriault D, et al. 2015. Dynamical feature extraction at the sensory periphery guides chemotaxis. Elife 4 10.7554/eLife.06694 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Seymour B, Singer T, Dolan R. 2007. The neurobiology of punishment. Nat Rev Neurosci 8: 300–311. [DOI] [PubMed] [Google Scholar]
  49. Tanimoto H, Heisenberg M, Gerber B. 2004. Experimental psychology: event timing turns punishment to reward. Nature 430: 983. [DOI] [PubMed] [Google Scholar]
  50. Tempel BL, Bonini N, Dawson DR, Quinn WG. 1983. Reward learning in normal and mutant Drosophila. Proc Natl Acad Sci 80: 1482–1486. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wystrach A, Lagogiannis K, Webb B. 2016. Continuous lateral oscillations as a core mechanism for taxis in Drosophila larvae. Elife 5: e15504. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material
supp_24_5_191__index.html (1.6KB, html)
supp_24.5.191_Supplemental_Figure_Legends.docx (124.6KB, docx)
supp_24.5.191_Supplemental_FigureS7.tif (32.8MB, tif)
supp_24.5.191_Supplemental_FigureS8.tif (28.7MB, tif)
Download video file (2.5MB, mp4)
supp_24.5.191_Supplemental_FigureS1.tif (35.8MB, tif)
supp_24.5.191_Supplemental_FigureS4.tif (20.5MB, tif)
supp_24.5.191_Supplemental_FigureS2.tif (26.4MB, tif)
supp_24.5.191_Supplemental_FigureS5.tif (32.9MB, tif)
supp_24.5.191_Supplemental_FigureS3.tif (30.2MB, tif)
supp_24.5.191_Supplemental_FigureS6.tif (32.9MB, tif)

Data Availability Statement

Data sets were uploaded on a repository and will be made publicly available upon publication.

Articles from Learning & Memory are provided here courtesy of Cold Spring Harbor Laboratory Press

RESOURCES