Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Feb 11.
Published in final edited form as: Cell. 2009 Oct 16;139(2):405–415. doi: 10.1016/j.cell.2009.08.034

WRITING MEMORIES WITH LIGHT-ADDRESSABLE REINFORCEMENT CIRCUITRY

Adam Claridge-Chang 1,3, Robert D Roorda 1, Eleftheria Vrontou 1, Lucas Sjulson 1,4, Haiyan Li 2,5, Jay Hirsh 2, Gero Miesenböck 1,*
PMCID: PMC3920284  NIHMSID: NIHMS144154  PMID: 19837039

Abstract

Dopaminergic neurons are thought to drive learning by signaling changes in the expectations of salient events, such as rewards or punishments. Olfactory conditioning in Drosophila requires direct dopamine action on intrinsic mushroom body neurons, the likely storage sites of olfactory memories. Neither the cellular sources of the conditioning dopamine nor its precise postsynaptic targets are known. By optically controlling genetically circumscribed subsets of dopaminergic neurons in the behaving fly, we have mapped the origin of aversive reinforcement signals to the PPL1 cluster of 12 dopaminergic cells. PPL1 projections target restricted domains in the vertical lobes and heel of the mushroom body. Artificially evoked activity in a small number of identifiable cells thus suffices for programming behaviorally meaningful memories. The delineation of core reinforcement circuitry is an essential first step in dissecting the neural mechanisms that compute and represent valuations, store associations, and guide actions.

INTRODUCTION

Having to decide, moment by moment, what to do next is the price of motility. Mobile agents must continuously evaluate their circumstances and choose actions based on predicted consequences. “Intelligence” subserving these decisions is found even in the simplest organisms. The flagellar motors of E. coli, for example, are coupled to chemosensors via a biochemical circuit that enables the bacterium to chase nutrients and avoid toxins (Berg, 2004).

Fruit flies, too, are attracted to some chemicals and repelled by others. But in contrast to E. coli’s hardwired responses, a fly’s reactions to chemical signals are plastic: the valence of most odors is neither innate nor invariant but influenced by experience. When a scent is paired repeatedly with electric foot shock, it acquires persistent negative valence—an aversive memory is formed (Quinn et al., 1974; Tully and Quinn, 1985). Omitting electric shocks during further odor presentations gradually restores the odor’s original hedonic valence—the aversive memory is extinguished (Quinn et al., 1974; Tully and Quinn, 1985). The fly thus keeps a record of its experience, which it uses to inform its actions.

Olfactory-driven action choices in Drosophila require two brain centers, the lateral protocerebrum and the mushroom body. Of these, only the mushroom body has been implicated in response plasticity (Heisenberg et al., 1985; de Belle and Heisenberg, 1994; Zars et al., 2000; McGuire et al., 2003). Olfactory learning depends acutely on cyclic AMP (cAMP) signaling in the intrinsic mushroom body neurons (Zars et al., 2000; McGuire et al., 2003), called Kenyon cells (KCs).

An increase in cAMP levels coincident with odor-evoked activity is thought to modify KC output synapses (Dubnau et al., 2001; McGuire et al., 2001; Heisenberg, 2003; Davis, 2005; Keene and Waddell, 2007). The calcium/calmodulindependent adenylyl cyclase encoded by the rutabaga gene (Livingstone et al., 1984; Levin et al., 1992) has been proposed to function as a logic gate integrating sensory and reinforcement signals (Levin et al., 1992; Heisenberg, 2003; Davis, 2005; Keene and Waddell, 2007). cAMP production by this enzyme is thought to be regulated (Abrams et al., 1991; Levin et al., 1992) by calcium influx (due to odor-evoked KC depolarization) AND coupling to active GS protein [due to reinforcing dopamine action (Schwaerzel et al., 2003) on receptors expressed by KCs (Kim et al., 2007)].

Dopamine’s role as a putative aversive reinforcer in fly olfactory learning mirrors, with reversed polarity, its rewarding role in mammals (Wise and Rompre, 1989; Schultz et al., 1997). While the evidence implicating dopamine as an aversive reinforcement signal in Drosophila is substantial (Schwaerzel et al., 2003; Schroll et al., 2006; Kim et al., 2007), many mechanistic questions remain. First, the sources of the conditioning dopamine among the 200–300 dopaminergic neurons in the central fly brain (Budnik and White, 1988) are undefined; dopaminergic neurons might even communicate by volume transmission, not the localized activity of specific synapses. Second, although experiments in larvae point to an instructive role in memory formation (Schroll et al., 2006), it is unconfirmed whether dopamine acts in an instructive or merely permissive capacity in the much better characterized adult olfactory system. Third, neither the synaptic targets of dopaminergic projections in the mushroom body nor the effects of dopamine on the physiology of these cells are known. Here, we show that genetically targeted optical activation (Zemelman et al., 2002; Lima and Miesenböck, 2005; Sjulson and Miesenböck, 2008) of dopaminergic neurons is, in itself, sufficient for writing aversive olfactory memories. The origin of the conditioning dopamine is not the entire population of dopaminergic neurons but a specific cluster of 12 cells. The axonal projections of these neurons target exclusively mushroom body neurites in the vertical lobes and heel. Aversive dopamine signals thus act on restricted domains within a compartmentalized memory system.

RESULTS

Drosophila’s ability to learn and remember has been extensively probed by training and analyzing groups of flies in the olfactory T-maze (Quinn et al., 1974; Tully and Quinn, 1985). Though statistically powerful, this population assay suffers from several disadvantages: it is blind to individual behavioral variation and its physiological causes; it may report collective influences on individual decision-making (Quinn et al., 1974; Couzin, 2009); and it does not allow the animal’s behavior to control the rate and timing of reinforcement. To overcome these drawbacks, we designed an assay in which the odor choices of single flies could be monitored and altered. Olfactory preference was measured by tracking a fly's movements in two air streams converging from opposite ends of a narrow 50-mm chamber (Figures S1–S3). Flies paced the full length of the chamber in the absence of added odors but restricted their movements according to preference when odors were introduced (Figure 1). Untrained odor preferences varied widely among individuals but fluctuated little when the same flies were tested repeatedly (p < 0.0001, permutation test; Figures 1A and 2A); untrained preferences are thus individual invariants.

Figure 1. Pavlovian Olfactory Conditioning.

Figure 1

Open in a new tab

(A and B) Positions in a behavioral chamber (horizontal dimension) as a function of time (vertical dimension) of 20 Canton-S flies choosing between MCH (blue) and OCT (orange). The traces are sorted by untrained preference. Bar graphs on the right indicate population averages of decisions in favor of the left and right chamber halves before and after conditioning (pre and post), in the presence of odors (colored bars) or air (white and gray bars).

(A) Mock conditioning without electric shock preserves individual pre-training preferences.

(B) Pairing the presentation of MCH with electric shock causes conditioned avoidance of MCH.

Figure 2. Performance of Individually Trained Flies.

Figure 2

Open in a new tab

(A) Percentage of time spent in OCT before and after mock-conditioning (left) and Pavlovian training against MCH (right). **, p < 0.0001; permutation test. The graphs summarize data from Figures 1A and 1B; black lines connect data points corresponding to the same individual.

(B) Canton-S flies trained against MCH (column b) or OCT (column d) avoid the shock-associated odor, but trained rutabaga2080 flies (column c) perform indistinguishably from mock-conditioned Canton-S animals (column a). p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from mock-conditioned controls in post-hoc comparison (n = 20 flies per condition; means ± SEM).

(C) Persistence of memory (n = 20 flies per timepoint; means ± SEM).

Individual odor preferences, however, could be modified by Pavlovian shock conditioning (Figure 1B). Training cycles consisted of two epochs in random order: a 1-min presentation of 3-octanol (OCT) without shock, and a 1-min presentation of 4-methylcyclohexanol (MCH) with twelve 60-VDC electric shocks (Tully and Quinn, 1985). Two such training cycles caused profound changes in behavior: animals exhibiting an untrained bias in favor of MCH reversed their versed their preference; animals lacking an untrained MCH bias acquired a preference for OCT (Figures 1B and 2A). Across individuals, the rank order of preference was preserved even after training (p = 0.03, permutation test; Figure 2A), suggesting that olfactory conditioning operates on top of the individually variable base valence of a scent.

Individually trained flies exhibited many of the characteristics seen in population learning (Tully and Quinn, 1985). First, aversive memories could be formed against both MCH and OCT (Figure 2B, columns b and d), yielding an approximate performance index [PI (Quinn et al., 1974; Tully and Quinn, 1985)] of 0.68 (half-PIs 0.54 for anti-OCT learning and 0.81 for anti-MCH learning). Second, flies carrying the rutabaga2080 mutation, which impairs a calcium/ calmodulin-dependent adenylyl cyclase (Livingstone et al., 1984; Levin et al., 1992), failed to learn (Figure 2B, column c). Third, memories persisted for at least 3 h (Figure 2C).

In contrast to populations in a T-maze, individuals could also be trained under automated closed-loop conditions. Here, a fly’s actions—its entry into and exit from one of two odor streams—controlled the delivery of electric shock (Figures 3A and 3B). Learning in this regime is action-contingent but distinct from pure operant conditioning in the strictest sense (Brembs and Heisenberg, 2000; Brembs and Plendl, 2008), as the animal is guided by predictive sensory cues and subjected to sustained, quasi-Pavlovian punishment if it lingers in the reinforced odor. Action-contingent olfactory learning depended, like Pavlovian olfactory conditioning (Figure 2B) and some forms of visually cued learning (Liu et al., 2006; Brembs and Plendl, 2008), on a functional rutabaga gene (Figure 3C, column b), but it required less reinforcement to achieve the same level of performance as Pavlovian training (Figure 3D). This “operant advantage” puts a nuance on the prevalent model of olfactory learning, which proposes that aversive conditioning is entirely a result of coincident inputs to KCs from the sensory and reinforcement pathways. That action-contingent training yields better performance despite fewer coincident stimuli (Figure 3D) suggests that factors other than temporal coincidences have a significant effect. One possibility is that stimulus and action representations are bound together more tightly, facilitating correct action selection during recall (Brembs and Heisenberg, 2000; Gallistel and Gibbon, 2000).

Figure 3. Action-Contingent Olfactory Conditioning.

Figure 3

Open in a new tab

(A) Positions in a behavioral chamber (horizontal dimension) as a function of time (vertical dimension) of 20 Canton-S flies choosing between MCH (blue) and OCT (orange). The traces are sorted by untrained preference. During four 1-min training periods, entries into MCH are punished by electric shock. Bar graphs on the right indicate population averages of decisions in favor of the left and right chamber halves before and after conditioning (pre and post), in the presence of odors (colored bars) or air (white and gray bars). The MAT-file used to generate this figure can be downloaded for further analysis (Data S1).

(B) Locomotion traces at an expanded scale of 10 individuals (see corresponding numbers in A) during four epochs of action-contingent conditioning. A red tick mark to the left of a trace indicates the delivery of one electric shock. Animals receive 2–17 shocks during training; the selected individuals represent the minimum (trace 0) and 9 deciles (traces 1–9) in the frequency distribution of shock consumption. Fast learners (traces 0–4) tend to consume reinforcement only during the first two training epochs, whereas slow learners (traces 5–9) are reinforced throughout.

(C) Effective conditioning requires a functional rutabaga gene product (column b) and contingency between olfactory choice behavior and electric shock; learning does not occur when this contingency is broken by randomizing reinforcement (column c). p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from Canton-S animals in post-hoc comparison (n = 20 flies per condition; means ± SEM).

(D) Comparison of the performance of Canton-S flies after 2 and 4 min of Pavlovian and action-contingent training, normalized to the number of electric shocks consumed. p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from 2 min of Pavlovian conditioning in post-hoc comparison (n = 20 flies per condition; means ± SEM).

Optical Implantation of Memory

A forceful extension of these experiments is to replace the external reinforcer— electric foot shock—with direct manipulations of the brain’s internal valuation systems. If electric foot shock drives olfactory learning through a final common path of dopaminergic inputs to the mushroom bodies, it should be possible to write memories directly by activating these inputs in the presence of odors. To test whether this would result in conditioned odor avoidance, a fly’s entry into one odor stream was used to trigger a laser pulse that opened, via release of ATP from a previously microinjected caged precursor, ATP-gated P2X2 channels expressed selectively in circumscribed groups of cells (Zemelman et al., 2003; Lima and Miesenböck, 2005).

When targeted optical activation of dopaminergic neurons in TH-GAL4:UASP2X2 flies (Friggi-Grelin et al., 2003; Lima and Miesenböck, 2005) was made contingent upon entry into one odor, an aversive memory specific to that odor indeed formed (Figures 4 and 5A, columns a and b). The performance of flies instructed via light-evoked dopamine release matched that of animals trained conventionally via electric shock (Figure 5A). The high efficacy of dopaminergic instruction, combined with the absence of short-term memory when dopaminergic synapses are blocked during training (Schwaerzel et al., 2003), argues that signaling from dopaminergic neurons is the primary means by which aversive odor associations are stored.

Figure 4. Optical Implantation of Memory.

Figure 4

Open in a new tab

(A) Examples of conditioned odor avoidance in TH-GAL4:UAS-P2X2 flies after genetically targeted photostimulation of dopaminergic neurons. Positions in a behavioral chamber (horizontal dimension) as a function of time (vertical dimension) of 10 flies choosing between MCH (blue) and OCT (orange). The traces are sorted by untrained preference. During four 1-min training periods, entries into MCH activate 10-ms laser pulses. Laser pulses are repeated at 0.2 Hz while the fly remains in the reinforcement zone. Note the conditioned avoidance of MCH (blue) after training.

(B) Bar graphs indicate population averages (n = 68 flies) of decisions in favor of the left and right chamber halves before and after conditioning (pre and post), in the presence of odors (colored bars) or air (white and gray bars).

Figure 5. Sources of Dopaminergic Reinforcement Signals.

Figure 5

Open in a new tab

(A) Action-contingent photoactivation of P2X2 in dopaminergic neurons under TH-GAL4 control produces conditioned odor avoidance (columns a and b). Optically reinforced flies achieve the same level of performance as animals trained conventionally via electric shock (horizontal shaded band; mean ± SEM). Effective conditioning requires a functional rutabaga gene product (column c) and contingency between olfactory choice behavior and optically evoked dopamine release; learning does not occur when this contingency is broken by randomizing reinforcement (column d). Activation of P2X2 in dopaminergic neurons under HL9-GAL4 control (column e) or ATP uncaging in flies lacking P2X2 expression (columns f–h) are equally ineffective. p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from electric shock conditioning in post-hoc comparison (n = 20–68 flies per condition; means ± SEM).

(B) Temperature-induced expression of Kir2.1 in dopaminergic neurons under TH-GAL4 control (dark gray columns), but not under HL9-GAL4 control (medium gray columns), blocks action-contingent conditioning (column b). p = 0.0062; Kruskal-Wallis ANOVA; **, significantly different from permissive temperature in post-hoc comparison (n = 19–58 flies per condition; means ± SEM).

(C) Temperature-induced expression of Kir2.1 in dopaminergic neurons, under either TH-GAL4 control (dark gray columns) or HL9-GAL4 control (medium gray columns), inhibits locomotion (columns b and d). p < 0.0001; Kruskal-Wallis ANOVA; **, significantly different from permissive temperature in post-hoc comparison (n = 82–120 flies per condition; means ± SEM).

Common molecular and timing requirements for conditioning suggest that reinforcement via light-evoked dopamine release and electric shock draw on the same neural mechanisms: both forms of learning depend on the adenylyl cyclase encoded by the rutabaga gene (Figures 3C, column b, and 5A, column c), and both require that the delivery of the optical or electrical reinforcer follows the reinforced olfactory choice behavior in time (Figures 3C, column c, and 5A, column d). When this contingency was broken (by delivering the same average number of 20.3 laser pulses that are consumed during 4 min of effective training, but at random times rather than in an action-contingent fashion), olfactory preference remained unchanged (Figure 5A, column d).

Pairing an odor with photorelease of ATP in the absence of P2X2 expression, or with laser illumination alone (not shown), induced only minimal changes in preference (Figure 5A, columns f–h). These changes are in all likelihood due to a small intrinsically aversive effect of intense ultraviolet irradiation.

Sources of Dopaminergic Reinforcement Signals

The dual roles of dopaminergic neurons in regulating movement (Lima and Miesenböck, 2005) and providing reinforcement (Figures 4 and 5) suggest a functional subdivision of the dopaminergic system. It is not known how this division of labor is realized at the level of individual cells. Rendering subsets of dopaminergic neurons light-addressable opens a route into this problem: if function segregates along anatomical boundaries, it should be possible genetically to separate reinforcement from locomotor circuits.

The dopa decarboxylase enhancer fragment carried by the HL9-GAL4 line marks a subset of dopaminergic neurons that differs from TH-GAL4 (Figure 6 and Table S1). Whereas TH-GAL4 runs in six of the 7 paired dopaminergic clusters (Budnik and White, 1988) in the central fly brain (the exception being the paired anterior medial cluster, PAM; Figures 6A and 6C and Table S1), HL9-GAL4 shows majority coverage of PAM neurons (Figures 6B and 6D), but minority or no expression in both paired posterior lateral clusters (PPL1, PPL2) and two of the three paired posterior medial clusters (PPM1, PPM3) (Figures 6E and 6F and Table S1). Projections from TH-GAL4 and HL9-GAL4 neurons appear to target preferentially the vertical and horizontal mushroom body lobes, respectively (Figures 6G and 6H; Movies S1 and S2).

Figure 6. Anatomy of Two Functionally Distinct Sets of Dopaminergic Neurons.

Figure 6

Open in a new tab

(A and B) TH-GAL4 (top row) and HL9-GAL4 (bottom row) mark distinct but partially overlapping clusters of dopaminergic neurons.

(C to F) Maximum intensity projections of confocal sections reveal 7 paired neuronal clusters expressing tyrosine hydroxylase in the central brain (red pie charts in A and B, cell numbers in parentheses; see Table S1 for statistics); the fractions of neurons co-expressing mCD8-GFP in the two GAL4 lines are indicated in green. Neuropil was stained with nc82 antibodies (blue), dopaminergic neurons with antibodies against tyrosine hydroxylase (red), and mCD8-GFPexpressing neurons with antibodies against mCD8 (green). Scale bar, 50 µm. (G and H) Maximum intensity projections of confocal sections through the mushroom body. KCs express the mb247-DsRed transgene; dopaminergic projections are labeled by mCD8-GFP. Scale bar, 10 µm.

Dopa decarboxylase is also expressed in serotonergic (5-HT) cells (Johnson et al., 1989). To exclude a confound due to serotonergic neurons that might potentially be captured by one or another of the GAL4 lines used in our experiments, we increased 5-HT levels ~11-fold above baseline by feeding the 5-HT precursor 5-hydroxytryptophan (5-HTP; 3 days of 5-HTP treatment raised the mean 5-HT content per head from 0.20 to 2.27 pmol). Tests of olfactory memory revealed identical performance of 5-HTP treated and untreated animals (avoidance change 58.16 ± 4.22 vs. 62.77 ± 3.45 % in the presence and absence of 5-HTP, respectively; n = 99–105 flies per condition; means ± SEM; p = 0.20, permutation test). Dramatic changes in 5-HT levels thus have no effect on aversive olfactory conditioning, allowing us to ascribe driver line effects on learning to differential transgene expression in subsets of dopaminergic neurons.

When neurons labeled by either the TH-GAL4 or the HL9-GAL4 driver were silenced by inducible overexpression (McGuire et al., 2003) of the inwardly rectifying potassium channel Kir2.1 (Baines et al., 2001), spontaneous locomotor activity dropped to <25 % of baseline (Figure 5C, columns b and d). In striking contrast to these equally pronounced locomotor effects, activity in TH-GAL4 neurons, but not in HL9-GAL4 neurons, was necessary (Figure 5B, compare columns b and d) and sufficient (Figure 5A, compare columns a and e) for instructing aversive memories. The two driver lines thus differentiate between two subsets of dopaminergic neurons: TH-GAL4 labels cells involved in both locomotor control and reinforcement, whereas HL9-GAL4 labels only cells involved in locomotor control.

Because HL9-GAL4 drives expression in more dopaminergic neurons than THGAL4 (76 vs. 51 cells; Figure 6 and Table S1), manipulations of HL9-GAL4 neurons are expected to have a larger impact on extracellular dopamine levels than manipulations of TH-GAL4 neurons. Yet neither photostimulation (Figure 5A, column e) nor silencing (Figure 5B, column d) of all HL9-GAL4 cells had a detectable effect on memory. Reinforcement is thus not due to volume transmission of dopamine; rather, it requires a specific circuit formed by a specific subset of neurons that is missing in HL9-GAL4.

Targets of Dopaminergic Reinforcement Signals

Which of the four dopaminergic cell clusters captured by TH-GAL4 but absent from HL9-GAL4 (PPL1, PPL2, PPM1, and PPM3; Figure 6 and Table S1) are part of this circuit? Because dopamine must act directly on receptors expressed by KCs of the mushroom body to provide reinforcement (Kim et al., 2007), we are able to constrain the four candidate clusters further by their projection patterns. To highlight individual cell clusters, dopaminergic neurons were biosynthetically loaded with a photoactivatable variant of GFP (PA-GFP; (Patterson and Lippincott-Schwartz, 2002), which was subsequently switched on in identified somata by two-photon photoconversion (Datta et al., 2008). The fluorescent marker filled the illuminated neurons by diffusion, permitting their arborizations to be visualized against DsRed-counterstained (Riemensperger et al., 2005) mushroom bodies.

Of the four candidate clusters present in the TH-GAL4 line, only PPL1 neurons were found to target the mushroom body lobes (Figure 7), forcing the conclusion that this cluster of 12 cells contains the source of aversive reinforcement. PPM1 and PPM2 neurons, which were not discriminated further, ramify extensively in a region posterior to the mushroom bodies (Figure 7E); PPM3 neurons innervate the central complex (Figure 7F); and PPL2 neurons elaborate two principal branches: one extending into undefined synaptic neuropil posterior to the mushroom body, and another traced to the vicinity of the PPL1 cluster (Figure 7C).

Figure 7. Projections of Dopaminergic Neurons.

Figure 7

Open in a new tab

Three-dimensional reconstructions of PA-GFP-labeled dopaminergic arborizations (green) and mb247-DsRed expressing KCs (gray). The expression of PAGFP is controlled by TH-GAL4 (A, C to F) or HL9-GAL4 (B). Only PPL1 neurons (A, captured by TH-GAL4) and PAM neurons (B, captured by HL9-GAL4) innervate the mushroom body lobes. The two cell clusters target the vertical and horizontal lobes, respectively. In each panel, a right-handed coordinate system near the left mushroom body indicates the anterior (A), lateral (L), and dorsal (D) directions.

PPL1 somata sit immediately lateral of the mushroom body calyx and project a bundle of neurites medially between peduncle and vertical lobe (Figure 7A). Here, the neurons elaborate dense (possibly dendritic) ramifications that remain confined to the same hemisphere. Long-range fibers extend bilaterally to the tips and stalks of the vertical (α and α’) lobes, the heel, and the peduncle of the mushroom body and arborize within these regions (Figure 7A). Close inspection of confocal image stacks reveals an additional PPL1 projection to the central complex. The targets of PPL1 neurons thus include structures implicated in both olfactory (Zars et al., 2000; McGuire et al., 2003; Krashes et al., 2007; Wang et al., 2008) and visual memory (Liu et al., 2006).

Notably, PPL1 neurons are not the sole sources of dopaminergic input to the mushroom body lobes. A second dopaminergic projection originates from cells in the PAM cluster and terminates bilaterally in the medial portions of the horizontal (β) lobes (Figure 7B). The function of these inputs is currently unknown, but it is safe to say that they play no role in short-term olfactory learning, as comprehensive manipulations of PAM neuron activity via the HL9-GAL4 driver are without consequence for memory (Figures 5A and 5B).

DISCUSSION

Nervous systems transform sensory signals and internal states into actions. Learning is a higher-order process by which this transformation is altered, producing different actions from the same initial state. Associative learning, by definition, uses an inherently valued stimulus, such as pain, to modify behavioral responses to sensory cues. When flies form aversive olfactory memories, they associate at least two sets of external inputs, conveyed by the olfactory and reinforcement pathways. As the presumed sites of memory storage, KCs lie at the intersection of these neural pathways. Signals from odorant receptors reach KCs, which are third-order olfactory neurons, via an extensively characterized sensory processing stream (Heisenberg, 2003; Keene and Waddell, 2007). By contrast, little is known, after more than 30 years of research, about the neural circuits that transmit nociceptive inputs from peripheral sensors and transform them into reinforcement signals.

We do know that dopamine is an essential mediator of aversive reinforcement in adult flies (Schwaerzel et al., 2003; Kim et al., 2007) and that its broad release can substitute for aversive salt conditioning in larvae (Schroll et al., 2006). The data presented here advance our understanding from this purely pharmacological level of analysis to a resolution of the relevant circuitry. Dopaminergic neurons display a remarkable degree of anatomical target specificity and functional specialization. Of the seven paired cell clusters in the central fly brain, only two (PAM and PPL1) innervate the mushroom body lobes, where their projections are segregated into discrete, non-overlapping domains (Figure 7).

Of these two cell clusters, only one (PPL1) carries aversive reinforcement information. Artificial manipulation of these reinforcement signals is, in itself, sufficient for re-programming a fly’s olfactory choice behavior (Figures 4 and 5). Neurons of the PPL1 cluster thus appear to link the brain’s valuation systems, where neural measures of attractiveness or aversiveness are constructed, to memory systems that associate these measures with predictive sensory cues.

Presynaptic Inputs to PPL1 Neurons: Learning Algorithms

In primates, midbrain dopaminergic neurons signal not the absolute magnitudes of rewards but the extent to which these rewards are unpredicted, and it is this “prediction error” that is thought to control synaptic weight changes during learning (Schultz et al., 1997). The algorithm ensures that the learning process terminates, as it should, when an animal has understood a regularity in its environment: accurate prediction reduces the error signal to zero, and no further adjustments to the animal’s model of the environment are made.

Comparing the responses of PPL1 neurons to predictable and unpredictable punishment will provide a clear test of whether olfactory learning in flies is also driven by prediction error: if PPL1 neurons encode error signals, their responses to aversive stimuli that are forecast by learned olfactory cues are expected to be attenuated or abolished. Previous attempts to perform such a test, by imaging calcium signals in dopaminergic projections to the mushroom bodies (Riemensperger et al., 2005), proved inconclusive, perhaps due to an inability to focus on what we only now appreciate are the memory-relevant PPL1 neurons.

Measuring the physiological responses of these neurons may uncover further functional heterogeneity within the PPL1 cluster and thereby extend the resolution limit imposed by currently available genetic tools. For example, it is conceivable that reinforcement signals are only emitted by some subset of PPL1 neurons, or in the extreme, a single cell.

The ability to pin reinforcement to a small, identified set of dopaminergic neurons provides a starting point for the dissection of upstream circuits that regulate the activity of these cells. Engineers (Kalman, 1960), researchers in machine learning (Samuel, 1959; Holland, 1986; Sutton and Barto, 1998), and psychologists (Rescorla and Wagner, 1972) have developed models of adaptive behavior in which error signals play a central role; they have also designed efficient algorithms for their computation (Sutton and Barto, 1998). Ironically, however, the biological systems that provided the inspiration for these models are still poorly understood. The classic temporal difference (TD) algorithms of machine learning suggest that error signals are calculated by a bootstrapping method, which assigns credit by comparing successive predictions (Sutton and Barto, 1998). A variety of “neuromorphic” circuits for computing such signals have been proposed (Houk et al., 1995; Montague et al., 1995; Schultz et al., 1997; Wörgötter and Porr, 2005), but they remain speculative until actual biological examples are solved. The simplicity of the fly, which like other invertebrates (Hammer, 1993; Brembs et al., 2002) employs only a handful of cells to convey reinforcement, raises hopes that the mechanisms controlling PPL1 activity are coming within reach.

Postsynaptic Targets of PPL1 Neurons: Memory Circuits

High-resolution mapping of the instructive PPL1 terminations within their mushroom body target domains will help disclose how olfactory memories are organized, written, and read. Most, if not all, memory devices are constructed from four circuit components: data storage locations, an addressing system that activates these locations for reading or writing, data input lines, and data output lines (Kanerva, 1988). In the case of the mushroom bodies, there is every expectation (though not yet formal proof) that information is stored in the weights of synapses between KCs and presently unidentified mushroom body output neurons (Dubnau et al., 2001; McGuire et al., 2001; Heisenberg, 2003; Davis, 2005; Keene and Waddell, 2007). The addressing system that activates these storage locations is odor-evoked KC activity patterns. Memories are written when a data input line—identified here as a PPL1 projection—releases dopamine onto active storage locations. The ensuing surge in cAMP production is thought to strengthen connections between the active KC ensemble and mushroom body output neurons (Levin et al., 1992; Heisenberg, 2003; Davis, 2005; Keene and Waddell, 2007). Memories are retrieved when the addressing system re-activates these data locations—that is, when a fly re-encounters a learned odor—and a mushroom body output neuron reads the contents of memory by pooling its synaptic inputs.

A key detail of this memory architecture is that data input and output lines supply the same storage locations in matched pairs (Kanerva, 1988). This has two important implications. First, the synaptic terminals of PPL1 projections mark the locations of input to the associative machinery. Although much effort has been devoted to identifying the mushroom body subdomains in which associative memories first form, different studies have reached apparently irreconcilable conclusions (Zars et al., 2000; Pascual and Preat, 2001; McGuire et al., 2003; Akalal et al., 2006; Krashes et al., 2007; Wang et al., 2008). The restricted distribution of PPL1 neurites within the mushroom bodies limits the potential sites of short-term plasticity to the vertical lobes, the heel, and the peduncle and can thus further focus attempts to localize and monitor the changes accompanying memory formation. The ability to write behaviorally relevant memories at will, simply by activating a small set of dopaminergic neurons, opens the possibility of performing the same manipulation in preparations where memory formation can be observed directly.

Second, in order to retrieve information from memory, the dendritic arbors of mushroom body output neurons must visit the same storage locations as the dopaminergic data input lines. This anatomical constraint will facilitate the search for these currently unknown circuit elements, which are expected to hold the key to understanding how remembered experience biases action choice.

EXPERIMENTAL PROCEDURES

Fly Strains

Transgenic strains were backcrossed to Canton-S for >4 generations. For photostimulation experiments, balanced UAS-P2X2 animals (Lima and Miesenböck, 2005) were crossed with balanced driver lines TH-GAL4 (Friggi-Grelin et al., 2003) or HL9-GAL4 to yield UAS-P2X2/CyO ; GAL4-driver/TMSb males. These males were paired with Canton-S or rutabaga2080 virgins to generate experimental animals. The HL9-GAL4 line carries an insertion of the GAL4 coding sequence into a fragment of the dopa decarboxylase (Ddc) gene. Genomic Ddc sequence from bp -2702 (numbering relative to transcription start site) to bp +970 in exon B was fused to yeast GAL4 and flanked by the hsp70 termination/polyA region and the Ddc exon D 3’ UTR and 3’ flanking region (bp +3521 to +4643). The initiating AUG in Ddc exon B was mutated to prevent interference with the translation of GAL4.

Where indicated, 5-HT levels were elevated by growing flies on food containing 50 mM 5-HTP and 25 mg/100 ml ascorbic acid for 3 d. The effectiveness of treatment was verified amperometrically (LC-4B detector, Bioanalytical Systems) after fractionating frozen head homogenates by reverse-phase HPLC (Jennings et al., 2006).

To deliver caged ATP (Lima and Miesenböck, 2005), flies were briefly iced and injected (Nanoject II, Drummond) through the ocelli with 9.2 nl of 60 mM P3-[1- (4,5-dimethoxy-2-nitrophenyl)ethyl]-ATP (DMNPE-ATP; Molecular Probes) in artificial hemolymph (5 mM Na-HEPES, pH 7.3, 115 mM NaCl, 5 mM KCl, 2 mM CaCl2, 8 mM MgCl2, 4 mM NaHCO3, 1 mM NaH2PO4, 5 mM trehalose, 10 mM sucrose). Injected flies were allowed to recover for >5 min and used within 30 min.

For temperature-induced silencing of neuronal activity, balanced tubP-GAL80ts flies were crossed with a balanced UAS-Kir2.1 line (Baines et al., 2001) to yield tubP-GAL80ts/CyO ; UAS-Kir2.1/TMSb offspring, which were paired with THGAL4 or HL9-GAL4. Experimental animals were maintained at the permissive temperature of 20 °C, at which expression of the Kir2.1 potassium channel is repressed by GAL80ts (McGuire et al., 2003). Channel expression was induced by shifting flies for 16 h to the restrictive temperature of 30 °C, at which GAL80ts inactivates (McGuire et al., 2003).

Expression patterns were visualized by crossing GAL4 driver lines to responder strains carrying one copy of a UAS-mCD8-GFP transgene or two copies of a UAS-PA-GFP transgene (Datta et al., 2008); KCs expressed mb247-DsRed (Riemensperger et al., 2005).

Training Apparatus and Software

Olfactory choice behavior was evaluated and conditioned in custom-built single- fly chambers (Figure S1A). The chambers, designed to confine movement to one dimension, measured 50 mm in length, 5 mm in width (sufficient for flies to turn freely), and 1.3 mm in height (to prevent flies from walking on side walls and thereby escape electric shock). The chambers were fabricated from clear polycarbonate to allow tracking of the animals’ back-lit silhouettes in side-on video images. Floors and ceilings consisted of printed circuit boards (PCBs) with 1-mm electrodes spaced at 1-mm intervals. Air/odor mixtures entered the chambers through two inlets (5.0×0.3 mm) at the left and right ends and exited through four outlets (5.0×0.3 mm) at the midpoint. As in a traditional T-maze, two converging air/odor streams meet at central outflow vents, defining a narrow (~5×5 mm) choice zone. To eliminate any spatial bias, odor streams were randomly alternated between the left and right chamber halves during test, training, and re-test.

In electric shock conditioning experiments (Figure S1B), 20 chambers were arranged into two side-by-side stacks by slotting their metal air/odor inlet tubes into gas distribution manifolds; the gas tubes also served as electrical leads connecting the PCBs via solid-state relays (Fairchild HSR312L) to a 60-VDC source. The array was backlit by 940-nm LEDs (TSAL6100, Vishay) with plastic diffusers and viewed by a Sunell SN-425M CCD camera equipped with a Navitar1212 lens and connected to a video acquisition board (PCI-1409, National Instruments). A virtual instrument written in LabVIEW 7.1 (National Instruments) extracted position data from video images and controlled gas flow and the solid-state relays. Fly positions were calculated by subtracting the current video frame from a continuously updated background image that retained the brightest pixels from the previous frame, setting negative pixels in the difference image to zero, and deriving the center coordinates of the silhouettes with the centroid location function. Electric shocks of 1.25 s duration were delivered at fixed 0.2-Hz frequencies, either during pre-determined training intervals (Pavlovian conditioning) or when the virtual instrument detected entry of the fly into the reinforced odor stream (action-contingent conditioning).

In optical conditioning experiments (Figure S1C), four flies were tested simultaneously in four chambers with 50×5-mm apertures in the top and bottom cover plates of the assemblies; the PCB floors and ceilings were replaced with quartz glass windows. The chambers were arranged flat on a clear acrylic platform and back-illuminated by 940-nm LEDs (TSAL6100, Vishay) through plastic diffusers. The chambers were viewed from above by a Sunell SN-425M CCD camera equipped with a Navitar Zoom7000 lens. A virtual instrument written in LabVIEW 7.1 (National Instruments) extracted position data from video images and controlled gas flow and laser exposures. Optical stimuli were delivered by a Q-switched, frequency-tripled Nd:YVO4 laser emitting at 355 nm (DPSS Lasers, model 3520-30). The beam was switched and intensity-modulated with an acousto-optic deflector (AOD; IntraAction model ASN-802832 with ME- 802 driver), directed into a pair of scan mirrors (GSI Lumnonics VM2000 with MiniSAX servo controllers), and merged with the observation path by a UV cold mirror (CVI). Upon entry of a fly into the reinforced odor stream, its coordinates were automatically fed to the scan mirrors to position the beam, and the Q-switch and AOD were activated for 10 ms, delivering a light pulse of 0.08 mJ/mm2. Optical stimuli were repeated at 0.2 Hz while the fly remained in the reinforcement zone. During such intervals, the scan mirrors were repositioned every 100 ms.

The photostimulation conditions used here are milder than those in our earlier experiments (Lima and Miesenböck, 2005). Previously, flies were injected with 41 nl caged ATP and exposed to 10 mJ/mm2 of light (four 150-ms pulses at 17 mW/mm2). In the current protocol, flies were injected with 9.2 nl caged ATP. Individual light pulses delivered optical energies of 0.08 mJ/mm2; flies consumed an average of 20.3 such pulses during training. The mean cumulative optical energy delivered is therefore 1.62 mJ/mm2, corresponding to 16 % of that used previously. Conditioned behavior was assayed >5 min after photostimulation, when any potential acute locomotor effects of dopaminergic activation have subsided (Lima and Miesenböck, 2005).

Odors

Compressed air was passed through desiccant and activated charcoal before manifolding into mass flow controllers (MFCs; CMOSens PerformanceLine, Sensirion). Two 5,000-ml capacity MFCs controlled left and right arm carrier air flows; six 500-ml capacity MFCs regulated OCT, MCH, and compensatory air flows in the left and right chamber arms, respectively. Flow-controlled carrier air was re-humidified by passage through a gas washing tube containing distilled water; stimulus streams were drawn through vials filled with pure liquid odorants. Measurements with a ppbRAE photoionization detector (PID) showed that the vials delivered saturated vapor at flow rates of up to 360 ml/min for MCH and 480 ml/min for OCT. To determine the range of appropriate experimental odor concentrations, PID measurements were made from a conventional T-maze odor cup (Tully and Quinn, 1985; Connolly and Tully, 1998); these corresponded to 14 % MCH and 11 % OCT. We adjusted these odor concentrations slightly to shift the untrained preference toward the odor to be made aversive; this yielded more stable baselines. Aversive training against MCH used 5 % MCH versus 12 % OCT, while aversive training against OCT used 10 % MCH versus 8 % OCT. Flow rate differences were compensated with air. The combined air/odor flow rates were 100 ml/min per half chamber, roughly corresponding to the air speed experienced by a fly in flight (David, 1979).

Data Analysis

Experimental data were logged to ASCII text files and analyzed off-line in MATLAB 7.4 (The MathWorks; see Data S1 for an example). Odor preference was quantified as the percentage of decisions in favor of odor A during a test interval (Figures S2–S4). A decision was counted every time a fly entered and exited the central choice zone. Entry into odor A from the choice zone, either as the result of a reversal or a traversal, was tallied in favor of that odor's choice percentage: (decisionsodorA/decisionsinterval)×100 %. Learning was scored by subtracting the post-training preference from the pre-training preference, giving a percentage change. The results of aversive conditioning are reported as negative preference—that is, positive avoidance—changes. Test intervals lasted 2 min; memory was assayed following a 5-min rest period after conditioning. Only data from flies making at least one choice per test interval were analyzed.

Pairwise hypotheses were evaluated by permutation test. Kruskal-Wallis ANOVA and post-hoc analysis using Tukey’s honestly significant difference criterion were used to test hypotheses involving multiple groups.

Immunofluorescence Microscopy

Synaptic neuropil was decorated with monoclonal antibody nc82 (Developmental Studies Hybridoma Bank) and goat anti-mouse Alexa 568 conjugate (Molecular Probes). Tyrosine hydroxylase was detected with antiserum Ab152 (Chemicon, 1: 100) and goat anti-rabbit Alexa 635 conjugate (Molecular Probes). Mouse CD8 expressed from the UAS-mCD8-GFP transgene was stained with rat anti-mCD8a (Invitrogen Caltag, 1: 100) dilution and goat anti-rat Alexa 488 conjugate (Molecular Probes). Stained brains were embedded in FocusClear and MountClear (CelExplorer Labs) and imaged on a Zeiss LSM710 laser-scanning confocal microscope.

Two-Photon Photoconversion of PA-GFP

Fly brains were dissected in artificial hemolymph and positioned under a custom- built two-photon microscope equipped with a Chameleon Ultra II laser (Coherent) and Hamamatsu H7422-40 GAsP detectors. Dopaminergic cell bodies expressing two copies of a UAS-PA-GFP transgene (Datta et al., 2008) under TH-GAL4 or HL9-GAL4 control emitted faint basal fluorescence when excited at 925 nm. Individual cell clusters could be identified by their anatomical position relative to the mushroom bodies, which expressed mb247-DsRed (Riemensperger et al., 2005). To photoconvert PA-GFP to its intensely fluorescent form (Patterson and Lippincott-Schwartz, 2002; Datta et al., 2008), a small volume of tissue containing the dopaminergic cell cluster of interest was illuminated with 5–30 mW of laser power at 710 nm in a scan configuration that delivered 9–30 µJ of optical energy per µm3-voxel. Following a 1-h waiting period to allow diffusion of PA-GFP, brains were recovered, fixed with 4 % (w/v) paraformaldehyde in PBS containing 0.1% (w/v) Triton X-100, embedded in FocusClear and MountClear (CelExplorer Labs), and imaged on a Zeiss LSM710 laser-scanning confocal microscope. Three-dimensional models were rendered in Volocity 5.0.3 (Improvision).

Supplementary Material

01
02
Download video file (637.6KB, mov)
03
Download video file (659KB, mov)

ACKNOWLEDGMENTS

We thank Trevor Sharp and Paul Overton for 5-HT measurements, Simon Reeve for experiments on rutabaga mutants, Richard Axel, Michael Bate, Ronald Davis, André Fiala, and Scott Waddell for fly strains, and Shankar Srinivas for access to a confocal microscope. Supported by grants from the Medical Research Council (G. M.), the NIH (G. M. and J. H.), the Office of Naval Research (G. M.), the Dana Foundation (G. M.), and EMBO (E. V.).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

REFERENCES

  1. Abrams TW, Karl KA, Kandel ER. Biochemical studies of stimulus convergence during classical conditioning in Aplysia: dual regulation of adenylate cyclase by Ca2+/calmodulin and transmitter. J Neurosci. 1991;11:2655–2665. doi: 10.1523/JNEUROSCI.11-09-02655.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Akalal DB, Wilson CF, Zong L, Tanaka NK, Ito K, Davis RL. Roles for Drosophila mushroom body neurons in olfactory learning and memory. Learn Mem. 2006;13:659–668. doi: 10.1101/lm.221206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baines RA, Uhler JP, Thompson A, Sweeney ST, Bate M. Altered electrical properties in Drosophila neurons developing without synaptic transmission. J Neurosci. 2001;21:1523–1531. doi: 10.1523/JNEUROSCI.21-05-01523.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Berg HC. Ecoli in motion. New York: Springer; 2004. [Google Scholar]
  5. Brembs B, Heisenberg M. The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator. Learn Mem. 2000;7:104–115. doi: 10.1101/lm.7.2.104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Brembs B, Lorenzetti FD, Reyes FD, Baxter DA, Byrne JH. Operant reward learning in Aplysia: neuronal correlates and mechanisms. Science. 2002;296:1706–1709. doi: 10.1126/science.1069434. [DOI] [PubMed] [Google Scholar]
  7. Brembs B, Plendl W. Double dissociation of PKC and AC manipulations on operant and classical learning in Drosophila. Curr Biol. 2008;18:1168–1171. doi: 10.1016/j.cub.2008.07.041. [DOI] [PubMed] [Google Scholar]
  8. Budnik V, White K. Catecholamine-containing neurons in Drosophila melanogaster: distribution and development. J Comp Neurol. 1988;268:400–413. doi: 10.1002/cne.902680309. [DOI] [PubMed] [Google Scholar]
  9. Connolly JB, Tully T. Behaviour, learning, and memory. In: Roberts DB, editor. Drosophila. A practical approach. Oxford: Oxford University Press; 1998. pp. 265–317. [Google Scholar]
  10. Couzin ID. Collective cognition in animal groups. Trends Cogn Sci. 2009;13:36–43. doi: 10.1016/j.tics.2008.10.002. [DOI] [PubMed] [Google Scholar]
  11. Datta SR, Vasconcelos ML, Ruta V, Luo S, Wong A, Demir E, Flores J, Balonze K, Dickson BJ, Axel R. The Drosophila pheromone cVA activates a sexually dimorphic neural circuit. Nature. 2008;452:473–477. doi: 10.1038/nature06808. [DOI] [PubMed] [Google Scholar]
  12. David CT. Optomotor control of speed and height by free-flying Drosophila. J Exp Biol. 1979;82:389–392. doi: 10.1242/jeb.82.1.389. [DOI] [PubMed] [Google Scholar]
  13. Davis RL. Olfactory memory formation in Drosophila: from molecular to systems neuroscience. Annu Rev Neurosci. 2005;28:275–302. doi: 10.1146/annurev.neuro.28.061604.135651. [DOI] [PubMed] [Google Scholar]
  14. de Belle JS, Heisenberg M. Associative odor learning in Drosophila abolished by chemical ablation of mushroom bodies. Science. 1994;263:692–695. doi: 10.1126/science.8303280. [DOI] [PubMed] [Google Scholar]
  15. Dubnau J, Grady L, Kitamoto T, Tully T. Disruption of neurotransmission in Drosophila mushroom body blocks retrieval but not acquisition of memory. Nature. 2001;411:476–480. doi: 10.1038/35078077. [DOI] [PubMed] [Google Scholar]
  16. Friggi-Grelin F, Coulom H, Meller M, Gomez D, Hirsh J, Birman S. Targeted gene expression in Drosophila dopaminergic cells using regulatory sequences from tyrosine hydroxylase. J Neurobiol. 2003;54:618–627. doi: 10.1002/neu.10185. [DOI] [PubMed] [Google Scholar]
  17. Gallistel CR, Gibbon J. Time, rate, and conditioning. Psychol Rev. 2000;107:289–344. doi: 10.1037/0033-295x.107.2.289. [DOI] [PubMed] [Google Scholar]
  18. Hammer M. An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature. 1993;366:59–63. doi: 10.1038/366059a0. [DOI] [PubMed] [Google Scholar]
  19. Heisenberg M. Mushroom body memoir: from maps to models. Nat Rev Neurosci. 2003;4:266–275. doi: 10.1038/nrn1074. [DOI] [PubMed] [Google Scholar]
  20. Heisenberg M, Borst A, Wagner S, Byers D. Drosophila mushroom body mutants are deficient in olfactory learning. J Neurogenet. 1985;2:1–30. doi: 10.3109/01677068509100140. [DOI] [PubMed] [Google Scholar]
  21. Holland JH. Escaping brittleness: The possibilities of general purpose learning algorithms applied to parallel rule-based systems. In: Michalski RS, Carbonell JG, Mitchell TM, editors. Machine learning. Los Altos: Morgan Kaufmann; 1986. pp. 593–623. [Google Scholar]
  22. Houk JC, Adams JL, Barto AG. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Houk JC, Davis JL, Beiser DG, editors. Models of information processing in the basal ganglia. Cambridge: MIT Press; 1995. pp. 249–270. [Google Scholar]
  23. Jennings KA, Loder MK, Sheward WJ, Pei Q, Deacon RM, Benson MA, Olverman HJ, Hastie ND, Harmar AJ, Shen S, et al. Increased expression of the 5-HT transporter confers a low-anxiety phenotype linked to decreased 5-HT transmission. J Neurosci. 2006;26:8955–8964. doi: 10.1523/JNEUROSCI.5356-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Johnson WA, McCormick CA, Bray SJ, Hirsh J. A neuronspecific enhancer of the Drosophila dopa decarboxylase gene. Genes Dev. 1989;3:676–686. doi: 10.1101/gad.3.5.676. [DOI] [PubMed] [Google Scholar]
  25. Kalman RE. A new approach to linear filtering and prediction problems. J Basic Eng Trans ASME. 1960;82:35–45. [Google Scholar]
  26. Kanerva P. Sparse distributed memory. Cambridge: MIT Press; 1988. [Google Scholar]
  27. Keene AC, Waddell S. Drosophila olfactory memory: single genes to complex neural circuits. Nat Rev Neurosci. 2007;8:341–354. doi: 10.1038/nrn2098. [DOI] [PubMed] [Google Scholar]
  28. Kim YC, Lee HG, Han KA. D1 dopamine receptor dDA1 is required in the mushroom body neurons for aversive and appetitive learning in Drosophila. J Neurosci. 2007;27:7640–7647. doi: 10.1523/JNEUROSCI.1167-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Krashes MJ, Keene AC, Leung B, Armstrong JD, Waddell S. Sequential use of mushroom body neuron subsets during Drosophila odor memory processing. Neuron. 2007;53:103–115. doi: 10.1016/j.neuron.2006.11.021. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Levin LR, Han PL, Hwang PM, Feinstein PG, Davis RL, Reed RR. The Drosophila learning and memory gene rutabaga encodes a Ca2+/calmodulin-responsive adenylyl cyclase. Cell. 1992;68:479–489. doi: 10.1016/0092-8674(92)90185-f. [DOI] [PubMed] [Google Scholar]
  31. Lima SQ, Miesenböck G. Remote control of behavior through genetically targeted photostimulation of neurons. Cell. 2005;121:141–152. doi: 10.1016/j.cell.2005.02.004. [DOI] [PubMed] [Google Scholar]
  32. Liu G, Seiler H, Wen A, Zars T, Ito K, Wolf R, Heisenberg M, Liu L. Distinct memory traces for two visual features in the Drosophila brain. Nature. 2006;439:551–556. doi: 10.1038/nature04381. [DOI] [PubMed] [Google Scholar]
  33. Livingstone MS, Sziber PP, Quinn WG. Loss of calcium/ calmodulin responsiveness in adenylate cyclase of rutabaga, a Drosophila learning mutant. Cell. 1984;37:205–215. doi: 10.1016/0092-8674(84)90316-7. [DOI] [PubMed] [Google Scholar]
  34. McGuire SE, Le PT, Davis RL. The role of Drosophila mushroom body signaling in olfactory memory. Science. 2001;293:1330–1333. doi: 10.1126/science.1062622. [DOI] [PubMed] [Google Scholar]
  35. McGuire SE, Le PT, Osborn AJ, Matsumoto K, Davis RL. Spatiotemporal rescue of memory dysfunction in Drosophila. Science. 2003;302:1765–1768. doi: 10.1126/science.1089035. [DOI] [PubMed] [Google Scholar]
  36. Montague PR, Dayan P, Person C, Sejnowski TJ. Bee foraging in uncertain environments using predictive hebbian learning. Nature. 1995;377:725–728. doi: 10.1038/377725a0. [DOI] [PubMed] [Google Scholar]
  37. Pascual A, Preat T. Localization of long-term memory within the Drosophila mushroom body. Science. 2001;294:1115–1117. doi: 10.1126/science.1064200. [DOI] [PubMed] [Google Scholar]
  38. Patterson GH, Lippincott-Schwartz J. A photoactivatable GFP for selective photolabeling of proteins and cells. Science. 2002;297:1873–1877. doi: 10.1126/science.1074952. [DOI] [PubMed] [Google Scholar]
  39. Quinn WG, Harris WA, Benzer S. Conditioned behavior in Drosophila melanogaster. Proc Natl Acad Sci U S A. 1974;71:708–712. doi: 10.1073/pnas.71.3.708. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Rescorla RA, Wagner AR. Black AH, Prokasy WF. In Classical conditioning II: Current research and theory. New York: Appleton-Century-Crofts; 1972. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement; pp. 64–69. [Google Scholar]
  41. Riemensperger T, Völler T, Stock P, Buchner E, Fiala A. Punishment prediction by dopaminergic neurons in Drosophila. Curr Biol. 2005;15:1953–1960. doi: 10.1016/j.cub.2005.09.042. [DOI] [PubMed] [Google Scholar]
  42. Samuel AL. Some studies in machine learning using the game of checkers. IBM J Res Dev. 1959;3:211–229. [Google Scholar]
  43. Schroll C, Riemensperger T, Bucher D, Ehmer J, Völler T, Erbguth K, Gerber B, Hendel T, Nagel G, Buchner E, et al. Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae. Curr Biol. 2006;16:1741–1747. doi: 10.1016/j.cub.2006.07.023. [DOI] [PubMed] [Google Scholar]
  44. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  45. Schwaerzel M, Monastirioti M, Scholz H, Friggi-Grelin F, Birman S, Heisenberg M. Dopamine and octopamine differentiate between aversive and appetitive olfactory memories in Drosophila. J Neurosci. 2003;23:10495–10502. doi: 10.1523/JNEUROSCI.23-33-10495.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Sjulson L, Miesenböck G. Photocontrol of neural activity: biophysical mechanisms and performance in vivo. Chem Rev. 2008;108:1588–1602. doi: 10.1021/cr078221b. [DOI] [PubMed] [Google Scholar]
  47. Sutton RS, Barto AG. Reinforcement learning. Cambridge: MIT Press; 1998. [Google Scholar]
  48. Tully T, Quinn WG. Classical conditioning and retention in normal and mutant Drosophila melanogaster. J Comp Physiol [A] 1985;157:263–277. doi: 10.1007/BF01350033. [DOI] [PubMed] [Google Scholar]
  49. Wang Y, Mamiya A, Chiang AS, Zhong Y. Imaging of an early memory trace in the Drosophila mushroom body. J Neurosci. 2008;28:4368–4376. doi: 10.1523/JNEUROSCI.2958-07.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wise RA, Rompre PP. Brain dopamine and reward. Annu Rev Psychol. 1989;40:191–225. doi: 10.1146/annurev.ps.40.020189.001203. [DOI] [PubMed] [Google Scholar]
  51. Wörgötter F, Porr B. Temporal sequence learning, prediction, and control: a review of different models and their relation to biological mechanisms. Neural Comput. 2005;17:245–319. doi: 10.1162/0899766053011555. [DOI] [PubMed] [Google Scholar]
  52. Zars T, Fischer M, Schulz R, Heisenberg M. Localization of a short-term memory in Drosophila. Science. 2000;288:672–675. doi: 10.1126/science.288.5466.672. [DOI] [PubMed] [Google Scholar]
  53. Zemelman BV, Lee GA, Ng M, Miesenböck G. Selective photostimulation of genetically chARGed neurons. Neuron. 2002;33:15–22. doi: 10.1016/s0896-6273(01)00574-8. [DOI] [PubMed] [Google Scholar]
  54. Zemelman BV, Nesnas N, Lee GA, Miesenböck G. Photochemical gating of heterologous ion channels: Remote control over genetically designated populations of neurons. Proc Natl Acad Sci U S A. 2003;100:1352–1357. doi: 10.1073/pnas.242738899. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

01
NIHMS144154-supplement-01.pdf (566.8KB, pdf)
02
Download video file (637.6KB, mov)
03
Download video file (659KB, mov)

RESOURCES