Do the right thing: neural network mechanisms of memory formation, expression and update in Drosophila

Highlights

• •Recurrent connectivity is anatomically and functionally prevalent in fly memory circuits.
• •Sustained reverberant activity is necessary for memory consolidation.
• •Feedforward inhibitory neurons impose state control on memory retrieval and behavior.
• •Recurrent circuits enable re-evaluation and updating of memory.

Abstract

When animals learn, plasticity in brain networks that respond to specific cues results in a change in the behavior that these cues elicit. Individual network components in the mushroom bodies of the fruit fly Drosophila melanogaster represent cues, learning signals and behavioral outcomes of learned experience. Recent findings have highlighted the importance of dopamine-driven plasticity and activity in feedback and feedforward connections, between various elements of the mushroom body neural network. These computational motifs have been shown to be crucial for long term olfactory memory consolidation, integration of internal states, re-evaluation and updating of learned information. The often recurrent circuit anatomy and a prolonged requirement for activity in parts of these underlying networks, suggest that self-sustained and precisely timed activity is a fundamental feature of network computations in the insect brain. Together these processes allow flies to continuously adjust the content of their learned knowledge and direct their behavior in a way that best represents learned expectations and serves their most pressing current needs.

Current Opinion in Neurobiology 2018, 49:51–58

This review comes from a themed issue on Neurobiology of behavior

Edited by Kay Tye and Nao Uchida

For a complete overview see the Issue and the Editorial

Available online 16th December 2017

https://doi.org/10.1016/j.conb.2017.12.002

0959-4388/© 2017 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

If an experience of a pleasant or disagreeable nature (a feature known as positive or negative valence) occurs in coincidence with an otherwise neutral stimulus, such as an odor plume, it may be that this stimulus is learned as a cue, a predictor, for the desirable or unwelcome outcome. In associative learning, the cue is associated with the valence of its predicted aftermath, so that the previously neutral stimulus is now approached or avoided. This linking requires that some component of the brain's structure or activity is changed by the learning experience: identifying the nature of this plasticity and how it yields the changes in behavior that arise after training is a core goal of neuroscience.

Drosophila has been used as a model for associative learning for nearly half a century [1]: from the anatomical structures [2], to the cell types [3, 4] and their precise connectivity [5••, 6, 7••], the physical substrates of memory in flies are being elucidated to finer and finer detail [8, 9••, 10••, 11, 12••, 13, 14, 15]. Anatomy and function have been linked by experiments in which specific neurons are targeted genetically and their firing activity or synaptic release artificially altered, using temperature or light to control the timing of intervention. Within an associative learning paradigm, a neuron can therefore be prevented from functioning: if learning is disrupted, that neuron is likely to play a role. Conversely, if the artificial activation of the neuron can replace some component of the training paradigm, its function in the network is likely to represent that particular component [16, 17, 18, 19, 20, 21, 22, 23, 24]. In parallel with these loss-of-function and gain-of-function experiments, the activity of neurons can be directly monitored using fluorescent calcium indicators. Identifying changes brought about by learning (memory traces) can then point to the location of memory-induced plasticity.

Mushroom body cell types represent the components of learning experiences

A model arises from these experimental approaches [25, 26]: olfactory cues to be learned are represented by activity within sparse subpopulations of the overall array of cholinergic Kenyon cells (KCs) – 2000 cells per hemisphere forming the neuropil of the mushroom body [27, 28, 29]. Their downstream partners, output neurons (MBONs), which belong to various anatomically distinct classes using acetylcholine, glutamate or GABA (γ-aminobutyric acid) as neurotransmitter, project to the rest of the brain and promote either cue driven approach or repulsion [10••, 12••, 26, 30] (Figure 1). During learning, dopamine is released to induce plasticity at the synapse between odor-activated Kenyon cells and output neurons [10••, 12••, 31, 32•, 33••]. The presynaptic terminals of different subsets of dopaminergic neurons (DANs), encoding positive or negative valence, occupy distinct, non-overlapping compartments of the mushroom body neuropil which are precisely matched by the dendritic fields of discrete MBONs (Figure 1) [16, 18, 19, 22, 23, 34••, 35, 36]. Dopamine released from particular DANs alters the efficacy of specific KC-MBON connections, which imposes a skew in the overall drive of the output network and so tips the balance of behavior towards approach or avoidance [26] (Figure 2). This simple model displays a satisfying symmetry with the elements of associative learning and could explain, minimally, how to produce experience-dependent changes in behavior; but, as is often the case, recent experimental evidence suggests that the reality is more complex.

Figure 1.

The mushroom body is the center for associative learning. (a) Sensory cues are represented as activity in sparse populations of cholinergic Kenyon cells (KCs, grey). KCs send their neurites into the lobes of the mushroom body (light grey background), where they make en passant synapses with the output neurons (MBONs). Mushroom body-innervating dopaminergic neurons (DANs) provide the reinforcement signal during aversive and appetitive associative learning. (b) KCs are organized into three subtypes which make up the lobes of the mushroom body neuropil (individual representative KCs shown in dark grey). (c) The presynaptic fields of the different DAN classes tile the mushroom body into non-overlapping compartments. (d) The dendritic tufts of distinct MBONs match the compartmentalization of the DAN teminals. Aversively reinforcing DANs in the paired posterior lateral 1 (PPL1) cluster (red) overlap with approach-promoting MBONs (blue), while DANs of the protocerebral anterior medial (PAM) cluster, which are largely appetitively reinforcing (green), overlap with avoidance-promoting MBONs (orange). The transmitters used by each class of neuron is noted: ACh, acetylcholine; DA, dopamine; GABA, γ-aminobutiryc acid; Glu, glutamate.

Figure 2.

Dopamine drives plasticity at KC output synapses to alter behavior upon learning. During training (top), cue-driven KC activation (dark grey) coincides with the activity of either aversive (red, left) or appetitive (green, right) DANs and induces specific synaptic plasticity at the KC to MBON synapse (shaded green/red semicircle). Upon memory retrieval (bottom), the cue-driven response of the MBON network is skewed because of the synaptic modification imposed by learning (smaller semicircle): to avoidance behavior by the reduced drive of approach-promoting MBONs in the case of aversive learning (left), and towards approach behavior as a consequence of the reduced drive of avoidance-promoting MBONs in appetitive learning (right).

Memory consolidation requires ongoing activity

In the minimal model of associative learning (Figure 2), all activity is confined to the time of the learning experience, and plasticity occurs during training. It was therefore surprising that, in the characterization of the amnesiac memory-defective mutant [4, 37], the dorsal paired medial (DPM) neurons were identified to be required not during, but after training for memory to persist [38, 39]. This work provided the first neural network insight into why the stabilization of memory into a persistent form, or consolidation of memory, requires activity that extends beyond the duration of the learning experience [40, 41]. For processing to persist after the initial stimulus-driven phase, self-sustained, reverberant activity may come to dominate the network [42, 43]. During this period, interference such as anesthesia by cold shock (which halts neural activity) or the inhibition of protein synthesis disrupt the consolidation process and lead to the apparent loss of memory [41, 44, 45, 46]. Once consolidation is complete, intervention no longer affects memory [38, 39, 43, 45]. Output from the α′β′ subset of KCs is also required during the consolidation period and DPM connectivity to α′β′ KCs appears to be of particular importance [38, 43]. Since DPM neurons only receive input and deliver output within the mushroom body it was proposed that, in addition to representing sensory cues, the mushroom body uses feedback connectivity to generate self-sustained, reverberant activity that persists for minutes to hours and is crucial for memory stability [38, 39, 42, 47, 48, 49].

Recent work [50^•] exploring the anatomy of appetitive memory consolidation suggested the involvement of self-sustained activity in a specific compartment, α1, of the mushroom body output network. The αβ KCs, the DANs that innervate the α1 compartment and the MBONs that there receive their input all appear to be required shortly after training with odor and sugar for the memory to consolidate correctly. In this period, no overt sensory cues are presented to the flies so the question arises of how ongoing activity in Kenyon cells or MBONs can be generated. Experiments combining optogenetic stimulation and calcium reporters suggest that the release of dopamine can directly alter the activity of both MBONs [5^••] and KCs [32^•] in the absence of obvious stimulus-driven activity. DANs could therefore provide gates in the network to create localized nodes of activity, which may be relevant for the ongoing processes required for consolidation. DANs might also gate activity to be constrained within closed feedback circuits. Interestingly, the glutamatergic α1 MBONs project to the brain region where α1 DANs receive their inputs (Figure 3b), and could plausibly establish a recurrent circuit [50^•]. This recurrent anatomical motif — in which MBONs synapse onto DANs, that release dopamine onto that MBON's dendrites — may be a common one, having been observed in other parts of the mushroom body [9••, 10••] and predicted by anatomical overlap for many of the different output neuron groups [34^••]. Although feedback loops are prevalent in the adult mushroom body anatomy and appear abundant in a full reconstruction of the larval mushroom body from an electron microscope volume [7^••], it is not clear that the mode of all the connections is appropriate to generate sustained activity. Moreover, MBONs in the adult and larval mushroom body are also connected to more distant DANs that innervate other compartments [7••, 9••]. Models of memory consolidation, and other mushroom body functions, based on feedback activity must therefore be able to maintain specificity of both the learned stimulus and the network compartment and to avoid instability, such as runaway excitation. Feedback inhibition is used to refine the specificity of odor representations in the KC ensemble [28].

Figure 3.

Feed-forward, feed-back and feed-across networks in the mushroom body. (a) Feed forward inhibition regulates state dependent expression of food memory retrieval. Hunger state controls odor-driven behavior by relaying hunger-dependent dNPF signaling (purple) through the PPL1-MP1 DANs (red) and the GABAergic MVP2 neuron (blue). The MVP2 neuron feeds forward inhibition to the M4/M6 group of glutamatergic MBONs (orange). In a satiated fly, approach behavior is counterbalanced by the avoidance-promoting M4/M6 MBONs, which are not inhibited by MVP2. In a hungry fly, the MVP2 neuron is active and inhibits the M4/M6 MBONs, reducing avoidance and facilitating approach behavior. (b) A closed feedback loop involving the MBON from the α1 compartment, the αβ Kenyon cells and the DANs innervating the α1 compartment has been proposed to stabilize memory after learning. Blocking any of these neurons immediately after sugar reward learning cripples behavioral approach measured 24 h later [50^•], although a functional connection between the glutamatergic α1 MBON and α1 DANs has not been demonstrated. (c) A feedback and feed-across recurrency is crucial for memory reconsolidation. Experiencing training-related cues can switch stable memories back into a labile state and reconsolidation is then required for the memory to return to a persistent condition [9^••]. The reconsolidation process requires the activity of the cholinergic MBON-γ2α′1 which in turn recruits two different sets of dopaminergic neurons: the PPL1-γ2α′1 DANs, which are required during memory retrieval, and a group of PAM-DANs which innervate different compartments and are required in the period following retrieval. These cholinergic MBON-DAN connections are excitatory [9^••].

Internal states gate behavior through dopamine and feedforward neural circuitry

It is possible that sustained activity within recurrent circuits is necessary to bridge between the fast timescales of neuronal firing and the hours-long processes of protein synthesis-dependent memory consolidation. By extending the period in which the memory is being constructed, later information can be collected and integrated. This may be particularly useful when the experience includes delayed or long-lasting effects. When learning involves feeding, sweet but indigestible sugars will only form short-lived memories, while those that can be metabolized yield longer lasting memories [51]. Although flies can learn to distinguish between these sugars within minutes [51], it would seem likely that the sugar has to be ingested and absorbed for the fly to evaluate nutritional quality. Recent studies have indicated that sweet taste and nutrient reinforcement signals are provided by different DANs [36, 52] and that it is possible to artificially uncouple the taste and nutritious value by manipulating these neurons, or using a sweet but not nutritious sugar in an associative learning paradigm, then delivering nutritious food up to several hours later [53^•]. This delayed nutritious addition seems sufficient to ‘complete’ the original learning and establish a long term memory, suggesting that the original memory trace must remain accessible to the nutritional input. Rhythmic activity has been recorded in specific DANs (MP1, also known as PPL1-γ1pedc DANs) during this phase [53•, 54] which may be indicative of an underlying recurrent network. The MP1 DANs can relay information about nutritional state because they are modulated by dNPF (Drosophila neuropeptide F), a neuropeptide representing hunger state [21].

Internal state representations in the mushroom body are also required to limit the behavioral expression of memories to situations when their content is most appropriate. Flies that have associated an odor cue with a sugar reward only show robust preference for this cue when they are hungry, or if the hunger signal dNPF is artificially triggered [21, 45]. The compartmentalized architecture of the MBON network allows different memories to be linked to their relevant internal states, creating meaningful modules of learning and behavior. For example, a memory of a water reward is preferentially expressed when flies are thirsty, but not hungry, while a previously sugar-associated cue is approached when flies are hungry, but not thirsty [55]. In mechanistic terms, the hunger signal dNPF sits at the top of a hierarchical inhibitory pathway comprised of the MP1 DANs [21] and a pair of GABAergic neurons, called MVP2 (or MBON-γ1pedc > αβ), that exert feedforward inhibition across the MBON network [12^••] (Figure 3a). In hungry flies, inhibition from MVP2 tips the balance of the MBON network to facilitate expression of the odor approach behavior from the circuit configuration that was imposed by dopamine-induced plasticity during learning. This mechanism therefore allows flies to match the expression of food-seeking memories to hunger state.

Context, experience and novelty are represented in the mushroom body output network

Dopamine, then, is not only tasked with the writing of memories by inducing synaptic plasticity during a learning experience. The expression of memory, and especially its modulation by conditions such as hunger, is also under the direction of DANs. It is clear that these two tasks are not entirely distinct at the network level. Most strikingly, in addition to providing hunger-dependent control, MP1 DANs can write odor-specific aversive memories at the KC to MVP2 connection of the MBON network [12^••]. In addition, DANs that are artificially activated during the presentation of an odor and so implant a fictive olfactory memory, also suppress expression of learned behavior if they are activated when memory is tested [5^••]. By affecting both the formation of memory and its retrieval, dopamine is able to optimize current and future behavior, integrating accumulated experience with the particular environmental and internal conditions experienced by the fly. By re-routing information throughout the MBON network, the mushroom body can therefore integrate context and internal state with sensory stimuli to direct the most appropriate behavior [21, 56]. Such a pathway gating model is nicely supported by a live imaging study which demonstrated that behavioral states can correlate with differential routing of activity through the MBON network [32^•]. Moreover, the pattern of active MBON pathways is mirrored by activity in combinations of complementary DANs and dynamic adjustments of synaptic activity in the corresponding compartments along the length of KC arbors [32^•]. This study also demonstrated the existence of recurrent and broad network connectivity from MBONs to DANs. Artificial stimulation of the output network caused widespread and complex changes in the activity of DANs [32^•].

Since reinforcing DANs receive extensive recurrent input from MBONs, any change to the KC drive to MBONs, such as that imposed by prior experience, should change how recurrence drives the DANs. As a consequence, any stimulus that has been previously encountered may be processed differently, and engage different DAN pathways, if encountered again. The simplest example of such a situation is the case of behavioral adaptation that occurs when a fly becomes familiar with an odor. For any stimulus to be considered as familiar the first experience must leave a trace in the network that impinges on the processing of the second: a computation that is perfectly suited for a recurrent architecture. In flies, familiarization of odor requires plasticity of the KC to α′3 MBON connection that is driven by the DANs that release dopamine within that compartment of the MBON network. Although α′3 MBONs are cholinergic, no specific recurrent MBON-DAN connection was identified [57^••]. Familiarity is a short-lived process, with behavioral responses to a repeatedly presented stimulus returning to their original strength after an hour [57^••]. In contrast, network plasticity that underlies learning that a stimulus is a predictive cue for an important event needs to be relatively stable so that an expectation can be retrieved when the stimulus is encountered again.

Recurrent connectivity enables memory re-evaluation and reconsolidation

Sometimes learned predictions turn out to be wrong and these experiences provide the animal with conflicting information. Memory systems therefore need to have the capacity to constantly update and incorporate new information into the animal's model of the world. Such a memory re-evaluation can be observed in Drosophila that have been trained to associate an odor with a sugar reward. If the odor is later presented without reward, learned behavior is extinguished — the previously learned odor is no longer approached in a test situation [9••, 41]. This neutralizing of the previously learned behavior, requires a subset of MBONs and DANs that represent aversive valence. Thus, after reward learning the mushroom body network appears to represent and code the omission of expected reward in a similar manner to how aversive events such as electric shock punishment reinforce learning [9^••]. If instead the odor returns with the expected reward, preference is maintained, but the DANs that were responsible for the formation of the original memory are no longer necessary, suggesting that even in this case, the (identical) information is processed through different pathways if it is already known. Alternatively, flies can be re-exposed to partial elements of the training trial that do not strongly conflict with the prior learned expectation. This triggers a cycle of destabilization and re-stabilization, or reconsolidation, during which memory is thought to be updated [9••, 58, 59]. Memory reconsolidation requires a specific cholinergic MBON pathway and the sequential action of two sets of DANs innervating different compartments [9^••] (Figure 3c). Together, the extinction and reconsolidation experiments reveal that plasticity triggered by the initial learning episode reconfigures the MBON network, so that when the learned stimulus is encountered again it is processed through different parts of the circuit. This allows the network to code multiple events and their relationships, so that behavior is driven according to the best information available. Recurrent connections in the MB network appear to play a critical role in comparing past and present, accumulating experience over time, and re-assessing the validity of past decisions in light of new information.

Conclusions

Network recurrence challenges the simple model integrating cues and valence into preference behavior (Figure 1, Figure 2). It is becoming clear that the mushroom body is able to perform several more complex recursive computations that are critical for the fly to orchestrate a more coherent behavioral strategy (Figure 3). Feedback architecture appears not only at the network level, but also within the components of the tripartite synapse where KCs, DANs and MBONs are in contact: synapses between KCs and DANs are found in both directions, and DANs contact both the presynaptic KCs and the postsynaptic MBONs [5••, 7••] (Figure 4). These reciprocal micro-circuits could generate persistent activity within a highly localized region, thus retaining the sparse synapse specificity that is assumed to underlie olfactory learning [60]. It will be important to decipher how these different levels of network architecture contribute to the computational power of the mushroom body.

Figure 4.

KCs, MBONs and DANs form microcircuits within a compartment [5••, 7••]. KCs (black) form cholinergic excitatory connections with MBONs (orange) and also synapse onto DANs (green). KC to DAN connections appear to be excitatory and may modulate the dopaminergic reinforcement signal within a compartment to influence learning [60]. DANs synapse onto KCs and MBONs. Activation of DANs seems to lead to a slow but direct activation of MBONs which potentially allows a local excitatory feedback loop between KCs, DANs and MBONs [5^••].

Conflict of interest statement

Nothing declared.

References and recommended reading

Papers of particular interest, published within the period of review, have been highlighted as:

• • of special interest

• •• of outstanding interest