Recurrent loops: Incorporating prediction error and semantic/episodic theories into Drosophila associative memory models

Abstract

In 2003, Martin Heisenberg et al. presented a model of how associative memories could be encoded and stored in the insect brain. This model was extremely influential in the Drosophila memory field, but did not incorporate several important mammalian concepts, including ideas of separate episodic and semantic types of memory and prediction error hypotheses. In addition, at that time, the concept of memory traces recurrently entering and exiting the mushroom bodies, brain areas where associative memories are formed and stored, was unknown. In this review, I present a simple updated model incorporating these ideas, which may be useful for future studies.

A model of Drosophila associative memory incorporating prediction error hypotheses and episodic/semantic memory forms.

1. INTRODUCTION

We learn from experiences, and memories of these experiences mold our personalities to make us who we are. Thus, the question of how memories are stored and recalled in the brain is one of fundamental interest to us. Recent advances have succeeded in identifying networks or engrams that are formed during learning, and necessary for recall (for reviews, see References 1, 2, 3). However, we have yet to understand how these identified engrams actually encode memories and guide behaviors.

Drosophila offer an excellent model to bridge the connection between memory engrams and experience‐dependent behavior. Several easy to understand, and easily measured, learning and memory‐based behaviors are available,4, 5, 6, 7 and the structure of the mushroom bodies (MBs), the brain structure where plasticity critical for many of these behaviors occurs, is well understood.8, 9, 10

In 2003 and 2004, Martin Heisenberg et al.11, 12 suggested a model, based on data and ideas from various groups,13, 14, 15, 16 of how olfactory associative behaviors may be encoded in the MBs. While this model has been extremely useful in guiding research up to now, many research advances have been made since this model was first proposed. Thus, new models incorporating these advances need to be considered. Here, I present a model for olfactory associations that incorporates prediction error theory and semantic and episodic memory theories into the Heisenberg model.

2. OLFACTORY ASSOCIATIVE BEHAVIOR IN DROSOPHILA

The ability to accurately measure learning and memory‐dependent behaviors has been highly beneficial for the study of memory in Drosophila. In particular, Drosophila can be trained to form Pavlovian‐type odor associations.7, 17 In a typical training regimen for aversive olfactory associations, flies are simultaneously exposed to an odor, referred to as the paired conditioned stimulus or CS+, and an aversive unconditioned stimulus (US), often consisting of electrical shocks. Flies are next exposed to a second odor, the unpaired conditioned stimulus or CS−, this time in the absence of the US. Flies thus learn to associate the CS+ odor, but not the CS−, with an aversive stimulus. Learning and memory of this association can be measured at various time points after training by placing flies at the choice point of a T maze and allowing them to choose between the CS+ and CS− odors (Figure 1). Flies that learned prefer the CS− over the CS+, while flies that did not learn or do not remember, distribute evenly between the odors. Similar to aversive associations, appetitive associations can be formed by pairing the CS+ with an appetitive US, often a sucrose reward, instead of an aversive US.18, 19 These easy to understand, memory‐based behaviors, with well‐characterized sensory inputs and behavioral outputs, are highly suited for mapping the relationship between memory engrams, the sensory inputs that they respond to, and the behaviors they regulate.

Figure 1.

Training and testing of an aversive olfactory association in Drosophila. Flies are trained and tested in a “teaching machine,”7 consisting of an upper training chamber and a lower testing chamber. Training consists of two steps. During the first step, flies are exposed simultaneously to an odor (odor A, shown in purple), and electrical shocks (yellow). At step 2, flies are exposed to a second odor (odor B, green), in the absence of shocks. Flies learn to associate odor A, but not odor B, with aversive shocks. Learning and memory of this association can be measured at various time points after training by testing flies by placing them at the choice point between odors A and B, and allowing them to choose between these odors. Memory‐associated behaviors can be quantified as a performance index, calculated by subtracting the percentage of flies choosing the shock‐paired odor from the percentage of flies choosing the unpaired odor. Appetitive associations can be measured in a similar fashion in starved flies, by replacing electrical shocks with sucrose rewards

3. THE HEISENBERG MODEL FOR PLASTICITY UNDERLYING OLFACTORY ASSOCIATIVE BEHAVIORS

Olfactory associations are formed in the MBs, brain structures where olfactory CS, and aversive or appetitive US information converge (Figure 2A). Odors are first detected by olfactory receptor neurons in the antennae and maxillary palps, which extend axons to glomeruli located in the antennal lobes.20, 21 Projection neurons then transmit odor information from the antennal lobes to the MBs and the lateral horn.22, 23 In the MBs, projection neurons synapse onto the calyx, the dendritic region of intrinsic MB Kenyon cells, and individual odors are thought to be encoded in the MBs by activation of different sparse subsets of Kenyon cells.24, 25, 26 While the specific network connections transmitting aversive and appetitive US information to the MBs is less characterized, aversive information has been proposed to be transmitted to the MBs through PPL (protocerebral posterior lateral) dopaminergic neurons, and appetitive information is proposed to be transmitted by the PAM (paired anterior medial) cluster of dopaminergic neurons (Figure 2B).27, 28

Figure 2.

Schematic diagrams of the mushroom bodies and their connections. A, A schematic diagram of the Drosophila brain (shown in gray) with the mushroom body and antennal lobe of one hemisphere shown in green and blue, respectively. B, An expanded view of the boxed region of A. Kenyon cell (KC) bodies are represented as small green circles. KCs extend dendrites to the calyx, and extend axons down the peduncle to α/β, α′/β′ and γ lobes. Olfactory sensory information is transmitted to the calyx and the lateral horn (LH) from the antennal lobe (AL) via projection neurons, and reward and punishment information has been proposed to be transmitted to the mushroom body lobes via PAM and PPL1 dopaminergic neurons. Mushroom body output neurons (MBONs) transmit signals from the mushroom bodies to various other brain regions. C, Summary of the Heisenberg model. Odor information from the AL stimulates a subset of MB Kenyon cells represented as large green circles. Odor‐activated synaptic connections are shown as small blue circles. Kenyon cells connect to MBONs that influence conditioned approach or avoidance responses. However, odor‐dependent activation of KCs is not sufficient to activate approach or avoidance programs in naïve animals. When odor pathways are activated at the same time as aversive electrical shock pathways (shown as activating synapses represented by red circles), or appetitive reward pathways (not shown), the combined CS and US activation strengthens synaptic connections between odor‐encoding KCs and appropriate avoidance or approach‐inducing MBONs (filled green circles). Due to this synaptic strengthening, when flies are re‐exposed to the odor, they activate appropriate motor response pathways. (Modified from Heisenberg et al., 2003)12

MB Kenyon cells synapse onto 34 MB output neurons (MBONs) consisting of 21 different types.29, 30 Importantly, artificial activation of different MBONs (eg, through light activation), induces either avoidance or approach behaviors.

In previous models proposed by Heisenberg and others, Kenyon cells activated by a neutral odor that a fly has not previously experienced, have weak connections to different MBONs.11, 12 Thus flies neither approach nor avoid the odor in an MB‐dependent manner. However, if the odor is presented at the same time as an aversive or appetitive stimulus, coincident activation of reinforcement neurons responding to the US, strengthens synaptic connections between Kenyon cells and specific MBONs through a Hebbian mechanism.31 Thus, a motor response to an odor is modulated by its association with aversive or appetitive stimuli (Figure 2C). Recent work has determined that activation of PPL and PAM subsets of dopaminergic neurons can replace US presentation during formation of aversive and appetitive associations, suggesting that these neurons act to induce plasticity between Kenyon cells and appropriate MBONs.27, 28, 32, 33

4. PROGRESS SINCE THE HEISENBERG MODEL

While this model of fly learning is compelling, various new results have emerged since its proposal. For example, when Heisenberg proposed his model, the requirements for MB output during memory formation, storage and recall had not been well characterized. Studies inhibiting synaptic vesicle recycling in Kenyon cells had suggested that MB output was only required during memory recall, and not during memory formation or storage.15, 34 Thus, Heisenberg and others at the time concluded that sensory information needs to enter, but does not have to leave the MBs, for learning to occur. However, the MBs consist of various different types of Kenyon cells that can be broadly classified into α/β, α′/β′ and γ cells that project axons to α/β, α′/β′ and γ MB lobes.8, 35 Studies using more precise, type‐specific inhibition later showed that neuronal output from γ and α′/β′ neurons is required during learning and shortly thereafter,36, 37, 38 while output from α/β neurons is required during recall.15, 34 This indicates that sensory information converges in the MBs during training and likely activates a memory trace that then shifts between various Kenyon cell types.

Supporting the idea that memory traces exit and reenter the MBs recurrently, MB Kenyon cells are innervated by various MB extrinsic neurons, including anterior paired lateral (APL),39 dorsal paired medial (DPM)40 and dorsal anterior lateral (DAL)41 neurons. These neurons not only innervate the MBs, but also, directly or indirectly, receive input from the MBs.25, 42, 43 Activity of MB extrinsic neurons affects processes such as learned odor discrimination,25 maintenance of labile memory44 and consolidation of long‐term memories,42 suggesting that recurrent MB activity is necessary at various stages of learning and memory.

The different types of Kenyon cells have long been known. However, it has more recently become apparent that besides different cell types, individual Kenyon cell axons can be separated into different compartments with distinct dopaminergic inputs and MBON outputs.29, 30, 45, 46 This adds further layers of complexity to Kenyon cells that was not apparent when Heisenberg formulated his model.

Finally, Heisenberg's model does not take into account mammalian data on reward prediction errors and executive decision‐making.47, 48, 49 Heisenberg's model essentially focuses on plasticity in a single type of synaptic connection, the connection between Kenyon cells and MBONs, to explain experience‐dependent changes in behavior. However, its simplicity does not allow it to explain how learning and plasticity change over multiple rounds of training, or how executive decision‐making functions may interact with memory pathways to regulate behaviors.

5. PAIRING AN UNCONDITIONED STIMULUS WITH A CONDITIONED STIMULUS IS NOT SUFFICIENT TO INDUCE PLASTICITY AND LEARNING

In a simple Hebbian model of plasticity, coincident converging neuronal activity can enhance the synaptic connection between a weak coincident input and some neuronal output.50 Thus, a weak synaptic connection between odor‐activated Kenyon cells and behavior‐regulating MBONs can be strengthened by coincident activation of a neural pathway responding to a US, that converges onto this weak synapse. However, this type of model suggests that simply pairing a US with a CS is sufficient to induce a behavioral response, a conclusion that is inconsistent with several memory models, including mammalian prediction error models.47, 49, 51, 52

In prediction error theory, learning should only occur when there is a difference between what an animal is currently experiencing and what the animal expected to experience.47, 49, 51, 52 Thus pairing of an aversive US with a CS should be sufficient to form an aversive association in a naïve animal, which does not have a prior expectation regarding the CS. However, in an animal that has been sufficiently exposed to the US‐CS pairing paradigm and is already expecting the US, little learning should occur. If this is the case, dopamine, or any other neurotransmitter that functions to induce plasticity or reinforce an association, should not be released strictly upon presentation of the US.49 Instead, reinforcement signals should only be released after a comparison mechanism determines that a current US is different from the US that the animal was expecting when exposed to the CS.

6. EX VIVO STUDIES INDICATE THAT DOPAMINE DOES NOT CONVEY INFORMATION ABOUT THE UNCONDITIONED STIMULUS TO THE MUSHROOM BODIES

Minoru Saitoe's group from the Tokyo Metropolitan Institute of Medical Science has used an ex vivo dissected brain system53 to examine learning‐associated plasticity in the MBs (Figure 3).54 In this system, dissected Drosophila brains are maintained in culture media. In place of in vivo odor exposure, the ALs are directly stimulated using glass electrodes, and in place of electrical shock application, the ascending fibers of the ventral nerve cord (AFV), which transmit somatosensory information from the body to the brain, are stimulated. Ca²⁺ responses in the MBs are measured by expressing a fluorescent calcium sensor in the MBs. In brains from naïve flies, both AL stimulation and AFV stimulation induce Ca²⁺ influx in the MBs. However, after ex vivo conditioning, which consists of coincident AL and AFV stimulation, a robust increase in AL‐dependent MB responses is observed. This plasticity shares many similarities with in vivo aversive olfactory learning, and is referred to as long‐term enhancement (LTE) of MB Ca²⁺ responses.

Figure 3.

Schematic diagram of the ex vivo plasticity system. A dissected Drosophila brain is immobilized between nylon fibers on a platinum grid and placed in a recording chamber with flowing media. A glass electrode is placed against the AL to mimic odor activation, and a second electrode is placed around the AFV to mimic somatosensory (shock) activation. A fluorescent Ca²⁺ sensory is expressed in the MBs to measure MB activity. Co‐incident stimulation of the AFV and the AL induces long‐term enhancement (LTE) of AL‐dependent MB Ca²⁺ responses. (Modified from Ueno et al., 2017)55

Similar to learning, LTE requires dopaminergic activity.54 However, dopamine signaling is not required for transmission of sensory information from either the ALs or the AFV to the MBs. In dissected brains, signal transmission from the ALs to the MBs is mediated by cholinergic receptors, a result consistent with previous in vivo results. Transmission from the AFV to the MBs also does not require dopaminergic receptors in the MBs, and instead requires N‐methyl‐D‐aspartate (NMDA)‐type glutamate receptors.55 Dopamine is released onto the MBs at a step after coincident sensory input to the MBs and is required for plasticity. Plasticity only occurs at locations where dopamine is released, and dopamine release is restricted to MB lobes that have been coincidently stimulated. The finding that transmission of US information to the MBs involves glutamate rather than dopamine, allows us to consider a model that separates, and later integrates, US sensory information from dopaminergic reinforcement signals.

7. INCORPORATING PREDICTION ERRORS INTO DROSOPHILA MODELS

In this review, I define an activity trace (or trace) as a neuronal network that is activated in the fly brain upon exposure to a stimulus. Thus odor exposure will activate an activity trace including, but not limited to, olfactory receptor neurons, projection neurons, MB Kenyon cells and possibly MBONs, while shock exposure will activate a less defined trace from somatosensory receptor neurons to Kenyon cells and possibly MBONs. When flies learn an odor association, plastic changes occur in the odor trace that lead to altered MBON activity and behavioral changes. These changes have previously been referred to as memory engrams or memory traces and are incorporated into specific connections in the odor trace. While my definition of an activity trace is broader than traditional definitions of memory engrams and memory traces, it is useful for this review since not all neuronal connections in my model are associated with plastic changes.

Endel Tulving first proposed in 1972 that human explicit memory could be separated into two categories, episodic memories, which consist of memories of specific events or experiences, and semantic memories, which consist of general knowledge of facts, ideas and concepts.56 While these two types of memory are often studied separately in mammals, they have been shown to be interrelated and they are both thought to contribute to behaviors.57, 58 Indeed, it seems highly likely that knowledge of rules and concepts must be continually modulated by new episodic experiences. While I do not propose that Drosophila have actual episodic and semantic memories per se, this type of separation of memory types can be useful for the calculation of prediction errors. Thus, I propose that a particular odor stimulus activates two traces in the MBs, one, which incorporates associations from the current experience and has similarities to mammalian episodic memory (Figure 4A), and a second, which encodes the conclusions from previous experiences with the odor and has similarities to semantic memory (Figure 4B). The episodic trace in this case does not actually store a memory of the latest episode, but instead contributes to updating the information stored in the semantic trace.

Figure 4.

An updated model for olfactory associative learning. In order to incorporate prediction error theory into learning models, I propose that two separate neural traces exist: an episodic trace that encodes the current or latest experience (A), and a semantic trace that encodes a predicted outcome based on previous experiences (B). The episodic trace does not directly regulate behaviors, but instead transmits the results of the latest experience to the semantic trace, via dopaminergic neurons. The semantic trace regulates behaviors based on previous experiences, and also transmits information regarding expected outcomes to the same dopaminergic neurons as the episodic trace, but with opposite valence. The strength of outputs from the semantic trace is modulated by a prediction error signal. (C) Integrating both episodic and semantic traces, a putative Kenyon cell 1 transmits information about a current experience to dopaminergic neurons, while Kenyon cell 2 transmits expected outcomes to the same neurons with opposite valence. Dopaminergic neuron calculates the difference between these two inputs and transmits the result as a prediction error signal that modulates expected outcomes and subsequent behaviors

Activation of the episodic trace requires coincident exposure to both a US and an odor (Figure 4A). In the case of aversive associations, glutamate signals convey aversive information to a large subset of Kenyon cells. However, in the absence of coincident odor activation, these signals are not transmitted further. In the subset of Kenyon cells that are coincidently activated by an odor, odor activation functions as a gating mechanism allowing aversive US information to be transmitted to downstream dopaminergic neurons.

The semantic trace influences behaviors, and may be activated by odor exposure alone, depending on prior odor experiences (Figure 4B). In this trace, the strength of the connection between odor activation and downstream MBONs is modulated by a prediction dopaminergic error signal regulated in part by the episodic trace. Output from the semantic trace regulates behaviors through connections to executive areas of the brain. In addition, the semantic trace also sends output to the same dopaminergic neurons as the episodic trace, only of the opposite valence. Thus, dopaminergic neurons receive information about the strength of a current US from the episodic trace, as well as information about the predicted US based on previous experiences from the semantic trace. The opposing valences of these signals allow dopaminergic neurons to calculate the difference, or prediction error. This information is then sent back to the semantic trace to update the connection strength between odor‐activated Kenyon cells and outputs to both behavioral and predicted US pathways (Figure 4C).

8. THE EPISODIC AND SEMANTIC TRACE MODEL IS CONSISTENT WITH PREVIOUS DOPAMINE STUDIES

Numerous previous studies have showed that artificial activation of specific subsets of dopaminergic neurons can replace US exposure during memory formation.27, 28, 32 In particular, activation of the PPL subclass of dopaminergic neurons can replace aversive stimuli, and activation of the PAM subclass can replace appetitive stimuli. These results, at first seem inconsistent with findings that glutamate conveys US information to the MBs, but this apparent discrepancy can be reconciled by considering the separate, episodic and semantic traces. While glutamate conveys US information to the episodic trace, dopamine is the neuromodulator that induces plasticity of the semantic trace. Thus, artificial activation of appropriate dopaminergic neurons during odor exposure should bypass the episodic trace to directly enhance semantic memory. As the semantic trace regulates behaviors, odor‐paired activation of either appropriate glutamatergic or dopaminergic neurons should induce US‐associated behaviors. Interestingly, direct artificial activation of dopaminergic neurons should also bypass the mechanism that measures prediction error. Thus, memory after repeated odor‐shock pairing should reach an asymptote as an animal learns to expect a punishment. However, repeated pairing of an odor with PPL activation may not be subject to the same ceiling effect.

9. ODOR SPECIFICITY IN THIS MODEL

Although there are approximately 2000 to 2500 MB Kenyon cells in Drosophila, 59, 60 there are only a few hundred dopaminergic neurons that innervate these Kenyon cells.46 This suggests that while distinct subsets of Kenyon cells encode specific odors and associations, reinforcement of associations requires activity of a similar or overlapping set of dopaminergic neurons. If this is the case, how can dopamine release be restricted to induce plasticity in appropriate subsets of Kenyon cell synapses? Ex vivo studies from Ueno et al suggest that dopamine release is limited to Kenyon cells that have been activated by odor stimuli. If the antennal lobe on only one side of the brain is electrically stimulated (mimicking odor stimulation) while concurrently pharmacologically activating glutamate receptors bilaterally (mimicking shock application), dopamine is released only onto the MBs ipsilateral to the activated antennal lobe.55 Since dopaminergic neurons innervate the MBs bilaterally,46 this indicates that there must be a mechanism that restricts dopaminergic release to activated postsynaptic cells. Ueno et al propose that Kenyon cell that has been activated by an odor send a retrograde signal to pre‐synaptic dopaminergic terminals to gate dopamine release.55 Thus, release requires the convergence of two signals, first, activation of dopaminergic cells from the episodic trace, and second, a gating signal generated by odor‐activated post‐synaptic Kenyon cells. This idea is consistent with previous behavioral results showing that odor‐specific memories can be formed when US presentation is replaced by activation of specific dopaminergic neurons.27, 28, 32 If dopamine release depends only on presynaptic activity, plasticity should occur throughout the MBs, rather than at specific odor‐activated synapses.

10. ARE EPISODIC AND SEMANTIC MEMORIES STORED IN THE SAME OR DIFFERENT KENYON CELLS?

If episodic and semantic memory traces are both located in the MBs, and output from the episodic trace is required to update the semantic trace, synaptic output from the MBs must occur during learning. Consistent with this, output from γ and α′/β′ cells has been shown to be required during acquisition, or shortly thereafter,36, 61 while activity of α/β cells is required during recall.15, 34 Thus γ or α′/β′ cells may encode episodic memory traces, while α/β cells may encode semantic traces.

Alternatively, recent work shows that axons of MB Kenyon cells can be separated into different compartments with distinct inputs and outputs. All axonal compartments of a Kenyon cell are proposed to be activated similarly by odor‐dependent signals from the calyx, but individual compartments have been shown to have distinct dopaminergic inputs and MBON outputs.29, 45 This suggests that episodic and semantic memory traces can be stored in different axonal compartments in the same odor‐activated γ or α′/β′ Kenyon cells (Figure 5).

Figure 5.

A two trace model utilizing different Kenyon cell axonal compartments. A, The model in Figure 4 requires two different Kenyon cells populations responding to the same odor. However, because Kenyon cell axonal compartments behave independently, episodic and semantic traces may be stored in the same Kenyon cells in different compartments. B, When naïve flies are exposed to an odor, odor information flows through appropriate Kenyon cell compartments but does not activate dopaminergic neurons or motor responses. C, When naïve flies are exposed to an odor paired with an unconditioned stimulus (in this case, electrical shocks), the episodic trace located in compartment 1 sends a stimulatory signal to dopaminergic neurons, while the semantic trace located in compartment 2 does not send an inhibitory signal. The resulting prediction error signal from dopaminergic neurons strengthens connections between the semantic trace and predicted outcome and motor response pathways. This causes an increased motor response upon subsequent exposure to the odor (D). In flies that have already learned the association, the current experience and the predicted outcome are balanced, preventing further plastic changes (E)

Information about current experiences needs to be of the opposite valence to that of predicted outcomes in order for dopaminergic neurons to calculate the differences between these inputs. In the models shown in Figures 4 and 5, this prediction error is shown to enhance an odor‐dependent behavioral output. However, various groups31, 45, 62, 63 have showed that dopamine induces synaptic depression of odor‐dependent MBON activity, with aversive associations inhibiting MBONs associated with appetitive responses, and appetitive associations inhibiting MBONs associated with aversion. This information has been incorporated in the model shown in Figure 6.

Figure 6.

A model integrating aversive and appetitive associations. Recent data indicate that aversive and appetitive associations for the same odor are stored in different MB compartments. In this model, responses to an odor are regulated by a balance between appetitive and aversive signals. Aversive responses are induced by an inhibition of appetitive signals, while appetitive responses are induced by an inhibition of aversive signals. Small green and black circles in semantic memory encoding compartments represent synapses whose activity is modulated by prediction error signals. Dotted lines represent other putative connections that may regulate the balance between appetitive and aversive responses

11. INTEGRATION OF APPETITIVE AND AVERSIVE CIRCUITS

Different Kenyon cell axonal compartments are associated with either appetitive or aversive valences.30 PAM dopaminergic neurons, whose activation can replace appetitive stimulation during associative training, project to medial compartments in the MB horizontal lobes, while PPL dopaminergic neurons, whose activation can replace aversive stimulation, project to lateral compartments.28, 33, 46 Thus, odor‐responsive Kenyon cells are able to form both aversive and appetitive associations in different axonal compartments. To prevent both associations from being formed simultaneously to the same odor, it is likely that compartments storing aversive semantic memories inhibit formation of appetitive semantic memories, and vice versa. Indeed, odors with opposing valences show negative correlations in MBON activity.64 Cohn et al, have recently identified a possible mechanism for this effect by showing that activation of dopaminergic inputs to MB compartments associated with aversion inhibit dopaminergic inputs to compartments associated with attraction.45 Conversely, activation of inputs to compartments associated with attraction inhibit activity of inputs to compartments associated with aversion (dotted lines in Figure 6).

12. A SIMPLIFIED MODEL UTILIZING LOCALIZED SYNAPTIC INTERACTIONS

Thus far, we have examined models with two separate episodic and semantic memory traces because this allows dopaminergic neurons to easily calculate the differences between actual and predicted outcomes. In these models, odor‐activated Kenyon cells function to gate the transmission of both US information and predicted outcomes to dopaminergic neurons, thus providing the specificity that allows a small set of dopaminergic neurons to calculate separate prediction errors for many different odors. However, it remains uncertain whether Drosophila is able to actually encode and store episodic memories. Thus, I next consider whether dopaminergic neurons could theoretically calculate prediction errors for various specific odors in the absence of a stored episodic memory trace (Figure 7). In this situation, US information may cause direct depolarization of dopaminergic PPL or PAM neurons. However, in order to prevent a general increase in plasticity at all dopaminergic terminals, this depolarization should be insufficient on its own to induce dopamine release. Instead, release of dopamine at specific synapses should require both US‐dependent depolarization at presynaptic terminals and post‐synaptic depolarization of Kenyon cells responding to the specific CS. I propose that odor exposure activates synapses between odor‐specific Kenyon cells and dopaminergic neurons. This activity controls a localized gating mechanism in dopaminergic neurons, allowing US‐dependent depolarization to induce dopamine release from local presynaptic terminals, but not others. Similar to the model presented in Figure 6, released dopamine inhibits activity of MBONs to toggle appropriate behaviors. In addition, to modulate predicted outcomes, released dopamine also inhibits Kenyon cell‐dependent gating of dopamine release. Thus, when an odor is first paired with a US, a large amount of dopamine is released, but as the number of trainings/pairings increases, and the fly learns the association, odor‐dependent gating of dopamine gradually decreases.

Figure 7.

A more parsimonious model in which a separate episodic trace is replaced with an odor‐activated gating signal. In this model, US information is transmitted directly to dopaminergic neurons (PPL for aversive information, and PAM for appetitive). The relevance of this US information to a specific odor‐associated semantic trace is determined by whether the US is paired with the specific odor. If the US is unpaired, its signal is not transmitted to MB axons. However, upon pairing, odor stimulation activates a local gating signal placing local dopaminergic to MB synapses in an activatable state where coincident US stimulation can induce dopamine release. Dopamine regulates plasticity between MB axons and MBONs to regulate behaviors, and also inhibits the strength of the odor‐activated gating signal, weakening this signal as learning progresses. This release is local and only occurs on synapses from odor‐activated axons

Consistent with the model presented in Figure 7, synapses between Kenyon cells and MBONs, Kenyon cells and dopaminergic neurons, and dopaminergic neurons and MBONs have been recently identified in the MBs a lobes.65, 66 In addition, in mammals, it has been shown that presynaptic dopamine release cannot be explained simply by pre‐synaptic activity.67

13. EXECUTIVE FUNCTION: RECURRENT MB CONNECTIONS MAY FUNCTION DURING CHOICE SELECTION

The olfactory learning and behavior assay first developed by Quinn and Benzer17 has been critical for the study of memory, but it is necessary to keep in mind that this assay measures behavior, rather than memory. While behaviors may be strongly influenced by memories of previous experiences, they also require an executive function to compare memories, make a decision, and activate a motor response. Thus, the memory of an association between an odor A with aversive electrical shocks will produce opposite behaviors depending on whether the alternative choice, odor B, is associated with better or worse outcomes. Where does this decision‐making process occur in the Drosophila brain? While activation of different MBONs induces approach and avoidance behaviors, this result cannot be used to determine whether decision‐making occurs in the MBs upstream of MBON activation, or whether activity of different MBONs is integrated in a downstream executive area of the brain. Heisenberg's model, developed before recurrent feedback loops to the MBs were known, suggests that executive function occurs downstream of the MBs. In contrast, the models that I have presented here suggest that recurrent loops between the MBs and dopaminergic neurons are useful in comparing two different memory traces and measuring their differences. While these models propose that this comparison mechanism integrates new experiences into semantic memories as an update mechanism, similar recurrent loops may be used to compare memories associated with two different olfactory choices to identify an optimal choice. This comparison must occur when the flies are exposed simultaneously to two odors during testing. As output from α/β neurons is required during testing or recall, part of an executive decision‐making function may occur in recurrent loops between α/β MB neurons and extrinsic dopaminergic neurons.

14. PERSPECTIVES

Memory‐associated changes in neuronal activity have been identified in mammals and higher organisms, but the relationship between memory traces, networks and behaviors is still unclear. Drosophila offer an unparalleled opportunity to integrate learning and memory studies from the molecular to the behavioral levels. In particular, critical connections and plasticity in the MBs, from sensory inputs to behavior‐regulating MBONs, are being identified, and it is necessary to determine how these network behaviors correspond to memory systems and organismal behaviors. Here, I have presented simple models for how recurrent connections between MB Kenyon cells and extrinsic dopaminergic neurons may function to encode prediction errors, episodic memories and semantic rules. The central premise of these models is that recurrent loops can be used to compare the strengths of two different Hebbian outputs. Thus, they compare the results of a current experience with the predicted results from previous experiences and use the differences from these outputs to update the predictive network. This comparison mechanism can also be used to compare many different memories. Behavioral testing requires flies to compare the outcomes associated with two different odors and choose the more favorable odor. This likely requires an executive function that compares two associative memories and flexibly and reversibly chooses a motor behavior. A recurrent loop similar to the one we propose may function in this type of function.

Flies exhibit various olfactory associative behaviors that cannot be explained by the models presented here. For example, similar to other organisms, flies can form associations between stimuli that are not presented simultaneously, but instead presented in a predictable temporal order, a type of memory known as trace memory.68, 69, 70 Thus, flies can form associations between a CS and a US when the US is presented after termination of the CS. This type of memory also requires the MBs, suggesting that the MBs maintain a CS memory trace in the absence of a US. It also suggests that the MBs have a mechanism for measuring time. Activity of dopaminergic neurons influences estimation of time in mammals, suggesting that recurrent loops with dopaminergic neurons may also function in trace memory function. In addition, associative memories can be transferred to different CSs.71 Thus, if one CS is paired with a US, and later this CS is paired with a second CS, animals can infer that the second CS is predictive of the US. While the network connections allowing these types of associations is still unknown, recurrent loops that function to compare and modify associations will likely be important for these associations as well.

CONFLICT OF INTEREST

The author declares no conflicts of interest.

Horiuchi J. Recurrent loops: Incorporating prediction error and semantic/episodic theories into Drosophila associative memory models. Genes, Brain and Behavior. 2019;18:e12567 10.1111/gbb.12567