Re-evaluation of learned information in Drosophila

Abstract

Animals constantly reassess the reliability of learned information to optimize their behavior. On retrieval, consolidated long-term memory can be neutralized by extinction if the learned prediction was inaccurate ¹. Alternatively, retrieved memory can be maintained, following a period of reconsolidation during which it is labile ². Although extinction and reconsolidation provide opportunities to alleviate problematic human memories ^3–5, we lack a detailed mechanistic understanding of memory updating processes. Here we identify neural operations underpinning re-evaluation of memory in Drosophila. Reactivation of sugar-reinforced olfactory memory can lead to either extinction or reconsolidation, depending on prediction accuracy. Each process recruits activity in specific parts of the mushroom body output network and distinct subsets of reinforcing dopaminergic neurons. Memory extinction requires output neurons with dendrites in the α and α′ lobes of the mushroom body, which drive negatively reinforcing dopaminergic neurons that innervate neighbouring zones. The aversive valence of these new extinction memories neutralizes previously learned odor preference. Memory reconsolidation requires the γ2α′ 1 mushroom body output neurons. This pathway recruits negatively reinforcing dopaminergic neurons innervating the same compartment and re-engages positively reinforcing dopaminergic neurons to reconsolidate the original reward memory. These data establish that recurrent and hierarchical connectivity between mushroom body output neurons and dopaminergic neurons enables memory re-evaluation driven by reward prediction error.

Valence of Drosophila olfactory memory is coded as a skew in the mushroom body (MB) output network ^6,7 (Extended Data Fig. 1a-f). Reward memories are written by dopaminergic neurons (DANs) that innervate the horizontal MB lobes ^8,9, which depress odor drive to mushroom body output neuron (MBON) pathways that direct avoidance ¹⁰. Aversive memories are reinforced by DANs that primarily innervate the vertical MB lobes ^11,12 and reduce odor drive to MBONs directing approach ^6,13. Learned preference is weakened (or extinguished) in flies that experience the conditioned odor without reinforcement ^14–16, suggesting that flies reassess learned predictions (Extended Data Fig. 1g-i). We investigated mechanisms of re-evaluation of sugar-rewarded appetitive memory (Fig. 1a). Food-deprived flies were trained with one odor without reinforcer (Conditioned Stimulus minus, CS-), followed by air, then another odor (Conditioned Stimulus plus, CS+) with sugar reward ¹⁷. This training establishes long-term memory that is consolidated within hours of training ¹⁷. We challenged memory 3 h after training with two presentations of CS+, CS-, a novel odor (specificity control) or air (handling control), and then 3 h later tested the flies’ preference between CS+ and CS-. Flies exposed to CS-, air or novel odor displayed robust 6 h memory performance (Fig. 1a, Extended Data Fig. 2a). However, CS+ exposure at 3 h abolished 6 h performance. Memory was similarly reduced following one or three CS+ presentations at 3 h (Extended Data Fig. 2b) and remained depressed for at least 24 h (Extended Data Fig. 2c). Importantly, odor exposure alone in naïve flies did not change preference measured 3 or 21 h later (Extended Data Fig. 2d). Therefore, sugar-rewarded memories can be extinguished by presenting the conditioned stimulus after training.

Figure 1. Extinction of reward memory requires negatively reinforcing dopaminergic neurons.

a, Only CS+ evoked memory reactivation at 3 h leads to extinction of appetitive memory (n≥8). b, Blocking rewarding DANs in the protocerebral anterior medial (PAM) cluster during 3 h CS- or CS+ re-exposure did not alter extinction or 6 h learned approach (n≥15). c, Blocking aversive DANs in the paired posterior lateral 1 (PPL1) cluster during CS+ reactivation significantly impairs extinction, while block during CS- reactivation leads to loss of memory (n≥10). d, Blocking PPL1 DANs without reactivation does not alter 6 h performance (n≥10). Unless otherwise noted, in all figures data represent the mean ± standard error of the mean (s.e.m.). Asterisks (*) denote significant difference (p<0.05, ANOVA) between groups of same genotype treated differently, hash (#) denotes significant difference (p<0.05, ANOVA) between different genotypes treated identically. A break in the x-axis indicates independent experiments.

Since distinct DANs reinforce aversive ^11–12 and appetitive memories ^8,9 we tested each group’s involvement in memory extinction. We blocked DAN output with dominant temperature-sensitive uas-shibire^ts1 (shi^ts1) ¹⁸. At temperatures >29°C, shi^ts1 blocks membrane recycling and thus synaptic vesicle release; at <25°C, synaptic release is restored. All flies were trained at permissive 23°C, raised to 33°C 30 min before and during reactivation at 3 h, immediately returned to 23°C and tested for memory at 6 h. Blocking rewarding DANs did not impair extinction (Fig. 1b): R58E02; uas-shi^ts1 flies retained no measurable 6 h memory following CS+ exposure at 3 h but displayed robust performance following CS- exposure. We also tested whether, without a DAN-mediated reward signal to the MB, appetitive memory would be extinguished, by retraining flies at 3 h while blocking rewarding DANs. Perhaps surprisingly, R58E02; uas-shi^ts1 flies retained appetitive memory performance that was indistinguishable from their controls (Extended Data Fig. 2e), consistent with attenuation of rewarding DAN signals when learned performance is maximal ¹⁹.

We blocked aversively reinforcing DANs specifically during memory reactivation using TH-GAL4 and more restricted MB504B-GAL4 to express uas-shi^ts1 (Fig. 1c). Extinction driven by CS+ exposure was compromised in TH-GAL4; uas-shi^ts1 and MB504B; uas-shi^ts1 flies. In addition, when TH-GAL4 or MB504B output was blocked during 3 h CS- exposure, 6 h memory was decreased. Neither disruption was evident in experiments at 23°C (Extended Data Fig. 2f). Moreover, blocking aversive DANs did not impact 6 h memory in flies exposed to no odor or a novel odor at 3 h, (Fig. 1d and Extended Data Fig. 2g) nor odor preference of untrained flies 3 h after odor exposure (Extended Data Fig. 2h). Therefore, aversive DANs play a prominent role in appetitive memory extinction and blocking them also uncovers an unexpected CS- role in memory re-evaluation.

Prior work suggests reward omission during CS+ exposure might alter predictability of learned information and drive extinction learning. Small changes in a subsequent training trial instead trigger reconsolidation, which maintains the original memory ^20,21. Since CS- exposure represents part of a learning trial, we tested whether it induced reconsolidation. Drosophila reward memory consolidation is apparent as a time-dependent resistance to cold-shock anesthesia ¹⁷. We therefore followed 3 h CS- driven memory reactivation with 2 min of cold-shock (Fig. 2a). Cold-shock immediately or 30 min after CS- exposure abolished 6 h appetitive memory, but not if anesthesia was applied without reactivation or 90 min after CS- exposure (Fig. 2a-b, Extended Data Fig. 3a). Day old memories also became sensitive to cold-shock when reactivated with CS- exposure 21h after training (Extended Data Fig. 3b). CS- exposure therefore induces time-dependent memory reconsolidation. Blocking aversive DANs during CS- exposure (Fig. 1c) revealed a similar effect to that of cold-shock afterwards (Fig. 2a-b), suggesting reconsolidation ordinarily requires aversive DANs.

Figure 2. Reconsolidation of reward memory is triggered by CS- exposure and requires MV1/PPL1-γ2α′1 dopaminergic neurons.

a, Re-exposing trained flies to CS- odor renders reward memory sensitive to cold-shock anesthesia (n≥7) b, Memory remains sensitive 30 min after CS- reactivation but returns to a cold-shock resistant state by 90 min (n=10). c, Blocking MB-MV1 DANs during CS- reactivation abolishes 6 h learned approach but blocking during CS+ reactivation leaves extinction intact (n≥12). d, MB-MV1 block after CS- reactivation does not significantly impair 6 h performance (n≥23). e, MB-MV1 output is dispensable during 3 h memory retrieval (n≥14). f, Blocking MB-MV1 DANs during CS- reactivation abolishes 6 h approach towards the CS+ (n≥11).

To define aversive DANs responsible for reconsolidation and extinction we blocked MB-MP1 (PPL1-γ1pedc), MB-MV1 (PPL1-γ2α′1), MB-V1 (PPL1-α′2α2) or PPL1-α3 DANs during memory reactivation with either CS+ or CS- odors and measured 6 h memory (Fig. 2c and Extended Data Fig. 2c-f). None of these manipulations significantly impaired CS+ driven memory extinction suggesting extinction does not rely on individual aversive DANs. In contrast, blocking MB-MV1 DANs significantly reduced 6 h memory when done during CS- memory reactivation (Fig. 2c and Extended Data Fig. 2f), but not immediately or 1.5 h after CS- exposure (Fig. 2d and Extended Data Fig. 3g) or in experiments at 23°C (Extended Data Fig. 3 h). Furthermore, MB-MV1 DANs are dispensable for learning and expression of 3 h appetitive memory (Fig. 2e and Extended Data Fig. 3i). We tested whether the CS+ odor memory was affected after presenting the CS- while blocking MB-MV1 DANs, by testing the flies’ preference between CS+ and a novel odor, rather than CS+ and CS-. Whereas controls robustly chose CS+ over the novel odor, flies in which MB-MV1 DANs were blocked during CS- re-exposure displayed no CS+ memory (Fig. 2f). Therefore output from MB-MV1 DANs throughout, but not after, CS- driven memory reactivation is necessary to reconsolidate memory of the CS- and CS+ relationship.

Individual DANs tile the MB lobes into 15 discrete compartments, each containing the dendrites of a corresponding MBON ²². Anatomy suggests MBONs then connect to DAN dendrites, forming recurrent MB-MBON-DAN-MB feedback loops ²² (Extended Data Fig. 1c). Since sugar-reward learning inhibits relative CS+ odor drive to horizontal lobe tip MBONs ¹⁰ we tested MBON involvement in extinction and reconsolidation. We again restricted disruption to CS+ or CS- driven memory reactivation at 3 h and tested 6 h memory. MB052B-GAL4 labels cholinergic V2 MBONs (α2sc, α′1, α2p3p, α′3ap and α′3m) whose dendrites overlap with aversive DANs required for extinction. Blocking V2 MBONs impaired CS+ driven memory extinction and not CS- driven reconsolidation (Fig. 3a). No defect was apparent when adjacent, MVP2 MBONs were blocked during memory reactivation (Extended Data Fig. 4a) or in V2 MBON experiments at 23°C (Extended Data Fig. 4b). Moreover, V2 MBONs are dispensable for 3 h memory expression (Fig. 3b), suggesting they serve a function distinct from behavioral direction.

Figure 3. Reward memory extinction requires V2 MBONs that drive aversively reinforcing dopaminergic neurons.

a, Blocking V2 cluster MBONs significantly impairs CS+ driven extinction but spares CS- induced reconsolidation (n≥9). b, V2 MBONs are not essential to express 3 h memory performance (n≥9). c-e, Light-triggered activation (red bar) of lexAop-CsChrimson expressing R71D08-LexA V2 MBONs reproducibly evoked calcium responses in 2 of 3 MB504B-GAL4/ uas-GCaMP6f PPL1 DAN cell bodies per animal (Region of interest, ROI, indicated on schematic as dashed box). Data points from cells recorded in same individual fly shown in same color. One neuron was activated (cell 1), one inhibited (cell 3) and responses in the other (cell 2) did not reach significance. Time points (arrows) quantified before and after LED. * denotes significant difference (p<0.05, t-test) between pre- and post activation responses.

To test functional connectivity between V2 MBONs and aversive PPL1 DANs we light-stimulated V2 MBONs expressing lexAop-CsChrimson, while recording activity in PPL1 DANs with uas-GCaMP6f. V2 MBON activity produced a robust calcium response in 2 of 3 PPL1 DAN cell bodies (Fig. 3 c-e and Extended Data Fig. 4c-d), with one activated and the other inhibited. Dopaminergic plasticity in γ1 and α2 MB zones has been implicated in aversive odor memory ^13,23,24 and DANs respond differently to odors after aversive training ²⁵. However, we detected no obvious change in odor-drive of V2 MBONs or PPL1 DANs after reward training (Extended Data Fig. 4e-f). Considering connectivity and behavioral data with prior demonstration of reduced drive to horizontal lobe MBONs after reward training ¹⁰ (Extended Data Fig. 1d-f), we propose that CS+ in trained flies drives an MBON network skewed towards V2 MBONs. This in turn preferentially activates MB-MP1, and perhaps MB-MV1 and PPL1-α3, DANs which write a new, parallel and competing CS+ specific aversive memory. This manifests behaviorally as extinction of the original reward memory (Extended Data Figure 1g-i).

We next investigated cholinergic γ2α′1 MBONs whose dendrites overlap with MB-MV1 DANs required for reconsolidation. These MBONs have to potentially accommodate two functions: explain how CS- becomes represented after training and how CS- induces reconsolidation of the CS+ memory. Like MV1 DANs, blocking γ2α′1 MBONs during, but not after, CS- driven memory reactivation impaired memory reconsolidation (Fig. 4a and Extended Data Fig. 5a), while blocking γ2α′1 MBONs during CS+ reactivation did not disrupt extinction (Fig. 4a). However, unlike MB-MV1 DANs, or V2 MBONs, blocking the γ2α′1 MBONs abolished learned approach at 3 h (Fig. 4b and Extended Data Fig. 5b) demonstrating that γ2α′1 MBONs direct memory-guided behavior as well as odor re-evaluation.

Figure 4. The γ2α′1 MBONs orchestrate CS- triggered reconsolidation.

a, Blocking γ2α′1 MBONs during CS- reactivation significantly impairs reconsolidation of 6 h memory but spares CS+ driven extinction (n≥12). b, γ2α′1 MBON output is required for 3 h memory expression (n≥15). c, Light-triggered activation (red bar) of R25D01-LexA or MB077C-GAL4 γ2α′1 MBONs evokes calcium responses in aversive MB296B-GAL4 MB-MV1 DANs and in d, rewarding R58E02-LexA DANs. Arrows, time points quantified before and after LED. For c and d * denotes significant difference (p<0.05, t-test) between pre- and post activation responses. e, Sugar-reward training specifically enhances CS- odor-evoked calcium responses in γ2α′1 MBONs (n≥13). Responses to CS-, CS+, novel odor were measured in projections outside the MB (example traces, lower left panel). Calcium transients during CS- and CS+ re-exposure were normalized to responses recorded in the same preparation to novel odor (IAA). Solid lines, mean response; shaded area, s.e.m., color background, odor presentation. Groups trained with MCH or OCT are shown separately. f, Blocking rewarding DANs immediately, but not 1.5 h, after CS- re-exposure abolishes reconsolidation of 6 h memory (n≥9).

Anatomy suggests that γ2α′1 V2 MBONs connect with dendrites of MB-MV1 and rewarding DANs that innervate the horizontal MB lobes ²². We therefore assessed functional connectivity by stimulating γ2α′1 MBONs while monitoring activity in either MB-MV1 or rewarding DANs. Light-triggered activation of γ2α′1 MBONs evoked robust calcium responses in MB-MV1 (Fig. 4c) and rewarding DANs (Fig. 4d). To then determine whether γ2α′1 MBONs contained a trace of reward learning, we expressed uas-GCaMP6f in γ2α′1 MBONs and performed two-photon functional calcium imaging of CS- and CS+ odor responses (normalized to those of novel odor) 3 h after sugar-reward or mock training (Fig. 4e). Responses evoked after training in γ2α′1 MBONs by either of two CS- odors were significantly increased over those in mock-trained flies. CS+ evoked responses were not statistically different between groups. Therefore, reward learning produces a CS- specific increased response in γ2α′1 MBONs.

Since γ2α′1 MBONs also connect to rewarding DANs (Fig. 4d), we tested whether reconsolidation involved signaling from rewarding DANs after memory reactivation. Flies were trained and 3 h memory was reactivated with CS+ or CS- odors (Fig. 1b). Blocking rewarding DANs immediately, but not 90 min, after memory reactivation abolished 6 h memory (Fig. 4f). Therefore, reconsolidation results from increased CS- drive to γ2α′1 MBONs after training that recruits recurrent MB-MV1 DANs and also re-engages rewarding DANs to reinstate the original CS+ reward memory. We cannot yet establish what generates different temporal requirements for MB-MV1 and rewarding DANs, or the nature of updating memory.

Our data demonstrate that flies re-evaluate memory at retrieval using discrete modules of their MB-directed dopaminergic system. Different pathways are engaged depending on whether the difference between the learned expectation and the actual experience is large or small, yielding either memory extinction or reconsolidation, respectively. It will be important to determine whether dopamine based error-prediction ¹⁹ and reassessment of learned information in mammals ^26–30 utilizes similar neural network motifs and operating logic.

Methods

Fly strains

All fly strains (Drosophila melanogaster) were raised on standard cornmeal-agar food with additional molasses and active dried yeast at 25°C in 40-50% humidity. The wild-type strain was Canton-S. GAL4 lines used (Extended Data Fig. 6) have been described: TH-GAL4 ³¹, R58E02-GAL4 ⁸, MB052B-, MB058B-, MB077C-, MB083C-, MB112C-, MB296B-, MB308B- and MB504B-GAL4 ²², R73F07-GAL4 ³² and c061-GAL4;MBGAL80 ³³, uas-shi^{ts1 18}. LexA lines are described: R24H08-, R25D01-, R65B09- and R71D08-LexA ³², R58E02-LexA ⁸. The reporter uas-GCaMP6f, lexAop-GCaMP6m ³⁴ and optogenetic trigger flies uas-CsChrimson::mVenus ³⁵, lexAop-CsChrimson::tdTomato,uas-GCaMP6f ³⁶ are described.

Behavior

For behavior experiments males from GAL4 lines were crossed to uas-shi^ts1 females, except in the case of the c061-GAL4;MBGAL80 crosses where uas-shi^ts1 males were crossed to c061-GAL4;MBGAL80 females. For heterozygous controls GAL4 or uas-shi^ts1 flies were crossed to Canton-S. All flies were raised at 25°C and mixed sex populations of 4-9 day old flies were used in all experiments. Flies were starved for 18-24 h on 1% agar prior to training and were kept starved for the entire experiment. Appetitive training was performed and quantified as described ¹⁷. In brief, flies were exposed to the CS- for 2 min followed by 30 s of room air and then 2 min of the CS+ in the presence of dry sucrose. For memory reactivation experiments flies were given 2X 2 min exposures (spaced by 15 min) of the CS- odor, the CS+ odor (without reward), a novel odor, or room air. During the final memory test flies were given 2 min to choose between two odors in a T-maze: either the CS+ versus the CS-, or the CS+ versus a novel odor. 3-octanol (OCT, 7 µl in 8 ml mineral oil) and 4-methylcyclohexanol (MCH, 7-12 µl in 8 ml mineral oil) were used as trained odors and isoamyl acetate (IAA, 7 µl in 8 ml mineral oil) and ethyl butyrate (EB, 7 µl in 8 ml mineral oil) were used as novel odors in all experiments, except Fig. 2f and Extended Data Fig. 2d and 3a. In Fig. 2f ethyl butyrate was used as CS- and 3-octanol and 4-methylcyclohexanol were used as CS+ or as novel odor. In Extended Data Fig. 2d and 3a, IAA and EB were used as trained odors. For cold-shock anesthesia flies were transferred into pre-chilled plastic vials and kept on ice for 2 min as described previously ^17,37. For neural inactivation experiments with uas-shi^ts1 all flies were shifted to restrictive 33°C. Restrictive temperature was imposed immediately after reactivation by transferring flies to pre-heated vials containing 1% agar.

Explant Brain Two-Photon Calcium Imaging

Prior to all optogenetic experiments flies were housed for 1–3 days on standard cornmeal food supplemented with 1 mM retinal. Combined optogenetic and calcium imaging experiments were conducted using a two-photon microscope (Scientifica). Explant brains were placed on a polylysine-coated glass cover slip bathed in carbogenated (95% O₂, 5% CO₂) buffer solution (103 mM NaCl, 3 mM KCl, 5 mM N-Tris, 10 mM trehalose, 10 mM glucose, 7 mM sucrose, 26 mM NaHCO₃,1 mM NaH₂PO₄, 1.5 mM CaCl₂, 4 mM MgCl₂, osmolarity 275 mOsm [pH 7.3]) following dissection in cold calcium-free buffer. For light stimulation a high-power LED (Multicomp OSW-6338, 630 nm) was relayed onto the specimen via a 50 mm diameter lens with focal length 60 mm filtered through a 632/10 bandpass filter (Edmund Optics). Power at the specimen was measured to be 0.85 mW/mm². The LED was triggered using a microcontroller (Arduino MEGA). After rapid identification of focus on the respective field of view, brains were left to settle for 5 min before being imaged. Following 10 sec of baseline recording, a light pulse was delivered at 40 Hz, with 10 ms duration for a total of 200 ms (Fig. 4d) or 500 ms (all other experiments). Fluorescence (F) was excited using 140 fs pulses, 80 MHz repetition rate, centered on 910 nm generated by a Ti-Sapphire laser (Chameleon Ultra II, Coherent). Images of 256 X 256 pixels were acquired at 5.92 Hz, controlled by ScanImage 3.8 software ³⁸. PPL1 DANs were imaged at the level of the cell body in order to avoid inadvertent CsChrimson stimulation from the two-photon imaging laser. PAM DANs were imaged at the level of the γ5 compartment. In general, light exposure to the brain was kept at a minimum. Two-photon fluorescence images were manually segmented using ImageJ and further analyzed using custom-written MATLAB scripts. For quantification, F₀ was defined as the mean F from the 9 s prior until 1 s before stimulation. ΔF/F₀ was compared between 1 s before stimulation with 1 s after stimulation onset, using a paired t-test.

In Vivo Two-Photon Calcium Imaging

Two-photon imaging of odor-evoked calcium responses was performed as described ¹⁰. 3-8 day old flies were briefly (5-10 sec) immobilized on ice and mounted in a custom chamber 2.5-3.5 h after appetitive training using MCH or OCT (see Behavior section) or mock treatment (odor presentation without sugar). Legs and proboscis were immobilized with wax to reduce movement artifacts. The head capsule was opened under room temperature carbogenated buffer (see above). Odors were delivered on a clean air carrier stream using a custom-designed system ³⁹, which also synchronizes timing of odor delivery and two-photon image acquisition. Two-photon fluorescence images were acquired as described above and manually segmented using ImageJ. Movement of the animal was small enough such that images did not require registration. All subsequent analyses utilized custom-written Matlab routines. After 40 s of clean air, flies were exposed to 5 s of either trained odor (MCH or OCT; air stream passing over 100X odor dilution in mineral oil, and then further blended 1:9 with a clean air stream), then 15 s clean air, followed by 5 s of the other odor (MCH or OCT) pulse, then 15 s clean air, followed by 5 s of a novel odor (IAA). The protocol was repeated three times and only the first presentation is shown in the figures. We excluded flies from further analysis if they did not show any visible odor responses. In Figure 4e, 15 out of 73 flies were excluded (mock group: 9, trained group: 6). In Figure S4e, 9 out of 56 flies were excluded (mock: 5, trained: 4). In Figure S4f, 3 out of 34 flies were excluded (mock MB-MP1: 1, trained MB-MP1: 3). For quantification, baseline fluorescence F₀ was defined for each stimulus response as the mean F from 1 s before up to the point of stimulation. F/F₀ accordingly describes the fluorescence relative to this baseline. The area under the curve (a.u.c.) was measured as the integral of F/F₀ during the 5 s between odor stimulation onset and offset. In order to reduce the level of fly-by-fly variance, CS+ and CS- were further normalized by dividing each a.u.c. by the novel odor a.u.c. from the respective trial and the same fly, thus obtaining a normalized a.u.c.

Immunohistochemistry

Brains were dissected in ice-cold PBS and fixed for 20 min in 4% paraformaldehyde in PBS, then stained as described ⁴⁰. Brains were incubated for 1-2 days with primary antibodies against GFP (chicken, abcam 13970, diluted 1:2000) and Bruchpilot (mouse, DSHB nc82, diluted 1:50) followed by 1-2 days with secondary antibodies (anti-chicken Alexa 488 diluted 1:1000 and anti-mouse Alexa 633 diluted 1:200, Life Technologies), interspersed with washes in PBS + 0.3% Triton. Brains were mounted in Vectashield medium (Vector Labs), then the immunostained fluorescent signal and (when present) native RFP fluorescence were imaged using a Leica SP5 confocal microscope at 25X magnification, using manually adjusted laser and gain settings.

Statistics

Statistical analyses were performed in GraphPad Prism 6. All behavioral data was tested for normality using the D’Agostino and Pearson omnibus test. Normally distributed data were analyzed with one-way ANOVA followed by Tukey’s honest significant difference (HSD) post hoc test. For non-Gaussian distributed data, Kruskall-Wallis test was performed followed by Dunn’s multiple comparison test. Odor response normalized a.u.c. was compared between groups using multiple t-tests with Holm-Sidak correction. The data supporting the findings of this study are available from the corresponding author upon reasonable request.

Extended Data

Extended Data Figure 1 related to Figure 1, 2, 3 and 4. Extinction and reconsolidation of reward memory requires distinct subsets of dopaminergic neurons that are directed by recurrent and hierarchical connections within the mushroom body output network.

a, Aversively reinforcing DANs in the paired posterior lateral 1 (PPL1) cluster innervate discrete regions of the vertical MB lobe whereas individual rewarding DANs in the protocerebral anterior medial (PAM) cluster innervate unique zones on the horizontal lobe. b, Each zone innervated by a particular DAN houses the dendritic field of a corresponding MBON. Aversive DANs overlap with the dendrites of MBONs directing behavioral approach whereas rewarding DANs overlay the dendrites of MBONs driving avoidance. c, The presynaptic fields of many MBONs overlap with the dendrites of DANs that innervate the same MB zones suggesting the presence of local recurrent feedback loops. d, The weight of behavioral drive to approach and avoidance directing MBONs is balanced in naïve flies (orange and blue circles of equal size). e, Sugar conditioning engages rewarding DANs that innervate the tips of the horizontal MB lobes and that drive depression of synaptic connections between odor-activated MB Kenyon cells and MBONs. f, Following reward conditioning the CS+ drive to avoidance directing MBONs is reduced (smaller orange circle) thereby favoring activation of odor-driven behavioral approach pathways. g, The reward learning induced skew in the MBON network is expressed when flies re-encounter the CS+ odor. Preferential CS+ drive of approach directing MBONs in turn activates aversively reinforcing PPL1 DANs which feed back to encode a competing aversive odor memory. h, We propose that, after this extinction process, reduced CS+ drive to approach directing MBONs (smaller blue circle) equals, and so neutralizes, the previously coded approach memory (smaller orange circle). i, Example of fly behavior. Naïve flies approach odors equally as a result of equal drive to avoidance and approach MBON pathways. Reward conditioned flies exhibit odor preference as a result of reduced CS+ drive to avoidance MBONs. Extinction restores the balance by reducing the CS+ drive to the approach MBONs. j, Sugar conditioning establishes enhanced CS- odor-drive to γ2α′1 MBONs. k, During memory reactivation the CS- odor drives the γ2α′1 MBON which activates the MV1 DAN that feeds back and releases dopamine within the same MB compartment. This activity is required at the time of odor re-exposure to induce memory reconsolidation. l, CS- memory reactivation of the γ2α′1 MBON also activates rewarding DANs that innervate the horizontal MB lobe tips, and that were earlier required for the formation of the original reward memory. The output of these rewarding DANs is required for a restricted period of time after odor exposure to reconsolidate memory.

Extended Data Figure 2 related to Figure 1. Extinction of reward memory requires negatively reinforcing dopaminergic neurons.

a, 6 h reward memories for other odors (IAA and EB) can also be extinguished with two CS+ odor exposures 3 h after training (n≥4). b, One or three odor exposures 3 h after training, varying the memory reactivation regimen of Fig. 1, abolish odor preference behaviour of trained flies measured 3 h later (n≥6). c, Memory extinguished with two odor exposures at 3 h remains low 24 h after training (n=8). Spontaneous recovery of the initial reward memory is not obvious with our current training and extinction protocols ⁴¹. d, Two odor exposures, matching the memory reactivation regimen of Fig. 1, do not change the odor preference behaviour of naïve flies measured 3 or 21 h later (n=7). e, Blocking a reinforcement signal from rewarding R58E02-GAL4 DANs during retraining does not induce memory extinction (n≥9). f, Permissive temperature control for Fig. 1c. No differences in CS+ directed extinction or approach behavior following CS- exposure are apparent when the experiment in Fig. 1c is performed at permissive 23°C throughout (n≥7). g, Exposing flies to novel odors, IAA or EB, while MB504B-GAL4 PPL1 aversive DANs are blocked does not significantly impact 6 h memory performance (n≥7). h, Blocking aversive PPL1 DANs during odor pre-exposure in naïve flies does not attach a value to the pre-exposed odor (n≥9).

Extended Data Figure 3 related to Figure 2. Reconsolidation of reward memory is triggered by CS- exposure and requires MB-MV1 dopaminergic neurons.

a, Reward memories formed with other odors (IAA and EB) can also be rendered sensitive to cold-shock by reactivating them with CS- exposure 3 h after training (n=9). b, Reward memories can also be made labile by reactivation 21 h after training (n=10). c, Extinction of reward memory is insensitive to blocking small groups (<3 neurons per hemisphere) or individual classes of aversive PPL1 DANs during CS+ driven memory reactivation. Blocking c, MB-MP1 (n≥10); d, MB-V1 (b, n≥9) or e, PPL1-α3 and PPL1-α′3 (n≥6) during CS- reactivation leaves 6 h memory performance unaltered. f, Manipulating the MB-MV1 DANs with the alternative driver R73F07-GAL4 during reactivation confirms a specific role in CS- driven memory reconsolidation as seen with MB296B-GAL4 in Fig. 2c. Blocking R73F07-GAL4 neurons during CS+ reactivation does not affect reward memory extinction (n≥14). g, Blocking MB-MV1 DANs (MB296B-GAL4) 1.5 h after CS- exposure does not impair reconsolidation (n≥12). h, Permissive temperature control for Fig. 2c and Supplementary Fig. 3f. CS- reactivation at permissive temperature does not change 6 h approach memory performance (n≥8). i, MB-MV1 neurons are not required to form a 3 h sugar-rewarded memory (n≥8).

Extended Data Figure 4 related to Figure 3. Reward memory extinction requires V2 MBONs that drive negatively reinforcing dopaminergic neurons.

a, Blocking the GABAergic MVP2 MBONs (MB112C-GAL4) during CS- or CS+ triggered memory reactivation does not significantly impact 6 h conditioned approach behavior or CS+ driven extinction (n≥8). b, Permissive temperature control for Fig. 3a. Presenting the CS+ exposure at 23°C does not change the extinction of reward memory in V2 MBON MB052B GAL4; uas-shi^ts1 flies (n≥8). c, Light-triggered activation (red bar) of R65B09-LexA V2 MBONs or d, R24H08-LexA V2 MBONs evokes calcium responses in PPL1 DANs. For c and d * denotes significant difference (p<0.05) between pre- and post activation responses. e, Sugar-reward training does not alter CS+ or CS- odor-evoked calcium responses in V2 MBONs (n≥11). Responses to CS-, CS+ and novel odor were measured in a section through the α2 region of the vertical MB lobe (example traces, lower left panel). Calcium transients during CS- and CS+ re-exposure were normalized to responses recorded in the same preparation to novel odor (IAA). f, Sugar-reward training does not alter CS+ or CS- odor-evoked calcium responses in MB-MV1 or MB-MP1 DANs (n≥7). Responses to CS-, CS+ and novel odor were measured in a section through the α2 or γ1 region of the MB (example traces, lower left panel). Calcium transients during CS- and CS+ re-exposure were normalized to responses recorded in the same preparation to novel odor (IAA). N.B. Order of CS+ and CS- odor presentation is reversed for MB-MV1 and MB-MP1 experiments.

Extended Data Figure 5 related to Figure 4. The γ2α′1 MBONs orchestrate CS- triggered reconsolidation.

a, Blocking the cholinergic MBON-γ2α′1 (MB077C-GAL4) after CS- exposure does not impair memory reconsolidation (n≥10). b, Permissive temperature control for Fig. 4b. No defect in 3 h memory performance is apparent when the entire experiment is conducted at permissive 23°C (n≥11).

Extended Data Figure 6 related to Figures 1-4. The expression patterns of all GAL4 and LexA lines used in this study.

Panels a-k show GFP expression driven by the relevant GAL4 (green), LexA driven RFP expression in MB Kenyon cells (red) and general neuropil stained with an antibody to the Bruchpilot presynaptic marker (blue). a, R58E02-GAL4 broadly labels rewarding DANs in the PAM cluster including PAM-α1, PAM-β1(MVP1), PAM-β1ped, PAM-β2, PAM-β'1ap, PAM-β'1m, PAM-β'2a PAM-β'2m, PAM-β'2p, PAM-γ3, PAM-γ4<γ1γ2, PAM-γ4, PAM-γ5. b, TH-GAL4 broadly labels DANs throughout the brain including all six MB innervating PPL1-DANs, PPL1- γ1pedc (MB-MP1), PPL1- γ2α'1, PPL1-α'2α2 (MB-V1), PPL1-α3, PPL1-α'3. c, MB504B-GAL4 labels PPL1- γ1pedc (MB-MP1), PPL1- γ2α'1, PPL1-α'2α2 (MB-V1), PPL1-α3. d, MB296B-GAL4 and e, R73F07-GAL4 label PPL1- γ2α'1 neurons. f, c061-GAL4:MBGAL80 labels PPL1-γ1pedc (MB-MP1). g, MB058B-GAL4 labels PPL1-α'2α2 (MB-V1). h, MB308B-GAL4 includes PPL1-α'3 and weak expression in PPL1-α3. i, MB122C labels MBON-γ1pedc>α/β (MB-MVP2). j, MB052B-GAL4 labels MBON-α'1, MBON-α2sc (MB-V2α), MBON-α2p3p, MBON-α'3ap (MB-V2α'3) and MBON-α'3m (MB-V2α'3). k, MB077C-GAL4 labels MBON- γ2α'1. Panels l-p show GFP expression driven by the relevant LexA (green) and general neuropil stained with an antibody to the Bruchpilot presynaptic marker (blue). l, R65B09-LexA labels MBON-α'1, MBON-α2sc (MB-V2α), MBON-α2p3p, MBON-α'2 (MB-V4), MBON-α'3ap (MB-V2α'3) and MBON-α'3m (MB-V2α'3). m, R71D08-LexA includes MBON-α2sc (MB-V2α), MBON-α'3ap (MB-V2α'3) and MBON-α'3m (MB-V2α'3). n, R24H08-LexA includes MBON-α'1, MBON-α'3ap (MB-V2α'3) and MBON-α'3m (MB-V2α'3). o, R58E02-LexA includes PAM-α1, PAM-β1(MVP1), PAM-β1ped, PAM-β2, PAM-β'1ap, PAM-β'1m, PAM-β'2a, PAM-β'2m, PAM-β'2p, PAM-γ3, PAM-γ4<γ1γ2, PAM-γ4, PAM-γ5. p, R25D01-LexA includes MBON-γ2α'1.