Prediction error and trace dominance determine the fate of fear memories after post-training manipulations

Joaquín M Alfei; Roque I Ferrer Monti; Victor A Molina; Adrián M Bueno; Gonzalo P Urcelay

doi:10.1101/lm.038513.115

. 2015 Aug;22(8):385–400. doi: 10.1101/lm.038513.115

Prediction error and trace dominance determine the fate of fear memories after post-training manipulations

Joaquín M Alfei ^1,⁴, Roque I Ferrer Monti ^1,⁴, Victor A Molina ², Adrián M Bueno ¹, Gonzalo P Urcelay ^3,^✉

PMCID: PMC4509917 PMID: 26179232

Abstract

Different mnemonic outcomes have been observed when associative memories are reactivated by CS exposure and followed by amnestics. These outcomes include mere retrieval, destabilization–reconsolidation, a transitional period (which is insensitive to amnestics), and extinction learning. However, little is known about the interaction between initial learning conditions and these outcomes during a reinforced or nonreinforced reactivation. Here we systematically combined temporally specific memories with different reactivation parameters to observe whether these four outcomes are determined by the conditions established during training. First, we validated two training regimens with different temporal expectations about US arrival. Then, using Midazolam (MDZ) as an amnestic agent, fear memories in both learning conditions were submitted to retraining either under identical or different parameters to the original training. Destabilization (i.e., susceptibly to MDZ) occurred when reactivation was reinforced, provided the occurrence of a temporal prediction error about US arrival. In subsequent experiments, both treatments were systematically reactivated by nonreinforced context exposure of different lengths, which allowed to explore the interaction between training and reactivation lengths. These results suggest that temporal prediction error and trace dominance determine the extent to which reactivation produces the different outcomes.

Memory storage was traditionally conceived as a fixed and static process (McGaugh 1966). However, a large body of evidence in experimental psychology suggested malleability for previously stored memories through different manipulations (e.g., Misanin et al. 1968), and this issue has recently regained interest (Nader et al. 2000; Nader 2013). Data from a variety of experimental preparations has firmly established that consolidated memories become vulnerable to amnestics (i.e., sensitive to different manipulations), if reactivated under appropriate conditions (Piñeyro et al. 2013).⁵ Understanding the dynamics of memory reactivation and extinction has strong theoretical and clinical value, offering avenues to advance translatable knowledge on how human and nonhuman animals use stored information and pointing to potential therapeutic targets for the treatment of pathogenic memories (Urcelay 2012).

Memory reactivation was conceptualized by Lewis (1979) as a process that reactivates a stored, inactive memory into an active state that can encourage appropriate behavioral responses. Under some conditions, reactivation procedures lead to memory destabilization, rendering the mnemonic trace vulnerable to the effects of amnestic treatments. However, reactivation, retrieval, destabilization, and reconsolidation are not identical constructs (Guisquet-Verrier and Riccio 2012). In the context of this study, memory reactivation is used as a generic term which refers to the process through which a memory in a latent (inactive) state is placed in an active state by the reexposure to the conditioned stimulus. It is from this active state that memory can be retrieved leading to a behavioral output.

Because reactivation does not always lead to destabilization–reconsolidation, studies on human and nonhuman animals have been concerned with the specific conditions that lead to destabilization of a consolidated memory. One candidate mechanism widely developed in theories of learning (Bush and Mosteller 1955; Rescorla and Wagner 1972; Sutton and Barto 1981) that has been suggested to be critical for memory destabilization is prediction error (Pedreira et al. 2004).The psychological intuition is that a discrepancy between what is expected (based on information stored in long-term memory) and what is experienced (during reactivation) triggers the destabilization process to update the memory, which then requires reconsolidation in order to stabilize the newly incorporated information (Pedreira et al. 2004). This implies that memory reactivation will lead to memory destabilization if, and only if, a prediction error is experienced by the animal. This hypothesis has recently found support both in humans and rats through two manipulations that induce an error prediction during memory reactivation: (a) varying the amount of CS or response exposure (Exton-McGuinness et al. 2014b, 2015; Reichelt et al. 2012, 2013; Sevenster et al. 2012, 2013, 2014), and (b) varying the temporal expectation of the US presentation (Diaz-Mataix et al. 2013).

In the case of associative memories, an often used reactivation protocol to induce memory destabilization involves the presentation of a previously conditioned stimulus (CS) in the absence of the unconditioned stimulus (US), which is by definition a standard procedure to induce extinction learning (Pavlov 1927; Bouton 2004). In other words, memory reactivation without US presentation can lead to different mnemonic outcomes: mere retrieval, destabilization–reconsolidation or extinction. When reviewing the literature to better understand these seemingly contradictory outcomes, a pattern is seen: for example, Pedreira and Maldonado (2003) (also see Eisenberg et al. 2003) observed that reactivation duration influences in part which process takes place. This pattern has been replicated by other groups that further found that short presentations only produce memory retrieval, intermediate reactivations destabilize the memory and long reactivations generate extinction learning (Suzuki et al. 2004; Bustos et al. 2009; Piñeyro et al. 2013).

It is well established that extinction learning is best captured as new CS–NoUS learning (Bouton 1993) and thus many researchers have assessed what effect, if any, do amnestics have after extended post-training exposure to a CS. The results suggest that with enough CS exposure the new (extinction) memory is dominant, and administration of an amnestic will have the effect of disrupting the extinction memory, leading to an increase in responding at test relative to the appropriate control conditions (Eisenberg et al. 2003; Lee et al. 2006). Intriguingly, recent studies have shown that intermediate amounts of post-training CS exposure, a point at which presumably neither CS–US nor CS–NoUS trace is dominant, have revealed insensitivity to amnestic (Flavell and Lee 2013; Merlo et al. 2014; Sevenster et al. 2014).

We hypothesized that the occurrence of these four possible outcomes from a memory reactivation, mere retrieval, destabilization–reconsolidation, transitional insensitive period, and extinction learning depends upon the combination of both conditions, the initial learning experience and the CS-reactivation length. We decided to address this hypothesis experimentally, using contextual fear conditioning (CFC) and rats as experimental subjects.

In Experiment 1 we varied the amount of context exposure immediately before shock presentation (1 or 5 min) in order to establish two single-event learning experiences with different expectations about US arrival. A minute-by-minute analysis of behavior during a nonreinforced test 3 d later revealed clear temporal control of the conditioned response (CR, freezing), for it peaked and decreased according to the time point at which the US was delivered during training. This suggests the existence of different expectations about US arrival as a function of training parameters (also see Savastano and Miller 1998). Then, by varying the temporal expectation of the US presentation during reinforced reactivations (CS–US) and using MDZ as an amnestic agent, we replicated the aforementioned results of Díaz-Mataix et al. (2013), showing that US temporal prediction error is critical for memory destabilization–reconsolidation (Exps 2 and 3). In subsequent experiments (Exps 4–8), by a systematic combination of both temporal-specific memories with different CS-reactivation lengths, we demonstrated that the necessary time to reveal these four possible outcomes (mere retrieval, destabilization–reconsolidation, transitional insensitive period, and extinction) from a post-training CS presentation is highly dependent on the conditions established during training.

It is important to mention that throughout these experiments the two learning conditions (1 and 5 min of context exposure in training) were run separately, except in Experiment 1 which included both conditions thereby providing some justification for the subsequent between-experiment comparisons. To avoid confounds that can arise from cross-experiment comparisons, the corresponding statistical analyses were performed separately within each experiment. However, the experiments are presented in pairs, according to the reactivation duration, since this would best highlight how different learning conditions produce different mnemonic processes with equivalent amounts of reactivation.

Results

Experiment 1

Animals are capable of learning when the US arrives based on CS duration (Savastano and Miller 1998; Balsam et al. 2001; Gallistel and Balsam 2014). Indeed, temporally specific CRs have been observed in fear-potentiated startle after a single training experience (Davis et al. 1989), and a similar temporal specificity as a function of training parameters has been replicated for different CRs in CFC (Bevins and Ayres 1995).The aim of Experiment 1 was to establish two different temporal predictions about US arrival, as a necessary step to test our hypotheses concerning the role of US temporal prediction error in memory destabilization–reconsolidation. Rats were trained with 1 and 5 min of context exposure prior to shock presentations, and tested 72 h later during 6 min in the training context, without shock. Freezing behavior was analyzed minute by minute.

Figure 1 depicts the experimental protocol (top panel) and the percentage of freezing behavior of both groups throughout the 6 min of reactivation (i.e., test) time (bottom panel). A mixed ANOVA on total freezing behavior during reactivation (training condition × time as factors) revealed no effect of training condition (F_(1,12) = 0.71, P = 0.41), a significant effect of time (F_(5,60) = 10.16, P < 0.05), and a significant interaction between both factors (F_(5,60) = 11.58, P < 0.05). Thus, this result suggests that both learning conditions resulted in similar amounts of conditioned responding (i.e., memories of similar strength—see inset), although differentially distributed in time during the test session. Post hoc analyses revealed that groups did not differ between them during the first, third, or sixth minute of reactivation (P > 0.98 in all cases), but did so during the second, fourth, and fifth (P < 0.01 in all cases). Further post hoc analysis for the 1-min learning condition revealed that CR in the second minute was higher than in the fourth and fifth minute (P < 0.001 in all cases; Cohen's f = 0.87 and Cohen's f = 1.22 for comparison involving responding at the second versus fifth minute and second versus fourth minute, respectively) which correlates with the moment at which the US was delivered during training (within the first and second minute). Thereafter, CR decreased progressively. In the case of the 5-min condition, the post hoc analysis revealed that CR in the fifth minute was higher than in the first and second minute (P < 0.001 in all cases; Cohen's f = 1.55 and Cohen's f = 1.08 for comparison involving responding at the second versus fifth minute and first versus fifth minute, respectively). In other words, a different pattern emerged with the 5-min condition: the CR progressively increased over time, reaching its maximum level within the fifth minute before falling sharply in the sixth minute, which correlates with US arrival during training. Such a distinctive variation of CR curves along testing time strongly suggests differential shock expectancies or predictions, determined by learning conditions. Moreover, in a nonreactivation control experiment (see Supplemental Fig. S1) these temporal distributions of freezing were replicated, suggesting that the particular form of the CR in each single-event learning condition was not altered when MDZ or SAL were administered in the absence of memory reactivation.

Experiment 2

Experiment 1 suggests that the 1- and 5-min treatments lead to fear memories of similar strength but different temporal distributions. In order to confirm that a temporal prediction-error is critical to induce memory destabilization, two experiments were conducted in which shocks were administered during the reactivation session. In Experiment 2, rats trained in the 1- or 5-min conditions were retrained with identical parameters 72 h after the first training experience. In Experiment 3, shocks were presented 30 sec earlier than during original training, thus producing a discrepancy between expected and experienced shock arrival. Immediately after retraining, subjects were administered with MDZ or SAL. We hypothesized that under identical retraining conditions, MDZ would not lead to amnesic effects (Exp. 2), for the amnesic should only be effective when the retraining conditions are different from the previously stored temporal information (Exp. 3).

In addition to the above-mentioned objectives, we wanted to assess what effect, if any, does MDZ have on temporal control of behavior after post-training manipulations. Such analyses are available in the Supplemental Material under the heading Supplementary analysis (SA). A previous study used a design in which animals were retrained under similar or different temporal parameters and subsequently administered anisomycin in the lateral amygdala (Díaz-Mataix et al. 2013). Although they observed that the amnestic only exerted an effect when US timing was changed (by presenting shock before or after it was previously trained), they did not observe a precise temporal control. The present experiments investigated in CFC whether temporal control of behavior is preserved with the administration of MDZ after the induction of a US temporal prediction error.

The top panel of Figure 2 depicts the experimental protocol. Two groups of rats were trained (1- or 5-min conditions) and 3 d later retrained using identical parameters. Immediately after, half of the rats in each learning condition received a systemic injection of 3 mg/kg MDZ (i.p.), while the other half received an equivalent amount of SAL. Twenty-four hours and 7 d later, rats were exposed to the training context without shock for 5 min (test and retest, respectively).

Figure 2A shows the data for the 1-min condition, during retraining, test and retest. T-test analysis for the period of context exposure prior to shock during the retraining phase revealed no difference between groups t₍₁₄₎ = 0.09, P = 0.92. A mixed ANOVA on test and retest data (drug × phase as factors) revealed no effect of drug (F_(1,14) = 0.36, P = 0.55), phase (F_(1,14) = 0.48, P = 0.49), or interaction between these factors (F_(1,14) = 0.12, P = 0.73). Bayesian analysis found that the null hypothesis predicting no difference between SAL and MDZ groups in the test phase was 3.37 times more probable than the alternative hypothesis. Figure 2B shows the results for the 5-min condition during retraining, test, and retest, respectively. T-test analysis for the period of context exposure prior to shock during the retraining phase revealed no difference between groups t₍₁₄₎ = 0.27, P = 0.78. A mixed ANOVA on test and retest data (drug × phase as factors) revealed no effect of drug (F_(1,14) = 0.17, P > 0.68), phase (F_(1,14) = 1.08, P = 0.31), or interaction between both factors (F_(1,14) = 1.61, P = 0.22). Again, Bayesian analysis found that the null hypothesis predicting no difference in the CR in the test phase according to the drug treatment was 3.62 times more probable than the alternative hypothesis. For analyses of temporal control of behavior of both CFC protocols during test, see Supplemental Material SA1. Similar results were observed for both memories (Fig. 2C,D): the maximum levels of CR expression correlates with the moment at which the US was delivered both during the initial training and retraining. This suggests that the particular control of CR of each learning condition was preserved after MDZ administration, provided there was no US temporal prediction error during the reinforced reactivation.

Thus, the results suggest that an identical retraining session did not trigger plasticity processes sensitive to MDZ, presumably because no change in the temporal predictions between the previously stored information and the current experience (during retraining) was detected. In addition to similar levels of CR during test of both memories, the analyses of CR curves suggest that neither the application of MDZ nor SAL alter the expression of the specific temporal profiles.

Experiment 3

Experiment 3 was similar to Experiment 2, except that during retraining the parameters were altered by presenting the USs 30 sec earlier, compared with initial training. Therefore, animals were removed from the chamber 30 sec earlier compared with the total exposure time of each learning condition during training. All other parameters were identical to previous experiments. We hypothesized that changing the retraining parameters would lead to a US prediction-error and MDZ would exert its amnesic effects regardless of training condition. It was unclear however, whether MDZ would simply decrease conditioned responding without altering temporal control of behavior, or if temporal control would be lost with the administration of MDZ.

The top panel of Figure 3 depicts the experimental protocol. Figure 3A shows the amount of freezing in the 1-min condition, during retraining, test, and retest. T-test analysis for the period of context exposure prior to shock during the retraining phase revealed no difference between MDZ and SAL groups: t₍₁₂₎ = 0.16, P > 0.87. A mixed ANOVA on test and retest data (drug × phase as factors) revealed a main effect of drug (F_(1,14) = 174.49, P < 0.01), no effect of phase (F_(1,14) = 0.43, P = 0.83) and no interaction between these factors (F_(1,14) = 0.32, P = 0.57). Post hoc analysis revealed that MDZ group expressed reduced freezing behavior in both test (Cohen's f = 3.07) and retest (Cohen's f = 2.31), relative to SAL group (P < 0.001 in all cases). Figure 3B shows the amount of freezing in the 5-min condition during retraining, test and retest, respectively. A t-test analysis for the period of context exposure prior to shock during retraining phase revealed no difference between groups t₍₁₄₎ = 0.75, P = 0.47. A mixed ANOVA on test and retest data (drug × phase as factors) revealed a main effect of drug (F_(1,14) = 164.20, P < 0.01), an effect of phase (F_(1,14) = 5.79, P < 0.05), and no interaction between these factors (F_(1,14) = 3.82, P = 0.07). The post hoc analyses revealed that freezing behavior of MDZ group was significantly reduced during both test (Cohen's f = 2.46) and retest (Cohen's f = 2.75), when compared with SAL group (P < 0.001 in all cases). A minute-by-minute analysis of both CFC treatments was used to assess whether a change in temporal parameters would reveal an effect of the amnestic that is related or not to temporal control of behavior (see Supplemental Material SA2). Notably, for both learning conditions the analyses revealed temporal control of CR expression (SAL groups), for it peaked and decreased according to the time point at which the US was delivered during the initial training. MDZ not only decreased CR, but also altered the temporal control of behavior (Fig. 3C,D).

These results show that post-reactivation plasticity processes (i.e., memory destabilization–reconsolidation) can be induced when all the conditions presented during initial training are available, as long as there is a US prediction error based on stored temporal estimations and current (new) experience. Hence, these results strengthened the hypothesis that US temporal error detection is critical to trigger memory destabilization, rather than the mere reactivation of the already consolidated memory; and they are also consistent with a recent report by Díaz-Mataix et al. (2013). However, in that report, temporal control of behavior was not precise and thus it is unclear whether the amnestic altered memory strength or temporal control. The present results suggest that, in addition to lower CR, temporal control was lost only when there was a change in temporal parameters from training to retraining.

Experiment 4

According to the prediction-error hypothesis, the processes of memory destabilization–reconsolidation will only take place if a discrepancy between stored information and current experience is detected during reactivation (Exp. 3). It has been reported by several laboratories (Eisenberg et al. 2003; Pedreira and Maldonado 2003; Suzuki et al. 2004; Bustos et al. 2009) that a short memory reactivation only produces memory retrieval (with or without CR expression). In a recent study (Piñeyro et al. 2013; Exp. 1) we also observed in this preparation that a short reactivation did not result in memory destabilization Therefore, we predicted that a 30-sec reactivation session (without shock) in the conditioned context would not allow animals to detect any discrepancy with the already encoded information about US arrival, since in both single-event training conditions the US arrived at later time points (60 sec for the 1-min condition, 300 sec for the 5-min condition). As the secondary aim, we wanted to assess whether a short reactivation alters the temporally specific pattern of CR we observed in Exp. 1.

The top panel of Figure 4 depicts the experimental protocol. Rats were trained under 1- or 5-min conditions (as in previous experiments) and 3 d later received a brief context reexposure of 30 sec. This was followed immediately by a systemic injection of 3 mg/kg MDZ or SAL (i.p.). Twenty-four hours and 7 d later, rats were exposed to the training context for 5 min without shock (test and retest, respectively). Figure 4A shows the results of the 1-min condition during reactivation, test and retest. T-test analysis of the reactivation phase revealed no difference between MDZ and SAL groups: t₍₁₄₎ = 0.35, P = 0.72. A mixed ANOVA on test and retest data (drug × phase as factors) revealed no effect of drug (F_(1,14) = 2.00, P = 0.17), no effect of phase (F_(1,14) = 0.49, P = 0.49), and no interaction between these factors (F_(1,14) = 1.44, P = 0.24). A Bayesian analysis found that the null hypothesis predicting no difference in the CR during the test phase according to the drug treatment was 3.62 times more probable than the alternative hypothesis. Figure 4B shows the results for the 5-min condition during reactivation, test, and retest. T-test analysis of the reactivation phase revealed no difference between MDZ and SAL: t₍₁₄₎ = 0.13, P = 0.89. A mixed ANOVA on test and retest data (drug × phase as factors) revealed no effect of drug (F_(1,14) = 0.84, P = 0.37), phase (F_(1,14) = 1.35, P = 0.26), or interaction between these factors (F_(1,14) = 0.79, P = 0.38). Again, using a Bayesian analysis of the MDZ and SAL data in the test phase, we found that the null hypothesis was 3.70 times more probable than the alternative hypothesis, proposing a difference based on the drug treatment. We additionally analyzed temporal control of behavior during test using a minute-by-minute mixed ANOVA for each training condition (see Supplemental Material SA3). Notably, in both cases the CR peak correlates with the period of time at which the US was delivered in each learning condition (Fig. 4C,D). Thus, a short reactivation that does not result in memory destabilization, does not alter the temporal profile of the fear memories.

In conclusion, these results suggest that MDZ does not have amnesic effects on CFC memory when administered after a brief (30 sec) reactivation, regardless of the initial training condition. In line with our reasoning, 30 sec of context reexposure was insufficient time to violate the predictions about the US arrival for either learning condition, and destabilization–reconsolidation processes were not observed.

Experiment 5

If the temporal prediction about US arrival is an attribute of the reactivated memory, we expected that 2 min of context reexposure (without shock) should be sufficient to destabilize the memory trained with 1-min of context exposure, but not the one trained with 5-min of context exposure. Thus, we predicted amnesic effects of MDZ only for animals trained in the 1-min condition. In addition, we wanted to assess whether memory destabilization induced by US temporal prediction error has an effect on the temporal distribution of CR at test.

The top panel of Figure 5 depicts the experimental protocol. It was similar to that used in Experiment 4, except that context reexposure (i.e., reactivation) lasted for 2 min, instead of 30 sec. Figure 5A shows the results for the 1-min condition during reactivation, test, and retest. T-test analysis for the reactivation phase revealed no difference between MDZ and SAL groups: t₍₁₄₎ = 1.04, P = 0.31. A mixed ANOVA on test and retest data (drug × phase as factors) revealed a main effect of drug (F_(1,14) = 408.19, P < 0.01), an effect of phase (F_(1,14) = 18.07, P < 0.01), but no interaction (F_(1,14) = 2.72, P = 0.12). Post hoc analysis revealed that freezing behavior for MDZ group was significantly reduced during both test (Cohen's f = 3.99) and retest (Cohen's f = 4.00), relative to SAL group (P < 0.001 in all cases). Figure 5B shows the results for the 5-min condition during reactivation, test, and retest. T-test analysis for the reactivation phase revealed no difference between MDZ and SAL: t₍₁₄₎ = 0.10, P > 0.92. A mixed ANOVA on test and retest data (drug × phase as factors) revealed no effect of drug (F_(1,14) = 0.04, P = 0.82), phase (F_(1,14) = 0.23, P = 0.63), or interaction between these factors (F_(1,14) = 0.63, P = 0.44). A Bayesian analysis supported the null hypothesis, it was found to be 3.41 times more likely than the alternative.

In sum, Experiment 5 suggests that, after a 2-min context reactivation, MDZ only produced amnesic effects in animals trained with 1 min of context exposure, having no amnesic effect in animals trained under the 5-min condition. These conclusions are strengthened by the minute-by-minute analyses (see Supplemental Material SA4), which indicate that 2 min of nonreinforced context exposure for the 5-min condition does not alter the temporal profile of the CR (Fig. 5D), presumably because 2 min still does not allow the detection of a temporal prediction error about the US arrival, as observed in Experiment 4 for the 1 min condition in which the precise temporal controls of CR were unchanged after a mere retrieval. In contrast, a temporal predictive error seems to have occurred for 1-min rats, as inferred by the lack of temporal precision of the CR of both MDZ and SAL groups. Notably, the occurrence of an US temporal prediction error during the nonreinforced reactivation of context reveals an amnesic effect of MDZ which is also related to temporal control of behavior (Fig. 5C). This suggests that a 2-min nonreinforced context reactivation leads to prediction error only for animals trained in the 1-min condition. Consequently, if temporal prediction error about US arrival is necessary for memory destabilization, extending the reactivation duration should induce memory destabilization–reconsolidation processes for animals trained in the 5-min condition. Testing this prediction was the aim of Experiment 6.

Experiment 6

In line with the reasoning of Experiment 5, we hypothesized that 6 min of reactivation would be sufficient for the animals trained in the 5-min condition to detect a relevant difference relative to their initial learning (i.e., US absence at the expected point in time). Furthermore, we assessed whether 6 min of context reactivation would induce extinction learning in the animals trained under the 1-min condition. As mentioned in the Introduction, previous studies have found that, with intermediate amounts of exposure which are beyond the period sufficient for memory destabilization, but not long enough to induce extinction learning, amnestics have no effect (Flavell and Lee 2013; Merlo et al. 2014; Sevenster et al. 2014). Accordingly, our prediction was that, after 6 min reactivation, MDZ would block memory reconsolidation for animals in the 5-min condition, but perhaps have no effect on animals in the 1-min condition.

The top panel of Figure 6 depicts the experimental protocol. The only difference with Experiments 4 and 5 was that reactivation lasted 6 min. Figure 6A shows the results for the 1-min condition during reactivation, test, and retest. T-test analysis for the reactivation phase revealed no significant difference between MDZ and SAL groups: t₍₁₂₎ = 2.01, P = 0.059. A mixed ANOVA on test and retest data (drug × phase as factors) revealed no effect of drug (F_(1,12) = 0.84, P = 0.37), phase (F_(1,12) = 1.07, P = 0.32) or interaction between these factors (F_(1,12) = 1.89, P = 0.19). A Bayesian analysis found that the null hypothesis predicting no difference between vehicle and MDZ conditions during the test phase was 3.47 times more probable than the alternative. Figure 6B shows the results for the 5-min training condition during reactivation, test, and retest. T-test analysis for the reactivation phase revealed a significant difference between the groups MDZ and SAL: t₍₁₄₎ = 2.86, P < 0.01. Although significant, the difference between MDZ and SAL groups during reactivation was minimal (54% versus 57% of freezing, respectively) and likely the result of random variation, for no difference should have arisen before any pharmacological treatment was administered. Accordingly, we decided to analyze the data as in previous experiments. A mixed ANOVA on test and retest data (drug × phase as factors) revealed a main effect of drug (F_(1,14) = 438.09, P < 0.01), and no effect of phase (F_(1,14) = 3.07, P = 0.10) or interaction between these factors (F_(1,14) = 1.55, P = 0.23). The post hoc analysis revealed that the MDZ group expressed significantly less freezing behavior throughout the test (Cohen's f = 4.15) and retest (Cohen's f = 4.11) phases (P > 0.01 in all cases). For analyses of temporal control of behavior of both CFC protocols during test, see Supplemental Material SA5. These analyses suggest that in the 1-min condition, the CR peaked and decreased according to the initial training parameters, whereas for the 5-min condition neither SAL nor MDZ groups showed the temporal control of the CR during test according to the previously established training parameters (Fig. 6C,D).

The aforementioned findings suggest two main conclusions: first, a 6-min reactivation session is sufficient to trigger destabilization–reconsolidation processes in animals trained in the 5-min condition, as revealed by the amnesic effects of MDZ over the total percentage of CR expression and also in the temporally indistinct form of the CR distribution (Fig. 6D). Importantly, these results replicate the findings for the 1-min condition after 2 min of nonreinforced reactivation (Exp. 5; i.e., destabilization–reconsolidation). Overall, these results provide support for the critical role of US temporal prediction error in triggering destabilization–reconsolidation processes. Second, the same reactivation duration was neither long enough to generate extinction nor appropriate to induce destabilization–reconsolidation processes in animals trained in the 1-min condition. Consistent with findings from other laboratories (Flavell and Lee 2013; Merlo et al. 2014; Sevenster et al. 2014), these results suggest that there is a period of insensitivity to MDZ agent in which destabilization–reconsolidation processes are no longer taking place (deduced from the lack of amnesic effects of MDZ) and extinction learning has not yet been sufficiently developed. Thus, we suggest that this particular “null” outcome results from an interaction between excitatory (CS–US) and inhibitory (CS–NoUS) memory traces, which, at this particular parametric point, are of equivalent strength and thus neither is sensitive to MDZ. Surprisingly, MDZ administered during this transitional insensitive period did not alter the temporal expectations about the US arrival that were encoded as a product of the specific training parameters (Fig. 6C).

Experiment 7

Previous experiments demonstrated that memory destabilization–reconsolidation processes can be modulated by different initial learning conditions and reactivation lengths (Exps 5 and 6). In other words, a reactivation procedure which destabilizes a memory formed under certain training parameters may be ineffective when other parameters were used, because predictions about the US arrival were differentially encoded in each single-event training experience. These experiments also suggested the existence of a transition phase between destabilization–reconsolidation and extinction learning in which MDZ has no effect on memory expression. Since memory reactivation can also lead to extinction learning after extended CS exposure (Eisenberg et al. 2003; Briggs and Riccio 2007), it follows that when the CS–US and CS–NoUS traces are of similar strength they may cancel each other and result in neither memory trace being dominant—hence the amnestic is ineffective. In the next experiment, we attempted to answer two questions: first, what is the necessary time to trigger an extinction learning process for animals trained under the 1-min condition? Second, is there also a transitional insensitive period between destabilization–reconsolidation and extinction for the 5-min condition?

In Experiment 7, animals trained in the 1- and 5-min conditions were subsequently exposed to a 15-min reactivation session, and followed by MDZ or SAL. Based on previous studies, we expected that 15 min should induce extinction learning (which would be blocked by MDZ) for animals trained under the 1-min condition (see Piñeyro et al. 2013). For animals trained in the 5-min condition, two possible outcomes were expected: the transitional insensitive period as observed in the 1-min condition (Experiment 6), particularly operationalized in the lack of MDZ amnesic effects and intact CR expression for SAL group, or extinction (reduced CR expression for SAL group, which would be blocked by MDZ).

The top panel of Figure 7 depicts the experimental protocol. Figure 7C shows the results for animals trained in the 1-min condition during reactivation, test and retest. T-test analysis for the reactivation phase revealed no difference between the MDZ and SAL groups, t₍₁₄₎ = 0.68, P = 0.63. A mixed ANOVA on test and retest data (drug × phase as factors) yielded a main effect of drug (F_(1,14) = 128.06, P < 0.01), a significant effect of phase(F_(1,14) = 72.81, P < 0.01), and also a significant interaction between both factors (F_(1,14) = 114.64, P < 0.01). Post hoc analyses revealed that groups differed from each other only during the test phase (P < 0.01; Cohen's f = 4.12): SAL group showed less CR expression when compared with MDZ group 24 h after the 15-min reactivation session, then this difference in the test between MDZ and SAL groups was lost 7 d later (retest; P = 0.99). This was due to the SAL group expressing increased freezing from test to retest (P < 0.01; Cohen's f = 3.71) suggesting spontaneous recovery, which is consistent with the typical pattern of extinction learning (Bouton 2004). The MDZ group, however, showed intact CR expression, a consequence of the amnesic agent over the consolidation of extinction memory, as previously reported with MDZ (Bustos et al. 2009) hypothermia (Briggs and Riccio 2007), and other pharmacological agents (Eiserberg et al. 2003; Pedreira and Maldonado 2003; Suzuki et al. 2004). Figure 7A shows the course of extinction minute by minute over the 15 min of reactivation time. For a minute-by-minute analysis of temporal control of behavior during reactivation and test sessions, see Supplemental Material SA6. During the test session neither SAL nor MDZ groups showed the temporal control of the CR according to the initial learning parameters (Fig. 7E).

Figure 7D shows the results for the 5-min condition. T-test for the reactivation phase revealed no difference between the MDZ and SAL groups, t₍₁₄₎ = 0.04, P > 0.96. A mixed ANOVA on test and retest data (drug × phase as factors) showed no effect of drug (F_(1,14) = 1.55, P = 0.23), no effect of phase (F_(1,14) = 0.85, P = 0.36), and no interaction between these factors (F_(1,14) = 0.32, P = 0.57). Bayesian analysis found that the null hypothesis predicting no difference between drug treatments in the test phase was 3.37 times more probable than the alternative hypothesis. Figure 7B shows minute by minute the 15 min of reactivation time. Temporal control of behavior was assessed during test and reactivation sessions using a minute-by-minute mixed ANOVA (see Supplemental Material SA6). In test session, the analysis showed that the temporal control of CR was unaltered in either group (Fig. 7F).

The behavioral pattern that emerges from this experiment is similar to the previous results, but using a slightly different set of parameters. In Experiment 6, we observed that 6 min of context reexposure for rats trained in the 1-min condition failed to reveal any effect of MDZ, and furthermore the analysis of the CR form suggested that both MDZ and SAL groups showed specific temporal control of the CR. The current experiment replicated these results: 15 min of context reexposure for rats trained in the 5-min condition showed insensitivity to MDZ, it failed to induce extinction, and the minute-by-minute analysis also indicates that the particular temporal control of the CR was not altered.

Experiment 8

In Experiment 8, we wanted to determine whether extended context exposure would result in extinction learning for the 5-min condition. Only animals trained in the 5-min condition were used in this experiment, and we expected that extinction after 30 min of context exposure would be evident 1 d after training, but recover afterward (7 d later).

The top panel of Figure 8 depicts the experimental protocol. Rats trained with 5-min of context exposure prior to US arrival were subjected to 30 min of context reexposure and immediately after received MDZ or SAL (i.p.). Test and retest were identical to previous experiments. Figure 8B shows the results during the different phases. T-test for the reactivation phase showed, as expected, no difference between groups: t₍₁₂₎ = 1.44, P = 0.17. A mixed ANOVA on test and retest data (drug × phase as factors) revealed a main effect of drug (F_(1,12) = 53.54, P < 0.01), a significant effect of phase (F_(1,12) = 22.38, P < 0.01), and also a significant interaction effect between both factors (F_(1,12) = 11.20, P < 0.05). Post hoc analyses revealed that the groups differed from each other during the test (SAL expressed significantly less freezing behavior than MDZ; P < 0.01; Cohen's f = 4.58), but not during retest (P = 0.97). At the same time, SAL group expressed increased freezing from test to retest (P < 0.01; Cohen's f = 2.88) suggesting spontaneous recovery. Figure 8A shows the course of extinction minute by minute over the 30 min of reactivation time. For a minute-by-minute analysis of temporal control of behavior during reactivation and test sessions, see Supplemental Material SA7. Critically, during the test session, the analyses suggested that the temporal specificity of CR form seems not to be strongly modified in the MDZ group (Fig. 8C). These results suggest that extinction learning can be achieved through a 30 min unreinforced reactivation for animals trained under the 5-min condition. This was evidenced by a reduced CR expression during test that recovered spontaneously a week later, an effect blocked by MDZ. It is worth noting that during long extinction sessions, neither learning condition (Figs 7A,B, 8A) showed a complete reduction in levels of CR. This is consistent with other studies using single-session extinction in rodents (Cain et al. 2003) and humans (Tsao and Craske 2000). It is unclear whether the reduction of fear during extinction predicts fear expressed during subsequent tests (for review, see Craske et al. 2014).

Discussion

The results in this study suggest a number of conclusions. First, as previously suggested, time is encoded as a component of associative memories and critically determines the timing of conditioned responding, possibly through temporal predictions about the reinforcer onset (Savastano and Miller 1998; Gallistel and Balsam 2014). In addition, US temporal prediction-error determines the extent to which reactivation destabilizes a memory trace thus making it vulnerable to the effects of an amnestic agent (Díaz-Mataix et al. 2013).

Second, whichever of four possible outcomes occurs after memory reactivation depends on specific training conditions. This conclusion was reached by a comprehensive combination of learning and reactivation conditions that shed light on the relationship between the initial learning conditions (1 or 5 min) and the four different outcomes during the reactivation of previously encoded memories. We first observed that the induction of an US temporal prediction error can occur when the US was presented during memory reactivation, provided that the temporal parameters of training were altered during retraining (Díaz-Mataix et al. 2013). This demonstrated that a difference between what is expected and what is experienced (i.e., US temporal prediction error) is critical for memory destabilization and subsequent reconsolidation. This suggestion was strengthened by the fact that the induction of an US temporal prediction error does not only occur in the presence of the US. During nonreinforced reactivation, memory destabilization (and subsequent vulnerability to MDZ) was only observed when subjects were exposed to the trained context long enough for omission of the expected reinforcement to be experienced. The working hypothesis that unreinforced reactivation of such temporally specific memories may differentially result in mere retrieval, destabilization–reconsolidation, transitional insensitive period or extinction depending on the conditions of training and the amount of reactivation, was supported by the different reactivation lengths that trigger destabilization–reconsolidation processes for each CFC memory. We also observed that during post-training CS exposure, less exposure than that used during training is sufficient for memory retrieval (i.e., expression of CR) but it does not lead to memory destabilization, as inferred from the lack of effect of MDZ upon subsequent memory expression (Exps 4 and 5). Finally, similar results emerged in the case of the transitional insensitive period and extinction learning. That is, different amounts of nonreinforced reactivation were needed to reveal these memory outcomes by each CFC protocol (Exps 6A, 7, and 8). We return to the implications of these later.

Consistent with the present results, a study has also suggested that US temporal prediction error triggers memory destabilization (Díaz-Mataix et al. 2013). In the present research we first replicated the findings of Díaz-Mataix et al. (2013; Exps 2 and 3), but then we studied prediction error after different amounts of nonreinforced reactivations, which demonstrated that the US temporal prediction error can induce memory destabilization in the absence of shock. In addition, the measurement of US expectancies operationalized by the temporal CR changes during nonreinforced reactivation could provide an independent measure of the memory reconsolidation process itself. This is in line with recent suggestions that the period of time of maximum US expectancy (i.e., fear) is a key factor to improve therapeutic strategies such as enhancing inhibitory learning based on an expectancy violation model (Craske et al. 2014). In addition, this is also in agreement with reports in humans (Sevenster et al. 2013, 2014) which strongly indicate that post-retrieval changes in US-expectancy can be used to reveal the different processes triggered by nonreinforced memory reactivation.

This study is first to demonstrate that when a US temporal prediction error is detected during nonreinforced reactivation, temporal-specific control of behavior is lost, as also does extinction learning (for a summary of temporal control of CR throughout Exps 2–8, see Supplemental Material Table S1). On the contrary, when US prediction error is not detected after insufficient reactivation time or when reactivation length is beyond reconsolidation (but not enough to establish an extinction memory) the temporal-specific control of CR is preserved, without altering the distribution of responding as a function of each particular single-event training parameter. However, it is important to note that additional minutes of context reexposure which are beyond the period sufficient for memory destabilization, but not long enough to induce extinction learning, restore temporal control that was lost after memory destabilization. This point should be addressed in future studies.

The present experiments were designed under the premise that multiple outcomes are possible after memory reactivation, and that US temporal prediction error can serve as indicative of when a memory will be most vulnerable to the effects of an amnestic. Simply put, when the training conditions are known, US temporal prediction error can dictate when the amnestic will be effective, following a relatively simple model such as that proposed by Sutton and Barto (1981). However, this model does not explain the lack of sensitivity to amnestic we observed with intermediate amounts of post-training CS exposure (Exps 6 and 7), because according to this model, there should still be some error in such circumstances. Neither can simple temporal prediction error account for the vulnerability of extinction memories to MDZ (Exps 7 and 8), because at the end of extinction prediction error should be minimal. For decades now, it has been suggested that extinction is best captured by new inhibitory learning, which is context dependent (Bouton 1993). Thus, a modification of Sutton and Barto's architecture that assumes both excitatory CS–US and inhibitory CS–NoUS links has some advantages (Pan et al. 2008). This model is within the realm of temporal difference (TD) architecture, but by assuming two separate memory traces with different decay functions, it can better approximate critical aspects of extinction learning such as spontaneous recovery. A similar two-trace approach, but focused on attention was initially developed by Pearce and Hall (1980).

It is not entirely clear, however, how this model handles the lack of sensitivity to amnestics observed under both training conditions following intermediate amounts of exposure (also see Exton-McGuinness et al. 2014a for a potential explanation in terms of attentional processes to the CS). This finding is consistent with a report by Flavell and Lee (2013), using an appetitive learning task and MK-801 as amnesic agent. Merlo et al. (2014) also reported a similar pattern in an aversive preparation. We suggest that for these models to handle the insensitivity to amnestics, a weighting rule such as that proposed by Devenport (1998) could be used to determine which (if any) memory trace will be dominant and hence vulnerable to the effects of amnestics.

Overall, the present results can best be accommodated appealing to two processes, namely temporal prediction-error (Sutton and Barto 1981) and trace dominance (Devenport 1998; Eisenberg et al. 2003). Prediction-error seems to be critical for memory destabilization, as suggested by the present results during both reinforced and nonreinforced memory reactivations. During nonreinforced reactivations, memory destabilization (and subsequent vulnerability to the effects of the amnestic MDZ) was only observed when subjects were exposed on extinction until the reinforcer was expected and not experienced (Exps 5 and 6). In addition, a change in shock expectancy by presenting the shock sooner than expected also revealed memory vulnerability to the amnesic (Exp. 3). However, prediction error does not explain the amnestic effects observed when post-training exposure was increased, assuming error was minimal when extinction learning has taken place. Presumably, when parameters favor extinction learning (or in other words, the formation of a new competing trace), it is this (CS–NoUS) mnemonic trace which is vulnerable to the amnestic effects, suggesting that, when multiple memory traces compete, prediction-error itself is insufficient to explain memory vulnerability. Instead, memory trace dominance seems to be the principle governing memory vulnerability (Eisenberg et al. 2003).

In summary, if basic knowledge on learning and memory mechanisms is to be translated to clinical settings (Urcelay 2012), it is imperative to know in detail the conditions and parameters that determine the occurrence of different outcomes when a previously consolidated memory is reactivated. The results from these studies indicate that the initial learning experiences can determine the reactivation length needed to trigger each of the four possible mnemonic outcomes, and point at two potential processes, namely temporal prediction-error and trace dominance.

Materials and Methods

Subjects

Subjects were experimentally naïve, adult male Wistar rats (60–65 d old, weighing 270–320 g at the beginning of the experiments). The animals were bred in our colony in the Laboratorio de Psicología Experimental, Facultad de Psicología, Universidad Nacional de Córdoba, Argentina. All animals were housed in standard laboratory Plexiglas cages (60 cm long × 40 cm wide × 20 cm high) in groups of 3–4 per cage. Food and water were available ad libitum. Animals were maintained on a 12-h light–dark cycle (lights on at 8 a.m), at room temperature of 21°C–23°C. The standards of the NIH Guide for the Care and Use of Laboratory Animals were respected. The number of animals and their suffering was kept to the minimum possible to achieve the goals of this research.

Drugs

Midazolam (MDZ, Gobbi Novag SA, Buenos Aires, Argentina) was diluted in sterile isotonic saline (SAL, 0.9% w/v) to a concentration of 3 mg/mL, and administered intraperitoneally (i.p.). The total volume of drug or equivalent amount of SAL was 1.0 mL/kg in all cases. This dose of MDZ was selected on the basis of previous reports demonstrating its ability to block contextual fear memory reconsolidation in Wistar rats (Bustos et al. 2006, 2009).

Apparatus

Contextual Fear Conditioning was conducted in a 24 long × 22 wide × 22 cm high Plexiglas chamber with opaque gray walls and a removable transparent ceiling, the floor consisting of 20 parallel stainless steel grid bars, each measuring 3 mm in diameter, spaced 1 cm apart and connected to a device to provide adjustable footshocks (Automatic Reflex Conditioner 7501, Ugo Basile, Milan, Italy). The chamber was cleaned with water and dried with paper towels before and after each experimental session. Recording of behavior (for off-line analysis) was made with a DCR-SR21 Sony Handycam digital video camera placed 50 cm above the conditioning chamber. Background noise was supplied with ventilation fans. All procedures occurred in a sound-attenuated experimental room. Experiments were always performed during the light phase of the diurnal cycle.

Behavioral procedures

In all experiments, rats were first identified, weighed and handled for 5 min on two separate days to habituate them to being handled. In those experiments involving i.p. injections, rats were also injected with 1mL/kg SAL after preexperimental handling was complete to habituate them to this procedure.

Contextual fear conditioning

One day after habituation, rats were taken individually from their home cage, transported into the experimental room and exposed to the conditioning chamber. Depending on the experiment: A—the animals were exposed for 1 min (group: “1-min”) after which two footshocks (1.0 mA, 3 sec duration, with an intershock interval of 30 sec) serving as US were delivered. B—The animals were exposed for 5 min (group: 5-min), after which two footshocks (1.0 mA, 3 sec duration, with an intershock interval of 30 sec) serving as US were delivered. In both groups, immediately after the second shock ended, rats were removed from the chamber, transported back to the colony room and placed back in their home cages.

Retraining session

Seventy-two hours after conditioning, 1- and 5-min rats were fear conditioned again. Depending on the experiment, Experiment 2—The groups were retrained using similar parameters as that used in the initial training relative to context preexposure time (1 or 5 min), total exposure time to CS-context (1 min 36 sec, and 5 min 36 sec), number (two footshocks), duration (3 sec), intensity (1.0 mA), and intershock interval (30 sec) between the US (group “Ret CS–US identical”). Experiment 3—the groups were retrained using the same number, duration, intensity, and intershock interval between the USs, however both the context preexposure time and the total exposure time to CS were modified. For each learning condition the USs onsets were given “30 sec before,” relative to the initial training parameters (group: “Ret CS–US 30 sec before). That is, for the 1-min condition the first US was given after 30 sec of context exposure (instead of 60 sec); for the 5-min condition the first US was given after 4 min and 30 sec (instead of 5 min). The animals were removed from the chamber immediately after the second US. Thus, the animals were removed 30 sec earlier than the total exposure time of each learning condition during training.

Reactivation sessions

Reactivations were always carried out 72 h after conditioning. Rats were reexposed to the conditioning chamber (i.e., context) without footshock, for different periods of time (0.5, 2, 6, 15, and 30 min only for the group 5-min condition), depending on the experiment.

Drug administration

MDZ 3 mg/kg or an equivalent amount of SAL was injected i.p. immediately after reactivation sessions.

Test

Testing consisted of a 5-min exposure session to the conditioning chamber, without shock.

Retest

Retest was identical to test, but 7 d later (context-NoUS).

Behavioral scoring

All experiments were video-taped for later off-line analyses. Freezing behavior, defined as the total absence of body and head movements except for that associated with breathing (Blanchard and Blanchard 1969), was scored minute-by-minute with a stop-watch by an observer blind to the experimental condition of each animal, and expressed as % of time (in seconds). Inter-observed reliability was established with a different set of data (Pearson's r = 0.95).

Experiments

Due to the fact that throughout the experiments both learning conditions (1- and 5-min rats) were reactivated under similar amounts of time (0.5, 2, 6, and 15 min), the experiments are presented in pairs, according to the reactivation context duration. However, with exception of Experiment 1, which included both conditions thereby providing some justification for the subsequent between-experiment comparisons, 1- and 5-min conditions were run separately. To avoid confounds that can arise from cross-experiment comparisons, the corresponding statistical analyses were performed separately within each experiment. This sort of presentation facilitates observing how different learning conditions produce different consequences for memory after similar amounts of reactivation.