Novelty-facilitated extinction: Providing a novel outcome in place of an expected threat diminishes recovery of defensive responses

Abstract

Background

Experimental extinction serves as a model for psychiatric treatments based on associative learning. However, the effects of extinction are often transient, as evidenced by post-extinction return of defensive behaviors. From a therapeutic perspective, an inherent problem with extinction may be that mere omission of threat is not sufficient to reduce future threat uncertainty. The current study tested an augmented form of extinction that replaced- rather than merely omitted- expected threat outcomes with novel non-threat outcomes, with the goal of reducing post-extinction return of defensive behaviors.

Methods

Thirty-two healthy male Sprague-Dawley rats and 47 human adults underwent threat conditioning to a conditioned stimulus paired with an electrical shock. Subjects then underwent a standard extinction protocol with shock omitted, or an augmented extinction protocol wherein the shock was replaced by a surprising tone. Tests of post-extinction recovery occurred 24 hours later in the absence of the tone.

Results

Replacing the shock with a novel non-threat outcome, as compared to shock omission, reduced post-extinction recovery (freezing in rats and anticipatory skin conductance responses in humans) when tested 24 hours later. Self-reported intolerance of uncertainty was positively correlated with recovery following standard extinction in humans, providing new evidence that post-extinction recovery is related to sensitivity to future threat uncertainty.

Conclusions

These findings provide cross-species evidence of a novel strategy to enhance extinction that may have broad implications for how to override associative learning that has become maladaptive, and offer a simple technique that could be straightforwardly adapted and implemented in clinical situations.

Introduction

Without effort or intention, individuals retain information associated with highly aversive experiences. In extreme cases, this leads to persistent intrusive memories and unwanted physiological responses to cues associated with the event. Investigations on how to mitigate the psychological and physiological impact of negative events borrows heavily from research on extinction of classical threat (fear) conditioning, whereby the omission of an aversive event reduces defensive behaviors, like increases in sympathetic arousal and freezing. Laboratory research routinely shows, however, that defensive behaviors return post-extinction (1–3). That learned defensive behaviors return even in laboratory experiments with healthy subjects reveals that threat conditioning is a powerful form of learning, and that extinction is a rather unsatisfactory way to acquire safety. As the principles of extinction continue to serve as a model for clinical treatments like exposure therapy (4, 5), there is strong motivation to discover innovative behavioral techniques to prevent post-extinction return of defensive behaviors.

The weakness of extinction may be owed to a number of evolutionary and environmental factors (see 6, 7). One limiting factor is that extinction does not eliminate the association between a conditioned stimulus (CS) and an unconditioned stimulus (US). Instead, extinction is a form of retroactive interference in which the new safety association competes for expression against the original threat association (6, 8). This secondary association is fragile, as evidenced by the return of conditioned responses (CR) following the passage of time (spontaneous recovery), following presentation of the US (reinstatement), or when cues are encountered outside the extinction context (renewal) (see 9 for review). Preventing the return of CRs remains a challenge (10). Models that conceptualize extinction as another form of associative learning propose that animals are provoked to learn by the surprising absence of the US (8, 11). However, omission does not guarantee an effective learning signal. This is especially true in relapse following exposure treatment, where highly feared outcomes are repeatedly disconfirmed but continue to be expected following treatment, e.g., a persistent fear of dying in an airplane crash after exposure therapy for fear of flying. In other words, the mere absence of the US is not sufficient to provoke a durable safety memory.

Another limit to extinction is that US omission can render the meaning of the CS ambiguous (12). That is, the CS takes on an additional meaning where it no longer predicts the US (or, predicts “no US”). Resolving this ambiguity tends to favor expression of the original memory, perhaps since this association was learned first and/or is simply more salient than the extinction memory. For example, expression of threat conditioning readily generalizes to related cues and across multiple contexts, whereas extinction tends to be confined to the extinguished cue in the extinction context (12–14). The unexpected absence of the US could also promote a general sense of future threat uncertainty. If so, then individuals who have difficulty coping with uncertainty may be especially prone to return of defensive behaviors following extinction.

In the present study, we reasoned that extinction could be straightforwardly and effectively augmented if the aversive US was instead replaced by a surprising non-threat outcome. Here, an expected electric shock was replaced by a novel and affectively neutral tone. We predicted that replacing shock with a novel outcome would reduce post-extinction spontaneous recovery more effectively than shock omission alone for two principal reasons: First, unlike the mere absence of the shock, a novel outcome should be more effective at generating a mismatch between the predicted and received outcome, therefore signaling a clear change in the environment to promote the acquisition of new learning (8, 15). Second, a novel perceptible outcome should help resolve some ambiguity generated by shock omission by providing a more substantive alternative association for the CS than “no shock.”

It is important to note up-front certain methodological similarities to another form of outcome interference: counterconditioning. In counterconditioning, the outcome switches between opposing reinforcement qualities (appetitive and aversive) and thus opposing behavioral responses (approach and avoidance). Counterconditioning serves as a model for some behavior therapies (16, 17), but contemporary research on aversive-to-appetitive counterconditioning in humans is scant (18, 19). However, there is ample evidence in rats showing strong return of defensive CRs following counterconditioning (20–25). Unlike counterconditioning, the current protocol replaces the US with a surprising stimulus that does not reinforce any overt behavior.

The effect of this modified extinction protocol was assessed 24 hours after extinction in rats (Experiment 1) and humans (Experiment 2), with the prediction that a novel non-threat outcome would diminish spontaneous recovery. We additionally explored for the first time whether individual differences in self-reported intolerance of uncertainty predicts post-extinction recovery in humans.

Methods and Materials

Experiment 1

Animal subjects

Thirty-two male Sprague-Dawley rats (Hilltop Lab Animals, Scottsdale, PA) were used in Experiment 1. Further details regarding the animal subjects and threat conditioning procedures are included in the Supplemental Materials.

Rodent Threat Conditioning Procedures

Two contexts were created (‘A’ and ‘B’) and made distinct with visual, tactile and olfactory cues. Subjects were placed in context A on the first day of the study (Figure 1A), wherein the front panel light was illuminated for 30-seconds and co-terminated in the delivery of a 1 second 0.7 mA footshock on 100% of trials. The acquisition session included 5 trials spaced apart by an approximate 180-second intertrial interval. Two minutes following the last trial, subjects were removed from the chambers and returned to the colony for the remainder of the day.

Figure 1.

(A) Procedure for Experiment 1 in rats. Two groups underwent identical differential threat conditioning to initiate freezing to a conditioned stimulus (CS) paired with an electric shock unconditioned stimulus (US). Twenty-four hours later in a new context, one group underwent standard extinction in which the US was omitted on CS trials, while another group underwent a modified extinction procedure in which the US was replaced by a novel non-aversive tone. Spontaneous recovery was tested for both groups 24-hours later in the extinction context. (B) Freezing during extinction in the two groups was equivalent. (C) The NFE group showed significantly less evidence of recovery 24 hours later than the standard extinction group. ** P < .01; error bars reflect standard error. CS duration was 30 seconds.

Subjects were randomly assigned to one of two groups that differed only in regard to the type of extinction training: extinction through shock omission (EXT) or extinction modified by a surprising and novel non-threat outcome in place of the shock, a procedure we refer to as novelty-facilitated extinction (NFE). Rats in both groups were placed in context B on day two of the study. Subjects in the EXT group received only the 30-second light presentation. For NFE subjects, the final 10-seconds of the light presentation were accompanied by a 5-kHz tone. This session included 30 trials separated by 15-seconds with an additional 2 minutes following the final trial. No shocks were delivered during this session. The day after extinction, all subjects were returned to context B for a test of extinction retention to the extinguished light CS alone (5 trials). Exploratory tests of conditioned inhibition and contextual renewal were then conducted in the same subjects to investigate the influence of NFE on response inhibition. Data from these tests were mixed, and are not included here to ease the interpretation of the spontaneous recovery data.

Rodent Behavior Analysis

A rater blind to experimental conditions manually scored freezing behavior. Freezing is presented as a percentage of the total CS duration. Inspection of acquisition indicated successful acquisition, i.e., all subjects demonstrated shock reactivity and high rates of freezing (data not presented). Data from extinction were analyzed with split-plot repeated measures analysis of variance (ANOVA) procedures using Extinction-block (average of 5 CS trials) and Group (EXT, NFE) as within and between subject factors respectively. Planned t-tests were used to directly compare freezing between groups during extinction and recovery tests, and were considered significant at α < .05, two-tailed.

Experiment 2

Human Participants

All participants gave written informed consent approved by the University Committee on Activities Involving Human Subjects at New York University. Sixty healthy volunteers participated in Experiment 2. Thirteen participants were excluded from the final analysis due to technical problems (N = 2), failure to attend to the task (N = 2), or failure of threat conditioning as defined by a positive difference in mean skin conductance responses (SCR) to the CS+ versus the CS− in the 2nd half of threat conditioning (N = 9). Removing these subjects (non-learners) is justified as this investigation is focused on recovery from extinction, which is predicated on participants having evinced acquisition, and is standard practice in studies investigating extinction manipulations (e.g., 26, 27, 28). The final sample included 47 participants (25 females; Mean ± SD: 24.06 ± 6.79). A third group was run to investigate the effects of avoidance responses on post-extinction recovery, but data from this group is not included here. Following informed consent, participants completed the State and Trait Anxiety Inventory [STAI; (29)] and the Intolerance of Uncertainty Scale [IUS; (30)], and then randomly assigned to the EXT group (N = 23; 13 females) or the NFE group (N =24, 12 females)

Human Threat Conditioning Procedures

Experiment 2 occurred over 2 days separated by 24 hours (Figure 2A). Conditioned stimuli were two angry faces (31) presented for 6 seconds that signaled the presence (CS+) or absence (CS−) of an aversive electric shock to the right wrist during threat conditioning. Faces serving as CS+ and CS− were counterbalanced. Each trial was separated by an intertrial interval of 12 seconds. Subjects in all experimental groups wore headphones (Sennheiser HD-280 PRO), which they were told were needed to block out background noise. This provided a convenient cover story for the NFE group who later received surprising tones through the headphones during the augmented extinction session.

Figure 2.

(A) Procedure for Experiment 2 in humans. Two groups underwent identical differential threat conditioning to initiate conditioned skin conductance responses (SCR) to a conditioned stimulus (CS+) paired with an electric shock unconditioned stimulus (US). An unpaired CS (CS−) served as a within-subjects control condition. One group then underwent standard extinction in which the US was omitted on CS+ trials, while another group underwent a modified extinction procedure in which the US was replaced by a novel non-aversive tone. Spontaneous recovery of SCRs to the CS+ versus CS− was tested 24 hours later. (B) SCR results showed equivalent acquisition (CS+ minus CS−) and extinction on Day 1. The NFE group showed no evidence of recovery 24 hours later, whereas the standard extinction group showed robust recovery. * P < .05; ** P < .01; μS = square root transformed SCRs, in microsiemens; n.s. = not significant; error bars reflect standard error.

Conditioning occurred over two runs that each included 8 presentations of the CS+ and CS− (32 total trials). An additional 8 CS+ trials (4 in each run) co-terminated with the US (33% reinforcement rate). CS+ trials paired with shock were not included in the data analysis to mitigate any potential confound in the psychophysiological analysis introduced by the shock; thus, the number of CS+ and CS− trials in the analysis were equal. A short habituation phase preceded acquisition to reduce initial orienting responses, which contained 4 trials each of the CS+ and CS− (data not reported).

Following acquisition, subjects were randomly assigned to group EXT, or group NFE in which a low-volume (> 60 decibel) 440 Hz-tone was unexpectedly presented and co-terminated with each CS+ trial in place of the electric shock for 1.5 s. In both groups, extinction occurred over two runs that included 8 presentations each of the CS+ and CS− (32 total trials) in the absence of the US. Subjects returned 24 hours later, at which time the shock electrodes were re-attached and the intensity level was set to the level reached the previous day. The shock intensity was not recalibrated so as to avoid reinstatement prior to the spontaneous recovery test. The recovery test included the CS+ and CS− (10 trials of each) in the absence of shock. Additional details on the electrical shock, collection of skin conductance responses, and an exploratory test of reinstatement are included in the Supplemental Materials.

Human Psychophysiology Data Analysis

SCRs were analyzed separately for early and late trials, as differences in responding between the CS+ and CS− tend to vary over the course of training (see also 32). The late phase of threat conditioning was defined as the mean of the last 8 CS+ and 8 CS− trials. The late phase of extinction was defined as the last 3 CS+ and 3 CS− trials. As the late phase of acquisition and extinction are the most relevant to show learning effects, only the late phase for acquisition and extinction are reported. Additionally, as prior research has shown that extinction manipulations tend to affect the earliest presentations of the CS+ versus the CS− in recovery tests (33), the early phase of spontaneous recovery was defined as the mean of the first 3 CS+ trials and first 3 CS− trials. The late phase of extinction retention was defined as the mean of the last 3 trials of the CS+ and 3 CS−. Results from each phase were analyzed by repeated measures ANOVA with stimulus (CS+, CS−) as a within subjects factor and group as a between subjects factor, considered significant at α < .05, two-tailed. We provide a complementary analysis of recovery that accounts for individual differences in acquisition, adapted from Milad et al. (34). We calculated a “spontaneous recovery index” by dividing each subject’s mean SCRs to the CS+ during early recovery by the largest SCR to the CS+ during acquisition on Day 1. This value provides a percent of recovery as a function of initial acquisition, with higher values indicating stronger recovery as a function of initial conditioning, whereas lower values indicate lower recovery as a function of initial conditioning.

Results

Experiment 1: Novelty-facilitated extinction versus standard extinction in rats

Extinction

Freezing diminished over the course of extinction training in both groups (Figure 1B). There was a significant main effect for trial block (average of 5 trials; F _{5, 15} = 9.16, P < .001), and no significant effect of group (P = .21); thus, extinction proceeded comparably for both groups. Furthermore, there was no difference in freezing in the last 5 trials of extinction between the two groups (t₃₀ = 1.28, P = 0.21), demonstrating that they were at an equivalent level of freezing by the end of extinction.

24 hour extinction retention test

Extinction retention tests showed that subjects in the EXT group froze significantly more than subjects in the NFE group (t₃₀ = 2.70, P = .01, d = .87), indicating that the NFE protocol diminished recovery of defensive responses compared to mere omission of shock (i.e., standard extinction) in rats (Figure 1C).

Experiment 2: Novelty-facilitated extinction versus standard extinction in humans

Acquisition

SCRs from late threat conditioning showed equivalent levels of acquisition between groups (Figure 2B). There was a main effect of CS (F _{1, 45} = 63.11, P < .001, η²_p= .58), but no Group interaction (P = .99). Planned t-tests on differential SCRs (CS+ minus CS−) confirmed acquisition in group EXT (t₂₂ = 5.26, P < .001) and group NFE (t₂₃ = 6.02, P < .001), with no difference between groups (P = .93).

Extinction

By late extinction, there was no effect of CS (P = .13) and no Group interaction (P = .15). As confirmation of extinction, t-tests performed on the late extinction SCR difference scores were not significant for either group (Ps > .05).

24-hour spontaneous recovery test

The primary prediction in this experiment was that replacing the US with a novel non-threat outcome would reduce post-extinction recovery more effectively than merely omitting the US altogether. The early phase of spontaneous recovery was characterized by a main effect of CS (F _{1, 45} = 18.72, P < .001, η²_p= .29) as well as a CS X Group interaction (F _{1, 45} = 5.94, P = .019, η²_p= .12). Planned t-tests revealed greater differential SCRs (CS+ minus CS−) in the EXT relative to the NFE group (t₄₅ = 2.43, P = .019, d = .71). Consistent with the CS X Group interaction, paired t-tests showed no SCR difference between the CS+ (Mean ± Standard Error: .41 ± .07) and CS− (.33 ± .08) in the NFE group (t₂₃ = 1.35, P = .19), but significantly greater SCRs to CS+ (.74 ± .09) than to the CS− (.43 ± .06) in the EXT group (t₂₂ = 4.73, P < .001). These effects persisted into late recovery, where there remained a main effect of CS type (F _{1, 45} = 7.33, P = .01, η²_p= .140) and a CS X Group interaction (F _{1, 45} = 6.96, P = .01, η²_p= .13). Again, this effect was driven by a greater SCR difference score in the EXT relative to NFE group (t₄₅ = 2.63, P = .011, d = .77). Consistent with analysis on early recovery SCRs, the spontaneous recovery index also showed greater recovery in the EXT (57.28% ± 5.96%) relative to the NFE (31.23% ± 5.34%) group (t₄₅ = 3.37, P < .01) (Figure 3A). Overall, these results provide evidence that a novel non-threat outcome during extinction reduced post-extinction recovery relative to standard extinction in which the US was merely omitted.

Figure 3.

(A) A complementary index of recovery was created for each subject by dividing mean SCRs to the CS+ during early recovery by the subject’s largest SCR to the CS+ during acquisition on Day 1. This value was multiplied by 100 providing an index of recovery as a function of initial responding for each subject. The standard extinction group showed significantly greater recovery by this index than the novelty-facilitated extinction group. (B) Index of 24-hour spontaneous recovery correlated with self-reported intolerance for uncertainty (IUS) in the standard extinction group. These correlations were selective to the standard extinction group. ** P < .01; error bars reflect standard error.

Correlations between spontaneous recovery and Intolerance of Uncertainty

One potential explanation for the fragility of standard extinction is that the omission of the US could in some individuals trigger a general sense of future threat uncertainty (35). That is, rather than instilling a sense of safety, omission renders the probability of shock less predictable in the long-term. This uncertainty could interfere with the formation of a persistent extinction memory and lead to threat recovery when the CS is reencountered. One approach to test this hypothesis is to examine whether threat recovery following extinction is related to individual subject’s inability to cope with ambiguity and uncertainty, as measured by subjective report on the Intolerance of Uncertainty Scale (IUS). Intolerance of uncertainty (30, 36) is characteristic of individuals who tend to worry excessively (37), a trait characterized by interpreting ambiguous events as threatening and overestimating risk (38). There was a positive correlation between SCRs during early spontaneous recovery and IUS scores in the EXT group (r²₂₁ = .23, P = .022) but not in the NFE group (P = .89) (Figure 2C). The correlation in the EXT group remained significant after controlling for trait anxiety levels, r²₂₀ = .21, P = .034. A correlation with IUS scores was also observed using the spontaneous recovery index (r²₂₁ = .29, P = .008) but not in the NFE group (P = .89) (Figure 3B). This association also held after controlling for trait anxiety levels, r²₂₀ = .24, P = .019), and a test of the difference between correlation coefficients revealed a significant difference between the EXT and NFE group (P = .04).

Discussion

Results show that replacing an aversive shock with a novel non-aversive outcome was more effective at diminishing recovery of conditioned responses in rats and humans than extinction that relied on omission alone. Effective extinction learning necessitates that the initial CS-US association is updated and modified by new information. Surprise encourages this new learning by signaling change (15). Thus, when extinction fails to yield a persistent memory, it could be due to the ineffectiveness of omission alone signaling a clear change in the meaning of the CS. Consequently, reductions in defensive responses seen over the course of standard extinction training do not necessarily reflect the formation of a durable safety memory (39, 40).

In conditioning research, two heavily investigated forms of retroactive interference are extinction and counterconditioning (41). Both types of new learning interfere with the original CS-US association, but the original CR is prone to recovery (20). Thus, the results obtained here are in contrast to those predicted from a counterconditioning protocol. Theoretical accounts of counterconditioning have proposed that the effect relies on competing motivational states and competing expectations between different reinforcements with incompatible responses (17). In the case of aversive-to-appetitive counterconditioning, reinforcement by a new appetitive outcome upholds the CS as an important cue in the environment that commands a response - the manner determined by which memory representation is retrieved at the time, approach or withdrawal. Indeed, counterconditioning is also referred to as cross-motivational transfer in the learning literature (e.g. 22), implying the goal of the technique is to transfer responses from one set of motivated behaviors to an opposing set of motivated behaviors. Here, we incorporated an outcome interference design that did not reinforce a behavioral response. Consequently, subjects may have reinterpreted the CS as behaviorally irrelevant through association with a novel neutral outcome. If the goal of new learning is to reduce the relevance of threat conditioned cues altogether, then counterconditioning may not be the most appropriate technique.

The technique used here can also be distinguished from response inhibition techniques that focus on reducing the aversiveness of the US itself, including US habituation (42) and US revaluation (43). In the present study, the meaning or intensity of the US was not targeted. Rather, we focused on the uncertainty inherent to omission of the US during extinction by replacing the US with a novel outcome.

The cross-species design of this study provides generalizability across different levels of behavior, from freezing in rats to autonomic arousal in humans. It has been suggested that SCRs are a cognitive expression of conditioned learning (44). That the NFE procedure was effective in rats, however, provides evidence that these effects are not due to some cognitive factor specific to humans. Importantly, in humans we were able to assess subjective ratings of intolerance of uncertainty to test the hypothesis that recovery following standard extinction is related to individual differences in IU, affording a different level of analysis than is possible from an animal model. Positive findings of the NFE technique in rats also opens the possibility for more extensive future research on the neurophysiology underlying this effect than is possible using humans, with a likely focus on regions implicated in novelty enhanced learning, including the hippocampus and striatum (45).

The inclusion of human subjects also provides a closer step towards understanding the clinical implications of the present results. The NFE technique may be appropriate for therapeutic purposes when trying to diminish the perceived significance of situations or stimuli inappropriately associated with danger. In other words, an association with an unremarkable outcome could restore the neutrality of everyday events and objects. Clinical applications could include augmenting exposure therapy to increase surprise using novel but unremarkable outcomes, in line with clinical models that highlight the critical role for belief disconfirmation for the success of exposure treatments (46, 47). For example, in exposure treatments, it may be more beneficial to have negative beliefs disconfirmed by experiencing an unexpectedly mundane event, rather than simply experiencing the absence of a negative event. These findings lend support to the idea that updating beliefs for negative outcomes may occur more efficiently through this sort of surprise, and also fit with associative learning models that emphasize the importance of surprise on updating previously held knowledge (8, 15).

While much research describes post-extinction return of defensive responses as an unwanted consequence of unsuccessful extinction, it is important to point out that the failure to completely override threat associations serves an adaptive purpose. That is, it is advantageous to maintain a memory that certain cues signal danger despite some evidence to the contrary, because even one encounter with a threat (e.g., a predator) threatens survival. In other words, fully effective extinction could lead to the incorrect conclusion that a cue is entirely safe, with disastrous results. Extinction learning provides animals with a repertoire of flexible behaviors that are (ideally) generated under the appropriate circumstances. In order to utilize the principles of extinction as a therapeutic tool, one must overcome the limits to extinction, which are inherently beneficial under survival circumstances, but maladaptive in pathological anxiety. Learning through surprise that a CS with a prior history of predicting danger now predicts something trivial may help overcome these limits.

We note some limitations to the present studies. First, future research using the NFE paradigm in clinical populations is needed to generalize these results to anxiety patients. Future experiments will also be needed to determine how NFE targets other post-extinction phenomena, including contextual renewal and reinstatement. Future studies in humans should consider incorporating immersive virtual reality to emulate characteristics of real world environments (48). Finally, an intriguing possibility is that NFE transforms the CS+ into a conditioned inhibitor (i.e., safety signal). Such an effect would be a powerful demonstration that the NFE technique is far more effective than standard extinction, which does not result in the CS taking on the properties of a safety signal (see also (49))

Finally, these results provide possible explanations as to why extinction often fails to instill persistent memories, as individuals who express an inability to cope with ambiguity and uncertainty were more susceptible to threat recovery following standard extinction. The novelty-facilitated extinction technique presented here provides a straightforward technique to help mitigate uncertainty inherent to extinction procedures, and may be especially beneficial to reduce relapse in individuals with high to intolerance to uncertainty. Insofar as the principles of threat conditioning and extinction have been successfully extended to understand and treat clinical anxiety (5), this finding offers new insights into the types of individuals who may be more likely to relapse following exposure therapy, and establishes a strategy that is simple to implement and may provide more effective clinical outcomes than strategies based on merely omitting aversive events.