Prepared stimuli enhance aversive learning without weakening the impact of verbal instructions

Abstract

Fear-relevant stimuli such as snakes and spiders are thought to capture attention due to evolutionary significance. Classical conditioning experiments indicate that these stimuli accelerate learning, while instructed extinction experiments suggest they may be less responsive to instructions. We manipulated stimulus type during instructed aversive reversal learning and used quantitative modeling to simultaneously test both hypotheses. Skin conductance reversed immediately upon instruction in both groups. However, fear-relevant stimuli enhanced dynamic learning, as measured by higher learning rates in participants conditioned with images of snakes and spiders. Results are consistent with findings that dissociable neural pathways underlie feedback-driven and instructed aversive learning.

Human emotion and behavior is influenced by evolutionarily adapted mechanisms that are conserved across organisms. However, humans also learn directly from language and rule-based knowledge. A growing body of research aims to dissociate conserved, biologically prepared processes from those that are sensitive to higher-order influences, such as verbal instruction. The goal of the present study was to test formally how biological preparedness influences aversive learning and its modulation by instructions.

Fear-relevant stimuli such as snakes and spiders are thought to engage biologically prepared mechanisms that shape responses and behavior (Seligman 1971; Mineka and Ohman 2002). The extant literature on fear conditioning suggests two main ways in which preparedness can impact conditioning. First, fear-relevant stimuli enhance fear acquisition and retention. Relative to fear-irrelevant stimuli (e.g., flowers, mushrooms), fear-relevant stimuli have been shown to facilitate classical conditioning (Ho and Lipp 2014) and slow extinction learning (Fredrikson et al. 1976; Ohman et al. 1975a,b). Work in nonhuman primates suggests that innate responses to fear-relevant stimuli are mediated by the amygdala and orbitofrontal cortex (Meunier et al. 1999; Kalin et al. 2001; Murray and Izquierdo 2007). These structures play crucial roles in fear acquisition (Davis 1992; Maren 2001), extinction (Schiller and Delgado 2010; Milad and Quirk 2012), and value-based learning (Schoenbaum et al. 1998; Holland and Gallagher 2004), and thus their preferential recruitment by prepared stimuli might enhance dynamic aversive learning.

Second, a distinct literature indicates that fear-relevant stimuli may be less responsive to instructions about safety. Studies of instructed extinction (Hugdahl and Ohman 1977; Hugdahl 1978) indicate that when individuals are informed that shocks will no longer be delivered following standard Pavlovian conditioning, conditioned responses are abolished immediately in people exposed to neutral stimuli, whereas conditioned responses remain elevated in those conditioned with biologically prepared stimuli. A dominant interpretation of these findings is that fear-relevant stimuli preferentially engage subcortical pathways that are less sensitive to higher-order knowledge, rendering them impervious to instructions. However, results are mixed. Studies that measured the effects of contingency instructions surrounding reversals (McNally 1981) or prior to acquisition (Mertens et al. 2016) found that instructions did indeed influence responses to fear-relevant stimuli, although McNally lacked a fear-relevant control condition and therefore could not assess whether responses were impacted by preparedness. Thus the question of whether and how biological preparedness impacts cognitive influences on aversive learning remains unanswered.

In this paper, we jointly tested the effects of biological preparedness on both (a) dynamic aversive learning and (b) its modulation by cognitive knowledge via instruction. Over the past three decades, quantitative models of reinforcement learning have revealed new insights into conditioning and associative learning by allowing researchers to isolate dynamic components of the learning process. These models capture how organisms develop and update expectations in response to salient events in the environment. The models incorporate a “learning rate,” which captures the speed at which expectations update in response to outcomes and expectancy violations. Quantitative learning models differ from traditional analyses that average across trials over time (e.g., ANOVAs), as they assess how quickly responses update following a surprising event (e.g., an unexpected shock) on a trial-by-trial basis.

We recently introduced a new quantitative model that simultaneously isolates learning rates and additionally quantifies the effects of instructions on aversive reversal learning (Atlas et al. 2016). We found dissociable effects of instructions on aversive reversal learning such that the orbitofrontal cortex, striatum, and skin conductance responses (SCRs) updated immediately with instructions, whereas the amygdala updated based on aversive outcomes irrespective of instructed knowledge. These findings are consistent with theories of evolutionarily adapted automatic threat detection in the amygdala (Ohman 2005) that suggest that amygdala responses (and, by extension, amygdala-dependent processes) may be impervious to cognitive instruction (Ohman and Mineka 2001). In the present study, we extended this work to measure the influence of stimulus preparedness, or fear relevance. The aim of our study was to test formally whether biological preparedness modulates flexible aversive learning and the impact of instructed knowledge.

As shown in Figure 1, participants assigned to a Prepared Stimulus Group (PG; n = 20) viewed images of snakes or spiders, while a Neutral Stimulus Group (NG; n = 20) viewed abstract fractals. Visual stimuli were paired with aversive electric shocks in a standard discriminative conditioning procedure, and contingencies reversed halfway through the task. We instructed all participants about the updated contingencies several trials before the reversal, which allowed us to dissociate feedback-driven and instructed learning, similar to Atlas et al. (2016). Skin conductance responses (SCRs) served as a measure of aversive learning. We used computational models and standard statistical approaches to measure the effects of stimulus preparedness across all participants. In Supplemental Material, we also provide results of secondary analyses restricted to participants who demonstrably acquired conditioned fear prior to the reversal (“responders”). Restricting analyses to participants who display differential responses prior to reversal ensures that reversal instructions are meaningful, and verifies that conclusions from quantitative analyses (which were fit to SCR) are not driven primarily by participants without measurable SCR. All results from the full sample reported below remained significant when analyses were restricted to responders, unless stated otherwise. Please see Supplemental Material for complete details on participants, experimental design, skin conductance acquisition and analysis, and results of analyses restricted to responders.

Figure 1.

Experimental design. (A) Prior to the experiment, participants were randomly assigned to either the Neutral Group (n = 20), which saw two fractals, or the Prepared Group, which saw images of either snakes (n = 10) or spiders (n = 10). All participants were told that their task was to pay attention to the screen and to try to learn the relationship between the stimuli they saw and the shocks that they felt. (B) Both groups underwent the same Pavlovian fear conditioning task with a single instructed reversal, adapted from Atlas et al. (2016). (I) During Acquisition, the original CS+ (depicted in yellow) was paired with a shock on 30% of trials, while the original CS− was never paired with a shock. (II) After 20 trials, all participants were told that the contingencies had reversed. (III) Each stimulus was then presented, unreinforced, at least twice before (IV) the previous CS−/new CS+ was paired with a shock. (V) Learning then proceeded with the new, reversed contingencies until the end of the task. As in our previous work (Atlas et al. 2016), our design allowed us to test whether conditioned responses update with instructions, or whether they require reinforcement in order to update. If responses update with instructions we would expect to see SCR reversals immediately upon instruction, whereas if learning is driven by aversive feedback alone, SCRs would not reverse until the new context is reinforced (i.e., when the previous CS− is paired with a shock).

We fit our recently developed quantitative model of instructed aversive reversal learning (Atlas et al. 2016) to SCRs from both groups. As explained in Supplemental Material, the model is an adapted Rescorla–Wagner reinforcement learning model that describes how expected value updates in response to unexpected outcomes (aversive shocks, in this case). We assume that dynamic expected value correlates with trial-by-trial SCR and fit models to unreinforced trials, consistent with other quantitative models of aversive learning (Li et al. 2011; Atlas et al. 2016; Zhang et al. 2016). Our model assumes that both groups start with the same expected value (0.5) for both CS cues, as participants were not informed about contingencies prior to learning. We then use observed data to fit a learning rate, α, which dictates the speed of learning over time (i.e., both pre- and post-reversal). Our model includes one additional parameter, ρ, which captures the extent to which responses reverse upon instruction. If ρ = 1, the expected value of the previous CS+ becomes the expected value associated with the previous CS− at the time of instruction, whereas if ρ = 0, each cue retains its current expected value. While standard reinforcement learning models allow the expected value of the CS+ and CS− to update independently, our reversal parameter implements a direct link between the quantities at the time of the reversal (and only at this time) because participants are informed that the cue contingencies are exchanged at the time of instructions. Outside of the reversal phase, the value of each CS updates independently based on experiential learning, consistent with standard models.

We used an iterative jackknife model fitting procedure (Wu 1986; Miller et al. 1998) to estimate group-level parameters that allow for statistical comparison. On each iteration, one participant was left out from each group, and we estimated parameters for the remaining participants. This generates a distribution of estimates for each group that can be used for statistical comparison and are less likely to be driven by noise than models separately to each participant. See Supplemental Material and Atlas et al. (2016) for full details on the model and model fitting procedures.

Quantitative model fits revealed that PG participants learned significantly faster than NG participants as measured by higher learning rates (see Fig. 2), based on jackknife estimates from models fit across subjects within each group, whether models were fit across all participants (α_PG: M = 0.53, SD = 0.02; α_NG: M = 0.14, SD = 0.02; t₍₃₈₎ = 69.74, P < 0.001) or restricted to responders (see Supplemental Material). We also found higher learning rates in the PG group when we restricted our model to data from the acquisition phase, i.e., prior to reversal (α_PG: M = 0.65, SD = 0.01; α_NG: M = 0.49, SD = 0.03; t₍₃₈₎ = 24.59, P < 0.001), and when initial expected value was modeled as a free parameter (α_PG: M = 0.38, SD = 0.03; α_NG: M = 0.21, SD = 0.03; t₍₃₈₎ = 16.45, P < 0.001). This suggests that individuals update expected value (indicated by SCR) more quickly when aversive outcomes are paired with prepared, as opposed to neutral, stimuli. However, there were no Group differences in the extent to which instructions influenced responses (see Fig. 2), as captured by the instructed reversal parameter, whether models were fit across all subjects or restricted to responders (ρ_PG = 1.0; ρ_NG = 1.0). Both groups reversed responses immediately upon instruction, irrespective of stimulus preparedness.

Figure 2.

Quantitative models reveal enhanced feedback-driven learning from prepared fear stimuli. Our quantitative learning model allows us to isolate both dynamic learning rate (α; left) and the extent to which responses/expectations reverse immediately upon instruction (ρ; right). As shown in the left panel, we found that individuals who undergo conditioning with images of snakes or spiders (orange) learn faster from aversive outcomes than individuals who are exposed to neutral fractals (blue), based on an iterative jackknife model fitting procedure (P < 0.001). This was the case whether models were fit to responders (above), all subjects, and when restricted to the acquisition phase. However, SCRs in both groups updated immediately in response to instructions that contingencies had reversed, as indexed by ρ parameters (right). Thus stimulus preparedness enhanced learning rates, but did not impact the effect of instructions on aversive learning.

To verify the conclusions of our quantitative model, we used linear mixed models to examine SCRs on trials that immediately followed instructions, prior to any reinforcement (see Fig. 1), compared with trials that immediately preceded instructions, as in Atlas et al. (2016). This analysis is independent of model fit, and tests whether SCR reverses immediately upon instruction, and whether the magnitude of the instructed reversal differs across groups (see Supplemental Material for details). As shown in Figure 3, both groups showed reversals of SCR immediately in response to instructions (Stimulus × Reversal: β = 0.13, P < 0.001) although we also found elevated responses to the original CS+ relative to the original CS− across the phase surrounding the reversal (Stimulus effect: All subjects: β = 0.03, P < 0.05) during this window. However, there were no Group differences in these effects (NG versus PG, P > 0.1), providing further support that prepared stimuli are not less sensitive to immediate effects of instructed reversals, consistent with our findings of equivalent instructed reversal parameters in the quantitative models reported above.

Figure 3.

Skin conductance responses on unreinforced trials. Normalized skin conductance responses (SCRs) to unreinforced trials in participants who acquired the conditioned response (n = 33) as a function of Group and Condition. (A) Mean normalized SCRs on unreinforced trials prior to the instructed reversal. (B) Mean normalized SCRs on unreinforced trials following the instructed reversal. (C) Normalized SCRs on all unreinforced trials across time. Black vertical dashed lines depict the delivery of instructions regarding the reversal, and red dashed lines denote when the initial CS− was paired with a shock. Both groups reversed immediately upon instruction and there were no group differences in the magnitude of the differential response or its reversal with instructions.

Finally, we examined potential group differences using ANOVAs that tested for main effects of group and condition as a function of phase (acquisition versus reversal). Unlike our quantitative models, which measure trial-to-trial influences both pre-and post-reversal, ANOVAs average responses across time. Consistent with our quantitative models, participants in both groups showed SCRs that were larger for the current CS+ than CS−, both pre- and post-reversal (F_(1,38) = 75.42, P < 0.001). Participants also showed larger SCRs during the first half of the task relative to the second half (effect of Phase: F_(1,38) = 6.74, P < 0.05). When we averaged across all trials, we did not observe main effects of Group, nor Group × Condition interactions (all P’s > 0.3), although we found a marginal Group × Time (Early versus Late) interaction (F_(1,38) = 3.89, P = 0.056), suggesting responses habituated faster within each half of the task in the Neutral Group. We note that the effect of Phase and the Group × Time interactions were not significant when we limited analyses to responders (see Supplemental Material). Thus our basic ANOVAs revealed no effects of stimulus preparedness when we averaged across trials. This suggests that the group differences isolated with our quantitative models are driven by dynamic, trial-by-trial responses to outcomes, rather than sustained effects of stimulus preparedness on the acquisition, expression, or reversal of conditioned fear.

Taken together, our findings demonstrate that prepared stimuli enhance dynamic learning from aversive outcomes, but that verbal instructions immediately update responses irrespective of stimulus type. Prepared stimuli such as the snakes and spiders we used here are thought to draw on basic evolutionarily conserved mechanisms (Seligman 1971; Mineka and Ohman 2002). They elicit preferential attention and enhanced learning early in development in human infants (LoBue and DeLoache 2010) and primates (Cook and Mineka 1989; Kalin et al. 2001; Murray and Izquierdo 2007), and the specific phobia rate for such stimuli is elevated relative to nonprepared stimuli (de Silva et al. 1977; de Silva 1988; Fredrikson et al. 1996). Consistent with these population-based findings, experimental work has shown that prepared stimuli elicit increased arousal and preferential attention relative to neutral stimuli (for review, see Ohman and Mineka (2001)). Neurobiologically, their effects are thought to be mediated by the amygdala as well as the orbitofrontal cortex (Meunier et al. 1999; Kalin et al. 2001; Murray and Izquierdo 2007). Our quantitative models reveal that discriminative fear conditioning occurred faster when humans were exposed to images of snakes or spiders than when they were exposed to images of abstract fractals, replicating prior work (Ho and Lipp 2014). We conducted a follow-up experiment (n = 41) to ensure that differences in learning rate were not driven by differences in perceptual discriminability and found that the fractals were actually perceived as “more” different than the pairs of snakes or spiders (see Supplemental Material), indicating that the enhanced learning was due to fear-relevance/preparedness, not ease of discriminability.

Our quantitative model revealed effects of stimulus preparedness on dynamic learning rate during acquisition and following reversal, which replicates and extends previous work: Ho and Lipp (2014) observed faster acquisition with fear-relevant stimuli in a within-subjects design, although conclusions were based on averaging across blocks of trials rather than actual learning rates per se. Interestingly, we did not observe effects of stimulus preparedness on the magnitude of the differential response when we averaged across trials within each phase, unlike previous studies (Fredrikson et al. 1976; Siddle et al. 1988; Ohman and Soares 1993; Ohman and Mineka 2001), although results have been mixed (McNally 1987). Our study design differs from prior work in this area in that we (1) manipulated stimulus preparedness between, rather than within, subjects; (2) used a low reinforcement rate in order to measure the dynamics of conditioned responses on unreinforced trials; and (3) measured reversal, rather than extinction. Perhaps most importantly, previous studies applied standard statistics that average responses across time to test for differences in overall magnitude of the differential response, whereas we used dynamic learning models that evaluate responses as a function of time. These models allow us to isolate differences in the speed at which outcome measures react to events in the environment (i.e., in response to instructions or reinforcement), which is not possible in standard approaches that average responses over time. To our knowledge, this is the first paper to use quantitative methods to measure effects of stimulus preparedness on dynamic aversive learning.

While prepared stimuli enhanced the speed of learning, SCR responses (a measure of arousal) updated immediately when individuals were instructed that contingencies had reversed, irrespective of stimulus type. These findings stand in stark contrast to previous reports that prepared stimuli are impervious to instructions (Hugdahl and Ohman 1977; Hugdahl 1978), and are consistent with findings that prepared stimuli facilitate classical conditioning (Ho and Lipp 2014). One important distinction is that we examined the effects of instructed reversals, rather than instructed extinction. Instructed reversals examine the flexibility of conditioned responses and require the acquisition of a new differential response, whereas instructed extinction measures only safety learning. Likewise, our instructed reversal task differs from standard instructed fear paradigms (Hugdahl and Ohman 1977; Olsson and Phelps 2007), which generally measure effects of instructions in the absence of reinforcement and therefore cannot dissociate the effects of instructions from the effects of feedback-driven learning, although some important variants exist (Mertens and De Houwer 2016). We also note that our instructed reversals create a direct reciprocal relationship between the CS+ and CS−, which differs from standard associative learning. Future studies might dissociate the effects of instructions on fear learning and safety learning by using our quantitative approach to estimate the effects of instruction on instructed fear and instructed extinction. In addition, we acknowledge that we only measured SCR in the present study. Future studies should consider other measures of arousal and/or defensive responses, such as startle, which has been shown to be less sensitive to instructed extinction (Sevenster et al. 2012).

The dissociation between the effects of stimulus preparedness on experiential versus instructed learning parallels our recent findings of neural dissociations between these two forms of learning (Atlas et al., 2016). We found that amygdala tracked experiential learning irrespective of instructions, whereas the ventromedial prefrontal/orbitofrontal cortex (as well as the striatum and autonomic arousal) updated immediately with instructions. Interestingly, work in nonhuman primates has shown that lesions of the orbitofrontal cortex and the amygdala both abolish monkeys’ innate tendency to avoid snake-like objects (Murray and Izquierdo 2007). Our findings suggest that prepared stimuli may enhance amygdala-dependent feedback-driven learning without impacting the effect of instructions on cortico-striatal interactions and downstream processes such as arousal. Future studies should directly test these hypotheses and potential dissociations using functional magnetic resonance imaging or other neuroscientific techniques.