Skip to main content
PMC home page
Social Cognitive and Affective Neuroscience logoLink to Social Cognitive and Affective Neuroscience
. 2016 Oct 31;12(1):106–115. doi: 10.1093/scan/nsw157

Neural correlates of appetitive extinction in humans

Onno Kruse 1,2,✉, Isabell Tapia León 1,2, Rudolf Stark 1,2, Tim Klucken 1,2,3
PMCID: PMC5537618  PMID: 27803289

Abstract

Appetitive extinction receives attention as an important model for the treatment of psychiatric disorders. However, in humans, its underlying neural correlates remain unknown. To close this gap, we investigated appetitive acquisition and extinction with fMRI in a 2-day monetary incentive delay paradigm. During appetitive conditioning, one stimulus (CS+) was paired with monetary reward, while another stimulus (CS−) was never rewarded. Twenty-four hours later, subjects underwent extinction, in which neither CS was reinforced. Appetitive conditioning elicited stronger skin conductance responses to the CS+ as compared with the CS−. Regarding subjective ratings, the CS+ was rated more pleasant and arousing than the CS− after conditioning. Furthermore, fMRI-results (CS+ − CS−) showed activation of the reward circuitry including amygdala, midbrain and striatal areas. During extinction, conditioned responses were successfully extinguished. In the early phase of extinction, we found a significant activation of the caudate, the hippocampus, the dorsal and ventral anterior cingulate cortex (dACC and vACC). In the late phase, we found significant activation of the nucleus accumbens (NAcc) and the amygdala. Correlational analyses with subjective ratings linked extinction success to the vACC and the NAcc, while associating the dACC with reduced extinction. The results reveal neural correlates of appetitive extinction in humans and extend assumptions from models for human extinction learning.

Keywords: fMRI, reward, conditioning, extinction, nucleus accumbens, amygdala

Introduction

Appetitive conditioning and extinction receive increasing attention as a model for psychiatric disorders (Martin-Soelch et al., 2007). Especially, the extinction of reward associated cues plays an important role in the ability to flexibly interact with the environment and, in clinical cases, in the treatment of addiction-related disorders (Taylor et al., 2009; Everitt, 2014). However, most research on the neural mechanisms of appetitive extinction is conducted in animals (Millan et al., 2011). In humans, only the neural mechanisms of fear extinction have been extensively investigated (Quirk and Mueller, 2008; Sehlmeyer et al., 2009).

Appetitive conditioning in humans can be studied using a classical (Pavlovian) conditioning paradigm or the monetary incentive delay paradigm (MID), which adds an operant conditioning component to the task. In both cases, a neutral stimulus (e.g. a colored rectangle; CS+) is paired with the chance to get a reward (UCS; e.g. money), whereas another neutral stimulus (CS−) is never paired with the chance to win. Additionally, in the MID paradigm, to gain the reward, subjects are required to press a button in reaction to an operant cue that appears in all trials with a varying delay after the CS. The repeated pairing of the CS+ with the UCS (reward) in appetitive conditioning paradigms results in conditioned responses (CRs). CRs comprise positive affective ratings of the CS+ and elevated skin conductance reactions (SCRs) as compared with the CS− (Kirsch et al., 2003; Klucken et al., 2013b, 2015; Andreatta and Pauli, 2015). The MID, however, has rarely been used as a learning task as subjects have in most cases been instructed about the CS-UCS contingencies beforehand.

Brain areas involved in appetitive conditioning mainly include the nucleus accumbens (NAcc) in the ventral striatum, the amygdala, the orbitofrontal cortex (OFC), as well as the caudate nucleus in the dorsal striatum and the anterior cingulate cortex (ACC) (Martin-Soelch et al., 2007; Chase et al., 2015). The amygdala mediates the association between CS and UCS during acquisition of conditioning (Balleine et al., 2003). The NAcc as part of the mesolimbic dopamine system is thought to encode the salience of the reward associated stimulus and plays an important part in mediating reward, while the dorsal striatum is involved in goal-directed behavior (dorsomedial striatum) and habit learning (dorsolateral striatum) (O’Doherty et al., 2004; Burton et al., 2015). The ventral ACC plays a role in early discriminative learning, while the dorsal ACC encodes the outcome of a CS+ (Gabriel et al., 2003; Alexander and Brown, 2011). The roles of other areas like the OFC are still debated since different studies ascribed different roles to the OFC including the encoding of expected UCS-value, recall of reward and appetitive extinction (Cox et al., 2005, Moorman and Aston-Jones, 2014; Stalnaker et al., 2015).

In appetitive extinction, the CS+ is no longer paired with a reward. This leads to the formation of a new extinction memory able to inhibit the previously learned CS-UCS association but not deleting it (Bouton, 2002). In effect, extinction learning reduces the CRs (Andreatta and Pauli, 2015). With respect to its neural correlates, animal studies highlight the role of the basolateral amygdala (Tye et al., 2010; Portero-Tresserra et al., 2013) and the ventral striatum (Janak et al., 2004; Millan et al., 2010) in appetitive extinction learning. In these brain areas, different neural populations encode the previously learned reward association as well as the extinction memory. The only fMRI-study using an appetitive extinction paradigm was conducted by Tobler et al. (2007). However, the authors focused on the association of reward learning-related activations in the striatum and the midbrain and individual finances and found slower learning in richer as compared with poorer participants. Regarding fear extinction in humans, the reported neural correlates including the amygdala, the NAcc, hippocampus and ventromedial prefrontal cortex are comparable to those reported for appetitive extinction in animals (LaBar et al., 1995; Schiller et al., 2008; Sehlmeyer et al., 2009). A more detailed insight in the neural correlates of human appetitive extinction can help to translate animal findings and give insight into similarities and differences to fear extinction.

The aim of this study was the identification of the neural correlates of human appetitive extinction using the MID-task. After an appetitive conditioning paradigm on the first day, an extinction training was conducted 24 h later without any reinforcement of the CS+ and the CS−. We were interested in examining, whether among others the amygdala or the NAcc are involved in appetitive extinction learning in humans as they are in animals. We further explored correlations between neural activations and changes in subjective ratings from post-acquisition to post-extinction.

Materials and methods

Subjects

Twenty-one male subjects (M = 23.33 years; s.d. = 2.50; range = 18–28 years) were recruited to take part in the study. All subjects were right-handed, native German speakers with European background and had normal or corrected-to-normal vision. Subjects reporting any current or past mental problems or a consumption of psychotropic drugs were excluded as well as subjects with chronic illnesses or treatments preventing them from entering the MRI scanner. Because the experiment was part of an ongoing project exploring endocrinological effects on appetitive conditioning, only male subjects were recruited. All subjects gave written informed consent and received the money they won during the acquisition phase in addition to monetary compensation or course credit for their time. The study was conducted in accordance with the declaration of Helsinki and approved by the local ethics committee. Subjects also filled out the Barratt Impulsiveness Scale (BIS-15, German version, Meule et al., 2011). The mean score was 32.24 (s.d. = 5.22), similar to the evaluation sample (M = 30.04; s.d. = 6.13).

Procedure

Acquisition and extinction of the appetitive conditioning paradigm took place on two consecutive days roughly 24 h apart and always between 1 and 6 p.m. in the afternoon. As part of a larger study, all subjects took part in a placebo version of the trier social stress test (TSST) on the day of the acquisition (Het et al., 2009). This entails thinking of and subsequently talking about a pleasant topic (e.g. a favorite movie) and counting upwards in steps of five in an empty room for 15 min in sum. The placebo TSST was performed after the first subjective rating and a practice run of the experiment outside the scanner and directly before entering the scanner. Cortisol analyses confirmed that the placebo TSST did indeed not lead to a stress reaction. Moreover, PANAS (Positive and Negative Affect Schedule; Krohne et al., 1996) ratings collected before and after the placebo TSST did not indicate any positive [t(20) = 1.67; P = 0.11] or negative [t(20) = 0.99; P = 0.33] mood change (for correlations of positive mood and acquisition of conditioning see the supplementary information).

Acquisition

A modified version of the MID paradigm (Knutson, 2000) was used as conditioning procedure and was conducted in the MRI (Figure 1). It consisted of 21 CS+ with partial reinforcement (∼62% of trials followed by a reward) and 21 CS− trials, with a yellow or a blue rectangle serving as CS+ or CS−. No more than two CS trials of a kind were presented in succession and CS trials were distributed evenly in two blocks of 20 trials (10 CS+ and 10 CS−), with one CS+ and one CS− trial presented at the beginning. All trials were presented in succession; the blocks were not discernible to the subjects. The first trials of the CS+ and the CS− were later excluded from the analyses because learning could not have taken place yet (Phelps et al., 2004; Klucken et al., 2013a).

Fig. 1.

Fig. 1.

Open in a new tab

MID-task. Subjects first saw a CS− or a CS+ (colored rectangle). After a variable delay, a target appeared for a short time. Subjects were instructed to push a button as soon as the target appeared. During the acquisition phase in trials that began with a CS+, fast reactions led to a win. After the target had vanished, feedback about the win and the current total was displayed. During extinction, the same procedure was used but without wins after the CS+.

Every trial started with a variable intertrial interval ranging from 6 to 12 s followed by the presentation of a CS+ or a CS− for 6 s. Following the CS+ or CS−, after a variable (1–3 s) interstimulus interval displaying a fixation cross, the target (a white square) was presented for at least 16 ms up to a maximum of 750 ms (Figure 1). Subjects were instructed to press the button every time as soon as the target was presented regardless of the CS presented before, which they did (no significant differences in response frequency between CS or phases). Pressing the reaction button while the target was visible always resulted in a win of 0.50€, when a CS+ had been presented beforehand. To ensure a similar reinforcement for all subjects (aim: 6.50€ for wins in 62% of CS+ trials) distributed across the whole acquisition phase, the presentation time of the target was adjusted to individual reaction times (RT) in a multistep procedure (see later). Pressing the button during a target presentation that was preceded by the CS− never resulted in a win. Immediately after the target vanished, feedback on the win of money and the current balance was presented for 2.5 s.

Presentation time of the target was based on a training session with different stimuli and was longer in trials with a planned reward (win: MRT + 2 × s.d.RT loss: MRT − 2 × s.d.RT; Dillon et al., 2008; Balodis et al., 2012). If subjects did not win as planned during the acquisition phase, the presentation time of the target was adapted online based on the subjects’ performance (−20 ms if subjects won unplanned, +20 ms if subjects did not win as planned; Hahn et al., 2009; Hermans et al., 2010). CS+ trials that did not result in wins as planned or vice versa were adaptively repeated in scheduled CS+ trials with the according duration of target presentation. The adaptive online corrections of cue presentation times were necessary for only 4.1% of CS+ trials across all subjects or an average of <1 CS+ trial per subject. The target presentation times in CS− trials were taken from the CS+ trials and adapted accordingly.

Extinction

The extinction phase was conducted in the scanner 24 h later. It consisted of 40 trials with 20 CS+ and 20 CS− trials. Both early and late phase consisted of 10 CS+ and 10 CS− trials in pseudorandomized order with no more than two trials of the same condition presented directly after one another. As before, subjects were instructed to always press the button when they saw the target. In contrast to the acquisition phase, subjects could not win any money regardless how fast they reacted to the target.

Subjective ratings

Subjects were asked to rate the stimuli used as CS+ and CS− on the scales arousal, valence and UCS-expectancy prior to the acquisition phase, after the acquisition phase and after the extinction phase. To avoid reactivation of the memory trace before extinction (Schiller et al., 2010, 2013; Agren et al., 2012; Agren, 2014), ratings were not collected again right before extinction. For the affective ratings, 9-point self-assessment manikin scales were used (Bradley and Lang, 1994), while UCS-expectancy was rated in 10% steps from 0 to 100%. Subjective ratings were analyzed in 2 (CS) × 3 (Time) analyses of variance (ANOVA) using SPSS 22 (SPSS 22.0 for Windows, SPSS Inc., Chicago, IL, USA). Significant interactions were followed up with paired t-tests and were corrected for multiple comparisons.

Skin conductance measuring

Skin conductance was measured during acquisition and extinction with reusable Ag/AgCl electrodes filled with isotonic (0.05M NaCl) electrolyte medium placed on the non-dominant left hand. Data were collected with a sampling rate of 1 kHz. Ledalab 3.4.4 was used for preprocessing and data analysis (Benedek and Kaernbach, 2010). For preprocessing, the data were downsampled to 100 Hz and smoothed with a 32 sample FWHM Gaussian kernel. All data were visually screened. Technical artifacts (e.g. short negative spikes likely caused by the fMRI) were interpolated with spline interpolation. Due to technical difficulties, three subjects had to be excluded from SCR analysis, leaving a sample of 18 subjects for SCR analysis. A time window of 1–6 s following the onset of CS+ and CS− was defined as analysis window. The extracted response was defined as the highest difference between a maximum and the minimum that directly preceded it. The preceding minimum had to be within the analysis window for the response to be counted. Responses smaller than 0.01 µS were considered zero responses. To screen for non-responders, we checked whether all participants showed at least two responses > 0.05 µS to the UCS, which was the case for all participants. All maximum responses were log (µS + 1) transformed to correct for violation of normal distribution of the data. Mean SCRs for CS+ and CS− were calculated for the acquisition phase as well as for the early and late phase of the extinction phase. Skin conductance data were then analyzed separately for the acquisition phase in a paired t-test (CS+ vs CS−) and for the extinction phase in a 2 (CS+ vs CS−) × 2 (time: early phase vs late phase) using SPSS 22 (SPSS 22.0 for Windows, SPSS Inc., Chicago, IL, USA).

fMRI

All MRI images were acquired using a 3 Tesla whole-body tomograph (Siemens Prisma) with a 64-channel head coil. The structural images consisted of 176 T1-weighted sagittal slices (slice thickness 0.9 mm; FoV = 240 mm; TR = 1.58 s; TE = 2.3 s). For the functional images, a total of 432 images was acquired on the first day (acquisition phase), while 420 images were acquired on the second day (extinction phase). This difference in the number of images is caused by two additional trials (1 CS+/1 CS−) during the acquisition phase compared with the extinction phase. Images were acquired with a T2*-weighted gradient echo-planar imaging (EPI) with 36 slices covering the whole brain (voxel size = 3 × 3 × 3.5 mm; gap = 0.5 mm; descending slice acquisition; TR = 2 s; TE = 30 ms; flip angle = 75; FoV = 192 × 192 mm; matrix size = 64 × 64; GRAPPA = 2). The field of view was positioned automatically relative to the AC-PC line with an orientation of −40°. Preprocessing, first and second level analysis was done using Statistical Parametrical Mapping (SPM8, Wellcome Department of Cognitive Neurology, London, UK; 2008) implemented in Matlab 7.14 (Mathworks Inc., Sherbourn, MA). For preprocessing, all EPI images were coregistered to an EPI template, realigned and unwarped, slice time corrected, normalized to MNI standard space and smoothed with a Gaussian Kernel at 6 mm FWHM. Functional data were analyzed for outlying volumes using a distribution free approach for skewed data (Schweckendiek et al., 2013). Each resulting outlying volume was later modeled within the general linear model as a regressor of no interest.

The experimental conditions entered during the acquisition phase were CS+, CS−, UCS+, NoUCS+ (no win feedback following a CS+) and UCS− (no win feedback following a CS−). Because the target was presented for a few milliseconds before every feedback and inseparable from the feedback itself, it was not separately modeled. The experimental conditions during the extinction phase were similar to the acquisition but without the UCS+ condition because the CS+ was no longer reinforced. During acquisition, the first CS+ and CS− trial was modeled separately as learning could not have taken place yet. CS regressors were split into an early phase (CS+early/CS−early) and a late phase (CS+late/CS−late), to more clearly discern between early and late effects (LaBar et al., 1998; Phelps et al., 2004; Milad et al., 2007). CS and UCS events were modeled using the duration the respective stimulus was presented (CS = 6 s; UCS = 2 s). The maximum collinearity between a CS+ and a UCS regressor was <0.20. All regressors were convolved with the canonical hemodynamic response function. The six movement parameters were entered as covariates alongside regressors of no interest for the identified outlying volumes. The time series was then filtered with a high pass filter (time constant = 128 s). For acquisition, a CS+ − CS− contrast was calculated for every subject, while for extinction CS+early − CS−early and CS+late − CS−late were computed.

On the group level, one-sample t-tests were performed for the computed first level contrasts to examine neural differences in appetitive conditioning and extinction. Whole brain analyses were conducted with P < 0.05 family-wise-error (FWE) corrected and k > 10 voxels. Region of interest (ROI) analyses were conducted using the small volume correction in SPM8 with P < 0.05 (FWE) and k > 5 voxels. The ROI masks for NAcc, amygdala and caudate were taken from the ‘Harvard-Oxford cortical and subcortical structural atlases’ provided by the Harvard Center for Morphometric Analysis. The masks for OFC, ventral ACC and dorsal ACC were created in MARINA (Walter et al., 2003). To explore associations of neural activations and subjective measures of extinction, difference scores of CS+ ratings from post-acquisition to post-extinction were calculated. These were correlated with the extracted β estimates for significant peak voxels from the early and late extinction phase.

Results

Subjective ratings

ANOVA of the subjective rating scales (Table 1) revealed main effects of CS and Time as well as significant CS × Time interactions in valence [CS: F(1, 20) = 7.33; P = 0.014; Time: F(2, 40) = 7.87; P = 0.001; CS × Time: F(2, 40) = 5.15; P = 0.01], arousal [CS: F(1, 20) = 12.67; P = 0.002; Time: F(2, 40) = 5.49; P = 0.008; CS × Time: F(2, 40) = 8.40; P = 0.001] and UCS-expectancy ratings [CS: F(1, 20) = 78.10; P < 0.001; Time: F(2, 40) = 76.75; P < 0.001; CS × Time: F(2, 40) = 165.31; P < 0.001]. CS+ and CS− differed significantly on all three rating scales [Valence: t(20) = 3.63; P = 0.001; Arousal: t(20) = 4.13; P < 0.001; UCS-expectancy: t(20) = 19.80; P < 0.001] after conditioning while they did not differ before the acquisition phase [Valence: t(20) = 0; P = 1; Arousal: t(20) = −0.13; P = 0.900; UCS-expectancy: t(20) = 0.19; P < 0.851].

Table 2.

ROI activations during the acquisition phase (CS+ − CS−)

structure Side k x y z zmax r Pcorr
Amygdala L 121 −24 −2 −12 3.24 0.65 0.038
R 106 20 −2 −12 3.37 0.66 0.029
Caudate L 493 −8 16 −2 4.86 0.84 <0.001
R 517 10 10 4 5.05 0.85 <0.001
dACC L 1299 −6 10 42 5.19 0.86 <0.001
R 1426 4 18 38 5.38 0.88 <0.001
Midbrain 621 4 −30 −2 4.80 0.83 <0.001
NAcc L 104 −8 16 −2 4.53 0.81 <0.001
R 85 8 8 −4 4.67 0.78 <0.001
OFC L 120 14 16 −12 3.56 0.69 0.050
Open in a new tab

Localization, cluster size (k), effect size (r) and statistics (FWE-corrected) of the peak voxel in the respective ROI.

Table 3.

ROI activations during the early phase and late phase of extinction (CS+ − CS−)

contrast Structure side k x y z zmax r Pcorr
CS + (early) − CS − (early) dACC L 616 −10 2 34 4.18 0.77 0.007
vACC L 330 16 38 20 3.58 0.69 0.034
Caudate R 189 8 16 8 3.52 0.69 0.033
Hippocampus L 194 −24 38 −4 3.33 0.66 0.073
R 72 22 −38 6 3.53 0.69 0.041
CS + (late) − CS − (late) Amygdala L 85 −12 −6 −18 3.43 0.67 0.024
R 130 16 −4 −18 3.36 0.66 0.032
Nacc L 71 −12 8 −8 3.24 0.64 0.020
R 30 12 14 −6 2.99 0.61 0.033
Open in a new tab

Localization, cluster size (k), effect size (r) and statistics (FWE-corrected) of the peak voxel in the respective ROI.

Table 1.

Mean (s.d.) subjective ratings of CS+ and CS − 

Pre- acquisition Post- acquisition Post- extinction
Arousal CS +  3.19 (1.86) 6.00 (1.92)*,† 3.95 (2.25)*
CS −  3.24 (1.92) 3.52 (1.54) 3.52 (1.99)
Valence CS +  5.43 (1.94) 6.76 (1.67)*,† 4.71 (1.98)*
CS −  5.43 (1.60) 4.67 (1.62) 4.24 (2.05)
UCS-expectancy CS +  5.62 (1.75) 8.19 (1.50)*,† 1.52 (2.02)*
CS −  5.52 (1.57) 0.76 (0.83)* 0.90 (1.18)
Open in a new tab

*indicates a significant difference to the mean rating of the same CS at the previous time point (P < 0.05).

†indicates a significant difference to the mean rating of the CS- at the same time point (P < 0.05).

After the extinction phase, ratings of the CS+ dropped [valence: t(20) = 3.75; P < 0.001; arousal: t(20) = 4.02; P < 0.001; UCS-expectancy: t(20) = 16.49; P < 0.001] while ratings of the CS− remained unchanged [valence: t(20) = 1.34; P = 0.20; arousal: t(20) = 0; P = 1; UCS-expectancy: t(20) = −0.45; P = 0.658]. In addition, CS+ and CS− did no longer differ significantly on ratings of UCS-expectancy, arousal and valence after the extinction phase [valence: t(20) = 1.11; P = 0.280; arousal: t(20) = 1.12; P = 0.275; UCS-expectancy: t(20) = 2.03; P = 0.056].

Skin conductance responses

During the acquisition phase, SCRs showed a significant effect of conditioning [t(17) = 3.01; P = 0.004] (Figure 2). During the extinction phase, analysis of SCRs in a 2 × 2 ANOVA did neither show a main effect of CS [F(1, 17) < 1] nor a main effect of Time [F(1, 17) = 1.90; P = 0.186]. There was a significant CS × Time interaction [F(1, 17) = 4.58; P = 0.047]. The interaction was mainly driven by a significant reduction in mean SCRs toward the CS+ from the early to the late phase [t(17) = 2.41; P = 0.015] pointing to a successful extinction of conditioning.

Fig. 2.

Fig. 2.

Open in a new tab

SCRs (µS log-transformed) for the early and late acquisition phase and the early and late extinction phase. During acquisition, there is a main effect of CS with higher reactions to the CS+. During extinction, there is a CS × Time interaction with higher reactions to the CS+ during the early phase than the late phase. All error bars indicate SEM. *< 0.05; **< 0.01.

Hemodynamic responses

Acquisition

The acquisition phase on the first day reliably activated the reward network implicated in the acquisition of appetitive conditioning, including NAcc, amygdala and OFC (Table 2). In addition, we computed a model separating CS + presentations, which were later reinforced, from those that were not. A comparison of reinforced and unreinforced CS+ trials showed no trends or significant differences (all P > 0.31). A separate analysis of the early and late phase of the acquisition can be found in the supplementary information.

Extinction

In the early extinction phase, activations were found in the left dorsal and ventral ACC as well as the right caudate nucleus and hippocampus (Table 3, Figure 3). In the late phase, bilateral activation of NAcc and amygdala reached significance (Figure 4). There were no significant differences in the contrast CS− − CS+ in the early or the late phase of extinction.

Fig. 3.

Fig. 3

Open in a new tab

Significant ROI activations during the early phase of extinction (CS + early − CS − early) on voxel level P < 0.05 (FWE-corrected). (a) caudate nucleus, (b) hippocampus, (c) dACC (d) and vACC. Displayed t-values are thresholded at t < 1.5. (Lower row) Mean β-weights of the respective CS+ and CS− activations. All error bars indicate SEM.

Fig. 4.

Fig. 4

Open in a new tab

Significant ROI activations during the late phase of extinction (CS + late − CS − late) on voxel level P < 0.05 (FWE-corrected). (a) Bilateral activation of the amygdala and (b) bilateral activation of the NAcc. Lines on the sagittal slices on the right side indicate the coronal slices depicted on the left. Displayed t-values are thresholded at t < 1.5. (Right) Mean β-weights of the respective CS+ and CS− activations. All error bars indicate SEM.

Fig. 5.

Fig. 5

Open in a new tab

Significant correlations of extracted β-weights in significant ROIs during early and late extinction (CS+ − CS−) with differences in subjective ratings to the CS+ (post-extinction − post-acquisition). Higher differences indicate extinction success. (a) Negative correlation of neural activations in the dACC with extinction of UCS-expectancy to the CS+. (b) Positive correlation of neural activations in the vACC with extinction of CS+ arousal. (c) Positive correlation of neural activations in the NAcc with extinction of CS+ valence.

Correlations of BOLD responses during extinction with subjective ratings

Extinction indices were calculated separately for UCS-expectancy, valence and arousal by subtracting the post-acquisition ratings for the CS+ from the respective post-extinction ratings. These differences were then correlated with the extracted β estimates at the peak voxel coordinates of the significant neural activations from the early and late extinction phases.

We found activation of the dorsal ACC to be correlated with UCS-expectancy to the CS+ (r = −0.64; P = 0.002): increased activations of the dACC during early extinction were linked to less reduction of UCS-expectancy from post-acquisition to post-extinction (Figure 5). In contrast, higher activations of the ventral ACC during early extinction were correlated with greater reductions of CS+ arousal ratings (r = 0.48; P = 0.029). Moreover, activation of the left NAcc was positively correlated with a reduction in CS+ valence ratings (r = 0.44; P = 0.048).

Following a worthwhile suggestion from an anonymous reviewer, we also investigated the association of impulsivity with the extinction-related BOLD responses. It was hypothesized that higher impulsivity scores might impair extinction-related neural activation. Correlational analyses of BIS-15 scores with the BOLD responses in the late extinction phase did indeed reveal reduced neural activations in the left amygdala (r = −0.37; P = 0.049), the right amygdala (r = −0.52; P = 0.008), the NAcc (r = −0.40; P = 0.038) and a trend in the right NAcc (r = −0.31; P = 0.083) (see supplementary for further analyses).

Discussion

This study aimed to investigate the neural correlates of appetitive extinction in humans. Using the MID paradigm, neural, subjective and psychophysiological responses were measured on the first (acquisition phase) and second day (extinction phase). On the first day, subjects successfully acquired CRs to the CS+, which were reflected in increased SCRs to the CS+, increased subjective ratings and in an increased involvement of the neural circuit for appetitive conditioning as compared with the CS−. On the second day, CRs were extinguished as subjects did no longer receive a reward following the CS+. This led to a decline in SCRs and in subjective ratings of the CS+. fMRI-results showed activations of the dACC and vACC, as well as the caudate nucleus and the hippocampus during early extinction, while the NAcc and the amygdala were involved during late extinction. Moreover, activation of the dACC was associated with less extinction of subjective ratings, while activations of the vACC and the NAcc were associated with greater extinction of subjective ratings. Because there are still only few fMRI studies on the acquisition of appetitive conditioning, we will briefly discuss these results before discussing appetitive extinction in detail.

Acquisition phase

The results for the acquisition of appetitive conditioning are in line with previous studies on this topic, which also found conditioned SCRs and subjective ratings (Delgado et al., 2008; Andreatta and Pauli, 2015). Neural activations mainly comprised the dorsal and ventral striatum, the amygdala, the midbrain, as well as the dorsal ACC and the OFC, which together form the reward circuitry and are considered to play key roles in appetitive conditioning (Haber and Knutson, 2010; Chase et al., 2015). The striatum is thought to encode incentive salience as well as changes in valence (O’Doherty et al., 2003; Berridge and Kringelbach, 2015). As this study used secondary reinforcement, i.e. money, this might have had an influence on the pattern of neural activations as well. It has previously been shown that different reinforcement types lead to the engagement of different but overlapping subregions in the reward network, especially the ventral striatum (Valentin et al., 2007). In case of the amygdala, its involvement in conditioning with primary reinforcement has consistently been shown, while its involvement in appetitive conditioning with secondary reinforcers is still under debate (Martin-Soelch et al., 2007). Some studies that used money as reinforcer did not find activation of the amygdala (Kirsch et al., 2003, Cox et al., 2005), while others did (Jiang et al., 2014; Kumar et al., 2014). The findings of the current study support the involvement of the amygdala in classical conditioning with secondary reinforcers. In general, the amygdala is thought to encode the association of CS and unconditioned reaction facilitating the conditioned reaction (Martin-Soelch et al., 2007).

The present paradigm could be regarded as a more ‘active conditioning design’ because subjects had to press a button in each trial. In contrast, previous studies also used completely ‘passive conditioning designs’, in which subjects received rewards (e.g. money, food and pleasant pictures) without any personal effort (Delgado et al., 2008; Andreatta and Pauli, 2015; Schweckendiek et al., 2016). This active learning design could also have impacted the CRs. It could be assumed that an active learning paradigm may lead to a stronger CS+/CS− differentiation. Nevertheless, it would be interesting to directly compare active and passive conditioning designs and compare CRs and other relevant features like contingency awareness.

Appetitive extinction

The main research aim was to investigate the extinction of appetitive conditioning and to explore possible mechanisms through its neural correlates. Subjective ratings of UCS-expectancy, valence and arousal showed successful extinction of appetitive conditioning. This is in line with a recent study by Andreatta and Pauli (2015), who were also able to successfully extinguish appetitive conditioning in subjective ratings of valence and arousal. However, other studies on appetitive extinction found resistance to appetitive extinction especially in subjective ratings of valence (Baeyens et al., 2005; Dwyer et al., 2007). In contrast to this study, these two studies used a classical conditioning paradigm, without requiring the subjects to perform a task to obtain the reward. This may have increased the importance in addition to the effects on subjective ratings, the observed appetitive extinction also extended to psychophysiological recordings of skin conductance. This supports the study by Andreatta and Pauli (2015), who also report successful appetitive extinction in SCRs and startle responses.

Early phase of appetitive extinction

During the early phase of extinction, the contrast CS+ − CS− revealed significant activations in the caudate, the hippocampus, as well as the dorsal and the ventral ACC. At this point, SCRs to the CS+ were still higher than in the late phase and the reported neural correlates might also be part of a recall of the consolidated appetitive conditioning (Milad et al., 2007). Although a few fear conditioning studies report increased activation to the CS− as compared with the CS+ (Hermann et al., 2012; Merz et al., 2014), no increased activation to the CS− was found in this study. As one of the main areas implicated in goal-directed behavior (O’Doherty et al., 2004; Burton et al., 2015), activation of the caudate/dorsomedial striatum during the early phase of extinction can be viewed as part of the recall of appetitive conditioning 24 h after acquisition. Previous experiments in rodents found that the inactivation of the dorsomedial striatum after appetitive conditioning mimicked the effect of appetitive extinction (Yin et al., 2005). Activation of the hippocampus has also been associated with the recall of (reward) associative learning (Wolosin et al., 2012; Hattori et al., 2015). Studies on fear extinction, however, discuss a role for the hippocampus in encoding contextual information in extinction learning (Abraham et al., 2014). Moreover, we found activations of the dorsal and ventral ACC. In a computational model based on a range of studies, the dorsal ACC has been proposed to encode predictions of appetitive and aversive outcomes based on previous experiences (Alexander and Brown, 2011). Increased activation of dorsal ACC in substance dependent subjects has been shown to reflect reduced sensitivity to omission of reward (Alexander et al., 2015). Activation of the ventral ACC on the other hand is assumed to play a role in early discriminative learning or the updating of previously learned associations and might therefore be a part of the extinction learning process (Gabriel et al., 2003; Schiller et al., 2008). This functional difference between the dorsal and the ventral ACC has already been discussed for fear extinction (Etkin et al., 2011). The authors review converging evidence showing correlations between dACC and expression of fear conditioning and vACC and inhibition of fear conditioning. This pattern is reflected in the presented correlations of activity in dACC and vACC with changes in subjective ratings and therefore seems to extend to appetitive extinction. Although higher activation of the dorsal ACC was associated with lasting UCS-expectancy following the CS+, higher activation of the ventral ACC was associated with reductions of arousal ratings of the CS+. Taken together with previous studies on fear conditioning, the present findings suggest that for both appetitive and aversive conditioning dACC encodes previously acquired conditioning while the vACC is part of early extinction processes.

Late phase of appetitive extinction

In the late phase, SCRs showed a decline in reactions to the CS+ indicating successful appetitive extinction. Neural correlates in the late phase can therefore be interpreted as part of the processing of extinction (Milad et al., 2007). On a neural level, the amygdala and the NAcc were significantly activated during the late phase of extinction (CS+ − CS−). Based on animal studies, Quirk and Mueller (2008) already ascribed the (basolateral) amygdala an important role in human appetitive extinction. Studies with monkeys and rats showed that after functional inactivation of the (basolateral) amygdala, animals continued to show CRs throughout extinction. It has been hypothesized that the amygdala encodes the changing CS-UCS association during extinction (Martin-Soelch et al., 2007). Tye et al. (2010) reported neuron populations in the basolateral amygdala that became active only once extinction learning had begun. Notably, the neurons encoding extinction did not react to omission of reward during a partial reinforcement schedule or to the absence of reward at the beginning of extinction. Similarly, this study also employed a partial reinforcement schedule and found an activation of the amygdala only in the late phase of extinction. Therefore, the activation of the amygdala does not seem to reflect an early reaction to each omission of a possible reward but rather the formation of (new) extinction memory. Moreover, one study on fear extinction that investigated the late phase of extinction separately also found activation of the amygdala in the late phase (Milad et al., 2007) while other studies report mixed results (for a systematic review see: Sehlmeyer et al., 2009).

In addition to the amygdala, we found increased activation of the NAcc, which is one of the main research foci in animal studies on appetitive extinction (Janak et al., 2004; Millan et al., 2010; for a review see: Millan et al., 2011). These studies show that the NAcc core mainly plays a role in acquisition of conditioning while the NAcc shell is involved in extinction of appetitive conditioning. Moreover, in human studies the NAcc has also been associated with the acquisition of new or changing contingencies (Klucken et al., 2009a; Sehlmeyer et al., 2009). Interestingly, activations of the NAcc were only found during the late phase of extinction but not in the early phase. Continued involvement of the NAcc in the MID paradigm, however, has not yet been explored after a 24-h period of consolidation. Animal studies already showed inactivation of the NAcc after the acquisition phase only impaired the reversal of contingencies and extinction of conditioning but not the expression of the consolidated CS-UCS association (Lorens, 1971). Thus, extinction of appetitive conditioning was no longer possible without the NAcc (Reading and Dunnett, 1991). The explored associations of neural activations and changes in subjective ratings also support the involvement of the NAcc in mediating extinction and valence: activation of NAcc was associated with reductions of valence ratings of the CS+ from post-acquisition to post-extinction. This supports the involvement of the NAcc in appetitive extinction and is further in line with reports on the NAcc mediating stimulus valence (Cooper and Knutson, 2008; Berridge and Kringelbach, 2015). In sum, the present results support a role of the NAcc in encoding the update of conditioned valence during acquisition and extinction.

Several authors assume that conditioning is an important model for the development and maintenance of addiction, while extinction processes mirror the treatment of addiction. A central element in the treatment of addiction is therefore the extinction of drug-related cues (Millan et al., 2011). We found increased activation of the NAcc during extinction. The NAcc has repeatedly been associated with changes in the relationship between CS and UCS (Schiller et al., 2008; Klucken et al., 2009b), which is an important element in exposure-based therapy. Another central element of exposure-based therapy is the inhibition of conditioned reactions, which is thought to be mediated by the amygdala (Quirk and Mueller, 2008).

Moreover, in line with current models of addiction, activation of the NAcc and the amygdala during the late phase of extinction was negatively correlated with the participants’ impulsivity. This supports assumptions that impulsivity is a risk factor for addiction (Potenza and Taylor, 2009; Gullo and Potenza, 2014; Potvin et al., 2015). Although previous studies showed associations of impulsivity with reward anticipation (Beck et al., 2009; Balodis et al., 2012), this study indicates that impulsivity is also associated with lower activations related to the extinction of previously learned reward associations. Because appetitive extinction serves as a model for the treatment of addiction, these findings indicate that low impulsivity might be a predictor for the success of exposure-based interventions or that these interventions might benefit from impulse control trainings.

A few limitations need to be addressed. First, the sample of this study consisted only of male subjects. Studies of the neural correlates of human fear conditioning showed effects of sex hormones on fear extinction (Merz et al., 2012). Therefore, further studies need to include female subjects to allow for a generalization of the present findings. Second, in the early extinction phase, effects of recall of appetitive conditioning and aspects of early extinction learning probably overlap. This study cannot entirely disentangle these two processes during the early extinction phase. Based on animal findings on appetitive extinction, regions involved in the expression of conditioning and extinction were expected to overlap. Although animal studies are able to disentangle these neuron populations in multicell recordings, the spatial resolution of fMRI does not allow this. Therefore, differences between acquisition, early extinction phase, and late extinction phase on a neural level could not be studied by direct comparison but its neural correlates had to be analyzed separately.

In conclusion, this study aimed to investigate extinction of appetitive conditioning and its neural correlates. Processes during the early phase of extinction can reflect the recall of the acquired appetitive conditioning and early processes of extinction learning. These differing processes were most pronounced in the ACC. Although the dACC was associated with stable expectance of rewards, the vACC was associated with extinction of subjective ratings. This extends previous aversive conditioning results to appetitive conditioning leading to the assumption that the ACC is involved in emotional learning regardless of the specific valence. The activation of amygdala and NAcc during the late phase of extinction supports their role in encoding the new extinction memory, which is in line with animal studies on appetitive extinction. Thus, our study provides a first insight into the neural correlates of appetitive extinction in humans.

Funding

This work was supported in part by a scholarship from the Justus Liebig University Giessen to Onno Kruse.

Supplementary data

Supplementary data are available at SCAN online.

Conflict of interest. None declared.

Supplementary Material

Supplementary Data
Click here for additional data file. (131.3KB, zip)

References

  1. Abraham A.D., Neve K.A., Lattal K.M. (2014). Dopamine and extinction: a convergence of theory with fear and reward circuitry. Neurobiology of Learning and Memory, 108, 65–77. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Agren T. (2014). Human reconsolidation: a reactivation and update. Brain Research Bulletin, 105, 70–82. [DOI] [PubMed] [Google Scholar]
  3. Agren T., Engman J., Frick A., et al. (2012). Disruption of reconsolidation erases a fear memory trace in the human amygdala. Science, 337, 1550–2. [DOI] [PubMed] [Google Scholar]
  4. Alexander W.H., Brown J.W. (2011). Medial prefrontal cortex as an action-outcome predictor. Nature Neuroscience, 14(10),1338–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Alexander W.H., Fukunaga R., Finn P., Brown J.W. (2015). Reward salience and risk aversion underlie differential ACC activity in substance dependence. NeuroImage. Clinical, 8, 59–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Andreatta M., Pauli P. (2015). Appetitive vs. aversive conditioning in humans. Frontiers in Behavioral Neuroscience, 9, 128.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Baeyens F., Díaz E., Ruiz G. (2005). Resistance to extinction of human evaluative conditioning using a between-subjects design. Cognition & Emotion, 19(2),245–68. [DOI] [PubMed] [Google Scholar]
  8. Balleine B. W., Killcross A. S., Dickinson A. (2003). The effect of lesions of the basolateral amygdala on instrumental conditioning. The Journal of neuroscience : the official journal of the Society for Neuroscience, 23,666–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Balodis I.M., Kober H., Worhunsky P.D., Stevens M.C., Pearlson G.D., Potenza M.N. (2012). Diminished frontostriatal activity during processing of monetary rewards and losses in pathological gambling. Biological Psychiatry, 71(8),749–57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Beck A., Schlagenhauf F., Wüstenberg T., et al. (2009). Ventral striatal activation during reward anticipation correlates with impulsivity in alcoholics. Biological Psychiatry, 66, 734–42. [DOI] [PubMed] [Google Scholar]
  11. Benedek M., Kaernbach C. (2010). A continuous measure of phasic electrodermal activity. Journal of Neuroscience Methods, 190(1),80–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Berridge K.C., Kringelbach M.L. (2015). Pleasure systems in the brain. Neuron, 86(3),646–64. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bouton M.E. (2002). Context, ambiguity, and unlearning: sources of relapse after behavioral extinction. Biological Psychiatry, 52(10),976–86. [DOI] [PubMed] [Google Scholar]
  14. Bradley M.M., Lang P.J. (1994). Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1),49–59. [DOI] [PubMed] [Google Scholar]
  15. Burton A.C., Nakamura K., Roesch M.R. (2015). From ventral-medial to dorsal-lateral striatum: neural correlates of reward-guided decision-making. Neurobiology of Learning and Memory, 117, 51–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Chase H.W., Kumar P., Eickhoff S.B., Dombrovski A.Y. (2015). Reinforcement learning models and their neural correlates: an activation likelihood estimation meta-analysis. Cognitive, Affective & Behavioral Neuroscience, 15(2),435–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Cooper J.C., Knutson B. (2008). Valence and salience contribute to nucleus accumbens activation. NeuroImage, 39(1),538–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Cox S.M.L., Andrade A., Johnsrude I.S. (2005). Learning to like: a role for human orbitofrontal cortex in conditioned reward. Journal of Neuroscience, 25, 2733–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Delgado M.R., Gillis M.M., Phelps E.A. (2008). Regulating the expectation of reward via cognitive strategies. Nature Neuroscience, 11(8),880–1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Dillon D.G., Holmes A.J., Jahn A.L., Bogdan R., Wald L.L., Pizzagalli D.A. (2008). Dissociation of neural regions associated with anticipatory versus consummatory phases of incentive processing. Psychophysiology, 45(1),36–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Dwyer D.M., Jarratt F., Dick K. (2007). Evaluative conditioning with foods as CSs and body shapes as USs: no evidence for sex differences, extinction, or overshadowing. Cognition & Emotion, 21(2),281–99. [Google Scholar]
  22. Etkin A., Egner T., Kalisch R. (2011). Emotional processing in anterior cingulate and medial prefrontal cortex. Trends in Cognitive Sciences, 15(2),85–93. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Everitt B.J. (2014). Neural and psychological mechanisms underlying compulsive drug seeking habits and drug memories—indications for novel treatments of addiction. The European Journal of Neuroscience, 40(1),2163–82. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gabriel M., Burhans L., Kashef A. (2003). Consideration of a unified model of amygdalar associative functions. Annals of the New York Academy of Sciences, 985, 206–17. [DOI] [PubMed] [Google Scholar]
  25. Gullo M.J., Potenza M.N. (2014). Impulsivity: mechanisms, moderators and implications for addictive behaviors. Addictive Behaviors, 39, 1543–6. [DOI] [PubMed] [Google Scholar]
  26. Haber S.N., Knutson B. (2010). The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology, 35(1),4–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hahn T., Dresler T., Ehlis A.C., et al. (2009). Neural response to reward anticipation is modulated by Gray’s impulsivity. NeuroImage, 46(4),1148–53. [DOI] [PubMed] [Google Scholar]
  28. Hattori S., Chen L., Weiss C., Disterhoft J.F. (2015). Robust hippocampal responsivity during retrieval of consolidated associative memory. Hippocampus, 25(5),655–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hermann A., Kupper Y., Schmitz A., et al. (2012). Functional gene polymorphisms in the serotonin system and traumatic life events modulate the neural basis of fear acquisition and extinction. PLoS One, 7, e44352. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hermans E.J., Bos P.A., Ossewaarde L., Ramsey N.F., Fernández G., van Honk J. (2010). Effects of exogenous testosterone on the ventral striatal BOLD response during reward anticipation in healthy women. NeuroImage, 52(1),277–83. [DOI] [PubMed] [Google Scholar]
  31. Het S., Rohleder N., Schoofs D., Kirschbaum C., Wolf O.T. (2009). Neuroendocrine and psychometric evaluation of a placebo version of the ‘trier social stress test’. Psychoneuroendocrinology, 34(7),1075–86. [DOI] [PubMed] [Google Scholar]
  32. Janak P.H., Chen M.T., Caulder T. (2004). Dynamics of neural coding in the accumbens during extinction and reinstatement of rewarded behavior. Behavioural Brain Research, 154(1),125–35. [DOI] [PubMed] [Google Scholar]
  33. Jiang Y., Kim S.-I., Bong M. (2014). Effects of reward contingencies on brain activation during feedback processing. Frontiers in human neuroscience, 8,656, doi:10.3389/fnhum.2014.00656. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kirsch P., Schienle A., Stark R., et al. (2003). Anticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system. NeuroImage, 20(2),1086–95. [DOI] [PubMed] [Google Scholar]
  35. Klucken T., Alexander N., Schweckendiek J., et al. (2013a). Individual differences in neural correlates of fear conditioning as a function of 5-HTTLPR and stressful life events. Social Cognitive and Affective Neuroscience, 8(3),318–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Klucken T., Kagerer S., Schweckendiek J., Tabbert K., Vaitl D., Stark R. (2009a). Neural, electrodermal and behavioral response patterns in contingency aware and unaware subjects during a picture–picture conditioning paradigm. Neuroscience, 158(2),721–31. [DOI] [PubMed] [Google Scholar]
  37. Klucken T., Kruse O., Wehrum-Osinsky S., Hennig J., Schweckendiek J., Stark R. (2015). Impact of COMT Val158Met-polymorphism on appetitive conditioning and amygdala/prefrontal effective connectivity. Human Brain Mapping, 36(3),1093–101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Klucken T., Tabbert K., Schweckendiek J., et al. (2009b). Contingency learning in human fear conditioning involves the ventral striatum. Human Brain Mapping, 30, 3636–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Klucken T., Wehrum S., Schweckendiek J., et al. (2013b). The 5-HTTLPR polymorphism is associated with altered hemodynamic responses during appetitive conditioning. Human Brain Mapping, 34(10),2549–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Knutson, B., Westdorp, A., Kaiser, E., Hommer, D. (2000). FMRI visualization of brain activity during a monetary incentive delay task. NeuroImage, 12,20–7. doi:10.1006/nimg.2000.0593. [DOI] [PubMed] [Google Scholar]
  41. Krohne, H. W., Egloff, B., Kohlmann, C.-W., Tausch, A. (1996). Untersuchungen mit einer deutschen Version der “Positive and negative Affect Schedule” (PANAS). Diagnostica, 42,139–56. [Google Scholar]
  42. Kumar P., Berghorst L.H., Nickerson L.D., et al. (2014). Differential effects of acute stress on anticipatory and consummatory phases of reward processing. Neuroscience, 266, 1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. LaBar K.S., Gatenby J.C., Gore J.C., LeDoux J.E., Phelps E.A. (1998). Human amygdala activation during conditioned fear acquisition and extinction: a mixed-trial fMRI study. Neuron, 20(5),937–45. [DOI] [PubMed] [Google Scholar]
  44. LaBar K.S., LeDoux J.E., Spencer D.D., Phelps E.A. (1995). Impaired fear conditioning following unilateral temporal lobectomy in humans. The Journal of Neuroscience, 15(10),6846–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Lorens S.A. (1971). Operant responding for food following lesions in the nuclei accumbens of the rat. Physiology & Behavior, 7(3),449–50. [DOI] [PubMed] [Google Scholar]
  46. Martin-Soelch C., Linthicum J., Ernst M. (2007). Appetitive conditioning: neural bases and implications for psychopathology. Neuroscience & Biobehavioral Reviews, 31(3),426–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Merz C.J., Hermann A., Stark R., Wolf O.T. (2014). Cortisol modifies extinction learning of recently acquired fear in men. Social Cognitive and Affective Neuroscience, 9, 1426–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Merz C.J., Tabbert K., Schweckendiek J., et al. (2012). Neuronal correlates of extinction learning are modulated by sex hormones. Social Cognitive and Affective Neuroscience, 7(7),819–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Meule A., Vögele C., Kübler A. (2011). Psychometrische Evaluation der deutschen Barratt Impulsiveness Scale – Kurzversion (BIS-15). Diagnostica 57, 126–33. [Google Scholar]
  50. Milad M.R., Wright C.I., Orr S.P., Pitman R.K., Quirk G.J., Rauch S.L. (2007). Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert. Biological Psychiatry, 62(5),446–54. [DOI] [PubMed] [Google Scholar]
  51. Millan E.Z., Furlong T.M., McNally G.P. (2010). Accumbens shell-hypothalamus interactions mediate extinction of alcohol seeking. Journal of Neuroscience, 30(13),4626–35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Millan E.Z., Marchant N.J., McNally G.P. (2011). Extinction of drug seeking. Behavioural Brain Research, 217(2),454–62. [DOI] [PubMed] [Google Scholar]
  53. Moorman D.E., Aston-Jones G. (2014). Orbitofrontal cortical neurons encode expectation-driven initiation of reward-seeking. The Journal of Neuroscience, 34(31),10234–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. O’Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304(5669),452–4. [DOI] [PubMed] [Google Scholar]
  55. O’Doherty J.P., Dayan P., Friston K., Critchley H., Dolan R.J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38, 329–37. [DOI] [PubMed] [Google Scholar]
  56. Phelps E.A., Delgado M.R., Nearing K.I., LeDoux J.E. (2004). Extinction learning in humans: role of the amygdala and vmPFC. Neuron, 43(6),897–905. [DOI] [PubMed] [Google Scholar]
  57. Portero-Tresserra M., Martí-Nicolovius M., Guillazo-Blanch G., Boadas-Vaello P., Vale-Martínez A. (2013). D-cycloserine in the basolateral amygdala prevents extinction and enhances reconsolidation of odor-reward associative learning in rats. Neurobiology of Learning and Memory, 100, 1–11. [DOI] [PubMed] [Google Scholar]
  58. Potenza M.N., Taylor J.R. (2009). Found in translation: understanding impulsivity and related constructs through integrative preclinical and clinical research. Biological Psychiatry, 66, 714–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Potvin S., Tikasz A., Dinh-Williams L.L.A., Bourque J., Mendrek A. (2015). Cigarette cravings, impulsivity, and the brain. Frontiers in Psychiatry, 6, 125.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Quirk G.J., Mueller D. (2008). Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology, 33(1),56–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Reading P.J., Dunnett S.B. (1991). The effects of excitotoxic lesions of the nucleus accumbens on a matching to position task. Behavioural Brain Research, 46(1),17–29. [DOI] [PubMed] [Google Scholar]
  62. Schiller D., Kanen J.W., LeDoux J.E., Monfils M.H., Phelps E.A. (2013). Extinction during reconsolidation of threat memory diminishes prefrontal cortex involvement. Proceedings of the National Academy of Sciences of the United States of America, 110, 20040–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Schiller D., Levy I., Niv Y., LeDoux J.E., Phelps E.A. (2008). From fear to safety and back: reversal of fear in the human brain. The Journal of Neuroscience, 28, 11517–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Schiller D., Monfils M.H., Raio C.M., Johnson D.C., LeDoux J.E., Phelps E.A. (2010). Preventing the return of fear in humans using reconsolidation update mechanisms. Nature, 463, 49–53. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Schweckendiek J., Klucken T., Merz C.J., et al. (2013). Learning to like disgust: neuronal correlates of counterconditioning. Frontiers in Human Neuroscience, 7, PubMed ID 23847514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Schweckendiek J., Stark R., Klucken T. (2016). Neuroticism and extraversion moderate neural responses and effective connectivity during appetitive conditioning. Human Brain Mapping, 37,2992–3002. doi:10.1002/hbm.23221. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Sehlmeyer C., Schöning S., Zwitserlood P., et al. (2009). Human fear conditioning and extinction in neuroimaging: a systematic review. PLoS One, 4(6),e5865.. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Stalnaker T.A., Cooch N.K., Schoenbaum G. (2015). What the orbitofrontal cortex does not do. Nature Neuroscience, 18(5),620–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Taylor J.R., Olausson P., Quinn J.J., Torregrossa M.M. (2009). Targeting extinction and reconsolidation mechanisms to combat the impact of drug cues on addiction. Neuropharmacology, 56, 186–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Tobler P.N., Fletcher P.C., Bullmore E.T., Schultz W. (2007). Learning-related human brain activations reflecting individual finances. Neuron, 54, 167–75. [DOI] [PubMed] [Google Scholar]
  71. Tye K.M., Cone J.J., Schairer W.W., Janak P.H. (2010). Amygdala neural encoding of the absence of reward during extinction. Journal of Neuroscience, 30(1),116–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Valentin V.V., Dickinson A., O’Doherty J.P. (2007). Determining the neural substrates of goal-directed learning in the human brain. The Journal of Neuroscience, 27, 4019–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Walter B., Blecker C., Kirsch P., et al. (2003). MARINA: an easy to use tool for the creation of MAsks for Region of INterest Analyses [abstract]. In: Presented at the 9th International Conference on Functional Mapping of the Human Brain. Available on CD-Rom in NeuroImage, Vol. 2, No. 19.
  74. Wolosin S.M., Zeithamova D., Preston A.R. (2012). Reward modulation of hippocampal subfield activation during successful associative encoding and retrieval. Journal of Cognitive Neuroscience, 24(7),1532–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  75. Yin H.H., Ostlund S.B., Knowlton B.J., Balleine B.W. (2005). The role of the dorsomedial striatum in instrumental conditioning. The European Journal of Neuroscience, 22(2), 513–23. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Data
Click here for additional data file. (131.3KB, zip)

Articles from Social Cognitive and Affective Neuroscience are provided here courtesy of Oxford University Press

RESOURCES