Role of the mesolimbic dopamine system in relief learning

Abstract

The relief from an aversive event is rewarding. Since organisms are able to learn which environmental cues can cease an aversive event, relief learning helps to better cope with future aversive events. Literature data suggest that relief learning is affected in various psychopathological conditions, such as anxiety disorders. Here, we investigated the role of the mesolimbic dopamine system in relief learning. Using a relief learning procedure in Sprague Dawley rats, we applied a combination of behavioral experiments with anatomical tracing, c-Fos immunohistochemistry, and local chemogenetic and pharmacological interventions to broadly characterize the role of the mesolimbic dopamine system. The present study shows that a specific part of the mesolimbic dopamine system, the projection from the posterior medial ventral tegmental area (pmVTA) to the nucleus accumbens shell (AcbSh), is activated by aversive electric stimuli. 6-OHDA lesions of the pmVTA blocked relief learning but fear learning and safety learning were not affected. Chemogenetic silencing of the pmVTA-AcbSh projection using the DREADD approach, as well as intra-AcbSh injections of the dopamine D2/3 receptor antagonist raclopride inhibited relief learning. Taken together, the present data demonstrate that the dopaminergic pmVTA-AcbSh projection is critical for relief learning but not for similar learning phenomena. This novel finding may have clinical implications since the processing of signals predicting relief and safety is often impaired in patients suffering from anxiety disorders. Furthermore, it may help to better understand psychological conditions like non-suicidal self-injury, which are associated with pain offset relief.

Introduction

The motivational opponent-process theory suggests that an emotionally relevant event induces two opponent emotions [1, 2]. In the case of an aversive event, initially the fear emotion is triggered. However, after the offset of the aversive event, fear disappears and is followed by its opponent emotion, namely relief.

Both of these emotions, fear and relief, can be associated with environmental cues [3, 4]. These learning phenomena are called fear learning [5, 6] and relief learning [7, 8], respectively. Thereby, humans and animals can learn what made aversive events happen and cease, which help them to better cope with future aversive events [3]. Another type of learning is safety learning, which also can be observed during the experience of an aversive event. Safety learning is the association of a reoccurring environmental cue with the absence of the aversive event [9, 10]. In typical safety learning protocols, the cue and the aversive event are presented in an explicitly unpaired manner. Furthermore, in contrast to relief learning, the inter-stimulus interval is often pseudo-randomized (see also below). Nonetheless, some authors consider relief learning as a specific form of safety learning [11].

There is a debate in literature whether these three types of learning contribute to the development of anxiety disorders by causing or supporting some of the symptoms [12–15]. Because increasing evidence suggests that safety learning is impaired in anxiety disorders [11, 16, 17], the behavioral and neural characterization of safety and relief learning gained more and more interest in basic and applied neurosciences during the last decade [9, 10, 18–20].

Regarding relief learning, there is already a good deal of information available [3]: Human imaging studies found the nucleus accumbens to be activated during both acute relief [21, 22] and learned relief [23, 24]. Studies in laboratory rodents confirmed the crucial role of the nucleus accumbens in relief learning [25, 26] and showed that relief learning is mediated and modulated by glutamatergic and dopaminergic transmission within the nucleus accumbens [26–29].

Relief from an aversive event is rewarding [22, 24, 30] and the mesolimbic dopamine system is an established mediator of natural reward [31]. In rats, Brischoux et al. [32] identified dopaminergic neurons in the posterior medial region of the ventral tegmental area (pmVTA) that are activated by the offset of an aversive event (an electric footshock), i.e. in the moment of relief. Based on these findings, we hypothesized that the dopaminergic projection from the pmVTA to the nucleus accumbens shell (AcbSh) mediates relief learning.

The aim of the present study was to investigate this hypothesis. Therefore, we used a combination of anatomical tracing, c-Fos immunohistochemistry, neurotoxic lesions, and chemogenetic and pharmacological approaches. We report that dopaminergic pmVTA neurons projecting to the AcbSh were activated by foot shocks and that lesions of these neurons prevented relief learning but not fear or safety learning. Furthermore, chemogenetic inactivation of the pmVTA-AcbSh projection as well as local blockade of dopamine D2/3 receptors within the AcbSh inhibited the acquisition of conditioned relief. Notably, we previously published an important role of accumbal dopamine D1 receptors in the acquisition and expression of conditioned relief [28]. Taken together, these findings indicate that the dopaminergic pmVTA-AcbSh projection and accumbal dopamine receptors are crucially involved in relief learning.

Materials and methods

Animals

One hundred and thirty-two male Sprague Dawley rats aged between 2 and 3 months were used. They were bred in our animal facility (original breeding stock; Taconic Biosciences, Denmark), group-housed under standard conditions, and all experiments were performed during the light phase. Our study was carried out in accordance with international guidelines for the use of animals in experiments (2010/63/EU) and was approved by the local ethical committee (Landesverwaltungsamt Sachsen-Anhalt, Az. 42502-2-1309 UniMD).

Surgeries

For surgeries, the rats were anesthetized with 2–3% isoflurane (mixed with pure oxygen). For the different injections and implantations, a stereotaxic apparatus and coordinates from a rat brain atlas were used [33]. Post-surgical analgesia was provided by carprofen (Rimadyl, Zoetis, Berlin, Germany; 5 mg/kg, s.c.).

Tracer injections (experiment 1)

The anterograde tracer Phaseolus vulgaris-leucoagglutinin (PHA-L; Vector Laboratories, Burlingame, CA, USA; 2.5% in NaPBS;) was iontophoretically (5 µA, 10 min, on–off: 7 s) injected into the pmVTA (6.3 mm caudal, 0.1 mm lateral, and 8.5 mm ventral from bregma). In another set of rats, the retrograde tracer fluorogold (FG; Fluorochrome, Denver; 4% in saline; 4 µA, 7 min, on–off: 7 s;) was injected into the AcbSh (1.6 mm rostral, 2.0 mm lateral, and 8.2 mm ventral from bregma).

Dopamine-specific lesions (experiment 2)

Rats were pretreated with desipramine (Sigma-Aldrich, Taufkirchen, Germany; 25 mg/kg, i.p.) for the protection of noradrenergic neurons [34]. Using a 2 µl microsyringe, vehicle or the neurotoxin 6-hydroxydopamine (6-OHDA; Sigma-Aldrich, Taufkirchen, Germany; 8 µg/µl, dissolved in saline with 0.02% ascorbic acid) was injected into the pmVTA (see above for coordinates) with a volume of 1 or 2 µl and a rate of 0.2 µl/min.

Injections of the viral DREAAD construct (experiment 3)

Rats received injections of 1 µl pAAV8-CaMKIIa-hM4D(Gi)-mCherry (titer: 3.3 × 10¹² vg/ml, dissolved in 350 mM NaCl + 5% d-sorbitol in phosphate-buffered saline (PBS); Virus Vector Core, University of North Carolina, Chapel Hill, NC, USA) or pAAV2-CaMKIIa-mCherry, i.e. a construct without DREADD, into the pmVTA. Then, guide cannulas aiming at the AcbSh were implanted (see the next section).

Implantations of guide cannulas (experiments 3 and 4)

To perform local injections in awake rats, guide cannulas (8 mm length, 0.7 mm diameter) aiming at the AcbSh (1.6 mm rostral, ± 2.0 mm lateral, and 8.2 mm ventral from bregma) were bilaterally implanted.

Startle apparatus and stimulus parameter

To measure the startle response (experiments 2, 3, and 4) and rats’ reactivity to electric stimuli (experiment 4), a startle system with piezoelectric motion sensors was used (SR-LAB; San Diego Instruments, CA, USA).

For the different conditioning procedures, aversive electric stimuli (unconditioned stimulus, US) and either light or tone stimuli (conditioned stimulus, CS; duration: 5 s) were used. The light CS was produced by a 10 W bulb (~1000 lux), the tone CS by a loudspeaker (intensity: 75 dB SPL). The US (footshock) was administered via a floor grid (intensity: 0.4 mA, duration: 0.5 s). A second loudspeaker was used to present a continuous background noise (55 dB SPL) and the acoustic startle stimuli (noise bursts, 96 dB SPL, 40 ms duration). For testing the reactivity to electric stimuli, stimulus intensities of 0.0, 0.1, 0.2, 0.3, and 0.4 mA were used.

Behavioral procedures

Experiment 1

PHA-L-injected rats were perfused 1 week after the injections without any behavioral tests. FG-injected rats were put into the startle system 2 weeks after the surgery. After 2 min habituation, four of them received 15 foot shocks with an inter-trial interval (ITI) of 2 min, five of them did not receive electric stimuli. One hour later they were perfused.

Experiment 2

Ten days after injections of vehicle or 6-OHDA, rats were subjected to a relief-conditioning procedure as previously described [28]. After 5 min acclimation time, 15 pairings of an electric stimulus with the light stimulus were presented (mean ITI: 2 min, range: 1.5–2.5 min). The stimuli were backward paired, i.e. the light CS was presented 3 s after the US onset.

On the following day, the retention test on conditioned relief was performed. After 5 min acclimation, 10 startle stimuli were presented with an ITI of 30 s to habituate the rats. Then, 20 further startle stimuli were presented, 10 of them without the light CS (startle alone trials) and 10 of them upon presentation of the light CS (CS-startle trials). The order of the trials with and without light CS was pseudo-randomized.

After a break of at least 3 days, the same rats were safety-conditioned as previously described [25]. After 5 min acclimation, the CS (5 kHz tone) as well as the US was presented 15 times. For this procedure, in order to explicitly unpair CS with US, the US was never presented less than 12 s before or after the tone CS, otherwise presented with a random inter-stimulus interval.

On the following day, the retention test on conditioned safety was performed. The protocol was identical to the retention test on conditioned relief which is described above.

After a further break of at least 3 days, the same rats were fear-conditioned as previously described [35]. After 5 min acclimation, 15 forward pairings of the CS (now a 10 kHz tone) and the US were presented. The electric stimuli were presented 4.5 s after the onset of the CS and both stimuli ended simultaneously.

On the following day, the retention test on conditioned fear was performed. The protocol was identical to the retention test on conditioned relief which is described above. After this test, the animals were perfused.

Experiment 3

Two weeks after the injections of the DREAAD construct, the rats received bilateral injections of either saline or 1 mM clozapine-N-oxide (CNO; Sigma-Aldrich, Taufkirchen, Germany; dose according to mahler et al. [36]) into the AcbSh. A volume of 0.5 µl was injected with a speed of 0.2 µl/min.

The procedure for relief conditioning and also for the retention test on the following day (without injections) was identical to the one described above (experiment 2). After the retention test, the rats were perfused.

Experiment 4

One week after guide cannula implantation, the rats were relief-conditioned and tested on conditioned relief as described above. However, one group of rats were injected before the conditioning and tested without treatment (pre-training injections), whereas rats in the other group were conditioned without treatment and received injections before testing (pre-testing injections). Injections of 0, 0.5, or 1.0 µg raclopride (dissolved in 0.3 µl saline; Sigma-Aldrich, Taufkirchen, Germany; doses according to Lex and Hauber [37]) were performed as described above. A between-subject design was used for pre-training injections (saline; 0.5 or 1.0 µg raclopride). For the pre-testing injections, a within-subject design was used, i.e. each rat was conditioned and tested three times (with a break of 1–2 days in between) and received each treatment in a pseudo-randomized order.

Eighteen randomly chosen rats of experiment 4 were used to measure potential effects of raclopride injections on the locomotor response to the foot shocks. They were injected with vehicle or 1 µg raclopride, put into the test chamber, and exposed to different shock intensities.

Immunohistochemistry and histology

After the end of the behavioral experiments, rats of experiments 1, 2, and 3 were anesthetized with 350 mg/kg chloral hydrate and transcardially perfused with PBS followed by 4.0 % paraformaldehyde and 0.2% 0.1 M picric acid. Rats from experiment 4 were euthanized by CO₂ asphyxia. The rats’ brains were removed, post-fixed, and coronal sections were cut (40 µm). Sections of experiments 1 and 2 were immunostained with the following primary antibodies: rabbit anti-PHA-L (1:1000; Vector Laboratories Cat# AS-2300, RRID:AB_2313686), mouse anti-tyrosine hydroxylase (1:5000; Millipore Cat# MAB318, RRID:AB_2201528), and rabbit anti-c-Fos (1:500; Santa Cruz Biotechnology Cat# sc-253, RRID:AB_2231996). Staining was amplified as described previously [38]. For the analysis of FG and mCherry expression, the native fluorescence of these two chromophores was used. Brain sections (AcbSh region) of experiments 3 and 4 were Nissl-stained (1% cresyl violet). Brain sections of experiment 3 (pmVTA region) were not stained. Last, all sections were mounted on glass slides, dehydrated, and coverslipped with DPX.

Slices were examined under a microscope. Photomicrographs of the regions of interest were taken and analyzed with ImageJ software (double- and triple-stained neurons; experiment 1). Injections sites (experiments 1, 3, and 4) and 6-OHDA-induced lesions (experiment 2) were analyzed under a light microscope. Only rats with confirmed injection or lesion sites in the intended target areas were included in further analysis. In sum, 23 rats had to be excluded because of missing startle responses or misplaced injection sites.

Descriptive and statistical analyses

For statistical analysis, Prism 6.0 (GraphPad Software Inc., La Jolla, CA, USA) was used. The mean startle magnitudes of the startle trials in the absence (startle alone) and in the presence of the conditioned light stimulus (CS-startle), as well as the difference between these means were calculated for each animal. Normal distribution of the data was checked with the D’Agostino-Pearson omnibus normality test. ANOVAs with the factors (within-subject if appropriate) startle trial type, shock reactivity or/and treatment were calculated. C-Fos data were analyzed with Student’s t-tests. The significance level was set at p < 0.05 for all statistical tests. The results are shown as mean ± SEM.

Results

Foot shocks activate dopaminergic pmVTA-AcbSh projection neurons

The anterograde tracer PHA-L was iontophoretically applied into the pmVTA (for injection sites see Supplementary Figure 1), whose neurons are partly activated by the offset of foot shocks [32], to confirm that this brain site projects to the nucleus accumbens, a crucial brain area for relief learning (e.g. [25]). As shown in Fig. 1a, PHA-L-immuno-positive cells were found in the dorsomedial part of the AcbSh but not in other subregions of the nucleus accumbens.

Fig. 1.

The pmVTA projection to the AcbSh was activated by exposure to foot shocks. a Triangles indicate terminals in AcbSh arising from the pmVTA, as labeled by an iontophoretic injection with the anterograde tracer PHA-L into the pmVTA (inlay; see also Supplementary Figure 1). Scale bar: 1 mm. b FG was injected into the AcbSh (inlay; see also Supplementary Figure 2). Then one group of rats were exposed to foot shocks (n = 4) whereas another group received no shocks (n = 5). The photomicrograph shows a representative section of the pmVTA with TH-, FG-, and c-Fos-positive neurons. White triangles indicate triple-labeled neurons. The bar diagram depicts the percentage of pmVTA-AcbSh projection neurons (i.e. fluorogold-positive) which is also c-Fos immunoreactive (ir) or c-Fos and TH-ir. Scale bar: 100 µm. **p < 0.01, Student's t-test, comparison with the no-shocked group. c In experiment 2, saline (n = 9) or 6-OHDA (n = 23) was injected into the pmVTA (inlay). The photomicrographs are from representative TH-immunostained coronal sections. 6-OHDA injections strongly reduced TH-immunoreactivity within the pmVTA (right panel) and the AcbSh (left panel). Based on histological analyses, rats were divided into groups with misplaced lesions (n = 7) and lesions restricted to the pmVTA (here indicated by triangels; n = 16). For behavioral data, see Fig. 2

Next, we exposed rats (n = 4; for injection sites see Supplementary Figure 2), which had received FG injections into the AcbSh 1 week before, to foot shocks. Another group of rats with FG injections (n = 5) received no foot shocks and served as a control group. After the exposure, the rats were euthanized and the brain (pmVTA) slices immunostained for c-Fos, a marker of neural activity [39], and TH, a marker of dopaminergic neurons in the ventral midbrain [40]. Our analyses focused on the percentage of pmVTA-AcbSh projection neurons (i.e., FG-labeled) that were activated by foot shocks (i.e. c-Fos-immunoreactive). As depicted in Fig. 1b, only a slight amount (0.8 ± 1.0 %) of pmVTA-AcbSh projection neurons were c-Fos-immunoreactive under control conditions (no exposure to foot shocks). Exposure to foot shocks significantly increased the percentage of c-Fos-immunoreactive projection neurons to 12.8 ± 4.3% (Student’s t-test: t₍₇₎ = 5.40, p = 0.001). Nearly all of these neurons were also TH-immunoreactive.

Dopaminergic pmVTA lesions block relief learning

Our first experiment showed that dopaminergic neurons projecting from the pmVTA to the AcbSh are activated by foot shocks. To see whether these dopaminergic neurons are involved in relief learning, we performed 6-OHDA lesions of the pmVTA and submitted these rats to the relief conditioning procedure.

Histological analyses of the pmVTA revealed integrity of the pmVTA after saline injections (sham lesions, n = 9, Fig. 1c and Supplementary Figure 3). Injections of both volumes of 6-OHDA successfully induced lesions of the pmVTA with very similar extensions, and the data were therefore pooled. In 16 rats, these lesions included only the pmVTA, i.e. the interfascicular, the paranigral, and the parainterfascicular nuclei of the VTA. Notably, these lesions also lead to a down-regulation of TH-immunoreactivity in the nucleus accumbens (Fig. 1c, left panels). Seven rats had misplaced lesions; not the pmVTA, but more dorsal sites of this region were lesioned (e.g., red nucleus, parabrachial pigmented nucleus). These rats built the group “misplaced”.

The rats were first subjected to a relief learning protocol, then to a safety learning protocol, and last to a fear learning protocol. An overall ANOVA with protocol, lesion group, and startle trial type as factors showed that the lesions had no general effects on startle magnitude (F_(2,87) = 0.99, p = 0.37) and that—as expected—the different protocols leads to different CS effects on the startle response (F_(2,87) = 23.55, p < 0.0001). Notably, this interaction was differently affected by the lesions (lesion group × protocol × trial type: F_(4,87) = 2.54, p = 0.046). To evaluate the particular effects of pmVTA lesions on the different types of learning, separate ANOVAs were performed in the following.

Figure 2a depicts the data from the retention test on conditioned relief, measured by modulation of the startle response by the relief CS (for individual startle data see Supplementary Figure 4). The mean startle magnitudes (±SEM) in startle alone and CS-startle trials, as well as their difference are shown. An ANOVA using trial type as within-subject and group as between-subject factors revealed a significant main effect of trial type (F_(1,29) = 15.30, p = 0.0005) indicating startle attenuation by the relief CS, i.e. successful relief conditioning. Notably, the lesions did not generally affect the startle magnitude (main lesion effect: F_(2,29) = 0.38, p = 0.69). However, there was a significant interaction between trial type and group (F_(2,29) = 6.74, p = 0.004), indicating that the pmVTA lesions affected relief conditioning. This was confirmed by post hoc Sidak’s tests demonstrating a significant startle attenuation by the relief CS in the sham and misplaced groups (t₍₉₎ = 3.20, p = 0.01 and t₍₇₎ = 3.36, p = 0.007, respectively) but not in the pmVTA-lesioned group (t₍₁₆₎ = 0.54, p = 0.93). Furthermore, a separate analysis of the difference scores showed a significant difference between sham-lesioned and pmVTA-lesioned rats (t₍₂₃₎ = 2.90, p = 0.01; Dunnett’s post hoc comparison after ANOVA, F_(2,29) = 6.78, p = 0.004).

Fig. 2.

6-OHDA lesions of the pmVTA blocked relief but not safety or fear learning. Rats were pmVTA-lesioned (see Fig. 1c and Supplementary Figure 3) and then submitted to relief, safety, and fear conditioning. In the retention tests, the startle paradigm was used to measure CS effects. The bar diagrams depict the mean startle magnitudes (±SEM) in startle alone and CS-startle trials, as well as their difference. a Relief learning was significantly blocked by pmVTA lesions (ANOVA: p = 0.004). ^#p = 0.013, post hoc Dunnett’s comparison with difference scores of the sham group. b In contrast, lesions did not affect safety conditioning (ANOVA: p = 0.80) or c fear conditioning (ANOVA: p = 0.86). **p < 0.01, *p < 0.05, post hoc Sydak’s multiple comparisons startle alone vs. CS-startle

This pmVTA lesion effect was specific to relief learning since neither safety learning (Fig. 2b) nor fear learning (Fig. 2c) was affected by the lesions. ANOVAs revealed successful conditioning in these tests (main trial type effects: F_(1,29) = 12.12, p = 0.001 and F_(1,29) = 17.25, p = 0.0004, respectively) but no interactions between pmVTA lesion and CS effects on startle (F_(2,29) = 0.23, p = 0.80 and F_(2,29) = 0.15, p = 0.86, respectively). In addition, the startle response was not generally affected by the lesions in these tests (main lesion effect: F_(2,29) = 1.58, p = 0.22 and F_(2,29) = 1.04, p = 0.37, respectively).

Chemogenetic inactivation of the pmVTA-AcbSh projection blocks relief learning

Our next experiment had the aim to demonstrate that the blockade of relief learning, observed in experiment 2, is mediated by the projection from the pmVTA to the AcbSh. To specifically inactivate the pmVTA-AcbSh projection, we made use of chemogenetics [41], i.e. we injected a DREADD vector (pAAV8-CaMKIIa-hM4D(Gi)-mCherry) into the pmVTA (Fig. 3, inlay). Furthermore, to be able to later specifically inactivate the projection from the pmVTA to the AcbSh, we implanted intracranial cannulas targeting the AcbSh (for injection sites see Fig. 3a). Histological analysis revealed robust mCherry expression of the pmVTA neurons as well as of terminals within the AcbSh (Supplementary Figure 5).

Fig. 3.

Chemogenetic inactivation of the projection from the pmVTA to the AcbSh blocked relief learning. a The inlay indicates the injection sites of the viral DREADD construct (AAV8-CaMKIIa-hM4D(Gi)-mCherry) into the pmVTA and of vehicle (n = 10) and CNO (n = 14) into the AcbSh (see also Supplementary Figure 4). b CNO injections, i.e. inactivations of the pmVTA-AcbSh projection, blocked startle attenuation by the relief CS in the rats in which DREADD expression was histologically confirmed, but not in rats which received pmVTA injections of an inactive viral construct (i.e. without DREADD; CNO/control; n = 9). ^#p < 0.05 post hoc Dunnett’s comparison with vehicle difference score; *p < 0.05, post hoc Sydak’s multiple comparisons startle alone vs. CS-startle

Immediately before relief conditioning, the rats were either injected with saline (n = 10) or with CNO (n = 14) into the AcbSh. CNO is a synthetic ligand that activates the inhibitory hM4D receptors and thereby specifically inactivates the projection from the pmVTA to the AcbSh in DREADD vector-injected rats. After the behavioral experiment, expression of the DREADD construct was histologically confirmed by analyzing mCherry expression in the pmVTA and the AcbSh. Notably, a group of rats (n = 9) received pmVTA injections of a viral construct without DREADD. These rats built the group “CNO/control” and were used to control for unspecific CNO effects.

The effect of intra-AcbSh CNO injections on relief learning is depicted in Fig. 3b (for individual startle data see Supplementary Figure 6). These CNO injections had no general unspecific effects on startle (main treatment effect: F_(2,19) = 0.14, p = 0.71). A significant interaction between treatment and trial type (F_(2,19) = 5.42, p = 0.03) revealed that the CNO injections affect the modulation of startle by the relief CS. This is confirmed by post hoc analyses demonstrated successful startle attenuation by the relief CS in vehicle-injected rats (t₍₁₀₎ = 2.96, p = 0.02) but not in those rats with validated expression of the viral DREADD construct (t₍₁₁₎ = 0.27, p = 0.96). In addition, startle difference scores are significantly reduced after CNO injections (t-test: t₍₁₈₎ = 2.22, p = 0.04). Notably, CNO had no effects on relief learning (paired t-test, startle alone vs. CS-startle: t₍₈₎ = 2.39, p = 0.048) in rats which received injections of the inactive viral construct (i.e., without DREADD) indicating that the CNO effects in the rats which received injections of the active virus (with DREADD) are mediated by the DREADDs.

Accumbal dopamine D2/3 receptor blockade prevents relief learning

To evaluate the role of accumbal dopamine D2/3 receptors in the acquisition of conditioned relief, two doses of the dopamine D2/3 receptor antagonist raclopride were injected immediately before submitting the rats to the relief conditioning procedure (vehicle: n = 9; 0.5 µg raclopride: n = 8; 1.0 µg raclopride: n = 9; for injection sites see Fig. 4a). The retention test on the next day was performed without any injections. As shown in Fig. 4c (for individual startle data see Supplementary Figure 7), there were no main effects of the injections on the startle response (F_(2,23) = 1.97, p = 0.16). The relief CS attenuated the startle response in vehicle-injected rats (trial type: F_(2,23) = 12.66, p = 0.002). Notably, injections of raclopride into the AcbSh blocked relief learning indicated by a significant interaction between treatment and trial type (F_(2,23) = 4.90, p = 0.02). Post hoc tests demonstrated significant startle attenuation after vehicle injections (t₍₉₎ = 4.68, p = 0.0003) but not after 0.5 and 1.0 µg raclopride injections (t₍₈₎ = 0.63, p = 0.90 and t₍₉₎ = 0.94, p = 0.74 respectively). This is confirmed by the separate analysis of the difference scores showing significant differences between vehicle and raclopride-injected rats (vehicle vs. 0.5 µg raclopride: t₍₁₅₎ = 2.73, p = 0.02; vehicle vs. 1.0 µg raclopride: t₍₁₆₎ = 2.64, p = 0.03; Dunnett’s post hoc comparison after ANOVA, F_(2,23) = 4.87, p = 0.02).

Fig. 4.

Intra-accumbal injections of the dopamine D2/3 receptor antagonist raclopride blocked the acquisition but not the retention of conditioned relief. a Histological verification of the AcbSh injection sites of vehicle and raclopride into the AcbSh. In separate groups of rats, injections were performed either before conditioning (filled circles), or before testing on conditioned relief (filled squares). b Reactivity to foot shocks (FS) after vehicle and raclopride injections (n = 9/group), measured by the mean locomotor response during shock application (duration: 1 s). Increasing footshock intensities resulted in higher reactivity; however, raclopride injections had no effects. c Intra-AcbSh raclopride injections before relief conditioning significantly blocked relief learning (vehicle: n = 9; 0.5 µg raclopride: n = 8; 1.0 µg raclopride: n = 9), whereas d raclopride injections before the retention test (n = 16) had no effects. ^#p < 0.05 post hoc Dunnett’s comparison with vehicle difference score; **p < 0.01, *p < 0.05, post hoc Sydak’s multiple comparisons startle alone vs. CS-startle

In contrast, raclopride injections into the AcbSh before the retention test did not affect expression of conditioned relief (Fig. 4d). An ANOVA revealed main effects of trial type (F_(1,15) = 22.91, p = 0.0002) but not of treatment (F_(2,30) = 0.63, p = 0.54), as well as no interaction between these two factors (F_(2,30) = 0.87, p = 0.43).

In a separate experiment, we also tested whether intra-accumbal raclopride injections affect the locomotor reactivity to foot shocks which would indicate impaired perception (Fig. 4b). There was only a main effect of stimulus intensity (F_(4,64) = 25.93, p < 0.0001) but neither a main effect of treatment (F_(1,16) = 1.80, p = 0.20) nor an interaction between treatment and footshock intensity (F_(4,64) = 0.56, p = 0.69). Notably, also a focused analysis of the reactivity to 0.4 mA foot shocks alone revealed no effects of raclopride (Student's t-test: t₍₁₆₎ = 1.21, p = 0.24).

Discussion

The goal of the present study was to explore whether the mesolimbic dopamine system, in particular the projection from the pmVTA to the AcbSh, is involved in relief learning.

We used a behavioral procedure of relief learning in rats which was previously established by our group [23]. During conditioning, the CS is presented 3 s after the US onset, a condition that leads to the most robust attenuation of the acoustic startle response [23, 42], a bivalent and translational behavioral measure of emotional valence [43, 44]. Several control experiments of our group and of others demonstrated that this startle attenuation is based on an associative status of the CS, since the CS does not affect the startle response before conditioning or after different types of sham conditioning such as totally randomized presentations of the CS and the US [3, 23, 42, 45].

Relief and safety learning seem to be two distinct learning types [3]. Even though our relief learning protocol has similarities with certain safety learning protocols [46], it has striking differences with other safety learning protocols [10, 18]. In the latter protocols, the CS and the US are presented explicitly unpaired without any fixed temporal relationship, yet our protocol relies on backward pairing of the CS with the US with a very short and fixed inter-stimulus interval (3 s). Furthermore, we previously demonstrated that the nucleus accumbens is crucial for relief learning but not for safety learning [25], which suggests different neural underpinnings for these learning types.

Role of the pmVTA-AcbSh projection in relief learning

The scientific question of the present study is based on findings that the nucleus accumbens is crucially involved in relief learning [23, 24, 28, 29, 47] and that some neurons of the pmVTA are activated by the offset of an aversive event [32], i.e. in the moment of relief. First, using retrograde and anterograde tracing, we confirmed literature data (e.g. [31]) demonstrating that the pmVTA projects to the AcbSh (Fig. 1). Furthermore, using immunohistochemistry for TH, a critical enzyme in dopamine synthesis, and for c-Fos, an indirect marker of neuronal activity [48], we showed that a decent number of the dopaminergic pmVTA-AcbSh projection neurons are activated by foot shocks (Fig. 1b). Whether this activation is triggered by the foot shocks themselves or by the relief from the foot shocks cannot be determined by the used methods. However, this finding indicates that the pmVTA neurons that are activated by the offset of foot shocks [32] are dopaminergic and they project to the AcbSh.

To examine whether the dopaminergic pmVTA neurons are not only activated by foot shocks but also involved in relief learning, we performed dopaminergic lesions of the pmVTA by local injections of the neurotoxin 6-OHDA. Such 6-OHDA-induced lesions of the VTA were described to impair several reward-related behaviors such as self-administration of rewarding drugs [49] or sucrose intake [50]. In the present study, pmVTA lesions prevented relief learning, measured by startle attenuation of the relief CS (Fig. 2a). This strongly supports the idea that the dopaminergic projection from the pmVTA to the AcbSh is mediating relief learning. Importantly, this lesion effect seems to be specific to relief learning since startle attenuation after safety learning (Fig. 2b) and startle potentiation after fear learning (Fig. 2c) were not affected. Here, it should be noted that our animals were repeatedly tested with these three different learning protocols and a previous type of learning might affect performance in one of the consecutive types of learning. We tried to prevent such potential interferences by using different to-be-learned stimuli but generalization processes cannot be excluded. Nevertheless, the findings that our lesion affect relief but not fear or safety learning confirm previous studies demonstrating the relief learning requires a different neural network than fear learning [23] and safety learning [25]. In these studies, the nucleus accumbens was temporary inactivated by local muscimol injections. Importantly, the rats were not repeatedly tested in different learning protocols but different groups of rats were used for each type of learning.

Next, we made use of chemogenetics [41, 51] to examine whether the direct projection from the pmVTA to the AcbSh is mediating relief learning. The DREADD vector, AAV8-CaMKIIa-hM4D(Gi)-mCherry, was injected into the pmVTA. During the behavioral experiment, CNO—the drug activating the inhibitory DREADD—was locally infused into the AcbSh in order to specifically silence the pmVTA-AcbSh projection. These CNO injections clearly prevented relief learning (Fig. 3b), which was intact in vehicle-injected rats. Notably, CNO had no effects on relief learning in rats which were injected with an inactive viral construct, i.e. in which DREADDs were not expressed. The latter control experiment is of importance since CNO—at least after systemic injections—is metabolized into clozapine [52] which has not only a high affinity to the DREADD, i.e. hM4D(Gi), but also to a number of other receptors [53]. However, it is not known so far whether this metabolism is of significance after local CNO injections into the brain.

It should also be noted that promoter which was used for the viral DREADD construct is not very specific, i.e. most probably not only dopaminergic VTA neurons were infected. However, Melchior et al. [54] demonstrated in an optogenetic experiment that using the CaMKIIa promotor for an optogenetic construct was equally efficient regarding stimulation-induced dopamine release in the nucleus accumbens than using TH:Cre as a promoter. Furthermore, they found their CaMKIIa-driven construct strongly expressed in TH-positive VTA neurons.

The previous experiments showed that (a) a dopaminergic projection from the pmVTA to the nucleus accumbens shell is activated by aversive events, (b) dopaminergic lesions of the pmVTA block relief learning, and (c) chemogenetic silencing of the projection from the pmVTA to the AcbSh blocks relief learning. We further asked whether a blockade of dopamine receptors within the AcbSh would have similar effects. We recently demonstrated that accumbal injections of the selective dopamine D1 receptor antagonist SCH23390 inhibit the acquisition and expression of conditioned relief [28]. However, an involvement of other dopamine receptor subtypes was not investigated so far. In the present study, we therefore injected raclopride, a dopamine D2/D3 receptor antagonist [55], into the AcbSh, either before the relief conditioning procedure or before the retention test on conditioned relief. Interestingly, only the injections before conditioning (Fig. 4c) but not the ones before the retention test (Fig. 4d) reduced relief learning. This indicates that accumbal D2/D3 receptors are selectively important for the acquisition but not the expression of relief memory. Notably, the locomotor response to the foot shocks was not impaired by the raclopride injections (Fig. 4b), suggesting that footshock perception was not impaired.

Role of dopamine in relief and safety learning

All four experiments of the present study support our working hypothesis that a dopaminergic projection from the pmVTA to the AcbSh mediates relief learning. Our data strongly support findings from a completely different procedure of pain relief learning. In this procedure [26, 56], subchronic pain in rats is induced by incision of the skin and underlying hind-paw tissue. Peripheral nerve blockade by local injections of lidocaine resulted in pain relief, which was associated with the context in which relief took place. In a following place preference test, the relief-associated context is preferred towards the context of a sham treatment. Using this behavioral procedure, it was shown that temporary inactivation of the nucleus accumbens or the VTA impairs pain relief-induced place preference learning [26]. Furthermore, an increase of accumbal dopamine release during pain relief was measured by microdialysis, and blockade of dopamine receptors within the nucleus accumbens inhibited relief learning in this procedure [26].

These data and the present data strongly argue for an important role of mesolimbic dopamine system in relief learning. This nicely fits to the general concept of relief learning [3]: Relief from an aversive event is rewarding [22, 24, 30] and the mesolimbic dopamine system is well known for its role in mediating reward (e.g. [31]). We here focused on the pmVTA, since some of the pmVTA neurons are activated by the relief from foot shocks [32], however, also other parts of the VTA might be involved in relief learning. Furthermore, we here show an crucial role of accumbal dopamine D2/3 receptors in the acquisition but not the expression of relief learning, whereas accumbal dopamine D1 receptors are important for both acquisition and expression of conditioned relief [28]. This fits to the findings that activation of these receptors in the nucleus accumbens induce reward [57], whereas blockade of these receptors inhibits reward-related learning (e.g. [37]).

Literature data suggest that the dopaminergic brain system also plays a significant role in safety learning. Pollak et al. [58] showed that systemic applications of drugs acting on the dopamine D2/3 receptors strongly modulate the retention of learned safety in mice. Interestingly, the authors discussed that a VTA projection to the amygdala mediates these effects. It is not known whether D2/3 receptors are also involved in the acquisition of learned safety, and D1 receptors were not investigated so far in safety learning. Future studies should have a closer look on the role of the different dopamine receptors and the different VTA projections in safety and relief learning.

Both, relief learning as it was investigated in the present study and safety learning as investigated by Daniela Pollak and others, are very similar to classical conditioned inhibition procedures [59]. In fact, the retention test of both procedures constitutes a type of summation test: animals are tested in the conditioning context, which induces contextual fear, and the relief or safety stimuli can be understood as conditioned inhibitors of this contextual fear which lead to an attenuation of the startle response or freezing behavior. However, the robust findings that (1) the nucleus accumbens is involved in relief learning but not safety learning (present study; [25]) and (2) learned relief but not learned safety is rewarding [22, 60, 61] indicates that relief and safety learning are different on a neural and conceptual level.

Potential clinical implications

What are potential implications of the present findings? For animals and humans, cues are important not only for signaling potential danger [4], but also for predicting the relief or the absence of danger [3, 18]. Increasing evidence during the last decade implies that psychiatric diseases such as depression or anxiety disorders are associated with an impaired processing of signals predicting relief and safety [11, 17, 18, 62]. Our findings suggest that these deficits could be caused by a hypofunction of the mesolimbic dopamine system, as it is observed in post-traumatic stress disorder [63]. Based thereon, it would be interesting to also investigate whether relief learning is affected in diseases with a hyperfunction of the mesolimbic dopamine system such as schizophrenia [64] or addiction [65]. Last, the present data may also help to better understand trichotillomania (hair pulling), excoriation (skin picking) disorder, or non-suicidal self-injury since these conditions are associated with the phenomenon of pain offset relief [66, 67].

Conflict of interest

The authors declare that they have no conflict of interest.