Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2014 Apr 30;34(18):6286–6293. doi: 10.1523/JNEUROSCI.4146-13.2014

Free-Operant Avoidance Behavior by Rats after Reinforcer Revaluation Using Opioid Agonists and d-Amphetamine

Anushka Fernando 1,2,✉, Gonzalo Urcelay 1,2, Adam Mar 1,2, Anthony Dickinson 1,2, Trevor Robbins 1,2
PMCID: PMC4004814  PMID: 24790199

Abstract

The associative processes that support free-operant instrumental avoidance behavior are still unknown. We used a revaluation procedure to determine whether the performance of an avoidance response is sensitive to the current value of the aversive, negative reinforcer. Rats were trained on an unsignaled, free-operant lever press avoidance paradigm in which each response avoided or escaped shock and produced a 5 s feedback stimulus. The revaluation procedure consisted of noncontingent presentations of the shock in the absence of the lever either paired or unpaired with systemic morphine and in a different cohort with systemic d-amphetamine. Rats were then tested drug free during an extinction test. In both the d-amphetamine and morphine groups, pairing of the drug and shock decreased subsequent avoidance responding during the extinction test, suggesting that avoidance behavior was sensitive to the current incentive value of the aversive negative reinforcer. Experiment 2 used central infusions of D-Ala2, NMe-Phe4, Gly-ol5]-enkephalin (DAMGO), a mu-opioid receptor agonist, in the periacqueductal gray and nucleus accumbens shell to revalue the shock. Infusions of DAMGO in both regions replicated the effects seen with systemic morphine. These results are the first to demonstrate the impact of revaluation of an aversive reinforcer on avoidance behavior using pharmacological agents, thereby providing potential therapeutic targets for the treatment of avoidance behavior symptomatic of anxiety disorders.

Keywords: avoidance, d-amphetamine, morphine, nucleus accumbens shell, periacqueductal gray, revaluation

Introduction

Appetitive studies have provided good evidence that instrumental behavior is directly sensitive to the current incentive value of the reinforcer. This conclusion is based on a procedure known as reinforcer revaluation, which involves changing the incentive value of the reinforcer before testing whether instrumental responding is sensitive to this revaluation when assessed in extinction in the absence of the reinforcer. If instrumental responding is sensitive to the reinforcer value, then changes in this value should modulate instrumental behavior during the extinction test. Appetitive instrumental behavior has been shown to be sensitive to reinforcer revaluation in rats (Adams and Dickinson, 1981) and, more recently, in both primates (Rhodes and Murray, 2013) and humans (Valentin et al., 2007). The theoretical inference from this finding is that the instrumental behavior is goal directed in the sense of being mediated by representations of the response-reinforcer contingency and the current incentive value of the reinforcer.

The processes underlying avoidance behavior have received less attention. In contrast to appetitive behavior, the performance of an avoidance response is maintained by a negative contingency between the response and the aversive reinforce because the avoidance response prevents its presentation. The outcome of an avoidance response is therefore the omission of the aversive reinforcer, which is thought to be the critical event in establishing and maintaining instrumental avoidance (Mackintosh, 1983). Instrumental avoidance is therefore sensitive to revaluation only if mediated by a representation of the negative action-reinforcer contingency and the current value of the aversive reinforcer. Three studies have assessed the effects of revaluation of an aversive reinforcer on avoidance behavior, one in rats (Hendersen and Graham, 1979) using heat as a reinforcer and two in humans, one using monetary loss as the reinforcer (Declercq and DeHouwer, 2008) and the other an electric shock (Gillan et al., 2013). Although these studies demonstrate that it is possible to revalue an aversive, negative reinforcer of avoidance behavior, none has assessed the neurobiological mechanisms of reinforcer revaluation. Experiment 1 in the present study, however, provides evidence for these mechanisms using analgesic agents to revalue an aversive foot shock reinforcer. After free-operant lever press avoidance training, the rats received revaluation sessions during which the analgesic agents morphine and d-amphetamine (Burrill et al., 1944; Abbot et al., 1995) were administered before noncontingent presentations of the foot shock reinforcer in a paired group or before sessions without the shock in the unpaired group. If avoidance responding is goal directed, we predicted that, during the extinction test session, the paired group would reduce responding after experience of foot shock under analgesia. In Experiment 2, selective infusions of D-Ala2, NMe-Phe4, Gly-ol5]-enkephalin (DAMGO), a mu-opioid receptor agonist, in the periacqueductal gray (PAG), a substrate of the pain circuitry (McNally, 1999), and the nucleus accumbens shell (NacS), a region rich in opioid receptors involved in the processing of emotional stimuli (Mansour et al., 1995; Barrot et al., 2002), were administered during the revaluation procedure in paired and unpaired groups to identify brain substrates involved in the mediation of shock revaluation.

Materials and Methods

Subjects.

Experiment 1 was conducted in two groups sequentially. The first group consisted of 17 male Lister hooded rats that received treatment with morphine; the second group consisted of 16 male Lister hooded rats that received treatment with d-amphetamine. Experiment 2 was conducted in a group of 42 rats. Rats, which were obtained from Charles River Laboratories, weighed 300 g at the start of the experiment. They were housed in groups of four per cage in a reverse light cycle room (12 h light/dark cycle with lights on at 19:00). Training and testing occurred during the dark phase and complied with the statutory requirements of the UK Animals (Scientific Procedures) Act of 1986.

Apparatus.

Fourteen operant conditioning chambers (Med Associates) were used, each measuring 29.5 × 32.5 × 23.5 cm with a Plexiglas ceiling, front door and back panel, and metal paneling on the sides of the chamber. The floor of the chamber was covered with a metal grid with a metal tray beneath. Med Associates shock generators (ENV-224AMWN, 115 V AC, 60 Hz) were connected to the metal grid and used to produce scrambled 0.5 s, 0.5 mA foot shocks. Each chamber was placed within a sound- and light-attenuating box and interfaced to a computer through Whisker control software (Cardinal and Aitken, 2010). The feedback stimulus was either a 2900 Hz tone produced by a Med Associates tone generator (ENV-223AM) or a white noise produced by a Med Associates white noise generator (ENV-2255M) counterbalanced. Both of these generators were attached to the same wall of the chamber and the stimuli were set to 8 dB above background level. Levers could be extended either side of a central food magazine on the opposite side wall, but no pellets were ever delivered.

Pretraining.

Rats were first habituated to the chamber and the levers for 4 d. During the first 2 d, either the left or the right lever was randomly chosen at the start of the session. The designated lever was then extended as the session began and any responses resulted in its retraction for 1 s, followed by its immediate extension back into the chamber. For the last 2 d, the opposite lever was extended and the number of responses was limited so that the number of retractions and extensions of the 2 levers was equated. The houselights remained on until the end of each daily 1 h session.

Training.

The start of the session was marked with the illumination of the house light and the extension of a single lever, which was randomly chosen as either the right or the left lever at the start of daily sessions. The lever remained extended for the entire session. The session began with an unsignaled avoidance period ranging between 120 and 140 s that, in the absence of a lever press response, was followed by intermittent foot shocks (0.2 mA). During this shock period, the shock–shock interval was 5 s. After three presentations of the shock, the shock period terminated and was followed by the next avoidance period. The maximum number of shocks that could be presented in the session was limited to 30, at which point the session ended. Any lever press during the avoidance or shock periods immediately terminated these periods with a 120 s auditory feedback stimulus, which was then followed by the next avoidance period. The use of a feedback stimulus facilitates the performance of the avoidance with this procedure (Fernando et al., unpublished data). Lever presses during the feedback stimulus had no consequence and did not contribute to the assessment of avoidance responding. The variability and duration of the avoidance period was increased over the course of training to a final mean avoidance interval of 120 s (range 10–230), the feedback stimulus duration was gradually reduced to 5 s, and the shock intensity increased in 0.1 mA increments across training to a final value of 0.5 mA. Subjects were trained for ∼30 sessions.

Surgery.

Rats in Experiment 2 received surgery after avoidance training. Twenty-four rats were implanted in the PAG with 26 Ga unilateral guide cannulae (Plastics One) according to the stereotaxic coordinates of AP +0.1, ML +0.8, DV −5.6 (from lambda, DV measured from skull). The cannulated hemisphere was counterbalanced between right or left hemisphere across subjects. Eighteen rats were cannulated in the NacS with 26 Ga bilateral guide cannulae (Plastics One) to the stereotaxic coordinates of AP +1.7, ML +1.0, DV −2.0 (from bregma, DV measured from skull). After surgery, rats were individually housed and left to recover with both food and water ad libitum. After 1 week of recovery, subjects were retrained on the avoidance task until 3 d of stable baseline was observed before the revaluation procedure.

Revaluation.

The revaluation procedure lasted 4 d (1 session/d). Rats in the paired group received 2 sessions in which the analgesic drug was administered before a session in which 15 presentations of 0.5 mA foot shock were experienced in the absence of the lever and feedback stimulus. Three shocks were presented with a shock–shock interval of 5 s, the next 3 shocks were then presented after an average interval of 588 s (range 240–348 s). During the other two sessions, rats received vehicle injections before sessions in which nothing occurred in the chamber for the 30 min. In the unpaired group, the drug was administered before sessions in which nothing occurred in the chamber and vehicle was administered before sessions with shock.

Drugs and administration procedure.

All rats receiving systemic treatments (Experiment 1) received 4 d of intraperitoneal injections, 2 d with the drug (morphine 10 mg/kg or d-amphetamine 1.5 mg/kg, calculated as free base) and 2 d with vehicle (0.9% filtered saline). Doses were chosen for their analgesic effects in the absence of motor depressant effects and selective behavioral effects (morphine: Babbini and Davis, 1972; Babbini et al., 1979; Kuribara et al., 1985; d-amphetamine: Abbot et al., 1995; Fernando et al., 2013b). Subjects were left in single cages in the dark after injections and then placed in the testing chambers either 20 min (morphine) or 15 min (d-amphetamine) after injection.

Central treatments.

All rats in Experiment 2 received infusions of the mu-opioid agonist DAMGO at a dose of 0.5 μg/μl and a flow rate of 0.25 μl/min with a 2 min diffusion period. Doses were chosen based on studies of central infusions of opioids in the PAG and NacS (Peciña and Berridge, 2000; Iordanova et al., 2006). Unilateral infusions in the PAG of 0.5 μl were performed with the injector extending 1 mm below the guide cannulae for 2 min. Bilateral infusions in the NacS were performed with the injector extending 5.25 mm beyond the guide cannulae for 1 min, resulting in a total bilateral volume of 0.5 μl of drug. Rats were placed in the test chamber 5 min after infusion. Infusions were only conducted for 2 of the 4 d with no vehicle infusions.

Before test infusions, all rats received mock infusions in which they were habituated to the infusion procedure and infusion room. This procedure was performed so that any behavioral effects of tissue damage mechanically induced by the first infusion occurred before the test session.

Extinction test.

The day after the 4 d revaluation procedure, a single 30 min drug-free test session was conducted, which was the same as baseline sessions except in the absence of the foot shock (i.e., extinction).

Reinforced test.

The next day after the test session, a reinforced test was conducted in which the revalued shock was presented in the absence of an avoidance response; the session was therefore the same as a baseline session and lasted 1 h.

Data analysis.

For all experiments in this study, the rates of lever press responding were square root (SQRT) transformed for statistical analysis as the variance increased with mean responding. ANOVA was conducted on the mean of the SQRT of the rate of avoidance responses per minute during the extinction test, with a between-subjects factor of revaluation condition (paired vs unpaired) and, in Experiment 2, a between-subject factor of infusion site (PAG vs NacS). A rejection criterion of p < 0.05 was used and the Huynh-Feldt adjustment was applied if sphericity was violated.

Results

The final numbers of subjects in each group are reported in Table 1.

Table 1.

Final group sizes for each treatment

Revaluation group Morphine d-Amphetamine PAG NacS
Paired 8 8 8 9
Unpaired 9 8 8 8
Open in a new tab

Experiment 1

Training

Analysis of both the d-amphetamine group and morphine group revealed no differences in rates of avoidance responses between the assigned revaluation conditions (paired vs unpaired) during the last training session before revaluation (both F < 1). Means (SEMs) of avoidance response rates during the baseline session before revaluation are reported in Table 2.

Table 2.

Mean of the SQRT transformed avoidance responses per minute of the d-amphetamine group and morphine group during the baseline session prior to revaluation

Drug Treatment condition Mean SE
d-amphetamine Paired 2.8 0.2
Unpaired 2.9 0.2
Morphine Paired 2.4 0.2
Unpaired 2.5 0.2
Open in a new tab

Extinction test

Figure 1 illustrates the reduction in avoidance behavior in rats that had pairings (relative to their unpaired controls) of either systemic morphine (revaluation F(1,15) = 14.0, p < 0.005) or d-amphetamine (revaluation F(1,14) = 14.7, p < 0.005) with shock before an extinction test session.

Figure 1.

Figure 1.

Open in a new tab

Prior pairings of either systemic morphine (A) or d-amphetamine (B) successfully revalued the foot shock, as evidenced by the reduction in rate of avoidance responses only in the paired groups during the drug-free extinction test. Bars represent the mean SQRT-transformed avoidance responses per minute ± SEM.

Reinforced test

Figure 4, A and B, provides evidence for the effectiveness of the revaluation procedure. On first experience of the revalued foot shock drug free, the paired group demonstrated a reduced rate of avoidance behavior for 5 min with respect to their unpaired controls. Statistical analysis revealed a significant reduction in the rate of avoidance responding in the paired group with respect to the unpaired group after revaluation with systemic d-amphetamine and a trend to the same effect with systemic morphine (d-amphetamine: revaluation F(1,14) = 4.6, p < 0.05; morphine: revaluation F(1,15) = 1.9, p > 0.15, N.S.). Analysis of the entire reinforced test session, in which the rate of avoidance responding was averaged in 5 min time bins, demonstrates the transitory nature of these effects. Rats in the paired groups increased their rates of avoidance responding across the session (morphine: time F(7.0, 105.7) = 2.9, p < 0.01; d-amphetamine: time F(6.8, 95.0) = 5.1, p < 0.001) to result in no differences in avoidance behavior between the two revaluation groups (morphine: revaluation F < 1; d-amphetamine: revaluation (F(1,14) = 1.6, p > 0.2, N.S.). The transitory nature of the revaluation effect is reflected in the inability to detect the change in rates of avoidance behavior in the revaluation groups over the course of the reinforced test session (morphine: revaluation × time F (11,165) = 1.2, p > 0.3, N.S.; d-amphetamine: revaluation × time F(11,154) = 1.6, p > 0.9, N.S.) Revaluation of shock with central infusions of DAMGO may result in more persistent effects during the reinforced tests.

Figure 4.

Figure 4.

Open in a new tab

Reinforced tests confirm the revaluation effects observed during the extinction tests with decreased avoidance responding in paired groups on their first drug-free experience of the foot shock. Reinforced tests were conducted after prior revaluation of shock with morphine (A), d-amphetamine (B), DAMGO in the PAG (C), or DAMGO in the NacS (D). Bars represent the mean SQRT-transformed avoidance responses in the first 5 min after first shock experience ± SEM.

Experiment 2

Histology

Figure 2, A and B, are representative microphotographs of the location of cannulae in the PAG and the NacS, respectively. Four rats were excluded from the study due to injector cannulae being positioned outside the PAG. Two rats were excluded due to procedural errors in the PAG group and three rats lost guide cannulae through the course of the procedure in the PAG group (two rats) and NacS group (one rat). There was no gross tissue damage in the local vicinity of the injector tracks of the approved placements.

Figure 2.

Figure 2.

Open in a new tab

Schematic representations of the positions of injector tips in the PAG (A) and the NacS (B).

Training

Analysis of the baseline session before the extinction test revealed no differences in responding either between the surgical groups or between treatment groups (infusion site, revaluation, infusion site × revaluation, all F < 1). Means (SEMs) of avoidance response rates during baseline are presented in Table 3.

Table 3.

Mean of the SQRT transformed avoidance responses per minute during the baseline session prior to the revaluation procedure in the PAG and NacS group

Injection site Treatment condition Mean SE
PAG P 2.6 0.2
UP 2.9 0.2
NacS P 2.7 0.2
UP 2.7 0.2
Open in a new tab

Extinction test

Figure 3 demonstrates the successful revaluation of shock with infusions of DAMGO in both the PAG and NacS. Infusions in these areas before shock revaluation significantly reduced rates of avoidance responding during the drug-free extinction test only when DAMGO was paired with shock (revaluation F(1,29) = 80.1, p < 0.001). A main effect of infusion site was revealed with the ANOVA (F(1,29) = 5.6, p < 0.05), which did not, however, interact with the effect of revaluation (revaluation × infusion site F < 1). Post hoc pairwise comparisons revealed a significant difference in the rate of avoidance responding between the paired and unpaired revaluations for each region infused (PAG: revaluation p < 0.001; NacS: revaluation p < 0.001).

Figure 3.

Figure 3.

Open in a new tab

Central infusions of DAMGO in either the PAG (A) or the NacS (B) revalued the foot shock, resulting in a reduction in the rate of avoidance responding in the paired groups during the drug-free extinction test. Bars represent the mean SQRT-transformed avoidance responses per minute ± SEM.

Reinforced test

Figure 4, C and D, illustrates the reduced rate of avoidance responding of the paired groups with respect to their unpaired controls after shock revaluation with central infusions of DAMGO in the PAG and NacS, respectively. Statistical analysis confirmed the observed reduced rate of avoidance responding in the paired group with respect to the unpaired group (F(1,35) = 31.9, p < 0.001), which was consistent across both brain regions (infusion site F < 1, N.S.; infusion site × revaluation F(1,35) = 2. 1, p > 0.1, N.S.). Furthermore, analysis of rates of avoidance behavior averaged in 5 min time bins across the 1 h session revealed a significant effect of time (F(9.1, 319.3) = 4.3, p < 0.01), with rates of avoidance behavior differing between the two revaluation groups across the session (time × revaluation F(11,385) = 4.2, p < 0.001). Despite the different rates of avoidance behavior seen between the NacS and PAG across the reinforced test session (time × infusion site F(11,385) = 1.9, p < 0.5), this did not result in differences in the rate of avoidance behavior between the revaluation groups nor between regions (infusion site F < 1, N.S.; infusion site × group F(1,35) = 2.1, p > 0.1, N.S.; time bin × infusion site × group F < 1, N.S.). Central administration of the mu-opioid agonist DAMGO may have produced a more persistent revaluation effect that was resistant to repeated exposure to foot shock.

Discussion

This study demonstrates that free-operant lever press avoidance behavior is sensitive to the current value of the aversive foot shock, indicating that responding is mediated by a representation of the negative contingency between the response and reinforcer. The reduction in avoidance responding during the extinction test session was only observed in groups that received prior pairings of analgesic drugs and foot shock during the revaluation procedure. These results will be discussed in terms of their neurobiological mechanisms and their implications for theories of avoidance behavior.

Systemic morphine and d-amphetamine have been shown to produce analgesia in a variety of models of pain sensitivity (Abbot and Guy, 1995; Abbot et al., 1995; Babbini et al., 1979; Tricklebank et al., 1984; Connor et al., 2010, Sohn et al., 2000). Experience with either analgesic drug prior only to sessions with foot shock presentations decreased avoidance responding during the subsequent drug-free extinction test. This finding strongly suggests that these drugs diminished the pain experienced with presentations of the foot shock, leading to its revaluation.

Infusions of DAMGO both in the PAG and NacS in the paired revaluation groups again resulted in a decrease in avoidance responding during the drug-free test session. Both regions have been implicated in mediating analgesia. Morphine is believed to inhibit the ascending transmission of nociceptive information from the spinal cord dorsal horn, leading to activation of descending circuits that include the PAG (Reynolds, 1969; McNally, 1999). Dopamine receptors (DARs) within the PAG have also been shown to mediate antinociception, because infusions of apomorphine, a DAR agonist, have a direct antinociceptive effect during hot plate tests when infused into the PAG (Meyer et al., 2009). The involvement of DA in analgesia has also been demonstrated in the Nac, where DA release parallels antinociceptive responses in drug-naive and morphine-pretreated rats (Schmidt et al., 2002) and fluctuates in healthy human controls to predict the magnitude of pain.

Pain can, however, be conceptualized as more than simply nociception. Therefore, the experience of pain can be influenced not only by its sensory properties, but also by the motivational state of the subject (Leknes and Tracey, 2008). The influence of sensory and motivational influences on the experience of pain is therefore flexible and has been formalized using the signal detection theory, which assumes the detection of a stimulus above a background “noise” of stimuli in our environment and requires a statistical decision by the subject (Lloyd and Appel, 1976; Rollman, 1979). The sensitivity in detection of this stimulus from the background noise is a statistical parameter that could reflect the sensory properties of a painful stimulus. Determining whether a stimulus differs to background stimuli, that is, whether a stimulus is painful respective to the current state of the individual, can be altered by the subject's response bias or criterion, which is influenced by their motivational state. These processes, the sensory perception of pain and the motivation of the subject to report a stimulus as painful, may be represented in the PAG and NacS, respectively.

The sensory aspects of pain are likely to be mediated by the PAG, a region within the midbrain that acts as a supraspinal site to produce opioid-mediated analgesia (Pert and Yaksh, 1975; Yaksh et al., 1976; Sohn et al., 2000). The NacS has also been implicated in aversive processing and the mediation of pain (Becerra et al., 2001; Pezze et al., 2001; Aharon et al., 2006; Martinez et al., 2008; Levita et al., 2009; Badrinarayan et al., 2012). Baliki et al. (2010) observed activation of the putative NacS in humans during application of a painful stimulus before ratings of pain and thus potentially acting as a predictive signal of pain experience. Infusions of DAMGO within the NacS may have revalued the sensory experience of shock via an analgesic mechanism, as predicted for the PAG; however, they could also have activated an appetitive reward system involving the NacS (Parkinson et al., 1999; Corbit and Balleine, 2011), changing the motivational state of the subject.

Lesions of the NacS have been shown to eliminate appetitive-outcome-specific Pavlovian instrumental transfer (Corbit and Balleine, 2011). Lesions of the NacS also abolished the potentiating effect of d-amphetamine on preferential lever pressing to produce a stimulus previously paired with food (Parkinson et al., 1999). Potential activation of the appetitive reward system in this study with infusions of DAMGO in the NacS may have altered the motivational state of the subject, changing the criterion at which subjects report a stimulus as painful with respect to background stimuli. Support for this prediction is seen in a study by Navratilova et al. (2012) using a model of experimental postsurgical pain in rats. The investigators blocked afferent input from the site of injury, producing chronic pain with a local anesthetic; this resulted in a change in the motivational state of the rats reflected in the observed conditioned place preference (CPP) by these animals. The elicitation of CPP was associated with increased activity of dopaminergic cells within the ventral tegmental area and enhanced DA release specifically in the medial NacS. Our previous studies have indicated that free-operant avoidance behavior is affected by catecholamine manipulations of the NAcS, but not the NAc core subregion, suggesting some specificity of the NacS in avoidance behavior supported by appetitive-aversive interactions (Fernando et al., 2013c). Furthermore, connected regions of the NacS, such as the ventromedial prefrontal cortex and amygdala, have also been implicated in avoidance behavior, acting as interfaces for appetitive and aversive influences (Wilensky et al., 2000; Kim et al., 2006; Prévost et al., 2011; Moscarello and LeDoux, 2013; Fernando et al., 2013a). The dual role of the NacS in both appetitive and aversive processing suggests that the mechanism by which revaluation of the shock occurred with infusions of DAMGO differed between the NacS and PAG, with each region potentially mediating a different aspect of pain experience. This may be reflected in Figure 5, C and D, in which paired infusions of DAMGO in the PAG during revaluation of shock resulted in a more persistent reduction of avoidance behavior during the reinforced test than in the other groups, a result predicted if the sensory experience of the shock had been altered.

Figure 5.

Figure 5.

Open in a new tab

Rates of avoidance responding across the reinforced test session after revaluation of shock with morphine (A), d-amphetamine (B), DAMGO in the PAG (C), or DAMGO in the NacS (D). Bars represent the mean SQRT-transformed avoidance responses per minute ± SEM.

The diverging mechanisms of these two regions in revaluation of aversive stimuli could still be indirectly linked anatomically, as in previous studies of pain report activation of both regions (Becerra et al., 2001; Leknes et al., 2013). Specifically, the revaluation of a painful stimulus has been shown in humans to result in greater functional connectivity between the PAG and ventral striatum (Leknes et al., 2013).

These experiments, at the very least, demonstrate that the value of the negative reinforcer is encoded in the associative representations that mediate avoidance behavior. The precise nature of these processes, however, remains undetermined. Cognitive theory (Bolles, 1970; Seligman and Johnston, 1973; Lovibond, 2008) argues that subjects learn the outcomes of responding and of not responding and then make decisions about whether to respond or withhold responding on the basis of a comparison between the two expected outcomes. Cognitive theory assumes that the aversive valence of the reinforcer motivates avoidance behavior through knowledge of the negative contingency between response and reinforcer. The implications for the present results is that revaluation of the foot shock may have resulted in a reduced preference to perform the lever press that was negatively correlated with the presentation of the aversive reinforcer with respect to not performing the avoidance behavior.

Alternatively, classic two-factor theory (Mowrer, 1947; Konorski, 1967) could also predict the sensitivity of avoidance behavior to revaluation of the aversive reinforcer if it is assumed that fear motivation generated by Pavlovian conditioning to the context in the case of free-operant avoidance is mediated by some representation of the shock. Aversive Pavlovian reinforcer revaluation studies have demonstrated inflation of the value of the unconditioned stimulus (US). For example, Rescorla (1974) reported that noncontingent exposure to a more intense shock US than that used during Pavlovian conditioning can enhance subsequent conditioned responding, also suggesting a role for the representation of the value of the US (Rescorla, 1974).

Whatever the merits of these two theories, the present study confirms that avoidance conditioning is mediated by a representation of the aversive reinforcer. Changing the value of a reinforcer of behavior allows the experimenter to test whether the animal has knowledge of the causal relationship between the response and reinforcer. The reduction in avoidance behavior during the drug-free extinction test in this study suggests that the rats had learned the negative contingency between the avoidance response and aversive foot shock. This negative contingency could thus engender goal-directed processes that underlie avoidance behavior even after an extensive degree of avoidance training.

This study provides evidence for the successful revaluation of a negative aversive reinforcer of free-operant lever press avoidance behavior, suggesting that rats learned the negative contingency between their avoidance response and the presentation of the aversive foot shock. This is the first study, to our knowledge, to demonstrate the revaluation of an aversive reinforcer on free-operant avoidance behavior. This study is not only novel in assessing the neurobiological basis of this process in the brain, but also in its use of analgesic drugs to revalue the aversive reinforcer. The sensitivity of free-operant avoidance behavior to this revaluation procedure could thus provide a useful tool with which to study avoidance habits (e.g., after overtraining of the avoidance response), which may be relevant to anxiety disorders such as obsessive compulsive disorder (Gillan et al., 2013).

Footnotes

This work was supported by the Wellcome Trust (Grant 089589/Z/09/Z to T.R., B.J. Everitt, A.C. Roberts, J.W. Dalley, and B.J. Sahakian) and was conducted at the Behavioural and Clinical Neuroscience Institute, which is supported by a joint award from the Medical Research Council (G1000183) and the Wellcome Trust (093875/Z/10?Z). A.F. was supported by a Medical Research Council Case studentship.

The authors declare no competing financial interests.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution and reproduction in any medium provided that the original work is properly attributed.

References

  1. Abbott FV, Guy ER. Effects of morphine, pentobarbital and amphetamine on formalin-induced behaviours in infant rats: sedation versus specific suppression of pain. Pain. 1995;62:303–312. doi: 10.1016/0304-3959(94)00277-L. [DOI] [PubMed] [Google Scholar]
  2. Abbott FV, Franklin KB, Westbrook RF. The formalin test: scoring properties of the first and second phases of the pain response in rats. Pain. 1995;60:91–102. doi: 10.1016/0304-3959(94)00095-V. [DOI] [PubMed] [Google Scholar]
  3. Adams CD, Dickinson A. Instrumental responding following reinforcer devaluation. The Quarterly Journal of Experimental Psychology Section B. 1981;33:109–121. [Google Scholar]
  4. Aharon I, Becerra L, Becerraa L, Chabris CF, Borsook D, Borsooka D. Noxious heat induces fMRI activation in two anatomically distinct clusters within the nucleus accumbens. Neurosci Lett. 2006;392:159–164. doi: 10.1016/j.neulet.2005.09.054. [DOI] [PubMed] [Google Scholar]
  5. Babbini M, Davis WM. Time-dose relationships for locomotor activity effects of morphine after acute or repeated treatment. Br J Pharmacol. 1972;46:213–224. doi: 10.1111/j.1476-5381.1972.tb06866.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Babbini M, Gaiardi M, Bartoletti M. Stimulus-response relationships in a quickly learned escape from shock: effects of morphine. Pharmacol Biochem Behav. 1979;11:155–158. doi: 10.1016/0091-3057(79)90007-8. [DOI] [PubMed] [Google Scholar]
  7. Badrinarayan A, Wescott SA, Vander Weele CM, Saunders BT, Couturier BE, Maren S, Aragona BJ. Aversive stimuli differentially modulate real-time dopamine transmission dynamics within the nucleus accumbens core and shell. J Neurosci. 2012;32:15779–15790. doi: 10.1523/JNEUROSCI.3557-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Baliki MN, Geha PY, Fields HL, Apkarian AV. Predicting value of pain and analgesia: nucleus accumbens response to noxious stimuli changes in the presence of chronic pain. Neuron. 2010;66:149–160. doi: 10.1016/j.neuron.2010.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Barrot M, Olivier JD, Perrotti LI, DiLeone RJ, Berton O, Eisch AJ, Impey S, Storm DR, Neve RL, Yin JC, Zachariou V, Nestler EJ. CREB activity in the nucleus accumbens shell controls gating of behavioral responses to emotional stimuli. Proc Natl Acad Sci U S A. 2002;99:11435–11440. doi: 10.1073/pnas.172091899. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Becerra L, Breiter HC, Wise R, Gonzalez RG, Borsook D. Reward circuitry activation by noxious thermal stimuli. Neuron. 2001;32:927–946. doi: 10.1016/S0896-6273(01)00533-5. [DOI] [PubMed] [Google Scholar]
  11. Bolles RC. Species-specific defense reactions and avoidance learning. Psychological Review. 1970;77:32–48. doi: 10.1037/h0028589. [DOI] [Google Scholar]
  12. Burrill DY, Goetzl FR, Ivy AC. The pain threshold raising effects of amphetamine. Journal of Dental Research. 1944;23:337–344. doi: 10.1177/00220345440230050601. [DOI] [Google Scholar]
  13. Cardinal RN, Aitken MR. Whisker: a client-server high-performance multimedia research control system. Behav Res Methods. 2010;42:1059–1071. doi: 10.3758/BRM.42.4.1059. [DOI] [PubMed] [Google Scholar]
  14. Connor J, Makonnen E, Rostom A. Comparison of analgesic effects of khat (Catha edulis Forsk) extract, d-amphetamine and ibuprofen in mice. J Pharm Pharmacol. 2000;52:107–110. doi: 10.1211/0022357001773580. [DOI] [PubMed] [Google Scholar]
  15. Corbit LH, Balleine BW. The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J Neurosci. 2011;31:11786–11794. doi: 10.1523/JNEUROSCI.2711-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Declercq M, De Houwer J. On the role of US expectancies in avoidance behavior. Psychon Bull Rev. 2008;15:99–102. doi: 10.3758/PBR.15.1.99. [DOI] [PubMed] [Google Scholar]
  17. Fernando AB, Murray JE, Milton AL. The amygdala: securing pleasure and avoiding pain. Front Behav Neurosci. 2013a;7:190. doi: 10.3389/fnbeh.2013.00190. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fernando AB, Urcelay GP, Mar AC, Dickinson A, Robbins TW. Comparison of the conditioned reinforcing properties of a safety signal and appetitive stimulus: effects of d-amphetamine and anxiolytics. Psychopharmacology (Berl) 2013b;227:195–208. doi: 10.1007/s00213-012-2952-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fernando AB, Urcelay GP, Mar AC, Dickinson A, Robbins TW. The role of the nucleus accumbens shell in the mediation of the reinforcing properties of a safety signal in free-operant avoidance: dopamine-dependent inhibitory effects of d-amphetamine. Neuropsychopharmacology. 2013c doi: 10.1038/npp.2013.337. doi: 10.1038/npp.2013.337. Advance online publication. Retrieved Dec. 12, 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gillan CM, Morein-Zamir S, Urcelay GP, Sule A, Voon V, Apergis-Schoute AM, Fineberg NA, Sahakian BJ, Robbins TW. Enhanced avoidance habits in obsessive-compulsive disorder. Biol Psychiatry. 2013;75:631–638. doi: 10.1016/j.biopsych.2013.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hendersen RW, Graham J. Avoidance of heat by rats: effects of thermal context on rapidity of extinction. Learning and Motivation. 1979;10:351–363. doi: 10.1016/0023-9690(79)90038-9. [DOI] [Google Scholar]
  22. Iordanova MD, McNally GP, Westbrook RF. Opioid receptors in the nucleus accumbens regulate attentional learning in the blocking paradigm. J Neurosci. 2006;26:4036–4045. doi: 10.1523/JNEUROSCI.4679-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Kim H, Shimojo S, O'Doherty JP. Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain. PLoS Biol. 2006;4:e233. doi: 10.1371/journal.pbio.0040233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Konorski J. Integrative activity of the brain. Chicago: University of Chicago; 1967. [Google Scholar]
  25. Kuribara H, Haraguchi H, Tadokoro S. Comparisons between discrete lever-press and shuttle avoidance responses in mice: acquisition processes and effects of psychoactive drugs. Jap J Pharmacol. 1985;38:141–151. doi: 10.1254/jjp.38.141. [DOI] [PubMed] [Google Scholar]
  26. Leknes S, Tracey I. A common neurobiology for pain and pleasure. Nat Rev Neurosci. 2008;9:314–320. doi: 10.1038/nrn2333. [DOI] [PubMed] [Google Scholar]
  27. Leknes S, Berna C, Lee MC, Snyder GD, Biele G, Tracey I. The importance of context: when relative relief renders pain pleasant. Pain. 2013;154:402–410. doi: 10.1016/j.pain.2012.11.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Levita L, Hare TA, Voss HU, Glover G, Ballon DJ, Casey BJ. The bivalent side of the nucleus accumbens. Neuroimage. 2009;44:1178–1187. doi: 10.1016/j.neuroimage.2008.09.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Lloyd MA, Appel JB. Signal detection theory and the psychophysics of pain: an introduction and review. Psychosom Med. 1976;38:79–94. doi: 10.1097/00006842-197603000-00002. [DOI] [PubMed] [Google Scholar]
  30. Lovibond PF, Saunders JC, Weidemann G, Mitchell CJ. Evidence for expectancy as a mediator of avoidance and anxiety in a laboratory model of human avoidance learning. Q J Exp Psychol (Hove) 2008;61:1199–1216. doi: 10.1080/17470210701503229. [DOI] [PubMed] [Google Scholar]
  31. Mackintosh NJ. Conditioning and associative learning. New York: Oxford UP; 1983. [Google Scholar]
  32. Mansour A, Fox CA, Akil H, Watson SJ. Opioid-receptor mRNA expression in the rat CNS: anatomical and functional implications. Trends Neurosci. 1995;18:22–29. doi: 10.1016/0166-2236(95)93946-U. [DOI] [PubMed] [Google Scholar]
  33. Martinez RC, Oliveira AR, Macedo CE, Molina VA, Brandão ML. Involvement of dopaminergic mechanisms in the nucleus accumbens core and shell subregions in the expression of fear conditioning. Neurosci Lett. 2008;446:112–116. doi: 10.1016/j.neulet.2008.09.057. [DOI] [PubMed] [Google Scholar]
  34. McNally GP. Pain facilitatory circuits in the mammalian central nervous system: their behavioral significance and role in morphine analgesic tolerance. Neurosci Biobehav Rev. 1999;23:1059–1078. doi: 10.1016/S0149-7634(99)00040-8. [DOI] [PubMed] [Google Scholar]
  35. Meyer PJ, Morgan MM, Kozell LB, Ingram SL. Contribution of dopamine receptors to periaqueductal gray-mediated antinociception. Psychopharmacology. 2009;204:531–540. doi: 10.1007/s00213-009-1482-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Moscarello JM, LeDoux JE. Active avoidance learning requires prefrontal suppression of amygdala-mediated defensive reactions. J Neurosci. 2013;33:3815–3823. doi: 10.1523/JNEUROSCI.2596-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Mowrer OH. On the dual nature of learning–a re-interpretation of ‘conditioning' and ‘problem-solving.’. Harvard Educational Review. 1947;17:102–148. [Google Scholar]
  38. Navratilova E, Xie JY, Okun A, Qu C, Eyde N, Ci S, Ossipov MH, King T, Fields HL, Porreca F. Pain relief produces negative reinforcement through activation of mesolimbic reward-valuation circuitry. Proc Natl Acad Sci U S A. 2012;109:20709–20713. doi: 10.1073/pnas.1214605109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive Pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine. J Neurosci. 1999;19:2401–2411. doi: 10.1523/JNEUROSCI.19-06-02401.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Peciña S, Berridge KC. Opioid site in nucleus accumbens shell mediates eating and hedonic ‘liking’ for food: map based on microinjection Fos plumes. Brain Res. 2000;863:71–86. doi: 10.1016/S0006-8993(00)02102-8. [DOI] [PubMed] [Google Scholar]
  41. Pert A, Yaksh T. Localization of the antinociceptive action of morphine in primate brain. Pharmacol Biochem Behav. 1975;3:133–138. doi: 10.1016/0091-3057(75)90092-1. [DOI] [PubMed] [Google Scholar]
  42. Pezze MA, Heidbreder CA, Feldon J, Murphy CA. Selective responding of nucleus accumbens core and shell dopamine to aversively conditioned contextual and discrete stimuli. Neuroscience. 2001;108:91–102. doi: 10.1016/S0306-4522(01)00403-1. [DOI] [PubMed] [Google Scholar]
  43. Prévost C, McCabe JA, Jessup RK, Bossaerts P, O'Doherty JP. Differentiable contributions of human amygdalar subregions in the computations underlying reward and avoidance learning. Eur J Neurosci. 2011;34:134–145. doi: 10.1111/j.1460-9568.2011.07686.x. [DOI] [PubMed] [Google Scholar]
  44. Rescorla RA. Effect of inflation of the unconditioned stimulus value following conditioning. Journal of Comparative and Physiological Psychology. 1974;86:101–106. doi: 10.1037/h0035964. [DOI] [Google Scholar]
  45. Reynolds DV. Surgery in the rat during electrical analgesia induced by focal brain stimulation. Science. 1969;164:444–445. doi: 10.1126/science.164.3878.444. [DOI] [PubMed] [Google Scholar]
  46. Rhodes SE, Murray EA. Differential effects of amygdala, orbital prefrontal cortex, and prelimbic cortex lesions on goal-directed behavior in rhesus macaques. J Neurosci. 2013;33:3380–3389. doi: 10.1523/JNEUROSCI.4374-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Rollman GB. Signal detection theory pain measures: empirical validation studies and adaptation-level effects. Pain. 1979;6:9–21. doi: 10.1016/0304-3959(79)90136-2. [DOI] [PubMed] [Google Scholar]
  48. Schmidt BL, Tambeli CH, Barletta J, Luo L, Green P, Levine JD, Gear RW. Altered nucleus accumbens circuitry mediates pain-induced antinociception in morphine-tolerant rats. J Neurosci. 2002;22:6773–6780. doi: 10.1523/JNEUROSCI.22-15-06773.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Seligman ME, Johnston JC. A cognitive theory of avoidance learning. In: McGuigan FJ, Lumsden DB, editors. Contemporary approaches to conditioning and learning. Washington, DC: Winston-Wiley; 1973. [Google Scholar]
  50. Sohn JH, Lee BH, Park SH, Ryu JW, Kim BO, Park YG. Microinjection of opiates into the periaqueductal gray matter attenuates neuropathic pain symptoms in rats. Neuroreport. 2000;11:1413–1416. doi: 10.1097/00001756-200005150-00012. [DOI] [PubMed] [Google Scholar]
  51. Tricklebank MD, Hutson PH, Curzon G. Involvement of dopamine in the antinociceptive response to footshock. Psychopharmacology. 1984;82:185–188. doi: 10.1007/BF00427770. [DOI] [PubMed] [Google Scholar]
  52. Valentin VV, Dickinson A, O'Doherty JP. Determining the neural substrates of goal-directed learning in the human brain. J Neurosci. 2007;27:4019–4026. doi: 10.1523/JNEUROSCI.0564-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Wilensky AE, Schafe GE, LeDoux JE. The amygdala modulates memory consolidation of fear-motivated inhibitory avoidance learning but not classical fear conditioning. J Neurosci. 2000;20:7059–7066. doi: 10.1523/JNEUROSCI.20-18-07059.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Yaksh TL, Yeung JC, Rudy TA. Systematic examination in the rat of brain sites sensitive to the direct application of morphine: observation of differential effects within the periaqueductal gray. Brain Res. 1976;114:83–103. doi: 10.1016/0006-8993(76)91009-X. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES