Skip to main content
PMC home page
Learning & Memory logoLink to Learning & Memory
. 2016 Nov;23(11):607–617. doi: 10.1101/lm.042820.116

Lateral, not medial, prefrontal cortex contributes to punishment and aversive instrumental learning

Philip Jean-Richard-dit-Bressel 1, Gavan P McNally 1,✉
PMCID: PMC5066604  PMID: 27918280

Abstract

Aversive outcomes punish behaviors that cause their occurrence. The prefrontal cortex (PFC) has been implicated in punishment learning and behavior, although the exact roles for different PFC regions in instrumental aversive learning and decision-making remain poorly understood. Here, we assessed the role of the orbitofrontal (OFC), rostral agranular insular (RAIC), prelimbic (PL), and infralimbic (IL) cortex in instrumental aversive learning and decision-making. Rats that pressed two individually presented levers for pellet rewards rapidly suppressed responding to one lever if it also caused mild punishment (punished lever) but continued pressing the other lever that did not cause punishment (unpunished lever). Inactivations of OFC, RAIC, IL, or PL via the GABA agonists baclofen and muscimol (BM) had no effect on the acquisition of instrumental learning. OFC inactivations increased responding on the punished lever during expression of well-learned instrumental aversive learning, whereas RAIC inactivations increased responding on the punished lever when both levers were presented simultaneously in an unpunished choice test. There were few effects of medial PFC (PL and IL) inactivation. These results suggest that lateral PFC, notably OFC and RAIC, have complementary functions in aversive instrumental learning and decision-making; OFC is important for using established aversive instrumental memories to guide behavior away from actions that cause punishment, whereas RAIC is important for aversive decision-making under conditions of choice.

Aversive stimuli have two distinct consequences for learning. First, they support Pavlovian learning about their environmental antecedents to imbue stimuli with the ability to elicit species-typical defense reactions. Second, they support instrumental learning about their behavioral antecedents to alter the probability that these behaviors will be emitted again in the future. In this way, animals learn an association between an action and aversive outcome (Bolles et al. 1980; Mackintosh 1983; Goodall and Mackintosh 1987) to withhold behaviors that cause the outcome. Whereas much is known about the brain mechanisms of Pavlovian aversive learning, little is known about these mechanisms for instrumental aversive learning.

Given its central role in Pavlovian aversive learning, we recently studied the role of the basolateral amygdala (BLA) in instrumental aversive learning and decision-making (Jean-Richard-dit-Bressel and McNally 2015). We showed that BLA, in particular its caudal region, encodes the aversive value of punishers and so contributes to instrumental aversive learning and decision-making when the punisher is present but not when it is absent. However, beyond BLA it is unknown whether common or distinct regions and circuits encode these different kinds of learning about aversive events.

Prefrontal cortex (PFC) is another region where there may be considerable overlap between Pavlovian and instrumental aversive learning. Medial PFC, in particular prelimbic (PL) and infralimbic (IL) PFC, has key roles in regulating Pavlovian fear. PL regulates expression of Pavlovian fear responses (Corcoran and Quirk 2007; Courtin et al. 2013a,b; Senn et al. 2014; Do-monte et al. 2015) and IL suppresses these responses after inhibitory treatments such as extinction (Milad and Quirk 2002; Quirk et al. 2006; Santini et al. 2008; Senn et al. 2014). These roles are due to direct (Paz and Paz 2012; Senn et al. 2014; Likhtik and Paz 2015) and indirect (Do-monte et al. 2015) interactions with amygdala. The role of PFC in instrumental aversive learning is less clear. Medial PFC contributes to appetitive decision-making, behavioral control, and instrumental learning (Balleine and Dickinson 1998; Arana et al. 2003; Corbit and Balleine 2003; O'Doherty et al. 2003; Ostlund and Balleine 2005; Tanaka et al. 2008; Ostlund et al. 2009) and thus may contribute to aversive instrumental learning and decision-making, but this lacks experimental support (Pelloux et al. 2013). Lateral PFC, namely orbitofrontal cortex (OFC) and rostral agranular insular cortex (RAIC), has not been reliably established as important for Pavlovian fear learning (Morgan and Ledoux 1999; Lacroix et al. 2000; Zelinski et al. 2010; Tian et al. 2011), but do show variations in activity linked to this learning (Furlong et al. 2010). However, OFC, in particular, has a key role in appetitive decision-making via encoding of outcome value (Arana et al. 2003; O'Doherty et al. 2003; Schoenbaum and Roesch 2005; Schoenbaum et al. 2009), as well as response choice and inhibition (Schoenbaum et al. 2009). These make OFC a strong candidate for punishment learning and behavior, but this role remains unsubstantiated (Pelloux et al. 2013) and a recent study showed that rats were more, not less, sensitive to punishment after OFC lesion (Orsini et al. 2015).

The aim of these experiments was to examine the role of OFC, RAIC, PL, and IL in aversive instrumental learning. We trained rats to respond for reward on two levers and then introduced a punishment contingency on one lever but not the other. We assessed the effects of reversible inactivation of OFC, RAIC, PL, and IL on the acquisition and expression of this instrumental aversive learning as well as on aversive choice.

Results

Experiment 1: role of the orbitofrontal cortex

Histology

Eighteen rats received bilateral cannulations of the OFC. Three rats had misplaced cannulae—these animals were excluded from all analyses. The location of microinjection cannulae tips in the OFC is shown in Figure 1A.

Figure 1.

Figure 1.

Open in a new tab

Effects of orbitofrontal cortex (OFC) inactivations. (A) OFC cannula placements as verified by Nissl-stained sections. Black dots represent the ventral point of the cannula tract, indicated on coronal sections adapted from Paxinos and Watson (2007). (B) Mean ± SEM lever-presses on the punished and unpunished levers during the last day of lever-press training (T) and punishment acquisition (sessions 1–5). Arrows indicate days that rats received infusions of either saline (n = 6) or baclofen and muscimol (BM) (n = 9) immediately prior to the session. (C) Mean ± SEM latency to initially press the punished and unpunished lever (averaged across trials) during punishment acquisition. (D) Mean ± SEM lever-press ratios of BM on lever pressing during punishment expression (n = 15). (E) Mean ± SEM latency to initially press the punished and unpunished lever (averaged across trials) during punishment expression after infusions of saline or BM. (F) Mean ± SEM magazine entries during punishment expression after infusions of saline or BM. (G) Mean ± SEM lever–press ratios of BM on lever pressing during aversive choice (n = 15). (H) Mean ± SEM latency to initially press the punished and unpunished levers during choice test after infusions of saline or BM. (I) Mean ± SEM magazine entries during choice test after infusions of saline or BM. (*) P < 0.05.

Lever-press training

In daily 40-min training sessions, rats received alternating periods of 5 min access to two levers whereby each lever was reinforced with a food pellet on a VI30s schedule for 7 d. The mean ± SEM responses and response latencies on the to-be-punished and to-be-unpunished levers for the last day of training are shown in Figure 1B–C (data point T). There was no significant overall difference between saline (n = 6) and BM (n = 9) groups in lever pressing at the end of lever-press training (F(1,13) < 1; P > 0.05), no overall difference in responding on the to-be punished and to-be unpunished levers (F(1,13) = 2.67; P > 0.05), and no group × lever interaction (F(1,13) < 1; P > 0.05).

Punishment acquisition

Following lever-press training, rats received daily 40 min daily punishment sessions consisting of alternating periods of 5 min access to two levers for 5 d. Pressing these levers was reinforced with food pellets via the same VI30s schedule used during training, and one of these levers was also punished on an FR10 schedule with delivery of 0.5 mA footshock.

Mean ± SEM lever pressing during these punishment sessions are shown in Figure 1B. Over the course of these sessions, there was a significant effect of lever (punished vs. unpunished) (F(1,13) = 57.0; P < 0.05), and the difference in responding on the levers increased across days (F(1,13) = 40.1; P < 0.05). Across sessions, there was an increase in responding on the unpunished lever (F(1,13) = 16.7; P < 0.05) and a decrease in responding on the punished lever (F(1,13) = 23.7; P < 0.05).

Rats received infusions of BM (n = 9) or saline (n = 6) (between subjects) prior to the first 2 d of punishment to determine the effect of OFC inactivation on punishment acquisition. During infusion days, there was no effect of BM on pressing of the punished (F(1,13) < 1; P > 0.05) or unpunished lever (F(1,13) < 1; P > 0.05) (Fig. 1B). Latencies to emit initial responses (averaged across trials) were also assessed (Fig. 1C). During infusion days, latencies to respond on the punished lever increased (F(1,13) = 24.1; P < 0.05), whereas latencies to respond on the unpunished lever did not change (F(1,13) < 1.1; P > 0.05). BM had no effect on latencies to press the punished or unpunished lever (all F(1,13) < 1.3; P > 0.05).

Punishment expression

Following the three noninfusion days of punishment acquisition, rats received two within-subject tests (BM vs. saline) for the expression of punishment. The contingencies in effect during acquisition remained in effect during these tests. We examined lever presses, latencies to press, as well as a ratio of responding on each lever between the BM and saline tests (ratio = A/(A + B)). When this ratio equals 0.5, responding on the lever did not change between the BM and saline tests, whereas values >0.5 indicate an increase in responding on the BM test and values <0.5 indicate a decrease in responding on the BM test.

Mean ± SEM lever-press ratios, lever-press latencies, and magazine entries during expression are shown in Figure 1D–F. Rats responded more on the unpunished lever than the punished lever (F(1,14) = 100.0; P < 0.05). Infusion of BM (mean = 23.1, SEM = 3.92) significantly increased punished lever pressing compared with saline (mean = 16.5, SEM = 3.87) (F(1,14) = 6.1; P < 0.05), while there was no difference in unpunished lever pressing (F(1,14) < 1; P > 0.05). This increase in punished lever pressing following BM was also observed in lever-press ratios (t(14) = 2.75; P < 0.05), whereas unpunished lever-press ratios did not differ from 0.5 (t(14) = −0.74; P > 0.05) (Fig. 1D), and resulted in significantly more shock deliveries during the BM (mean = 2.13; SEM = 0.38) compared with the saline (mean = 1.40; SEM = 0.38) test (F(1,14) = 7.56; P < 0.05). OFC inactivation significantly increased magazine entries (F(1,14) = 7.64; P < 0.05) (Fig. 1F) and this did not interact with lever (F(1,14) < 1; P > 0.05).

Rats were significantly slower to respond on the punished lever than the unpunished lever (averaged across trials) (F(1,14) = 61.3; P < 0.05) (Fig. 1E). BM had no effect on punished (F(1,14) < 1; P > 0.05) or unpunished latencies (F(1,14) < 1; P > 0.05).

Aversive choice

Next, rats were assessed in a choice procedure that involved concurrent presentations of punished and unpunished levers across two within-subjects test sessions (BM vs. saline). Each lever was reinforced with food pellets on a VI60s schedule, but no punishment was delivered. Figure 1G–I shows mean ± SEM lever-press ratios, lever-press latencies, and magazine entries for choice tests. Rats responded significantly more on the unpunished than the punished lever (F(1,14) = 43.2; P < 0.05). There was no difference in responding between BM and saline tests for the punished (F(1,14) = 1.77; P > 0.05) or unpunished lever (F(1,14) = 2.44; P > 0.05); nor was there an effect of BM on lever-press ratios for punished (t(14) = −0.73; P > 0.05) or unpunished (t(14) = −1.27; P > 0.05) levers (Fig. 1G). Rats were significantly slower to respond on the punished relative to the unpunished lever (F(1,14) = 10.1; P < 0.05) (Fig. 1H). There was no effect of BM on lever-press latencies (all F(1,14) < 1; P > 0.05) (Fig. 1H). BM into the OFC also had no effect on magazine entries (F(1,14) < 1; P > 0.05) (Fig. 1I).

Taken together, these data suggest that OFC is not important for learning about the aversive consequences of actions but is important for using this learning, once established, to guide behavior away from actions that cause aversive outcomes.

Experiment 2: role of the RAIC in punishment

Histology

Twenty rats received bilateral cannulations of the RAIC. Five rats were excluded due to misplaced cannulae, leaving 15 with microinjection cannulae tips in the RAIC (Fig. 2A).

Figure 2.

Figure 2.

Open in a new tab

Effects of insular cortex (RAIC) inactivations. (A) RAIC cannula placements as verified by Nissl-stained sections. Black dots represent the ventral point of the cannula tract, indicated on coronal sections adapted from Paxinos and Watson (2007). (B) Mean ± SEM lever-presses on the punished and unpunished levers during the last day of lever-press training (T) and punishment acquisition (sessions 1–5). Arrows indicate days that rats received infusions of either saline (n = 7) or baclofen and muscimol (BM) (n = 7) immediately prior to the session. (C) Mean ± SEM latency to initially press the punished and unpunished levers (averaged across trials) during punishment acquisition. (D) Mean ± SEM lever-press ratios of BM on lever pressing during punishment expression (n = 15). (E) Mean ± SEM latency to initially press the punished and unpunished levers (averaged across trials) during punishment expression after infusions of saline or BM. (F) Mean ± SEM magazine entries during punishment expression after infusions of saline or BM. (G) Mean ± SEM lever-press ratios of BM on lever pressing during aversive choice (n = 15). (H) Mean ± SEM latency to initially press the punished and unpunished levers during choice test after infusions of saline or BM. (I) Mean ± SEM magazine entries during choice test after infusions of saline or BM. (*) P < 0.05.

Lever-press training

The mean ± SEM responses on the to-be-punished and to-be-unpunished levers for the last day of training are shown in Figure 2B,C (data point T). One rat was excluded from between-subject analyses due to a cannulae patency issue that was resolved prior to within-subject tests. There was no significant overall difference between saline (n = 7) and BM (n = 7) groups in lever pressing at the end of lever-press training (F(1,12) < 1; P > 0.05), no overall difference in responding on the to-be punished and to-be unpunished levers (F(1,12) < 1; P > 0.05), and no group × lever interaction (F(1,12) < 1; P > 0.05).

Punishment acquisition

Mean ± SEM lever pressing during the first five punishment sessions are shown in Figure 2B. Over the course of these sessions, there was a significant effect of lever (punished vs. unpunished) (F(1,12) = 42.8; P < 0.05), and the difference in responding on the levers increased across days (F(1,12) = 21.7; P < 0.05). Across sessions, there was an increase in responding on the unpunished lever (F(1,12) = 62.3; P < 0.05) and a decrease in responding on the punished lever (F(1,12) = 25.4; P < 0.05).

During infusion days, there was no effect of BM on pressing of the punished (F(1,12) < 1; P > 0.05) or unpunished lever (F(1,12) < 1; P > 0.05) (Fig. 2B). Latencies to respond on the punished lever increased (F(1,12) = 64.0; P < 0.05), whereas latencies to respond on the unpunished lever did not change (F(1,12) < 1; P > 0.05) (Fig. 2C). There was no effect of BM infusions on latencies to respond on either the punished or unpunished lever (all F(1,12) < 1.2; P > 0.05).

Punishment expression

Mean ± SEM lever-press ratios, lever-press latencies, and magazine entries during expression are shown in Figure 2D–F. There was a significant main effect of lever: rats responded more on the unpunished lever than the punished lever (F(1,14) = 74.5; P < 0.05). There was no difference in responding between BM and saline tests (F(1,14) = 2.23; P > 0.05) or unpunished lever (F(1,14) < 1; P > 0.05); BM did not significantly alter punished (t(14) = 0.40; P > 0.05) or unpunished (t(14) = 1.07; P > 0.05) lever-press ratios from 0.5 (Fig. 2D).

Rats were significantly slower to respond on the punished lever than the unpunished lever (averaged across trials) (F(1,14) = 29.9; P < 0.05) (Fig. 2E). BM infusion had no significant effect on punished (F(1,14) < 1; P > 0.05) or unpunished latencies (F(1,14) = 1.30; P > 0.05). However, BM into the RAIC was not completely without effect—there were significantly more magazine entries following BM compared with saline (F(1,14) = 5.18; P < 0.05) (Fig. 2F).

Aversive choice

Figure 2G–I shows mean ± SEM lever-press ratios, lever-press latencies, and magazine entries for choice tests. Three rats were excluded from choice analyses due to a program error on their second choice test (remaining n = 12). Rats responded significantly more on the unpunished than the punished lever (F(1,11) = 36.7; P < 0.05). There was no difference in responding between BM and saline for either lever (F(1,11) = 1.41; P > 0.05), but there was a significant drug × lever interaction (F(1,11) = 6.94; P < 0.05). Consistent with this, analyses of lever-press ratios revealed greater punished responding (t(11) = 2.57; P < 0.05) but not unpunished responding (t(11) = −1.34; P > 0.05) following BM (Fig. 2G).

As these choice tests were conducted without shock presentations, it is possible increased punished lever pressing was caused by augmented extinction. To examine this possibility, within-session changes in lever pressing were analyzed. There was no change in punished lever pressing across choice tests (F(1,11) < 1; P > 0.05), and this did not interact with infusion (F(1,11) < 1; P > 0.05). This suggests that there is no extinction of punishment during these choice tests, and that increased pressing of the punished lever during choice did not derive from augmented extinction.

Although there was a trend toward rats pressing the unpunished lever faster than the punished lever, this difference did not reach statistical significance (F(1,11) = 4.12; P = 0.067) (Fig. 2H). There were also no significant effects of BM on lever-press latencies (all F(1,11) < 1.2; P > 0.05) (Fig. 2H). There was a nonsignificant trend toward an increase in magazine entries following infusions of BM into the RAIC (F(1,11) = 3.78; P = 0.078) during aversive choice (Fig. 2I).

Taken together, these data show that RAIC serves a complementary role to OFC in aversive decision-making: whereas OFC appears to guide behavior away from actions that cause aversive outcomes, RAIC is important for learning when this decision-making involves choice.

Experiment 3: role of the prelimbic cortex in punishment

Histology

Twenty rats received bilateral cannulations of the prelimbic cortex (PL). Five rats had misplaced cannulae that did not bilaterally target the PL. These animals were excluded from the analyses, leaving 15 animals. The locations of accurate microinjection cannulae tips in the PL are shown in Figure 3A.

Figure 3.

Figure 3.

Open in a new tab

Effects of prelimbic cortex (PL) inactivations. (A) PL cannula placements as verified by Nissl-stained sections. Black dots represent the ventral point of the cannula tract, indicated on coronal sections adapted from Paxinos and Watson (2007). (B) Mean ± SEM lever-presses on the punished and unpunished levers during the last day of lever-press training (T) and punishment acquisition (sessions 1–5). Arrows indicate days that rats received infusions of either saline (n = 7) or baclofen and muscimol (BM) (n = 8) immediately prior to the session. (C) Mean ± SEM latency to initially press the punished and unpunished levers (averaged across trials) during punishment acquisition. (D) Mean ± SEM lever-press ratios of BM on lever pressing during punishment expression (n = 15). (E) Mean ± SEM latency to initially press the punished and unpunished lever (averaged across trials) during punishment expression after infusions of saline or BM. (F) Mean ± SEM magazine entries during punishment expression after infusions of saline or BM. (G) Mean ± SEM lever-press ratios of BM on lever pressing during aversive choice (n = 15). (H) Mean ± SEM latency to initially press the punished and unpunished levers during choice test after infusions of saline or BM. (I) Mean ± SEM magazine entries during choice test after infusions of saline or BM.

Lever-press training

There was no significant overall difference between saline (n = 7) and BM (n = 8) groups in lever pressing at the end of lever-press training (F(1,13) = 1.3; P > 0.05), no overall difference in responding on the to-be-punished and to-be-unpunished levers (F(1,13) < 1; P > 0.05), and no group × lever interaction (F(1,13) < 1; P > 0.05) (Fig. 3B, data point T).

Punishment acquisition

Mean ± SEM lever pressing during the first five punishment sessions are shown in Figure 3B. Over the course of these sessions, there was a significant effect of lever (punished vs. unpunished) (F(1,13) = 134.2; P < 0.05), and the difference in responding on the levers increased across days (F(1,13) = 116.1; P < 0.05). Across sessions, there was an increase in responding on the unpunished lever (F(1,13) = 95.1; P < 0.05) and a decrease in responding on the punished lever (F(1,13) = 45.4; P < 0.05).

During infusion days (days 1 and 2), there was no effect of BM on punished (F(1,13) < 1; P > 0.05) or unpunished lever (F(1,13) = 1.7; P > 0.05) responses (Fig. 3B). Latencies to respond on the punished lever increased (F(1,13) = 52.1; P < 0.05), whereas latencies to respond on the unpunished lever did not change (F(1,13) < 1; P > 0.05) (Fig. 3C). There was no effect of BM on these latencies for either the punished or unpunished lever (all F(1,13) < 1; P > 0.05).

Punishment expression

Mean ± SEM lever-press ratios, lever-press latencies, and magazine entries during expression are shown in Figure 3D–F. There was a significant main effect of lever, such that rats responded more on the unpunished than the punished lever (F(1,14) = 172.7; P < 0.05). There was no difference in responding between BM and saline tests for either lever (all F(1,14) < 1; P > 0.05); BM did not significantly alter punished (t(14) = 1.22; P > 0.05) or unpunished (t(14) = 0.80; P > 0.05) lever-press ratios from 0.5 (Fig. 3D).

Rats were significantly slower to respond on the punished lever than the unpunished lever (F(1,14) = 55.1; P < 0.05) (Fig. 3E). BM had no significant main effect on punished (F(1,14) = 1.8; P > 0.05) or unpunished (F(1,14) = 1.4; P > 0.05) latency. There was also no effect of PL inactivation on magazine entries (F(1,14) = 1.1; P > 0.05) (Fig. 3F).

Aversive choice

Figure 3G–I shows mean ± SEM lever-press ratios, lever-press latencies, and magazine entries for choice tests. Rats responded significantly more on the unpunished lever than the punished lever (F(1,14) = 84.1; P < 0.05). There was no difference in responding between BM and saline tests for the unpunished lever or the punished lever (F(1,14) < 1; P > 0.05); BM did not significantly alter punished (t(14) = −0.11; P > 0.05) or unpunished (t(14) = −1.01; P > 0.05) lever-press ratios from 0.5 (Fig. 3G).

Rats were significantly slower to respond on the punished relative to the unpunished lever (F(1,14) = 7.0; P < 0.05). There was no effect of BM infusions on lever-press latencies (all F(1,14) < 1; P > 0.05) (Fig. 3H). BM into the PL also had no effect on magazine entries during aversive choice (F(1,14) < 1; P > 0.05) (Fig. 3I).

So, taken together, the inactivation of PL PFC had no effect on acquisition or expression of punishment or on aversive choice.

Experiment 4: role of the infralimbic cortex in punishment

Histology

Thirty-three rats received bilateral cannulations of the infralimbic cortex (IL). Fifteen rats had misplaced cannulae, with a majority of these misplacements being too ventral and within the dorsal peduncular cortex (Paxinos and Watson 2007). These animals were excluded from the analyses, leaving 18 animals with correct bilateral placement. The locations of microinjection cannulae tips in the IL are shown in Figure 4A.

Figure 4.

Figure 4.

Open in a new tab

Effects of infralimbic cortex (IL) inactivations. (A) IL cannula placements as verified by Nissl-stained sections. Black dots represent the ventral point of the cannula tract, indicated on coronal sections adapted from Paxinos and Watson 2007. (B) Mean ± SEM lever-presses on the punished and unpunished levers during the last day of lever-press training (T) and punishment acquisition (sessions 1–5). Arrows indicate days that rats received infusions of either saline (n = 7) or baclofen and muscimol (BM) (n = 11) immediately prior to the session. (C) Mean ± SEM latency to initially press the punished and unpunished levers (averaged across trials) during punishment acquisition. (D) Mean ± SEM lever-press ratios of BM on lever pressing during punishment expression (n = 18). (E) Mean ± SEM latency to initially press the punished and unpunished levers (averaged across trials) during punishment expression after infusions of saline or BM. (F) Mean ± SEM magazine entries during punishment expression after infusions of saline or BM. (G) Mean ± SEM lever-press ratios of BM on lever pressing during aversive choice (n = 18). (H) Mean ± SEM latency to initially press the punished and unpunished levers during choice test after infusions of saline or BM. (I) Mean ± SEM magazine entries during choice test after infusions of saline or BM. (*) P < 0.05.

Lever-press training

The mean ± SEM responses on the to-be-punished and to-be-unpunished levers for the last day of training are shown in Figure 4B,C (data point T). There was no significant overall difference between saline (n = 7) and BM (n = 11) groups in lever pressing at the end of lever-press training (F(1,16) < 1; P > 0.05), no overall difference in responding on the to-be-punished and to-be-unpunished levers (F(1,16) < 1; P > 0.05), and no group × lever interaction (F(1,16) < 1; P > 0.05).

Punishment acquisition

Mean ± SEM lever pressing during the first five punishment sessions are shown in Figure 4B. Over the course of these sessions, there was a significant effect of lever (punished vs. unpunished) (F(1,16) = 65.2; P < 0.05), and the difference in responding on the levers increased across days (F(1,16) = 41.1; P < 0.05). Across sessions, there was an increase in responding on the unpunished lever (F(1,16) = 22.4; P < 0.05) and a decrease in responding on the punished lever (F(1,16) = 54.7; P < 0.05).

During infusion days (days 1 and 2), there was no effect of BM on the punished (F(1,16) < 1; P > 0.05) or unpunished lever (F(1,16) = 1.7; P > 0.05) (Fig. 4B). Latencies to respond on the punished lever increased (F(1,16) = 46.3; P < 0.05), whereas latencies to respond on the unpunished lever did not change (F(1,16) < 1; P > 0.05) (Fig. 4C). There was no effect of BM infusions on these latencies (all F(1,16) < 1; P > 0.05).

Punishment expression

Mean ± SEM lever-press ratios, lever-press latencies, and magazine entries during expression are shown in Figure 4D–F. There was a significant main effect of lever, with significantly more responses on the unpunished lever than the punished lever (F(1,17) = 101.4; P < 0.05). There was no difference in responding between BM and saline tests for the punished or unpunished lever (all F(1,17) < 1; P > 0.05); BM did not significantly alter punished (t(17) = 0.05; P > 0.05) or unpunished (t(17) = 1.22; P > 0.05) lever-press ratios from 0.5 (Fig. 2D).

Rats were significantly slower to respond on the punished lever than the unpunished lever (F(1,17) = 48.2; P < 0.05) (Fig. 4E) (averaged across trials). BM infusion into IL had no significant effect on punished (F(1,17) < 1; P > 0.05) or unpunished latencies (F(1,17) < 1; P > 0.05). There was no effect of IL inactivation on magazine entries (F(1,17) = 2.0; P > 0.05) (Fig. 4F).

Aversive choice

Figure 4G–I shows mean ± SEM lever-press ratios, lever-press latencies, and magazine entries for choice tests. Rats responded significantly more on the unpunished than the punished lever (F(1,17) = 70.5; P < 0.05). There was significantly greater overall lever pressing following BM compared with saline (F(1,17) = 5.42; P < 0.05). There was also a drug × lever interaction (F(1,17) = 5.77; P < 0.05), driven by significantly more unpunished lever pressing following BM compared with saline (F(1,17) = 6.58; P < 0.05), while there was no significant change in punished lever pressing following BM (F(1,17) = 1.72; P > 0.05). However, BM did not significantly alter punished (t(17) = 0.598; P > 0.05) or unpunished (t(14) = 1.77; P = 0.095) lever-press ratios from 0.5 (Fig. 4G).

Rats were significantly slower to respond on the punished relative to the unpunished lever (F(1,17) = 11.8; P < 0.05). BM significantly and indiscriminately reduced lever-press latencies (F(1,17) = 4.76; P < 0.05) (Fig. 4H), with no interaction effect of drug and lever (F(1,17) = 2.93; P > 0.05). BM into the IL had no effect on magazine entries during aversive choice (F(1,17) < 1; P > 0.05) (Fig. 4I).

Discussion

Here we examined the roles of the medial and lateral PFC in aversive decision-making in the rat. In all experiments, rats learned to reduce responding on a lever that yielded both reward and occasional mild punishment, and the latencies with which animals responded on the punished lever increased. Responding on a second, unpunished lever increased and latencies to respond on this lever were low. When confronted with a choice between the two levers, but in the absence of any punishment, rats showed a clear preference for the unpunished lever both in terms of total lever-presses and latencies to respond on the two levers. We studied the effects of reversible inactivation of different PFC regions during the acquisition and expression of this aversive instrumental learning and during an aversive choice task. Surprisingly, given its well-documented role in both appetitive instrumental learning and aversive Pavlovian learning, there was little effect of reversible inactivation of medial PFC, either PL or IL. In contrast, there were complementary effects of reversible inactivation of lateral PFC, namely OFC and RAIC.

A primary finding here is that OFC is important for expression of aversive instrumental memories, specifically guiding behavior away from actions that cause aversive outcomes, whereas RAIC is important for aversive decision-making when this involves choice. The selective increase in punished responding following inactivation of the OFC is consistent with previous reports that patients with OFC damage are impaired in tasks involving punishment (Bechara et al. 1999, 2000) and could be used to support the proposed role of OFC in encoding aversive outcomes (Morrison and Salzman 2011; O'Doherty et al. 2001, 2003). However, the contribution of OFC to aversive instrumental learning and decision-making is more complex than simply encoding aversive value. For example, OFC inactivation had no effect on the acquisition of punishment, when learning about such value is critical for performance, and Pelloux et al. (2013) failed to find any effect of pretraining OFC lesions in rats using a task involving punishment of cocaine seeking. Moreover, Orsini et al. (2015) reported increased sensitivity in rats to shock-induced punishment of lever pressing for a large reward and, in monkeys, OFC inactivation augmented the impact of a weak punisher that was otherwise unable to affect behavior (Clarke et al. 2015). This pattern of results is inconsistent with OFC simply encoding outcome value in aversive decision-making tasks. Rather, our results suggest that OFC may contribute to comparison of the relative values of reward versus punishment or the cost versus benefit of responding during punishment (Shiba et al. 2016). Consistent with this, we observed an increase in magazine entries concurrent with the increase in punished lever pressing, possibly indicating greater reward salience relative to punishment salience after OFC inactivation. Moreover, choice lever pressing, when the punisher was absent, was unaffected by OFC inactivations. This profile of performance is consistent with OFC as a locus for matching incentive information to guide behavior (Levy and Glimcher 2012). It is likely that this role of OFC in punishment relates to its strong reciprocal connectivity with BLA (Carmichael and Price 1995; Ghashghaei and Barbas 2002), which we have shown previously to encode the aversive value of the punisher in this task (Jean-Richard and McNally 2015)

The RAIC has been strongly implicated in multiple aspects of aversion-coding (Coghill et al. 1994; Flynn et al. 1999; Simmons et al. 2004, 2006; Preuschoff et al. 2008; Franciotti et al. 2009; Furlong et al. 2010; Menon and Uddin 2010; Hayes and Northoff 2011; Wiech and Tracey 2013) and behavioral control (Cai et al. 2014; Ghahremani et al. 2015). Despite this, RAIC inactivations had no effect on either the acquisition of expression of punishment, suggesting that RAIC is not required for appropriate suppression of a response that causes an aversive shock. Rather, RAIC inactivation selectively increased responding on the previously punished lever during a choice task. This suggests that RAIC is involved in avoiding a previously punished response under conditions of explicit choice and/or when the punisher is omitted. This latter possibility is not attributable to punishment extinction. This role in choice is unique among the brain regions we have examined in this task (PL, IL, RAIC, OFC, BLA, lateral habenula; Jean-Richard and McNally 2014, 2015) and is consistent with the proposed roles for RAIC in encoding relative response value (Talmi and Pine 2012) and/or retrieving representations of outcome value (Balleine and Dickinson 2000; Parkes and Balleine 2013).

The lack of effect of PL or IL inactivations on punishment acquisition, expression, and aversive choice suggests that neither PL nor IL is required for punishment learning or behavior. These infusions were not without effect though: IL infusions of BM nondiscriminately increased lever pressing during choice test and also reduced the latencies with which animals responded on the levers. Moreover, similar and lower doses of BM into PL or IL attenuate lever-press suppression in other behavioral paradigms (Bossert et al. 2011; Fuchs et al. 2007; Peters et al. 2008, 2009), while this same manipulation of BLA attenuates punishment suppression using the current protocol (Jean-Richard and McNally 2015). Thus, it is difficult to attribute null results to an ineffective inactivation procedure or task insensitivity. This broad lack of effect on punished responding is consistent with Pelloux et al. (2013) finding that neither PL nor IL lesions affected punishment of responding for cocaine. However, it is also somewhat surprising given that PL is implicated in formation of response–outcome associations (Corbit and Balleine 2003; Ostlund and Balleine 2005; Tran-Tu-Yen et al. 2009) and the expression of aversive Pavlovian associations (Blum et al. 2006; Vidal-Gonzalez et al. 2006; Corcoran and Quirk 2007; Sierra-Mercado et al. 2011; Lammel et al. 2012), whereas IL has been implicated in reductions of instrumental responding following extinction (Peters et al. 2008, 2009; Marchant et al. 2010; Peters and De Vries 2013). Our results suggest that these roles for PL and IL in behavioral control do not extend to aversive instrumental (response-punisher) associations. An important caveat on this conclusion, and the lack of effect of PL and IL manipulations, is that we studied primary punishment via a mild and occasional footshock. We did not study conditioned punishment produced by a cue that signaled shock. This distinction may be important given the roles for PL and IL in regulating Pavlovian fear responses to such cues.

In conclusion, these results show that lateral PFC, notably OFC and RAIC, has complementary functions in aversive instrumental learning and punishment, with OFC important for expression of aversive instrumental memories and guiding behavior away from actions that cause aversive outcomes and RAIC important for aversive decision-making when this involves choice and/or when punishment is absent. In marked contrast, there was no evidence here for a role of medial PFC, namely IL and PL, in these processes.

Materials and Methods

Subjects

Subjects were 91 experimentally naive male Sprague-Dawley rats (260–470 g) obtained from a commercial supplier (Animal Resources Centre). Rats were housed in groups of four in plastic cages and maintained on a 12 h light–dark cycle (lights on at 7:00a.m.). The procedures used were approved by the Animal Care and Ethics Committee at the University of New South Wales and were conducted in accordance with the National Institutes of Health (NIH) Guide for the Care and Use of Laboratory Animals (NIH Publications No. 80–23, revised 1996).

Apparatus

All behavioral training was conducted in a set of eight identical experimental chambers (24 cm [length] × 30 cm [width] × 21 cm [height]; Med Associates Inc.). Each chamber was enclosed in sound- and light-attenuating cabinets (55.9 cm [length] × 35.6 cm [width] × 38.1 cm [height]) and fitted with fans for ventilation and background noise. The chambers were made up of a Perspex rear-wall, ceiling and hinged front-wall, and stainless steel sidewalls. The chamber floors were made of stainless steel rods (4 mm in diameter) spaced 15 mm apart. Each chamber stood 35 mm above a tray of corncob bedding.

A recessed magazine (3 cm in diameter) within a 4 × 4 cm hollow in the right-side chamber wall received 45 g grain pellets from an external automatic hopper. Infrared photocells detected entries into the magazine. There were two retractable levers on the same right-side chamber wall, on either side of the magazine. The punisher was a 0.5 sec, 0.5 mA footshock delivered through the grid floor. All chambers were connected to a computer with Med-PC IV software (Med Associates), which controlled lever, pellet, and shock presentations and recorded the lever-presses and magazine entries.

General procedure

Surgery

Rats were anaesthetized with 1.3 mL/kg ketamine (100 mg/mL; Ketapex; Apex Laboratories) and 0.2 mL/kg muscle relaxant, xylazine (20 mg/mL; Rompun; Bayer) (i.p.) and placed in stereotaxic apparatus (Model 900, Kopf), with the incisor bar maintained at ∼3.3 mm below horizontal to achieve a flat skull position. Twenty-six gauge guide cannulas (6 mm in length; Plastics One, Virginia) were implanted bilaterally according to the coordinates AP: +3.25, ML: ±0.75, DV: −3.3 mm from bregma when targeting the PL (experiment 1; n = 20), AP: +3.1, ML: ±0.5, DV: −4.7 mm from bregma when targeting the IL (experiment 2; n = 33), AP: +4.5, ML: ±1.7, DV: −3.9 mm from bregma (at a 10° angle, laterally) when targeting the OFC (experiment 3; n = 18) and AP: +2.65, ML: ±3.7, DV: −5.1 mm from bregma when targeting the RAIC (experiment 4; n = 20) (Paxinos and Watson 2007). The guide cannulae were fixed in position with dental cement and jeweller's screws. Dummy cannulae were kept in the guide at all times except during microinjections. Rats were allowed to recover for a minimum of 5 d prior to the start of the experimental procedure.

Lever-press training

Commencing at least 5 d after surgery and persisting for the duration of the experiment, rats received daily access to 10–15 g of food and unrestricted access to water in their home cages. Three days after commencement of this feeding schedule, rats were placed in the experimental chambers for 30 min to acclimatize and were then given lever-press training, which consisted of two levers (left and right) being extended and reinforced with grain pellets on a fixed ratio-1 (FR-1) schedule for 1 h, or until each lever had been pressed 25 times each (each lever would retract after 25 presses). Houselights were on throughout the session. All rats received another day of lever-press training, and any rats that did not acquire lever pressing were manually shaped until lever pressing was acquired. All rats were then given 7 d of lever-press training. Levers were presented individually in an alternating pattern so that one lever was extended for 5 min while the other lever was retracted. This prevented response competition between the two levers. After 5 min, the extended lever was retracted and the retracted lever was extended, such that each lever was always presented on its own. This alternation occurred throughout the 40-min session. Both levers were reinforced with a pellet on a VI 30-sec schedule.

Punishment

For sessions 1–8, rats were trained and tested in the punishment task. Punishment sessions were identical to acquisition sessions, except that a designated lever was also punished with a 0.5 sec, 0.5 mA footshock on an FR-10 schedule. The same lever (left or right) was designated as “punished” throughout the experiment for each rat (whether left or right was punished was counterbalanced between rats). Immediately before the first 2 d of punishment, rats received bilateral infusions of 0.9% phosphate-buffered saline or of the GABA agonists baclofen and muscimol (BM; 1 mM baclofen, 0.1 mM muscimol; Sigma–Aldrich) to assess the role of the target region in the acquisition of punishment. For microinjections, a 33-gauge microinjection cannula (Plastics One) was inserted into the guide cannula and connected to a 10-µL glass syringe (Hamilton Company) operated by an infusion pump (World Precision Instruments). The microinjection cannula projected a further 1 mm ventral to the tip of the guide cannula. Drugs were infused at a rate of 0.25 µL/min over 2 min, and the microinjection cannula was left in place for a further 1 min to permit diffusion of the injectate. Rats also received bilateral infusions of either saline or BM on days 6 and 7 (counterbalanced within subject) to test for the effect of PFC region inactivation on expression of punishment.

Choice test

Rats received a choice test for sessions 9 and 11. This involved both levers being extended for 30 min. Responses on either lever were rewarded on a VI 60-sec schedule such that pressing only one lever or both levers over the course of the session yielded no benefit. No shocks were delivered. Rats were tested twice, once after bilateral infusions of BM and once after infusions of saline (within subject, counterbalanced). Between the two choice tests, rats received a reminder punished session, under the same conditions as the previous punished sessions, to reduce any effects the initial nonpunished choice session might have had on performance or lever preference.

Histology

At the end of the experiment, the rats were injected i.p. with sodium pentobarbital (100 mg/kg) and their brains were removed. Unfixed brains were quickly frozen and sectioned coronally (40 µm) through the PFC using a cryostat (Microm 560). Each section was collected and subsequently stained with cresyl violet for histological examination. The boundaries of PFC regions were determined according to Paxinos and Watson (2007).

Data analysis

The dependent measures were punished and unpunished lever-presses, and latency to lever-press. Between × within-subjects ANOVAs were used to analyze lever-press training and punishment acquisition data, with lever (punished vs. unpunished) and day (for punishment acquisition, using linear contrasts) as the within-subjects factors, and drug group (saline vs. BM) as the between-subjects factor. Within-subjects ANOVAs were used to analyze lever-presses and lever-press latencies for punishment expression and aversive choice. In these analyses, lever (punished vs. unpunished) was one within-subjects factor and infusion (saline vs. BM) was the other. Lever-press ratios were analyzed using a one-sample t-test, using 0.5 (no change in lever pressing after BM compared with after saline) as the test value. Within-session choice data were analyzed using a within-subjects ANOVA, with infusion (saline vs. BM) as one factor, and a linear contrast for each minute of the session as another. For all analyses, type I error rate (α) was controlled at 0.05.

Acknowledgments

These experiments were supported by a grant from the Australian Research Council (DP130100310) to G.P.M. P.J.R.D.B. was supported by a UNSW International Research Scholarship and G.P.M. was supported by a Future Fellowship (FT120100250) from the Australian Research Council.

Footnotes

References

  1. Arana FS, Parkinson JA, Hinton E. 2003. Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J Neurosci 23: 9632–9638. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Balleine BW, Dickinson A. 1998. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37: 407–419. [DOI] [PubMed] [Google Scholar]
  3. Balleine BW, Dickinson A. 2000. The effect of lesions of the insular cortex on instrumental conditioning: evidence for a role in incentive memory. J Neurosci 20: 8954–8964. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bechara A, Damasio H, Damasio AR, Lee GP. 1999. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J Neurosci 19: 5473–5481. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bechara A, Damasio H, Damasio AR. 2000. Emotion, decision making and the orbitofrontal cortex. Cereb Cortex 10: 295–307. [DOI] [PubMed] [Google Scholar]
  6. Blum S, Hebert AE, Dash PK. 2006. A role for the prefrontal cortex in recall of recent and remote memories. Neuroreport 17: 341–344. [DOI] [PubMed] [Google Scholar]
  7. Bolles RC, Holtz R, Dunn T, Hill W. 1980. Comparisons of stimulus learning and response learning in a punishment situation. Learn Motiv 11: 78–96. [Google Scholar]
  8. Bossert JM, Stern AL, Theberge FR, Cifani C, Koya E, Hope BT, Shaham Y. 2011. Ventral medial prefrontal cortex neuronal ensembles mediate context-induced relapse to heroin. Nat Neurosci 14: 420–422. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cai W, Ryali S, Chen T, Li CSR, Menon V. 2014. Dissociable roles of right inferior frontal cortex and anterior insula in inhibitory control: evidence from intrinsic and task-related functional parcellation, connectivity, and response profile analyses across multiple datasets. J Neurosci 34: 14652–14667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carmichael ST, Price JL. 1995. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J Comp Neurol 363: 615–641. [DOI] [PubMed] [Google Scholar]
  11. Clarke HF, Horst NK, Roberts AC. 2015. Regional inactivations of primate ventral prefrontal cortex reveal two distinct mechanisms underlying negative bias in decision making. Proc Natl Acad Sci 112: 4176–4181. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Coghill RC, Talbot JD, Evans AC, Meyer E, Gjedde A, Bushnell MC, Duncan GH. 1994. Distributed processing of pain and vibration by the human brain. J Neurosci 14: 4095–4108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Corbit LH, Balleine BW. 2003. The role of prelimbic cortex in instrumental conditioning. Behav Brain Res 146: 145–157. [DOI] [PubMed] [Google Scholar]
  14. Corcoran KA, Quirk GJ. 2007. Activity in prelimbic cortex is necessary for the expression of learned, but not innate, fears. J Neurosci 27: 840–844. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Courtin J, Bienvenu TCM, Einarsson EÖ, Herry C. 2013a. Medial prefrontal cortex neuronal circuits in fear behavior. Neuroscience 240: 219–242. [DOI] [PubMed] [Google Scholar]
  16. Courtin J, Chaudun F, Rozeske RR, Karalis N, Gonzalez-Campo C, Wurtz H, Abdi A, Baufreton J, Bienvenu TCM, Herry C. 2013b. Prefrontal parvalbumin interneurons shape neuronal activity to drive fear expression. Nature 505: 1–17. [DOI] [PubMed] [Google Scholar]
  17. Do-monte FH, Quiñones-Laracuente K, Quirk GJ. 2015. A temporal shift in the circuits mediating retrieval of fear memory. Nature 519: 1–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Flynn FG. 1999. Anatomy of the insula functional and clinical correlates. Aphasiology 13: 55–78. [Google Scholar]
  19. Franciotti R, Ciancetta L, Della Penna S, Belardinelli P, Pizzella V, Romani GL. 2009. Modulation of α oscillations in insular cortex reflects the threat of painful stimuli. Neuroimage 46: 1082–1090. [DOI] [PubMed] [Google Scholar]
  20. Fuchs RA, Eaddy JL, Su ZI, Bell GH. 2007. Interactions of the basolateral amygdala with the dorsal hippocampus and dorsomedial prefrontal cortex regulate drug context-induced reinstatement of cocaine-seeking in rats. Eur J Neurosci 26: 487–498. [DOI] [PubMed] [Google Scholar]
  21. Furlong TM, Cole S, Hamlin AS, McNally GP. 2010. The role of prefrontal cortex in predictive fear learning. Behav Neurosci 124: 574–586. [DOI] [PubMed] [Google Scholar]
  22. Ghahremani A, Rastogi A, Lam S. 2015. The role of right anterior insula and salience processing in inhibitory control. J Neurosci 35: 3291–3292. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Ghashghaei HT, Barbas H. 2002. Pathways for emotion: interactions of prefrontal and anterior temporal pathways in the amygdala of the rhesus monkey. Neuroscience 115: 1261–1279. [DOI] [PubMed] [Google Scholar]
  24. Goodall G, Mackintosh NJ. 1987. Analysis of the pavlovian properties of signals for punishment. Q J Exp Psychol B 39: 1–23. [PubMed] [Google Scholar]
  25. Hayes DJ, Northoff G. 2011. Identifying a network of brain regions involved in aversion-related processing: a cross-species translational investigation. Front Integr Neurosci 5: 49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Jean-Richard-dit-Bressel P, McNally GP. 2014. The role of the lateral habenula in punishment. PLoS One 9: e111699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Jean-Richard-Dit-Bressel P, McNally GP. 2015. The role of the basolateral amygdala in punishment. Learn Mem 22: 128–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lacroix L, Spinelli S, Heidbreder CA, Feldon J. 2000. Differential role of the medial and lateral prefrontal cortices in fear and anxiety. Behav Neurosci 114: 1119–1130. [DOI] [PubMed] [Google Scholar]
  29. Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC. 2012. Input-specific control of reward and aversion in the ventral tegmental area. Nature 491: 212–217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Levy DJ, Glimcher PW. 2012. The root of all value: a neural common currency for choice. Curr Opin Neurobiol 6: 1027–1038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Likhtik E, Paz R. 2015. Amygdala-prefrontal interactions in (mal)adaptive learning. Trends Neurosci 38: 1–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Mackintosh NJ. 1983. Conditioning and associative learning. Oxford University Press, Oxford. [Google Scholar]
  33. Marchant NJ, Furlong TM, McNally GP. 2010. Medial dorsal hypothalamus mediates the inhibition of reward seeking after extinction. J Neurosci 30: 14102–14115. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Menon V, Uddin LQ. 2010. Saliency, switching, attention and control: a network model of insula function. Brain Struct Funct 214: 655–667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Milad MR, Quirk GJ. 2002. Neurons in the medial prefrontal cortex signal memory for fear extinction. Nature 420: 70–74. [DOI] [PubMed] [Google Scholar]
  36. Morgan MA, Ledoux JE. 1999. Contribution of ventrolateral prefrontal cortex to the acquisition and extinction of conditioned fear in rats. Neurobiol Learn Mem 72: 244–251. [DOI] [PubMed] [Google Scholar]
  37. Morrison SE, Salzman CD. 2011. Representations of appetitive and aversive information in the primate orbitofrontal cortex. Ann N Y Acad Sci 1239: 59–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. O'Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C. 2001. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci 4: 95–102. [DOI] [PubMed] [Google Scholar]
  39. O'Doherty J, Critchley H, Deichmann R, Dolan RJ. 2003. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci 23: 7931–7939. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Orsini CA, Trotta RT, Bizon JL, Setlow B. 2015. Dissociable roles for the basolateral amygdala and orbitofrontal cortex in decision-making under risk of punishment. J Neurosci 35: 1368–1379. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Ostlund SB, Balleine BW. 2005. Lesions of medial prefrontal cortex disrupt the acquisition but not the expression of goal-directed learning. J Neurosci 25: 7763–7770. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Ostlund SB, Winterbauer NE, Balleine BW. 2009. Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. J Neurosci 29: 8280–8287. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Parkes SL, Balleine BW. 2013. Incentive memory: evidence that the basolateral amygdala encodes and insular cortex retrieves outcome values to guide choice between goal-directed actions. J Neurosci 33: 8753–8763. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Paxinos G, Watson C. 2007. The rat brain in stereotaxic coordinates. Elsevier, New York, NY. [DOI] [PubMed] [Google Scholar]
  45. Paz ULR, Paz R. 2012. Amygdala-prefrontal synchronization underlies resistance to extinction of aversive memories. Neuron 75: 133–142. [DOI] [PubMed] [Google Scholar]
  46. Pelloux Y, Murray JE, Everitt BJ. 2013. Differential roles of the prefrontal cortical subregions and basolateral amygdala in compulsive cocaine seeking and relapse after voluntary abstinence in rats. Eur J Neurosci 38: 3018–3026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Peters J, De Vries TJ. 2013. D-cycloserine administered directly to infralimbic medial prefrontal cortex enhances extinction memory in sucrose-seeking animals. Neuroscience 230: 24–30. [DOI] [PubMed] [Google Scholar]
  48. Peters J, LaLumiere RT, Kalivas PW. 2008. Infralimbic prefrontal cortex is responsible for inhibiting cocaine seeking in extinguished rats. J Neurosci 28: 6046–6053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Peters J, Kalivas PW, Quirk GJ. 2009. Extinction circuits for fear and addiction overlap in prefrontal cortex. Learn Mem 16: 279–288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Preuschoff K, Quartz SR, Bossaerts P. 2008. Human insula activation reflects risk prediction errors as well as risk. J Neurosci 28: 2745–2752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Quirk GJ, Garcia RE, Gonz a lez-Lima F. 2006. Prefrontal mechanisms in extinction of conditioned fear. Biol Psychiatry 60: 337–343. [DOI] [PubMed] [Google Scholar]
  52. Santini E, Quirk GJ, Porter JT. 2008. Fear conditioning and extinction differentially modify the intrinsic excitability of infralimbic neurons. J Neurosci 28: 4028–4036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Schoenbaum G, Roesch M. 2005. Orbitofrontal cortex, associative learning, and expectancies. Neuron 47: 633–636. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK. 2009. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci 10: 885–892. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Senn V, Wolff SBE, Herry C, Grenier F, Ehrlich I, Gründemann J, Fadok JP, Müller C, Letzkus JJ, Lüthi A. 2014. Long-range connectivity defines behavioral specificity of amygdala neurons. Neuron 81: 428–437. [DOI] [PubMed] [Google Scholar]
  56. Shiba Y, Santangelo AM, Roberts AC. 2016. Beyond the medial regions of prefrontal cortex in the regulation of fear and anxiety. Front Syst Neurosci 10: 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Sierra-Mercado D, Padilla-Coreano N, Quirk GJ. 2011. Dissociable roles of prelimbic and infralimbic cortices, ventral hippocampus, and basolateral amygdala in the expression and extinction of conditioned fear. Neuropsychopharmacology 36: 529–538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Simmons A, Matthews SC, Stein MB, Paulus MP. 2004. Anticipation of emotionally aversive visual stimuli activates right insula. Neuroreport 15: 2261–2265. [DOI] [PubMed] [Google Scholar]
  59. Simmons A, Strigo I, Matthews SC, Paulus MP, Stein MB. 2006. Anticipation of aversive visual stimuli is associated with increased insula activation in anxiety-prone subjects. Biol Psychiatry 60: 402–409. [DOI] [PubMed] [Google Scholar]
  60. Talmi D, Pine A. 2012. How costs influence decision values for mixed outcomes. Front Neurosci 6: 146. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Tanaka SC, Balleine BW, O'Doherty JP. 2008. Calculating consequences: brain systems that encode the causal effects of actions. J Neurosci 28: 6750–6755. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Tian S, Huang F, Gao J, Li P, Ouyang X, Zhou S, Deng H, Yan Y. 2011. Ventrolateral prefrontal cortex is required for fear extinction in a modified delay conditioning paradigm in rats. Neuroscience 189: 258–268. [DOI] [PubMed] [Google Scholar]
  63. Tran-Tu-Yen DA, Marchand AR, Pape JR, Di Scala G, Coutureau E. 2009. Transient role of the rat prelimbic cortex in goal-directed behaviour. Eur J Neurosci 30: 464–471. [DOI] [PubMed] [Google Scholar]
  64. Vidal-Gonzalez I, Vidal-Gonzalez B, Rauch SL, Quirk GJ. 2006. Microstimulation reveals opposing influences of prelimbic and infralimbic cortex on the expression of conditioned fear. Learn Mem 13: 728–733. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Wiech K, Tracey I. 2013. Pain, decisions, and actions: a motivational perspective. Front Neurosci 7: 46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Zelinski EL, Hong NS, Tyndall AV, Halsall B, McDonald RJ. 2010. Prefrontal cortical contributions during discriminative fear conditioning, extinction, and spontaneous recovery in rats. Exp Brain Res 203: 285–297. [DOI] [PubMed] [Google Scholar]

Articles from Learning & Memory are provided here courtesy of Cold Spring Harbor Laboratory Press

RESOURCES