Abstract
Although several studies have examined the subcortical circuitry underlying Pavlovian-to-instrumental transfer (PIT), the role of medial prefrontal cortex in this behavior is largely unknown. Elucidating the cortical contributions to PIT will be key for understanding how reward-paired cues control behavior in both adaptive and maladaptive context (i.e., addiction). Here we use bilateral lesions in a rat model to show that infralimbic prefrontal cortex (ilPFC) is necessary for appropriate expression of PIT. Further, we show that ilPFC mediates this effect via functional connectivity with nucleus accumbens shell (NAcS). Together, these data provide the first demonstration that a specific cortico-striatal circuit is necessary for cue-invigorated reward seeking during specific PIT.
Cue-motivated reward seeking often requires integration of instrumental response–outcome (R–O) associations with Pavlovian stimulus–outcome (S–O) associations (Rescorla 1990). Experimentally, this integration can be assessed using the Pavlovian-to-instrumental transfer (PIT) task, which measures the ability of reward-associated stimuli to invigorate instrumental responding, even when the R–O and S–O associations have been acquired independently (Estes 1948; Rescorla and Solomon 1967; Lovibond 1983). PIT tasks can be structured to test the generalized arousing effects of Pavlovian cues on behavior (i.e., general PIT) or to test the transfer of Pavlovian information about a specific reward onto the precise behavior that produces that reward (i.e., specific PIT) (Balleine and Ostlund 2007; Holmes et al. 2010). A growing body of work suggests that the specific form of PIT depends on distinct neural substrates, including the basolateral amygdala (BLA) and nucleus accumbens shell (NAcS) (Corbit and Balleine 2005, 2011). However, in contrast to amygdalo-striatal circuitry, the role of medial prefrontal cortex (mPFC) in these behaviors is unknown. The mPFC integrates affective information and instrumental output to attribute incentive value to stimuli, and is therefore well situated to play a key role in cue-guided behavior (Christakou et al. 2004; Homayoun and Moghaddam 2009). While dorsal regions of mPFC—anterior cingulate cortex (ACC) and prelimbic cortex (plPFC)—are not required for PIT (Cardinal et al. 2003; Corbit and Balleine 2003), a role for the more ventral infralimbic cortex (ilPFC) has not been investigated. IlPFC regulates flexible reward seeking and may facilitate cue-controlled behavior (Barker et al. 2014). IlPFC sends a dense glutamatergic projection to NAcS, a structure that is itself necessary for specific PIT (Corbit and Balleine 2011), and this projection is known to inhibit inappropriate reward seeking (Vertes 2004; Peters et al. 2008, 2009; Bossert et al. 2012). Such behavioral inhibition may also be an important element of specific PIT, as responding is not invigorated by a cue that has been paired with a different reward and response (Cartoni et al. 2013).
We therefore hypothesized that ilPFC and its projection to the NAcS are required for the expression of specific PIT. To directly the necessity of ilPFC for specific PIT, we first performed either bilateral ilPFC lesions (“Bilat”) or bilateral ilPFC sham lesions (1× PBS, “Sham”). In a second experiment, we performed either contralateral ilPFC/NAcS (“Contra”) or ipsilateral ilPFC/NAcS lesions (“Ipsi”). Since ilPFC projections to NAcS are mostly lateralized (Vertes 2004), lesioning ilPFC in one hemisphere and NAcS in the opposite hemisphere functionally disconnects these two structures. Conversely, lesioning both structures in the same hemisphere leaves the opposite hemisphere intact and can therefore be used as a control for noncircuit-specific effects (see Fig. 1). Male Sprague-Dawley rats (275–325 g, Charles River) were anesthetized with sodium pentobarbital (55–65 mg/kg i.p.) and lesions were performed by infusing 0.3 µl NMDA (20 µg/µL) or 1× PBS (2-min injection, 5-min diffusion) into ilPFC (from bregma: AP +3.2, ML ±0.6, DV −5.0) or NAcS (AP +1.5, ML ±0.85, and DV −7.6). Animals were housed in pairs on a 12-h light–dark cycle. Rats were allowed to fully recover after surgery before being food restricted to 90%–92% of their ad libitum weights. All procedures were approved by the Yale University IACUC and are consistent with the NIH Guidelines on the Care and Use of Laboratory Animals.
Animals first underwent 12 d of Pavlovian conditioning, where they learned to associate two distinct cues (tone or white noise, 75–80 dB each) with two different appetitive reinforcers. Specifically, we used two forms of sucrose—45 mg pellet and 0.1 mL 10% solution—to control for the potential effects of taste on behavior. Reinforcers were delivered into a magazine port and head entries into the port were detected by a photocell. Animals next underwent 14 d of instrumental training, where each reinforcer could be earned by pressing either a right or left lever (see Table 1 for details on the behavioral paradigm). On test day, only one lever was presented (either left or right, counterbalanced) and both cues were played in alternating 2-min bins with 2-min intervals. For each animal, one CS was the “same” (i.e., predicted the same reinforcer as the extended lever) and the other CS was “different” (was not associated with the same reinforcer as the extended lever). Lever pressing was recorded throughout the session but not reinforced (i.e., test was conducted in extinction), and same/different PIT scores were calculated by dividing response rates during each CS by response rates during pre-CS intervals.
Table 1.
All cue/reward and lever/reward pairings were counterbalanced, and preliminary analyses showed no significant difference between the two stimuli, levers, or reinforcers (F values <1). Thus, all data are collapsed across these conditions. For test day data, average rates of lever pressing during each condition (pre-, same-, or different-CS) were log transformed to correct for skewness and maintain homogeneity of variance. Repeated-measures ANOVAs were used for all analyses, and significant interactions were followed-up with appropriate post hoc tests.
Bilateral ilPFC lesion did not affect acquisition of Pavlovian or instrumental conditioning (Fig. 2A,B). Both Sham and Bilat animals entered the magazine more when the CS was on than off (Fig. 2A; F(1,17) = 65.186, P < 0.001) and this difference increased over the course of training (day × cue: F(11,187) = 3.23, P < 0.001), with no main effect of group and no significant interactions with group (maximum F(11,187) = 0.881, P = 0.56). Similarly, both groups increased lever presses over the course of training (Fig. 2B; F(13,195) = 13.465, P < 0.001), with no difference between groups (F(1,15) = 2.437, P = 0.139) and no group × day interaction (F(13,195) = 0.302, P = 0.991). Contra and Ipsi animals also showed similar behavior during Pavlovian and instrumental conditioning (Fig. 3A,B). Both groups entered the magazine more when the CS was on (Fig. 3A; F(1,14) = 94.9, P < 0.001) and this difference increased over training (day × cue: F(11,154) = 5.074, P < 0.001), with no main effect of group and no interactions with group (maximum F = 3.844). All groups acquired instrumental conditioning (Fig. 3B; F(13,247) = 18.296, P < 0.000), with no day × group interaction (F(26,246) = 0.493, P = 0.983,) and no effect of group (F(2,19) = 1.683, P = 0.212).
However, bilateral ilPFC lesion significantly disrupted performance on PIT test day. A rmANOVA of log response rates revealed main effects of group (Fig. 2C; F(1,16) = 7.995, P = 0.012) and condition (F(2,32) = 9.935, P < 0.001) and a group × condition interaction (F(2,32) = 3.557, P = 0.040). Sham animals responded more during the same CS (F(2,16) = 10.837, P < 0.001) but Bilat animals did not (F(2,16) = 1.029, P = 0.380). Bilat animals responded more than Shams during pre-CS intervals (t16 = −3.053, P = 0.008) and different-CS intervals (t16 = −3.032, P = 0.008), but not during same-CS intervals (t16 = −1.183, P = 0.254). Response rates for the first pre-CS were also analyzed separately from total responding to evaluate responding at baseline (before any cue presentation) and differed significantly between groups (t15 = −2.52, P = 0.024). Additionally, rmANOVA of lever pressing across the test session, binned into 2-min intervals, revealed main effects of time bin (F(11,176) = 2.129, P = 0.021) and group (F(1,16) = 5.790, P = 0.029), but no time × group interaction (F(11,176) = 1.203, P = 0.288), suggesting that both Bilat and Sham animals decreased lever pressing across the test session, with no significant difference between rates of extinction.
PIT scores were calculated for the same and different cues as described above. rmANOVA revealed a main effect of cue pairing (Fig. 2D, F(1,16) = 8.884, P = 0.009) and a group × cue pairing interaction (F(1,16) = 5.136, P = 0.038), with no main effect of group (F(1,16) = 2.359, P = 0.144). Follow-up analyses revealed a significant effect of CS-type in Sham (paired-sample t-test: t8 = 2.956, P = 0.018) but not Lesion animals (paired-samples t-test: t8 = 0.775, P = 0.460). Presentation of the same CS resulted in a greater invigoration of responding (i.e., higher responding than in pre-CS intervals) for Sham animals (mean PIT score ± SEM = 2.31 ± 0.458) versus Lesion animals (mean PIT score ± SEM = 1.3 ± 0.112).
PIT behavior was also disrupted in Contra versus Ipsi animals. rmANOVA of log response rates revealed main effects of condition (Fig. 3C; F(2,30) = 8.749, P = 0.003) and group (F(1,15) = 192.089, P < 0.001), and a condition × group interaction (F(2,30) = 4.074, P = 0.027). Responding was elevated during presentation of the same CS in Ipsi animals (F(2,14) = 9.966, P = 0.002) but not Contra animals (F(2,16) = 0.573, P = 0.575). Contra animals responded at a higher rate than Ipsi animals during pre-CS intervals (t14 = −2.708, P = 0.017) and different-CS intervals (t15 = −2.392, P = 0.031), but not during same-CS intervals (t15 = −1.072, P = 0.302). As with bilateral lesion, responding was significantly elevated in Contra versus Ipsi animals during the first pre-CS interval (t14 = −2.612, P = 0.02), suggesting that responding was heightened at baseline and not as a result of the first cue presentation. rmANOVA of lever pressing across the test session, binned into 2-min intervals, revealed a main effect of time bin (F(11,154) = 2.813, P = 0.002) and a nonsignificant trend toward an effect of group (F(1,14) = 3.623, P = 0.078), with no time × group interaction (F(11,154) = 0.575, P = 0.874), suggesting that there was no significant of disconnection in the rate of extinction during the transfer test.
A rmANOVA analysis on same and different PIT scores revealed a trend toward a main effect of group (Fig. 3D; F(2,20) = 3.423, P = 0.053) and a significant group × condition interaction (F(2,20) = 3.909, P = 0.037). PIT scores were significantly higher for the same versus different CS in Ipsi animals (t7 = 2.772, P = 0.028) but not in Contra animals (t8 = 0.185, P = 0.858). PIT scores were >1 in the Ipsi (t7 = 3.183, P = 0.015) but not Contra animals (t8 = 0.9,41, P = 0.374), indicating that responding in Contra animals was unaffected by CS presentation.
These results demonstrate that ilPFC and its functional connectivity with NAcS are necessary for the cue-elicited invigoration of instrumental behavior as measured by outcome-specific PIT. Both sham and ipsilaterally lesioned animals exhibited a specific increase in responding in the presence of a cue that was previously associated with the same outcome (i.e., specific PIT). Conversely, animals with either bilateral ilPFC or contralateral ilPFC/NAcS disconnection responded equally during pre-CS, same CS, and different CS intervals, and, interestingly, showed an overall higher rate of responding versus controls that was selective to the PIT test session. Thus, the ilPFC–NAcS circuit is necessary for the invigoration of instrumental behavior by Pavlovian cues, and may mediate this role—at least in part—by providing a necessary inhibitory function.
These results are the first to identify a region of mPFC, the ilPFC, as necessary for PIT. Previous work has shown that lesions of the more dorsal plPFC do not disrupt cue-guided behavior in the outcome-specific PIT task (Cardinal et al. 2003; Corbit and Balleine 2003). Another study, however, showed that plPFC neurons track the degree of behavior invigoration due to general arousal in a way that striatal neurons do not (Homayoun and Moghaddam 2009), suggesting that plPFC does encode some aspects of Pavlovian and instrumental integration. Our results provide further evidence for the distinct roles of of ilPFC and plPFC in cue invigoration of behavior. While these regions have distinct projection targets that often subserve their dissociable functions, they are also highly interconnected (Vertes 2004). It is therefore possible that proposed neural representations of PIT in plPFC are in fact derived from ilPFC input, which itself is the necessary substrate for integrating Pavlovian and instrumental associative learning processes. This hypothesis is supported by extensive ilPFC interconnectivity with structures that encode emotional valence and process visceromotor responses, including the amygdala and NAcS (Sesack et al. 1989; Condé et al. 1995; Vertes 2004).
Indeed, our findings suggest that the ilPFC's interaction with NAcS is key for the expression of outcome-specific PIT. While it is known that NAcS is necessary for this behavior (Corbit and Balleine 2011), the upstream structures mediating this effect were unclear. IlPFC sends a strong glutamatergic projection to the NAcS (Sesack et al. 1989), and evidence suggests that this circuit is necessary for extinction of reward seeking (Peters et al. 2008). While we did not observe a difference in within-session lever pressing during our transfer test (which was conducted in extinction), the overall increased response rates in lesioned and disconnected animals suggests that inhibitory mechanisms were likely impaired. Further, NAcS has a well-known role in cue-induced reinstatement of reward-seeking behavior and is important for discriminative instrumental training (Ambroggi et al. 2011; Bossert et al. 2013). Thus, the interaction of these two regions may similarly drive outcome-specific cue-guided behavior, which combines the incentive motivational impact of cues with inhibition of previously learned R–O relations to produce a selective invigoration of behavior.
Interestingly, animals with both bilateral ilPFC and contralateral ilPFC–NAcS lesions showed a higher overall rate of responding during the PIT test session than controls: they pressed more in both the pre-CS and different CS intervals, while pressing at the same level as controls during the same CS interval. In ilPFC lesioned and ilPFC–NAcS disconnection animals, instrumental behavior was not invigorated by the presentation of reward-paired cues, consistent with a complete lack of specific PIT. While it remains possible that the absence of invigoration of responding may result from a ceiling effect, our finding that all groups responded at higher rates during RI60 training (see Fig. 2B, 3B) suggests that this is not the case. It is also possible that our results reflect a behavioral deficit that is more complex than just a loss of PIT alone. For example, disruption of the ilPFC/NAcS circuit may result in diffuse attribution of incentive motivational properties to all contextual features—including the behavioral chamber and extended levers—rather than discretely attributing incentive motivational properties to the auditory stimuli.
A complimentary interpretation is that the ilPFC, via its connectivity with NAcS, exerts inhibitory control over behavior that is necessary for observing cue-elicited responding in the outcome-specific PIT test. When this circuit is disrupted, action–outcome relations dominate behavior, masking the influence of learned stimulus–outcome relations. The ilPFC is necessary for the acquisition and expression of extinction, or the suppression of reward seeking when the reward is no longer available (Bouton 2004; Millan et al. 2011). It is also known to mediate habitual behavior, as indexed by persistent responding even after reward contingencies or values have been disrupted (Killcross and Coutureau 2003). Thus, the ilPFC can be thought of as actively suppressing action–outcome contingencies in both extinction and habitual behaviors (Barker et al. 2014). In the same way ilPFC suppresses inappropriate goal-directed behavior in extinction and habit paradigms, it may also be suppressing “inappropriate” responding in PIT, i.e., responding during pre-CS and different-CS intervals. Indeed, ilPFC inhibition of responding during these intervals may then interact with the excitatory effects of reward-paired cues that are mediated by subcortical regions, resulting in cue-specific, elevated reward-seeking behavior. In the absence of ilPFC function, this suppression of reward-seeking behavior is lost, resulting in responding that is insensitive to previously learned cues. This is supported by the fact that responding was elevated even during the first pre-CS interval, before any cue presentation. This interpretation is consistent with a recent theoretical model of PIT put forth by Cartoni et al. (2013), which argues that behavioral inhibition is in fact necessary for specific PIT since responding is not affected by a cue that has been paired with a different action and reward. Our findings add to this model that behavioral inhibition is necessary not only in the presence of unpaired cues (the basis for specific PIT), but also at baseline within the reward-associated context. Thus, ilPFC–NAcS disconnection could result in loss of PIT inhibition mechanisms and produce disinhibited responding at baseline and during same and different CS presentations, as we observed here. While the present study was constructed to test the role of this circuit on outcome-specific PIT, future experiments could test whether these inhibitory mechanisms may also be necessary for general PIT.
In summary, our results provide the first demonstration that disconnection of a cortico-striatal circuit can block the expression of outcome-specific incentive motivational properties of a cue on reward seeking. These data expand our understanding of how the ilPFC–NAcS functions as an inhibitory “gate” for motivated reward-seeking behavior. Given the likelihood of a relationship between cue-reactivity and addiction in humans (Bray et al. 2008; Talmi et al. 2008), these results have important implications for improving addiction therapies and broadening our understanding of motivated behavior.
Acknowledgements
This study was supported by PHS P50 AA012870 and RO1 DA011717, and the CT Department of Mental Health.
Footnotes
Article is online at http://www.learnmem.org/cgi/doi/10.1101/lm.038810.115.
References
- Ambroggi F, Ghazizadeh A, Nicola SM, Fields HL. 2011. Roles of nucleus accumbens core and shell in incentive-cue responding and behavioral inhibition. J Neurosci 31: 6820–6830. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Balleine BW, Ostlund SB. 2007. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann N Y Acad Sci 1104: 147–171. [DOI] [PubMed] [Google Scholar]
- Barker JM, Taylor JR, Chandler LJ. 2014. A unifying model of the role of the infralimbic cortex in extinction and habits. Learn Mem 21: 441–448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bossert JM, Stern AL, Theberge FRM, Marchant NJ, Wang H-L, Morales M, Shaham Y. 2012. Role of projections from ventral medial prefrontal cortex to nucleus accumbens shell in context-induced reinstatement of heroin seeking. J Neurosci 32: 4982–4991. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bossert JM, Marchant NJ, Calu DJ, Shaham Y. 2013. The reinstatement model of drug relapse: recent neurobiological findings, emerging research topics, and translational research. Psychopharmacology (Berl) 229: 453–476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouton ME. 2004. Context and behavioral processes in extinction. Learn Mem 11: 485–494. [DOI] [PubMed] [Google Scholar]
- Bray S, Rangel A, Shimojo S, Balleine B, O'Doherty JP. 2008. The neural mechanisms underlying the influence of Pavlovian cues on human decision making. J Neurosci 28: 5861–5866. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cardinal RN, Parkinson JA, Marbini HD, Toner AJ, Bussey TJ, Robbins TW, Everitt BJ. 2003. Role of the anterior cingulate cortex in the control over behavior by Pavlovian conditioned stimuli in rats. Behav Neurosci 117: 566–587. [DOI] [PubMed] [Google Scholar]
- Cartoni E, Puglisi-Allegra S, Baldassarre G. 2013. The three principles of action: a Pavlovian-instrumental transfer hypothesis. Front Behav Neurosci 7: 153. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Christakou A, Robbins TW, Everitt BJ. 2004. Prefrontal cortical-ventral striatal interactions involved in affective modulation of attentional performance: implications for corticostriatal circuit function. J Neurosci 24: 773–780. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Condé F, Maire-Lepoivre E, Audinat E, Crépel F. 1995. Afferent connections of the medial frontal cortex of the rat. II. Cortical and subcortical afferents. J Comp Neurol 352: 567–593. [DOI] [PubMed] [Google Scholar]
- Corbit LH, Balleine BW. 2003. The role of prelimbic cortex in instrumental conditioning. Behav Brain Res 146: 145–157. [DOI] [PubMed] [Google Scholar]
- Corbit LH, Balleine BW. 2005. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. J Neurosci 25: 962–970. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Corbit LH, Balleine BW. 2011. The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J Neurosci 31: 11786–11794. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Estes WK. 1948. Discriminative conditioning; effects of a Pavlovian conditioned stimulus upon a subsequently established operant response. J Exp Psychol 38: 173–177. [DOI] [PubMed] [Google Scholar]
- Holmes NM, Marchand AR, Coutureau E. 2010. Neuroscience and biobehavioral reviews. Neurosci Biobehav Rev 34: 1277–1295. [DOI] [PubMed] [Google Scholar]
- Homayoun H, Moghaddam B. 2009. Differential representation of Pavlovian-instrumental transfer by prefrontal cortex subregions and striatum. Eur J Neurosci 29: 1461–1476. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Killcross S, Coutureau E. 2003. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex 13: 400–408. [DOI] [PubMed] [Google Scholar]
- Lovibond PF. 1983. Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process 9: 225–247. [PubMed] [Google Scholar]
- Millan EZ, Marchant NJ, McNally GP. 2011. Extinction of drug seeking. Behav Brain Res 217: 454–462. [DOI] [PubMed] [Google Scholar]
- Peters J, LaLumiere RT, Kalivas PW. 2008. Infralimbic prefrontal cortex is responsible for inhibiting cocaine seeking in extinguished rats. J Neurosci 28: 6046–6053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Peters J, Kalivas PW, Quirk GJ. 2009. Extinction circuits for fear and addiction overlap in prefrontal cortex. Learn Mem 16: 279–288. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rescorla RA. 1990. The role of information about the response-outcome relation in instrumental discrimination learning. J Exp Psychol Anim Behav Process 16: 262–270. [PubMed] [Google Scholar]
- Rescorla RA, Solomon RL. 1967. Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning. Psychol Rev 74: 151–182. [DOI] [PubMed] [Google Scholar]
- Sesack SR, Deutch AY, Roth RH, Bunney BS. 1989. Topographical organization of the efferent projections of the medial prefrontal cortex in the rat: an anterograde tract-tracing study with Phaseolus vulgaris leucoagglutinin. J Comp Neurol 290: 213–242. [DOI] [PubMed] [Google Scholar]
- Talmi D, Seymour B, Dayan P, Dolan RJ. 2008. Human Pavlovian–instrumental transfer. J Neurosci 28: 360–368. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vertes RP. 2004. Differential projections of the infralimbic and prelimbic cortex in the rat. Synapse 51: 32–58. [DOI] [PubMed] [Google Scholar]