Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Oct 13.
Published in final edited form as: Neuroscience. 2010 Jul 16;170(2):468–477. doi: 10.1016/j.neuroscience.2010.07.002

External incentives and internal states guide goal-directed behavior via the differential recruitment of the nucleus accumbens and the medial prefrontal cortex

Justin M Moscarello 1, Osnat Ben-Shahar 1, Aaron Ettenberg 1
PMCID: PMC2933310  NIHMSID: NIHMS223998  PMID: 20638448

Abstract

Goal-directed behavior is governed by internal physiological states and external incentives present in the environment (e.g., hunger and food). While the role of the mesocorticolimbic dopamine (DA) system in behavior guided by environmental incentives has been well studied, the effect of relevant physiological states on the function of this system is less understood. The current study examined the role of the medial prefrontal cortex (mPFC) and the nucleus accumbens (NAcc) in the kind of food-reinforced behaviors known to be sensitive to the internal state produced by food deprivation conditions. Operant lever-press reinforced on fixed ratio 1 (FR1) and progressive ratio (PR) schedules was tested after temporary inactivation of, or DA receptor blockade in, the prelimbic mPFC or NAcc core of rats with differing levels of food deprivation (0, 12 and 36-h). Food deprivation increased PR breakpoints, as well as the number of lever-presses emitted on the FR1 schedule. Both temporary inactivation and DA blockade of NAcc reduced breakpoints across deprivation conditions, while temporary inactivation and DA blockade of mPFC reduced breakpoints only in food-deprived rats. Neither manipulation of mPFC and NAcc had any effect on behavior reinforced on the FR1 schedule. Thus, mPFC and NAcc were differentially relevant to the behaviors tested – NAcc was recruited when the behavioral cost per reinforcer was rising or high regardless of food deprivation conditions, while mPFC was recruited when food-deprived animals behaved through periods of sparse reinforcement density in order to maximize available gain.

Keywords: mesolimbic dopamine, GABA agonists, food-reinforced responding, food deprivation, operant behavior, motivation

Introduction

A large empirical literature implicates the mesocorticolimbic dopamine (DA) system in goal-directed behavior, with its component structures subserving different functions within this broad framework. The nucleus accumbens (NAcc) is thought to play a role in positive reinforcement and appetitive forms of learning (e.g., Ettenberg, 1989; Kelley, 2004; Wise, 2004, 2008), as well as in the motivation to seek reinforcing outcomes and in the allocation of behavioral effort (Berridge, 1996, 2007; Robinson & Beridge, 1993; Salamone et al., 1994; 2009; Salamone & Correa, 2002). The medial prefrontal cortex (mPFC), on the other hand, is thought to integrate task-relevant information used to make cost/benefit decisions in the pursuit of a goal or reinforcer (Bechara et al, 1994, 1999, 2005; Floresco et al., 2008; Walton et al., 2007). Thus, the mesocorticolimbic DA system guides behavior toward biologically relevant goal states via the complex interplay of reinforcement, motivation and decision-making processes.

These differing theoretical perspectives on the function of mPFC and NAcc share a common behavioral focus, emphasizing how behavior is organized around the distribution of reinforcers or environmental incentives. However, it has long been observed that relevant internal/physiological states act in conjunction with the reinforcement structure of an animal’s environment in order to produce goal-directed behavior (Bindra, 1969, 1974; Hodos, 1961; Hull, 1943). In spite of this, the vast majority of experiments examining the behavioral roles of mPFC and NAcc energize goal-directed action by inducing the same physiological state in all subjects (often by means of food deprivation/restriction), thus holding this variable constant across conditions. As such, the role of internal states is largely ignored in the behavioral analysis of the mesocorticolimbic DA system, even though these states have a profound impact on the output of the very behaviors this system is thought to govern.

Our laboratory has sought to fill this gap in the literature. We have previously reported that feeding-induced activity in NAcc and mPFC was modulated by food deprivation, suggesting that the mesocorticolimbic DA system processes information about deprivation conditions (Moscarello et al., 2007a). The current study was designed to extend our previous results by examining the effects of temporary inactivation and DA receptor blockade within prelimbic mPFC and NAcc core on food-reinforced behavior occurring after three periods of food deprivation (0, 12 or 36-hrs). The prelimbic and core subregions (of mPFC and NAcc, respectively) were chosen for their demonstrated relevance to operant responding (Bari & Pierce, 2005; Bezzina et al., 2008; Coutureau et al., 2009; Floresco et al., 2006, 2008). These manipulations were tested against both fixed ratio 1 (FR1) and progressive ratio schedules of reinforcement. FR1 schedules reinforce every single operant behavior emitted, whereas PR schedules of reinforcement require an animal to pay out ever-rising work costs for each successive reinforcer earned; both are sensitive to food deprivation/restriction (Aberman & Salamone, 1999; Hodos, 1961). Thus, alterations in internal state (deprivation level) and external factors (reinforcement schedule) were tested with respect to the mesocorticolimbic substrates widely thought to be involved in the generation of goal-directed behavior.

Materials and Methods

Subjects

The subjects were 127 male Sprague-Dawley rats obtained from Charles Rivers Laboratories (Hollister, CA) weighing 310 – 350 g at the beginning of each experiment. Animals were gentled through daily handling on each of 5 days prior to the initiation of testing. The rats were housed in pairs in hanging plastic tubs located within a temperature controlled vivarium (22°C) maintained on a 12:12 light-dark cycle (lights on at 0700h). Subjects had ad libitum access to food prior to food deprivation and ad libitum access to water in their home cages throughout the experiment. All procedures were executed in strict accordance with the NIH Guide for the Care and Use of Laboratory Animals and were reviewed and approved by the UCSB Institutional Animal Care and Use Committee.

Operant Chambers

Lever press training and testing occurred in 8 standard operant conditioning chambers (29 cm W X 25 cm L X 30 cm H; Med Associates, St. Albans, VT). Each chamber was equipped with one non-retractable (inactive) and one retractable (active) lever, both positioned 7.0 cm above the floor of the chamber. A pellet trough was situated 2.0cm above the chamber floor between the two levers and was connected to a food dispenser that was located outside the operant chamber. Each chamber was equipped with a 2.8 W house light. Equipment control and data collection were achieved via a desktop computer running Med Associates software (MED-PC for Windows).

Behavioral Training

For operant conditioning, all animals were placed on a restricted diet that reduced their body weight to 85% of free feeding levels. Then, during daily one-hour sessions, subjects were trained to lever press for sweetened 45 mg food pellets (Noyes) on an FR1 schedule of reinforcement. Animals were considered to be performing to criterion when they earned >100 reinforcers during each of four consecutive training sessions. Once this occurred, the training phase of the experiment was ended and rats again received ad libitum access to food for at least 5 days, during which all animals regained their free-feeding weight.

Surgery

Each rat was stereotaxically implanted with bilateral intracranial guide cannulae (22 gauge; Plastics One, Roanoke, Virginia, USA) aimed at either the nucleus accumbens core or the prelimbic medial prefrontal cortex. Deep anesthesia was induced by continuous inhalation of isoflurane gas (4% for induction and 1.5%–2.5% for maintenance). Each animal was also administered single injections of 0.04 mg/kg IM atropine to prevent respiratory congestion and 2.0 mg/kg SC of the non-opiate analgesic flunixin meglumine (FluMeglumine; Phoenix Pharmaceuticals, Belmont, California, USA), to control post-surgical pain. The intracranial guide cannulae were aimed at the prelimbic mPFC or the NAcc core using the following coordinates relative to bregma: for mPFC, AP = + 2.7, ML = +/− .75, DV = − 2.5 from skull surface; for NAcc, AP = +1.0, ML = +/− 1.5, DV = − 5.5 from skull surface (Paxinos and Watson, 2005). The injection cannulae used for drug infusion (described below) protruded 1 mm beneath the tip of the guide cannulae. An obdurator was inserted into each guide cannula to seal the opening and thereby maintain cannula patency and reduce the risk of infection. Finally, before returning animals to their home cages for recovery, each subject was injected (SC) with 3.0 ml of 0.9% physiological saline to prevent dehydration. Animals were permitted to recover from surgery for at least 7 days prior to the initiation of testing.

Drugs

To achieve temporary inactivation of prelimbic mPFC and NAcc core, these regions were infused with a mixed solution of the GABA A agonist muscimol, and the GABA B agonist baclofen (Sigma Aldrich), prepared in a vehicle solution of 0.9% physiological saline. Each drug was initially prepared in a separate solution of 500ng/μL so that the combination of the two drugs produced a single solution in which both drugs were at a concentration of 250ng/μL. An injection volume of 0.3μL/side was used, thus allowing the infusion of 75ng/drug/side. This means of targeted inactivation has been successfully employed by others investigating the functional significance of a number of brain structures, including prelimbic mPFC and NAcc core (Floresco et al., 2006, 2008; McFarland & Kalivas, 2001).

To block DAergic function in prelimbic mPFC and NAcc core, these regions were infused with a solution of the D1/D2 antagonist, cis-flupenthixol, prepared in a vehicle solution of 0.9% physiological saline at a concentration of 50μg/μL. Injection volume was 0.3μL/side resulting in a dose of 15μg/side. Previous reports demonstrate that this dose is behaviorally effective when administered to both prelimbic mPFC and NAcc core (Naniex et al., 2009; Simmons & Neill, 2009).

Procedure

This study involved four separate experiments. The design of each experiment was identical with regard to training and behavioral testing, and varied with regard to the brain area tested (prelimbic mPFC or NAcc core) and the pharmacological agents utilized (muscimol/baclofen [M/B] or cis-flupenthixol [cis-flu]). Thus, one experiment examined the behavioral effects of M/B delivered into mPFC, and one the effects of M/B administered into NAcc; another experiment tested the effects of cis-flu infused into mPFC, and yet another the effects of cis-flu injected into NAcc.

At the outset of all four experiments, animals were food restricted and brought to 85% of their baseline weight, trained to lever press and then returned to their free feeding weight as described above. Training was followed by surgical implantation of guide cannulae into either mPFC or NAcc and then a 7 day recovery period (again, as above). The rats in each experiment were distributed across three groups, each corresponding to one of three periods of food deprivation (0, 12, or 36-h; n = 8/group, and thus 24/experiment). Food deprivation occurred during the animals’ night cycle, and food was removed immediately prior to lights out for animals in the 12 and 36-h conditions. 0-h animals received ad-libitum access to food while in the home-cage. All animals underwent four test sessions in the operant chambers; each session was preceded by 0, 12 or 36-h of food deprivation, depending on group. Subjects were allowed 5 days of ad libitum access to food between each test session.

Each animal was tested twice on an FR1 and twice on a PR schedule, so as to examine the effects of drug and vehicle on behavior reinforced on both schedules; the two reinforcement schedules were presented in a counterbalanced order. The PR schedule was defined using the following equation: 5e (reinforcer number * .20) −5, rounded to the nearest integer (Richardson & Roberts, 1996), which produced the following response ratios as the session advanced: 1, 2, 4, 9, 12 … 50, 62, 77, 95, 118 … etc. Failure to complete a given ratio within 30 min resulted in the termination of the session. The FR1 sessions were 1 h long, as in the training phase of each experiment.

Immediately prior to each session, animals were fitted with a bilateral injection cannula (which protruded 1 mm beyond the tip of the guide cannula) and slowly infused with 0.3μL of vehicle or drug solution directly into mPFC or NAcc over a 3 min period. After each injection, an additional minute was allowed for the solution to diffuse away from the cannulae tip, after which time the injection cannula was removed, the obdurator replaced, the animals put directly into the operant chambers, and the session initiated. Each animal received four intracranial injections: two infusions of vehicle and two infusions of drug in a counterbalanced order, so that the effects of treatment (either cis-flu or M/B administered directly into either mPFC or NAcc) could be tested on both the FR1 and PR schedules of reinforcement.

Perfusion & Histology

After completion of behavioral testing, all animals were deeply anesthetized and transcardially perfused with a 10% formalin solution. Brains were removed and sliced in 40-micron frozen sections prior to being mounted and stained with Cresyl Violet. Cannula placements were verified using Paxinos and Watson (2005) as a guide.

Results

Cannula Placement

Figure 1 shows the regions of mPFC and Fig 2 the regions of NAcc that contained all correctly placed cannulae. All animals whose bilateral cannula placement was revealed to be incorrect by histological examination were removed from subsequent statistical analysis. 18 animals from the NAcc experiments and 13 animals from the mPFC experiments were thus disqualified, leaving 96 viable subjects and n=8 for all experimental groups tested.

Figure 1.

Figure 1

Open in a new tab

Cannula placement within the prelimbic mPFC. Figures represent coronal slices containing mPFC, with the distance from Bregma (in mm) indicated below each slice. Circles represent muscimol/baclofen injection sites, whereas squares represent injection sites for cis-flupenthixol. From The Rat Brain in Stereotaxic Coordinates, Paxinos & Watson (2007). Adapted with permission.

Figure 2.

Figure 2

Open in a new tab

Cannula placement within NAcc core. Figures represent coronal slices containing NAcc, with the distance from Bregma (in mm) indicated below the slices. Circles represent muscimol/baclofen injection sites, whereas squares represent injection sites for cis-flupenthixol. From The Rat Brain in Stereotaxic Coordinates, Paxinos & Watson (2007). Adapted with permission.

Inactivation of prelimbic mPFC

In order to understand the effects of temporary inactivation of mPFC on behavior reinforced on a PR schedule, a mixed design two-way ANOVA was conducted on animals’ breakpoints (number of ratios completed by session’s end), with a within-subjects factor of Drug (M/B or vehicle treatment) and a between-subjects factor of Group (0, 12 or 36-h food deprivation). Figure 3-A depicts the effects of food deprivation and M/B treatment on breakpoints. ANOVA revealed a significant main effect for both Group [F(2, 21) = 3.466, p<.05] and Drug [F(1, 21) = 3.319, p<.05]. Subsequent Tukey’s HSD post-hoc tests on the between-subjects factor revealed a significant difference between 0 and 36-h groups (p<.05), 0 and 12-h groups (p<.05), as well as 12 and 36-h groups (p<.05). There was also a significant Drug × Group interaction [F(2, 21) = 5.517, p<.05]. Two-tailed T-tests confirmed that this interaction was driven by significant differences between vehicle and M/B treatment condition in the case of animals from the 12-h group [t(12)=2.9, p<.05] and the 36-h group [t(12)=3.1, p<.05], but not the 0-h group. To ask whether or not food deprivation had any effect on animals under the influence of mPFC M/B, a simple effects analysis was conducted on breakpoint data from the M/B-treatment sessions. No significant differences were found. In summary, then, temporary inactivation of mPFC selectively reduced break points only in food-deprived animals, prompting food-deprived rats to behave comparably to non-deprived controls.

Figure 3.

Figure 3

Open in a new tab

Operant behavior occurring under different periods of food deprivation after intra-mPFC administration of either muscimol/baclofen (M/B; left column) or cis-flupenthixol (cis-flu; right column). Panel A -- Mean (+SEM) break-points on a progressive ratio schedule of food-reinforced responding during inactivation of mPFC: * indicates a significant difference between M/B and vehicle treatments in both 12 and 36-h groups (p<.05). Panel B -- Mean (+SEM) break-points in animals during DA antagonism of mPFC: * indicates a significant difference between cis-flu and vehicle treatments in both 12 and 36-h groups (p<.05). Panel C -- Mean (+SEM) lever-presses for a FR1 schedule of reinforcement during mPFC inactivation: + indicates a significant difference between 36-h and both 0 and 12-h groups (p<.05). Panel D -- Mean (+SEM) lever-presses during mPFC DA antagonism: + indicates a significant difference between 36-h and both 0 and 12-h groups (p<.05).

The effects of inactivation of mPFC were also tested on an FR1 schedule of reinforcement (Figure 3-C). A mixed design, two-way ANOVA computed on number of lever presses emitted by each animal during the one hour session revealed a significant main effect for Group [F(2,21) = 3.029, p<.05] only. Subsequent Tukey’s HSD post-hoc tests revealed a significant difference between 0 and 36-h (p<.05), as well as 12 and 36-h groups (p<.05). No main effect was found for Drug, nor was there a Group × Drug interaction. Thus, while lever-press performance on an FR1 schedule was sensitive to the effects of food deprivation, temporary inactivation of mPFC had no behavioral impact.

DA antagonism in prelimbic mPFC

A mixed design two-way ANOVA was conducted on data from animals performing on a PR schedule and subjected to mPFC DA blockade. Figure 3-B displays the effects of food deprivation and mPFC cis-flu treatment on breakpoints. This analysis revealed a significant main effect for both Group [F(2, 21) = 3.281, p < .05] and Drug [F(1,21) = 3.677, p < .05]. Subsequent Tukey’s HSD post-hoc tests uncovered a significant difference between 0 and 36-h groups (p<.05), 0 and 12-h groups (p<.05), as well as 12 and 36-h groups (p<.05). There was also a significant Drug × Group interaction [F(2,21) = 2.918, p < .05]. Two-tailed T-tests confirmed that this interaction was driven by significant differences between vehicle and cis-flu treatment in animals from the 12-h group [t(12)=3.0, p<.05] and 36-h group [t(12)=2.8, p<.05], but not the 0-h group. To ask whether or not food deprivation had any effect on animals administered cis-flu into mPFC, a simple effects analysis was conducted on breakpoint data from the cis-flu treatment sessions. No significant differences were found. Thus, DA blockade in mPFC produced comparable effects to those observed during M/B-induced activation – both treatments selectively reduced break points only in food-deprived animals, rendering theses subjects non-responsive to the effects of increasing levels of food deprivation.

The effects of food deprivation and cis-flu infused into mPFC on behavior reinforced on an FR1 schedule are depicted in Figure 3-D. A mixed design, two way ANOVA revealed a significant effect only for Group [F(2, 21) = 3.031, p < .05]. Subsequent Tukey’s HSD post-hoc tests revealed a significant difference between 0 and 36-h (p<.05), as well as 12 and 36-h groups (p<.05). No effects were observed for Drug, nor was there a Group × Drug interaction. Again, as observed during M/B inactivation of the mPFC, microinjection of the DA antagonist did not alter the subjects’ responding on the FR1 schedule of reinforcement, which remained sensitive to escalating levels of food deprivation.

Inactivation of NAcc core

Figure 4-A illustrates the effects of food deprivation and temporary inactivation of NAcc core on break-points obtained from animals behaving on a PR schedule of reinforcement. A mixed design, two-way ANOVA computed on these data revealed a significant main effect for Group [F(2, 21) = 2.714, p < .05] with progressively higher levels of deprivation producing reliably higher break points. Tukey’s HSD post-hoc tests confirmed significant differences between animals 0 and 12-h deprived (p<.05), between animals 0 and 36-h deprived (p<.05), and animals 12 and 36-h deprived (p<.05). ANOVA also revealed a significant main effect for Drug [F(1, 21) = 3.332, p<.05], although no significant Group × Drug interaction was found. Thus, temporary inactivation of NAcc reduced breakpoints in every group regardless of the animal’s motivational state.

Figure 4.

Figure 4

Open in a new tab

Operant behavior occurring under different periods of food deprivation after intra-NAcc (core) administration of either muscimol/baclofen (M/B; left column) or cis-flupenthixol (cis-flu; right column). Panel A -- Mean (+SEM) break-points on a progressive ratio schedule of food-reinforced responding during inactivation of NAcc: * indicates a significant difference between M/B and vehicle treatments in each of the three deprivation groups (p<.05). Panel B -- Mean (+SEM) break-points in animals during DA antagonism of NAcc core: * indicates a significant difference between cis-flu in all three deprivation groups (p<.05). Panel C -- Mean (+SEM) lever-presses for a FR1 schedule of reinforcement during NAcc inactivation: + indicates a significant difference between 36-h and both 0 and 12-h groups (p<.05). Panel D -- Mean (+SEM) lever-presses during NAcc DA antagonism: + indicates a significant difference between 36-h and both 0 and 12-h groups (p<.05).

The effects of food deprivation and NAcc inactivation were also tested on an FR1 schedule of reinforcement (Figure 4-C). Here, mixed design two-way ANOVA revealed a significant main effect for Group [F(2,21) = 3.204, p<.05] alone. Subsequent Tukey’s HSD post-hoc tests revealed a significant difference between 0 and 36-h (p<.05), as well as 12 and 36-h groups (p<.05). There was no main effect for Drug, nor was there a Drug × Group interaction. Thus, while performance on the FR1 schedule was sensitive to the effects of food deprivation, temporary inactivation of NAcc had no effect on operant behavior in this context.

DA antagonism in NAcc core

Figure 4-B depicts the impact of food deprivation and DA blockade in NAcc core on breakpoints of animals behaving on a PR schedule of reinforcement. Mixed design two-way ANOVA revealed significant main effects for Group [F(2, 21) = 2.714, p < .05] and Drug [F(1, 21) = 3.021, p<.05], but no Group × Drug interaction. Tukey’s HSD post-hoc tests confirmed significant differences in the performance of the 0 and 12-h deprived groups (p<.05), the 0 and 36-h deprived groups (p<.05), and the 12 and 36-h deprived groups (p<.05). The main effect for cis-flu treatment in the absence of any interaction with deprivation conditions suggests that DA blockade in NAcc reduced breakpoints regardless of motivational state.

The effects of food deprivation and NAcc DA antagonism on behavior reinforced on an FR1 schedule are shown in Figure 4-D. Mixed design two-way ANOVA revealed a significant main effect only for Group [F(2, 21) = 2.655, p < .05]; there was no reliable effect of Drug nor a Group × Drug interaction. Tukey’s HSD post-hoc tests confirmed significant differences between 0 and 36-h derived groups (p<.05) and the 12 and 36-h derpived groups (p<.05). So, once again, while performance on the FR1 schedule was sensitive to the effects of food deprivation, DA blockade in NAcc had no appreciable effect on operant responding in this context.

Discussion

As previously demonstrated by us, 12 and 36-h of food deprivation increased breakpoints, or number of ratios completed before the end of the session, relative to non-deprived controls in animals reinforced with food on a progressive ratio (PR) schedule (Moscarello et al, 2009). The drug treatments employed had different effects on breakpoints depending on whether they were applied to the prelimbic medial prefrontal cortex (mPFC) or the nucleus accumbens (NAcc) core. Both temporary inactivation with muscimol/baclofen (M/B) and D1/D2 receptor blockade with cis-flupenthixol (cis-flu) reduced breakpoints when administered directly to NAcc core, regardless of deprivation condition. In contrast, M/B and cis-flu applied to prelimbic mPFC reduced breakpoints as a function of the animal’s prior period of food deprivation. When behavior was reinforced on an FR1 schedule, food deprivation produced a relative increase in the number of lever-presses emitted, but only in animals 36-h deprived. Interestingly, neither M/B nor cis-flu applied to NAcc core or prelimbic mPFC produced behavioral changes relative to vehicle in the FR1 task. Thus, mPFC and NAcc were differentially relevant to the behaviors tested, recruited as a function of reinforcement schedule in the case of NAcc, and as a combined function of reinforcement schedule and food deprivation conditions in the case of mPFC.

The FR1 sessions controlled for a number of important variables, in particular reinforcer valuation and motor function. That drug-treated animals behaved vigorously for reinforcement on an FR1 schedule suggests that our manipulations left intact the subjects’ ability to assess food with its appropriate motivational valence. These results are commensurate with a body of evidence indicating that the directional aspects of motivation (i.e., the ability to appropriately identify an environmental stimulus as a goal object) are unperturbed by manipulations of NAcc (Aberman & Salamone, 1999; de Bochgrave et al, 2002; Salamone et al., 1994). These data further suggest that prelimbic mPFC is equally uninvolved in assigning reinforcing stimuli with their appropriate valence. The FR1 sessions also served as an important motor control. The statistically indistinguishable levels of behavior observed during drug and vehicle treatment conditions indicate that these manipulations of mPFC and NAcc produced no motor impediments. Thus, these results suggest that the prelimbic mPFC and NAcc core are critical for neither reinforcer valuation nor motor function.

One possibility is that our manipulations of mPFC and NAcc reduced breakpoints by disrupting the brain mechanisms of reinforcement learning. Indeed, animals were only exposed to the PR schedule during testing, and a body of evidence suggests that glutamatergic and DAergic signaling in both mPFC and NAcc are required for the acquisition of a new operant behavior (Baldwin et al., 2000, 2002; Smith-Roe & Kelley, 2000). When applied to mPFC, M/B and cis-flu only affected breakpoints in deprived animals; if these manipulations of mPFC prevented the animals from learning about the contingency structure of the PR schedule, then they should have reduced breakpoints in all subjects, regardless of deprivation condition. Thus, disrupted learning seems an unlikely explanation for our mPFC results.

Microinjection of M/B and cis-flu into NAcc, however, did reduce breakpoints in all groups. Though this observation may seem to support a learning-based interpretation of our results, further analysis makes this interpretation unlikely. Firstly, it has been demonstrated that lesions of NAcc (de Borchgrave et al., 2000) and of NAcc core selectively (Corbit et al., 2001) produce no effects in a contingency degradation paradigm, suggesting that NAcc is uninvolved in encoding changes in previously acquired action-outcome associations. Secondly, an effect of disrupted learning would likely produce a pattern of behavior different from what we observed. Many theories of reinforcement learning contain the notion of a prediction error, which is a teaching signal that occurs when the organism’s expectations are violated – negative prediction errors, for example, occur when outcomes are worse than expected and tend to attenuate the output of conditioned behavior (Pearce & Hall, 1980; Makcintosh, 1975; Sutton & Barto, 1981; Rescorla & Wagner, 1972). In our experiments, the move from an FR1 schedule (in which all operant behaviors are reinforced) to a PR schedule (in which reinforcement is increasingly sparse) would be expected to produce just such a negative prediction error. Blocking that signal in animals behaving on a PR schedule, then, would produce an increase in reinforced behavior by preventing the associated inhibitory feedback. Instead, the directionally opposite effect was observed – drug treatment decreased breakpoints across deprivation conditions. It seems reasonable to conclude that while NAcc and mPFC may be involved in the initial acquisition of action-outcome associations, these structures are not required for an animal to learn alterations in previously acquired instrumental contingencies.

A body of scholarship implicates NAcc DA in the expenditure of effort, which can be defined behaviorally as the observed willingness of a subject to perform an instrumental behavior on a high work-requirement schedule of reinforcement (Caul & Brindle, 2001; Salamone & Correa, 2002; Salamone et al., 2009). In general, animals behaving on these schedules will gradually increase their rate of behavior across sessions, presumably to maximize available reinforcement (Aberman & Salamone, 1999; Hamill et al., 1999). In the experiments reported here, NAcc was necessary for operant performance when changing contingencies required an increasingly high behavioral cost for each subsequent reinforcer (i.e. the PR schedule). Though animals did not necessarily behave at a higher rate during the PR trials (subjects reinforced on this schedule usually take several sessions to accelerate their rate of response; Aberman et al., 1998; Hamill et al., 1999), it seems that the escalating cost of effort/behavioral energy required for each reinforcer earned was sufficient to engage NAcc, as well as the DA projection to that structure. Thus, our data are commensurate with the notion of NAcc DA as a crucial mediator of effort functions.

Our NAcc data are also commensurate with those derived from a series of forced-choice lever-press paradigms (for review see Floresco et al., 2008). In these tasks, animals are required to decide between reinforcers of differing magnitudes that can be obtained with different degrees of effort (Ghods-Sharifi & Floresco, 2010), or with different probabilities of reinforcer delivery (Cardinal & Howes, 2005), or with different delays prior to receipt of reinforcement (Cardinal et al., 2001). Intact/control animals generally choose larger reinforcers delivered at a high cost of effort, or with some risk of receiving no reinforcement, or after a delay; NAcc lesions (Cardinal et al., 2001; Cardinal & Howes, 2005) or inactivations of NAcc (Ghods-Sharifi & Floresco, 2010) reverse this trend, prompting animals to choose a lower magnitude reinforcer that is administered absent these costs (it is worthwhile to note, however, that while NAcc DA lesions impact effort and risk functions, they do not affect delay discounting; Winstanely et al, 2005). From the perspective of the animal, the PR schedule described here involves not only an increasing cost of effort for each reinforcer, but also increasing uncertainty and delay in reinforcer delivery. Discounting occurs when the value of an outcome is diminished by some associated cost – such as effort, uncertainty, or delay (Floresco et al, 2008; Walton et al, 2007). It seems that NAcc is recruited to behavior in these circumstances of discounted net value, in which the continued pursuit of individual reinforcers necessitates the incursion of costs. Conversely, NAcc is unnecessary for instrumental action when individual reinforcers are relatively undiscounted. Commensurate with these data derived from animals, a number of recent imaging studies have found that activity in the human NAcc varies as a combined function of reinforcement conditions and a number of discounting factors, including effort (Botvinick et al., 2008; Croxson et al., 2009), risk (Levy et al., 2010), and delay (Kable & Glimcher, 2007).

NAcc seems to be specifically concerned with the investment of metabolic energy in the face of discounted reinforcer value, as behavior in a ‘fail point’ task, in which animals are required to wait in a stationary position for longer and longer periods for each subsequent reinforcer, is unperturbed by manipulations of NAcc function (Wakabayashi et al., 2004). Note that even in conditions of delay discounting experimental animals are asked to invest behavioral energy in the form of an operant response, even though the factor that discounts the reinforcer is time (Cardinal et al., 2001). Given that the output regions of NAcc are classical basal ganglia structures (Mogenson et al., 1980; O’Donnell et al., 1997; Zhou et al., 2003), this area is well-situated to exert influence over the substrates of motor control. Thus, NAcc seems to be involved in the implementation of cost/benefit decisions, increasing the probability that the organism will emit goal-directed action in the pursuit of a reinforcing outcome discounted by some behavioral obstacle.

A number of studies have assayed the role of rodent frontal cortical regions in effort-based decision-making. Lesions of the anterior cingulate cortex (aCC) shift the subjects’ preference to a lower effort alternative in a T-Maze task (Schweimer & Hauber, 2005; Walton et al., 2003), but seem to have no effect on breakpoints in animals performing on a PR schedule (Schweimer & Hauber, 2005). Interestingly, prelimbic mPFC lesions have no effect on an effort-based T-maze paradigm (Walton et al., 2003), which is also sensitive to manipulations of NAcc and NAcc DA (Cousins et al., 1996; Hauber & Sommer, 2009). These data, in concert with our own, suggest an interesting model of effort-based choice. aCC seems to mediate choices in which the organism must select between mutually exclusive options, while prelimbic mPFC may guide the choice to express effort as a matter of degree, or as a part of a broader behavioral policy (described in greater detail below). NAcc, then, may act as a final common pathway for both forms of choice, which is consistent with the anatomical connections between both of these cortices and ventral regions of the striatum (Berendse et al., 1992).

In studies of human clinical populations, Bechara and colleagues assayed patients with lesions of the ventromedial PFC (encompassing the gyri of the orbital convexity and extending dorsally through the ventral-anterior extent of the cingulate gyrus). Damage to this region of the brain has an interesting effect on the Iowa gambling task, in which subjects either win or lose money depending on cards selected from one of four decks, each associated with different risks and returns. Normal subjects eventually maximize their wins by drawing from the decks associated with smaller individual gains but less overall loss, which is the optimal strategy in this task. However, patients with lesions of the ventromedial PFC always favor the risky decks associated with larger individual wins, but a degree of overall loss that results in a net monetary deficit if consistently chosen (Bechara et al., 1994, 1999, 2005). Thus, damage to the ventromedial PFC prevents subjects from pursuing an approach in which gain is maximized slowly over time, prompting them to opt for an immediate, if risky, pay off. Subsequent work indicates that regions of the orbital frontal cortex are not implicated in this kind of decision-making, though dorolateral regions of PFC may also play a role (Fellows & Farrah, 2005).

The higher ratios of the PR schedule described here required a comparable behavioral strategy on the part of experimental animals. The 13th ratio, for example, required 77 lever presses in order to gain a single reinforcer, forcing the animal to proceed through periods of sparse reinforcement in order to obtain available food resources. Conversely, the FR1 schedule offered immediate reinforcement for each operant behavior emitted. Indeed, while a functioning mPFC was required for food-deprived animals to perform normally on the PR schedule, behavior reinforced on the FR1 schedule was unaffected by manipulations of mPFC. It seems that mPFC is necessary across mammalian taxa for behavioral strategies that maximize reinforcement through the slow accrual of gains, whereas this structure is not required in order for animals to exhibit behaviors that yield an immediate return.

Food deprivation alone is sufficient to engage mPFC in both humans (Tataranni et al., 1999) and rodents (Moscarello et al., 2007a), as well as to increase DA release and utilization in that structure (Carlsson et al., 1987, 1988; Moscarello et al., 2007b). These data, in combination with the results of the current study, suggest that deprivation conditions prompt the organism to remain engaged in the pursuit of a state-relevant goal through periods of sparse reinforcement by recruiting the mPFC. Food deprivation conditions, however, do not always exert their influence on reinforced behavior via the mPFC. When reinforcement is immediately available upon emission of an instrumental behavior, food deprivation remains influential even though mPFC is not required. Thus, deprivation conditions can likely engage a variety of task-relevant brain substrates, rather than acting via a single mechanism.

Data presented here suggest that NAcc core prompts the investment of behavioral effort in the pursuit of some high-cost reinforcer, while prelimbic mPFC kept food-deprived animals task engaged through periods of sparse reinforcement density. As such, it seems reasonable to conclude that NAcc was recruited to behavior by motivational information external to the organism, while mPFC was activated by a relevant physiological state to the extent that the environment required a specifc strategy in order to maximize available resources. Interestingly, both external incentives and internal states guided behavior reinforced on the FR1 schedule, though neither mPFC nor NAcc were behaviorally relevant in that context. Thus, it seems the structure of environmental incentives acts in a complex synergy with internal states to guide behavior via the fluid recruitment of a variety of task-relevant brain structures.

Acknowledgments

This work was supported by an NIH pre-doctoral NRSA fellowship awarded to JMM (F31-DA024505) and a National Institute of Drug Abuse grant awarded to AE (R01-DA05041).

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Aberman JE, Salamone JD. Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement. Neuroscience. 1999;92(2):545–52. doi: 10.1016/s0306-4522(99)00004-4. [DOI] [PubMed] [Google Scholar]
  2. Aberman JE, Ward SJ, Salamone JD. Effects of dopamine antagonists and accumbens dopamine depletions on time constrained progressive ratio performance. Pharmacol Biochem Behav. 1998;61(4):341–8. doi: 10.1016/s0091-3057(98)00112-9. [DOI] [PubMed] [Google Scholar]
  3. Bari AA, Pierce RC. D1-like and D2 dopamine receptor antagonists administered into the shell subregion of the rat nucleus accumbens decrease cocaine, but not food, reinforcement. Neuroscience. 2005;135(3):959–68. doi: 10.1016/j.neuroscience.2005.06.048. [DOI] [PubMed] [Google Scholar]
  4. Baldwin AE, Holahan MR, Sadeghian K, Kelley AE. N-methyl-D-asparate receptor-dependent plasticity within a distributed corticostriatal network mediates appetitive instrumental learning. Behav Neurosci. 2000;114(1):84–98. doi: 10.1037//0735-7044.114.1.84. [DOI] [PubMed] [Google Scholar]
  5. Baldwin AE, Sadeghian K, Kelley AE. Appetitive instrumental learning requires coincident activation of NMDA and dopamine D1 receptors with the medial prefrontal cortex. J Neurosci. 2002;22(3):1063–77. doi: 10.1523/JNEUROSCI.22-03-01063.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bechara A, Damasio AR, Damasio H, Anderson SW. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition. 1994;50(1–3):7–15. doi: 10.1016/0010-0277(94)90018-3. [DOI] [PubMed] [Google Scholar]
  7. Bechara A, Damasio H, Damasio AR, Lee GP. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J Neurosci. 1999;19(13):5473–81. doi: 10.1523/JNEUROSCI.19-13-05473.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bechara A, Van Der Linden M. Decision-making and impulse control after frontal lobe injuries. Curr Opin Neurol. 2005;18(6):734–9. doi: 10.1097/01.wco.0000194141.56429.3c. [DOI] [PubMed] [Google Scholar]
  9. Berridge KC. Food reward: brain substrates of wanting and liking. Neurosci Biobehav, Rev. 1996;20(1):1–25. doi: 10.1016/0149-7634(95)00033-b. [DOI] [PubMed] [Google Scholar]
  10. Berridge KC. The debate over dopamine’s role in reward: the case for incentive salience. Psychopharm. 2007;191:391–431. doi: 10.1007/s00213-006-0578-x. [DOI] [PubMed] [Google Scholar]
  11. Berendse HW, Galis-de Graaf Y, Groenewegen HJ. Topographical organization and relationship with ventral striatal compartments of prefrontal projections in the rat. J Comp Neurol. 1992;316(3):314–47. doi: 10.1002/cne.903160305. [DOI] [PubMed] [Google Scholar]
  12. Bezzina G, Body S, Cheung TH, Hampson CL, Deakin JF, Anderson IM, Szabadi E, Bradshaw CM. Effect of quinolinic acid-induced lesions of the nucleus accumbens core on performance on a progressive ratio schedule of reinforcement: implications for inter-temporal choice. Psychopharmacology. 2008;197(2):339–50. doi: 10.1007/s00213-007-1036-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Bindra D. A unfied interpretation of emotion and motivation. Ann N Y Acad Sci. 1969;159 (3):1071–83. doi: 10.1111/j.1749-6632.1969.tb12998.x. [DOI] [PubMed] [Google Scholar]
  14. Bindra D. A motivational view of learning, performance, and behavior modification. Psychol Rev. 1974;81(3):199–213. doi: 10.1037/h0036330. [DOI] [PubMed] [Google Scholar]
  15. Botvinick MM, Huffstetler S, McGuire JT. Effort discounting in human nucleus accumbens. Cogn Affect Behav Neurosci. 2009;9(1):16–27. doi: 10.3758/CABN.9.1.16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cardinal RN, Howes NJ. Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 2005;6:37. doi: 10.1186/1471-2202-6-37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Cardinal RN, Pennicott DR, Sugathapala CL, Robbins TW, Everitt BJ. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science. 2001;292(5526):2499–501. doi: 10.1126/science.1060818. [DOI] [PubMed] [Google Scholar]
  18. Carlson JN, Glick SD, Hinds PA, Baird JL. Food deprivation alters dopamine utilization in the rat prefrontal cortex and asymmetrically alters amphetamine-induced rotational behavior. Brain Res. 1988;454(1–2):373–7. doi: 10.1016/0006-8993(88)90840-2. [DOI] [PubMed] [Google Scholar]
  19. Carlson JN, Herrick KF, Baird JL, Glick SD. Selective enhancement of dopamine utilization in the rat prefrontal cortex by food deprivation. Brain Res. 1987;400(1):200–3. doi: 10.1016/0006-8993(87)90673-1. [DOI] [PubMed] [Google Scholar]
  20. Caul WF, Brindle NA. Schedule dependent effects of haloperidol and amphetamine: multiple-schedule task shows within subjects effects. Phamacol Biochem Behav. 2001;68(1):53–63. doi: 10.1016/s0091-3057(00)00431-7. [DOI] [PubMed] [Google Scholar]
  21. Corbit LH, Muir JL, Balleine BW. The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell. J Neurosci. 2001;21(9):3251–60. doi: 10.1523/JNEUROSCI.21-09-03251.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Cousins MS, Atherton A, Turner L, Salamone JD. Nucleus accumbens dopamine depletions alter relative response allocation in a T-maze cost/benefit task. Behav Brain Res. 1996;74(1–2):189–97. doi: 10.1016/0166-4328(95)00151-4. [DOI] [PubMed] [Google Scholar]
  23. Coutureau E, Marchand AR, Di Scala G. Goal-directed responding is sensitive to lesions to the prelimbic cortex or basolateral nucleus of the amygdala but not to their disconnection. Behav Neurosci. 2009;123(2):443–8. doi: 10.1037/a0014818. [DOI] [PubMed] [Google Scholar]
  24. Croxson PL, Walton ME, O’Reilly JX, Behrens TEJ, Rushworth MFS. Effort-based cost-benefit valuation and the human brain. J Neurosci. 2009;29(14):4531–41. doi: 10.1523/JNEUROSCI.4515-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. de Borchgrave R, Rawlins JNP, Dickinson A, Balleine BW. Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats. Exp Brain Res. 2004;144:50–68. doi: 10.1007/s00221-002-1031-y. [DOI] [PubMed] [Google Scholar]
  26. Ettenberg A. Dopamine, neuroleptics and reinforced behavior. Neurosci Biobehav Rev. 1989;13(2–3):105–11. doi: 10.1016/s0149-7634(89)80018-1. [DOI] [PubMed] [Google Scholar]
  27. Fellows LK, Farah MJ. Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cereb Cortex. 2005;15(1):58–63. doi: 10.1093/cercor/bhh108. [DOI] [PubMed] [Google Scholar]
  28. Fields HL, Hjelmstad GO, Margolis EB, Nicola SM. Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement. Annu Rev Neurosci. 2007;30:289–316. doi: 10.1146/annurev.neuro.30.051606.094341. [DOI] [PubMed] [Google Scholar]
  29. Floresco SB, Ghods-Sharifi S, Vexelman C, Magyar O. Dissociable roles for the nucleus accumbens core and shell in regulating set shifting. J Neurosci. 2006;26(9):2449–57. doi: 10.1523/JNEUROSCI.4431-05.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Floresco SB, McLaughlin RJ, Haluk DM. Opposing roles for the nucleus accumbens core and shell in cue-induced reinstatement of food-seeking behavior. Neuroscience. 2008;154(3):877–84. doi: 10.1016/j.neuroscience.2008.04.004. [DOI] [PubMed] [Google Scholar]
  31. Floresco SB, St Onge JR, Ghods-Sharifi S, Winstanley CA. Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making. Cogn Affect Behav Neurosci. 2008;8(4):375–89. doi: 10.3758/CABN.8.4.375. [DOI] [PubMed] [Google Scholar]
  32. Gabbott PL, Warner TA, Jays PR, Salway P, Busby SJ. Prefrontal cortex in the rat: projections to subcortical autonomic, motor, and limbic centers. J Comp Neurol. 2005;492(2):145–77. doi: 10.1002/cne.20738. [DOI] [PubMed] [Google Scholar]
  33. Ghods-Sharifi S, Floresco SB. Differential effects on effort discounting induced by inactivations of the nucleus accumbens core and shell. Behav Neurosci. 2010;124(2):179–91. doi: 10.1037/a0018932. [DOI] [PubMed] [Google Scholar]
  34. Hamill S, Trevitt JT, Nowend KL, Carlson BB, Salamone JD. Nucleus accumbens dopamine depletions and time-constrained progressive ratio performance: effects of different ratio requirements. Pharmacol Biochem Behav. 1999;64(1):21–7. doi: 10.1016/s0091-3057(99)00092-1. [DOI] [PubMed] [Google Scholar]
  35. Hauber W, Sommer S. Prefrontal circuitry regulates effort-related decision making. Cereb Cortex. 2009;19:2240–7. doi: 10.1093/cercor/bhn241. [DOI] [PubMed] [Google Scholar]
  36. Heidbreder CA, Groenewegen HJ. The medial prefrontal cortex in the rat: evidence for a dorsal-ventral distinction based upon functional and anatomical characteristics. Neurosci Biobehav Rev. 2003;27(6):555–79. doi: 10.1016/j.neubiorev.2003.09.003. [DOI] [PubMed] [Google Scholar]
  37. Hodos W. Progressive ratio as a measure of reward strength. Science. 1961;134:943–4. doi: 10.1126/science.134.3483.943. [DOI] [PubMed] [Google Scholar]
  38. Hull CL. Principles of Behavior. New York: Appleton-Century-Crofts; 1943. [Google Scholar]
  39. Kable JW, Glimcher PW. The neural correlates of subjective value during intertemporal choice. Nat Neurosci. 2007;10(12):1625–33. doi: 10.1038/nn2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Kelley AE. Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning. Neurosci Biobehav Rev. 2004;27:765–76. doi: 10.1016/j.neubiorev.2003.11.015. [DOI] [PubMed] [Google Scholar]
  41. Levy I, Snell J, Nelson AJ, Rustichini A, Glimcher PW. Neural representation of subjective value under risk and ambiguity. J Neurophysiol. 2010;103(2):1036–47. doi: 10.1152/jn.00853.2009. [DOI] [PubMed] [Google Scholar]
  42. Mackintosh NJ. A theory of attention: variations in the associability of stimuli with reinforcement. Psychol Rev. 1975;82(4):276–98. [Google Scholar]
  43. McFarland K, Kalivas PW. The circuitry mediating cocaine-induced reinstatement of drug-seeking behavior. J Neurosci. 2001;21(21):8655–63. doi: 10.1523/JNEUROSCI.21-21-08655.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Mogenson GJ, Jones DL, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol. 1980;14(2–3):69–97. doi: 10.1016/0301-0082(80)90018-0. [DOI] [PubMed] [Google Scholar]
  45. Moscarello JM, Ben-Shahar O, Ettenberg A. Dynamic interaction between medial prefrontal cortex and nucleus accumbens as a function of both motivational state and reinforcer magnitude: a c-Fos immunocytochemistry study. Brain Res. 2007a;1169:69–76. doi: 10.1016/j.brainres.2007.06.064. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Moscarello JM, Ben-Shahar O, Ettenberg A. Food presentation to hungry rats produces an immediate increase in DA and delayed reductions in GABA and glutamate within the medial prefrontal cortex: relevance to reinforcement and motivation processes. Soc Neurosci Abstr. 2007b program no. 749.18. [Google Scholar]
  47. Moscarello JM, Ben-Shahar O, Ettenberg A. Effects of food deprivation on goal-directed behavior, spontaneous locomotion, and c-Fos immunoreactivity in the amygdala. Behav Brain Res. 2009;197(1):9–15. doi: 10.1016/j.bbr.2008.07.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Naneix F, Marchand AR, Di Scala G, Pape JR, Coutureau E. A role for medial prefrontal dopaminergic innervation in instrumental conditioning. J Neurosci. 2009;29(20):6599–606. doi: 10.1523/JNEUROSCI.1234-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Nicola SM. The nucleus accumbens as part of a basal ganglia action selection circuit. Psychopharmacology. 2007;191(3):521–50. doi: 10.1007/s00213-006-0510-4. [DOI] [PubMed] [Google Scholar]
  50. O’Donnell P, Lavin A, Enquist LW, Grace AA, Card JP. Interconnected parallel circuits between rat nucleus accumbens and thalamus revealed by retrograde transynaptic transport of pseudorabies virus. J Neurosci. 1997;17(6):2143–67. doi: 10.1523/JNEUROSCI.17-06-02143.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Pearce JM, Hall G. A model for Pavlovian learning: variations in the effectiveness of conditioned but not unconditioned stimuli. Psychol Rev. 1980;87(6):532–552. [PubMed] [Google Scholar]
  52. Rescorla RA, Wagner AR. A theory of Pavlovian conditioning. Variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning II: current research and theory. New York: Appleton-Century-Crofts; 1972. pp. 64–99. [Google Scholar]
  53. Richardson NR, Roberts DC. Progressive ratio schedules in drug self-administration studies in rats: a method to evaluate reinforcing efficacy. J Neurosci Methods. 1996;66(1):1–11. doi: 10.1016/0165-0270(95)00153-0. [DOI] [PubMed] [Google Scholar]
  54. Robinson TE, Berridge KC. The neural basis of drug craving: an incentive salience sensitization theory of addiction. Brain Res Rev. 1993;18:247–91. doi: 10.1016/0165-0173(93)90013-p. [DOI] [PubMed] [Google Scholar]
  55. Salamone JD, Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res. 2002;137(1–2):3–25. doi: 10.1016/s0166-4328(02)00282-6. [DOI] [PubMed] [Google Scholar]
  56. Salamone JD, Correa M, Farrar AM, Nunes EJ, Pardo M. Dopamine, behavioral economics, and effort. Front Behav Neurosci. 2009;3:13. doi: 10.3389/neuro.08.013.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Salamone JD, Cousins MS, Bucher S. Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure. Behav Brain Res. 1994;65(2):221–9. doi: 10.1016/0166-4328(94)90108-2. [DOI] [PubMed] [Google Scholar]
  58. Schweimer J, Hauber W. Involvement of the rat anterior cingulated cortex in control of instrumental responses guided by reward expectancy. Learn Mem. 2005;12:334–42. doi: 10.1101/lm.90605. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Simmons DA, Neill DB. Functional interaction between the basolateral amygdala and the nucleus accumbens underlies incentive motivation for food reward on a fixed ratio schedule. J Neurosci. 2009;159(4):1264–73. doi: 10.1016/j.neuroscience.2009.01.026. [DOI] [PubMed] [Google Scholar]
  60. Smith-Roe SL, Kelley AE. Coincident activation of NMDA and dopamine D1 receptors within nucleus accumbens core is required for appetitive instrumental learning. J Neurosci. 2000;20(20):7737–42. doi: 10.1523/JNEUROSCI.20-20-07737.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Sutton RS, Barto AG. Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev. 1981;88(2):135–70. [PubMed] [Google Scholar]
  62. Tataranni PA, Gautier JF, Chen K, Uecker A, Bandy D, Salbe AD, Pratley RE, Lawson M, Reiman EM, Ravussin E. Neuroanatomical correlates of hunger and satiation in humans using positron emission tomography. Proc Natl Acad Sci. 1999;96(8):4569–74. doi: 10.1073/pnas.96.8.4569. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Wakabayashi KT, Fields HL, Nicola SM. Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward. Behav Brain Res. 2004;154(1):19–30. doi: 10.1016/j.bbr.2004.01.013. [DOI] [PubMed] [Google Scholar]
  64. Walton ME, Bannerman DM, Alterecu K, Rushworth MFS. Functional specialization within medial frontal cortex of the anterior cingualte for evaluating effort-related decisions. J Neurosci. 2003;23(16):6475–9. doi: 10.1523/JNEUROSCI.23-16-06475.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Walton ME, Rudebeck PH, Bannerman DM, Rushworth MF. Calculating the cost of acting in frontal cortex. Ann N Y Acad Sci. 2007;1104:340–56. doi: 10.1196/annals.1390.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Winstanely CA, Theobald DEH, Dalley JW, Robbins TW. Interactions between serotonin and dopamine in the control of impulsive choice in rats: therapeutic implications for impulse control disorders. Neuropsychopharmacology. 2005;4:669–82. doi: 10.1038/sj.npp.1300610. [DOI] [PubMed] [Google Scholar]
  67. Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5:483–94. doi: 10.1038/nrn1406. [DOI] [PubMed] [Google Scholar]
  68. Wise RA. Dopamine and reward: the anhedonia hypothesis 30 years on. Neurotox Res. 2008;14(2, 3):169–83. doi: 10.1007/BF03033808. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Zhou L, Furuta T, Kaneko T. Chemical organization of projection neurons in the rat accumbens nucleus and olfactory tubercle. Neuroscience. 2003;120(3):783–98. doi: 10.1016/s0306-4522(03)00326-9. [DOI] [PubMed] [Google Scholar]

RESOURCES