The Medial Orbitofrontal Cortex Regulates Sensitivity to Outcome Value

Abstract

An essential component of goal-directed decision-making is the ability to maintain flexible responding based on the value of a given reward, or “reinforcer.” The medial orbitofrontal cortex (mOFC), a subregion of the ventromedial prefrontal cortex, is uniquely positioned to regulate this process. We trained mice to nose poke for food reinforcers and then stimulated this region using CaMKII-driven G_s-coupled designer receptors exclusively activated by designer drugs (DREADDs). In other mice, we silenced the neuroplasticity-associated neurotrophin brain-derived neurotrophic factor (BDNF). Activation of G_s-DREADDs increased behavioral sensitivity to reinforcer devaluation, whereas Bdnf knockdown blocked sensitivity. These changes were accompanied by modifications in breakpoint ratios in a progressive ratio task, and they were recapitulated in Bdnf^+/− mice. Replacement of BDNF selectively in the mOFC in Bdnf^+/− mice rescued behavioral deficiencies, as well as phosphorylation of extracellular-signal regulated kinase 1/2 (ERK1/2). Thus, BDNF expression in the mOFC is both necessary and sufficient for the expression of typical effort allocation relative to an anticipated reinforcer. Additional experiments indicated that expression of the immediate-early gene c-fos was aberrantly elevated in the Bdnf^+/− dorsal striatum, and BDNF replacement in the mOFC normalized expression. Also, systemic administration of an MAP kinase kinase inhibitor increased breakpoint ratios, whereas the addition of discrete cues bridging the response–outcome contingency rescued breakpoints in Bdnf^+/− mice. We argue that BDNF–ERK1/2 in the mOFC is a key regulator of “online” goal-directed action selection.

SIGNIFICANCE STATEMENT Goal-directed response selection often involves predicting the consequences of one's actions and the value of potential payoffs. Lesions or chemogenetic inactivation of the medial orbitofrontal cortex (mOFC) in rats induces failures in retrieving outcome identity memories (Bradfield et al., 2015), suggesting that the healthy mOFC serves to access outcome value information when it is not immediately observable and thereby guide goal-directed decision-making. Our findings suggest that the mOFC also bidirectionally regulates effort allocation for a given reward and that expression of the neurotrophin BDNF in the mOFC is both necessary and sufficient for mice to sustain stable representations of reinforcer value.

Introduction

An essential component of goal-directed decision-making is assessing the value of a reinforcer and engaging response strategies accordingly. Electrophysiological studies in nonhuman primates and neuroimaging studies in humans suggest that the ventral and medial orbitofrontal cortex (mOFC) encodes the value of rewards during real or hypothetical tasks (Arana et al., 2003; Paulus and Frank, 2003; Padoa-Schioppa and Assad, 2006, 2008; Plassmann et al., 2007; Valentin et al., 2007). Furthermore, mOFC neurons are sensitive to satiety-specific reinforcer devaluation and notably more so than centrolateral OFC neurons (Bouret and Richmond, 2010). This observation is consistent with the perspective that, across species, ventromedial prefrontal cortical structures are concerned with representations of outcome value, as determined by internal means such as hunger/satiety, as well as inhibitory control processes; meanwhile, the lateral OFC integrates multisensory information to build representations of task states to guide optimal response selection strategies (Wallis, 2012; Wilson et al., 2014; Stalnaker et al., 2015; Gourley and Taylor, 2016).

What are the molecular mechanisms regulating mOFC function? One candidate is BDNF. BDNF belongs to a family of structurally and functionally related peptide growth factors and is expressed throughout the brain. BDNF facilitates synaptic transmission (Kang and Schuman, 1995, 1996) and long-term potentiation (LTP) in the adult hippocampus (Figurov et al., 1996; Korte et al., 1996; Patterson et al., 1996). In the cerebral cortex, BDNF regulates AMPA receptor subunit expression, experience-dependent synaptogenesis, and dendritic modeling (McAllister et al., 1995, 1997; Gorski et al., 2003; Genoud et al., 2004; Nakata and Nakamura, 2007). Bdnf^−/− mutant mice do not survive to adulthood, but Bdnf^+/− mutants are viable and grossly normal on multiple tests of “emotionality” and memory (Montkowski and Holsboer 1997; but see Linnarsson et al., 1997; MacQueen et al., 2001; Chourbaji et al., 2004). Nonetheless, hippocampal LTP is impaired in these animals (Korte et al., 1995), indicating that even incomplete loss of BDNF has functional consequences.

To assess the role of mOFC BDNF in reward-related decision-making, we first turned to reinforcer devaluation assays. In both primates and rodents, sensitivity to outcome value can be quantified using these procedures, e.g., allowing mice that have been trained previously to respond for a food reinforcer unconditional access to the food before testing. Sensitivity to outcome value is reflected by diminished responding for the now-devalued food (Colwill and Rescorla, 1986; Dickinson, 1980; Balleine and O'Doherty, 2010). In another task, the progressive ratio (PR) schedule of reinforcement can quantify perceived outcome value using instrumental response requirements that progressively increase with each reinforcer delivery (Hodos 1961). In this case, the highest response/reinforcer ratio—the “breakpoint ratio”—can serve as an indicator of perceived “value.” We report that viral-mediated mOFC-selective Bdnf knockdown decreases behavioral sensitivity to reinforcer devaluation and inflates responding on a PR schedule of reinforcement. Constitutive Bdnf^+/− mice also generate aberrant breakpoints, and mOFC-selective BDNF replacement in these mice rescues behavioral abnormalities and immediate-early gene expression in the downstream striatum.

Goal-directed response selection often involves predicting the consequences of one's actions and the value of future outcomes. Bradfield et al. (2015) recently reported that the mOFC sustains goal-directed behavior by retrieving outcome identity information to guide response strategies when reward value is not immediately observable (e.g., during certain stages of reinforcer devaluation tasks). Our data suggest that this property also helps to gate appropriate effort allocation (as in the PR task) and that the neurotrophin BDNF is a critical molecular substrate mediating these mOFC functions. To further support these perspectives, we additionally report that sensitivity to outcome value and PR schedules of reinforcement can be enhanced by stimulating CaMKII-driven G_s-coupled excitatory designer receptor exclusively activated by designer drugs (DREADDs) in the mOFC. These findings raise the possibility that the mOFC could be a viable target for therapeutic interventions aimed at correcting, normalizing, or enriching goal-directed behaviors. Examples include therapies aimed at bringing about behavioral change in depression or addiction, illnesses commonly characterized by behavioral rigidity and inflexibility.

Materials and Methods

Subjects

Mice included the following: (1) adult male C57BL/6 mice (10–12 weeks old) obtained from Charles River or The Jackson Laboratory; (2) Bdnf^+/− and littermate wild-type controls bred on a C57BL/6 background (The Jackson Laboratory); or (3) adult male mice homozygous for a floxed Bdnf gene (exon V) bred on a mixed BALB/c background (The Jackson Laboratory).

Mutant mice were tested between 10 weeks and 6 months of age, at which point body weights in Bdnf^+/− animals significantly increase (Kernie et al., 2000), even despite food restriction (S.L.G. and J.R.T., unpublished observations). Genotypes were determined by PCR of tail tissue and, in the case of Bdnf^+/− mice, confirmed by postmortem analysis of homogenized hippocampal tissue by BDNF ELISA (25 μl/well; Promega; methods below). Mice were food-restricted during instrumental conditioning, maintaining ∼93% original body weight unless otherwise noted. All tests were conducted during the light phase of the 12 h light cycle (7:00 A.M. lights on). The Yale and Emory University Animal Care and Use Committees approved procedures as appropriate.

Viral vector delivery

AAV8–CaMKII–HA–rM₃D(G_s)–IRES–mCitrine (AAV–G_s–DREADD–mCitrine) or AAV8–CaMKII–GFP (AAV–GFP) viral vectors were generated by the University of North Carolina Viral Vector Core and infused into wild-type mice. Lentiviral vectors expressing GFP or Cre recombinase (Cre) under the CMV promoter were generated by the Emory University Viral Vector Core and infused into floxed Bdnf mice. Mice were anesthetized with ketamine/dexdomitor. With needles centered at bregma, stereotaxic coordinates were located on the leveled skull using a digitized stereotaxic frame. Viral vectors were delivered to +2.3 mm anteroposterior (AP), −2.8 mm dorsoventral (DV), and ±0.1 mm mediolateral (ML; Gourley et al., 2010) in a volume of 0.5 μl/side. The microsyringe remained in place for 5 min after infusion. Mice were sutured and allowed to recover for at least 3 weeks before behavioral testing. After testing, mice were deeply anesthetized and transcardially perfused with 4% paraformaldehyde, then brains were sectioned into 40 μm sections, and mCitrine or GFP was imaged to confirm that infusions infected the mOFC.

Clozapine-N-oxide injection in DREADDs-expressing mice and general experimental design

The DREADDs ligand clozapine-N-oxide (CNO; 1 mg/kg, i.p., in 2% DMSO and PBS; Sigma) was prepared fresh daily and administered immediately before the 30 min prefeeding period for devaluation experiments and then again 30 min before testing for PR experiments. A final injection was delivered 30 min before a 1 h locomotor monitoring test. The 30 min inject-to-test interval allowed CNO time to penetrate the brain. All mice received CNO, which increases the excitability of infected neurons in G_s–DREADDs-expressing mice but does not affect control GFP-expressing neurons (Urban and Roth, 2015).

In these experiments, the acute stress resulting from injection would be expected to reduce sensitivity to reinforcer devaluation (Schwabe and Wolf, 2011), as was indeed observed. Mice were accordingly then trained for four additional sessions and then retested in the devaluation procedure to confirm that mice could ultimately develop sensitivity to reinforcer devaluation with habituation to the injection.

Instrumental response training

Instrumental response training was conducted as reported previously (Gourley et al., 2008a,b). Briefly, experimenters used standard aluminum operant conditioning chambers for mice (16 × 14 × 12.5 cm) controlled by MedPC software and equipped with two or three nose-poke recesses (Med-Associates). Each chamber was housed in a sound-attenuating outer chamber equipped with white noise generator, fan, and house light. A dispenser delivered grain-based food pellets (20 mg; Bio-Serv) into the magazine. Head entries into the one active nose-poke aperture and magazine were detected by photocell. Mice were initially trained to perform the operant response over several 25 min sessions, during which one, two, or three responses resulted in food reinforcement (a variable ratio 2 schedule of reinforcement). Once responding stabilized, experiments proceeded as indicated below. Response rates were compared by two-factor (genotype or treatment × session) ANOVA with repeated measures (RM). In the case of interactions, Tukey's post hoc comparisons were used for all experiments.

Satiety-specific reinforcer devaluation

To investigate behavioral sensitivity to reinforcer devaluation, trained mice were allowed unlimited access to the reinforcer pellets in a clean cage for 30 min immediately before a 15 min probe test conducted in extinction. Response rates were compared by two-factor ANOVA to responding in a session when ad libitum access to the reinforcer pellets had not been available, but rather mice were given regular chow. These “devalued” (pellets) and “value” (chow) sessions were counterbalanced, with one test per day on sequential days.

In one experiment, a main effect of testing order indicated that mice extinguished responding during the second test regardless of devalued or value condition. To address this issue, response rates after the first prefeeding period are compared with those generated on the final day of training, referred to as “baseline.”

PR test

PR testing was conducted in mice trained to nose poke as described above. We then applied a linearly increasing response/reinforcement requirement (1, 5, 9, x + 4 responses/reinforcement). The test ended when no active responses were emitted for 5 min or if the mouse had not “timed out” within 4 h. The highest response/reinforcement ratios achieved—termed breakpoint ratios—were compared by two-factor (genotype × session) RM-ANOVA or t test as appropriate. For mice used in viral vector experiments, this test followed reinforcer devaluation testing to conserve animal usage. In Bdnf^+/− mice, independent groups were used for devaluation and PR tests. Breakpoint ratios >2 SDs outside of the group means were excluded (Gourley et al., 2008a; throughout all experiments: n = 1 lenti-GFP; n = 3 intact wild-type; n = 1 Bdnf^+/− plus vehicle; n = 2 Bdnf^+/− plus BDNF. Additionally, two Bdnf knockdown mice were excluded because of experimenter error.)

The PR task is commonly used as an assay of “reward value.” Measuring post-reinforcement pausing (PRP) in animals with differing breakpoint ratios can, however, determine whether differences in primary motivation can instead account for group differences (Skjoldager et al., 1993). For example, with increased food restriction, presumably increasing “motivation,” PRPs shorten, whereas changes in reinforcer value (e.g., one pellet vs three pellets) do not affect PRPs (Skjoldager et al., 1993). Thus, this metric—the time between magazine head entry after reinforcer delivery and initiation of the next trial—was also compared between wild-type and Bdnf^+/− mice, which experience adult-onset obesity (Kernie et al., 2000). The first, median, and final pauses for each session were extracted and compared by RM-ANOVA. The ratio of responses on the active versus inactive apertures was also compared by two-factor RM-ANOVA to verify that mice distinguished the active from inactive apertures.

BDNF microinfusion

For BDNF microinfusions, mice were experimentally naive Bdnf^+/− and littermate wild-type mice trained to perform the instrumental response as described and then subjected to three PR test sessions. Based on breakpoint ratios, mice were matched and assigned to groups (wild-type plus vehicle, wild-type plus BDNF, Bdnf^+/− plus vehicle, Bdnf^+/− plus BDNF). Mice were then anesthetized with 1:1 2-methyl-2-butanol and tribromoethanol diluted 40-fold with saline. The head was shaved and placed in a digitized stereotaxic frame. The scalp was incised, skin retracted, and bregma and lambda identified. The head was leveled, and recombinant human BDNF (Millipore Bioscience Research Reagents) dissolved in saline (0.4 μg/μl; Gourley et al., 2008b, 2009a, 2012) was infused into the mOFC (+2.3 mm AP, −2.8 mm DV, ±0.1 mm ML; Gourley et al., 2010) in a volume of 0.2 μl over 6 min using a digital coordinate system with $\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$100$}\right.$ mm resolution (David Kopf Instruments).

Pilot experiments indicated that a single infusion of BDNF affects PR responding up to 8 d after infusion, so mice here were allowed 4 d recovery, followed by 3 consecutive days of PR testing, with a single PR session per day. Each animal's breakpoint ratios were averaged, yielding a single value per mouse, which were compared between groups by two-factor (genotype × infusion) ANOVA.

Mice were killed by rapid decapitation immediately after the last session, brains were rapidly sectioned, bilateral needle entry sites were documented, and tissues were frozen on dry ice for Western blot analyses. Additional groups of behaviorally naive wild-type mice were infused and killed 24 h after infusion for Western blot and immunostaining analyses.

Western blotting

A single experimenter dissected frozen tissue punches from the ventromedial prefrontal cortex (vmPFC), dorsal striatum, dorsal hippocampus, and nucleus accumbens (NAc) core and shell using 1.2 and 0.50 mm tissue cores (Fine Science Tools). vmPFC samples were collected with a single midline punch with the tissue core aimed at the rostroventral-most part of the vmPFC, containing the mOFC. Samples likely included both mOFC and ventral prelimbic PFC to generate sufficient protein concentrations for multiple blots. Hence, these samples are referred to as “vmPFC” tissue.

Tissue was sonicated in lysis buffer [75–200 μl: 137 mm NaCl, 20 mm Tris-Hcl, pH 8, 1% igepal, 10% glycerol, and 1:100 Phosphatase Inhibitor Cocktails 1 and 2 (Sigma)] and stored at −80°C. Protein concentrations were determined using a Bradford colorimetric assay (Pierce), and 20 μg of each sample was separated by SDS-PAGE on an 8–16% gradient Tris-glycine gel (Invitrogen). After transfer to nitrocellulose membrane, blots were blocked with 5% nonfat milk for 1 h. The following primary antibodies were used: anti-phosphorylated (p) ERK1/2 (mouse; 1:1000; Cell Signaling Technology), anti-ERK1/2 (rabbit; 1:2000; Cell Signaling Technology), anti-trkB (mouse; 1:1000; BD Biosciences), anti-p-trk (rabbit; 1:500; Cell Signaling Technology), anti-GluR1 (rabbit; 1:500; Millipore Bioscience Research Reagents), and anti-c-fos (rabbit; 1:500, Santa Cruz Biotechnology). Membranes were incubated with primary antibodies at 4°C for 1 h or overnight and then incubated with IRDye 700 Dx Anti-Rb IgG and IRDye 800 Dx Anti-Ms IgG (both 1:5000; Rockland Immunochemicals) for 1 h.

Bands were then quantified using infrared densitometry analysis (Odyssey Infrared Imaging System). Membranes were reprobed with anti-GAPDH, which served as a loading control (mouse; 1:20,000; Advanced Immunochemical). pERK1/2 was normalized to total ERK1/2, which was not changed in any comparison. Infrared values were converted to a percentage of control samples from the same membrane to control for variance between gels. Group means were then compared by t test or two-factor ANOVA as appropriate.

BDNF ELISA

Fresh frozen brain tissue was dissected and homogenized as for Western blotting experiments. BDNF was quantified by ELISA (Promega) in duplicate in accordance with the instructions of the manufacturer except for exclusion of the extraction step. BDNF concentrations were normalized to total protein concentrations in each sample.

pERK1/2 immunostaining

One group of wild-type mice was infused with BDNF or saline as described. Then, 24 h later, brains were harvested and stored in 4% paraformaldehyde for 48 h and then transferred to 30% w/v sucrose before being sectioned into 45 μm sections. Sections were blocked in a PBS solution containing 2% normal goat serum, 1% bovine serum albumin (BSA), and 0.3% Triton X-100 (Sigma) for 1 h at room temperature. Sections were then incubated in primary antibody solution containing 0.3% normal goat serum, 1% BSA, and 0.3% Triton X-100 at 4°C for 48 h. pERK1/2 (1:400; Cell Signaling Technology) served as the primary antibody. Sections were incubated in secondary antibody solution containing 0.5% normal goat serum and 0.3% Triton X-100, with Alexa Fluor 633 (1:200; Life Technologies) serving as the secondary antibody.

Sections were imaged on a Nikon 4550s SMZ18 microscope with settings held constant. Concentric rings 35 μm apart were generated in NIH ImageJ with the center of the first ring positioned at the base of the infusion site. Integrated intensity was measured along the perimeter of each ring to assess the spread of ERK1/2 phosphorylation after BDNF infusion, and values were compared by RM-ANOVA. Each mouse contributed a single value.

Experimental design for experiments using a MAP kinase kinase inhibitor

To evaluate whether suppressing MAP kinase kinase (MEK), a kinase directly upstream of ERK1/2, could regulate PR responding, intact wild-type mice were trained to perform an instrumental response for food reinforcement as described. During this period, mice were also habituated to injection by nightly handling and mock intraperitoneal injection to ensure injections before PR tests did not interfere with performance. Mice were then matched based on reinforcers acquired during training and assigned to either vehicle or PD0184161 (5-bromo-2-[(2-chloro-4-iodophenyl)amino]-N-(cyclopropylmethoxy)-3,4-difluorobenzamide; 30 mg/kg, generously provided by Dr. David Russell, Yale University, New Haven, CT) groups. Because the drug necessitated 100% DMSO for dissolution, mice were maintained at 29–31 g body weight to allow for accurate injection of very small injection volume (30 μl/mouse). In contrast to other experiments, this translates to maintaining mice at ∼100% of their original body weight. PD0184161 was administered within 48 h of dissolution and kept at 4°C when not in use.

Thirty minutes before the PR tests, mice were injected with either drug or vehicle alone. An additional control group was injected at the end of the session to evaluate whether MEK inhibition interfered with post-session memory consolidation. Breakpoint ratios were compared by RM-ANOVA. Three weeks after the last test, mice were tested without drug to identify whether PD0184161 had long-term consequences.

Locomotor monitoring

Ambulation in a clean cage was quantified using the automated Omnitech Digiscan Micromonitor system equipped with 16 photocells or a customized Med-Associates locomotor monitoring system identically equipped with 16 photocells. Mice were food restricted overnight to recapitulate locomotor activity levels during instrumental response training and testing. Consecutive photobeams broken across 60 min were compared by two-factor (group × time bin) RM-ANOVA.

Mice expressing DREADDs and their GFP-expressing counterparts were administered CNO and then placed in the locomotor monitoring chambers 30 min later. Given evidence that the mOFC may regulate repetitive behavior (Ahmari et al., 2013), ambulatory counts were segregated from repetitive interruption of the same photobeam, an indicator of stereotypy-like behavior. Both types of photobeam breaks were compared between groups by RM-ANOVA.

Additional behavioral testing in drug-naive mice

Cued reinforcer delivery.

A naive group of Bdnf^+/− and wild-type mice was used. We added discrete stimuli (a 2 s, 2.9 kHz tone and extinction of the house light during the 2 s between nose-poke response and food pellet delivery) signaling reinforcer delivery. This stimulus was delivered during both training and PR testing. Otherwise, training, testing, and analytic procedures were identical.

Extinction conditioning.

In trained mice, responding in the absence of reinforcement (extinction) was evaluated. Here, the tube connecting the food hopper and magazine was disconnected; daily tests were otherwise identical to those during response training. Response rates were compared by two-factor (group × session) RM-ANOVA.

Sucrose consumption.

Mice were habituated to drinking a 1% (w/v) sucrose (Sigma) solution in place of water for 2 d as part of an experimental protocol that was aimed at evaluating animals' hedonic response to a desirable food product. Mice were next fluid deprived for 4 h, followed by 1 h access to the sucrose solution. The deprivation periods were then extended to 14 and 19 h to habituate mice to water restriction. Finally, each mouse was allowed 1 h access to the solution in its home cage, whereas cage mates were housed in a clean cage in the colony room. Each mouse had 1 h access such that the average deprivation period was 14 h. ANOVA with testing order as the independent measure confirmed no effects on consumption (F < 1). Consumption values were then normalized to body weight and compared by t test. This procedure has been shown previously to be sensitive to other manipulations (Gourley et al., 2008b, 2013a; Gourley and Taylor, 2009).

Results

Stimulation of the mOFC enhances behavioral sensitivity to reinforcer devaluation

We first expressed CaMKII-driven AAV–GFP in the mOFC. GFP was mostly contained within the mOFC (Fig. 1a). Within the downstream dorsal striatum, labeled terminals were confined to the medial-most aspects of the rostral striatum, adjacent to the lateral ventricles (Fig. 1b). This innervation pattern is highly consistent with previous reports (Schilman et al., 2008; Hoover and Vertes, 2011). Innervation of the amygdala was also consistent with previous reports (Hoover and Vertes, 2011; Fig. 1c); relative to projections of the infralimbic and prelimbic cortices (Mcdonald et al., 1996), the mOFC has sparse innervation of the lateral capsular division of the central nucleus of the amygdala (CeA). Rather, projections mostly avoid the CeA, instead innervating the ventral portion of the lateral nucleus of the amygdala and the medial aspects of the basal nucleus (Fig. 1c).

Figure 1.

Chemogenetic stimulation of the mOFC enhances behavioral sensitivity to reinforcer value and PR response requirements. a, Viral vectors expressing GFP were infused into the mOFC, annotated by the anatomical boundaries outlined at right. b, Fluorescing axons were detectable in a stereotyped pattern hugging the medial wall of the dorsal striatum, particularly in the rostral portion highlighted by the gray dashed lines (cf. Schilman et al., 2008). c, Terminals were also detected in the medial compartment of the basal amygdala. The corresponding coronal sections are shown at right. Blue boxes outline the areas shown in the photomicrographs. d, Infection spread for viral vector experiments is represented on images from the Mouse Brain Library (Rosen et al., 2000), with black indicating the largest spread and white the smallest. e, Mice expressing CaMKII-driven AAV–GFP or AAV–G_s–DREADD–mCitrine acquired the instrumental response. Breaks in the response acquisition curves indicate tests for behavioral sensitivity to reinforcer devaluation. f, Mice were fed the reinforcer pellets or regular chow ad libitum before probe tests (devalued and value conditions). Groups did not differ in food consumption. g, Activation of G_s–DREADDs augmented behavioral sensitivity to reinforcer devaluation, decreasing response rates. Meanwhile, control mice did not modify their response patterns. h, A second experience with the reinforcer devaluation procedure and injection stress ultimately resulted in response inhibition in both groups (n = 4–6 per group). i, Activating G_s–DREADDs also reduced breakpoints in a PR test (n = 6–10 per group). j, Locomotor activity was not effected by G_s–DREADDs stimulation. Ambulatory and repetitive photobeam breaks are represented in 5 min bins (left) and 1 h total counts (right). Mice in both instrumental conditioning experiments were tested. Bars and symbols represent means ± SEMs. *p < 0.05, **p < 0.001.

Next, CaMKII-driven AAV–G_s–DREADD–mCitrine or AAV–GFP was infused into the mOFC of separate mice. Fluorescence distribution is represented in Figure 1d, with the majority of infusions selective to the mOFC. Mice were trained to nose poke for food pellets. We identified no group differences in response acquisition (interaction, F_(5,40) = 1.7, p = 0.15; effect of group, F < 1; Fig. 1e). We then delivered the DREADDs ligand CNO via systemic injection to all mice, regardless of viral vector, and gave the mice access to food ad libitum: the reinforcer pellets in one session and regular chow in another session. Throughout, food intake did not differ between groups (t₈ = −1.44, p = 0.19; Fig. 1f). Mice were then placed in the operant conditioning chambers. Control mice generated robust response rates throughout, insensitive to reinforcer devaluation. In contrast, G_s–DREADDs-expressing mice inhibited responding after prefeeding with the reinforcer pellets (session × group, F_(1,8) = 7.6, p = 0.03; Fig. 1g). Thus, G_s–DREADDs-mediated mOFC stimulation enhanced behavioral sensitivity to reinforcer devaluation.

Mice were then trained for four additional sessions (Fig. 1e), and the reinforcer devaluation procedure was repeated. With this additional identical test (and habituation to the injection stressor), all mice displayed sensitivity to reinforcer devaluation as expected, inhibiting responding after pellet prefeeding (main effect, F_(1,16) = 22.9, p < 0.001; Fig. 1h). These findings indicate that CNO does not itself impair sensitivity to prefeeding devaluation (i.e., in GFP control mice) but rather enhances sensitivity to reinforcer devaluation in G_s–DREADDs-expressing mice. Activation of mOFC G_s–DREADDs also does not appear to obviously affect extinction conditioning because overall response rates were indistinguishable between groups during the probe tests (Fig. 1g,h), which are conducted in extinction.

We hypothesized that stimulation of the mOFC may increase sensitivity to effort requirements, i.e., the effort required to obtain a reinforcer, relative to the value of the reinforcer. Thus, we tested the same mice in a PR task. In this test, the response requirements progressively increase for a reinforcer of fixed value. Consistent with our hypothesis, activation of G_s–DREADDs decreased breakpoint ratios (t₁₄ = 2.2, p = 0.04; Fig. 1i).

There is some evidence that hyperexcitation of the mOFC causes repetitive stereotypy-like behaviors (Ahmari et al., 2013). Previous experiments used optogenetic stimulation of the mOFC rather than DREADDs approaches, which instead regulate the firing threshold of neurons (Urban and Roth, 2015). We quantified spontaneous locomotor activity for 1 h 30 min after CNO administration (matching the timing of devaluation and PR testing). Mice engaged in ambulatory behavior more than repetitive stereotypy-like behavior in general (F_(1,28) = 103, p < 0.001), but we identified no group differences in either ambulation or repetitive stereotypy-like locomotor counts (F < 1; Fig. 1j). This same locomotor monitoring system has been used to document changes in psychostimulant-elicited locomotion and stereotypy-like behavior (Gourley et al., 2009b), increasing confidence in this null result.

Selective Bdnf knockdown in the mOFC decreases behavioral sensitivity to reinforcer devaluation

Excitatory pyramidal neurons—those targeted by CaMKII-driven AAV–G_s–DREADD—are a primary source of the pro-plasticity neurotrophin BDNF in the cortex. We hypothesized that BDNF may be a molecular mechanism by which the mOFC regulates reward-related decision-making. To test this perspective, we next reduced expression of Bdnf in the mOFC using a viral vector approach (for representation of viral vector spread, see Fig. 1d). Subsequently, mice acquired the nose-poke response with no group differences (interaction and main effect, F < 1; Fig. 2a). After pellet prefeeding (and in the absence of an injection stressor as in the studies above), control GFP-expressing mice decreased response rates as expected. Meanwhile, mice with selective Bdnf knockdown failed to modify response rates (interaction, F_(1,13) = 5.3, p = 0.04; Fig. 2b), insensitive to reinforcer devaluation. Instead, these mice responded identically to those that had been prefed with regular chow, leaving the value of the food pellet intact (value groups). (Note that response rates generated during the probe tests were compared with each animal's own baseline because an effect of testing order in this experiment indicated that mice extinguished responding during a second probe test not shown, regardless of devalued or value condition.)

Figure 2.

Selective Bdnf knockdown in the mOFC decreases behavioral sensitivity to reinforcer value and PR response requirements. a, Cre-expressing viral vectors were infused into the mOFC (as in Fig. 1d) of “floxed” Bdnf mice. Mice were trained to nose poke for food reinforcers, with no differences between groups. b, After prefeeding devaluation (and in the absence of an injection stressor as in Fig. 1), control GFP-expressing mice decreased response rates relative to baseline. Meanwhile, selective Bdnf knockdown mice failed to modify response rates. Instead, response rates were indistinguishable from those generated by mice that had access to regular chow before test (value groups). c, mOFC Bdnf knockdown also increased breakpoint ratios. d, Meanwhile, response extinction was not affected; e, furthermore, spontaneous locomotor activity was unaffected. Ambulatory and repetitive photobeam breaks are represented in 5 min bins (left) and 1 h total counts (right). Symbols represent means ± SEMs. *p < 0.05. n = 8 per group.

mOFC-selective Bdnf knockdown mice also generated higher breakpoint ratios in a PR test (t₍₁₃₎ = −2.1, p = 0.05; Fig. 2c), again the opposite pattern relative to mOFC G_s–DREADDs-expressing mice. This could not obviously be attributable to resistance to extinction because selective Bdnf knockdown did not affect extinction conditioning (main effect of session, F_(2,26) = 14.2, p < 0.001; effect of group and interaction, F < 1; Fig. 2d). The lack of effect on extinction conditioning replicates our previous findings on this topic (Gourley et al., 2009a), and this pattern overall is consistent with the suggestion that the mouse mOFC, like the primate mOFC, regulates behavioral sensitivity to outcome value.

As with the G_s–DREADDs mice, locomotor activity counts did not differ between groups (ambulation, F_(1,14) = 1.7, p = 0.2; repetitive photobeam breaks, F_(1,14) = 1.2, p = 0.3; interactions, F < 1; Fig. 2e).

Bdnf^+/− mice are behaviorally insensitive to reinforcer devaluation

Our findings suggest that mOFC BDNF serves as an inhibitory brake on reward-related responding. Bdnf^+/− mice are viable, meaning that we were next able to assess whether these mice develop the same phenotype as selective Bdnf knockdown mice and whether it could be rescued by selective replacement of BDNF in the mOFC.

We first confirmed that brain (hippocampal) BDNF expression in Bdnf^+/− mice was approximately half that in wild-type mice as expected (p < 0.001; Fig. 3a). Mice in this experiment successfully acquired the food-reinforced instrumental response. We detected no differences between groups (F < 1; Fig. 3b). Food consumption during the prefeeding periods was also unaffected by genotype (t₍₂₆₎ = 1.2, p = 0.25; Fig. 3c). Wild-type mice subsequently reduced responding associated with the now devalued reinforcer, but Bdnf^+/− mice failed to inhibit responding, showing insensitivity to reinforcer devaluation (interaction, F_(1,26) = 4.11, p = 0.05; within-group post hoc p = 0.8; Fig. 3d).

Figure 3.

Bdnf^+/− mice are behaviorally insensitive to reinforcer devaluation. a, Bdnf^+/− mice expressed approximately half as much BDNF in hippocampal lysates as wild-type littermates (n = 11–12 per group). b, Mice acquired the nose-poke response without differences between groups. c, Mice also consumed equivalent amounts of food under ad libitum conditions. d, After freely consuming reinforcer pellets, however, only wild-type mice significantly decreased response rates, whereas Bdnf^+/− mice failed to significantly modify response patterns. e, Nonetheless, the same mice consumed equivalent amounts of a palatable sucrose solution (SUC; n = 16 wild-type, 12 Bdnf^+/− littermates). Bars and symbols represent means ± SEMs per group, *p < 0.05; **p < 0.001. wt, Wild-type.

In addition to unchanged food intake during ad libitum feeding (Fig. 3c), consumption of a palatable sucrose solution was also indistinguishable between genotypes (p = 0.6; Fig. 3e). This pattern suggests that adult Bdnf^+/− mice are behaviorally insensitive to reinforcer devaluation, as opposed to, for example, the hedonic valence of the reinforcer. This is an important distinction, given that Bdnf^+/− mice develop late-life obesity (Kernie et al., 2000).

PR responding is elevated in Bdnf^+/− mice

As in our experiments with mOFC-selective Bdnf knockdown, responding on a PR schedule was also assessed. Here, we tested mice daily over the course of 1 week, revealing an interaction between genotype and session (F_(6,150) = 2.6, p = 0.02; Fig. 4a). Breakpoint ratios gradually grew in Bdnf^+/− mice, coupled with a modest decline in typical mice. However, the ratio of responses on the active versus inactive apertures was unchanged (main effect of genotype, F_(1,25) = 1; interaction, F_(6,150) = 1.6, p = 0.17; Fig. 4b), indicating that responding was equally selective for the active nose-poke aperture between groups.

Figure 4.

Responding on a PR schedule is inflated in Bdnf^+/− mice. a, Bdnf^+/− mice achieved higher breakpoint ratios on a PR schedule of reinforcement, particularly during test sessions 4–7. b, Despite this, responding was equally selective for the active, relative to inactive, apertures between groups. c, d, Comparisons of the average first, median, and last PRPs from sessions 1–3, when breakpoint ratios did not significantly differ, and sessions 4–7, when breakpoints differed, revealed main effects of time as expected, but no genotype effects. These patterns suggest that motivational differences between wild-type and Bdnf^+/− mice cannot obviously account for differences in breakpoints. e, vmPFC expression of p-trk, GluR1, and pERK1/2 were decreased in Bdnf^+/− mice. The latter could be attributed reduced phosphorylation of the ERK2 isoform. Representative bands are adjacent. Bars and symbols represent means ± SEMs per group. *p < 0.05; **p < 0.001. wt, Wild-type. n = 13–14 per group.

An increase in breakpoint ratios could conceivably be attributed to increased perceived value of the reinforcer or increased motivation to acquire the reinforcer. PRPs can dissociate these factors. PRPs decrease when the motivation to acquire an outcome increases, for example, rats more rapidly initiate a new trial after collecting a reinforcer when hungry (Skjoldager et al., 1993). In contrast, increasing the quantity of reinforcers, for example, by providing three pellets instead of one, also increases PR breakpoints but leaves PRPs unaffected. To determine whether motivational contributions potentially increased breakpoint ratios in Bdnf^+/− mice, we extracted the first, median, and last PRP for each mouse for each test session (Skjoldager et al., 1993). Several analytic approaches failed to identify an effect of genotype. For example, we averaged PRPs for sessions 1–3 (when breakpoints did not significantly differ) and compared them with sessions 4–7 (when breakpoints differed). These analyses failed to reveal any effects of genotype (F values ≤1; Fig. 4c,d). Only main effects of time were detected as expected (session 1–3, F_(2,23) = 6.8, p = 0.002; session 4–7, F_(2,23) = 6.9, p = 0.002). As an additional example, PRPs did not differ between groups during session 1 when breakpoints also did not differ, nor during session 7 when breakpoints did differ (F values ≤1; data not shown). In fact, during no test session did the PRPs differ as a function of genotype (all p values >0.05). This pattern of responding—increased breakpoint ratios, coupled with unaffected PRPs—suggests that differences in primary motivation do not account for breakpoint ratio differences between Bdnf^+/− and littermate wild-type mice.

pERK2 is reduced in Bdnf^+/− vmPFC

pERK1/2 has been proposed as a marker of neuronal activity. Furthermore, BDNF binding to its high-affinity receptor trkB activates the ERK MAP kinase signaling pathway. For these reasons, pERK1/2 was analyzed in vmPFC tissue samples, which include the mOFC, immediately after the last session. pERK1/2 was decreased in Bdnf^+/− mice as expected (t₍₂₂₎ = 3.3, p = 0.004; Fig. 4e). Analysis of the individual pERK1/2 isoforms indicated that pERK2, which is preferentially associated with activity-dependent neuroplasticity (English and Sweatt, 1996), was significantly reduced (p = 0.006). Expression of the primary BDNF receptor trkB was unchanged (p > 0.6), but phosphorylation of the receptor was decreased as expected, as was expression of the GluR1 subunit of the AMPA receptor (p < 0.05; Fig. 4e). pERK1/2 was also analyzed in the dorsal hippocampus and NAc; notably, there were no differences in expression levels in these regions (data not shown).

BDNF replacement in the mOFC blocks behavioral abnormalities

We next aimed to block behavioral abnormalities in Bdnf^+/− mice. We first developed a BDNF microinfusion protocol that, in genetically intact mice, increased levels of pERK1/2 and the immediate-early gene c-fos at the infusion site, detectable 24 h after infusion (all p < 0.05; Fig. 5a,b). To determine the anatomical distribution of BDNF-mediated ERK1/2 stimulation, we also immunostained for pERK1/2 24 h after infusion and quantified expression in 35 μm concentric rings around the infusion site in hemisected coronal sections (Fig. 5c). pERK1/2 expression was increased proximal to the infusion site in both groups, but pERK1/2 was higher in the BDNF group within 500 μm of the infusion terminus (interaction, F_(19,95) = 1.9, p = 0.05; Fig. 5d).

Figure 5.

mOFC BDNF infusion enhances pERK1/2 expression. a, Experimental timeline. Mice were infused into the mOFC with saline or BDNF, then brains were collected 24 h later. b, BDNF increased expression of c-fos and pERK1/2, as measured by Western blot. Representative bands are adjacent. n = 6 per group. c, In another cohort of mice, pERK1/2 expression was quantified in concentric rings radiating from the base of the infusion site in fixed, hemisected coronal sections. The schematic represents the rings overlaid on representations of the largest, smallest, and representative (middle line) pERK1/2 fluorescence signal. d, Quantitative immunostaining revealed increased pERK1/2 expression at the infusion site 24 h after infusion, and the pERK1/2 signal was further elevated in BDNF-infused mice. Representative image is adjacent. n = 3–4 per group. Bars and symbols represent means ± SEMs per group. *p < 0.05. sal, Saline.

Next, we trained a naive cohort of Bdnf^+/− and littermate wild-type mice to perform the instrumental response. We again detected no differences in response rates or reinforcement rates between groups (data not shown). Then, groups were assigned by matching breakpoint ratios collected during three initial PR test sessions (Fig. 6a). After BDNF infusion in the mOFC, mice were tested again, and an interaction between genotype and infusion was detected (F_(1,37) = 10.3, p = 0.003; Fig. 6b). mOFC BDNF replacement in Bdnf^+/− mice normalized breakpoint ratios compared with saline-infused Bdnf^+/− mice (p = 0.01) and wild-type mice infused with BDNF (p = 0.009). In other words, selective BDNF replacement fully rescued responding in Bdnf^+/− mice, and thus, mOFC BDNF is both necessary and sufficient for typical responding in this task. Interestingly, BDNF infusion in wild-type mice modestly increased breakpoint ratios (p = 0.08), suggesting that mOFC BDNF regulates action selection according to an inverted U-shaped curve. This finding, although unexpected, bears some similarity to the inverted U-shaped influence of BDNF met gene dosing on gray and white matter morphometry in humans (Forde et al., 2014). Also of note, synaptic scaling can occur after supra-physiological BDNF overexpression (Rutherford et al., 1998), which here, could conceivably impair optimal mOFC function and account for a modest inflation of breakpoints in wild-type mice administered BDNF.

Figure 6.

mOFC BDNF infusion rescues behavioral sensitivity to a PR schedule of reinforcement in Bdnf^+/− mice. a, Baseline responding on a PR schedule was established in Bdnf^+/− mice and wild-type littermates before intracranial BDNF infusion. b, A single mOFC BDNF infusion occluded the expected increase in PR responding in Bdnf^+/− mice. Mean breakpoint ratios achieved during three test sessions are shown. Inset, Needle tracks terminated within the anatomical boundaries drawn at approximately +2.46 and +2.1 mm from bregma on images from the Mouse Brain Atlas (Rosen et al., 2000). n = 8–13 per group. c, Mice were killed immediately after the last session. As expected, pERK2 expression was decreased by 30–40% in Bdnf^+/− mice infused with saline compared to wild-type mice infused with saline. BDNF infusion blocked this deficit. Representative blots are below. d, In the dorsal striatum, c-fos patterns mirrored behavioral response patterns (compare with b), with elevated c-fos associated with high breakpoint ratios. Representative blots are adjacent. e, Breakpoint ratios covaried with striatal c-fos. Bars and symbols represent means ± SEMs per group, except in e, in which each symbol represents a single mouse. *p ≤ 0.05, **p < 0.001. wt, Wild-type; sal, saline.

Immediately after the last session, mice were killed, and vmPFC tissue was homogenized to evaluate whether BDNF infusion normalized pERK2 expression, in parallel with behavioral responding. As expected, pERK2 was decreased in Bdnf^+/− mice infused with saline (interaction, F_(1,26) = 6.3, p = 0.02, post hoc p = 0.02; Fig. 6c). However, BDNF infusion restored pERK2 levels such that BDNF-infused Bdnf^+/− mice did not differ from control mice (p = 0.3).

As shown, rodents can learn to select actions according to the value of a reinforcer. Meanwhile, behavioral insensitivity to reinforcer value is associated with a dorsolateral striatal “habit” circuit (Yin et al., 2008, 2009). Thus, the dorsal striatum was also extracted and immunoblotted for the immediate-early gene c-fos. Expression patterns strongly resembled behavioral response patterns (interaction, F_(1,26) = 18, p = 0.003; Fig. 6d, compare with b): Bdnf^+/− mice had high c-fos expression levels (p < 0.05), whereas BDNF infusion normalized expression (p = 0.005 compared with saline-infused Bdnf^+/− mice). BDNF infusion in wild-type mice increased c-fos (p = 0.007). Even from a correlational perspective, high striatal c-fos was associated with high breakpoint ratios (r = 0.37, p = 0.045; Fig. 6e).

Discrete stimuli signaling reinforcer availability rescue responding

Dorsolateral striatal systems are associated with stimulus-dependent, as opposed to value-dependent, decision-making (Yin et al., 2008; Hart et al., 2014). Thus, we evaluated whether PR responding could be normalized if Bdnf^+/− mice were provided with discrete stimuli signaling reinforcer availability; this would presumably access cue-sensitive striatal systems. A light/tone stimulus was coupled with reinforcer delivery during both the response training (Fig. 7a) and PR testing phases (Fig. 7b). Response rates and breakpoint ratios were indistinguishable between Bdnf^+/− and wild-type mice, suggesting that Bdnf^+/− mice were able to use pavlovian stimuli to regulate responding (PR test main effect, F < 1; interaction, F_(4,56) = 1.6, p = 0.2; Fig. 7b). An alternative perspective is that stimulus–outcome associations energized responding in wild-type mice, but it does not necessarily account for the lack of group differences during training, when the stimuli were also present. Nevertheless, we replicated this experiment in a separate group of mice, and again, wild-type and Bdnf^+/− mice did not differ (main effect and interaction F values <1; Fig. 7b′).

Figure 7.

Cues signaling reinforcer delivery rescue PR responding in Bdnf^+/− mice. No differences in extinction conditioning or locomotor activity. a, A group of wild-type and Bdnf^+/− mice was tested on a PR schedule as in previous experiments but with the addition of discrete stimuli that signaled reinforcer delivery. Instrumental response acquisition (with stimuli presentation) was unaffected by genotype. b, When presented with stimuli signaling reinforcer delivery, Bdnf^+/− mice were able to regulate their responding identically to wild-type mice. Gray bars sit at the overall mean breakpoint for each group, with the width representing ± SEM. Light gray, Wild-type; dark gray, Bdnf^+/−. b′, We replicated this experiment in another group of naive mice, and again, breakpoints did not differ when mice were provided discrete stimuli signaling reinforcer delivery. c, Responding in extinction was also unaffected. d, Ambulation counts were also equivalent in food-restricted Bdnf^+/− and wild-type mice. Symbols represent means ± SEMs. wt, Wild-type. n = 6–18 per group throughout.

Extinction responding in Bdnf^+/− mice

To rule out insensitivity to non-reinforcement or general behavioral inflexibility as causal factors in behavioral abnormalities in Bdnf^+/− mice, reinforcement was withheld entirely (extinction). A main effect of session (F_(5,130) = 14.5, p < 0.001), but no effect of genotype or interactions, was detected (F values <1), again indicating that responding was indistinguishable based on genotype (Fig. 7c).

Finally, we also confirmed that locomotor activity was unaffected by genotype (F values <1; Fig. 7d). Overall, activity decreased across the session (F_(11,110) = 2.7, p = 0.004), indicating habituation to the novel environment, but we detected no differences between groups. Importantly, mice were food restricted during this test, recapitulating locomotor activity levels during PR and devaluation testing.

MEK inhibition elevates PR responding

ERK1/2 phosphorylation was blunted in Bdnf^+/− mice. To evaluate whether suppressing MEK, a kinase directly upstream of ERK1/2, could mimic Bdnf heterozygosity, adult male C57BL/6 mice were trained to perform an instrumental response for food reinforcement. Mice were matched based on reinforcements earned during training and assigned to either vehicle or PD0184161 groups (Fig. 8a). When mice were treated with PD0184161, breakpoint ratios differed (F_(3,27) = 4.1, p = 0.02). Specifically, wild-type mice injected with PD0184161 before the test achieved higher breakpoint ratios than mice injected with vehicle before the test (p = 0.02; pretest injections represented in Fig. 8b), whereas mice injected with PD0184161 after the test did not differ from corresponding control mice (p = 0.17; posttest injections represented in Fig. 8c). Also, when tested drug free, PD0184161-exposed mice were indistinguishable from control mice (p = 0.4; Fig. 8b, inset). This pattern suggests ERK1/2 regulates action selection online rather than via post-session memory consolidation (i.e., consolidating information regarding the PR schedule of reinforcement).

Figure 8.

Pretest but not posttest MEK inhibition reduces breakpoint ratios. a, Naive mice were initially trained to perform the instrumental response. b, When a MEK inhibitor [PD0184161 (PD)] was administered before the session, responding on a PR schedule was elevated. Mice were given a 3 week drug washout period and retested, revealing no persistent effects on responding. c, As a control, other mice were injected after the sessions; in this case, breakpoint ratios were unchanged. Bars and symbols represent means ± SEMs per treatment group. *p < 0.05 versus vehicle. n = 10–11 per group.

PR responding predicts vmPFC, but not striatal, BDNF in wild-type mice

Our findings suggest that BDNF within the mOFC regulates effortful instrumental response selection. However, an alternative possibility is that mOFC-derived BDNF acts via axonal transport from the mOFC to the dorsal striatum. To explore this possibility, we characterized responding on a PR schedule in several naive mice (Fig. 9a). Immediately after the final session, vmPFC and dorsal striatal samples were extracted. vmPFC BDNF significantly covaried with responding during the last session (r = 0.62, p = 0.02; Fig. 9b), whereas striatal BDNF did not (r = 0.05, p = 0.86; Fig. 9c). This outcome suggests that local mOFC BDNF is a determining factor in action selection strategies.

Figure 9.

Endogenous prefrontal, but not dorsal striatal, BDNF covaries with instrumental responding in intact mice. a, A large group of naive mice was trained to perform the nose-poke response, and then several PR tests were conducted. Total responses on the active aperture are represented. b, vmPFC BDNF expression correlated with response number during the last test session. c, In contrast, dorsal striatal BDNF did not covary with the same measure. Symbols represent means ± SEMs per group in a; symbols in b and c represent individual mice. n = 15.

Discussion

The mOFC is a primary component of an anatomically interconnected medial prefrontal cortical network and is considered distinct relative to the central and lateral compartments of the OFC (Ongür and Price, 2000; Wallis, 2012). These compartments are instead part of a “sensory integration” network classically associated with behavioral flexibility during reversal conditioning (Iversen and Mishkin, 1970; Schoenbaum et al., 2002; McAlonan and Brown, 2003; for contemporary models, see Stalnaker et al., 2015). In contrast, vmPFC structures may be essential to determining behavioral response strategies based on general representations of outcome value (Wallis, 2012). Goal-directed action selection after reinforcer devaluation correlates significantly with neural activity in the human mOFC (Valentin et al., 2007). Furthermore, the mOFC is activated during willingness-to-pay calculations (Plassmann et al., 2007), a finding that may be particularly germane to our current study, because we find that the rodent mOFC is essential to behavioral sensitivity to outcome value and the appropriate “pay” (i.e., effort expenditure) for a given reinforcer in a PR task. Lesions or DREADD-mediated inactivation of the mOFC in rats induce failures in retrieving outcome identity memories (Bradfield et al., 2015), suggesting that the healthy mOFC serves to access outcome value information when it is not immediately observable and thereby guide goal-directed decision-making. We argue that BDNF is essential for this function, given that both tasks used in Bdnf-deficient mice here require mice to sustain a stable representation of reinforcer value to appropriately inhibit responding (after reinforcer devaluation) or gate responding when response requirements escalate (as in the PR task).

Bidirectional regulation of behavioral sensitivity to reinforcer value

We initiated these studies by expressing in the mOFC G_s-coupled DREADDs, engineered G-protein-coupled receptors activated by the otherwise inert ligand CNO (Urban and Roth, 2015). We then decreased reinforcer value using a prefeeding procedure wherein mice can freely consume reinforcer pellets before a probe test conducted in extinction. Additionally, instrumental responding according to a PR schedule of reinforcement was tested. G_s–DREADDs stimulation enhanced response inhibition after reinforcer devaluation, evidence of increased sensitivity to decreased outcome value. Mice were also more sensitive to the escalating demands of the PR schedule, achieving lower breakpoints. mOFC stimulation did not affect ambulation or stereotypy-like behaviors, contrary to evidence that optogenetic stimulation of a mOFC-striatal circuit induces repetitive stereotypy (Ahmari et al., 2013). Repeated burst-like activity caused by optogenetic stimulation, relative to slow depolarization caused by G_s–DREADDs, could account for different behavioral consequences.

What molecular factors might regulate mOFC function? Excitatory pyramidal neurons are a primary source of BDNF in the cortex, and high-frequency stimulation induces synaptic BDNF secretion (Hartmann et al., 2001). To determine the role of locally synthesized BDNF in mOFC-dependent decision-making, we reduced local Bdnf, revealing the opposite behavioral profile. Specifically, mOFC-selective Bdnf knockdown interfered with behavioral sensitivity to reinforcer devaluation, and PR responding was inflated. Bdnf^+/− mice developed the same aberrantly elevated PR breakpoints as those with site-selective knockdown, which allowed us to next confirm that selective infusion of BDNF into the mOFC normalized responding in constitutive Bdnf^+/− mice. Thus, mOFC BDNF is both necessary and sufficient for appropriate response inhibition. Notably, PRPs did not differ between Bdnf^+/− and wild-type littermates, suggesting that exaggerated primary motivation to acquire the reinforcer does not account for breakpoint differences (Skjoldager et al., 1993). Moreover, elevated breakpoints could not be attributed to motoric hyperactivity or enhanced hedonic sensitivity to food. These are important measures, given that Bdnf^+/− mice develop late-adulthood obesity (Kernie et al., 2000).

Involvement of ERK/MAP kinase

The ERK MAP kinase signaling cascade is coupled to multiple receptor systems, including trkB, the high-affinity receptor for BDNF. ERK1/2 is also implicated in several forms of learning, memory, and neuroplasticity (Mazzucchelli and Brambilla 2000; Rodrigues et al., 2004), and furthermore, LTP induction preferentially activates ERK2 (English and Sweatt, 1996). Phosphorylation of both trkB and ERK1/2 was decreased in the Bdnf^+/− vmPFC, driven specifically by decreased pERK2, suggesting that ERK2-mediated signaling may be essential to mOFC function. Pharmacologically inhibiting MEK, immediately upstream of ERK1/2, recapitulated the effects of mOFC-selective Bdnf knockdown, augmenting breakpoints in the PR task. Notably, post-session MEK inhibition had no effects, implicating ERK1/2 signaling in online action selection rather than post-session consolidation processes. This perspective is further supported by our DREADDs studies, given that CNO was onboard during testing and is in general agreement with the argument that the mOFC retrieves outcome identity/value information during task performance (Bradfield et al., 2015). Interestingly, this selective function contrasts with that of the ventrolateral orbitofrontal cortex, in which BDNF–trkB appears to also regulate the consolidation or retention of response–outcome associative memory (Zimmermann et al., 2015).

Replacement of BDNF selectively within the mOFC rescued pERK1/2 in Bdnf^+/− mice, and immunostaining for pERK1/2 revealed a sustained elevation in expression fanning from the infusion site. BDNF-induced pERK1/2 extended in some mice into the dorsally situated prelimbic cortex. We believe that the behavioral effects of BDNF infusion can nonetheless be attributed to actions in the mOFC, because Bdnf knockdown in the prelimbic PFC decreases, rather than increases, PR responding (Gourley et al., 2012). Prelimbic-selective knockdown also facilitates extinction conditioning, whereas extinction was spared here (Fig. 2; Gourley et al., 2009a). These opposing roles for BDNF in the mOFC and prelimbic cortex may account for why mOFC-selective Bdnf knockdown rapidly modified PR response patterns, whereas genotypic differences between Bdnf^+/− mice and wild-type littermates emerged only with repeated testing.

Corticostriatal interactions in reward-related decision-making

BDNF is subject to anterograde transport. For example, cortical pyramidal neurons are a predominant source of BDNF in the striatum, which contains little Bdnf mRNA (Altar et al., 1997). Furthermore, BDNF infusion in the dorsal PFC can increase striatal and amygdalar BDNF (McGinty et al., 2010). Conversely, PFC-selective Bdnf knockdown reduces BDNF in downstream structures (Gourley et al., 2009a, 2013c; Zimmermann et al., 2015). Thus, it is conceivable that mOFC BDNF regulates action selection by binding locally or in distal targets, such as the dorsomedial striatum (Fig. 1; Schilman et al., 2008). However, we found that BDNF in the vmPFC, but not striatum, predicted response patterns. Thus, BDNF subjected to axonal transport from the mOFC to dorsal striatum may serve important purposes, but striatal BDNF was a poor predictor of response patterns here.

Unlike BDNF, striatal immediate-early gene expression closely mirrored response patterns, with high c-fos expression in Bdnf^+/− mice that responded persistently, despite escalating response demands in the PR task. Why might this be? Classically, goal-directed behaviors are defined as those sensitive to outcome value. mPFC damage (lesions, inactivations) and stressors can induce a shift from goal-directed to “habitual” modes of response that are, in contrast, insensitive to reinforcer value (Balleine and O'Doherty, 2010; Schwabe and Wolf, 2011). Converging neuroanatomical models characterize this process as a transition from PFC–striatal systems that act in concert to a cue-sensitive dorsolateral striatum neurocircuit (Yin et al., 2008, 2009; Kimchi et al., 2009; Gourley et al., 2013b). Thus, c-fos expression here may reflect the recruitment of a dorsolateral striatal system that drives stimulus-dependent responding in the absence of BDNF-mediated signaling in the mOFC. To test this perspective, we provided discrete stimuli signaling reinforcer availability, “bridging” the association between the nose-poke response and reinforcer. These stimuli eliminated response differences between groups. This effect was robust, detected in multiple experiments and despite cohort variances, suggesting that stimulus-dependent response regulation is intact in Bdnf^+/− mice.

In marmosets, vmPFC lesions including the mOFC increase breakpoints (Pears et al., 2003), and mice with mOFC lesions develop nearly identical patterns of responding on a PR schedule relative to Bdnf^+/− mice (Gourley et al., 2010). However, we have also reported previously that mOFC lesions do not affect behavioral sensitivity to reinforcer devaluation (Gourley et al., 2010). How might we reconcile these findings? Previously, mice were trained before lesion placement, whereas here, Bdnf was knocked down first. It is thus possible that information acquired before insult can help to sustain inhibitory control.

Effects of acute BDNF infusion

We used multiple complementary genetic, pharmacological, viral, and chemogenetic approaches to provide evidence that the mOFC inhibits instrumental responding when response demands escalate and that BDNF is necessary for this function. One additional discovery was that a single BDNF infusion into the mOFC had sustained behavioral consequences, normalizing response patterns and pERK1/2 in Bdnf^+/− mice multiple days after infusion (Fig. 6). In conceptually similar studies, BDNF infusion into the dorsomedial PFC after cocaine self-administration in rats reduced the reinstatement of cocaine seeking several days later (Berglind et al., 2007, 2009; Whitfield et al., 2011), a sustained response. However, the relationship between mPFC BDNF and reward seeking is likely quite complex, e.g., cocaine abstinence causes progressive increases in mPFC BDNF above control levels (Lu et al., 2010; McGinty et al., 2010; Giannotti et al., 2014), and the precise behavioral significance of cocaine-induced BDNF overexpression is debated (Sadri-Vakili et al., 2010; Pitts et al., 2016). Ongoing studies using targeted manipulation of BDNF in specific circuits and structures such as the mOFC will help to clarify how BDNF regulates the function of corticolimbic regions in balancing reward seeking versus behavioral inhibition.