Functional Heterogeneity within Rat Orbitofrontal Cortex in Reward Learning and Decision Making

Abstract

Rat orbitofrontal cortex (OFC) is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area, ventral orbital area, ventrolateral orbital area, lateral orbital area, dorsolateral orbital area, and agranular insular areas. Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of rat OFC. While collectively this extensive body of work has provided great insight into the functions of OFC, leading to theoretical and computational models of its functions, one issue that has emerged relates to what is defined as OFC because targeting of this region can be quite variable between studies of appetitive behavior, even within the same species. Also apparent is that there is an oversampling and undersampling of certain subregions of rat OFC for study, and this will be demonstrated here. The intent of the Viewpoint is to summarize studies in rat OFC, given the diversity of what groups refer to as “OFC,” and to integrate these with the findings of recent anatomical studies. The primary aim is to help discern functions in reward learning and decision-making, clearing the course for future empirical work.

Introduction

Over the past 20 years, there has been a marked increase in the number of publications focused on the functions of the orbitofrontal cortex (OFC) (Fig. 1). While these studies have provided penetrating insight into the various functions of this region of the prefrontal cortex, one issue that has emerged relates to what is defined as OFC, as the specific cortical regions that have been studied are variable between experiments, even within the same species. The precipitous rise in the number of OFC reports in the past decade makes it even more necessary to carefully assess stereotaxic targeting in experimental studies for firm conclusions on appetitive, reward-related functions. Comparative studies of OFC function have yielded a long history of this brain region as a locus for inhibitory control, and there is convincing recent evidence of this as well (Torregrossa et al., 2008; Mansouri et al., 2014; Bryden and Roesch, 2015; Meyer and Bucci, 2016; Hardung et al., 2017). Yet this classic role for OFC (Gourley and Taylor, 2016) has become more nuanced with the findings of more recent studies of various other functions, positioning OFC as the locus of relative and economic value (Tremblay and Schultz, 1999; Padoa-Schioppa and Assad, 2006), expected or predicted outcomes (Tremblay and Schultz, 2000; Rudebeck and Murray, 2014), confidence estimates (Kepecs et al., 2008), imagination and insight (Takahashi et al., 2013; Lucantonio et al., 2014, 2015), Bayesian belief (Jang et al., 2015), cognitive map of task space (Wilson et al., 2014), state space (Schuck et al., 2016; Nogueira et al., 2017), regret (Camille et al., 2004; Steiner and Redish, 2014), and credit assignment (Walton et al., 2011; Chau et al., 2015; Akaishi et al., 2016; Noonan et al., 2017). Given many high-quality and in-depth analyses on what the OFC does and does not do (Noonan et al., 2012; Rudebeck and Murray, 2014; Stalnaker et al., 2015), there appears to be no shortage of theoretical viewpoints on the subject.

Figure 1.

Number of reports on OFC in all species. PubMed search terms “OFC,” “Orbitofrontal Cortex,” and “Orbital Prefrontal Cortex” from 1948 to 2014. The steepest rise in the number of publications occurred in the early 2000s, which was preceded by a more gradual increase in the 1990s. Rat studies, although increasing in number over the past decade, still account for a minority of reports on OFC.

A review of OFC function across species (Wallis, 2011) revealed that studies of monkey OFC function have primarily focused on anterior rather than posterior OFC, and that rat studies have instead emphasized lateral over medial OFC. Indeed, many reports of OFC function do not specify the subregion within OFC that is the focus of the work, an important detail that is sometimes buried in the methods. Also apparent is that there is an oversampling of certain subregions of rat OFC for study, and this will be demonstrated here. The primary intent of the present Viewpoint is to summarize recent targeting in rat OFC (given the diversity in coordinates used) and, secondarily, to integrate these with the findings of recent anatomical studies. Collectively, the goal is to help clear the course for future empirical work.

What is rat OFC?

Rat OFC is located in the dorsal bank of the rhinal sulcus, and is divided into the medial orbital area (MO), ventral orbital area (VO), ventrolateral orbital area (VLO), lateral orbital area (LO), dorsolateral orbital area (DLO), and agranular insular (AI) areas (Ray and Price, 1992; Ongür and Price, 2000; Rempel-Clower, 2007). Although all prefrontal regions are agranular in the rat brain, these sectors of rat OFC are thought to be homologous to approximately one-third of monkey caudal orbitomedial prefrontal cortex because these regions share similar positioning and connectivity to subcortical structures (Price, 2007). In particular, mediodorsal thalamus is reciprocally connected to all parts of OFC in both rats and monkeys (Rose and Woolsey, 1948; Groenewegen, 1988; Uylings and van Eden, 1990; Ray and Price, 1992; Ongür and Price, 2000; compare Markowitsch and Pritzel, 1979). Additional criteria have since been included to designate the “parts” of prefrontal cortex across species: namely, by outlining behavioral, neurochemical, and electrophysiological properties (Markowitsch and Pritzel, 1979), and by studying phylogenetic (Passingham and Wise, 2012) and ontogenetic relationships (Goldman, 1971) of these areas.

A recent comprehensive comparative analysis of collections of monkey and rat anatomical tracing experiments enable us to make “connectivity-based inferences about homologies” across these species (Heilbronner et al., 2016). Specifically, there is similar OFC-striatal organization across rat and monkey, with comparable efferent hubs: MO in rat and medial OFC in monkey projecting to ventromedial segments of striatum, and LO in rat and central-lateral OFC in monkey projecting to central and lateral segments of striatum in both species. Interestingly, primarily on the basis of their striatal terminal fields, Heilbronner et al. (2016) clustered rat VO and LO more closely together than VO and MO, and equated rat VO and LO (and this likely extends to VLO) as collectively homologous to monkey central-lateral OFC.

Hoover and Vertes (2011) came to slightly different conclusions for rat MO and VO following a study of brain-wide projections from these OFC sectors. They found that rat VO and MO send high density fibers to the same cortical areas (MO to VO, VO to MO, and both MO and VO to anterior cingulate, prelimbic, and infralimbic), caudate-putamen, mediodorsal thalamus, lateral hypothalamus, most areas of the hippocampus, substantia nigra, and ventral tegmental area. However, there are surprising and notable differences as well: MO projects densely to basolateral and central amygdala, whereas VO projects less so, and MO projects more densely to nucleus accumbens than VO. In general, MO sends fibers more broadly throughout limbic areas than does VO. Interestingly, both MO and VO do not project densely to LO. On this basis, Hoover and Vertes (2011) proposed instead that MO and VO have an important shared role in the affective and cognitive integration of goal-directed behavior. As for the distinctions between rat VLO, LO, DLO, and AI (as labeled in the atlas by Paxinos and Watson, 2007), there are no brain-wide systematic tracer studies of projections arising from these specific sectors like that conducted by Hoover and Vertes (2011) for MO and VO, but we can rely on earlier anatomical work (Price, 2007; Rempel-Clower, 2007) and behavioral distinctions. The similarity in the functional (behavioral) data I present here supports the view that MO and VO may constitute a functional unit or “network” (Price, 2007) in reward learning and decision making. Additionally, consistent with recent anatomical studies (Hoover and Vertes, 2011; Heilbronner et al., 2016), as one moves medially to laterally in rat OFC, there is less involvement of systems linked to affective and motivational states and more involvement of systems supporting sensory integration (i.e., in the associations of stimuli/cues and sensory events to outcomes). This will also be reflected in the functional evidence I review here.

In the following sections, I outline evidence for clusters of function in rat OFC, based in part on a study of stereotaxic coordinates used across reports, and considering recent anatomy papers of rat OFC (Hoover and Vertes, 2011; Heilbronner et al., 2016). Included in this analysis are behavioral studies partnered with a variety of approaches: excitotoxic lesions, electrophysiological recordings, pharmacological inactivations, as well as chemogenetic and optogenetic techniques to target cell populations within rat OFC. As will be noted, chemogenetic and optogenetic targeting of rat OFC is, as yet, quite scarce relative to work using older technologies. In general, the pattern of functional heterogeneity within rat OFC summarized here lends credence to the idea proposed a decade ago by Rempel-Clower (2007), that special attention be paid to the medial-lateral organization of rat OFC (Fig. 2), as this dimension has the most correspondence to the rostral-caudal organization of monkey OFC in goal-directed behavior (Price, 2007).

Figure 2.

Parcellation of rat OFC as determined by anatomical tracing studies. Rat OFC is depicted in this ventral view of the right hemisphere. Adapted with permission from Rempel-Clower (2007). Rat OFC is divided in the MO, VO, VLO, LO, DLO, and AI areas. Numerals on upper right indicate the anterior (right)/posterior (left) distance (in millimeters) from bregma (Paxinos and Watson, 2007).

I include in this Viewpoint only studies focused on appetitive behavior, specifically investigations of food reward learning and decision-making. Here I also focus on discerning functions in rat OFC, exclusively, although it should be mentioned that there are numerous noteworthy investigations on the involvement of mouse OFC in reinforcement processes (Bissonette et al., 2008, 2015; Brigman et al., 2013; Burguière et al., 2013; Gourley et al., 2013, 2016; Gremel and Costa, 2013; Bakhurin et al., 2017; Radke et al., 2017), and associated reviews highlighting contributions of distinct rodent frontocortical subregions to reward-related behavior (Jentsch et al., 2014; Hamilton and Brigman, 2015). One limitation of studying stereotaxic targeting in any species is that the coordinates alone are of limited use without parallel review of the unintended damage or additional spread of infusion that typically occurs with lesions or pharmacological manipulations, respectively. These are typically found in the representative photomicrographs and reconstructions within each report, and can also be gauged from infusion volumes. All of these factors have been considered here and are part of the evaluation in each section below. Finally, I summarize lesion effects together with temporary inactivation effects, although it is known that recovery of function has been demonstrated after cortical lesions where behavioral effects resolve spontaneously, perhaps due to adaptations or “repurposing” of other interconnected cortical regions (Otchy et al., 2015). Such a consideration is indeed a limitation for direct comparison of lesions with transient inactivations. Experimental design is evaluated even more carefully in these cases, to better ascertain the possibility of recovery or transient effects. Standard conventions apply when referring to stereotaxic coordinates: anteroposterior (AP), mediolateral (ML), dorsoventral (DV), with reference to distance in millimeters from bregma according to Paxinos and Watson (2007).

Multiple OFC regions (MO + VO + LO + DLO + AI)

The largest area targeted in rat OFC, by volume, includes very anterior portions of MO and VO frontal cortex (AP 4.0; ML ±0.8, one group had the most anterior target of 4.7) (Boulougouris et al., 2007), VO and VLO (at AP 3.7, and ML within ±2.0), and more posterior VLO (at AP 3.2, and ML ±2.6) (Chudasama and Robbins, 2003; Rudebeck et al., 2006, 2007; Ostrander et al., 2011; Izquierdo et al., 2013). Many of the studies using these coordinates are summarized in Table 1. Only a handful of investigations reported effects following administration of NMDA or ibotenic acid microinfusions (Mar et al., 2011; Ostrander et al., 2011; Izquierdo et al., 2013), whereas most groups delivered quinolinic acid. Although quinolinic acid acts as an agonist at NMDA receptors, specifically NR2A and NR2B (de Carvalho et al., 1996), its excitotoxicity has been linked to other mechanisms beyond activity at the presynaptic receptor, including oxidative stress, inflammation, and mitochondrial dysfunction (Lugo-Huitrón et al., 2013). As such, administration of quinolinic acid may result in longer-lasting, more potent lesion effects.

Table 1.

Targeting of rat MO, VO, LO, DLO frontal cortex and AI cortex^a

Reference	Sex and strain	OFC target	Method	Behavioral task	Finding
Chudasama and Robbins, 2003	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	Visual discrimination and reversal learning	Impaired Pavlovian autoshaping and increased perseveration in reversal learning
Chudasama et al., 2003	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	5-choice serial reaction time	Perseveration when the intertrial interval was long and unpredictable
Winstanley et al., 2004	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	Delay discounting	Increased preference for larger, delayed rewards
Rudebeck et al., 2006	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	Cost-benefit decision making	Intact effort-based choices, increased impulsive choices
Rudebeck et al., 2007	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	Emotional and social behavior	Heightened responses in fear context, increased aggression, correlated with impulsivity
Boulougouris et al., 2007b	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	Spatial reversal learning	Early phase impairment with perseveration
Eagle et al., 2008	Male, Lister Hooded	MO + VO + LO + DLO + AI	Quinolinic acid lesions	Stop signal task	Slower reaction times to stop signal
Ostrander et al., 2011	Male, Long Evans	MO + VO + LO + DLO + AI	Ibotenic acid lesions	Effort-based decision making	Intact effort discounting, impaired cue-guided effort choices
Izquierdo et al., 2013	Male, Long Evans	MO + VO + LO + DLO + AI	Ibotenic acid lesions	Visual discrimination and reversal learning	Intact discrimination learning, impaired reversal learning
Mar et al., 2011	Male, Lister Hooded	MO + VO + LO + DLO + AI	NMDA lesions	Delay discounting	Transient delay aversion, recovered with training

^aStereotaxic coordinates for MO + VO + LO + DLO + AI: Site 1, APf 4.0, ML ±0.8, DV −3.4; Site 2, AP 3.7, ML ±2.0, DV −3.6; Site 3, AP 3.2, ML ±2.6, DV −4.4.

^bThis group had the most anterior site at 4.7.

As might be expected from the coordinates, the studies beginning in the early 2000s listed in Table 1 generally report broad effects on reward-related behaviors. Additionally, behavior across these studies was assessed only in male rats and in two predominant strains, Long–Evans or Lister Hooded. Importantly, although the stereotaxic coordinates were used to target MO, VO, and VLO, visual inspection of reconstructions and representative photomicrographs in these studies demonstrates that LO, DLO, and ventral AI areas also constituted part of the lesions. In the first demonstration of the functional heterogeneity of rat frontal cortex, Chudasama and Robbins (2003) showed that large lesions of OFC resulted in impaired approach to the conditioned stimulus associated with reward (CS⁺) and increased perseveration in reversal learning, defined as repeated errors according to past rules of reinforcement. This was in line with an older literature of large lesions of OFC in monkeys resulting in behavioral inflexibility, a hallmark feature of OFC damage or dysfunction (Butter et al., 1963; Jones and Mishkin, 1972; Kolb, 1984). Many subsequent studies in rats using MO + VO + LO + DLO + AI coordinates corroborate a behavioral rigidity effect across paradigms ranging from the stop signal reaction time task (Eagle et al., 2008), 5-choice serial reaction time task (Chudasama et al., 2003), left-right lever (spatial) reversals (Boulougouris et al., 2007), and reversal of reward contingencies associated with visual stimuli presented on touchscreens (Chudasama and Robbins, 2003; Izquierdo et al., 2013). Although our group found that OFC-lesioned rats required a greater number of sessions to reach criterion for the reversal phase compared with sham-operated controls (Izquierdo et al., 2013), we found no evidence of increased perseveration as reported by others (Chudasama and Robbins, 2003; Chudasama et al., 2003; Boulougouris et al., 2007). Because lesion placements were identical, and given that experimental design and timeline were similar across these studies, it may be that ibotenic acid lesions produced more subtle effects than quinolinic acid lesions, as described above.

These large lesions also produced effects on cost-benefit decision-making, specifically on delay-based tasks. In some cases, groups reported increased choice of smaller-sooner rewards over larger-later rewards (Rudebeck et al., 2006, 2007; Mar et al., 2011) indicating impulsive decision making, yet others found the opposite effect (Winstanley et al., 2004). Later studies clarified a role for OFC following a more specific inactivation of LO in choices involving delay costs (Zeeb et al., 2010): LO inactivations produce impulsive choice when cues are present versus a delay tolerance (or willingness to wait) when cues are absent. One approach to pinpoint OFC function is to assay a comparison behavior in the same animals or within the same experiment, using the same coordinates for OFC. As examples, conclusions can be drawn when investigators report both spared and impaired functions within the same study: Rudebeck et al. (2006) and Ostrander et al. (2011) both showed that rats exhibited intact effort discounting following OFC lesions (e.g., the effort requirement did not produce a steeper discounting for a larger reward relative to controls), but OFC-lesioned rats did show impairments on delay- and cue-guided appetitive behavior, respectively. In summary, the studies involving this broad targeting of OFC in rats reveal involvement in the following: delay, not effort discounting (e.g., selective modulation of impulsive choice via wait time); deterministic (fully predictive) reversal learning, not initial discrimination learning (Izquierdo et al., 2017); and in cue-guided choices involving cost-benefit evaluation. Notably, there is no published evidence of probabilistic reversal learning assessment using these coordinates in rats. The task effects are mapped onto coronal sections of rat OFC (Fig. 3).

Figure 3.

Coronal sections depicting functional heterogeneity within rat OFC. Behavioral task effects are mapped onto coronal sections of rat OFC. Numerals on top of each section indicate the AP distance (in millimeters) from bregma (Paxinos and Watson, 2007).

Other groups have administered similarly large lesions in rats but included much more lateral portions of OFC (>1 mm lateral to those described above) to include neurons in VLO, LO, and both ventral AI cortex (Pickens et al., 2003, 2005). Those effects will be described separately in subsequent sections.

MO

Few investigations of rat OFC have selectively probed MO function (Table 2). Located at the most anterior portion of OFC (AP between 4.4 to 4.0) and with ML between ±0.5 and ±1.6, it is perhaps the most difficult to target from a flat skull surface due to the overlying midsagittal sinus, causing some to use an angled approach (Lopatina et al., 2016). There is diversity in findings from lesions and inactivations of MO. In contrast to results following large OFC lesions, one group reported an increased preference for large delayed rewards following NMDA lesions of MO (Mar et al., 2011) and another found intact delay discounting but increased choice of a risky option following inactivation of MO (Stopper et al., 2014). Again, it is difficult to compare lesion and inactivation methods directly, however, the task methods in these two studies were similar, except for the amount of training received: choice of larger, delayed reward occurred after extensive experience with the task in Mar et al. (2011), but MO-lesioned rats were indistinguishable from controls in early phases of the task, consistent with Stopper et al. (2014). Together, the findings support the idea that MO may be important in updating value (and consequently a cost-benefit ratio) over time.

Table 2.

Targeting of rat MO frontal cortex^a

Reference	Sex and strain	OFC target	Method	Behavioral task	Finding
Fuchs et al., 2004	Male, Sprague Dawley	MO	NMDA lesions or baclofen/muscimol microinfusions	Cue-induced and drug-primed reinstatement of cocaine self-administration	Intact cue-induced reinstatement; attenuated drug-primed reinstatement
Mar et al., 2011	Male, Lister Hooded	MO	NMDA lesions	Delay discounting	Increased preference for large delayed rewards; improved reversal learning
Stopper et al., 2014	Male, Long Evans	MO	Baclofen/muscimol microinfusions	Probabilistic discounting and delay discounting	Intact delay discounting; increased choice of risky option
Burton et al., 2014	Male, Long Evans	MO	Electrophysiological recording	Choice between different reward magnitude and delays	Neurons signal cues predicting lower value outcomes
Bradfield et al., 2015	Male, Long Evans	MO	NMDA lesions and inhibitory DREADDs	Action selection of observable and unobservable outcomes	Reliance on the value of only observable outcomes
Dalton et al., 2016	Male, Long Evans	MO	Baclofen/muscimol microinfusions	Probabilistic discrimination and reversal learning	Impaired probabilistic discrimination learning, increased perseveration, and reduced sensitivity to positive and negative feedback
Lopatina et al., 2016	Male, Long Evans	MO	Electrophysiological recording	Odor-based unblocking	Neurons acquired responses to cues predicting decreases in reward value
Lopatina et al., 2017	Male, Long Evans	MO	Electrophysiological recording	Behavioral states in and across cue-outcome trials	Neurons track different value trial types

^aStereotaxic coordinates for MO: AP 4.4, ML ±0.7, DV −3.2 from dura (Stopper et al., 2014); AP 4.2, ML ±0.6, DV −4.3 (Fuchs et al., 2004); AP 4.2, ML ±0.7, DV −3.2 from dura (Dalton et al., 2016); AP 4.0, ML ±0.5, DV −4.1 (Bradfield et al., 2015); single-unit recordings at coordinates approached at a 13° angle, AP 4.4, ML ±1.58, DV −2.78 from brain surface (Lopatina et al., 2016, 2017).

There is better agreement on the role of MO in value, in conditions of uncertainty. Two groups (Burton et al., 2014; Lopatina et al., 2016), using different behavioral paradigms, found that neurons in MO acquire responses to cues predicting decreases, not increases, in reward value. Bradfield et al. (2015) administered MO lesions or inhibition of MO via Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) and tested rats on action selection involving observable and unobservable outcomes (food rewards, in this case). This group found that taking MO “offline” selectively impaired rats' performance in the unobservable condition. There was also an impairment in responses to instrumental outcome devaluation. Importantly, rats were not impaired if they had access to the outcomes during test, generally supporting the idea that MO is required for maintaining a representation of outcomes when the outcomes are not available. Consistent with the proposal that MO supports behavior in uncertain situations, Dalton et al. (2016) found that inactivation of MO in well-trained rats resulted in impaired probabilistic discrimination and reversal learning, more perseverative errors, and fewer reversals completed per session. One possible explanation is that inactivating MO could render rats more stimulus-dependent (i.e., more reliant on observable stimuli and their outcomes). Overall, the available evidence suggests that MO has an important role in risk and probability discounting, and in supporting value choices when outcomes are either ambiguous or changing in value. One way in which MO may support all these functions is by contributing a representation (e.g., memory) of reward value for use in guiding actions. This may be achieved by stable neuronal ensemble activity in this region (Lopatina et al., 2017).

Is value representation a specialized function of MO or is it part and parcel of a broader OFC function? This remains to be fully uncovered given the relatively few studies on MO to date. The fact that MO and VO share robust corticocortical connectivity (Hoover and Vertes, 2011) helps us predict, however, that MO and VO will share more complementary functions in support of value learning compared with more lateral sectors.

VO and VLO

Few groups have selectively targeted VO (reviewed below). Of all sectors of OFC, VO and VLO are supported by the most mixed evidence. Here, the distinction drawn between VLO versus LO was primarily for the purpose of simplifying comparisons and was determined on the basis of ML extent: VLO constituted coordinates that extended no more than ±2.5 from the midline, at any AP level through OFC (between 4.2 and 3.3; Table 3). Mobini et al. (2002) and Kheramin et al. (2002, 2003, 2005) administered quinolinic acid lesions that also included VO regions predominantly, but with additional intrusion into MO. These investigators tested rats on delay and probability tasks and found that VLO-lesioned rats preferred smaller-sooner rewards (i.e., a delay-averse phenotype) and more certain rewards (i.e., risk-aversion). This is consistent with the effects of large OFC lesions but inconsistent with MO lesions or inactivations that produce no effect or even improvements over time. This suggests it is VLO (not MO) that supports value over increasing delays, contributing to normal delay discounting performance.

Table 3.

Targeting of rat VO and VLO frontal cortex^a

Reference	Sex and strain	OFC target	Method	Behavioral task	Finding
Mobini et al., 2002; Kheramin et al., 2002, 2003, 2005	Female, Wistar	VO + LO	Quinolinic acid lesions	Delayed and probabilistic reinforcement; intertemporal choice	Preference for small immediate, certain rewards; increased sensitivity to ratio of option magnitude differences
Young and Shapiro, 2011	Male, Long Evans	VLO	Electrophysiological recording	Spatial reversals and strategy switches in a plus maze	Firing distinguished rewarded paths and correlated with performance as rats learned newly rewarded outcomes
Steiner and Redish, 2012, 2014	Male, Fisher Brown Norway	VLO	Electrophysiological recording	Continuous loop multiple choice maze	Firing signals expected outcomes; firing signals a “missed” option
McAlonan and Brown, 2003; Tait and Brown, 2007; Chase et al., 2012	Male, Lister Hooded	VLO	Ibotenic acid lesions	Attentional set formation and shifts and reversal learning	Slower to acquire attention set, more trials to shift, and impaired reversal learning
Meyer and Bucci, 2016	Male, Long Evans	VLO	Inhibitory DREADDs	Negative occasion setting	Underactivity decreases inhibitory learning
Jean-Richard-Dit-Bressel and McNally, 2016	Male, Sprague Dawley	VLO	Baclofen/muscimol microinfusion	Aversive instrumental learning and decision making	Impaired use of aversive instrumental memory
Riceberg and Shapiro, 2017	Male, Long Evans	VLO	Electrophysiological recording	Spatial learning in a plus maze that varied frequency of reversal	Firing signals past, present, and pending events when contingencies are stable
Stolyarova and Izquierdo, 2017	Male, Long Evans	VO	NMDA lesions	Learning value distributions and adjusting to changes in those distributions	Failure to represent the full delay distribution and inability to adjust to better than expected outcomes
Hardung et al., 2017	Male, Sprague Dawley	VO	Optogenetic inhibition and electrophysiological recording	Stop signal task	Slower reaction times to stop signal

^aStereotaxic coordinates for VO + LO: Site 1, AP 3.7, ML ±1.2, DV −4.8; Site 2, AP 3.7, ML ±2.8, DV −4.4. Stereotaxic coordinates for VLO: tetrode recordings at AP 4.0, ML ±2.4, DV −3.0 to −3.5 (Young and Shapiro, 2011), and similar coordinates for tetrode recordings at AP 3.5, ML ±2.5 (Steiner and Redish, 2012, 2014). Lesion coordinates: AP 4.0, ML ±2.0, DV −4.5, from skull surface (McAlonan and Brown, 2003; Tait and Brown, 2007; Chase et al., 2012). Expression of DREADDs reporter: Site 1, AP 4.5, ML ±0.5, DV −4.9; Site 2, AP 4.2, ML ±2.2, DV −4.8; Site 3, AP 3.3, ML ±2.2, DV −5.6 (Meyer and Bucci, 2016). Guide cannulae implanted at a 10° angle at AP 4.5, ML ±1.7, DV −3.9 (Jean-Richard-Dit-Bressel and McNally, 2016). Stereotaxic coordinates for VO: one group selectively targeted VO (separately from MO and LO), AP 3.24, ML ±2.0, DV −4.4 (Hardung et al., 2017). Our laboratory targeted VO, but with additional intrusion in MO: AP 3.7; ML ±2.0; DV −4.6 (Stolyarova and Izquierdo, 2017).

VLO has also been the focus of neurophysiological recording studies by several different groups (Young and Shapiro, 2011; Steiner and Redish, 2012, 2014; Riceberg and Shapiro, 2017). In general, there is converging evidence that neurons in VLO signal an expected outcome (Steiner and Redish, 2012), a missed outcome (Steiner and Redish, 2014), and that neuronal firing in VLO is engaged when stimulus-outcome contingencies become stable with experience (Riceberg and Shapiro, 2012, 2017). This is a difference with MO, where changes are a more important feature than stability. A mechanism for this “stabilizing” feature in VLO is yet to be determined, but one possibility is that it involves formation of a learning set (described by Izquierdo et al., 2017). For example, Brown and colleagues reported evidence that OFC (in particular, VLO) is important for the acquisition of attention sets and overcoming nonreward in reversal learning (McAlonan and Brown, 2003; Tait and Brown, 2007; Chase et al., 2012). Lesions of VLO render the animal in need of more evidence to adjust behavior (i.e., more trials to shift) (Chase et al., 2012), which authors explain as a failure in linking relevant cues to unexpected outcomes but could also be attributed to an impaired ability to stabilize stimulus-outcome contingencies over time.

An interesting recent finding (Meyer and Bucci, 2016) probing the role of VLO made use of a “negative occasion setting” paradigm, a behavioral task that requires the rat to monitor the context in which it needs to act or withhold response. Using an inhibitory DREADDs approach, Meyer and Bucci (2016) found that inhibiting VLO function disrupts the ability to discriminate the context in which to act (i.e., withhold response). Similarly, another group (Jean-Richard-Dit-Bressel and McNally, 2016) trained rats to earn food rewards on two levers presented individually. One of the levers delivered mild punishment, the other, only reward. VLO inactivation did not impair initial learning (rats decreased their responses to punished lever) but did increase responding to the punished lever once stimulus-outcome associations had already been well established. It is still unclear how VLO supports such flexible responding. Supporting an earlier result by Eagle et al. (2008), Hardung et al. (2017) found that selective optogenetic inhibition of VO impaired inhibitory control on the stop signal reaction time task, which authors describe as a lack of “reactive movement control.” Indeed, this finding is consistent with the behavioral rigidity or enhanced perseveration resulting from large lesions of OFC, yet also consistent with the idea that VLO supports either the “occasion” on which to act, or the remapping of relevant cues to outcomes, as described above.

Our laboratory has a recent finding (Stolyarova and Izquierdo, 2017) demonstrating that VO may be particularly important in learning about value under conditions of delay uncertainty. In our task, rats were required to select between two visual stimuli associated with different delay distributions to reward. The stimuli were associated with the same mean wait times to reward and imposed the same reward rate over the session, only the variance of the delays differed. One stimulus was associated with a highly variable wait time and another with a relatively stable wait time. Unlike control animals, rats with NMDA lesions of VO visited the food magazine closest to the average time when reward was expected, failing to represent the more extreme instances of the highly variable distribution (i.e., shortest and longest wait times possible). Lacking access to a representation of the richness of the outcome distribution, VO rats over-rely on the average cached value. We interpret these findings as providing additional evidence that this sector of OFC may be involved in more than motor inhibition but rather that it exerts a more complex function in value learning when outcomes are not easily predicted.

Given the evidence reviewed here, VO and VLO may support reward learning and performance over increasing delays (i.e., delay discounting), the learning of the variance of value distributions over time (i.e., learning of risk), and the conditions (context or state) in which outcomes are expected. All these functions likely support the stabilization of expectations necessary for appropriate behavioral responses to changing or surprising events. These are similar to the functions of MO, yet there is appreciably more emphasis on cue-guided learning in VO and VLO studies, a trend that continues as one moves more laterally through rat OFC. Critically, this may result from an overemphasis on the study of cue-outcome association learning, with few investigations probing the role of VO and VLO in cue-action and action-outcome coding in mouse (compare Gremel and Costa, 2013).

LO

By far the most well explored area of OFC, investigated by a number of different groups, is LO. The Schoenbaum laboratory, in particular, has made significant progress in understanding the role of this region, as well as the role of LO-AI, which I define in the next section as a more lateral target than LO for convenience, to simplify comparisons and discussion. It should be noted, however, that LO and AI are frequently targeted in tandem (i.e., LO targets often also include AI). For the purpose of this Viewpoint, studies were classified as primarily LO studies on the basis of ML coordinates that extended more laterally than VLO, between ±2.5 to ±3.2 from the midline, at any AP level. If studies are clustered in this way (Table 4), it becomes clear that many investigations of LO have probed reversal learning, outcome prediction, and confidence estimates in rats. Many of these cognitive processes have been recently reviewed (Kepecs et al., 2008; Kepecs and Mainen, 2012; Rudebeck and Murray, 2014; Izquierdo et al., 2017), revealing a great deal of convergence between rat and monkey studies. In the rat, it has now been shown that LO is selectively involved in reversal learning (both deterministic and probabilistic), but not discrimination learning (Kim and Ragozzino, 2005; Dalton et al., 2016; Amodeo et al., 2017). Impaired reversal performance following LO inactivation also occurs in the context of Pavlovian responding (Burke et al., 2009), not just instrumental responding. LO also has a role in delay discounting: increasing preference for small, immediate over large, delayed rewards (Mar et al., 2011). As mentioned before, it was Zeeb et al. (2010) that made the important determination that the involvement of LO in delay discounting depends on whether there are explicit cues to signal delays to reward.

Table 4.

Targeting of rat LO frontal cortex^a

Reference	Sex and strain	OFC target	Method	Behavioral task	Finding
Fuchs et al., 2004	Male, Sprague Dawley	LO	NMDA lesions or baclofen/muscimol microinfusions	Cue-induced and drug-primed reinstatement of cocaine self-administration	Impaired conditioned cue-induced reinstatement; enhanced perseveration in drug-primed condition
Kim and Ragozzino, 2005	Male, Long Evans	LO	Muscimol microinfusions	Two- and four-choice discrimination and reversal learning	Intact discrimination learning, impaired two- and four-choice reversal learning with increased perseverative errors
Kepecs et al., 2008	Male, Long Evans	LO	Electrophysiological recording	Postdecision confidence estimation in willingness to wait	Firing rates of single neurons match predictions of confidence models
Takahashi et al., 2009; Burke et al., 2009	Male, Long Evans	LO	Baclofen/muscimol microinfusions	Pavlovian overexpectation; Pavlovian learning	Impaired learning from expected outcomes; impaired reversal of Pavlovian responding
Zeeb et al., 2010	Male, Long Evans	LO	Baclofen/muscimol microinfusions	Cued and noncued delay discounting	Increased impulsive choice when the delay was cued; decreased impulsive choice when delay was uncued
Mar et al., 2011	Male, Lister Hooded	LO	NMDA lesion	Delay discounting	Increased preference for small immediate rewards; attenuated reversal learning
Takahashi et al., 2013; Lucantonio et al., 2014	Male, Long Evans	LO	Electrophysiological recording and optogenetic inhibition/activation	Pavlovian overexpectation	Activity to cue-responsive neurons increases with a compound cue and declines with learning; a pattern abolished with prior cocaine history
Stott and Redish, 2014	Male, Fisher Brown Norway	LO	Electrophysiological recording	Spatial-adjusting delay-discounting task	Reward-related signals occurred after choice
Lak et al., 2014	Male, Long Evans	LO	Muscimol microinfusions	Postdecision confidence estimation in willingness to wait	Intact decision accuracy, disrupted waiting-based confidence report
Moorman and Aston-Jones, 2014	Male, Long Evans	LO	Electrophysiological recording	Context-based response inhibition	Activity most robust to reward-predicting stimuli; activity tracked reward when contingencies reversed
Lopatina et al., 2015	Male, Long Evans	LO	Electrophysiological recording	Odor-based unblocking task	Firing early in learning to valued cues consistent with outcome features
Ramírez-Lugo et al., 2016	Male, Wistar	LO	Baclofen/muscimol microinfusions	Conditioned taste aversion choice/no choice	Impaired choice behavior guided by learned taste aversion, but not retrieval of taste memory
Dalton et al., 2016	Male, Long Evans	LO	Baclofen/muscimol microinfusions	Probabilistic discrimination and reversal learning	Intact discrimination learning, impaired reversal learning
Nogueira et al., 2017	Male, Wister	LO	Electrophysiological recording	Auditory time-interval task	Signals choice before stimulus onset
Amodeo et al., 2017	Male, Sprague Dawley	LO	Electrophysiological recording	Probabilistic reversal learning	Signals reflect changes in contingencies

^aStereotaxic coordinates for LO: guide cannulae implanted AP 3.7, ML ±2.6, DV −4.7 (Ramírez-Lugo et al., 2016) or AP 3.8, ML ±2.6, DV −2.9 from dura (Zeeb et al., 2010). Tetrode recordings (Kepecs et al., 2008; Stott and Redish, 2014) at AP 3.5, ML ±2.5, with similar coordinate recordings at AP 3.7, ML ±2.5 (Nogueira et al., 2017). Lesions at AP 3.2, ML ±2.5, DV −3.6 (Fuchs et al., 2004; Mar et al., 2011). Inactivations at AP 3.8, ML ±2.6, and DV −3.2 from dura (Dalton et al., 2016). Inactivations slightly more anterolaterally at AP 4.0, ML ±3.2, and DV −4 from dura (Kim and Ragozzino, 2005). More posterolateral inactivations at AP 3.0, ML ±3.2, and DV −5.0 (Burke et al., 2009; Takahashi et al., 2009). Dual guide cannulae were implanted 1.5 mm above LO at AP 4.1, ML ±2.8, DV −1.8 from dura, and AP 3.2, ML ±3.2, DV −2.8 from dura (Lak et al., 2014). Recording electrodes and optic fibers placed at AP 3.0, ML ±3.2, DV −4.2 from brain surface (Takahashi et al., 2013; Lucantonio et al., 2014) and slightly more dorsally at −3.9 (Lopatina et al., 2015) and −4.0 (Amodeo et al., 2017). Electrodes placed at similar coordinates, although more ventrally, AP 3.2, ML ±3.2, DV −5.2 (Moorman and Aston-Jones, 2014).

Highlighting the relevance of cues in LO function are addiction studies (Fanous et al., 2012). There are relatively few published reports of rat LO versus MO in studies involving drug reward (Fuchs et al., 2004; Lucantonio et al., 2014). One such finding was by Fuchs et al. (2004) showing that NMDA lesions of LO impaired cue-induced reinstatement of cocaine self-administration, whereas MO lesions left cue-induced reinstatement intact. Another group studied Pavlovian overexpectation, a task where outcomes (food rewards) may not be directly experienced during training but instead must be inferred or estimated when the rat is later exposed to a compound cue. Using this paradigm, it was determined that activity in cue-responsive neurons in LO increases initially with a compound cue and then declines with learning. Interestingly, this pattern is abolished in rats with a history of cocaine self-administration and is partnered with behavioral impairments that are rescued via optogenetic activation of LO (Lucantonio et al., 2014).

In many studies of LO function, cues are present in the task as prominent stimuli that enable the animal to predict outcomes. Some research groups focus their study of neural activity before and during cue onset (Moorman and Aston-Jones, 2014; Lopatina et al., 2015; Nogueira et al., 2017), whereas others are more interested in the signal after the rat selects a response (the “postdecision” epoch) (Kepecs et al., 2008; Lak et al., 2014; Stott and Redish, 2014). Overall, there is convincing evidence that LO supports cue-based outcome prediction as well as decision confidence estimates in rats, as measured by rats' willingness to wait for reward.

Given the evidence that MO and VO project rather sparsely to LO compared with other cortical regions (Hoover and Vertes, 2011), LO may rely less on these corticocortical motivational or affective state signals and more on inputs directly from piriform cortex, rhinal cortex, and basolateral amygdala conveying sensory features about cues that predict specific outcomes. This could support high-resolution cue-based outcome predictions.

LO-AI

More lateral LO-AI is also well explored and features much converging evidence. Designation of LO-AI was determined on the basis of the most lateral ML coordinates in rat OFC: between ±3.2 to ±4.2 from the midline, at any AP level of OFC, although targeting of this subregion is generally more caudal. Several studies have probed ventral and dorsal AI in addition to LO (Table 5), which collectively receive dense projections from piriform cortex and basolateral amygdala (Price, 2007; Wassum and Izquierdo, 2015). NMDA lesions more restricted to LO-AI have resulted in intact discrimination learning but impaired reversal learning (Schoenbaum et al., 1999, 2000, 2003; Bohn et al., 2003a, b). Additionally, LO lesions do not produce impairments in the ability to discriminate between different reward magnitudes (Bohn et al., 2003a; Orsini et al., 2015). More recent studies have refined the role of LO and LO-AI to support reversal learning only under particular conditions, specifically in low-frequency reversals (Riceberg and Shapiro, 2012). For example, if rats have fewer experiences with switching reward contingencies, OFC lesions impair reversal learning and lead to more perseverative errors; but if rats already have many more experiences with reversals beforehand, OFC lesions actually improve performance. This suggests that the most lateral sectors of OFC are important in (1) keeping track of reward statistics and/or (2) stabilizing the reward distribution to represent the average outcome. Both possibilities would be important in updating behavior in changing environments. There are several similarities here with findings in VO and VLO. When recording from VLO, Riceberg and Shapiro (2017) found evidence that OFC does indeed maintain a record of reward history: signaling in VLO was predictive of future behavior only if the reward statistics were stable.

Table 5.

Targeting of rat LO frontal cortex and AI cortex^a

Reference	Sex and strain	OFC target	Method	Behavioral task	Finding
Gallagher et al., 1999; Pickens et al., 2003, 2005	Male, Long Evans	LO-AI	NMDA lesions	Reinforcer devaluation	Impaired ability to associate cues with updated value; difficulty maintaining representations
Schoenbaum et al., 1999, 2000, 2003	Male, Long Evans	LO-AI	NMDA lesions	Odor discrimination and reversal learning	Intact discrimination learning, impaired reversal learning
Bohn et al., 2003a,b	Male, Sprague Dawley	LO-AI	NMDA lesions	Reaction time task with different magnitude outcomes	Intact discrimination, impaired stimulus-reward magnitude reversal learning
Ostlund and Balleine, 2007	Female, Long Evans	LO-AI	NMDA lesions	Instrumental outcome devaluation and Pavlovian-instrumental transfer	Intact instrumental outcome devaluation, impaired Pavlovian-instrumental transfer
McDannald et al., 2011	Male, Long Evans	LO-AI	NMDA lesions	Odor-based unblocking	Impaired learning of reward identity
McDannald et al., 2014	Male, Long Evans	LO-AI	Electrophysiological recordings	Odor-based unblocking	Firing to identity cues, independent of value information
Riceberg and Shapiro, 2012	Male, Long Evans	LO-AI	NMDA lesions	Spatial learning in a radial arm maze that varied frequency of reversal	Slower to learn low-frequency reversals but quicker to learn high-frequency reversals
Orsini et al., 2015	Male, Long Evans	LO-AI	NMDA lesions	Punishment risk	Intact discrimination of reward magnitude, decreased risk-taking

^aStereotaxic coordinates for LO-AI: Site 1, AP 4.0, ML ±2.2 and ±3.7, DV −3.8 from skull surface; Site 2, AP 3.0, ML ±3.2 and ±4.2, DV −5.2 from skull surface (Gallagher et al., 1999; Schoenbaum et al., 1999, 2000; Pickens et al., 2003; Schoenbaum et al., 2003; Pickens et al., 2005; McDannald et al., 2011; Riceberg and Shapiro, 2012) for lesions. One group used a similar site for their lesions, AP 3.5, ML ±3.2, DV −4.2 (Ostlund and Balleine, 2007), but more medially restricted. Electrodes were also implanted at a similar site, AP 3.0, ML ±3.2, DV −4.0 from skull surface (McDannald et al., 2014; Jo and Jung, 2016). A similar set of coordinates (Riceberg and Shapiro, 2012) was used by Bohn et al. (2003a, b) for lesions at identical AP and ML coordinates, but different DV (−5.0 and −5.8 from skull surface). Another group administered lesions at AP 3.7, ML ±2.1 and ±3.6, DV −4.4 and −4.5, respectively; AP 3.0, ML ±2.6 and ±3.9, DV −5.2 and −5.0, respectively (Orsini et al., 2015).

Another function of LO-AI is in adaptive responses to reinforcer devaluation, when rats are assessed after selective satiation with a particular reward. Lesions to this region impair the ability to associate cues with an updated value (Gallagher et al., 1999; Pickens et al., 2003; Pickens et al., 2005; Ostlund and Balleine, 2007). More recent investigations of this region have probed Pavlovian unblocking (McDannald et al., 2011, 2014). In this procedure, rats are trained to associate different cues with either an increase in reward magnitude (a change in both reward identity and value) or a change in the flavor of the reward (change in reward identity without a change in value, confirmed by behavioral test). The cues are then presented in compound with novel cues where the reward magnitude or identity is changed. Unblocking was assessed by presenting the cues alone in a probe test. It was determined that some LO neurons acquire responses to cues predicting both value and “valueless” (simple reward identity) changes. This is support that rat LO/LO-AI is involved in signaling expected outcomes based on sensory and/or perceptual features, even apart from their value.

How do the roles of these more lateral sectors of OFC in responses to reinforcer devaluation and in reversal learning fit with the proposed roles for MO and VO/VLO described earlier? To respond appropriately to both changes in value and changes in contingencies, one would need access to value representations and stable expectations acquired through experience. So even if MO and VO projections to LO are relatively sparse compared with other cortical regions, this information appears to be important to LO function. Another intriguing possibility is that the functions of MO and VO are redundant to LO (considered in more detail below).

In summary for LO-AI, there may be little functional difference between LO and more laterally targeted LO-AI in reward learning and decision making in the rat, although there is still a need to distinguish the role of LO from AI in appetitive behavior.

Future directions in rat OFC

What remains to be studied in rat OFC? As shown in Figure 3, reversal learning effects have been reported following targeting of most sectors of OFC, in various laboratories, and across sensory modalities. Of the variants used, deterministic, fully predictive reversal learning tasks have been administered following manipulations and recordings in much of rat OFC (for review, see Izquierdo et al., 2017), with many fewer assessments of probabilistic reversal learning (Dalton et al., 2016). Indeed, many of the studies discussed here assessed well-trained rats for their performance after extensive experience on task. More experiments assessing OFC function in rats while learning about uncued or unobservable outcomes would be welcome. Similarly, there appears to be a shortage of investigations of responses to devaluation in rat VO and LO, confidence estimates in rat MO, reward magnitude and reward identity discrimination in MO and VO, and strategy/set shifting and delay discounting in rat LO. Importantly, as shown in Tables 1–5, almost all of the experiments included in this Viewpoint have been conducted in male, pigmented strains (mostly in Long–Evans), with very few female rat subjects. This may change with a new awareness that female rats are not more variable than males and should be included in future research (Becker et al., 2016).

Moving forward, the biggest concern for the future of rat OFC is the influence of confirmation bias because there are, at present, many theoretical viewpoints driving the empirical work, with increasingly sophisticated computational models fit to behavior. Additionally, as described earlier, there is an oversampling of certain subregions for study, which I have attempted to highlight. Greater exploration should be conducted in MO and VO, and knowledge of rat neuroanatomy should remain at the center of future experimental design and inquiry.

Related to the point above, a decade ago, Price (2007) expressed a need for the systematic study of corticocortical connections in rat OFC. At present, these corticocortical connections still remain to be more comprehensively investigated, although there has been progress (Hoover and Vertes, 2011). Several neuroanatomists have now proposed that MO and VO may function as a bridge between LO and the medial wall of the PFC (mPFC) (Price, 2007; Hoover and Vertes, 2011; Heilbronner et al., 2016). The behavioral evidence summarized here also supports the idea that MO is most linked to motivational states, with best access to areas coding incentive value, such as the nucleus accumbens. As such, MO and perhaps also VO have better access to values that are in flux or outcomes that are uncertain, as in the salience of early learning or in rapidly changing, volatile environments. One possibility mentioned above is that the functions of MO and VO are redundant to LO. This is an interesting possibility in light of the idea that cortical representations are hierarchical and recurrent in supporting reward choices: value computations are emergent properties that take shape across time and across cortical regions (Hunt and Hayden, 2017), with no real functional localization or modules per se. Alhough derived largely from studies of human and nonhuman primate cortex, moving forward, exploration of rat OFC could benefit from this refreshing perspective to generate testable hypotheses.

Also required for the future is a more direct comparison of LO and AI. Importantly, some groups have initiated the much-needed investigation of the distinction between LO and AI in appetitive behavior (Jean-Richard-Dit-Bressel and McNally, 2016; Jo and Jung, 2016; Parkes et al., 2017). Finally, although the list is growing quickly (Lichtenberg et al., 2017; Wikenheiser et al., 2017), there are far fewer cell-specific circuit dissection techniques directed at rat OFC compared with more traditional approaches. These newer technologies would ideally be leveraged with concurrent assessment of sophisticated rat behavior.