Behavioral flexibility in rats and mice: Contributions of distinct frontocortical regions

Abstract

Research examining the contribution of genetics to behavior is increasingly focused on higher order behavioral and cognitive processes including the ability to modify behaviors when environmental demands change. The frontal cortices of mammals, including rodents, subserve a diverse set of behavioral and cognitive functions including motor planning, social behavior, evaluation of expected outcomes, and working memory which may be particular sensitive to genetic factors and interactions with experience (e.g. stress). Behavioral flexibility is a core attribute of these functions. This review orients readers to the current landscape of the literature on the frontocortical bases of behavioral flexibility in rodent laboratory experiments. Studies are divided into three broad categories: reversal learning, inhibitory learning, and set-shifting. Functional dissociations within the broader scope of behavioral flexibility are reviewed, followed by discussion of the associations between specific components of frontal cortex and specific aspects of relevant behavioral processes. Finally, the authors identify open questions that need to be addressed to better establish the constituents of frontal cortex underlying behavioral flexibility.

1. INTRODUCTION

Research examining the contributions of genetics to behavior has become increasingly focused on higher order behavioral and cognitive processes, including the ability to plan actions, attend to relevant stimuli and modify behaviors when environmental demands change. The capacity for behavioral flexibility has been widely used as a measure of “executive” control in a broad range of mammalian species including humans, monkeys, rats, and mice. Further, the frontal cortices of mammals, including rodents, subserve a diverse set of behavioral and cognitive functions for which behavioral flexibility is a core attribute, including motor planning, social behavior, evaluating expected outcomes, and working memory.

Based on clinical case studies and elegant work done in non-human primates, the last 20 years have seen an increased focus on understanding the neuronal circuits and cortical regions underlying behavioral flexibility. Rodent studies are critically important for identification of neural systems/circuits and genetic factors relevant to behavioral flexibility, understanding abnormal processes, and evaluating therapeutic approaches. They are also an important tool for examining how genetics may interact with environmental factors including developmental insult, learning experience and stress. Inducible and conditional manipulation of genes is becoming increasingly more refined as a tool for investigating relationships between genes, nervous system processes, and behavior. There is a commensurate escalation in the need for basic research using rodents to be guided by consideration of how distinct areas of the frontal cortex contribute to different forms of flexible behavior.

Functional Divisions of Rodent Frontocortex

Broadly speaking, rodent frontal cortex can be divided into medial and lateral/orbital regions. Medial frontal cortex (MFC) can further be subdivided into anterior cingulate (Ac), infralimbic (IL) and prelimibic (PrL) subregions. Similarly, orbital frontal Cortex (OFC) is by convention subdivided into medial (MO), ventral (VO) and lateral (LO) components as well as the contiguous agranular insular (AI) cortex (Figure 1). MFC and OFC and the subregions that comprise them are generally similar in terms of cytoarchitectonics and connectivity in mice and rats. Studies examining frontocortical contribution to behavior commonly target whole regions (e.g., MFC or OFC), or one or more specific subregions within these areas (Uylings et al., 2003, Van De Werd et al., 2010). For the purposes of this review we discriminate subregions of OFC and MFC using well established coordinate systems and associated atlases of rat and mouse brain (Paxinos & Franklin, 2001, Paxinos & Watson, 2005) with the goal of addressing which aspects of OFC and MFC are required for and/or engaged by tasks that require behavioral flexibility.

Figure 1. Coronal sections demonstrating characterization of location of recording or manipulation in frontal cortex subregions in rat and mouse.

Serial coronal sections identified by coordinate (anterior to Bregma) showing the representative frontocortical subregions in the rat and mouse targeted in manipulation and recording studies examining flexible behavior. The areas denoted are: prelimbic cortex (PrL), infralimbic cortex (IL), anterior cingulate cortex (Ac), medial orbital cortex (MO), ventral orbital cortex (VO), lateral orbital cortex (LO) and agranular insular cortex (AI). Adapted from Paxinos & Franklin (2001; 2005).

Homology of Rodent and Primate Frontal Cortex

Homologies between rodents and primates are important to any discussion regarding cortical function in rats and mice. A comprehensive treatment of this issue would divert the major goals of the present review. However, the apparent conservation of processes involved in behavioral flexibility, and the associated neural bases, across species suggests that current debate on the functions of primate frontal cortex should guide thinking and advances in research on rodent MFC and OFC function. Recent studies in primates have called into question the long-held belief that the OFC is critically important for successful reversal learning. In comparison to surgical ablation, neurotoxic lesions that spare fibers of passage do not impair reversal learning, except when targeting a specific area of posterior OFC (Rudebeck et al., 2013). These results suggest that the traditional role of OFC in reversal learning may not be quite so clear, and also suggest that lesion studies in rodents need to be carefully examined both for technique, and the targeted subregion of OFC (Young et al., 2013). This review will strive to summarize the rodent literature with a close examination of both factors, and shed further light on the role of rodent OFC both in reversal, as well as value updating in other tasks of behavioral flexibility (Baxter & Croxson, 2013).

Scope of Current Review

Modular approaches have been used successfully to understand the diverse functions of the subregions of frontal cortex. Although important and relevant to a broader range of issues than addressed here, these issues are not addressed directly in the present review. We refer readers to extant review or opinion articles on these topics as they may prove useful for guiding neurogenetics research (Bukalo et al., 2014, Churchwell & Kesner, 2011, Floresco et al., 2009, Schoenbaum et al., 2009a, Schoenbaum & Setlow, 2001, Uylings et al., 2003).

Rather than emphasizing specific cognitive domains (e.g., working memory, spatial problem solving, social cognition), the goal of the current review is to orient readers to the current landscape of the literature on the frontocortical bases of behavioral flexibility in rodent laboratory experiments. We provide an overview of functional dissociations, discussing which components of frontal cortex are linked with specific aspects of relevant behavioral and cognitive processes. The behavioral domains of interest are broadly divided into three general classifications: reversal learning, inhibitory learning (e.g., extinction) and set-shifting. For each of these categories we review behavioral preparations and associated behavioral modifications that are observed when previously established contingencies are modified. An additional goal is to orient the reader to important open questions related to the regional specificity of brain-behavior relationships. Among these are questions arising from the selection of non-trivial parameters including stimulus modalities, response forms, outcome types, and task difficulty. While available data allow for some conclusions regarding specificity and generality of MFC and OFC function, there are important gaps that need to be addressed in future research. Successfully addressing these issues will better establish which constituents of frontal cortex are critically recruited and required for specific processes involved in behavioral flexibility, and should be considered when designing and evaluating studies on the neurogenetic bases of behavioral flexibility.

2. REVERSAL LEARNING

The studies discussed in this section share in common that a single response among at least two options was reinforced (or elicited) during an initial learning phase, and after a criterion level of performance was achieved the response-outcome or stimulus-outcome contingency was reversed. Emphasis is placed on studies that include both initial acquisition data and data for individual contingency reversal phases, particularly during an initial reversal. For lesion studies we primarily include studies that induced damage to the circuits of interest bilaterally, using neurotoxins that damage cells within the region while leaving fibers of passage intact. Findings from reversible inactivation, stimulation, electrophysiological recording, and neurotransmitter depletion studies are also included when available. Data from studies of the role of the distinct frontal regions of interest (AI, LO, VO, MO, IL, PrL, and Ac) are summarized in Figure 2 by subregion, species/strain, approach (e.g. lesion), and stimulus type. Because of the breadth and diversity of stimulus modalities used in reversal learning tasks we have organized this section around the various modalities that have been utilized.

Figure 2. Effects of frontal cortex subregion manipulation or recording on contingency reversal tasks.

Closed circles (●) denote a manipulation of a particular subregion or collection of subregions yielded a pattern of impaired reversal performance during reversal training, or that recording data were related to contingency reversals. Open circles (○) denote lack of an impairment on reversal learning or lack of relationship between recordings and reversal performance. Lines that connect individual circles (●--●) indicate a manipulation or recording preparation including all subregions connected by the line. Grey circles (●) and lines (●--●) indicate results that require additional clarification of the deficit that are more fully discussed in corresponding section 2 text. Species/Strain: R= Rat, M= Mouse, LE= Long Evans, LH= Lister Hooded, SD= Sprague Dawley, B6= C57Bl/6J, 129= 129S1/SvIMJ, Balb= BALB/c. Approach: C= context, V= visual, A= auditory, O= olfactory, T= tactile, S= spatial. Areas: PrL= prelimbic cortex, IL= infralimbic cortex, Ac= anterior cingulate cortex, MO= medial orbital cortex, VO= ventral orbital cortex, LO= lateral orbital cortex and AI= agranular insular cortex.

Olfactory and Tactile

A considerable number of reversal learning studies in rodents have utilized olfactory and/or tactile stimuli to train an initial discrimination followed by contingency reversal. For example, McAlonan and Brown (2003) found that neurotoxic lesions of combined VO/LO in the rat impaired reversal learning, but not initial learning, in a digging task (for food reinforcement), where the relevant stimuli were odors or tactile stimuli. In contrast, using a similar task Birrell and Brown (2000) found that IL/PrL lesions in the rat did not impair acquisition or reversal learning, suggesting a dissociation between MFC and OFC with respect to reversal learning involving tactile or olfactory stimuli. Using a similar task, Bissonette et al. (2008) reported evidence for an OFC-MFC dissociation in reversal learning in the olfactory/tactile domain in the mouse. NMDA lesions of LO, that also included damage to VO in some subjects, impaired reversal of an initial contingency for both cue modalities, whereas damage to IL/PrL did not impair reversal for either modality. Importantly, data consistent with these outcomes has also been obtained following inactivation of the regions of interest. For example, Churchwell et al. (2009) inactivated LO or PrL in rats with the GABA agonist muscimol during reversal of olfactory discriminations and observed impairments with LO inactivation but not PrL inactivation.

Other studies have observed similar results utilizing different response forms (e.g., lever pressing) in olfactory discrimination reversal learning tasks. Using a task in which rats pressed a lever for water reinforcement cued by odor stimuli, Schoenbaum et al. (2002) reported that lesions of LO/VO did not impair initial learning but significantly impaired learning when the odor stimuli were reversed. Thus, the presence of reversal learning deficits following OFC lesions does not appear to be specific to response form (lever pressing vs. digging) or reinforcer type (e.g., food vs. water). Schoenbaum et al. (2002) also evaluated performance when the contingencies were reversed serially two additional times. On the final contingency reversal rats with LO/VO lesions met criterion faster than controls, suggesting a savings effect for S-R contingencies that had already been established. A subsequent experiment revealed that neurotoxic lesions of AI/LO/VO also impaired reversal learning with odor discriminations in this task (Schoenbaum et al., 2003a). Recently, Cordova et al. (2014) reported no deficits in reversal learning with olfactory stimuli following MFC stroke that resulted in damage to IL/PrL/AC, further suggesting that reversal learning with olfactory stimuli does not require MFC.

Observations from electrophysiological studies have also indicated a role for OFC neurons in reversal learning with odor stimuli. For example, Schoenbaum et al. (2000) performed simultaneous recordings of neurons in LO/VO and basolateral amygdala during an odor discrimination task and observed increases in correlated activity during reversal of an initially established stimulus-outcome association (odor-sucrose). A recent report from Bissonette et al. (2014b) also found that LO/VO neurons in the mouse respond to odor discrimination performance, reversal performance, or exclusively with respect to the expected outcome.

Collectively, the available data provide multiple examples from several laboratories using mice or rats that demonstrate a dissociation between MFC and OFC with respect to reversal learning with olfactory or tactile stimuli. One notable exception was reported by Ferry et al. (2000), who trained rats to discriminate between two odors for water reinforcement. After successful learning, rats were given lesions of either PrL/IL/MO or VO/LO/AI. Both lesions impaired learning during reversal, however, we note that the MFC lesion included MO, which could account for the failure to observe a clear dissociation.

Visual Discrimination Tasks

A large number of studies have examined reversal learning with distinct, conspicuous visual stimuli that signal which responses are reinforced or not. The majority of these studies have consistently demonstrated that manipulations of OFC disrupt reversal learning, while, with few exceptions, manipulations of MFC do not or actually enhance performance. For example, using a touchscreen apparatus in which mice performed a visual discrimination (contact S+), Graybeal et al. (2011) found that NMDA lesions of LO impaired reversal learning, resulting in a greater number of perseverative errors during the reversal phase. Neurotoxic lesions of LO, VO/MO or AI in the rat have also been found to disrupt reversal of nose-poke responses directed toward visual stimuli (Chudasama & Robbins, 2003). Another study used a variation on the visually cued discrimination task that trained rats to press and hold a lever to initiate visual stimulus that signaled whether either a large (5 pellets) or small (1 pellet) magnitude reinforcer would be delivered upon release. Bohn et al. (2003) found that combined lesions of rat LO/VO did not affect faster responding to the large reinforcer during training but impaired a reversal of the release response when the outcome magnitude contingencies were reversed.

Whereas OFC manipulations disrupt reversal learning for visual discriminations, the results of MFC manipulations in visual discrimination tasks have been mixed. For example, Graybeal et al. (2011) found that combined NMDA lesions of IL/PrL actually enhanced performance during reversal learning, which the authors attribute to disrupted modulation of other subcortical systems (e.g., dorsolateral striatum) that allowed new contingencies to be acquired through facilitation of habit learning. Chudasama and Robbins (2003) found that lesions of IL and PrL impaired performance during reversal of a nose-poke response, however, the authors argue that this was due to learning-related errors rather than perseveration errors, as these animals also had significantly shorter response latencies during the reversal phase. Similarly, Salazar et al. (2004) reported mixed results on the effects of neurotoxic MFC lesions (IL/PrL/Ac) on reversal learning where the reinforced lever was cued by a light stimulus. Rats with MFC lesions performed significantly more correct responses and faster responses during the early sessions of reversal training compared to controls, but required more trials to meet criterion and had slower response latencies in the latter half of the reversal training.

Difficulty of stimulus discrimination has also been reported to alter recruitment of frontocortical regions during reversal. Bussey et al. (1997b) reported that quinolinic acid lesions of IL/PrL or IL/PrL/Ac in the rat impaired reversal learning for visual stimuli that were characterized as difficult to discriminate. In a similar study, Brigman and Rothblat (2008) found that lesions to IL/PrL in mice did not impair reversal learning for comparatively easy discriminations (based on stimulus luminance), however, impairments in reversal learning were observed for line figure stimuli that were more difficult to discriminate. However, other studies have observed null results of MFC manipulations on reversal learning. For example, inactivation of IL/PrL with bupivacaine in the rat has also been shown to leave reversal learning intact for responses (lever presses) that were signaled by the presence or absence of a light cue above the lever (Floresco et al., 2008). Thus, MFC manipulations spare reversal learning for simple visual discriminations, impair performance by altering learning processes, or impair reversal learning for difficult discriminations.

Auditory Cued Tasks

Reversal learning with auditory stimuli also appears to require and engage OFC. For example, Burke et al. (2009) tested whether the OFC in the rat is critical to reversal learning for classical (Pavlovian) conditioning of auditory associations in which one auditory stimulus signals food delivery. After Pavlovian approach to the site of food delivery was established for one stimulus, the stimulus-outcome contingencies were reversed. Inactivation of AI/LO, via GABA agonism, impaired conditioned responding to the previously unrewarded cue during reversal, but had no impact on inhibiting responding to the previously rewarded cue.

Recent electrophysiological evidence is also consistent with a role for OFC neurons in auditory reversal learning. Moorman and Aston-Jones (2014) recorded from AI neurons while rats performed a task in which a discriminative stimulus (SD) signaled that responding would or would not result in access to sucrose. The activity of AI neurons was maximal when an SD+ (i.e., one predictive of reinforcement) was both presented and responded to, and when rats approached the location where sucrose was delivered. Activity was weak during consummatory behavior, as well as when an SD+ was not responded to or was presented during extinction. When the contingencies (SD+ or SD-) were reversed the activity of AI neurons underwent a corresponding reversal. Taken together, these data suggest that OFC subregions are engaged by and required for reversal learning with auditory stimuli. At present there do not appear to be data on the role of MFC in auditory reversal learning.

Spatial Tasks

Reversal learning has also been extensively investigated in paradigms where the spatial location of manipulanda (e.g., levers) is utilized to signal which response is reinforced. For example, Boulougouris et al. (2007) trained rats to press a single lever among two alternatives for food reinforcement and then gave rats quinolinic acid lesions of AI/LO, IL or PrL. Damage to AI/LO impaired performance when the spatial locations of the reinforced and non-reinforced levers were reversed, whereas IL or PrL lesions resulted in similar levels of reversal performance to controls. Interestingly, when the initial contingencies were reinstated, rats with AI/LO lesions made fewer errors than sham, PrL, or IL animals suggesting that at least a portion of the reversal learning impairment following OFC lesions is due to an inability to extinguish responding to the initially reinforced lever. Similar patterns of reversal learning deficits in the absence of initial learning deficits have been observed in rats using deep brain stimulation (DBS) targeted at LO 10 min prior to and throughout the behavioral session (Klanker et al., 2013). There were, however, no effects of DBS in LO on a subsequent reversal to the original contingency, suggesting that responding for the original problem was retained. Consistent with this observation, Boulougouris and Robbins (2009) reported that lesions of the OFC given after discrimination learning and two reversals had no effects on retention or four more subsequent serial reversals.

The effects of reversing reinforcement contingencies with respect to spatial locations of manipulanda has also been investigated in delay discounting procedures, where the outcomes associated with each lever vary in magnitude and delay following the instrumental behavior. Mar et al. (2011) trained rats on a task where pressing one lever led to a small but immediate food reinforcer while pressing a second lever in a different spatial location led to a delayed but large reinforcer delivery. Under these conditions rats typically display a preference for the smaller, immediate reinforcer. Neurotoxic lesions of LO, VO, or combined MO/VO/LO impaired reversal learning in this task, whereas rats with lesions restricted to the MO actually displayed faster reversal learning compared to sham animals. Importantly, when the delay for the larger magnitude reinforcer was removed all animals responded almost exclusively to the lever associated with higher magnitude reinforcement, indicating that impaired reversal performance in rats with lesions cannot be attributed to insensitivity to reinforcer magnitude. These results suggest that reversal of responding based on spatial position of levers is impaired by manipulations of OFC and spared or possibly enhanced by manipulation of MFC. However, Kosaki and Watanabe (2012) reported that MFC lesions lead to increased perseveration errors in a three-lever choice protocol in which the reinforced lever was modified repeatedly. One caveat is that this study used multiple contingency changes and the three possible responses were reinforced in multiple sessions, suggesting the possibility that animals with MFC lesions were confused given the varied reinforcement history, and/or that the three-lever choice problem is a more difficult spatial discrimination.

In addition to spatial discrimination based on lever position a number of studies have examined reversal learning in spatial tasks performed in mazes. Several studies have evaluated the effects of OFC and/or MFC manipulations on reversal learning in the T-Maze, in which navigating to one arm of the maze is reinforced. Successful performance in this task can be achieved by multiple strategies (Dudchenko, 2001, Sutherland & Hamilton, 2004) including turning responses (left or right turn), the spatial location or direction of reinforcement within the maze (Blodgett et al., 1949), and, when present, other cues co-localized with the arms of the maze. Meunier et al. (1991) found that Ac lesions in mice did not impair initial acquisition, but did impair subsequent reversals in a T-maze, whereas, posterior cingulate lesions impaired both initial acquisition and the first reversal. Neonatal ibotenic acid lesions of IL/PrL have also been shown to disrupt reversal learning in the T-Maze (Schwabe et al., 2004). Ghods-Shariffi et al. (2008) found that inactivation of LO with bupivacaine in the rat impaired reversal of spatial responses in a T-maze, however, modifying responses from a visually cued to a spatial response was not altered. Shaw et al. (2013) also observed deficits in reversal learning in the rat following infusion of muscimol in PrL in a T-maze where the reinforced arm was signaled by the visual and tactile aspects of the maze floor.

Ragozzino et al. (1999) failed to observe deficits on place or left vs. right motor response reversal in a plus maze following IL/PrL inactivation with tetracaine in the rat. In contrast, Young and Shapiro (2009) found that inactivation of LO/VO/MO but not IL/PrL with muscimol impaired retention of reversal learning in a plus maze. Using the Morris water task (Morris, 1981), in which rats navigate to an escape platform in a circular pool of cool water, Lacroix et al. (2002) found that NMDA lesions that included IL/PrL/MO did not impair initial learning, but slowed the rate of reversal learning when the escape platform was relocated to a new position. This was only apparent during the second of three reversal blocks and lesion animals ultimately performed at the same level as controls. Thus, the available data on reversal learning in the T-Maze, plus maze, or Morris water task are somewhat mixed, with some studies reporting deficits following MFC or OFC damage, and others failing to detect deficits. One possibility is that reversals in these tasks can be sensitive to MFC or OFC damage because the tasks involves multiple potential relevant response strategies that may complicate evaluation of reversal performance. Though not a spatial reversal task, we note that Li and Shao (1998) rotated the maze during training to eliminate other strategies, and the reinforced arm was marked by a conspicuous visual stimulus. These authors found deficits in reversal of visual discriminations in a T-maze in the rat following knife cuts to either IL or PrL but not AC, however, the effects of OFC damage were not evaluated.

Overall, the majority of available data suggest a clear role for OFC in reversal learning when the relevant stimulus dimension is spatial location of levers at either side of a chamber. Manipulations of MFC appear to largely spare this form of reversal learning, however, the MFC in more clearly implicated in reversal learning for tasks that require movement through space, which differentiates these tasks from others discussed here. The available data on effects of OFC manipulations also suggest a role in reversal learning in such tasks, however, more data are needed to arrive at a more thorough understanding of this relationship.

Reversal Learning Summary

The most consistent result in studies evaluating the effects of contingency reversals is that loss of OFC function is associated with impaired reversal learning. Further, neural firing in OFC is correlated with reversal learning and performance. The available data also clearly implicate the LO subregion of the OFC in reversal learning, however there is a need for more studies examining lesions or recordings restricted to the VO and AI selectively. The involvement of the OFC is observed for rats (of various strains) and mice, for multiple stimulus domains (visual, olfactory, tactile, spatial, auditory) and for varied forms of motor responses (lever pressing, nose-poking, Pavlovian magazine approach) suggesting that the function of OFC in reversal learning is generalized with respect to stimulus modality and response form. It is important to point out that much of the available data have been obtained using food, water, or sucrose reinforcement (along with deprivation). Thus, the available data cannot rule out the influence of food-related reinforcers, and food-related motivation, and the interaction of these factors with other components of the behavioral methods. While the majority of contingency reversal tasks are not sensitive to MFC damage, some reversal learning deficits are observed and appear to fall into two classes: Difficult visual discrimination problems and visuospatial learning (T-maze, Morris water task).

3. INHIBITORY LEARNING: EXTINCTION AND REINFORCER DEVALUATION

Inhibitory learning refers to behavioral modifications that occur as a result of negative contingencies between a stimulus and a response, a stimulus and an outcome (as in Pavlovian conditioning), or a response and an outcome. Reversal learning as characterized in section 2 can be considered to consist of a critical shift in control of behavior from initial excitatory learning (e.g., for an S+) to inhibitory learning during reversal training. The experimental results from reversal learning studies following OFC or MFC manipulation summarized in the prior section suggest that initially-learned responses may not extinguish during reversal training. Impairments of extinction learning could contribute to perseveration errors during a first reversal, and possibly superior performance when initial contingencies are reinstated in subsequent reversals. Because extinction requires an excitatory learning history it can be considered a particular form of reversal learning. Similarly, devaluation of an outcome effectively alters the valence of the outcome (e.g., appetitive to aversive), representing a salient form of contingency modification. Extinction and devaluation of reinforcers following manipulations of MFC or OFC have been evaluated in several rodent studies described in this section. Figure 3 provides a graphic summary of these studies with respect to the targeted subregions of MFC or OFC, species/strain, experimental approach, and stimulus type.

Figure 3. Effects of frontal cortex subregion manipulation or recording on extinction and devaluation tasks.

Closed circles (●) denote a manipulation of a particular subregion or collection of subregions yielded a pattern of impaired extinction or devaluation, or that recording data were related to extinction or devaluation. Open circles (○) denote lack of an impairment on extinction or devaluation or lack of relationship between recordings and extinction or devaluation. Lines that connect individual circles (●--●) indicate a manipulation or recording preparation including all subregions connected by the line. Species/Strain: R= Rat, M= Mouse, LE= Long Evans, LH= Lister Hooded, SD= Sprague Dawley, B6= C57Bl/6J, 129= 129S1/SvIMJ, Balb= BALB/c. Approach: C= context, V= visual, A= auditory, O= olfactory, T= tactile, S= spatial. Areas: PrL= prelimbic cortex, IL= infralimbic cortex, Ac= anterior cingulate cortex, MO= medial orbital cortex, VO= ventral orbital cortex, LO= lateral orbital cortex and AI= agranular insular cortex. Additional abbreviations: BOLD = Blood oxygen level dependent signal.

Extinction Tasks

Pavlovian autoshaping procedures have been used in several studies to address the role of MFC and OFC subregions in extinction learning. For example, Bussey et al. (1997a) trained rats to approach one visual stimulus (S+) for food reinforcement, whereas approaching the other was not reinforced (S-). Lesions of the Ac impaired initial learning of this behavior, but also impaired discriminative approach to the previous S+ when the contingency was altered during extinction training. Using a similar paradigm in the mouse, Sparta et al. (2014) reported that selective stimulation of PrL interneurons, resulting in a net inhibition of PrL activity, did not alter initial learning of a Pavlovian consummatory response but accelerated the rate of extinction learning, suggesting that suppression of PrL activity may enhance behavioral flexibility. Similarly, Francois et al. (2014) found significantly increased activity in IL neurons in the rat during initial acquisition and extinction in a goal-directed behavioral task. Together, results from Pavlovian conditioning paradigms consistently suggest a role for the MFC in extinction of a learned response, and in some cases, acquisition of that response.

Extinction of a conditioned fear response has also been utilized to examine the ability of rodents to flexibly modify a well-learned response. While some studies using targeting lesions of MFC subregions encompassing IL, PrL, and Ac in rat have shown no effect on acquisition or extinction of fear conditioning (Garcia et al., 2006, Zelinski et al., 2010), recording studies have suggested that subregions of MFC are involved in these behaviors. For example, Santini et al. (2008) found that excitability of IL neurons in the rat increases during extinction of fear responding, whereas a reduction in excitability was observed in these neurons during fear learning. Burgos-Robles et al. (2009) also reported that PrL neurons in the rat increased responding to stimuli associated with a fear response, and persistent increases in the activity of these neurons was associated with failure of extinction. These results were further supported by findings of Cho et al. (2013) that fear extinction in the mouse reduces the efficacy of synaptic transmission from the IL/PrL to the amygdala. Mouse strains that show intact or impaired fear extinction have also been utilized to elucidate the contribution of MFC to these behaviors. Fitzgerald et al. (2014) recorded activity simultaneously in the IL and PrL of mice strains showing normal and impaired extinction learning and found that mice with deficient extinction learning showed elevated single-unit firing in the PrL and the IL compared to mice that display normal extinction.

Studies using stimulation paradigms also provide evidence of a role for MFC in extinction learning. A study by Maroun et al. (2012) showed that high frequency stimulation of IL in rats given immediately, but not 3-hours, after retrieval of a fear or taste aversion memory significantly facilitated extinction of the learned responses. In contrast, low frequency stimulation of IL impairs extinction of these paradigms (Shehadi & Maroun, 2013). Therefore, there appears to be a critical distinction between the recruitment of MFC regions during extinction of fear responding and the necessity of these circuits for extinction learning. Several studies have also examined the effects of frontal cortex damage on the consolidation of extinction. These studies involve damaging or inactivating the circuit of interest after initial extinction training to disrupt ongoing consolidation processes. Impairments in the consolidation of fear extinction in the rat have been observed following blockade of NMDA receptors in PrL (Burgos-Robles et al., 2007), blockade of GluN2B receptors in IL (Laurent & Westbrook, 2008, Sotres-Bayon et al., 2009), mGluR5 receptors in IL (Fontanez-Nuin et al., 2011), protein synthesis inhibition in IL and PrL (Santini et al., 2004), suppression of action potentials (with TTX) in IL and PrL (Sierra-Mercado et al., 2006), and inactivation of IL (but not PrL) with the GABA agonist muscimol (Sierra-Mercado et al., 2011). Importantly, only the study of Sierra-Mercado et al. (2011) observed impairments in extinction learning, while all other studies only observed deficits in the consolidation of extinction.

The role of OFC in extinction learning has not been as thoroughly examined. In one study, Zelinski et al. (2010) performed neurotoxic lesions (NMDA) of either combined LO/VO or combined IL/PrL. After recovery, rats were trained in a fear conditioning paradigm that involved context discrimination (i.e., shock was delivered in one context) followed by extinction training. Extinction learning proceeded normally in IL/PrL lesion rats, however, LO/VO lesion rats did not display extinction of fear responding, and instead displayed overgeneralization of fear responses. Interestingly, IL/PrL lesion rats did not display spontaneous recovery after a post-extinction delay of 14–24 days. A similar study by West et al. (2013) examined the effects of combined NMDA lesions of AI/LO/VO or 5-HT depletion of the same areas in the rat on extinction learning and devaluation (discussed below). This study utilized a task that required discrimination of visual stimuli for reinforcement during initial learning. NMDA lesions of AI/LO/VO caused a mild impairment in extinction learning, whereas 5-HT depletion did not affect extinction. These observations suggest that extinction learning depends on subregions of OFC for both appetitive and aversive outcomes, and the critical processes required for extinction learning in OFC do not appear to depend upon 5-HT receptors.

Devaluation Tasks

Although fewer in number, some studies have examined the effects of frontal cortex damage in the rat on responding when reinforcers are devalued. In devaluation procedures either a Pavlovian or instrumental response is established with an appetitive outcome such as food after which the reinforcer is devalued, typically by pairing the flavor with illness induced by injection of lithium chloride (LiCl). Responding for the now devalued outcome is compared to the responding for an outcome that has not been devalued. Responses for the devalued stimulus are expected to decrease, though perhaps not entirely, whereas responding for outcomes that are not devalued typically are unaffected (Colwill & Rescorla, 1985, Colwill & Rescorla, 1990).

The contribution of OFC to behavioral flexibility following alterations in outcome value was examined by Gallagher et al. (1999), who evaluated outcome (food) devaluation in Pavlovian conditioning in rats with NMDA lesions of AI/LO. Compared to controls, rats with AI/LO lesions were not impaired at learning a conditional response to a light CS that signaled food delivery. The reinforcer was then devalued via LiCL injection. Control animals displayed appropriate reduction in responding, whereas responding by rats with AI/LO lesions was not altered. Reductions in the effectiveness of devaluation to reduce responding following OFC damage in the rat was subsequently replicated by Pickens et al. (2003). Failures of reinforcer devaluation to reduce conditional responding can be understood in terms of failures in modifying the reinforcer value. It is important to note that unaltered rats will perform instrumental responses for brief presentations of CSs associated with appetitive outcomes, and the OFC was critical for this form of behavioral modification (Burke et al., 2008). This observation suggests that at least part of the OFC contribution to response modification following devaluation may be related to the representation of outcome-specific representations that are not present in animals with OFC damage.

The effects of OFC function on devaluation via satiety have also been examined. West et al. (2013) performed NMDA lesions of AI/LO/VO or 5-HT depletion within the same areas and examined the effects on responding following devaluation of a food reinforcer. Both the neurotoxic lesions of OFC and 5-HT depletion disrupted the normal devaluation-related reduction in responding, as both manipulations resulted in comparable levels of responding for a particular food reinforcer before and after devaluation. These findings are particularly interesting as satiety has been linked with reduced neural responding to food-related cues in the OFC of non-human primates as well (Critchley & Rolls, 1996). In addition to suggesting that rodent OFC is critical for devaluation based on satiety of a food reinforcer, the findings of West et al. (2013) indicate that devaluation of this type can be dissociated from extinction learning in that the former depends on 5-HT receptors. A possible interpretation of this finding is that 5-HT receptors in OFC play a critical role in outcome valuation, but not extinction when the outcome is absent (rather than devalued).

More complex behavioral tasks involving varying probability of reinforcement have also been used in combination with devaluation via satiety. Zeeb and Winstanley (2013) examined the effects of either unilateral quinolinic acid lesions of LO/VO/MO or the basolateral amygdala (BLA), or combined lesions of LO/VO/MO in one hemisphere and BLA on the contralateral hemisphere (disconnecting the circuit) on a task patterned after the Iowa Gambling Task for humans (Bechara et al., 1999). Because of the difficulty incorporating the results of unilateral lesions into the present discussion, we focus on the results of the complete disconnection of BLA and OFC in both hemispheres. When satiated on sucrose pellets control animals reduced their responding accordingly, however, animals with disconnection of BLA and OFC did not display a reduction in responding, consistent with the notion that OFC and related circuitry are involved in behavioral flexibility when reinforcers are devalued.

Effects of MFC damage on devaluation have not been examined as extensively as OFC. A study by Tran-Tu-Yen et al. (2009), however, did evaluate the effects of inactivation of PrL via muscimol infusion on devaluation. When PrL was inactivated at test (i.e., after learning) animals displayed normal sensitivity to the effects of devaluation following pairings of flavor and illness. PrL deactivation during training, however, was found to make animals insensitive to later devaluation, implicating MFC in goal-directed learning during the initial training.

Extinction and Devaluation Summary

The available data suggest two basic conclusions regarding the contributions of frontocortical regions to extinction and devaluation. Lesions of the regions comprising OFC disrupt extinction of conditioned fear and extinction of responding in relation to appetitive outcomes, as well as modification of behavior following devaluation of reinforcers. The latter form of behavioral flexibility also appears to critically depend on 5-HT signaling. In contrast, manipulations of MFC (IL/PrL) do not appear to influence extinction learning or to directly impact behavioral modifications following devaluation. MFC manipulations do reliably disrupt consolidation of extinction. Considering the limited data on the role of MFC in behavioral flexibility following reinforcer devaluation and the limited data on extinction learning and OFC more studies are clearly needed.

4. EXTRADIMENSIONAL SET-SHIFTING

Distinct from the ability to reverse or extinguish a choice behavior, the ability to change or shift a response rule when learned has been widely used to measure behavioral flexibility. Based on the clinical Wisconsin Card Sorting Task (WCST, Grant & Berg, 1948), set-shifting tasks have been one of the more commonly employed assays for measuring cognitive flexibility in rodents. These tasks involve initially training animals to discriminate stimuli that vary in multiple modalities. For example, in rodents reinforcement is often delivered based on differential responding to one stimulus dimension such as odor, in which responding to one odor stimulus is reinforced and responding to another is not. Another stimulus dimension (e.g., visual or tactile) is established as irrelevant with respect to outcome contingencies. After the initial discrimination is established, an intradimensional shift (ID) is conducted by providing a novel discrimination with a stimulus in the same dimension as the initial discrimination (e.g., from cinnamon to chocolate odor). The next phase of training typically involves a reversal, with the goal of establishing attention to the relevant stimulus dimension. In an extradimensional shift (ED, Shepp & Eimas, 1964), the correct dimension is changed altogether, such that choices must be guided by the new dimension (texture) while the previously learned dimension is now irrelevant. In this section, studies that have examined the role of MFC and OFC subregions in ED set-shifting are discussed. Figure 4 provides a graphic summary of these studies with respect to the targeted subregions of MFC or OFC, species/strain, experimental approach and stimulus type.

Figure 4. Effects of frontal cortex subregion manipulation or recording on set-shifting tasks.

Closed circles (●) denote a manipulation of a particular subregion or collection of subregions yielded a pattern of impaired set-shifting during ED shifts, or that recording data were related to set-shifting. Open circles (○) denote lack of an impairment on set-shifting or lack of relationship between recordings and set-shifting. Grey circles (●) and lines (●--●) indicate results that require additional clarification the nature of the deficit that are more fully discussed in corresponding section 4 text. Lines that connect individual circles (●--●) indicate a manipulation or recording preparation including all subregions connected by the line. Species/Strain: R= Rat, M= Mouse, LE= Long Evans, LH= Lister Hooded, SD= Sprague Dawley, B6= C57Bl/6J, 129= 129S1/SvIMJ, Balb= BALB/c. Approach: C= context, V= visual, A= auditory, O= olfactory, T= tactile, S= spatial. Areas: PrL= prelimbic cortex, IL= infralimbic cortex, Ac= anterior cingulate cortex, MO= medial orbital cortex, VO= ventral orbital cortex, LO= lateral orbital cortex and AI= agranular insular cortex.

Olfactory-Tactile Tasks

First developed by Birrell and Brown (2000), the olfactory/tactile version of the set-shifting task involves training rodents to dig in a substrate for food reinforcement. The use of multiple compound olfactory/tactile stimuli allow for training of multiple, novel discriminations in a single dimension in order to establish the attentional set before subsequent ED set-shifting. In their initial study, Birrell and Brown (2000) showed that ibotenic acid lesions of combined IL/PrL in rats impaired behavioral flexibility when the relevant stimulus dimension was changed during the ED shift. Cordova et al. (2014) also recently reported that stroke (induced by endothelin-1 infusion), which resulted an infarct that typically included IL, PrL, and Ac, also impaired ED shifts in rats. It should be noted that only a single discrimination was trained prior to reversal and ED shift in this study, whereas previous studies have typically established 3–4 separate ID discriminations prior to the ED shift. Thus, animals may not have formed an attentional set prior to the ED shift. McGaughy et al. (2008) also demonstrated that noradranergic (NE) lesions (via dopamine beta hydroxylase Saporin infusion), but not ACh depletion (via 192 IgG Saporin infusions) in IL/PrL impaired ED set-shifting in rats, indicating that NE afferents in IL/PrL are critical for this form of behavioral flexibility. Importantly, other studies established that neurotoxic lesions of VO/LO in the rat spare ED set-shifting, consistent with a dissociation between MFC and OFC with respect to this form of behavioral flexibility (McAlonan & Brown, 2003). Bissonette et al. (2008) later found that lesions of IL/PrL in mice impaired ED set-shifting in the odor/tactile set-shifting task, whereas damage to LO/VO did not. Overall, the available data provide consistent evidence that ED set-shifting with olfactory/tactile stimuli depends on MFC, but is spared following damage to constituents of OFC.

Visual-Spatial Tasks

Set-shifting behavior has also been extensively studied in rodents using discrete visual stimuli and visuospatial information as dimensions. Floresco et al. (2008) employed a task that required rats to press a lever (among two choices) for food reinforcement, where the reinforced lever was reliably signaled by either a light stimulus or the spatial location of the lever. Control animals performed significantly worse when shifted from the light stimulus to the spatial location, compared to shifting from the spatial location to light dimension, indicating the former was more difficult. Rats with IL/PrL inactivated with bupivacaine displayed impaired learning for the more difficult light to spatial response shift, but acquired the easier spatial response to light shift similarly to controls. Interestingly, these results suggest that the necessity of IL/PrL circuitry for shifting is modulated by task difficulty. This conclusion was further supported by separate experiments in which rats with IL/PrL lesions were impaired on ED shifts, when the ED shift was made more difficult by habituating animals to the initially irrelevant light stimuli prior to training.

Several studies have also examined the role of the frontal cortex in spatial navigation tasks in which reinforcement is contingent upon utilization of distinct strategies based on interoceptive or exteroceptive cues. Using a set-shifting paradigm in the plus maze that could be solved based on motor responses or visual cues, Ragozzino et al. (1999a) found that inactivation of IL/PrL in rats significantly impaired the ability to shift between response strategies. Inactivation, however, did not impair acquisition of either strategy or reversal learning within each response domain. In a related study, this group evaluated the effects of IL/PrL inactivation with tetracaine on shifting in the cheeseboard task, which requires responding to a particular spatial location using visual cues (Ragozzino et al., 1999b). Rats with IL/PrL lesions trained to perform the task using a spatial strategy or to navigate to a target location based on a conspicuous visual cue localized at the goal were impaired when the relevant strategy was switched compared to control or animals with inactivation of posterior cingulate.

Using a similar paradigm, Block et al. (2007) examined the effects of disconnecting IL and PrL from the mediodorsal nucleus (MD) of the thalamus or nucleus accumbens (NAc) in rats. Bupivacaine was infused into MD or NAc of one hemisphere and IL/PrL of the contralateral hemisphere after learning a turn response. Disconnection of NAc or MD from IL/PrL resulted in increased trials to criterion and perseverative errors when the response rule was altered so that a visual stimulus, and not turn direction, predicated reinforcement. Similar results have been found in studies using the Y-maze. Oualian and Gisquet-Verrier (2010) trained rats with IL, PrL or combined IL/PrL lesions to navigate to a reinforced location based on either a motor-response (left or right) or a visual cue (light vs. dark stimulus). Compared to controls, animals with lesions were not impaired at learning these discriminations, however, all lesion groups were impaired when the rule (dimension) was reversed. These observations indicate that set-shifting involving visual stimuli and turn responses is impaired following damage to or disconnection of MFC subregions.

Impairments in the retention of “shifted” responses after manipulation of MFC have also been reported. For example, Rich and Shapiro (2007) found that muscimol inactivation of IL/PrL did not impair learning ED shifts (place v. motor response) in the plus maze, but did impair 24 hour retention of the shifted behavioral response. Young and Shapiro (2009) replicated this observation for IL/PrL inactivation and further demonstrated that complete OFC (MO/LO/VO/AI) inactivation impaired retention of ID reversals, but not ED set-shifting. Initial learning was not disrupted by inactivation in either brain region or for either dimension. Similar results have been reported with VO inactivation alone (Ghods-Sharifi et al., 2008). Although these studies highlighted effects of IL/PrL inactivation on retention, Rich and Shapiro (2009) demonstrated that activity of IL and PrL neurons in the rat acutely code for strategy switches between place and response-based strategies. Based on these results, IL and PrL appear to be engaged during set-shifting for place versus response strategies and code for strategy switches in the plus maze. Constituents of OFC do not appear to be required for or engaged by set-shifting with visual stimuli.

Set-Shifting Summary

Most of the available data indicate that lesions or other manipulations of MFC impair set-shifting, both for shifts between odor and tactile stimulus dimensions, visual and spatial dimensions, or spatial versus response strategies. Deficits in set-shifting are not observed with MFC manipulations that involve acetylcholine (Ach) depletion, GABA agonism or comparatively easy strategy shifts. The latter point is important because while most available studies have counterbalanced across initial learning dimension or otherwise examined stimulus difficulty, the available results suggest that balance of these dimensions can drive behavioral results during ED set-shifting if one dimension is more salient (Baxter & Gaffan, 2007). Additionally, studies examining set-shifting need to ensure that attentional set-formation has actually been established during initial training if ED problem performance is to be properly interpreted (Young et al., 2009).

A limitation of the currently available findings is that the effects of MFC or OFC manipulations have primarily been evaluated with food reinforcers, thus, the degree to which these effects are related to the type of reinforcement cannot be firmly established. There are very limited data on manipulations of OFC on set-shifting, however, these data indicate that ED set-shifting does not depend on OFC. More studies are needed to examine neurobiological double-dissociations between reversal-learning and ED set-shifting with respect to OFC and MFC.

5. DISCUSSION

The available data on reversal learning, extinction, devaluation and set-shifting in a broad range of rat and mouse strains suggest specific relationships between flexible behavior and distinct subregions of the frontal cortex. First, reversal learning, when reinforcement contingencies are altered within a single stimulus domain, is consistently reported to recruit and engage regions comprising the OFC in rodents (see Figure 2), whereas MFC regions do not appear to be critical for most forms of contingency reversal within a single stimulus dimension (with notable exceptions discussed below). In contrast, extinction learning appears to more critically depend on MFC. Some dissociations, however, have clearly been observed suggesting that OFC and not MFC are critical for extinction of fear learning. Regarding reinforcer devaluation, appropriate reductions in responding appear to depend on OFC and not MFC. However, this conclusion is based on limited data and more research is needed to solidify this relationship (see Figure 3). Set-shifting behavior, which requires the ability to flexibly shift between responses, perceptual dimensions, or rules, has consistently been shown to depend on MFC (see Figure 4). The available studies examining OFC contributions to set-shifting have not reported deficits following damage to OFC subregions, however, these are few in number. In addition to lesion or inactivation findings, each of these the broader conclusions on reversal learning, extinction and set-shifting have also received support from recordings of neural activity during behavior.

Collectively, the observations outlined here suggest a clear double dissociation between OFC and MFC with respect to reversal learning and set-shifting, as has been suggested by numerous empirical reports. It is important, however, to recognize that only a few studies have examined neurobehavioral double dissociations within the same study. More studies that utilize this approach are needed, and future studies on the genetics of behavioral flexibility should be designed with this consideration in mind.

Functional Specificity of Frontocortical Subregions

There is strong evidence from the current literature that subregional specificity of MFC and OFC is important for understanding the neurobiology of behavioral flexibility. For example, region LO is targeted in many studies (see Figure 2), and therefore, has been the most represented subregion in the available empirical reports. The roles of adjacent subregions, including AI and VO, in isolation are less well established and need to be examined more thoroughly in future research. This is particularly important in light of recent findings regarding the role of OFC in flexible behavior and reward updating in primates (Rudebeck et al., 2013). Similarly, relatively few studies have examined the relationship of OFC subregions to extinction, devaluation, or set-shifting.

With respect to MFC, many set-shifting and reversal learning studies have manipulated or targeted both IL and PrL, however, studies that targeted individual subregions suggest that manipulation of either may yield similar outcomes. One exception in MFC is Ac, which has not been reported to impair reversal when specifically targeted. However, there is evidence that Ac loss can impair the flexibility required to successfully extinguish learned responses. Given evidence that Ac is involved in learning and reward updating, more work should focus on parsing the role of this area in a wider variety of tasks, including set-shifting behaviors (Bussey et al., 1997b, Kosaki & Watanabe, 2012, Meunier et al., 1991, Parkinson et al., 2000). As researchers begin to target more specific cell populations within OFC and MFC, it will be important to consider potential differences in regional function.

Flexible Behavior and Initial Learning

It is also important to consider the issue of what behavioral modifications should be considered to reflect behavioral flexibility. For example, it is evident from the studies discussed in the present review that considerable attention has been devoted to investigating the role of the frontal cortex in modifying behaviors that have been previously established based on an individual organism’s learning history. However, it is also important to acknowledge that initial learning itself represents a critical form of behavioral flexibility. The majority of studies discussed here failed to observe learning deficits for initial discriminations, however, several studies have characterized learning deficits following manipulations (e.g., lesion, inactivation) of rodent frontal cortex regions, in several tasks. These are briefly summarized here in the broader context of the more common null findings from reversal learning, extinction, and set-shifting studies.

Some studies utilizing discriminations of spatial location or visual cues that signal reinforcement have failed to detect deficits in initial learning following lesions of the MFC, (Brigman & Rothblat, 2008, van Der Plasse et al., 2007) or OFC (Klanker et al., 2013) In contrast, MFC lesions have yielded deficits when the task also involves learning which of two outcomes is associated with a particular behavioral response (Corbit & Balleine, 2003), or when multiple responses among more than two alternatives must be modified based on changing reinforcement contingencies (Kosaki & Watanabe, 2012). MFC lesions have also produced deficits when more difficult visual discrimination problems such as learning multiple concurrent discriminations or rules is required (Bussey et al., 1997a, Peters et al., 2013, Rivalan et al., 2011).

Additionally, Pavlovian approach behaviors are impaired by frontocortical damage. Deficits in learning approach behavior have been observed following lesions to Ac alone (Bussey et al., 1997a, Parkinson et al., 2000) and combined lesions of AI, LO, VO and MO (Chudasama & Robbins, 2003). Other studies, however, have found that approach actually proceeded faster in animals with MFC lesions when the CS+ was a retractable lever introduced into the chamber prior to food delivery (van Haaren et al. (1988). Furthermore, other studies have failed to observe deficits in Pavlovian approach following IL lesions (Chudasama & Robbins, 2003) or lesions of OFC (Pickens et al., 2003).

Most available studies have failed to detect effects of MFC damage on reversal learning based on arbitrary contingencies determined by the experimenter. Interestingly, van Der Plasse et al. (2007) found that responding by unaltered control animals is affected when the contingencies between odor cues or lever position and outcomes that vary in preference (preferred or non-preferred) are reversed. Destruction of serotonergic terminals in the MFC eliminated the sensitivity to reversal of preferred outcomes. That is, serotonergic de-afferentation of the MFC spared reversal learning, but eliminated an affective component (preference) as a factor in responding and reversal learning.

Several studies have reported deficits in spatial learning and behavior following lesions of MFC. For example, van Haaren et al. (1988) found that MFC lesions impaired spatial delayed alternation between two levers and Sloan et al. (2006) reported impairments in a delayed matching to position task. In the T-maze Meunier et al. (1991) found initial acquisition deficits following Ac damage and Schwabe et al. (2004) found impaired acquisition of continuous alternation following neonatal IL, PrL damage. Some failures to observe effects in comparable tasks include a lack of deficits in learning in a plus maze following inactivation of IL/PrL (Ragozzino et al., 1999) and a lack of deficits in delayed matching-to-position following IL/PrL lesions (Lacroix et al., 2002).

Lesions of MFC and OFC have also been found to alter acquisition of spatial learning in the Morris water task (MWT, Morris, 1981) with large deficits in MWT learning following removal of MFC, with more modest deficits following OFC removal (Kolb et al., 1983). Large lesions of MFC that include IL, PrL, Ac and MO have also result in deficits in spatial learning in the MWT (Brown et al., 2000, Hoh et al., 2003, McDonald et al., 2008), or learning to use an egocentric spatial strategy (Mogensen et al., 2005). Some studies have, however, failed to yield robust deficits in MWT learning following lesions of IL, PrL, and Ac (Lacroix et al., 2002, Sloan et al., 2006).

Together, these findings suggest that lesions of MFC produce deficits in Pavlovian approach (discrimination learning), concurrent discrimination learning, and spatial learning in several tasks, while lesions of OFC appear to have few reliable effects on visual discrimination learning and olfactory discrimination learning. However, the presence of some initial learning deficits highlights the importance of establishing that initial learning is spared prior to evaluating the effects of manipulations of frontal cortex on reversal learning.

Frontal Cortex and Working Memory

The present review has focused on the role of rodent OFC and MFC on modification of behavior following reversal or extinction of contingencies, devaluation of outcomes, and set-shifting. However, the contributions of working memory to these functions and associated deficits in these domains following manipulations have not been covered here in detail. In considering the contributions of rodent OFC and MFC to working memory, it is important to distinguish tasks that measure the amount of information (i.e., capacity or span) that can be held online (and possibly manipulated) simultaneously and tasks that require information, often about a single item (e.g., spatial location, object) or action, to be maintained during an imposed delay period (Dudchenko, 2004). The importance of this distinction for translational efforts has been addressed in several reviews (Dudchenko, 2004, Dudchenko et al., 2013, Young et al., 2009), however, the major thrust of these authors’ arguments are recapitulated here. Of primary importance is the distinction between processes that are tapped during tests of human working memory, primarily working memory span and manipulation of items held online for active processes, and processes that are recruited in tests with rodents and other non-human animals, namely the maintenance of some information during a delay period. The distinction between capacity and delay is evident in patient populations (e.g., schizophrenia) where working memory capacity is affected in the absence of delay-dependent memory impairments (Gold et al., 2010). Classic tests of human working typically lack a critical delay period, whereas imposed delays are more common in tasks that depend on rodent OFC or MFC. This creates some difficulty in integrating findings from human and rodent studies of working memory. Below we have, however, attempted to briefly highlight the available data on the relationships between rodent frontal cortex and working memory capacity or maintenance of information during delays.

Tasks that measure working memory span in rodents have not been used extensively to evaluate the frontocortical contributions to this aspect of working memory, however, the available data suggest a link between MFC and span. For example, inactivation of MFC has been shown to impair the performance of rats in an odor span task designed to assess working memory capacity (Davies et al., 2013b). Performance in this task has also been linked to GluN2BRs and AMPARs in MFC (Davies et al., 2013a). Data regarding the role of OFC in this task do not appear to be available at present. The majority of available studies have examined the role of rodent OFC or MFC in tasks that require information to be maintained during a delay period (without an obvious ‘span’ component). For example, lesions of AI, but not Ac or IL/PrL, impair memory for reward value in a delay-dependent manner (Decoteau et al., 1997, Ragozzino & Kesner, 1999). In contrast, delay-dependent spatial memory is disrupted following PrL lesions (Fritts et al., 1998, Ragozzino et al., 1998) (but see, Delatour and Gisquet-Verrier (2000)) or combined Ac/PrL lesions (Dias & Aggleton, 2000), but is spared following AI damage (Ragozzino & Kesner, 1999). Similarly, inactivation of PrL/Ac with muscimol, but not AI, disrupts spatial delayed alternation performance in the rat (Horst & Laubach, 2009). Further, neural activity in MFC neurons during spatial delayed alteration appear to reflect coding for not only spatial information, but the outcomes of recent trials in the service of current and future behavior (Horst & Laubach, 2012). Collectively, these observations suggest that the dorsal components of rodent MFC perform functions homologous to those observed in non-human primates with respect to maintenance of spatial memory (Rushworth et al., 2003).

The activity of individual neurons in MFC is also correlated with performance in tasks that require maintenance of spatial information during a delay (Jung et al., 1998), and PrL neurons display stimulus selective activity during delays where auditory stimuli predict subsequent reward (Cowen & McNaughton, 2007). Because MFC and OFC are implicated in maintenance of some forms of information during delays, it is important to consider that deficits in set-shifting and reversal learning may reflect deficits in this ability. This consideration could help resolve some apparent inconsistencies in the dissociation between OFC and MFC with respect to reversal learning, particularly the associations between MFC and reversal learning with difficult visual discriminations and spatial reversals (see Figure 2). One possibility is that both spatial and difficult visual discriminations are related to increased requirements for maintenance of information online. Many, though not all, spatial tasks that involve movement through space to a location of reinforcement have such requirements which requires time to complete, during which information might be held online. Further, some tasks require that relevant information be maintained within a given trial until the behavior can be performed (Dudchenko, 2004). Similarly, discrimination between two competing visual stimuli may require online maintenance of information if both stimuli cannot be sampled simultaneously. Sensitivity to disruptions in this ability may increase commensurate with increases in stimulus complexity, which might help explain why reversal of difficult visual discriminations critically depends on MFC.

Effects of Sex, Stimulus and Reward Modality

Although the role of the various frontocortical regions in the major categories of behavioral flexibility that we emphasized here appear to generalize across stimulus modality and response domains (i.e., forms of motor responding), virtually all of the studies, with the exception of fear conditioning, utilize food, sucrose or water reinforcement. Additionally, most include restriction procedures to enhance motivation. Thus, establishing whether the phenomena characterized in this body of reports are limited to food reinforcement and consummatory behaviors is difficult. This is particularly important because there are gustatory representations in the frontocortical circuits of interest, and these circuits are involved in feeding, representation of food-related stimuli, and consummatory behavior (Critchley & Rolls, 1996, Mena et al., 2011, Rolls et al., 1996, Whishaw & Kolb, 1989). To address this gap in the current landscape future studies should examine a broader range of reinforcer types (e.g., sex, brain stimulation reward, etc.). Additionally, all the studies characterized here examined behavioral flexibility in male animals. Thus, it is not possible to suggest conclusions about the influence of organizing and circulating effects of sex hormones on the frontocortical bases of behavioral flexibility. Bissonette et al. (2012) found that male and female mice performed similarly on a reversal learning task, but female mice did not acquire an attentional set unless a larger number of discrimination tasks were used to established the set. Whether there are sex differences in the neural circuits involved in reversal learning, extinction, and set shifting is currently not known. Obviously, future studies need to include examination of sex effects and, when possible, exploit them to better inform our understanding of the neurobiology underlying behavioral flexibility.

Additionally, a large number of studies aimed at evaluating the role of frontal cortex in flexible behavior have utilized olfactory stimuli to control distinct behavioral responses. Importantly, nearly all of these studies have found that learning of initial olfactory discriminations is not impaired following damage to the constituents of OFC (Bissonette et al., 2008, Churchwell et al., 2009, Schoenbaum et al., 2002, Schoenbaum et al., 2003a, Smith et al., 2010) or MFC (Birrell & Brown, 2000), although Bissonette et al. (2008) reported slight impairments in initial olfactory discrimination learning. More complex configural learning involving associations between olfactory and tactile stimuli are, however, sensitive to OFC damage (Whishaw et al., 1992), although the available data indicate that simple olfactory discrimination problems appear to be unaffected by frontal cortex damage.

It is also important to recognize that reversal learning, extinction, devaluation and set-shifting engage and require other circuits, both cortical and subcortical (Calu et al., 2007, Kosaki & Watanabe, 2012, Roesch et al., 2007). For example, lesions of the BLA have been shown to impair reversal learning (Churchwell et al., 2009, Schoenbaum et al., 2003a). The BLA and OFC are interconnected and the BLA is critical for OFC neurons to code for predicted outcomes (Schoenbaum et al., 2003b). Evaluating the role of frontocortical subregions in the larger context of the networks in which they participate will also be important.

Conclusions

The purpose of this review was to characterize the current landscape of the literature on the role of the rodent frontal cortex in behavioral flexibility. In addressing this goal, we focused on several task domains and specific subregions of frontal cortex implicated in a larger body of literature. Because a large number of excellent review articles and books have been written on the topic of frontal cortex which cover a broader range of topics, species, and approaches to fragmenting the regional differences in function we provide a targeted, and of course non-exhaustive, list of these references here for the readers’ consultation (Bissonette et al., 2014a, Bissonette & Powell, 2012, Chudasama & Robbins, 2006, Fuster, 2008, Kesner & Churchwell, 2011, Kolb, 1984, Ragozzino, 2007, Schoenbaum et al., 2009b, Schoenbaum & Setlow, 2001, Uylings et al., 2003). Our hope is that this review further organizes thinking about the role of the frontal cortex in behavioral flexibility and motivates future work to better understand the neural bases of flexible behavior.