Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2010 Oct 1.
Published in final edited form as: Behav Neurosci. 2009 Oct;123(5):1028. doi: 10.1037/a0016822

Double Dissociation and Hierarchical Organization of Strategy Switches and Reversals in the Rat PFC

James J Young 1, Matthew L Shapiro 1
PMCID: PMC2821059  NIHMSID: NIHMS157228  PMID: 19824768

Abstract

Prelimbic-Infralimbic cortex (PL-IL) and Orbitofrontal cortex (OFC) influence behavioral flexibility in the rat. We tested the effects of PL-IL or OFC infusion with the GABA agonist muscimol in the context of two flexible responding tasks: strategy switching and reversal. Muscimol infusion into PL-IL impaired retention of strategy switches but not reversals, whereas muscimol infusion into OFC impaired retention of reversals but not switches. However, while training in repeated reversals did not remove the requirement of PL-IL for switch retention (Rich and Shapiro, 2007), training in repeated switches did remove the requirement of OFC for reversal retention. Thus, activity during strategy switches was sufficient to initiate learning and remove the requirement of OFC in later reversals.

Keywords: strategy switching, reversal, prelimbic, infralimbic, orbitofrontal cortex

Lesions to the prefrontal cortex (PFC) in humans (Bechara, Damasio, Damasio & Anderson, 1994; Milner, 1963), primates (Dias, Robbins, & Roberts 1996a, 1996b;Izquierdo, Suda, & Murray, 2004; Jones & Mishkin, 1972), and rats (Birrell & Brown, 2000; de Bruin, Sanchez-Santed, Heinsbroek, Donker, & Postmes, 1994; McAlonan & Brown, 2003; Ragozzino, Wilcox, Raso, & Kesner, 1999a; Schoenbaum, Nugent, Sadoris, & Setlow, 2002; Schoenbaum, Setlow, Nugent, Saddoris, & Gallagher, 2003; Stalnaker, Franz, Singh, & Schoenbaum, 2007) impair behavioral flexibility: the ability to adapt rapidly to novel situations. Cross-species comparison of these lesion studies suggests that two types of behavioral flexibility can be dissociated: strategy switching and reversal learning.

Strategy switching requires the animal to change behavior across task dimensions or cognitive sets. In a +-maze, for example, a strategy switch might require rats to shift from a spatial navigation maze task (e.g. always go to the North arm) to an egocentric body turn response task (e.g. always make right turns). In multiple paradigms, lesions or inactivation of the rat PL-IL produce deficits in strategy switching (Birrell & Brown, 2000; de Bruin et al., 1994; Ragozzino et al., 1999a; Ragozzino, Detrick, & Kesner, 1999b; Ragozzino, Kim, Hassert, Minniti, & Kiang, 2003; Rich & Shapiro, 2007).

In contrast, reversal learning requires animals to withhold previously rewarded responses and produce previously unrewarded ones within the same strategy or cognitive set. Lesions or inactivation of the rat OFC are associated with deficits in reversal learning. OFC dysfunction reduces sensitivity to changes in the reward value of an odor cue (Kim & Ragozzino, 2005; Schoenbaum et al., 2002, 2003; Stalnaker et al., 2007) as well as impairs body-turn reversal (Ghods-Sharifi, Haluk, & Floresco, 2008) and texture discrimination reversal (McAlonan & Brown, 2003).

The present experiments investigated the mechanisms of strategy switching and reversal learning in the rat PFC. If strategy switching and reversal mechanisms are completely independent processes, then PL-IL inactivation should only impair strategy switching and OFC inactivation should only impair reversal. Prior training in strategy switches should not affect subsequent reversal learning performance, and vice versa. Here, PL-IL or OFC were each infused with muscimol in the context of repeated strategy switches and reversals. We confirm that PL-IL infusion impairs the retention of strategy switches (Rich & Shapiro, 2007) and found that OFC infusion impairs the retention of reversals. Furthermore, the impairment associated with OFC infusion disappeared after repeated reversals, suggesting that training altered the neural systems supporting reversal learning. Finally, strategy switching and reversal learning were affected asymmetrically by training history. While repeated reversal learning did not alter the strategy switching deficits produced by PL-IL muscimol infusions (Rich & Shapiro, 2007), repeated strategy switching eliminated the reversal deficit associated with OFC infusions. These results imply that strategy switching and reversal learning are not completely dissociable processes. Rather, as suggested originally by learning theorists (Sutherland and Mackintosh, 1971), reversal learning in strategy switching are organized hierarchically, and the hierarchy is implemented in the structural heterogeneity of the rat PFC. Strategy switching includes overlapping task demands and information processing with reversal, and switching activates the neural systems associated with reversal. Thus, prior training on strategy switching is sufficient to initiate learning processes and change the neural systems required for normal reversal learning.

Methods

The experimental protocol was adapted from (Rich & Shapiro, 2007).

Animals

40 male, 2 month-old Long–Evans rats weighing approximately 300 g at the beginning of testing were housed individually in a colony room held on 12 hr light/dark cycle. After acclimating to the colony room for at least one week, rats were food-restricted to 85% of their ad libitum body weight and maintained on a food-restricted diet for the duration of the experiment. All procedures with animals were performed in accordance with National Institutes of Health (NIH) and Institutional Animal Care and Use Committee (IACUC) guidelines.

Maze

An elevated +-shaped maze consisting of four wooden arms (65×8 cm) meeting at 90° angles was used (Fig. 1A). A wooden block prevented access to the unused Start arm on each trial. On each trial, both Goal arms were open, but only one held food. Food wells drilled into the ends of each arm held cereal reward. The bottoms of the wells were made of mesh screen, below which an inaccessible food reward was placed to minimize the influence of odor cues in the task. A waiting platform was placed next to the maze. The maze and waiting platform were open to the testing room, the walls of which had several distal visual cues.

Figure 1.

Figure 1

Open in a new tab

(A) The +-maze has two Start arms (East or West). On each trial, the unused Start arm is blocked. The animal chooses between two possible Goal arms (North or South), only one of which is rewarded. (B) The +-maze provides 4 different tasks. Two spatial tasks are solved by entering the goal arm in one spatial location (North or South). Two response tasks are solved by executing a specific body turn response (Right or Left) to enter a goal arm. A strategy switch occurs when the task is changed from spatial to response or vice versa. A spatial reversal occurs when the task is changed from one spatial task to the other.

Behavior Testing

On a given trial, either the East or the West arm was designated as the “Start” arm, and either the North or South arm was designated the “Goal” arm. In all tasks, the rat was placed on a Start arm and trained to enter one of the Goal arms for food reward (half of a piece of Froot Loops cereal; Kellogg’s, Battlecreek, MI). This study distinguished paths, tasks, and strategies as hierarchically-related, increasingly abstract descriptions of behavior in the + maze (Rich & Shapiro, 2009; Fig. 1B). A path describes a specific trajectory through the maze, e.g. East arm-to-North arm (E->N). A task is a rule describing the set of paths leading to reward, e.g. in the spatial task "go North" both paths ending in the North goal arm (E->N and W->N paths) are rewarded. A strategy is an abstract rule that does not dictate reward contingencies, but rather the stimulus-response category relevant to solving the task, e.g., the spatial tasks "go North" and "go South" require approaching specific allocentric locations, while the response tasks “turn Right” and “turn Left” require executing specific egocentric body turns. Thus, the +-maze includes four potential tasks divided into two strategies. Previous experiments have shown that different strategies on the +-maze require different memory systems (White & McDonald, 2002). Place tasks and response tasks required similar overt behaviors, but depended on the hippocampus or dorsal striatum, respectively, for correct performance.

Maze acclimation

Before surgery, rats were handled, acclimated to the testing environment, and allowed to forage for food spread over the maze until the rat would consumed all the food twice in less than 10 minutes during a single testing day.

Surgery

Rats were implanted with cannulas bilaterally into either PL-IL or OFC. Rats were anesthetized with continuous-flow isoflurane and mounted in a stereotaxic frame. The scalp was anesthetized with local application of .2–.3 cc of lidocaine with epinephrine, shaved, sterilized with betadine, and incised and retracted. After cleaning the skull, burr holes were drilled at the following stereotaxic coordinates: PL-IL: +3.0 mm AP, ±1.8 mm ML; OFC: +4.0 mm AP, ±2.4 mm ML. Twenty-six-gauge guide cannulas (Plastics-One, Inc.; Roanoake, VA) were implanted - 2.0 mm DV, 14° for PL-IL lateral from vertical and -1.5 mm DV, 10° lateral from vertical for OFC. The guide cannulas were affixed to the skull with dental acrylic and skull screws. Dust caps with dummy cannulas (Plastics-One, Inc.; Roanoake, VA) were inserted to ~1 mm below the cannula tips to maintain patency. Rats were allowed to recover for 5–10 d after surgery before beginning maze training.

Pretraining

After surgery rats were re-acclimated to the maze with another day of foraging for randomly distributed food rewards. The next day cereal rewards were placed only in the food cups on both Goal arms and one Start arm was blocked. Rats were placed on each Start arm twice and given access to both Goal arms for a total of four trials. The direction of their first turn was recorded on each trial and three or more turns in the same direction was noted as a turning bias. If a rat displayed a turning bias and was required to learn a response task as their initial strategy, it was trained on the response task opposite to its turning bias as its initial task.

Training on Initial Task

Each rat was pseudo-randomly assigned to one of the four place or response tasks. On each trial, the rat was placed at the distal end of a Start arm facing the center of the maze and allowed to enter one of the Goal arms. Entering one full body-length into a Goal arm defined a choice, but the trial did not end until the rat either proceeded to the end of the arm or attempted to turn around and enter the other arm. If the rat chose the correct arm, it was allowed to consume the food and was placed on the waiting platform until the maze could be re-baited for the next trial. If the rat entered the incorrect arm, it was returned to the waiting platform with no reward. Inter-trial intervals were 5–8 s. During training, the West Start arm was used until the rat chose the correct Goal on two consecutive trials. Then, the Start arm was changed to East, and the series of trials continued from the East Start arm until the rat again made two consecutive choices correct. The Start arm was then changed back to the West start arm. Training proceeded in this manner (West until two in a row correct, East until two in a row correct, etc.) until the rat reached the training criterion of six consecutive correct trials. The animal was limited to 40 trials on each training day. Twenty-four hours after reaching the 6-in-a-row correct criterion, the rats were tested for 24 trials with pseudo-randomly ordered Start arms such that no more than three consecutive trials used the same Start arm. Rats were required to perform at least 80% of trials correctly (four or fewer errors) each day until they met this performance criterion on two consecutive days. During initial training no infusions were administered, and no rat was given either reversal or switching training prior to the infusion tests described next.

PL-IL or OFC Inactivation

Prior to each strategy switch or reversal testing day, rats were infused into either PL-IL or OFC. Either 100 ng/µL muscimol (Sigma, St Louis, MO) in 0.9% NaCl or just 0.9% NaCl vehicle was bilaterally infused. For all infusions, rats were briefly anesthetized with inhaled isoflurane. The dummy cannulas were removed and 33-gauge infusion cannulas (Plastics-One, Inc.; Roanoake, VA) extending 1.5 mm below the tip of the guide cannulas were inserted. Infusion cannulas were connected by Silastic tubing to 2 µL Hamilton syringes (Hamilton; Reno, NV) mounted on an infusion pump. The infusion was delivered at a rate of 0.25 µL/min for 2 min. Infusion cannulas were left in place for 3 min to allow for diffusion. Rats were then placed back in their home cage to recover from the anesthesia. Twenty minutes after the infusions were completed the rats were tested on the maze.

Strategy Switch/Reversal Training and Evaluation

Rats were initially assigned to perform either repeated spatial reversals (Experiments 1 and 3) or strategy switches (Experiments 2 and 4) (Fig. 2). Within each experiment, animals were randomly assigned in equal numbers to initial tasks and treatment orders. For all subsequent switches or reversals, the muscimol and saline infusions were alternated within a single animal. Strategy switching occurred when the task was changed from spatial task to response task or vice versa. Reversal occurred when the task was changed within a strategy, i.e. from one spatial task to another. Only spatial reversals – changing between North to South or South to North Goals – were used because animals would not reliably perform response reversals in a single testing day. The rats were tested on the new task by changing the rewarded Goals arms from the beginning of the testing day. The order of Start arms on each testing day was West until two correct, East until two correct, etc. until the animal scored 6 in a row correct. Once this criterion was reached, rats were tested for 24 trials with pseudo-randomly ordered Start arms (never a run of more than three). The following day, rats were tested for retention of the new task for 24 trials with pseudo-randomly ordered Start arms. The combination of testing day and the following day of retention trials constituted a single strategy switch or reversal. If the animal performed 4 errors or less (>80% correct) on the second day, it was switched or reversed again the next day. If the animal made more than 4 errors (<80% correct), it was remediated the following day with 24 trials with pseudo-randomly ordered Start arms. Remediation continued on subsequent days – 24 trials each day – until the animal performed fewer than 4 errors on a single day of testing.

Figure 2.

Figure 2

Open in a new tab

Experimental Design. Initial training was identical for all animals with the exception of cannula placement into either PL-IL or OFC and the balanced assignment of initial tasks. All animals were infused with either muscimol (gray boxes) or saline vehicle (white boxes) on each switch or reversal. The top and bottom rows for each experiment indicate two groups: one that received muscimol on the first switch/reversal and the other that received vehicle. Arrows indicate when animals in one experiment were used in subsequent experiments. After the first switch/reversal, muscimol and saline infusions alternated across multiple switches and reversals for each animal. Experiment 1 tested animals on a single spatial reversal. Experiment 2 tested animals on a single strategy switch. Experiment 3 tested the persistence of OFC deficits in reversal by performing 3 additional spatial reversals with the OFC-implanted animals from Experiment 2. Experiment 4 tested performance over 4 or 5 additional switches with the OFC implanted animals from Experiment 2. The switches were followed by a single spatial reversal. Trials to criterion and Errors at t=0 hrs were measured on the first day of testing (T0), and Errors at t=24 hrs was measured on the following day of testing (T24) for all switches and reversals.

Statistical Analysis

Trials to criterion (TTC) measured the rate of acquisition: the number of trials required for the animal to perform 6 in a row correct during the first day of testing on a strategy switch or reversal. Errors t=0 hrs measured the within-day performance of the new task: the number of errors the rat performed during the 24 trials on the first day of testing. Errors t=24 hrs measured the retention of the new task the following day: the number of errors the rat performed during the 24 trials on the day after the first day of testing. Statistical analysis was performed using ANOVA looking for significant drug, infusion region, and reversal/switch number effects as well as interactions. Post hoc tests compared different groups using Fisher’s Least Squared Difference.

Histology

After testing was complete, rats were deeply anesthetized with isoflurane and pentobarbital (50 mg/ml, ip) and transcardially perfused with ice-cold PBS followed by 4% paraformaldehyde. Brains were removed and postfixed in 4% paraformaldehyde for at least 24 hr and then cryoprotected in 18% followed by 30% sucrose solution. 50 µm coronal sections were cut on a cryostat and mounted on slides. Sections were stained with formol-thionin and compared with a standard brain atlas (Paxinos & Watson, 1998) to confirm correct cannula placement.

c-Fos staining

Sections were deparaffinized with xylene and rehydrated with 100%, 95%, and 70% EtOH followed by distilled water. Antigens were reclaimed by boiling the slides in 10 mM sodium citrate pH 6.8 for 10 min. Endogenous peroxides were quenched by incubating for 30 min in .3% H2O2. The slides were stained for a Rabbit anti-c-Fos primary antibody (Abcam, Inc.; Cambridge MA; ab7963) using a RTU Vectastain Elite ABC Kit (Vector Laboratories; Burlingame, CA; PK-7200) according to the manufacturer’s protocol. The slides were then counterstained for Nissl substance.

Results

Histology

Cannula tracks were visible in coronal sections, and their tips were located by comparison to a brain atlas (Paxinos & Watson, 1998). The tips of cannulas in the PL-IL implanted animals clustered on the PL-IL border, whereas the tips of cannulas in the OFC implanted animals clustered in the cleft immediately above the olfactory bulb (Fig. 3).

Figure 3.

Figure 3

Open in a new tab

(A) Cannula tip locations of PL-IL (●) and OFC (○) implanted animals. Coronal sections from +2.7 to +4.7 mm AP reproduced from a brain atlas (Paxinos and Watson, 1998). PL-IL is indicated by light gray, and OFC is indicated by dark gray. Representative formol-thionin stained sections through (B) PL-IL and (C) OFC. The larger diameter tracks of the guide cannulas and the smaller tracks of the infusion cannula tips are visible.

Confirmation of Inactivation

One PL-IL- and one OFC-cannulated animal were infused unilaterally with muscimol using the same infusion protocol as the behavioral experiments. The right hemisphere was infused with muscimol, and the left hemisphere was infused with saline vehicle. Brain sections from these animals including PL-IL and OFC were stained for the immediate early gene c-Fos as a marker of activity. c-Fos activity in the right hemisphere in both animals was markedly reduced compared to the left hemisphere (Supplementary Fig. 1). The reduction in c-Fos staining suggests that muscimol infusion was effective at reducing activity in the targeted region at the dosage used in the behavior experiments.

Acquisition

All animals acquired initial tasks in 1–3 days. Animals acquired spatial tasks more quickly than response tasks (TTC ± SEM: Spatial 28.5 ± 2.7 vs. Response 45.9 ± 3.3; Student’s t-test, t(18.3) = 2.224, p = .039) (data not shown).

Experiment 1: OFC, but not PL-IL, Muscimol Impairs Retention of Spatial Reversals

Muscimol infusion in either PL-IL (N = 8) or OFC (N = 8) did not impair initial acquisition (Fig. 4A) or within-day performance (Fig. 4B) of the first spatial reversal (Fig. 2). However, drug infusion in OFC significantly impaired retention of the new task the following day (Fig. 4C). Muscimol infused into the OFC had significantly greater retention errors (LSD(12) = 3.25–4.5, p < .01 for all intergroup comparisons).

Figure 4.

Figure 4

Open in a new tab

1st spatial reversal learning. Rats were infused with either muscimol (MUSC) or saline vehicle (SAL) into either PL-IL or OFC. Drug infusion did not affect the rate of acquisition (A) or the within-day performance of the new task (B). Muscimol infusion into OFC but not PL-IL significantly impaired reversal retention (C). ±SEM,* p<.05, Fisher’s LSD for all inter-group comparisons, N = 4.

Experiment 2: PL-IL, but not OFC, Muscimol Impairs Retention of Strategy Switches

Neither PL-IL (N = 12) nor OFC (N = 12) inactivation impaired acquisition (Fig. 5A) or within-day performance (Fig. 5B) of the first strategy switch (Fig. 2). However, muscimol infusion into PL-IL impaired retention of the new task the following day (Fig. 5C). Rats with muscimol infusion into PL-IL had significantly more retention errors (LSD(20) = 3.17–5, p <.05 for all intergroup comparisons). Experiments 1 and 2 reveal a double dissociation between strategy switching and reversal in the rat PFC. OFC, but not PL-IL, is required for retention of reversal learning; PL-IL, but not OFC, is required for retention of strategy switching.

Figure 5.

Figure 5

Open in a new tab

1st strategy switch. Rats were infused with either muscimol (SAL) or saline vehicle (VEH) into either PL-IL or OFC. Drug infusion did not affect the rate of acquisition (A) or the within-day performance of the new task (B). Muscimol infusion into PL-IL but not OFC significantly impaired strategy switching retention (C). ±SEM,* p<.05, Fisher’s LSD, for all inter-group comparisons, N = 6.

Experiment 3: Reversal Learning after Extensive Training

To establish whether OFC inactivation impaired spatial reversals after repeated training, we tested the 8 OFC-implanted rats from Experiment 1 on three additional spatial reversals (Fig.2). As in Experiment 1, OFC muscimol infusion did not significantly affect acquisition or within-day performance of the new task. However, the rate of reversal acquisition significantly improved over multiple spatial reversals, regardless of infusion (Fig. 6A), so that TTC was negatively correlated with reversal number (p <.001, R = −.659). Reversal retention was impaired by muscimol infusion on the first, but not subsequent, reversals (Fig. 6B) (LSD(24) = 3.25–5.5, p <.001 for all intergroup comparisons).

Figure 6.

Figure 6

Open in a new tab

Multiple spatial reversals. Drug infusion did not affect the rate of acquisition, but learning speed improved over multiple reversals (p <.001, R = −.659) (A). Muscimol infusion into OFC significantly impaired retention of the new task only after the 1st spatial reversal (B). ±SEM, * p<.05, Fisher’s LSD for all intergroup comparisons, N = 4.

Experiment 4: Reversal Learning after Training in Strategy Switches

Even after training on repeated reversals, animals infused with muscimol into PL-IL still showed a deficit in retention of strategy switches (Rich & Shapiro, 2007). If strategy switching and reversal learning are parallel processes, then analogous results should be obtained here: training on repeated switches should not affect the deficit observed in reversal with OFC muscimol infusion. OFC-implanted animals from Experiment 2 (N = 10) were tested on four or five additional strategy switches – such that their last strategy switch ended on a spatial task. These animals were then tested on a single spatial reversal learning task. Note that these animals had received no prior training on reversals. As expected, OFC inactivation did not impair acquisition or retention during the additional strategy switches, as shown in Experiment 2. Muscimol infusion impaired the retention of spatial reversals in animals without prior switching training (LSD(14) = 2.75–3.75, p <.05 for all intergroup comparisons) (Fig. 7). Muscimol infusion into OFC impaired retention of spatial reversals, but this impairment disappeared when the animals were trained on repeated strategy switches.

Figure 7.

Figure 7

Open in a new tab

Muscimol infusion into OFC significantly impaired retention following spatial reversal (NAIVE), but not after prior training on strategy switches (SWITCH-TRAINED). ±SEM, * p<.05, Fisher’s LSD for all intergroup comparisons, N = 4 for NAIVE groups, N = 5 for SWITCH-TRAINED groups.

Discussion

These results confirm a double dissociation between strategy switching and reversal in the medial and orbital regions of the rat PFC. Further, they extend our previous research by showing that OFC – in addition to PL-IL – is only transiently required for flexible responding. However, though strategy switching and reversal can be double dissociated, they are not fully independent. Training in strategy switching changes the neural systems required for the retention of reversals. These results show that strategy switching and reversal are organized hierarchically with respect to learning.

PL-IL and OFC have Distinct Actions in Facilitating Behavioral Flexibility

Experiments 1 and 2 verified a double dissociation between strategy switching and reversal learning in the rat PFC suggested by previous work (Birrell & Brown, 2000; de Bruin et al., 1994; Ghods-Sharifi et al., 2008; Kim & Ragozzino, 2005; McAlonan & Brown, 2003; Ragozzino et al., 1999a, 1999b, 2003; Rich & Shapiro, 2007; Schoenbaum et al., 2002, 2003; Stalnaker et al., 2007). This dissociation between strategy switching and reversal learning has been observed in primates (Dias et al., 1996a, 1996b; Izquierdo et al., 2004; Jones & Mishkin, 1972) and humans (Bechara et al., 1994; Milner, 1963) suggesting that observations made about behavioral flexibility in rodents may generalize to other species.

The task demands in the +-maze used in these experiments provide important insight into the implementation of strategy switching and reversal learning and help better to define the neural basis of a strategy. According to an operational definition, strategy switching requires modifying the "analyzer" (Macintosh, 1975), attentional set (Birrell & Brown, 2000) or a higher order rules (Wise, Murray, & Gerfen, 1996) required for correct performance, whereas reversal requires reassigning the reward association within a given set, rule, or strategy. However, the link between attentional sets, higher order rules, or strategies to any specific functional anatomy is unclear. In these experiments, spatial reversal required the animal to reassign the reward location in the room, but did not require changing either the relevant stimulus dimension (e.g. from allocentric to egocentric cues) or the memory system (i.e. the hippocampus to caudate) required for the basic task. Hence, reversal did not require a change between any of the neuronal systems needed to perform correctly; rather, the task-dependent activity within a system had to be reassigned to reflect new reward contingencies. By contrast, strategy switching required modifying both the relevant stimulus dimension (from allocentic to egocentric cues and vice versa) and the required memory system (from hippocampus for the spatial task to dorsal striatum for the response task and vice versa) (White & McDonald, 2002). Switching required a change in the neuronal systems needed to perform correctly. By extension, “cognitive sets” or “strategies” could be thought of as representing different neuronal systems. From this view, PL-IL and OFC functions are distinguished by the different operations they perform on these systems. PL-IL facilitates strategy switches by modulating the efficacy or relative control by one neural system or another, and OFC facilitates reversal by helping to reassign the association between reward and representations within a given neural system.

Multiple Learning Processes Improve Flexible Performance

Experiment 3 showed that OFC is only required for the retention of the first spatial reversal. Similarly, retention deficits in strategy switching after PL-IL muscimol were limited to the first three strategy switches (Rich & Shapiro, 2007). The changing effects of muscimol infusions cannot be explained by reduced sensitivity caused by multiple infusions. Rats given as many as twelve muscimol infusions into PL-IL in the context of spatial reversals were impaired subsequently when muscimol was infused during strategy switching (Rich & Shapiro, 2007). That a PFC region is required for retention of novel tasks changes, but not to familiar ones implies that the neural systems supporting flexible responding are themselves plastic. One explanation for the attenuated effectiveness of muscimol is that with training the animal changes its approach to solving the task and that this changing approach correlates with changes in the brain regions required for performance. Prior work found that muscimol infusions into the PL-IL lost effect on repeated switches because, rather than using memory to guide the most recently learned strategy, the rats responded to immediate changes in task contingency on a given day (Rich and Shapiro, 2007) . A similar learning process may occur during reversal. OFC may no longer be required after the first reversal because the animal changes its approach: rather than using memory to follow the most recently learned task, it responds to the task contingencies on each testing day. Alternatively, reversal learning may require OFC transiently because the OFC helps to establish novel associations among learned responses and reward valences (McAlanon & Brown, 2003). From this view, once a memory item has been linked to multiple reward valences, then OFC’s contribution is no longer needed for rapid reversal learning. Modifications in the neural systems required for responding to task changes – leading to resistance to impairment by PFC inactivation – is one way in which training improves behavioral flexibility.

These experiments also identified other learning processes that improve switch and reversal learning. TTC declined significantly over multiple spatial reversals (Experiment 3) and strategy switches (Experiment 4), and these improvements were unaffected by OFC infusion. The acquisition rate in Experiment 3 continued to improve into the 4th reversal, when OFC-inactivation no longer impaired reversal retention, suggesting that learning processes other than those involved in the improvements in retention are engaged. The mechanism of these improvements was not assessed in these experiments, but prior findings suggest they may have resulted from more rapid activity changes in the memory structures necessary for the performance of spatial and response tasks. The hippocampus is required for spatial tasks, and the dorsal striatum is required for response tasks (White & McDonald, 2002). The improved acquisition rates of spatial reversals may result from more rapid changes in hippocampal coding in response to new task demands. Similarly, faster acquisition of strategy switches may result from a more rapid transition between task-dependent representations in the hippocampus and striatum. Though no study, to our knowledge, has tested changes in the speed of task-dependent activity changes with training, task switches and reversals do change hippocampal and striatal neuronal codes (Eschenko & Mizumori, 2007; Ferbinteanu & Shapiro, 2003). PFC-independent mechanisms of transition within and between multiple memory systems may improve flexible responding through more rapid changes in task-dependent activity.

Hierarchical Learning of Strategy Switching and Reversal

PL-IL and OFC mediate fundamentally different types of behavioral transitions, but these experiments show that they are hierarchically organized with respect to learning. Prior training in reversals does not reduce the deficit in switching associated with PL-IL muscimol infusion (Rich & Shapiro, 2007). In contrast, Experiment 4 showed that after repeated strategy switches OFC-muscimol animals show normal retention of spatial reversals. That training in strategy switching is sufficient to support learning and thereby change the brain regions required for an unfamiliar task, reversal, suggests that switching and reversal have some similar task demands – e.g. reward reassignment to a novel task. Similar task demands lead to activation of the neural systems associated with reversal, including OFC, and supports learning even though these systems are not required for switching. Thus, with respect to learning mechanisms, strategy switching and reversal learning are organized hierarchically. The results imply that switching engages reversal mechanisms, even if these processes are not required for maintaining new strategies.

Several interpretations may explain how strategy switching activates changes in activity in regions such as OFC and supports learning. First, animals that switch strategies repeatedly experience at least one spatial task, albeit not the one learned during the spatial reversal. If all spatial tasks are represented as a single spatial “strategy code,” then the previous experience activating this representation could explain the absence of deficit. Second, animals subjected to repeated strategy switches learned a common path that was tested during spatial reversal. For example, an animal repeatedly switched between the “South” and “Left” tasks learns a common path with the “North” task; the path from the West Start arm is the same for both “Left” and “North.” If tasks are represented in terms of rewarded or reinforced paths (e.g. West Start arm → North Goal arm), then the previous experience with one of the paths could explain the improved performance in spatial reversal after multiple switches. These interpretations suggest that activity changes in regions such as OFC during switching may facilitate learning and improve performance on later spatial reversals. Further experiments will assess these predictions by recording OFC activity during strategy switches.

Conclusion

The experiments report a double dissociation between strategy switching and reversal learning in the rat PFC. OFC, but not PL-IL, is required for the retention of reversals, whereas PL-IL, but not OFC, is required for the retention of strategy switches. The experiments extend previous results to show that both rat PFC regions are required transiently for flexible responding. Finally, though strategy switching and reversal are dissociable, they are hierarchically-related with respect to learning. Strategy switching engages neuronal mechanisms, possibly within the OFC, that alter the neural systems required for subsequent reversals.

Supplementary Material

suppl data

Footnotes

Publisher's Disclaimer: The following manuscript is the final accepted manuscript. It has not been subjected to the final copyediting, fact-checking, and proofreading required for formal publication. It is not the definitive, publisher-authenticated version. The American Psychological Association and its Council of Editors disclaim any responsibility or liabilities for errors or omissions of this manuscript version, any version derived from this manuscript by NIH, or other third parties. The published version is available at www.apa.org/journals/bne.

References

  1. Bechara A, Damasio AR, Damasio H, Anderson SW. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition. 1994;50:7–15. doi: 10.1016/0010-0277(94)90018-3. [DOI] [PubMed] [Google Scholar]
  2. Birrell JM, Brown VJ. Medial frontal cortex mediates perceptual attentional set shifting in the rat. Journal of Neuroscience. 2000;20:4320–4324. doi: 10.1523/JNEUROSCI.20-11-04320.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. de Bruin JP, Sanchez-Santed F, Heinsbroek RP, Donker A, Postmes P. A behavioural analysis of rats with damage to the medial prefrontal cortex using the Morris water maze: evidence for behavioural flexibility, but not for impaired spatial navigation. Brain Research. 1994;652:323–333. doi: 10.1016/0006-8993(94)90243-7. [DOI] [PubMed] [Google Scholar]
  4. Dias R, Robbins TW, Roberts AC. Dissociation in prefrontal cortex of affective and attentional shifts. Nature. 1996a;380:69–72. doi: 10.1038/380069a0. [DOI] [PubMed] [Google Scholar]
  5. Dias R, Robbins TW, Roberts AC. Primate analogue of the Wisconsin Card Sorting Test: effects of excitotoxic lesions of the prefrontal cortex in the marmoset. Behavioral Neuroscience. 1996b;110:872–886. doi: 10.1037//0735-7044.110.5.872. [DOI] [PubMed] [Google Scholar]
  6. Eschenko O, Mizumori SJ. Memory influences on hippocampal and striatal neural codes: effects of a shift between task rules. Neurobiology of Learning and Memory. 2007;87:495–509. doi: 10.1016/j.nlm.2006.09.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Ferbinteanu J, Shapiro M. Prospective and Retrospective Memory Coding in the Hippocampus. Neuron. 2003;40:1227–1239. doi: 10.1016/s0896-6273(03)00752-9. [DOI] [PubMed] [Google Scholar]
  8. Ghods-Sharifi S, Haluk DM, Floresco SB. Differential effects of inactivation of the orbitofrontal cortex on strategy set-shifting and reversal learning. Neurobiology of Learning and Memory. 2008;89:567–573. doi: 10.1016/j.nlm.2007.10.007. [DOI] [PubMed] [Google Scholar]
  9. Izquierdo A, Suda RK, Murray EA. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. Journal of Neuroscience. 2004;24:7540–7548. doi: 10.1523/JNEUROSCI.1921-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Jones B, Mishkin M. Limbic lesions and the problem of stimulus--reinforcement associations. Experimental Neurology. 1972;36:362–377. doi: 10.1016/0014-4886(72)90030-1. [DOI] [PubMed] [Google Scholar]
  11. Kim J, Ragozzino ME. The involvement of the orbitofrontal cortex in learning under changing task contingencies. Neurobiology of Learning and Memory. 2005;83:125–133. doi: 10.1016/j.nlm.2004.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. McIntosh NJ. A theory of attention: variations in the associability of stimuli with reinforcement. Psychological Review. 1975;82(4):276–298. [Google Scholar]
  13. McAlonan K, Brown VJ. Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behavioral Brain Research. 2003;146:97–103. doi: 10.1016/j.bbr.2003.09.019. [DOI] [PubMed] [Google Scholar]
  14. Milner B. Effects of different brain lesions on card sorting: The role of the frontal lobes. Archives of Neurology. 1963;9:100–110. [Google Scholar]
  15. Paxinos G, Watson C. The Rat Brain in Stereotactic Coordinates. San Diego, CA: Academic Press, Inc.; 1998. [Google Scholar]
  16. Ragozzino ME, Wilcox C, Raso M, Kesner RP. Involvement of rodent prefrontal cortex subregions in strategy switching. Behavioral Neuroscience. 1999a;113:32–41. doi: 10.1037//0735-7044.113.1.32. [DOI] [PubMed] [Google Scholar]
  17. Ragozzino ME, Detrick S, Kesner RP. Involvement of the prelimbic-infralimbic areas of the rodent prefrontal cortex in behavioral flexibility for place and response learning. Journal of Neuroscience. 1999b;19:4585–4594. doi: 10.1523/JNEUROSCI.19-11-04585.1999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Ragozzino ME, Kim J, Hassert D, Minniti N, Kiang C. The contribution of the rat prelimbic-infralimbic areas to different forms of task switching. Behavioral Neuroscience. 2003;117:1054–1065. doi: 10.1037/0735-7044.117.5.1054. [DOI] [PubMed] [Google Scholar]
  19. Rich EL, Shapiro ML. Prelimbic/infralimbic inactivation impairs memory for multiple task switches, but not flexible selection of familiar tasks. Journal of Neuroscience. 2007;27:4747–4755. doi: 10.1523/JNEUROSCI.0369-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Rich EL, Shapiro ML. Rat prefrontal cortical neurons selectively code strategy switches. Journal of Neuroscience. 2009;29:7208–7219. doi: 10.1523/JNEUROSCI.6068-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Schoenbaum G, Nugent SL, Saddoris MP, Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport. 2002;13:885–890. doi: 10.1097/00001756-200205070-00030. [DOI] [PubMed] [Google Scholar]
  22. Schoenbaum G, Setlow B, Nugent SL, Saddoris MP, Gallagher M. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learning and Memory. 2003;10:129–140. doi: 10.1101/lm.55203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Stalnaker TA, Franz TM, Singh T, Schoenbaum G. Basolateral amygdala lesions abolish orbitofrontal-dependent reversal impairments. Neuron. 2007;54:51–58. doi: 10.1016/j.neuron.2007.02.014. [DOI] [PubMed] [Google Scholar]
  24. Sutherland NS, Mackintosh NJ. Mechanisms of Animal Discrimination Learning. New York: Academic Press; 1971. [Google Scholar]
  25. White NM, McDonald RJ. Multiple parallel memory systems in the brain of the rat. Neurobiology of Learning and Memory. 2002;77:125–184. doi: 10.1006/nlme.2001.4008. [DOI] [PubMed] [Google Scholar]
  26. Wise SP, Murray EA, Gerfen CR. The frontal cortexbasal ganglia system in primates. Critical Reviews in Neurobiology. 1996;10:317–356. doi: 10.1615/critrevneurobiol.v10.i3-4.30. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

suppl data
NIHMS157228-supplement-suppl_data.ppt (266.5KB, ppt)

RESOURCES