Operant Procedures for Assessing Behavioral Flexibility in Rats

Abstract

Executive functions consist of multiple high-level cognitive processes that drive rule generation and behavioral selection. An emergent property of these processes is the ability to adjust behavior in response to changes in one’s environment (i.e., behavioral flexibility). These processes are essential to normal human behavior, and may be disrupted in diverse neuropsychiatric conditions, including schizophrenia, alcoholism, depression, stroke, and Alzheimer’s disease. Understanding of the neurobiology of executive functions has been greatly advanced by the availability of animal tasks for assessing discrete components of behavioral flexibility, particularly strategy shifting and reversal learning. While several types of tasks have been developed, most are non-automated, labor intensive, and allow testing of only one animal at a time. The recent development of automated, operant-based tasks for assessing behavioral flexibility streamlines testing, standardizes stimulus presentation and data recording, and dramatically improves throughput. Here, we describe automated strategy shifting and reversal tasks, using operant chambers controlled by custom written software programs. Using these tasks, we have shown that the medial prefrontal cortex governs strategy shifting but not reversal learning in the rat, similar to the dissociation observed in humans. Moreover, animals with a neonatal hippocampal lesion, a neurodevelopmental model of schizophrenia, are selectively impaired on the strategy shifting task but not the reversal task. The strategy shifting task also allows the identification of separate types of performance errors, each of which is attributable to distinct neural substrates. The availability of these automated tasks, and the evidence supporting the dissociable contributions of separate prefrontal areas, makes them particularly well-suited assays for the investigation of basic neurobiological processes as well as drug discovery and screening in disease models.

Download video file ^{(22.8MB, mp4)}

Introduction

High-level cognitive processes including rule generation, behavioral selection, and strategy evaluation are collectively referred to as “executive function” or “behavioral flexibility¹.” Such processes are crucial to normal cognitive function, and may be impaired in such diverse disorders as schizophrenia, alcoholism, depression, stroke, and Alzheimer’s disease^2-7. The regulation of executive function processes is primarily mediated by areas in the frontal cortex, including the dorsolateral prefrontal cortex and the orbitofrontal cortex in humans^8-10.

The development of tasks to assess executive function and/or behavioral flexibility in nonhuman animals, particularly rodents, has greatly advanced the understanding of the neurobiology of cognition^11-14. Such tasks have made it possible to separately measure distinct components of behavioral flexibility, including strategy shifting and reversal learning. Strategy shifting refers to the ability to actively suppress a previously learned response strategy while acquiring a new, competing strategy, particularly across stimulus dimensions (extradimensional shift) — e.g., switching from performing a visually-based discrimination (red vs. green, where red is “correct” and tactile stimuli are irrelevant) to performing a tactile discrimination (smooth vs. rough, where smooth is “correct” and visual stimuli are now irrelevant). On the other hand, reversal learning also involves a change in response strategy, but within the same stimulus dimension — e.g., in the “red vs. green” example, if red was previously correct, a reversal would dictate that green is now correct, while tactile stimuli would remain irrelevant.

Several tasks have been developed to investigate behavioral flexibility in rodents. The cross-maze task requires an animal to first learn either a direction-based rule (e.g., “always turn right”) or a visual-based rule (e.g., “always turn toward the visual cue”) to a certain criterion of performance. Then, the animal is required to unexpectedly shift either across modality to the opposite rule (strategy shifting, originally referred to as a “nonreversal shift”¹⁵) or shift within modality to the opposite contingency (reversal learning)^13,14,16. Such tasks are sensitive to disruptions in cortical and subcortical networks, involving the prefrontal cortex, thalamus, and striatum^{1,13,14,16-18}. Another type of attentional set-shifting task (sometimes referred to as the digging task) requires training animals to discriminate between two containers that differ along two or three stimulus dimensions (digging media, odor, and/or external texture). Similar to the cross-maze task, animals are then required to shift either across dimensions (strategy shifting) or within the same dimension (reversal learning), and these tasks are similarly sensitive to frontal cortex manipulations^11,19. An advantage of this task is that during the extra-dimensional strategy shift, rats are presented with novel sets of stimuli (exemplars), which ensures that performance impairments during this stage are likely attributable to disruptions in the ability to shift attentional set to different aspects of compound stimuli, rather than an impaired ability to stop approaching a specific stimulus previously associated with reward. However, this feature also makes it more difficult to ascertain the specific nature of a deficit during a set shift.

Although the tasks described above have been well documented in the literature, they both suffer from a number of procedural disadvantages, primarily the length of time it takes to test animals. In both the cross-maze task and the digging task, only one animal may be tested at a time; furthermore, testing must be administered in real time by a dedicated experimenter, and may take up to several hr per day per animal. In addition, the presentation of stimuli and the recording of behavioral responses in both types of tasks are manually controlled by an experimenter, and are thus vulnerable to human error and subjective interpretation.

Here, we describe an automated method for assessing strategy shifting and reversal learning in the rat, using operant procedures that streamline stimulus control and data presentation, and dramatically improve the rate of data collection and throughput^20,21. The methods used to shape and train rats are described, as well as the components of the task itself and the analysis of the resulting data. We have found that like the cross-maze and digging tasks, these automated tasks are sensitive to disruptions in prefrontal and subcortical circuitry, as well as to a neurodevelopmental manipulation that models schizophrenia^20-23.

Protocol

NOTE: All procedures described here were approved by the Institutional Animal Care and Use Committee (IACUC) at St. Mary’s College of Maryland, or the Canadian Council on Animal Care at the University of British Columbia.

1. Animals

1. Use adult male Sprague-Dawley or adult male Long-Evans rats. NOTE: Although differences in performance across these two strains have not been formally tested, Long-Evans rats tend to acquire a visual cue discrimination (described below) slightly faster than Sprague-Dawley rats (Floresco, unpublished observations).

2. Upon arrival in the colony, house adult rats singly or in groups, depending on the needs of the experiment and the limitations of the facility. Use single housing for experiments in which animals are food-restricted, to provide better control of food intake. Allow animals to acclimate to the colony (without handling or food restriction) for at least 3 days after arrival. NOTE: A recent report has suggested that experimenter gender can adversely affect measures of pain and anxiety behavior²⁴, findings which may extend to other stress-sensitive behaviors including cognition. Across our studies, we have not observed any obvious performance differences in rats trained by male vs. female handlers, although we have not formally assessed this.

3. Handle animals daily for approximately 3-5 min each, for at least 3 days before beginning behavioral testing. On the first day of handling, obtain a free-feeding weight for each rat. The target weight for food restriction, if used, will be 85-90% of this free weight. Note this target weight, e.g., in a lab notebook or on the animal’s cage card. In some cases, food restriction may not be required if the reinforcer used is highly palatable (e.g., sweetened milk), although animals may become satiated too quickly with the use of such reinforcers.

4. On each of the handling days, as each animal’s handling is finished, place approximately 10-20 reward pellets inside the animal’s home cage to acclimate rats (who are typically neophobic) to the reinforcement that will be used in the task (see Section 2.1.3 below).

5. Over the three days of handling, gradually reduce animals’ daily food intake to bring them to their target weights. Be sure to indicate on the cage cards or other documentation that the animals are now on a restricted diet. NOTE: Food restriction in particular requires approval of the institutional IACUC or other regulatory body before any procedures may begin. The general procedures below were approved by the authors’ respective institutional agencies; be sure to consult any appropriate local and/or national guidelines for additional requirements at individual institutions.

6. Weigh animals at least twice a week to monitor health and ensure that animals do not drop substantially below the target weight. Ensure that water is freely available at all times.

2. Equipment and Software

1. Use operant chambers equipped with (at a minimum) two retractable levers, two stimulus lights, a houselight, and a reinforcement dispenser for these tasks.

   1. Place the levers on either side of a central reinforcement delivery area with one stimulus located above each lever.
   2. Ensure that the houselight illuminates the entire chamber while not interfering with detection of the stimulus lights, e.g., place the houselight on the wall opposite the levers and stimulus lights.
   3. Use palatable food (e.g., sucrose pellets^20,21 or sucrose solutions²⁵) for reinforcement. Wherever it is indicated to “reinforce the animal” below, deliver one 45 mg sucrose pellet or one prescribed amount of sucrose solution.

2. Control stimulus presentation, lever operation, and data collection via an interface with a computer. Contact the authors for specific information regarding task programs written with MED-PC software, a program specifically designed for behavioral testing and data collection. NOTE: A crucial feature of the programs used for behavioral testing is the recording of key variables on a trial-by trial basis, including the position of the cue light, the lever selected by the animal, whether the animal made a correct, incorrect or no response (omission) and the latency to make a choice. These data are critical for assessment of the specific types of errors made during different portions of the choice sequence, as will be described below.

3. At the beginning of pre-training, assign each animal to an operant chamber where it will be tested each day throughout the experiment. Test animals at approximately the same time of day throughout the experiment.

4. Clean operant chambers regularly (at least once a week) with soap and water and/or an antimicrobial solution.

3. Pretraining

NOTE: Once animals have reached their target food-restricted weight, they may begin shaping in the operant chambers. Pretraining procedures typically take about 10-20 days, with substantial variability between rats. See Figure 1C for an overview of procedures.

1. Shape animals to lever press.

   1. Train animals under a fixed-ratio (FR)-1 schedule of reinforcement, i.e., one reinforcement is delivered for each lever press. Shaping may be administered either with both levers extended (a press on either is reinforced), or on one lever at a time (e.g., one lever per day) with the order (left/right) counterbalanced across animals and/or experimental conditions.

      1. For shaping with both levers extended, continue shaping sessions (one 30-min session per day) until animals meet a minimum criterion of at least 50-60 presses per session for two consecutive days. This typically takes about 3-6 days.
      2. For shaping on each lever individually, continue sessions on the first lever until animals respond at least 50-60 times on the first lever presented. Subsequent shaping sessions should use the opposite lever until the rat again achieves this criterion. Typically, this second criterion is quickly acquired after rats have learned to press the first lever. NOTE: Shaping on one lever at a time will take longer (the animal must meet criterion twice, once for each lever), but will ensure that animals obtain experience responding on and alternating between the two levers, a critical component of the tasks described below.
2. Give animals retractable lever training sessions to familiarize them with the extension and retraction of the levers, and to ensure that rats are making relatively few omissions (typically <5) by the time they proceed to the main test phases of the task.

   1. On each trial, determine which lever to extend. Alternate lever extensions in a pseudorandom order such that there are 45 left-lever trials and 45 right-lever trials, but no more than two consecutive trials extend the same lever.
   2. Extend the selected lever. Reinforce the animal for a press on this lever within 10 sec, after which the lever is retracted.
   3. If the animal does not respond within 10 sec, retract the lever and record an omission.
   4. Begin trials every 20 sec throughout the session. NOTE: During retractable lever training, pre-exposure to stimulus lights (illumination of both the left and right stimulus lights upon each lever extension) may be employed to decrease the novelty and salience of the panel lights, and thus increase the difficulty of the subsequent set-shifting task²⁰. Using this procedure will markedly increase the number of trials required to achieve criterion performance on the visual cue discrimination described below, and animals may require multiple days to learn this rule under these conditions.
   5. Continue retractable lever training sessions (one 30-min session per day) for a fixed number of days, or until animals meet a minimum criterion of five or fewer omissions for two consecutive days. This typically takes about 5-10 days.

      1. For studies using acute manipulations (e.g., drug testing), use a fixed number of days (e.g., 5 days) to ensure that all rats receive similar exposure to the levers.
3. Assess animals for side preference.

   1. Conduct side preference testing immediately after the last session of retractable lever training (on the same day; see Figure 1C). The side preference task consists of seven trials, each of which is composed of between two and eight sub-trials separated by a fixed 20-sec intertrial interval (ITI).
   2. On each sub-trial, extend both levers into the chamber for 10 sec or until a lever press response is made. Do not illuminate the stimulus lights during this phase of training.
   3. Reinforce a response on either lever on the first sub-trial of each trial, and record it as the “initial response.”

      1. Do not reinforce responses on the same lever on subsequent sub-trials within the same trial. Allow up to six subsequent responses on the same lever with a trial, after which give a forced sub-trial. A forced sub-trial consists of only the opposite lever being extended for 10 sec or until a response is made.
   4. Following the initial response on each trial, reinforce the first response on the opposite lever, and then terminate that trial. Thus, within each trial (containing up to eight sub-trials), an animal is required to respond at least once on each lever.
   5. Define each animal’s side preference as the side on which the majority of initial responses took place (at least four of seven trials).
   6. However, if an animal disproportionately responds to one lever throughout the session (defined as greater than a 2:1 ratio), record that side as the animal’s preference.
   7. Begin testing on the next consecutive day after the side preference test. NOTE: In our experiences, most animals do not display a strong side preference. For those that do, requiring them to press the lever opposite their bias during response discrimination training ensures that they are learning the specific response-reward contingencies associated with that lever, rather than merely responding on a preferred lever.

4. Testing

NOTE: Animals may be tested in one of three sequences, each of which involves two different tasks. Strategy shifting is assessed using (1) Set-Shifting from Cue to Response and/or (2) Set-Shifting from Response to Cue; reversal learning is assessed using (3) Reversal of Response. (A fourth possible sequence, Reversal of Cue, is not recommended for reasons discussed below.)

1. General information about tasks and sequences.

   1. Conduct each sequence on consecutive days. Each sequence will take at least two days (initial discrimination learning and then shift or reversal).
   2. Use a maximum of between 150 and 200 trials in a single task, depending on the nature of the experiment. (Note that using higher numbers of trials will necessarily increase the total session time to 60 min or greater, which may be a factor to consider for pharmacological tests using compounds with shorter durations of action.)
   3. For each sequence, test animals in one task (“Set”) followed by a second task (“Shift” or “Reversal”). Test animals for a maximum of 3 days (i.e., 450-600 trials) on each task, for a maximum of 6 days total.

      1. Remove animals that do not reach criterion within 3 days on the first task (“Set”) from the experiment.
      2. For animals that do not reach criterion within 3 days on the second task (“Shift” or “Reversal”), assign a maximal score for trials to criterion that represents the number of trials experienced (i.e., 450 trials for 3 days of 150 trials each).
      3. If the task parameters have been modified so that control animals can achieve criterion performance on a single day, then modify the task so that all rats are given only one test session, and give those that do not reach criterion within the allotted number of trials the maximum score (150-200 trials, depending on how the task has been configured).
      4. During the shift or reversal task, have the animals either start a session having to perform the new rule immediately, or give them 20 “reminder” trials where they perform the task using the rule learned during the first phase of training, and then the rule switches during the session²². NOTE: This latter procedure is particularly useful for experiments assessing potential pro-cognitive compounds that may improve flexibility, as it can be used to clarify whether better performance during a shift/reversal is due specifically to enhanced flexibility or impaired retrieval of the previous rule that may facilitate learning of a new one during the shift.
2. Set-shifting: Cue Task to Response Task.

   1. Begin testing animals on the Cue task (see Figure 1A), which reinforces animals for responding on the lever below the illuminated stimulus light (cue). The Cue task is the “Set” task in this sequence.

      1. Begin each trial with both levers retracted.
      2. Illuminate either the left or right stimulus light for 3 sec; then extend both levers into the chamber for 10 sec or until a response occurs.
      3. Reinforce only a correct response on the signaled lever. Upon a response on either lever, retract the levers.
      4. Begin trials every 20 sec throughout the session. Pseudorandomly determine the order of trials such that no more than two consecutive trials occur with the same stimulus light (left or right) illuminated.
      5. Continue trials until an animal has reached criterion (completed 10 consecutive correct responses) and has completed a minimum of 30 trials, or until 150-200 trials are completed without reaching criterion.
      6. If criterion is not reached on the first day, test the animal on the Cue task again on the second day, but remove the requirement to complete a minimum of 30 trials. If criterion is not reached on the second day, test the animal on the third day following the same procedure.
   2. On the next day after reaching criterion on the Cue task, shift animals to theResponse task (see Figure 1B), which reinforces animals for responding on the lever opposite their side preference, regardless of stimulus light (cue) illumination. The Response task is the “Shift” task in this sequence.

      1. Begin each trial with both levers retracted.
      2. Illuminate either the left or right stimulus light for 3 sec; then extend both levers into the chamber for 10 sec or until a response occurs. (Note that the position of the stimulus light is irrelevant for this task.)
      3. Reinforce only a response on the correct position lever (left or right, opposite of the animal’s side preference). Upon a response on either lever, retract the levers.
      4. Begin trials every 20 sec throughout the session. Pseudorandomly determine the order of trials such that no more than two consecutive trials occur with the same stimulus light (left or right) illuminated.
      5. Continue trials until an animal has reached criterion (completed 10 consecutive correct responses) or until 150 trials are completed without reaching criterion.
      6. If criterion is not reached on the first day, test the animal on the Response task again on the second day. If criterion is not reached on the second day, test the animal on the third day following the same procedure.

Figure 1: Discrimination Tasks Used in the Set-Shifting Sequence. This figure displays the tasks as performed in the Cue-to-Response sequence; note that the tasks are the same, simply in opposite order, in the Response-to-Cue sequence. (A) During visual-cue discrimination learning, animals are reinforced for a response on the lever under the illuminated stimulus light. (B) During response discrimination learning, animals are reinforced for responding on one lever (either left or right) regardless of the position of the stimulus light. (C)Flowchart depicting sequence of training phases for a typical strategy shifting experiment, from pretraining to testing. Please click here to view a larger version of this figure.

1. Set-shifting: Response to Cue. NOTE: This sequence is the one that benefits most from the addition of the visual-cue light pre-exposure condition to pretraining^20,21 (see Step 3.2.4 above). Previous studies have shown that pre-exposing rats to the lights during retractable lever training makes the response-to-cue shift more difficult and dependent on the medial prefrontal cortex. Conversely, prefrontal inactivations do not impair this type of shift if these pretraining procedures are not employed²⁰.

   1. Begin animals’ testing on the Response task, which reinforces animals for responding on the lever opposite their side preference, regardless of stimulus light (cue) illumination. The Response task is the “Set” task in this sequence.

      1. Proceed with testing as detailed in Step 4.2.2 (Response task) above.
      2. Have animals complete a minimum of 30 trials on this task, since it is the “Set” task.
   2. On the next day after reaching criterion on the Response task, shift animals to the Cue task, which reinforces animals for responding on the lever below the illuminated stimulus light (cue). The Cue task is the “Shift” task in this sequence.

      1. Proceed with testing as detailed in Step 4.2.1 (Cue task) above. The minimum of 30 trials completed is not needed when this task is the “Shift” task.
2. Reversal of Response.

   1. Begin animals’ testing on the Response task, which reinforces animals for responding on the lever opposite their side preference, regardless of stimulus light (cue) illumination. The Response task is the “Set” task in this sequence.

      1. Proceed with testing detailed in Step 4.2.2 (Response task) above.
      2. Have animals complete a minimum of 30 trials on this task, since it is the “Set” task.
   2. On the next day after reaching criterion on the Response task, test animals on a reversal of the Response task, which reinforces animals for responding on the opposite lever as on the first task, i.e., the lever corresponding to their original side preference. This new Response task is the “Reversal” task in this sequence.

      1. Proceed with testing as detailed in Step 4.2.2 above, with the exception that the reinforced lever position is now equal to the animal’s original side preference.

5. Behavioral Measures

1. Record the trials to criterion on both the “Set” task and the “Shift” task. Trials to criterion is the main measure of accuracy, defined as the number of trials required to complete 10 consecutive trials, including those 10 trials. Note that the number of omissions should be factored out of this measure (e.g., if a rat requires 100 trials to achieve criterion and makes 10 omissions, the actual trials to criterion is 90).

2. Count the number of errors made before criterion was reached on both the “Set” task and the “Shift” task. Errors to Criterion is a complementary measure of accuracy that can be more sensitive than Trials to Criterion and is not affected by increased omission rates.

3. For shift error types, further analyze the types of errors made on the “Shift” task of a set-shifting sequence.

   1. Count an error as “perseverative/regressive type” when an animal responds incorrectly on the “Shift” task according to the rule that was correct on the previous day’s “Set” task. Then, use the following guidelines to divide errors of this type into perseverative and regressive, respectively.

      1. Divide the “Shift” session into blocks of 16 consecutive completed trials (do not include omitted trials). Within each block, identify which errors fit the definition of this type, i.e., the rat made an incorrect response that corresponds to the “Set” task rule. There will be a maximum of 8 possible errors of this type in each block of 16 trials.
      2. Score identified errors as perseverative until less than six of them are made within a block.
      3. Beginning with the next block and continuing through the end of the task, score errors of this type as regressive.
      4. If the animal was tested in the “Shift” task on more than one day, continue scoring errors as if the blocks were contiguous.
   2. Count an error as “never-reinforced” when an animal responds incorrectly on the “Shift” task with a response that was not correct on either the “Set” or the “Shift” task.
4. For reversal error types, further analyze the types of errors made on the “Reversal” task of a reversal learning sequence. Reversal errors are broken down along two dimensions: (1) into perseverative and regressive errors, and (2) into toward-distractor and away-from-distractor errors.

   1. Divide total errors into perseverative and regressive.

      1. Divide the “Reversal” session into blocks of 16 consecutive completed trials. Count the errors in each block (a maximum of 16 errors is possible).
      2. Score errors as perseverative until fewer than 10 of them are made within a block.
      3. Beginning with the next block and continuing through the end of the task, score errors as regressive.
      4. If the animal was tested in the “Reversal” task on more than one day, continue scoring errors as if the blocks were contiguous.
   2. Divide total errors into toward-distractor (stimulus light was illuminated above the incorrect, pressed lever) and away-from-distractor (stimulus light was illuminated above the correct, unpressed lever).
5. Record the number of omitted trials to provide a broad measure of the animal’s motivation level.

   1. Examine omission scores after each daily test session to rule out equipment malfunction, which can be indicated by high omission scores.
   2. In the absence of equipment malfunction, exclude omission data from animals with abnormally high numbers of omissions (generally, ≥3 standard deviations above the mean) on either task.

6. Record response latencies by measuring the time elapsed between lever extension and a response. Latencies provide a rough measure of motor function and/or speed of processing.

Representative Results

Acute, reversible inactivation of the prefrontal cortex can be accomplished by infusion of the local anesthetic bupivacaine hydrochloride (0.75%, 0.5 µl) into the prelimbic region via a surgically implanted infusion cannula²⁰ approximately 10 min prior to testing. Furthermore, the effects of inactivation during either the first task (“Set”) or the second task (“Shift” or “Reversal”) can be assessed to investigate possible general effects on learning. Figure 2 illustrates the results of such inactivations on animals performing the Cue-to-Response strategy-shifting sequence. Prefrontal inactivation on the first day, the Cue/“Set” task, did not impair performance (Figure 2A), suggesting that the medial prefrontal cortex is not necessary for initial discrimination learning. However, prefrontal inactivation on the second day, the Response/“Shift” task, significantly impaired performance in that animals required a substantially greater number of trials to reach criterion performance (Figure 2B). When the prefrontal cortex was inactivated, animals made more perseverative-like errors than never-reinforced errors on the Shift task (Figure 2C). These findings replicate previous data regarding the importance of the medial prefrontal cortex for strategy shifting and, in particular, in suppressing a previously learned strategy^13,20.

Conversely, animals trained in the Reversal of Response sequence did not show this prefrontal dependency. Animals receiving inactivation of the prefrontal cortex on the “Reversal” day did not differ from saline-infused animals on either the initial response discrimination (Figure 3A) or the subsequent reversal (Figure 3B)²⁰.These findings are consistent with previous research showing that the orbitofrontal cortex, not the medial prefrontal cortex, regulates reversal learning on a variety of tasks^12,19,26, including an operant task similar to the one described here²⁷.

Figure 2. Inactivation of the Prefrontal Cortex Impairs Strategy Shifting. A, Trials to criterion on the initial Cue discrimination task (“Set”) by rats receiving infusions of saline or bupivacaine (Bupi) into the medial prefrontal cortex on the set day. Prefrontal inactivation had no effect on initial acquisition. B, Trials to criterion on the shift to the Response task (“Shift”) following medial prefrontal infusions of either saline or bupivacaine on the shift day. Inactivation of the prefrontal cortex impaired the strategy shift to the response task. C, Types of errors committed by animals on the shift day. Prefrontal inactivation prior to the shift task (“sal-bupi” group) led to an increase in perseverative-like errors. *, p < .05 vs. saline-saline. This figure has been modified from Floresco et al.²⁰ Please click here to view a larger version of this figure.

Figure 3. Inactivation of the Prefrontal Cortex Leaves Reversal Learning Intact. A, Trials to criterion during the initial response discrimination training by rats that would subsequently receive infusions of saline or bupivacaine (Bupi) into the medial prefrontal cortex prior to reversal training. No differences were seen. B, Trials to criterion during the reversal of the response discrimination, following medial prefrontal infusions of either saline or bupivacaine. Prefrontal inactivation did not impair reversal learning. This figure has been modified from Floresco et al.²⁰

Data presented in Figure 4 provides an example of how requiring rats to perform “reminder” trials using the old rule prior to a strategy shift can aid in data interpretation. In this study (Enomotor and Floresco, unpublished observations), rats were matched for performance on acquiring a visual cue rule on Day 1 (Figure 4A). On Day 2, rats received vehicle or 0.2 mg/kg haloperidol. At the start of the Day 2 test session, they were given 20 trials where they were required to respond using the visual cue rule acquired on Day 1, after which the rule switched mid-session to a response discrimination. As displayed in Figure 4B, this treatment impaired retrieval of the visual cue rule during the first 20 reminder trials of the session. Subsequently, haloperidol-treated rats required fewer trials to achieve criterion (Figure 4C) and made fewer perseverative errors (Figure 4D) on the strategy shift. Note that had we not used the reminder trials prior to the shift, these data may have been interpreted as an improvement in set shifting by haloperidol. However, the impairment during the rule retrieval phase suggests that these effects are better understood as impaired memory for the previously acquired rule, which may have led to less response conflict when rats were required to learn a novel rule and thus faster shifting.

Figure 4. Impaired Rule Retrieval and Facilitated Set-Shifting Induced by Haloperidol Treatment. A, Trials to criterion on visual cue discrimination from rats that were to receive vehicle (saline) or the dopamine D2 antagonist haloperidol (0.2 mg/kg) prior to the strategy shifting sequence on the following day. Animals in both groups showed comparable pre-drug performance. B, At the beginning of testing on Day 2, rats received 20 reminder trials where they were required to continue to respond using the visual cue rule from Day 1. Treatment with haloperidol significantly decreased accuracy during these reminder trials. C, After the 20 reminder trials, the rule shifted mid-session to a response discrimination. Haloperidol treated rats required fewer trials to achieve criterion during the shift. D. Haloperidol treatment also reduced perseverative errors. Although these data could suggest improved performance, the impairment in rule retrieval display in B indicates that the apparent “enhanced strategy shifting” is more likely attributable to reduced interference from the previously acquired rule. Enomoto and Floresco, unpublished observations. *, p < .05 vs. vehicle.

The neonatal ventral hippocampal lesion (NVHL) manipulation has been used to model some aspects of schizophrenia in rats²⁸, particularly cognitive impairments^29,30. Briefly, an excitotoxic lesion is administered to the hippocampus of 7-day old rats, and testing is carried out in adults (60+ days postnatal). This models the hypothesized developmental trajectory of schizophrenia²⁸. Figure 5 illustrates performance of NVHL and control rats on the pre-exposed version of the Set-Shifting: Response to Cue sequence. NVHL rats are unimpaired at learning the first rule (Response/”Set”, Figure 5A), but are dramatically impaired at shifting to the new rule (Cue/”Shift”) as shown by the increase in the number of trials required to reach criterion (Figure 5B). Moreover, this deficit was due primarily to an increase in perseverative errors, as shown in Figure 5C, suggesting prefrontal abnormalities^20,21. These results confirm previous findings of impaired strategy shifting in NVHL animals using the cross-maze task²⁹.

Similar to the data from prefrontally-inactivated animals shown above, NVHL animals were not impaired at reversal learning (Figure 6A,B), although they were slower to respond (Figure 6C,D). This negative finding implies that the observed strategy shifting deficits are not attributable to a simple inability to switch between stimuli²¹.

Figure 5. Impaired Set-Shifting in the NVHL Model of Schizophrenia. Performance on the pre-exposed version of the set-shifting sequence (Response-to-Cue) in NVHL and sham control animals. A, NVHL animals were unimpaired on the Response (“Set”) task. B, However, NVHL animals required significantly more trials than shams to reach criterion on the Visual Cue (“Shift”) task. C, Errors on the “Shift” day. NVHL animals made more perseverative errors than sham animals, but did not differ on regressive or never-reinforced errors. *, p < .05 vs. sham. This figure has been modified from Placek et al.²¹

Figure 6. Lack of NVHL Impairment on Reversal Learning. A,B, NVHL and sham animals did not differ in their ability to acquire either the initial Response Learning task (“Set”), or the Response Reversal. C,D, NVHL animals were slower than shams to respond on both the “Set” and the “Reversal” tasks. This figure has been modified from Placek et al.²¹

Finally, pilot testing has indicated that animals are virtually unable to learn a reversal of the Cue task, i.e., to press the lever opposite the illuminated stimulus light. Five of six animals tested completed 450 reversal trials (3 days) without reaching criterion, and the sixth animal required 418 trials (Brady, unpublished observations; data not shown). This is likely because the stimulus lights are very salient and attractive cues that make it very difficult for rats to direct responding away from them. Thus, this testing sequence is not recommended.

Discussion

The development of behavioral tasks to measure higher-order cognitive constructs in rodents is essential to advance knowledge of the neurobiology of cognition. With well-constructed and validated tasks, rodents can be assessed on tasks of complexity rivaling those of primates or even humans. Here we have shown how two aspects of executive function, strategy shifting and reversal learning, can be investigated in rodents using automated operant techniques. Using these automated tasks, we have replicated previous findings in cross-maze and digging tasks regarding the neural substrates of set-shifting and reversal learning^{11,13,18-21,27,29}, suggesting that the operant tasks are valid assessments of these constructs.

These automated tasks have a number of benefits and advantages over existing non-automated cross-maze and digging tasks. Most compelling is the superior rate of data collection in the automated operant version. Each day’s training or testing takes only 30-60 minutes, and is fully computer-controlled requiring minimal supervision by the experimenter. Moreover, several animals can be tested simultaneously with a multi-chamber operant setup. Each task series, from shaping to final testing, can be completed in approximately 2-3 weeks. Another important advantage of the automated tasks is the precise control of stimulus presentation, thus minimizing the possibility of experimenter error. For example, the order of presentation of cue location on each trial is randomized and controlled by the computer, rather than by an experimenter manually consulting a trial-by-trial list. The timing between trials is precisely measured and consistent, and is not confounded by the time it takes an experimenter to, e.g., remove a rat from the cross-maze or rearrange the digging containers. Reinforcement delivery is automatic and is not subject to experimenter error (e.g., forgetting to bait the correct arm of a cross-maze). Data collection is similarly improved, with automatic recording of response patterns including the measurement of exact response latencies. In the absence of other motor abnormalities, changes in response latencies can be used to infer evidence of altered processing speed and/or to judge the level of cognitive complexity of a task^21,22.

The automated tasks also retain one important advantage of the cross-maze tasks: the ability to conduct a detailed analysis of the types of errors made on the shift or reversal day. Distinguishing between set-shifting errors that replicate the previous day’s strategy (perseverative or regressive errors) and errors that represent previously untried strategies (never-reinforced errors) can assist in characterizing specific deficits in behavioral flexibility. In particular, perseverative errors occurring early in testing reflect an animal’s inability to abandon the previous strategy, while later-occurring regressive errors reflect an animal’s inability to maintain the new strategy once perseveration has ceased²⁰. Never-reinforced errors may indicate a failure to acquire the new strategy, or an inability to respond systematically according to a rule²⁰. Previous findings^16,17,20 demonstrating dissociable neuroanatomical bases of these types of errors are also valuable in interpreting the results of these tasks.

Our procedures have been developed and optimized for use with rats. This being said, other groups have used similar procedures for testing set-shifting abilities in mice³¹. However, certain modifications need to be employed with mice to accommodate for species differences. These include longer presentation of the visual cue light prior to lever extension, training over multiple days using 30 trials/day and incorporation of a time-out punishment after incorrect choices. Although these modifications make this assay less amenable for use with pharmacological challenges, it could prove useful for assessing cognitive flexibility in genetically altered mice (although it is unclear whether these modifications would preserve the frontal cortex sensitivity of the task).

Of course, there are also limitations to these tasks. Some of these limitations arise from the automated nature of the task, while others are related to the parameters of the task itself. With regard to the latter, the set-shifting task described here (as well as the cross-maze set-shifting task²⁶) utilize a restricted set of stimuli and responses. Unlike the digging task, on which novel exemplars (e.g., unfamiliar scents or digging media) can be used to construct new attentional sets at each stage^11,19, the operant set-shifting task necessarily requires choosing between two stimuli that are familiar to the animal — either the left vs. right cue light, or the left vs. right position. This means that the operant and cross-maze set-shifting tasks involve response conflict as well as strategy shifting, although the concept of shifting one’s strategy to a new, previously irrelevant stimulus dimension is preserved^20,23. On a related note, the set-shifting and reversal operant tasks as described here do not allow for a third stimulus dimension, as in the digging task which may include digging media, odor, and texture^11,19. However, we do not consider this a fatal flaw, as the operant set-shifting task still requires the animal to suppress the previously relevant discrimination strategy and attend to a previously ignored stimulus dimension. Additionally, it seems conceivable that modifications to the equipment and task parameters could support the addition of a third stimulus dimension, such as auditory cues or odor, although these additions would likely make learning more difficult and less amenable to single-day pharmacological tests.

Finally, a potential limitation of any operant-based task is the loss of direct information regarding rat behavior — i.e., the experimenter is no longer watching the rat. We feel that the advantages in objectivity and data collection speed conferred by automation more than make up for this loss, and cameras mounted in the operant chambers are a relatively easy way to restore individual visual access if desired.

There are a number of steps that can be taken to maximize success using these operant tasks. First, the importance of handling the animals before training begins cannot be overstated; as with any behavioral task, well-handled animals are easier to work with, are less stressed, and tend to produce less variable data. Second, some pilot testing may be necessary to determine the best time of day to conduct testing; we test during the light cycle, and find that performance is optimal when animals are tested near the end of this cycle (e.g., approximately 4:00 pm for a light cycle ending at 7:00 pm). Third, care should be taken to confirm that stable performance is established at each pretraining stage before an animal is advanced to the next step. For example, consistent and robust performance at the retractable lever training stage is an excellent predictor of proficient performance on the “set” discrimination task. Regarding the equipment, although all steps are automated, experimenter intervention remains necessary to confirm that all components are in working order. For example, an equipment check should be run daily (or more than once a day, if large number of animals are being tested) to ensure that all lights, levers, and reward delivery systems are operational. In particular, malfunctions in reward delivery systems (particularly pellet dispensers) can drastically affect performance. An unusually high number of omissions on a given day may indicate an issue with reward delivery equipment, and thus data output should be checked every day by an experimenter familiar with the task and expected performance levels. In the absence of an equipment malfunction, a high number of omissions may indicate other problems with motivation or animal health. If an animal is otherwise healthy, food restriction may be increased to take the animal to 80-85% of the free-feeding weight for a short time until performance recovers.

These set-shifting and reversal tasks can be used in a variety of experimental paradigms. For example, the effects of manipulations such as lesions, developmental treatments, dietary manipulations, long-term pharmacological treatment, or genetic modifications could be investigated. While the effect of a treatment on the set-shifting or reversal stage may be of primary interest, note that since such chronic or permanent treatments must necessarily be administered before training begins, effects on multiple stages of performance (particularly on the initial discrimination or “set”) must also be examined²¹. The use of acute manipulations, such as pharmacological treatments or temporary neuroanatomical inactivations, are particularly well-suited to these tasks. In such cases, the addition of a third group (as illustrated in Figure 2) is useful; thus, the primary experimental group receives the manipulation of interest on the day of shift or reversal, while one control group receives the manipulation on the day of initial discrimination or “set” to test for broad effects on learning, and a second control group receives no manipulations (or sham treatments) on both days^20,22. Note that for such acute manipulation studies, it is advisable to match rats for performance during the learning of the initial set and allocate them to the experimental group and (second) control group accordingly. This minimizes the possibility that treatment-induced differences in performance may be confounded by individual variations in how readily rats learn to discriminate between stimuli. Furthermore, if an experiment requires testing of multiple cohorts over weeks or months, each cohort should include animals from all experimental groups. For example, a study testing the effects of acute pharmacological manipulations during a shift may require 48 rats in total and 3 experimental groups, tested in three cohorts of 16 animals each. In this case, each cohort should contain 5-6 rats in each experimental group. Ideally, the statistical analyses should include a factor that confirms there were no differences in performance across each cohort of rats. Finally, these operant tasks may be particularly useful for applying in vivo recording techniques, including microdialysis, voltammetry, and electrophysiology, due to components such as the controlled environment, precise timing of stimulus presentation and responses, and restricted movements of the animals which are not available or practical in the cross-maze or digging tasks.

Disclosures

Publication of and free access to this manuscript was supported by Med Associates, Inc.