Pavlovian Contextual and Instrumental Biconditional Discrimination Learning in Mice

Abstract

Genetically-modified animal models are a powerful tool for investigating the link between neurological and behavioral changes and for the development of therapeutic interventions. Executive function deficits are symptomatic of many human clinical disorders but few tasks exist for studying executive functions in mice. To address this need, we describe procedures for establishing Pavlovian contextual and instrumental biconditional discriminations (BCDs) in C57BL/6J mice. In the first experiment, contextual cues disambiguated when two short duration stimulus targets would be followed by food pellets. In the second experiment, discrete visual cues signaled when lever press or nose poke responses would be continuously reinforced with food pellets. Mice learned both BCDs as evidenced by differential responding in each cue during training and, more critically, during extinction testing. The implications of these findings for using BCD tasks to analyze the neural substrates of executive processing in animal models are discussed.

1. Introduction

Recent advances in transgenic and knockout mouse technology provide unparalleled opportunities to probe the molecular mechanisms involved in learning and memory [1-5]. To take full advantage of the insights that genetically-modified mice might yield, however, requires the development of more sophisticated behavioral assays [3, 6]. To this end, we describe procedures for obtaining relatively rapid acquisition of a Pavlovian contextual biconditional discrimination (BCD) and an instrumental BCD in mice. Versions of these BCD tasks have been analyzed quite extensively in the rat from the perspectives of contemporary learning theory [7- 14] and the neuroscience of brain function [15-17]. Our studies used the inbred C57BL/6J mouse because this strain is widely used for the generation and analysis of transgenic and knockout models and its genome has been completely sequenced.

In a standard Pavlovian biconditional discrimination task, four stimuli (A, B, X, and Y) of equal and typically short duration are presented in pairs correlated with reinforcement (AX+ and BY+) and nonreinforcement (AY- and BX-) so that the compounds but not the elements reliably signal the trial outcomes [18]. To optimize this task to study executive functions [19], researchers increase the duration of A and B (the cues) relative to X and Y (the targets) thus creating asynchronous compounds. Onset asynchrony between a cue and its target is thought to discourage configural conditioning [20-22] in favor of a modulatory process whereby a cue either acts on the association between a target and its outcome [23-24] or alters the threshold for outcome activation by a target [25]. Onset asynchrony may involve strictly serial compounds in which the cues A and B completely precede the targets X and Y, partially overlapping compounds where the cue onsets precede target onsets but where A and B co-terminate with X and Y, and embedded compounds where the cues, usually different contexts, completely straddle the targets.

Although some studies have reported differential responding between reinforced and nonreinforced asynchronous compounds, mean rates of conditioned responding are rarely provided for each of the four compounds. Instead, a combined response rate to the reinforced targets is statistically compared with a combined response rate to the nonreinforced targets. Such a comparison is inadequate when appropriate differential target responding occurs in only one of the two cues and is either absent or even reversed in the other cue. To demonstrate true biconditional discrimination performance with asynchronous compounds, responding to AX+ must be greater than to AY- and responding to BY+ must be greater than BX-. This principle also applies to instrumental BCD tasks where one response (R1) is reinforced in cue A but not in cue B and the other response (R2) is reinforced in cue B but not in cue A. True instrumental biconditional discrimination learning depends on showing that R1 is greater than R2 in cue A and that R2 is greater than R1 in cue B. Few studies have met this criterion.

A second criterion that is also rarely met by contextual and instrumental BCD studies is the demonstration of differential responding between reinforced and nonreinforced targets or responses, respectively, in extinction. Testing in the absence of outcome presentations is crucial for eliminating the use of potential within-session rules. For example, if contextual cues differing in physical location or in features such as odor or substrate are used to disambiguate when a target is reinforced and when it is not, target training with each contextual cue has to be conducted in separate sessions. Consequently, during training, target outcomes at the start of a session could guide responding for the remainder of that session [26]. Similarly, in an instrumental BCD, a win-stay, lose-shift strategy could generate reliable and appropriate differential responding during training. However, neither rule would be able to guide differential responding in extinction when the outcomes are not delivered.

We report two experiments in which C57BL/6J mice successfully acquired a contextual BCD and an instrumental BCD. In both experiments, differential responding was obtained in each biconditional cue during training and extinction testing. Both tasks were acquired relatively rapidly and true BCD performance was observed in the majority of subjects making these procedures particularly suitable for using transgenic and knockout mice to investigate the molecular mechanisms underlying executive functions and their disturbance in clinical populations.

2. Materials and methods

2.1 Subjects

The subjects were 24 experimentally naïve male C57BL/6J mice (Jackson Laboratories, Bar Harbor, ME) approximately 50 days old at the start of the experiment. They were pair- housed in ventilated plastic tubs (28.6 cm × 13.2 cm × 13.0 cm) in a temperature-controlled shared rodent facility on a 12:12-h light:dark cycle. The tail of one mouse in each pair was marked with a black stripe for individual identification. Mice were fed daily and maintained at 85% of their free-feeding weight. Water was always available in the home cage.

2.2 Apparatus

The apparatus consisted of eight identical mouse conditioning chambers (Med Associates, St. Albans, VT) measuring 24.0 cm × 20.1 cm × 18.6 cm. The two end walls of each chamber were aluminum, and the side walls and ceiling were clear polycarbonate. Each chamber contained a recessed food magazine in the center of one end wall. Activation of a pellet dispenser delivered one 20-mg food pellet (Dustless Precision Pellets, BioServ, NJ) into the magazine. An infrared detector and emitter system was mounted on the inside walls of the magazine, permitting automatic recording of head entries into the magazine. A retractable lever, 1.5 cm wide and protruding 0.6 cm into the chamber, was mounted 4.2 cm from the left hand side of the food magazine and 2.0 cm above the chamber floor. A circular recess, 1.3 cm in diameter and 1.0 cm deep, was located 4.4 cm from the right hand side of the magazine and 0.8 cm above the chamber floor. When not in use, access to these manipulanda was prevented by placing metal covers over the openings. The chamber floor was composed of 0.4 cm stainless steel rods spaced 0.6 cm apart. A plastic sheet measuring 21.3 × 18.5 × 0.03 cm could be inserted over the grid floor to create a solid flat surface.

Each chamber was housed in a soundproof, light-resistant shell, with a 1-cm spy hole in the front door. A houselight (L) with diffuser (Med Associates, St. Albans, VT) was mounted on the ceiling directly above the chamber. A 28-V yellow panel light (Y) was mounted on the rear polycarbonate wall 2 cm above the grid floor and 3 cm from the right aluminum wall containing the food magazine. Two speakers, mounted vertically on the wall opposite the magazine, were used to present a white noise (N) and a 2900 Hz tone (T). Experimental events were controlled and recorded automatically by interfacing (Med Associates, St. Albans, VT) and a 486 computer located in a room adjacent to the experiment boxes.

2.3 Contextual BCD

Previous studies of contextual BCD learning have consistently used contexts that differed not only in appearance but in their physical location [11, 26-30]. We eliminated differences in physical location by using the same box in the same location with the grid floor exposed (Context A) or covered with a plastic insert (Context B) to create the two distinctive contextual cues. We also included a novel control group to assess directly the potential role of a within- session rule based on initial target outcomes in acquisition of a contextual BCD task.

Sixteen mice were randomly assigned to Group BCD and trained in two contexts (A and B) distinguished only by their floors. In each context, two 10-s auditory stimuli (noise and tone) were presented in a randomized order. In one context, the tone but not the noise was paired with food; in the other context, the noise but not the tone was paired with food. The remaining 8 mice were assigned to Group NC and received the same treatment as Group BCD except that their training always occurred in Context A. With this arrangement, Group NC could use a within- session rule based on initial target outcomes to respond preferentially to reinforced targets in training but not in extinction.

2.3.1 Magazine training

Subjects were given two sessions of magazine training. For Group BCD, one session was in Context A and the other was in Context B. For Group NC, both sessions were in Context A. In each session, the magazine was preloaded with 3 food pellets and 15 food pellets were delivered on a variable-time (VT) 60-s schedule.

2.3.2 Contextual BCD training

There were 24 days of contextual biconditional discrimination training. On each day, Group BCD received 8 trials with each auditory stimulus in Context A and 8 trials with each auditory stimulus in Context B. For half the subjects, 10-sec presentations of N were immediately followed by food pellets and 10-sec presentations of T were nonreinforced in Context A (A: N+, T-); these contingencies were switched in Context B (B: T+, N-). For the other half, T but not N was reinforced in Context A (A: T+, N-); these contingencies were switched in Context B (B: N+, T-). The order in which context training occurred each day was determined by an ABBABAAB Gellerman sequence and counterbalanced across animals and contingencies.

For Group NC, subjects received two sessions a day in Context A. In one daily session, 10-sec presentations of N were immediately followed by food pellets and 10-sec presentations of T were nonreinforced; in the other daily session, these contingencies were switched so that T was reinforced and N was not. The order of which session was first each day was determined by an ABBABAAB Gellerman sequence.

For both groups, an equal number of reinforced and nonreinforced trials occurred in the first half of a session and the order was randomized across days. The order of trials in the second half of a session was the mirror image of the order during the first half of that session. The mean intertrial interval (ITI) was 120 sec. The number of photobeam interruptions by head entry into the magazine was recorded during each 10-sec presentation of an auditory stimulus, during the 10 seconds immediately preceding the onset of an auditory stimulus, and during the ITI.

2.3.3 Extinction

Subjects were given one extinction test in each context that was identical to their training session with two exceptions: there were only 4 presentations of each target and no foods were delivered. The order of testing was counterbalanced across contexts and target-outcome contingencies.

2.4 Instrumental BCD

Sixteen days after completing the contextual BCD task, mice from Group BCD were trained with two 30-s visual cues and two instrumental responses, lever press and nose poke. Lever presses but not nose pokes were correct in one visual cue and nose pokes but not lever presses were correct in the other visual cue. Each correct response was reinforced with a food pellet; incorrect responses had no programmed consequence. One simple solution to this task is a win-stay, lose-shift strategy. Following a light onset, either one of the two responses would be executed. If a food pellet followed that response, then the response would be repeated (win-stay), but if no pellet was delivered, the subject would switch to the other response (lose-shift). Deployment of this within-session rule during cue presentations would support differential responding in training but not in subsequent extinction testing.

2.4.1 Response training

There were four sessions of response training, two with the lever and two with the nose poke. In each session, responses were reinforced on a continuous reinforcement (CRF) schedule until 15 pellets had been earned. All sessions were conducted in the dark. The order of training was lever, nose, nose and lever.

2.4.2 Instrumental BCD training

Subjects received 12 daily sessions of instrumental BCD training with two visual cues, L and Y. Each session contained eight 30-sec presentations of L and eight 30-sec presentations of Y which flashed on and off every second. During L and Y presentations, one response was reinforced on a CRF schedule and the other response was not reinforced. For half the subjects, lever presses were reinforced in the presence of L and nose pokes were reinforced in Y; for the other half this was reversed. These contingencies were orthogonal to those experienced in the contextual BCD task. The mean ITI was increased incrementally across sessions, as has been our practice in related studies with rats [8, 31]. The mean ITI was 15 seconds in sessions 1 and 2, 30 seconds in sessions 3 through 8, and 60 seconds in sessions 9 through 12. An equal number of L and Y presentations occurred in the first half of a session and the order was randomized across days. The order of L and Y presentations in the second half of a session was the mirror image of the order during the first half of that session. Lever press and nose poke responses were recorded during the first and second 15 seconds of each stimulus and during the ITI.

2.4.3 Extinction testing

Subjects were given a single session of extinction testing that was identical to the final training session with two exceptions. First, responses were never reinforced, and second, there were only 4 presentations of each cue.

3. Results

3.1 Contextual BCD Acquisition

Acquisition of the Pavlovian contextual BCD is shown in Figure 1 for Group BCD. Mean responses per minute to the reinforced and nonreinforced auditory targets and in the 10-s period immediately preceding the targets are displayed separately in 4-day blocks for Context A (top panel) and Context B (bottom panel). Following an initial steep rise and then a gradual decline in responding on both reinforced and nonreinforced trials, robust differential responding emerged as responses to the reinforced targets increased whereas those to the nonreinforced targets continued to decline. A repeated measures ANOVA of the final block of training for Group BCD revealed a significant main effect of Reinforcement, F(1,15) = 29.3, p < .01, indicating that the rate of responding was higher when the targets were reinforced than not reinforced. There was no main effect of Context (F<1) and no significant Reinforcement × Context interaction (F<1). Paired sample t-tests on the final block of training showed that mean response rates were significantly higher to reinforced than to nonreinforced targets in both Context A, t(15)=4.6, p<.01, and Context B, t(15)=5.0, p<.01. The magnitude of this differential responding did not differ significantly between the first and second halves of the sessions for either context (Fs<1). For Context A, mean response rates to reinforced and nonreinforced targets were 19.0 and 5.8 responses per min in the first half and 20.2 and 6.4 responses per min in the second half; for Context B, mean response rates were 20.0 and 6.2 responses per min in the first half and 21.2 and 6.1 responses per min in the second half. Throughout training, mean response rates during the 10-s periods preceding the reinforced and nonreinforced targets were consistently low and indistinguishable in both contexts.

Figure 1.

Contextual BCD. Acquisition is shown in blocks of 4 days for Group BCD. Mean responses per minute to reinforced (Tar+) and nonreinforced targets (Tar-) and during the 10 sec periods preceding reinforced (preTar+) and nonreinforced (preTar-) targets are shown separately in Context A (top panel) and Context B (bottom panel).

Figure 2 shows the mean response rates in Group NC to the auditory targets for sessions in which the noise but not the tone was paired with food (top panel) and for sessions in which the tone but not the noise was paired with food (bottom panel). Mean response rates increased slightly to both targets initially but then remained at a stable low rate to the noise while gradually declining to the tone, regardless of when the targets were paired with food. A repeated measures ANOVA of the final block of training for Group NC revealed a significant Reinforcement × Contingency interaction, F(1,7) = 7.8, p<.05, but no significant main effects (Fs<1). Paired sample t-tests on the final block of training revealed that mean response rates were significantly higher to the noise than to the tone both when the noise was reinforced, t(7)=3.1, p<.05, and when it was not, t(7)=2.4, p<.05.

Figure 2.

Contextual BCD. Acquisition is shown in blocks of 4 sessions for Group NC. Mean responses per minute to the targets are shown when N was reinforced and T was not (top panel) and when T was reinforced and N was not (bottom panel). Mean response rates in the 10 sec periods before the reinforced (Pre+) and nonreinforced (Pre-) targets are also shown.

3.2 Contextual BCD Extinction

The results for Group BCD are shown in Figure 3. Mean responses per minute to the reinforced and nonreinforced targets in each context are plotted separately for the final day of training and for extinction testing. Mean response rates were consistently higher on reinforced than on nonreinforced trials. This difference was relatively unaffected by the omission of food during extinction testing although there was numerically less responding overall in extinction testing. A repeated measures ANOVA revealed a significant main effect of Reinforcement, F(1,15) = 46.2, p < .01, but no significant main effect of Day, F(1,15) = 3.7, p >.07 or Reinforcement × Day interaction, (F<1). A series of paired sample t-tests revealed that response rates were higher on reinforced than on nonreinforced trials during training in Context A, t(15)=4.7, p<.01, and Context B, t(15)=5.2, p<.01, and during extinction testing in Context A, t(15)=7.2, p<.01, and Context B, t(15)=6.3, p<.01.

Figure 3.

Contextual BCD. Extinction testing is shown for Group BCD. Mean responses per minute to reinforced (+) and nonreinforced (-) targets during the last day of training (Training) and during extinction testing (Testing) are shown separately in Context A (left bars) and Context B (right bars).

Inspection of individual subject data on the final day of training revealed that 12 of the 16 mice in Group BCD responded more to the reinforced than nonreinforced targets in both contexts. During extinction testing, all but one of the 16 mice in Group BCD responded more to the reinforced than nonreinforced targets in both contexts.

On the final day of training, Group NC responded significantly more to the noise than to the tone target in both sessions regardless of whether the noise was followed by food (9.2 vs 4.4 responses per min, t(7)=3.0, p <.05) or whether the tone was followed by food (9.8 and 4.4 responses per min, t(7)=2.5, p<.05). Six subjects showed this pattern while the other two were indifferent to the targets. During extinction testing, Group NC continued to show a higher rate of responding to the noise than to the tone target (9.6 and 5.1 responses per min, t(7)= 2.7, p<.05). Again, six subjects displayed this pattern while the other two were indifferent to the targets.

3.3 Instrumental BCD Acquisition

Figure 4 shows mean correct and incorrect responses per minute during L (top panel) and Y (bottom panel) in blocks of 2 sessions. For both cues, the rate of correct responses increased with training whereas the rate of incorrect responses declined. This improvement may be an effect not only of increased exposure to the reinforcement contingencies but of increased ITI duration which has been reported to promote conditional discrimination performance in rats [32]. A repeated measures ANOVA of the final block of training revealed a significant main effect of Reinforcement, F(1,15) = 392.5, p<.01, but no main effect of Cue and no significant Reinforcement × Cue interaction (Fs<1). Paired sample t-tests revealed that subjects made significantly more correct than incorrect responses during L, t(15)= 14.0, p<.01, and during Y, t(15)=12.4, p<.01. Every subject showed this biconditional response pattern.

Figure 4.

Instrumental BCD. Mean responses per minute in blocks of two sessions of training. Correct and incorrect responses are shown separately during Cue L (top panel) and Cue Y (bottom panel).

3.4 Instrumental BCD Extinction

Figure 5 shows the results of the extinction test. With the omission of food, there was, not surprisingly, an overall increase in response rates compared to training. This increase was more pronounced in L than in Y. Of more importance, however, is the fact that the discrimination remained intact and correct response rates were substantially higher than incorrect response rates in both cues. A repeated measures ANOVA revealed a significant main effect of Reinforcement, F(1,15) = 73.1, p<.01, a significant main effect of Cue, F(1,15) = 41.7, p<.01 but no Reinforcement × Cue interaction, F(1,15) = 3.0, p>.10. Paired sample t-tests revealed that subjects made significantly more correct than incorrect responses during L, t(15)=6.7, p<.01, and during Y, t(15)=6.6, p<.01. Every subject showed this biconditional response pattern.

Figure 5.

Instrumental BCD. Mean responses per minute in the extinction test. Correct and incorrect responses are shown separately during Cue L (left bars) and Cue Y (right bars).

4. Discussion

Two experiments with C57BL/6J mice demonstrated true BCD performance using Pavlovian and instrumental procedures. In the first experiment, contextual cues disambiguated when two short duration stimulus targets would be followed by food pellets. In the second experiment, discrete visual cues signaled when lever press or nose poke responses would be continuously reinforced with food pellets. In both experiments, differential responding emerged in each biconditional cue during training in the majority of subjects. Importantly, this pattern of differential responding persisted during extinction testing indicating that BCD performance was not dependent on within-session rules based on outcome presentations.

Our findings extend the results of previous BCD studies in three novel ways. First, in our demonstration of successful acquisition of a Pavlovian contextual BCD, we used simple contexts that were constructed by adding or removing a floor cover in the same box located in the same physical space. Other BCD studies have used more complex multidimensional contexts comprised from one or more sensory modalities and located in different rooms or areas of a room [11, 26-30]. An important advantage of simplifying the differences between contextual biconditional cues is that any variations in contextual BCD performance between control and genetically-modified mice cannot be attributed to their use of different features of a multimodal context or to differences in their ability to integrate those features into an internal representation of that context.

Second, the failure of the mice in Group NC to acquire differential responding over the course of their training suggests that mice are normally unlikely to adopt a within-session rule based on target outcomes to solve contextual BCDs. This type of concern has been addressed in other studies by examining BCD performance in extinction [26, 33] or on the first trial of the day [28]. An alternative approach used in our study was to see if mice could in fact learn to use such a rule to support differential responding during training when no other solution was available. We found that the mice in Group NC did not acquire this within-session rule. Instead, they responded more to the noise target than to the tone target regardless of the prevailing reinforcement contingencies. Given these results, it seems improbable that the mice in Group BCD acquired multiple strategies using the within-session rule to generate biconditional performance in training and a different rule to generate biconditional performance in extinction.

Third, our demonstration of successful acquisition of an instrumental BCD extends in two ways the results of two other studies using related procedures to examine cognitive deficits in chimeric and transgenic mice [33-34]. Johnson et al. [33 2005] used a differential outcomes procedure in which correct responses (left and right nose pokes) were reinforced on a random interval schedule with pellets in one auditory cue and with sucrose in the other auditory cue. Dickson et al. [34 2010] used the same outcome to reinforce correct responses in a discrete trial procedure in which the orientation of a line (horizontal or vertical) signaled whether the left or right lever was correct. First, in contrast to our study, neither of these studies nor others that implemented a BCD procedure with mazes in mice [28, 35] reported performance scores separately for the two biconditional cues. Second, our study used different instrumental response manipulanda (lever press and nose poke) whereas these other studies with mice used identical manipulanda in different places. This feature of our study may prove invaluable in studies with mutant mice. By using a nose poke and a lever press, we reduced the potential for generalization between the responses and we allowed the mice to encode details about the nature of the response and not simply its location.

It is clear from our extinction data that, despite the sequence of cue presentations for the contextual BCD and the use of a CRF schedule for the instrumental BCD, both problems were solved using a strategy other than within-session rules based on outcome presentations. However, the present data do not distinguish among the various associative structures that might have been used to solve the BCDs. Elsewhere, we have discussed how different permutations of binary or hierarchical associations might subserve Pavlovian contextual BCD performance [26]. We have also provided a comprehensive analysis of instrumental BCD performance in rats when the same outcome is used to reinforce correct responses in each cue [8]. Using three different behavioral probes to dissect the nature of the learned associations, we concluded that rats were using a hierarchical structure to represent the relations among the biconditional cues (S), the instrumental responses (R) and the rewarding outcome (O). Specifically, we proposed an S-(R- O) mechanism in which rats learned to associate the S with specific R-O associations. This hierarchical association is thought to develop when cues resolve the ambiguity inherent in stimulus targets and instrumental responses that have multiple outcomes [7-8, 36]. On this basis, it seems quite likely that a common strategy involving hierarchical associations was used to solve both the Pavlovian contextual BCD and the instrumental BCD.

Executive deficits are associated with frontal lobe dysfunction and appear in several clinical disorders including schizophrenia and Alzheimer's disease, as well as in cancer patients undergoing chemotherapy. In developing gene- and molecular-based treatments for executive deficits, the mouse is arguably the best mammalian genetic system to study the causal relationship between genotype and the cognitive endophenotype [3]. Results of recent lesion and inactivation studies in rats suggest that the instrumental BCD task is an especially promising candidate for assessing components of executive function in animals. For example, lesions of the dorsolateral striatum did not disrupt instrumental BCD performance [15] but lesions of the medial prefrontal cortex, and inactivation of the prelimbic cortex in particular, did impair the ability of rats to use contextual cues to resolve conflicting biconditional cue information about which instrumental response to perform [17, 37-38]. These findings support the feasibility of using the BCD procedures devised here with mice to identify specific executive deficits induced by targeted gene and molecular alterations. Additionally, the BCD task is unique in that experimental parameters can be systematically manipulated to bias animals to use nonhierarchical, configural solutions that engage other brain areas such as the hippocampal system and the striatum. Strategic comparison of performance on different versions of the BCD task can be used to rule out any general sensory, motor and motivational deficits in knockout and transgenic mouse strains and may yield theoretical insights and practical applications for translational research.

In summary, our demonstrations of Pavlovian contextual and instrumental BCDs in C57BL/6J mice satisfied the two criteria for true BCD performance in that appropriate differential responding was observed in both biconditional cues and during extinction. Our procedures established robust and relatively rapid acquisition of what is widely considered to be a challenging task making them both suitable for studies using mutant mice to examine the molecular mechanisms of executive functions. However, a truly meaningful analysis of potential differences between the BCD performance of control and genetically-modified mice will depend on the judicious application of behavioral tools that contemporary learning theorists have developed to distinguish among the different associative structures that animals might use to represent their experiences.

Highlights.

• C57BL/6J mice acquired a Pavlovian contextual biconditional discrimination

• C57BL/6J mice acquired an instrumental biconditional discrimination

• Food pellets were used to reinforce Pavlovian targets and instrumental responses

• Differential responding was observed in each cue in training and in extinction

• Performance did not depend on within-session rules based on outcome presentations