Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Jan 1.
Published in final edited form as: Behav Processes. 2013 Sep 8;101:15–22. doi: 10.1016/j.beproc.2013.08.015

Temporal Maps in Appetitive Pavlovian Conditioning

Kathleen M Taylor 1, Victory Joseph 2, Alice S Zhao 3, Peter D Balsam 4,1
PMCID: PMC3943977  NIHMSID: NIHMS530083  PMID: 24021946

Abstract

Previous research suggests animals may integrate temporal information into mental representations, or temporal maps. We examined the parameters under which animals integrate temporal information in three appetitive conditioning experiments. In Experiment 1 the temporal relationship between 2 auditory cues was established during sensory preconditioning (SPC). Subsequently, rats were given first order conditioning (FOC) with one of the cues. Results showed integration of the order of cues between the SPC and FOC training phases. In subsequent experiments we tested the hypothesis that quantitative temporal information can be integrated across phases. In Experiment 2, SPC of two short auditory cues superimposed on a longer auditory cue was followed by FOC of either one of the short cues, or of the long cue at different times in the cue. Contrary to our predictions we did not find evidence of integration of temporal information across the phases of the experiment and instead responding to the SPC cues in Experiment 2 appeared to be dominated by generalization from the FOC cues. In Experiment 3 shorter auditory cues were superimposed on a longer duration light cue but with asynchronous onset and offset of the superimposed cues. There is some evidence consistent with the hypothesis that quantitative discrimination of whether reward should be expected during the early or later parts of a cue could be integrated across experiences. However, the pattern of responding within cues was not indicative of integration of quantitative temporal information. Generalization of expected times of reward during FOC seems to be the dominant determinant of within-cue response patterns in these experiments. Consequently, while we clearly demonstrated the integration of temporal order in the modulation of this dominant pattern we did not find strong evidence of integration of precise quantitative temporal information.

Keywords: temporal integration, associative learning, temporal map

Introduction

Learning about time is an integral part of associative learning (Balsam, Drew, and Gallistel 2010; Diaz-Mataix et al. 2013). Whether one considers such learning as the acquisition of a temporal map (Balsam and Gallistel 2009; Honig and Urcuioli 1981) or as the encoding of an attribute of the conditioned stimulus (CS) (Arcediano and Miller 2002; Denniston and Miller 2007; Molet and Miller 2013) it is clear that temporal parameters have a large impact on learning and performance. The timing of events alters the speed with which anticipatory conditioned responses (CRs) emerge (Balsam and Gallistel 2009; Gallistel and Gibbon 2000; Gibbon and Balsam 1981), the pattern of CR expression within the CS (Balsam, Drew, and Yang 2002; Brandon, Vogel, and Wagner 2003; Bitterman 1965; Drew et al. 2005; Kirkpatrick and Church 2003), and even the topography of the CR itself (Silva and Timberlake 1997; Vogel, Brandon, and Wagner 2003; Smith 1968; Holland 1980). Additionally, once this temporal learning has occurred the information can be used in flexible ways. One feature of this flexibility makes it analogous to spatial maps; subjects can integrate temporal information across separate experiences when there are common elements in the learning episodes (Molet et al. 2012). Spatial maps of large areas are built up through sequential experiences of overlapping subsets of the total map (Collett, Harland, and Collett 2002; Gallistel and King 2009; O’Keefe and Nadel 1978; Shapiro, Tanila, and Eichenbaum 1997). Though acquired sequentially, an integrated representation of the information can guide behavior to new locations (Blaisdell and Cook 2005) or be used to infer novel routes to a goal (Foo et al. 2007; Foo et al. 2005; Gallistel 1990; Tolman 1948).

Evidence for a similar ability to integrate temporal information across separate experiences comes primarily from both second-order conditioning (SOC) and sensory pre-conditioning (SPC) studies which have shown that animals have the ability to superimpose temporal maps from different training phases provided there are common elements in each phase (Molet et al. 2012). In a sensory preconditioning experiment animals are first presented with forward pairings of two neutral CS’s, A and B, where A immediately precedes B (A → B). In the next phase the value of one of these stimuli (B) is changed by pairing it with a motivationally significant event, for example a food unconditioned stimulus (US), B → Food. Once the CR has been established to B (B → CR), the integration of information across phases is evident when the changed value of B is reflected in a change in the value of A, even though A has never been directly associated with the US (A → CR). This integration reflects the animal’s knowledge of the order and perhaps timing of events. For example, in a variant of the SPC procedure Miller and colleagues (Arcediano, Escobar, and Miller 2003; Cole, Barnet, and Miller 1995; Molet et al. 2012) showed that when B is backward paired with the US, B is not excitatory. Nevertheless, A controls a strong excitatory response as would be expected if subjects can infer that a US that comes before B would be expected just after A. Data like these encourage the view that animals are capable of integrating temporal information across experiences. Most studies have employed aversive conditioning paradigms and demonstrated that subjects can integrate order information. However the encoding and use of order information does not necessarily require a quantitative appreciation of time. The integration of temporal order and integration of quantitative temporal information may be separable processes mediated by different neural substrates (Buhusi and Meck 2005; Eichenbaum 2013; MacDonald et al. 2011; Ivry and Schlerf 2008; Shapiro and Eichenbaum 1999). Thus it is important to explore in more detail whether quantitative temporal information, like metric spatial information, can be integrated across experiences.

Only one appetitive conditioning study indicates that integration of quantitative temporal knowledge is possible (Leising, Sawa, and Blaisdell 2007). Here we explore this possibility in more detail. Experiment 1 sought to demonstrate temporal order integration in appetitive conditioning with a method similar to the aversive conditioning procedures of Arcediano, et al. (2003). In subsequent experiments we explored the conditions that give rise to integration of quantitative temporal information across experiences.

Experiment 1

Experiment 1 sought to establish that temporal order information could be integrated across experiences in appetitive conditioning as demonstrated in aversive conditioning. Subjects were exposed to temporal information in two separate training phases as illustrated in Figure 1. During the first phase of training two auditory cues (A and B) were presented in sequence. Two groups were exposed to these cues in forward order (A→B) as in a standard SPC experiment, while two groups experienced backward pairings (B → A). First-order conditioning occurred in the second phase, where B was presented either before or after the delivery of a food pellet. One forward-ordered SPC group and one backward-ordered SPC group received forward CS-US pairings (B → US) while the two remaining groups received backward CS-US pairings (US → B) of B with food. From the traditional associative point of view which assumes weak learning as a result of backward pairing, only the subjects that received forward pairings in both phases of the experiment should show an excitatory response when tested with A. However, if animals integrate temporal knowledge across experiences, then each group of animals would each have different expected times of reinforcement (Figure 1). Group Post-A received SPC of A→B followed by first order conditioning of B →Food. If the animal is responding based on temporal expectation, there may be little excitatory conditioning to A as the temporal expectation for food would be well after the end of the A. However, Group Late-A, with identical SPC training (A→B), received backward pairings of the food US and B (Food→B). In this case, the integrated temporal information would lead to the expectation of food at the end of the A. The prediction here is somewhat counterintuitive. We expected greater excitatory conditioning to A, even though there is very little excitatory conditioning to B based on the backward pairing of food and B (Food→B). In the groups given backward SPC training with B→A, similar predictions follow. During FOC, Group Pre-A receives backward pairings of Food→B, and the temporal expectation induced by the presentation of A is that the time for the delivery of the food US has passed, as B precedes A in SPC and occurs after food is delivered in FOC. Therefore, we expected to see very little response to A. In Group Early-A animals are given forward paired B→Food during FOC, and the temporal expectation would be for food to be available at the beginning of A. Consequently, if the animals integrate temporal information across phases we predict higher responding when tested with A in Groups Early-A and Late-A, both of which should expect food during A, and less responding in groups Pre-A and Post-A, both of which should not expect the food to be available during A. These predictions were tested in the first experiment.

Figure 1. Procedure of Experiment 1.

Figure 1

Open in a new tab

Four groups of animals were trained with procedures depicted in the figure.

Rats were given sensory pre-conditioning (SPC) training with Cue A and Cue B. In groups A Early and Pre A B precedes A. In other groups the order was reversed. All groups then experienced first order condition (FOC) in which cue B was either forward or backward paired with food. Cue C was used to bracket the food during FOC. The groups are named to indicate where the subjects might come to expect food relative to the Cue A if they integrate temporal information across the two phases. The potential inferred time of food is indicated by the bracketed asterisk in the third row references the test phase for each group.

Method

Subjects

Male Sprague Dawley rats (CD:Crl Charles River Laboratory, Raleigh, NC; N=40) were pair housed in clear polycarbonate cages, and maintained on a 12:12-h light/dark cycle in a controlled environment with temperatures of 22 ± 2° C. Animals were acclimated to the animal colony for at least one week after arrival and were weighed and handled at least 3 times prior to training. Beginning 3 days prior to training, rats were restricted to 1 hour per day of food with unlimited access to water. Training consisted of one session per day. All behavioral training occurred after light onset (7am), and feeding occurred after all animals had completed the training for that day but prior to light offset. These animal husbandry procedures were maintained for each of the experiments reported here. All husbandry and testing procedures were approved by the Columbia University Institutional Animal Care and Use Committee.

Behavioral Apparatus

In each experiment the subjects were trained and tested in modular operant test chambers (MedAssociates, Georgia, VT, model ENV-008, 30 × 24 × 21 cm). The chambers were individually housed inside light and sound-attenuating isolation boxes equipped with fans providing ventilation and a background noise level of approximately 65 dB. The floor of the operant chamber consisted of stainless steel rods spaced 15 mm apart. A food trough (5 × 5 cm) was centered in the right wall of the chamber, 2 cm above the floor. Head entries to the trough were detected by an infrared photocell. A response lever, not used in this experiment, was located to the left of the food trough. The chambers were dimly illuminated throughout the session by a red stimulus light located above and to the left of the food trough, 8 cm above the floor and 2 cm from the back wall. A speaker (6 × 7 cm) was mounted in the top rear corner of the left wall and a clicker module (Med Associates, ENV-135M) was mounted in the top front corner of the left wall.

Procedure

Pretraining

All animals were trained during one daily session occurring at approximately the same time each day. Subjects were first trained to retrieve pellets from the food trough over three daily sessions during which twenty 45 mg food pellets (Dustless Precision Pellets, BioServ, Frenchtown, NJ) were delivered on a variable time (VT-45 s) schedule. Rats were subsequently trained to make a head poke into the trough during the presentation of a cue. Pilot studies indicated that the response rates of the rats increased more rapidly during training and remained at higher levels when a response requirement was imposed on the delivery of a reward. By increasing the overall response rate we were able to both decrease the number of first order conditioning (FOC) training sessions and minimize the delay from sensory preconditioning (SPC) to the test. A flashing light was used as the head poke training cue to decrease the possibility of generalization to the focal auditory cues. For Sessions 1 through 3 of training animals were exposed to a 12 s flashing white stimulus light cycling on and off at a rate of .3s followed by delivery of the food US consisting of two 45 mg pellets. In the next phase, the US was immediately delivered if the rat poked during the last 4 s of the flashing light. If no head poke occurred the food was delivered at the end of the cue. Once rats earned 50% of the pellets by head poking (typically within 2 sessions), the criterion changed such that a head poke during the last 4 s of the flashing light cue was required for reward; this training continued for 5 sessions. Each session during the head poke training phase consisted of 10 trials separated by a variable inter-trial interval (ITI) with a mean duration of 300 s. By the end of training (a total of 10 sessions) all rats were receiving pellets on at least 9 out of 10 trials.

Cue Training

Rats were randomly assigned to one of four training procedures, (n=10/group): Pre-A, Early-A, Late-A and Post-A (Figure 1), where A represents the cue tested for temporal information integration and the prefix represents the putative expected US delivery relative to the cue presentation. In the SPC phase, each group was given either forward paired, A→B (Post-A and Late-A) or backward paired, B→A (Pre-A and Early-A) cues. A was a 16 s 80 dB white noise, and B was a 12 s click train consisting of 5 clicks/s. Each session consisted of 4 trials of paired A and B presentations with trials separated by a variable ITI with a mean duration of 446 s. SPC training lasted for 2 sessions, totaling 8 trials.

The FOC phase began on the day following SPC training. All groups were trained with 2 auditory cues separated by a 4 s empty period. Two groups receive forward pairings (B → Food), and two groups received backward pairings (Food → B). Groups Early-A and Post-A received forward pairings of B→ Food followed by a 4 s delay when the US was available, then a 12 s presentation of a novel 12 s 80 dB tone (C). Groups Pre-A and Late-A received backward pairings of Food→ B by presenting C followed by 4 s delay then B. C was used to signal the animal about the impending reinforcement opportunities in the backward condition. In all conditions rats were reinforced only if they made a head poke into the food trough during the 4 second period between the two auditory cues. Each session consisted of 10 trials with a mean ITI of 305 s. FOC training continued for 15 sessions and all rats were consistently earning pellets on at least 9 trials per session by the end of FOC.

Beginning on the day after the completion of FOC, all groups were tested on each of the cues presented during SPC and FOC across three consecutive days with A presented on Day 1, B on Day 2 and C on Day 3. Each of these test sessions consisted of 10 presentations of one auditory cue but no food delivery. Cue presentations were separated by avariable ITI with a mean of 420 s.

Results

Head entries into the food trough were recorded during the sessions and accumulated in 1 s bins. Responding to the test cues was measured using elevation scores computed by subtracting the mean rate of head entries during 16 s of ITI preceding each cue presentation from the rate of head entries during cue presentation in each of the 1 s bins. Total elevation, the sum of the elevations scores from each second, was analyzed by one-way ANOVA with planned comparisons. Contrasts were selected to compare groups based on the predicted response during the presentation of A (Figure 1); groups Early-A and Late-A were expected to respond at a higher rate than Groups Pre-A and Post-A. Additional contrasts were used to compare responding between groups expected to respond more to A: Groups Early-A vs. Late-A and the groups expected to show little responding to A: Groups Pre-A vs. Post-A.

A One-way ANOVA of total responding to A showed a significant effect of Group [F(3,39)=3.238, p<. 05] (Figure 2). Planned contrasts indicated that there was significantly greater responding in the Early-A and Late-A groups, [t(36)= 3.082, p<.01] compared to Pre-A and Post-A groups, but no difference between Early-A vs. A Late [t(36)=0.123, NS] and no difference between Pre-A vs. Post-A [t(36)=0.123, NS].

Figure 2. Results of Experiment 1.

Figure 2

Open in a new tab

Total elevation scores represent the rate of head poking during the total cue presentation after subtracting the rate of head poking during the 16 s of ITI immediately preceding each trial. Bars represent the mean total elevation score for each group. Temporal integration of cues was evidence by the significantly higher response to A in groups with a temporal expectation of food during the A cue (Early-A and Late-A) as compared to groups without a temporal expectation of food during the A Cue (Pre-A and Post A).

Tests of B and C reveal that the groups did learn which cues predicted the availability of food in FOC. There was a significant effect of FOC training on responding to the B [F(3,39)=9.232, p<.05] and C [F(3,39)=13.525, p<.05], Table 1. Planned comparisons show more responding to B in groups Post-A and Early-A, the groups trained with forward pairings of B→ Food as compared to groups Pre-A and Late-A, who received backwards pairing of Food→ B [t(36)=5.023, p<. 01]. The inverse pattern of responding to C was also found [t(36)=6. 311, p<. 01]. There were no differences in responding to B in Post-A vs. Early-A Early and no difference in responding to C between Pre-A and Late-A groups.

Table 1.

Total Elevation Responding to First Order Cues

Total elevation scores represent the rate of head poking during the cue presentation after subtracting the rate of head poking during the 16 s of ITI immediately preceding each trial. Mean total elevation score for each group is presented followed by the standard error of the mean. First order conditioning is demonstrated by significantly higher responding to the B cue in Groups Early-A and Post-A (where B→Food→C) as compared to Pre-A and Late-A. The opposite pattern was observed when C→Food→B as demonstrated by higher rates of responding to the C cue in Groups Pre-A and Late-A.

Groups Cue B (SEM) Cue C (SEM)
Pre-A 0.69 (0.276) 3.21 (0.337)
Early-A 5.44 (1.200) 0.89 (0.196)
Late-A 2.27 (0.645) 3.70 (0.722)
Post-A 4.99 (0.723) 0.76 (0.312)
Open in a new tab

If the subjects were able to integrate quantitative information about the expected time of the US across phases then we expected that the Early-A and Late-A groups would show a different pattern of responding across the duration of the cue. The Early-A group should show responding early in the cue that decreases over time, while the Late-A group should show a ramping up of responding with maximal responding near the end of the cue. We tested this prediction by examining responding in each second of A and comparing the two groups with a two-way (Group X Time) repeated measures ANOVA. There was no significant difference between groups [F(1,18)=0. 012, NS], and no significant interaction [F(15,270)=0. 587, NS], however, there was a significant effect of time [F(15,270)=2. 272, p<. 05], data not shown. Both groups began responding with approximately the same latency and showed elevated responding for the remainder of the cue. We looked for further evidence of quantitative temporal encoding by examining responding in the ITI after the presentation of the cue. Groups Pre-A Post-A could have formed a temporal expectation of US delivery relative to A, with the expectation in Pre-A that the reinforcer would be delivered prior A and the expectation in Post-A that the US would be delivered after A (see Figure 1). Given the variable ITI, Pre-A would be unable to predict the delivery of the US, however Post-A could show an increase in responding after the offset of A. However there were no differences in responding in the ITI before or after cue presentation in any group. It appears that subjects in these groups were able to infer that food could be expected during A but there is no strong evidence that any knowledge of the specific interval at which the US had been presented was manifest in the test performance. Had the subjects used specific quantitative information to infer an expected time of the US subjects in Group Early-A and Late-A should have differed in the within-CS pattern of responding to A. Instead both groups responded more during A than the other groups but the within-CS response pattern did not differ.

Experiment 2

While the results of the previous experiment provide evidence of the integration of information, we did not find evidence of integration of the quantitative temporal relationship between the cues. Both Early-A and Late-A groups demonstrated similar high levels of responding throughout A. While this argues that the temporal order of A and B was preserved and integrated with the temporal location of the reward, it does not suggest that the quantitative temporal location of reward within A was used to make the inference. One possibility is that the 16s duration of A was not long enough to demonstrate the expected increase in responses during the cue in the Late-A group compared to high responding early in the cue expected in the Early-A group. In the next experiments we modified the procedure in an attempt to provide us with a greater opportunity to see the integratation of quantitative temporal information across experiences. Two auditory cues (A and B) were sequentially superimposed onto a longer third cue (C) during SPC, as shown in Figure 3. In different groups of subjects FOC consisted of US delivery early in C, late in C, or US delivery only during A or B. If a rat integrated quantitative information about when the reward was expected during C then rats trained to expect a reward early in C should respond more to A than to B. This pattern should be reversed in subjects trained to expect reward at the end of C. Similarly rats reinforced during A should show increased responding early in C which would taper off towards the end of the cue and rats reinforced during B should come to expect the food reward at the end of C, and thus show an increase in responding across the duration of C. These predictions were tested in the experiments below.

Figure 3. Procedure of Experiments 2 and 3.

Figure 3

Open in a new tab

Experiments 2 and 3 employed similar procedures, with SPC followed by FOC and test. The A and B cues were completely embedded in the C cue in Experiment 3, but had asynchronous onset and offset in Experiment 3. In each experiment the groups only differed with respect to their treatment during FOC as reflected in the group names.

Methods

Pretraining

This experiment was conducted in the same behavioral chambers as described above and magazine training was performed as described in Experiment 1. Following magazine training rats were trained to head poke for a reward during a flashing white stimulus light. A food reward was delivered when a head poke occurred, or 12s after cue onset if there was no head poke. The cue was terminated with reward delivery. Rats were trained for 2 sessions with 10 rewards per session. For the next 2 training sessions the reward was available during the flashing light cue on a variable interval schedule (VI) of VI-8 and VI-16 on successive days. Rewards were delivered at the end of the cue light if the rat had not made a head poke. All rats were consistently head poking by the end of training on the flashing light cue.

SPC and FOC

The design of this experiment is presented in Figure 3. All rats in this experiment (N=40, Charles River Breed Labs, Kingston, NY) received identical SPC training across 2 sessions with 4 trials per session. Each presentation of C consisted of a 32 s click train (5 clicks/s) with two additional cues superimposed on the longer cue. The shorter cues consisted of a 16 s 85 dB 1000 Hz tone, and a 16 s 85 dB white noise. A (tone or noise, counterbalanced) started synchronously with C. B (tone or noise, opposite to A) began immediately at the end of A, and ended synchronously with C. Following SPC, animals were randomly assigned to one of four FOC groups (n=10). Rats in Groups A-Plus and B-Plus received pellet deliveries during A or B, respectively. Rats received a pellet if a head poke into the food trough was made during seconds 6–10 of the 16s cue. Pellets were delivered 10 s into the cue presentation in the absence of a head poke. Rats in Groups C-Early and C-Late received pellets during C. Group C-Early subjects were rewarded for the first head poke during seconds 4–8 and Group C-Late subjects were rewarded for the first head poke during seconds 24–28 of the 32 s clicker. Pellets were delivered at the end of the availability period if no head poke was made. Training consisted of 15 sessions with 10 trials per session and each trial was separated by a variable ITI with a mean duration of 305s.

Test

The test phase consisted of two sessions across consecutive days. The first test session consisted of 10 trials of C (Groups A-Plus and B-Plus) or 10 randomly ordered presentations of A and B (Groups C-Early and C-Late). In the second test session all animals were tested on the cues that had not been presented during the first test. Test trials were separated by a variable ITI with a mean of 420 s.

Results

The number of head pokes into the food trough was recorded throughout the session and accumulated in 1 s bins. The ITI response rate was calculated across an interval equivalent to the duration of the test cue immediately preceding the cue presentation. Elevation scores were calculated by subtracting the mean ITI response rate from responding during each second of the cue. If animals can integrate quantitative temporal information then the pattern of responding during cues should be different in the different groups. Consequently, we compared responding during C between A Plus and B Plus in a Group x Time repeated measures ANOVA. There was a significant change in responding over time [F(31, 558)=20.518, p<.05]. However, contrary to predictions there was no Group x Time interaction and no difference between groups. Both groups showed their highest response rates soon after the onset of C and response rate declined after about 10 s of the test stimulus (Figure 4).

Figure 4. Experiment 2 Results.

Figure 4

Open in a new tab

Responding to Cue C during the test phase in groups A-Plus and B-Plus. Cue C elevation scores were calculated by subtracting the mean rate of responding during the 32 s of the ITI (ITI rate) preceding from the mean rate of responding during each second of cueC presentation. Based on the SPC training it was predicted that Group A-Plus would show increased responding earlier in the C cue than group B-Plus. However the identical pattern of responding within CS responding was observed in both groups suggesting that the pattern was likely based on generalization from FOC rather than based on temporal integration.

We predicted that the C-Early and C-Late would differ in their responding during A and B. To test this prediction we summed the elevation scores within each trial type, and compared the mean total elevation scores across groups. A Group x Cue repeated measures ANOVA showed significantly greater responding by C-Early [F(1,18) = 7. 051, p<. 05], but no significant difference in responding between A and B [F(1,18) = 2. 658, p=0. 12, NS] and no Group x Cue interaction [F(1,18)=0. 286, p<. 05, NS] (Figure 5).

Figure 5. Experiment 2 results.

Figure 5

Open in a new tab

Total elevation scores represent the total rate of head poking during the cue presentation after subtracting the rate of head poking during the 16 s of ITI immediately preceding each trial. Bars represent the mean total elevation for each group. There was a significant difference in responding to the A and B cues in Group C-Early as compared to Group C-Late. There were no significant differences between the A and B cues within each of the groups. It appears then that responding to both the A and B cues was mainly determined by the FOC treatments..

Discussion

Contrary to our predictions the subjects did not show integration of temporal information across the phases of this experiment. In groups where food was paired with the shorter cue (A-Plus and B-Plus) the pattern of responding during the test of C was identical and likely reflects generalization across cues. When examining the responding of groups trained to receive a reward in C (C-Early and C-Late) we expected to see an interaction in responding to A and B during the test phase, with higher responding to A in C-Early and higher responding to B in C-Late, and lower responding to the other short cue in both groups. The increased responding in the C-Early group can be explained by a difference in learning during FOC. Group C-Early learned to respond soon after the onset of the FOC cue, while C-Late learned to respond later in the cue. The overall greater responding by C-Early than C-Late reflects the level of responding that each group showed during the first 16s of C during FOC. Thus as in the A-Plus and B-Plus groups, responding by the C-Early and C-Late groups appears to reflect generalization from FOC.

Experiment 3

Though Experiment 1 showed that animals could integrate temporal order information, responding to the SPC cues in Experiment 2 appeared to be dominated by generalization from the FOC cues. One possible reason for the difference between experiments may be the differing relation between cue onsets and offsets. In Experiment 1 all cues were non-overlapping, each with unique times of onset and offset. However in Experiment 2 A and B were completely embedded in C; the time of onset is shared between A and C while offset times are shared by B and C. These common attributes could promote generalization between cues. Additionally, while rats may attend to the onsets of asynchronous cues, cues with shared onsets (A and C) or cues that are entirely embedded in other ones (B and C) may not be processed differently due to attentional competition. If this is the case then making A and B begin and end asynchronously with C should promote the integration of temporal maps. We tested this in the Experiment 3 by using asynchronous onset and offset of the cues superimposed on a longer duration light cue.

Methods

Pretraining

This experiment was conducted in the same behavioral chambers as described above, and rats (N=24) were magazine trained as described in Experiment 1. Following magazine training, all animals were trained to perform a head poke into the food during a click train. Rats were reinforced on a variable interval (VI) schedule and the click train terminated with pellet delivery. If the rat did not make a head poke, the cue continued for twice the VI duration and a reward was delivered 1 s prior to the termination of the click train. Rats were trained for one session on VI-8 s, one session of VI-16 s and 2 sessions of VI-32 s. By the end of training all rats were receiving pellets after head pokes on at least 90% of trials. Each session consisted of 10 trials separated by a variable ITI with a mean duration of 300 s.

SPC and FOC

All rats in this experiment received two identical SPC training sessions each consisting of 4 trials. Each trial consisted of the presentation of 3 cues. A and B were 10 s auditory cues (1000 Hz 85 dB tone or 85 dB white noise) superimposed on C, a 32 s flashing white stimulus light cycling on and off every .3s. Trials were separated by a variable ITI with a mean duration of 450 s. A (tone or white noise, counterbalanced) started 4 s prior to the onset of C, and terminated 10 s later. B began 26 s after the start of C and terminated after 10 s (4 s following the termination of C). This timing of both auditory cues resulted in a 6 s overlap with C, and 4 s of presentation without C.

Following sensory preconditioning, animals were randomly assigned to one of two first order conditioning groups. Rats in Group C-Early and C-Late were trained to expect a reinforcer if a head entry was made during seconds 4–8, or 24–28 of the 32 s flashing light cue, respectively, according to the procedures described in Experiment 2. FOC was conducted across 8 daily sessions with 15 trials per session, and variable ITI with a mean of 180 s. On the day following the final FOC session all rats were given an extinction test consisting of 5 trials each of A and B presented in random order. Test trials were separated by a variable ITI with a mean of 420 s.

Results

The number of head pokes into the food trough was measured throughout the session and accumulated in 1s bins. Elevation scores were calculated by subtracting the mean responding during an interval equal to the test cue duration immediately prior to the cue presentation from the mean responding during the cue. A 2×2 repeated measures ANOVA was used to compare responding to A and B. There was a significant Cue x FOC interaction [F(1,22) = 8.135, p<. 05], but no main effect of Cue [F(1,22) = 0.110, NS] or FOC [F(1,22)=0.050, NS]. One-tailed, post hoc comparisons indicate significantly greater responding to A in the C-Early group [t(22)=1. 74, p<. 05;] and greater responding to B in the C-Late group [t(22)=1. 91, p<. 05; one-tailed] as predicted (Figure 6).

Figure 6. Experiment 3 Results.

Figure 6

Open in a new tab

Total elevation scores represent the total rate of head poking during the cue presentation after subtracting the rate of head poking during the 16 s of ITI immediately preceding each trial. Bars represent the mean total elevation for each group. When rats were trained to expect food early in the C cue they responded significantly more to the A cue, while groups trained to expect food later in the C cue responded significantly more to the B cue. This pattern of results suggests that when the cue onsets and offsets are asynchronous the subjects could integrate temporal information across the two phases of the experiment.

Discussion

Previous research in aversive conditioning suggests that animals are capable of forming temporal maps and integrating information from different experiences to update those maps (Balsam, Drew, and Gallistel 2010; Molet et al. 2012). The current studies along with previous work (Leising, Sawa, and Blaisdell 2007) represent demonstrations that temporal maps established from independent experiences with appetitive conditioning can also be integrated when those experiences involve common cues. An alternative account of many experiments, including the current ones, is based on the idea of representation mediated conditioning (Holland 2008; Holland 1981). For example, in Experiment 3, this account would posit that A and B become associated with the early and late parts of C, respectively. In FOC the representation of A is reactivated by the early part of C. This representation would be directly paired with the US in group C-Early. Similarly, the late part of C would reactivate the representation of B which would then be directly paired with the US in group C-Late. While the current experiments cannot rule out this explanation, recent experiments (Balsam, Drew, and Gallistel 2010; Molet et al. 2012) demonstrate that in experiments like the ones conducted here the integration of information across experiences does not take place during FOC, rather it occurs during the test session when only the test cue is present.

In the previous aversive conditioning experiments with rats (see (Molet and Miller 2013), subjects clearly integrated the order in which cues occurred but these experiments were not designed or suited to demonstrate the integration of quantitative temporal information across experiences. Fear responses are often long lasting and thus not well suited for pinpointing temporal expectations even when those expectations are shown to be quite specific in transfer tasks (Amundson and Miller 2008; Denniston, Blaisdell, and Miller 2004; Diaz-Mataix et al. 2013; Blaisdell, Denniston, and Miller 1998). Temporal expectations in appetitive conditioning on the other hand, are fairly precisely manifest in anticipatory behavior (Balsam, Drew, and Gallistel 2010; Molet, Jozefowiez, and Miller 2010). Thus we hoped to show the integration of precise temporal information across different experiences. The sensory pre-conditioning methods however, also appear to have pitfalls for revealing integration of temporal information.

In appetitive experiments it appears that the dominant influence on the pattern of responding evoked by the SPC cues is generalization of the pattern of responding established during FOC. Thus any impact of the inferred temporal expectations established during SPC are modulations of a response pattern established in FOC. In Experiment 1, subjects whose inferred expectations occurred during the test cue responded more during the test cue than subjects whose inferred expectations were for reward delivery before or after the test cue. However, the pattern of responding in all groups was similar to the within-cue pattern established in FOC. In Experiment 2, the pattern in all groups was determined completely by generalization from FOC. As in Experiment 1, Experiment 3 demonstrated a modulation of the FOC pattern by the inferred expectation. Subjects in C-Early responded more to A than B and the opposite pattern was found in C-Late. This implies that subjects were able to quantitatively discriminate the early and late portions of C and use that temporal knowledge to infer whether reward could be expected in A or B. However, the general pattern of responding within the A and B cues was similar. These results are much like those reported by Leising et al. (2007). Their experiments employed methods very similar to those we used in Experiments 2 and 3 and the pattern of responding in their test cues was largely determined by the pattern of responding established during FOC. Like us, they found that an inferred expectation early or late in the test cue produced a slightly higher response rate at the appropriate temporal portion of the test cue. Thus the dominance of the FOC pattern of responding makes it difficult to see the use of precise temporal information established during the SPC phase of these experiments.

Though the Leising et al. (2007) experiments and one experiment with human subjects (Molet, Jozefowiez, and Miller 2010) show somewhat more precise temporal modulation than those reported here, the most cautious conclusion that we can draw from all the experiments is that subjects can integrate ordinal temporal information from different experiences. Our Experiment 1 shows that subjects could infer that a reward could occur during a pretrained cue as compared to either before or after the cue. Our Experiment 3, like Leising et al. (2007), shows that information about whether a reward occurred early or late in a cue can be integrated across experiences suggesting that quantitative temporal information can be integrated in a manner similar to that seen in spatial maps. Future studies will have to develop new methods that allow for the expression of very precise temporal knowledge acquired through the integration of multiple experiences. Perhaps, transfer tests that reinforce the animal for responding at a time that could be inferred to be appropriate based on a number of past experiences would provide a more sensitive measure of the capacity to integrate temporal maps.

Highlights.

  • Rats can integrate temporal order information across learning experiences.

  • Rats may integrate quantitative temporal information across learning experiences.

  • Generalization of learning appears to dominate responding across experiences.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Amundson JC, Miller RR. CS-US temporal relations in blocking. Learning & behavior. 2008;36:92–103. doi: 10.3758/lb.36.2.92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Arcediano F, Escobar M, Miller RR. Temporal integration and temporal backward associations in human and nonhuman subjects. Animal Learning & Behavior. 2003;31:242–256. doi: 10.3758/bf03195986. [DOI] [PubMed] [Google Scholar]
  3. Arcediano F, Miller RR. Some constraints for models of timing: A temporal coding hypothesis perspective. Learning and Motivation. 2002;33:105–123. [Google Scholar]
  4. Balsam PD, Drew MR, Gallistel CR. Time and Associative Learning. Comp Cogn Behav Rev. 2010;5:1–22. doi: 10.3819/ccbr.2010.50001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Balsam PD, Drew MR, Yang C. Timing at the start of associative learning. Learning and Motivation. 2002;33:141–155. [Google Scholar]
  6. Balsam PD, Gallistel CR. Temporal maps and informativeness in associative learning. Trends Neurosci. 2009;32:73–8. doi: 10.1016/j.tins.2008.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bitterman M. Phyletic differences in learning. The American psychologist. 1965;20:396. doi: 10.1037/h0022328. [DOI] [PubMed] [Google Scholar]
  8. Blaisdell AP, Cook RG. Integration of spatial maps in pigeons. Animal Cognition. 2005;8:7–16. doi: 10.1007/s10071-004-0223-1. [DOI] [PubMed] [Google Scholar]
  9. Blaisdell AP, Denniston JC, Miller RR. Temporal encoding as a determinant of overshadowing. J Exp Psychol Anim Behav Process. 1998;24:72–83. doi: 10.1037//0097-7403.24.1.72. [DOI] [PubMed] [Google Scholar]
  10. Brandon S, Vogel E, Wagner A. Stimulus representation in SOP:I. Theoretical rationalization and some implications. Behavioural Processes. 2003;62:5–25. doi: 10.1016/s0376-6357(03)00016-0. [DOI] [PubMed] [Google Scholar]
  11. Buhusi CV, Meck WH. What makes us tick? Functional and neural mechanisms of interval timing. Nature Reviews Neuroscience. 2005;6:755–765. doi: 10.1038/nrn1764. [DOI] [PubMed] [Google Scholar]
  12. Cole RP, Barnet RC, Miller RR. Temporal encoding in trace conditioning. Animal Learning & Behavior. 1995;23:144–153. [Google Scholar]
  13. Collett M, Harland D, Collett TS. The use of landmarks and panoramic context in the performance of local vectors by navigating honeybees. Journal of Experimental Biology. 2002;205:807–814. doi: 10.1242/jeb.205.6.807. [DOI] [PubMed] [Google Scholar]
  14. Denniston JC, Blaisdell AP, Miller RR. Temporal coding in conditioned inhibition: Analysis of associative structure of inhibition. J Exp Psychol Anim Behav Process. 2004;30:190–202. doi: 10.1037/0097-7403.30.3.190. [DOI] [PubMed] [Google Scholar]
  15. Denniston JC, Miller RR. Timing of omitted events: an analysis of temporal control of inhibitory behavior. Behav Processes. 2007;74:274–85. doi: 10.1016/j.beproc.2006.11.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Diaz-Mataix L, Ruiz Martinez RC, Schafe GE, Ledoux JE, Doyere V. Detection of a temporal error triggers reconsolidation of amygdala-dependent memories. Curr Biol. 2013;23:467–72. doi: 10.1016/j.cub.2013.01.053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Drew MR, Zupan B, Cooke A, Couvillon PA, Balsam PD. Temporal control of conditioned responding in goldfish. J Exp Psychol Anim Behav Process. 2005;31:31–9. doi: 10.1037/0097-7403.31.1.31. [DOI] [PubMed] [Google Scholar]
  18. Eichenbaum H. Memory on time. Trends Cogn Sci. 2013;17:81–8. doi: 10.1016/j.tics.2012.12.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Foo P, Duchon A, Warren WH, Jr, Tarr MJ. Humans do not switch between path knowledge and landmarks when learning a new environment. Psychological research. 2007;71:240–251. doi: 10.1007/s00426-006-0080-4. [DOI] [PubMed] [Google Scholar]
  20. Foo P, Warren WH, Duchon A, Tarr MJ. Do humans integrate routes into a cognitive map? Map-versus landmark-based navigation of novel shortcuts. Journal of Experimental Psychology-Learning Memory and Cognition. 2005;31:195–214. doi: 10.1037/0278-7393.31.2.195. [DOI] [PubMed] [Google Scholar]
  21. Gallistel CR. Representations in animal cognition: An introduction. Cognition. 1990;37:1–22. doi: 10.1016/0010-0277(90)90016-d. [DOI] [PubMed] [Google Scholar]
  22. Gallistel CR, Gibbon J. Time, rate, and conditioning. Psychol Rev. 2000;107:289–344. doi: 10.1037/0033-295x.107.2.289. [DOI] [PubMed] [Google Scholar]
  23. Gallistel CR, King AP. Memory and the computational brain: Why cognitive science will transform neuroscience. Wiley-Blackwell; 2009. [Google Scholar]
  24. Gibbon J, Balsam P. Spreading association in time. Autoshaping and conditioning theory. 1981:219–253. [Google Scholar]
  25. Holland PC. CS-US interval as a determinant of the form of Pavlovian appetitive conditioned responses. J Exp Psychol Anim Behav Process. 1980;6:155–74. [PubMed] [Google Scholar]
  26. Holland PC. Acquisition of Representation-Mediated Conditioned Food Aversions. Learning and Motivation. 1981;12:1–18. doi: 10.1016/j.lmot.2008.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Holland PC. A comparison of two methods of assessing representation-mediated food aversions based on shock or illness. Learn Motiv. 2008;39:265–277. doi: 10.1016/j.lmot.2008.08.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Honig WK, Urcuioli PJ. The legacy of Guttman and Kalish (1956): Twenty-five years of research on stimulus generalization. J Exp Anal Behav. 1981;36:405–45. doi: 10.1901/jeab.1981.36-405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Ivry RB, Schlerf JE. Dedicated and intrinsic models of time perception. Trends in cognitive sciences. 2008;12:273–280. doi: 10.1016/j.tics.2008.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kirkpatrick K, Church RM. Tracking of the expected time to reinforcement in temporal conditioning procedures. Learn Behav. 2003;31:3–21. doi: 10.3758/bf03195967. [DOI] [PubMed] [Google Scholar]
  31. Leising KJ, Sawa K, Blaisdell AP. Temporal integration in Pavlovian appetitive conditioning in rats. Animal Learning & Behavior. 2007;35:11–18. doi: 10.3758/bf03196069. [DOI] [PubMed] [Google Scholar]
  32. MacDonald CJ, Lepage KQ, Eden UT, Eichenbaum H. Hippocampal “time cells” bridge the gap in memory for discontiguous events. Neuron. 2011;71:737–49. doi: 10.1016/j.neuron.2011.07.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Molet M, Jozefowiez J, Miller RR. Integration of spatial relationships and temporal relationships in humans. Learn Behav. 2010;38:27–34. doi: 10.3758/LB.38.1.27. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Molet M, Miguez G, Cham HX, Miller RR. When does integration of independently acquired temporal relationships take place? J Exp Psychol Anim Behav Process. 2012;38:369–80. doi: 10.1037/a0029379. [DOI] [PubMed] [Google Scholar]
  35. Molet M, Miller RR. Timing: An Attribute of Associative Learning. Behavioural Processes. 2013;95:xx. doi: 10.1016/j.beproc.2013.05.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Molet M, Miller RR. Timing: An Attribute of Associative Learning. Behav Processes. 2013;95:xx. doi: 10.1016/j.beproc.2013.05.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. O’Keefe J, Nadel L. The hippocampus as a cognitive map. Clarendon Press; Oxford: 1978. [Google Scholar]
  38. Shapiro ML, Eichenbaum H. Hippocampus as a memory map: synaptic plasticity and memory encoding by hippocampal neurons. Hippocampus. 1999;9:365–384. doi: 10.1002/(SICI)1098-1063(1999)9:4<365::AID-HIPO4>3.0.CO;2-T. [DOI] [PubMed] [Google Scholar]
  39. Shapiro ML, Tanila H, Eichenbaum H. Cues that hippocampal place cells encode: dynamic and hierarchical representation of local and distal stimuli. Hippocampus. 1997;7:624–642. doi: 10.1002/(SICI)1098-1063(1997)7:6<624::AID-HIPO5>3.0.CO;2-E. [DOI] [PubMed] [Google Scholar]
  40. Silva KM, Timberlake W. A behavior systems view of conditioned states during long and short CS-US intervals. Learning and Motivation. 1997;28:465–490. [Google Scholar]
  41. Smith MC. CS-US interval and US intensity in classical conditioning of the rabbit’s nictitating membrane response. J Comp Physiol Psychol. 1968;66:679–87. doi: 10.1037/h0026550. [DOI] [PubMed] [Google Scholar]
  42. Tolman EC. Cognitive maps in rats and men. Psychological Review. 1948;55:189–208. doi: 10.1037/h0061626. [DOI] [PubMed] [Google Scholar]
  43. Vogel EH, Brandon SE, Wagner AR. Stimulus representation in SOP: II. An application to inhibition of delay. Behav Processes. 2003;62:27–48. doi: 10.1016/s0376-6357(03)00050-0. [DOI] [PubMed] [Google Scholar]

RESOURCES