The influence of multiple temporal memories in the peak-interval procedure

Abstract

Memories for when an event has occurred are used to anticipate future occurrences of the event, but what happens when the event is equally likely to occur at two different times? In this study, one group of rats was always reinforced at 21 sec on the peak interval procedure (21-only group), whereas another group of rats was reinforced at either 8 or 21 sec, varying daily (8–21 group). At the beginning of each session, the behavior of the 8–21 group largely lacked temporal control, but by the end of the session, temporal control was re-established. When both groups were reinforced at 21 seconds, the pattern of responding was indistinguishable after subjects in the 8–21 group experienced 13 reinforcement trials. Lastly, the reinforcement time of previous sessions affected the 8–21 group, such that subjects were biased depending on the reinforcement time of the prior session. These results show that when the reinforcement time is initially ambiguous, rats respond in a way that combines their expectation of both possibilities, incrementally adjust their responding as they receive more information, but still use information from prior sessions to bias their initial expectation for the reinforcement time. Combined these results imply that rats are sensitive to the age of encoded temporal memories in an environment where the reinforcement time is variable. How these results inform the scalar expectancy theory, the currently accepted model of interval timing behavior, is discussed.

Introduction

To track the passage of time, organisms employ at least three physiologically distinctive mechanisms that are each separately affected by several physiological (Wearden, Philpott, & Win, 1999) and pharmacological (Meck, 1996) factors. Which mechanism is used to complete a task is dependent on the length of time that must be monitored. Two of these timing mechanisms, millisecond timing (employed when sub-second durations must be monitored) and circadian timing (employed when durations close to 24 hours must be monitored), have high precision but are limited to a small range of durations in the durations that may be accurately timed and may be slow to re-synchronize to a change in environmental conditions. In contrast, interval timing is used to track a wide range of durations lasting seconds or minutes and this mechanism can be rapidly adjusted to respond to changing environmental conditions, but as a consequence, this mechanisms is highly variable (Buhusi & Meck, 2005). Despite the flexibility of interval timing processes, the predominant theory of interval timing, the scalar expectancy theory (Gibbon, 1977; Gibbon, Church, & Meck, 1984), focuses on well-learned or steady-state performance. This focus may be due to the fact that investigating temporally-governed behavior (especially in non-human animals) requires one to first capture evidence of temporal control. Temporal control is defined here as behavior that has been shaped to the reinforcement time of the current session. Experimentally inducing a loss of temporal control after a subject has learned a temporal relationship would also cause the experimenter to lose, temporarily, access to the very behavior they are attempting to study.

The scalar expectancy theory posits that temporally governed behavior is the result of three interconnected but separate components, the clock, memory, and comparison components (Gibbon, 1977; Gibbon et al., 1984). The clock component marks the currently elapsing passage of time by semi-randomly emitting pulses. The accumulator, also part of the clock component, tracks the number of pulses that occur between the initiation of a stimulus and the delivery of a reward. The memory component stores, in reference memory, the total number of pulses registered in the accumulator at the time of reward. Finally, the comparison component is responsible for a decision process that assesses the relative difference between an accumulating number of pulses and a value sampled from memory; when the relative difference between these two values passes below a threshold, the animal begins to respond, but then ceases to respond once the absolute value of this difference exceeds the threshold again.

The scalar expectancy theory is often used to explain the behavior of individual rodents performing the peak interval procedure. The peak interval procedure engages rodent subjects in two trial types that occur randomly throughout a session: 1) fixed interval (FI) trials where subjects can depress a response lever after a specific delay has elapsed to earn a reinforcer, and 2) peak interval (PI) trials where reinforcement is withheld resulting in the subject increasing their response rate around the expected time of reinforcement, but then decreasing their response rate when the elapsed interval of time sufficiently exceeds this expectation. While the scalar expectancy theory models the cognitive processes that subjects utilize on individual trials of the peak interval procedure, performance on a single trial is highly variable and does not necessarily reflect temporal control (Gibbon & Church, 1990; Gibbon et al., 1984). As a result, measures of a subject’s expectation of the reinforcement time are often extracted from the average of all PI trials that occur in a session. The high variability on individual PI trials stems from two potential sources of noise in the model of scalar expectancy theory: 1) pulses emitted by the pacemaker are not always registered by the accumulator (Gibbon et al., 1984) and 2) the temporal memory used by the memory component is randomly selected from a pool of all stored memories of the task (Gibbon & Church, 1990).

This second source of variability implies that a recently formed memory of the reinforcement time exerts no greater influence on expectation than memories formed many sessions ago. On the contrary, memories from a current session should have, based on probability, less influence on current behavior than the much larger pool of memories from numerous prior sessions. Discerning the influence of newer memories (from the current session) versus older memories (from prior sessions) on temporally-governed behavior can be investigated experimentally by changing the delay to reinforcement, as these two sources of information (i.e., older and newer memories) would then predict different temporal performance. However, such a manipulation would induce a loss of temporal control, and as mentioned above, scalar expectancy theory has yet to be extended to explain such behavior.

Some prior studies have attempted to investigate instances where temporal control is disrupted due to the time of reinforcement being changed. These prior studies have shown that, in these semi-variable environments (e.g., environments where the time of reinforcement is stable for a period of time, but then changes to a new stable reinforcement time), subjects abruptly shift from using older memories (of the previous reinforcement duration) to newer memories (of the new reinforcement duration) to guide temporally-governed behavior.

In Simen et al. 2011, human participants engaged in a time-based “beat the clock” task where a square was displayed on a computer screen for a set amount of time during individual trials and random numbers overlaid to dissuade participants from counting. During individual trials, participants earned larger amounts of money (max $ 0.25 per trial) the closer their first response was to the time when the square disappeared, but a response that occurred after the square disappeared was not rewarded. The time the square was displayed (mean = 8 seconds) was constant for only a few trials, and un-signaled changes in the interval (>50% of the current duration) occurred rapidly over the course of the session. The average correlation between the participant’s response and the actual deadline exceeded an R² of 0.9 after only a single trial since an interval change had occurred, meaning that human participants could rapidly adjust their responding as soon as a discrepancy was detected: this could not have been done had all prior memories had an equal chance of influencing the subject’s expectation.

Similarly, in three rodent studies using the peak interval procedure, subjects changed their temporal behavior within a session when (1) they were re-trained with a new duration (Meck, Komeily-Zadeh, & Church, 1984), or (2) they experienced a sequence of changing reinforcement times over the course of several sessions (Lejeune, Ferrara, Simons, & Wearden, 1997); this abrupt change in responding makes it unlikely that rodent subjects were equally likely to sample from older versus newly formed (i.e., within session) memories. However, the transition from a shorter to a longer duration produced a different pattern of responding relative to a transition from a longer to a shorter duration (Higa, 1997). This may be due to the fact that on long sessions, subjects’ experienced the absence of reinforcement at the short time prior to being reinforced at the long time, whereas on short sessions, reinforcement trials ended before the longer reinforcement durations are experienced. In either case, if recently formed memories were equally likely to be sampled as memories formed earlier, then the transitioning behavior of these rats in all three of the above mentioned studies should have been gradual.

The above studies suggest that subjects can regain temporal control quickly after detecting a change in the reinforcement time when in a variable environment. Humans were able to use a single presentation of a duration to guide their behavior while ignoring all preceding trials, while rodent subjects are capable of transitioning to a new reinforcement time relatively quickly as well. However a number of questions still remain about how, specifically, subjects reestablish temporal control in a variable environment. First, what is the least amount of information needed before a subject will re-establish temporal control; for example, would it be possible to re-establish temporal control after experiencing a single reinforcement trial? Secondly, if the current reinforcement time is shorter than the subject’s initial expectation, would temporal control be re-established faster compared to when the reinforcement time is longer than the initial expectation? Finally, would subjects always use prior experiences to guide behavior? In other words, if the reinforcement time of a current session was selected at random, would the reinforcement time of the preceding session still affect behavior? Answering these questions would not only inform current understanding of an under-researched topic in the interval timing literature, but may also provide avenues in which scalar expectancy theory could be updated to better explain how temporal control is re-established in a variable environment; given that interval timing has been shown to be more sensitive to environmental change than other timing processes (Buhusi & Meck, 2005), such an addition would improve the external validity of this model.

The current study investigated these questions using a group of rats (the 8–21 group) trained to expect food availability at either one of two durations (i.e., either 8 or 21 seconds after the start of a trial), and a second group (the 21-only group) that could only earn food at a single duration (i.e., 21 seconds after the start of a trial). For the subjects in the 8–21 group, the duration used was randomly selected at the start of each daily session, but did not change over the course of the session. In this paradigm, the subjects in the 8–21 group start the session in an initially ambiguous state (i.e., they cannot fully predict the reinforcement time), as characterized by a lack of temporal control, but develop accurate temporal control after experiencing several reinforcement trials during the session.

Methods

Subjects

Sprague-Dawley rats were restricted to 15 g of food per day throughout the experiment, but had ad libitum access to water. A 12:12 reverse light-dark cycle was used. Sessions occurred in the dark phase of the cycle with lights turning off at 7 am and turning on at 7 pm. Twenty rats were separated into two groups with the rats being tested in two shifts (counterbalanced across subjects). Shortly after this separation, however, one rodent from the 8–21 group had to be euthanized due to health problems. The first shift was started at approximately 9:00 am and the second at approximately 11:00 am. Rats were trained 5 days per week. All procedures were approved by the Institutional Animal Care and Use Committee at the University of Georgia and followed the guidelines of the National Research Council Guide for the Care and Use of Laboratory Animals.

Materials

Ten stainless steel operant chambers (30.5 cm × 24.1 cm × 29.2 cm; Med Associates) each encased in a sound attenuating cabinet (66 cm × 35.6 cm × 55.9 cm) were used. A feeder dispensed 45 mg dustless grain pellets (Bio-serv F0165) into a food trough and a photobeam tracked when the rat’s head entered the trough. A retractable lever, that the rat could depress when extended, was also in the chamber, along with a small hole that allowed a sipper tube from a water bottle to be inserted into the chamber; subjects had free access to water at all times. The experimental procedure was programmed using the MED-PC IV (St. Albans, VT USA) software package on two computers, both of which were housed in a separate room.

Procedure

Rats received food magazine directed training for one day followed by two days of training where food could be earned by depressing a response lever located to the left of the food magazine on a fixed-ratio 1 schedule. Rats were then trained on the peak interval procedure. At the onset of a trial, the lever was extended and a white noise was presented until the subject either earned a pellet or the trial ended. In this procedure a subject received an equal number (i.e., 50 %) of randomly selected trial types, fixed interval trials and peak interval trials. During fixed interval (FI) trials, the subject had the opportunity to respond and earn one pellet of food after the white noise stimulus was on for a specific amount of time. The first response after the interval elapsed produced the pellet, terminated the signal, and retracted the lever. During peak interval (PI) trials, reinforcement was omitted, and the white noise remained on for a randomly selected time that was between 4 and 5 times longer than the FI schedule used in the current session (8 second sessions = between 32 and 40 seconds; 21 second sessions = between 84 and 105 seconds) at which point the lever was retracted and the noise ended. After either trial type, an inter-trial interval (in the absence of noise) occurred for a randomly selected interval between 60 and 100 seconds.

For the subjects in the 21-only group, the reinforcement duration of FI trials was always 21 seconds. In contrast, the 8–21 group subjects received two different reinforcement durations. On randomly selected days the FI was either 8 or 21 seconds, and the reinforcement duration was held constant within a session. For randomization, a list containing an equal number of session types was created and a random number assigned to each item, the list of random numbers (and corresponding items) was then sorted in ascending order. To maintain equivalent training, the subjects in the 21-only group either completed a session where they were reinforced at 21 seconds or remained in their home cage and were not run. The same randomization procedure as described above was used to select the reinforcement time the subjects in the 8–21 group experienced within a session, thereby equating the number of sessions where reinforcement occurred at 21 seconds was equated across groups. Two hours after the session ended a supplement of food was given to the animals; the supplement plus food received during the daily session equaled 15 grams of food.

Data Analysis

Data from sessions 31 to 130 were pooled to create the final data set. The PI trials for a session were fit with a Gaussian-linear model that was initially developed by Buhusi and Meck (2000), with the parameters of the model estimated using the error correcting Marquardt-Levenburg algorithm (Marquardt, 1963). Measures of peak time and coefficient of variation were derived from the Gaussian-linear model using Matlab (Mathworks, Natick, MA).

On individual trials, a rat’s lever pressing behavior conforms to a break-run-break or square-wave pattern (Gibbon & Church, 1990). The transition between the first break and the run is termed the start time (denoting the start of a high rate of responding) and the transition between the end of the run and the second break is termed the stop time. The start and stop times from individual PI trials were assessed using the methods described in Church, Meck, and Gibbon (1994) where all possible start and end times were considered and the combination that minimized the error between observed and expected rates was selected.

The data were grouped by the number of preceding FI trials that had been experienced within the session. Put another way, all PI trials that occurred before any FI trials were grouped together and called “0FI” trials, whereas all PI trials that occurred after a single FI trial had been reinforced were grouped together and called “1FI” trials. Two separate analyses were completed on this generated data set. First the PI data was further split according to whether the prior session used the same or a different reinforcement time to capture whether subjects biased their responding based on the reinforcement time of the prior session. Secondly, an inflection point analysis was conducted on the start and stop times of all sessions for the 8–21 subjects to quantify the point at which subjects transitioned from being initially uncertain about the reinforcement time at the beginning of the session to expressing stable temporal control. In this inflection point analysis, one line was fit to the data points from those PI trials that occurred prior to reinforcement (i.e., “0FI” trials) to those that occurred before a specified number of FI trials had occurred: slope and intercept were free parameters. A second model was then fit as a flat line (i.e., slope = 0), with an intercept equal to the average of all remaining data points ranging from the same number of FIs used in the first model to the last data point. This two model analysis was then iteratively completed on all possible FIs experienced and the residual sum of squares (RSS) for each model calculated; the optimal inflection point (our operational definition of the number of FIs experienced when subjects first expressed stable temporal control) was then taken from the model-set with the lowest RSS value. All trials that occurred before temporal control was re-established were removed, and this subset of data was then submitted to the same Gaussian-linear model fit as describe above to determine peak time and the coefficient of variation.

Results

Peak interval response distributions

The 8–21 subjects and the 21-only subjects had equivalent experience on the 21 second duration; the only difference between these two groups was that one group had also experienced sessions where reinforcement could be earned by responding 8 seconds after the onset of the same signal. The presence of this additional memory caused the distribution of responses of the 8–21 subjects to be earlier and broader than that of the 21-only subjects (Figure 1A) on 21 second sessions. During these sessions the mean peak time of the 8–21 subjects was 17.62 ± 1.09 (mean ± s.e.m.) seconds and the peak time for the 21-only subjects was 19.65 ± 1.32 seconds; the coefficient of variation of the 8–21 subjects was 0.62 ± 0.05, while the coefficient of variation for the 21-only subjects was 0.49 ± 0.06. Between subjects t-tests revealed that significant differences between these two groups were observed in both the peak time (t(17) = −3.57, p < 0.05) and the coefficient of variation (t(17) = 4.95, p< 0.05; see Figure 1B). Further, when the 8–21 rats’ peak functions from 8 second and 21 second sessions were compared, within-subjects t-tests revealed that the coefficient of variation (i.e., the relative breadth of the response distribution after being normalizing by the peak time) from the short (8 second) sessions (0.53 ±0.06) was significantly narrower than that of the long (21 second) sessions (0.62 ± 0.05; t(8) = −5.06, p<0.05; see Figure 1B). Finally, the peak time of the 8–21 subjects on short sessions (8.51 ±0.87) was significantly earlier than the peak time obtained when these subjects experienced long sessions (17.62 ± 1.09; t(8) = −37.50, p<0.05; see Figure 1A) demonstrating that 8–21 subjects did indeed respond sooner on short sessions compared to long sessions.

Figure 1.

Figure 1A shows the normalized response rate (y-axis) for the entire data set of the 8–21 subjects and 21-only subjects as a function of trial duration (x-axis), while (B) shows this same data with relative time (actual time/the averaged peak time) on the x-axis. A two-model fit was used to find the the number of reinforcement trials that had to be experienced before temporal control was re-established by the subjects in the 8–21 group on 8 and 21 second sessions. Peak trials that occurred before these points were excluded so that only data (normalized response rate) from trials occurring after these timepoints were compared to the 21-only group as a function of (Figure 1C) real time (x-axis) and (Figure 1D) relative time (x-axis).

Individual Trials Analysis: 8–21 group vs. 21-only group

As shown in Figure 2A (bottom lines), start times differed between the two groups in the beginning of a session, but converged after several trials. These visual impressions were confirmed by a 2 × 21 mixed effects ANOVA that considered both group and experience (i.e., the number of preceding FI trials), respectively. The ANOVA revealed a significant main effect of experience (F(20,340) = 11.19, p < 0.05), no effect of group (F(1,17) = 0.74, p = 0.40), but an experience by group interaction (F(20,340) = 2.43, p < 0.05). Probing the interaction with a between subjects post-hoc t-tests performed on each experience level (i.e., 0FI, 1FI, etc.) revealed that only the 0FI and 1 FI experience levels were significantly different, meaning that start times for the two groups converged after one reinforcement trials had been experienced. For stop times (Figure 2A, top lines), the main effect of experience (F(20,340)= 1.97, p < 0.05) and the interaction of group x experience (F(20,340)= 7.77, p < 0.05) were significant whereas the main effect of group (F(1,17) = 0.30, p = 0.59) was non-significant. Post-hoc, between-subjects t-test performed on this interaction revealed that the stop times in the two groups converged after two reinforcement trials. The effect seen in the stop times followed the same general pattern as the start times; the pattern of responding in these two groups converged after experiencing only one or two reinforcement trials, as measured by start or stop times, respectively.

Figure 2.

The lever pressing behavior of the 8–21 subjects was different from the 21-only subjects early in the sessions; after experiencing several FI trials the behavior of these two groups overlapped; error bars represent the standard error of the mean. Figure 2B: the lever pressing behavior of the 8–21 subjects when reinforced at 8 seconds and 21 seconds was indistinguishable early in the session and diverged after experiencing a few FI trials.

Individual Trials Analysis: 8–21 group (8 vs. 21 second sessions)

The start and stop times for the 8–21 group on 8 and 21 second sessions are plotted as a function of experience in a session in Figure 2B. As can be seen, early in a session, subjects began (Figure 2B, bottom lines) and terminated (Figure 2B, top lines) responding at roughly the same time. However, as they experienced feedback about the current session’s reinforced duration, their start and stop times progressively diverged. To quantify the 8–21 group’s responding on 8 and 21 second sessions, start time and stop time were analyzed separately, and 2 × 21 within-subjects ANOVAs that considered both reinforcement duration (either 8 or 21 seconds) and experience were conducted. For start times, the main effects of duration (F(1,8) = 108.68, p < 0.05) and experience (F(20,160) = 9.37, p < 0.05) as well as the interaction of duration x experience (F(1,8) = 9.58, p < 0.05) were significant. Probing the interaction with a post-hoc, within-subjects t-test on all experience levels (i.e., 0FI, 1FI, etc.) revealed that the start times diverged after a single FI trial. Likewise for stop times, the main effects of duration (F(1,8) = 1132.02, p < 0.05), experience (F(20,160)= 4.53, p < 0.05) and their interaction (F(20,160) = 31.57, p < 0.05) were significant (Figure 2B, top lines). Post-hoc, within-subjects t-tests performed on all experience levels revealed that the stop times also diverged after a single FI trial. Similar to the individual trials analysis comparing the 21-only and 8–21 group, the behavior of the 8–21 subjects diverged after a small number of reinforcement trials, in this case one reinforcement trial, as measured by start or stop times.

Effect of the previous session on performance in the 8–21 subjects

The data from the 8–21 subjects was sorted according to whether the current session’s reinforcement time was the same or different from the previous session. When reinforced at 8 seconds on the current session (see Figure 3A), a 2 (same vs. different reinforcement time in the prior session) x 21 (experience) within-subjects ANOVA on start times yielded a significant main effect of same vs. different (F(1,8) = 6.74, p < 0.05) with responding beginning later if the previous session was reinforced at 21s, a non-significant main effect of experience (F(20,160) = 1.08, p = 0.37), and a non-significant interaction (same vs. different x FI experience (F(20,160) = 1.37, p = 0.14). However, a 2 (same vs. different) x 21 (experience) within-subjects ANOVA on the stop times revealed significant main effects of both same vs. different (F(1,8) = 14.5, p < 0.05) and experience (F(20,160) = 63.42, p < 0.05) as well as a significant interaction (F(20,160) = 2.04, p < 0.05). Probing the interaction with post-hoc, within-subjects t-tests revealed significant differences at every level of experience prior to receiving 16 reinforcement trials, with the exception of trials preceded by 4, 6, 9, and 10 FI trials. These data demonstrate that the reinforcement time of the previous session caused a difference in both the subject’s start and stop times on the current session. The finding of a significant difference in start and stop times due to the reinforcement time of the prior session documents that memories formed approximately 24 hours in the past affect the behavior of the subject on the current session.

Figure 3.

The lever pressing behavior of the 8–21 subjects when reinforced at 8 seconds was dependent on the previous session; error bars represent the standard error of the mean. If they were reinforced at 21 seconds, they began the current session biased towards this time, and if they were reinforced at 8 seconds, they began the current session biased towards 8 seconds. Figure 3B: Like their behavior on days when the reinforcement time was 8 seconds, lever pressing behavior of the 8–21 subjects when reinforced at 21 seconds was dependent on the previous session. If they were reinforced at 21 seconds, they began the current session biased towards this time, and if they were reinforced at 8 seconds, they began the current session biased towards 8 seconds.

On 21 second sessions (see Figure 3B), single trial responding was earlier when the previous session had been reinforced at 8s compared to when it had been reinforced at 21s. A 2 (same vs. different) x 21 (experience) within-subjects ANOVA on start times revealed that the main effects of both same vs. different (F(1,8) = 18.36, p < 0.05) and experience (F(20,160) = 9.20, p < 0.05) were significant, but the interaction was non-significant (F(20, 160) = 1.3, p = 0.18). For stop times, the main effects of same vs. different (F(1,8) = 14.74, p < 0.05) and experience (F(20, 160) = 4.49, p < 0.05) were significant, but the interaction was non-significant (same vs. diff x experience, F(20, 160) = 0.96, p = 0.51). As with the data from the 8-second sessions, we can conclude that memories formed over 24 hours earlier bias current behavior.

Inflection point analysis

The start and stop times from the 8–21 subjects, when reinforced at 8 seconds and grouped by the number of FI trials that had been experienced, stabilized (as determined by the inflection point) after 9 and 8 FI trials, respectively. When analyzed in a similar manner, the start and stop times of 8–21 subject reinforced at 21 seconds stabilized after 13 and 3 FI trials had occurred, respectively. All PI trials that occurred before temporal control was re-established (i.e., PI trials that occurred before the 9th FI trial had occurred on 8 second sessions, and all PI trials that occurred before the 13th FI trial had occurred on 21 seconds sessions) were removed, and the new peak time and coefficient of variation for this subset of data was: 8 second session – peak time 7.99 ± 0.29 (mean ± s.e.m.) and coefficient of variation 0.69 ± 0.05 (mean ± s.e.m.); 21 second session - peak time 18.9 ± 0.38 (mean ± s.e.m.) and coefficient of variation 0.54 ± 0.02 (mean ± s.e.m.; See Figure 1C). In comparing the 21-only group to the 8–21 subjects when under temporal control and reinforced at 21 seconds, there was no significant difference in the peak time (t(17) = −1.27, p = 0.22) or the coefficient of variation (t(17) = −1.90, p = 0.074, see Figure 1D). This finding suggests that after experiencing several trials within a long session, memories from prior (i.e., short) sessions did not influence the behavior of the subjects in the 8–21 group. Finally, there was a significant difference between the 21-only group and the 8–21 subjects when reinforced at 8 seconds (and temporal control re-established) on peak time (t(17) = −22.5, p <0.05) and the coefficient of variation (t(17) = −3.84, p <0.05), meaning that on short sessions, when temporal control had been re-established in the 8–21 group, the relative breadth of responding was still greater in this group compared to the 21-only control group.

Discussion

Two groups of rats were trained to expect food for responding after an auditory signal had been played for a specific amount of time. One group, the 21-only group, was only reinforced after the signal had been played for 21 seconds. The other group, the 8–21 group, could earn food after either 8 or 21 seconds had elapsed, but during a session only one of these durations was reinforced. There are three main observations from the results. First, the behavior of subjects trained to expect reinforcement at one of two durations (8–21 group) was initially different from another group of rats that were trained to time only one of these durations (21-only group), but these two groups eventually expressed similar responding after the subjects in the 8–21 group experienced multiple (i.e., approximately 13) reinforcement trials. As the reinforcement time of the current session was randomly selected and constant within a session, rats could have used the first reinforcement trial to determine whether the reinforcement time of the current session was either 8 or 21 seconds (and had this happened, there would have been only a marginal difference between the 21-only and 8–21 subjects when reinforced at 21 second). However the 8–21 rats appeared to need several, redundant, reinforcement trials before temporal control was re-established. Secondly, the development of accurate temporal control in the 8–21 group was dependent on whether the reinforcement time occurred at either 8 or 21 seconds. Third, and finally, memories from the session just prior affected current session performance as there was a significant difference between data sets composed of sessions where the prior session was either the same or different from the reinforcement time of the current session. This means that despite the reinforcement time of the current session being selected at random, subjects were sensitive to the reinforcement time of the last session.

Our results suggest that rats may be sensitive to the age of temporal memories because newer memories appear to have a greater impact on temporally-governed behavior. We note that, although we provided random variation in the age of memories from prior sessions, we did not experimentally manipulate age in systematic fashion. The currently predominant theory of interval timing, the scalar expectancy theory, does not specify a role for the age of temporal memories or the process(es) by which a subject may transition from uncertainty to certainty about the reinforcement time. As noted in the introduction, relatively few studies in the interval timing literature have investigated this topic. However, several studies within the matching law literature have experimentally disrupted the reinforcement ratio between two available reinforcement schedules mid-training to track how behavioral control is re-established (Kyonka, 2008; Kyonka & Grace, 2007, 2010). Indeed, the concatenated generalized matching law (Davison & McCarthy, 1988) states that the degree to which trials with different reinforcement ratios from prior experiences change behavior is independent and additive; this assertion resonates with the behavior of the subjects in the 8–21 group because 1) behavior on PI trials changed as a function of the number of preceding FI trials, and 2) the reinforcement time of the prior session affected performance on the current session.

Likewise a number of cognitive models of human memory propose that the age of a memory plays a large role in the impact it will have on a subject’s behavior (Bjork & Whitten, 1974; Brown, Neath, & Chater, 2007; Rubin & Wenzel, 1996). Indeed, without such a mechanism one would expect estimates corresponding to both 8 and 21 seconds to be drawn throughout the session since both criterion times have been reinforced many more times in the past, relative to the number received in the current session. While our study suggests that older memories exert less of an influence on a rat’s expectation as more memories from the current session are formed, we cannot conclude that the memories are forgotten or that a memory distribution only has a finite capacity. In fact, it would be difficult to account for the gradual change in temporally governed behavior observed in the subjects if older memories had no influence.

The current findings suggest that with slight modification to the scalar expectancy theory, specifically to the memory component of the model, these data and previous similar findings could be modelled by this theory. First, the idea that memories associated with a single cue are retrieved at random should be revised as the results here show that rats are sensitive to the age of a temporal memory. Secondly, a mechanism that allows subjects to selectively use a truncated set of memories to inform behavior, specifically a mechanism where newly formed memories exert greater influence than memories from prior sessions, could be added to the memory component of the scalar expectancy theory and potentially account for the re-establishment of temporal control. While memories from prior sessions do indeed play a role in the behavior captured in this study, the ~20 reinforcement trials from the prior session do not appear to influence the subject’s behavior as much as a single reinforcement trial from the current session, as evidenced by the relatively large change in responding that occurs between 0FI (where there are no memories for a current session reinforcement trial) and 1FI (where a single memory from the current session is present) in Figures 2A and 2B. Conversely, during “same” sessions (i.e., when the reinforcement time of the prior session was the same as the current session) the other duration had not been reinforced for at least 48 hours, yet the subject’s initial pattern of responding was clearly influenced by memories from these prior sessions (see Figures 3A and 3B). The exact mechanism for how prior session memories inform this initial pattern of responding would be better informed by a replication of the current study in which the number of prior sessions where the reinforcement time is either the same or different from the reinforcement time of the current session is experimentally controlled.

Any replication of the current study should be cognizant, however, that the difference between the durations used in this experiment (a 2.63 ratio of difference) may have impacted the results. In Experiment 2 of Cheng, Westwood, and Crystal (1993), pigeons were reinforced at one of two durations for pecking the same response key (randomly chosen at each trial), and were presented with test trials in which reinforcement was omitted. Under these conditions, separate peaks of responding occurred when the ratio of difference between the two durations was large (specifically the larger duration was 5 times longer than the shorter duration), but only a single peak of responding that overlapped with both durations when the ratio of difference was small (the long duration was 2.33 times longer than the shorter duration). If the current study was replicated using two durations that had a greater ratio of difference, and were therefore more easily discriminable by the subjects, temporal control may be re-established sooner.

The details necessary for the scalar expectancy theory to be amended so as to account for the re-establishment of temporal control are not fully captured in this study. The only necessary change to scalar expectancy theory identified, based on the current data, is that temporal memories used by the memory component are not chosen at random, and newer memories exert a greater influence on subjects’ expectation compared to older memories. However, future investigations that would lead to the identification of these parameters should be a priority of researchers in this field. Interval timing ability has been identified as a mechanism that is more sensitive to environmental change than other timing processes (Buhusi & Meck, 2005). With slight modification scalar expectancy theory has the potential ability of being able to model this important behavior, and such a change would increase the external validity of this important and widely-accepted theoretical construct.