Timing and Anticipation: Conceptual and Methodological Approaches

Abstract

Anticipation occurs on timescales ranging from milliseconds to hours to days. This paper relates the theoretical and methodological developments in the study of interval timing in the seconds, minutes and hours range to research on the anticipatory activity induced by regularly timed daily meals. Daily food anticipatory activity (FAA) is entrained by procedures which are formally identical to procedures studied in Pavlovian and Operant conditioning except for the long duration of the interval between feeding opportunities. As in FAA, the conditioning procedures induce orderly anticipatory activity in advance of food presentation. During the interval between foods the behaviors that express anticipation change as the interval progresses. Consequently, no single response represents a pure measure of anticipation. The ability to distinguish between properties of general anticipatory timing mechanisms such as the scalar property (Gibbon, 1977) and dynamic properties of specific response output systems has been facilitated by teaching animals to use arbitrary anticipatory responses like bar pressing to obtain food. Interval timing research highlights the importance of identifying the mechanisms of perception, memory, decision making and motivation that all contribute to food anticipation. We suggest that future work focused on the similarities and differences in the neural bases of FAA and interval timing may be useful in unravelling the mechanisms mediating timing behavior.

When we anticipate we take action based on our beliefs about the future state of the world. Implicit in this capacity is the ability to discriminate the passage of time. Minimally, an ordinal sense of now and later is required for anticipation but the sense of time in all species that have been studied is more acute; it provides a quantitative sensitivity to the passage of time. In fact, it appears that multiple timing mechanisms have evolved to guide anticipation and thus insure that the right adaptive response occurs at right time.

Timing and anticipation takes place on different scales. Execution of fine motor actions requires precise timing in ranges on the order of milliseconds. To pick up a piece of seed a bird must open it’s beak and reclose it exactly at the moment when the beak surrounds the target - an action which must be precisely timed if it is to be successful (Klein et al., 1985; Bermejo & Zeigler, 1998). The rich literature on rapid motor timing suggests that temporal control arises, at least in some cases, directly from motor control circuits (Karmarkar & Buonomano, 2007; Ivry & Schlerf, 2008). On a much larger scale is the timing controlled by circadian rhythms. These rhythms regulate sleep patterns, metabolic rates, body temperature and hormone secretion, to name just a few aspects of circadian contol. The body anticipates when in a 24 hour cycle waking, feeding and social interactions will occur. The suprachiasmatic nucleus (SCN) coordinates much of this timing but the papers in this volume on food anticipatory activity (FAA) document that there are other mechanisms for timing these long intervals that do not depend on the SCN. There is a third kind of timing that also appears to be SCN independent, namely that underlying the timing of arbitrary intervals from seconds, to minutes, to hours to days(Innis & Vanderwolf, 1981; Lewis et al., 2003; Cordes & Gallistel, 2008). We learn to anticipate being able to use our computer seconds after booting, that water will boil in a matter of minutes and that our faculty meeting is likely to take hours. Because these intervals are arbitrarily selected we lump them all into a category known as interval timing.

In this paper we focus on the relation between FAA and interval timing procedures in which animals learn to anticipate food at arbitrary intervals. There is a strong similarity in the methods. Yet, these two areas have evolved fairly independently. We contrast the methods and conceptualizations of interval timing with those used in the study of FAA in the hopes of stimulating research that deepens our understanding of the similarities and differences between these two types of anticipatory responding.

The standard procedure for studying FAA is to arrange for an animal to have a single meal at a regular time each day. After a few days on such a regimen animals become active prior to food presentation. This method is identical to that used since the time of Pavlov to study learned anticipation at much shorter intervals. In his best known experiments, Pavlov (Pavlov, 1927) would use an external cue such as a metronome as a signal for food, but in some experiments, he did not add an external cue between food presentations. In those experiments he repeatedly presented food to his dogs after a fixed interval of time had elapsed since the prior food delivery. He termed this procedure temporal conditioning because he believed that an endogenous temporal cue served as a signal for the impending reward. Just as a bell paired with food served as a conditioned stimulus this unobserved temporal cue came to be a learned predictor of food.

In more recent and systematic versions of the temporal conditioning experiment Kirkpatrick and Church (Kirkpatrick & Church, 2003) have examined food anticipation over intervals ranging from 45 to 360 s. Here, food pellets are delivered to hungry rats at fixed intervals between each presentation. Anticipation is measured by the likelihood that a rat will poke its head into the feeder where a pellet will be presented. Figure 1 shows that as the intervals elapse, the average level of anticipatory headpoking increases. There are two important features of this anticipation that are sensitive to the interval between food presentations. The ramping function is shallower and the final level of anticipation is lower the longer the time between food presentations. Pavlov believed that temporal conditioning was no different than conditioning with external cues. He thought that the neural representation of sensory events (traces) decayed over time. Consequently, the strength of the trace provided information about how much time had elapsed since the onset of a sensory change. When food followed a tone, learning occurred because the neural representation of the tone conditioned stimulus (CS) was activated contiguously with the representation of the food unconditioned stimulus (US). Thus if a conditioned stimulus was of a fixed duration the dynamic representation of the CS would change over time and the representation most closely paired with the US would be one correlated with a specific time since the onset of the CS. As a consequence, conditioned responses (CRs) would be timed. Similarly, if the only cue that signaled an impending US was the prior US, the decaying neural representation of the prior US would be paired with the subsequent US. Thus anticipatory CRs would become more likely as the interval between US’s elapses.

Figure 1.

Effect of varying the interfood interval on timed food anticipation in rats. Food was delivered at a fixed time after the previous food presentation. The time between food presentations was varied across different groups of rats. The figure shows responses per minute (head pokes into the food receptacle) as a function of time since the last food delivery. A burst of head poking that occurred after pellet delivery has been omitted from this replot of the data. Modified from Kirkpatrick and Church (2003) with permission.

Recent studies of conditioning affirm this aspect of Pavlov’s predictions. When subjects are exposed to a fixed CS-US interval from the outset, they rapidly form a temporally based expectation of the US. In one study (Drew et al., 2005) we exposed goldfish to an aversive conditioning procedure in which a brief shock (US) was presented 5 s after the onset of a light (CS). On a few trials in each session the light remained on for 45 s and no shock was presented. This procedure allowed us to see when the CR occurred during trials over the course of training. Figure 2 shows the development and timing of anticipatory (that is “conditioned”) activity as conditioning progressed. The main effect of training was to change the magnitude of the CR; the time at which CR’s occur did not change. Modeling of the distributions over the course of training confirmed this conclusion. The peak height changes but its location does not. In the same vein, there is evidence that the very first occurrences of CR’s are timed in many preparations, including eyeblink conditioning in rabbits (Ohyama & Mauk, 2001), appetitive head-poking in rats (Kirkpatrick & Church, 2003) and autoshaping in birds (Balsam et al., 2002). In fear conditioning preparations, temporal control of conditioned responding can occur after just one CS-US pairing (Davis et al., 1989). That is, after one trial, the CR timing reflects the CS-US interval. This line of thinking has led us to suggest that Pavlovian conditioning should be conceptualized as learning to anticipate outcomes (such as food or shock) based on the information that a cue provides about when an outcome will occur (Balsam & Gallistel, 2009).

Figure 2.

Timing of conditioned anticipation in goldfish during Pavlovian training. A 5-s visual CS terminated in mild shock (US). Training trials were intermixed with probe trials, during which the CS remained on for 45 s with no US. Magnitude of activity as a function of time in the CS is shown in blocks of 5 sessions during the long, unreinforced probe trials. The overall amount of anticipatory activity increased as training progressed, but CRs were timed appropriately from the outset of training. Modified from Drew et al. (2005) with permission.

It is customary in the FAA literature to measure a single anticipatory behavior (e.g. activity) as a reflection of the underlying timing mechanisms. Studies of interval timing, however, suggest that new information can be gleaned when more than one measure of anticipation is assessed. In Pavlovian conditioning, the topography of the anticipatory response is determined in part by the time until the next US. Cues that signal different temporal distances to US’s control qualitatively different responses. Holland (Holland, 1980) exposed groups of rats to CS’s of different durations paired with a food US. He found that when auditory CS’s were brief, they tended to evoke head-jerk CR’s; when they were long, they evoked less head-jerking but much more approach to the feeder. Timberlake (Timberlake, 2001) has suggested that motivational modes change with proximity to reinforcement. Immediately after food presentation an animal’s feeding related behavior may be inhibited. As food becomes more imminent, the subject first enters a general search mode followed by a focal search mode and then a handling/consummatory mode. Even when one food presentation is the only signal for when the next food is to be expected, as in temporal conditioning (and perhaps FAA), behavior changes systematically as the interval elapses. During the early post-food interval animals are likely to withdraw from feeding areas. During the middle of the interval animals are likely to be very active and engage in exploration and during the last part of the interval one is likely to see focal search and/or consummatory behaviors such as investigation of feeding sites and anticipatory physiological responses that prepare the animal for food ingestion (Staddon & Simmelhag, 1971; Silva & Timberlake, 1997; Silva & Timberlake, 1998; Silva & Timberlake, 2005). Thus if one only looks at a single behavior as an assay of timing, accuracy and precision conclusions may be limited to the response system that is being measured. In a similar vein, the focus on a single measure of anticipation might mislead one into thinking that a particular timing system was restricted in the range of times over which it could accurately guide anticipation when in actuality the anticipation was timed accurately but the form of the anticipatory response had changed (and therefore went unmeasured) due to changes in the motivational mode of the organism over the course of the timed interval. For this reason, focusing on a single response system to measure timed anticipation can be problematic.

One way to confront this problem is to teach the animal an arbitrary anticipatory response such as pressing a bar in order to obtain food. This response can then be used to study anticipatory responding under a wide range of intervals. When the duration of the interval from one food until the next can be obtained is fixed (known as a Fixed Interval (FI) reinforcement schedule) animals begin responding about half way through the interval and the likelihood of responding increases as the expected time of reward approaches. As in the study of FAA one does not know if behavior is timed when the entraining stimulus is present. In both FAA and FI schedules food presentation might suppress activity. As the time since the last food elapses the likelihood of responding might increase monotonically but this might simply be interrupted by the next food. This would provide the appearance of anticipation but would not depend on the animal having learned when to expect food. In order to see if the animals are timing they must be tested in the absence of the entraining cue. In both FAA and FI studies the animals needs to be tested in the absence of food to see if the animal has learned the expected time of reward. To achieve this in FI schedules, the procedure is modified into what has come to be called a Peak Interval (PI) task (Bitterman, 1964; Catania, 1970). Animals are exposed to two kinds of trials in this task. On FI trials reward becomes available for the first response following a fixed time after stimulus onset (e.g., tone). Intermixed with these FI training trials are unreinforced probe trials, called peak trials, in which the same stimulus remains on for a long duration (usually 3 or 4 times the FI value) and no rewards are available. On the standard FI trials the likelihood of responding increases until the time when reinforcement is earned. During peak trials, the average response rate increases up to a maximum at the scheduled time of reinforcement, and then decreases, providing an estimate of the time at which the subject expected to be reinforced. The peak time and the shape of the response rate curve provide indices of the accuracy (peak location) and precision (variability) with which subjects time.

There is remarkable species and duration generality to this timing. Figure 3 shows the performance of humans, pigeons and mice on this task. The figures on the left show the average performance in real time. The figures on the right have been rescaled with respect to both peak location and height. The superposition of the curves indicates that this interval timing has the scalar property: The standard deviation increases linearly with the mean of the interval being timed (Gibbon, 1977). This is a hallmark of interval timing and seems to hold across intervals ranging from a second to almost an hour.

Figure 3.

Timed anticipation in humans (modified from Rakitin et al., 1998, with permission), pigeons (from Gibbon, Fairhurst & Goldberg, 1997 with permission), and mice (from Brunner, personal communication with permission) shows the scalar property. The left panels show mean response rate as a function of absolute time in the interval, the right panels show mean percentage of maximum response rate as a function of relative time in the interval. The superposition of the relative response rate distributions shows the scalar property- the standard deviation of the distributions increases linearly with the mean.

Though the complete peak procedure has been utilized to study anticipation over seconds and minutes, the FI procedure (without peak trials) has been used to study anticipation over a range of intervals from seconds to many hours. Figure 4 shows performance on FI schedules ranging from 30 seconds to 50 minutes. These typical results again show the scalar property as shown by the superposition of the curves when they are rescaled with respect to absolute time and response rate. It should be noted that while this scalar property is frequently observed across a broad range of intervals there are local maxima and minima in the variability of timed anticipation suggesting that the clock mechanism may be tuned to some intervals better than others (see Crystal, this volume) (Crystal, 2003; Crystal, 2006).

Figure 4.

Timed anticipation in fixed interval schedules shows the scalar property. The figure shows the rate of responding as a function of time in the fixed interval. Both measures have been rescaled relative to the absolute response rates and interval duration. Note the superposition of the relative response rate functions over the range from 30 seconds to 50 minutes. Modified from Dews (1970) with permission.

The ability to perform accurately on these food anticipation tasks involves more than having a clock. Figure 5 shows all of the cognitive/behavioral processes that underlie anticipation on interval timing tasks. To perform well the subject needs to attend to the relevant aspects of the task, learn an association between a response and the delivery of reinforcement, perceive and remember time, decide whether a given interval has elapsed by comparing the elapsed time during a trial with the memory for previously experienced durations and act on the comparison by executing or inhibiting responses.

Figure 5.

Schematic illustration of cognitive/motivational processes that are thought to underlie and to modulate anticipation in interval timing tasks. See text for details.

Specifically, an environmental event must first trigger (or entrain) a timing mechanism that generates a quantitative measure of time used to create and store representations of the duration. Information about elapsed time (t) is stored in working memory and when the event ends or another event happens the value of that duration (T*) is stored in long-term or working memory. On subsequent occasions when this event is encountered the elapsing time can be compared to the remembered time (t/T*) and depending on the comparison of this value to behavioral thresholds appropriate behavior can be selected.

For example, when we stop at a familiar red light we automatically store the duration of the interval until it turns green. On subsequent occasions when we sit at that red light we compare the elapsing duration to our memory of how long it stayed red in the past. If that duration exceeds our remembered duration we eventually decide that the light is broken and go through the red light. Similarly, if an animal is fed once a day it must perceive the time on a “food clock” (t) and store a memory of the meal time (T*). On subsequent days it must compare the elapsing time to the remembered time. When this comparison reaches a threshold value (b) the animal shows in increase in anticipatory activity.

It is worth noting that Figure 5 should not be taken to mean that there is a single interval timer. Indeed the limits of our ability to simultaneously time different arbitrary intervals are not known nor do they seem bounded. In the red light example presented in the previous paragraph not only do we know the duration of the red light but we also know, how long it took to reach the light since the previous one, how long it has been since we began the trip as well as almost any arbitrary interval that could be computed along the way. An alternative conceptualization to a specific clock is that the time of occurrence of all remembered events is encoded as part of that memory (Gallistel, 1990). Such a system would allow for the computation of an unlimited number of arbitrary intervals through subtraction. How the nervous system might accomplish this is of course a great mystery.

There are also state and motivational factors that affect the accuracy and variability of anticipation. When motivation is low, timed anticipation becomes less accurate and more variable (Ward & Odum, 2006; Ward & Odum, 2007; Galtress & Kirkpatrick, 2009; Ward, in press). Attention also affects anticipatory timing. When subjects are distracted when producing a time interval their peak anticipation is delayed (Fortin, 2003; Buhusi & Meck, 2006a; b; Champagne & Fortin, 2008). Similarly performing different tasks while doing an anticipatory timing task causes overestimation of the interval (Zakay & Block, 1997; Fortin & Couture, 2002) – even when the alternate task is to simultaneously time another interval(Brown et al., 1992) Emotional states also alter the accuracy of anticipation. When subjects are in fearful states, such as viewing an angry face (Droit-Volet & Meck, 2007; Gil & Droit-Volet, 2009), arachnophobics viewing images of spiders (Watts & Sharrock, 1984), or people walking towards the edge of a rooftop (Langer et al., 1961), time is overestimated in comparison to non-fearful states. In a broad analysis of how mood affects timing, Angrilli (Angrilli et al., 1997) found that highly arousing negative states and mildly arousing positive lead to overestimation, while mildly arousing negative states and highly arousing positive emotions lead to underestimation. Drugs of abuse also alter timing in a variety of ways (Mattell et al., 2001; Buhusi, 2003a; Matell et al., 2006; Cheng et al., 2007; Taylor et al., 2007) probably through both state modulation as well as direct actions on timing mechanisms. These factors may also modulate timing and food anticipation on a circadian scale; alternatively circadian timers, unlike interval timers may be impervious to local fluctuations in environmental conditions and organismic states.

Figure 5 also implies that the processes that underlie anticipation might best be understood by studying behavior on a trial by trial basis. Up to this point all of the figures suggest that anticipation gradually increases as intervals progress. However, all of the smooth curves shown above are averages. They are a composite of anticipation across a number of trials. The behavior on any single trial may not be well represented by the composite performance and, in fact, the performance when examined in real time has a quite a different character. Individual peak trials are better characterized by abrupt transitions in response rates. On a typical trial, rates begin low, followed by an abrupt transition to a high relatively steady rate prior to the expected time of reward and then return to a low rate after the expected time of reward has passed (Gibbon & Church, 1992; Church et al., 1994; Balci et al., 2009). This pattern is depicted in Figure 6. This transition from low to high responding (start time) and back to low responding (stop time) can be estimated using a variety of methods (Gallistel et al., 2004a). The midpoint of that interval (middle time) is then taken as the subject’s estimate of the expected time of reinforcement on that trial. This methodological consideration is important when characterizing the effects of some manipulation or treatment that is predicted to alter timing. For example, Taylor et al (2008) employed the single-trials analysis in an investigation of the effects of amphetamine on timing. When performance was averaged across many trials the peak time of responding shifted to an earlier time in the peak trial. Such leftward shifts in the peak function could be interpreted as indicating that the speed of an internal clock had increased (Meck, 1983; Buhusi & Meck, 2002). However, performance on single trials looked like the representative performances depicted in Figure 6. While amphetamine caused an increase in the number of responses, the change in the temporal pattern of anticipatory behavior was not produced by an overall change in timing. Instead, amphetamine caused the subjects to begin responding earlier in the trial but stop times did not change. The data are more consistent with an explanation in terms of a shift in the threshold to start anticipatory behavior than in an explanation based on a shift in clock speed which should have produced both earlier starts and stops. This dissociation between starts and stops suggests that there might be distinct mechanisms that trigger the onset and offset of anticipation. Other studies that have examined performance on individual trials have reached a similar conclusion that the starting and stopping of timed anticipation represent independent decision processes (Gallistel et al., 2004b; Balci et al., 2008; Balci et al., 2009). It would be of interest to know whether the controls for the start and stop of FAA are also independent. One could approach this question by implementing FAA studies that are analogous to the peak procedure. Scheduled feeding periods could be omitted on unpredictable days. This would not only allow for the study of mechanisms that end anticipation but it would also allow for the repeated observation of anticipation on the days following food omission. Typically, FAA does not persist for very long after the removal of the entraining cues. This suggested variation of the peak procedure would in principle allow for unlimited observations of endogenous FAA rhythms.

Figure 6.

Amphetamine decreases start times in the peak interval task. The top panel shows responding during a single baseline peak trial. The majority of responses occur between 14 and 33 s into the peak trial. The bottom panel shows the responding for the same animal during a peak trial after administration of .5 mg/kg amphetamine. Responding begins at 6 s but ends at 33 s, showing an earlier start time but the same stop time as the baseline trial (modified from Taylor et al., 2007 with permission).

Another comparison of interest will be whether the brain structures and networks important for timed anticipation of arbitrary intervals share any commonality with those that underlie circadian FAA. With respect to the timing of the arbitrary intervals, it is not known how networks receive or generate a temporally correlated signal (the clock) but there are some areas of the brain that are consistently implicated in interval timing. In a range of imaging, lesion, recording and pharmacological studies cerebellum has been shown to play a role in timing and perhaps an especially important one in the timing of short intervals less than one second (Mangels et al., 1998; Coull, 2004; Coull et al., 2004; Nobre & O’Reilly, 2004). Coriticostriatal circuits also seem to play a large role in timing and anticipation of arbitrary intervals. Lesions (Meck, 2006), pharmacological (Meck, 1996; Buhusi, 2003b; Drew et al., 2003; Taylor et al., 2007) and genetic (Drew et al., 2007) manipulations as well as electrophysiological recordings (Matell et al., 2003; Matell & Meck, 2004) of this pathway all point to the importance of these circuits in temporal anticipation and timing. Hippocampus seems important for timing tasks with working memory components (Buhusi & Meck, 2005). Lastly, cortical areas such as SMA (Lewis & Miall, 2003), PFC ((Mangels et al., 1998; Koch et al., 2003; Koch et al., 2004) and posterior parietal cortex (Leon & Shadlen, 2003) have all been implicated in timed anticipation in different tasks. It is clear that from even this brief and incomplete review that many structures and circuits in the brain are involved in timing and anticipation of arbitrary intervals. It is not yet clear how to definitely map this neurobiology on to the functions shown in Figure 5. The answer will be complex but perhaps as the work on FAA and interval timing progresses an understanding of how perception, memory, decision and response selection processes will begin to emerge and be applicable to many kinds of timing and anticipation systems.

In summary, there are lessons learned in the study of interval timing that could inform research on FAA. First, there is no single “pure” measure of anticipation. The manifestations of anticipation change as the time for the next meal approaches. Thus multiple measures of anticipation will need to be studied. Second, it is useful to characterize anticipation as a real-time change in behavior. When behavior is averaged over days (or trials) the action of the mechanisms that control anticipation on a moment-to-moment basis may be obscured. This sort of analysis on interval timing data made it clear that we are likely to find independent starting and stopping mechanisms for different manifestations of anticipation as intervals elapse. A third related point is that there are multiple, simultaneously active clock mechanisms that underlie anticipation. For example, in the Pavlovian conditioning experiments the animals learn the times from the CS onset to the food, from the CS offset to the food and the interval from the previous US until the next one (Kehoe & Napier, 1991; Kirkpatrick & Church, 2000; Balsam & Gallistel, 2009). The level of anticipation at any point in time is a composite of all the relevant expectations. Similarly, if food is available during a single restricted period each day it is likely that animals learn the time of feeding with respect to a light entrainable oscillator, a food entrainable oscillator(s), interval timers triggered by light onset or offset and perhaps an interval timer that record the duration of one meal to the next. FAA, like interval timing, may involve the coordinated use of temporal information from several sources for multiple functions (also see Crystal, this volume). Lastly, the processes that alter timing accuracy and variability in FAA remain to be detailed. Is a change in the slope of the anticipation function as a result of a genetic or pharmacological manipulation, the result of a change in variability in the clock mechanism, threshold or memory? Such questions which have come to guide the study of interval timing could provide new strategies and methods for investigating FAA.