Interval timing, temporal averaging, and cue integration

Abstract

In a series of recent experiments, we found that if rats are presented with two temporal cues, each signifying that reward will be delivered after a different duration elapses (e.g., tone-10 seconds / light-20 seconds), they will behave as if they have computed a weighted average of these respective durations. In the current article, we argue that this effect, referred to as “temporal averaging”, can be understood within the context of Bayesian Decision Theory. Specifically, we propose and provide preliminary data showing that, when averaging, rats weight different durations based on the relative variability of the information their respective cues provide.

Introduction

Time is central to our experience of the world. For example, both basic functions, such as effectively executing motor movements [1,2], and higher-level processes, such as perceiving causality [3–5], require an accurate representation of the passage of time. Interval timing refers to the perception of time in the seconds-to-minutes range [6]. Recently, interval timing studies have begun to investigate how behavior is affected when conflicting temporal information is present within the environment [7**–10]. Results from these experiments often resemble findings from studies investigating Bayesian Decision Theory. Here, we review and attempt to integrate these two areas of research.

Temporal Averaging

Interval timing is often examined in animals using the peak interval (PI) procedure [11–14]. In a PI task, the presentation of a stimulus (e.g., tone) indicates that reward can be earned for making an operant response after a fixed “criterion duration” has elapsed (e.g., 10 seconds). Probe trials are also included, during which the stimulus is presented for 3–4 times the length of the criterion duration and no reinforcement is provided. When plotted as a function of time, average response rate during probe trials typically resembles a Gaussian distribution, with a mode centered over the time at which reward is usually earned (i.e., “peak time”) (for examples, see Figure 1).

Figure 1.

Probe trial data from a temporal averaging experiment adapted with permission from Kurti et al. [7]. Short and long refer to trials where the short and long cues were presented in isolation. Compound refers to responding during trials in which both the short and long cues were presented simultaneously.

The spreads of these response distributions reflect variability in temporal estimates, and grow in direct proportion to the duration being timed [15]. For example, if the criterion duration were doubled, both the peak time and spread of the original distribution would increase by a factor of two. Consequently, when the spread of a response distribution is divided by its peak time, the resulting value, referred to as the coefficient of variation (CV), will remain constant across different durations. This pattern of variability is observed in a variety of timing tasks and is referred to as the scalar property of interval timing [16,17].

Recently, we conducted a PI study in which rats were trained to associate two cues (tone and light) with two different durations (e.g., 10 and 20 seconds, respectively) [8]. After the rats were well trained, we added “compound” probe trials in which both the tone and light were presented simultaneously. We included these trials to assess how rats would respond when presented with conflicting temporal information regarding when reward would become available. When cues were presented in isolation, responses were centered over their respective criterion durations. However, during compound trials, rats produced scalar response distributions that fell in between the two criterion times. In other words, rats appeared to time a single temporal expectation that reflected a combination of the two component durations. Therefore, this effect was referred to as “temporal averaging”.

Further work showed that compound peak times usually fall closer to the long duration, suggesting rats time a weighted average during compound trials with more weight being given to the long cue (see Figure 1) [7**,9,15,18*]. This increased weight has been attributed to the fact that experimenters often assign the long duration’s cue with a higher reinforcement probability (ratio of rewarded trials to probe trials) to equate response rates for the two cues, as response rates tend to decrease with increasing duration. In support of this, several studies have shown that compound peak times can be predicted well by computing an average of the component peak times when each is weighted by its respective reinforcement probability [7**,9,15].

Averaging behavior and Bayesian Decision Theory

Averaging behavior has also been observed during investigations of spatial navigation [19–22]. For example, if pigeons are presented with multiple landmarks that provide discrepant information regarding the location of a hidden goal, they will navigate based on an average of self-to-goal vectors derived from each landmark [23]. Importantly, landmarks that have been closer to the goal in the past will be weighted more heavily, presumably because closer landmarks provide more reliable spatial information. We will refer to this effect as “spatial averaging”.

Recently, Cheng et al. [24] proposed that spatial averaging can be explained using Bayesian Decision Theory (BDT); a prominent framework used to study how information is integrated in order to optimize behavior. According to BDT, when multiple sources of information (e.g., A and B) can be used to inform a judgment, observers should compute a weighted average of estimates derived from each source, as given in the equation below:

$$I_{AB}=W_{A}A+W_{B}B$$ (1)

A and B refer to estimates from each source; W_A and W_B correspond to the weights given to each estimate; and I_AB refers to the integrated estimate. More reliable information should be weighted more heavily during this averaging process. Mathematically, the “reliability” of a given source is defined as the inverse variance of the information it provides, and weights are computed as follows:

$$W_A=\frac{1/{{\sigma }^2}_A}{1/{{\sigma }^2}_A+1/{{\sigma }^2}_B}$$ (2)

$$W_B=\frac{1/{{\sigma }^2}_B}{1/{{\sigma }^2}_A+1/{{\sigma }^2}_B}$$ (3)

BDT can be applied to spatial averaging by assuming A and B represent navigation vectors derived from two landmarks [24]. If the closer of the two landmarks provides more reliable (i.e., less variable) spatial information, its corresponding vector would be weighted more heavily, as seen in the data [23]. To the best of our knowledge, the relationship between spatial averaging and BDT has not been formally tested. However, this integration process has been extensively evaluated during studies investigating a phenomenon referred to as “cue integration” [for reviews, see 25**–28].

An example of cue integration that is particularly relevant to the topic of spatial averaging comes from Alias and Burr [29], who asked human subjects to judge the spatial location of briefly presented auditory and/or visual cues. The experimenters blurred the visual stimulus to different degrees on different trials in order to vary its reliability. During single-modality trials, cues were presented individually. This allowed Bayesian weights to be computed for each cue-type and blur-level based on variance in participants’ judgments. During multi-modality trials, both cues were presented simultaneously and in different spatial positions. During these trials, participants appeared to compute an average of the two cue-locations (i.e., “integrate” both locations into a single estimate), and the weights given to each cue matched closely with the Bayesian predictions.

Studies using similar tasks have shown that cue integration occurs within [30–33] and across [34–38] a variety of modalities, and generalizes to a variety of judgments such as shape [33,37], size [38], slant [30,32,34], and height [36]. Collectively, this area of research suggests that cue integration is a general computational framework that observers use to combine information from different sources.

Temporal Averaging and Bayesian Decision Theory

Given the similarities between temporal averaging, spatial averaging, and cue integration, we sought to re-evaluate our past temporal averaging data within a Bayesian framework. As described above, rats weight the longer duration more heavily during compound trials. From a Bayesian standpoint, this suggests the long duration’s cue provides more reliable information than the short. If so, responses should be less variable when the long cue is presented in isolation, relative to the short cue. This poses a potential problem, as absolute variability in temporal estimates increases with increasing duration. However, across several of our experiments, CVs for the short cue have been significantly larger than those of both the long and compound cue [7**–9]. In other words, relative, as opposed to absolute, variability in responding has been higher for the short cue than the long.

These CV differences likely relate to the fact that we typically assign the short and long cues with low and high reinforcement probabilities, respectively, in order to generate equivalent response rates for both stimuli. Specifically, during PI tasks, a cue’s reinforcement probability is known to impact the amount of behavioral variation it elicits. For example, using a PI procedure, Roberts [39] showed that decreasing the reinforcement probability of a 40s cue caused spreads to increase [for similar results see 40,41]. This effect is not specific to interval timing behavior, as recent studies have begun to establish that reinforcement probability is inversely related to variation in various forms of operant behavior [42–44]. Many have suggested that, with lower reinforcement probabilities, increases in response variation allow subjects to discover better strategies for obtaining reward [45,46]. However, Stahlman, Young, and Blaisdell [47] recently documented this effect during a Pavlovian task (i.e., where no response strategy is required) calling this explanation into question. This finding suggests that, as a general rule, variation in conditioned behavior may increase as a cue’s predictiveness of reward decreases. If so, manipulations of reinforcement probability would have an interesting parallel to manipulations of reliability during cue integration studies. Specifically, during cue integration studies, decreasing a cue’s reliability typically entails corrupting its appearance [e.g., 35,36,38]. In other words, the ability to perceive an event (i.e., the cue) is altered. Manipulating reinforcement probability may have a similar effect with the primary difference being that the ability to predict, rather than perceive, an event (i.e., reward) is altered.

To evaluate whether BDT can be used to explain temporal averaging behavior, we reanalyzed all of our prior compounding data in which temporal averaging was observed. We computed Bayesian weights for each rat, with component cue CVs serving as a measure of variation, and used these to compute a weighted average of the component peak times (for further description, see Figure 2). Predicted and actual compound peak times are plotted against one another in Figure 3, and the accuracy of the Bayesian predictions for each study are plotted in Figure 4. Both figures show that temporal averaging behavior is well accounted for by BDT.

Figure 2.

Schematic of Bayesian weighting during compound trials. Top left: Hypothetical short and long probe trial responding. Top right: Data from the same short and long distributions. However, for each cue, the proportion of maximal response rate across time is expressed as a proportion of the peak time (100% corresponds to the maximums of either function). Plotting the data in this way shows that relative variability in short cue responding is higher than the long cue, despite the fact that the opposite is true in terms of absolute variability. Bottom right: The equation used to compute predicted compound peak times based on the relative variation (CV) in responding to the short and long cue. Bottom left: Same as top left; however, predicted compound responding is now plotted. Note that increased relative variation for the short cue causes the compound function to fall closer to the long duration.

Figure 3.

Scatterplot of actual compound peak times from rats in all prior temporal averaging experiments and predicted compound peak times under BDT based on relative variation in component cue responding. A line of best fit with a constrained intercept of zero is also plotted (slope=1.08, R=.891).

Figure 4.

Average accuracy of Bayesian predictions for rats in each study and across all rats. Accuracy was computed for each rat by dividing the predicted compound peak time by the actual compound peak time. Values of 1 represent perfect accuracy, whereas values above and below 1 correspond to over and under-prediction, respectively. The criterion durations and reinforcement probabilities of the short and long cues are included next to each study citation.

Conclusions and Implications

Establishing a connection between temporal averaging and cue integration could have important implications. For example, the neural mechanisms that mediate interval timing are currently unclear. However, if temporal averaging is indeed a form of cue integration, it may be possible to constrain neural models of interval timing using well-established neurophysiological theories of cue integration.

For example, according to the “probabilistic population coding” hypothesis of cue integration [48–50], the cortex represents stimuli as probability distributions, via collections of neurons with tuning curves that differ in regard to their preferred stimulus value. When multiple stimuli are presented simultaneously, Ma et al. [48] showed that cue integration can easily occur by linear combination of the distributions produced by each stimulus (i.e., by multiplying the two distributions element-wise). If temporal averaging and cue integration are related, this would imply the existence of neurons with “temporal tuning curves” (i.e., neurons that respond differentially when exposed to different durations). This conclusion has been reached based on behavioral data before [51,52] and is supported by evidence from electrophysiology studies. For example, Merchant et al. [53**] recently observed neurons in the medial premotor cortex with Gaussian tuning curves that responded preferentially to specific durations, regardless of the modality of the stimulus being timed. Similarly, studies using PI tasks or comparable designs have documented neurons that exhibit Gaussian-like or ramping firing profiles during elapsing intervals that appear to be tuned to the programmed criterion duration [54–58**].

However, the variability of these firing profiles grows in proportion to the duration being timed [57]. Consequently, while multiplication of the tuning curve activity associated with a short and long duration across time during a compound trial would lead to an intermediate peak, it can be shown that the compound peak time would fall closer to the short duration, which is inconsistent with the data. For such a mechanism to work, there would have to be neurons that were tuned according to the relative variability of their preferred duration. It is possible that such neurons exist. However, given the predominance of scalar variability across interval timing tasks [16], this possibility seems unlikely.

An alternative is that there are temporally tuned neurons that are responsive to the duration associated with a cue, but do not generate timed behavior directly. For example, upon presenting a temporal cue, neurons representing its associated duration could become responsive and activate a downstream timing process that generates temporally controlled behavior. If two cues were presented simultaneously, the population activity associated with their respective durations could be combined, causing the downstream timing process to time an intermediate interval. This proposal is similar to the conceptual explanation of temporal averaging that we have relied on in the past. Specifically, we have proposed that, at the start of a compound trial, the temporal memories associated with the short and long cues are retrieved and integrated into an averaged temporal expectation, which is then timed in an otherwise normal manner [15]. In support of this proposal, Mita et al. [59] found neurons in the presupplementary motor area that fired transiently at the onset of a temporal cue. Importantly, the nature of these responses differed as a function of the duration associated with the cue being presented.

Finally, pursuing the relationship between cue integration and temporal averaging might help explain instances where temporal averaging is not observed. Specifically, while not discussed in detail here for sake of space, temporal averaging is usually only observed when the tone and light are associated with the short and long durations, respectively. Counterbalanced groups (light-short/tone-long) show only partial signs of averaging during compound trials, as responses overlap considerably with the light’s associated duration [7**,9,60, for further discussion see 61]. It is unclear why this “modality effect” occurs. However, several cue integration studies have reported similar effects, in which cues presented in certain modalities are weighed more heavily than expected by BDT [35,62–65]. Authors have accounted for this effect by proposing subjects have a prior probability distribution that creates a tendency to favor information from one modality over another [35]. Models of interval timing that incorporate Bayesian priors are beginning to be developed [66*–70]. Therefore, it would be interesting to see if these models can be used to account for this effect.

Highlights.

1. When presented with multiple temporal cues, rats will time an average of their respective durations

2. Non-temporal averaging behavior has been observed during cue integration studies

3. Cue integration findings have been interpreted using Bayesian Decision Theory

4. We evaluate temporal averaging using this Bayesian perspective