Significance
Humans rely on their inner sense of time to judge the duration of events or decide when to initiate actions. Still, they also often use movements to improve their timing accuracy. In both humans and animals, it is unclear whether accurate timing relies on disembodied internal representations or is externalized in motor routines. We investigated this issue in rats performing a timing task while running on a motorized treadmill. Our setup allowed us to create conditions that either favored or limited the usage of motor routines. We found that timing accuracy was improved when the environment afforded cues that rats can incorporate into motor routines. Timing, at least in animals, may thus be fundamentally embodied and situated.
Keywords: timing, embodiment, internal clock
Abstract
How animals adapt their behavior according to regular time intervals between events is not well understood, especially when intervals last several seconds. One possibility is that animals use disembodied internal neuronal representations of time to decide when to initiate a given action at the end of an interval. However, animals rarely remain immobile during time intervals but tend to perform stereotyped behaviors, raising the possibility that motor routines improve timing accuracy. To test this possibility, we used a task in which rats, freely moving on a motorized treadmill, could obtain a reward if they approached it after a fixed interval. Most animals took advantage of the treadmill length and its moving direction to develop, by trial-and-error, the same motor routine whose execution resulted in the precise timing of their reward approaches. Noticeably, when proficient animals did not follow this routine, their temporal accuracy decreased. Then, naïve animals were trained in modified versions of the task designed to prevent the development of this routine. Compared to rats trained in the first protocol, these animals didn’t reach a comparable level of timing accuracy. Altogether, our results indicate that timing accuracy in rats is improved when the environment affords cues that animals can incorporate into motor routines.
The ability of animals to adapt their behavior to periodic events is critical for survival, as the appearance of a sensory cue can predict the timing of food availability, predator attack, or mating opportunity (1–4). It has been postulated that humans and other animals use a dedicated internal clock to evaluate the duration of behaviorally relevant time intervals and sensory cues, or to produce well-timed movements (5–10). However, time is a critical parameter for a wide range of behaviors engaging distinct brain regions. In addition, temporal representations are intrinsic to the activity of ensembles of neurons [i.e., neuronal population activity dynamically evolves in time (11)]. Thus, more recently, it has been proposed that the ability of animals to judge the duration of sensory stimuli or to produce well-timed movements emerges from self-sustained dynamics of task-specific neuronal populations (11–13).
A common assumption of the dedicated and intrinsic/emergent models is that timing is primarily emerging from neuronal activity. Consequently, a significant effort has been made to isolate neuronal representation of time independent of other variables that may covary with time, such as the production of movements or sensory information. Such an approach, which is equivalent to considering that brains can tell time, is problematic for two reasons (14): 1) The body and the brain are inseparable in their function; and 2) animals have their own goals, and their behavior should not be reduced to a linear process in which neuronal computation (here time measurement) is intercalated between a stimulus and a behavioral response. The close relation between body and brain is particularly relevant to the question of timing, as humans display poor temporal judgment accuracy when prevented to count covertly or overtly (15), and several studies have reported that movements improve the perception of recurring intervals (16–18). In nonhuman primates, birds, and rodents, early investigations using a variety of suprasecond-long motor timing tasks (typically with an interval longer than 10 s) reported that their performance was associated with the production of relatively stereotyped chains of actions between the operant responses delivering the reward (19–22). These so-called collateral behaviors (sometimes referred to as superstitious, adjunctive, or interim behaviors) have also been observed in humans, but, as their names indicate, it is usually assumed that they do not mediate timing per se. Indeed, the duration and order of the collateral behaviors can substantially vary during suprasecond-long intervals, making them relatively unreliable external clocks (22). Consequently, even in the few theories that considered that collateral behaviors are important for timing, their duration and transition times were assumed to be largely determined by some sort of internal clock (23, 24) or habitual processes (25, 26). Alternatively, the variable collateral behaviors reported during time intervals could reflect attempts of the animals to produce a long motor sequence (routine) to facilitate timing. Importantly, motor routines do not require an internal clock but can be driven by associative learning (27). Because neither the amount of procedural information that an animal can store nor working memory is infinite, it can be expected that motor routines can only facilitate timing when intervals are relatively short. Recently, the continuous video monitoring of rats performing different timing tasks with shorter intervals demonstrated that proficient animals developed highly stereotyped motor routines (28, 29), raising the possibility that timing accuracy could be improved by the performance of motor routines. Still, it is possible that animals can rely on disembodied/internal time representations when their environment is not conducive to the use of motor routines.
To investigate whether motor routines contribute to timing accuracy, we used a 90-cm-long motorized treadmill, in which rats had to wait for a 7-s-long time interval (or delay) before approaching a reward port located at the front of the treadmill (30). We observed that rats took advantage of the task parameters (treadmill speed, direction, and length) to learn, by trial and error, a simple wait-and-run motor routine whose execution resulted in their front–back–front trajectory on the treadmill and an accurate timing of their reward approaches. By manipulating the duration of the waiting time and the speed of the treadmill (its magnitude and reliability across trials) and interfering with the initiation of this simple motor routine, we created conditions that prevented its development or usage. In such conditions, rats were always less accurate in timing their reward approaches. We conclude that, in our task, the level of timing accuracy depends, on the one hand, on the animals’ ability to move and to learn and, on the other hand, on physical features of the environment that afford cues that facilitate the execution of simple routines adapted to the temporal challenge faced by the animal.
Results
To investigate how animals adapt their behavior to temporal regularities in their environment, we challenged Long–Evans rats in a treadmill-based behavioral assay that required them to wait for 7 s (goal time [GT]) before approaching a “reward area” located at the front of the treadmill (SI Appendix, Fig. S1A and Methods). The front wall of the treadmill was equipped with a device delivering rewards (a drop of sucrose solution). An infrared beam, located at 10 cm from this device, defined the limit of the reward area and was used to record the first time, relative to trial onset, the animals entered the reward area (entrance time []). Animals were first familiarized with the apparatus and trained to lick drops of the sucrose solution while the treadmill was immobile. Then, rats were trained once a day for 55 min in the proper waiting task. Each daily session contained 130 trials interleaved with resting periods of 15 s (intertrial, motor off). Each trial started by turning the treadmill motor on at a fixed speed of 10 cm/s. The conveyor belt moved toward the rear of the treadmill. The infrared beam was not active during the first 1.5 s (), to give the opportunity to the animals to leave (passively or actively) the reward area at the beginning of each trial. Three trial types were defined based on the in the reward area relative to the GT: correct trials in which , error trials in which , and omission trials in which the infrared beam was not interrupted in 15 s (SI Appendix, Fig. S1 B–D). A penalty period of extra running started when the animals erroneously crossed the infrared beam before GT (), and its duration varied between 10 s and 1 s, according to the error magnitude (SI Appendix, Fig. S1 C, Inset). Thus, to maximize reward collection and minimize running time, animals should cross the infrared beam just after the GT.
During the first training sessions, animals started most trials in the front of the treadmill, mostly ran in the reward area, and interrupted the infrared beam before the GT (Movie S1 and Fig. 1 A, Top and C, Left). Progressively, across training sessions, animals waited longer, and, after 15 sessions, they reliably entered the reward area just after the GT (Fig. 1B). Interestingly, for a large majority of animals, the ability to precisely wait 7 s before entering the reward area was associated with the performance of a stereotyped motor sequence on the treadmill (Movie S2 and Fig. 1 A, Bottom and C, Right). First, animals began each trial in the reward area. Then, when the treadmill was turned on, they remained largely still while being pushed away from the reward area until they reached the rear wall. Finally, after reaching the rear wall, they ran across the treadmill, without pause, and crossed the infrared beam. The percentage of trials for which animals used this motor routine increased during learning (Fig. 1D). Even though a strong preference for the reward area was observed for both correct and error trials, the probability of starting a trial in the frontal portion of the treadmill was higher for correct trials compared to error trials (Fig. 1E), a tendency that developed progressively during training (SI Appendix, Fig. S2). In addition, if an animal started a trial in the frontal portion of the treadmill, the probability of reaching the back of the treadmill was higher in correct trials than in error trials (Fig. 1F), confirming that correct trials were associated with the animals following the wait-and-run routine and effectively reaching the back of the treadmill before running forward toward the reward area. However, a significant fraction of the animals (14/54) did not develop such a strategy (Fig. 1 C, Right and SI Appendix, Fig. S3A). Compared with these animals, those following, regularly, the wait-and-run routine entered the reward area later, and displayed reduced variability and an increased percentage of correct trials (SI Appendix, Fig. S3 B–D). While we cannot exclude that animals that did not follow the wait-and-run routine also used a more subtle stereotyped motor routine not captured by tracking the average body positions along the treadmill length, the above results suggest that following a front–back–front trajectory through the “wait-and-run” routine is the most reliable strategy to accurately enter the reward area just after 7 s.
Our results indicate that animals naturally took advantage of cues in their environment to perform a motor routine during the time interval they had to respect. Still, it is unclear whether the level of timing accuracy is truly facilitated by such a routine or whether animals can rely on a disembodied perception of time when the environment is not conducive to the usage of routines. To address these questions, we examined how accurately animals respected the GT, when distinct task parameters were modified such as to prevent the use of this simple wait-and-run motor sequence. First, we trained a new group of rats in a version of the task in which, for each trial, the speed of the treadmill was selected randomly from a uniform distribution between 5 cm/s and 30 cm/s (Fig. 2A). We found that, during the course of training, these animals consistently failed to wait as long as the animals trained in the control version of the task (“control” group; Fig. 2B). Still, the average trajectories of animals extensively trained in this “variable speed” condition revealed that they followed a front–back–front trajectory (Fig. 2C). Accordingly, the probability of performing a correct trial, given different speeds, fell rapidly from 5 cm/s to 15 cm/s and was lowest for the fastest treadmill speeds (Fig. 2D). Indeed, when the treadmill speed was fast, performing the wait-and-run strategy resulted in error trials, as animals reached the back region of the treadmill earlier than when the treadmill speed was slow. We also found that the probability of entering the reward area at the GT s sharply peaked for a treadmill speed (11.5 cm/s) that is suitable to perform the wait-and-run motor sequence (Fig. 2D). Finally, when rats that were extensively trained in the control version of the task underwent a single probe session with variable speeds (Fig. 2E), all measures of performance dropped significantly (Fig. 2F). Examining the probability of correct trials and accurate s ( s) given the treadmill speed suggested that animals kept performing the wait-and-run routine they previously learned in the control condition (compare Fig. 2 G and D).
In the control condition, 80% of correct trials started while animals were positioned in the reward area (Fig. 1E). We next investigated whether accurate timing was dependent on this initial position. We trained a group of rats in a modified version of the task that penalized them when they started the trials in the front region of the treadmill. This was done by activating the infrared beam as soon as the motor was turned on (in the control condition, the infrared beam was inactive during a timeout period that lasted 1.5 s after treadmill onset). In this “no-timeout” condition, error trials corresponded to s occurring between 0 s and 7 s after motor onset (Fig. 3A). Animals trained in this condition never reached the level of timing accuracy displayed by animals in the control condition (Fig. 3B). Still, no-timeout animals followed a front–back–front trajectory (Fig. 3C), and correct trials were associated with the animals starting the trials just behind the infrared beam (Fig. 3D). The stereotyped reliance on the wait-and-run strategy to facilitate accurate timing was also demonstrated by the fact that rats extensively trained in control condition kept performing the exact same trajectory when tested during a single probe session under no-timeout condition, leading to a sharp decrease in performance (Fig. 3 E–G).
We next examined how animals behaved when the GT was set to 3.5 s (Fig. 4), a condition in which the performance of the wait-and-run strategy would lead to late s (and smaller rewards) because it takes up to 8 s for the animals to passively travel from the front to the rear portion of the treadmill. Animals successfully entered the reward area after 3.5 s and reduced their variability across training sessions (Fig. 4A), but, as a group, they displayed an increased variability compared to animals trained in the control condition, with GT set to 7 s (Fig. 4E). From the averaged trajectories of “short GT” animals measured once their performance plateaued, it appeared that three subjects out of seven followed a front–back–front trajectory by running toward the rear portion of the treadmill. The other four animals remained still when the treadmill was turned on and tried to run forward before reaching the rear wall (Fig. 4B). Interestingly, after training, in 67% of the error trials, the rats started running forward before reaching the middle of the treadmill (Fig. 4C; compare with red histogram in Fig. 1F). Conversely, after initiating a trial in the reward area, the probability of visiting a deeper portion of the treadmill was much stronger in correct than error trials, reinforcing the idea that timing accuracy was improved by exploiting the most salient physical features of the environment (Fig. 4C). Accordingly, the three rats that actively performed the front–back–front trajectory were less variable than those that passively stayed still before running toward the reward area from the middle of the treadmill (Fig. 4E, individual data points, same color code as in Fig. 4B). In addition, among animals trained in the short GT condition, we found that the magnitude of the backward displacement on the treadmill was negatively correlated with variability (, Pearson’s correlation). In the short GT condition, animals became proficient more rapidly than in the control condition (compare Fig. 4A with Fig. 1B). Could the increased variance when the GT is 3.5 s be explained by the fact that the task is easier in this condition and that animals do not need to be very precise? To test this possibility, we increased the penalty for early s and decreased reward size for late s. In this “sharp reward” condition, the performance of the animals trained in the short GT was even more variable (Fig. 4 D–E). This result confirms that, under short GT condition, animals are less accurate in timing their entrance in the reward area than when the GT was set to 7 s. Finally, another group of animals was trained with GT set to 3.5 s and treadmill speed at 20 cm/s (i.e., twice as fast, such that following the front–back–front trajectory through the wait-and-run motor sequence would lead to s close to the GT; Fig. 4F). These animals displayed reduced variability compared to animals trained at 10 cm/s, and, after treadmill onset, they stayed immobile until reaching the end of the treadmill, similar to animals trained in the control condition (Fig. 4 G and H).
The above results suggest that, in a task requiring animals to produce a motor response according to a fixed temporal constraint, the possibility to perform a stereotypical motor sequence adapted to salient features of the environment (here, taking advantage of the full treadmill length and its physical boundaries) improves temporal accuracy. To further investigate this idea, we trained another group of animals in a version of the task in which the treadmill was never turned on (trial onset was signaled by turning the ambient light on). In this condition, animals were less accurate compared to animals trained in the control condition (Fig. 5 A and B). On average, animals reached the reward area later and later across sessions but displayed a constant high variability in (Fig. 5C). We also noticed that correct trials preferentially occurred when animals crossed the treadmill from the rear wall to the reward area (Movie S3 and Fig. 5 A, D, and E). Accordingly, after extensive training, a robust correlation was observed on a session by session basis between the percentage of correct trials and displacement of the animal on the treadmill (Fig. 5F). Moreover, the probability of a correct trial increased for higher values of displacement (Fig. 5G).
Lastly, animals trained in the immobile treadmill condition for several weeks were challenged in the control condition (i.e., by simply setting the treadmill speed at 10 cm/s). These animals improved their behavior at the same pace and with the same wait-and-run routine as naïve animals (SI Appendix, Fig. S4). Thus, animals that previously learned to wait in one version of the task did not learn faster than naïve animals when challenged in a second version of the task with distinct movement requirement but identical GT, demonstrating, again, that task proficiency relied primarily on the acquisition of a motor sequence rather than an abstract knowledge of time.
Altogether, our results suggest that rats adapted to the 7-s waiting time constraint by taking advantage of the space available on the treadmill to converge to the wait-and-run motor routine as a recipe of task proficiency. We assumed that animals developed this strategy through a trial-and-error learning process without disembodied representation of time. To assess the validity of this assumption, we developed a reinforcement learning model to simulate how artificial agents would learn a simplified version of our task (see SI Appendix, Methods for details). Briefly, the treadmill was divided into five regions (with the most frontal region corresponding to the reward area), and time was also discretized. At each time step, the agent had to choose one of the three following actions: stay immobile to move one region backward, run at the treadmill speed to remain in the same region, or run at twice the treadmill speed to move forward (Fig. 6 A, Top). Critically, each region corresponded to two different states, depending on whether the agent had reached the back of the treadmill or not (Fig. 6 A, Bottom). The position of the agent at the beginning of each trial was biased toward the reward area, to mimic the tendency of rats, following habituation, to remain close to the reward area during intertrials. We found that, across training sessions, an artificial agent waited longer and longer and reduced its ET variability by performing the front–back–front trajectory (Fig. 6 B and C). A similar strategy was developed by agents endowed with different learning parameters and exploration/exploitation rates (SI Appendix, Fig. S5). After learning, agents performed the front–back–front routine independently of their initial positions. Thus, they arrived in the reward area too early when their initial position was near the middle or the end of the treadmill, similar to animals. Next, we examined how artificial agents would perform the same task if they had access to the time elapsed since trial onset (Fig. 6D). Such agents also learned, progressively, to enter the reward area at the right time and reduced their variability (Fig. 6E). However, after learning, they did not perform the front–back–front trajectory. Rather, they mainly stayed behind the infrared beam and were capable of respecting the GT independently of their initial position (Fig. 6F). Finally, different agents developed idiosyncratic position trajectories (even if they remained close to the reward area during most of the trial duration), due to stochastic variations in their learning dynamics (Fig. 6G). These results support the hypothesis that the behavior developed by most rats in our task is consistent with the usage of a motor sequence adapted to the agent’s environment, rather than a disembodied time representation.
Discussion
In this study, we challenged rats in a task in which they had to wait for 7 s before approaching a reward area located at the front of a motorized treadmill. A large majority of rats took advantage of the task parameters (waiting duration and treadmill speed, direction, and length) to develop a simple wait-and-run routine whose execution resulted in accurate timing of their reward approaches. Noticeably, when proficient animals occasionally failed to follow this routine, the timing of their approaches was systematically poor. In a second series of experiments, we manipulated the speed of the treadmill or key parameters of this waiting task. We found that rats were less accurate in timing their reward approaches when they could not use the wait-and-run routine because of constant variability in the treadmill speed (Fig. 2) or when the task parameters did not allow them to transform salient features of the treadmill into cues that adaptively constrain the execution of the routine (Figs. 3–5).
Altogether, the results of our study indicate that rats were less accurate when they were forced to time their reward approach independently of their movements on the treadmill (i.e., without relying on a motor routine). However, whether rats are able to time without moving (i.e., in their head) was not directly addressed by our experimental design. Indeed, in most versions of our task, the motorized treadmill forced the animals to move during the waiting period. In the only version of the task in which the treadmill remained immobile, the timing performance was the poorest, although there was no incentive for the animals to remain immobile. The fact that nonhuman primates restrained on a chair successfully perform timing tasks seems to indicate that brains can tell time independent of movements (31–33). Still, it is hard to rule out that timing accuracy in this type of task is not facilitated by subtle movements or muscle contractions, as shown in some early works (34). In head-restrained apparatus, rodents continuously perform a variety of movements (orofacial and tail movements, postural adjustments). Recently, it has been shown that these movements strongly influence brain-wide neuronal dynamics, even in the context of the perception of a simple stimulus or during decision-making (35, 36). It has also been reported that sensory discrimination decisions could be decoded by small head rotations of rats while they maintained their nose in a nosepoke during a delay period between stimuli presentation and discriminative operant response (37). Thus, even when animals are relatively immobile, their decision may, in fact, rely on their ability to embody the task rules into stereotyped interactions with their environment. Attempting to train rodents to perform timing tasks while preventing them from moving may seem to be the most direct way to demonstrate the presence of disembodied timing mechanisms. But, as discussed above, it is unclear whether this approach will yield conclusive results (and see also ref. 14).
It could also be argued that rats never understood the temporal dimension of the task and treated it as a motor learning challenge. Firstly, we agree that animals persisted in trying to solve our task using a motor strategy, but whether this could have been predicted before performing the experiments is not clear. Indeed, our experimental design was influenced by models postulating that timing depends on neuronal timekeeping mechanisms that operate independently of the movements occurring during the waiting time (e.g., ref. 7). Such models predicted that rats could solve this type of task by continuously running at the treadmill speed and accelerate to enter the reward area once 7 s had elapsed since trial onset. This logic was confirmed using a simple simulation of the task in the reinforcement learning theory framework (Fig. 6). Secondly, there will always be a limit to the inferences that an experimenter can make regarding the mental state of a behaving animal or its internal model of a task. If the usage of sentences such as “subjects estimate (or measure) time” are valid for humans who have an abstract notion of time, it may not be meaningful from the viewpoint of rodents (38). In fact, even in tasks explicitly designed to require rats to estimate the duration of sensory cues, it is unclear whether they do so (whatever this could mean for them), and video quantification supported the hypothesis that the animals’ performance in this type of task is also facilitated by overt timing strategy (see below).
Another issue regarding our conclusion is whether it is relevant beyond the specifics of our experimental protocols. Interestingly, in a study in which rats had to perform two lever presses interleaved by 700 ms, each animal slowly developed an idiosyncratic motor sequence (e.g., 1) first press on the lever with the left paw; 2) touching the wall above the lever with the right paw; 3) second press on the lever with the left paw), lasting precisely 700 ms (29). Importantly, virtually all animals developed such interactive motor strategy, even if, compared to our task, the time interval was much shorter ( s), the terminal operant response (a single lever press) was perhaps more obvious than ours, and the animals did not need to move between the presses. In one of the rare studies that continuously recorded and quantified the full body dynamics of rats performing a sensory duration categorization choice task, it was reported that animals developed highly stereotyped motor sequences during presentation of the sensory cues and that perceptual report of the animals could be predicted by these motor sequences (28). This result is reminiscent of an earlier study showing that the prediction of rats’ temporal judgment (a 6-s-long versus a 12-s-long luminous signal) was always better if based on the collateral behavior performed by the animal at the end of the signal than if based on time (39). Thus, in such temporal discrimination tasks, the choice of the animals might be primarily determined by what the animal is doing when a sensory cue disappears, rather than by a disembodied neuronal representation of the duration of that cue. Altogether, these studies support the idea that animals resort to motor strategies to adapt to temporal constraints in a wide range of timing tasks.
An important question raised by our study is why animals seem to default to embodied/situated strategies to solve tasks with temporal constraints, instead of relying on a disembodied neuronal representation of time. A first potential line of explanation is related to fundamental principles of behavioral control conserved throughout vertebrate evolution (40), and the type of temporal challenges met by animals in their natural environment (41). Indeed, outside laboratory experiments, timing is almost always implicit and occurs while animals perform other actions reflecting their goals as living organisms (foraging while avoiding predators, reproducing). To fulfill such goals, animals must interact with the world in order to explore and exploit available opportunities from which successful ones will be reinforced. The acquisition of idiosyncratic motor sequences to respect a short delay between two lever presses (29), gauge the duration of a stimuli (28), or remember a sensory stimulus during a delay (37), or the development of the wait-and-run motor strategy in our task, are well accounted for by the natural tendency of animals to explore their environment until a solution is found (e.g., obtention of food). An important aspect of our study is to demonstrate that the level of timing accuracy is not just dependent on driving muscles in a stereotyped order. If this were the case, when the treadmill was immobile or when the GT was shortened, animals would have used any kind of motor routine to time their reward area approaches with high-level accuracy. What seems most critical to improve timing is the ability of rats to interact with physical features of the treadmill that could provide cues to effectively constrain the execution of the wait-and-run routine and drive the animals in the reward area just after 7 s. In our task, those cues were the extremities of the treadmill which, given the treadmill speed, were at the right distance to develop the wait-and-run strategy. In that sense, our results provide experimental support for the concept of affordances (42) and a view of cognition described in terms of agent–environment interactions rather than in terms of representations (43, 44). The large interindividual variability reported in ref. 29 is likely to reflect the multiple possibilities of brief interactive sequences that rats can perform with salient features of the wall in which the lever was inserted, during the 700-ms-long delay. If the interval between the two lever presses had been longer (e.g., several seconds), it is likely that rats would have started using locomotion-based navigation sequence, such as running back and forth to the opposite wall between two presses, as shown in earlier studies (22). The animals’ ability to conform to very long intervals () is likely to be limited by procedural and working memory capacity. Thus, learning and executing longer sequences of actions is expected to lead to higher temporal variability. While we could not find studies that precisely quantified timing performance and movement variability across a range of intervals, early works showed that collateral behaviors during long intervals ( s) became less structured and often consisted of the repetitive execution of a single action (22, 45). Another reason why interactive embodied strategies seem to be preferred for accurate timing might be related to the intrinsically irregular activity of cortical neurons in behaving animals (46). In neuronal networks with recurrent and complex connectivity, such irregularity can lead to chaotic dynamics (47), a phenomenon that will be amplified by small perturbations (13). This could prevent reliable temporal representation by self-sustained dynamics, especially in the case of rodents, whose restlessness has been shown to strongly influence cortical and subcortical activity (36). One solution to this problem could be to constrain the initial conditions that trigger a neuronal sequence and provide checkpoints, through sensory-evoked deterministic patterns of neuronal activity. This is exactly what is provided when animals perform stereotyped interaction with their environment from the beginning to the end of a fixed interval.
That timing could be primarily embodied and situated might seem contradictory to our inner rooted feeling of time and the abundant literature reporting neuronal correlates of time (48–54). However, just because time can be decoded from neuronal activity does not mean that animals measure time with these representations, like we read time from a clock (38, 55, 56). This raises the intriguing question of whether humans (and maybe nonhuman primates), on the one hand, and other animals, on the other hand, have a totally distinct relationship with time (i.e., respectively, explicit and implicit). The answer might not be so simple, as it has been recently proposed that the explicit perception of time in humans may also be constructed implicitly through the association between the duration of an interval and its sensorimotor content (57).
Materials and Methods
All experimental procedures were conducted in accordance with standard ethical guidelines (European Communities Directive 86/60-EEC) and were approved by the relevant national ethics committee (Ministère de l’Enseignement Supérieur et de la Recherche, France, Authorizations #00172.01 and #16195).
Behavior.
One hundred eleven male Long–Evans rats were used (the number of animals in each experimental condition is shown in its respective figure). Before being trained in the different versions of the timing task, animals were habituated to the treadmill for three to five daily sessions of 30 min, while the treadmill remained immobile and a drop of reward was delivered every minute (see SI Appendix, Methods for detailed information on the subjects, apparatus, and behavioral procedures).
Data Analysis.
Position information derived from video tracking (sampling rate 25 frames per second) was scaled to the treadmill length, and smoothed (Gaussian kernel, s). Trials were considered routine if all of the following three conditions were met: 1) The animal started the trial in the front (initial position ), 2) the animal reached the rear portion of the treadmill after trial onset (maximum trial position ), and 3) the animal completed the trial (i.e., they crossed the infrared beam). The same criteria were applied to the median trajectories after training (session #30) to classify animals into two groups: those that used the front–back–front trajectory and those that did not (SI Appendix, Fig. S3).
All statistical comparisons were performed using resampling methods (permutation test and bootstrapping) inspired from ref. 58 (see SI Appendix, Methods for details).
Reinforcement Learning Models.
We took advantage of the Markov Decision Process formalism to analyze how artificial agents learned to perform a simplified version of the treadmill task. A simple Q-learning algorithm was used to model how agents, with or without access to information on the elapsed time since the beginning of the trial, learned an optimal policy (ref. 59; see SI Appendix, Methods for details).
Data Availability
All of the Jupyter Notebooks, as well as the raw data necessary for full replication of the figures and videos, are publicly available via the Open Science Framework (https://osf.io/7s2r8/?view_only=7db3818dcf5e49e88d708b2597a21956).
Supplementary Material
Acknowledgments
We thank Drs. I. Bureau and J. Epsztein for critical reading of the manuscript. This work was supported by European Research Council (Grant ERC-2013-CoG–615699–NeuroKinematics; D.R.) and a Centuri postdoctoral fellowship (S.S.).
Footnotes
The authors declare no competing interest.
This article is a PNAS Direct Submission.
Data deposition: All of the Jupyter Notebooks, as well as the raw data necessary for full replication of the figures and videos, are publicly available via the Open Science Framework (https://osf.io/7s2r8/?view_only=7db3818dcf5e49e88d708b2597a21956).
This article contains supporting information online at https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1921226117/-/DCSupplemental.
References
- 1.Kacelnik A., Brunner D., Timing and foraging: Gibbon’s scalar expectancy theory and optimal patch exploitation. Learn. Motiv. 33, 177–195 (2002). [Google Scholar]
- 2.Gallistel C. R., The Organization of Learning (MIT Press, 1990). [Google Scholar]
- 3.Balsam P. D., Gallistel C. R., Temporal maps and informativeness in associative learning. Trends Neurosci. 32, 73–78 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Nobre A. C., van Ede F., Anticipated moments: Temporal structure in attention. Nat. Rev. Neurosci. 19, 34–48 (2018). [DOI] [PubMed] [Google Scholar]
- 5.Gibbon J., Scalar expectancy theory and weber’s law in animal timing. Psychol. Rev. 84, 279–325 (1977). [Google Scholar]
- 6.Gibbon J., Church R. M., Meck W. H., Scalar timing in memory. Ann. N. Y. Acad. Sci. 423, 52–77 (1984). [DOI] [PubMed] [Google Scholar]
- 7.Buhusi C. V., Meck W. H., What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765 (2005). [DOI] [PubMed] [Google Scholar]
- 8.Wittmann M., The inner sense of time: How the brain creates a representation of duration. Nat. Rev. Neurosci. 14, 217–223 (2013). [DOI] [PubMed] [Google Scholar]
- 9.Allman M. J., Teki S., Griffiths T. D., Meck W. H., Properties of the internal clock: First-and second-order principles of subjective time. Annu. Rev. Psychol. 65, 743–771 (2014). [DOI] [PubMed] [Google Scholar]
- 10.Kononowicz T. W., Van Rijn H., Meck W. H., “Timing and time perception: A critical review of neural timing signatures before, during, and after the to-be-timed interval” in Stevens’ Handbook of Experimental Psychology and Cognitive Neuroscience (Wiley, 2018), vol 1, pp. 1–38. [Google Scholar]
- 11.Goel A., Buonomano D. V., Timing as an intrinsic property of neural networks: Evidence from in vivo and in vitro experiments. Phil. Trans. Biol. Sci. 369, 20120460 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Paton J. J., Buonomano D. V., The neural basis of timing: Distributed mechanisms for diverse functions. Neuron 98, 687–705 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Buonomano D. V., Laje R., Population clocks: Motor timing with neural dynamics. Trends Cognit. Sci. 14, 520–527 (2010). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gomez-Marin A., Ghazanfar A. A., The life of behavior. Neuron 104, 25–36 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Rattat A. C., Droit-Volet S., What is the best and easiest method of preventing counting in different temporal tasks?. Behav. Res. Methods 44, 67–80 (2012). [DOI] [PubMed] [Google Scholar]
- 16.Su Y. H., Pöppel E., Body movement enhances the extraction of temporal structures in auditory sequences. Psychol. Res. 76, 373–382 (2012). [DOI] [PubMed] [Google Scholar]
- 17.Manning F., Schutz M., “Moving to the beat” improves timing perception. Psychon. Bulletin Rev. 20, 1133–1139 (2013). [DOI] [PubMed] [Google Scholar]
- 18.Wiener M., Zhou W., Bader F., Joiner W. M., Movement improves the quality of temporal perception and decision-making. eNeuro 6, ENEURO.0042-19.2019 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Skinner B. F., ‘Superstition’ in the pigeon. J. Exp. Psychol. 38, 168–172 (1948). [DOI] [PubMed] [Google Scholar]
- 20.Wilson M. P., Keller F. S., On the selective reinforcement of spaced responses. J. Comp. Physiol. Psychol. 46, 190–193 (1953). [DOI] [PubMed] [Google Scholar]
- 21.Falk J. L., The nature and determinants of adjunctive behavior. Physiol. Behav. 6, 577–588 (1971). [DOI] [PubMed] [Google Scholar]
- 22.Richelle M., Lejeune H., Time in Animal Behaviour (Pergamon Press, 1980). [Google Scholar]
- 23.Killeen P. R., Fetterman J. G., A behavioral theory of timing. Psychol. Rev. 95, 274–295 (1988). [DOI] [PubMed] [Google Scholar]
- 24.Machado A., Learning the temporal dynamics of behavior. Psychol. Rev. 104, 241–265 (1997). [DOI] [PubMed] [Google Scholar]
- 25.Dragoi V., Staddon J., Palmer R. G., Buhusi C. V., Interval timing as an emergent learning property. Psychol. Rev. 110, 126–144 (2003). [DOI] [PubMed] [Google Scholar]
- 26.Staddon J., Higa J., Time and memory: Towards a pacemaker-free theory of interval timing. J. Exp. Anal. Behav. 71, 215–251 (1999). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Berns G. S., Sejnowski T. J., A computational model of how the basal ganglia produce sequences. J. Cognit. Neurosci. 10, 108–121 (1998). [DOI] [PubMed] [Google Scholar]
- 28.Gouvêa T. S., Monteiro T., Soares S., Atallah B. V., Paton J. J., Ongoing behavior predicts perceptual report of interval duration. Front. Neurorob. 8, 10 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Kawai R., et al. , Motor cortex is required for learning but not for executing a motor skill. Neuron 86, 800–812 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Rueda-Orozco P. E., Robbe D., The striatum multiplexes contextual and kinematic information to constrain motor habits execution. Nat. Neurosci. 18, 453 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Leon M. I., Shadlen M. N., Representation of time by neurons in the posterior parietal cortex of the macaque. Neuron 38, 317–327 (2003). [DOI] [PubMed] [Google Scholar]
- 32.Mita A., Mushiake H., Shima K., Matsuzaka Y., Tanji J., Interval time coding by neurons in the presupplementary and supplementary motor areas. Nat. Neurosci. 12, 502–507 (2009). [DOI] [PubMed] [Google Scholar]
- 33.Wang J., Narain D., Hosseini E. A., Jazayeri M., Flexible timing by temporal scaling of cortical responses. Nat. Neurosci. 21, 102–110 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hodos W., Ross G. S., Brady J. V., Complex response patterns during temporally spaced responding. J. Exp. Anal. Behav. 5, 473–479 (1962). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Stringer C., et al. , Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, eaav7893 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Musall S., Kaufman M. T., Juavinett A. L., Gluf S., Churchland A. K., Single-trial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677–1686 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cowen S. L., McNaughton B. L., Selective delay activity in the medial prefrontal cortex of the rat: Contribution of sensorimotor information and contingency. J. Neurophysiol. 98, 303–316 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Gomez-Marin A., A clash of umwelts: Anthropomorphism in behavioral neuroscience. Behav. Brain Sci. 42, e229 (2019). [DOI] [PubMed] [Google Scholar]
- 39.Fetterman J. G., Killeen P. R., Hall S., Watching the clock. Behav. Process. 44, 211–224 (1998). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Cisek P., Resynthesizing behavior through phylogenetic refinement. Atten. Percept. Psychophys. 81, 2265–2287 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.van Rijn H., Towards ecologically valid interval timing. Trends Cognit. Sci. 22, 850–852 (2018). [DOI] [PubMed] [Google Scholar]
- 42.Gibson J. J., The Ecological Approach to Visual Perception: Classic Edition (Psychology, 2014). [Google Scholar]
- 43.Pfeifer R., Bongard J., How the Body Shapes the Way We Think: A New View of Intelligence (MIT Press, 2006). [Google Scholar]
- 44.Chemero A., Radical Embodied Cognitive Science (MIT Press, 2011). [Google Scholar]
- 45.Laties V. G., Weiss B., Weiss A. B., Further observations on overt “mediating” behavior and the discrimination of time 1. J. Exp. Anal. Behav. 12, 43–57 (1969). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Shadlen M. N., Newsome W. T., The variable discharge of cortical neurons: Implications for connectivity, computation, and information coding. J. Neurosci. 18, 3870–3896 (1998). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Van Vreeswijk C., Sompolinsky H., Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996). [DOI] [PubMed] [Google Scholar]
- 48.Kraus B. J., Robinson R. J. II, White J. A., Eichenbaum H., Hasselmo M. E., Hippocampal ”time cells”: Time versus path integration. Neuron 78, 1090–1101 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Bakhurin K. I., et al. , Differential encoding of time by prefrontal and striatal network dynamics. J. Neurosci. 37, 854–870 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Gu B. M., Kukreja K., Meck W. H., Oscillation patterns of local field potentials in the dorsal striatum and sensorimotor cortex during the encoding, maintenance, and decision stages for the ordinal comparison of sub-and supra-second signal durations. Neurobiol. Learn. Mem. 153, 79–91 (2018). [DOI] [PubMed] [Google Scholar]
- 51.Mello G. B., Soares S., Paton J. J., A scalable population code for time in the striatum. Curr. Biol. 25, 1113–1122 (2015). [DOI] [PubMed] [Google Scholar]
- 52.Villette V., Malvache A., Tressard T., Dupuy N., Cossart R., Internally recurring hippocampal sequences as a population template of spatiotemporal information. Neuron 88, 357–366 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Pastalkova E., Itskov V., Amarasingham A., Buzsáki G., Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Heys J. G., Dombeck D. A., Evidence for a subcircuit in medial entorhinal cortex representing elapsed time during immobility. Nat. Neurosci. 21, 1574–1582 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Buzsáki G., Llinás R., Space and time in the brain. Science 358, 482–485 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Buzsáki G., Tingley D., Space and time: The hippocampus as a sequence generator. Trends Cognit. Sci. 22, 853–869 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Coull J. T., Droit-Volet S., Explicit understanding of duration develops implicitly through action. Trends Cognit. Sci. 22, 923–937 (2018). [DOI] [PubMed] [Google Scholar]
- 58.Fujisawa S., Amarasingham A., Harrison M. T., Buzsáki G., Behavior-dependent short-term assembly dynamics in the medial prefrontal cortex. Nat. Neurosci. 11, 823 (2008). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Sutton R. S., Barto A. G., Introduction to Reinforcement Learning (MIT Press, 1998), vol. 135. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
All of the Jupyter Notebooks, as well as the raw data necessary for full replication of the figures and videos, are publicly available via the Open Science Framework (https://osf.io/7s2r8/?view_only=7db3818dcf5e49e88d708b2597a21956).