Time and the Brain: How Subjective Time Relates to Neural Time

Abstract

Most of the actions our brains perform on a daily basis, such as perceiving, speaking, and driving a car, require timing on the scale of tens to hundreds of milliseconds. New discoveries in psychophysics, electrophysiology, imaging, and computational modeling are contributing to an emerging picture of how the brain processes, learns, and perceives time.

Brains have a difficult problem to solve. Signals from different modalities are processed at different speeds in distant neural regions, but to be useful to the organism as a whole, these signals must become aligned in time and correctly tagged to outside events (Eagleman, 2005b). Understanding the timing of events, such as a motor act followed by a sensory consequence, is critical for moving, speaking, determining causality, and decoding the barrage of temporal patterns at our sensory receptors.

Despite its importance to behavior and perception, the neural bases of time perception remain shrouded in mystery. Scattered confederacies of investigators have been interested in time for decades, but only in the past few years has a concerted effort been applied to old problems. Now, experimental psychology is striving to understand how animals perceive and encode temporal intervals, whereas physiology, functional magnetic resonance imaging (fMRI), and EEG unmask how neurons and brain regions underlie temporal computations. In this review, we sketch parts of an emerging picture and highlight remaining confusions about time in the brain. Some of the overarching questions are as follows: How do brains encode and decode information that streams in through time? How are signals entering various brain regions at varied times coordinated with one another? What is the temporal precision with which perception represents the outside world? How are intervals, durations, and sequences coded in the brain? What factors (causality, attention, adrenaline, or eye movements) influence temporal judgments and why? Does the brain constantly recalibrate its time perception? In this mini-symposium, we illustrate different experimental approaches that attempt to shine light on these questions and others.

Psychophysics

Much of what we know about time in the brain comes from psychophysical experiments. One class of studies involves ways in which time perception distorts: for example, during brief, dangerous events, such as car accidents and robberies, many people report that events pass in slow motion as if time slowed down. Recent studies have been able to quantify distorted time judgments during rapid eye movements (Eagleman, 2005a; Morrone et al., 2005) or after adaptation to flickering or moving stimuli (Johnston et al., 2005; Kanai and Verstraten, 2005).

Although such examples are probably related to very low-level processes, other investigators have reported time distortions they believe are related to attentional shifts (Tse et al., 2004). For example, Tse and colleagues have shown that, when many stimuli are shown in succession, a low-probability “oddball” stimulus in the series tends to last subjectively longer than the high-probability stimuli, even when they are presented for the same objective duration. This is true in both vision and audition. The dynamics of the distortion (it takes some hundreds of milliseconds to reach its peak effect) support the view that attentional orienting underlies distortions in perceived duration. Tse et al. propose that a simple “counter” model can account for distortions of time perception over short timescales. That is, it is proposed that, instead of an accurate internal “clock,” the brain simply has access to the approximate constant rate of its own information processing. Imagine that one bit of internal information processed is interpreted as one unit of objective time having passed. When the rate of internal information processing suddenly goes up to two bits per unit of objective time (as when one pays more attention because of an imminent crash into another car), a counter would count more bits. If the assessment of duration by the brain is the result of the output of such a counter, it would come to the wrong conclusion that more objective time had passed, creating the illusion that time and motion had slowed down.

Although the illusion is clear, the idea of a counter is in debate (Buonomano and Karmarkar, 2002; Mauk and Buonomano, 2004). In contrast to a counter that integrates events, the “state-dependent networks” model proposes that the ubiquity of time-varying neuronal properties allows networks to inherently encode temporal information (Buonomano and Merzenich, 1995). In other words, the way the network evolves through time can code for the time itself. Foremost among the time-dependent properties is short-term synaptic plasticity. Buonomano and his colleagues propose that, in the same way that long-term plasticity may provide a memory of a learning experience, short-term plasticity may provide a memory of what happened a few hundred milliseconds ago (Buonomano and Merzenich, 1995). This brief memory is expressed by complex changes in the population response of the network. A prediction from this class of models is that time is not encoded explicitly in a metric. Rather, time is encoded in a temporal pattern. This theoretical prediction has been tested psychophysically by asking subjects to judge an interval that was preceded by a distractor presented at a variable or fixed interval. For short intervals of 100 ms, thresholds were double those for the fixed intervals. This was not the case, however, for long intervals (1 s). These results suggest that, for short intervals, time is encoded as a temporal object in a high-dimensional network state. In contrast, long intervals may be explicitly encoded in a manner consistent with internal clocks. Note that the state-dependent network model suggests that temporal processing is distributed throughout the brain rather than relying on a centralized timing area, as has been suggested in internal clock models. Additional experiments will be needed to cleanly separate the domains of integrator models and state-dependent network models. Understanding the difference will be critical to our search for how brains tell time.

Whatever the mechanisms, a related question is whether brains do anything to keep their timing judgments calibrated so that distortions are the exception rather than the rule. It has been found recently that duration judgments are distorted during slow-motion video sequences of natural biological motion (Eagleman, 2004). That is, a flash presented during a slow-motion sequence of a movie is erroneously perceived as having a shorter duration than an identical flash presented during the same sequence at normal speed. The time-distortion illusion is only found when the future positions of objects in the movie are predictable by Newtonian dynamics. The illusion suggests that the speed of subjective time can be modulated by sensory feedback. That is, predictions about future positions of a moving object are compared against sensory feedback, and the difference can modulate the nervous system to speed or slow perceived time to match the physics of the sensory feedback. Thus, the brain may ease its task of consistent timekeeping by constantly calibrating its time estimation against physical laws in the outside world (Eagleman, 2004).

Another example of time calibration comes from studies of causality: some studies suggest that, when we perceive our actions to cause an event, it seems to occur earlier than if we did not cause it (Haggard et al., 2002; Eagleman and Holcombe, 2002). As above, this illusion may reflect an ongoing temporal calibration. Judging the order of action and sensation is essential for determining causality. Accordingly, the nervous system must be able to recalibrate its expectations about the normal temporal relationship between action and sensation to overcome changing neural latencies. A novel illusion in this domain shows not only that the perceived time of a sensation can change but also that temporal order judgments of action and sensation can become reversed as a result of a normally adaptive recalibration process.

When a fixed delay is consistently injected between the participant's key press and a subsequent flash, adaptation to this delay induced a reversal of action and sensation: flashes appearing at delays shorter than the injected delay were perceived as occurring before the key press (Stetson et al., 2005). This illusion appears to reflect a recalibration of motor-sensory timing that results from a neural previous expectation that sensory consequences should follow motor acts with little delay.

Aside from temporal distortions, another vital clue in our search for understanding time in the brain is the basic temporal limits on perceiving various aspects of the visual world. Holcombe and his colleagues have shown that specialized processors in our visual system allow us to perceive certain changes rapidly, on timescales of a few dozen milliseconds or less (Holcombe and Cavanagh, 2001; Clifford et al., 2004). However, when a specialized detector is not available for a visual timing judgment, the brain shows very poor temporal resolution. As an example of poor resolution, imagine a colored patch alternating between red and green adjacent to bars alternating between horizontal and vertical. Surprisingly, we can only perceive whether red is temporally paired with the horizontal bars or with vertical at rates of slower than six stimuli per second (Holcombe and Cavanagh, 2001). The speed of these sorts of visual judgments seems to rely on slow shifts of attention. As we move forward in studying time perception, it is important to keep in mind that there is no single speed at which the brain processes information. Given the variety of psychophysical effects described above, it is likely that a diverse group of neural mechanisms mediates temporal judgments.

Neural bases and models

We have so far highlighted a growing collection of psychophysical findings that suggest that time judgments can distort, recalibrate, reverse, or have a range of resolutions depending on the stimulus and on the state of the viewer. However, what do we know about the neural bases of these phenomena?

At the level of the behaving animal, experiments in monkeys have shown that posterior parietal neurons can encode signals related to the perception of time. Neurons in lateral intraparietal area (LIP) seem to represent the passage of time relative to a remembered standard duration (Leon and Shadlen, 2003). Janssen and Shadlen (2005) recorded the activity of LIP neurons while monkeys made saccades to peripheral targets after a variable delay period. The timing of the “Go” signal (dimming of the fixation point) was a random value whose probability distribution was fixed throughout a block of trials. The conditional probability of an event given that it has not yet occurred is termed the hazard rate. The subjective hazard rate is a blurred version of the theoretical hazard rate based on the assumption that time is known with uncertainty that scales with elapsed time (Weber's law), a ubiquitous property of time perception. Many LIP neurons modulated their spike rate as a function of elapsed time in a manner that mimicked the subjective hazard rate of the Go signal. Thus, LIP activity appears to signal the animal's subjective perception of time.

Of course, humans also anticipate and deploy their attention when an important event is expected: it is critical for brains to know not only where to place attention, but when. Hemodynamic and electrophysiological imaging studies are uncovering the neural systems involved (Nobre, 2001). Intuition suggests that knowing when something will happen helps us to focus resources at that expected point in time to enhance our behavior. By now, many studies indicate that we can use temporal information flexibly and across multiple sensory modalities to orient attention selectively to specific intervals (Griffin et al., 2001; Lange et al., 2003; Correa et al., 2004). Positron emission tomography and fMRI studies have shown that control of temporal orienting in speeded-response tasks involves brain areas that participate in spatial orienting of attention as well as areas that participate in motor control (Coull and Nobre, 1998; Coull et al., 2000), including posterior parietal cortex, in which cellular correlates of temporal predictability have been identified (Janssen and Shadlen, 2005). Despite the sizeable overlap of brain areas participating in temporal and spatial orienting, the neural mechanisms involved in anticipating and modulating stimulus processing when each type of orienting occurs in isolation can differ substantially (Nobre, 2004). For example, event-related potential studies show that, in most visual attention tasks, temporal expectations alone do not modulate early perceptual analysis of target objects but instead optimize motor-related mechanisms (Miniussi et al., 1999; Griffin et al., 2002). However, when combined, temporal and spatial orienting interact synergistically to potentiate attentional mechanisms during early perceptual analysis (Doherty et al., 2005). Overall, the influence of temporal attention on our perception and action is therefore likely to be much more general than previously considered, and many thus far undetected effects of temporal orienting are likely to be pervasive in the behavioral and neuroscientific literature (Griffin and Nobre, 2005).

Conclusions

It is clear that the brain exploits knowledge of elapsed time to anticipate sensory events and to prepare appropriate actions. Collectively, this research begins to shed light on features of time perception and its neural bases. Timing is critical to almost every behavior we engage in, from neural computation to driving a car to playing piano.

Furthermore, over the past decade, we have come to view certain disorders, such as aphasias and dyslexias, as potentially being disorders of timing rather than disorders of language (Efron, 1963; Merzenich et al., 1996). Other deficits in time perception are found in a variety of disorders such as Parkinson's disease (Riesen and Schnider, 2001), attention-deficit/hyperactivity disorder (Kerns et al., 2001), and schizophrenia (Davalos et al., 2003). As we continue to understand more about time in the brain, we hope to discover other contact points with clinical neuroscience.

The relatively new study of time in the brain is exceptionally cross-disciplinary, blurring the borderlines of many subfields. The investigations in this field have traditionally been loosely bound. However, recent studies are beginning to establish a fertile middle ground in which to combine experimental techniques using electrophysiology, psychophysics, EEG, fMRI, and computational modeling. Few fields enjoy the opportunity to apply so much of the armamentarium of neuroscience to a scientific problem.