Human time perception and its illusions

SUMMARY

Why does a clock sometimes appear stopped? Is it possible to perceive the world in slow motion during a car accident? Can action and effect be reversed? Time perception is surprisingly prone to measurable distortions and illusions. The past few years have introduced remarkable progress in identifying and quantifying temporal illusions of duration, temporal order and simultaneity. For example, perceived durations can be distorted by saccades, by an oddball in a sequence, or by stimulus complexity or magnitude. Temporal order judgments of actions and sensations can be reversed by exposure to delayed motor consequences, and simultaneity judgments can be manipulated by repeated exposure to non-simultaneous stimuli. The confederacy of recently discovered illusions points to the underlying neural mechanisms of time perception.

Introduction

The visual system brags a long history of parlaying illusions into an understanding of the neurobiology [1], but only recently has the study of temporal illusions begun to blossom. New illusions of duration, order and simultaneity illustrate that temporal introspection can often be a poor guide to the timing of physical events in the world. Temporal judgments are constructions of the brain, and, as we will see below, surprisingly easy to manipulate experimentally [2].

Time perception is a term that encompasses many scales. For the purpose of this review, we will only address illusions of time perception at the ‘automatic’ or ‘direct sensation’ time scales – that is, sub-second timing. Timing of longer scales, such as second and minutes and months, are categorized as ‘cognitive’ and appear to be underpinned by entirely different neural mechanisms [3-6].

Short interval durations

Duration judgments at short intervals are subject to several types of illusions. Here is a do-it-yourself demonstration to set the stage: look at your own eyes in a mirror and move your point of focus back and forth so that you're looking at your right eye, then at your left eye, and back again. Your eyes take tens of milliseconds to move ballistically from one position to the other – but here's the mystery: you never see your own eyes move. What happens to the gaps in time while your eyes are moving? Why doesn't your brain care about the small absences of visual input?

In recent years, several groups looked at time perception around eye movements more carefully. This began with an examination of the ‘stopped clock’ illusion: upon first glance, the second hand of a clock sometimes seems to be stopped in place momentarily before it continues to tick at a normal pace. Yarrow et al (2001) proposed that the scene the eyes land upon fills the time gap retrospectively [7], such that the eye movement is an integral part of the sense of time. Morrone and her colleagues then discovered that duration judgments were compressed during saccades [8]: when subjects were asked to judge an interval between two flashes near in time to a saccade (by comparison to two more targets well after the saccade), durations were underestimated by about a factor of two (Figure 1a). More recently, Terao et al (2008) suggested a possible explanation for the saccade results, showing more generally that stimuli with reduced visibility (as stimuli are during a saccade) lead to the same sort of duration compressions [9]. While the data are clear, the mechanisms are still a subject of debate [10].

Figure 1.

Depictions of temporal illusions. (a) Compressed duration judgments occur around saccades [8] and moments of diminished stimulus visibility [9] (blue range). Under normal circumstances, we do not notice the temporal gap in sensory experience caused by a blink; the continuity of input is also a temporal illusion of sorts. Figure adapted from [10]. (b) The first [20] and oddball [19,23,25] stimuli appear expanded in duration compared to their neighbors. While this has traditionally been interpreted as an expansion of the unexpected stimulus experience duration dilations, it is equally feasible that the repeated stimuli are experiencing duration contractions due to repetition suppression [19]. (c) Recalibration of perceived timing. Given delayed sensory consequences (a button press followed by a delayed flash, top), temporal expectations are dynamically adjusted in order to bring sensory consequences closer toward simultaneity (bottom). As a result of recalibration, unexpected events occurring after the flash may be perceived to have occurred beforehand [40].

More generally, duration distortions can be induced by properties of the stimuli themselves. For example, it was shown early last century that subjective duration is dilated by motion [11] or sequence complexity [12,13], and these observations grew into proposals that the brain estimates time based on the number of ‘events’ that occur [14-16] – in essence, the occurrence of many events is interpreted by the brain as a longer duration. In support of this hypothesis, Kanai et al (2006) explored the basis of motion-induced time dilation and concluded that temporal frequency was the critical element in the distortion rather than the motion per se; in support of this, they demonstrated that duration dilation could be induced simply by a flickering stimulus [17]. Although temporal frequency—or more generally, the ‘event density’ in a stimulus—appears to be a modulating factor, it cannot be the only factor determining duration: after all, we are quite capable at judging the timing of non-dynamic stimuli.

Moreover, there are simple ways to dilate durations independent of dynamic changes to the stimulus. Xuan et al (2007) demonstrated that duration is dilated by the magnitude of the stimulus. Larger, brighter, and higher numerosity stimuli were all perceived to have a longer duration than equal-length stimuli of smaller magnitudes along those axes [18]. Whether the same neural mechanisms underlie magnitude-induced and event-based distortions remains to be seen. One recently suggested possibility is that subjective duration mirrors the amount of neural energy used to encode a stimulus [19], which could in theory account for both types of results. We will return to this possibility below.

Predictability modulates duration

The examples above appear to be related to low-level visual processes, but the story of subjective duration grows more interesting. When a stimulus is shown repeatedly, the first appearance is judged to have a longer duration than successive stimuli [19-22]. Similarly, an ‘oddball’ stimulus in a repeated series will also be judged to have lasted longer than others of equal physical duration (Figure 1b) [19,23-25]. These dilations of perceived duration have been called a subjective ‘expansion of time’ [23]; however, it is important to note that the psychophysical results could equally be interpreted as a duration contraction of the repeated stimuli, rather than an expansion of the first or oddball stimulus.

With this in mind, Pariyadath and Eagleman (2007) have pointed out that this pattern of duration distortions seems to parallel the pattern of neural activity seen with repetition [19]. That is, neuronal firing rates in higher cortical areas quickly become suppressed after repeated presentations of a stimulus [26-28], an effect generally known as repetition suppression [29,30] and measured in humans using EEG [31], fMRI [32], PET [33] and MEG [34,35]. It has been proposed that repetition suppression reflects increasing efficiency of representation [36,37]. In that view, with repeated presentations of a stimulus, a sharpened representation or a more efficient encoding is achieved in the neural network that codes for the object, affording lower metabolic costs. We have previously speculated that this differential response to novel versus repeated stimuli maps on to perceived duration: a suppressed neural response corresponds to a shorter perceived duration [19].

Note that the duration distortions also occur with higher-level predictability. For example, if the series 1-1-1-1-1 is presented, the first stimulus appears longer because of the putative duration contraction of the succeeding stimuli; critically, the same illusion also occurs for the sequence 1-2-3-4-5 [19], presumably because the successive stimuli are predictable, even while their low-level shapes differ. This finding indicates that the predictability of successive stimuli involves higher cortical areas than the primary visual cortex, and that repetition suppression may be a special case of prediction suppression.

Temporal order judgments dynamically recalibrate

A challenge for the brain is that afferent signals from the different sensory modalities are processed at different speeds. When receiving signals from several modalities, how does the brain determine the timing correspondences? The answer seems to be that the brain dynamically recalibrates its expectations.

In 2002, Haggard and colleagues noticed that when a subject made a motor act (such as a button press), subsequent events (such as a beep 250 msec later) appeared to be ‘pulled’ slightly closer in time to the button press [38,39]. This might have been explained by a compression of perceived time between the button press and flash, but a few years later Stetson et al (2006) ruled that out in favor of a different explanation: the timing expectations of motor acts and sensory consequences can shift in relation to one another, even to the extent that they can switch places [40]. Specifically, imagine that you can trigger a flash of light by pressing a button. Now imagine that we inject a slight delay—say, 100 ms—between your press and the consequent flash. After pressing the button several times, your nervous system adapts to this delay, such that the two events seem slightly closer in time, as Haggard and colleagues had suggested. Now that you are adapted to the delay, we now surprise you by presenting the flash immediately after you press the button: in this condition, you will believe the flash happened before your action—in other words, you experience an illusory reversal of action and sensation [40] (Figure 1c). We hypothesize that this illusion reflects a recalibration of motor-sensory timing which results from a prior expectation that sensory consequences should follow motor acts without delay. Note that temporal order recalibrations can be demonstrated in passive conditions as well (i.e., without the motor act): repeated exposure to non-simultaneous external sensory events can alter subsequent simultaneity judgments [41] and temporal order judgments [40,42,43]. However, the shift in these judgments in the sensory-sensory case is less than half of that for the motor-sensory case [40], indicating that the best way to calibrate timing expectations of incoming signals is to interact with the world: each time a person kicks or knocks on something, the brain can make the assumption that the sound, sight and touch should be simultaneous. If one of the signals arrives with a delay, the brain can adjust its expectations to better approach subjective simultaneity.

Dynamically recalibrating the temporal interpretation of motor and sensory signals is not merely a party trick of the brain – it is critical to solving the problem of causality. At bottom, causality requires a temporal order judgment: did my motor act precede or follow the sensory input? The only way this problem can be accurately solved in a multisensory brain is by keeping the expected time of signals well calibrated, so that ‘before’ and ‘after’ can be accurately determined even in the face of different sensory pathways of different speeds.

Is time one thing?

An open question is whether subjective time is a unitary phenomenon, or instead whether it is underpinned by separate neural mechanisms that usually work in concert but can be dissociated under the right circumstances. In other words, when one temporal judgment changes, do the others necessarily follow suit? We give three examples that indicate the answer is ‘no’.

First, returning to the Morrone et al (2005) finding of duration compression around the time of a saccade (Figure 1a), can one assume that subjective time in general has been compressed by a factor of two during the saccade? No, because the duration compression does not occur with auditory clicks, but only with flashes [8,10]. Therefore, it is not time in general that is compressed, only duration judgments of visual stimuli that are modulated.

Second, returning to the ‘oddball’ experiments described above (Figure 1b), if the duration dilation represented a general speeding of an internal clock, then other temporal judgments such as the pitch of an auditory tone or the rate of a flickering stimulus should be expected to change concomitantly with the oddball. However, measurements show clearly that other temporal judgments (e.g. pitches and flicker rates) do not change during the oddball duration distortion [19]. This simple experiment indicates that time is not one thing, but is instead composed of separate neural mechanisms that usually work together but can be teased apart in the laboratory.

Finally, to understand the meaning of the common anecdotal report that “time seems to have slowed down” during a life-threatening situation, Stetson et al (2007) ran an experiment to determine if the claim meaningfully captured actual subjective experience. They hypothesized that if time can slow down as a single unified entity (the way it does in movies) then the slow motion should entail consequences such as an ability for higher temporal resolution. (For example, watching a video of a hummingbird in slow motion allows finer temporal discrimination because more snapshots are taken of the rapidly beating wings). The experimenters measured time perception of participants who fell backward from a 50 meter tower into a net below. Participants retrospectively reported an increased perception of duration for their fall (as compared to others’ falls) – however, critically, they showed no evidence for increased temporal resolution when measured during the 3 second fall [44]. This result suggests a close intertwining of time and memory: during a frightening event, the amygdala is thought to contribute to denser-than-normal memory formation. In this way, frightening events become associated with dense memories, and the more memory one has of an event, the longer it is interpreted to have been [14-16,45].

These experiments provide rich evidence that time is not a single entity. Instead, it is likely that a diverse group of neural mechanisms mediates temporal judgments. Note that this framework for thinking about time perception places it in line with the history of vision research, in which it is understood that vision emerges as the collaboration of many subpopulations that code for different aspects of scenes (motion, position, color, and so on) [1]. These subpopulations usually work in concert, but they can be separated in the laboratory. In the domain of time perception, it is likely that duration, simultaneity, temporal order, flicker rate and other judgments are underpinned by different mechanisms that normally concur but are not required to.

Current models and their discontents

The most traditional model proposed to account for interval passage over short time scales is a simple ‘counter’ model, in which internal pulses are collected up and integrated during the presence of a stimulus [46,47]. This is thought to account for distortions in the following way: if we imagine that the brain has access to the roughly constant rate of its own information processing (say, one bit of internal information processed is interpreted as one unit of objective time having passed), then when the rate of internal information processing suddenly goes up to two bits per unit of objective time (as when one pays more attention because of an imminent crash into another car) a counter would count more bits. If the brain's assessment of duration is the result of the output of such a counter, it would come to the wrong conclusion that more objective time had passed, creating the illusion that duration had expanded. Several authors have appealed to versions of this counter model to explain the duration distortion triggered by the oddball or the first stimulus [20,23-25]. In this framework, an increase in arousal caused by the appearance of an unexpected (‘oddball’) stimulus leads to a transient increase in the ‘tick rate’ of an internal clock. Thus, the accumulator collects a larger number of ticks in the same time period, and the duration is judged as having lasted longer during the oddball.

However, the idea of a clock-like counter has found little support in the physiology, and in its place a new style of model has proposed that the passage of time can be encoded in the evolving patterns of activity in neural networks [4,48-50]. For example, imagine that every time a red cue light turns on, a specific spatiotemporal pattern of activity is triggered in the visual cortex. At 100 msec after the light comes on, a particular set of neurons will be active; shortly afterward, these neurons will activate other neurons, which will activate other neurons, and so on – leading to a specific pattern of neural activity that progresses into a different snapshot of active cells at every moment. When a salient event happens 100 msec after the cue light (say, the delivery of a juice reward), the snapshot of neurons that happened to be active at that moment is imprinted by the strengthening of their weights. In other words, the way the network evolves through time can code for the time itself. One twist on the model appeals to oscillating membrane potentials in individual neurons: if the phases of the oscillations are reset by a triggering event (the red light), then each successive moment in time can be encoded by the unique pattern of the relative phases of all the members of the population [5]. In another version of the model, the ongoing neural activity of the network is not encoded in continuous spiking, but instead is carried in the state of intracellular signals, such as calcium concentrations [49]; this expands the notion of the ‘state’ of the network from spikes to parameters which influence how spikes will be received and sent.

Although this model is appealing, it awaits experimental validation, and potentially suffers from the difficulty of making it work in noisy environments. In the context of this review, the major challenge to the state-dependent model seems to be the illusions of duration. It may be a major challenge of network engineering to speed or slow the passage of patterns through neural tissue without getting a new pattern entirely (but see [51]).

New experimental data reviewed above may point to surprisingly low-level properties of neurons. For example, Johnston et al (2006) demonstrated that adaptation to a flickering stimulus led to duration distortions of subsequent stimuli, and that the effect is spatially localized. The localization suggests a source of timing in early visual areas [52]. Similarly, the low-level importance of temporal frequency in duration distortion may also point to early levels [17]. Recently, Terao et al (2008) suggested that the transient response of neurons may be involved in very short time scales: when they manipulated stimulus visibility, perceived intervals were compressed [9]. Finally, from studies of duration and repetition, a non-exclusive suggestion is that the total amount of neural activity maps onto duration [19]: in its extreme form, duration is a signature of the amount of energy expended by neurons. All these non-exclusive hypotheses recommend future experiments in which low-level neural signatures are put directly to the test by combining psychophysics with physiology and neuroimaging.

Conclusions

The recent renaissance of temporal illusions is ripe to trigger a cross-disciplinary approach, establishing a fertile middle ground in which to combine experimental techniques employing electrophysiology, psychophysics, EEG, fMRI and computational modeling. Mechanisms are often exposed by their stresses and strains, and the hope is that these illusions will light the way to understanding general outstanding questions of time perception: How are the signals entering various brain regions at varied times coordinated with one another? How are durations, simultaneity and temporal order coded differently in the brain? How does the brain recalibrate its time perception on the fly? We hope the illusions presented here will provide a useful starting point for a neurobiological understanding of time.