The inner experience of time

Abstract

The striking diversity of psychological and neurophysiological models of ‘time perception’ characterizes the debate on how and where in the brain time is processed. In this review, the most prominent models of time perception will be critically discussed. Some of the variation across the proposed models will be explained, namely (i) different processes and regions of the brain are involved depending on the length of the processed time interval, and (ii) different cognitive processes may be involved that are not necessarily part of a core timekeeping system but, nevertheless, influence the experience of time. These cognitive processes are distributed over the brain and are difficult to discern from timing mechanisms. Recent developments in the research on emotional influences on time perception, which succeed decades of studies on the cognition of temporal processing, will be highlighted. Empirical findings on the relationship between affect and time, together with recent conceptualizations of self- and body processes, are integrated by viewing time perception as entailing emotional and interoceptive (within the body) states. To date, specific neurophysiological mechanisms that would account for the representation of human time have not been identified. It will be argued that neural processes in the insular cortex that are related to body signals and feeling states might constitute such a neurophysiological mechanism for the encoding of duration.

1. Introduction

Throughout history, philosophers have been intrigued by the nature of time and how we, as humans, experience its progression. The perception of time is part of human experience; it is essential for everyday behaviour and for the survival of the individual organism (Pöppel 1997; Wittmann 1999; Buhusi & Meck 2005). Yet, and surprisingly enough, its neural basis is still unknown. Temporal intervals, lasting only seconds or spanning a lifetime, are judged according to their perceived duration—often regarded as painfully long or, the reverse, as lasting too short. Everyday decisions we make, as simple as either waiting for the elevator or taking the stairs, are based on the experienced passage of time and anticipated duration (Wittmann & Paulus 2008). The importance of our temporal experiences for daily living is strikingly documented in individual neurological cases where patients report of an accelerated progression of time and, consequently, have troubles in adequately interacting with the environment, i.e. driving a car (Binkofski & Block 1996). Although we doubtless have a time sense, our bodies are not equipped with a sensory organ for the passage of time in the same way that we have eyes and ears—and the respective sensory cortices—for detecting light and sound. Time, ultimately, is not a material object of the world for which we could have a unique receptor system. Nevertheless, we speak of the perception of time. When we talk about time (‘an event lasted long’, ‘time flew by’), we use linguistic structures that refer to motion events and to locations and measures in space (Evans 2004); a further indication that time itself is not a property in the empirical world.

Despite a growing body of knowledge on the psychology and on the neural basis of the experience of time, the riddle for philosophers and scientists alike is still unsolved: how does the mind (or, for that matter, the brain) create time? Martin Heidegger's paraphrase of St Augustine's famous quotation1 ‘In you, my spirit, I measure times; I measure myself, as I measure time’ (Heidegger 1992) reflects a theoretical approach—founded in western philosophical tradition—which states that time is a construction of the self. Perceived time, thereafter, represents the mental status of the beholder. In terms of a functional equation, one could state that time t is a function f of the self, where the self stands for all possible psychological (i.e. empirical and theoretical) properties of an individual who perceives time,

$$t=f(\text{self}).$$ (1.1)

Psychological research has shown that cognitive functions such as attention, working memory as well as long-term memory determine our temporal judgements (Brown 1997; Zakay & Block 2004; Taatgen et al. 2007). Moreover, drive states, moods and emotions (Wittmann et al. 2006; Droit-Volet & Meck 2007; Noulhiane et al. 2007a) as well as factors of personality (Rammsayer 1997) influence duration estimates. For example, time intervals are judged to be longer when we pay more attention to time and when the load of varying experiences stored in memory is higher. Our subjective well-being also strongly influences how time is experienced. Time speeds up when we are involved in pleasant activities, but it drags during periods of boredom. Thus, our sense of time is a function of the intricate interplay between specific cognitive functions and of our momentary mood states.

The aforementioned psychological factors definitely influence the processing of duration. However, a specific neural timing mechanism—influenced by the aforementioned factors—nevertheless, could account for our ability to accurately process temporal intervals. Especially for shorter durations up to a few seconds, humans can accurately synchronize their movements to regular beats (Mates et al. 1994), discriminate tones with different durations (Rammsayer & Lima 1991) or reproduce presented intervals (Eisler & Eisler 1994). Yet, there is no consensus as to which temporal mechanisms account for these temporal-processing abilities. Over the last decades, the most successful models for such a mechanism have been variants of a pacemaker–accumulator clock, where an oscillator (a pacemaker) produces a series of pulses (analogous to the ticks of a clock) and the number of pulses recorded over a given timespan represents experienced duration (Pöppel 1971; Church 1984; Treisman et al. 1990; Meck 1996; Zakay & Block 1997). However, competing models assume neuronal system properties for encoding duration not related to a simple pacemaker–accumulator system (Matell & Meck 2004; Wackermann & Ehm 2006; Karmarkar & Buonomano 2007), or they propose that memory decay processes are involved in time perception (Staddon 2005; Wackermann & Ehm 2006). Related to this unsolved issue, the question of which areas of the brain process duration has also not yet been answered definitely. Among other regions, most prominently, the cerebellum (Ivry & Spencer 2004), the right posterior parietal cortex (Bueti et al. 2008a), the right prefrontal cortex (Rubia & Smith 2004; Lewis & Miall 2006) as well as fronto-striatal circuits (Harrington et al. 2004a; Hinton & Meck 2004) have been implicated as the neural substrates of a potential timekeeping mechanism.

In summary, as this brief outline suggests, there is still considerable uncertainty on how (regarding psychological and neurophysiological models) and where in the brain time is processed. This paper has several goals that are related to the issues raised: (i) to give an answer to the question of why so many different brain regions have been assigned to the neural basis for our experience of time, i.e. to explain some of the variation found across models, and (ii) to describe recent developments in the research on time perception, which are indicative of a strong involvement of emotions and mood states. These developments could be described as an ‘emotive turn’ in this area of research, which might follow decades of focusing on cognitive aspects of time perception. (iii) Recent conceptualizations and empirical findings, which have led to this emotive turn, might develop into a neural theory of time perception that will encompass subjective feeling and interoceptive (within the body) states. Specific neurophysiological processes in circumscribed regions of the brain, as they are related to these feeling states, might constitute a mechanism for encoding duration.

2. Cognitive and neural models of time perception

Investigations in the fields of experimental psychology, clinical neuropsychology and neuroimaging have resulted in an extensive literature on the mechanisms and underlying neural systems of temporal processing. Over the last decades, certain cognitive and neural models have dominated time perception research, but alternative models exist and the number of potential theories has to date increased considerably. To summarize the status of the research field in general terms: there is a lot of conflicting evidence and there are several competing conceptualizations. In the following, a short review of (i) predominant cognitive models of time perception and (ii) related neural theories of interval timing is provided. These models and theories will be contrasted with alternative conceptualizations and empirical evidence in order to provide an overview of the heterogeneity of ideas concerning mechanisms of time perception.

Cognitive models distinguish two fundamental perspectives in time estimation: prospective and retrospective time estimation (James 1890; Zakay & Block 2004).2 In the former, an observer judges the duration of an interval that is being presently experienced. In retrospective time estimation, by contrast, an observer estimates a timespan that has already been passed and to which he is only now paying attention. Models of prospective time estimation assume an ‘internal clock’ with a pacemaker producing a sequence of time units that are fed into an accumulator (Church 1984; Treisman et al. 1990). In a variant of these pacemaker–accumulator models, the attentional-gate model (Zakay & Block 1997), the time units produced are only registered when attention is directed to time. Thus, prospective timing is always a dual task since an observer has to divide attention between temporal and non-temporal processes (Grondin & Macar 1992; Taatgen et al. 2007). The number of units that have been recorded during a physical time period (being stored in working memory) is then compared (in a decision process) with long-term memory of stored representations of time periods, which can be verbalized as seconds or minutes (Pouthas & Perbal 2004; Wearden 2004). Thus, in addition to the pacemaker (the actual clock component), several cognitive processes such as working memory, long-term memory, attention and decisions are involved in prospective time perception.

In retrospective time estimation, the duration of a time interval that has already elapsed has to be judged. Then, an observer has to estimate a given duration in retrospect from the amount of processed and stored memory contents; that is, duration has to be reconstructed from memory (Ornstein 1970; Flaherty et al. 2005; Noulhiane et al. 2007b). The more changing experiences we have during a certain timespan—which are stored in memory and later retrieved—the longer the duration is subjectively experienced (Bailey & Areni 2006). In retrospect, routine activity when compared with novel activity leads to the perception of shorter time intervals (Avni-Babad & Ritov 2003). Thus, the subjective impression of a long time interval depends on the activity of a person with diverse experiences and recruits the activation of areas such as the medial temporal cortex, known to be involved in episodic memory (Noulhiane et al. 2007b). Retrospective duration judgements are based on temporal intervals spanning a few seconds (short-term memory) to, in principle, a whole lifetime (long-term memory) (Wittmann & Lehnhoff 2005). Pure prospective duration judgements, by contrast, are only conceivable over a limited and shorter time range where an observer attends to time for a period of seconds to minutes.

Prospective timing studies in animals and humans have yielded the general hypothesis that fronto-striatal circuits consisting of recurrent loops between frontal cortex (SMA), caudate–putamen, pallidum and thalamus, which are modulated by the dopamine system, are critical for the processing of duration (Hicks 1992; Harrington et al. 2004a). Support for the anatomical hypothesis comes from investigations in patients with brain lesions and from neuroimaging studies. For example, individuals with structural damage to the frontal lobes (Nichelli et al. 1995; Kagerer et al. 2002) or traumatic brain injury predominantly affecting frontal areas (Pouthas & Perbal 2004) show impaired estimates of temporal intervals. Neuroimaging studies with healthy volunteers find that the perception of duration is linked to activation in right prefrontal and striatal regions (Ferrandez et al. 2003; Nenadic et al. 2003; Coull et al. 2004). Regarding the involvement of neurotransmitter systems, D2 dopaminergic antagonists (such as haloperidol) impair duration discrimination abilities in healthy subjects (Rammsayer 1999). Moreover, studies with animals and humans indicate that both dopaminergic agonists, e.g. methamphetamine, and antagonists influence timing processes, presumably by increasing and decreasing clock speed, respectively (Mohs et al. 1980; Buhusi & Meck 2002; Cevik 2003). Individuals dependent on cocaine and methamphetamine, who have abnormal brain metabolism and structural changes involving dopaminergic target areas such as the striatum and the frontal cortex, exhibit impaired time processing on several timing tasks (Wittmann et al. 2007b). Additional evidence for the involvement of the dopamine system in time perception comes from patients with Parkinson's disease who have decreased dopaminergic function in the basal ganglia and show deficits discriminating and reproducing temporal intervals (Hellstrom et al. 1997; Harrington et al. 1998). Further evidence comes from a study in which subthalamic deep brain stimulation in patients with Parkinson's disease levelled temporal reproduction performance to that of control subjects (Koch et al. 2004). Thus, intact dopamine neurotransmission within striatal and frontal areas of the brain is an important prerequisite for accurate temporal-processing abilities.

However, some neuroimaging studies, revealing areas of activity while subjects estimate durations or time their movements, suggest that a core temporal-processing mechanism is located in the right prefrontal cortex (dorsolateral and ventrolateral areas) for both sub- and supra-second intervals (Rubia et al. 1998; Brunia et al. 2000; Lewis & Miall 2003a; Smith et al. 2003). In these (and other) studies, the basal ganglia did not show activation, thus leading to the conclusion that dopamine modulation in right prefrontal areas might be the basis for a primary timekeeping mechanism (Lewis & Miall 2006). Furthermore, and pointing to yet another different brain region, since patients with cerebellar dysfunctions are impaired in the precise timing of movements (Ivry et al. 1988) as well as in the sensory discrimination of duration (Ivry & Keele 1989), the cerebellum seemingly plays an important role in the processing of time. It has been speculated that separate, i.e. non-overlapping, neural elements in the cerebellum that have different delay properties could potentially encode duration (Ivry & Spencer 2004). Last but not least, the right posterior–inferior parietal cortex has been implicated (i) in integrating duration information as represented in the sensory modalities and (ii) as an interface between the perception of duration and timed actions (Bueti et al. 2008a,c). Repetitive transcranial magnetic stimulation (rTMS) selectively disrupts duration discrimination in the visual and auditory modality if the respective sensory cortices are affected, but stimulation of the right parietal cortex disrupts timing in both the visual and auditory domains (Bueti et al. 2008a,b). Bueti and colleagues thus support the idea of distributed, modality-specific, timekeeping processes that converge in the parietal cortex. Neurophysiological studies in monkeys complement these findings by showing that specific increasing (ramp-like) neuronal activity in the posterior parietal cortex encodes duration (Leon & Shadlen 2003). However, similar neural activity in the monkey brain and related to the timing of events has been recorded in the premotor and motor cortex as well (Lebedev et al. 2008).

One difficulty in deciding on which regions are the primary target areas for a suspected timekeeping mechanism comes from neuroimaging studies where, typically, multiple brain regions show activation during time perception tasks (Lewis & Miall 2003b). The involvement of all these brain areas is probably due to different cognitive processes not subtracted out in the contrasts between primary time perception and control tasks, the principal method used in functional magnetic resonance imaging (fMRI). Many active areas are not primarily related to the encoding of duration (i.e. an internal clock) but, nevertheless, take part in a complex timing system representing attention, working memory and decision-making processes (Rubia & Smith 2004; Livesey et al. 2007). A consequence of the complex architecture of an assumed timekeeping system is that disruptions in any component can result in timing impairments. That could explain why so many different patient groups with damage to the brain, i.e. lesions or degenerative processes in the basal ganglia, cerebellum or right parietal, as well as frontal cortex, can be impaired (Wittmann 1999). Likewise, it cannot be clearly decided whether experimental disruptions with rTMS are due to the transient impairment of a neural clock or of other processing components. For example, rTMS applied to the dorsolateral prefrontal cortex reliably impairs temporal reproductions of a few seconds as well as multiple second intervals, but it is unclear whether this accounts for the disruption of a neural clock or of working memory (Koch et al. 2003, 2007; Jones et al. 2004). rTMS applied to the cerebellum transiently impairs the temporal reproduction of intervals with durations of approximately half a second (but not approx. 2 s), a finding that could be interpreted by the disruption of either a millisecond timer or of a motor programme involved in millisecond timing (Koch et al. 2007).

Two further complicating factors in this discussion arise from neuropsychological studies with brain-injured patients. First, some studies report negative findings in which patients do not differ in their timing abilities from control subjects. Second, timing deficits, if registered, although significant from a statistical point of view, are not necessarily dramatic. For example, and contrasting earlier findings, patients with cerebellum lesions following a stroke were not impaired in a duration discrimination task and only mildly impaired in a motor-timing task (Harrington et al. 2004b). Patients with focal basal ganglia lesions, although impaired in performing with maximum tapping frequency, were as accurate in motor timing as were controls (Aparicio et al. 2005). Moreover, although these patient groups, as shown in several studies, on average, and according to the inferential statistics, differed from the performance of control subjects, the deviations were often minimal (Ivry & Keele 1989; Harrington et al. 1998; Kagerer et al. 2002; Wittmann et al. 2007b).

One suggested solution to the problem of not being able to pin down specific regions of the brain as representing a clock-type timing mechanism is to assume distributed neural networks where neural populations within each region would encode duration (Mauk & Buonomano 2004). If several neural units would possess intrinsic temporal-processing properties, many different brain areas could contribute to the perception of time depending on the modality and the type of task. These network or state-dependent models do not incorporate a dedicated timing system (with a centralized clock) but rather time-dependent neural changes, such as short-term synaptic plasticity (Karmarkar & Buonomano 2007). However, such intrinsic mechanisms would be limited to short time intervals up to several hundred milliseconds. Time perception in the range of seconds would still require additional processes (Ivry & Schlerf 2008). Similarly, it is conceivable that temporal-processing functions are embedded in several specialized and interacting neural circuits, where the timing function would not be the primary process in those brain regions (through a dedicated mechanism for temporal processes) but, for example, motor systems would implicitly be involved in the temporal processing of intervals (Nobre & O'Reilly 2004). Alternatively, an idea that has been very recently stated suggests that the effort made when we perceive and act might determine experienced duration (Marchetti 2009). An intuitive example fitting this idea is the phenomenon that novel experiences seem to last longer than routine events, which can be explained by the greater demand of mental activities involved in performing a task or analysing a situation for the first time. The idea of mental effort as a determinant of time experience is compatible with the notion of ‘fluency’, the subjective experience of ease or difficulty of a mental task (Oppenheimer 2008). The experience of time, according to this view, would not rely on clock processes but would be an epiphenomenon evolving from cognitive and emotional responses during a time interval, where a cognitively demanding task (a filled interval) and doing nothing (an empty interval, i.e. when sitting in the doctor's waiting room) would likewise lead to the impression of a slow passage of time (the act of waiting without any distraction can be filled with painful emotions) (Flaherty 1999).

Further conceptualizations, which argue against the idea of dedicated timing systems, add to the diversity of time perception models. Just as is argued for the retrospective perspective on duration, prospective timing could be governed by memory processes, where the decreasing memory strength over time, specifically modelled using leaky integrators, leads to the impression of passing time and, eventually, of duration (Staddon 2005). In a different model, the ‘dual klepsydra’ model, inflow and outflow processes of a leaky accumulator form properties of a timekeeping mechanism (Wackermann & Ehm 2006). Subjective duration is defined by the state of a leaky integrator that depends on inflow units (accumulation over time during the encoding of stimulus duration) and outflow units (loss over time). Parameters of that model fit well to individual temporal reproduction responses as well as to changes in the timing of behaviour induced by experimental manipulations (Wackermann et al. 2008).

In summary, a multitude of incompatible conceptualizations exists on how time is processed. Furthermore, there is no agreement on which brain areas or brain systems—dedicated to time or not—might underlie our perception of duration. The important conclusion so far is that different processing components are involved when we perceive duration or time our movements, which are not related to an internal clock. Nevertheless, through experimental manipulation or in certain patient populations, changes in these processes can affect perception and behaviour related to the time domain. In the following, one further factor will be discussed that may explain some of the variation found in studies on time perception, i.e. why so many different brain regions have been assigned with a central timekeeping function. The main point will be that different neural systems are involved in temporal processes (and related experiences) depending on the duration of the processed interval.

3. Time scales in the perception of time

It is intuitively most unlikely that one mechanism or one neural system would be responsible for all possible durations that an organism has to process. Different temporal processing mechanisms must be involved on different time scales (Trevarthen 1999; Wittmann 1999; Mauk & Buonomano 2004; Buhusi & Meck 2005). Experimental interventions have repeatedly shown duration-specific effects of psychopharmacological agents on interval timing. For example, the dopamine receptor antagonist haloperidol as well as the benzodiazepine midazolam impair duration discrimination of intervals ranging approximately 1 s, whereas processing of 50 ms intervals is affected by haloperidol only (Rammsayer 1999). According to these results, any pharmacological treatment that affects working memory capacity (e.g. midazolam) would interfere with temporal processing of intervals above 1 s. However, intervals with a length of up to a few hundred milliseconds (such as the 50 ms interval) are supposed to be processed based on brain mechanisms outside of motor and cognitive control and reflect pure timing processes (thus, not being influenced by midazolam). According to this view, additional processes such as attention and working memory, not related to time per se, come into play with intervals exceeding several hundred milliseconds in length and complementing dopamine-driven processes, which are involved both in shorter and longer time intervals (Rammsayer et al. 2001). A similar theoretical proposal, derived from a meta-analysis on neuroimaging data, suggests two distinct neural timing systems: (i) an automatic timing system for shorter intervals up to approximately 1 s, which recruits motor systems of the brain (SMA, basal ganglia and cerebellum) and (ii) a cognitively controlled timing system for supra-second intervals connected mainly to right prefrontal and parietal cortical areas (Lewis & Miall 2003b). This separation of time perception systems is to some extent mirrored by findings in motor-timing studies, where qualitative changes in tapping performance occur with inter-tap intervals of approximately 1 and 1.5 s duration (Madison 2001). Time ranges between 0.45 and 1.5 s seem to be automatically processed, i.e. not strongly affected by attentional demands, whereas attention and working memory processes (stimulated by secondary tasks) affect intervals in the range between 1.8 and 3.6 s (Miyake et al. 2004).

The notion that perception and motor behaviour are processed in discrete windows or processing epochs has been conceptualized for some time (White 1963; Pöppel 1970; Dehaene 1993; VanRullen & Koch 2003; Fingelkurts & Fingelkurts 2006). These temporal integration units fuse successive events into a unitary experience, ‘snapshots of experience’ or ‘psychological presents’ (Blumenthal 1977), which are characterized by co-temporality, meaning that events within this time zone have no temporal relationship (Ruhnau 1995). For example, the perception of temporal order of short stimuli in different modalities is only possible if the individual events are separated by at least 20–60 ms (Exner 1875; Hirsh & Sherrick 1961; Kanabus et al. 2002; Fink et al. 2006). If the two events are separated by smaller intervals, an observer is not able to tell which of the two appeared first. Moreover, since stimulus properties of speech stimuli are perceptually segmented into these sequential processing units, temporal information within a segment of a speech sound may not be relevant for decoding (Kiss et al. 2008). A longer time frame of approximately 200 ms determines the integration of auditory–visual input in speech processing (van Wassenhove et al. 2007). Fusion percepts were reported if the onset of lip movements and heard syllables did not exceed this time lag. Threshold values of approximately 250–300 ms have long been suggested to represent a specific integration process in perception (Münsterberg 1889; White 1963). A minimum stimulus duration (or minimum inter-onset interval) of 200–300 ms is necessary for detecting the temporal sequence of four acoustic events (Warren & Obusek 1972; Shrivastav et al. 2008) and for optimal effects in an oddball task that leads to the subjective expansion of duration (Tse et al. 2004). Temporal integration mechanisms in a time frame of approximately 250 ms also seem to be involved in sensorimotor processing, distinguishing maximum tapping speed from a personal, controlled motor speed (Peters 1989; Wittmann et al. 2001).

On a different time scale, a perceptual mechanism seems to exist that integrates separate successive events into a unit or perceptual gestalt (see Pöppel 2009). We do not just perceive individual events in isolation, but automatically integrate them into perceptual units with a duration of approximately 2–3 s (Fraisse 1984; Pöppel 1997). For example, while listening to a metronome at a moderate speed, we do not hear a train of individual beats, but automatically form perceptual units, such as 1–2–3, 1–2–3, etc. These are mental constructs—physically speaking, they do not exist. The duration of this temporal integration mechanism, referred to as the subjective present, seems to be limited to 2–3 s (Szelag et al. 1996; Wittmann & Pöppel 2000). Phenomenological approaches have revealed that temporal intervals shorter than 2 s are perceived qualitatively as different from longer intervals (Benussi 1913; Nakajima et al. 1980). Typically, intervals up to 2–3 s are reproduced accurately, whereas longer intervals tend to be underestimated (Ulbrich et al. 2007; Noulhiane et al. 2008). In one study measuring event-related potentials (ERPs) during the reproduction of visual stimuli ranging in duration between 1 and 8 s, accurate reproductions up to 3 s were accompanied by a contingent negative variation-like slow negative shift in the ERP signal. This shift was reduced or absent when durations exceeded 3 s (Elbert et al. 1991). Duration discrimination thresholds with base durations up to 2 s show constant Weber fractions (Δt/t, where Δt is the difference between the base duration t and the length of the comparison interval at which a duration difference is just noticeable), but with longer durations, Weber fractions rise rapidly3 (Getty 1975). Subjects can accurately synchronize their motor actions to a sequence of tones presented with a frequency of approximately 1–2 Hz. The ability to synchronize these tones becomes more difficult with increasing inter-tone intervals and finally breaks down when intervals exceed durations of approximately 2–3 s (Mates et al. 1994; Miyake et al. 2004). With respect to even longer intervals, it is also conceivable that distinct processes are involved for different durations. For example, the estimation of a 1 h interval seems to be related to the duration of wake time of an individual, whereas the estimation of seconds and minutes is related to body temperature (Aschoff 1998).

Experimental interventions selectively affect intervals of specific time ranges, i.e. the psychopharmacological substance psilocybin has an effect on duration longer than 2 s (Wittmann et al. 2007a), rTMS of the cerebellum affects solely millisecond timing and rTMS of the right dorsolateral frontal cortex only affects timing in the seconds range (Jones et al. 2004; Koch et al. 2007). Nevertheless, a reliable correlation between time scales and neural systems has not been accomplished so far. Also, individual neuroimaging studies often find similar brain areas activated in time perception tasks employing different durations (Pouthas et al. 2005; Jahanshahi et al. 2006).

To summarize §§ 2 and 3, several factors contribute to the diversity of different models on the question of how (which mechanism) and where (which neural system) in the brain duration is processed. First, there are several cognitive processes that are entangled with our perception of time (independent of a potential internal clock), which require the integrative processing of multiple modules distributed across the brain. Experimental manipulations of task load (in attention and working memory, for example), clinical studies with selected patients with dysfunctions in different brain areas (i.e. cerebellum, basal ganglia, frontal and parietal cortex) as well as neuroimaging studies show that many parts of the brain and multiple cognitive processes contribute to the perception of duration. Moreover, there is no single area of the brain on which functioning our temporal experiences would completely rely on; that is, patients with damage to the brain may be affected in the processing of duration but their performance hardly ever breaks down completely. Second, different neural mechanisms are most probably responsible for temporal processes and time experiences depending on the duration involved.

4. The emotional experience of time

Time does not pass with a steady-paced flow. Perceptual time is not isomorphic to physical time, meaning that the subjective passage of time and estimates of duration vary considerably. In uneventful or unpleasant situations, such as when nervously waiting for something to happen, we experience a slower passage of time and overestimate its duration. By contrast, if we are entertained and focus on rewarding activities, time seems to pass more quickly and duration is more likely to be underestimated. These examples of time judgements are inherently emotional. They point to an aspect of time perception, although part of everyday experience, which has been neglected in research over the past few decades. Only very recently, a body of evidence has accumulated which is indicative of the intricate interplay between mood states and perceived duration. In most cases, the influence of emotions (often leading to longer time estimate) is explained by the standard cognitive model of prospective time perception in which emotions affect the degree of attention to time or increased physiological arousal levels lead to a higher pacemaker rate (Droit-Volet & Meck 2007; Wittmann & Paulus 2008). Both increased attention to time and a higher pacemaker rate of an assumed internal clock would lead to the accumulation of more temporal units during a given timespan.

Regarding durations of multiple seconds to minutes, it has been shown that patients with depression (Bschor et al. 2004) and cancer patients with high levels of anxiety (Wittmann et al. 2006) report a slowing down of the pace of time and overestimate temporal intervals. It would appear that the psychological distress these individuals suffer from directs attention away from meaningful thoughts and actions to the passage of time. Also, boredom-prone individuals estimate time intervals to last longer than persons with low levels of boredom (Danckert & Allman 2005). Similarly, subjects who were socially rejected in a psychological experiment overestimated intervals of multiple seconds, a finding that was interpreted as stemming from a state of reactive emotional and cognitive deconstruction, which in turn resulted in a stronger attentional focus on the present (Twenge et al. 2003).

Positive correlations of general anxiety levels with duration estimates of multiple seconds have been reported in undergraduate students (Siegman 1962) and in patients with psychiatric diagnoses (Melges & Fougerousse 1966). Spiderphobics who had to look at spiders for 45 s also overestimated this exposure time when compared with controls (Watts & Sharrock 1984). These overestimations were interpreted as resulting from an arousal-induced increase in an internal pacemaker. Moreover, emotional stress such as the anticipation of a mild electric shock (Falk & Bindra 1954), when compared with control conditions, led to an overestimation of duration.

Generally speaking, time distortions are stress related as they are often experienced during dangerous or life-threatening situations such as road accidents or encountered violence (Hancock & Weaver 2005). In movies, scenes depicting combat are sometimes shown in slow motion to portray what the involved protagonist subjectively experiences. To date, two studies experimentally tested this phenomenon. Novice skydivers, for example, who reported a stronger fear during their first tandem jump, also experienced its duration (subjective estimates approx. 30 min) as lasting longer than did less fearful novice skydivers (Campbell & Bryant 2007). A similar time dilation effect was also seen on a shorter time scale. Subjects who experienced a free fall of 31 m before they landed safely in a net overestimated retrospectively the duration of that fall when compared with when watching others fall (Stetson et al. 2007). However, a slow-motion effect was not detected when probed with a special chronometer that had to be watched during the fall. Even in more conventional and less frightening experimental situations, subjects overestimate the duration of high arousing pictures with emotional valence (depicting angry faces or accidents), which last only several hundred milliseconds to a few seconds (Angrilli et al. 1997; Droit-Volet et al. 2004; Gil et al. 2007). These effects seem to be strongly tied to the embodiment of emotions. Participants seem to overestimate emotional faces only when they are able to spontaneously imitate the perceived emotions on the faces which they have to judge, thus showing that affect is embodied (Effron et al. 2006). To be more precise, however, paradoxical effects can occur in the way that emotional stimuli can cause over—as well as underestimations of time. For example, in one study (Angrilli et al. 1997), low-arousing and emotionally negative pictures led to an underestimation of duration (interpreted as resulting from the subjects' distraction from time and attention to the emotional content of the stimuli), whereas high-arousing emotional pictures led to an overestimation of duration (interpreted as resulting from an increased pacemaker). In another study (Noulhiane et al. 2007a), unpleasant sounds (e.g. sobs, a crying women) were judged to last longer than pleasant (e.g. laughs) or neutral stimuli. This finding was again interpreted through an increase in pacemaker rate. Furthermore, and contrasting with the reported findings above, high-arousing sounds were judged to be shorter than low-arousing sounds (interpreted as resulting from the distraction from time).

In summary, increased attention to time (such as in waiting situations) and an increase in physiological arousal (such as under stress) can lead to longer time estimates when judging intervals in the range of milliseconds to seconds and minutes. However, paradoxical effects of an underestimation of duration in emotional situations can occur, which are discussed as stemming from a distraction from time. It is difficult to decide which mechanism, attention related or activation induced, actually affects the sense of time (and in which direction). Physiological measurements would have to complement the employed time perception measures. For example, in one study, acute marijuana effects in healthy subjects corresponded with underproductions of time intervals (interpreted as acceleration of a pacemaker) and an increase in heart rate, indicative of an increase in physiological arousal (Tinklenberg et al. 1976). The two proposed mechanisms influencing time experience are not necessarily exclusive but could contribute in an additive way (Burle & Casini 2001). For example, overestimations of time intervals of approximately 1 min in duration as detected in many impulsive patient groups could be due to an increase in physiological arousal as well as caused by an increase in attention to time (Berlin & Rolls 2004; Wittmann et al. 2007b).

The impact of our emotional states on the experience of time is usually discussed in the framework of the standard cognitive model of prospective time perception, which proposes a pacemaker–accumulator that is embedded in a system of cognitive components. However, on reviewing the empirical evidence and as argued above, the basic questions of how our sense of time is created are still unresolved. The recent upsurge in empirical investigations on the relationship between emotions and time, perhaps signifying an emotive turn, might result in a new psychological and neural theory of time perception. So far, emotions and physiological states, similar to cognitive functions such as attention and working memory, have been treated as modulators of an assumed neural clock. What if mood processes and the representation of body sensations themselves function as a timekeeper? Since emotions and physiological states seem so fundamental to the experience of time, it is tempting to assign a pivotal role to these processes related to a core time-keeping system.

Since the late nineteenth century, based on the theory by James and Lange, it has been argued that affective states as well as experienced emotions are inseparable from autonomic responses (e.g. cardiovascular activity, abdominal sensations and breathing patterns; Saper 2002; Pollatos et al. 2005). According to this notion, bodily signals, visceral and somatosensory feedback from the peripheral nervous system, enact emotions (Damasio 1999; Wiens 2005). For example, people's heartbeat detection ability is positively related to subjective ratings of emotional pictures as more arousing (Pollatos et al. 2007a). Interoceptive awareness, as tested with heart rate detection tasks, is predicted (among other regions) by right anterior insula activity (Critchley et al. 2004; Pollatos et al. 2007b). The insular cortex of primates, structurally embedded in the extended limbic system, is considered as the primary receptive area for visceral input, i.e. for physiological states of the body (Craig 2002; Saper 2002). The capacity for the awareness of emotions is probably built on the anatomical organization and a progressive integration of information in the insula (Craig 2003). A posterior-to-anterior progression of representations in the human insular cortex is the basis for the sequential integration of body states and internal autonomic responses with cognitive and motivational conditions, the latter being instantiated by distributed neural processes across the brain. This progression culminates in the anterior insula and leads to the conscious awareness of complex feeling states. A direct link between the perception of time and physiological processes has been proposed by Craig (2008), who claims that our experience of time relates to emotional and visceral processes because they share a common underlying neural system, the insular cortex and the interoceptive system. He suggests that the insula, through the temporal integration of signals from within the body, produces a series of ‘emotional moments’ in time. The perception of duration thereafter would be defined by these successive moments of self-realization, formed by information originating within the body (see also Craig 2009).

Several attempts have been made to directly relate rhythms of the body to temporal processes in perception and action. For example, Münsterberg (1889), being his own subject, reported that when the onset and the offset of temporal intervals to be reproduced and ranging in duration between 6 and 60 s coincided with him starting to inhale, his temporal reproductions were more accurate than when temporal intervals started at other points in time not systematically related to his breathing cycle. Münsterberg, who did not count in his experiments, concluded that the sense of time relied on the sensation of tension in different organs which are caused by muscle contractions. Fraisse (1982) highlights findings showing that the periodicities of the heart, of walking, of the preferred tapping tempo as well as of preferred acoustical rates are of the same order of magnitude of 500–700 ms. However, he concludes that ‘we cannot assume that one phenomenon can be explained by the other. There is only a narrow range of frequencies of natural or voluntary rhythms and of preferred tempo’ (p. 154). Despite this conclusion, however, one study showed that participants' preferred tempo of successive tones was in a harmonic relation (with a ratio of 1, 1.5 and 2) to individual heart rates as measured during the presentation of the tone sequences (Iwanaga 1995).

Other studies, which attempted to relate heart rate (in some cases also breathing rate) to time estimates in the range of several seconds to minutes, some testing subjects before and after physical exercise, found no relationship (Schaefer & Gilliland 1938; Bell & Provins 1963) or associations that were weak or difficult to interpret (Lediett & Tong 1972; Osato et al. 1995). One study, nevertheless, employing shorter intervals with duration up to 4 s, found that drugs, which either stimulate or inhibit the central or peripheral sympathetic nerve system, lead to an increase or decrease in heart rate and breathing rate and an accompanying relative under- or overproduction of intervals, respectively (Hawkes et al. 1962). A relative underproduction of intervals can be interpreted as resulting from an increase in clock speed. Although this latter result points to the role of cardiac and respiratory cycle rates in the judgement of duration, one has to keep in mind that proponents of the standard cognitive models of time perception could still argue that it is not the heart rate per se that functions as an internal clock but that generally increased arousal levels affect the pacemaker of an internal clock in the brain (Wittmann & Paulus 2008). Especially regarding the short durations tested in the study by Hawkes et al. (1962) of just a few seconds, a heart rate cycle of typically approximately 700 ms could not accurately represent different temporal intervals.

Although hardly any convincing evidence exists that would show how specific physiological cycles function as an internal clock for judging time, body states as a whole could, nevertheless, form the building blocks of a timekeeping mechanism. Ultimately, interoceptive processes as registered in the insula encompass the physiological status of all body tissues and organs such as the skin, muscles and the viscera (Craig 2008). It has been shown that the self-regulation of emotions (Vohs & Schmeichel 2003) and of autonomic parameters in biofeedback procedures (Cohen 1981) lead to longer time percepts. One attempt to explain these and other findings is that the insular cortex, which integrates body signals, is the anatomical basis for the creation of emotions and the sense of time (Craig 2008). Being more strongly aware of one's own emotions and body processes would, at the same time, lead to a prolongation of subjective duration. When individuals are experimentally deprived of sensory stimuli (auditory–tactual–visual) their overall sense of duration over several days gets strongly impaired. However, these subjects, who have to rely solely on their inner sense, report that time passes painfully slowly (Schulman et al. 1967). Experienced practitioners of mindfulness meditation who concentrated ‘on the self across time and the present moment’ in an fMRI study showed stronger activity in a right lateralized network including the insula and somatosensory cortex (Farb et al. 2007).

An explicit assignment to a functional role for the insula in temporal processing was made by Ackermann et al. (2001), who showed in an fMRI study that a linear increase in left anterior insula activity was a function of presented click train rates (increasing up to 6 Hz). By contrast, a linear increase in right anterior insula activity was recorded when click rates slowed down to 2 Hz. This and other findings have led to the idea that the insula is an essential component in the sequencing of sounds and the perception of rhythm in music (Bamiou et al. 2003). Moreover, an involvement of the insular cortex has repeatedly been shown in neuroimaging studies on duration processing with different timing tasks (Brunia et al. 2000; Rao et al. 2001; Lewis & Miall 2003b; Pouthas et al. 2005; Livesey et al. 2007; Stevens et al. 2007); however, the significance of insula activation in the context of time perception is seldom discussed. Recently, however, evidence of neurophysiological activity in the posterior insula of the human brain has shown to be involved in the encoding of multiple-second durations. Time–activity curves of neural activation derived from event-related fMRI during a time reproduction task showed activation in bilateral posterior insula that linearly built up when subjects were presented with 9 and 18 s tone intervals (Wittmann et al. 2008). This build-up of neuronal activation peaked at the end of the respective intervals. Related to the functional role of the dorsal posterior insula, this finding of accumulator-type activity was, therefore, interpreted as representing the registration of physiological changes over time that eventually leads to the representation of duration. The flow of time, thereafter, corresponds to the flow of interoceptive signalling from the body as sensed in the dorsal posterior insula.

A shift in attention to emotional processes and insular cortex activity in the search for the underlying mechanisms of time perception has only recently started. The greatest obstacle to a sound theory on the neural bases of time perception has been the lack of evidence for a neurobiological mechanism. The finding on accumulator-type activity during the processing of multiple second intervals represents such a potential mechanism. This finding is also compatible with the pacemaker–accumulator model of time perception where pulses emitted by a pacemaker are transiently stored in an accumulator and the number of pulses defines duration (Pfeuty et al. 2005). In line with this conceptualization, it is conceivable that the number and rate of body signals accumulated in the posterior insula over a given timespan creates our perception of duration. To disclose mechanisms on different time scales, i.e. milliseconds and a few seconds, will be the challenge for future research. Interval processing over different time ranges may be controlled by different neurobiological mechanisms.

5. Conclusion

Despite the fact that time is an essential factor for understanding complex behaviour, the processes underlying the experience of time and the timing of action are incompletely understood. Too many contradictory theories exist—in psychology and neuroscience alike—that aim at explaining how we judge duration. One could argue that the conceptualization presented here of relating body processes to the experience of time represents yet another alternative approach among so many others. Only future empirical evidence, based on strong theoretical grounds, will tell whether a dedicated neural system actually exists for the representation of time and which neurobiological mechanism causes the experience of time.

One argument of plausibility as to why our inner sense, our body awareness, might be related to the sense of time should be mentioned here. The recent proposal of an intimate relationship between time and the awareness of an emotional and visceral self (see Craig 2008 2009) seems to be in line with a western philosophical tradition proposing that time is a creation of the self. ‘Inner time or duration is virtually indistinguishable from the awareness of the self, the experience of the self as an enduring, unitary entity that is constantly becoming’ (Hartocollis 1983, p. 17). Thereafter, the experience of our self is only possible as an entity across time. One attribute of the self is time, or differently expressed, the self is defined by its extension over time, a succession of moments that constitutes duration.4 This idea is probably most explicitly expressed by Henri Bergson who noted that ‘psychic states […] unfold in time and constitute duration’ (Bergson 1913). According to his view, the phenomenal self creates the sense of duration.

In neurobiological terms, we perceive signals from the body which create a ‘material me’ that has subjective feelings and is self-aware (‘I feel it's me!’; Craig 2002). Since the ascending pathways to the insular cortex inform us about the ongoing status of the body, a pacemaker-type signal (accumulating successive states of the material self) would be continuously present that could be employed in a timekeeping system. An accumulation of physiological states over time would be registered in the insula. This sketch of a processing model is, of course, borrowed from the cognitive models with a pacemaker–accumulator unit and is, to date, speculative. To summarize the main lines of argument put forward in this review:

1. Although the perception of time is an essential and inextricable component of everyday experience, no conclusive answers to the questions of which neural substrates and what kind of neurophysiological processes could account for the experience of duration have been established. That is, several areas of the brain have been identified as potential contributors to timekeeping (e.g. cerebellum, frontal cortices and basal ganglia), but none have been specifically implicated for this process and there is no consensus as to the precise neural mechanisms accounting for our sense of time.

2. Philosophical wisdom beginning in antiquity has related the experience of time to the feeling of a self. The body self and emotional self in modern biological terms is based on insular cortex activity. The signalling of body states, which define the material me and contribute to the ‘feeling me’, is a permanent and ongoing process over time and, thus, could function as an inner measure of duration by matching external temporal intervals with the duration of physiological changes.

3. The strong relationship between effect and time is not only an everyday experience of everyone but it is also well documented by many empirical studies. Since affective states are entangled with body states, insular cortex activity (the primary sensory area for visceral signals) may, therefore, cause the experience of the passage of time.