The Sensory Representation of Time

Time is embedded in many aspects of our sensory experience; sensory events unfold in time and often acquire particular meaning because of their specific temporal structure. The speed of a moving object, the words pronounced by a speaker and the tactile exploration of a texture, are all examples of temporally structured sensory experiences. Despite the ubiquitousness of the temporal dimension of our sensory experience, the understanding of the neural mechanisms underlying the temporal representation of sensory events, that is the capacity to estimate duration in milliseconds/seconds range, remains a controversial and complex issue. The controversy relates to the effective involvement of sensory-specific brain regions in the processing of temporal information. The complexity arises from the neurophysiological mechanisms underlying the representation of time in these areas and the functional interplay between sensory-specific and amodal temporal mechanisms (Harrington et al., 2011).

The idea that we time sensory signals via a single “centralized” and “amodal” clock dominated the field of temporal cognition over the last 30 years. More recently the universality of timing mechanisms has been challenged by new theoretical positions and a growing body of empirical data (Buhusi and Meck, 2005). From a theoretical perspective the challenge comes from “distributed” timing models. This is a broad class of models, which – although different regarding the neurophysiological mechanisms proposed for time processing – collectively share the idea that we have multiple timing mechanisms “distributed” across brain areas or circuits; and that the engagement of each single mechanism depends on the psychophysical task, sensory modality, and lengths of temporal intervals (Ivry and Richardson, 2002; Durstewitz, 2003; Matell and Meck, 2004; Buonomano and Maass, 2009). The idea that sensory-specific timing mechanisms exist is supported by studies showing that the ability to discriminate temporal information depends on the modality of the signals. For example, temporal discrimination thresholds are lower for auditory compared to visual signal durations (Grondin, 1993; Grondin et al., 2005; Merchant et al., 2008); and the capacity to keep in memory multiple intervals improves if the temporal signals belong to different modalities and therefore rely on different memory resources (Gamache and Grondin, 2010). The existence of independent sensory-specific clocks is also suggested by the observation that the perceived duration of a sensory event can be distorted by modality-specific properties of the stimuli such as visual adaptation (Johnston et al., 2006; Ayhan et al., 2009), spatial, and temporal frequency (Kanai et al., 2006; Kaneko and Murakami, 2009); or by the observation that such distortions are limited to a single sensory domain, like in case of saccadic eye movements causing compression of the perceived duration of visual but not of auditory stimuli (Morrone et al., 2005; Burr et al., 2011). From the neurophysiological point of view, electrophysiological recordings in animals as well as neuroimaging and magnetic stimulation studies in humans suggest that both modality-specific and supramodal mechanisms underlie the estimation of temporal intervals (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bosco et al., 2008; Bueti et al., 2008b; Sadeghi et al., 2011). For example, it has been demonstrated that the extrastriate visual area MT/V5 is necessary for temporal discrimination of visual, but not of auditory durations (Bueti et al., 2008a) and that duration estimation to predict expected visual and auditory events involves secondary as well as primary visual and auditory cortices (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bueti and Macaluso, 2010; Bueti et al., 2010).

Taken together these behavioral and neurophysiological data highlight the functional contribution of sensory-specific cortices and support the existence of modality-specific timing mechanisms. However, how temporal information is actually represented in these cortices and what is the neurophysiological mechanism behind it, remain unclear. A few interesting theoretical hypotheses have been advanced. “Intrinsic” timing models for example, describe time as a general and inherent property of neural dynamics. A consequence of this assumption is that any area of the brain is in principle able to encode time. Temporal computations according to these models rely on inherent temporal properties of neural networks like short-term synaptic plasticity [i.e., state-dependent networks (SDNs) model; Buonomano and Maass, 2009] or arise either from the overall magnitude of neural activity (Eagleman, 2008) or from the linear ramping of neuronal firing rate (Durstewitz, 2003; Reutimann et al., 2004). “Intrinsic models” of temporal coding are particularly suitable to describe the functional organization of sensory timing mechanisms because they assume that time is encoded by the same circuits encoding other stimulus properties such as color or motion in the visual modality. However the explanatory power of some of these models, like for example the SDNs model, is constrained to durations of a few hundred milliseconds (i.e., <500 ms; Buonomano et al., 2009; Spencer et al., 2009); this is indeed a strong limitation, given that most of the neurophysiological evidence in favor of modality-specific timing mechanisms deal with durations from hundreds of milliseconds to a few seconds. An alternative possibility is that temporal computations in sensory cortices engage wider and specialized temporal circuit (s), where time signals from sensory cortex are sent to “dedicated” timing areas where these signals are integrated and used to guide action for example (Coull et al., 2011). In this latter case the relationship between sensory-specific and sensory independent timing areas need to be elucidated. Many cortical (parietal, premotor, prefrontal, and insular cortices) and subcortical (basal ganglia and cerebellum) brain structures have indeed been implicated in the processing of temporal information independently from the sensory modality of the stimuli (see Spencer et al., 2003; Coull et al., 2004; Koch et al., 2008; Wiener et al., 2010 for a review; Wittmann et al., 2010). Although there is only a partial agreement regarding the relevance of all these structures to time processing, the challenge is now to explore whether these areas have dissociable or interchangeable/overlapping functional roles and therefore whether these areas support the same or different temporal mechanisms compared to sensory-specific areas. A very special case of multimodal timing area is represented by the auditory cortex, a sensory-specific area. It has been recently demonstrated indeed that the auditory cortex is important for temporal discrimination not only of auditory but also of somatosensory and visual stimuli (Bolognini et al., 2009; Kanai et al., 2011). The supramodal involvement of auditory areas in temporal tasks has been associated with a strategic use of auditory-based mental representations for time estimation (Franssen et al., 2006). An interesting hypothesis, suggested by Kanai and colleagues, is that given the dominance of the auditory system over vision in temporal tasks (Walker and Scott, 1981; Burr et al., 2009), visual information is converted into an auditory code for temporal computation(Kanai et al., 2011). This hypothesis is interesting because offers new insight into the relationship between visual and auditory timing systems and highlights a possible link between modality independent and modality-specific temporal mechanisms.

It is therefore clear that the study of the functional architecture of sensory timing mechanisms poses a few more theoretical and experimental challenges. A few important questions are still open. It is, for example, unclear whether the organizational principles that apply to space also apply to time and whether the temporal dimension of visual stimuli is processed by the same or distinct networks compared to those for space. Is time coding in visual cortex retinotopic specific? Do we encode all possible temporal intervals at each retinotopic position? In which context do sensory-specific temporal mechanisms work? Is temporal information encoded in sensory cortices automatically or does it require explicit attention? Are sensory areas engaged only during duration encoding or are also active during working memory maintenance?

The already complex scenario of the neural representation of time is getting even more intricate. From the idea of a single “amodal” mechanism we moved into the idea of multiple “modality-specific” and “modality independent” temporal mechanisms (Wiener et al., 2011). The challenge is now to find out the functional architecture of these mechanisms as well as the interaction between them. As a concluding remark, I would like to emphasize that the focus of the majority of studies exploring the neural correlates of temporal processing has been so far to identifying the key components of internal timing networks (i.e., the “where” of timing mechanisms). The result of this approach has been, for example, an exponential increase of the number of neuroimaging studies on this topic that has lead to a substantial disagreement regarding the structures that are relevant to time processing (Wiener et al., 2010 for a review). It is time to adopt new experimental approaches that pose more mechanistically oriented questions about the underlying timing mechanisms while at the same time attempting to link computational models and neurophysiology (Portugal et al., 2011).