Multiple channels of visual time perception

Abstract

The proposal that the processing of visual time might rely on a network of distributed mechanisms that are vision-specific and timescale-specific stands in contrast to the classical view of time perception as the product of a single supramodal clock. Evidence showing that some of these mechanisms have a sensory component that can be locally adapted is at odds with another traditional assumption, namely that time is completely divorced from space. Recent evidence suggests that multiple timing mechanisms exist across and within sensory modalities and that they operate in various neural regions. The current review summarizes this evidence and frames it into the broader scope of models for time perception in the visual domain.

Introduction

The identification of neural substrates that are selectively tuned for time is a massive challenge for researchers and, to date, has failed to produce many clear answers. The first and most influential model of timing assumed that the duration of an interval, regardless of the modality of the embedded sensory stimulus, could be measured by a centralized cognitive mechanism (the ‘internal clock’) if one accessed a periodic brain signal (pacemaker) and integrated the number of pulses between two markers [1–3]. Within this framework, changes in perceived duration of a fixed-duration stimulus are solely explained in terms of changes in arousal or attention, which would speed up or slow down the clock rate. While fruitful, this model does not predict some basic features of time perception, such as scalar variability (i.e. a linear relationship between judgment uncertainty and interval length [4]). Also, a clear biological substrate has been elusive. Alternatives include network models, in which the rich interplay of excitation and inhibition in the cortex endows neural networks with the intrinsic capability of being time-evolving systems [5,6••,7,8] and ramping firing rate models [9–11,12•], which propose that elapsed time is encoded in the increased rate of firing of neurones before an event. According to another model, multiple oscillators tuned to different frequencies create distinct patterns of activity as time elapses [13–15]. Finally, it has been proposed that the perceived duration of a stimulus might mirror the amount of energy used to encode it, possibly tuned by the natural statistics of temporal information [16–20].

In this review, we will focus on visual time processing in the sub-second range often called ‘perceptual timing’ [21], which is of a highly perceptual nature and it is not accessible to cognitive control [22], rather than supra-second time estimation, which is likely to rely on higher cognitive systems or memory [21]. By reporting results obtained with different paradigms, we aim to address the following questions: (1) Is time processed independently from space? (2) Can adaptation occur specifically for one duration? (3) Do eye-movement planning and execution interfere with time processing? (4) Is attention necessary to monitor multiple timing mechanisms? (5) Does the recent stimulus history affect subsequent interval duration estimates?

Visual time perception models have to account for the observation that the perceived duration of an interval can be altered by various forms of adaptation, by properties of the embedded stimulus itself and by the recent evolution of the temporal environment. All these point to specific temporal modules within the visual modality.

Spatially specific changes in apparent duration follow visual adaptation

The idea that, in our brain, there might be a single universal clock to determine duration at a cognitive level seems to imply that time estimation is divorced from sensory processing. Alternatively, it would reasonable to expect to find modality-specific timing mechanisms with a sensory component that might be spatially localized and adaptable. Distortions in apparent duration were reported after adaptation to visual motion or flicker (see Figure 1a and b): adapting to a 20 Hz flickering or drifting stimulus resulted in an apparent temporal compression of subsecond intervals containing 10 Hz flickering or drifting stimuli presented at the adapted location [23••]. This compression cannot be attributed to a general change in arousal or attention since both the adapted and the unadapted stimuli would be affected. Adaptation effects are specific to duration judgments: changes in perceived temporal frequency or perceived onset/offset of the adapted stimulus could not account for the observed duration bias [23••,24,25]. The effect was limited to high temporal frequencies (5 Hz adaptation induced a negligible change in apparent duration) and it disappeared at equiluminance [26]. These findings suggest an involvement of the magnocellular pathway [27], possibly through a ‘predict and compare’ strategy [28••,29,30]. The temporal tuning of the parvocellular pathway appears to be unaffected by this kind of adaptation. Time distortions were also reported for high temporal frequency stimuli [31–33] during saccadic eye-movements [34,35••], after luminance contrast adaptation [36], after dark adaptation [37] and for an interval defined by two bars within a sequence of high-frequency random dynamic luminance flicker [38]. These experimental conditions again point to a critical role of the magnocellular pathway. This does not imply that the parvocellular pathway is not involved in the processing of subsecond intervals [39]. There is some evidence that adaptation causes changes in apparent duration when the adaptor contains translating motion, but not when it contains radial, circular or biological motion [40]. This might be at odds with a magnocellular interpretation, as early magno areas respond equally to translating, radial and circular motion.

Figure 1. Adaptation-induced changes in perceived duration.

(a) Adapting a limited region of the visual space to 20 Hz induced a reduction of the apparent duration of a 10 Hz stimulus subsequently displayed in the same location as the adaptor, relative to an identical test stimulus presented in an unadapted location. (b) The effect was limited to high adapting frequencies and was orientation-independent. Adapted from Johnston et al. [23••]. (c) After repeated and prolonged exposure to a given interval duration, apparent duration was measured for two test stimuli of different modalities. (d) A repulsive after-effect was observed: the test duration (320 ms) was perceived as having a shorter duration after adapting to longer intervals and vice versa, but only for a limited duration range. Adapted from Heron et al. [57••].

At present, the evidence fails to point to a single brain site where this kind of adaptation takes place. Adaptation-induced changes are orientation-independent [23••] and are tightly tuned (less than one degree of visual angle) to the adapted location [41•]. Moreover, duration compression was also reported after adaptation to a temporal frequency higher than the critical flicker fusion threshold [24]. These findings point to a precortical locus in the visual pathway, where the receptive fields are small, do not show orientation selectivity and respond to higher temporal frequencies than in the visual cortex. However, studies that investigated the spatial coordinates and the direction specificity of the adaptation-induced duration effects have yielded inconsistent results. On one hand, evidence exists that the effects of adaptation are spatiotopic [40,42,43•,44,45], that they transfer from the adapted eye to the opposite one [43•,44] and that they are specific to the direction of adapting motion at low speed (3°/s) [46] and to the translating motion profile [40]. All these results would implicate a more cortical locus, where stimuli are represented in head- or world-centric coordinates, cells receive inputs from both eyes and are selective for motion direction. On the other hand, at odds with the cortical proposal, retinotopic effects [40,45,47•,48] and lack of interocular transfer [47•,48] have also been reported; the direction-specificity was replicated under retinotopic, but not spatiotopic, viewing conditions [49] and, at higher speeds (>8°/s), compression was found after adaptation to the opposite direction of drifting motion [50].

One may argue that the discrepancy between the results supporting a pre-cortical locus and those supporting a more central locus are due to different manipulations that tap into different processes, which might not regard the representation of time per se, but some more generic decision bias. However, most of the studies reviewed above involved the same adaptation paradigm with very similar stimuli and, at times, both retinotopic and spatiotopic effects emerged within the same study [40,45]. Notably, adaptation selectively caused a shift in the Point of Subjective Equality (PSE, defined as the 50% point on the psychometric function), which was used as a measure of perceived duration, but no change in the slope, indicating that the different experimental manipulations did not really affect the noise levels associated with decision processes in different ways.

Most of the aforementioned studies reported duration compression for dynamic stimuli after adaptation. Duration expansion was reported for static Gabors after adaptation to flicker [51]. The effect did not depend on the adapting frequency (5 or 20 Hz), it did not transfer interocularly and it was reduced when the orientation of adaptor and test differed of 45°, suggesting that low level neurons (V1) might be affected by this kind of adaptation. At this stage, it is not clear whether the two phenomena (compression and dilation after adaptation) share the same characteristics (for instance, if they show similar spatial specificity) and, therefore, whether they can be traced back to the same underlying mechanisms.

Taken together, these results seem to suggest that spatially localized adaptation occurs at multiple levels of the visual hierarchy. We note that, to this point, we have explored this issue by focusing exclusively on studies that use psychophysical methods. Specific neurophysiological and neuroimaging studies could provide us with the key for a deeper understanding of the brain sites and mechanisms involved.

Repulsive duration after-effects

Do separate channels dedicated specifically to duration processing exist in our brain? And, if they do, are they tuned to a preferred duration the way neurons are to visual orientation [52] or auditory pitch [53]? The duration biases described in the previous section were induced by adaptation to motion or flicker and not to a given duration or range of durations. Temporal frequency was the main parameter that was varied across conditions, whereas adaptation time remained constant. Assuming that time processing shares some of the same components employed for the processing of motion or temporal change can account for the aforementioned results.

There is evidence showing repulsive duration after-effects for subsecond intervals after repeated exposure to an interval of a given duration both in the auditory [54,55] and in the visual [56] modality: an interval of intermediate length appeared longer after adaptation to a shorter duration, whereas a longer adapter induced compression. More recently, Heron et al. [57••] conducted a systematic investigation of these effects. Their participants adapted to a repeated sequence of intervals of identical duration containing, in separate sessions, Gaussian blobs or white noise bursts (see Figure 1c). A test phase followed, where two intervals in two modalities (a Gaussian Blob for vision, a white noise burst, for audition) were sequentially presented. Participants were required to compare the duration of these two tests. As in the earlier studies, test duration was perceived in a direction away from the adapting duration (see Figure 1d). Furthermore, they showed that the effect of adaptation was limited to the adapting modality, that the duration bias was tuned to the reference test interval and that the magnitude of the bias effect depended on the adapting duration. In other words, a 320 ms interval appeared compressed or expanded to various degrees within a limited range of adapting durations (roughly, between 80 and 1000 ms), whereas the distortions tended to disappear for much longer or shorter adapting durations. The effect of duration adaptation is not limited to the spatial location of the adaptor and it transfers across visual hemifields [58].

These findings seem consistent with recent results on perceptual learning. In fact, some studies reported that temporal generalization is strictly selective for the trained interval duration [59••,60]. Confirming a previous observation [56], duration adaptation did not transfer across modalities [57••,61], suggesting an earlier processing stage than that for multisensory integration [62•]. Neurons that showed responses tuned to a preferred interval duration were found in a few species in the auditory modality [63–65]. Neurons in the premotor cortex in the monkey showed tuning to subsecond intervals for both visual and auditory stimuli [66], suggesting the existence of a supramodal representation of time intervals. Using a fMRI paradigm, Hayashi and colleagues have recently observed neuronal adaptation in the inferior parietal lobule (IPL) to repeated presentation of visual intervals of the same duration [67••]. This finding provides evidence for the existence of duration-tuned neurons in humans.

Eye movements and temporal distortions

The possible existence of spatially localized visual clocks makes it important to consider how the passage of time is integrated across eye movements or spatial shifts of attention. Morrone et al. [35••] investigated the perception of brief intervals marked by equiluminant bars in the proximity of an eye movement. Consistently with the idea that spatial references are fundamental for visual time perception, they found strong compression of time with intervals of 100 ms being compressed by about 50% [35••] and even a transient reversal of temporal order during saccadic preparation [34,35••,68]. These effects were specific for the visual modality as auditory intervals were not distorted. Temporal compression was accompanied by a decrease in the just-noticeable difference (JND), complying with scalar variability, suggesting that the input to the timing mechanisms might be altered in proximity of the saccade. It has been proposed that the distortions of time occurring at the time of eye movements could result from the suppression of luminance information that accompanies saccades [38]. However, the original experiment was obtained with equiluminant stimuli for which no suppression occurs [35••]. Further the experiments that attempted to emulate the phenomenon and lowered stimulus visibility, either via continuous flicker [38] or via presentation of a mask [69], induced smaller effects, which unfolded over a briefer time course [69]. Interestingly, compression of visual time occurs also when stimuli are presented during hand movements [70], a condition which is not accompanied by suppression of magnocellular activity, indicating that the compression induced by action exists independently of changes in visual sensitivity.

The rapid shift of the retinal image from one fixation to the next causes disruptions both in space and time [34,71]. The time course temporal distortions is remarkably similar to that of space distortions, suggesting that they may be produced by the same mechanisms [72,73].

Several follow up studies have shown that temporal compression also occurs for pursuit eye movements [74], where equiluminant stimuli are compressed of 15%, and for movements of the hand [70,75,76], for which the voluntariness of the movement was found to be essential [75–78].

Monitoring and attending multiple clocks

Popular sentences such as ‘the watched pot never boils’ indicate that conscious estimation of long temporal intervals can be manipulated by the availability of cognitive resources [78–81]. Researchers have also investigated whether a similar effect takes place with sub-seconds intervals (i.e. perceptual timing) and if the availability of attentional resources is a prerequisite to monitor the passage of time unfolding in different spatial locations.

Evidence of a role for attention in time perception comes from studies that show marked performance decrements when people are required to monitor multiple intervals concurrently, even when the stimuli are presented at different spatial positions [79,80]. The interference between intervals delivered in several positions may seem at odds with the idea that multiple independent clocks exist in the visual modality [23••]. However, Morgan and colleagues have shown that even for mechanisms which are definitely mediated by localized neural substrates, such as those to perceive object size, interference does take place [80]. Overall these data indicate that people cannot time two events simultaneously. This might be due either to the fact that attention is necessary to accumulate and handle information over time (i.e., before the duration processing itself) or that interference arises between representations of the different intervals. Ayhan and colleagues have shown that averaging duration across multiple elements lowers duration discrimination regardless of the set size and that this pattern remains the same for both sequential and simultaneous presentations, suggesting that the decline in performance is due to limits in central rather than divided attention [81].

In order to investigate what happens if attention is withdrawn only transiently, Cicchini and Morrone measured perception of time marked by two stimuli in different position while subjects had to shift their attention rapidly toward a peripheral stimulus [82••]. Consistently with previous findings on the withdrawal of attention, they found that time was grossly underestimated if one of the two markers was in temporal proximity to the stimulus of the primary task. In a further condition, however, they repeated the experiment by delivering the two visual stimuli marking the interval in the same spatial position.

Surprisingly, in this case, no underestimation occurred. This shows that the availability of attentional resources is necessary only when integrating information delivered across multiple sites. Somehow, the delivery of an interval in the same position does not require attention, as if an automatic timing mechanism was engaged.

Multiple time channels revealed by optimization strategies

The idea that temporal estimates can be biased by previous exposure to either a moving stimulus or to a repeated interval, suggests that temporal perception is far from being monolithic and it is in a constant state of calibration and fine tuning. Another piece evidence that supports this view is that the perceived duration of an interval exhibits a strong regression toward the average (Figure 2a).

Figure 2. Temporal judgments are affected by context.

(a) Reproduction of visual intervals for three experimental conditions (short, medium and long intervals). Data for each condition show a strong regression toward the average. The effect is stronger for longer intervals, which bear more uncertainty [83••]. (b) The strength of the regression toward the average depends on the precision of sensory representation. Expert percussionists display the best precision and show nearly no regression [82••]. (c) Perceived duration in a comparison task shows a strong assimilation toward an irrelevant interval presented either before or after. (d) The effect, however, disappears if distracter and test are presented in different modalities. For details see [90••].

In a typical temporal reproduction paradigm, the final estimate of an interval depends as strongly as 40% on the stimulus sequence to which it belongs [83••], as if participants formed a prior guess on stimulus duration which distorts the perception and reproduction of the new stimulus. Consistent with the idea that these mechanisms represent a way to optimize performance under conditions of uncertainty [83••,84–86], the effects are stronger for narrow stimulus distributions [87], noisy sensory estimates [83••] and depend upon the precision of the sensory modality and musical expertise [84,88] (Figure 2b).

The timescale and the locus where such optimization takes place are not known. In a recent unpublished report, Roach et al. [89] delivered a sequence of intervals containing stimuli from different modalities in the same session and found that, instead of regressing toward specific means, the two intervals regressed toward a common average. This suggests that the averaging can occur supramodally, if not at the response stage itself.

Burr et al. [90••] have investigated the perception of intervals marked by two events in the presence of a third distracting event which marked a second interval that was irrelevant to the task [7]. Even though the task was purely perceptual, the authors found strong assimilative effects (up to 80% when the distracting interval lead the test interval) even for very brief intervals (Figure 2c). Moreover, the magnitude and temporal properties of the effect depended on the modality of stimulation with the most reliable one (audition) leading to smaller effects (Figure 2d). This effect is consistent with a perceptual optimization process aimed to reduce errors under conditions of uncertainty. The effects reported for longer temporal intervals (in the range of 500–1500 ms) are rather modest (i.e. less than 10%). Whilst it is highly likely that optimization can occur both in perception as well as in response planning, it is still possible the effect reported in the reproduction tasks, often measured with longer stimuli, may be different than that observed with brief visual intervals.

Conclusions

Despite the fact that time appears as a unitary dimension, evidence is building for the idea that brief intervals are handled differently by different modalities. Within the visual modality, the changes in apparent duration described here can be caused by adaptation at multiple stages of the visual pathway suggesting that several parallel clocks exist, which estimate time independently across the visual field. Many of these findings are also consistent with the idea that specific temporal modules can be formed with relatively simple neural mechanisms. Buonomano et al. demonstrated that even a simple network comprising of hundred neurons is capable of encoding temporal intervals [6••] and, similarly, simulations of multiple oscillators indicate that reasonable timing performance can be obtained with about hundred oscillators [13]. Different clocks might exist not only for different sensory modalities [91], but they might also be specifically selected according to planned actions or movements [77,92]. Having a distributed network of adaptable local mechanisms to process temporal information might be useful for, at least, two reasons: first, to reconstruct a unified account of a fragmented perceptual experience [93,94], and, second, for stimuli of different modalities as well as for different objects in the visual scene, to extract not only their static features (such as size, shape, color, position), but also to have specific information on their temporal properties (such as knowing how rapidly they change).