Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2013 Jul 24;33(30):12423–12429. doi: 10.1523/JNEUROSCI.5146-12.2013

How the Visual Brain Encodes and Keeps Track of Time

Paolo Salvioni 1, Micah M Murray 1,2,3, Lysiann Kalmbach 1, Domenica Bueti 1,✉
PMCID: PMC6618677  PMID: 23884947

Abstract

Time is embedded in any sensory experience: the movements of a dance, the rhythm of a piece of music, the words of a speaker are all examples of temporally structured sensory events. In humans, if and how visual cortices perform temporal processing remains unclear. Here we show that both primary visual cortex (V1) and extrastriate area V5/MT are causally involved in encoding and keeping time in memory and that this involvement is independent from low-level visual processing. Most importantly we demonstrate that V1 and V5/MT come into play simultaneously and seem to be functionally linked during interval encoding, whereas they operate serially (V1 followed by V5/MT) and seem to be independent while maintaining temporal information in working memory. These data help to refine our knowledge of the functional properties of human visual cortex, highlighting the contribution and the temporal dynamics of V1 and V5/MT in the processing of the temporal aspects of visual information.

Introduction

Time is a key feature of any sensory experience. Sensory events unfold in time and the way we perceive this temporal unfolding is crucial for our understanding of these events. An approaching hand can be construed as an impending slap or caress, depending on its speed. If and how, time in the range of hundreds of milliseconds is processed by our sensory systems is controversial regarding the actual engagement of sensory-specific cortices and the underlying neurophysiological mechanisms.

The contribution of visual cortices to temporal computations has been suggested by psychophysical observations showing for example, that the perceived duration of a visual stimulus can be distorted by modality-specific features of the stimuli, such as visual motion and/or temporal frequency (Morrone et al., 2005; Kanai et al., 2006). Empirical support for the role played by visual cortices in temporal computations comes from animal electrophysiology and human neuroimaging. These studies show, for example, that the firing rate of V1 and V4 neurons is modulated by the time of an expected reward (Ghose and Maunsell, 2002; Shuler and Bear, 2006) or that, in humans, activity of V1 and extrastriate visual regions correlates with the temporal expectation of a visual event and with the encoding of a learned temporal interval (Bueti et al., 2010, 2012). However, none of these measurements allows for inferences regarding any causal contribution of these visual regions to temporal computations. The few existing magnetic stimulation studies on this subject highlight the importance of area V5/MT in temporal judgments of visual moving and static stimuli, but leave open the issues of the causal involvement of V1 in temporal computations and of the independence of this contribution from low-level visual processing (Bosco et al., 2008; Bueti et al., 2008; Kanai et al., 2011). To date, none of the above mentioned lines of evidence has determined precisely to which temporal computations V1 and V5/MT contribute (i.e., duration encoding versus duration short-memory), the temporal dynamics of these computations within each region, and the extent of functional interplay between them.

Across five paired-pulse transcranial magnetic stimulation (TMS) experiments we asked: first, whether and when V1 and V5/MT play a causal role in discriminating temporal intervals in the range of hundreds of milliseconds independent of the processing of low-level visual features of the interval; second, the extent to which V1 and V5/MT contribute to the encoding of temporal intervals and/or their maintenance in working memory; and third, the functional interplay versus independence of V1 and V5/MT in these temporal computations. Healthy volunteers discriminated either the temporal interval between or the brightness of visual stimuli. Paired-pulse TMS was applied over V1 or V5/MT at different delays during either the encoding or the short-term memory maintenance of the visual information. This design enabled us to establish whether and when V1 and V5/MT contribute to different phases of the task. The use of nontemporal discrimination tasks, including the test of visual perception, determined the specificity of this engagement in the processing of the temporal aspects of visual information.

Materials and Methods

Subjects.

Participants in Experiments 1–5, respectively, included 14,14,10,14 and 10 healthy right-handed adults (9, 10, 9, 8, and 9 females; mean age was 25.1, 26.2, 26.5, 26.1, and 23.3 years, range was 28–22, 38–22, 38–21, 38–22, and 21–29 years for Experiments 1–5, respectively) with normal or corrected-to-normal vision. All participants gave written informed consent to participate in this study, which was approved by the Ethics Committee of the Faculty of Biology and Medicine at the University Hospital of Lausanne. Among the tested participants, five participated in Experiments 1 and 3, six participated in Experiments 2 and 4, and five participated in Experiments 3 and 5.

Stimuli and procedure.

Experiments 1–2 involved a temporal discrimination task of empty intervals, each marked by two brief (16.7 ms each) light blue disks (0.78° diameter) presented at the center of the screen (resolution was 1024 × 768 pixels and refresh rate was 60 Hz). A black asterisk, (0.39° size) presented 0.78° above the center of the screen, served as the fixation point and was continuously displayed for the entire duration of the trial. Each trial consisted of the sequential presentation of the two temporal intervals separated by a brief gap (i.e., a random value taken from a uniform distribution ranging from 900 to 1200 ms); one of the two intervals was the “standard duration” and the other the “comparison duration.” The duration of the standard interval (T) was fixed (200 ms). The duration of the comparison interval was the standard plus a variable, always positive, ΔT value (i.e., comparison duration = T + ΔT). The presentation order of the standard and the comparison intervals was randomized and counterbalanced across trials. In half of the trials the standard was presented first, in the other half it was presented second. The volunteers performed an interval-discrimination task that consisted in judging which one of the two intervals lasted longer (first or second). Subjects responded by pressing one of two keys on the keyboard (Fig. 1A). Visual feedback was provided at the end of each trial: the fixation asterisk turned green or red signaling whether the response was correct or incorrect. The duration of the feedback was 1 s, whereas the duration of the intertrial interval was a random value taken from a uniform distribution ranging from 1.8 to 2.5 s. The feedback, although not essential for the task execution helped participants to set an internal discrimination criterion and, as other studies suggested, has likely influenced subjects discrimination accuracy (Droit-Volet and Izaute, 2005).

Figure 1.

Figure 1.

Open in a new tab

Results of the temporal tasks. A, Schematic representation of the experimental paradigm (detailed in Materials and Methods). B, Average (SEM indicated) of individual discrimination thresholds (i.e., Weber fractions: ΔT/T) after paired-pulse TMS of V1 and V5/MT at three different delays (different shades of blue and red for V1 and V5/MT, respectively) after either interval onset (encoding) or offset (working memory). In the top thresholds are indexed as the change with respect to the vertex stimulation (i.e., average of the two vertex blocks) as follows: site-vertex/vertex. The bottom shows the averaged thresholds not normalized to the vertex; gray and white bar are respectively the vertex and no-TMS condition. C, Left, correlations across subjects between discrimination thresholds obtained after TMS of V1 and V5/MT 85 ms (light gray) and 120 ms (black) after interval onset. Right, correlations across subjects between discrimination thresholds obtained after TMS of V1 at the 50 ms delay and V5/MT at the 85 ms delay after interval offset. Asterisks represent the results of the paired t test significant at *p < 0.05 and **p < 0.01. Plus symbols represent the results of the one-sample t tests significant at +p < 0.05 and ++p < 0.01 (Table 1).

The duration of the comparison interval (T + ΔT) was adjusted adaptively across trials, to obtain the ΔT threshold leading to 79% correct discrimination. For this, the duration of the comparison interval was adjusted by decreasing the ΔT after every three consecutive correct responses and increasing the ΔT after each incorrect response. The ΔT was changed in steps of 32 ms until the third reversal and 16 ms thereafter. The ΔT values at which the direction of the change was reversed (decreasing to increasing or vice-versa) were noted. The first three reversals of each block of trials were discarded, and the 79% correct point on the psychometric function was estimated by taking the average value of the remaining reversals (Levitt, 1971; Bueti et al., 2012). To ensure reliability, no estimate was retained if there were fewer than four reversals. None of the participants had <4 reversals. The final threshold was expressed as Weber fraction, i.e., the ΔT needed to achieve 79% correct discrimination divided by T. Participants were not informed about the adaptive procedure or that one of the durations was kept constant. The use of only positive ΔTs although enabled a relatively quick and accurate estimation of individual discrimination thresholds, prevented us from the possibility to have a measure of perceptual bias (i.e., the point of subjective equality, the value at which the two intervals would have been subjectively perceived as equal). The measure of the point of subjective equality was relatively unimportant for us first, because our aim was to perturb temporal sensitivity and second, because TMS has been proved to be ineffective in inducing temporal perceptual biases (Bueti et al., 2008).

In Experiments 3 and 4, we used the same task structure as Experiments 1 and 2; the only differences were that we kept the interval length constant (i.e., 200 ms), we changed the brightness of one of the four disks and asked participants to decide which pair of disks was on average brighter. The disk that changed in brightness could be in either the first or the second pair of flashes and be either the first or the second flash within each pair. The position of the changed disk was randomized and counterbalanced across trials. To obtain individual discrimination thresholds leading to 79% correct discrimination, we used the same adaptive procedure (i.e., rule “three up one down”) used in Experiments 1–2. The brightness was changed decreasing the original luminance value (358 cd/m2) by 5% until the third reversal and 1% thereafter. The luminance of the monitor background was 182 cd/m2).

In the fifth experiment we used the same stimuli used in Experiments 1 and 2 but a different task structure. Each trial consisted of the sequential presentation of two visual flashes, each lasting 16 ms, and separated by an empty interval of 200 ms. The intertrial interval was a value randomly chosen between 1.8 and 2.5 s. In 75% of the trials there were two flashes, and in the remaining 25% of the trials there was just a single flash. The subject's task was to decide whether there were 1 or 2 flashes.

In each experimental session of Experiments 1–4, participants performed a minimum of 12 blocks (60 trials each) of the visual task. Of the 12 blocks, eight were with TMS (i.e., 2 sites × 3 delays plus 2 blocks of vertex stimulation), the remaining were without TMS. The no-TMS blocks were used to obtain stable and reliable individual discrimination thresholds before applying TMS. Each participant performed at least four blocks (range 4–8) of the task without TMS plus 20 initial practice trials to familiarize with the procedure. In Experiment 5, participants performed eight blocks of the simple visual detection task (i.e., 2 sites × 3 delays plus 2 blocks of vertex stimulation), each block comprised 40 trials. In this last experiment there was no reason to have no-TMS blocks; however, at the beginning of each experimental session participants performed 10 trials to familiarize with the task. All Experiments were conducted in an acoustically isolated and dark room, sitting 45 cm from the computer's monitor.

Transcranial magnetic stimulation.

In all experiments TMS was delivered by a Magstim Rapid2 Stimulator and by a 70 mm figure-of-eight coil. We used paired-pulse TMS to take advantage of the summation properties of TMS pulses; double-pulse TMS gives larger effects than single-pulse, but still provides a reasonable temporal resolution defined by the temporal distance between the two pulses (Pascual-Leone and Walsh, 2001; Walsh and Pascual-Leone, 2003; Silvanto et al., 2005).

In choosing the paired-pulse TMS protocol, we took as methodological reference the paper by Silvanto et al.(2005). In this work, the authors applied paired-pulse TMS (interpulse interval was 20 ms, stimulus intensity 60% of the maximum stimulator output) over V1 and V5/MT to disrupt (i.e., reduce sensitivity as measured by d′) visual motion detection. Similarly to Silvanto et al. (2005), our aim was to interfere with the normal function of V1 and V5/MT by increasing neural noise (Walsh and Pascual-Leone, 2003). Compared with single pulse TMS, paired-pulse TMS is known to increase visual cortex excitability as measured by phosphene threshold (i.e., phosphene threshold decreases) when TMS pulses are applied at an interstimulus interval (ISI) ranging from 2 to 100 ms (Ray et al., 1998; Gerwig et al., 2005; Kammer and Baumann, 2010). The increase of visual cortical excitability is stronger when the intensity of two pulses is 100% of the phosphene threshold and decrease for intensities that are <80% of the threshold (Sparing et al., 2005). In cats' primary visual cortex, paired-pulse TMS has been shown to produce both facilitation and suppression of single-unit activity. These effects last for 150–200 ms after TMS onset and, at least within the range of ISIs tested (2–30 ms), are more pronounced when the intensity of both stimuli is equal to phosphene threshold (Moliadze et al., 2005).

Similar to Silvanto et al. (2005), we did not estimate individual phosphene thresholds and we used pairs of magnetic stimuli of the same intensity. The intensity of stimulation chosen was 55% of the maximum stimulator output; an intensity value at which none of the participants reported phosphenes. This value was indeed below that required to induce the clear perception of phosphenes (i.e., ≈70%; Cowey and Walsh, 2000), to disrupt psychophysical performance or to induce scotomas (i.e., >70%; Amassian et al., 1989, 1994; Kamitani and Shimojo, 1999).

The interpulse interval used in our experiments was 35 milliseconds; this was the shortest possible time delay (i.e., minimum pulse interval) between successive pulses at a stimulator intensity of 55% of the maximum stimulator output. This was a hardware limitation of the magnetic stimulator used.

In different blocks of trials, paired-pulse TMS was applied over V1 and right V5/MT at three different delays (50–85, 85–120, and 120–155 ms) from the offset of the first flash (i.e., beginning of the first interval) in Experiments 1, 3, and 5 and from the offset of the second flash (i.e., end of the first interval and beginning of the retention period) in Experiments 2 and 4. In the temporal discrimination tasks (Experiments 1–2), because the presentation order of standard and comparison duration was counterbalanced, TMS was applied on half of the trials in coincidence with the standard (200 ms) and on the other half in coincidence with the comparison duration (200 + ΔT).

In choosing those delays of stimulation, we were motivated by two distinct considerations. First, we wanted delays that in the encoding phase could cover different segments of the temporal window preceding the end of the judged interval (i.e., 200 ms and 200 + ΔT). This choice was motivated by a neurophysiological work in rats showing temporal modulations in the firing rate of V1 neurons preceding the time of an expected reward (Shuler and Bear, 2006). To have equivalent and comparable conditions across experiments, we decided to keep the same delays also in the working memory experiments.

The second consideration relates to the well known links between time and motion perception. Visual motion perception is known to bias the perception of time (Kanai et al., 2006; Brown, 1995), whereas the motion sensitive region V5/MT has been shown to be causally involved in temporal discrimination judgments (Bosco et al., 2008; Bueti et al., 2008). Because of these empirical observations, we decided to remain as close as possible to the timings relevant for visual motion detection (Silvanto et al., 2005).

In all experiments, each experimental session had a total of six TMS blocks i.e., two sites by three different delays, plus two additional blocks of vertex stimulation used as control site for nonspecific effects of TMS, such as acoustic and somatosensory artifacts. The vertex blocks were always the first and the last TMS block, whereas the order of V1 and V5/MT blocks were randomized across subjects. The reason to have just two vertex blocks was purely practical: reducing the length of the experiment and consequently minimizing the fatigue of the subject. During vertex stimulation the TMS pulses were delivered randomly at the three different delays from either interval onset (Experiments 1, 3, and 5) or from interval offset (Experiments 2 and 4). The delays were intermixed in the vertex blocks to have stimulation conditions as close as possible to the visual blocks.

At this point, it is worth emphasizing that although the vertex stimulation is one important control condition in our experiments, it is neither the only nor the most important control. The contrast between TMS blocks of the same area at the three different delays is in itself a crucial control for unspecific TMS artifacts. This difference is indeed the most strict and appropriate because it is between conditions that are identical in every aspect (site, acoustic noise, somatosensory stimulation, and coil position) except for the experimental manipulation (i.e., the delay).

Apart from the general acoustic and somatosensory TMS artifacts there was another aspect in our experimental paradigm that needed control: a potential bias in duration perception induced by the regular sounds produced by the TMS pulses. Psychophysical observations show indeed that trains of acoustic stimuli played regularly just before the presentation of a stimulus can bias the duration perception of that stimulus (Treisman et al., 1990). To prevent this temporal bias, we recorded the sound of a TMS pulse and played it twice via loudspeakers at either the onset (Experiments 1 and 3) or the offset (Experiments 2 and 4) of the second interval. The timings at which this “fake” paired-pulse TMS was played was always congruent with the timing of the real pulses (i.e., 50–85, 85–120, and 120–155 ms). The same fake TMS procedure was used in a previous TMS work on temporal judgments (Bueti et al., 2008).

Finally, to minimize the acoustic impact of both real and fake TMS, participants wore both earplugs and headphones. Although attenuated the sounds were still perceived. The coil handle was oriented upward for V1 stimulation and leftwards with respect to the subject's midline for V5/MT stimulation. Both V1 and V5/MT were localized using a functional method that consisted in eliciting phosphenes from the site (for review, see Walsh and Pascual-Leone, 2003). For V1, the starting point of stimulation was 2 cm dorsal from the inion and for V5/MT it was 2 cm dorsal and 4 cm lateral from the inion. The coil was then moved slightly to find a region from which the clearest static (V1) or moving (V5/MT) phosphenes could be obtained. This location was an average 2.0 cm dorsal and 0.5 cm lateral (to the right) from the inion for V1 and 3 cm dorsal and 5 cm lateral (to the right) to the inion for V5/MT. During the functional localization of V1 and V5/MT, we used single-pulse TMS at stimulation intensity equal to 70% of the maximum stimulator output. This intensity value was chosen to obtain clear phosphenes (Cowey and Walsh, 2000; Silvanto et al., 2005) and was increased only if subjects failed to perceive it.

Data analysis.

For Experiments 1–4 the individual discrimination thresholds (i.e., Weber fractions: ΔT/T) obtained in each V1 and V5/MT TMS block were used to calculate an index of change with respect to the vertex stimulation (i.e., the average of the two vertex blocks) as follows: (site-vertex)/vertex. To check for differences in the discrimination thresholds obtained during visual cortex stimulation with respect to vertex stimulation, we performed on these normalized values 3 two-tailed one-sample t tests (i.e., one for each delay). Then, the same normalized values were entered into a site (V1, V5/MT) by delay (50–85, 85–120, and 120–155 ms) repeated-measures ANOVA to test for any difference in discrimination thresholds between the different visual sites and delays.

In Experiment 5, performance accuracy (% of hits) as well as d′ was computed for each TMS block. Both of these values were first normalized to the average of the two vertex blocks (i.e., [site-vertex]/vertex) and then entered in two separate site (V1,V5/MT) by delay (50–85, 85–120, and 120–155 ms) repeated-measures ANOVA.

Concerning the additional test of the vertex performed in Experiment 1 (N = 7), individual Weber fractions obtained in the different vertex conditions (i.e., intermixed delays, 50–85, 85–120, and 120–155 ms) were first entered in a one-way ANOVA to test for differences between the different vertex stimulation conditions, and then used to perform a delay (50–85, 85–120, and 120–155 ms) by site (V1,V5/MT, vertex) ANOVA to check for differences between the different delays and stimulation conditions.

As post hoc tests, we used two-tailed paired t tests. For all t tests (one-sample and paired) the α value was set to 0.05 and the Bonferroni correction for multiple comparisons was applied (three comparisons leads to a p corrected <0.016).

Results

In Experiments 1 and 2, healthy participants performed a temporal discrimination task. Experiment 1 (N = 14) investigated the functional role and the temporal interplay between V1 and V5/MT during the encoding of temporal intervals. Paired-pulse TMS (interpulse interval = 35 ms) was applied on different blocks of trials over right V1 and V5/MT at three different delays (50, 85, and 120 ms) from the onset of the first interval (i.e., offset of the first flash). Compared with vertex stimulation and with the earliest stimulation delay (i.e., 50 ms; Table 1), discrimination thresholds were significantly higher following TMS of both V1 and V5/MT at delays of 85 and 120 ms (effect of delay: F(2,26) = 18.39, p < 0.001; effect of site: F(1,13) = 0.10, p = 0.92; interaction delay by site: F(2,26) = 0.25, p = 0.78; Fig. 1B, left). We also observed a significant difference between vertex stimulation and the no-TMS condition (t(13) = 2.72, p = 0.02; Fig. 1B, left; nonnormalized data). These changes in discrimination threshold following stimulation of V1 and V5/MT were positively correlated across subjects (Fig. 1C, left; r = 0.36, p = 0.10 [r2 = 0.13, b = 0.35, p = 0.21, stderr = 0.43] and r = 0.68, p = 0.004 [r2 = 0.46, b = 0.60, p = 0.008, stderr = 0.34] for the 85 and 120 ms delays, respectively) indicating a possible functional coupling between these two areas.

Table 1.

T values of paired (left) and one-sample (right) t test computed on discrimination thresholds (i.e., Weber fractions: ΔT/T) normalized to the vertex stimulation

TMS delays Paired t test
 One-sample t test
1–2 1–3 2–3 1 2 3
Exp 1
    V1 2.27 4.32 1.76 −1.23 1.91 4.40
0.01** <0.001** 0.045* 0.88 0.03* <0.001**
    V5 2.03 3.02 1.01 −0.49 2.18 3.33
0.03* 0.003** 0.16 0.68 0.02* 0.003**
Exp 2
    V1 2.64 0.82 −1.60 1.93 −1.85 0.69
0.007** 0.21 0.94 0.04* 0.96 0.25
    V5 2.61 1.19 1.09 −1.01 2.70 0.72
0.007** 0.12 0.14 0.83 0.01** 0.24
Open in a new tab

Degrees of freedom = 13;

*p values uncorrected,

**p values Bonferroni corrected for multiple comparisons.

At this point we wondered whether the difference in discrimination thresholds observed between vertex stimulation and V1 and V5/MT was due to the difference between these conditions: during the vertex stimulation the delays were intermixed, whereas during the stimulation of visual areas each delay was tested in a different block of trials. For this reason, we decided to test in a subgroup (N = 7) of volunteers who took part in Experiment 1, the vertex stimulation at the three different delays. The results of this new testing revealed first, the absence of a significant difference between the different vertex conditions (one-way ANOVA, F(3,24) = 0.1, p = 0.96) and second, a substantial replication of the original results. Discrimination thresholds were on average higher after TMS of V1 and V5/MT compared with vertex stimulation (effect of site, F(2,12) = 5.77, p = 0.02; p = 0.08 and p = 0.02 for respectively V1 and V5/MT), and were on average higher at the last compared with the earliest delay (effect of delay, F(2,12) = 4.21, p = 0.04; last vs earliest p = 0.05). We also found significantly higher discrimination thresholds for V1 stimulation at the intermediate and the latest delay. The same effects, although only marginally significant, were also found for V5/MT stimulation (interaction site by delay; F(4,24) = 1.76, p = 0.17).

These last results clearly support our previous findings by strengthening the hypothesis that the observed increase of discrimination thresholds during interval encoding was genuinely due to an interference with temporal processing in visual cortices rather than caused by unspecific differences between the different stimulation conditions.

Although Experiment 1 showed that V1 and V5/MT were both causally involved in the accurate encoding of temporal intervals in a temporally simultaneous manner, they did not clarify whether these areas also play a role in the retention period, when the first interval has just been presented and needs to be retained in memory before the presentation of the second interval. To address this issue we asked a different group of healthy participants (N = 14) to perform the same discrimination task, with the sole difference being that paired-pulse TMS was applied at three different delays from the offset of the first interval (i.e., offset of the second flash). Compared with vertex stimulation and with different stimulation delays we observed higher discrimination thresholds following TMS over V1 at the 50 ms stimulation delay and following TMS over V5/MT at the 85 ms stimulation delay (Table 1; Fig. 1B, right; interaction site by delay: F(2,26) = 6.75, p = 0.004; effect of site: F(1,13) = 0.11, p = 0.75; effect of delay: F(2,26) = 0.85, p = 0.92). In contrast to the results during the encoding phase, there was no evidence of correlated effects across subjects (r = 0.18, p = 0.27 [r2 = 0.03, b = 0.23, p = 0.54, stderr = 0.41]; Fig. 1C, right), suggesting functional independence of these two areas during working memory maintenance of temporal information.

To ensure that the observed effects during time encoding and working memory maintenance were not caused by interference with either low-level visual processing or with unspecific task requirements, we conducted three additional experiments. In Experiments 3 and 4, we used the same task structure of Experiments 1 and 2 but we changed adaptively the brightness of one of the four visual markers and asked participants to decide which pair of disks was on average brighter. Paired-pulse TMS was applied at either the encoding (Experiment 3; N = 10) or the working memory (Experiment 4; N = 14) stage of the task. In both experiments we found that TMS was ineffective in disrupting brightness discrimination threshold at any site or stimulation delay (Fig. 2; Experiment 3, effect of site: F(1,9) = 0.57, p = 0.47; effect of delay: F(2,18) = 0.40, p = 0.68; interaction site by delay: F(2,18) = 0.26, p = 0.77; Experiment 4, effect of site: F(1,13) = 2.05, p = 0.17; effect of delay: F(2,26) = 1.06, p = 0.36; interaction site by delay: F(2,26) = 0.59, p = 0.56).

Figure 2.

Figure 2.

Open in a new tab

Results of the visual nontemporal tasks. Average (SEM indicated) of individual discrimination thresholds for V1 and V5/MT paired-pulse TMS at the three different delays (different shades of blue and red for V1 and V5/MT, respectively) after either interval onset (encoding) or offset (working memory). Top, Thresholds are indexed as the change with respect to the vertex stimulation (i.e., average of the two vertex blocks) as follows: site-vertex/vertex. Bottom, Averaged thresholds not normalized to the vertex; gray and white bar are, respectively, the vertex and no-TMS condition.

Finally, to make sure that the TMS effects in the temporal discrimination tasks were not due to an interference with the visual perception of the stimuli, we ran a fifth experiment (Experiment 5, N = 10) in which we tested simple visual detection under experimental conditions identical to those of the temporal discrimination tasks (Experiments 1–2, see Materials and Methods). In this last experiment we found d′ values well above 0 in all experimental conditions (on average 4.2, range 3.1–4.5) and a performance accuracy that was on average 99% (range 95–100%, stdev = 0.3%). Both performance accuracy and d′ were better after V1 stimulation compared with V5/MT (effect of site: F(1,9) = 5.01, p = 0.05; F(1,9) = 6.45, p = 0.03 for respectively, accuracy and d′ data) but did not differ at the different delays (effect of delay: F(2,18) = 0.30, p = 0.74; F(2,18) = 0.04, p = 0.66 for respectively, accuracy and d′ data; interaction delay by site: F(2,18) = 0.31, p = 0.97; F(2,18) = 0.84, p = 0.92, for respectively, accuracy and d′ data).

Overall the results of these last three experiments indicate that the effects observed in Experiments 1 and 2 were not simply due to interference with low-level visual processing or with unspecific task requirements. Instead and in general, the role played by both V1 and V5/MT during encoding and working memory appears genuinely linked to temporal processing and not to aspects of the task common to any discrimination processes.

Discussion

The results of the five experiments showed that V1 and extrastriate area V5/MT were causally involved in encoding and keeping time in memory and that this involvement was independent from unspecific task requirements and low-level visual processing. The two visual areas became involved in interval encoding simultaneously and their engagement in this phase of the task correlated across subjects. During the short-term memory of temporal information the two areas came into play at different times and their engagement was uncorrelated across subjects.

Compared with previous neuroimaging and electrophysiological works (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bueti et al., 2010), here we were able to show the causal contribution of both V1 and V5/MT to different aspects of temporal processing i.e., temporal encoding and working memory maintenance. The contribution of distinct brain regions to these two different functional stages of temporal computation has not been studied before. The engagement of V1 and V5/MT in duration encoding has been suggested by previous electrophysiological and neuroimaging data (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bueti et al., 2010). However, none of these previous works had clarified that V1 and V5/MT play a causal role in temporal processing, that this role is at the encoding stage and it is independent of low-level visual processing. Only two previous TMS studies have shown a causal relationship between V5/MT activity and temporal discrimination judgments (Bosco et al., 2008; Bueti et al., 2008). However, the first of these studies failed to clarify the computational stage at which V5/MT is engaged during duration discrimination (Bueti et al., 2008); the second failed to demonstrate that the role played by V5/MT during stimulus encoding was time specific (Bosco et al., 2008).

V1 and V5/MT became involved in time encoding simultaneously, 85 ms after interval onset. Interestingly, this engagement became stronger at later times (120–155 ms from interval onset) close to the “target” duration. Our hypothesis, given the relatively early engagement of these areas in temporal encoding, is that time signals are generated locally, in the neural activity of V1 and V5/MT (Buonomano and Maass, 2009; Johnson et al., 2010). Hypotheses relative to time coding in these areas come from animal electrophysiological and human neuroimaging. These studies show that activity in visual cortices increases monotonically, whereas temporal information is tracked and peaks at the expected time of a behaviorally relevant event or reward (Ghose and Maunsell, 2002; Shuler and Bear, 2006; Bueti et al., 2010). In particular, three classes of “reward timing” neurons have been described in rats V1: neurons that show a sustained increase of response to reward time, neurons that show a sustained decrease of response to reward time, and finally neurons that peak at the time of the reward. Our results are compatible with a temporal coding via either the first or the third class of these neurons. If time is encoded via either a sustained increase or a peak of neural activity at the time of the target duration, TMS stimulation should be more effective if delivered toward the end of the interval, close to the time of the maximum engagement of these areas in the temporal process. If the stimulation occurs too early, when the areas are either not fully engaged in the process or when they have enough time to recover from the stimulation, the behavioral effects of TMS are expected to be null (Walsh and Cowey, 2000). This last point is crucial to explain the null effect at the earliest delay and the stronger effects while approaching the target duration.

According to the hypothesis of time encoded in the firing rate of neurons and to the above considerations relative to the effectiveness of TMS, the timing of the TMS interference are strongly dependent from the target duration. If the target duration would be longer we expect the most effective delays to be later too. Concerning the timing of the TMS interference and its interpretation, a few considerations are important here. The time course of paired-pulse TMS on human visual cortex is still unclear (Hoffken et al., 2008; Kimura et al., 2013). In cats' primary visual cortex, suprathreshold paired-pulse TMS has been shown to affect visual-evoked single unit responses for at least 150–200 ms after TMS onset (Moliadze et al., 2005). Although TMS frequency and intensity used here were lower than those used in this electrophysiological work, we cannot exclude the possibility that the neurophysiological effects of each pair persisted for longer than the time-window stimulated. This is not an insurmountable problem for the first two delays of the encoding experiment where, assuming that magnetic stimulation effects lasted for 150 ms after TMS onset, we were still within the range of the target interval (i.e., extended effects <270 ms; T = 200 ms and T + ΔT was on average 266 ms, ranging from 220 to 300 ms). For the last delay of the same experiment (120–155 ms extended effects = 305 ms), we admit that the TMS effects might have extended to the interstimulus interval. The timing of TMS effects were less problematic for the working memory experiments, where the total duration of the interstimulus interval exceeded the duration of the paired-pulse effects.

The fact that both V1 and V5/MT were simultaneously involved in duration encoding suggests a form of redundancy in this process. This result is line with neuroimaging data showing that V1 and extrastriate areas V2/V3 respond to temporal expectations with analogous activation profiles (Bueti et al., 2010). The reason of this redundancy is unclear and our results cannot establish whether these areas perform similarly or different computations or whether their activity is functionally synchronized. The fact that the TMS effects over V1 and V5/MT correlated across subjects seems to suggest though the existence this functional link.

The involvement of V1 and V5/MT during the short-term memory stage of the task is also a new finding. Previous electrophysiological works have mainly focused on the encoding stage of temporal computation (Ghose and Maunsell, 2002; Leon and Shadlen, 2003; Shuler and Bear, 2006), whereas neuroimaging studies, due to the poor temporal resolution of the technique, have failed to draw a clear distinction between time encoding and short-term memory (Rao et al., 2001; Coull et al., 2008). Although it has never been shown before, the role of sensory-specific cortices in temporal short-term memory has been suggested by many psychophysical observations showing that the capacity to keep time in memory depends on the sensory modality of the temporal signals (Penney et al., 2000; Gamache and Grondin, 2010; Ogden et al., 2010; Rattat and Picard, 2012; Takahashi and Watanabe, 2012). Our data, in line with these behavioral findings, represent the first demonstration that sensory-specific visual regions V1 and V5/MT are engaged in the short-tem memory of visual temporal intervals.

Although V1 and V5/MT are simultaneously active in interval encoding, they work with different temporal dynamics (i.e., first V1 and subsequently V5/MT) during working memory. These timing differences may suggest that V1 and V5/MT process different aspects of the temporal information and that the two areas are functionally linked. Alternatively, we can hypothesize that the computations in the two areas are identical and that V1 and V5/MT are completely independent. This last possibility seems to be supported by the correlational results. Compared with the encoding phase, the TMS effects over V1 and V5/MT were indeed uncorrelated across subjects. This means that the magnitude of the effects in one area was independent from the magnitude of the effects in the other one. These differences might be linked to the different strategies used to keep time in memory, which may in turn reflect the existence of separate and independent routes for temporal memory (Takahashi and Watanabe, 2012). Further experiments are necessary to better specify the nature of these different memory pathways and to clarify the potential differences between V1 and V5/MT at the encoding stage.

An important aspect of our findings is represented by the temporal specificity of the observed effects. The stimulation of V1 and V5/MT at the three different delays did not interfere with either stimulus detection or brightness discrimination at any stage of the visual processing. This result may appear bizarre in the first place, especially because the stimulation delays used in our experiments were those that previous TMS works have indicated as critical to disrupt visual perception (Amassian et al., 1989). A possible explanation for the lack of any TMS effect in our nontemporal visual tasks is that the stimulus intensity used was below that required to interfere with psychophysical performance and to induce scotomas (i.e., stimulus intensity required >70% of the maximum stimulator output; Amassian et al., 1989; Kamitani and Shimojo, 1999; Kammer et al., 2005). The low-stimulus intensity coupled with easily detectable visual features was likely the cause of the null effect in the simple detection task.

Overall we can conclude that the stimulation timings and the TMS parameters used in our experiments, although were not sufficient to disrupt low level visual processing like brightness discrimination and stimulus detection, were powerful enough to interfere with the processing of the temporal aspects of visual information.

Footnotes

Financial support has been provided by the Italian Ministry of Health (Grant RF-2010-2319059 to D.B.) and by the Swiss National Science Foundation (Grant #3100B0-133136 to M.M.M.).

The authors declare no competing financial interests.

References

  1. Amassian VE, Cracco RQ, Maccabee PJ, Cracco JB, Rudell A, Eberle L. Suppression of visual perception by magnetic coil stimulation of human occipital cortex. Electroencephalogr Clin Neurophysiol. 1989;74:458–462. doi: 10.1016/0168-5597(89)90036-1. [DOI] [PubMed] [Google Scholar]
  2. Amassian VE, Maccabee PJ, Cracco RQ, Cracco JB, Somasundaram M, Rothwell JC, Eberle L, Henry K, Rudell AP. The polarity of the induced electric field influences magnetic coil inhibition of human visual cortex: implications for the site of excitation. Electroencephalogr Clin Neurophysiol. 1994;93:21–26. doi: 10.1016/0168-5597(94)90087-6. [DOI] [PubMed] [Google Scholar]
  3. Bosco G, Carrozzo M, Lacquaniti F. Contributions of the human temporoparietal junction and MT/V5+ to the timing of interception revealed by transcranial magnetic stimulation. J Neurosci. 2008;28:12071–12084. doi: 10.1523/JNEUROSCI.2869-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brown SW. Time, change, and motion: the effects of stimulus movement on temporal perception. Percept Psychophys. 1995;57:105–116. doi: 10.3758/bf03211853. [DOI] [PubMed] [Google Scholar]
  5. Bueti D, Bahrami B, Walsh V. Sensory and association cortex in time perception. J Cogn Neurosci. 2008;20:1054–1062. doi: 10.1162/jocn.2008.20060. [DOI] [PubMed] [Google Scholar]
  6. Bueti D, Bahrami B, Walsh V, Rees G. Encoding of temporal probabilities in the human brain. J Neurosci. 2010;30:4343–4352. doi: 10.1523/JNEUROSCI.2254-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bueti D, Lasaponara S, Cercignani M, Macaluso E. Learning about time: plastic changes and interindividual brain differences. Neuron. 2012;75:725–737. doi: 10.1016/j.neuron.2012.07.019. [DOI] [PubMed] [Google Scholar]
  8. Buonomano DV, Maass W. State-dependent computations: spatiotemporal processing in cortical networks. Nat Rev Neurosci. 2009;10:113–125. doi: 10.1038/nrn2558. [DOI] [PubMed] [Google Scholar]
  9. Coull JT, Nazarian B, Vidal F. Timing, storage, and comparison of stimulus duration engage discrete anatomical components of a perceptual timing network. J Cogn Neurosci. 2008;20:2185–2197. doi: 10.1162/jocn.2008.20153. [DOI] [PubMed] [Google Scholar]
  10. Cowey A, Walsh V. Magnetically induced phosphenes in sighted, blind and blindsighted observers. Neuroreport. 2000;11:3269–3273. doi: 10.1097/00001756-200009280-00044. [DOI] [PubMed] [Google Scholar]
  11. Droit-Volet S, Izaute M. The effect of feedback on timing in children and adults: the temporal generalization task. Q J Exp Psychol A. 2005;58:507–520. doi: 10.1080/02724980443000025. [DOI] [PubMed] [Google Scholar]
  12. Gamache PL, Grondin S. The lifespan of time intervals in reference memory. Perception. 2010;39:1431–1451. doi: 10.1068/p6652. [DOI] [PubMed] [Google Scholar]
  13. Gerwig M, Niehaus L, Kastrup O, Stude P, Diener HC. Visual cortex excitability in migraine evaluated by single and paired magnetic stimuli. Headache. 2005;45:1394–1399. doi: 10.1111/j.1526-4610.2005.00272.x. [DOI] [PubMed] [Google Scholar]
  14. Ghose GM, Maunsell JH. Attentional modulation in visual cortex depends on task timing. Nature. 2002;419:616–620. doi: 10.1038/nature01057. [DOI] [PubMed] [Google Scholar]
  15. Höffken O, Grehl T, Dinse HR, Tegenthoff M, Bach M. Paired-pulse behavior of visually evoked potentials recorded in human visual cortex using patterned paired-pulse stimulation. Exp Brain Res. 2008;188:427–435. doi: 10.1007/s00221-008-1374-0. [DOI] [PubMed] [Google Scholar]
  16. Johnson HA, Goel A, Buonomano DV. Neural dynamics of in vitro cortical networks reflects experienced temporal patterns. Nat Neurosci. 2010;13:917–919. doi: 10.1038/nn.2579. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Kamitani Y, Shimojo S. Manifestation of scotomas created by transcranial magnetic stimulation of human visual cortex. Nat Neurosci. 1999;2:767–771. doi: 10.1038/11245. [DOI] [PubMed] [Google Scholar]
  18. Kammer T, Baumann LW. Phosphene thresholds evoked with single and double TMS pulses. Clin Neurophysiol. 2010;121:376–379. doi: 10.1016/j.clinph.2009.12.002. [DOI] [PubMed] [Google Scholar]
  19. Kammer T, Puls K, Erb M, Grodd W. Transcranial magnetic stimulation in the visual system: II. Characterization of induced phosphenes and scotomas. Exp Brain Res. 2005;160:129–140. doi: 10.1007/s00221-004-1992-0. [DOI] [PubMed] [Google Scholar]
  20. Kanai R, Paffen CL, Hogendoorn H, Verstraten FA. Time dilation in dynamic visual display. J Vis. 2006;6(12):1421–1430. doi: 10.1167/6.12.8. [DOI] [PubMed] [Google Scholar]
  21. Kanai R, Lloyd H, Bueti D, Walsh V. Modality-independent role of the primary auditory cortex in time estimation. Exp Brain Res. 2011;209:465–471. doi: 10.1007/s00221-011-2577-3. [DOI] [PubMed] [Google Scholar]
  22. Kimura T, Katzuya O, Tobimatsu S. Repetitive paired-pulse transcranial magnetic stimulation over the visual cortex alters visual recovery function. Brain Stimul. 2013;6:298–305. doi: 10.1016/j.brs.2012.05.005. [DOI] [PubMed] [Google Scholar]
  23. Leon MI, Shadlen MN. Representation of time by neurons in the posterior parietal cortex of the macaque. Neuron. 2003;38:317–327. doi: 10.1016/S0896-6273(03)00185-5. [DOI] [PubMed] [Google Scholar]
  24. Levitt H. Transformed up-down methods in psychoacoustics. J Acoust Soc Am. 1971;49(Suppl 2):467. [PubMed] [Google Scholar]
  25. Moliadze V, Giannikopoulos D, Eysel UT, Funke K. Paired-pulse transcranial magnetic stimulation protocol applied to visual cortex of anaesthetized cat: effects on visually evoked single-unit activity. J Physiol. 2005;566:955–965. doi: 10.1113/jphysiol.2005.086090. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Morrone MC, Ross J, Burr D. Saccadic eye movements cause compression of time as well as space. Nat Neurosci. 2005;8:950–954. doi: 10.1038/nn1488. [DOI] [PubMed] [Google Scholar]
  27. Ogden RS, Wearden JH, Jones LA. Are memories for duration modality specific? Q J Exp Psychol (Hove) 2010;63:65–80. doi: 10.1080/17470210902815422. [DOI] [PubMed] [Google Scholar]
  28. Pascual-Leone A, Walsh V. Fast backprojections from the motion to the primary visual area necessary for visual awareness. Science. 2001;292:510–512. doi: 10.1126/science.1057099. [DOI] [PubMed] [Google Scholar]
  29. Penney TB, Gibbon J, Meck WH. Differential effects of auditory and visual signals on clock speed and temporal memory. J Exp Psychol Hum Percept Perform. 2000;26:1770–1787. doi: 10.1037/0096-1523.26.6.1770. [DOI] [PubMed] [Google Scholar]
  30. Rao SM, Mayer AR, Harrington DL. The evolution of brain activation during temporal processing. Nat Neurosci. 2001;4:317–323. doi: 10.1038/85191. [DOI] [PubMed] [Google Scholar]
  31. Rattat AC, Picard D. Short-term memory for auditory and visual durations: evidence for selective interference effects. Psychol Res. 2012;76:32–40. doi: 10.1007/s00426-011-0326-7. [DOI] [PubMed] [Google Scholar]
  32. Ray PG, Meador KJ, Epstein CM, Loring DW, Day LJ. Magnetic stimulation of visual cortex: factors influencing the perception of phosphenes. J Clin Neurophysiol. 1998;15:351–357. doi: 10.1097/00004691-199807000-00007. [DOI] [PubMed] [Google Scholar]
  33. Shuler MG, Bear MF. Reward timing in the primary visual cortex. Science. 2006;311:1606–1609. doi: 10.1126/science.1123513. [DOI] [PubMed] [Google Scholar]
  34. Silvanto J, Lavie N, Walsh V. Double dissociation of V1 and V5/MT activity in visual awareness. Cereb Cortex. 2005;15:1736–1741. doi: 10.1093/cercor/bhi050. [DOI] [PubMed] [Google Scholar]
  35. Sparing R, Dambeck N, Stock K, Meister IG, Huetter D, Boroojerdi B. Investigation of the primary visual cortex using short-interval paired-pulse transcranial magnetic stimulation (TMS) Neurosci Lett. 2005;382:312–316. doi: 10.1016/j.neulet.2005.03.036. [DOI] [PubMed] [Google Scholar]
  36. Takahashi K, Watanabe K. Short-term memory for event duration: modality specificity and goal dependency. Atten Percept Psychophys. 2012;74:1623–1631. doi: 10.3758/s13414-012-0347-3. [DOI] [PubMed] [Google Scholar]
  37. Treisman M, Faulkner A, Naish PL, Brogan D. The internal clock: evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. Perception. 1990;19:705–743. doi: 10.1068/p190705. [DOI] [PubMed] [Google Scholar]
  38. Walsh V, Cowey A. Transcranial magnetic stimulation and cognitive neuroscience. Nat Rev Neurosci. 2000;1:73–79. doi: 10.1038/35036239. [DOI] [PubMed] [Google Scholar]
  39. Walsh V, Pascual-Leone A. Transcranial magnetic stimulation: a neurochronometrics of mind. Cambridge, Mass: MIT; 2003. [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES