The Temporoparietal Junction as a Part of the “When” Pathway

Accurate perception of the timing of sensory events is crucial in normal functioning and plays a key role in numerous higher-order cognitive functions, ranging from speech comprehension to motor coordination. Accordingly, poor temporal processing has been implicated in several psychiatric and neurologic conditions, including dyslexia, schizophrenia, autism, and attention deficit disorder (Buhusi and Meck, 2005). Although the encoding of temporal information on the subsecond time scale has been extensively studied over the last several decades using temporal order judgment paradigms, its neural basis remains largely unresolved, because the vast majority of this research involved psychophysical and/or electrophysiological methods with poor spatial resolution (Lewandowska et al., 2008). A recent study by Davis et al. (2009) constitutes a major advance toward understanding how the brain encodes temporal information. First, it helps to resolve conflicting results of previous literature regarding the relative roles of the left and right hemispheres in the processing of event order (Battelli et al., 2007). Some of these inconsistencies stem from divergent theories concerning hemispheric specialization in attention and the resultant influence of any such lateralization on temporal order judgments (e.g., Spence et al., 2001). Second, Davis et al. (2009) provide new insight into the existence of a “when” processing stream and shed light on its multisensory nature.

Using functional magnetic resonance imaging, Davis et al. (2009) identified brain regions involved in judging the temporal order of rapidly presented, spatially separated visual stimuli. In experiment 1, subjects were presented with sequences of two rectangles (one red, one green), the onset, size, and location of which were randomized across trials. Before each trial, the task subjects had to perform was cued by the color of a centrally presented fixation point. In the shape task, participants were instructed to report whether the red or the green rectangle was more square. In the temporal order judgment condition, they reported which of the two items appeared first. This paradigm allowed the researchers to compare the brain's responses to physically identical stimulation; the experimental condition being defined by the feature (shape or timing) on which subjects were instructed to focus their attention. The results of the statistical contrast between the two conditions revealed differences only between the tasks in which subject were engaged, not differences between the stimuli.

A possible confound of experiment 1 (that was acknowledged by the authors) was that resolving the temporal order required subjects to process differences in the onset of the items, whereas such temporal selectivity was not necessary in the shape discrimination task. In addition, the shape discrimination task called for longer visual search than the temporal order judgment task. These differences in the temporal window during which attention was required in the two tasks, as well as differences in the duration of visual search in which the participants engaged, were controlled in a second experiment. In experiment 2, attention to stimulus onset was necessary in both the temporal order and shape condition. The design was similar to that in experiment 1, except that the shape of the rectangles was kept constant and that a gray line of variable width was initially superimposed on experiment 1 items. The shape task of experiment 2 was based on these gray lines, thereby compelling participants to attend to the onset of the items, as in the temporal order judgment condition. This control experiment is of particular importance in a temporal order judgment paradigm, as the perception of succession has been shown to be strongly influenced by attentional factors (Eagleman, 2008). It is noteworthy that the relative luminance of the red, green, and gray colors used by Davis et al. (2009) were individually adjusted to ensure that the items' salience did not bias responses. Moreover, the difficulty of the shape and temporal order judgment tasks was matched in a pilot study to avoid possible confounds induced by differences in attentional load across experimental conditions.

Behaviorally, the results of experiments 1 and 2 indicate that the difficulty levels of both temporal order judgment and shape discrimination tasks were comparable (∼85%), indicating that differences in brain activity related to the two experimental conditions were unlikely attributable to differences in task difficulty.

Brain activation in experiment 1 included stronger bilateral activity in the temporoparietal junction, right frontotemporal cortices, and frontal eye fields in the temporal order judgment task than in the shape task [Davis et al. (2009), their Fig. 2]. However, as mentioned above, the possibility that this statistical contrast reflects differences in the length of visual search required in the two tasks (i.e., subjects had to pay attention to the stimulus onset in the temporal order judgment but not shape task) cannot be excluded. In experiment 2, in which this potential confound was controlled for by matching the temporal window over which the task-relevant information could be extracted in the two tasks (i.e., the onset of the stimuli), a similar activation pattern to that in experiment 1 was observed except that the temporoparietal junction activity was restricted to the left hemisphere [Davis et al. (2009), their Fig. 3].

By providing direct evidence for the brain regions involved in processing the order of spatially separated visual events, the study by Davis et al. (2009) is a major step toward understanding the functional organization underlying temporal processing. Hence, their results contribute to a better understanding of how the brain integrates the timing of sensory events, an issue receiving growing interest, as it becomes more evident that the temporal dimension is important in sensory and motor processing (Battelli et al., 2007).

In contrast to the classical conception that the anatomofunctional organization of the visual system consists of two main processing streams subserving the processing of semantic (what) and spatial/action-oriented (where) information, Davis et al. (2009) provide strong support for the emerging view that the encoding of time relies on a third, partially segregated, “when” network. By contrasting brain activity associated with a “what” (shape discrimination) and a “when” (temporal order judgment) task, the authors demonstrated that the processing of the temporal features is not intrinsic to “what” processing, but rather relies on a partially independent module comprised within the temporoparietal junction.

More importantly, the results of experiment 2 revealed the selective involvement of the left temporoparietal junction in temporal order judgment. This result not only extends, but also runs counter to previous transcranial magnetic stimulation (TMS) and neuropsychological findings attributing a crucial role to right inferior parietal cortex and/or inferior parietal lobule in the perception of events' order (Battelli et al., 2008). The new insight provided by Davis et al. (2009) into the issue of hemispheric dominance in fast temporal processing was enabled by the joint use of a correlational neuroimaging method and their experimental design, which avoided two confounds. First, contrary to previous TMS or neuropsychological studies that tested for the role of a predetermined brain area of interest in temporal order judgment (i.e., right temporoparietal regions); Davis et al. (2009) screened for temporal order judgment-related activity throughout the whole brain, in a purely data-driven approach, allowing them to reveal unexpected activation patterns. Second, because the regions of interest selected in the lesion or TMS studies are also involved in spatial attention and visual search duration (Shulman et al., 2007), right temporoparietal junction interference possibly impacted temporal order judgment by perturbing top-down spatial attentional processes, resulting in misinterpretation of the role of right hemispheric structures in temporal order judgment [see also Wittmann et al. (2004) for discussion]. The control for attentional factors used in Davis et al. (2009) is particularly important because previous data revealed that attentional variables can induce striking distortion in temporal order judgment (Eagleman, 2008). This hypothesis is supported by the finding that right temporoparietal junction activity was not observed in experiment 2 of Davis et al. (2009), in which the attentional window required to resolve the tasks was balanced across the temporal order judgment and shape discrimination tasks. Davis et al. (2009) correctly argue that the effects of attention could be discerned in TMS-induced and real lesion studies as such induces biases but not decreases in temporal order judgment performance. Additional ways for controlling the influence of spatial attention include avoiding the use of spatially varying stimuli for the temporal order judgment task by presenting stimuli at a single location and/or along the midline; both of which would minimize a potential interaction between the location of the stimuli and any hemispheric specialization for the processing of the attended feature or for spatial attention in general.

As mentioned above, the results of Davis et al. (2009) are relevant with regard to current models of the anatomofunctional organization of the visual system. Given the evidence suggesting the organization of sensory processing in dual what/where streams constitutes a general principle across sensory systems that might encompass multisensory aspects [see, e.g., De Santis et al. (2007) for discussion], it could be hypothesized that “when” processing follows the same architecture. While the temporal order judgment task by Davis et al. (2009) has been conducted in the visual modality, the striking result of a specific role they found for the left temporoparietal junction in temporal order judgment provides insight into the multisensory nature of the “when” pathway. Damage in the left temporoparietal junction causes Wernicke's aphasia, a syndrome characterized by language comprehension deficits, which in turn have been explained by impaired integration of the order within and/or between phonemes or more generally in auditory temporal order judgment (von Steinbüchel et al., 1999). Impaired responsiveness along the posterior superior temporal cortex and temporoparietal junction has also been linked to deficient integration of letters and speech sounds in dyslexic individuals (Blau et al., 2009), who often also exhibit impaired temporal processing (Tallal, 2004). Together, lesion data on temporal order judgment and the study by Davis et al. (2009) therefore suggest that the left temporoparietal junction may comprise a multisensory (or at least auditory and visual) temporal order processing unit within an extended “when” pathway. Although further investigations are needed to confirm that the left temporoparietal junction indeed encodes temporal order in a multisensory manner, the work by Davis et al. (2009) supports this hypothesis.