The neurophysiology of human biological motion processing: A high-density electrical mapping study

Abstract

The neural processing of biological motion (BM) is of profound experimental interest since it is often through the movement of another that we interpret their immediate intentions. Neuroimaging points to a specialized cortical network for processing biological motion. Here, high-density electrical mapping and source-analysis techniques were employed to interrogate the timing of information processing across this network. Participants viewed point-light-displays depicting standard body movements (e.g. jumping), while event-related potentials (ERPs) were recorded and compared to ERPs to scrambled motion control stimuli. In a pair of experiments, three major phases of BM-specific processing were identified: 1) The earliest phase of BM-sensitive modulation was characterized by a positive shift of the ERP between 100 and 200 ms after stimulus onset. This modulation was observed exclusively over the right hemisphere and source-analysis suggested a likely generator in close proximity to regions associated with general motion processing (KO/hMT). 2) The second phase of BM-sensitivity occurred from 200 to 350 ms, characterized by a robust negative-going ERP modulation over posterior middle temporal regions bilaterally. Source-analysis pointed to bilateral generators at or near the posterior superior temporal sulcus (STS). 3) A third phase of processing was evident only in our second experiment, where participants actively attended the BM aspect of the stimuli, and was manifest as a centro-parietal positive ERP deflection, likely related to later cognitive processes. These results point to very early sensory registration of biological motion, and highlight the interactive role of the posterior STS in analyzing the movements of other living organisms.

Introduction

Humans, and indeed all creatures, have a need to rapidly detect and process sensory percepts that suggest the presence of another living organism. Perhaps one of the richest sources of such information comes through visual processing of the movements of others, commonly known as “biological motion” (BM). In recent years, considerable effort has gone into trying to understand the neural underpinnings of biological motion processing, in large part because it appears to be disordered in a number of clinical populations such as those with schizophrenia (e.g. Kim et al., 2005) or autism (e.g. Blake et al., 2003; See, however, Freitag et al., 2008; Parron et al., 2008). Behavioral and neuroimaging data have demonstrated, at least in humans, a profound sensitivity to everything from gender (e.g. Mather and Murdoch, 1994; Troje, 2002) to mood (e.g. Atkinson et al., 2007; Pollick et al., 2001, 2002), even in cases of highly impoverished information, such as those using Johansson’s (1973) point-light displays (PLDs). In these displays, the movement of a body is reduced to the motion of dots that represent the key joints. The purpose of the current study was to use high-density electrical mapping to assess the relative timing of BM processing, and to assess the role of attention in this processing. First, we will briefly review what is already known from neuroimaging and electrophysiological studies.

Brain circuits and biological motion

Most recent BM research has focused on localizing the cortical and subcortical brain areas involved in general BM processing (for an excellent review, see Blake and Shiffrar, 2007). Although quite a number of regions have been implicated thus far, the area most prominently associated with BM processes is the posterior superior temporal sulcus (pSTS) (e.g. Bonda et al., 1996; Puce et al., 1998; Grossman et al., 2000), with some evidence suggesting a right-hemisphere bias in pSTS (e.g. Peuskens et al., 2005). Another area often implicated is the nearby extrastriate body area (EBA), which is also active during the processing of static images of the human body (e.g. Downing et al., 2001; Taylor et al., 2007). EBA was shown to be more strongly activated for canonical biological motion than for scrambled PLDs (Peelen et al., 2006; see however Grossman and Blake, 2002), although Downing et al. (2006) have suggested that stronger activation of EBA to BM stimuli might simply reflect that EBA is involved in static structural information processing rather than actual “motion dynamics”.

Some debate also surrounds the roles of general motion processing areas, such as the human homolog of the middle temporal gyrus in monkeys (hMT/V5) and the kinetic occipital (KO) region (located posterior and medial to hMT). For example, a number of studies have reported differential activation of KO for BM stimuli (e.g. Vaina et al., 2001; Santi et al., 2003; Peuskens et al., 2005). Similarly, Vaina et al. (2001) and Ptito et al. (2003) both found significant BM-related effects in area hMT. In contrast, Grossman and Blake (2002) and Downing et al. (2001) found no significant differences between canonical BM stimuli and their scrambled counterparts in these regions. Perhaps most compellingly, Grossman et al. (2005) reported that while transcranial magnetic stimulation (TMS) over STS impaired BM perception, it had no effect on BM perception when applied over hMT.

A number of additional form-processing cortical regions have also been implicated, including the fusiform gyrus (FFG) and the occipital face area (OFA) (e.g. Vaina et al., 2001; Grossman and Blake, 2002; Michels et al., 2005). Similarly, Vaina et al. (2001) reported activation of the ventral surface of the temporal lobe. Beauchamp et al. (2003) found this activation to be more pronounced for whole body displays than for PLDs. Michels et al. (2005) found that areas traditionally associated with the processing of static human images were differentially activated by different levels of form information in their BM stimuli. In contrast, activation in these areas remained unchanged in response to differing local motion information. An additional area plausibly responsive to BM stimuli is premotor cortex. Saygin et al. (2004) used fMRI to determine that putative mirror neuron networks in premotor cortex respond to PLDs of human BM. In support, Ulloa and Pineda (2007) found significant suppression of electrophysiological mu rhythms (8–13 Hz) in response to BM PLDs, which they also associated with mirror neuron activity in premotor cortex.

In addition to cortical selectivity, cerebellar activity in response to BM stimuli has also been reported. Grezes et al. (1998) implicated right cerebellum in the visual processing of “meaningful” and “meaningless” actions and Grossman et al. (2000) found cerebellar activity in response to BM stimuli in the anterior portion, starting near the midline. Vaina et al. (2001) found selective activation for BM stimuli in the lateral cerebellum. More recently, Sokolov et al. (2010) reported that patients with left lateral cerebellar lesions, as opposed to medial lesions, show BM processing deficits. The cerebellum has also previously been associated with visual motion-percept processing (see Gao et al., 1996) as well as with action judgments (e.g. Parsons et al., 1995; but see Grezes et al., 2001).

Timing of processing in the biological motion network

In contrast to the abundant and ever-growing body of work regarding the localization of BM processes, there is relatively little consistent data regarding the precise timing of events across this network of implicated regions. Such information is valuable with regard to understanding feedback-feed-forward connections between STS and putative mirror neuron networks in premotor cortex and higher associated social cognition areas such as orbitofrontal cortex and the amygdala (see e.g. Gallagher and Frith, 2003; Stone et al., 2003).

Hirai et al. (2003) reported a significant right occipito-temporal amplification of their “N200” component in response to BM stimuli, as well as a bilaterally amplified “N240”. Similarly, Hirai et al. (2005) observed a later “N200” for BM stimuli when they were masked by additional, scattered, slowly moving dots. Jokisch et al. (2005) found amplifications of the negative event-related potential (ERP) components at 180 ms (N1) and 230–360 ms (N2) for biological motion PLDs relative to their scrambled counterparts. They also reported that the N1 and N2 effects were greatest before their respective components peak. Using inverse source localization methods (LORETA-analysis), they suggested that generators of the N1-effect were based in the posterior cingulate gyrus and in the left lingual gyrus and that the N2-effect arose from sources in the right fusiform gyrus (FFG), right superior temporal gyrus, as well as in the orbitofrontal cortex.

In a later study, Hirai and Hiraki (2006a) reported a significantly greater negativity in the 0–100 ms time-window for their scrambled condition vs. their normal BM PLDs, while the converse was true regarding the 200–300, 300–400, and 400–500 ms time-windows. Also, they reported no significant BM effects in the 100–200 ms N1 time-window. In a recent experiment involving both children and adults, Hirai et al. (2009) found main effects of larger and later bilateral N1 peaks, larger bilateral N2s, as well as larger amplitudes between the peaks, for BM relative to scrambled motion (SM), over occipito-temporal sites.

Using magnetoencephalography (MEG), Pavlova et al. (2004) concentrated on responses in the frequency domain, finding enhanced responses between 25 and 30 Hz, as early as 100 ms for both upright and inverted BM PLDs over left occipital cortex, with additional effects for upright PLDs over parietal and right temporal cortices at 130 and 170 ms, respectively. Scrambled displays did not affect “gamma” responses. In a more recent study, Pavlova et al. (2006) found these effects as early as 80 ms over left parieto-occipital cortex. They also reported right-hemisphere effects due to attention to BM stimuli at 120 ms over parietal cortex and at 155 ms over temporal cortex (see next section). Also using MEG, Virji-Babul et al. (2007) recorded significantly increased oscillatory responses between 15 and 35 Hz over the left posterior temporal area between 250 and 350 ms when subjects viewed PLDs of human motion, which was not found in response to PLDs of object motion.

Effects of attention

Early research supported spontaneous, early, bottom-up BM processing models (e.g. Johansson, 1973; Mather et al., 1992). More recent studies, however, have also implicated the role of top–down attentional-processes in the perception of biological motion. For example, Cavanagh et al. (2001) found that attentional load delayed detection of an oddball PLD. Similarly, Thornton et al. (2002) demonstrated the need for focused attention to detect biological motion under certain noisy conditions (see next section).

Using electrophysiology, Hirai et al. (2005) found significant amplification of their N330 component when subjects attended-BM stimuli rather than concurrently-presented geometric stimuli (see also Hirai and Hiraki, 2006b). Also, as mentioned earlier, Pavlova et al. (2006) found MEG effects in the gamma response as early as 80 ms for both attended and unattended tasks. However, only their attended biological motion stimuli produced results over right cortices parietally at 120 ms and temporally at 155 ms. In addition, both attended stimuli yielded effects fronto-temporally at 180–200 ms, a result they suggested implicates working memory.

More recently, an fMRI/EEG study (Safford et al. 2010) used a “double-exposure” paradigm in which either tool motion (TM) and BM, TM and SM, or BM and SM overlaid each other and subjects attended either TM or BM. Attention to TM suppressed the BOLD response of the right STS/MTG, while attention to BM suppressed the BOLD response of the left ITS/MTG. Additionally, category-based cortical current source density modulations began relatively late (after ~ 450 ms), probably in large part because of the subtle nature of the stimuli.

The emerging picture

While the results of much of the research are still not entirely consistent, a general model of BM processing does appear to emerge. Low level, feed-forward systems play a prominent role, particularly when the stimuli are presented in non-noisy conditions and at short interstimulus intervals (ISI) (see Mather et al., 1992). Yet BM can still be perceived when presented at display rates faster than those usually associated with low-level, local motion processes (Thornton, 1998). Thornton et al. (2002) reported that while attention was necessary to perceive BM in “dynamic noise” at long ISIs, short ISIs still yield a BM percept in the absence of attention (see also Thornton and Vuong, 2004). As such, both top–down and bottom–up processes likely play a role in BM perception, with attention playing a greater role in integrating BM information that cannot simply be processed automatically.

Similarly, and as evidenced by the aforementioned neuroimaging studies, both motion and form processes appear to interact dynamically in BM detection and perception (see e.g. Beintema and Lappe, 2002; Pinto and Shiffrar, 1999). Basing themselves on the physiological and neuroimaging data, Giese and Poggio (2003) formulated a feed-forward model of parallel ventral-form and dorsal-motion processes that analyze “snapshots” of human forms and “optic-flow” (OF) patterns, in an increasingly global manner, as they converge toward STS and associated areas (see also Peuskens et al., 2005). According to this feed-forward model, BM processing involves the two visual pathways. Motion information traverses dorsally from local motion detectors in V1 to V2 and hMT. It then ascends to local OF pattern-detectors in hMT, MST, and/or KO. The information is then further processed by complex OF-pattern detecting neurons in STS and/or FA, as well as by motion pattern neurons in STS, FA, and/or ventro-lateral premotor cortex. Form information is conveyed ventrally from simple (and complex) cells in V1 and V2 to complex cells in V4. View-tuned snapshot neurons in inferotemporal cortex, EBA, STS, and/or FA further process the information before relaying it to the motion pattern neurons of STS, FA, and/or F5, where it can be integrated with dorsally-processed information.

The present study

In the current study, we implement two basic BM tasks to more fully corroborate and clarify the electrophysiological spatiotemporal processing of biological motion stimuli. We use higher density electrode arrays (168 channels) to aid in localization analyses. In contrast to previous electrophysiological studies which focused on just two components, our analyses explore effects both at component peaks as well as between them. A clearer picture of the actual timing of BM processes will enable a clearer understanding of how the different brain areas involved in BM processing interact. As of yet there appears to be no EEG literature addressing precisely when BM processing begins. Such information is valuable in more accurately evaluating potential feed-forward–feedback flow in BM processes and social cognition. Along these lines, we also explore the differences in attended vs. unattended tasks as manifested in our electroencephalographic data. In doing so, our intention is to establish a baseline for comparison with clinical populations.

Materials and methods

Subjects

Fourteen (4 female) volunteers (mean age= 28.6 years.; SD= 5 years), with no reported neurological impairments, participated in this study. All subjects provided written informed consent after the procedures of the experiment were fully explained to them, and all procedures were approved by the Institutional Review Boards of the Nathan Kline Institute and the City University of New York. All subjects received a modest fee for their participation.

Stimuli and tasks

Displays were presented on either an 18″ Ilama Pro VisionMaster 502 (nine subjects) or a 30 × 40 cm MultiSync FE2111SB (five subjects) monitor controlled by Presentation™ software. All experiments were conducted in a sound-attenuated electrically shielded room illuminated only by light from the video screen. In both tasks, all stimuli appeared black against a white background. Subjects were instructed to maintain fixation on a central fixation-cross and eye-position was monitored by vertical and horizontal electroocculogram.

Video-clips of an adult human engaged in common activities (e.g. running, kicking, climbing, throwing, and jumping) were imported to a computer to create the biological motion stimuli. Markers were placed on the actor’s joints in each frame of the sequence, such that the final clips were only composed of up to seven moving dots (i.e. point-light displays). Scrambled motion (SM) sequences were created from the normal biological animations and consisted of the same individual dots undergoing the same local motions as the biological counterparts. Scrambling was produced by randomizing the temporal phases and spatial locations of the dots in a given animation, thereby skewing the hierarchical, pendular motions that are characteristic of biological motion (see Fig. 1). The methodology behind the generation of biological motion sequences is discussed more fully in Grossman and Blake, (1999); Blake et al., (2003).

Fig. 1.

Sample stimuli: on the left are still-frames depicting normal biological activity in point-light animation sequences. On the right are the scrambled counterparts of the biological motion sequences.

The experiment consisted of two tasks in each of which subjects were presented with six or seven five-minute blocks of randomized, repeating video-clips, with 110 clip-presentations per block (55 BM+55 SM; total time=~35 min per task). In total, twenty distinct video-clips were used, ten of which represented point-light displays of canonical biological motion (see below), and ten of which were scrambled images thereof. These twenty clips were selected from a larger pool of 100 clips to match for retinal displacement (see Supplementary Materials). Each clip was composed of 29 frames presented at the monitor refresh-rate of 60 Hz. Inter-stimulus interval was randomized between 500 and 1000 ms.

In the first task, in random clips (nine percent of total clips), one of the dots would briefly turn red. Only a single dot changed color on these target trials, the position of which within the moving object was randomized, and this only occurred after the movement clips had already begun, never beginning before frame 4 (i.e. 54 ms after onset). The duration of the color-change was then very brief, lasting just 4 or 5 frames (67–83 ms). The onset of the color-change was also randomized such that it could appear at any time from frame 4 to frame 23 (383 ms). As such, participants needed to attend across the entire stimulus presentation period to rule out target presence. Subjects were instructed to respond to these “target” clips by depressing a mouse key. Subjects were not explicitly informed that some of the clips portrayed human motion, although this was immediately obvious to subjects upon debriefing. Subjects were also instructed to delay their responses until the completion of each video-clip in order to diminish the impact of motor response-related artifacts. Target-trials were excluded from later analysis enabling a contrast between non-target BM and non-target SM without additional motor response artifacts or target-related processing effects.

In the second task, subjects were once again presented with the same video-clips (minus the red-dot target clips of the first task). This time, however, subjects were asked to judge whether the clips depicted human motion or scrambled motion. Following each trial, the subject indicated whether or not the animated dots portrayed “human” activity by pressing one of two pre-assigned computer keys. A forced-choice paradigm was used to control for target-effects. As such, differences in the response would reflect the difference between attended target BM and attended target SM, and not motor planning or inhibition. The second task always followed completion of the first task to maintain presumed naïveté in the first task regarding the presence of BM in the displays.

Over the course of both tasks, subjects were encouraged to take breaks between blocks whenever they deemed it necessary, in order to maintain high concentration and reduce fatigue.

Measurements and analyses

Continuous EEG was acquired through the ActiveTwo BioSemi™ electrode system from 168 scalp electrodes, digitized at 512 Hz. For display purposes, data were filtered with a low-pass 0-phase shift 96 dB 40 Hz filter after acquisition. With the BioSemi™ system, every electrode or combination of electrodes can be assigned as the reference, which can be done offline. BioSemi™ replaces the ground electrodes used in conventional systems with two separate electrodes: Common Mode Sense (CMS) active electrode and Driven Right Leg (DRL) passive electrode. These two electrodes form a feedback loop, rendering them references. For a detailed description of the referencing and grounding conventions used by the BioSemi™ active electrode system, visit www.biosemi.com/faq/cms&drl.htm.

After acquisition, data were re-referenced to a medial-frontal site (FPz) for analysis. After each recording session, and before the electrode cap was removed from the subject’s head, the 3D coordinates of all 168 electrodes with reference to anatomic landmarks on the head (nasion and preauricular notches) were digitized with a Polhemus Magnetic 3D digitizer. Data were analyzed and artifacts were rejected offline using BESA™ multimodal neuroimaging analysis software package (MEGIS Software GmbH, Munich, Germany). Because of the relatively long duration of each video-stimulus, artifacts were only rejected before 600 ms. Accepted trials were epoched (100 ms prestimulus to 1300 ms post-stimulus) and then averaged separately for each condition. To control for low-level stimulus properties, only non-target trials were included in the averages for the color-detection task. We defined baseline as the mean voltage over −50 ms to 0 ms preceding the onset of the stimulus. Trials with blinks and large eye movements were rejected offline on the basis of horizontal and vertical electro-oculogram recordings. An artifact rejection criterion of 80–100 μV was used at all other electrode sites to exclude periods of high EMG and other noise transients. From the remaining artifact-free trials, we computed averages for each subject.

Analysis strategy

Because there is little consistent literature regarding the precise timing of the electrophysiological response to BM stimuli, we took a two-stage approach to our statistical analyses. The first stage was a simple three-way ANOVA (factors: Task: attended/unattended; Hemisphere: left/right; and Motion: BM/SM) based on the findings of past studies. The second stage comprised a more comprehensive analysis of all time-points and sites to more fully explore the scalp effects in response to the two tasks. What follows is a brief description of these two analyses. See also Wylie et al. (2003), who employed a similar methodology.

Stage one analysis: regions-of-interest and ERP components

For our initial analysis, and basing ourselves on findings in the previous literature (e.g. Hirai et al., 2003; Jokisch et al., 2005), we defined bilateral regions-of-interest comprising three adjacent electrode sites on or near the temporo-parieto-occipital junctions bilaterally, roughly corresponding to underlying higher order visual processing areas such as STS. We then generated waveforms averaged from each set of three electrodes. Componentry was defined based on waveforms collapsed across both canonical BM and scrambled conditions (see Fig. 2), i.e. unbiased by observation of any possible effects.

Fig. 2.

Event-related componentry defined for the initial region-of-interest analysis.

Waveforms were largely similar to those reported in the aforementioned literature (e.g. Jokisch et al., 2005; Hirai et al., 2003) with higher-frequency, large-amplitude P110 and N180, as well as lower-frequency, lower amplitude P280 and N360. The area under each curve (AUC) was computed for seven consecutive time-windows. For the sharper P1 and N1 components, 20 ms time-windows centered at the peaks were computed. In addition, and based on the aforementioned findings in the EEG and MEG literature (e.g. Jokisch et al., 2005; Hirai and Hiraki, 2006a; Pavlova et al. 2004, 2006), we also looked at the 20 ms time-window before the P1 peak (“eP1”), between the P1 and N1 peaks (“P1–N1”), and in the post-peak, late N1 (“N1–P2”). For the later, low-frequency components, we computed the consecutive 80 ms time-windows that spanned the components (“P2” and “N2”) (see Fig. 2). We conducted a three-way repeated-measures ANOVA with factors of task-type (attend BM vs. unattended), hemisphere (right vs. left), and motion type (BM vs. SM). One subject was excluded from this analysis due to the fact that the second task had been executed with a GoNoGo paradigm, rather than as the forced-choice paradigm used by the remaining subjects. Our critical value was set at α=0.05.

Stage 2 analysis. Exploratory statistical cluster plots and source modeling

In order to incorporate more fully the wealth of information provided by our high-density electrophysiological dataset, we also computed statistical cluster plots for each task (see Molholm et al., 2002). These maps were created using pointwise, paired, two-tailed t-tests between the VEP responses to our two conditions (BM and SM). As such, we can assess more fully an approximation of the entire differential activation between the conditions across the 500 ms post-stimulus-onset epoch. Since the potential for a Type I error is high with such an approach due to the high number of statistical comparisons, we restrict our analysis to an alpha criterion of 0.01 and, additionally, only accept as significant those data that reach this threshold for 11 consecutive time-points (N 20 ms at our 500 Hz sampling rate; See e.g. Guthrie and Buchwald, 1991; Foxe and Simpson, 2002, for similar approaches).

Using these statistical cluster plots as a framework, dipole source analyses were then implemented using the BESA software suite (version 5.0.4) to estimate the intracranial generators underlying the spatio-temporally discrete effects. BESA models the best-fit location and orientation of multiple intracranial dipole generator configurations to produce the waveforms observed at the scalp, using iterative adjustments to reduce the residual variance between the solution and the observed data (see e.g. Scherg and Von Cramon, 1985). For the purpose of the modeling, an idealized three-shell spherical head model with a radius of 85 mm and scalp and skull thickness of 6 mm and 7 mm was assumed. The upper bound of the number of modeled dipole sources was determined using an unconstrained test dipole (see Scherg and Picton, 1991). When the number of modeled sources, m, is sufficient, addition of another source (test dipole) and solving for m + 1 sources would not be expected to further reduce the residual variance, above that attributable to noise. Similarly, when scalp effects appeared to be bilaterally symmetric, dipoles were constrained for symmetry, provided unconstrained sources were unable to reduce residual variance beyond that attributable to noise. In order to maintain a high signal-to-noise ratio, as well as to generalize our results across subjects, group-averaged VEP data were used. It is worth mentioning that as the modeled equivalent current-dipole represent simplifications of activity in the area, they should be considered as indicators of centers-of-gravity and not necessarily distinct neural sites.

Results

Behavioral

Hit rates for the first, color-detection task were 91.7% (S.D.= 0.08), with false alarm rates at 11% (S.D.= 0.01). Similarly, in the second, motion-categorization task, accuracy was 92.4% (S.D.= 0.08).

Regions-of-interest analyses

To more clearly demonstrate the overall distribution of the electrophysiological response, VEPs from key scalp sites are shown in Fig. 3. Our initial analysis focused on the areas corresponding to PO7 and PO8 in the diagram. As can be seen from the figure, BM generated greater positivity than SM over the right parieto-occipital site from the peak of P1 (~ 110 ms) and continuing toward the N1 peak (~ 180 ms), at which point BM generated greater negativity than SM for upwards of 150 ms. This later effect appeared to have a greater amplitude when BM is explicitly attended, as well as a later offset.

Fig. 3.

VEPs for the unattended (a) and attended (b) biological motion (BM) tasks. Blue lines indicate the response to BM stimuli, red lines indicate the response to scrambled stimuli, and green lines represent the difference waves.

In what follows, we will step systematically through our predefined componentry, analyzing each in their turn (eP1, P1, P1–N1, N1, P2, and N2).

Our analysis of the earliest time-window (‘eP1’=80–100 ms) yielded no significant main effects nor significant interactions.

P1 (100–120 ms) had a significant interaction between hemisphere and motion (F_(1,12) = 5.38; p= 0.04) with increased amplitudes in right scalp sites in response to BM vs. SM. (See Supplementary Materials with regard to potential confounds regarding this early effect.)

The time-window between P1 and N1 (‘P1–N1’= 120–170 ms) had a significant main effect for motion type (F_(1,12) = 10.00; p= 0.01) as well as a significant interaction between motion and hemisphere (F_(1,12) = 10.10; p= 0.01). Post-hoc analyses indicated that this was due to a greater negativity over the right hemisphere in response to BM, as well as a reduced negativity in the left.

N1 (170–190 ms) yielded a significant interaction between task-type and hemisphere (F_(1,12)=5.52; p=0.04) with bilateral amplifications of the negativity in response to “attended” BM. N1 approached significance for the main effect of hemisphere (F_(1,12)=3.93; p=0.07), as well as for the interaction between hemisphere and motion (F_(1,12)=4.07; p=0.07).

The time-window between N1 and P2 (‘N1–P2’= 190–240 ms) had a significant main effect of task (F_(1,12) = 6.12; p= 0.03), a significant main effect of motion (F_(1,12) = 20.60; p = 0.001), as well as significant two-way interactions between task and hemisphere (F_(1,12) = 8.44; p = 0.01), and task and motion (F_(1,12) = 10.76; p= 0.01). Post-hoc tests revealed a bilateral amplified negativity for the attended task, an amplified negativity in response to BM, as well as a greater task-related effect in the left hemisphere. The difference between BM and SM was greater for the attended task.

Similarly, the P2 (240–320 ms) showed a significant effect for task (F_(1,12) = 5.47; p= 0.04) and motion (F_(1,12) = 17.24; p= 0.001), as well as for the interaction between the two (F_(1,12) = 6.66; p= 0.02). Post-hoc tests revealed a bilateral amplified negativity for the attended task as well as an amplified negativity in response to BM.

The N2 (320–400 ms) had a significant main effect of motion type (F_(1,12) = 6.90; p= 0.02), as well as a significant interaction between motion and hemisphere (F_(1,12) = 7.32; p= 0.02). Post-hoc analyses demonstrated that the response to BM after N1 was more negative particularly in the attended task as well as in the left hemisphere (see Table 1).

Table 1.

Summary of results of 3-way ANOVA with independent variables of task-type (attended vs. unattended), hemisphere, and motion-type (BM vs. SM). (* indicates significant results at α= 0.05.)

Component	ms	Task	Hem	Motion	Task X Hem	Task X Motion	Hem X Motion	Task X Hem X Motion
eP1	80–100	0.94	0.86	0.78	0.57	0.42	0.76	0.46
P1	100–120	0.99	0.17	0.15	0.86	0.52	0.04*	0.34
P1–N1	120–170	0.65	0.91	0.01*	0.67	0.5	0.01*	0.79
N1	170–190	0.14	0.07	0.55	0.04*	0.28	0.07	0.60
N1–P2	190–240	0.03*	0.36	0.001*	0.01*	0.01*	0.77	0.88
P2	240–320	0.04*	0.19	0.001*	0.22	0.02*	0.23	0.25
N2	320–400	0.64	0.31	0.02*	0.44	0.39	0.02*	0.18

Exploratory statistical cluster plots and source modeling

As described earlier, we also conducted statistical cluster plots for each task to measure for the effects between the two stimulus-classes (BM vs. SM) (see Fig. 5). Our plots over all electrodes and for all time-points showed the most significant effects for the unattended task in the 120–160 ms time-window over parieto-occipital (PO), parietal, and central areas, and at around 200 ms in occipital and PO areas (Fig. 5A). In the attended task, our probability maps showed effects which were fairly similar to those in our unattended task for the 100–200 ms time-window (see Fig. 5B). In addition, we saw prolonged effects from 200 ms to 350 ms over parieto-occipital areas and from 400 ms onward over parietal and central areas, presumably related to attentional amplification.

Fig. 5.

Color-plot of t-values for the differences between canonical biological motion point-light displays and their scrambled counterparts in the unattended (a) and attended (b) tasks.

We then estimated the intracranial generators of the scalp electrophysiology seen in the cluster plots with dipole source analyses using the BESA software suite (see earlier in Methods). The early effect in both tasks appeared as a greater positivity over the right occipito-parietal area from 120 to 160 ms. A single dipole over that time period localized at Talairach coordinates: x= 35, y= −69, and z= −2, accounted for 91% of the scalp electrophysiological variance for the unattended task (Fig. 6A). For the attended task over that time period, a dipole localized at Talairach coordinates: x= 23, y= −80 and z= 20, accounted for 81% of the variance (Fig. 6C).

Fig. 6.

a. Scalp map of the difference between BM and SM responses at ~140 ms in the unattended task and the corresponding source localization for the 120 ­ 160 ms time-window (Talairach: x= 35, y= −69, z = −2; explained variance [EV] = 91%). b. Scalp map of the difference between BM and SM responses at ~275 ms in the unattended task and the corresponding symmetric sources localized for the 200 ­ 350 ms time-window (Talairach: x=±40, y= −69, z = 13; EV= 80%). c. Scalp map at ~140 ms of difference-waves between scrambled and canonical biological motion for the attended task and the corresponding source localized for the 120 ­ 160 ms time-window (Talairach: x= 23, y= −80, z = 20; EV= 81%). d. Scalp map at ~275 ms of the difference between scrambled and canonical biological motion for the attended task and the corresponding symmetric sources localized for the 200 ­ 350 ms time-window (Talairach: x=±40, y= −65, z = 7; EV= 89%). e. Scalp map at ~450 ms of the difference between scrambled and canonical biological motion for the attended task and the corresponding sources localized for the 400 ­ 500 ms time-window (Talairach: x= −37, y= −76, z= 16; x= 32, y = −77, z = 10; EV = 93%).

The second major effect occurred over what appeared to be approximately symmetric sites over the bilateral occipito-temporal cortices from ~ 200 to 350 ms. Two symmetrically-constrained dipoles at Talairach coordinates: x=± 40, y= −69, and z= 13, accounted for 80% of the scalp variance, for the unattended task (Fig. 6B). Similarly, we obtained Talairach coordinates of: x=± 40, y= −65 and z= 7, which accounted for 89% of the variance, for the attended task (Fig. 6D).

The third, attention-related effect (400–500 ms) yielded sources at: x = − 37, y = − 76, z = 16; x = 32, y = − 77, z = 10, which accounted for 93% of the scalp variance in that time-window (Fig. 6E).

Discussion

In the two experiments reported here, we sought to provide a comprehensive picture of the spatiotemporal dynamics of biological motion processing and their modulation by attention. The use of high-density electrode arrays allowed for a detailed characterization of evoked responses over time and a more precise estimation of their cortical sources. It was found that biological motion affected neural processing as early as ~ 100 ms after the onset of the first frame of stimulation (see Supplementary Materials), with robust modulation of the ongoing response thereafter that continued past 320 ms, irrespective of whether participants specifically attended to the motion aspect of the stimuli or not. We identified three distinct phases of modulation and we will treat of each of them in their turn in what follows.

Phase I effects (100–200 ms)

The earliest phase of this BM-sensitive modulation was characterized by a positive shift of the ERP in the BM condition in the time-window between 100 and 200 ms after stimulus onset. The timing and topographical distribution of this effect is in relatively close correspondence to early occipital P1 modulations reported by Hirai et al. (2009) and perhaps with the early onset of differences in gamma band oscillations found by Pavlova et al. (2004, 2006; see above). This would seem to suggest that the brain detects BM very early, since the timing of the onset of this effect is such that no more than the first three frames of the animation (~ 30 ms each) could realistically have been registered in cortex before the modulation emerged (see Foxe and Simpson, 2002). Source localization of this activity suggested a likely cortical generator in the dorsal visual processing stream, in close proximity to regions associated with general motion processing (KO/hMT) (see Fig. 7). Significantly, this modulation was observed exclusively over the right hemisphere suggesting that right hemispheric neural networks underlying general motion processing may be specialized for early detection of BM. This effect is also consistent with the well-established role of right hemisphere temporo-parietal regions in so-called global processing (e.g. Robertson et al., 1988; Fink et al., 1997a; see, however, Fink et al., 1997b), since a major aspect of processing the point-light-display stimuli used here lies in constructing a global percept from the coordinated movements of an array of local disconnected elements.

Fig. 7. SuThThary of findings in soThe recent neuroiThaging studies as related to our source localizations.

See [KO] Orban et al, 1995; Dupont et al, 1997; Van Oostende et al, 1997; Tyler et al, 2005; [hMT] Tootell et al, 1995; Watson et al, 1993; Van Oostende et al, 1997; Culham et al, 1998; Sack et al, 2006; Kourtzi et al, 2002; Becker et al, 2008; [pSTS] Kontaris et al, 2009; Peelen et al., 2006; Ahlfors et al,1999; Beauchamp et al, 2004; Calvert et al, 2000; Materna et al, 2008.

In order to elucidate the effects of attention on BM processing, we conducted Experiment 1 with naive subjects who were instructed to respond to a simple non-BM-related cue (briefly appearing dot-color changes), while in Experiment 2 participants were explicitly asked to make judgments about the presence or absence of BM in the stimulus. The early BM effect (100–200 ms) does not appear to be task-dependent in that it was observed with similar scalp topographies in both paradigms. Thus, these data point to a relatively involuntary process unrelated to explicit attention to the BM aspect of the stimuli. Of course, this effect could also reflect exogenous engagement of attentional-processes and since the color-detection task used in Experiment 1 could not be classified as an especially demanding task, it is entirely possible that subjects were able to devote some attentional resources to processing this aspect of the stimuli. However, the very early timing of this effect would argue against such an interpretation, and the large-scale differences in later cognitive components as a result of task make it clear that subjects did engage very differently in both tasks. Nonetheless, there is some experimental evidence for relatively automatic activation of BM processes. For example, Thornton and Vuong (2004) found that when task-irrelevant BM figures flanked a target BM figure, response times regarding the perceived direction of the centrally-presented figure were significantly prolonged, particularly when the flanking distractors’ motion direction was incongruous with the target. That is, participants were clearly unable to ignore the flanker BM stimuli. However, unlike the design used in Experiment 1 here, participants in the Thornton and Vuong study were explicitly attending for BM stimulus direction which complicates interpretation somewhat. Nonetheless, the task-independent early effects demonstrated here may represent the underlying neural processes behind such behavioral delays, as the brain involuntarily detects and processes irrelevant and potentially distracting BM signals.

Phase II effects (200–350 ms)

The second major phase where BM processing effects were evident occurred between 200 and 350 ms and was characterized by a robust negative-going modulation of the ERP over the posterior middle temporal regions of both hemispheres. This negative deflection was evident in both Experiments 1 and 2, but it was also clearly amplified in Experiment 2 when the BM aspect of the stimuli was explicitly attended for. Differences between conditions in this middle phase of neuronal activity were in rough correspondence to the second component reported by Hirai et al. (2003) and what Jokisch et al. (2005) termed “N300”. Source analysis of this activity resulted in an excellent fit by a pair of equivalent current dipoles located bilaterally at or near the posterior STS. Given that these dipoles fall precisely between known activation locations within hMT and the pSTS, we think it very likely that they represent compound activity across an extended region comprising both of these regions (see Fig. 7). Given the limited spatial resolution of scalp recordings, it was not possible to tease apart specific contributions from both regions using the point-dipole approach and one must be careful not to over-interpret the precision of such localizations. Rather, dipole locations are best thought of as centers-of-gravity for net local current flow rather than discrete generator locations. These locations are highly consistent with pSTS effects previously reported in the neuroimaging literature (see Introduction section).

The pSTS has been implicated in many biological motion studies involving articulated human motion (Vaina et al., 2001; Beauchamp et al., 2002; Grossman and Blake, 2002). However, neuroimaging studies have also shown that pSTS is engaged by considerably less complex motion stimuli, such as when the motion of simple two-dimensional objects depict social interactions (Heider and Simmel, 1944; Castelli et al., 2000; Schultz et al., 2003; Ross and Olson, 2010). This is interesting since the two-dimensional motion of these geometric shapes is very different in terms of kinetic and perceptual properties to the point-light displays that were used here and in other biological motion experiments. Here, the points mark locations on human body parts that contort (e.g. arms and knees bending) when the body is in natural motion and that move in reference to one another evoking a vivid three dimensional impression. The fact that pSTS-regions are also activated in so-called “theory-of-mind” tasks, some of which employ static images or lexical tasks, suggests that pSTS-regions may be involved in processing of information that is more broadly related to social interactions (see Carrington and Bailey, 2009). Similarly, regions along the STS into the temporo-parietal junction are engaged in a variety of language related tasks (see Binder et al., 2009). Given the multitude of tasks for which pSTS involvement is indicated, Hein and Knight (2008) suggested that this brain region may support different functions depending on task-dependent network connections. In this view, pSTS activity is determined by coactivations of cell populations in other parts of a distributed neural network, and in the current work, it is likely the interaction with nearby hMT that determines the pSTS role in processing BM.

Given the above evidence it seems plausible that networks within the STS are part of the semantic system supporting knowledge about the meaning of motion patterns and sequences. These structures can be engaged not only by dynamic BM stimuli, but virtually by any task involving or evoking meaningful motion such as static images or lexical or verbal descriptions of moving entities or agents. This could explain why this area is implicated in many tasks that are so different in nature, but also the consistency with which it is activated in experiments involving point-light displays that are often very similar in the types of activities they display.

BM and non-BM stimuli differ in basic and complex aspects of motion that can impact on early and late stages of the information processing stream. On a basic level, the dynamics of the point-lights in BM motion display a more patterned motion coherence and motion opponency (see Jastorff and Orban, 2009; Casile and Giese, 2005) which leads to the emergence of a Gestalt on higher perceptual levels and will eventually engage neural networks involved in identification and conceptual knowledge encoded in higher order semantic networks. According to this notion, early lateral-occipital and occipitoparietal effects may be associated with differences in basic aspects of motion such as the spatiotemporal coherence of the motion of the point lights. Further, it is possible that there are automatic attentional mechanisms at play that are related to the binding of the point lights into a form similar to processes that precede the closure of fragmented objects in static displays (e.g. Snodgrass and Corwin, 1988). Indeed, the observed bilateral negativities observed during this second phase of BM processing bear strong resemblance to bilateral lateral-occipital negativities previously described over both hemispheres during so-called perceptual closure tasks (e.g. using fragmented line drawings of common objects) that have been associated with the emergence of “objectness”, wherein associated fragments of a visual image are bound into a coherent and meaningful form (see e.g. Doniger et al., 2000; Sehatpour et al., 2006, 2008).

Phase III effects (400+ ms)

The primary focus of this study was on sensory-perceptual stages of BM processing, but we also observed a robust later phase of processing that was BM-sensitive from approximately 400 ms onwards. This third phase of BM processing was only observed during Experiment 2 when the BM aspect of the stimuli was explicitly attended (see Figs. 4 and 5). This attention-driven effect was seen as a greater positivity in response to BM stimulation over midline central-parietal scalp. We speculate that this later sustained difference is associated with cognitive processes involved in decoding the meaning of the activity displayed by the motion stimulus. These higher order representations coding semantic features and associations, as well as their integration into abstract conceptual knowledge, are hypothesized to be widely distributed over the cortex according to an ‘embodied cognition’ view (Patterson et al., 2007). It seems likely that they involve parts of the premotor cortex which have been implicated in biological motion processing (Deen and McCarthy, 2010) and are considered to be part of a wider mirror neuron system (see Van Overwalle and Baetens, 2009). Such a widely-distributed network of activation is not easily modeled using the dipole source-modeling technique. Here, we found that a pair of bilateral parietal sources provided a good fit for this late effect but this solution likely represents a significant over-simplification.

Fig. 4.

Posterior topographic scalp maps of the response during both experimental conditions and the difference maps between them at selected time-points.

Conclusion

The detection and integration of biological motion (BM) information is a fundamental process of social cognition and involves a specialized cortical network. The present study used high-density electrical mapping and source-analysis techniques to provide a timeframe of information processing across this network. Scalp electrophysiology was recorded in response to canonical BM vs. scrambled motion (SM) stimuli in both a “BM-unattended” task and a forced-choice, attended-BM task. Our analyses resolved early effects beginning at ~ 100 ms with continuous significance achieved through 400 ms after stimulus onset, except for at the brief N1-peak time-window. The first phase of differential activation (110–170 ms) elicited a probable source in the dorsal stream superior to the KO/hMT complex. The second phase (200–350 ms) suggested bilateral sources between hMT and pSTS. An additional late (320 ms onward), occipital “positivity” occurred only when the distinction between BM and SM was explicitly attended. These results hopefully provide a framework for comparing the subtler information implicit in BM processing, such as familiar, complex motion processing, theory-of-mind processes, intentionality and perceived attention.

List of abbreviations.

AUC

area under the curve

BM

biological motion

CRT

cathode ray tube

EBA

extrastriate body area

EEG

electroencephalography

ERP

event-related potential

FFG

fusiform gyrus

fMRI

functional magnetic resonance imaging

ISI

interstimulus interval

KO

kinetic occipital area

MEG

magnetoencephalography

hMT

homolog of monkey middle temporal gyrus

OF

optic-flow

OFA

occipital face area

PLDs

point-light displays

PO

parieto-occipital areas

pSTS

posterior superior temporal sulcus

SM

scrambled motion

TMS

transcranial magnetic stimulation

ToM

theory of mind

VEP

visual evoked potential