Interactions between visual and motor areas during the recognition of plausible actions as revealed by magnetoencephalography

Abstract

Several studies have shown activation of the mirror neuron system (MNS), comprising the temporal, posterior parietal, and sensorimotor areas when observing plausible actions, but far less is known on how these cortical areas interact during the recognition of a plausible action. Here, we recorded neural activity with magnetoencephalography while subjects viewed point‐light displays of biologically plausible and scrambled versions of actions. We were interested in modulations of oscillatory activity and, specifically, in coupling of oscillatory activity between visual and motor areas. Both plausible and scrambled actions elicited modulations of θ (5–7 Hz), α (7–13 Hz), β (13–35 Hz), and γ (55–100 Hz) power within visual and motor areas. When comparing between the two actions, we observed sequential and spatially distinct increases of γ (∼65 Hz), β (∼25 Hz), and α (∼11 Hz) power between 0.5 and 1.3 s in parieto‐occipital, sensorimotor, and left temporal areas. In addition, significant clusters of γ (∼65 Hz) and α/β (∼15 Hz) power decrease were observed in right temporal and parieto‐occipital areas between 1.3 and 2.0 s. We found β‐power in sensorimotor areas to be positively correlated on a trial‐by‐trial basis with parieto‐occipital γ and left temporal α‐power for the plausible but not for the scrambled condition. These results provide new insights in the neuronal oscillatory activity of the areas involved in the recognition of plausible action movements and their interaction. The power correlations between specific areas underscore the importance of interactions between visual and motor areas of the MNS during the recognition of a plausible action. Hum Brain Mapp 35:581–592, 2014. © 2012 Wiley‐Periodicals, Inc.

INTRODUCTION

Action recognition plays an important role for effective communication and interaction with other people [Blakemore and Frith, 2005; Kokal et al., 2009; Schippers and Keysers, 2011]. Action recognition occurs at different levels and over distinctive time scales. On a lower level and a shorter time period, sensory information will be processed [Blake and Shiffrar, 2007; Grossman et al., 2000; Michels et al., 2009; Pavlova and Sokolov, 2003]. This incorporates the ability to integrate form and motion but it can also rely on the ability to distinguish form from motion [Lange et al., 2006; Michels et al., 2005; Oram and Perrett, 1994]. Several recent studies have argued that action recognition also relies on higher, nonsensory areas of the mirror neuron system (MNS) [Schippers and Keysers, 2011; Urgesi et al., 2010]. Mirror neurons were first discovered in area F5 of the macaque monkey premotor cortex (PMC) [Di Pellegrino et al., 1992]. They are a particular class of neurons that fire when a monkey performs a goal‐oriented action but also when it passively observes that same action [Gallese et al., 1996; Rizzolatti et al., 1996]. Areas frequently considered as being part of the MNS in humans are the PMC, supplementary motor area, somatosensory areas, the inferior parietal lobe, inferior frontal gyrus, and indirectly the superior temporal sulcus (STS), a visual area known to respond to biological actions without being a standard part of the MNS [Bonda et al., 1996; Buccino, 2004; Dinstein et al., 2007; Filimon et al., 2007; Gazzola et al., 2007; Pelphrey et al., 2005; Rizzolatti and Craighero, 2004; Schippers and Keysers, 2011].

The proposed mechanism of how mirror neurons mediate recognition of actions is to compare visual information of an action to one's own motor repertoire [Rizzolatti and Craighero, 2004]. In other words, when one observes an action performed by another person, neurons that represent that action in the observer's repertoire of possible actions are triggered in the PMC [Buccino et al., 2004a; Rizzolatti et al., 2001]. Actions belonging to the movement repertoire of the observer are mapped in their PMC. Actions that do not belong to this repertoire are recognized predominantly on a visual basis. In line with this model, studies have shown that the observers' ability to perform an observed action modulates activation in mirror neuron areas (e.g., Calvo‐Merino et al., 2005; Orgs et al., 2008).

An effective and frequently used method for studying action recognition is the point‐light display (PLD) method [Johansson, 1973]. Although PLD represents a human body and its action with only a handful of dots, observers can easily recognize the actions of these PLD (e.g., Grossman et al., 2000; Johansson, 1973). As PLDs are easy to present and manipulate, they are a useful tool in neuroimaging to study the cortical areas involved in action recognition. By changing the spatial configuration of the dots, while keeping the motion trajectories intact, the configural and holistic impression of the action can be destroyed while keeping low‐level information such as motion signals, stimulus size, and number of point‐light dots constant. Such “scrambled” PLDs are often used as control stimuli to unravel action recognition from basic low‐level visual perception [Grossman et al., 2000; Michels et al., 2005; Pavlova et al., 2004]. Neuroimaging studies in human and nonhuman primates have identified the visual areas to be primarily involved in the process of PLD actions compared to scrambled PLD [Grossman et al., 2000; Michels et al., 2005; Oram and Perrett, 1994; Pavlova et al., 2004]. More recently, studies have also identified the PMC to be involved in the recognition of PLD actions compared to scrambled PLD [Candidi et al., 2008; Kemenade et al., 2012; Saygin et al., 2004]. These findings have led to the interpretation that visual as well as motor areas contribute to the recognition of actions. Most of these studies have been performed using functional magnetic resonance imaging (fMRI). Little is known, however, about the role of neuromagnetic oscillatory activity and how these cortical areas dynamically interact during the process of action recognition.

To investigate the dynamic modulations and interactions between visual and motor areas during the process of action recognition, we used the PLD method similar to the above‐mentioned fMRI studies and magnetoencephalography (MEG). We created different PLD action representations and scrambled versions of these PLD actions. MEG's high temporal and good spatial resolution enabled us to examine the dynamics in the frequency domain within‐ and between‐sensory and motor areas during the process of action recognition.

METHODS

Subjects

Twelve right‐handed subjects with normal or corrected to normal vision (six males, mean age ± SD = 27.6 ± 2.87) and with no known neurological disorders participated in this study. All subjects gave informed consent in accordance to the declaration of Helsinki and the local Ethics Committee.

Stimuli

Point‐light biological motion animations were generated by recording the movements of human actors with sensors attached to their main joints (head, shoulders, elbows, wrists, hips, knees, and feet) using a motion tracking system (MotionStar; Ascension Technology, Burlington, VT; [Lange and Lappe, 2007]). The main joints were represented by 14 small white dots (5 × 5 pixels) against a black background.

Stimuli were offline manipulated using MATLAB (MathWorks, Natick, MA). First, actions were cut into segments representing one cycle of the action, lasting between 0.6 and 1.0 s. Next, cycles of each action were repeated five times. To compute a seemingly continuous movement of each action, transitions between cycles were smoothed [Lange et al., 2006]. We manipulated the different stimuli to create three different stimulus conditions with different degrees of action representation, whereas leaving low‐level visual information as constant as possible (Fig. 1).

Figure 1.

Experimental setup. Examples of stimuli used (I) Plausible, (II) Scrambled, (III) Implausible. Connecting lines were not present in the actual experiment. For details, see Experimental Procedures section. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Originally 20 animations depicting a human action were recorded. In a pretest, we presented plausible, implausible, and scrambled versions of the animations and asked subjects to rate the stimuli as plausible, implausible, or scrambled. Eight animations, which were clearly distinguished based on the three‐scale rating, were selected and used in the MEG experiment. The selected animations depicted eight actions: walking (viewed from the front, walking toward the screen), walking (viewed from the side walking either toward the left or toward the right), running, throwing, boxing, skipping (on one leg), skipping (side to side), and a high kick into the air.

Plausible condition (I): Each animation in its original form as recorded. In the pretest, subjects reported to perceive the stimuli as normal, biomechanically probable biological motion.

Scrambled condition (II): Scrambled versions of each animation were created by randomizing the spatial positions of all dots within the field of the original figure [Grossman et al., 2000; Pavlova and Sokolov, 2003; Saygin et al., 2004]. Again, the net movement of the dots is unchanged, whereas the spatial configuration of a human figure is completely destroyed. In the pretest, subjects rated these stimuli as meaningless movements of dots.

Implausible condition (III): Implausible versions of each animation were created by randomizing the starting positions of two dots from the upper body and two from the lower body, whereas leaving their motion paths unchanged. This manipulation leaves the overall movement of all dots unchanged and alters the configural structure only minimally. In the pretest, subjects reported to perceive the stimuli as “somehow human” but the actions as biomechanically implausible.

Experimental Procedure

Subjects were seated comfortably with their head placed inside the MEG helmet. Visual stimuli were projected on the backside of a translucent screen positioned 100 cm in front of the subjects using a projector (PT‐DW700E; Panasonic) with a refresh rate of 60 Hz placed outside the shielded room. Each trial started with the presentation of a central red cross (0.4 × 0.4 cm; visual angle. 0.23°). After a randomized period of 800–1,300 ms, in which only the red fixation cross was visible, the point‐light animation (8.4 × 3.4 cm; visual angle, 4.81° × 1.95°) appeared for a period of 3,600–5,000 ms (five cycles). The red fixation cross was centrally present throughout the duration of the stimuli to minimize eye movements. After another random period of 0–1,000 ms, in which only a black screen was visible, response instructions were visually presented on the screen. Subjects were asked to rate the animation using a 1–4 rating scale as either plausible (1), implausible (2–3), or scrambled (4) by button presses. Once a response was given, a new trial started. The assignment of the four‐fingers to the four configurations of the rating scale was randomized for each trial and response hands were balanced across subjects (Fig. 1). If no response was given within 2,000 ms, or if a response was given too quickly (before the response instructions appeared), the trial was discarded from analysis and repeated at the end of the block. No feedback was given. The estimated duration of a trial was 4,400–7,300 ms, followed by the individual response period (maximum of 2,000 ms). Stimuli were presented in pseudo‐random order within a block. One block consisted of 31 trials, so that each block had an estimated duration of 136.4–226.3 s, respectively, without individual response times (max. 2,000 ms) taken into account (no. of trials × duration). If response times are taken into account, each block had an estimate duration of ∼5 min. Overall, five blocks were presented, with self‐timed breaks of ∼2 min in between blocks. On the whole, the experiment lasted ∼25–30 min. Subjects performed a training session of ∼5 min before the start of the MEG experiment. Stimulus presentation was controlled using Presentation Software (Neurobehavioral Systems, Albany, NY).

Data Acquisition and Analysis

While subjects performed the task, neuromagnetic activity was recorded continuously at a sampling rate of 1,000 Hz with a 306‐channel whole head MEG system (Neuromag Elekta Oy, Helsinki, Finland). This system includes 204 planar gradiometers and 102 magnetometers arranged in a helmet configuration. In the present study, data analysis was carried out only with the planar gradiometers. In addition, vertical and horizontal electrooculograms were recorded simultaneously for offline artifact rejection. Subjects' head position within the MEG helmet was registered by four coils placed at subjects' forehead and behind the left and right ear. A 3T MRI scanner (Siemens, Erlangen, Germany) was used to obtain individual full brain high‐resolution standard T1‐weigthed structural magnetic resonance images (MRIs). MRIs were offline aligned with the MEG coordinate system using the coils and anatomical landmarks (nasion, left, and right preauricular points).

Data were analyzed offline with the open source toolbox FieldTrip for Matlab http://www.ru.nl/donders/fieldtrip) [Oostenveld et al., 2011]. Continuously recorded data were cut into epochs as defined by the trials. All epochs were first semi‐automatically and then visually inspected for artifacts. Artifacts caused by eye movements or muscle activity were removed. Power line noise was removed by applying a Fourier transformation of 10‐s long signal periods and subtracting the 50, 100, and 150 Hz components.

Time–Frequency Analysis

Time–frequency representations were computed separately for two frequency windows: For frequencies ranging from 4 to 40 Hz (in steps of 2 Hz), we applied a Fourier transformation on 500‐ms windows moved in steps of 50 ms. Data segments were tapered with a single Hanning taper, resulting in a spectral smoothing of ±2.0 Hz. For the frequencies from 40 to 100 Hz (in steps of 5 Hz), a Fourier transformation was applied on 400‐ms windows moved in steps of 50 ms, using the multitaper approach [Walden et al., 1995]. Data segments were tapered with seven tapers, resulting in a spectral smoothing of ±10.0 Hz around each center frequency.

As we were interested in the development of power over time and power correlations across frequencies (for details, see correlation analysis), we used a Fourier transformation on constant time window and tapering for all frequencies within a frequency band. This approach ensures that the same data set and same tapers are used within a frequency band. Any changes observed are thus attributed to the frequency components, rather than changes in time windows and/or tapers. We used different time windows and tapering for low and high frequencies because low‐frequency bands are relatively narrow and closely spaced. We therefore aimed at a high spectral resolution in the low frequency range of roughly ±2 Hz (i.e., 1/500 ms). In the higher frequency range, frequency bands are broader and spaced more far apart so that we applied a spectral smoothing of ±10.0 Hz. This approach provided an acceptable trade‐off between capture of physiological frequency bands and comparability within‐ and between‐frequencies. Previous studies have identified parieto‐occipital, left and right temporal, and sensorimotor areas as crucial areas in the recognition of PLD actions [Grossman et al., 2000; Michels et al., 2009; Saygin et al., 2004; Schippers and Keysers, 2011]. To identify these regions of interest in sensor space in our study, we applied a combined data driven and a priori approach. First, we pooled all trials together irrespective of stimulus conditions (plausible, implausible, and scrambled) and determined which sensors showed clear perturbations of oscillatory activity in response to PLD relative to baseline (−400 to −250 ms). Six sensors in the right hemisphere showing a sustained decrease in α (7–13 Hz) and β (13‐23 Hz) power as well as a selective sustained increase in γ (55–95 Hz) power where selected over parieto‐occipital areas (Fig. 2A). In addition, 10 sensors, five in the left hemisphere and symmetrically five in the right hemisphere, showing a sustained decrease in low α (7–11 Hz) and a selective increase in high α (11–13 Hz) power as well as a sustained decrease in β (15–23 Hz) power where selected over the sensorimotor cortex (Fig. 2B). Finally, owing to the vast reports on the importance of STS and temporal areas in action recognition (e.g., Dinstein et al., 2007; Grossman and Blake, 2001, 2002; Grossman et al., 2000; Pavlova et al., 2004; Pelphrey et al., 2004), eight sensors in the left and symmetrically eight sensors in the right hemisphere were selected over the temporal cortices. Although temporal cortices showed similar effects in the lower range frequencies (4–40 Hz) as observed in parieto‐occipital areas, the difference in the γ‐band effects between the two suggests that both process action representations differently. To asses the different roles of the parieto‐occipital, temporal, and sensorimotor areas in action recognition, we next investigated the contrasts between the conditions.

Figure 2.

Stimulation effects of PLDs. Top row shows our sensors of interest for (A) parieto‐occipital (x), (B) sensorimotor (*), (C) left‐temporal (+), and (D) right‐temporal (+) areas, respectively. Color maps illustrate changes in power relative to baseline (−400 to −250 ms), which were calculated separately for low (4–40 Hz) and high frequencies (40–100 Hz) by pooling all trials together irrespective of conditions.

Condition Contrasts

We assessed differences in spectral power between stimulus conditions in the four above‐mentioned regions of interest (parieto‐occipital, left and right temporal, sensorimotor). To this end, we averaged spectral power over the sensors of interest for each stimulus condition separately. Next, we compared stimulus conditions for each subject by subtracting power of both conditions and dividing the difference by the variance (equivalent to an independent‐sample t‐test). This step serves as a normalization of interindividual differences [Hoogenboom et al., 2010; Lange et al., 2011]. This comparison was carried out for each time–frequency sample independently, resulting in a time–frequency map of pseudo‐t‐values for each subject. To minimize influences of motor activity owing to response preparation, statistical analyses were restricted to the first 3 s. Next, we analyzed the consistency of pseudo‐t‐values over subjects by means of a nonparametric randomization test. This statistical test effectively corrects for multiple comparisons [Maris and Oostenveld, 2007]. To this end, time–frequency pseudo‐t‐values exceeding a threshold (P < 0.05) were identified and neighboring significant time–frequency pseudo‐t‐values were combined to a cluster. For each cluster, the sum of the t‐values was used in the second‐level cluster‐level test statistics. We used the Monte Carlo approach to estimate the permutation P‐value of the cluster by comparing the cluster‐level test statistic with a randomization null distribution. The null distribution was computed by randomly assigning data to different conditions, under the null hypothesis of no difference between conditions and thus exchangeability of the data. The random reassignment of the data to conditions was performed 1,000 times. For each of these 1,000 repetitions, a group t‐value was calculated. Finally, a P‐value was estimated for each cluster as the proportion of the elements in the randomization null distribution exceeding the observed maximum cluster‐level test statistic (for details, see Lange et al., 2011). This group level statistics results in time–frequency clusters which reveal differences between conditions that were significant at the random effects level after correcting for multiple comparisons along both the time and the frequency dimension [Maris and Oostenveld, 2007].

In the present study, we were interested in how processing of plausible actions differs from processing of nonactions. As discussed in the Introduction section, most fMRI studies, to date, on PLD action recognition have dealt with a similar question by comparing actions to scrambled versions of these actions. Based on our main research question and for the sake of comparability, we will focus in our present study on the main contrast of plausible (actions) versus scrambled (nonactions) conditions. The comparison between plausible and implausible actions engages different research questions and thus presumably different cortical networks and mechanisms which lie beyond the scope of the present study.

Source Analysis

To determine the neuronal sources, we applied dynamic imaging of coherent sources (DICS), an adaptive spatial filtering beamforming technique [Gross et al., 2001]. To this end, a regular three‐dimensional 1‐cm grid in the Montreal Neurological Institute (MNI) template brain was created and the structural MRI of each subject was linearly warped onto this template brain. The inverse of this warp was applied to the template grid, resulting in individual grids. This approach allowed us to average source parameters over subjects by simply averaging over grid points. For each grid point then, a forward model based on a realistic single shell volume conductor based on the individual MRI was used to calculate the lead‐field matrix [Nolte, 2003]. We next applied a Fourier transformation on time–windows of interest and computed the cross‐spectral density (CSD) matrix between all MEG sensor pairs for the frequency bands of interest, which were determined by the significant time clusters on sensor level. Spatial filters were constructed for each individual grid point using the CSD and lead field matrix. These filters pass activity from the location of interest, whereas suppressing activity from all other locations. Spatial filters w(r,f) were computed from the following formula:

$$w(\mathrm{r}\mathrm{,}\mathrm{f})=(L^{\prime }(r)C(f)+\lambda \hspace{1em}x\hspace{1em}I){}^{-1}{L}^{\prime }(r){}^{-1}{L}^{\prime }(r)C(f)+\lambda \hspace{1em}x\hspace{1em}I){}^{-1},$$

where L′(r) is the inverse of lead‐field matrix (forward model) at location of interest r, C(f) is the CSD matrix between all MEG signals at frequency f, λ is the regularization parameter, and I is the identity matrix [de Lange et al., 2008; Gross et al., 2001].

First, we pooled all conditions (pre‐ and post‐stimulus period for stimulation effects; plausible and scrambled conditions for condition contrast) and computed common filters. Next, CSD matrices of single trials were projected through those filters, providing single trial estimates of source power [Hoogenboom et al., 2010; Lange et al., 2011]. In line with the analysis on sensor level, we computed a relative change to baseline for stimulation contrasts and a between‐condition t‐value for condition contrasts for each subject. Statistical testing on group level for time–frequency representations of stimulation effects (P < 0.05, cluster corrected) and condition contrasts (P < 0.05, uncorrected) was carried out in the same way as on sensor level (see above). Results were displayed on the MNI template brain and neuronal sources were identified using the AFNI atlas (http://afni.nimh.nih.gov/afni), integrated into FieldTrip.

Cross‐frequency Correlations

To investigate the interaction between visual and motor areas during the recognition of plausible actions, we calculated the crossfrequency coupling over the specific time course of our significant clusters. Cross‐frequency coupling refers to the coupling of the neuronal signal between distinct frequency bands in the same or different cortical regions [Jensen and Colgin, 2007]. Here, we investigated the power correlation between the significant time–frequency clusters of the above‐mentioned time–frequency analysis. For each trial, we averaged spectral power across the time and frequency bins defined by the significant clusters on group level (Fig. 4A). Next, we computed correlations between sensorimotor β‐power on the one hand and parieto‐occipital γ and temporal α‐power on the other hand. Power correlations were determined per subject on a trial‐by‐trial basis by computing Pearson correlation coefficient. Individual correlation coefficients were converted to z‐values using the Pearson's r‐to‐z transform to attain a normally distributed variable [Choi, 1977]. The distribution of correlation coefficients across subjects was statistically tested against the null hypothesis of no correlation, that is, r = 0 by using a two‐sided t‐test. To test for a temporal specificity of the correlations, frequency bands of interest were shifted in steps of ±100 ms and correlations were recomputed as described above.

Figure 4.

Condition contrasts for plausible versus scrambled PLD: (A) Representations of significant clusters (P < 0.05) found on sensor level for (x) parieto‐occipital, (*) sensorimotor cortex, and (+) left‐temporal, and (+) right‐temporal. Red denotes higher power for plausible, whereas blue denotes higher power for scrambled. Source reconstruction of the significant clusters found on sensor level for (B) parieto‐occipital γ increase, (C) right temporal γ decrease, (D) sensorimotor β‐increase, (E) parieto‐occipital β‐decrease and, (F) left temporal α increase. Color map represents t‐values for source reconstruction. Red denotes higher power for plausible, whereas blue denotes higher power for scrambled. Arrows and r‐values represent significant (P < 0.05) positive trial‐by‐trial correlations for the plausible condition between sensorimotor β and (I) parieto‐occipital γ‐power as well as (II) left temporal α‐power.

RESULTS

Behavioral Data

The subjects rating of the PLD motion as plausible or scrambled indicated that they could easily distinguish both stimuli with an average rating of 1.5 (±0.19) for all plausible, and 3.8 (±0.14) for all scrambled. Statistical testing revealed highly significant differences between both conditions (P < 0.001).

Stimulation Effects

We first determined the effects of stimulation by pooling all trials irrespective of condition (plausible, implausible, and scrambled) and computing time–frequency representations of neural oscillatory activity in response to the PLD relative to baseline (−400 to −250 ms). We focused on four main areas, which showed clear perturbations of spectral activity in response to visual stimulation:

Parieto‐occipital areas: PLD elicited an increase of power in the θ‐band (5–7 Hz) immediately after stimulus onset (0–0.3 s). In addition, we observed a sustained decrease in the α (7–13 Hz) and β (13–21 Hz) band power after stimulus onset (0.2–4.5 s), as well as a sustained increase in γ‐power (70–95 Hz) between 0.1 and 4.5 s poststimulus onset. All stimulation effects showed a clear bilateral distribution in parieto‐occipital areas, with the γ‐band effect more strongly pronounced to the right hemisphere (Fig. 2A).

Sensorimotor areas: PLD elicited a weak increase in θ‐band (5–7 Hz) power after stimulus onset (0.0–0.3 s), which, however, is most likely owing to spatial smearing from the parieto‐occipital areas. In addition, we observed a distinct and sustained increase in high α (11–13 Hz) power between 0.4 and 4.0 s and a sustained decrease in low α (7–11 Hz) and β (15–23 Hz) power between 0.5 and 4.5 s poststimulus onset in bilateral sensorimotor areas (Fig. 2B).

Temporal areas: PLD elicited a bilateral increase in θ‐band (5–7 Hz) power after stimulus onset (0.0–0.3 s). In addition, we observed a sustained bilateral decrease in low α (7–11 Hz) and β‐power (13–23 Hz) between 0.5 and 4.5 s, as well as a bilateral increase in high α (11–13 Hz) between 0.3 and 0.6 s (Fig. 2C,D). These effects are highly similar to the effects found in sensors over parieto‐occipital and sensorimotor areas (see above) but with lower amplitude. In contrast to the results from parieto‐occipital sensory, we observed a robust early increase (90–100 Hz) between 0 and 1.5 s and a sustained decrease in oscillatory γ‐power (50–80 Hz) between 0 and 4.5 s poststimulus onset in right temporal cortex (Fig. 2D).

Next, we identified the cortical sources of the sustained effects, found in the time–frequency representations on sensor level. To this end, we performed source localization using a beamformer on four distinct frequency bands, based on the results on sensor level (i.e., for low α [7–11 Hz], high α [11–13 Hz], β [13–25 Hz], and γ [50–100 Hz] band). Strongest cortical sources were identified in visual as well as sensorimotor areas (for details, see Fig. 3).

Figure 3.

Stimulation effects on source level. Cortical sources of relative change for low α (7–11 Hz), high α (11–13 Hz), β (13–25 Hz), and γ (70–100 Hz) power, respectively. Color maps illustrate changes in power relative to baseline. Only significant sources (P < 0.05; cluster corrected) are shown.

Condition Contrast

We assessed differences between plausible versus scrambled stimuli in the four regions of interest (parieto‐occipital, left, and right temporal, and sensorimotor areas). We found a significant increase in γ (55–90 Hz) power at 500–800 ms poststimulus onset in parieto‐occipital areas (P < 0.05), followed by a significant increase in β (20–35 Hz) power at 700–1,200 ms poststimulus onset in sensorimotor areas as well as a significant increase in high α (9–13 Hz) power at 900–1,300 ms in left temporal areas (Fig. 4A). In addition, we found a significant decrease in γ (50–80 Hz) and α/low β (10–22 Hz) power, between 1,300 and 2,000 ms in right temporal and parieto‐occipital (Fig. 4A) areas (P < 0.05), respectively.

Next, we identified the cortical sources of these significant clusters. For the increase in γ‐power (55–90 Hz) between 500 and 800 ms, the sources were identified in the primary visual cortex (V1). Additional sources were identified in the right medial and inferior temporal gyrus, as well as right dorsolateral prefrontal cortex (DLPFC) (Fig. 4B).

The sources of the significant effects between 20–35 Hz and 700–1,200 ms were located in the bilateral sensorimotor areas of the brain and more specifically the PMC and right primary motor cortex (M1) (Fig. 4D).

The sources for the significant effects between 9–13 Hz and 900–1,300 ms were located in the left temporal areas of the brain and more specifically the STS. Additional sources were identified in the left somatosensory areas (Fig. 4F).

The sources for the significant effects between 50–80 Hz and 1,300–1,600 ms were located in the right temporal areas of the brain and more specifically the right medial and inferior temporal cortex. Additional sources were identified in the right DLPFC (Fig. 4C).

Finally, the sources of the significant cluster between 10–22 Hz and 1,600–2,000 ms were localized in bilateral parieto‐occipital areas and more specifically visual areas V1 and V2 as well as right parietal posterior (Fig. 4E).

Cross‐frequency Correlations

To assess the interactions between visual and sensorimotor areas during the recognition of actions, we calculated the trial‐by‐trial cross‐frequency correlation between the significant time–frequency clusters (Fig. 4A). A significant positive correlation was observed between sensorimotor β (averaged between 20–35 Hz and 700–1,200 ms) and parieto‐occipital γ (averaged between 55–75 Hz and 500–800 ms) power (r = 0.09; P < 0.05) as well as between sensorimotor β and left temporal α (9–13 Hz and 900–1,300 ms) power (r = 0.20; P < 0.05) for the plausible action condition, but not for the scrambled one. No significant correlation was observed when the time windows of the significant clusters were moved in steps of ±100 ms. In addition, a significant positive trial‐by‐trial correlation was observed between parieto‐occipital β (10–22 Hz and 1,600–2,000 ms) and right temporal γ (50–80 Hz and 1,300–1,600 ms) power (r = 0.16; P < 0.05) for the scrambled condition. A significant positive trial‐by‐trial correlation was still visible when the time windows of the significant clusters were simultaneously moved in steps of −100 ms (r = 0.19; P < 0.05), but not for other time shifts. Finally, a significant negative trial‐by‐trial correlation was observed between sensorimotor β (20–35 Hz and 700–1,200 ms) and parieto‐occipital β (10–22 Hz and 1,600–2,000 ms) power (r = 0.08; P < 0.05) for the scrambled condition that was not present when the time windows were moved in steps of ±100 ms.

DISCUSSION

The present study aimed at determining the dynamic modulations of neuronal oscillatory activity in the cortical networks involved in the recognition of plausible actions. PLDs elicited sustained effects in θ (5–7 Hz), α (7–13 Hz), β (15–25 Hz), and γ (50–100 Hz) power within cortical areas of the MNS. We were particularly interested how these dynamic modulations as well as the interactions between areas of MNS changed when we compared plausible and scrambled actions. We will first discuss the observed stimulation‐induced effects with respect to earlier hemodynamic and electrophysiological reports. The main focus of this article is the comparison of our two conditions and their interactions between cortical areas of the MNS, which will then be applied to current theories of the action recognition process.

Presentation of PLD (pooled over all conditions) induced a sustained decrease of spectral power in the α‐ and β‐band in parieto‐occipital regions. The decrease started at ∼200 ms poststimulus onset and was sustained throughout the trial. The decrease as well as its timing is in line with the previous reports on visual stimulation (e.g., de Lange et al., 2008; Hoogenboom et al., 2006; Koelewijn et al., 2008; Singh et al., 2002). The decrease of α/β‐power was also found in sensorimotor areas, starting at around ∼500 ms poststimulus and lasting until the end of the trial, in agreement with the earlier reports of α/β suppression during action preparation, action execution, and motor imagery tasks (de Lange et al., 2008; Hari and Salmelin, 1997; Koelewijn et al., 2008; Oberman et al., 2005; Orgs et al., 2008; Schnitzler et al., 1997; Ulloa and Pineda, 2007). Moreover, somatosensory areas have been suggested to play a role in the internal simulation of the sensory consequences of observed actions or embodiment [Caetano et al., 2007; de Lussanet et al., 2008]. In contrast to the suppression of low α band‐power, sensorimotor areas revealed an increase of high α (11–13 Hz) band power between 400 and 4,000 ms poststimulus. While a decrease of α/β‐band power has been linked to engagement of sensorimotor areas, an increase has been suggested to reflect inhibition or disengagement of the sensorimotor system [Hummel et al., 2002; Jensen et al., 2002; Nachev et al., 2008; Neuper and Pfurtscheller, 2001]. The early observed increase in high α‐band power might thus reflect subjects' active inhibition of finger and/or eye movements during stimulus presentation or suppression of task‐irrelevant areas during initial stimulus presentation. Finally, we observed a sustained increase of high γ‐band power in a wide range of areas including frontal and posterior regions of the brain (for details, see Fig. 3). This increase of γ‐power is visible between 100 and 4,500 ms, that is it starts slightly earlier than the other sustained effects, similar to the previous reports involving visuomotor tasks [Aoki et al., 1999; de Lange et al., 2008; Pavlova et al., 2004, 2006; Pfurtscheller and Neuper, 1992].

When comparing plausible to scrambled condition, we observed an early increase of γ (55–75 Hz) band power between 500 and 800 ms in right V1 and temporal cortex. Other electrophysiological studies report an increase in γ‐power as early as 80–170 ms when subjects passively viewed point‐light walkers [Pavlova et al., 2004, 2006]. One reason for the differences in timing might be owing to the different definition of γ‐band activity: Although we observed γ‐band effects between 55 and 75 Hz, Pavlova et al. found effects in the lower γ‐band between 25 and 40 Hz. In addition, differences might be owing to the different experimental designs between Pavlova et al. (passive viewing of normal, scrambled, inverted PLD) and our study (active evaluation of normal, implausible, and scrambled PLD). This increase of γ‐band, however, is in line with increased hemodynamic responses in parieto‐occipital and temporal areas for plausible versus scrambled PLD (e.g. Grossman and Blake, 2002; Grossman et al., 2000; Michels et al., 2009, 2005; Pelphrey et al., 2004, 2005). Neuronal activity, especially γ‐band activity, in right temporal areas reflects the processing of the global form of the PLD, which is only recognizable in the plausible condition [Michels et al., 2009, 2005; Pavlova et al., 2004]. As the γ‐band effect was the earliest significant cluster, the result suggests that discrimination between plausible and scrambled PLD starts at early, low‐, and high‐level visual stages of the action recognition process (e.g. Pavlova et al., 2004).

The increase of γ‐band power was followed by an increase of power in the β (20–35 Hz) band between 700 and 1,200 ms in bilateral sensorimotor areas (PMC and M1). Similar to the timing of the sensorimotor β‐effect, previous electrophysiological studies reported sensorimotor α/β decreases to differentiate during action observation or motor imagery in the time period of ∼450–1,500 ms poststimulus onset [de Lange et al., 2008; Orgs et al., 2008; Schnitzler et al., 1997]. Previous fMRI studies demonstrated that sensorimotor areas and more specifically the PMC, responded to both human (plausible) and nonhuman (scrambled) actions, but much stronger for human actions belonging to the observer's own motor repertoire [Buccino et al., 2004b; Saygin et al., 2004]. In addition, Calvo‐Merino et al. (2005) observed a stronger hemodynamic response in STG, premotor, and parietal areas when capoeira and classical ballet dancers observed movements from their own repertoire. The observed positive β‐cluster in sensorimotor areas reflects a stronger suppression of power for scrambled than plausible actions. In contrast to this observation, one previous study revealed a stronger suppression of sensorimotor β‐power when subjects viewed actions within their own repertoire compared to other plausible, but clearly distinguishable movements [Orgs et al., 2008]. Interestingly however, stronger suppression of sensorimotor β‐band power has been reported for the observation of incorrect versus correct button presses [Koelewijn et al., 2008]. Although subjects in the study by Koelewijn et al. had to distinguish between correct and incorrect button presses, subjects in our study had to distinguish between normal and scrambled actions. Despite these notable differences in the experimental setup, we observed a similar pattern of β‐decrease as reported by Koelewijn et al. We therefore speculate that stronger β‐band suppression in our study might thus be related to the recognition of the scrambled action movements as incorrect. Future studies, however, are needed to support this speculation.

The sensorimotor β‐increase was followed by an α (9–13 Hz) band increase between 900 and 1,300 ms in left S1 and STS. An increase in α power might reflect suppression of task‐irrelevant areas during initial stimulus presentation, as well as active inhibition or disengagement of the cortical areas involved (e.g. Jensen and Mazaheri, 2010; Jensen et al., 2002). The observed α‐power increase over left STS and somatosensory areas, two areas known to be involved in the processing [Allison et al., 2000] and internal simulation [de Lussanet et al., 2008] of biological actions, might thus reflect active inhibition of these areas. Previous electrophysiological studies reported α activity of temporal areas peaking at around ∼750 ms during a visual attention task [Pantazis et al., 2009]. The observed left hemisphere activity might reflect visual attention of the local details of the PLD when differentiating between plausible and scrambled conditions [Bonda et al., 1996; Fink et al., 1997; Lamb and Robertson, 1988].

Interestingly, we observed a significant positive trial‐by‐trial correlation between sensorimotor β‐power and parieto‐occipital γ‐power as well as left temporal α‐power. This correlation was observed only for plausible PLD but not for scrambled PLD, and the correlation was observed only at specific time points, namely at time points where we found the significant power increase for plausible PLD. This finding illustrates a crossfrequency coupling between visual and motor areas during recognition of plausible actions operating at large spatio‐temporal scales. The temporal profiles of the power changes suggest a functional interaction proceeding from visual areas to sensorimotor areas and back projecting to STS.

At a later time point, we observed an additional negative cluster in the β‐band in parieto‐occipital areas, reflecting a stronger β‐band power for the scrambled than the plausible condition. This finding is in line with fMRI studies which suggest that parieto‐occipital areas are more sensitive to image scrambling (for review, see Grill‐Spector and Malach, 2004). Trial‐by‐trial correlations between this late parieto‐occipital β‐band power and early sensorimotor power revealed a negative correlation for the scrambled PLD, but no significant correlation for the plausible PLD. This finding reveals crossfrequency coupling between sensorimotor and visual areas over several hundred milliseconds. We suggest that this effect reflects feedback projections from sensorimotor areas to visual areas, possibly updating visual processing [Schippers and Keysers, 2011]. Interestingly, all correlations have been observed between sensorimotor β‐power and other frequencies in other areas. Oscillations in the β‐band have been widely observed in sensorimotor areas in relation to motor behavior [Haegens et al., 2011; Salenius and Hari, 2003] and have been proposed as a mechanism for synchronization over long transmission delays and long ranges [Bibbig et al., 2002; Gross et al., 2004; Kopell et al., 2000; Schnitzler and Gross, 2005]. We suggest that β‐oscillations supply a mechanism that combines visual and motor areas into a functional network [Brovelli et al., 2004].

The power correlations, although low in absolute value, are statistically significant and consistent across all subjects. Studies investigating working memory with intracranial EEG (iEEG) have reported correlation with absolute values >0.3 (e.g. Axmacher et al., 2010). This difference might be owing to a higher signal‐to‐noise ratio for iEEG when compared to MEG. The absolute values of the correlation values (0.07–0.20) of our study, however, are in line with the previous MEG studies, reporting power correlations in the range of 0.01–0.07 (e.g., Hipp et al., 2012; Hoogenboom et al., 2010).

Interestingly, we also observed a much stronger γ‐power for scrambled PLD in right DLPFC. DLPFC activity has been linked to the process of evaluating other people's behavior (e.g. Saygin, 2007; Saygin et al., 2004). It has been, therefore, suggested that DLPFC is an important contributor to cognitive control in a social domain, as its role is to maintain intentions of our actions in working memory, and subsequently using feedback to evaluate whether our actions match those intentions [Weissman et al., 2008]. The stronger suppression of γ‐power for plausible than scrambled actions might thus reflect DLPFC efforts in trying to evaluate the intentions of the scrambled actions that do not match the intentions stored in working memory.

In summary, our results reveal a widespread cortical network involved in the recognition of plausible actions, including areas of the MNS operating at different frequency bands, extending previous fMRI and MEG studies. We demonstrate interactions between these areas by revealing power correlations between visual and motor areas during the recognition of plausible and scrambled actions at specific spatial‐temporal scales. We propose that these results reveal a functional coupling of visual and motor areas, predominantly coupled to the sensorimotor β‐frequency, in support to current models of motor control that propose the presence of internal models (inverse and forward) involving visual and motor interactions.