Cortical regions involved in the observation of bimanual actions

Abstract

Although we are beginning to understand how observed actions performed by conspecifics with a single hand are processed and how bimanual actions are controlled by the motor system, we know very little about the processing of observed bimanual actions. We used fMRI to compare the observation of bimanual manipulative actions with their unimanual components, relative to visual control conditions equalized for visual motion. Bimanual action observation did not activate any region specialized for processing visual signals related to this more elaborated action. On the contrary, observation of bimanual and unimanual actions activated similar occipito-temporal, parietal and premotor networks. However, whole-brain as well as region of interest (ROI) analyses revealed that this network functions differently under bimanual and unimanual conditions. Indeed, in bimanual conditions, activity in the network was overall more bilateral, especially in parietal cortex. In addition, ROI analyses indicated bilateral parietal activation patterns across hand conditions distinctly different from those at other levels of the action-observation network. These activation patterns suggest that while occipito-temporal and premotor levels are involved with processing the kinematics of the observed actions, the parietal cortex is more involved in the processing of static, postural aspects of the observed action. This study adds bimanual cooperation to the growing list of distinctions between parietal and premotor cortex regarding factors affecting visual processing of observed actions.

several studies have reported human cortical regions involved in action observation, i.e., the visual processing of others' actions. Some such actions involved the whole body (e.g., Bonda et al. 1996; Grossman et al. 2000; Peuskens et al. 2005), but manipulative actions investigated thus far have generally been unimanual actions, performed with the right hand (Binkofski et al. 1999; Buccino et al. 2001; Gazzola et al. 2007; Grafton et al. 1996; Hari et al. 1998; Iacoboni et al. 1999; Jastorff et al. 2010; Montgomery et al. 2007; Peeters et al. 2009; Pelphrey et al. 2005; Rizzolatti et al. 1996; Shmuelof and Zohary 2006; Turella et al. 2009; but see Schubotz and von Cramon 2009). Thus, despite the predominance of bimanual movements in human activities, the cortical regions involved in bimanual manipulative action observation are unknown, as no study has directly compared observation of uni-and bimanual actions. In monkeys, mirror neurons in F5 selective for the observation of hand interactions have been described (Gallese et al. 1996). However, these interactions consisted of one hand holding food and moving toward the other hand to touch it, which falls short of bimanual manipulative hand actions, in which, in general, one hand is holding an object to allow the other hand to manipulate it more effectively.

What predictions can we make for the comparison of observing bi- and unimanual manipulative actions? Predictions can either be based on the nature of the visual stimuli, action performed by conspecifics, or on the interaction between two visual stimuli, moving left and right hands. Using the nature of the stimuli, some predictions can be derived from bimanual motor studies, since observation and execution networks largely overlap at the parietal and frontal level, at least for unimanual actions (Aziz-Zadeh et al. 2006; Caspers et al. 2010; Dinstein et al. 2007; Iacoboni et al. 1999; Turella et al. 2009). Such predictions must be made with some caution, as thus far mirror neurons, which are the substrate for common regions involved in planning of actions and observation of actions, have been reported only for grasping and similar manipulative actions (Gallese et al. 1996). In addition, such neurons have been reported in premotor and parietal cortex (Fogassi et al. 2005) but not in the primary motor cortex of monkeys, even if congruence between M1 neuronal activity during observation of cursor movement and control of cursor movement has been reported (Tkach et al. 2007). Thus not all processes engaged during execution of actions are necessarily also involved in observation of actions. The absence of mirror neurons in M1 (Gallese et al. 1996) strongly suggests that one such process is the control of force exerted during a movement. It would indeed be difficult to imagine what visual cue could be used to visually retrieve small differences in force exerted with the fingers of the two hands. Other processes specific to action execution are monitoring of the ongoing action (Andersen and Cui 2009) and reafferent signals to inform sensory systems, and in particular the visual system (Iacoboni et al. 2001), about the ongoing action and the changes it produces in the environment. A number of studies have investigated motor control of bimanual actions, highlighting the role of cortical areas involved in controlling the contralateral hand (Debaere et al. 2003, 2004; Diedrichsen et al. 2006; Immisch et al. 2001; Jancke et al. 2000; Johansson et al. 2006; Sadato et al. 1997; Stephan et al. 1999; Swinnen 2002; Swinnen and Wenderoth 2004; Toyokura et al. 1999, 2002; Tracy et al. 2001; Ullen et al. 2003; Wenderoth et al. 2005b, 2006). Since the contralateral cortex also dominates unilateral hand actions, these motor studies imply that no specific region is involved in the motor planning of bimanual actions. However, it may also be that a region involved in the control or planning of bimanual actions has thus far escaped detection (Tanji et al. 1988). The results of Wenderoth et al. (2005a) may be taken as a suggestion that anterior cingulate and precuneus play such a role, although these areas may also be engaged by any complex action. Hence, execution studies at best suggest that an area might specifically be involved in the observation of bimanual compared with unimanual actions.

Predictions can also be derived from visual studies in which two stimuli are combined. One such type of study is the cue combination study. In general, cue combination studies, whether multimodal or intramodal, suggest (for review, see Green and Angelaki 2010) that responses to the cues combined should at least equal (additivity) or even exceed (superadditivity) the sum of the responses to the single cues. This would predict that the responses to observing bimanual actions should at least be equal to the sum of the responses to observing unilateral actions. Different predictions can be derived from single-cell studies of sensory interactions between stimuli presented in the same receptive field (RF) (Reynolds et al. 1999). Indeed, beyond MT/V5 most neurons in STS and parietal cortex have large RFs, extending into the ipsilateral visual field (Ben Hamed et al. 2001; Desimone and Ungerleider 1986; Duhamel et al. 1997). Hence, one might expect that observing bimanual action of an actor facing the subject would cause two stimuli to fall onto single RFs straddling the vertical meridian of many neurons in homologs of these areas in humans. The single-cell study has shown that when the stimuli are different, with one stimulus driving the neuron and the other only doing so weakly, the combined response is not the sum but the average of the two responses to the single stimuli. Thus this would predict that responses to observing bimanual actions would fall halfway between the responses to observing the unimanual actions.

The present study was undertaken to address the issue of observing bimanual actions directly and to compare the cortical regions involved in unimanual and bimanual action observation. More specifically, we intended to investigate whether any cortical area was specialized for the observation of bimanual actions, as some motor studies seem to suggest, or, alternatively, whether regions involved in unimanual action observation might function differently during bimanual action observation. In the latter case, opposite predictions are generated from visual studies: either bimanual responses exceed those to the strongest unimanual action, as predicted from cue combination, or the strongest unimanual action exceeds the bimanual action, as predicted by the sensory interaction between two stimuli of similar nature.

To address these questions, we scanned human subjects while they observed bimanual actions and their unimanual components, along with control conditions matched for low-level motion properties. Our results, relying on both whole-brain and region of interest (ROI)-based analyses, did not reveal any cortical area specialized for the observation of bimanual actions. However, the fronto-parietal regions involved in unimanual action observation did indeed function differently during the observation of bimanual actions.

METHODS

Subjects

Nineteen healthy adult subjects (10 women and 9 men; mean age 23.4 yr, SD 4.5 yr, range 18–31 yr) participated in the study. All volunteers were right-handed as assessed with an adapted version of the Edinburgh Handedness Inventory (mean laterality = 95.8, SD = 8.4, range 67–100) (Oldfield 1971), had normal or corrected-to-normal vision, and had no history of mental illness or neurological diseases. The study was approved by the Ethical Committee of the Katholieke Universiteit Leuven Medical School, and all volunteers gave their written informed consent in accordance with the Helsinki Declaration before the experiment.

Stimuli and Conditions of Main Experiment

The stimuli were projected with a liquid crystal display projector (Barco Reality 6400i, 1,024 × 768, 60-Hz refresh frequency; Barco) onto a translucent screen positioned in the bore of the magnet at a distance of 36 cm from the eyes. Participants viewed the stimuli through a mirror tilted at 45° that was attached to the head coil. Prior to scanning, participants were familiarized with the different stimuli outside the scanner and were instructed to maintain their gaze on a red fixation target throughout the experiment. To reduce the amount of head motion during the scanning sessions, the participants were asked to use a bite bar fixed onto the head coil. Throughout the scanning session, the participant's eye movements were recorded with an ASL5000 eye tracking system (60 Hz; Applied Science Laboratories, Bedford, MA).

Factorial design.

The action observation stimuli consisted of video clips (duration of 4 s, size of 22.5 × 16.8°) displaying a frontal view of a human actor (i.e., the observer viewed the actor from a 3rd-person perspective) sitting at a white table in front of a black background (Fig. 1A) and performing one of six motor acts involving the manipulation of various objects, using either both hands [bimanual (“Bi”) condition] or only the left or right hand [unimanual left (“UniL”) or unimanual right (“UniR”) condition]. In the bimanual condition, the two hands simultaneously carried out specific actions assigned to each hand. These actions were then repeated in the unimanual conditions, in which the same objects were manipulated by using either the right or left hand only, with that hand performing the same specific motor actions as those assigned to it in the bimanual conditions. The hand and forearm motions in the unimanual videos were matched as closely as possible to the equivalent motions in the bimanual movies. For each of the three hand-action conditions, a control condition was created. In these control videos, the first frame of each action video was presented but the motion of the hands was replaced by the motion of spherical control objects, “balls,” presented to the left and right of the actor's hands. These control balls were also present in the action videos but remained immobile. Thus the six experimental conditions corresponded to a factorial design with factors Type of stimulus (with two levels: action video vs. moving ball control videos) and Hand (with three levels: two hands, right hand, and left hand) (Fig. 1B). The seventh condition was a fixation baseline condition in which only a fixation square (0.2° side) was presented on a black background for 4 s. This fixation target was also present in the other six conditions.

Fig. 1.

Stimuli and experimental design. A: examples of bimanual action observation stimuli with fixation target. Top left: start frame of fixation “Banana peeling” video with fixation dot low and control balls low. Top right: start frame of “Bottle opening” video with fixation dot low and control balls high. Bottom left: final frame of the “Grapes” video with fixation dot high and control balls low. Bottom right: final frame of “Peanut-box” video with fixation dot high, control balls high. B: experimental design with factors Hand and Type of video. Bi, bimanual; UniR, unilateral right; UniL, unilateral left; ContBi, control bimanual; Cont UniR/L, control unilateral right/left.

Action videos.

All six motor acts were chosen with consideration to using these or very similar stimuli in experiments with monkeys. These actions are described for the bimanual action condition (see Fig. 1A) and included 1) picking up and peeling a banana (left hand grabs and holds the banana above the table, holding the lower third of the fruit, while the right hand then peels the fruit, dividing the initially intact peel into 4 separate “strips” and peeling down to just above the fingers of the left hand); 2) picking up a full transparent 500-ml disposable plastic water bottle and opening it (left hand grabs and holds the bottle above the table while the right hand then opens it by turning the cap, followed by removing and holding the bottle cap away from the screw top); 3) picking up a bunch of green grapes and removing one grape from that bunch (left hand grabs and holds the cluster above the table, with the right hand then plucking one grape and holding it a short distance away from the cluster of grapes held in the left hand); 4) picking up a peanut and cracking it (left hand picks up the peanut, holds it above the table with the right hand, then joins the left hand to crack the shell); 5) picking a peanut out of a small transparent plastic box filled to the top with peanuts (left hand moves to open and tilt the lid of the closed box away—maintaining contact with the lid after opening it—with the right hand then moving to take a peanut out of the box and holding it away from the box above the table); and 6) taking a peanut out of a ceramic coffee cup filled with peanuts (left hand moves to tilt the vertically positioned cup toward the actor, allowing visual access to the contents, with the right hand then taking a peanut out of the cup and holding it away from the cup above the table).

At the start of each video clip, the object was held in position by a three-pronged clamp ∼15 cm above the table surface to allow unimanual manipulation of the object. Each video started with the actor's hands resting on the table close to his or her body, with the palms facing downward. In the bimanual actions the left hand took the object from the clamp and held it while the right hand manipulated the object. In the unimanual actions, the inactive hand remained at this starting position on the table top for the entire duration of the video. In the unimanual right-hand actions, the right hand manipulated the object held by the clamp. In the unimanual left-hand actions, the left hand lifted the object from the clamp as it did in the bimanual actions. Thus the unimanual actions matched the components of the bimanual actions. However, because of the use of the three-pronged clamp, the unimanual actions were performed in a natural manner, since the clamp replaced the hand that remained immobile at the starting position. During video recording, special care was taken to ensure that only the actor's arms and hands moved and that no motion was noticeable in any other body part or in the actor's face. All video clips were recorded twice, i.e., once with a female actor and once with a male actor. In the beginning of each action video clip, the hands of the actor were positioned to either side of the object, at ∼2° laterally. Since subjects were required to fixate a target that was located directly vertically below or above the object, most of the hand trajectories were restricted to a single visual field: the left visual field for the right hand and the right visual field for the left hand (Fig. 2C).

Fig. 2.

Kinematics and trajectories in action and control videos. A: examples of motion paths for hand (green lines, left) and control balls (red lines, right) within a 4-s stimulus movie. B: example of the x (blue) and y (red) coordinates of hands (left and right, top) and balls (left and right, bottom) taken from a 4-s stimulus movie (see A) plotted as a function of time. The temporal order of the hand kinematics was reshuffled for the animation of the control balls, as is apparent by comparing the plot patterns of top and bottom panels for the left and the right hand/ball, respectively. C: superimposed trajectories of right and left hands (green lines) and right and left balls (red lines) across all stimuli (n = 42), plotted as a function of vertical and horizontal position relative to the lower fixation point. Trajectories of balls were collapsed over the upper and lower ball positions. Negative positions correspond to down and left. Below each group of trajectories, the distribution of azimuths is plotted. Vertical differences in position of the trajectories in the visual field were reduced by averaging 2 ball positions and 2 fixation target positions. The azimuths of the balls averaged 3° greater than those of the hands.

Control conditions.

In the control videos, the actor was present but did not move and the hands remained in the starting position for the entire video. The motion of the hands was replaced by the motion of control objects presented to the left and right of the actor's body (Fig. 2A). These objects were spheres covered with black-and-white patterns and subtended 2.2° of visual angle (see Fig. 1). The choice of this control condition was inspired by the demonstration of Jastorff et al. (2010) that a local motion control was a more stringent control for manipulative action observation than static or scrambled conditions. This control is based on the property of low-level motion-selective neurons in MT/V5, which are selective for direction and speed of motion but relatively insensitive to the shape of the moving stimulus in the monkey (Albright 1984; Zeki 1974), a finding that can be extended to homologous areas in the human (Kolster et al. 2010). The motion animating the control “balls” was based on the kinematics of the bimanual or unimanual hand motions as applicable (i.e., with only the left, only the right, or both balls moving). The temporal order of the speed-time curve was reshuffled for the animation of the control ball (see Fig. 2B for an example), so that the motion of the control balls differed from the original biological motion pattern while still retaining the kinematics in terms of overall distance moved per movie/stimulus condition. To that end, the motion of the hand in the corresponding videos was tracked manually frame by frame. A motion vector list based on this information was calculated for each video. Each vector list was then randomly split into four to six temporal segments, depending on how rapidly the vectors were changing, and the order of these segments was then shuffled to break up the contextual nature of the motion. The resulting speed curves were smoothed to minimize speed transients that could occur by juxtaposition of slow and fast segments. The resulting new motion vector lists were then used to animate the motion of the control balls: the ball moved at the same speeds as the hand but did so in a different temporal order. Thus local motion was conserved, as shown by the similarity of the horizontal and vertical speed distributions of hands and balls. Indeed, mean speeds were very similar: median values were 0.64 and 0.29°/s for the horizontal speed of right and left hands compared with 0.63 and 0.35°/s for right and left balls and 1.20 and 0.38°/s for the vertical speed of right and left hands compared with 1.22 and 0.56°/s for right and left balls. None of the distributions differed significantly (Wilcoxon test, all 4 P > 0.20). Since motion duration was the same for hands and balls, the total distance traveled by the hands and balls was also similar, but the trajectories were different, as the cumulative graphs show (Fig. 2C).

The control balls were presented in the video displays at one of two possible positions: 1) lower visual field, i.e., superimposed on the table surface, to the left and right of the object held by the three-pronged clamp (5° left and right of the midline and 2° below the position of the lower fixation dot), and 2) upper visual field, i.e., approximately even with the actor's shoulders (5° left and right of the object and 1° above the position of the upper red fixation dot). In the action movies where hands were moving and performing the motor acts, the pair of control balls was always present (either in a high or a low position) but did not move. In the bimanual control conditions (ContBi) both balls moved, whereas in the unimanual control conditions (ContUniR and ContUniL) only the appropriate control ball moved while the other ball remained stationary. Given that the ball's starting position was 3° more lateral than that of the hands, all ball trajectories were also confined to a single visual field (Fig. 2C). In Fig. 2C the ball trajectories were averaged over ball up and down starting positions, and therefore the trajectories of hands and balls can be compared. Because of the use of the two different ball starting positions, the average elevations of hands and balls were identical. However, the azimuth remained different; ball position averaged 3° degrees more lateral than the hands. It should be noted that even at 3–5° eccentricity the diameter of RFs in MT/V5 of the monkey is 5–6° at half-height (Raiguel et al. 1995). The RFs of the satellites of MT/V5, which receive input from MT/V5 (Ungerleider and Desimone 1986) and the homologs of which are in all likelihood included within hMT/V5+ (Kolster et al. 2010), are an order of magnitude larger (Desimone and Ungerleider 1986). Thus the slight difference in azimuths of hands and control balls is unlikely to have an effect at the level of hMT/V5+ and, even less so, at the level of the action observation regions, which are further downstream (Nelissen et al. 2011).

Number of video versions.

While watching the movies, subjects were instructed to fixate a small red square displayed ∼3.3° of visual angle above or below the manipulated object. These two positions were chosen to control for possible retinotopic effects arising from presentation of the control motion in the upper or lower visual field (see below). To reduce any possible influences of sharp peripheral borders in the visual stimulus videos, the edges of the videos were blurred with an elliptical mask (25° × 18°), leaving the actor and objects in the video unchanged but gradually blending the periphery with the black background.

Because the pair of control balls or fixation target could appear in two possible locations, each motor act was created in four versions, defined by the combinations of fixation target and control “ball” positions in the visual field: up/up, up/down, down/up, and down/down (Fig. 1A). The actions were performed by a male and a female actor. Combined with the permutations of the visual field, this yielded eight versions of each action: bottle, grapes, peanut-break, peanut-box, peanut-cup. For the banana stimulus, however, there were 16 movie versions, as the action of peeling the banana had to be divided into two sections because of the 4-s time constraint of the video clips. The number of versions applied to all six conditions: Bi, UniL, and UniR and their respective controls.

Block design.

Each time series (run) of the experiment included seven different types of blocks corresponding to the six conditions (6 experimental conditions—Bi, UniL, UniR, ContBi, ContUniL, ContUniR—and 1 fixation condition), and all experimental conditions pertained to a single motor act. Within a run, presentation of each condition was repeated twice. Blocks lasted 16 s, including the four versions of a given action video generated by the four different fixation target positions, with male and female actors shown in alternating blocks with the sequences M/F/M or F/M/F, counterbalanced across runs. Runs lasted 336 s (16 s × 7 conditions × 3 repetitions). The order of the conditions was randomized for each run and counterbalanced across subjects. Every run started with the acquisition of four dummy volumes to ensure that the MR signal had reached its steady state. For each action, the standard data acquisition included four time series for each subject, corresponding to the four permutations in the visual field.

Kinematics Across Stimulus Conditions

The metric for comparing hand kinematics was the mean movement speed (°/s) for each hand action and its corresponding control stimulus video. This measure of global movement was used because it applies both to the hands and the balls, unlike more complex measures related to relative movements of the fingers. A comparison of the kinematics of the left and right hand motions across the different actions (Fig. 3), using repeated-measures ANOVA (factors Hand, Action, and Sex), revealed a significant main effect of Hand, with the right hand being more active than the left (F_1,399 = 113.13, P < 0.0001), a significant effect of Action (F_5,1995 = 113.06, P < 0.0001), as well as a significant Hand × Action interaction effect (F_5,1995 = 136.42, P < 0.0001). There was no significant sex difference. Subsequent analyses for each stimulus modality using t-tests for independent samples revealed significant differences in the kinematics of the right and left hands for the banana stimuli (right ± SE 12.3 ± 0.42 vs. left ± SE 3.3 ± 0.15, P < 0.00001) and bottle (5.8 ± 0.36 vs. 2.5 ± 0.18, P < 0.00001), peanut-box (4.4 ± 0.28 vs. 7.2 ± 0.42, P < 0.00001), and peanut-cup (6.8 ± 0.43 vs. 2.3 ± 0.26, P < 0.00001) stimuli. No differences between left and right kinematics were apparent for the grapes (3.2 ± 0.24 vs. 3.4 ± 0.26, P = 0.48) or nuts (2.5 ± 0.22 vs. 3.0 ± 0.24, P = 0.17) stimuli. It should be noted that the temporal shuffling in the control conditions did not influence the mean movement speed, and hence these kinematics also applied to the left and right moving balls.

Fig. 3.

Quantitative comparison of kinematics measured by mean speed across the 6 different stimulus conditions for left and right hands. Error bars show SE. Since the temporal shuffling applied in the control conditions did not affect the mean speed, these kinematics also applied to the left and right balls.

Control Tests: Visual Motion Localizer

To specifically map motion-sensitive areas, a standard visual motion localizer (Sunaert et al. 1999) was presented to the subjects in two additional time series. The stimuli were circular (7° diameter) random-textured patterns consisting of 50% white dots (4.6 arcmin) on a black background. The subject had to fixate a small red square (0.2°) at the center of the stimulus throughout the entire run. Each localizer run included three conditions corresponding to 1) a motion condition in which the random dots were moving coherently at 6°/s in one of four possible directions (0°, 45°, 90°, and 135°), 2) a static condition in which the random dots remained still, and 3) a fixation condition in which only the red fixation square was present. Within each run, conditions were repeated four times, yielding runs 360 s (24 s × 3 conditions × 5 repetitions) in duration.

Data Collection

Scanning was performed with a 3-T MR scanner (Intera; Philips Medical Systems) located at the University Hospital of Katholieke Universiteit Leuven. Functional images were acquired by gradient-echo planar imaging with the following parameters: 50 horizontal slices (2.5-mm slice thickness: 0.25-mm gap), repetition time (TR) 3 s; time of echo (TE) 30 ms; flip angle 90°; 80 × 80 matrix with 2.5 × 2.5-mm in-plane resolution; and SENSE reduction factor of 2. The 50 slices of a single volume covered the entire brain from the cerebellum to the vertex. A three-dimensional high-resolution T1-weighted image covering the entire brain was acquired in one of the scanning sessions and used for anatomical reference (TE/TR, 4.6/9.7 ms; inversion time 900 ms; slice thickness 1.2 mm; 256 × 256 matrix; 182 coronal slices; SENSE reduction factor 2.5). Because of the number of runs to be collected, data acquisition was split into three fMRI sessions on different days (mean ± SE time difference between scan sessions was 5.8 ± 0.9 days).

Data Analysis

Data analysis was performed with the SPM5 software package (Wellcome Department of Cognitive Neurology, London, UK) running under MATLAB (MathWorks, Natick, MA). The preprocessing steps involved realignment of the images and spatial normalization of all images to a standard stereotaxic space [Montreal Neurological Institute (MNI)] with a voxel size of 2 × 2 × 2 mm.

Whole-brain group analysis.

Before the group analyses, the functional data were smoothed with an isotropic Gaussian kernel of 8 mm. For every participant, the onset and duration of each condition were modeled by a general linear model (GLM). The design matrix was composed of seven regressors modeling the seven conditions (6 movement conditions, i.e., bimanual, UniL, and UniR and their control conditions, plus the fixation condition) plus six regressors obtained from the motion correction during the realignment process. The latter were included to account for variations in voxel intensity due to head movement. All regressors were convolved with the canonical hemodynamic response function. The contrasts of interest at the first level included “action − control” (termed main contrasts) and “action − fixation,” separately for the bimanual and unimanual conditions. A second-level random-effects analysis (1-sample t-test) was then performed on the contrasts obtained from the subjects' first-level analyses. An inclusive mask of “action − fixation,” itself derived from a second-level random-effects analysis and thresholded at P = 0.01 uncorrected, was applied to each of the “action − control” main contrasts (Bi − ContBi, UniL − ContUniL, UniR − ContUniR). Except when contrasts were used for masking, threshold was set at P < 0.05 FDR corrected. To examine interaction effects between the bimanual and unimanual observations, we first calculated the interaction contrasts [(action 1 − control 1) − (action 2 − control 2)] for each subject, where action 1 is the bimanual action condition (Bi) and action 2 one of the unimanual action condition (UniR or UniL), or the reverse. Similar interactions were also computed for the two unimanual action conditions. A second-level random-effects analysis (1-sample t-test) was then performed on the contrasts obtained from the subjects' first-level analyses while applying inclusive masks of (action − control) and (action − fixation), both thresholded at P = 0.01 uncorrected. The threshold for these second-level analyses was also set at P < 0.05 FDR corrected. The resulting activations were projected (enclosing voxel projection) onto the population average, landmark- and surface-based (PALS) atlas surface (Van Essen 2005) with the Caret software package (Van Essen et al. 2001).

ROI analysis.

The whole-brain analysis was followed by a ROI analysis concentrating on 12 ROIs in premotor, parietal, and temporo-occipital cortical regions involved in action observation (action-observation ROIs). The main aim of this analysis was to analyze the MR responses to action observation and control conditions separately, in order to evaluate the predictions of additivity and averaging. The action-observation ROIs were spheres 4 mm in radius, centered on the local maxima of the average action-observation network obtained by averaging the bimanual and two unimanual contrasts (Bi + UniL + UniR − ContBi − ContUniL − ContUniR) at the first, or individual-subject, level. Six parietal ROIs, three in each hemisphere, were defined in this way: anterior intraparietal sulcus (antIPS), near the border of putative human AIP (phAIP), middle medial IPS (mmIPS, falling within dorsal intraparietal sulcus anterior, DIPSA), and posterior medial IPS (pmIPS) near superior parieto-occipital cortex (SPOC) (see, e.g., Cavina-Pratesi et al. 2010). Two occipito-temporal ROIs were positioned at the local maxima within the two activation segments, one extending from hMT/V5+ into the posterior middle temporal gyrus (pMTG) and the other located in the posterior occipito-temporal sulcus (pOTS). These two branches of occipito-temporal action-observation activation are likely to correspond to the upper and lower bank of the superior temporal sulcus (STS) of the monkey (Jastorff and Orban 2009). The ROI in pMTG was positioned slightly forward of the local maximum of the average action-observation network to avoid overlap with hMT/V5+.The two premotor ROIs, one in each hemisphere, complemented the action-observation ROIs.

Four ROIs, defining V1 and hMT/V5+ bilaterally, were derived from the results of the visual motion localizer task using the contrast (moving dots − static dots) and applying a threshold of 0.001 (uncorrected). Given the large sizes of these activation sites, spheres with a 10-mm radius were placed around the coordinates defining the peak activations of V1 and hMT/V5+ in each hemisphere. Given the size of V1 (Dougherty et al. 2003), the V1 ROIs may have included some parts of V2, but this would still qualify as early cortex, distinct from hMT/V5+.

For each ROI in each subject, we then extracted the mean percent MR signal change relative to fixation in each observation condition (Bi, ContBi, UniR, ContUniR, UniL, ContUniL) for each of the six motor acts. The resulting data, averaged over motor acts, represent the average visual activation for a given subject/ROI. These data were then entered into repeated-measures ANOVAs examining the factors Hemisphere, ROI, Type (hand action vs. moving balls control), and Hand (both, right, left). Significant main and interaction effects were further explored by post hoc tests using the Tukey correction.

RESULTS

Frequency of Eye Movements Across Conditions

Eye movement data were available for 16 subjects. Subjects averaged 7 (±3) saccades/min. A repeated-measures ANOVA comparing the number of saccades across experimental conditions showed no significant differences (F_5,75 = 1.7, P = 0.15).

Whole-Brain Analysis: Interaction Between Factors Hand and Type (of stimulus)

Bimanual action-observation activation map.

The random-effects analysis of the contrast bimanual hand action vs. control object motion revealed a network that included primarily parietal and occipito-temporal cortical regions in both hemispheres (Fig. 4). Plotting the activation pattern on the flattened hemispheres showed that the parietal activation sites were located in the vicinity of the DIPSA region (Fig. 5A). The occipito-temporal activation extended forward from hMT/V5+ in both an upper branch crossing the pMTG and a lower branch extending into the pOTS. Little premotor activation was present for bimanual action observation, with the exception of a small site in the right precentral gyrus (arrow in Fig. 5A). A comparison with similar contrasts for unimanual actions (Fig. 5, B and C) revealed that this was also true of the unimanual conditions and indicated that the weakness of premotor activation was not characteristic of the sole bimanual action-observation network.

Fig. 4.

Cortical activations during the observation of bimanual actions: statistical parametric maps (SPMs) of a random-effects analysis (n = 19) plotting voxels more active during observation of the hand actions than for observation of moving control balls projected onto the rendered views of the left and right hemispheres (MNI template). Activated voxels (FDR corrected P < 0.05, masked with an inclusive mask resulting from an equivalent random-effects analysis of “action observation − fixation” at P < 0.01 uncorrected) are colored red to yellow.

Fig. 5.

Cortical activations during the observation of bi- and unimanual actions: SPMs of a random-effects analysis (n = 19) plotting voxels more active during observation of the hand actions than for moving control balls, projected onto flatmaps (PALS atlas) of left and right hemispheres for bimanual (A) and unimanual right (B) and left (C) hand actions. Same significance levels as in Fig. 4. Olive-colored arrows point to activations in the posterior medial intraparietal sulcus (IPS); yellow arrows point to the premotor activations. Orange, green, and blue ellipses, confidence area for local maxima of putative human AIP (phAIP), anterior dorsal IPS (DIPSA), and medial dorsal IPS (DIPSM), respectively; purple dot, local maximum of hMT/V5+. prCS, precentral sulcus; CS, central sulcus; SF, sylvian fissure; STS, superior temporal sulcus. Color and spatial scales are indicated.

Interactions between bimanual and unimanual action observation.

To detect regions significantly more activated, relative to the control conditions, by bimanual than unimanual action observation, we tested the interactions between bimanual and each of the unimanual conditions (see methods). These interactions imply, as a definition of specialization for the visual processing of bimanual action observations, a region that is more active for the observation of bimanual action than the observation of either unimanual action. We used interaction tests, rather than simple subtractions between Bi and UniL or UniR, to ensure that any differences in activation between the observation conditions could not be accounted for by lower-order visual motion factors. The masking of these interaction tests (see methods) ensured that the significant effects were due to the difference between the observation conditions and also that they reflected visual responses. No voxel reached significance in both interactions, when tested for stronger activation in bimanual than either unimanual condition. Three sites were significantly activated in the interaction comparing Bi to UniL, relative to their controls: right mmIPS, left pMTG, and a site in right occipital cortex, near the occipital pole. None of these sites reached significance in the other interaction, comparing Bi to UniR, relative to their controls. In fact, this interaction yielded no significant voxels at all. Even decreasing the threshold to P < 0.05, uncorrected in the interaction, yielded but a single cortical region in left occipital cortex, at a location symmetric with respect to the region found in the other interaction. This absence of an interaction, when comparing Bi to UniR, indicates that many of the differences in z scores reached by a given cortical area in the subtractions listed in Table 1 are small and not significant. Thus the interaction analysis did not provide any indication for a specialized neural substrate for the visual processing of observed bimanual as opposed to unimanual actions.

Table 1.

Anatomical position, MNI coordinates, and t scores of local maxima of the main contrasts of the whole-brain analysis

	Bimanual				Unimanual L				Unimanual R
Brain Region	x	y	z	t	x	y	z	t	x	y	z	t
LH sup. front. gyrus	—	—	—	—	—	—	—	—	−18	34	52	4.12
LH sup. front. gyrus	—	—	—	—	—	—	—	—	−12	56	38	3.53
LH SMA	—	—	—	—	—	—	—	—	−6	4	56	3.90
LH inf. orbitofront. cortex	—	—	—	—	—	—	—	—	−22	12	−16	3.85
RH sup. front. gyrus	—	—	—	—	—	—	—	—	14	−8	74	4.83
RH inf. orbitofront. cortex	—	—	—	—	—	—	—	—	56	38	−8	3.29
RH gyrus rectus	—	—	—	—	—	—	—	—	6	50	−16	3.18
RH sup. medial-front. gyrus	—	—	—	—	—	—	—	—	4	60	36	3.24
LH inf. front. gyrus p. triangularis									−52	34	6	3.92
RH inf. front. gyrus p. triangularis	—	—	—	—	—	—	—	—	58	26	0	3.78
LH precent. gyrus (premotor)	—	—	—	—	—	—	—	—	−38	−4	54	2.81
RH precent. gyrus (premotor)	36	−2	66	4.12	—	—	—	—	—	—	—	—
RH precent. gyrus (motor)				—	—	—	—	—	34	−16	66	3.54
LH paracent. lobule	—	—	—	—	—	—	—	—	−6	−34	60	3.60
RH postcentral gyrus	42	−30	46	3.14	—	—	—	—	—	—	—	—
LH anterior IPS	−38	−40	52	3.46	−40	−40	54	4.61	−38	−40	50	7.23
RH anterior IPS	—	—	—	—	36	−46	56	3.36	32	−42	54	10.41
LH middle medial IPS	−36	−56	62	4.85	−30	−54	60	3.42	—	—	—	—
RH middle medial IPS	32	−52	68	6.36	—	—	—	—	—	—	—	—
LH posterior medial IPS	—	—	—	—	−24	−86	32	3.66	−22	−88	32	4.21
RH posterior medial IPS	—	—	—	—	30	−76	26	3.00	26	−82	36	5.36
LH precuneus	—	—	—	—	—	—	—	—	−12	−60	66	3.33
LH inf. temp. gyrus	—	—	—	—	—	—	—	—	−32	0	−42	4.17
LH sup. temp. pole	—	—	—	—	—	—	—	—	−44	18	−14	2.87
LH sup. temp. pole	—	—	—	—	—	—	—	—	−48	12	−22	4.37
LH supramarginal gyrus	—	—	—	—	—	—	—	—	−52	−38	30	3.76
RH supramarginal gyrus	—	—	—	—	—	—	—	—	66	−18	32	5.47
LH post. middle temp. gyrus	−48	−74	4	8.48	−48	−62	−4	5.57	−50	−74	6	13.01
RH post. middle temp. gyrus	50	−72	0	9.33	48	−72	0	6.57	52	−70	0	14.41
LH occipito-temp. sulcus	−46	−46	−22	7.26	−48	−38	−20	4.55	—	—	—	—
RH occipito-temp. sulcus	42	−52	−20	4.91	42	−50	−14	3.69	42	−50	−20	8.28
RH fusiform gyrus	—	—	—	—	32	−62	−14	4.15	—	—	—	—
LH middle occipital gyrus	−20	−94	6	5.30	—	—	—	—	—	—	—	—
LH middle occipital gyrus				—	−12	−103	6	7.31	—	—	—	—
RH middle occipital gyrus	26	−94	10	4.74	—	—	—	—	—	—	—	—
RH inf. occipital gyrus	—	—	—	—	38	−90	−4	5.24	—	—	—	—
RH inf. occipital gyrus	—	—	—	—	42	−76	−4	6.14	—	—	—	—

MNI, Montreal Neurological Institute; RH, right hemisphere; LH, left hemisphere.

Comparison of unimanual and bimanual action observation maps.

The cortical activation patterns of the contrasts between uni- and bimanual action observation and ball motion control conditions were, in fact, relatively similar: strong activation in occipito-temporal and parietal regions with little premotor activation. Activation of the temporo-occipital areas extended into the pMTG and pOTS in both the bimanual and unimanual observation conditions and was approximately equal in strength in the two hemispheres in both the bimanual and unimanual conditions (Fig. 5). In all conditions, this activation also extended caudally toward the center of early visual areas, reflecting the slightly more central positions of the hands compared with the spheres in all videos (Fig. 2C), despite the use of two fixation positions.

The parietal activations, however, showed a notable difference between bimanual and unimanual conditions regarding hemispheric bias. The strength and extent of parietal activation was bilaterally balanced in the bimanual condition (Fig. 5A, Table 1): t score at peak parietal activation and number of activated voxels measured 4.85 with 316 voxels for the left hemisphere, with a t score of 6.36 and 394 voxels for the right hemisphere. However, parietal peak activations in the unimanual conditions differed in both level and extent in left and right hemispheres (Fig. 5, B and C). The t scores in the hemispheric maxima and the extent of parietal activation for the UniR condition were, respectively, 7.23 and 1,851 voxels for the left hemisphere compared with 10.41 and 2,725 for the right. For UniL the values were t = 4.61 and 611 voxels for the left hemisphere and t = 3.36 and 90 for the right. The difference in the numbers of significant voxels in the two hemispheres reversed between the unimanual conditions. More importantly, the ratio of voxels in the two hemispheres differed between the Bi and both the UniR [χ² (df = 1) = 4.18, P < 0.05] and the UniL [χ² (df = 1) = 248, P < 0.0001] conditions. Using identical t scores to define the number of activated voxels in all three contrasts yielded similar results. For t = 2.87 (P < 0.005 uncorrected) the numbers of voxels in left/right hemispheres were 164/291 for the Bi condition, 837/2,262 for the UniR condition, and 344/49 for the UniL condition; χ² values were 16 (P < 0.0001) and 232 (P < 0.0001) for the differences Bi − UniR and Bi − UniL, respectively. Using t = 2.55 (P < 0.01 uncorrected) yielded extremely similar results (data not shown), indicating that the exact level was not important. Thus, for each of the unimanual conditions, the main parietal activation was seen in the hemisphere contralateral to the observed hand action, as the right (left) hand moved within the left (right) hemifield. In the bimanual conditions parietal activations were more similar in the two hemispheres.

Very little premotor activation was seen in any of the action observation conditions (Fig. 5, yellow arrows). Only two sites showed an action-specific level of activation (Table 1): one site in the left premotor cortex for the UniR condition (80 voxels) and a small focus seen for bimanual observation in the right premotor area (46 voxels).

Thus the whole-brain analysis revealed the classical three-level action-observation network, involving occipito-temporal, parietal, and premotor cortex, although with a weak premotor component. Interaction between factors Type and Hand was observed in the direction of stronger bimanual activation, but only between bimanual and unimanual left and not between bimanual and unilateral right, even at very low thresholds. Thus no cortical region was more activated relative to control conditions in the bimanual compared with both unimanual conditions. In that sense, we did not observe any region specifically devoted to bimanual action observation. One explanation for the absence of such regions could be a lack of power in the measurements. However, this is unlikely, insofar as the interaction between the two unimanual conditions (in the direction UniR > UniL) reached significance in occipito-temporal regions bilaterally and in the right parietal cortex (Fig. 6). The latter activation corresponds to the contralateral bias typical of the parietal regions mentioned above, but also to the overall stronger activation in the right-unimanual compared with the left-unimanual condition. This latter difference in all likelihood reflects the stronger kinematics of right-handed actions over actions of the left hand (Fig. 3).

Fig. 6.

SPM plotting the voxels significant (P < 0.05 FDR corrected) in the interaction right − left unimanual action observation (i.e., both relative to their control condition) in left (A) and right (B) hemispheres. Blue outlines, edge of activation pattern in UniR − control (Fig. 5B). Other conventions as in Fig. 5.

ROI Analysis: Activation Levels Among Hand Conditions

We performed a ROI-based analysis because it is more sensitive than the whole-brain approach, the outcome of which was negative with respect to specificity for bimanual action observation. Furthermore, the ROI analysis allows testing of MR responses to the observation conditions separately from those in the control conditions. This is essential to evaluate the predictions derived from visual studies and testing for additivity of unimanual observation conditions or the averaging of these responses in the bimanual observation condition. If the pattern of observation responses matches one of the predictions, one can compare them with the MR responses to the corresponding control conditions to test the specificity of the bimanual observation effects.

Definition of ROIs.

The six bilateral action-observation ROIs were defined from the average action-observation network obtained by contrasting the average of all three hand conditions with their three control conditions (Fig. 7, C and D). The three parietal ROIs, antIPS, mmIPS, and pmIPS, and the two pairs of occipito-temporal ROIs, pMTG and pOTS, were centered on local maxima at P < 0.001 uncorrected, which corresponded to a t score of 3.61. This score exceeded that (t = 2.33) corresponding to P < 0.05 FDR corrected, as higher thresholds segregated local maxima more effectively. The pMTG ROI was moved rostrally to avoid overlap with the hMT/V5+ ROI (center indicated by green dot in Fig. 7, C and D). Although the antIPS and mmIPS ROIs were centered on local maxima (see Fig. 7), their segregation was not complete, as t scores did not decrease much in the intervening interval (Fig. 7, C and D), but this may reflect the smoothing and averaging over subjects. The premotor pair was centered on the local maxima taken at P < 0.05 uncorrected, given the weakness of the premotor activations (Fig. 7, A and B).

Fig. 7.

Regions of interest (ROIs) in parietal and temporo-occipital cortical areas. Blue circles indicate the positions of the 12 action-observation ROIs in the flatmaps of left and right frontal cortex (A and B) and of left and right occipito-temporo-parietal cortex (C and D). Red to yellow colored voxels (color code in inset) indicate voxels reaching significance in the subtraction average action conditions − average control conditions. The threshold is t= 1.73, P < 0.05 uncorrected in A and B and t = 3.61, P < 0.001 uncorrected in C and D. Green dots, local maximum of hMT/V5+; black marker in the left hemisphere, location of an activation focus in the superior parieto-occipital cortex (SPOC) in the study by Cavina-Pratesi et al. (2010). Coordinates of ROI centers in left and right hemispheres: anterior IPS (antIPS): −38, −38, 52 and 34, −42, 48; middle medial IPS (mmIPS): −36, −48, 64 and 32, −58, 60; posterior medial IPS (pmIPS): −24, −86, 32 and 28, −80, 36; posterior occipito-temporal sulcus (post OTS): −42, −48, −24 and 42, −46, −22; posterior middle temporal gyrus (post MTG): −50, −62, 10 and 54, −54, 6; premotor: −36, 2, 48 and 38, −2, 64.

Figure 8B plots the levels of activation, averaged over actions, compared with fixation for the bimanual (green diamond) and the two unimanual action observation conditions (blue square for UniR and red triangle for UniL) in each of the 12 action-observation ROIs. To visualize the complete processing of visual actions at the cortical level, we also included early areas V1 and hMT/V5+ (Fig. 8A). Figure 8, C and D, plot activation levels in the same ROIs for the control ball conditions with respect to fixation. Thus the MR signals plotted in the various panels of Fig. 8 correspond to visual responses in any given experimental or control condition.

Fig. 8.

Activation levels for the 3 hand conditions in the 16 ROIs. The % MR signal changes relative to fixation baseline in the 12 action-observation ROIs (B and D) and 4 early visual ROIs (A and C) are plotted for observation of the hands (A and B) and the control balls (C and D). Blue squares, UniR; green diamonds, Bi; red triangles, UniL. Error bars show SE across subjects. *P < 0.05, **P < 0.01, ***P < 0.001, ****P < 0.0001. Blue shading indicates interaction Hand-Hemisphere; horizontal lines denote a comparison of left and right ROI; vertical lines between boxes indicate an interaction between Hand and Type, and vertical lines within a box refer to post hoc tests between hand conditions. The colors of the boxes framing the activity levels of a given ROI indicate the cluster to which they belong.

No ROI specialized for bimanual action observation.

In all 12 action-observation ROIs, the activation by the bimanual condition was at best equal to the strongest activation by a unimanual action observation. Indeed, in two ROIs, left pOTS and right pMTG, the activation level for bimanual action observation matched that for the strongest unimanual activation, UniR, in both cases. In all other ROIs the bimanual observation response was actually lower than the strongest response to unimanual action observation. Since the interactions between these conditions, the tests used in the whole-brain analysis, imply that the activities in the experimental conditions differ, these interactions cannot be significant in any of the ROIs. Thus the ROI analysis confirms the results of the whole-brain analysis, which did not reveal any region specific for bimanual action observation.

Since in all 12 ROIs the MR response to the weaker unimanual action observation response was nonzero, in particular in left pOTS and right pMTG, it also follows that the prediction of additivity, derived from multisensory integration studies, was met in none of the 12 action-observation ROIs. We were interested in additivity of responses to hand action observation that could not be explained by lower-order visual factors. Hence we defined the ROIs by contrasting action observation with ball motion control conditions, averaged across the hand conditions.

The other prediction, averaging of responses to the unimanual action observation in the bimanual action observation, seems to be met in some of the 12 ROIs, most notably in the parietal ROIs. However, there was quite a diversity of activation patterns across ROIs, raising the question of the significance of these patterns.

Diversity of ROI activation patterns: a synthesis.

Figure 8, A and B, reveal a diversity of activation patterns for bi- and unimanual across action-observation ROIs. To test whether any of the diverse activation patterns illustrated in Fig. 8, A and B, represented significant differences, either among the ROIs for the action-observation conditions themselves or between the action-observation and control conditions within a given ROI, we conducted a four-way repeated-measures ANOVA for the percent MR signal change of the hand and ball conditions across 16 ROIs. Most importantly, the four-way interaction Hemisphere × ROI × Hand × Type was significant (Table 2). This result shows that the relative activation levels for the three hand-action conditions differed with respect to each other, not only between the ROIs of the two hemispheres but also from the activation levels of the ball control conditions in the respective ROIs. Of course, it can be argued that the four-way interaction is driven by the relative activation levels of the two unimanual action observation conditions. Indeed, restricting the four-way ANOVA to these two experimental conditions and their controls still yielded a significant four-way interaction (F_7,142 = 4.6, P < 10⁻⁴). However, restricting the ANOVA to the bimanual action observation and one of the unimanual action observation conditions and their controls yielded four-way interactions that were close to significant (both F_7,142 = 1.9, P < 0.07). Thus the four-way interaction in the main ANOVA (Table 2) in all likelihood is not just driven by the differences between the unimanual action observation activation levels. Importantly, this four-way interaction indicates that the activation for the different experimental conditions differed across ROIs and hemispheres.

Table 2.

Four-way repeated-measures ANOVA of individual % MR signal changes across 16 ROIs

Effect	df	F	P
Hemisphere	1,142	0.6	0.439
ROI	7,142	23.2	<0.0001
Hand factor	2,284	13.4	<0.00001
Type	1,142	307.9	<0.0001
Hemisphere × ROI	7,142	1.9	0.079
Hemisphere × Hand factor	2,284	193.1	<0.0001
Hemisphere × Type	1,142	0.2	0.697
Hand factor × ROI	14,284	2.9	0.0004
Hand factor × Type	2,284	9.4	0.0001
Type × ROI	7,142	11.9	<0.00001
Hemisphere × Hand factor × ROI	14,284	13.0	<0.00001
Hemisphere × Type × ROI	7,142	2.0	0.062
Hemisphere × Hand factor × Type	2,284	7.0	0.001
Hand factor × Type × ROI	14,284	0.7	0.800
Hemisphere × Hand factor × Type × ROI	14,284	2.7	0.0009

ROI, region of interest.

The next step is to show that this diversity has some structure and in particular that the activity patterns in the parietal action-observation ROIs stand out. To that end, we performed a dimensionality reduction by a cluster analysis (k-means method using Euclidean distances, 4 clusters determined by cross validation) on the normalized percent signal changes for the different conditions (for normalization procedure, see Jastorff et al. 2009). This analysis was performed over 16 data points (8 ROIs separately for action and control conditions) in a six-dimensional space defined by the MR signals associated with each ROI (the 3 hand conditions in the 2 hemispheres). Indeed, the whole-brain analysis clearly indicated the importance of considering for a given cortical area the differences in hand conditions across hemispheres. The four clusters included 1) the action conditions of the V1, antIPS, mmIPS, and pmIPS1 ROIs (red boxes in Fig. 8), 2) the action conditions of hMT/V5+ together with the control conditions of hMT/V5+ and pmIPS (green boxes in Fig. 8), 3) the action conditions of pOTS, pMTG, and premotor ROIs together with the control conditions of the premotor ROI (blue boxes in Fig. 8), and 4) the control conditions of the V1, pOTS, pMTG, antIPS, and mmIPS ROIs (yellow boxes in Fig. 8). Thus the activation pattern for the action conditions in Fig. 8 falls into three clusters of ROIs, one containing the occipito-temporal and premotor nodes of the action-observation network, a second cluster containing the parietal nodes of the action-observation network together with V1, and a third cluster including hMT/V5+. Importantly, this analysis indicates that indeed the parietal ROIs stand out among the action-observation ROIs.

What are the distinctive features leading to this particular clustering?

In the parietal ROIs, the activation level for bimanual action observation was close to the average of the two unimanual conditions in both hemispheres. This corresponds to the more nearly equal parietal activations observed in the whole-brain analysis for bimanual compared with unimanual action observation. However, Fig. 8 reveals another striking feature of the action-related activation patterns in the parietal ROIs that was not apparent in the whole-brain analysis: In the left hemisphere, the level of activation by UniL equals or even exceeds that for the UniR in the right hemisphere, even though the amount of movement in the left unilateral stimuli was much smaller (Fig. 3). Furthermore, in the three parietal ROIs overall activation levels are at least equal in the two hemispheres, and in one of them the overall activation was significantly larger for the left hemisphere compared with the right (antIPS: F_1,18 = 4.9, P < 0.05). Both results indicate that, at the parietal level, action-observation activity is not related to the kinematics of the stimulus, which was clearly stronger for the right hand (Fig. 3), visible in the left visual field and thus activating the right hemisphere. This suggests that activity in the parietal ROIs might be related to the static, postural aspects of the action. Indeed, these postural visual signals are at least as strong for the left, nondominant hand, which is holding the object, as it is for the right, dominant hand manipulating the object. This may explain the systematic reversal in activation levels between hemispheres for the unimanual action observation conditions (UniL activation as high in left hemisphere as UniR in right hemisphere) that was specific for all the parietal ROIs. The parietal ROIs also stand out as the only ones in which the Hand and Type of stimulus factors interact (right antIPS F_2,36 = 4.7, P < 0.02; left pmIPS F_2,36 = 3.5, P < 0.05).

hMT/V5+ activation reflects the kinematics of the stimuli present in the contralateral visual field rather well, as the bimanual and UniR conditions, in which the right hand moved in the left visual field, evoked strong activity in the right hemisphere, while bimanual and UniL conditions, in which the left hand moved in the right visual field (Fig. 2C), drove left hMT/V5+, but at a lower level. Further support for a representation consisting mainly of kinematics in hMT/V5+ comes from the result showing that activations for the control conditions (having identical variations in kinematics) were grouped into the same cluster. Typically, ROIs of this cluster exhibit significantly higher activations for the bimanual action observation condition than for observation of the unimanual action in the ipsilateral visual field (Fig. 8)

Activations of the occipito-temporal and premotor nodes of the action-observation network seem to some degree similar to the activation of hMT/V5+, with stronger activations for the UniR and bimanual conditions compared with the UniL condition in the right hemisphere. However, the difference in activity of these nodes in the right hemisphere was reduced compared with hMT/V5+, with the bimanual activation level lying closer to midway between unimanual activation levels. In addition, the activation levels of these nodes were often significantly (pOTS/action F_1,18 = 5.2, P < 0.04; premotor/action F_1,18 = 4.9, P < 0.05) weaker in the left than in the right hemisphere and nearly equal in strength for the three hand conditions in the left hemisphere. The similarity of activation levels in the left hemisphere distinguishes the members of this cluster from the parietal ROIs, in which these levels were clearly segregated. This difference underlies the interaction between ROI and Hand factors observed in the left hemisphere between the two rostral parietal ROIs and the action-observation ROIs likely to be connected with them (Table 3). Thus these occipito-temporal and premotor ROIs can be described as exhibiting an asymmetric, kinematic-related pattern in the right hemisphere.

Table 3.

Two-way repeated measures ANOVA ROI × Hand for selected ROI pairs

		ROI			Type (Hand/Ball)			ROI × Type
ROI Pair	Hemisphere	df	F	P	df	F	P	df	F	P
pOTS/antIPS (hands)	LH	1,221	22.4	<0.0001	2,442	0.9	0.403	2,442	4.4	0.012
	RH	1,220	121.9	<0.0001	2,440	10.8	<0.0001	2,440	0.1	0.930
pOTS/mmIPS (hands)	LH	1,218	0.8	0.380	2,436	2.5	0.082	2,436	5.2	0.006
	RH	1,212	14.6	0.0002	2,424	11.5	<0.0001	2,424	1.3	0.278
Premotor/antIPS (hands)	LH	1,220	11.0	0.001	2,440	5.4	0.005	2,440	2.6	0.072
	RH	1,214	0.3	0.557	2,428	6.4	0.002	2,428	0.4	0.664
Premotor/mmIPS (hands)	LH	1,217	35.7	<0.0001	2,434	5.3	0.005	2,434	4.0	0.019
	RH	1,206	9.3	0.003	2,412	8.3	0.0003	2,412	0.2	0.832
pOTS/antIPS (control balls)	LH	1,221	26.7	<0.0001	2,442	5.1	0.006	2,442	0.2	0.816
	RH	1,220	95.0	<0.0001	2,440	2.3	0.097	2,440	1.2	0.294
pOTS/mmIPS (control balls)	LH	1,218	3.4	0.065	2,436	10.9	<0.0001	2,436	1.9	0.155
	RH	1,212	16.6	<0.0001	2,424	2.7	0.067	2,424	2.1	0.128
Premotor/antIPS (control balls)	LH	1,220	0.02	0.887	2,440	0.6	0.528	2,440	2.4	0.096
	RH	1,214	0.8	0.364	2,428	0.1	0.861	2,428	1.7	0.193
Premotor/mmIPS (control balls)	LH	1,217	9.6	0.002	2,434	5.2	0.006	2,434	8.1	0.0003
	RH	1,206	2.9	0.091	2,412	0.5	0.601	2,412	2.3	0.103

Values are 2-way repeated-measures ANOVA ROI × Hand for selected ROI pairs (based on likely connectivity, see discussion). pOTS, posterior occipito-temporal sulcus; antIPS, anterior intraparietal sulcus; mmIPS, middle medial intraparietal sulcus.

Thus we have observed several patterns in the activation levels elicited by the bimanual and unimanual action observation conditions in the action-observation ROIs, but none identifies any region specialized for processing bimanual actions. The ROI analysis, however, singled out the parietal regions, in which the hemispheric asymmetry strongly depended on the hand conditions: dominance by the hemisphere contralateral to the acting hand in the unimanual conditions but hemispheric balance in the bimanual conditions. This distinction in the degree of interhemispheric balance was also evident in the whole-brain analysis, but the ROI-based analysis clarified the differences with the other levels of the action-observation network, which belonged to different clusters of activity patterns across hand conditions and hemispheres.

DISCUSSION

The present study has clearly given a negative answer to the first prediction made in the introduction, the conjecture regarding the existence of a specific bimanual action region. The study also dismissed the prediction of additivity of the unimanual observation responses in bimanual action observation. It did, however, provide some partial support for the last prediction, averaging of the unimanual action observation responses in bimanual action observation. As a consequence, the common manipulative action-observation network appeared organized differently under bi- and unimanual conditions. This difference was reflected in dissimilar interhemispheric balances for visual action-observation activations, particularly in parietal cortex. Before discussing these two main results, we shall briefly comment on the characteristics of the unimanual action-observation networks revealed by the present study.

The Unimanual Action-Observation Networks

Observation of unimanual manipulative actions activated a cortical network including occipito-temporal, parietal, and premotor regions, in agreement with earlier studies (Buccino et al. 2001; Jastorff et al. 2010; Newman-Norlund et al. 2010; Turella et al. 2009; see Caspers et al. 2010 and Grosbras et al. 2012 for review). The stronger activation for right than left unimanual action observation likely reflects the differences between the kinematics of the right and left unimanual actions (Fig. 3). Since distances traveled and mean speeds of the movements were confounded, it is unclear which of these parameters determined the level of cortical activation.

Overall, the unimanual action-observation networks were similar to our previous observations (Jastorff et al. 2010, 2011; Peeters et al. 2009), except for a weaker premotor activation. Importantly, fMRI studies in the monkey (Nelissen et al. 2005) indicate that premotor activations differ depending on whether the actor performing the grasping is visible in the video or not. Many studies reviewed by Caspers et al. (2010) and Grosbras et al. (2012) included studies showing just hands moving. In the present study the actor was visible, as in Jastorff et al. (2010, 2011). The lesser premotor activation, however, may reflect differences in control conditions, although the moving ball control was directly inspired by the moving car control of Jastorff et al. (2010). In that study a small object (the car) was present in the video, which in the control condition was animated with the motion of the hand, extracted from the action videos, exactly as done here. Casile et al. (2010) have shown that the biological nature of the kinematics is reflected in premotor activity. Hence it is conceivable that in the control conditions the movement of the spheres activated premotor cortex more strongly than that of the car, despite the temporal shuffling. Indeed, the balls were animated with exactly the same speed variations as those of the hands, save for the coarse temporal shuffling. In Jastorff et al. (2010), because the hand movement was minimal, and mainly limited to a vertical or horizontal translational component, the car movement resembled a translation. In the present study the movement of the balls clearly differed from a translation and remained reminiscent of the hand kinematics. The effect of the control condition is consistent with weak premotor activation in both uni- and bimanual conditions (Fig. 8). It is unlikely that other factors, such as the viewpoint or the absence of a task involving the visual stimuli (e.g., a one-back task), explain the weak premotor activation. While most of our previous studies used lateral views (Jastorff et al. 2010; Peeters et al. 2009), another of our studies (Jastorff et al. 2011) used front views as here, and none of them used a task, yet premotor activations were observed in all of our studies. Furthermore, while addition of a task may enhance prefrontal activation by visual stimuli (e.g., Jastorff and Orban 2009), addition of a task modifies the bottom-up visual processing (Dehaene and Cohen 2011), complicating the interpretation of results.

The parietal regions involved in manipulative hand observation revealed in the present study included both lateral (antIPS and mmIPS) and medial (pmIPS) parietal regions, in keeping with the predominance of finger and wrist components in the actions presented. This suggests that just as there are lateral and medial dorsal stream regions involved in the visual control of finger/hand and wrist/lower arm motor acts, respectively (Cavina-Pratesi et al. 2010; Rizzolatti and Matelli 2003; Tunik et al. 2007), similar lateral and medial parietal regions may be involved in the observation of these types of acts.

The similarity of the right unimanual action-observation network to our recent results for observation of typical right-handed manipulative actions (Abdollahi et al. 2012; Jastorff et al. 2010) suggests that the unimanual actions of the present study were fairly natural, probably because of the use of the three-pronged clamp to stabilize the objects manipulated. It is thus unlikely that the unimanual actions attracted attention in any unusual manner, which could have explained the absence of any bimanual action-observation activation that exceeded the unimanual activation.

Same Network for Bi- and Unimanual Action Observation

Neither the whole-brain nor the ROI-based analysis supported the existence of a cortical region specifically devoted to the observation of bimanual actions. The use of two different analysis strategies decreases the likelihood that this negative result reflects a lack of power in our experiments. The significance of the interaction between right and left unimanual actions, as well as the many significant effects observed in the ROI analysis (Table 2), clearly demonstrate that our experiments possessed sufficient power. A lack of power is also unlikely given the large number of subjects (n = 19) and number of runs/subject (n = 24) collected in three sessions. Although it remains difficult to interpret negative results obtained in functional imaging, our findings are in line with those of Schubotz and von Cramon (2009), who reported similar activation for observing bimanual actions as those usually reported for observing unimanual actions. The negative result is unlikely to be due to the weakness of the premotor activations, because in the ROI-based analysis the activity in the different ROIs was studied separately for the experimental and control conditions. Furthermore, despite the weak activation level, the difference in activity between left and right premotor cortex was significant (Fig. 8).

The present result only partially matches those of bimanual motor control studies. Those investigations involved arbitrary manual coordination (Debaere et al. 2003; Wenderoth et al. 2005a, 2005b), rhythmic finger tapping (Immisch et al. 2001; Jancke et al. 2000), or exerting forces on a tool to control a cursor (Johansson et al. 2006), but not manipulation of natural objects. The last study revealed a stronger activation in frontal cortex, contralateral to the dominant hand, that switched hemispheres according to the hand used. In the bimanual coordination studies, anterior cingulate and precuneus were more active during bimanual tasks than in unimanual control tasks (Wenderoth et al. 2005a). Thus these motor studies revealed differences in activation levels between uni- and bimanual conditions not observed here for action observation. However, a recent study (Abdollahi et al. 2012) may help us understand this apparent discrepancy. It should be noted that manipulative actions are only one type of bimanual action; climbing actions are another example, in which both hands actually move more and do so independently, as in the coordination study of Wenderoth et al. (2005a). Interestingly, observation of climbing actions activated parietal and premotor regions close to those reported by Wenderoth et al. (2005a). The fact that observations of both unimanual and bimanual manipulative actions activate similar regions in parietal cortex fits with current views that parietal cortex is organized according to type of actions, not effector (Heed et al. 2011; Jastorff et al. 2010). Indeed, in the present study actions were of the same type, manipulation, while effectors differed in number.

The activation by action observation was asymmetric at the premotor level, with a stronger activation for the right premotor cortex than the left, both in observation of unimanual right actions and in observation of bimanual actions. The premotor asymmetry is in line with the findings that for bimanual actions, double-pulse transcranial magnetic stimulation (TMS) of the premotor cortex in the nondominant hemisphere had a more disruptive effect than TMS of the dominant premotor cortex (Van den Berg et al. 2010). These effects were attributed to suppressive effects that the premotor cortex in the nondominant hemisphere exerts on the dominant motor cortex (M1) to uncouple the tunings of cortical motor neurons for ipsi- and contralateral movements (Rokni et al. 2003). This combination of excitatory and suppressive effects by premotor cortex during the planning of bimanual actions makes it difficult not only to relate simple population activities, as measured with fMRI, to underlying neuronal activities but even to relate activation during observation of bimanual actions to activations during execution of bimanual actions if they require mainly force coordination (Diedrichsen et al. 2012; Johansson et al. 2006).

Differences in the Common Uni- and Bimanual Action-Observation Network Under Bi- and Unimanual Conditions

The most conspicuous difference between the bimanual and unimanual versions of the hand action-observation networks is the greater degree of hemispheric balance under bimanual conditions, apparent in the voxel- and ROI-based analyses of activity levels. In the voxel-based analysis the number of activated voxels was more equal across hemispheres in the bimanual condition than in the unimanual conditions. In the ROI analysis the four-way (Type × Hand × Hemisphere × ROI) interaction was significant. Reducing the four-way ANOVA to the bimanual condition and one of the unimanual conditions yielded marginally significant four-way interactions. Since these two interactions were independent, their combination strongly suggests that the four-way interaction in the complete ANOVA does not just reflect the differences between the unimanual conditions. These analyses indicate that although uni- and bimanual action observation largely share the same cortical network, this network functions differently under bi- and unimanual conditions.

The increased balance between activation levels in the two hemispheres under bimanual conditions was most robust in the parietal cortex, much more so than at the two other levels of the action-observation network. Indeed, at the parietal level of the network the averaging prediction for bimanual action observation was borne out. Hence the response to bimanual action observation was lower than the stronger of the two responses to unimanual action observation. The two parietal ROIs antIPS and mmIPS overlap with phAIP and DIPSA (Fig. 5) and are therefore likely to be the homologs of monkey AIP (Durand et al. 2009). Unfortunately, few single-cell studies have been performed in AIP, and limited information is available about RF size in this area (Romero et al. 2012). This latter study shows that the RFs near the fixation point are ∼5–10° in diameter. Thus it is plausible that in the present study the images of the two hands frequently impinged together upon the RFs of individual neurons.

Although it is plausible that averaging of responses to stimuli impinging upon single RFs operates in the antIPS and mmIPS regions, the pattern of activation across hand conditions at all levels of the action-observation network and its afferent stages cannot be explained by this sole averaging at the neuronal level. Indeed, V1 is likely to house neurons with small RFs, as in the monkey, yet it shows a pattern similar to parietal ROIs. In contrast, both hMT/V5+ and occipito-temporal action-observation ROIs in all likelihood house neurons with large RFs, probably larger in size than those in parietal ROIs, yet the activity pattern in these ROIs is different from that of parietal ROIs. A possible solution is the presence of antagonistic surrounds, which have been documented even in V1 (Cavanaugh et al. 2002; Shushruth et al. 2009; Sillito et al. 1995) and which by definition may extend much further into the ipsilateral visual field than the excitatory RF (ERF). In the bimanual action observation the hand moving in the ipsilateral visual field would suppress the activity evoked by the hand moving in the contralateral visual field. Given their size, the antagonistic surrounds of V1 (Shushruth et al. 2009) can extend as far as 2–3° into the ipsilateral visual field, and could thus indeed be triggered by the moving hands in the action conditions but not the moving balls in the control conditions (see trajectories in Fig. 2C). Assuming similar values for human V1 neurons, this is consistent with the difference in activity patterns between the action and control conditions of the V1 ROI (Fig. 8, A and C).

In addition, one would have to assume that the ERFs of premotor and occipito-temporal cortex have substantial extensions into the ipsilateral visual field, while the majority of parietal RFs would have antagonistic regions in the ipsilateral visual field. In fact, it is well documented that the ERFs of neurons in MST or FST extend considerably into the ipsilateral visual field (Desimone and Ungerleider 1986; Raiguel et al. 1997), a finding that has a counterpart in human imaging (Huk et al. 2002). In addition, the Romero et al. (2012) study and studies of RFs in neighboring LIP (Ben Hamed et al. 2001) indicate that the RFs in antIPS of the monkey are smaller in size and extend less into the ipsilateral field than the RFs of monkey STS areas, such as MST or FST (Desimone and Ungerleider 1986; Raiguel et al. 1997). On the other hand, the only study documenting antagonistic surrounds in a parietal area, LIP (Falkner et al. 2010), reports that while the peaks of the ERFs in this area are clearly restricted to the contralateral visual field, the most sensitive points of the surrounds are located almost equally frequently in ipsi- as in contralateral visual field. Furthermore, unlike surrounds in MST and FST, stimulating the suppressive surround also acts on the spontaneous activity in LIP, predicting a stronger effect in the fMRI. No study has examined the antagonistic surrounds in monkey AIP, but the Romero et al. (2012) study indicates that such surrounds do occur in this area. Some of the inhibitory effects evoked in antIPS neurons by stimulation of the ipsilateral visual field might rely on inhibitory effects transiting the corpus callosum, which have been described for the neighboring, somato-sensory parietal cortex (Reed et al. 2011).

Differences Between the Parietal and Premotor Levels of the Action-Observation Network

Finally, it is worth pointing out that the present study revealed further differences between the premotor and parietal levels of the action-observation network. Indeed, the activation was biased toward the hand present in the contralateral hemifield in the parietal cortex, but not in premotor cortex. The complete reversal of the activation pattern with the hand appearing in the contralateral visual field fits with earlier findings stressing the contralateral bias reported for visual action activation in parietal cortex (Jastorff et al. 2010; Pelphrey et al. 2004; Shmuelof and Zohary 2006).

Considering the complete patterns of activation for all three hand conditions, the parietal pattern differed from those in occipito-temporal and premotor cortex, as shown by the cluster analysis. The occipito-temporal and premotor ROIs could be described as exhibiting an asymmetric, kinematics-related pattern in the right hemisphere. Both the involvement with kinematics and the rightward bias of some of these areas have been reported previously (Casile et al. 2010; Grossman and Blake 2002; Jastorff et al. 2010; Jastorff and Orban 2009). In contrast, the parietal ROIs' action-observation activity was not related to the kinematics of the stimulus. This difference becomes even more interesting in view of the fact that, anatomically, these levels are probably directly linked. Given the three following homologies: 1) of DIPSA/phAIP, which overlapped the antIPS and mmIPS ROIs, with monkey AIP (Durand et al. 2009), 2) of pMTG and pOTS with upper and lower banks of STS, respectively (Jastorff et al. 2012), and 3) between the ventral premotor cortices of both species (Rizzolatti and Craighero 2004) and the anatomical connection linking these action-observation regions in the monkey (Borra et al. 2008; Nelissen et al. 2011), it is likely that in humans antIPS/mmIPS relays action information from pOTS to ventral premotor cortex. The differences in the respective activation patterns of these two interconnected regions in response to the different hand conditions, documented by the cluster analysis and Table 3, are reminiscent of the changes in functional properties of interconnected areas in the equivalent network in the monkey (Nelissen et al. 2011).

The activation pattern in the parietal cortex indicates that activity at this level of the action-observation network does not at all reflect the kinematics of the observed action. The strong activation by the observation of actions by the nondominant, stabilizing hand and the similarity between the activation patterns in parietal ROIs and V1 suggest that the parietal ROIs are involved in processing the static, postural aspects of the action. Indeed, V1 is known to respond well to static stimuli (Tootell et al. 1995). Figure 8 further showed that the activation pattern across hand conditions was clearly different in the hand observation and control responses of the parietal ROIs, as documented by the ANOVA and the cluster analysis. These visual postural aspects of the action, once processed, can then be compared in the same regions to the visual shape of the object, the target of the action, which is known to be processed in antIPS (Murata et al. 2000; Orban 2011; Romero et al. 2012; Sakata et al. 1998). Such a comparison can provide a rich source of information about the intention of the actor as well as his level of skill. This view is consistent with the suggestion (Fogassi et al. 2005) that parietal levels are involved in extracting the intentions of the actor and the goal of the observed action.

The premotor cortex belongs to the category of asymmetric kinematics-related action-observation ROIs, together with the occipito-temporal ROIs. This fits with increasing evidence that the premotor cortex is primarily concerned with the effector used in the action observed and their kinematics. Indeed, Jastorff et al. (2010) reported that activation in parietal and premotor cortex relates to the type of action observed and the effector used, respectively. Casile et al. (2010) have shown that the premotor visual activation reflects not only the effector used but also whether its kinematics matches that of biological movements. The weakness of the premotor activations, however, calls for caution when drawing conclusions about premotor function from the present experiments, although the difference between premotor and parietal levels was statistically robust.

Conclusions

Observation of bi- and unimanual manipulative actions involves the same occipito-temporal, parietal, and premotor network. However, it is organized differently under these two conditions, with the more balanced activation occurring in the bimanual condition, especially at parietal level, reflecting largely the stimulation of two versus a single hemifield at a time. Further work is needed, however, to extend these observations to bimanual actions in which both hands display similar levels of movement and to provide further evidence for different roles of parietal and premotor stages of the action-observation network.

GRANTS

This study was supported by grant IAP 6/29 to G. A. Orban and grants from the research fund of KU Leuven (OT/11/071) and FWO Flanders (G.0483.10 to S. P. Swinnen and G.0730.09 to G. A. Orban).

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the author(s).

AUTHOR CONTRIBUTIONS

Author contributions: M.H.H., M.J.M., and J.J. analyzed data; M.H.H., M.J.M., J.J., S.P.S., and G.A.O. interpreted results of experiments; M.H.H. and M.J.M. prepared figures; M.H.H., M.J.M., J.J., S.P.S., and G.A.O. drafted manuscript; M.H.H., M.J.M., J.J., S.P.S., and G.A.O. edited and revised manuscript; M.H.H., M.J.M., S.P.S., and G.A.O. approved final version of manuscript; M.J.M., S.P.S., and G.A.O. conception and design of research; M.J.M. performed experiments.