Object visibility alters the relative contribution of ventral visual stream and mirror neuron system to goal anticipation during action observation

Abstract

We used fMRI to study the effect of hiding the target of a grasping action on the cerebral activity of an observer whose task was to anticipate the size of the object being grasped. Activity in the putative mirror neuron system (pMNS) was higher when the target was concealed from the view of the observer and anticipating the size of the object being grasped requested paying attention to the hand kinematics. In contrast, activity in ventral visual areas outside the pMNS increased when the target was fully visible, and the performance improved in this condition. A repetition suppression analysis demonstrated that in full view, the size of the object being grasped by the actor was encoded in the ventral visual stream. Dynamic causal modelling showed that monitoring a grasping action increased the coupling between the parietal and ventral premotor nodes of the pMNS. The modulation of the functional connectivity between these nodes was correlated with the subject’s capability to detect the size of hidden objects. In full view, synaptic activity increased within the ventral visual stream, and the connectivity with the pMNS was diminished. The re-enactment of observed actions in the pMNS is crucial when interpreting others’ actions requires paying attention to the body kinematics. However, when the context permits, visual-spatial information processing may complement pMNS computations for improved action anticipation accuracy.

1. Introduction

Mirror neurons in the ventral premotor cortex and inferior parietal lobe of the monkey fire when it performs an action, and when it sees or hears similar actions performed by others (di Pellegrino et al., 1992; Fogassi et al., 2005; Fujii et al., 2008; Gallese et al., 1996; Keysers et al., 2003; Kohler et al., 2002; Kraskov et al., 2009; Rozzi et al., 2008; Umiltà et al., 2001). In humans, the presence of cells with mirroring properties has been confirmed with intra-cranial recording (Mukamel et al., 2010). The existence of a human mirror neuron system is further supported by the finding that the increases in BOLD signal recorded during action observation and execution overlap in a variety of brain regions, including in particular the ventral premotor cortex (areas BA6 and BA44), and several parietal regions along the postcentral and intraparietal sulci (areas BA2, PF/PFt, hIP1-3, BA7) (Gazzola and Keysers, 2009; Iacoboni et al., 1999; Shmuelof and Zohary, 2006). Moreover, it was found that the neuronal activity in these parietal and premotor regions is attenuated when observing an action that was recently executed relative to one that was not (Chong et al., 2008; Kilner et al., 2009), which suggests that the same neuronal ensembles are recruited during both action observation and execution (but see Lingnau et al., 2009). Collectively, these regions have thus been termed the putative mirror neuron system (pMNS), where putative refers to the fact that, in humans, mirror neurons have not been recorded from all of these regions (Keysers, 2009). The mirror neuron discovery has offered a biological explanation for many findings suggesting that action perception and execution share a common coding system (Hommel et al., 2001), like for instance the fact that action execution is perturbed by the simultaneous perception of an incongruent movement (Kilner et al., 2003).

The functional role of parietal and premotor activations during action perception is however still hotly debated (Dinstein et al., 2008; Goodale, 2005; Hickok, 2009; Keysers, 2009; Mahon and Caramazza, 2005; Rizzolatti and Craighero, 2004; Rizzolatti and Sinigaglia, 2008; Turella et al., 2009a), and very little is known about the exchanges of information between the pMNS and other brain regions contributing to action perception, like the extrastriate body area and the posterior superior temporal sulcus involved in the perception of body shape and biological motion (Beauchamp et al., 2002, 2003; Bonda et al., 1996; Downing et al., 2001, 2006; Grossman and Blake, 2002; Grossman et al., 2000; Jastorff and Orban, 2009; Peelen et al., 2006; Pelphrey et al., 2003), or other visual areas carrying information about the environment in which the action is taking place. The notion that the re-activation of parietal and premotor cortices necessary for action execution contributes to the perception of some aspects of the actions of others is supported by studies showing impairments in action perception following neurological conditions that affect regions of the pMNS (Fazio et al., 2009; Kalénine et al., 2010; Pazzaglia et al., 2008a, 2008b). It is also supported by the results of a transcranial magnetic stimulation study in which participants had to judge the weight of a box lifted by an actor (Pobric and Hamilton, 2006). Virtual lesions to the vPM impaired weight estimation in this condition but did not influence the participants’ capacity to estimate the weight of a bouncing ball, suggesting that the pMNS participate to perception most, when the source of information is the kinematics of a human body. This notion is consistent with recent findings showing that humans can successfully derive the intentions of others from an analysis of their arm movement, even if only the initial part of the action is shown (Becchio et al., 2012; Sartori et al., 2011). Undoubtedly however, the pMNS is not the only process contributing to action perception, and a great deal of it can be accomplished satisfactorily without re-enacting other’s actions within the pMNS. As a matter of fact, patients with apraxia show severe problems in motor planning (Rumiati et al., 2001) or motor execution (Rapcsak et al., 1995), but although some apraxic patients show deficits in the perception of the actions of others (Pazzaglia et al., 2008a, 2008b) others do not (Rapcsak et al., 1995; Rumiati et al., 2001; for a review Mahon and Caramazza, 2005). Furthermore, the reverse dissociation has been documented as well, with patients being able to execute the actions that they could not recognize when seeing them done by someone else (Cubelli et al., 2000; Rothi et al., 1986). In the tasks used with these neurological patients, action perception is typically measured by requiring the patients to arrange sets of pictures depicting everyday actions (like preparing tea) in the correct sequence, to discriminate between correctly and incorrectly executed actions, or to view an action and choose the word that best describes the action (e.g. hammering) or the image of the tool most associated with the action (e.g. a hammer). In the light of these findings, the most balanced account of the existing literature is that the pMNS provides just a part of the information that constitutes the multifaceted perception of others’ actions. The challenge is therefore to determine (i) to what facets of perception the pMNS contribute to, (ii) how that compares to the contribution of other regions, and (iii) how these regions communicate with each other to solve particular tasks. Accordingly, here we explore the hypothesis, fuelled by the general properties of the regions composing the pMNS (Keysers, 2011) and the abovementioned TMS and lesion studies (Fazio et al., 2009; Kalénine et al., 2010; Pazzaglia et al., 2008a, 2008b; Pobric and Hamilton, 2006; Rapcsak et al., 1995; Rumiati et al., 2001), that pMNS computations are most important when the kinematics of the human agent are the only source of information to determine the goal of the action. In contrast, pMNS computations become less important when other, non-biological cues are also informative.

To test this hypothesis, we harness the observation that in monkeys passively watching reach-to-grasp actions, about half of the mirror neurons also respond to the sight of reaching to grasp when the final part of the action, the grasp itself, is concealed from view (Kraskov et al., 2009; Umiltà et al., 2001). This opens the possibility to show only the first part of a reach-to-grasp action and yet preserve activations in at least some mirror neurons. We can then test the effect of hiding the target object on the parietal and pre-motor activations of participants requested to anticipate the size of the object being grasped by the actor. Specifically, we asked participants to watch blocks of 6 videos of an actor reaching to grasp one of two balls and to report how often the actor reached for the larger ball (Fig. 1A). In one condition, the two balls were visible in each movie (Full-View), in the other they were both concealed inside a box (Occlusion, Fig. 1B). In both cases, the movies were stopped 120–200ms before the hand contacted the object (Fig. 1B, rightmost frame of each movie), and this in order to keep the visual information regarding the kinematics of the actor similar across conditions, while manipulating how much non-kinematic information was present: only in the Full-View condition, analysing the shape of the ball closest to the hand would help solve the task. To explore the effect of attending to a grasping action on connectivity, we included a visual control condition in which an object was sometimes touched but no grasping ever occurred. To constrain whether brain activations could be considered as belonging to the pMNS, we also included an action execution condition in which participants had to grasp balls in the scanner to approximate the actions of the actor. Hence, if our hypothesis is true and pMNS computations are most important when the only available information to interpret the action is in the body kinematics, pMNS activations should be stronger in the Occlusion than the Full-View condition, since in this condition there is no other source of information available to infer what ball is being grasped by the actor. In contrast, activity in brain regions of the ventral temporal cortex involved in processing object shape should be stronger in the Full-View condition and the performance should improve in this condition, if the participants actually take advantage of the additional information provided by the sight of the target object on the table.

Figure 1. Experimental design and movie stimuli.

(A) Design of an observation block including a warning sign, an instruction screen, a 12-s movie composed of 6 videos, a response screen with 2 possible answers, and a resting period. There were 12 such blocks per observation session, and 3 consecutive sessions for a total number of 72 volumes per condition (1 volume per TR = 2.0 s). (B) Illustration of the movie stimuli. The 6 videos composing a movie were of variable length (between 0.8 and 2.9 s). The length of the 6 videos was pair-matched across the 3 conditions for every actor. In the Full-View and the Occlusion conditions, the videos stopped 3-5 frames (120-200ms) before the hand touched the ball. The outcome of the action had therefore to be anticipated in both conditions, and the main difference was whether the target of the grasping action was visible or not.

2. Methods

Participants

Seventeen young adults (9 females and 8 males between 18 and 34 years of age; mean 22.4 ± 4.0) participated in the main experiment. Sixteen male adults (22 to 57 years of age; mean 34.1 ± 8.7) took part in the control experiment for task difficulty (see below). All were right handed according to the Edinburgh handedness inventory, and had normal or corrected-to-normal vision. None had a history of neurological or psychiatric disorder. The study was approved by the local ethical committee, and volunteers gave written informed consent to participate.

MRI-scanning

The study was carried out on a Philips 3T Intera MR-scanner. The main experiment was performed with a synergy SENSE 8-channel head coil, the control experiment with a 32-channel head coil. Functional images were acquired parallel to the bi-commissural plane using a T2*-weighted single-shot echo-planar sequence with TR = 2.0 s, TE = 28 ms, and 39 axial slices 3.5 mm thick (with no gap) covering the whole brain (flip angle 70 degrees, FOV 224 x 136.5 x 224 mm, Reconstructed Matrix size 64 x 64, isometric 3.5 mm voxels). A T1-weighted anatomical image consisting of 160 slices 1mm-thick was acquired at the beginning of the experiment for normalization purpose, and to visualize the activity at the individual level (TR 7.6 ms, TE 3.5 ms, flip angle 8 degrees, FOV 224 x 160 x 256 mm, Reconstructed Matrix size 256 x 256, isometric 1 mm voxels).

Movie stimuli (main experiment)

Twelve actors were videotaped in 3 different conditions: Full-View, Occlusion, and Touch. The actors (6 females and 6 males, between 6 and 55 years of age) sat behind a table and only the torso and the hands were visible on the video (examples on Fig. 1). Viewing only part of the body has no influence on the human pMNS activity (Turella et al., 2009b), and this was used in our movies to ensure that the subjects would pay attention to the relevant part of the stimuli. In the Full-View condition, the actors were instructed to grasp a big or a small ball on the table (10 versus 3.5 cm diameter). In the Occlusion condition, the balls were placed inside a box and the actors were again requested to grasp one of the two balls. The balls in the box were placed on a stand in order to force the actors to shape their hand before the hand was hidden. In both the Full-View and the Occlusion condition, the position of the balls was randomly assigned, with the big ball on the left side half of the time. In the Touch condition, the actors were asked to move their hand freely above the box, without obvious purpose. On some occasions the actors touched the box with their hand. The Touch condition was included to investigate what if anything was specific to the observation of a grasping action, and no any grasping movement was displayed in this condition. For each condition and actor, six short movies were selected. The blocks for the fMRI experiment were then constructed by pasting the six videos of the same condition and actor together, with a red cross separating the videos within a block. The same video was never used more than once. For the grasping actions, the videos always stopped 3 to 5 frames (120 to 200 ms) before the ball was touched, and the outcome of the grasping action had therefore to be anticipated in both the Full-View and the Occlusion conditions. For each actor, the same numbers of frames were removed at the end of the grasping actions in the two conditions, and this to ensure that on the last frame, the hand aperture was identically shaped. The hand aperture of two different actors grasping a big and a small ball is exemplified in Supplementary Figure S1. The number of frames of the 6 videos was paired-matched across the 3 conditions for each actor. The average length of a video representing a single action was about 1400 ms (35.8 frames, Min 20 and Max 72 frames, with a sampling rate of 25 frames/s). The duration of the cross between the videos was set to reach a constant length of 12s for every block.

Tasks (main experiment)

Participants were requested to answer a question about every block. In the Full-View and the Occlusion conditions, they were asked to estimate how many times (out of six) the actor was going to pick the big ball. In the touch condition, participants had to estimate how many times (out of six) the actor touched the box. Responses were given on a response box by selecting one of two numbers presented on the screen at the end of the block. We opted for a two-alternative forced choice in order to be able to randomise the side of the correct response across trial hence preventing the participants to prepare their response before the end of the movie. The target number appeared on the right side of the screen in half of the cases, and was the largest number in approximately half of the cases. A short question (“Big Ball?” or “Touching Box?”) was displayed for 2.5 s before every block. The question was preceded by a warning sign presented for 0.5 s. The response period was predetermined to 4 s, and was followed by a rest period of 10 s. The total duration of one block was 29 s, including 12 s of action observation (Fig. 1A). The 12 blocks times 3 conditions were pseudo-randomly split over 3 runs, ensuring that each run contained exactly 4 blocks of each condition, that the same actor never appeared in two blocks of the same run, and that the same condition never appeared twice in a row. Each run lasted for 6 minutes during which 185 volumes were acquired. Stimuli were back-projected on a tinted screen positioned at the end of the tunnel, and sustained a visual angle of about 15 degrees. The subject’s responses were recorded by the computer used to display the stimuli. The computer also recorded the occurrence of the pulses sent by the scanner at the beginning of each new volume acquisition. Before to start the experiment, the participants were shown the two balls grasped by the actors in the movies as well as the box. Since in the Occlusion condition, the participants could never see the balls hidden inside the box, and mirror neurons do not fire reliably in front of a pantomime action (Kraskov et al., 2009; Umiltà et al., 2001), the participants were explicitly told at the beginning of the experiment that the two balls were actually inside the box and that the actors were indeed grasping one of the balls. In humans, knowledge of the presence/absence of the object has been shown to affect the response of the somatosensory component of the pMNS (Turella et al., 2012).

During one additional scanning session at the end of the experiment, participants were requested to grasp a ball on the left or on the right side of their body upon hearing a low (340 Hz) or a high tone (440 Hz) respectively. The tones were presented binaurally for 500 ms through magnetic compatible headphones. The two balls (10 cm diameter) were placed at the level of the hips. Only one ball size was used because the participants lying in the scanner could not see the balls next to the scanner bed, and including a small ball next to the big one in the scanner would have increased significantly the time spent in tactile exploration. The participants were requested to reach and grasp the ball, hold it for one second, put it back, and come back in a resting position with the hands on their stomach. The inter-stimuli interval was fixed to 18 s, and the experimenter manually pressed a button on the response box as soon as the subject was back in the resting position in order to later determine the actual length of the execution period. There were 15 grasping actions to be performed on each side. The side of grasp was randomized, with no more than 3 consecutive grasps on the same side, and about half of the participants grasping first on the right side. The duration of the execution session was 9 min 30 s during which 285 volumes were acquired. The total duration of the experiment was approximately 45 minutes.

Data pre-processing (main experiment)

Data were analysed with SPM5 (Wellcome Department of Cognitive Neurology, London, UK; http://www.fil.ion.ucl.ac.uk/spm) implemented in MatLab 7.6 (Mathworks Inc., Sherborn, MA). Functional images from the 3 action observation sessions were realigned to the first volume of the first session, and co-registered to the anatomical scan. For the execution task, since participants were more likely to move, and these movements were likely to be correlated with the task, images were also unwrapped. Then, the anatomy was segmented, and functional images were normalized to the MNI standard space using the segmentation parameters (with a final voxel size of 3x3x3 mm and a 7^th-degree B-spline interpolation). Finally, data were smoothed with an 8x8x8 mm FWHM Gaussian filter. Inspection of the realignment parameters for every subject indicated that no head movement created a displacement exceeding the voxel size. The maximum displacement cumulated over an observation session never exceeded 1.9 mm and 2.6 degrees for the translation and the rotation parameters respectively. The maximum displacement cumulated over an execution session never exceeded 2.4 mm and 2.6 degrees.

Statistical parametric mapping (main experiment)

At the 1^st-level, the instruction period preceding the movies, the 12s-block of 6 movie-clips, and the response period were modelled separately in a factorial design with the diversity and the jerkiness of the movements in every block as two additional factors in the analysis. To estimate the diversity and jerkiness of the movements, we asked 3 independent judges to watch the 3x12 blocks of 6 movie-clips and to report after each block the number of different sorts of hand movements (diversity) and the number of pauses or changes of direction (jerkiness). The judges were allowed to watch the block of movies multiple times if they wanted to. Spearman correlations between the ratings were above 0.90 for diversity and above 0.70 for jerkiness. The jerkiness and diversity for each of the 36 blocks was estimated with the median of the 3 ratings. On average, the movements of the actors were judged to be more diverse and jerkier in the Touch than in the two Grasping conditions. The mean movement diversity ratings over 12 movies were 2.3, 2.3, and 4.8 for the Occlusion, Full-View, and Touch condition respectively. The mean jerkiness was 1.0, 0.1, 3.7. At the end however, entering or not diversity and jerkiness in the 1st-level analysis lead to identical results at the group level. Contrasts between each condition and the baseline were entered in a factorial design at the second level (random effect analysis, one single Factor with 3 levels, non-independent measurements with unequal variances, and masked with a binary image from a smoothed average of the 17 grey matter volumes). Activity related to the three conditions was contrasted with the baseline activity and with activity in the other conditions at the voxel level. To correct for multiple comparisons, the threshold was set to P_FWE = 0.05 when comparing the active tasks to the baseline, and to P_FDR = 0.005 when comparing the conditions with each other (P_FWE values are provided in the Tables). In both cases the minimum cluster size (k) was set to 20 voxels. Several strategies were developed in order to examine the impact of task difficulty. Firstly, the analysis was repeated with the number of errors of every subject on the three conditions as an additional factor (a single vector with the number of errors of the 17 subjects for the three conditions successively). Secondly, an analysis was conducted modelling-out every trial associated with an erroneous response at the 1^st-level (with a separate discard condition for every experimental conditions), and the general linear model was then computed at the group level on correct trials only. Finally, we analysed data from a control experiment involving an Occlusion condition and a control condition more difficult than the Occlusion. During that more difficult control condition, participants saw the same movie stimuli as during Occlusion, but did not have to focus on hand kinematics (see below).

For the execution task, the tone and the execution period were modelled separately at the subject level, taking into account the side of grasp. The actual duration of the grasping action was used to determine the number of volumes to be included in every execution block and in the following rest period. The contrasts between left grasp and baseline, and between right grasp and baseline were entered in a factorial design at the second level (non-independent measurements). The execution map showing the areas active above baseline during either left- or right-hand grasp (P_FDR < 0.005, k = 20) was overlaid on the map resulting from the contrasts between the 3 observation conditions in order to determine whether the areas identified could be regarded as belonging to the putative human mirror neuron system.

Control experiment for task difficulty

There were three conditions for the participants in this control experiment: an Occlusion, a Letter and a Theory of Mind condition. The last condition will not be reported here. The stimuli used for all 3 conditions were similar to the ones used in the Occlusion condition of the main experiment except that a letter q or x was flashed briefly on the left or right shoulder of the actor during the grasping action. This was achieved by superimposing the letter on 2 or 3 successive frames in the original movie clips, so the letter appeared to be flashed for 50 or 75ms during the course of the action. Each block lasted for 12 seconds with 6 actions of a given actor being displayed per block like for the main experiment. During the Letter condition, the participants had to decide how many times the letter q appeared on the t-shirt of the actor; during the Occlusion condition, how many times the big ball had been grasped. Responses were given as in the main experiment by selecting the number on the screen corresponding to the number of letter q or the number of big ball being grasped out of 6. There were 3 sessions with 4 blocks of every condition per session, randomized as for the main experiment. Images were pre-processed in SPM8 using the same procedures as described above. At the first-level, the instruction, the response, and the movie periods were modeled separately, and the parameter estimates for Occlusion & Letter were brought to the second level for a direct comparison between conditions. Finally, the mean contrast estimates were extracted for the Letter and the Occlusion conditions in 4 ROIs of the pMNS defined on the basis of the results of the contrast between the Occlusion and the Full-View condition in the main experiment.

Repetition suppression analysis

In a subsequent analysis, we used the repetition suppression technique (Grill-Spector et al., 2006) to test whether certain areas pertaining to the regions identified in the GLM, habituated their activity to the size of the ball being grasped. At the 1^st-level, three sorts of grasping actions were modelled for the Full-View and the Occlusion conditions separately. One Discard condition contained the first video-clip of every block, one Repeat condition contained all the video-clips where the ball that was grasped had the same size as the ball grasped in the previous clip (33 actions for the Full-View and 31 for the Occlusion condition), and one Non-Repeat condition contained the video-clips where the ball that was grasped had a different size from the one grasped in the previous clip (27 and 29 actions for the Full-View and the Occlusion condition respectively). The instruction period, the response period, and the Touch condition movies were modelled separately, for a total of 9 conditions in the factorial design. The repetition suppression (RS) effect was computed at the subject level, for the Full-View and the Occlusion conditions separately, by computing the contrasts Full-View_(Non-Repeat) – Full-View_(Repeat) and Occlusion_(Non-Repeat) – Occlusion_(Repeat). These contrast volumes were then brought to the second level separately for the Full-View and Occlusion condition and analysed using two one-sample t-tests. These voxel-wise t-tests were performed in the whole brain at P < 0.001 uncorrected, and a correction for cluster size was then applied to report only clusters that were unlikely to occur by chance (P < 0.05 corrected). Because we had strong hypotheses that we would find RS for the Full-View condition in the ventral visual pathway and for the Occlusion condition in the pMNS, whenever RS occurred in these regions, the cluster size significance was tested in a ROI defined on the basis of the results of the main GLM (Full-View > Occlusion contrast for the ventral visual pathway, and Occlusion > Full-View contrast for the pMNS, all at P_FDR < 0.005). The approach to threshold at the voxel level at p<0.001, then correct for cluster extend in the ROI was inspired from recent publications (e.g., Andics et al., 2013; Press et al., 2012). In addition to testing for repetition suppression, we also looked for repetition enhancement (Repeat > Non-Repeat) using the same statistics and same threshold. Finally, in order to test where RS was larger for the Full-View than for the Occlusion condition, RS in these two conditions was compared using paired-sample t-tests (i.e. contrast Full-View_(Non-Repeat) – Full-View_(Repeat) – Occlusion_(Non-Repeat) + Occlusion_(Repeat)).

Dynamic causal modelling

A five-node model was fitted to the data using Dynamic Causal Modelling (DCM) as implemented in SPM5. The set of brain areas entering the DCM was chosen a priori on the basis of the results of the main GLM results as recommended (Stephan et al., 2010). It included the parastriate cortex as the main source of visual input. It then included two routes fed by the parastriate cortex: the dorsal route composed of bidirectional connections between the lateral occipitotemporal cortex (LOT) and the anterior portion of the intraparietal sulcus (aIPS) and between the aIPS and the ventral premotor cortex (vPM), and the ventral route consisting in the medial fusiform gyrus associated with the recognition of man-made objects (Chouinard and Goodale, 2010; Shmuelof and Zohary, 2005). We did not include a specific node in the pSTS for the processing of biological motion because, although it was active, this region did not form a separate cluster from the lateral occipitotemporal cortex, which also contains neurons with a preference for biological motion in addition to ‘snapshot’ neurons responding equally strongly to dynamic and static presentation of bodies (Vangeneugden et al., 2009). Based on a plausible biophysical model of the fMRI signal, DCM uses Bayesian inferences to provide posterior estimates of both the effective strength of synaptic connections among the pre-specified set of neuronal populations, and the context-dependent modulations of these connections (Friston et al., 2003; Stephan et al., 2010). Our aims were twofold: (1) measuring the effect of attending to a grasping action on the connectivity between the parietal and the pre-motor nodes of the pMNS, and (2) studying the modulation of the connectivity between the ventral and the dorsal streams by the presentation of the object in full view. In terms of intrinsic connections, we only modelled forward connections from the parastriate cortex to the lateral occipitotemporal cortex (LOT) and to the medial fusiform gyrus. Backward connections certainly exist, but do not contribute to our capacity to test our hypotheses and are therefore not modelled. We modelled reciprocal connections between LOT, aIPS and vPM based on the available literature and the importance given to both forward and backward connections in recent theories of vicarious motor activity (Keysers and Perrett, 2004; Kilner et al., 2007; Miall 2003; Wolpert et al., 2003). We also modelled reciprocal connections between the dorsal route’s LOT cortex and the medial fusiform gyrus, which were evidenced by a previous work (Jastorff and Orban, 2009).

The coordinates of the five regions were determined on the basis of the group GLM on correct responses only. The vPM and the aIPS coordinates were defined from the contrast between the Occlusion and the Full-View conditions. The coordinates in the medial fusiform gyrus were taken from the contrast between the Full-View and the Occlusion conditions. Finally, the coordinates in the parastriate cortex and the lateral occipitotemporal cortex (LOT) were defined from the conjunction between the Full-View and the Occlusion condition relative to baseline, since these two nodes were equally active in both conditions. There are however inter-individual differences in the exact location within a macro-anatomical region where a given cognitive process takes place. Accordingly, to capture the BOLD signal relating to specific information processing in different participants, we did not extract brain activity from the exact same location across individuals. Instead, we identified the local maxima in subject-specific SPMs nearest to the maximum in the group SPM using the most revealing t-contrast for each region (Stephan et al., 2010). We looked for the most active voxel within the same anatomical structure at a maximum distance of 20 mm from the group maximum (this criterion had however to be relaxed to 25 mm for 6 subjects in the medial fusiform gyrus). The time course in the regions of interest was summarized by the first eigenvariate in an 8-mm sphere centred on the subject-specific maxima, adjusting for the effects of interest with the omnibus F-test including the 3 visual conditions relative to baseline. We modelled only one hemisphere in order to keep the number of parameters to be estimated and interpreted as low as possible, and because increasing the number of nodes can negatively affect parameters estimation (Daunizeau et al., 2011; Seghier and Friston, 2013). We selected regions of interest in the right hemisphere because the activity was more reliable and significantly higher in the right than in the left hemisphere (Supplementary Figure S2). This allowed us to localize our nodes of interest consistently in all participant’s single subject data in close vicinity to the group maxima, providing the strongest testing ground for our model. In addition, the right hemisphere activity was largely immune to task difficulty, and the RS analysis led to significant results in the right hemisphere only.

A hierarchical design matrix with three conditions was specifically created for the DCM analysis. The first condition, Video, corresponded to the presentation of a movie and consisted of the Occlusion, the Full-View, and the Touch conditions of the initial design. The second condition, Grasp, corresponded to the monitoring of a grasping action and comprised the Occlusion and the Full-View conditions together. Finally, the third condition, Full-View corresponded to the effect of seeing the target of the grasping action in full view. We hypothesized that the presentation of a video would have a direct effect on the neuronal activity in the parastriate cortex. The monitoring of a grasp was expected to have an influence on the reciprocal connections between the nodes of the pMNS (aIPS and vPM) and on the reciprocal connections between the LOT cortex and the aIPS. Finally, revealing the object (Full-View) was expected to have a direct effect of the neuronal activity in the fusiform gyrus, and to modulate the bidirectional connectivity between the neuronal populations in the fusiform gyrus and the LOT cortex.

The DCM was conducted only for those sessions where the subjects made the same number of errors in the occlusion and the Full-View condition. There were 29 such sessions, 27 with no error in any of the conditions, and 2 with one single error in both conditions. Every participant contributed at least one session to this analysis (2 subjects contributed three sessions, 8 subjects contributed two sessions, and 7 subjects contributed only one session). Posterior estimates (in Hz) of the connections strength and of the modulating effects of the 3 conditions were obtained for the 29 sessions separately. T-tests were computed on the posterior estimates of 17 independent subjects to assess the significance of the intrinsic connectivity and the effects of the conditions (against 0 Hz). The median value was selected for the t-test when 2 or 3 sessions were available from the same participant. P values were corrected for multiple comparisons using Bonferoni correction (P = 0.05/8 for both the intrinsic and the modulation effects). Furthermore, we also counted the number of participants showing positive, negative, or mixed evidence between sessions vis-à-vis the group results (P ≥ 0.8 in either direction).

3. Results

Behavioural results

The average number of correct responses for the Occlusion, the Full-View and the Touch condition are shown of Figure 2. As we had predicted, the participants’ ability to detect the actors’ intentions (i.e. big or small ball) was significantly higher in the Full-View than in the Occlusion condition (t₁₆ = 5.19, P < 0.001). This result is consistent with the hypothesis that the participants took advantage of the possibility to locate the balls on the table in Full-View to solve the task more accurately (Fig.1). Performance on the Touch condition was not significantly different from the performance on the Occlusion condition (t₁₆ = 0.11), and was lower than the performance on the Full-View condition (t₁₆ = 2.88, P < 0.05). The number of participants who scored 10/12 correct or above (exact binomial probability P = 0.0193) was 14, 17, and 12/17 in the Occlusion, the Full-View, and the Touch condition respectively.

Figure 2. Task performance in the 3 conditions.

Mean number of correct responses (out of 12) and standard deviation for the 17 participants in the Occlusion (OC), Full-View (FV), and Touch (TO) conditions.

Whole brain analyses

The brain areas increasing their activity above baseline in the Occlusion, the Full-View and the Touch conditions are represented in Figure 3. As can be seen in the figure, all three visual conditions triggered activation of regions of the occipital and temporal cortex involved in visual processing of the human body (Downing et al., 2001, 2006; Vangeneugden et al., 2009) and the perceptual analysis of biological motion (Beauchamp et al., 2002, 2003; Blake and Shiffrar, 2007; Bonda et al., 1996; Grossman and Blake, 2002; Grossman et al., 2000; Jastorff and Orban, 2009; Peelen et al., 2006; Pelphrey et al., 2003; Vangeneugden et al., 2009). These regions particularly included the extrastriate body area (EBA) in the lateral occipitotemporal cortex and the posterior superior temporal sulcus (pSTS) involved in the perception of biological motion. In addition, activity was found in the premotor and parietal cortices bilaterally for every visual condition relative to baseline (Fig. 3). In the premotor cortex, activity increased along the precentral sulci (BA6) and in the inferior frontal gyrus pars opercularis (BA44). In the parietal lobe, increases of activity were seen along the intraparietal sulci (areas hIP1-3, PF, BA7, BA2), in the postcentral sulci (areas BA2, PF, PFt), and in the supramarginal gyrus (areas PFm, PF). In between the pre- and postcentral regions, the primary motor cortex did not increase its activity above baseline during the observation of the movies.

Figure 3. Activity above baseline in the 3 conditions.

Areas increasing their activity above baseline in (A) the Occlusion, (B) the Full-View, and (C) the Touch condition. Statistical maps are rendered on a smoothed average brain surface with a corrected threshold of P_FWE < 0.05 (t₄₈ = 5.27). The premotor cortex (1), the pre-supplementary motor area (2), and the intra-parietal and postcentral cortices (3) were active in every conditions in addition to the occipital cortex (4), and temporal areas involved in the processing of biological motion (5, 6). At the cytoarchitectonic level, the premotor activity involved BA44 and BA6, and the parietal activity concerned BA2, hIP1-3, the PF complex, and BA7.

Contrasting the activation for the Occlusion and the Full-View conditions revealed significant differences in line with our hypotheses. We expected that activity would be enhanced in the pMNS during the Occlusion condition, when the only relevant information for inferring the goal of the actor was in the analysis of the hand/fingers kinematics. We also expected that part of the activity would switch towards visual brain regions outside the pMNS in Full-View when additional information about the location of the balls on the table was available. Inspection of Figure 4A and Table 1 shows that indeed, relative to the Full-View condition, an increase of activity was found bilaterally in the ventral premotor cortex (BA6/44), the postcentral sulci (BA2, PF/PFt), and the anterior intraparietal sulci (hIP1-3, PF) considered to be part of the pMNS when the balls were concealed inside the box (Occlusion). Analysis of the activation maps for the execution task (grasping a ball) confirmed that the cortical areas increasing their activity during the Occlusion condition overlapped with the cerebral network involved in the execution of a grasping action. Regions of overlap were found bilaterally in the ventral premotor cortex, the postcentral sulcus and the anterior intraparietal sulcus (Supplementary Fig. S3). For the Full-View condition on the other hand, as can be seen in Figure 4B and Table 2, the contrast Full-View > Occlusion showed a relative increase of activity in the ventral visual stream bilaterally, with peaks in the medial fusiform gyri, the lingual gyri, and the parahippocampal place area (Epstein et al., 1999) when the balls were fully visible. This contrast also revealed relative increases of activity in regions including the medial prefrontal cortex and the pre-cuneus associated with the default mode network (Raichle et al., 2001), and the temporo-parietal junction which is associated with thinking about the believes of others (Saxe and Kanwisher, 2003) (Table 2). Comparing the level of activity in these latter regions relative to baseline for all three visual conditions revealed that this was due to larger de-activations during the more difficult Occlusion condition (Supplementary Fig. S4). In contrast, in the ventral visual stream, all conditions caused increases of activation relative to baseline excepted in the most anterior parts of the cluster (in the parahippocampal gyri) where the activity was above baseline for the Full-View condition only (Supplementary Fig. S4). Because the Touch condition mainly served as control to inform the connectivity analyses, activation maps for this condition are only presented in the supplementary materials (Supplementary Table S1 and Fig. S5). Activity in the putative mirror neuron system was as strong for this condition that did not involved any grasping action as for the two other conditions where the subjects had to monitor grasping actions. In addition to pMNS activity, a relative increase of activity was found in the secondary somatosensory cortices of the participants when they were monitoring how many times the actor touched an object (Supplementary Table S1 and Fig. S5).

Figure 4. Contrasts between the Occlusion and the Full-View condition.

(A) Regions showing increased activity for the Occlusion relative to the Full-View condition (P_FDR = 0.005; t₄₈ = 3.91). The axial section on the top-left corner (z = 44) shows the activity in the intraparietal and postcentral sulci in both hemispheres and in the pre-SMA. The axial section on the bottom-left corner (z = 27) shows the activity in the ventral pre-motor cortex, centered on the inferior pre-central sulcus in both hemispheres. On the right of the panel, the postcentral and ventral pre-motor clusters of activity are also represented on sagittal sections (x = -48 and x = 54 for the left and right hemispheres respectively). (B) Regions showing increased activity for the Full-View relative to the Occlusion condition (P_FDR = 0.005; t₄₈ = 3.82). The axial section on the top-left corner (z = -15) shows the clusters of activity in the inferior lingual gyrus (O5) and the parahippocampal gyrus (T5) of both hemispheres. The bottom-left section (z = -6) shows the clusters of activity in the superior lingual gyri (O5). The relative increase of activity on the ventral visual pathway is also represented on three coronal sections on the right, with from top to bottom the activity in the parahippocampal gyri (y = -36), in the medial fusiform and parahippocampal gyri (y = -42), and in the medial fusiform and lingual gyri (y = -52). iPrC: inferior pre-central sulcus, aIPS: anterior part of intra-parietal sulcus, PoCS: post-central sulcus (ascending segment of IPS), preSMA: pre-supplementary motor area, OPF: frontal operculum, BA44: inferior frontal gyrus pars opercularis, pHiG: parahippocampal gyrus, mFG: medial fusiform gyrus, LG: inferior and superior lingual gyrus.

Table 1.

Occlusion minus Full-View

Cluster		Voxel
Pcorr	k (E)	PFWE	PFDR	T	x	y	z	Area (Cytoarchitectonic region)
0.000	335	0.000	0.000	7.18	54	6	21	R inferior precentral sulcus (BA44/BA6)
		0.010	0.000	5.81	42	21	-3	R short insular gyrus
		0.075	0.001	5.12	45	18	12	R inferior frontal g. pars opercularis (BA44)
0.000	389	0.000	0.000	6.80	60	-24	45	R postcentral sulcus (BA2/PFt/PFop)
		0.000	0.000	6.68	54	-24	39	R postcentral sulcus (BA2/PFt/PFop)
		0.001	0.000	6.37	39	-42	42	R anterior intraparietal sulcus (hIP1-3)
0.000	115	0.005	0.000	6.01	-60	-24	39	L postcentral sulcus (BA2/PFt/hIP1-3)
		0.027	0.000	5.48	-42	-36	42	L anterior intraparietal sulcus (PFt/hIP1-3/BA2)
		0.043	0.000	5.32	-57	-30	45	L anterior intraparietal sulcus (PF/hIP2/BA2)
0.001	58	0.007	0.000	5.92	36	-84	-6	R inferior/middle occipital gyrus (BA18)
0.006	35	0.013	0.000	5.73	6	24	42	Medial superior frontal gyrus, pre-SMA
0.000	123	0.013	0.000	5.73	42	42	6	R middle frontal gyrus
		0.209	0.001	4.74	51	33	24	R middle frontal g. / inferior frontal sulcus
0.000	94	0.026	0.000	5.49	-48	6	21	L inferior precentral sulcus (BA44)
		0.070	0.000	5.15	-60	9	27	L inferior precentral sulcus (BA44/6)
0.016	27	0.041	0.000	5.34	-33	-84	-12	L inferior occipital gyrus (BA18)
0.033	21	0.286	0.001	4.60	45	-60	-6	R middle occipital gyrus – inferior part (BA19)

Height threshold: T = 3.909; P = 0.000 (P FDR = 0.005). Extent threshold: k = 20; P = 0.019 (0.038). Expected voxels per cluster <k> : 3.343. Expected number of clusters <c> : 0.04. DF = 1,48. FWHM (mm) = 11.5, 11.7, 11.4. Volume = 41632 voxels = 566.50 resels. Voxel size (mm) = 3.0, 3.0, 3.0 (1 resel = 56.84 voxels). Coordinates are in MNI space here and throughout the paper. Cytoarchitectonic regions in the parietal cortex where defined using the SPM Anatomy Toolbox which provides a more precise map of the supramarginal gyrus (BA40), and intraparietal sulcus. BA: Brodmann Area.

Table 2.

Full-View minus Occlusion

Cluster		Voxel
Pcorr	k (E)	PFWE	PFDR	T	x	y	z	Area
0.000	442	0.000	0.000	7.40	21	-42	-12	R fusiform gyrus (BA36) / collateral sulcus
		0.010	0.000	5.81	36	-27	-15	R parahippocampal g. (BA35) / collateral sulcus
		0.409	0.002	4.44	33	-18	-24	R parahippocampal gyrus, T5 (BA36)
0.000	541	0.000	0.000	7.26	-9	-69	-6	L lingual gyrus (BA18)
		0.001	0.000	6.36	-24	-51	-12	L fusiform gyrus (BA37)
		0.005	0.000	6.00	-15	-42	-9	L lingual gyrus (BA19)
0.000	325	0.008	0.000	5.88	-3	-54	39	Posterior cingulate g. (BA31) / subparietal sulcus
		0.017	0.000	5.63	0	-51	48	Precuneus (BA7) / subparietal sulcus
		0.275	0.001	4.62	12	-54	66	R superior parietal gyrus (BA7)
0.000	184	0.022	0.000	5.55	-12	63	21	L superior frontal gyrus (BA9) – medial prefrontal
		0.146	0.001	4.88	-12	63	12	L superior frontal g. (BA10) – medial prefrontal
		0.154	0.001	4.86	-3	60	6	Frontopolar gyrus (BA10) – medial prefrontal
0.003	46	0.292	0.001	4.59	-51	-69	30	L angular g. (BA39/temporo-parietal junction)
		0.525	0.002	4.31	-48	-60	21	L superior temporal sulcus – posterior segment

Height threshold: T = 3.821; P = 0.000 (P FDR = 0.005). Extent threshold: k = 20; P = 0.022 (0.055). Expected voxels per cluster <k> : 3.585. Expected number of clusters <c> : 0.006. DF = 1,48. FWHM (mm) = 11.5, 11.7, 11.4. Volume = 41632 voxels = 566.50 resels. Voxel size (mm) = 3.0, 3.0, 3.0 (1 resel = 56.84 voxels).

Control for task difficulty

The relative increase of activity in the parietal and premotor cortices for the Occlusion condition seems to confirm that the re-enactment of the grasping action in the pMNS was most important to distinguish big from small ball grasps in the Occlusion condition when the balls were concealed inside the box. An alternative interpretation however would be that the increase of activity in the parieto-frontal network was the consequence of the higher level of difficulty of the Occlusion condition. In order to evaluate the effect of task difficulty on pMNS activity, we first repeated the analysis including only correct trials in the model at the first level. As can be seen in Figure 5, there was nearly as much activity in the parieto-frontal network when correct trials only were included in the analysis (Fig. 5B) as with all trials (Fig. 5A). Given that the Occlusion condition was likely to be more difficult than the Full-View condition for correct trials as well as incorrect ones, we conducted another analysis including the number of errors of each subject in both tasks as a cofactor at the second level (see Methods). As shown in Figure 5C, significant increases of activity for the Occlusion relative to the Full-View condition remained in the right ventral pre-motor cortex (vPM), postcentral sulcus (PoCS) and anterior intraparietal sulcus (aIPS) after including the number of errors as a cofactor. The Full-View condition is nonetheless inevitably easier than the Occlusion condition since additional information about the location of the balls is available to the subject.

Figure 5. Controls for task difficulty.

(A) Increase of activity in the putative mirror neuron system for the Occlusion relative to the Full-View condition observed before controlling for task difficulty. The activation map is rendered on a template smoothed surface view (t₄₈ = 3.91, P_FDR < 0.005, see Table 1 for peaks coordinates and clusters size). (B) The same increase of activity was observed in the pMNS when the analysis was restricted to the movie clips that were followed by a correct decision (t₄₈ = 4.02, P_FDR < 0.005). (C) Activity in the right hemisphere pMNS remained significant after removing the influence of task difficulty with the inclusion of an additional factor in the analysis (t₄₇ = 3.83, P_FDR < 0.05). The largest cluster of activity, independently of task difficulty, was in the ventral premotor cortex (k = 69, corrected P < 0.001), with the main peak of activity in the inferior pre-central sulcus (t₄₇ = 5.18, P_FDR = 0.018; x = 51, y = 9, z = 27). Three peaks of activity were also found in the right parietal lobe, two of them in the postcentral sulcus (t₄₇ = 5.24, P_FDR = 0.018; x = 57, y = -24, z = 51, and t₄₇ = 4.81, P_FDR = 0.018; x = 54, y = -24, z = 42), and the other in the intraparietal sulcus (t₄₇ = 4.65, P_FDR = 0.020 FDR; x = 39; y = -39; z = 42). Finally, one region of the right occipital cortex in the inferior part of the middle occipital gyrus (/sulcus lunatus) was also more active in the occlusion than in the Full-View condition (t₄₇ = 4.91, P_FDR = 0.018; x = 33; y = -84; z = -6). This inferior occipital area was not active above baseline during action execution (Fig. S3). Inspection of the areas commonly active for the three observation conditions relative to baseline showed that this cluster fell at the border of a major cluster of activity centred anteriorly in the middle occipital gyrus extra-striate body area (see Fig. 3). iPrC: inferior precentral gyrus/sulcus, aIPS: anterior intraparietal sulcus, PoCS: postcentral sulcus (ascending segment of IPS), O2: inferior part of middle occipital gyrus. (D) Mean contrast values and 90% confidence interval for the Occlusion and the Letter conditions relative to baseline in the 4 regions of the pMNS. The Occlusion condition was associated with more activity in the 4 ROIs (all p<0.01, Bonferroni corrected for 4 comparisons) despite the fact that the Letter condition was more difficult and involved the same video stimuli as the Occlusion condition (see also Supplementary Table S2). IP: inferior parietal, vPM: ventral premotor cortex.

If task difficulty alone accounts for the Occlusion>Full-View effect observed in the parieto-frontal network, comparing the Occlusion condition to a significantly more difficult control task requiring attention to a different aspect of the same movies should lead to a reversal of the effect in the parieto-frontal network. In order to investigate this question, we analysed new data collected in an independent sample of 16 subjects who saw movies of actors grasping one of two balls inside a box, as in the Occlusion condition of the main experiment. In this control experiment however, a letter q or x was briefly flashed on the left or right shoulder of the actor performing the grasping action (see Methods). In the new Occlusion condition, participants had to report how often the big ball had been grasped, in the new control condition (the Letter condition), participants had to report how often the letter q had appeared on the t-shirt. The Letter condition in this experiment was significantly more difficult than the Occlusion condition (Letter: 10.9±1.1 vs. Occlusion: 11.6±0.6 correct answers out of 12 trials, paired t-test t₁₅ = 2.165, P < 0.05). We extracted the mean parameter estimates and 90% confidence intervals for the Occlusion and Letter conditions relative to baseline in the pre-motor and inferior parietal cortex bilaterally (4 ROIs defined on the basis of the contrast between the Occlusion and the Full View condition of the main experiment at P_FDR < 0.005, see Fig.5D). The results show that in all 4 ROIs of the pMNS, the Occlusion condition triggered more activity than the more difficult Letter condition during which the participants saw the exact same video stimuli but did not have to pay attention to the hand kinematics to detect the goal of the actor (Figure 5D). This was also confirmed by extracting the values for the direct contrast between the Occlusion and the Letter condition in the 4 ROIs (P < 0.01 corrected for all ROIs, Supplementary Table S2). This finding shows that task difficulty does not account for the increased parieto-frontal activity during the Occlusion condition and renders our initial interpretation more likely, that the additional pMNS activity derived from increased reliance on the processing of hand kinematics.

In Full View in contrast, there was not only relatively less activity in the pMNS, there was also more activity on the ventral visual stream. Including task difficulty as a covariate, slight increases in clusters size and t-values were noticed on the ventral visual pathway, but no additional cluster was found. Removing the blocks associated with an erroneous response did not change the activity in Full-View as no subject made more than one mistake in this condition. Since the same information about the hand and finger kinematics was available to the participants in the Full-View and the Occlusion conditions, the results suggest that the higher performance in Full-View and the increased activation of ventral visual stream regions reflect the utilization of the additional information provided by being able to see how large the ball was that was closest to the hand. An alternative, and less interesting interpretation however would be that the two balls on the table and the larger visible part of the actor’s torso triggered more ventral visual stream activation in Full-View without this activity encoding task relevant information (which ball was being grasped).

Repetition suppression

To test whether the ventral stream activity carried task relevant information in Full-View, we looked for voxels where the activation diminished when a ball of the same size was grasped repeatedly. At an uncorrected threshold of P < 0.001 (k = 5 voxels) in the entire brain, the only significant cluster (green in Figure 6; peak at (27, -45, -6); t = 4.45) falls within the region of the right ventral visual stream where the activity increased most for the Full-View relative to the Occlusion condition (our a priori ROI, blue contour Fig. 6). Cluster correction within this a priori ROI confirmed the significance of this cluster (cluster P_corrected = 0.036). A direct comparison between RS in Full-View versus Occlusion using a paired-sample t-test (Full-View_(Non-Repeat) – Full-View_(Repeat) – Occlusion_(Non-Repeat) + Occlusion_(Repeat), Supplementary Table S3) confirmed that RS in the right ventral visual pathway was stronger in the Full-View than the Occlusion condition (Fig. 6, red), and that this condition-specific RS overlapped with the one in Full-View within the ROI (Fig. 6, yellow). For the Occlusion condition, we could not find significant RS at the whole brain level nor in any of the ROIs at the threshold set for the analysis (P < 0.05 corrected for cluster size). Finally we also looked for clusters showing significant response enhancement in one of the two conditions, but could not find any using the same threshold as for RS.

Figure 6. Repetition suppression.

Repetition Suppression in the Full-View condition when observing the actor grasping a ball of the same size on consecutive trials is shown in Green (peak t₁₆ = 4.45, P = 0.001 uncorrected), and falls within the ROI that showed FullView>Occlusion in the main GLM (blue contour, cluster P corrected within blue ROI = 0.036). Regions showing a significantly larger RS during Full-View compared to Occlusion are shown in red, and their overlap with the RS during Full-View in yellow. All results are shown at P < 0.001 uncorrected on a sagittal slice of the average brain at x = 27.

Dynamic causal modelling

Dynamic Causal Modelling (DCM) was conducted in order to explore the changes in brain connectivity associated with the detection of big ball grasps in the Occlusion and Full View conditions. Our five-nodes model is illustrated on Figure 7. The coordinates of time course extraction in every subject for the five areas of the model are provided in the Supplementary Material (Tables S4-S8). The posterior estimates of the intrinsic connections in the model were all positive and significant, apart from the backward connection between the ventral premotor cortex (vPM) and the anterior segment of the intraparietal sulcus (aIPS), which was not significant (Table 3). Table 4 gives the posterior estimates of the direct task effects and the modulation of the connectivity by the condition for every participant. The modulations and direct effects that were significant at the group level are summarized on Figure 7. The size of the effects can be appreciated by inspecting the average value of the posterior estimates (in Hz), the Bonferroni corrected probabilities that these posterior estimates are different from zero at the group level, and the number of participants showing consistent positive evidence for the 3 sessions (Fig. 7, Table 4). The presentation of the video stimuli had a significant direct impact on the activity of the parastriate region (BA18, inferior lingual gyrus). More importantly, detecting big ball grasps in a series of grasping actions (compared to detecting the occurrence of touch in the visual control condition) increased the transfer of information from the aIPS to the vPM, suggesting that this circuit indeed caries information specific to grasping. Detecting big ball grasps also generated a negative backward connectivity between the vPM and the aIPS, and between the aIPS and the lateral occipitotemporal cortex (LOT). Finally, the presence of the object in Full View had a direct effect on the ventral visual stream (fusiform gyrus), and negatively modulated the information flow between the fusiform gyrus and the LOT cortex, suggesting that more processing occurred in the ventral stream for this condition and the information transfer towards the pMNS was reduced.

Figure 7. Results of the Dynamic Causal Modelling.

Schema of the 5-nodes model and mean posterior estimates of the modulation by the experimental conditions (Video, Grasp, Full-View). LG: lingual gyrus (parastriate cortex BA18), mFG: medial fusiform gyrus/superior lingual gyrus, LOT: lateral occipitotemporal cortex (extrastriate body area), aIPS: anterior part of the intra-parietal sulcus, vPM: ventral premotor cortex. All the posterior estimates of the intrinsic connectivity between the nodes of the model were significant but the one between vPM and aIPS represented by a dashed line (see also Table 3). The mean posterior estimates of the modulation of the connectivity be the experimental conditions is provided (in Hz) when significant: P < 0.05 (*) and P < 0.005 (**) Bonferoni corrected. The number of participants showing consistent positive evidence for the model is provided in parenthesis. The observation of a video stimulus had a direct effect on the neuronal activity in LG (BA18). The observation of a grasping action (Grasp) increased the connectivity between the neuronal populations in aIPS and vPM, in accordance with the idea that these two regions of interest contained mirror neurons specifically involved in the re-enactment of grasping actions. Furthermore, the backward connectivity between vPM and aIPS, and between aIPS and LOT was decreased during grasping observation, demonstrating the importance of backward connections during action perception. Finally, in accordance with the hypothesis of a partial shift of activity from a dorsal route for the ‘simulation’ of grasping actions, to a ventral route for the visual and spatial analysis of objects shape and location, the vision of the object had a direct effect on the synaptic activity in the ventral node (mFG), and diminished the connectivity between mFG and LOT, therefore isolating the ventral from the dorsal pathway.

Table 3.

Intrinsic connectivity estimates per subject (Hz)

	LG–mFG	LG–EBA	mFG–EBA	EBA–mFG	EBA–aIPS	aIPS–EBA	aIPS–vPM	vPM–aIPS
S-1	0.315	0.493	0.230	0.297	0.257	0.076	0.415	-0.049
S-2	0.091	0.399	0.180	0.275	0.567	0.320	0.473	0.055
S-3	0.046	0.300	0.135	0.307	0.475	0.156	0.681	-0.046
S-4	-0.014	0.489	-0.034	0.217	0.464	0.112	0.620	-0.069
S-5	0.092	0.516	0.022	0.096	0.361	0.104	0.208	-0.001
S-6	0.146	0.582	0.151	0.174	0.524	0.245	0.402	0.029
S-7	0.020	0.479	0.041	0.174	0.560	0.230	0.604	0.009
S-8	-0.002	0.593	-0.036	0.127	0.695	0.301	0.625	0.062
S-9	0.180	0.383	0.262	0.190	0.542	0.151	0.438	0.023
S10	0.270	0.241	0.198	0.226	0.587	0.060	0.659	-0.017
S-11	0.190	0.468	0.171	0.215	0.476	0.079	0.508	-0.050
S-12	0.088	0.506	0.114	0.196	0.492	0.183	0.479	-0.027
S-13	0.086	0.297	0.306	0.363	0.445	0.115	0.614	-0.051
S-14	0.066	0.302	0.059	0.156	0.480	0.117	0.578	0.021
S-15	0.028	0.343	0.075	0.154	0.394	0.106	0.537	-0.060
S-16	0.039	0.463	0.187	0.221	0.488	0.232	0.571	-0.002
S-17	0.182	0.444	0.262	0.366	0.754	0.455	0.555	0.152
Mean	0.107	0.429	0.137	0.221	0.504	0.179	0.528	-0.001
SD	0.094	0.104	0.103	0.077	0.116	0.106	0.118	0.056
T-value	4.71	17.02	5.47	11.75	17.86	6.95	18.49	-0.09
Sign.	**	**	**	**	**	**	**	ns
N posit. evidence	11	17	9	16	17	9	17	1

Estimates (in Hz) of the intrinsic connectivity between the five areas of the model for the 17 participants and summary statistics: Arithmetical Mean, standard deviation (SD), T-value (16 d.f.), Significance after Bonferoni correction with 2-tailed P corrected = 0.005 (**). The number of positive evidence was computed as the number of participants with an intrinsic connectivity significantly larger than 0 Hz (P ≥ 0.8) for one session at least. LG: lingual gyrus (parastriate cortex, BA18), mFG: medial fusiform gyrus (/superior lingual gyrus), EBA: extra-striate body area, aIPS: anterior segment of the intra-parietal sulcus, vPM: ventral premotor cortex.

Table 4.

Direct effects and modulation of the connectivity by the experimental conditions (Hz)

	Video on LG	Grasp on EBA–aIPS	Grasp on aIPS–EBA	Grasp on aIPS–vPM	Grasp on vPM–aIPS	F-V on mFG–EBA	F-V on EBA–mFG	F-V on mFG
S-1	0.704	-0.044	-0.004	0.134	-0.014	-0.083	-0.010	0.071
S-2	0.477	-0.039	0.035	0.209	-0.097	-0.172	-0.012	-0.019
S-3	0.535	-0.065	-0.280	0.136	-0.057	-0.156	-0.079	0.009
S-4	0.350	-0.124	-0.074	0.166	-0.056	-0.175	0.002	0.025
S-5	0.156	0.142	-0.027	0.147	0.001	-0.106	-0.091	0.029
S-6	0.195	-0.035	-0.230	0.179	-0.068	-0.113	0.012	0.005
S-7	0.436	-0.068	-0.181	0.064	-0.079	-0.178	0.030	0.006
S-8	0.227	-0.183	-0.122	-0.001	-0.104	-0.106	-0.004	0.039
S-9	0.144	-0.024	-0.168	0.109	-0.041	0.127	0.006	0.023
S-10	0.588	0.024	-0.233	0.114	-0.113	-0.158	-0.021	0.091
S-11	0.203	-0.059	-0.141	0.024	-0.054	-0.076	-0.032	0.027
S-12	0.280	-0.051	-0.181	0.103	-0.102	-0.072	0.005	0.001
S-13	0.714	-0.085	-0.073	0.127	-0.040	0.161	-0.193	0.115
S-14	0.506	0.063	-0.184	0.260	-0.076	-0.130	-0.013	0.064
S-15	0.516	-0.110	-0.067	0.173	-0.056	-0.067	-0.115	0.010
S-16	0.470	-0.121	-0.160	0.145	-0.060	-0.113	-0.154	0.007
S-17	0.314	0.026	-0.247	0.020	-0.068	-0.461	-0.008	0.019
Mean	0.401	-0.044	-0.137	0.124	-0.064	-0.110	-0.040	0.031
SD	0.184	0.077	0.090	0.068	0.031	0.131	0.064	0.035
T-value	8.99	-2.37	-6.26	7.48	-8.60	-3.47	-2.58	3.60
	**	NS	**	**	**	*	NS	*
N positive evidence	17	9	13	11	2	9	4	11
Inconsistent	0	2	2	0	0	0	0	0
Opposite	0	1	0	0	0	2	0	1

Estimates of the direct effect of viewing a video on the lingual gyrus (column 1), estimates of the effect of observing grasping actions on the reciprocal connexions between the aIPS and EBA (columns 2 and 3), and on the reciprocal connexions between aIPS and vPM (columns 4 and 5), estimates of the effect of viewing the target of the grasping action on the reciprocal connexions between mFG and EBA (columns 6 and 7), and estimates of the direct effect of the Full View on the firing rate in mFG. Values in Hertz for the 17 participants, and summary statistics: Arithmetical Mean, standard deviation (SD), T-value (16 d.f.), Significance after Bonferoni correction with 2-tailed P corrected = 0.05 (*), = 0.005 (**). The number of positive evidences was computed as the number of participants showing significant modulation of connectivity (P ≥ 0.8) in the same direction as the group, and is provided along with the number of participants showing inconsistent significant modulation across sessions, and the number of participants showing significant modulation in the opposite direction from the group. LG: lingual gyrus (BA18), mFG: medial fusiform gyrus (/superior lingual gyrus), EBA: extra-striate body area, aIPS: anterior segment of the intra-parietal sulcus, vPM: ventral premotor cortex.

Correlation with performance

Finally, if the performance in the Occlusion condition were indeed dependent upon the computations of the pMNS as we hypothesized, one would expect to find a positive correlation between the number of correct answers in the Occlusion condition and the functional connectivity between the pMNS nodes as quantified at the individual level in the DCM analysis. Specifically, we used the modulation by grasping of the aIPS->vPM connection as the proxy for pMNS deployment for two reasons. First, this parieto-frontal connectivity is considered to be the most important connection in most models of the pMNS (Nelissen et al., 2011; Rizzolatti and Luppino, 2001). Second, this connection showed a highly significant and consistent augmentation while witnessing a grasp across our participant showing that it was relevant to our experiment (see Table 4). We found a significant positive correlation between this connectivity and performance during Occlusion (r = 0.48, t₁₅ = 2.10, P < 0.027 one-tail; Spearman r = 0.45, P < 0.033 one-tail), indicating that performance in the Occlusion condition was higher in participants who recruited this connection more strongly when monitoring grasping actions, a finding that supports the idea that encoding the hand kinematics in the pMNS was particularly helpful for solving the task when the balls were concealed inside the box. Likewise, because participants made virtually no mistake in the Full-View condition, the abovementioned correlation with performance during Occlusion can also be interpreted as indicating that participants that recruits the pMNS through the aIPS->vPM connection when monitoring a grasp more strongly show a smaller drop in performance relative to Full-View when the object is occluded (r = -0.50, t₁₅ = -2.21, P < 0.022 one-tail; Spearman r = -0.60, P < 0.005 one-tail). This relationship was confirmed with a robust regression analysis (slope = -8.02 ±3.73, t₁₅ = -2.15, P < 0.048, inline Supplementary Figure S6).

4. Discussion

The participants in our fMRI experiment were requested to guess which of two objects an actor was aiming to grasp (big or small). We either occluded or showed the two objects in order to manipulate whether the kinematics of the actors’ movements were the only source of information or not. To equate the amount of kinematic information in the Occlusion and the Full-View conditions, we always stopped the movies 120-200ms before the hand contacted the object so that the outcome had to be anticipated in both conditions. Because in monkeys, 50 to 65% of vPM mirror neurons active for grasping fire even if the last part of the action, the grasp itself, is concealed (Kraskov et al., 2009; Umiltà et al., 2001), one can assume that in both conditions, about two thirds of the mirror neurons responding to grasping actions contributed to the BOLD signal. However, when the objects were fully visible, additional visual information could be used to solve the task, while when the objects were concealed from the view of the participant, the only relevant information was the hand kinematics. We hypothesized that activation in the parietal and premotor cortices would be strongest if the hand and finger movements were the only relevant information for inferring what object was going to be grasped. With the object fully visible however, we expected activity in the ventral visual stream to provide part of the task-relevant information.

The results of the experiment are in accord with our hypothesis. Activity increased in the inferior parietal (BA2/PF/hIP1-3) and the ventral premotor cortex (BA6/44) when the balls were hidden inside the box compared to when they were visible, i.e. when participants had to rely entirely on an analysis of the hand and fingers kinematics to decide which of the two balls was going to be picked. Areas of increased activity during the Occlusion condition overlapped with the network of cerebral areas active during the execution of a grasping action in the same subjects, making an interpretation of these differential activations as reflecting increased recruitment of the pMNS more likely. An alternative interpretation however could be that the increase of activity in the fronto-parietal network was the consequence of the subjects increasing their attention in the most difficult Occlusion condition. The additional activity would then reflect domain unspecific deployment of endogenous attention rather than processes specific to kinematic processing. Comparing the location of the clusters showing increased activity during Occlusion with those reported to be involved in domain unspecific endogenous attention (Corbetta and Shulman, 2002; Corbetta et al., 2008), reveals that our parietal activations are more anterior and our premotor activations more posterior than those typically reported for endogenous goal-directed attention. In order to further exclude the role of task difficulty, we analysed data from an independent sample of subjects who saw the same Occlusion movies but with a letter flashed very briefly on the (left or right) shoulder of the actor performing the grasping action. In one task, the new Occlusion task, they had to count the number of times the big ball was grasped (as in our main experiment), in the other, the Letter task, how often the letter q was flashed. Importantly, this Letter condition was significantly more difficult for the subjects than the new Occlusion condition. We found nonetheless that the activity in the pMNS regions was significantly increased for the Occlusion condition compared to the more difficult Letter condition. Taken together with the main experiment that found the Occlusion condition to trigger more activity than an easier control task, the most parsimonious explanation for these effects is thus that the increase of activity in the pMNS reflects increased demands on the processing of the hand kinematics of the actors. That a task focusing attention on kinematics increases pMNS activity even while viewing the exact same stimuli shows that processing within the pMNS is not impermeable to top down control. This may seem surprising given the existence of data showing automatic activation of the pMNS, even when this activity interferes with the on-going execution of a planned movement (e.g., Kilner et al., 2003), and thereby refines our understanding of the pMNS. This finding is however in perfect agreement with neurophysiological and brain imaging studies demonstrating that focusing the attention on a spatial location or a specific feature of a stimulus is associated with enhanced neuronal activity in the sensory areas devoted to process this feature, like for instance MT for motion (review in Kastner and Ungerleider, 2000; Maunsell and Treue, 2006).

In contrast, activity was increased in the lingual/fusiform, the parahippocampal, and the superior parietal gyri for the Full-View relative to the Occlusion condition. This increase of activity in visual and spatial cerebral areas, including the predicted increase in the medial fusiform gyrus particularly involved in the observation of tools (Chouinard and Goodale, 2010) was accompanied by a significant reduction of the number of errors. Since the hand kinematics were similarly visible during Occlusion and Full-View, the amelioration of performance certainly derived form the possibility to view the size of the ball closest to the hand at the end of the movie in the Full-View condition. Along with this activity on the ventral pathway, the Full-View condition was associated with a relative increase of activity in the default mode network and in a region at the temporo-parietal junction involved in representing the mental states of others. This relative increase of activity in the default network during the easiest active condition could be related to the spontaneous thoughts of the participants when their attention was not captured by the task (Mason et al., 2007). The temporo-parietal junction on the other hand, is not part of the default network, but its activity followed a similar pattern, with more deactivation during the most difficult (Occlusion) conditions. While speculative, the increase of activity in this region might indicate that parts of the thoughts of the participants during the easiest condition were directed towards the state of mind of the actors.

In order to confirm that in the Full-View condition the size of the ball being grasped was actually encoded in neuronal circuits pertaining to the ventral visual pathway, we looked for voxels where the BOLD signal diminished when the actor grasped a ball of the same size repeatedly (i.e. repetition suppression, RS). RS was observed in the right lingual/parahippocampal gyrus during the Full-View condition, while there was apparently no RS on the ventral stream for the Occlusion condition. Indeed, comparing RS across conditions revealed that it was stronger on the ventral stream during Full-View, particularly in the medial fusiform gyrus associated with the recognition of man-made objects (Chouinard and Goodale, 2010). Because this RS was found by comparing trials within the Full-View condition, it shows that at least part of the activity on the ventral pathway was not a mere by-product of the greater visual complexity of the stimuli in this condition. Furthermore, RS was observed despite the fact that the side on which the small ball was located was randomized, and the ball being grasped was hence not more likely to be on the same side of the table in same size repeats (Supplementary Table S9). This indicates that the repetition suppression effect reflected the processing of information about the size of the ball being grasped, not the side of grasp. The most straightforward account of the effect therefore would be that some neurons in the ventral temporal region encoded the size of the ball being grasped by the actor in Full-View and habituated their activity during repeated trials due to a phenomenon of synaptic fatigue/depression (Grill-Spector et al., 2006).

An alternative interpretation sometimes endorsed in the literature is that RS is the consequence of the participants' expectations (Summerfield et al., 2008). This account is based on the finding that repetition probability can influence the size of RS for facial identity (Summerfield et al., 2008, see however Kaliukhovich and Vogels, 2011; Kovács et al., 2013), raising the possibility that RS is observed because the brain anticipates the repetition of the same face on consecutive trials, leading therefore to a relative decrease in cerebral activity when the expectation is met. Applied to the present study, this interpretation of the RS effect would imply that the participants predicted that a ball of the same size (big or small) would be grasped on consecutive trials –i.e. the brain attached higher prior probabilities to repeat than non-repeat irrespective of ball size. It is not obvious however why the brain would attach higher priors to repeat of the same ball size in our experiment. In face perception studies, a large (potentially infinite) number of different faces are shown to the participant, and therefore the presence of repeats is extremely salient. In our experiment in contrast, with only two possible ball sizes, the equal probability of repeat and non-repeat (Supplementary Table S9) corresponds exactly to what would be expected by chance, and repetition was therefore not a salient feature. Actually, it was not more likely for the brain of our participants to anticipate repeats than non-repeats (since both predictions were to be disconfirmed half the time).

One may want to argue instead that the participants predicted a big ball grasp for every stimulus since their task was to estimate the number of big ball grasps in a block. In that sense, the RS effect would arise because of a reduction in prediction error when a big ball was grasped for a second time. This interpretation however cannot account for the RS in our experiment because in fact, there were as many big ball grasps in the repeat and non-repeat conditions, and therefore the subject's expectations of a big ball grasp were fulfilled the same number of times in the repeat and non-repeat conditions (67% and 65% of the time respectively). Our finding therefore demonstrates that the relevant task parameter, the size of the ball being grasped by the actor, was encoded in this ventral region. The habituation of the BOLD signal to the repeated presentation of the same stimuli may result from a decrease of activity at the synapses, as suggested for instance by a recent study that found no significant repetition suppression in the spiking activity of premotor neurons, but a significant decrease in local field potentials (Caggiano et al., 2013). Therefore, it has been argued that RS might not safely localize a certain type of neural representation and might localize instead a region that receives synaptic input from the one that contains neurons that encode the researched stimulus dimension (Bartels et al., 2008; Ewbank et al., 2012). However, this reasoning holds only for long-range connections, while in fact the vast majority of cortical neurons (especially in high order areas) do not send their output to neurons in remote areas but to neighboring neurons at 1 or 2 mm distance (Amir et al., 1993; Douglas and Martin, 2004). In consequence, RS effects are much more likely to emanate from neurons located within the voxel showing the effect than from neurons located in distant areas (Malach, 2012). Furthermore, even if the effect was driven by neurons located in different voxels, the chances are still high that these neurons, at one synapse away from the location of the effect, would still fall within the ventral visual pathway. Our data therefore suggest that some voxels in the parahippocampal and lingual gyri of our participants contained neurons that encoded the size of the ball being grasped during the Full-View condition.

We further used Dynamic Causal Modelling to investigate the effect of detecting big ball grasps on the functional connectivity between pMNS nodes, and to test whether the exchange of information between the dorsal and ventral streams was modulated by ball-visibility. This effort was guided by recent recommendations that DCM should not attempt to exhaustively model all nodes and connections involved in a task, but generate the simplest caricature that can test the hypothesis at hand (Stephan et al., 2010). Accordingly, we concentrated on a model including a common visual input from parastriate cortex that is sent ‘downwards’ to the medial fusiform gyrus for analysing the shape of the balls and ‘upwards’ to the LOT cortex, then the aIPS and vPM cortex of the pMNS for kinematics analysis. This model led to several observations about the pMNS itself and about the changes in connectivity as a function of ball-visibility. First of all, we found that monitoring a grasping action increased the transfer of information from the aIPS to the vPM (independently of ball visibility). Importantly, we further discovered that participants whose aIPS->vPM connectivity was more strongly modulated when monitoring grasping actions were better at telling how often the big ball was being grasped in the Occlusion condition. Together, these findings support the notion that a portion of the parietal-premotor circuit specifically involved in transmitting information about grasping was particularly useful to solve the task when the target object was concealed from the view of the subject.

Another DCM finding regarding the functioning of the pMNS was that monitoring a grasping action generated a negative backward connectivity between vPM and aIPS, and even more so between aIPS and LOT. This significant negative backward influence is to our knowledge the first evidence, that processing the actions of others involves feedback connections similar to those thought to serve forward models during action execution (Gazzola and Keysers, 2009; Keysers, 2011; Keysers and Perrett, 2004; Kilner et al., 2007; Kokal and Keysers, 2010; Miall, 2003; Schippers and Keysers, 2011; Wolpert et al., 2003). According to these models, viewing a grasping action first triggers motor programs in a forward stream of information. These motor programs then generate expectations of what should be seen next through a backwards stream of information (from vPM to aIPS, and from aIPS to LOT), with these expectations being subtracted from the actual visual input to generate a prediction error (Gazzola and Keysers, 2009; Keysers, 2011).

Lastly, the DCM confirmed a direct effect of ball visibility on the synaptic activity in the fusiform gyrus, and showed that this was accompanied by a concurrent diminution of the functional connectivity with the dorsal pathway. The increase of firing rate in the medial fusiform gyrus is in good agreement with the hypothesis that in the Full-View condition the participants took advantage of the additional information provided by the presence of the balls on the table, as revealed by RS in the ventral visual stream. That connection strength between the ventral and dorsal route decreased as the object became visible, on the other hand, suggests a degree of independence between the two streams when it comes to processing object shape for perception, in agreement with the classic dissociation between the dorsal and ventral visual routes (Goodale and Milner, 1992; Milner and Goodale, 2008).

The model we fitted to the data consisted of just five nodes, and was only concerned with the right hemisphere activity, in line with recommendations to keep models as simple as possible (Stephan et al., 2010). Most certainly, more exhaustive models incorporating the (pre-)SMA, cerebellum, other visual brain regions (like the pSTS) and the other hemisphere would fit the data even better, but it is known that including additional nodes (and thereby improving overall model fit) does not increase the reliability with which specific connections can be quantified (Stephan et al., 2010). Comparing similar models drawn from the two hemispheres could be interesting also, but since in the Bayesian framework adopted by DCM model evidence is based on the data, and the data is different for the two hemispheres, comparing two models involving different nodes is unfeasible (Stephan et al., 2010). For the present, the results of our modelling suggest a decoupling of the ventral and dorsal pathways when the balls were visible in the Full-View condition. The results further show that monitoring grasping actions changes the functional connectivity between aIPS and vPM, even when it is not associated with more activity in the system. In addition, we discovered that the modulation by our task of the rate of information transfer from aIPS to vPM was positively correlated with the performance of the participants in the Occlusion condition, strengthening the conclusion that pMNS computations were essential when anticipating big ball grasps could only be achieved by paying close attention to the hand/fingers kinematics.

Watching someone else performing an action is associated with increased activity in multiple cerebral areas including the lateral occipito-temporal cortex and the posterior superior temporal sulcus specialized in the processing of body shape and motion, as well as areas along the intraparietal and postcentral sulci, and the premotor cortex (see Fig.2). Increases of activity in the inferior parietal and ventral premotor cortex during action perception are often attributed to the firing of mirror neurons. One hypothesis has been that the human putative mirror neuron system is crucial for extracting the goal of observed actions (Fogassi et al., 2005; Gazzola and Keysers, 2009; Gazzola et al., 2007; Hamilton and Grafton, 2008, 2006; Keysers, 2011, 2009; Rizzolatti and Craighero, 2004; Thioux et al., 2008; Umiltà et al., 2001). This hypothesis is consistent with the fact that many mirror neurons in these parietal and premotor regions encode executed and observed actions in an abstract format, which corresponds to the goal of the action rather than the precise body kinematics (Fogassi et al., 2005; Rizzolatti and Craighero, 2004; Umiltà et al., 2008, 2001). It is also supported by several studies showing that (virtual) lesions to the inferior parietal and premotor cortex are associated with impairments on some action comprehension tests (Fazio et al., 2009; Kalénine et al., 2010; Pazzaglia et al., 2008a, 2008b; Pobric and Hamilton, 2006). However, the pMNS cannot function in isolation when it comes to understanding the intention behind someone else’s action. There are for instance many situations where the same goal can be achieved by different means, and more importantly, the same action can, in some circumstances, serve different goals (Kilner, 2011). Furthermore, brain lesions to the parietal and premotor circuits can compromise the ability to prepare and execute an action, while leaving unimpaired the ability to understand the same action performed by someone else (Mahon and Caramazza, 2005). It seems therefore that action comprehension can be achieved without re-activating the neuronal circuits involved in executing the same action.

One important question in the literature therefore remained: what are the circumstances under which the re-activation of parietal and premotor circuits is most important for making sense of others’ actions. Several empirical and theoretical studies suggest that computations taking place on the dorsal route can be useful when the outcome of an action needs to be anticipated by a few hundreds of milliseconds (Cattaneo et al., 2007; Fogassi et al., 2005; Keysers, 2011; Kilner et al., 2004; Kokal and Keysers, 2010; Schubotz, 2007; Schubotz and von Cramon, 2004; Schütz-Bosbach and Prinz, 2007; Urgesi et al., 2010; Wilson and Knoblich, 2005). Here we tested the hypothesis that pMNS computations are crucial when anticipating the outcome of someone else’s action requires paying attention to body kinematics because part of the visual information is not available. In line with our hypothesis, we found that when the target object was concealed from the view of the participants (1) activity increased on the dorsal stream of action execution, (2) these increases of activity remained significant after controlling for task difficulty, and (3) the ability to detect the size of the object being grasped was higher in participants showing stronger modulation of the information transfer from aIPS to vPM while watching grasping actions. We also predicted that cerebral areas outside the pMNS would come into play if the objects were fully visible. In accord with this prediction, (1) the performance improved when the balls were visible although the same amount of information about body kinematics was available as in the Occlusion condition, (2) activity increased in the medial fusiform and parahippocampal gyri on the ventral visual pathway, (3) repetition suppression revealed that the size of the ball being grasped was encoded in this region of the ventral visual pathway if and only if the balls were visible to the participant, and (4) viewing the object on the table had a direct effect on the synaptic activity in this ventral region, and the transfer of information towards the dorsal stream was diminished.

Our study therefore emphasizes that action perception, like object perception, requires the processing of multiple attributes of the scene that are processed in parallel by different visual pathways. If a visual analysis of the objects in the environment is informative, relatively more of goal identification involves visual and spatial brain regions (see Kilner, 2011, for a similar proposal). This was the case in the Full-View condition in which seeing what object was closest to the hand at the end of the movie could help solve the task. If such a simple analysis does not help to identify the goal of the observed action reliably, for instance because critical parts of the scene are not visible, then the brain relies more on brain regions involved in executing similar actions and the functional connectivity within this information stream predicts performance. This conclusion is in good agreement with the results of an rTMS study in which the authors showed that phoneme discrimination speed was slowed down by virtual lesion to the premotor mouth area, if the discrimination had to be made in a noisy environment, but not when the noise was removed and the signal was clear (D’Ausilio et al., 2012). Our results are also compatible with those of a recent rTMS study showing that perturbing the computations taking place in the visual cortices during action perception is associated with increased activity in the pMNS (Avenanti et al., 2013). The brain utilizes all the information available to interpret the actions performed by others. If part of the information is missing, the brain tries to compensate for the missing bit by relying on the other information available to interpret the action (or in this case, to anticipate which object was going to be grasped). This interpretation brings up a prediction that could be tested in patients with lesions to the fusiform/lingual gyrus and the parahippocampal place area: in such patients, brain activity in the pMNS and performance should not differ between the Occlusion and Full-View condition, because they cannot harvest the additional non-kinematic information we propose to cause the increase in performance and the relatively lower pMNS activity in our participants. Brain lesions affecting the ventral pre-motor cortex on the other hand, should have a detrimental effect on the Occlusion condition and on any other task requiring a detailed analysis of body parts kinematic (Candidi et al., 2008; Kalénine et al., 2010; Pazzaglia et al., 2008b; Pobric and Hamilton, 2006), while leaving the performance on the Full-View condition relatively unimpaired.