Action observation circuits in the macaque monkey cortex

Abstract

In both monkeys and humans the observation of actions performed by others activates cortical motor areas. An unresolved question concerns the pathways through which motor areas receive visual information describing motor acts. Using functional Magnetic Resonance Imaging (fMRI), we mapped the macaque brain regions activated during the observation of grasping actions, focusing on the superior temporal sulcus region (STS) and the posterior parietal lobe. Monkeys viewed either videos with only the grasping hand visible or videos with the whole actor visible. Observation of both types of grasping videos activated elongated regions in the depths of both lower and upper banks of STS, as well as parietal areas PFG and anterior intraparietal (AIP). The correlation of fMRI data with connectional data showed that visual action information, encoded in the STS, is forwarded to ventral premotor cortex (F5) along two distinct functional routes. One route connects the upper bank of the STS with area PFG, which projects, in turn, to the premotor area F5c. The other connects the anterior part of the lower bank of the STS with premotor areas F5a/p via AIP. While the first functional route emphasizes the agent and may relay visual information to the parieto-frontal mirror circuit involved in understanding the agent's intentions, the second route emphasizes the object of the action and may aid in understanding motor acts with respect to their immediate goal.

Introduction

The ability to understand actions performed by others is a fundamental aspect of social behavior. It is therefore no surprise that the topics of action observation and recognition have attracted a strong interest within the cognitive neurosciences. The work of Perrett and others showed an unequivocal role for the monkey anterior superior temporal sulcus (aSTS) in the visual analyses of actions performed by others (Bruce et al., 1981; Perrett et al., 1985, 1989, 1990, Oram and Perrett, 1994). Neurons in the aSTS respond to a wide variety of biological movements, including walking, bending the torso, head turning, arm movements and goal-directed hand actions (Perrett et al., 1989, 1990; Jellema et al., 2000).

The topic of action observation/recognition received increased attention after the discovery of mirror neurons in ventral premotor cortex of the monkey (area F5), a class of neurons discharging both when a monkey performs a goal-directed motor act and when it observes another individual performing the same or a similar motor act (Di Pellegrino et al., 1992; Gallese et al., 1996, Rizzolatti et al., 1996). Recently, Nelissen and others (2005), using fMRI, discovered that the convexity of F5 (F5c) responded to the observation of actions performed by others provided the agent was visible in the video, while the parts located in the posterior bank of the inferior arcuate sulcus (F5a and F5p) were also activated when only the hand and forearm performing the action were visible.

A fundamental, still unsolved question concerns the brain areas and connections conveying the visual descriptions of the observed actions from the aSTS to the premotor cortex (F5). The available anatomical data indicate that there are no direct anatomical connections between these two regions (Matelli et al., 1986; Ghosh and Gattera, 1995). It is generally accepted that the inferior parietal lobe (IPL), which is connected with both STS and F5, is the anatomical link between the two regions (Rizzolatti et al., 2001). This view is supported by single neuron studies reporting mirror neurons in IPL (PFG, Fogassi et al., 2005). Furthermore, recent imaging studies also point to a role of AIP in encoding hand grasping performed by others (Peeters at al., 2009, Rizzolatti and Sinigaglia, 2010).

The aim of the present study is threefold. First, to investigate which cortical areas of monkey STS and parietal cortex are involved in the visual analysis of the observed actions. Second, to assess to what extent these visual activations depend on the presence of the agent in the action videos. Third, to investigate the possible functional routes linking anterior STS with F5 by combining the results of the functional MRI experiments involving grasping observation with data from neural tracers injected into the parietal and frontal areas responsive to action observation.

The combination of functional and anatomical data allowed us to define two main functional routes by which the visual information about others' actions is projected from the STS, via the parietal cortex, to premotor area F5. One route uses PFG as an intermediate station, the other area AIP.

Experimental procedures

Subjects

The experiments were performed on five macaque monkeys (Macaca mulatta, 3–6 kg, 4–7 years of age), including four male (M3, M5, M6 and M15) and one female (M13). All animal care and experimental procedures met the national and European guidelines and were approved by the ethical committee of the K.U. Leuven medical school. The details of the surgical procedures, training of monkeys, image acquisition, eye monitoring and statistical analysis of monkeys scans have been described previously (Vanduffel et al., 2001; Fize et al., 2003; Nelissen et al., 2005, 2006), and will be described here only briefly.

During the experiments the monkeys sat in a sphinx position in a plastic monkey chair directly facing the screen. In the training and scanning sessions they were required to maintain fixation within a 2×2° window centered on a red dot (0.35 × 0.35°) in the middle of the screen. Eye position was monitored at 50Hz or 120 Hz (in later experiments) via the pupil position and corneal reflection (Iscan). During scanning the fixation window was slightly elongated in the vertical direction to 3°, to accommodate an occasional artifact on the vertical eye trace induced by the scanning sequence. The monkeys were rewarded (with apple juice) for fixating the small red dot within the fixation window for long periods (several minutes), while stimuli were projected in the background. Before each scanning session, a contrast agent, monocrystalline iron oxide nanoparticle (MION or Sinerem®), was injected into the femoral/saphenous vein (6-11mg/kg).

Visual stimuli

Visual stimuli were projected from a Barco 6300 liquid crystal display projector (1024 × 768 pixels; 60 Hz) onto a screen 54 cm from the monkeys' eyes. Unless otherwise mentioned, all tests included a simple fixation condition, in which the fixation target was shown on an empty gray screen, as baseline.

In Experiment 1, three monkeys (M3, M5 and M6) were scanned in a 1.5T scanner and one animal (M15) in a 3T scanner with custom-made 8-channel monkey Rx coils to achieve higher signal to noise ratios in the anterior STS. Stimuli (Figure 1) consisted of video-clips showing a hand (and forearm) grasping and picking up an object (‘isolated hand’ action, Supp. Video S1, 13 by 16° size) and video-clips showing a full view of a person performing the same actions (‘acting person’, Supp. Video S2, 18 by 20° size). Four different ‘isolated hand’ action video sequences were used: a male or female hand grasping and picking up a candy (precision grip) or a ball (whole hand grasp). These video sequences lasted 3.3 s and 11 randomly selected sequences were presented in a 36 s block. Six different ‘acting person’ video sequences, lasting 6s, were presented in random order in a block: a man or woman grasping and picking up an apple (whole hand grasp), a piece of carrot or a peanut (precision grip). The longer duration of the ‘acting person’ videos was due to the larger number of static frames at the beginning and end of the videos. The duration of the actual hand movement period was similar in the two movies: 2.1 s on average in the ‘hand action’ videos and 2.7 s in the ‘acting person’ videos. Two types of control stimuli were used: a) static single frames of the action videos, one from the middle of the video sequence when the hand is about to grasp the object and one from the end of the video sequence when the object has been picked up; b) scrambled videos produced by phase scrambling each frame of the video sequences (Supp. Video S3). Static stimuli were refreshed every 3.3 s or every 6 s by showing a frame selected from one of the four ‘isolated hand action’ videos or from one of the six ‘acting person’ videos respectively. The acting person videos differed from the static controls by the 2.7 s dynamic period, as shown by the strong differential activity in MT/V5 (see results).

Figure 1. Stimuli.

A,B. Frames taken from the isolated hand action (A) and person grasping videos (B), showing a male hand (A) or a female person (B) grasping an object. Monkeys fixated the small red spot at the center of the frames. Grasping actions were confined mostly to the central and right visual field.

The fixation condition used a white or green background, matched to those in the ‘acting person’ and ‘hand action’ videos, respectively. In each case, 5 different runs with different orders of conditions were used. Within a run, the order remained constant, and conditions were repeated once. The stimulus conditions are the same as those used in the Nelissen et al. (2005) study. For two monkeys (M5 and M6) we used the parietal and STS data obtained in this earlier study, while two additional monkeys (M3, M15) were scanned.

In Experiment 2, (M6, M13) we performed an additional action control test that contrasted the responses to goal-directed (object present) and mimicked (no object present) ‘isolated hand’ actions to either a translating hand (Supp. Video S4) or to a static single frame. The goal-directed hand action video-clips were the same as those used in Experiment 1. The mimicked hand action clips showed an isolated hand mimicking a grasping action (Supp. Video S5). The translation controls were introduced because many regions in the middle and posterior STS are involved in visual motion analysis (Zeki, 1974; Maunsell and Van Essen, 1983; Desimone and Ungerleider, 1986; Vanduffel et al., 2001; Nelissen et al., 2006). The action and static stimuli conditions were the same as those in Experiment 3 of Nelissen et al. (2005). The monkeys scanned in that experiment were different animals from those employed in the present study.

In the analyses of both experiments, regions of interest (ROIs) were defined in the anterior portion of STS and in the IPS using the hand action test of Experiment 2 of Nelissen et al. (2005) as an independent action localizer test (monkeys: M1, M3, M5). This action localizer, also used in Nelissen et al. (2006), consisted of videos showing ‘isolated hand’ actions (3.3 s for 1 cycle; 4 different video sequences presented in random order) and a static (single frames from the middle of the videos shown for 36 s) and scrambled control.

Scanning

Functional time series (runs) in Experiments 1 and 2 and in the action localizer test consisted of gradient-echo echoplanar whole-brain images acquired on a 1.5 T Siemens Sonata with a surface coil positioned over the head (1.5 T; repetition time (TR) 2400 ms; echo time (TE) 27 ms; 32 sagittal slices, 2 mm isotropic voxels). For one animal (M15) functional time series in Experiment 1 consisted of gradient-echo echoplanar whole-brain images acquired on a 3T Siemens TIM Trio with a custom-built 8-channel receive coil (TR 2000 ms, TE 17 ms, 30 horizontal slices, 1.5 mm isotropic voxels).

In the 1.5T scan sessions of Experiment 1, the ‘isolated hand action’ videos (and controls) and the ‘acting person’ videos (and controls) were presented in alternate runs. In addition, after six such runs, in which the monkey was passive, an active run was introduced in which the monkey had to detect the change in orientation of a small bar shown in the center of the screen while the action stimuli were presented in the background. The aim of this run was merely to enhance the alertness of the subject and these data were not analyzed. After this active run, another set of six passive runs were collected, with the cycle of six plus one run being repeated once or twice in a session. In the 3T scan sessions of Experiment 1 and in Experiment 2, only passive runs were collected, with little difference in the results.

For each monkey an anatomical (three-dimensional magnetization prepared rapid acquisition gradient echo, MPRAGE) volume (1×1×1 mm voxels) was acquired under anesthesia in a separate session.

Volume-based data analysis

Data were analyzed using SPM5 and Match software. Only runs in which the monkeys held fixation within the window for >85% of the time were analyzed. In these analyses, realignment parameters, as well as eye movement traces, were included as covariates of no interest to remove eye movements and brain motion artifacts. Spatial preprocessing consisted of realignment and rigid co-registration with a template anatomy (M12, corresponding to M1 in Ekstrom et al., 2008). To compensate for echo-planar distortions in the images and for inter-individual anatomical differences, the functional images were warped to the template anatomy using the non-rigid matching software, BrainMatch (Chef d'Hotel et al., 2002). The functional volumes were then re-sliced to 1 mm³ isotropic and smoothed with an isotropic Gaussian kernel (FWHM = 1.5 mm). Group analyses (fixed effects) were performed with an equal number of volumes per monkey, supplemented with single subject analysis, and the level of significance was set at p<0.05 corrected (family wise error) for multiple comparisons, unless stated otherwise.

Region of Interest (ROI) based analysis

Sixteen different ROIs, 10 within the STS and 6 in the parietal cortex, were defined onto the anatomical template (M12, Fig.2).

Figure 2. Localization of the IPL, IPS and STS ROIs.

A. Lateral view of a left hemisphere indicating the position of the four inferior parietal lobule (IPL) ROIs. From rostral to caudal: areas PF, PFG, PG and Opt. B,C. Flattened representation of the intaparietal sulcus (IPS) (B) with adjoining cortical convexity, and of the superior temporal sulcus (STS) (C), indicating position of areas PF, PFG, PG and Opt on the IPL convexity, area AIP and anterior portion of area LIP (LIPa) in the IPS, and MT/V5, MTp, MSTd, FST, LST, STPm, lower bank 1 (LB1), LB2, upper bank 1 (UB1) and UB2 in the STS. Thick solid lines indicate the lips and fundus of the sulcus, dotted lines coronal levels in one mm steps. D. Location of the ROIs indicated on the MR template anatomy of M12 in 6 different coronal slices (AP levels +12, +8, +2, -3, -8 and -14 mm respective to the interaural plane) as indicated by dashed lines in A. STS = Superior Temporal Sulcus, IPS = Intraparietal Sulcus. STPa encompasses the portion of the upper bank from +6 to +20 mm anterior to the interaural plane (Jellema et al. 2000, 2006; Baraclough et al. 2009).

The 10 STS ROIs (Fig.2C and D) consisted of 6 motion sensitive regions defined in Nelissen et al. (2006), plus 4 additional regions defined by the action localizer test. The motion sensitive regions are located in the caudal (MT/V5, MTp, dorsal part of MST- MSTd-, FST) and middle portion of the STS (LST, and STPm). These latter two regions are considered here part of the anterior STS (aSTS). The MTp region corresponds to the MSTv region of Nelissen et al. (2006). Here the Kolster' terminology (Kolster et al., 2010) is used, as a way of reconciling the motion sensitive regions defined by Nelissen et al. (2006) with the retinotopic results of Kolster et al. (2009). Indeed the region we refer to as MTp appears to be located more posteriorly than the MSTv described by Kolster et al. (2009), which shared a border with FST. The action localizer test yielded 2 separated activation sites in the anterior portion of the lower bank of the STS tentatively designated lower bank 1 (LB1, 6 to 10 mm anterior to interaural plane) and lower bank 2 (LB2, 11 to 15 mm anterior to interaural plane). These regions responded significantly to videos showing hand grasping actions (compared to static and scrambled controls). The action localizer test did not yield such activation sites in the corresponding portion of the upper bank. In order to have an equally sensitive analysis of the two banks of the STS, we tentatively defined two upper bank regions, upper bank 1 (UB1) and upper bank 2 (UB2), at the same anterior – posterior level as LB1 and LB2. Both UB regions are part of STPa (Bruce et al., 1981; Cusick et al., 1997; Oram and Perret, 1994; Jellema et al., 2000).

Six ROIs were defined in the parietal lobe (Fig.2 A, B and D), four in the inferior parietal lobule convexity (area PF, PFG, PG and Opt), following Gregoriou et al., 2006 and two in the lateral bank of the intraparietal sulcus (AIP and anterior portion of LIP). Area AIP was delineated on the basis of previous single cell (Murata et al., 2000) and fMRI studies (Durand et al., 2007). The action localizer test also yielded a significant, active site in the lateral bank of the IPS (Fig. 2B), in the anterior portion of the IPS region active during the execution of visual saccades (Wardak et al., 2010), which was termed LIPa (anterior portion of LIP), following Durand et al. (2007).

Since the grasping actions in the videos were performed near the fovea and mostly within the right visual field, the ROI analysis was restricted to the left hemisphere. For number of voxels and proportion of visually responsive voxels in each ROI see Supplemental Material.

Region-of-interest analysis was done using MarsBaR (version 0.41.1, http://marsbar.sourceforge.net). The significance threshold for the t-tests was set at p<0.05 one-tailed, Bonferroni corrected for the number of ROIs (Check). A ROI was considered to be significantly activated only if the action observation condition activated the ROI more than any of the control conditions (including the fixation baseline) at p<0.05 corrected, both in the group and at least 2/3 of the monkeys (i.e. 2/3 in experiment 1 and 2/2 in experiment 2).

The correlation analysis for heterogeneity of ROIs uses the method developed by Peelen et al. (2006). For each voxel of the ROI, we plot the differential activity in one contrast (e.g. isolated hand action– fixation only) as the function of differential activity in the second contrast (e.g. acting person – fixation only). These correlations were calculated using only the voxels that were activated in the subtraction defining the abscissa, e.g. acting person – fixation. Including both the deactivated and the activated voxels would have inflated these correlations.

Flattening procedure

To minimize distortions due to whole hemisphere flattening, the procedure developed by Durand et al (2007) was used to flatten the STS, IPS and adjoining IPL convexity and finally the inferior ramus of the arcuate sulcus (IAS). The trajectory in the flatmaps of the iso-AP levels corresponding to coronal sections depends on the overall shape of the sulcus. In the IPS (Fig.2B), iso-AP levels display a V shape, reflecting the increase in the depth of the IPS at more caudal levels. In the anterior STS, iso-AP lines run almost perpendicular to the elongated flattened shape of the STS (Fig.2C). Since the STS widens caudally, the posterior iso-AP lines there are strongly bent. Hence, MTp, for instance, might appear to be located completely posterior to MT/V5 on the STS flatmap, while it is actually located at similar anterior-posterior level as MT/V5.

Correlation with connectional data

To identify the possible pathways conveying action-related visual information from the STS to frontal lobe areas, functional MRI data were combined with data from nine representative macaque monkeys (five Macaca nemestrina- Cases 14, 20, 26, 27, 30, three Macaca fascicularis- Cases 13, 29, 36, and one Macaca fuscata- Case 17) in which neural tracers were injected into areas PFG (Case 13l, 27r, 29r), AIP (Cases 14r, 17r, 20l), 45B (Cases 26l, 30r, 36l) and F5a (Case 30r). All the cases of injections in areas PFG, AIP and 45B have been already presented in previous connectional studies to which the reader is referred also for details on surgical, histological and data analysis procedures (Rozzi et al., 2006; Borra et al., 2008; Gerbella et al., 2010). Data from the tracer injection in F5a have been presented only in abstract form.

In Cases 13l, 27r and 29r, peroxidase-conjugated wheat germ agglutinin, (WGA-HRP, 4%, one injection, 0.1 μl), cholera toxin B subunit, conjugated with Alexa 488 (CTBgreen, 1% in distilled water, two injections 1 μl each) or Fast Blue (FB, 3%, one injection, 0.2 μl) were injected into PFG. In Cases 14r, 17r and 20l, microruby (MR, 10% phosphate buffer 0.1 M, pH 7.4, two injections, 1 μl each), WGA-HRP (seven injections, 0.1 μl each) and Diamidino Yellow (2%, one injection, 0.2 μl) were injected into AIP using similar procedures. In Cases 26l, 30r and 36l, DY (one injection, 0.2 μl), FB (one injection, 0.2 μl) and FB (one injection, 0.2 μl) were injected into area 45B. Finally, in Case 30r, in addition to the FB injection into area 45B, DY (one injection, 0.2 μl) was injected into F5a.

In all cases, the distribution of retrograde labeling was plotted in sections every 600 μm and analyzed qualitatively and quantitatively, as the number of labeled neurons found in a given cortical subdivision outside the injected area, in percent of the overall labeling observed in the injected hemisphere. The distribution of the labeling observed after tracer injections in the same area was remarkably consistent across different cases and the quantitative analysis showed quite similar percent distributions of the labeling in the different animals (Rozzi et al., 2006; Borra et al., 2008; Gerbella et al., 2010).

For the purposes of the present study, the distributions of the retrograde labeling observed in the ipsilateral STS in all PFG, AIP and 45B injection cases and in the ipsilateral IPS in Case 30r (FB and DY) were re-analyzed as follows. The distributions were first visualized in two-dimensional (2D) reconstructions of the STS and IPS as described in Matelli et al. (1998). For purposes of comparison all reconstructions of injected hemispheres were shown as left hemispheres. These 2D reconstructions (flatmaps) were then warped to the flattened left STS and IPS isomaps. For the STS warping, reference points were placed every 2 mm along the lower lip, fundus and upper lip of the STS of both the input flatmap (tracer injection) and the reference flatmap (fMRI isomap). In each STS flatmap, AP 0 was set at the level of the posterior commissure. For the IPS warping, reference points were placed every 2 mm along the lower lip, the middle of the lateral bank and the fundus of the IPS of both the input flatmap (tracer injection) and the reference flatmap (fMRI isomap). The deformation of the input flatmap to conform to the reference flatmap was based upon a linear interpolation of a triangular mesh, formed by the reference points and the four corner points of the image, using the Matlab ‘griddata’ function (Watson, 1992) which is based on a Delaunay triangulation of the data.

To analyze the consistency of the labeling in a cortical region the following procedure, a method inspired by that of Borra et al. (2008), was used. The procedure introduces a slight smoothing of the labeling to overcome individual differences. Each dot of label was replaced by a disc of 100% luminance with a 10 pixel radius (roughly 600 micron in the flatmap, approximately the distance between sampled sections) and these luminance discs were added for the STS flatmap of a given individual. The composite luminance distribution was thresholded just above 100%, to reject isolated labeled neurons in the connectivity map of a single animal. The overlap between maps from different subjects was calculated and color coded. The results did not depend much on the exact size of the disc, nor on the threshold as long as it exceeded 100%. The fMRI data were also smoothed in the flatmaps, for the same reason, by replacing each local maximum with a disc of 20 pixels radius in the flat map. A union of the discs was computed for each animal and the overlap between individual activation patterns was calculated and color coded for each point of the flatmap.

Results

Cortical regions activated during action observation

The cortical regions activated by action observation are shown in figure 3. Figure 3B and D illustrate the activation sites for grasping performed by the ‘isolated hand’ (relative to static and scrambled controls), while Figure 3F shows those for grasping by the ‘acting person’. The network for hand action observation includes the STS, parietal, ventral premotor and lateral prefrontal cortices.

Figure 3. Overview of MR brain activations during the observation of grasping actions.

A. Top view of reconstructed (Caret) left hemisphere to illustrate locations of most lateral (x = -28 mm) and medial (x = -6 mm) sagittal slices shown in B. B. Statistical parametric maps (SPMs) for the contrast: hand action versus static control (p < 0.05, corrected), masked inclusively with the contrast hand action versus scrambled control, and with the contrast all stimuli versus fixation baseline (p < 0.001, uncorrected), overlaid on sagittal MR anatomical images of left hemisphere of M12. Group data of 3 monkeys (M3, M5 and M6; 1.5T). Numbers on each slice indicate x-coordinate (lateral to medial). C & E. Lateral view of reconstructed left hemisphere indicating the 6 different antero-posterior levels at which coronal slices are shown in D and F, respectively. D. SPMs for the same contrast as in B, but overlaid onto coronal anatomical sections of M12. Numbers on each slice indicate y coordinate (antero-posterior from interaural plane). Group data of 3 monkeys (M3, M5, M6). F. Statistical parametric maps for the same contrast shown in B and D, from a single subject (M15, 3T), overlaid on coronal sections of same anatomical template (M12). Numbers on each slice indicate y-coordinate. Abbreviations as in Fig. 2.

The actions were presented mostly in the monkey's right visual field (see Methods). It is therefore difficult to draw any conclusion regarding the lateralization of the action observation network in the monkey. However, as shown in Fig.3D and F, both STS and premotor cortex tended to respond bilaterally. The finding that the activation sites in the parietal cortex were more extensive in the left hemisphere than in the right, ie contralateral to the visual stimulation, is in agreement with previous results (Peeters et al. 2009).

Activation during grasping observation in the Superior Temporal Sulcus region

The ROI analysis of the two experiments showed that five of the ten STS ROIs were significantly more activated during the observation of grasping (isolated hand and active person) relative to the controls: MT/V5, FST, LST, LB2 and STPm (Table 1, Fig 4 & Suppl. Fig S1). In Experiment 1 the actions were contrasted with each of three control conditions (the two static and scrambled controls) and with the fixation baseline condition, while in Experiment 2 they were contrasted with the static and translation controls and the fixation baseline. Note that Experiment 2 shows that the five action responsive ROI are not merely motion sensitive regions. Table 1 lists the minimum t value obtained in the different contrasts in both experiments. This minimum value exceeds the significance threshold in all 5 ROIs mentioned above.

Table 1.

Minimum t value for the STS ROIs in the different experimental tests.

STS	MT/V5	FST	LST	LB1	LB2	MTp	MSTd	STPm	UB1	UB2
Exp.1 (1.5T) PA	11.58	20.12	17.99	8.22	7.08	-0.94	4.36	14.91	4.17	0.79
Exp.1 (3T) PA	16.08	31.83	19.18	1.20	19.65	2.93	9.40	23.06	-5.12	14.11
Exp.1 (1.5T) HA	15.45	15.75	16.81	7.89	6.65	5.29	7.04	20.04	2.69	2.65
Exp.1 (3T) HA	16.80	34.12	15.47	9.60	17.05	6.76	-1.42	36.00	-7.02	3.97
Exp.2 HA	9.96	13.13	10.95	7.75	9.34	-0.94	-3.06	10.03	2.58	3.62

Bold red numbers indicate significant activation: t exceeding 2.58 (p<0.05 corrected for 10 ROIs) in the group and in at least 2/3 animals; black numbers: non significant activation. PA: person acting videos; HA isolated hand action videos.

Figure 4. MR activations for grasping observation in the STS.

A & C. Percent MR signal change (baseline: fixation-only condition) during the observation of person grasping objects video-clips and their scrambled and static controls (see color inset) in 10 STS ROIs: MT/V5, FST, LST, LB1, LB2 in the lower bank, MTp in the fundus, and MSTd, STPm, UB1 and UB2 in the upper bank. Group data from 3 monkeys (M3, M5, M6), scanned at 1.5T (A) and data from single subject (M15) scanned at 3T (C). B & D. Percent MR signal change (baseline: fixation only condition) for observation of isolated hand grasping actions and scrambled and static controls (see color insets) in the same ROIs. Group data from 3 monkeys (M3, M5, M6) scanned at 1.5T (B) and data from single subject (M15) scanned at 3T (D). Asterisks indicate a significantly stronger (p < 0.05, corr.) response to action observation compared to all controls (in group and minimum of 2 out of the three single subjects). Error bars indicate variability over runs.

In experiment 1, the responses of MT/V5 to the acting person videos were much stronger than to either static condition (Fig. 4). This differential activity was very similar to that obtained with the isolated hand videos, clearly indicating that the main difference between the acting person videos and the static conditions is the motion sequence capturing the action.

In Experiment 2, we also presented video-clips of an isolated hand mimicking a grasp, without an object present (Suppl. Video S5). The activation elicited by the observation of hand action with and without objects was very similar in the STS ROIs, except for LB2, which exhibited a weak, non significant reduction in response for the mimicking conditions (Suppl. Fig. S1). LB2 is located near the region where Barraclough et al. (2009) observed stronger neuronal responses to goal-directed actions. No significant interaction between goal-directed and mimicked actions (compared to either their translational or static controls) was obtained in LB2 or in any other STS ROI.

All ROIs tested were located in the depth of the STS (Fig. 2). To complement the ROI analysis, we plotted all the local maxima (at p<0.001 uncorrected) of the four single subject SPMs for the contrasts acting person vs. the average of all three controls (Fig.5A) and isolated hand action vs. the average of all three controls (Fig.5B). The MR responses to action observation are indeed concentrated in the deeper portions of both banks. Most of the local maxima form a long ribbon starting at the caudal end of the lower bank STS (near MT/V5) and extending forward all the way up to the tip of STS. Comparatively few local maxima were located in the upper bank. Most STS regions yielded intermingled responses to both types of action videos.

Figure 5. Individual fMRI local maxima for observation of person grasping and hand grasping in STS.

A. Local maxima from all acting person observation tests (4 single subjects), overlaid onto the flattened representation of left STS. SPM local maxima are plotted from the contrast acting person versus the average of all three controls (scrambled, static middle and static end; p < 0.001, uncorr.). B. Local maxima from all isolated-hand action observation tests (same 4 single subjects), overlaid onto the flattened representation of left STS. Data from 10 different daily sessions are plotted (2 sessions each from M3, M5 and M6 and 4 sessions from M15). Location of the 10 STS ROIs are indicated with dashed lines. Darker grey area indicates fundus of the STS.

Activation during grasping observation in the IPL and IPS

The ROI analysis (Table 2, Fig 6, Suppl Fig 1) revealed that one parietal ROI, AIP, was significantly activated, using the criteria described for the STS, in the two experiments, for both types of action videos. Area PFG, in which visual responses to hand action have been reported (Fogassi et al 2005, Rozzi et al 2008) was also significantly activated in all these conditions, with the exception of the isolated hand tests at 1.5T in Experiment 1 and the comparison with translation control in experiment 2. This stands in sharp contrast to the other 3 IPL ROIs which were never activated. Finally, LIPa was significantly activated in most tests, except the isolated hand action at 3T.

Table 2.

Minimum t value for the parietal ROIs in the different experimental tests.

IPL/IPS	PF	PFG	PG	Opt	AIP	LIPa
Exp.1 (1.5T) PA	-0.32	2.96	-0.72	-1.08	4.9	9.43
Exp.1 (3T) PA	-	3.33	-11.93	-9.51	3.77	7.33
Exp.1 (1.5T) HA	-3.71	1.42	-2.33	1.17	4.48	11.85
Exp.1 (3T) HA	-	8.29	-13.10	-12.45	3.37	-4.59
Exp.2 HA	-1.95	-0.87	0.10	0.06	4.25	5.88

Bold red numbers indicate significant activation: t exceeding 2.41 (p<0.05 corrected for 6 ROIs) in the group and in at least 2/3 animals; black numbers: non significant activation. At 3T none of the PF voxels was visually responsive. PA: person acting videos; HA isolated hand action videos.

Figure 6. MR activations for grasping observation in IPL and IPS.

A & B. Percent MR signal change (baseline: fixation-only condition) for observation of acting persons (A) and observation of isolated hand actions (B) and their scrambled and static controls in parietal ROIs: PF, PFG, PG and Opt on the IPL convexity and AIP and LIPa in the IPS. Group data from 3 monkeys (M3, M5, M6). C & D. Percent MR signal change (baseline: fixation-only condition) for observation of acting persons (C) and observation of isolated hand actions (D) and their scrambled and static controls in IPL and IPS ROIs. Single subject data (M15; 3T). No visually driven voxels were present in PF, compared to an average of 29% of the voxels in PFG (see suppl. text). Asterisks indicate a significantly stronger (p < 0.05, corr.) response to action observation compared to all controls. Error bars indicate variability over runs.

In experiment 2 (Suppl. Fig.S1), area AIP showed a significant interaction between isolated hand grasping objects and mimicked hand actions, relative to either their translation or static controls (t= 3.57 and t=3.53 respectively, p<0.05 corrected for number of ROIs). Hence, AIP responded more strongly to the observation of the goal-directed (object present) than the mimicking (object absent) hand actions and this effect was specific for the action condition.

Anatomical connections between STS and parietal cortex

To investigate how visual areas processing grasping actions are connected with the different sectors of F5, we examined the anatomical connections of the key parietal regions active during the observation of hand grasping, PFG and AIP (Rozzi et al., 2006; Borra et al., 2008), and compared them to the present fMRI data, which were acquired in different subjects. The optimal approach would be to use the same animals, which is not feasible. Training a monkey for awake fMRI experiments is a long and intensive process, and a well-trained fMRI monkey can participate in several studies during an extended period of time. This renders it impractical to sacrifice these valuable animals for the tracer studies. As an alternative, we tested several macaque monkeys in the fMRI and anatomical experiments and searched for activation and projection sites that remained consistent across the majority of these subjects. Because evidence showed that PFG and AIP are connected with F5 (Luppino et al., 1999; Rozzi et al., 2006, Borra et al., 2008), we focused on the connections of these two parietal areas with STS.

Fig.7B shows that tracer injections into PFG (Fig.7 A) produced labeling concentrated in three sectors of the ipsilateral upper bank of the STS (Fig.7B): area MSTd (Nelissen et al., 2006), STPm and UB1. Note that the injections in all three hemispheres produced a similar labeling distribution, as indicated by the overlap of green, red and blue pixels (Fig. 7B). This was confirmed by the consistency analysis (Fig. 8A) in which these three sectors appear as white. The region of overlap in STPm is small because the label in one animal (blue label in Fig. 7B) was located slightly more anterior to that in the two other animals (red and green label in Fig. 7B). This is not to say that no other STS region is connected with PFG, as a few other regions, located mainly in the upper bank, are also labeled in a single animal. Even lowering the criterion to consistent labeling in two animals (light grey) would not substantially alter our conclusions, although the STS regions connected with PFG would be somewhat larger.

Figure 7. Distribution of STS labeling following tracer injections into area PFG.

A. Location of the PFG injection sites shown on the lateral view of the hemisphere (left side) and in a coronal section (right side) in three representative cases. B. Distribution of retrograde labeling in STS observed in the three cases, overlaid onto a flat-map of STS. For the sake of comparison all the reconstructions of the injected hemispheres in this and the subsequent figures are shown as left hemispheres. Color code corresponds to the three cases from A. STS ROIs indicated with dashed lines. Cg= cingulated sulcus; CS= central sulcus; IAS= inferior arcuate sulcus; LF= lateral fissure; Lu= lunate sulcus; PS= principal sulcus; SAS= superior arcuate sulcus.

Figure 8. Combination of fMRI grasping observation data in STS and anatomical connections between STS and parietal and prefrontal cortex.

A, B, C. Composite views the STS showing the consistency across animals of the labeled regions following injections in PFG (A), AIP (B) and 45B (C). Labeling dots shown in Figure 9-11 were smoothed (see methods) and superimposed onto the flattened STS fMRI template. STS regions consistently labeled in all 3 cases are shown in white, labeling present in 2 cases is shown in light grey and finally, labeling found in only one animal is shown in dark grey. D and E. Composite views of STS showing the consistency across animals of the regions activated by observation of isolated hand actions (D) and acting person (E). Functional MRI local maxima (action vs 3 controls, p < 0.001, uncorr.) from Fig. 6 were smoothed (see methods) and overlaid onto the STS template. Color code indicates consistent activations in all 4 fMRI subjects (red = 1 subject, orange = two subjects, yellow = three subjects; white = four subjects). Calibration bars indicate 20 and 40 pixels in A-C and D-E respectively. These values correspond to the diameters of the discs used to smooth the individual data (see methods). The arrow in A points to the small overlap region of injections in all three hemispheres in STPm.

Of these three labeled sectors only STPm showed a consistent activation during grasping action observation, being activated in 3 out of 4 animals (yellow voxels) for the observation of both types of hand actions (Fig. 8D,E). On the contrary, activation of MSTd during action observation was less consistent: only in one animal and for one type of video (Fig.8D,E). This suggests that STPm is the STS sector involved in the relaying of information concerning grasping actions to PFG.

Injections into area AIP (Fig. 9A) yielded a more widespread ipsilateral STS labeling (Fig.9B) than that observed following injection into PFG. In all three cases labeling was found in three sectors of the STS: one sector at the level of MSTd, extending further caudally, one in the middle portion of the upper bank near the fundus (anterior to STPm), extending into the lower bank, and one rostrally in LB2 (Fig.9B), extending more laterally. The consistency analysis (Fig. 8B) however, indicated that consistent labeling in all three subjects was only observed in LB2 and more laterally in the lower bank near the lip between +8 and +10.

Figure 9. Distribution of STS labeling following tracer injections into area AIP.

A. Location of the AIP injection sites shown on the lateral view of the hemisphere (left side) and in a coronal section (right side) in three representative cases. B. Distribution of retrograde labeling in STS observed in the three cases, overlaid onto a flat-map of STS. Color code corresponds to the three cases from A. STS ROIs indicated with dashed lines. Abbreviations and conventions as in Fig 7.

Of these two regions only one, LB2, was consistently activated by action observation. This was particularly true for the observation of isolated hand actions (Fig. 8D) to which LB2 responded significantly in all four animals. The sector lateral to LB2 and MSTd were hardly activated by action observation. Conversely most regions consistently activated by action observation were not labeled after AIP injections (STPm, LB1/LST border, FST and MT/V5). Thus, considering the fMRI data, LB2 appears to be the major source supplying AIP with visual information about grasping actions. This is not to say that there are no other connections between STS and AIP but those are unlikely to play a major role in action observation signals. This functional segregation amongst the anatomical connections supports the notion of a specific functional route by which we mean an anatomical connection that transmits specific types of information, here visual information on observed grasping. Other functional routes are likely to link the STS with AIP. For example the more lateral regions of the lower bank near LB2 belong to infero-temporal cortex and are close to region TEs where Janssen et al. (2000) recorded higher order disparity selective neurons. This lateral route may therefore have a different functional role, related to 2D and 3D shape (Borra et al., 2008). The same distinction between functional route and anatomical connection also applies to the STS-PFG connections described earlier. Indeed MSTd houses neurons selective for translation as well as optic flow components rotation or expansion (Tanaka et al., 1986; Lagae et al., 1994). Thus the MSTd-PFG connection may be a functional route, different from the STPm-PFG action observation route, sending translation and optic flow signals to PFG (Fig. 8A).

Anatomical connections between STS, parietal and frontal cortex

Since we had found in earlier studies that a third frontal region - area 45B - was activated by action observation (Nelissen et al., 2005) and that this region is connected with both STS and IPS (Gerbella et al., 2010), we also investigated the connections of area 45B with our functional regions in both IPL and STS. Fig. 10B presents the labeling pattern in the ipsilateral STS following injection into 45B (Fig.10A). The strongest labeling was found in the fundus and lower bank, at approximately the location of the border between LST and LB1, extending laterally into TE towards the lip of the lower bank. This is confirmed by the consistency analysis (Fig. 8C) which indicates a large region of projection to 45B in LB1, including the borders with UB1 and LST, and extending laterally in the lower bank. In addition, a small consistent region seems also to be present anterior to LB2. Note that the labeling pattern after 45B injection appears in respects complementary to that of the AIP and PFG injections, except for some overlap very laterally in the lower bank at +9mm.

Figure 10. Distribution of STS labeling following tracer injections into area 45B.

A. Location of the 45B injection sites shown on the lateral view of the hemisphere (left side) and in a coronal section (right side) in three representative cases. B. Distribution of retrograde labeling in STS observed in the three cases, overlaid onto a flat-map of STS. Color code corresponds to the three cases from A. STS ROIs indicated with dashed lines. Abbreviations and conventions as in Fig. 7.

LB1 and a small region just anterior to LB2 were consistently activated by action observation, both isolated hand actions (four animals, Fig. 8D) and acting persons (two animals, Fig. 8E). The other STS regions consistently activated by action observation have little labeling after 45B injections (Fig. 8C), except perhaps the rostral border of FST. Thus, action observation information from STS can reach 45B, directly, through connections arising mainly from LB1 and the rostral part of LST, and possibly supplemented by rostral FST and the rostral tip of the lower bank.

Finally, Fig. 11 shows the connections of area 45B and area F5a with the areas located in the ipsilateral IPS. The strongest labeling following injection into F5a is found in area AIP (Fig.11B, red labeling), while following injection into area 45B such labeling is located in the more posterior, oculomotor regions of the lateral bank, including LIPa (Fig.11B, green labeling). Thus 45B is connected with STS and IPS regions involved in action observation, but these regions appear largely segregated from those belonging to the STS-F5 routes.

Figure 11. Distribution of labeling in IPS following tracer injection into premotor area F5a and prefrontal area 45B.

A. Location of the injection sites in F5a and 45B shown on the lateral view of the hemisphere and in two coronal sections (a,b). B. Distribution of retrograde labeling in IPS, overlaid onto a flat-map of the IPS. Red color shows IPS labeling following F5a injection, green color shows IPS labeling pattern following injection into area 45B.

Heterogeneity of STS and parietal ROIs responding to action observation

The visual responses of F5c and F5a during grasping observation differ, the former requiring the agent performing the action to be visible, the latter not (Nelissen et al., 2005). This result has been replicated in M15 at 3T (Suppl. Fig. 2). However, the STS areas from which the two action observation routes originate (STPm and LB2) responded to either type of grasping videos. Thus, the F5c responses might be difficult to explain unless the neurons providing its main visual input (in PFG) show a certain degree of distinction between responses to hand actions performed with and without the agent visible.

In order to investigate the heterogeneity of the visual responses in the ROIs conveying visual information about actions to F5, we quantified in each of the four subjects the degree of correlation between the voxel activation levels (%MR signal change) during the observation of an acting person and isolated hand action. The analysis (Fig 12) showed that the STPm and LB2 voxels responding to acting person observation also responded to isolated hand action observation: for STPm the median R² equaled 0.78 and the correlation was significant in all four subjects, while for LB2 the median R² was 0.64 and the correlation was significant in 3 of the 4 subjects. Interestingly, for PFG the median R² equaled just 0.09 and the correlation between acting person and isolated hand actions was significant in only two subjects. In AIP, on the other hand, correlations were were significant in all four subjects, with a median explained variance of 85%. Differences in the degree of correlation present in PFG compared to its input STPm and its neighbor AIP were significant in three of the four subjects (Fig. 12).Thus, while there seems to be relative little difference in the neuronal selectivity for grasping observation with and without the agent visible in the STS regions conveying action information to F5 (LB2 and STPm), nor in AIP, the dissociation between these two visual stimuli is quite large in PFG.

Figure 12. Heterogeneity of voxels within ROI for acting person and isolated hand action observation.

A. Correlation between functional MRI activations for observing acting persons and isolated hand actions in STPm (left) and LB2 (right). Percent MR signal change values for observation of acting person and hand action (versus fixation-only baseline) are plotted from all voxels within ROI responding to acting person. Different colors correspond to 4 different monkeys (M3, M5, M6 and M15). Regression lines for different subjects in STPm: M3, y = 0.97× + 0.11; M5, y = 0.79× + 0.67; M6, y = 1.22× +1.26; M15, y = 1.35× − 0.50, and in LB2: M3, y = 1.14× + 0.06; M5, y = 0.68× + 0.66; M6, y = 0.83× +0.96; M15, y = 1.14× − 0.36. All correlations were significant at p<10^-6, except that in LB2 of M3 (p<0.06). B. Correlation between functional MRI activations for observing acting persons and isolated hand actions in PFG. Same conventions as in A. Regression lines for different subjects: M3, y = 0.001× + 0.26; M5, y = 0.31× - 0.08; M6, y = 0.20× + 0.50; M15, y = 1.31× − 0.11. Correlations were significant at p<0.001 in M5 and M15 but were non-significant (p>0.05) in M3 and M6. C. Correlation between functional MRI activations for observing acting persons and isolated hand actions in AIP. Same conventions as in A. Regression lines for different subjects: M3, y = 0.79× + 0.18; M5, y = 1.07× - 0.10; M6, y = 1.03× + 0.09; M15, y = 1.30× − 0.08. All correlations were significant at p<10^-6. The differences in the degree of correlation between STPm and PFG were significant at p<0.01 in M3, M5 and M6, and between PFG and AIP at p<0.002 in the same three animals.

The strong correlations between the levels of activity in the individual voxels of STPm and LB2 evoked by the acting person and the isolated hand actions indicate that despite the many low-level visual differences between the two types of movies, such as color or size, all voxels of these regions respond similarly. This finding strongly suggests that the responses in these STS regions are driven by the motion pattern, portraying the action. To verify whether this is also true at the earliest level in the STS, we also computed the correlations for the MT/V5 ROI. The median R² value here was also high: 0.72 (range 0.25-0.93, significant in all four subjects) indicating that even at this early level the responses to the action videos are driven predominantly by the motion pattern rather than other low level visual features.

Further evidence for the heterogeneity of PFG was provided by a correlation analysis of the action observation and translation responses. The PFG responses were not significantly stronger for the observation of hand actions compared to a translation control in experiment 2. Since PFG receives input both from STPm and MSTd and the latter region responds to translation (Duffy and Wurtz, 1991; Lagae et al., 1994) it is conceivable that these two types of responses belong to different sets of PFG voxels, reinstating a sensitivity of PFG for action observation at the voxel level. Consistent with this view, hand translation and object observation correlated far less in PFG (R² = 0.24, Suppl. Fig.3) than in STPm (R²= 0.82), or in LB2 and AIP (R² = 0.57, R² = 0.42, resp.), where the hand translation responses are systematically weaker than the action observation responses (Suppl. Figures S1 and S3). The difference in correlation between PFG and its input, STPm, was significant. This suggests that the voxels in PFG are more heterogeneous than those in their input region, STPm.

Discussion

The present study demonstrates that several regions in the STS are activated by action observation. They include, beyond MT/V5 and its FST satellite, STPm, LST, and more rostrally, LB2. Two of these regions, STPm and LB2 are the main starting points of two functional routes linking STS with F5: one via PFG and one through AIP. In the following sections we discuss these three aspects, STS and parietal activations and functional routes.

Functional routes conveying grasping-related visual information to the frontal lobe

Anatomical results indicate that area PFG receive consistent (across animals) connections from three STS regions. Of these three only one, STPm is systematically activated by action observation. Similarly, anatomical results consistently indicate that AIP receives input from LB2 and neighboring lower bank cortex, and from two small regions in the upper bank. Of these regions only one, LB2, displays consistent action observation responses. Both PFG and AIP have extensive connections with F5 (Borra et al., 2008; Rozzi et al., 2006), but the functional profiles of F5c and F5a/F5p (Nelissen et al., 2005, Suppl. Fig. S2) appear to be compatible only with those of PFG and AIP, respectively. Hence we propose that two main functional routes link STS with F5: one via PFG, the other through AIP (red and blue arrows in Fig. 13). These functional routes are a subset of the anatomical connections that link the ventral premotor cortex, posterior parietal cortex and STS. A third possible action observation pathway links STS with 45B (Fig. 13), suggesting that action observation signals can also reach the prefrontal cortex directly, perhaps for the purpose of oculomotor control (Flanagan & Johansson, 2003, Gerbella et al., 2010).

Figure 13. STS-F5 grasping observation routes in the monkey brain.

A. Lateral view of a macaque brain showing approximate locations of three regions involved in action observation. B. Flattened representation of STS, IPS/IPL and IAS with ROIs indicated. FEF = Frontal Eye Fields, F5c = F5 convexity, F5p = F5 (bank) posterior, F5a = F5 (bank) anterior. Visual information about observed actions can be sent forward from STS through parietal cortex to area F5 along two functional routes: a STPm - PFG – F5c route and a LB2 – AIP – F5a/p route, indicated with red and blue arrows, respectively. Area 45B receives parietal input from LIPa and also has direct connections with the lower bank STS (green arrows). The colored areas specify the functional routes. Other abbreviations as in Figures 2 and 7.

Although both STS-F5 functional routes convey visual grasping information, the PFG route appears to be more sensitive to the presence of the agent in the video. In contrast, the AIP route appears to be focused on the object, which is the goal of the action. Indeed, a strong interaction was observed in AIP depending on presence of the object and a weak, non significant effect of object presence was also found in LB2. This main effect reached significance in M6 (Suppl. Fig. S1), for whom a significant interaction was also observed in F5a. These functional differences suggest that the routes may serve different behavioral functions. The PFG route may be more important for extracting the intention behind the observed motor act. This view is supported by the single cell studies (Fogassi et al., 2005; Bonini et al., 2010) revealing that PFG neuronal activity depends on the type of grasping observed, but also on how the grasped object will subsequently be used by the actor. This property fits with the concept of motor intention, that is, the reason behind a performed motor act, e.g. grasping in order to eat or place. The AIP route may make the immediate goal of the motor act more explicit, not only in terms of how the hand shaping relates to the intrinsic object properties, but also in terms of different hand motor acts (grasping vs placing, dropping etc.). Of course these two functional routes are active simultaneously and exchange information by means of their reciprocal connections, providing an understanding of both the intention and the goal.

So far we have considered only the forward projections along the two functional routes. However, cortical connections are reciprocal. This implies that signals from F5 and PFG or AIP neurons can be transmitted backwards to the STS. We suggest that these backward connections are instrumental in binding the pragmatic meaning of actions, underpinned by the parieto-premotor activity, with their visual description in STS.

Superior temporal sulcus (STS) regions responsive to action observation

The cortex buried within the primate STS is known to be involved in the processing of observed actions (Allison et al., 2000; Puce and Perrett, 2003). Our data indicate that, anterior to MT/V5 and its satellites, three STS regions, STPm in the upper bank and LST and LB2 in the lower bank, are specifically activated by grasping observation. The predominance of action encoding in the lower bank, confirmed by the voxel-based analysis, is consistent with single-unit recordings in anterior STS. Although initial studies (Bruce et al., 1981; Oram and Perrett, 1994, 1996) emphasized the presence of neuronal responses to observed actions in the upper bank, several subsequent studies (Perrett et al., 1999; Barraclough et al., 2005, 2006; Vangeneugden et al., 2009; Singer and Sheinberg, 2010) have reported neurons responding to action observation also in the lower bank. Perrett et al. (1989) first documented neuronal responses specific for grasping in the lower bank. This finding was confirmed more recently by Barraclough et al. (2009), who reported that between AP levels +6 and +10 a 76% majority of neurons responding to hand actions were located in the lower bank, and by Singer and Sheinberg, (2010), who recorded at levels +12 to +15. Our results indicate that the proportion of hand action neurons in the two banks depends on AP levels and stimulus type (acting person vs. isolated hand actions).

Our results also indicate that the action observation activations are localized deep in the STS. These activation ribbons are reminiscent of the architectonic regions PGa and IPa of Seltzer and Pandya (1978). This resemblance, however, is only superficial, since the strips of action observation responsive cortex, especially in the lower bank, consist of several regions with distinct functional properties (Ungerleider and Desimone, 1986; Nelissen et al., 2006). In addition, the activation in the lower bank of aSTS clearly extends beyond PGa into area TE, hence the action observation network includes part of the infero-temporal cortex.

There are marked visual differences in the two types of videos used in the present study: amongst others are differences in size, duration, color and texture. Yet, the activities of individual voxels in the STS regions were strongly correlated for the two types of stimuli. In addition, the slope of the regression line was close to 1 in most regions, indicating similar mean levels of response to the two types of videos. This indicates that the responses to action observation in the STS mainly reflect the motion patterns portraying the action and were relatively little influenced by the low-level visual differences.

Inferior Parietal Lobule (IPL) regions responsive to action observation

Observation of hand actions activated three parietal regions: area PFG, AIP, and LIPa. The activation of PFG is in accord with single neuron studies showing that of the four IPL convexity areas, only PFG houses grasping mirror neurons (Fogassi et al., 2005; Rozzi et al., 2008). The present data also fit with the physiological studies in several other respects. In the 1.5T data, most PFG voxels appeared to be visually responsive, but with increasing resolution at 3T only a fraction of them (29%) proved visually responsive. Rozzi et al. (2008) observed that only 60% of the PFG neurons were visually responsive, but some of these neurons responded to stimuli not tested here. Furthermore, no PF voxel was visually responsive at 3T. Similarly Rozzi et al. (2008) observed less than 10% visual neurons in PF. Second, correlation analysis showed that the PFG voxels are visually heterogeneous, responding to at least three types of stimuli: acting person, isolated hand actions and mere translation. Rozzi et al. (2008) described seven types of visual responses, two of which, the mirror and motion neurons, responded to stimuli matching those we used. Third, although the PFG activation during action observation reached significance in most tests, there were two exceptions (Table 2). This fits with the relatively small proportion (15%) of mirror neurons in PFG (Rozzi et al., 2008). Finally, the presence of correlation between responses to acting person and isolated hand actions is STPm and its absence in PFG implies that novel neuronal responses to action observation may arise in PFG.

In addition to PFG, our data show that area AIP is also active during the observation of hand grasping actions, but, unlike PFG, the responses are not influenced by the visibility of the agent. According to Sakata et al. (1995), area AIP plays a fundamental role in visuo-motor transformations for grasping. Note, however, that Sakata et al. (1995), in addition to neurons responding to the observation of objects (‘object’ type neurons), also described a set of neurons related to the sight of the hand moving towards the target (‘non-object’ type neurons). Furthermore, recordings of single neurons in AIP have occasionally revealed the presence of grasping mirror neurons in this area (Rozzi et al., unpublished observations; Rizzolatti et al., 2009). Finally, action observation activated LIPa, the LIP sector where the central visual field is represented (Blatt et al., 1990; Fize et al., 2003) and fixation-related neurons have been described (Ben Hamed et al., 2001). Further single neuron recordings are needed to interpret this activation.

Conclusion

We have documented a double STS-F5 network using either PFG or AIP as parietal relay. Activity in this network may underlie the full perception of goal-directed actions, linking their goal and intention to their specific visual aspects.