Temporal dynamics of cortical representation for action

Abstract

Brain-imaging studies have shown that the human Broca's region and precentral motor cortex are activated both during execution of hand actions and during observation of similar actions performed by other individuals. We aimed to clarify the temporal dynamics of this cortical activation by neuromagnetic recordings during execution, on-line imitation, and observation of right-hand reaching movements that ended with a precision pinch of the tip of a manipulandum. During execution, the left inferior frontal cortex [Brodmann's area (BA) 44] was activated first (peak ≈250 ms before the pinching); this activation was followed within 100–200 ms by activation in the left primary motor area (BA4) and 150–250 ms later in the right BA4. During imitation and observation, the sequence was otherwise similar, but it started from the left occipital cortex (BA19). Activation was always strongest during action imitation. Only the occipital activation was detected when the subject observed the experimenter reaching his hand without pinching. These results suggest that the left BA44 is the orchestrator of the human “mirror neuron system” and is strongly involved in action imitation. The mirror system matches action observation and execution and probably contributes to our understanding of actions made by others.

When asked to perform movements, even simple ones, we often find them easier to do while observing another person's similar movements. This phenomenon suggests that observation of movements can facilitate execution of action. Interestingly, in the monkey, cerebral cortex area F5, located near the arcuate sulcus in the inferior frontal area (IFA), contains specific “mirror neurons” that fire both when the monkey performs and when he observes various goal-directed hand movements. Thus, the mirror neurons have been thought to compose an action execution/observation matching system (1–7).

Several recent studies have aimed at finding a similar action execution/observation matching system in humans. Positron emission tomography (PET) recordings have shown activation of the anterior left IFA, corresponding to Brodmann's area (BA) 45, and of the left superior temporal sulcus (STS) during observation of hand-grasping movements (8–10). Other PET and functional MRI studies have shown activation of the IFA region BA44/45 during self-paced hand actions (11, 12). However, none of these experiments has shown that exactly the same areas would be involved in both action execution and observation.

Convincing evidence for the existence of a human action execution/observation matching system was obtained from a transcranial magnetic stimulation study showing increased motor evoked potentials in the hand muscles when the subject observed movements of another individual (13). This study, however, was not able to identify the exact site of the effect, which could have been at either the cortical or the spinal cord level.

Recent neuromagnetic recordings have completed the picture by showing activation of the primary motor cortex during observation of explorative finger movements (14). Thus, there seems to exist a human analogue of the monkey observation/execution matching system. Such a mirror neuron system could play an essential role in our understanding of motor actions and intentions of other individuals, thereby providing a bridge from doing to communicating (5, 15).

The mirror neuron system could also be involved in imitation, which is a fundamental human ability, based on an inherent link between the cognitive and motor systems (16). Imitation is considered, on the basis of the understanding of actions, to be the mediator between execution and observation (8, 11, 12, 17).

Because none of the previous studies has revealed the relative timing of different areas of the mirror system, we aimed to elucidate the cortical temporal dynamics of action representation during execution, observation, and imitation of hand movements that ended with a pinch of the tip of a manipulandum (Fig. 1a); reaching movements with an open palm served as the control stimuli (Fig. 1 b and c).

Figure 1.

Experimental setup. (a) The experimenter is pinching the top of the manipulandum between his right index finger and thumb during the Imitation and Observation tasks. (b and c) The experimenter is extending his right hand toward the manipulandum without pinching it during the Control task; in the analysis the data from palm-up and palm-down movements were combined.

Methods

Subjects.

We studied seven healthy volunteers: four males and three females; ages 25–41 years (mean 30); five were right-handed and two were left-handed as assessed by the Edinburgh Inventory (18). Two subjects, assessed as left-handers, had been trained during childhood to use their right hand for writing, drawing, and eating and thus were at present “trained right-handers.” None of the subjects had a previous history of neurological or visual disorder. Informed consent was obtained from each subject after full explanation of the study.

Experimental Paradigm.

During the experiment, the subject was sitting in a magnetically shielded room. The head was positioned in a helmet-shaped Dewar and tightly pressed against its inner vault. The subjects were asked (i) to make self-paced movements once every 4–9 s to reach and pinch with the right index finger and thumb the tip of the manipulandum in front of them and then to return to the initial position as quickly as possible (Execution; Fig. 1a); (ii) to perform “on-line imitation” by making movements similar to those of the experimenter on the right in front of the subject (Imitation); (iii) to view similar pinching movements of the experimenter (Observation); or (iv) to view the experimenter extending and returning his right hand with open palm in front of the subject without pinching the manipulandum (Control; Fig. 1 b and c). In Execution, Imitation, and Observation tasks the analysis was time-locked to the pinch of the manipulandum that elicited a trigger pulse for the computer. In Control tasks, the experimenter pinched the tip of another manipulandum in front of the subjects; the other manipulandum was invisible to the subject.

During Execution and Imitation, all subjects moved their right hand from the resting position on the desk where the manipulandum was placed 50 cm in front of them; the tip of the manipulandum was 20 cm from the resting position of the right hand. During Imitation, the subjects extended their right hand to the tip of the manipulandum as soon as they could see the experimenter's hand extending, but the subjects did not touch the manipulandum. During Observation and Control, the subjects kept their right hand relaxed on the desk.

Data Acquisition.

The magnetic signals of the brain were measured with a helmet-shaped 122-channel neuromagnetometer (Neuromag, Helsinki). This system measures the two orthogonal derivatives of the radial magnetic field component (19) and typically detects the largest signal just above the corresponding cerebral source current. Head position with respect to the sensor array was measured with head position indicator coils placed on defined scalp sites. To allow alignment of the magnetoencephalogram (MEG) and MRI coordinate systems, the positions of three head position indicator coils with respect to anatomical landmarks were measured with a three-dimensional digitizer (Isotrak 3S1002, Polhemus Navigation Sciences, Colchester, VT). At the beginning of each recording session, the magnetic signals produced by the head position indicator coils on the scalp were measured by the sensors to obtain head position with respect to the sensor array. Head position was redefined at the beginning of each measurement. Head MRIs were obtained with a 1.5-T Siemens Magnetom system (Siemens Medical Systems, Erlangen, Germany).

To monitor eye movements and blinks, a vertical bipolar electrooculogram was recorded from electrodes placed below and above the left eye. A bipolar electromyogram was recorded from the right extensor carpi radialis and flexor pollicis brevis muscles to monitor right arm and hand actions. To monitor the subjects' vigilance, the waveforms of electrooculogram and selected channels of the MEG were inspected continuously during the recording. Short breaks were given to refresh the subject between the measurements, and the subject was requested to maintain head position as stably as possible during the intermissions.

The recording passband was 0.03–100 Hz (3-dB points) for MEG, 0.01–100 Hz (6-dB points) for electrooculograms, and 30 Hz to 3 kHz (6-dB points) for electromyograms. The sampling rate for digital conversion was 404 Hz, and the data were stored on an optical disk for later off-line analysis. The recording time for each condition was 5 min. To confirm the signal reproducibility, the measurement of each condition was repeated at least once, and the order of the experimental conditions was counterbalanced across subjects. When the measurements were contaminated with excessive artifacts (e.g., eye movements, blinks, or electromyogram signals) in any of the channels or when the subject was found to be drowsy, the measurement was excluded from analysis and an additional set of data was collected.

Signal Analysis.

The signals for each condition were averaged separately off-line from 1,500 ms before to 1,500 ms after the time when the subject or the experimenter pinched the tip of the manipulandum; the timing of the grip was detected by nonmagnetic force transducers in the manipulandum. Epochs containing MEG signals exceeding 1,500 fT/cm, electrooculogram signals exceeding 150 μV during all conditions, and electromyogram signals exceeding 10 μV during Observation and Control were omitted (<5% of all epochs). The first 10% of the whole 3-s time window (from −1,500 ms to −1,200 ms) served as the baseline for amplitude measurements at each channel.

Source Modeling.

Sources of the MEG signals were modeled as previously described (20–22). After the reproducibility of MEG waveforms had been confirmed for each individual subject, the averages of the two measurements within each condition were digitally low-pass filtered at 40 Hz. The processed data were used for construction of isocontour maps and for source analysis. The sources of the magnetic fields were modeled as equivalent current dipoles (ECDs) whose three-dimensional locations, orientations, and current strengths were estimated from the measured signals. A spherical head model was adopted, based on the individual MRIs obtained from each subject (21).

The ECDs that best explained the most dominant signals were determined by a least-squares search, based on data of 20–30 channels at areas including the local signal maximum. For each subset of channels, ECDs were calculated for every 1-ms segment over the time period of 50 ms, containing the peak of each main response. We accepted only ECDs accounting for >80% of the field variance at selected periods of time for each subset of channels and with confidence volume <1 cm³. ECDs with the highest goodness-of-fit value and the smallest confidence volume were accepted for further analysis. Thereafter, the analysis period was extended to the whole measurement time and to all channels, by using a multidipole model where the strengths of the previously found ECDs were allowed to vary as a function of time, while their locations and orientations were kept fixed (21).

The measured signals explained by the model were extracted with signal space projection (23), and a new ECD was identified on the basis of the residual field pattern. Each time a new ECD was obtained, the waveforms predicted by the multidipole model were compared with the measured signals. If the model left some dominant signals unexplained, the data were reevaluated for more accurate estimation of the generator areas but with a conservative attitude to explain only the dominant features of the field pattern. Finally, the estimated dipoles obtained through this procedure were superimposed on the subject's own MRI, according to the alignment of the MEG and MRI coordinate systems. The source locations were transformed into Talairach standard brain space (22, 24).

Statistical Analysis.

The latencies and strengths of the sources were compared with ANOVA with repeated measurements with factors of task, measurement, and location.

Results

Waveforms.

Fig. 2 shows typical MEG signals from subject 1. During Execution, Imitation, and Observation, prominent responses, time-locked to the right-hand pinching, were observed at the occipital area, the left IFA, and the bilateral primary motor area (M1). During Control, only the most posterior responses were seen. Although the major signals peaked at the same areas in all conditions, there were considerable differences in their timing. When this subject himself made the actions, the left IFA was activated first, peaking 265 ms before the time of the pinch, and activation peaked over the left sensorimotor area 135 ms later. During Imitation and Observation, the signals started from the left occipital area (peaks at −375 and −280 ms, respectively) and were followed by signals over the left IFA and both M1s. The last phasic signals were seen over the right M1. The waveforms of the two trained right-handed subjects showed patterns similar to those of the congenitally right-handed subjects.

Figure 2.

(Left) Whole-scalp magnetic responses of subject 1 during Imitation task. The head is viewed from above, and the upper and lower traces of each response pair show the latitudinal and longitudinal derivatives of the magnetic field perpendicular to the helmet at the measurement site. Signals from the left inferior frontal (a), the left and right sensorimotor (b and c), and the occipital (d) areas are enlarged on the Right, and the traces from the other three tasks are superimposed with different colors.

Source Distribution.

At the main response peaks, the magnetic field patterns were dipolar over several regions of both hemispheres, suggesting four main source areas (the left IFA, the left and right central areas, and the left occipital area).

The first ECD was defined on the basis of the strongest signals at the left central channels in all tasks except for Control (when the first process started at the left occipital area). As explained in Methods, the next ECD was identified by first removing the effect of the previous sources from the signal pattern and then searching for additional sources at the main response peaks of the residual waveforms.

Fig. 3 Upper shows, for subject 1, that the main sources were located in the left occipital area (BA19), the left posterior IFA (BA44), and the bilateral M1s (BA4); the same sources were active during Execution, Imitation, and Observation, but only the posterior source was seen during Control.

Figure 3.

(Upper) The main source locations for subject 1 during all conditions superimposed on his own three-dimensional MRI. Upper Left and Upper Right show the brain surface viewed from left (L) and right (R) sides, respectively. Occ, occipital area. (Lower Left) Strengths of the main four dipoles in the IFA and the occipital area of the left hemisphere and the bilateral sensorimotor areas as a function of time in the time-varying four-dipole model explaining the data of subject 1 during Execution, Imitation, and Observation. nAm, nanoampere·meter. (Lower Right) Mean (±SEM) peak latencies of source waveforms in the IFA and the occipital area of the left hemisphere and the bilateral sensorimotor area during Execution, Imitation, and Observation. ***, P < 0.001; *, P < 0.05.

In four of seven subjects, sources were also found in the left posterior temporooccipital area [corresponding to the middle temporal area/visual area 5 (V5) region] during Imitation and in three subjects during Observation. In two subjects, the right IFA was also activated during all conditions except Control, and in one subject, the right occipital area was activated during Imitation. These sources were not analyzed further. Activity in the STS region was specifically sought by first removing the effect of the other sources (occipital area, left IFA, bilateral M1s) from the signal distribution, but no prominent peaks remained in the lateral or in the centroparietal channels to suggest any reliable source areas in the STS or the inferior parietal lobule.

The four main sources were consistent across subjects and conditions, as is evident from Table 1, which summarizes the mean (±SEM) source locations of all subjects in Talairach coordinates (24). The values indicate that the main source areas were in the left occipital area (BA19), in the left BA44, and in the BA4 of both hemispheres.

Table 1.

Source locations in Talairach coordinates

	Left hemisphere				Right hemisphere
	x	y	z	n	x	y	z	n
Execution (n = 7)
IFA	−54 ± 2	11 ± 2	27 ± 2	7	53 ± 3	9 ± 4	27 ± 3	2
Somatomotor area	−35 ± 3	−25 ± 3	56 ± 3	7	35 ± 2	−17 ± 2	55 ± 3	7
Occipital area	−28 ± 5	−85 ± 6	18 ± 6	7
Imitation (n = 7)
IFA	−53 ± 2	10 ± 2	26 ± 4	7	53 ± 6	7 ± 2	30 ± 4	2
Somatomotor area	−36 ± 3	−22 ± 2	55 ± 3	7	34 ± 2	−24 ± 2	57 ± 5	7
Occipital area	−27 ± 4	−87 ± 6	18 ± 2	7	16	−82	18	1
V5 region	−44 ± 6	−67 ± 6	4 ± 4	4
Observation (n = 7)
IFA	−52 ± 1	9 ± 2	24 ± 3	7	55 ± 3	12 ± 3	25 ± 4	2
Somatomotor area	−39 ± 3	−24 ± 3	53 ± 3	7	36 ± 3	−21 ± 3	54 ± 3	6
Occipital area	−26 ± 4	−89 ± 5	20 ± 5	7
V5 region	−39 ± 3	−72 ± 6	5 ± 1	3
Control (n = 6)
Occipital area	−20 ± 2	−87 ± 4	20 ± 5	6
V5 region	−48 ± 5	−69 ± 2	1 ± 1	5	47 ± 2	−73 ± 5	2 ± 2	4

Coordinates x (left-to-right), y (posterior-to-anterior), z (inferior-to-superior) are in millimeters from an origin point situated at the anterior commissure. Results are means ± SEM from subjects whose number is given by n. 

Fig. 3 Lower Left shows, for subject 1, the strengths of the four dipoles, calculated with the same multidipole model for all conditions. The relative timing of different areas clearly varies according to the condition, in agreement with the raw signals shown in Fig. 2.

Fig. 3 Lower Right shows the mean (±SEM) peak latencies of activation in the four main source areas across all subjects. During all conditions, the peak latencies were statistically significantly different between the four areas (P < 0.001 or 0.05). During Execution, the earliest activation (on average 260 ms before the pinching) occurred in the left BA44, followed by the signals in the left BA4, the left occipital, and the right BA4 areas. During Imitation and Observation, the timing patterns were similar to each other, with the activation starting from the left occipital area, followed by activation of the left BA44, the left BA4, and the right BA4. Activation of the left BA44 preceded, by 100–150 ms, activation of the left BA4. The right BA4 was always activated last, typically 100–200 ms after activation of the left BA4.

Fig. 4 shows the mean (±SEM) peak strengths in the four source areas for all subjects. Activations of the left BA44 and the left BA4 were significantly stronger during Imitation than during other conditions (150–250%; P < 0.01 and 0.05, respectively). During Execution and Observation, the source strengths did not differ from each other. During Control, activity differing from the noise level (P < 0.05) was detected only in the occipital area.

Figure 4.

Mean (±SEM) source strengths during Execution, Imitation, Observation, and Control in the left IFA (L-IFA) and the left occipital area (L-Occ), and the bilateral M1s (L-M1 and R-M1).

Discussion

Our study has three important methodological aspects: (i) the source activations were evaluated without subtractions between the tasks; (ii) the source strengths were compared between conditions; and (iii) the relative timing of the source areas was studied with high temporal resolution. The common areas of activation observed during action Execution, Imitation, and Observation strongly support the existence of an action execution/matching system in humans, similar to that described in monkeys.

Brain Areas Involved in Action Representation.

The mean Talairach coordinates of the four main source areas of our subjects are in good agreement with previous findings, indicating that the human IFA and the primary motor cortex are activated during both execution and observation of hand actions (4, 5, 7–14, 25, 26).

In PET studies (8–10), grasp observation has also activated STS and the inferior parietal lobule, which contain many neurons selective to the sight of movement of meaningful biological stimuli and, in particular, to the sight of manipulation of three-dimensional objects (27). Simple and repetitive hand actions in our study did not elicit any prominent activation in the STS and inferior parietal lobule regions. Nor did we see signals from BA45, which Decety et al. (8) observed to be activated during imitation of meaningful actions. Grafton et al. (9), on the basis of PET data, suggested that BA44 reflects object-related grasping, whereas BA45 is related to action recognition. BA44 has, in several studies, been suggested to be the human homologue of the monkey F5 cortex (28–31), and our present data agree with such a proposal.

The activation in the left posterior temporooccipital area, observed in four subjects, agrees with the location of the V5 in previous PET and functional MRI studies (32, 33).

The observed clear left-hemisphere dominance of the mirror system could reflect a real left-sided dominance of the system; however, the tests could be biased, because the movements occurred mainly in the subject's right visual field. Furthermore, although the hand movements in our study appeared meaningless, we cannot totally exclude the possibility that responses were related to some semantic cues during Imitation and Observation (8).

Time Courses and Strengths of Activation.

The present study provided information about the time courses of the active areas involved in the mirror neuron system. The left BA44 was activated significantly earlier than the left BA4 area in all conditions. It thus seems that the BA44 acts as an orchestrator for the observation/execution matching system.

It is interesting that both BA44 and BA4 were activated about twice as strongly during Imitation as during Execution and Observation. This result has two possible interpretations. First, imitation and observation could activate distinct neurons in areas BA44 and BA4 such that when imitation and observation are simultaneous, as in our on-line Imitation condition, the observed response is approximately the sum of the single activations. The other possibility is that imitation of movements strongly facilitates the mirror system. Observing movements is the first step of imitation as a means of establishing contact with other individuals to acquire new skills from them. Even 2- to 3-week-old infants can imitate facial and manual gestures after a relatively long delay (16), and actions to be imitated are probably stored in the form of action rather than recognition commands. The observed actions could be transformed into potential actions in the mirror system (34) such that the perception–action link related to action representation is activated already during observation, and people probably imitate movements by using internal innate and learned models of movements (35).

Human Broca's Area and Hand, Mouth, and Language Representations.

BA44 and BA45 in the human left hemisphere form the well known Broca's area, classically considered the area of speech production. However, both our results and several earlier studies, including clinical data, indicate that the Broca's area also includes representation of hand actions. For example, aphasic patients are not able to recognize pantomimic hand actions (36). Thus, BA44/45 seems to be involved in both action recognition and speech processing.

Why would hand representation and the Broca's area overlap so heavily? The monkey F5 area contains, in addition to the hand field, a more laterally located large mouth–face field (3), stimulation of which elicits both mouth and hand movements (30). According to Rizzolatti and Arbib (15) the precursor of the caudal IFA might have played a crucial role in the interindividual communication by facial and hand gestures. Meaningless sounds might have gradually obtained specific meanings such that the mirror system, with accurate hand–mouth coordination, would have formed the gestural basis for the evolution of language and the precursor of speech.

In line with this hypothesis, hand gestures and speech are also heavily interdependent in modern humans. For instance, stutterers freeze in their hand gestures simultaneously as they start to stutter, and the onset of the gestures coincides with fluent speech (37). Furthermore, in bilingual children, speech-related hand gestures develop parallel to the development of each language (38).

In this context, it is interesting that the motor theory of speech perception, advocated by Liberman and Mattingly (39, 40), assumes that speech is perceived by recognition of phonetic gestures of the speaker rather than of sounds only; such a matching could well take place in the mirror neuron system, although the direct experimental evidence is still lacking.

Conclusions

The monkey mirror system is considered to be involved in action understanding and in assigning meaning to actions performed by others (6, 15, 30). Our results indicate that the observation and execution of action are carried out in humans much as they are in monkeys. The whole human mirror neuton system is most strongly activated when the subject observes hand actions made by another individual and simultaneously imitates the same actions. The observed temporal dynamics of cortical areas emphasizes the role of the posterior Broca's area, the left BA44, as an orchestrator of the whole mirror system.

Abbreviations

BA

Brodmann's area

ECD

equivalent current dipole

IFA

inferior frontal area

MEG

magnetoencephalogram

M1

primary motor area

PET

positron emission tomography

STS

superior temporal sulcus

V5

visual area 5