Reafferent copies of imitated actions in the right superior temporal cortex

Abstract

Imitation is a complex phenomenon, the neural mechanisms of which are still largely unknown. When individuals imitate an action that already is present in their motor repertoire, a mechanism matching the observed action onto an internal motor representation of that action should suffice for the purpose. When one has to copy a new action, however, or to adjust an action present in one's motor repertoire to a different observed action, an additional mechanism is needed that allows the observer to compare the action made by another individual with the sensory consequences of the same action made by himself. Previous experiments have shown that a mechanism that directly matches observed actions on their motor counterparts exists in the premotor cortex of monkeys and humans. Here we report the results of functional magnetic resonance experiments, suggesting that in the superior temporal sulcus, a higher order visual region, there is a sector that becomes active both during hand action observation and during imitation even in the absence of direct vision of the imitator's hand. The motor-related activity is greater during imitation than during control motor tasks. This newly identified region has all the requisites for being the region at which the observed actions, and the reafferent motor-related copies of actions made by the imitator, interact.

“No creature not endowed with divinatory power can perform an act voluntarily for the first time” (1). Voluntary movements must be preceded, as William James wrote, by “random, automatic, or reflex movements.” These movements leave a trace formed by kinesthetic impressions and by their outcome as perceived by the agent of the action [“remote effects” (1)]. The idea of an internal sensory copy of the executed action that in modern time has been reproposed in computer science [forward internal models (2–4)] and in psychology [ideomotor theory of learning (5, 6)] has far reaching consequences for understanding imitation. If the motor representation of a voluntary action indeed evokes an internal sensory representation of its consequences, imitation can be achieved by a mechanism relating this representation with the visual representation of the movement to be imitated and a subsequent re-activation of the relevant motor representations.

Evidence that the observed actions are mapped directly onto neurons coding actions has been provided recently by Rizzolatti and coworkers. They demonstrated that in the ventral premotor cortex [area F5 (7, 8)] and in the parietal area PFk of the monkey there are neurons that discharge both when the monkey makes a specific hand action and when it observes another individual making a similar action (mirror neurons). The issue, however, of whether there is a visual area that codes the observed actions as well as the remote effects of voluntary movements is open. Given its reciprocal connections with parietal area PF (and indirectly with F5), the superior temporal sulcus (STS) region, a cortical sector in which there is a large number of neurons responding to the observation of biological actions (9–11, see ref. 12 for review), is one of the most likely candidates.

The mirror system, given its observation/execution matching properties, very likely represents the evolutionary precursor of the human mechanism for imitation, a behavior fundamental for culture transmission (13–16). Evidence in favor of this hypothesis was provided recently by an experiment in which we studied imitative behavior by using functional magnetic resonance imaging (17). Our reasoning was the following: because mirror neurons are first of all motor neurons, a “mirror area” should be activated during execution of finger hand movements regardless of how the movement is actually triggered. Moreover, given that mirror neurons, unlike other cortical motor neurons, are triggered specifically by action observation, mirror areas should show an additional activation during imitation, compared with a control motor task. Finally, mirror areas should be activated by simple observation of the action. Two areas with these characteristics were found: area 44 and the rostralmost part of the superior parietal cortex. Note that in terms of comparative neuroanatomy, area F5, the area showing mirror properties in the macaque brain, corresponds to area 44 of the human brain (18, 19, see ref. 20 for review).

Mirror properties appeared to be present also in a third area located in the STS, thus anatomically compatible with the STS region of the macaque brain, as we reported preliminarily in abstract form.l This finding is rather surprising because unlike the first two areas, which are located in cortical sectors where movement-related activity is a characterizing functional property, this third area was located in the cortex mainly dominated by sensory processing (Fig. 1, peak coordinates: x = 57, y = −50, z = 16). Also, the activation was only marginally significant and given its unexpected location, additional empirical evidence on its functional properties was needed.

Figure 1.

Coronal, transverse, and sagittal views of the STS area (in red, 45 voxels) in the right hemisphere in which signal intensity is reliably bigger during imitation compared with motor control tasks and during action observation compared with visual control tasks.

We therefore performed a new experiment on a new group of volunteers by using the previously observed area as a search region of interest and instructing the subjects to observe and imitate both left and right hand movements. There is evidence from psychological studies that humans tend to imitate preferentially mirror-image movements (21–23). (A common experience is that when a person touches his right cheek with his right hand, telling another person that there is something on her cheek, the other person touches the left cheek, not the right cheek, with the left hand.) This behavioral evidence suggests a similar privileged neural link between opposite-side effectors. Thus, when using the right hand to imitate, observed left hand actions should produce a stronger activation of the area in which visual information and reafferent copy of the imitated action interact than observed right hand actions. The results corroborated this prediction, suggesting that this newly identified region in the human STS has all the requisites for being a region in which interactions occur between observed action and the reafferent motor-related copy of that action. Both the first experiment that allowed us to identify the region of interest and the second experiment in which we tested the reafferent-copy hypothesis are reported here.

Methods

Subjects.

In the first experiment, 12 normal right handers (9 males and 3 females) were studied. The mean age of this group was 25.4 (±5.8) years. In the second experiment, 10 additional right handers (6 males and 4 females) were enrolled. The mean age of this group was 26.8 (±6.3). All subjects were right handers, as assessed with a questionnaire modified from the Edinburgh Handedness Inventory (24), and had no neurological abnormalities identified at the neurological examination performed just before the scanning procedure. The subjects were enrolled according to UCLA Institutional Review Board guidelines.

Behavioral Tasks.

In the first experiment, the subjects viewed, through magnet-compatible goggles that prevented vision of their own hands, three types of stimuli: (i) an animated hand (left hand, the index or the middle finger of the animated hand was lifted at random); (ii) a static hand (left hand, a cross appeared on the index or middle finger at random); and (iii) a gray rectangle (a cross appeared on the left or right side of it at random; see Fig. 2, top graph). There were three “observation-only” and three “observation/execution” tasks. In the observation-only tasks, the instruction given to the participants was to pay attention to the stimuli. In the observation/execution tasks, the instruction was to lift the corresponding finger of the right hand in response to either the movements of the animated hand (imitation) or the appearance of the cross. In each condition half the trials involved the index finger, and half the trials involved the middle finger.

Figure 2.

Time series of the active STS area in the first (Top) and second (Bottom) experiments. These graphs represent the average time series of all runs in all subjects participating in the two different experiments. Thus, each data point in the top graph is the average of 48 data points, and each data point in the bottom graph is the average of 20 data points. The order of tasks was counterbalanced across subjects in the real experiments but is obviously displayed as a fixed order in this figure. The first graph is composed of seven rest periods alternated with six tasks periods. The first three task periods correspond to the observation/execution tasks, and the last three task periods correspond to the observation tasks. The small pictures correspond to the type of stimulus presented and are used here for display purposes only. The hand with the lifted finger corresponds to the animated hand, the geometric figure corresponds to the geometric figure condition, and the hand with the small black cross on the finger corresponds to the static hand condition. The second graph is composed of nine rest periods alternated with eight task periods. The first four task periods correspond to the observation/execution tasks, and the last four task periods correspond to the observation tasks. The hand with the lifted finger corresponds to the animated hand, and the hand with the small black cross on the finger corresponds to the static hand condition. There are two left hand stimuli (animated and static) and two right hand stimuli (animated and static). In both graphs, the signal is reliably higher for imitation tasks than relative motor control tasks and for action observation tasks relative to visual control tasks (see Results).

Second Experiment.

The subjects viewed, through magnet-compatible goggles that prevented vision of their own hands, four types of stimuli, in a 2 × 2 design: (i) an animated hand (left hand or right) displayed on the computer screen (the index or the middle finger of the animated hand was lifted at random); (ii) a static hand (left hand or right hand) displayed on the screen (Fig. 2, bottom graph). The stimuli were the same used in conditions i and ii of the first experiment. The right hand stimuli were made simply by horizontally flipping the frames used for stimulus presentation. There were four observation-only and four observation/execution tasks. In the imitative observation/execution conditions, participants had to imitate with their right hand the action they observed. The observed actions were either left or right hand actions. In the other two observation/execution tasks, the subjects performed the same movement triggered by the appearance of the cross, as in the previous experiment. In each condition half the trials involved the index finger, and half the trials involved the middle finger.

Imaging.

For both experiments we used a GE 3 Tesla scanner with an ANMR echo-planar imaging upgrade located in the Ahmanson-Lovelace Brain Mapping Center at UCLA. For both experiments, structural images were acquired as co-planar high resolution echo-planar imaging volumes (time to return = 4,000 ms; time to echo = 54 ms; flip angle = 90°; 128 × 128 matrix; 26 axial slices; 3.125-mm in-plane resolution; 4-mm thickness; skip 1 mm).

For both experiments, functional images were acquired with an echo-planar T₂*-weighted gradient echo sequence (time to return = 4,000 ms; time to echo = 70 ms; flip angle = 90°; 64 × 64 matrix; 26 axial slices; 3.125-mm in-plane resolution; 4-mm thickness; skip 1 mm). In the first experiment the subjects had seven rest periods interweaved with six task periods. Each period lasted 24 seconds, and the total scan time was 312 seconds. Four functional MRI scans were performed on each subject. In the second experiment there were nine rest periods interweaved with eight task periods. Each period lasted 20 seconds, and the total scan time was 340 seconds. Two functional MRI scans were performed on each subject. The task order was counterbalanced across subjects in both experiments.

Data Processing.

Intrasubject registration was performed by aligning the functional volumes onto the co-planar high resolution echo-planar imaging volume using a rigid body linear registration algorithm (25).

Intersubject registration was performed by using fifth order polynomial nonlinear (26) warping of each subject's images into a Talairach-compatible brain atlas (27). In-plane Gaussian filtering was applied, producing a final image resolution of 8.7 × 8.7 × 8.6 mm.

Data Analysis.

In both experiments, ANOVA was performed with the signal intensity at each voxel as the dependent variable. Because of the “blurred” hemodynamic response, the brain volumes acquired during each task period cannot be considered independent observations. Thus, we used the sum of the signal intensity at each voxel throughout each period as the dependent variable. In the first experiment subjects (n = 12), functional MRI scans (n = 4) task (n = 2: observation and observation/execution), and stimuli (n = 3: animated hand, static hand, and geometric figure) were included in the ANOVA. In the second experiment the included variables were subjects (n = 10), functional MRI scans (n = 2), task (n = 2: observation and observation/execution), stimulus type (n = 2: animated hand, static hand), and stimulus hand (n = 2: left hand, right hand).

Statistical Threshold.

For the first experiment, df = 66, t = 3.22, and P < 0.001 uncorrected for multiple spatial comparisons. For the second experiment, df = 9, t = 2.68, and P < 0.05 corrected for multiple spatial comparisons (28) by using the right temporoparietal region (216 mm³) observed in the first experiment and described in the Fig. 1 legend as hypothesis-driven search region of interest.

Results

Fig. 1 illustrates the location of the STS area observed in the first experiment (peak stereotaxic coordinates: x = 57, y = −50, z = 16) and used as the search region of interest for mirror activity during imitation in the second experiment. This area is located within the STS, where it divides in its two small posterior branches, the sulcus angularis and the sulcus horizontalis. In all voxels of our search region of interest, we observed significantly larger signal intensity (df = 9, t = 2.68, and P < 0.05 corrected for multiple spatial comparisons) during imitation as compared with the control motor tasks. The area was located more rostrally and slightly more dorsally than an area responding to motion that likely corresponds to the V5/MT complex in the right hemisphere (peak stereotaxic coordinates: x = 46, y = −64, z = 8; ref. 29).

Fig. 2 shows the time series of the STS active area in both the first and second experiments. Both time series show an overall greater activity for observation/execution than for observation only. They also show greater activity during imitation compared with the corresponding motor control tasks and greater activity during action observation compared with the corresponding visual control tasks. Note also that in experiment 2, during observation tasks (lower panel, right side) the response to the lifting of the fingers of the right hand (the one corresponding to that used by the subject to respond) was greater than the response to the lifting of the fingers of the left hand. In contrast, during execution tasks (lower panel, left side) the responses in imitation condition were greater when the triggering stimulus was the left hand than the when the triggering stimulus was the right hand. All these differences were statistically significant at corrected thresholds.

Discussion

The STS-activated area reported in the present study appears to correspond in its location to the monkey STS region. As shown by Perrett et al. (9–11) this region is characterized by a large number of neurons that selectively respond to the observation of biological stimuli (see also ref. 12 for review). Previous imaging studies in which volunteers observed actions such as hand or eye movements also showed that biological moving stimuli activate the human STS region (30–33). It therefore appears that both in humans and monkeys, the cortex around the STS is a visual region involved in the analysis of complex biological stimuli.

Given these findings, it is not surprising that in the present experiments activation was found in STS during the observation of finger movements. In previous studies in humans the STS region was observed as activated by biological motion in left, right, or both hemispheres (12). This difference in the laterality of activation of STS is likely caused by the type of biological actions used as stimuli. Left hemisphere activation was reported frequently in the case of object-oriented actions (e.g., ref. 30, see ref. 12). In our experiments the observed action was an intransitive action and required, presumably, a more fine-grained spatial processing, hence the right hemisphere prevalence. Regardless of the activation side, however, what is particularly interesting in our findings is that STS was activated during the execution of finger movements and that this activation was highest when there was a matching between the action that was prepared and the action that was observed.

There is general agreement that the temporal lobe processes visual information to give a semantic description of the external word. According to this view, the temporal lobe is the place where the “what” of a visual object is coded, as distinct from the “pragmatic” analysis of the “where” and “how” (34) performed in the parietal lobe (35, 36). A similar semantic role may be postulated for the STS region but with a specialization for biologically relevant stimuli including body and body-part movements.

If this general distinction between temporal and parietal lobe functions is accepted, then the activation of the temporal lobe during action execution can hardly be interpreted as a command to move or, more generally, as an activation causally related to action. Similarly, it is difficult to postulate that the STS activation may represent an intention to move, as it has been suggested for some sectors of the posterior parietal lobe (37, 38). It seems much more likely that the STS activation reported here represents a reflection of motor-related activity occurring in the frontoparietal circuits during action execution. The possible anatomical circuitry subserving this functional mechanism may be the connections from the inferior parietal lobe to STS (39).

It is interesting to note that the STS activation of the present study appears to be functionally different from the classical corollary discharges, the aim of which is typically that of canceling or modifying sensory information to maintain stable perception (40–44). On the contrary, the present data indicate that the activation in STS is maximal during imitation, i.e., in the condition under which there is a congruency between the observed action and the action to be executed. In other words, the visual representation of action coded in STS is potentiated during action execution, not canceled. This potentiation is not likely to be caused by unspecific attentional mechanisms. Attentional demands are generally higher for less “natural” tasks. Behavioral studies have demonstrated that in the case of imitation of hand movements, the movement that is imitated naturally is that of the hand of the actor facing the hand used by the imitator (21–23). That is, the motor activity evoked by the observation of left hand movement produces a tendency to move the right hand and vice versa. These considerations predict that an increase in attention is more likely to occur when subjects imitate in the less natural condition, which is the opposite of what we observed.

It is likely that the phenomenon of imitating in a mirror-like fashion occurs for a natural tendency to interact with other people by using a sector of space common to both actor and imitator. In contrast, there is no reason for this tendency to be present when the observer simply looks at another individual. Exactly this dissociation was found in the STS area reported in the present study. During observation tasks the activity in STS was greater when the right hand was the visual stimulus, compared with the left hand. During imitation, the activity in the STS area was greater when the imitators observed the hand mirror image of the hand they used (left hand as visual stimulus and right hand as motor effector). This reversal is likely caused by a modulatory role of the imitative behavior on STS visual activity that, in the absence of imitation requirements, reflects an implicit categorization of the moving hand as referred to the body of the observer. Although the hand used by subjects to imitate the actions is ipsilateral to the STS region reported here, the motor control at the parietal and premotor level is largely bilateral even for distal movements. It is interesting to note that in our previous report on imitation (17) we described a left inferior frontal area and a right posterior parietal area as endowed with mirror properties. At slightly lower statistical thresholds, however, we observed mirror-like activations also in the right inferior frontal and left posterior parietal cortex.

What we believe happens between the STS, inferior frontal, and posterior parietal cortices in terms of information processing is that STS neurons provide an early description of the action to parietal mirror neurons. These neurons add additional somatosensory information to the movement to be imitated. This more complex information is sent to the inferior frontal cortex, which in turn codes the goal of the action to be imitated. Sensory copies of the imitated actions are then sent back to the STS area for monitoring purposes (“my actions are like the actions I have seen”).

In conclusion, returning to the James' proposal that movements leave a trace formed not only by kinesthetic impression but also by their visual effects, our data indicate that this functional mechanism indeed may occur in the STS region. During action execution, and in particular during action imitation, the visual representation of biological motion located in STS is activated, and this activation has precisely those properties that an imitation mechanism must posses. It codes actions made by others and stores the remote effects of the movements made by the imitator (45).

Abbreviation

STS

superior temporal sulcus