Attention modulates motor system activation during action observation: evidence for inhibitory rebound

Abstract

Perceiving another individual’s actions activates the human motor system. We investigated whether this effect is stronger when the observed action is relevant to the observer’s task. The mu rhythm (oscillatory activity in the 8- to 13-Hz band over sensorimotor cortex) was measured while participants watched videos of grasping movements. In one of two conditions, the participants had to later report how many times they had seen a certain kind of grasp. In the other condition, they viewed the identical videos but had to later report how many times they had seen a certain colour change. The colour change and the grasp always occurred simultaneously. Results show mu rhythm attenuation when watching the videos relative to baseline. This attenuation was stronger when participants later reported the grasp rather than the colour, suggesting that the motor system is more strongly activated when the observed grasping actions were relevant to the observer’s task. Moreover, when the graspable object disappeared after the offset of the video, there was subsequent mu rhythm enhancement, reflecting a post-stimulus inhibitory rebound. This enhancement was again stronger when making judgments about the grasp than the colour, suggesting that the stronger activation is followed by a stronger inhibitory rebound.

Introduction

Vision and action systems evolved together to enable animals to rapidly gather information from the environment and produce the appropriate motor response. Avoiding the lunge of a predator, or grasping fleeing prey, requires exquisitely efficient vision-action processes. Indeed, Gibson (1979) long argued that the specific reason for vision to evolve was to serve action. One feature of such a system would be that visual information flowed automatically into actions, such that merely viewing a stimulus could evoke an action with little or no conscious intention to act (e.g., Arbib 1981; Bridgeman et al. 1979; Coles et al. 1985; Goldberg and Seagrave 1987; Simon 1969; Weiskrantz 1986; see Tipper 2001, 2004, for review).

Motor system activation in action observation

There is a large body of evidence showing that the action system becomes activated when viewing objects that afford actions, and when viewing other individuals manipulating objects. One line of evidence comes from single-cell recordings in monkeys. Cells have been found in the ventral premotor cortex (area F5) of monkey that are activated not only when the monkey grasps an object but also when the monkey merely sees graspable objects (canonical neurons; see Rizzolatti and Luppino 2001, for review). Other cells in area F5 have been found active when the monkey grasps or sees another individual grasping an object (‘mirror neurons’; DiPellegrino et al. 1992; see Rizzolatti et al. 2002, for review). Another line of evidence comes from imaging studies in humans, where activation of the motor system has been found when people viewed graspable objects or another person grasping an object (e.g., Buccino et al. 2001; Chao and Martin 2000; Grafton et al. 1996; Grezes and Decety 2002; Grezes et al. 2003), and recently single unit recording in patients has detected mirror cells in supplementary motor cortex and hippocampus in humans (Mukamel et al. 2010). Finally, TMS studies measuring motor cortex excitability confirm that visually perceiving an action leads to motor system activation (e.g., Fadiga et al. 2005; Strafella and Paus 2000).

The core idea behind all this research is that vision is converted automatically into action-based representations. However, it is not clear to date how automatic such processes are. Most of the above-mentioned studies do not require attending to the action properties of the visual stimulus. Subjects passively viewed the presented objects or actions, with no particular task instructions. Thus, action-based information seems to be represented even though it is irrelevant to a participant’s task. However, some recent behavioural studies have provided evidence that automatic action tendencies when observing actions or manipulable objects do critically depend on the observer’s task. For instance, Tipper et al. (2006), measuring compatibility effects between action properties of observed objects and performed actions, found that these compatibility effects depended on which stimulus properties were being attended. The compatibility effects were only found when the participants attended to an action-relevant feature. In particular, when viewing door handles oriented to the left or right and responding with the left or right hand, compatibility effects were found when the shape of the door handles was relevant to the participants’ task, but not when the colour of the door handles was relevant (see also Vainio et al. 2007). In another study, compatibility effects between observed actions and performed actions only occurred when the participants attended to the action related body site of the observed action (Bach et al. 2007).

Thus, there is preliminary evidence that attending to different aspects of observed actions modulates the extent of automatic action tendencies. However, a limitation of behavioural studies, such as those described above, is that they require overt responses to the stimuli, which might interact with the action tendencies evoked by the perception of visual stimuli (see e.g., Humphreys and Riddoch 2001; Symes et al. 2008, for examples of ongoing action influencing perception). Thus, in the current study, we sought to measure motor system activation in the absence of any overt behaviour. Hence, we used EEG to capture motor system activation during action observation.

A growing number of EEG and MEG studies have investigated the oscillatory activity of the cortex when performing actions and when observing actions. One kind of oscillatory activity, the mu rhythm, changes both when performing and when observing actions and has therefore been linked to mirror system activity (see Pineda 2005, for overview). The mu rhythm is activity in the alpha frequency band (8–13 Hz) over sensorimotor cortex and is thought to reflect downstream activity of the mirror system. The mu rhythm is most pronounced when a person is at rest and becomes suppressed during movement production, presumably reflecting desynchronisation of firing of neuron assemblies. Importantly, it also becomes suppressed when a person is merely observing the movement of another person, indicating motor system activation during action observation (e.g., Cochin et al. 1999; Gastaut and Bert 1954; Hari et al. 1998; Kessler et al. 2006; Kilner et al. 2006; Muthukumaraswamy et al. 2004; Muthukumaraswamy and Johnson 2004; Neuper et al. 2009; Nishitani and Hari 2000; Oberman et al. 2008; Pineda and Hecht 2009; Ulloa and Pineda 2007; see also Crawcour et al. 2009, for mu suppression during speech perception).

Motor system inhibition in action observation

It has further been observed that the desynchronisation in the alpha band over sensorimotor cortex (i.e. mu rhythm suppression) during action observation or action execution is followed by a period of increased synchronisation (i.e., mu rhythm enhancement) after the observed or executed event (e.g., Babiloni et al. 2002; Pfurtscheller et al. 2006).

This ‘post-stimulus rebound’ effect could reflect inhibition following the activation of the mirror system. In general, whereas desynchronisation of the neural activity in the alpha frequency range reflects activation, increased synchronisation in the alpha band reflects inhibition (see Klimesch et al. 2007, for review). Direct evidence that mu synchronisation is associated with inhibitory control comes from a study by Hummel et al. (2002). They showed that when people observed a cue associated with an action, but no overt action was to be produced, inhibitory control of the motor memory was confirmed by the reduction in motor evoked potentials (MEP). Critically, this reduction in MEP was accompanied by a significant increase in mu oscillations over sensorimotor areas (see also Sauseng et al. 2009; Zarkowski et al. 2006, for a link between mu power and MEP; but see Lepage et al. 2008, for lack of such a link). In another study (Pfurtscheller and Neuper 1997), it was shown that action of the foot increases mu power in the hand motor area, reflecting suppression of processing in the non-responding hand area while moving the foot. Thus, motor areas relevant to action are desynchronised (excitation), while other areas that might produce competing motor behaviour have increased synchrony (inhibition). Similar results have been observed during motor imagery without any overt movement (Pfurtscheller et al. 2006).

The above described post-stimulus rebound effect fits with the model of Houghton and Tipper (1994) and Houghton et al. (1996) describing the interactions between excitation and inhibition systems during the selective control of action. Assuming that vision flows fluently and automatically into action and that inhibition is necessary to prevent our actions from being captured by the dominant perceptual inputs of the moment, the model describes a reactive inhibitory feedback system. While there is perceptual input, this system is in a balanced state of excitation and inhibition. When a stimulus is offset, the excitatory inputs are terminated, and this reveals residual inhibitory feedback, resulting in a neural rebound for a short period, consistent with the mu rebound effect that occurs after the end of the observed action.

The Houghton and Tipper model also assumes that the level of inhibition feeding back onto activated representations is proportional to the level of activity in the representations. Thus, the model predicts that more potent or powerful inputs receive greater levels of inhibition to prevent them from controlling action, resulting in a greater inhibitory rebound after stimulus offset.

Present study

The current study investigates whether automatic action tendencies evoked by visual stimuli depend on whether the perceived action is relevant to the observer’s current task goal. Participants watched videos of grasping movements with an embedded colour change at the moment of the grasp. Whether they had to make a subsequent judgment about the grasp or about the colour was manipulated. A cup was either being grasped at the handle or at the top (i.e. with a precision grip or a power grip). Simultaneously, a grey X presented on top of the cup changed into either blue or green (see Fig. 1). The participants’ task was to either make a later judgment about the grasping movement or about the colour change in separate blocks of trials. Specifically, they had to estimate, at the end of the block, in what percentage of trials the cup had been grasped at the handle, or the X had changed into green. It is important to note that identical videos were shown in both conditions. Thus, the conditions differed only with respect to the task that had to be performed: whether the grasp or the colour change was relevant for later report. The participants did not perform any actions while watching the videos and only responded at the end of the block, after having watched 50 videos.

Fig. 1.

Paradigm. Participants watched short videos of grasping movements. In the stimulus-offset condition, the cup disappeared after the grasp. In the stimulus-maintenance condition, the cup remained visible after the grasp. In each trial, the cup was grasped at the handle or at the top (precision or power grip). The X on the cup changed colour at the moment of grasp (blue or green). Both grasp and colour varied randomly from trial to trial. One kind of grasp and one colour were more frequent than the other (80% versus 20%, respectively). Participants only responded at the end of a block of 50 videos. In the report-grasp condition, they had to estimate the percentage of trials with the infrequent grasp. In the report-colour condition, the percentage of trials with the infrequent colour had to be estimated (colour figure online)

We predict desynchronisation of the mu rhythm while observing the cup and the subsequent grasping movements, followed by a period of increased synchronisation of the mu rhythm after stimulus offset, as has been reported in previous studies (Babiloni et al. 2002; Pfurtscheller et al. 2006). The mu desynchronisation reflects activation of the motor system; the synchronisation reflects subsequent inhibition of the motor system. Mu desynchronisation is measured as a drop in power in the alpha band over sensorimotor cortex; the synchronisation by an increase in power, respectively.

To test the idea that the re-bound into the mu synchronisation state is evoked by stimulus offset, we compared two conditions. In the “stimulus-offset” condition, after observing the grasp action/colour change the display was terminated (see Fig. 1 Panel A). We predicted that stimulus offset would reveal an inhibitory re-bound. In the second, “stimulus-maintenance” condition, the cup stimulus remained visible (see Fig. 1 Panel B). We know that objects that afford action activate motor states (e.g., Tucker and Ellis 1998), and hence in this situation mu desynchronisation should continue, preventing inhibitory re-bound (see Houghton et al. 1996 for similar predictions).

When considering Fig. 1, we can make a number of specific predictions concerning mu power. During the pre-stimulus period (period i), mu synchronisation should be relatively high, and there will be no difference between attend-action and attend-colour conditions. In contrast, during the onset of the cup (period ii), action should be automatically evoked (e.g., Tucker and Ellis 2001), resulting in suppression of mu relative to the previous period i. However, at this time there should be no differences between attend action and attend colour, as the relevant stimuli have not yet been presented. In the third stage, when action is observed (period iii), mu suppression (relative to period i) should still be observed. However, if attention plays a role, we predict significantly greater mu suppression when attending to action than colour.

Note that these stages of a trial (columns i to iii) are identical for the stimulus-offset (Panel A) and stimulus-maintenance (Panel B) conditions. However, and perhaps of most interest, we predict differences between stimulus-offset and stimulus-maintenance conditions at the end of the trial (column iv). First, rebound into mu synchronisation should be greater when the stimulus is offset (Panel A) than when maintained (Panel B). Second, the inhibitory rebound in the stimulus-offset condition should be greater when previously attending to action than when attending to colour, as predicted by reactive inhibition mechanisms (Houghton et al. 1996). That is, the amount of inhibition is proportional to the amount of activation; thus, the more activation when attending to action, the stronger the subsequent inhibitory rebound.

Method

Participants

Twenty-eight right-handed females were tested; their mean age was 20 years (SD = 2.2 years). They gave written consent to take part in the study and received £18 in return for their participation. A female cohort was chosen because of a recent study showing stronger mu suppression in females than in males during the observation of hand actions (Cheng et al. 2008).

Design and procedure

Fourteen individuals participated in the ‘stimulus-offset’ condition and 14 participants in the ‘stimulus-maintenance’ condition. Participants watched short videos of a hand grasping a cup. A grey X was superimposed on the cup that changed its colour at the time of the grasp. The cup was either being grasped at the handle (precision grip) or at the top (power grip). The X changed into either green or blue. Two action observation conditions were compared. Participants had to either later report the grasp or the colour. In the former condition, they had to estimate at the end of the block in what percentage of trials the cup had been grasped with a precision grip. In the latter condition, they had to estimate at the end of the block in what percentage of trials the X had turned green.

The two conditions alternated blockwise, with overall four blocks being performed. Half of the participants started with reporting the grasp, the other half with reporting the colour. Each block consisted of 50 video trials presented in random order. Thirty-two of these contained a power grip and a blue X, 8 a precision grip and a blue X, 8 a power grip and a green X, and 2 a precision grip and a green X. For half of the participants, the frequent and infrequent events were reversed. That is, there were 32 videos with a precision grip and a green X, 8 with a power grip and a green X, 8 with a precision grip and a blue X, and 2 with a power grip and a blue X. Participants always had to report the proportion of the infrequent events.

Trial structure

Each trial started with a blank screen. After 1 s, the picture of a white cup on a black table appeared. A grey X was superimposed on the cup. The handle of the cup always pointed to the right. After 2 s, a right hand appeared from the right and grasped the cup. The grasp happened at 2.5 s after the onset of the video, at the same time as the X changed colour. The last frame of the video, showing the cup being grasped, and the coloured X, stayed on the screen for 1 more second. In the stimulus-offset condition, the picture was then replaced with a blank screen. In the stimulus-maintenance condition, the picture was then replaced by a cup with a grey X. This picture remained on the screen for 2.5 s and thereafter was replaced with a blank screen (see Fig. 1).

The total trial length was either 6.5 s or 7.0 s in the stimulus-offset condition, and either 8.5 s or 9.0 s in the stimulus-maintenance condition. The total trial length varied randomly from trial to trial; this jitter was included in order to ensure that the onset of the next video was not predictable.

Movement trials

In order to obtain measures of mu activity during performed actions, after the four action-observation blocks, participants performed 80 grasps themselves. A fixation cross appeared on the screen, indicating to the participants to reach and grasp a cup that was placed on the table in front of them. They were asked to lift the cup from the table and put it down again. The cup was the same as that observed in the videos. The handle was pointing to the right, and participants grasped with their right hand. They were instructed to vary between precision and power grip. The interval between one fixation cross and the next varied randomly and was 6.5 s or 7.0 s in the stimulus-offset condition, and 8.5 s or 9.0 s in the stimulus-maintenance condition.

Recording of eye movements

In order to subsequently correct for eye movement artefacts in the experiment, templates of the participants’ eye movements and blinks were created prior to the experiment. Four positions were marked on the frame around the screen: left, right, top, and bottom. A fixation cross was presented in the centre of the screen, followed by an arrow pointing to the left, right, up, or down. Participants were instructed to fixate the centre of the screen and then look in the direction of the arrow and fixate the indicated position on the frame. Participants performed 20 eye movements in each direction (first left, then right, then up, then down). Finally, they were asked to blink when a symbol appeared on the screen and performed 20 blinks.

EEG data recording

The EEG was recorded from 46 channels at the following scalp positions of the 10-10 system (American Encephalographic Society 1994): AFz, F1, F2, F5, F6, F9, F10, FC3, FC4, FCz, FP1, FP2, FT7, FT8, C1, C2, C3, C4, C5, C6, CP1, CP2, CP3, CP4, CP5, CP6, CPz, Iz, O1, O2, P1, P2, P5, P6, P9, P10, POz, T7, T8, TP7, TP8, TP9, TP10. In addition to these scalp electrodes, three facial electrodes were used: Nz, IO1, and IO2. An electrode at position Cz was used as reference, and an electrode at AF4 served as ground.

The EEG recordings took place in an electrically shielded and soundproof cabin. Ag/AgCl ring electrodes were used that were mounted in an elastic cap. Electrolyte gel was applied in the space between electrodes and skin. The impedances were below 5 kOhm. The signals were DC amplified and recorded at a sampling rate of 500 Hz with a 250 Hz low-pass filter.

Prior to the experiment, to demonstrate the importance of being still during EEG recording, participants were shown their own EEG signal and were asked to blink and crunch their teeth in order to see the effects of such movements on the EEG signal, so as to encourage them to refrain from doing so during the experiment. They were asked to relax during the experiment, fixate the centre of the screen, not to move their head, and to refrain from blinking during the presentation of the videos.

EEG data analysis

Eye movement artefact correction

The eye movement data that were recorded before the start of the experiment were split into 2,500-ms segments starting 500 ms before the trigger for an eye movement (i.e., the arrow or the blink symbol) and then filtered with a .01- to 30-Hz bandpass filter. An independent component analysis (ICA) was performed on the segmented and filtered data, and the components for horizontal eye movements, vertical eye movements, and blinks were identified for each participant by checking the event-related ICA data and topographies. The EEG raw data were corrected for eye movements by folding each participant’s EEG raw data with their ICA coefficient matrix and then back transforming the data using a reverse ICA matrix leaving out the eye movement components identified earlier.

Re-referencing, segmentation, filtering

The eye movement-corrected EEG data were average referenced. Then, the data were split into segments of 7 s starting 1 s before the onset of the cup in the video trials, and 1 s before the onset of the fixation cross in the movement trials. The resulting segments were visually inspected for artefacts, and segments with artefacts (e.g., due to DC correction) were excluded from further analysis. The included segments were then filtered with a bandpass filter of 1–30 Hz.

Computation of mu power

The mu power was computed for each trial, each subject, and each electrode site using Complex Demodulation. To this end, the power in the 8- to 13-Hz-frequency band was extracted via the “Frequency Extraction” feature implemented in the Brain Vision Analyzer Software. The extracted 8- to 13-Hz-frequency band power was then averaged across trials for each condition (there were 100 trials each per colour task and grasp task, and 80 trials in the movement condition). Finally, the 8- to 13-Hz-frequency band power was averaged across the four central channels C1, C2, C3, and C4 and was exported for statistical analysis.

Analysis of other frequency bands

To check for potential differences between the two action observation conditions in frequency bands other than the alpha band, power density spectra were computed for a range from 1 to 30 Hz. To this end, the re-referenced data were segmented into 512-ms intervals (each containing 256 data points), which overlapped by 12 ms. All intervals were 1- to 30-Hz bandpass filtered and then subjected to a Fast Fourier Transform (FFT). The resulting power-density spectra were averaged across trials, separately for the two observation task conditions, each interval, and each subject. Then, the power density spectra of the four central channels C1, C2, C3, and C4 were averaged. (To extract mu power from these power density spectra, the average power in the 8- to 13-Hz-frequency band of these central channels could be extracted by interpolating the spectral lines at the FFT band borders. The resulting power value for each time interval and each task condition could be taken as a measure of mu power.)

Analysis of other electrode sites

To analyse potential differences between the two action observation conditions at electrode sites other than the central channels, the power in the 8- to 13-Hz-frequency band was extracted at each electrode site, separately for the attend-grasp and attend-colour task. Then, the t values of the difference between the two observation conditions were computed and plotted in topographical maps.

Results

Behavioural data

Subjects had to estimate in what percentage of trials the rare event had occurred at the end of each block of 50 video trials. For half of the participants, the rare event was the green X in the report-colour condition and the precision grip in the report-grasp condition; for the other half it was the blue X and the power grip. There were 20% rare events in every block. In the stimulus-offset condition, participants’ mean estimates were 16.7% (SD = 1.4%) in the report-colour blocks and 15.3% (SD = 1.4%) in the report-grasp blocks. The estimates in the two conditions did not differ significantly, t(12) = 1.3, P = .21, two tailed. In the stimulus-maintenance condition, participants’ mean estimates were 18.0% (SD = 1.7%) in the report-colour blocks and 17.5% (SD = 2.0%) in the report-grasp blocks, t(13) = .3, P = .81. That is, overall, participants underestimated the frequency of the rare events. Importantly, there were no significant differences between the two tasks, suggesting that the task to later report colour change or later report grasp did not differ in task difficulty.

EEG data during action execution

The EEG data in the movement trials were analysed in order to determine whether mu suppression during action execution could be observed in each of the 28 participants. The onset of a fixation cross on the screen indicated to the participants to grasp a cup that was placed on the table in front of them, lift it up and put it back down again.

The pattern of mu power was visually inspected for all 28 subjects. Twenty-seven subjects showed reduced mu power during movement relative to rest, consistent with the literature reporting mu suppression during action execution (e.g., Hari et al. 1998; Muthukumaraswamy et al. 2004). One subject from the ‘stimulus-offset’ condition group failed to show mu suppression and was therefore excluded from all further analyses.

For statistical analysis, the 8- to 13-Hz-frequency band data extracted from the central channels (C1, C2, C3, and C4) were segmented into non-overlapping intervals of 500 ms of length. The starting point of the first interval was 1,000 ms before the onset of the fixation cross. The drop in mu power from interval 1 (baseline, 1,000 ms to 500 ms before onset of the fixation cross) to interval 4 (grasping action, 500 ms to 1,000 ms after onset of the fixation cross) was taken as a measure of mu suppression. Mu suppression was computed as the ratio of mu power during action execution (interval 4) to mu power during baseline (interval 1; see, e.g., Crawcour et al. 2009; Muthukumaraswamy et al. 2004; Oberman et al. 2008; Ulloa and Pineda 2007). Because ratio values are not normally distributed, a log transform was applied to the ratio. Significant mu suppression was obtained in the action execution trials, t(26) = 8.1, P < .01, two tailed (see Fig. 2).

Fig. 2.

Mu suppression during action execution and during action observation

EEG data during action observation

For statistical analysis of the action observation trials, the 8- to 13-Hz-frequency band data of the central channels C1, C2, C3, and C4 were segmented into non-overlapping intervals of 500 ms of length. The starting point of the first interval was 1,000 ms before the onset of the cup. The drop in mu power from Interval 1 (pre-stimulus, 1,000 ms to 500 ms before onset of the cup) to Interval 8 (observation of the grasping action, 2,500 ms to 3,000 ms after onset of the cup) was taken as a measure of mu suppression. The log ratio of mu power was computed separately for the report-colour and report-grasp condition (see Fig. 2).

Significant mu suppression was obtained in both action observation conditions; t(26) = 4.5, P < .01, two tailed, for the report-colour condition; t(26) = 5.0, P < .01, two tailed, for the report-grasp condition. Most importantly, the two observation tasks differed in the amount of mu suppression: Significantly more mu suppression was obtained when grasp was relevant than when colour was relevant, t(26) = 3.38, P < .01, two tailed.

Because the pre-stimulus intervals in the report-grasp and report-colour conditions tended to differ in the amount of mu power (see next section), mu suppression in the two observation conditions was also computed relative to a common baseline. To this end, the baseline from the action execution condition was used. The same data pattern was obtained as before: There was significant mu suppression in the report-colour condition relative to the common baseline, mean log ratio = −.17 (SE = .04), t(26) = 4.1, P < .01, two tailed, and in the report-grasp condition relative to the common baseline, mean log ratio = −.21 (SE = .04), t(26) = 5.2, P < .01, two tailed. Again, the mu suppression in the report-grasp condition was significantly larger than in the report-colour condition, t(26) = 2.1, P < .05, two tailed.

Analysis of whole trial length

In addition to the standard analysis of mu suppression (with either a separate pre-stimulus baseline for each condition, or a common baseline), mu power was analysed across the whole trial length (see Fig. 3). Fourteen intervals of 500 ms of length were computed, their starting points relative to the onset of the cup being −1,000 ms, −500 ms, 0 ms, 500 ms, 1,000 ms, 1,500 ms, 2,000 ms, 2,500 ms, 3,000 ms, 3,500 ms, 4,000 ms, 4,500 ms, 5,000 ms, and 5,500 ms. During the Intervals 1 and 2, there was a blank screen. At the beginning of Interval 3, the cup appeared and stayed on the screen during the Intervals 4, 5, and 6. During Interval 7, the hand started to appear from the right. The grasp happened at the beginning of Interval 8. During Interval 9, the picture of the hand grasping the cup (the last frame of the video) stayed on the screen. In the stimulus-offset condition, the picture disappeared at the beginning of Interval 10, and the screen remained blank during the Intervals 11, 12, 13, and 14, whereas in the stimulus-maintenance condition, the cup remained visible during Intervals 10 to 14. For the stimulus-offset condition, the first and the last interval were partly overlapping, because the entire trial length varied randomly between 6,500 ms and 7,000 ms. For the stimulus-maintenance condition, the first and the last interval did not overlap, because the overall trial length varied between 8,500 ms and 9,000 ms.

Fig. 3.

Mu power during action observation across the whole trial length, separately for the two tasks (later report grasp/later report colour) and stimulus offset/maintenance condition. Error bars denote the standard error of the mean. Mu power is computed as the mean power in the 8- to 13-Hz-frequency range over central channels (C1, C2, C3, and C4)

For statistical analysis, the intervals 1–2 were summarised as pre-stimulus period (column i in Fig. 1). Intervals 3–6 were summarised as observation of cup period (column ii), intervals 7–9 as observation of grasp period (column iii), and intervals 10–14 as post-grasp period (column iv). The mean mu power for each period was computed as the average mu power across the respective intervals, separately for the report-colour and report-grasp condition, and for stimulus-offset and stimulus-maintenance condition.

In all following analyses of variance (ANOVAs), significance was tested at an alpha level of .05. In case of violation of the sphericity assumption, the Huynh–Feldt correction was applied to the degrees of freedom (df). For sake of simplicity, the uncorrected df are reported, together with the corrected P values.

Combined analysis of stimulus-offset and stimulus-maintenance condition

A three-way ANOVA with the within-subject variables period (i–iv) and task (colour/grasp) and the between-subject variable condition (stimulus offset/maintenance) revealed a main effect of period, F(3,75) = 10.3, P < .01, η ² _p = .29, indicating that mu power differed across the different periods, and a significant interaction of period and task, F(3,75) = 3.9, P = .04, η ² _p = .14, indicating that the modulation of mu power across the periods was different for the report-colour and report-grasp task. No other main effects or interactions were significant; Fs < 1 for the main effects of task and condition, and the interaction of task and condition, F(3,75) = 1.38, P = .26, η ² _p = .05 for the interaction of period and condition, F(3,75) = 1.1, P = .32, η ² _p = .04, for the three-way interaction.

To further investigate the interaction of period and task, pre-planned contrasts were computed comparing adjacent periods (i.e., comparing periods i and ii, ii and iii, as well as iii and iv). There was a marginally significant contrast for the interaction of task and periods i and ii, F(1,25) = 4.0, P = .06, η ² _p = .14, indicating a steeper decline in mu power from period i to ii during the report-grasp task (from 4.75 μV² to 3.01 μV²) than during the report-colour task (from 4.41 μV² to 3.12 μV²). The contrast for the interaction of task and periods ii and iii was significant, F(1,25) = 6.3, P = .02, η ² _p = .20, again indicating a steeper decline in mu power during the report-grasp task (from 3.01 μV² to 2.45 μV²) than during the report-colour task (from 3.12 μV² to 2.76 μV²). The contrast for the interaction of task and periods iii and iv was also significant, F(1,25) = 14.8, P < .01, η ² _p = .37, indicating a steeper increase in mu power from period iii to period iv during the report-grasp task (from 2.45 μV² to 3.74 μV²) than during the report-colour task (from 2.76 μV² to 3.68 μV²).

Post-hoc t-tests comparing the two tasks directly in the different periods revealed that mu power differed significantly between the two tasks only during period iii (observation of grasp), t(26) = −2.5, P = .02, two tailed, with less mu power for the report-grasp task than for the report-colour task. Mu power did not differ significantly between the tasks during the other periods (period i: t(26) = 1.4, P = .18, two tailed; period ii: t(26) = −1.2, P = .26, two tailed; period iv: t(26) < 1).

Separate analysis of stimulus-offset and stimulus-maintenance condition in period iv

The above contrast analysis investigated the interaction of task and period, thereby averaging across the two conditions (stimulus offset and stimulus maintenance). This seems warranted for the periods i to iii, where the two conditions are identical. However, this kind of analysis obscures the differences between the conditions during period iv, where there is different visual input in the two conditions: The cup is no longer visible in the stimulus-offset condition but remains visible in the stimulus-maintenance condition.

To account for this, contrasts for the three-way interaction were computed, again comparing adjacent periods. As expected, the respective three-way contrasts comparing periods i and ii, as well as ii and iii, were not significant, Fs < 1. Importantly, the contrast for the interaction of condition, task, and periods iii and iv was significant, F(1,25) = 4.9, P = .04, η ² _p = .16, indicating that the above described steeper increase in mu power from period iii to period iv during the report-grasp task than during the report-colour task was modulated by condition. In the stimulus-offset condition, the mu power increase was larger during the report-grasp task (from 2.31 μV² to 4.31 μV²) than during the report-colour task (from 2.53 μV² to 3.94 μV²). In the stimulus-maintenance condition, the mu power increase was not different for the different tasks (from 2.57 μV² to 3.21 μV² for the report-grasp task, from 2.97 μV² to 3.44 μV² for the report-colour task). This was confirmed by statistical analysis conducted separately for the stimulus-offset and stimulus-maintenance condition. The two-way interaction of task and period (iii, iv) was highly significant in the stimulus-offset condition, F(1,12) = 12.4, P < .01, η ² _p = .51, but not in the stimulus-maintenance condition, F(1,13) = 2.3, P = .15, η ² _p = .15. In the stimulus-offset condition, there was significantly more mu power in period iv for the report-grasp task than for the report-colour task, t(12) = 5.7, P = .04, two tailed. In the stimulus-maintenance condition, mu power in period iv was not significantly different for the two tasks, t(13) < 1.

Analysis of frequency spectrum

To this point only the alpha band (8–13 Hz) has been analysed. To check for possible differences between the two tasks in other frequency bands, such as the beta band (14–20 Hz), the power density spectrum at the moment of observed grasp (interval 8) was computed separately for the two tasks (see “Method” section for details). As can be seen from Fig. 4, the two tasks only differ in the 8–13 Hz frequency band.

Fig. 4.

Mean power density spectrum of central channels (C1, C2, C3, and C4) at the moment of grasp (Interval 8 in Fig. 3) for the combined data of stimulus offset/maintenance condition, separately for the report-colour task and report-grasp task

Analysis of other electrode sites

To check for potential differences between the two tasks at other electrode sites, the t values of the difference between the two observation conditions were computed and plotted in topographical maps. Two critical time windows were chosen for this analysis: the observation of grasp (interval 8) and the post observation period (interval 12). Negative t values indicate less mu power in report-grasp than report-colour task, positive t values indicate more mu power in report-grasp than report-colour. Figure 5 shows that during the observation of the grasp, the difference between the two tasks was most pronounced over left central channels, confirming that the difference is largest over motor areas. That the difference is left-lateralised is presumably due to the stimulus material being lateralised, with a right hand appearing from the right of the screen to grasp the cup, and the cup handle pointing to the right).

Fig. 5.

Topographical map showing the t values of the difference in 8- to 13-Hz power between report-colour task and report-grasp task during Interval 8 (observation of grasp) for the combined data of stimulus offset/maintenance condition (df = 26). Negative t values indicate less mu power for the report-grasp task than report-colour task

The second critical time window is the post-grasp period (interval 12). As Fig. 6 shows, in the stimulus-offset condition the difference between the tasks is again most pronounced over central sites, confirming that the inhibitory rebound is a motor phenomenon. The rebound is observed over both left and right central channels. In the stimulus-maintenance condition, the cup remains on the screen during interval 12, and no rebound occurs. As can be seen from Fig. 6, there is no significant difference between the two tasks at any electrode sites in this condition.

Fig. 6.

Topographical maps showing the t values of the difference in 8-to 13-Hz power between report-colour task and report-grasp task during Interval 12 (post observation of grasp), separately for stimulus-offset condition (df = 12), and stimulus-maintenance condition (df = 13). Positive t values indicate more mu power for the report-grasp task than report-colour task

Discussion

Synopsis

In the current study, we measured participants’ motor system activation while they were observing grasping actions. It was investigated whether the motor system activation was greater when the observed grasps were relevant to the observer’s task than when they were not. We therefore manipulated whether the participants had to make a later judgment about the observed grasp, or about a visual feature (colour) not related to the action.

The visual input was identical in both conditions: Participants watched videos of a hand grasping a cup. At the moment of the grasp, a colour change occurred on the cup. The participants had to either make judgments about the grasp, or about the colour change.

We measured participants’ EEG activity while they were watching the videos. Motor system activity is indicated indirectly by changes in the power of the 8- to 13-Hz-frequency band over central channels. In particular, when the motor system is not active, there is oscillatory activity in the 8–13 Hz frequency band over central channels (mu rhythm). These oscillations become desynchronised when the motor system is activated, resulting in a drop of power in this frequency band (mu suppression; Babiloni et al. 2002; Cochin et al. 1999; Pineda 2005). Consistent with the literature, we found mu suppression when participants observed objects that evoked actions and subsequent grasping actions (Cochin et al. 1999; Hari et al. 1998; Muthukumaraswamy and Johnson 2004; Pfurtscheller et al. 2006). The reduction in mu was detected when subjects observed the action and when they performed the grasping actions. Note that when they performed the action, they also observed their own action; whereas in the action observation conditions, they only observed, but did not produce any overt action.

Importantly, the amount of mu suppression during the observation of the grasping actions differed between the two tasks. Participants showed stronger mu suppression during the grasping action when they were later judging the grasp than when later judging the colour change. There was significant mu suppression when initially observing the cup, which indicates motor system activation evoked by the observation of a graspable object (see Chao and Martin 2000; Grezes and Decety 2002; Grezes et al. 2003; Tucker and Ellis 1998). However, when comparing the colour- and grasp condition directly, there is no significant difference in mu power between attend-colour and attend-grasp during the pre-stimulus period or during observation of the cup, but there is a significant difference (less mu power when attending grasp) during observation of the action. This suggests that the impact of task relevance of the observed action is largest during the observed action itself. The difference between attend action or colour during the moment of grasp was predominant over left central channels. This lateralisation during action observation presumably reflects the observation of lateralised stimuli: the observed grasp was always a right-hand grasp viewed from an egocentric perspective, and the hand always appeared from the right.

We also examined mu at the end of the trial, in the between trial resting period. There were two conditions: “stimulus offset” where the visual display of the cup was terminated, and “stimulus maintenance”, here the cup remained visible. In the former stimulus-offset condition, participants showed an enhancement of mu power after the end of the video, indicating an inhibitory rebound effect. This rebound effect was stronger when participants had to identify the grasp rather than the colour. The rebound effect presumably reflects inhibition of the motor system when its activation is terminated, where the stronger the previous activation, the stronger the subsequent inhibition, consistent with existing theories of reactive self-inhibition of activated neural states (e.g. Houghton and Tipper 1994; Houghton et al. 1996). Further support for the reactive inhibition account was provided by the stimulus-maintenance condition, where no rebound into mu synchronisation, and no difference between attend action or colour, was observed when perceptual inputs were maintained. Interestingly, the difference between attend action versus colour during the rebound occurred bilaterally over left and right central channels, suggesting that attending to the grasp leads to a larger subsequent rebound of the motor system bilaterally, even when during the observation of the grasp, larger mu suppression when attending action versus colour was lateralised due to the lateralisation of the observed stimuli.

Task relevance and motor system activation during action observation

The goal of this study was to investigate whether action tendencies evoked by the observation of an action depend on the relevance of the perceived action to the observer’s task. We found mu suppression when participants observed a grasping movement even when their task was to later report a colour change. This supports the idea that vision is automatically converted into action. Note that colour is a stimulus dimension unrelated to action; moreover, there was no overt behaviour, so no possibility that produced actions could feedback and influence visual processes. Still, a mu suppression effect was obtained, showing that action tendencies are evoked automatically. This is consistent with a recent study showing mu suppression when attending to colour changes in point-light walker displays (Ulloa and Pineda 2007), and with a recent behavioural study showing compatibility effects between observed body parts (hands or feet) and the body parts used for responding, while attending to colour changes on the observed body parts (Bach et al. 2007).

In the present study, we directly compared mu suppression when colour was relevant and when the observed action was relevant. We found more motor activation when the observed action was relevant. This finding relates to a meta-analysis comparing fMRI data from action-observation tasks and imitation tasks (Molnar-Szakacs et al. 2005). The meta-analysis revealed differential activation patterns within the inferior frontal gyrus, which is thought to be part of the human mirror system. Again, the task relevance of the observed actions might have played a role: Whereas during imitation the observed action is clearly task relevant, it is not necessarily relevant when the task is to passively view the actions.

Observation of objects and observation of actions

In the current study, participants first saw the to-be-grasped object and only after 2 s did they see the hand appear and grasp the object. Mu suppression was found in both stages of processing. Somewhat surprisingly, the larger effect was found when the object appeared, and only marginally more mu suppression was found when the hand subsequently appeared and grasped the object. The influence of the object on the motor system was also observed at the end of the trial. That is, continued viewing of the cup prevented the usual rebound of mu power.

The finding that viewing an object automatically evokes the motor system supports previous work (Tucker and Ellis 1998). These authors reported that actions related to an observed object were automatically encoded and could influence unrelated behaviours. For instance, when viewing a coffee cup, and deciding whether the cup was the correct orientation or inverted, a right-hand key press response would be facilitated if the handle used to grasp the cup was oriented towards the right hand. Hence, grasping the object was irrelevant and not part of the task, yet information suitable for grasping the object seems to be made available to the motor system.

In the present study, there was only marginally more mu suppression when viewing the action relative to viewing the object. However, it is important to note that during action observation, the effects of attending to action versus attending to colour emerged, where mu suppression was greater in the former case. This study was not designed to discriminate mu suppression produced by vision of an action-evoking object such as a cup versus vision of an action directed to the object. Further work will be necessary to untangle the effects of viewing objects and actions on the motor system. Nevertheless, we have shown that manipulating task goal (discriminate action or colour) is one means of discriminating effects of objects from actions directed towards objects.

Motor system inhibition following activation

We compared two conditions, one where the stimulus display was offset and another where the stimulus was maintained. In the latter condition, while the action affording object remained visible there was no rebound of mu power, and the differences between attend action versus attend colour declined over Intervals 10 and 11 (see Fig. 3). In contrast, in the stimulus-offset condition, not only did the two task conditions (attend action and attend colour) differ during the observation of the grasping video, they also differed during the rest period after the offset of the display. The latter is a striking contrast, because at that time no visual stimulus was presented, and participants were not involved in any task while they were passively resting between trials. In particular, the increase in mu power after the end of the video was larger when the observed action was relevant to the observer’s task than when it was not. This rebound of mu power presumably reflects inhibition of previously activated actions, which is proportional to the previous activation state of the mirror system. Thus, when the observed action is relevant to the observer, the mirror system becomes more activated and subsequently more inhibited.

Interestingly, in the stimulus-offset condition the numerically largest difference between the two conditions of action or colour discrimination occurred after the event, during the inhibitory rebound period, not during the observation of the action. This is again consistent with the reactive inhibition model of Houghton and Tipper (1994) and Houghton et al. (1996), which holds that the inhibition is proportional to the excitation of a certain representation, and therefore excitation and inhibition states can be similar as long as the stimulus is present. After the offset of the stimulus, however, the excitatory input is absent, leading to a temporary overshoot of inhibition.1 Consistent with this model, the largest difference between the two conditions occurred after, not during, the observation of the cup and grasping action.

That inhibition is involved in action observation processes has been suggested in the literature (e.g. Brass et al. 2001). For example, in a recent study recording single unit responses in humans, Mukamel et al. (2010) observed two classes of cells. First, evidence for the classic mirror cell was obtained, where there was response both when producing an action and when merely observing the same action. But of more importance, in the same regions inhibitory control cells were observed. That is, although they were active when producing an action, they were suppressed when observing the same action.

The functional role of such cells is clear. Thus, if not controlled by inhibitory processes, the automatic activation of the motor system when observing an object that evokes an action, or an action directed towards an object, could result in overt mimicry of the viewed behaviour. This would cause serious problems, as an individual’s behavioural goals would be constantly hijacked by other behaviours that were viewed, and by action-evoking objects in their immediate environments. Therefore, there must be inhibitory control systems that prevent simulated motor processes producing overt motor acts. The frontal lobes appear to play a key role in this inhibitory control. For example, individuals with lesions of the frontal lobes produce utilisation behaviour, where they cannot prevent actions towards viewed objects, or imitation of viewed behaviour (e.g. Lhermitte 1983; DeRenzi et al. 1996; Luria 1980). These individuals are aware that no action should be produced, but they appear to lack the inhibitory control that blocks the automatic conversion from vision to action from capturing their own behaviour. In an fMRI study (Brass et al. 2001), it was confirmed that prefrontal areas such as right middle frontal gyrus are involved in response inhibition when viewing another person’s actions as well as involving connected brain regions in parietal cortex.

The approach of measuring quantitative EEG to investigate action observation processes might be of use in other studies where automatic action tendencies are not detected. In particular, measuring mu power at a time that is not typically investigated, which is after trial completion, might be especially revealing. For example, the idea that the human mirror system is in some way operating sub-optimally in people with autism spectrum disorders (ASD; Williams et al. 2001) has found support from studies showing less mu suppression in ASD (Bernier et al. 2007; Oberman et al. 2005, 2008). However, an examination of the subsequent rebound state might reveal larger effects in ASD, suggesting not reduced action tendencies, but larger inhibitory blocking of these processes that can only be detected after stimulus offset.

In conclusion, the present study provides evidence that although motor system activation during action observation can be automatic, the effect nevertheless can depend on the observer’s task to some extent. When the observed action is relevant to the observer, the mirror system is more strongly activated while the action is perceived, and more strongly deactivated afterwards. We feel that it is interesting that the clearest effects of the role of task relevance in action observation are revealed when no stimulus is actually presented and no overt behaviour is produced while participants are in a relaxed state between trials. Clearly, measurement of neural processes via EEG can detect processes such as automatic action tendencies and inhibitory feedback control that might not be detectable in behaviour.