Repetition suppression for performed hand gestures revealed by fMRI

Abstract

The functional components of the human motor system that are used to retrieve and execute simple intransitive hand gestures were identified with a repetition suppression (RS) paradigm. Participants performed movements with the right hand to text instructions in a rapid event related design with a pseudo‐random stimulus order. Brain areas associated with action retrieval were identified by comparing trials where an action was repeated to trials that involved a new action. Performance of a novel action, collapsed across individual actions, resulted in significantly greater activity in a left hemisphere predominant fronto‐parietal circuit involving inferior frontal gyrus and inferior parietal cortex (supramarginal gyrus). This is consistent with previous action retrieval tasks using go, no‐go paradigms and lesion studies of patients with apraxia that emphasize a role of these areas in action organization. In addition, RS effects were present in left primary sensorimotor cortex. These effects cannot be ascribed to kinematic differences, simple action related activity or differences of cognitive set. Significant RS effects for action retrieval could be identified with as little as 5 min of fMRI data and underscores the potential of using RS to characterize representational structure within the motor system. Hum Brain Mapp, 2009. © 2008 Wiley‐Liss, Inc.

INTRODUCTION

Performing even the simplest of hand gestures requires a complex cascade of control processes in the brain. For example, making a “thumbs up” sign is likely to require selection and retrieval of the proper hand posture, planning of the kinematic sequence which will achieve that posture, and ultimately, activation of many different muscles in a precisely timed progression. It has long been argued that this cascade of selection, planning and execution is distributed across dedicated premotor areas of the neocortex [Passingham, 1993]. In support of this argument neurophysiologic recordings of single neurons within motor areas of non‐human primates performing simple motor tasks such as reaching or grasping have identified neuronal selectivity that distinguish some of these processes, including planning, sequencing, and the execution of simple actions [Cisek et al. 2003; Crammond and Kalaska, 1994, 2000; Georgopoulos et al., 1999; Kalaska et al., 1989; Shen and Alexander, 1997a, b]. This functional decomposition of action, based on the temporal firing properties of individual neurons, can be readily identified in multiple premotor areas and motor cortex. A challenge is to extend this neurophysiologically based decomposition to natural actions that people perform. There are two hurdles that make this difficult. First, the complexities of many human behaviors, such as use of tools or communicative gestures have relatively few counterparts in the non‐human primates [Obayashi et al., 2001, 2002]. Second, the non‐invasive methods available for human neurophysiology lack the temporal resolution to establish neural selectivity analogous to what can be done with single unit recordings. The current paper investigates the utility of repetition suppression combined with functional magnetic resonance imaging (fMRI) as an alternative method to distinguish component planning processes without reliance on the high temporal resolution used in neural recordings.

The classic approach to investigate complex actions in humans is through the lens of neuropsychology and deficits from focal lesions. Behavioral studies of apraxia patients with left parietal or caudal middle frontal lobe lesions identify deficits in the retrieval and execution of actions based on verbal commands [Haaland et al., 2000; Heilman et al., 1982] imitation [Goldenberg et al., 1996] or object centered action goals [Buxbaum and Saffran, 2002]. It remains challenging to determine if the deficit in these cases is secondary to semantic access to a particular action, the organization of an action plan or the actual execution of a motor act. Studies using fMRI and conventional cognitive subtraction methodology in normal individuals show similar cortical areas activated during gesture execution and imitation [Chaminade et al., 2005; Muhlau et al., 2005]. Some studies have tried to tease apart planning and execution components required for the production of complex actions such as transitive and intransitive gestures [Fridman et al., 2006; Krams et al., 1998]. For example, one fMRI study had subjects make hand gestures to instructions denoting conventional tool actions such as “hammer” or “write.” By introducing a delay between the instruction and subsequent movement, combined with a go/no‐go paradigm it was argued that semantic retrieval and planning of the hand gestures could be isolated in the no‐go trials. Localization was largely concordant with classic patient studies [Johnson‐Frey et al., 2005]. Other approaches, such as contrasting precision versus power grasps, reaching versus grasping, executing simple versus complex actions or planning based on different types of instructions have all been used to dissect human planning systems [Culham et al., 2003; Ehrsson et al., 2000; Gorbet et al., 2004; Passingham et al., 1998]. While important, these approaches suffer from differences of movement kinematics, task difficulty or cognitive requirements, limiting the degree to which different motor control processing can be disambiguated.

Neither patient studies nor “cognitive subtraction” paradigms in fMRI are able to decompose action organization at a finer grained level of analysis. For example, these methods cannot distinguish goal selection from kinematic specification, because both of these two components are always required to act. Studies of visual neuroscience may provide novel ways to tackle this problem. One possible approach is to use multivariate pattern recognition, which can distinguish orientation coding in primary visual cortex as well as task selection from frontal cortex [Haxby et al., 2001; Kamitani and Tong, 2005, 2006]. However, this method requires distinct conditions for training the pattern recognition algorithm, which cannot easily be obtained for complex motor acts.

An alternative is to measure the repetition suppression (RS) of the fMRI blood oxygen level dependent (BOLD) signal when a particular stimulus feature is repeated from one trial to the next [Grill‐Spector et al., 2006]. The RS method, also known as fMRI‐adaptation, is promising because it is not limited to visual systems. It has been used to distinguish different levels of processing in many cognitive systems, including number [Naccache and Dehaene, 2001], syntax [Noppeney and Price, 2004] and semantics [Dehaene et al., 1998]. Recently, we and others have used the RS method to dissociate neural substrates for decoding different levels of action perception and action understanding. For example, we have demonstrate RS in anterior intraparietal sulcus when participants repeatedly observe the same action goal [Hamilton and Grafton, 2006], and RS in inferior frontal gyrus when participants repeatedly observe grasps with the same hand aperture [Hamilton and Grafton, 2007]. These results indicate that neural populations in aIPS selectively encode the goal of an action, while populations in IFG encode the kinematic features [Grafton and Hamilton, 2007]. Other studies of RS for observed actions have drawn similar conclusions [Shmuelof and Zohary, 2005].

In addition, two recent RS studies have examined the cortical systems supporting the excution of simple motor actions. In one study, Dinstein et al. [2007] had subjects play rock‐paper‐scissors against a video. They only saw the hand of the competitor. In this task repetition could occur for what was executed or what was observed. They demonstrated that repeated execution of a freely‐chosen hand action led to suppression of BOLD signal in a network of frontal and parietal regions associated with the mirror neuron system as well as the sensorimotor cortex contralateral to the performing hand. In this game situation subjects were playing for rewards and attention was high across trials. In this paradigm the repetition for an executed movement also included RS for strategic response selection, anticipatory guessing, hand gesture construction and task execution. In a second RS study, Kroliczak et al. [2008] had subjects grasp of simple natural objects and demonstrated a reduction of BOLD signal in bilateral anterior intraparietal sulcus whether repeating object orientation or object shape. The left ventral premotor cortex (PMv), and bilateral dorsal premotor cortex (PMd), showed only grasp‐selective adaptation.

The current study aims to extend these results to a gesture retrieval paradigm. To test this, subjects were required to retrieve, plan and execute simple intransitive hand gestures indicated by written instructions performed with the right hand. The task is simpler than the study by Dinstein [2007] because there is no strategic planning or game playing and there is no observed movement of another player. On the other hand, it is more complex than the study by Kroliczak [2008] in that subjects are required to generate intransitive gestures from memory, rather than automatic retrieval associated with visually presented objects.

It was predicted that RS for gesture retrieval would be overlap with the results of Dinstein, but would be more strongly associated with fronto‐parietal cortex linked to gesture retrieval: the inferior parietal lobule and inferior frontal gyrus in particular.

METHODS

Participants

Nineteen normal right‐handed young adult subjects (mean age 20) performed the RS motor task. All participants had corrected normal vision, and all gave informed consent as required by the local ethics committee.

Task

During fMRI, participants performed simple, right‐handed actions as instructed by a single written word in the center of the projection screen. Four possible instruction words were used: “FIST,” hold the hand in a fist; “THUMB” stick the thumb out with the hand in a fist; “STRETCH,” extend all the digits; “POINT,” point with the index finger. Each instruction was presented for 4 seconds, and participants had to move their hand to that posture when the instruction appeared and hold the posture. In between trials, the word “REST” appeared on the screen for 4 seconds, and participants adopted a relaxed posture of the right hand. Instructions were presented using Cogent running in Matlab and compliance was monitored by observing the participant's hand actions from the scanner control room.

A total of 36 trials were presented in four sets of nine. Each set used one pair of actions (e.g., Fist and Point), and the first action of the set was classified as “New” and not analyzed. The remaining 8 actions were ordered such that the set contained 4 novel actions (relative to the one trial before) and 4 repeated actions (relative to the one trial before) in a pseudorandom sequence. This one back RS structure provides a robust method for measuring changes in BOLD signal for repeated compared to novel performed actions. The total trial sequence took approximately 5 min to complete.

Imaging

One hundred and fifty whole brain images were acquired in a single, 5 min BOLD functional MRI scan. Data was collected using a 3T Philips Achieva Quasar Dual 8 channel scanner using an eight channel phased array coil and 30 slices per TR (4 mm thickness, 0.5 mm gap); TR: 2000 ms; TE: 35 ms; flip angle: 90o; field of view: 24 cm; matrix 80 × 80. In addition, a high resolution anatomical scan was collected of the whole brain was using a spoiled gradient recalled 3‐D sequence (TR = 9.9 msec; TE = 4.6 msec; flip angle = 88, FOV = 240 mm; slice thickness = 1 mm, matrix = 256 × 256).

Analysis

Data were realigned, unwarped and normalized to the MNI template with a resolution of 2 × 2 × 2 mm in SPM2. A design matrix was fitted for each subject with one regressor for each action type (novel or repeated; fist, stretch, point, thumb). Each trial was modeled as an event of 4 seconds duration, convolved with the standard hemodynamic response function. The design matrix weighted each raw image according to its overall variability to reduce the impact of movement artifacts [Diedrichsen and Shadmehr, 2005]. After estimation, 9 mm smoothing was applied to the beta images. To identify brain regions showing repetition suppression for repeated actions, we calculated a contrast for the main effect of repetition (novel action > repeated action) over all types of actions. Contrast images for all 19 participants were taken to the second level for a random effects analysis. We report regions that survive a threshold of P < 0.001 uncorrected in clusters of more than 100 voxels in the Table I and in the figures. The discussion focuses on regions which met the whole brain cluster‐corrected threshold [Friston et al., 1994].

Table I.

Coordinates of brain regions showing repetition suppression for performed actionsa

Region	MNI coordinates			P (cluster correction)	Number of voxels	P (peak voxel, FDR corr)	T (peak voxel)
	x	y	z
Cluster spanning parietal‐frontal cortex				<0.001	14052
Right medial parietal/precuneus	12	−78	46			0.002	8.17
Left mid‐intraparietal sulcus	−32	−38	40			0.002	6.91
Left anterior intraparietal sulcus/post‐central sulcus (aIPS)b	−54	−24	38			0.002	6.9
Left superior frontal sulcus (dorsal premotor)b	−26	2	56			0.002	6.29
Left posterior intraparietal sulcus	−28	−54	66			0.002	6.07
Right superior frontal gyrus	26	10	66			0.003	5.96
Right medial parietal/precuneus	14	−54	64			0.003	5.91
Right supramarginal gyrus	64	−24	20			0.003	5.89
Left posterior superior temporal sulcus	−46	−52	12			0.004	5.39
Right posterior intraparietal sulcus	34	−56	56			0.004	5.31
Left parieto‐occipital fissure	−18	−74	32			0.004	5.25
Left central sulcus (primary sensorimotor)b	−36	−22	58			0.004	5.19
Right mid intraparietal sulcus and inferior parietal lobule	40	−32	38			0.004	5.19
Right medial segment of superior frontal gyrus (SMA)b	6	12	62			0.004	5.17
Right precentral gyrus (PMv)	64	4	30			0.004	5.11
Left anterior cingulate sulcus	−10	20	38			0.005	4.63
Right precentral sulcus/middle frontal gyrus	42	6	40			0.007	4.23
Right precentral sulcus (PMd)	50	−4	58			0.008	4.09
Right posterior superior parietal lobule	20	−60	36			0.009	3.99
Left supramarginal gyrus	−54	−22	14			0.010	3.85
Other clusters
Right lateral cerebellar cortexb	46	−46	−32	<0.001	452	0.002	7.99
Left posterior inferior cerebellar cortex	−36	−54	−50	<0.001	1556	0.002	6.55
Left posterior inferior cerebellar cortex	−18	−76	−52			0.003	5.67
Left anterior lateral cerebellar cortex	−38	−74	−24			0.004	5.33
Left anterior insula/IFG	−34	20	−10	0.006	284	0.002	6.27
Left posterior orbitofrontal cortex	−12	16	−18			0.006	4.44
Right posterior inferior cerebellar cortex	12	−68	−50	0.132	132	0.004	5.32
Left rostral middle frontal gyrus	−34	50	30	0.267	100	0.004	5.18
Right anterior cingulate sulcus	4	4	38	0.033	197	0.004	5.06
Left insula	−38	−8	8	0.071	161	0.005	4.71
Left precentral gyrus (PMv)	−56	0	32	0.077	157	0.007	4.15

^aAll peaks in each cluster (more than 20 mm apart) are listed.

^bPeaks in predicted regions that are plotted in Figure 2.

RESULTS

Repetition suppression for generating the same action (novel action > repeated action), collapsed across all action types is shown in Figure 1. Note that these results do not show movement versus rest. They identify areas of greater activity when an action is no longer repeated, i.e. the escape from suppression. This contrast identified a large cluster of over 14,000 voxels spanning the bilateral frontal‐parietal cortex. This cluster included areas classically associated with action planning and execution, including intraparietal sulcus (IPS) and adjacent inferior parietal lobule (IPL) and supramarginal gyrus (SMG), middle frontal gyrus (MFG) and dorsal premotor cortex (PMd). In all these areas there is a greater involvement of the left hemisphere. Critically, there is also evidence for an RS effect in left primary motor cortex and somatosensory cortex, contralateral to the moving hand. RS is also observed in supplementary motor area (SMA) and cingulate motor area (CMA) on the medial wall of both hemispheres. Subcortical areas demonstrating an RS effect included the right anterior cerebellar cortex. We report the locations of all the activation peaks found within the RS for action contrast at P < 0.001 uncorrected and 100 voxels in Table I. Note that all the locations reported in Table I also met the FDR‐corrected threshold, and that five of the clusters met the whole brain cluster‐corrected threshold. Thus, the results we report are robust by several different standards. A supplementary analysis where each trial was modeled with a 1 s duration rather than a 4 s duration gave a very similar pattern of results.

Figure 1.

Repetition suppression for performed action over the whole brain. RS for performed hand actions was found in the classic visuo‐motor circuit including left primary motor cortex, left inferior parietal cortex, premotor cortex, SMA and right cerebellum. Results are thresholded at p<0.001 uncorrected and 100 voxels. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

To allow visualization of the BOLD signal within these large clusters, we generated regions of interest (ROI) at representative peak locations listed in Table I. Each ROI was defined as the mean of values for all the voxels showing an RS effect within 8 mm of the peak; data was extracted from each ROI and plotted as a post‐stimulus time histogram in Figure 2. As expected, the figure shows stronger responses when performing novel actions compared to repeated actions in all regions. The magnitude of the RS effect is less for motor cortex than other nodes of the motor network including IPS, PMd and SMA, but it is also significant.

Figure 2.

RS in specific clusters. Post‐stimulus time histograms are shown for five clusters. In each case, data was extracted from the yellow region to generate the plot. Red blobs on the brain slices illustrate all regions involved in the novel action > repeated action contrast. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

DISCUSSION

The results of this study clearly demonstrate that suppression of the BOLD signal occurs across the motor network when an action is repeated for a second time. This effect was observed throughout the motor system, including primary motor cortex, premotor cortex, supplementary motor area parietal cortex and cerebellum. These results have broad implications for interpreting functional imaging studies of motor control and learning. More importantly, the existence of RS effects in motor systems that is detectable with even a small number of trials creates enormous potential to spark a range of future experiments designed to further distinguish representational levels of planning and control. For example, it should be possible to use an RS design to discriminate different levels of the motor hierarchy, such as goal, kinematic and muscle representations. One innovative study recently localized the size‐weight illusion using this approach [Chouinard et al., 2008].

Methodological Issues

Before exploring the implications of our findings, we address methodological issues which could impact on the interpretation of the data. First, the current study involved only a brief (5 min) scan of each participant. While this relatively small amount of data might seem to limit our conclusions, it also reveals an advantage. Past experience suggests that protocols with long sequences of trials in an RS design have reduced power, because by the end of the sequence, even “novel” stimuli will have been experienced several times before. That is, RS can occur along multiple time constants. By using short protocols we previously demonstrated robust RS effects for action observation [Hamilton and Grafton, 2008]. The current results, with significant RS effects identified after only 36 trials, establishes the feasibility of using short sequences for action execution paradigms as well. Future studies can capitalize on the relative efficiency for measuring RS effects with abbreviated sets of trials.

Second, we used visual written words as instruction stimuli, and the words were repeated when the actions were repeated. Thus, it is possible that some of the RS effect we observe, in particular in visual regions, could be due to repetition of the visual stimulus. Prior studies have found RS for written words in extrastriate regions [Dehaene et al., 2001], but RS for words does not extend to motor regions in frontal and parietal cortex, primary motor cortex and cerebellum. This means that the major effects we report are unlikely to be due to visual RS. However, future studies could systematically vary the stimulus modality to investigate the interaction of visual and motor RS.

Third, the present study did not collect kinematic data during scanning. Performance was monitored by video to ensure overall task compliance and no systematic differences between trials could be detected. Normal movement variability from trial to trial was present, and the RS effect observed was robust to this variability. As an example, consider a sequence of three trials FIST1–FIST2–THUMB1, where we observe repetition suppression on trial 2 and a release from suppression on trial 3. The release from suppression on trial 3 occurs because the actions FIST and THUMB differ substantially, in both kinematics and planning. If the FIST1 and FIST2 trials also differed, we would see a similar failure of suppression on trial 2, and our contrast of novel action > repeated action would not yield a significant result. That is, movement variability within an action type could only act to reduce our effect size, not to generate false positives. The fact that robust RS effects can be observed even with normal movement variability means that RS in the human motor system does not depend on precise replication of an action from one trial to another. This is in itself an important result, because if motor RS could only be obtained in conditions with a perfect kinematic match, its utility as a research tool would be limited. This robustness is concurrent with the one previous report of RS in motor systems [Dinstein et al., 2007], which also found evidence of RS without precise kinematic control. It is also coherent with studies of visual systems, which find that RS is not disrupted by large changes in an irrelevant stimulus dimension [Grill‐Spector and Malach, 2001], nor by small changes in a relevant dimension [Naccache and Dehaene, 2001].

Neurophysiological Mechanisms of Motor RS

Our data raises the critical question of what changes in neuronal activity underlie the RS observed in the motor system. The neural mechanisms of RS in perceptual systems remain debated, as it is not yet clear how changes in the responses of single neurons to repeated stimuli relates to changes in the BOLD signal. The RS paradigm we use here and previously [Hamilton and Grafton, 2006, 2008] compares repeated trials where we predict suppression, to novel trials where we predict a release from the previous suppression. Such a release from suppression, when there is a change in stimulus, cannot be easily explained by current models of hemodynamic responses. Our own interpretation of RS focuses on patterns of neuronal activity at the population level. Specifically, we suggest that repetition of a stimulus causes a decrease in population activity, and a change to a new stimulus activates a different neural population that fires robustly [Grafton and Hamilton, 2007; Hamilton and Grafton, 2006, 2007, 2008]. This interpretation does not require a strict concordance between RS‐defined selectivity of the population response and the selectivity of single neurons [Grill‐Spector et al., 2006; Sawamura et al., 2006]. Instead, population level RS can be interpreted analogous to the double dissociation paradigm in neuropsychology. For population RS, evidence that two components of a motor task are supported by differing neural substrates is strong evidence that they are dissociable levels of action representation.

All previous studies of the neuronal mechanisms underlying RS have been conducted in visual systems [Desimone, 1996; Kohn and Movshon, 2004; Li et al., 1993; Sawamura et al., 2006]. However, our results suggest that similar effects should be observable in neurophysiological recordings from the motor system. Some might argue that, if RS is as universal as we claim, it is surprising that motor neurophysiologists have not already observed the phenomenon. We would propose that this is simply due to differences in experiment goals across domains. Systematic and carefully ordered single unit recordings have been needed in the visual system to observe subtle changes in neural tuning between novel and repeated stimuli [Kohn and Movshon, 2004]: casual observation would not have revealed the effects. In motor neurophysiological experiments, animals typically perform the same action many times at the start of a session while electrodes are placed, then during recording the stimuli are often pseudo‐randomized to avoid having the same trial twice in a row (Roger Lemon, personal communication); both of these factors preclude any observation of RS. We hope that our results will inspire researchers performing single cell experiments in monkey motor areas to study RS systematically.

The Functional Anatomy of Hand Action Planning and Performance

The present data identify a widely distributed set of cortical areas engaged in the retrieval, planning and execution of simple hand actions based on verbal instruction. RS might reflect more efficient motor processing, reduced planning for a second action compared to a first or simply non‐specific habituation within a given neural network. We note that our illustrative post‐stimulus time histograms (see Fig. 2) suggest that the magnitude of the RS effect differed between different brain regions. For example, BOLD was suppressed by around 20% in primary motor cortex, but by over 60% in premotor cortex and supplementary motor area. Formal analyses of these differences are not possible in the present design, but we speculate that they may reflect differences between planning processes in premotor/supplementary motor areas, and execution processes in primary motor cortex. For example, execution of an action might require a minimum level of signal, allowing less dynamic range for suppression in primary motor cortex even when, on a repeated trial, primary motor cortex might be receiving a smaller signal from the “upstream” premotor and supplementary motor areas.

The left hemisphere lateralization, with maximal effects in caudal middle frontal and inferior parietal lobule, is consistent with previous imaging studies of action production related to both arbitrary postures and also tools. However, the present study used only right handed participants performing right hand actions, so we do not draw strong conclusions about lateralization. The key contribution of the current result is that with the RS method there is no ambiguity in interpreting differences between specific types of actions, between action and a rest condition or in the introduction of a go, no‐go task to dissociate planning from planning with execution.

The data provide further evidence for a fronto‐parietal brain system for action retrieval and planning. This is concordant with an extensive patient literature. [Haaland et al., 2000; Heilman et al., 1997; Leiguarda, 2001; Ochipa and Gonzalez Rothi, 2000]. Ideomotor apraxia is most commonly associated with left inferior parietal or middle frontal lesions. The critical deficit in this disorder is the transformation of a verbal instruction into an appropriate motor action, despite adequate strength, dexterity, and comprehension. Future studies of motor tasks using RS may be useful in characterizing conceptual retrieval deficits such as selecting the appropriate tool to solve a motor act, or choosing an appropriate grasp to accomplish a motor goal.

The present findings replicate most of the findings described by Dinstein et al. [2007]. Whether gestures are executed in the context of a rock‐paper‐scissors game, or by verbal instructions, there is an RS effect for repeated hand actions in left sensorimotor cortex, throughout the intraparietal sulcus, in dorsal and ventral premotor cortex. Two relative differences were greater recruitment of cingulate motor area for self generated actions and more prominent recruitment of supplementary motor area and inferior parietal lobule for verbally instructed actions. We also report strong RS effects in the cerebellum, which was not examined by Dinstein. The cerebellar findings mean that RS is not limited to the cerebral cortex, and confirm the role of cerebellum in action planning. The RS effect in aIPS was also observed when subjects grasped the same object [Kroliczak et al., 2008] or any object with the same orientation, suggesting this area has a general involvement in forming hand configurations.

Broader Implications

Our results have several important implications for the design or interpretation of future fMRI studies of the motor system and possibly other cognitive systems. First, the data demonstrate that RS is a robust and broad phenomenon which likely applies across the entire motor system of the brain. In conjunction with previous data showing RS for visual, linguistic, number, motor and other tasks [Dehaene et al., 1998; Dinstein et al., 2007; Hamilton and Grafton, 2006, 2007, 2008; Kroliczak et al., 2008; Naccache and Dehaene, 2001; Noppeney and Price, 2004; Shmuelof and Zohary, 2005], this implies that RS may be ubiquitous across the cortex. This means that future fMRI studies should take into account the likely existence of RS and optimize stimulus sequences to either measures RS effects, or avoid them as confounds for other contrasts. The confound will be particularly problematic for repeated measures designs such as learning studies or pseudo‐random schedules with structured repetition interacting with task contrasts. By taking the RS effect into account experimental power and precision could potentially be improved.