Decoding Action Intentions in Parietofrontal Circuits

Decoding intended goals from sensorimotor pathways of paralyzed patients is an important feature for cognitive neural prosthetics. However, it is not clear which brain areas, or combination of areas, are optimal to guide the selection of recording sites and the design and implementation of decoding algorithms (Andersen et al., 2010). To date, the ability to predict goal-directed movements based on intention-related cortical signals has mainly been constrained to invasive neural recordings in nonhuman primates.

Single-unit studies in monkeys have implicated specialized parietofrontal circuits in processing sensorimotor information for goal-directed actions. The classical model of the neural control of reaching and grasping movements proposes that areas located in the posteromedial portion of the intraparietal sulcus (IPS) contribute to the planning of reaching movements toward an object, whereas a more anterolateral region of IPS integrates grasp-related information about an object. Specifically, in monkeys the medial intraparietal area (MIP) and V6A contain neurons associated with a particular direction of reach (Andersen and Buneo, 2002), while the anterior intraparietal area (AIP) contains visual and visuomotor neurons that are activated by a particular type of grasp (Castiello, 2005). Similar specialized areas also exist in the frontal cortex, namely the dorsal and ventral premotor cortex (PMd and PMv) for arm and hand movements, respectively. Recent findings, however, challenge the view that the reach and grasp components are processed independently (Fattori et al., 2010).

In humans, conventional functional magnetic resonance imaging (fMRI) studies have shown widespread, overlapping activations in the parietofrontal network during different goal-directed movements such as reaching and grasping (Filimon, 2010). These undifferentiated activations for distinct movements could be explained by the fact that traditional fMRI approaches show the average activity within a given area and may be unable to distinguish the neural information contained in distributed patterns of voxel activity. Surprisingly, little is known about whether goal-directed hand actions can be decoded from intention-related activity within different parietofrontal brain areas in humans. More specifically, can fMRI signal decoding unravel predictive neural activity underlying the planning of object-directed reach and grasp actions?

In a recent study published in The Journal of Neuroscience, Gallivan et al. (2011) took advantage of the relatively new neuroimaging method of voxel pattern classification to decode intention-related activity in the human parietofrontal network during different object-directed actions. This method differs from traditional fMRI analysis in that it uses a multivariate statistical technique to discriminate classes of stimuli by assessing differences in the elicited spatial patterns of fMRI signals. A strength of this method is that it can tell apart different target-directed behavior with a level of sensitivity previously unavailable.

The authors used a delayed movement task in which object presentation (preview phase), movement planning, and action execution phases were separated. Subjects previewed a graspable object (small or large) before receiving a cue indicating how to contact it. Subjects could either reach to grasp the object between the index and thumb or simply reach out and touch it (Gallivan et al., 2011, their Fig. 1). To localize the common brain areas in which to perform pattern analyses, the authors searched for regions that, at the group level, were involved in movement planning. To do this, they contrasted activity elicited by the planning of a hand action (i.e., after movement instruction) versus the transient activity elicited by visual presentation of the object before the instruction (planning versus preview phase). This group contrast allowed the authors to define 14 action-related regions of interest (ROIs) as well as three sensory-related ROIs that could then be reliably identified in single subjects with the same contrast (Gallivan et al., 2011, their Fig. 2). The fMRI analyses compared the reach-to-grasp versus the reach-to-touch conditions (large grasp vs touch or small grasp vs touch) or the two grasp conditions (large grasp vs small grasp). These contrasts allowed the fMRI pattern classifiers to decode either two different types of hand movements (reach-to-touch vs reach-to-grasp) or two different grasps, respectively.

Although the fMRI signal amplitude was similar in all defined areas of the parietofrontal network for the three movement contrasts, voxel pattern classifiers could accurately decode the different movement conditions in several areas. During movement planning, a subset of areas showed different voxel patterns for grasp (either small or large) versus touch, namely, superior parieto-occipital cortex (SPOC), anterior precuneus, medial IPS (mIPS), anterior IPS (aIPS), supplementary motor area (SMA), and preSMA (Gallivan et al., 2011, their Fig. 4). A second group of areas showed distinct activity patterns for different grasps (large versus small grasp) in addition to being selective for grasp versus touch, namely, posterior aIPS, caudal IPS (cIPS), PMv, PMd, and primary motor cortex (M1) (Gallivan et al., 2011, their Fig. 4). These results indicate that the fMRI pattern classifiers could discriminate two different circuits within the parietofrontal network. The first circuit, consistent with a dorsomedial organization, appears to encode reach-to-grasp and reach-to-touch movements differently. The second, consistent with a dorsolateral organization, appears to encode different grasps in addition to different hand movements. These results are important because they highlight distinct parietofrontal circuits with different encoding capabilities for specific types of hand movements in humans. It is noteworthy that, although decoding performance was statistically significant, it only reached an accuracy level of 60%, which is below the decoding accuracy levels found by neuronal recordings in monkeys (Carpaneto et al., 2011). Nevertheless, voxel pattern analysis could potentially identify brain target candidates for invasive recordings.

Although results from this study indicate that decoding of movement intentions from human brain signals can accurately identify and predict upcoming reach-to-touch and reach-to-grasp movements in different nodes of the parietofrontal network, this technique offers little insight into whether a brain area has a pivotal or merely subsidiary role in a given behavior. Moreover, it is important to recognize that the characteristic properties of a brain area are not intrinsic to that area, but arise from its specific interactions with other nodes of the network. Recently, advances in transcranial magnetic stimulation (TMS) techniques have revealed functional interactions between different areas of the parietofrontal network in humans (Rothwell, 2011). This allows us to probe corticocortical connectivity between different areas of the network, and, in turn, test whether these interactions are specific to upcoming hand movements (Fig. 1A). Using a twin-coil TMS approach, physiological interactions between PMv and M1 were tested while subjects prepared either a precision grip or a whole hand grasp. It was found that during movement preparation, the degree of facilitation of PMv–M1 interactions was correlated with the amount of muscle activity during the upcoming grasp (Davare et al., 2009; Fig. 1B). Interestingly, this is consistent with the distinct voxel patterns in PMv for different grasps found by Gallivan et al. (2011, their Fig. 4).

Figure 1.

A, Schematic of dual-site TMS protocol. This approach can be used to investigate the time course of interactions in a “two-node” neural circuit (a non-primary motor area and M1) using two coils. The idea is to stimulate a non-primary motor area with a conditioning pulse (i.e., PMv or PPC) to examine its effect on a subsequent suprathreshold test pulse to M1. This effect can be inferred by measuring any possible change in the amplitude of a hand muscle response to TMS using electromyography. Dual-site TMS can be used to test whether these connections can be modulated by task demands (Bolognini and Ro, 2010; Chouinard and Paus, 2010). B, Causal connectivity of different premotor and parietal regions to M1 identified by dual-site TMS studies in humans (posterolateral view). Dashed yellow line, IPS. Dashed arrows indicate possible pathways for reach, grasp, or both during planning. Asterisks indicate anatomical pathways where these interactions are indirect.

Another twin-coil TMS study, examined interactions between aIPS, cIPS, and M1 at different times during the preparation of reach-to-grasp movements in central or peripheral space. Koch et al. (2010) found that cIPS interacted with M1 early during preparation of movements requiring a whole-hand grasp in the peripheral space. In contrast, aIPS interacted with M1 at a later stage and only for a precision grip, regardless of object location (Fig. 1B). Again, this is consistent with the voxel pattern classification of parietal areas found by Gallivan et al. (2011). Whereas reach-to-touch versus reach-to-grasp intentions could be decoded from activity in caudal areas along the IPS, more lateral and anterior parietal regions showed activity patterns that were grasp selective. In light of these findings, it will be interesting to determine, using twin-coil TMS, whether functional connectivity between M1 and other parietal regions, such as SPOC, encode reaching, grasping, or both movements during planning (Fig. 1B, dashed arrows).

It also is worth noting that the temporal evolution of brain activity within the parietofrontal network for goal-directed actions operates on a very fast timescale (in the order of 100–150 ms). This fast timescale is a challenge for fMRI studies, because although reach- or grasp-specific changes in blood oxygenation level-dependent signal can be detected, these changes actually reflect fast processing within visuomotor circuits. TMS studies have the advantage of better temporal resolution to investigate the time course of the sensorimotor transformations that may occur at different times within the parietofrontal network. Future investigations should also apply TMS over several component regions of this circuit to determine the relative timing of computations performed by each specific region.

How functionally specialized are the parietofrontal circuits for reaching and grasping? Although recent findings support the existence of functionally distinct parietofrontal pathways for controlling reach-to-grasp movements, the overlap between these cortical circuits may reflect the different functional and computational constraints that need to be satisfied when planning reach-to-grasp movements. For instance, previous conventional fMRI work has found differences between arm transport and grip formation in distinct regions of the parietofrontal circuit during the execution phase of reach-to-grasp actions in humans (Cavina-Pratesi et al., 2010). Different regions may be involved in controlling proximal arm/shoulder muscles that drive the hand to the target location (transport component) and distal hand/finger muscles that exert fingertip forces to the object (grip component). Future studies using a pattern classification approach, along with TMS connectivity approaches, should examine the influence of reach-related hand movements and grip-specific responses to address these issues. Perhaps their functional specialization is not simply a matter of encoding reach versus grasp parameters. It appears that the dorsomedial pathway contributes to the integration of the grasp with the reaching component (Fattori et al., 2010), probably to ensure smooth coordination between both components. In contrast, the dorsolateral pathway may be specialized in the fast processing of the object properties, allowing on-line control of hand-object interactions.

In conclusion, a full understanding of the brain circuits subserving sensorimotor control cannot be achieved by studying the individual areas in isolation, but must combine fMRI with other techniques, such as TMS, to investigate causal interactions between different nodes of the network. The work of Gallivan et al. (2011) not only provides an important step in understanding the intention-related neural processes in the parietofrontal network during planning of object-directed actions, but also sheds light on how sensorimotor transformations could be achieved within this complex circuitry, and may inform neural inputs to drive neural prosthetic devices in complex goal-directed actions.