Context-Specific Grasp Movement Representation in the Macaque Anterior Intraparietal Area

Abstract

To perform grasping movements, the hand is shaped according to the form of the target object and the intended manipulation, which in turn depends on the context of the action. The anterior intraparietal cortex (AIP) is strongly involved in the sensorimotor transformation of grasping movements, but the extent to which it encodes context-specific information for hand grasping is unclear. To explore this issue, we recorded 571 single-units in AIP of two macaques during a delayed grasping task, in which animals were instructed by an external context cue (LED) to perform power or precision grips on a handle that was presented in various orientations. While 55% of the recorded neurons encoded the object orientation from the cue epoch on, the number of cells encoding the grip type increased from 25% during the cue epoch to 58% during movement execution. Furthermore, a classification of cells according to the time of their tuning onset revealed differences in the function and anatomical location of early- versus late-tuned cells. In a cue separation task, when the object was presented first, neurons representing power or precision grips were activated simultaneously until the actual grip type was instructed. In contrast, when the grasp type instruction was presented before the object, type information was only weakly represented in AIP, but was strongly encoded after the grasp target was revealed. We conclude that AIP encodes context specific hand grasping movements to perceived objects, but in the absence of a grasp target, the encoding of context information is weak.

Introduction

Humans and other primates are able to perform a wide range of complex hand movements and shape their hands both according to the target object, as well as depending on the intended manipulation. Since grasping movements are typically made to visually perceived targets, their neural control can perhaps be most easily understood in the framework of visuomotor transformations. However, such a framework needs to incorporate the fact that the same object, depending on internal goals or external context cues, can lead to different types of actions.

It has long been known that the parietal lobe plays an important role for the generation of hand grasping movements. Lesions in human parietal cortex lead to optic ataxia, a deficit in hand movement coordination (Balint, 1909; Jeannerod et al., 1984), while in the monkey, single-unit activity in the parietal lobe has been associated with the generation of hand movements (Hyvärinen and Poranen, 1974; Mountcastle et al., 1975). More recently, the group of Sakata described a region of the macaque parietal lobe, the anterior intraparietal area (AIP), which contains neurons that specifically encode the shape of the hand during grasping (Taira et al., 1990; Sakata et al., 1995, 1997; Murata et al., 2000). Moreover, the functional relevance of AIP for hand grasping was shown by inactivation (Gallese et al., 1994) and strong direct and reciprocal connections have been demonstrated between AIP and the ventral premotor area F5 (Luppino et al., 1999; Borra et al., 2008), an area that is also involved in hand movement control (Rizzolatti et al., 1988; Murata et al., 1997; Raos et al., 2006; Stark et al., 2007). Finally, there is evidence for a human homolog of AIP (Binkofski et al., 1998; Culham et al., 2003; Shikata et al., 2008).

In all these electrophysiological studies of AIP, a particular object was always grasped with the same grip. However, in everyday situations, several grip types are often possible for the same object, and we select an appropriate grip according to the intended goal of the manipulation. Such a goal-dependent grip selection can be regarded as a rule-based sensorimotor transformation, which has been attributed to the frontal cortex (White and Wise, 1999; Hoshi et al., 2000; Wallis et al., 2001; Amemori and Sawaguchi, 2006). However, signals representing action selection and task rules for eye and arm movements have also been found in the parietal cortex (Gottlieb and Goldberg, 1999; Kalaska et al., 2003; Gail and Andersen, 2006; Scherberger and Andersen, 2007).

In this study, we recorded single-unit activity in AIP while monkeys were instructed by an external context cue to grasp a handle either with a power or a precision grip. Additionally, we systematically varied a parameter of the object shape, by presenting it in five different orientations. The majority of neurons in AIP encoded the object orientation as well as the instructed grip type. We classified neurons according to the time of their tuning onset and found differences in function and anatomical distribution of early- and late-tuned cells.

Materials and Methods

Experimental setup.

Two female rhesus monkeys (Macaca mulatta) participated in this study (animals L and J). Procedures and animal care were in accordance with the regulations set by the Veterinary Office of the Canton of Zurich and the Guidelines for the care and use of mammals in neuroscience and behavioral research (National Research Council, 2003).

Animals were habituated to comfortably sit upright in individually adjustable primate chairs with the head post rigidly fixed to the chair. A grasp target was located at a distance of ∼30 cm in front of the animal at the level of its chest. The target consisted of a handle that could be grasped with two different grip types, either with a precision grip with index and thumb in opposition or a whole-hand power grip (Fig. 1A). To detect the contact of the animal's thumb and index finger during precision grips, two touch sensors were placed in small recessions. Their locations were well visible. Power grips were sensed by a light barrier. The handle was rotatable and 5 different handle orientations were tested in this experiment (upright and tilted 25 or 50° to the left or right). To illuminate the handle in the dark, two dedicated spotlights were positioned to the left and right of the handle (outside of the animal's reach). A halfway mirror was placed horizontally between the monkey's eye and the grasp target, such that the LED light stimuli used for eye fixation and task instructions (see below) were projected on the center of the handle. The mirror also ensured that the grasp target was only visible when illuminated by the spotlights. Two capacitive touch sensors (model EC3016NPAPL, Carlo Gavazzi) were fixed to the chair in front of the animal's hips to monitor the hand resting position for both hands. An optical eye tracking system (model AA-ETL-200; ISCAN) was used to monitor the animal's eye position. The animal's behavior and all stimulus presentations were controlled in LabView Realtime (National Instruments) with a time resolution of 1 ms using custom-written software. Finally, an infrared camera was used to monitor the monkey's behavior continuously throughout the entire experiment.

Figure 1.

Task paradigm and recording penetrations. A, Sketch of the handle (left) and photographs of a monkey performing a precision grip (middle) and a power grip (right). In the drawing, the red dotted line indicates a light barrier for detecting power grips, and the red oval indicates a touch sensor in a groove for sensing precision grips (a second sensor is located on the opposite side of the handle). The handle was presented in five different orientations. B, Delayed grasping task. Trials were divided into four epochs: fixation, cue, planning, and movement. Monkeys initiated trials by placing both hands on rest sensors and fixating a red LED in the dark. After a variable delay (fixation, 700–1100 ms), the handle was illuminated for 600 ms (cue), revealing its orientation. At the same time, a second colored LED (“context cue”) was illuminated, which instructed the animal about the required grip type (power or precision). After a variable delay (planning, 700–1100 ms), the dimming of the fixation light served as the go signal to initiate movement execution. All trial conditions were randomly interleaved. C, Cue separation task. Modified task from B, with the cues for grip type and orientation presented consecutively and with each cue followed by a separate planning period. In one version of this task (OT task), the orientation information preceded the grip type information, while in the other version the cue sequence was reversed (TO task). D, Coronal MRI section (monkey J) with the recording chamber on the right hemisphere filled with contrast medium. The red line indicates the position of the oblique section in E. E, F, Maps of recording electrode penetrations (yellow dots) in monkeys J and L, respectively. The yellow ruler indicates the median (long tick mark) and quartiles of the recording distribution along the intraparietal sulcus (IPS). CS, Central sulcus.

Tasks.

Monkeys were trained to perform a delayed grasping task, in which they were required to grasp a single object (handle) in one of five possible orientations with either a power grip or a precision grip. This led to a combination of 10 task conditions that were presented randomly interleaved. While the orientation of the handle became immediately apparent after illumination, the grip type was instructed by the color of an additional LED next to the fixation light that was green for a power grip and white for a precision grip.

The monkey initialized each trial by fixating a red LED and placing both hands on the hand rest sensors while otherwise sitting in the dark (Fig. 1B). The trial started with a baseline epoch (fixation) of variable length (700–1100 ms, mean: 900 ms), during which the animal had to maintain its resting position in the dark. The following cue epoch (length 600 ms) was dominated by visual input: the grasp target was illuminated, hence revealing the handle orientation, and the additional LED was shown, which informed the animal about the required grasp type (power grip or precision grip). Then, during the planning epoch of variable length (700–1100 ms, mean: 900 ms), the animal could plan, but was not allowed to execute the movement, until the dimming of the fixation light gave the go signal to reach and grasp (movement epoch). Planning and movement epochs were again in complete darkness except for the red LED light that the animal had to keep fixating. Only left-hand movements (contralateral to the right recording chamber) were allowed. All correctly executed trials were rewarded with a fixed amount of juice, and the animal could initiate the next trial after a short intertrial interval (1500 ms). Error trials were immediately aborted without giving a reward. To maintain a high motivation for reward, animals were restricted from access to water up to 24 h before the training and recording sessions.

Animals were also trained in a modified version of this task, in which the instructions regarding the grip type (colored LED) and the handle orientation (spotlight) were presented sequentially in two distinct cue periods (Fig. 1C). In this cue separation task, each cue epoch (duration 600 ms) was followed by its own planning period (length 600–1000 ms, mean: 800 ms) before the movement was executed. Animals were trained to perform this cue separation task in two variations: either with the object orientation shown in the first cue epoch and the grip type instruction in the second (OT task), or with the grip type instruction presented first and the object orientation in the second cue epoch (TO task). When testing neurons in this cue separation task, trials of both versions (OT and TO task) were always run randomly interleaved with each other and with trials of the (standard) delayed grasping task.

Surgical procedures.

To prepare for the recording experiments, a titanium head post was secured in a dental acrylic head cap, and a custom made oval-shaped recording chamber (material PEEK; outer dimensions: 40 × 25 mm) was implanted over the right hemisphere on top of AIP with the skull bone removed underneath the chamber. This allowed the insertion of recording microelectrodes through the dura in subsequent recording sessions without discomfort to the animal. The recording chamber and head post were fixed on the skull with bone cement (Refobacin Plus, Biomet Orthopaedics) and reinforced with titanium (Synthes) and ceramic bone screws (Thomas Recording).

All surgical procedures were performed under sterile conditions and general anesthesia (induction with ketamine 10 mg/kg, i. m., atropine 0.05 mg/kg, s.c., followed by intubation, isoflurane 1–2%, and analgesia with 0.01 mg/kg buprenorphine, s.c.). Heart and respiration rate, electrocardiogram, O₂ saturation, and body temperature were continuously monitored, and systemic antibiotics and analgesics were administered for several days after each surgery. Animals were allowed to recover for at least 1 week before behavioral training or recording experiments recommenced.

MRI scans.

Before surgical procedures, a structural magnetic resonance image (MRI) of the brain and skull was obtained from each animal to help guide the chamber placement. For this, animals were sedated (ketamine 10 mg/kg, i. m., atropine 0.05 mg/kg, s.c., and xylazine 0.5 mg/kg, i. m.), supplemented with O₂ (1 L/min), and their heart rate, O₂ saturation, and end-tidal CO₂ level continuously monitored. After placing in the scanner (GE Healthcare 1.5T) in a prone position, T1-weighted volumetric images of the brain and skull were obtained (3D IR-SPGR sequence, acquired voxel size 0.7 mm isometric, TR 7.6 ms, TE 3.16 ms, flip angle 12°, 400 ms inversion time) and realigned off-line in stereotaxic coordinates using AFNI 3.0 (for details see: Scherberger et al., 2003). The stereotaxic location of the tip of the right intraparietal sulcus was then obtained (approximate location: 8 mm anterior, 22 mm lateral) to guide the placement of the recording chamber over AIP.

Weeks after chamber implantation, a second MRI scan was obtained with the recording chamber filled with an MRI sensitive contrast medium (Gadolinium solution diluted in saline 1:2000). This allowed the mapping of cortical structures in the coordinates of the chamber, which greatly facilitated to target AIP with subsequent electrode penetrations (Fig. 1D–F).

Neural recording.

Single unit (spiking) activity was recorded using glass-coated tungsten electrodes (impedance: 1–2 MΩ at 1000 Hz) that were positioned by a 5-channel micromanipulator (MiniMatrix, Thomas Recording) that was directly attached to the recording chamber. Neural signals were amplified (400×), digitized with 16 bit resolution at 30 kS/s using Cerebus Neural signal processor (Bionics), and stored to disc together with the behavioral data. To coarsely monitor the tuning properties of the recorded neurons during data acquisition, spike detection was performed in real-time (Cerebus hardware) and analyzed for various task conditions using Matlab (MathWorks). However, all quantitative analysis reported here was performed off-line as described below.

Data analysis.

Raw data traces were bandpass filtered (600–8000 Hz) using Matlab and spikes were extracted and sorted using Offline Sorter (Plexon). The quality of single unit isolation was assessed with the following criteria: (1) the separation of waveform clusters in projections onto the first three principle component axes, (2) the homogeneity of waveforms, and (3) the presence of a refractory period in the interspike interval (ISI) distribution. A retrospective analysis revealed that <0.26% of all ISIs were shorter than 1 ms. Single units were included in our database if they were stably recorded for at least 7 trials per condition in the delayed grasping task (total of 70 trials) and at least 5 trials per condition in the cue separation task (total of 150 trials).

Peristimulus time histograms (PSTH) for the visualization of spike rates were generated by replacing each spike with a kernel function and then averaging all such functions across all spikes and trials. To obtain PSTHs that are continuous as well as causal (i.e., the PSTH at any given time point is not influenced by spikes that occur in the future), we chose the kernel to be a gamma-distribution function, hence replacing each spike at time t_s with the following time-shifted function:

The shape (α = 1.5) and rate parameter (β = 30) were chosen to achieve little delay (kernel peak at 16 ms) and an SD of ∼40 ms. It is important to note that PSTHs were only used for illustration; all quantitative analysis was based on exact spike counts. To obtain population averaged PSTHs, individual histograms were averaged across the cell population. For this, preferred and nonpreferred conditions were defined as follows:

For each neuron, the preferred grip type and orientation were determined from the mean firing rates in the delayed grasping task taken in the time interval from the cue onset to the end of the movement epoch, which was then averaged across all trials of the same grip type or the same object orientation, respectively. The preferred grip type was then defined as the grip for which the mean rate was largest while the off type was defined as the other grip. Likewise, the preferred orientation was defined as the object orientation for which the firing rate was maximal, while the off-orientation was taken as the object orientation at 75° angular distance from the preferred one. For neurons with preferred orientation of 0°, we randomly chose +50° or −50° as the off-orientation, since no 75° condition existed. This definition was chosen to select the off-orientation not exclusively from the two extreme orientations (±50°). However, all results stayed essentially the same if the off-orientation was defined as the orientation with maximal angular distance to the preferred orientation, or as the orientation with the lowest firing rate.

To test whether neurons were significantly tuned for grip type and/or orientation in a particular task epoch (fixation, cue, planning, or movement), we first determined in each trial the mean firing rate (spike count/length of epoch) and then applied a two-way ANOVA with group factors grip type and orientation. This compared the rate variance within conditions to across conditions. Neurons were considered to be significantly tuned for grip type or orientation for p values <0.01, and if they fired at least 5 spikes/s in the preferred condition.

In addition, tuning significance for grip type and orientation was tested at multiple time points t using a 2-way ANOVA on the spike count in a 200 ms window centered around t. This test was repeated in time steps of 50 ms (sliding window ANOVA). Due to the variable length of the planning period, trials were first aligned to cue offset up until 0.6 s after cue offset; after that they were aligned to movement start (release of the hand rest button). Criteria for significant tuning were the same as for the ANOVA analysis of the fixed time epochs.

For the tuning analysis in the cue separation task, we applied the same 2-way ANOVA as in the (standard) delayed grasping task, but with a significance level of p < 0.05 due the lower number of trials per condition (minimum 5, average 6.8; standard task: minimum 7, average 9.8). Since the cue separation task contained 2 planning periods of variable length, trials were aligned to the cue offset of the first and second cue as well as to the movement start (hand rest release), and realignments were placed 0.6 s after each cue offset.

To estimate the time when the number of significantly tuned cells for grip type or orientation sharply increased (during the cue and movement epoch), we determined the time in each epoch when the increase became half-maximal. For this, we first computed a linear interpolation of the number of significantly tuned cells (as obtained from the sliding window ANOVA) in steps of 2 ms (using Matlab command interp). We then determined, for each epoch, the time when this curve became half-maximal with respect to a baseline level. This baseline was set to 0 for the cue epoch and to the value of the curve at the time of the go signal for the movement epoch. To assess significance of possible time shifts in the increase of grip type and orientation tuning, we used a Monte Carlo procedure, in which 1000 repetitions of the same analysis were performed with random shuffling of the labels “grip type tuned” and “orientation tuned,” to determine the null distribution and its associated significance level.

Furthermore, we quantified the time in the task when each neuron first became significantly tuned for grip type or orientation. We called this the tuning onset of grip type and orientation tuning, and defined it as the first time when a neuron was significantly tuned in the sliding window ANOVA in at least five consecutive steps. If this occurred, tuning onset was set to the center of the first window; if not, it was set to infinity. Using this quantitative measure, we classified each neuron, separately for grip type and orientation, in one of the four categories: (1) early, (2) middle, (3) late, or (4) no tuning onset, corresponding to the tuning onset falling in the cue, planning, or movement epoch, or never occurring.

Furthermore, we quantified the number of cells preferring each of the two grip types and five grip orientations separately for the different task epochs. For this, the same definition of preferred grip type and orientation was used as for the calculation of population PSTHs, except it was restricted to the task epoch in consideration.

Finally, we applied a receiver operating characteristic (ROC) analysis (Britten et al., 1992) to various task epochs to assess for each individual cell, how well its firing rates during precision grip trials could be discriminated from those during power grip trials. We calculated the area between the ROC curve and the no-discrimination (diagonal) line as a measure of discriminatory power. To remove interaction effects of superimposed orientation tuning, we computed this measure separately for each orientation and averaged the five results. Significance levels were assessed by performing a Monte Carlo analysis for each cell as explained above, this time with random shuffling of the labels for power and precision grip between trials. The 95th percentile of the resulting distribution was then taken as the significance level.

Results

We recorded a total of 571 single cells in two monkeys (monkey L: 299 cells, monkey J: 272 cells) while the animals performed the delayed grasping task. Results were essentially the same for both animals and are therefore reported together.

Both monkeys performed the task with high accuracy. Errors due to the execution of the wrong grip type occurred only in ∼5% of all trials. Observation of the animals via infrared camera during task performance revealed that the handle was approached with the hand preshaped and in the matched orientation. Analysis of the movement times also suggested that the animals did not approach the target in a “standard” orientation and then adjusted the hand orientation based on sensory (tactile) feedback information: the influence of the object orientation on the movement time was quite small to allow for such feedback adjustments. The median movement times for precision grips/power grips were 0.53 s/0.22 s (−50° orientation), 0.47 s/0.21 s (−25°), 0.46/0.21 s (0°), 0.47/0.21 s (+25°), and 0.53 s/0.21 s (+50°). Monkeys kept holding the object on average for 0.36 s. No preshaping occurred before the go signal, and the hands were kept motionless on the sensor pads.

Tuning for grip type and orientation

A large majority of cells were modulated by the delayed grasping task. Three typical neurons are shown in Figure 2. Neuron A showed a sharp increase of its firing rate immediately after the movement instruction was given (cue epoch), in particular for objects oriented to the right (+25/+50°), and more strongly for power grips than for precision grips. This activity pattern was preserved throughout the task until the movement was executed. The example neuron was therefore modulated by grip type and orientation in all three task epochs (cue, planning, and movement). Note that the timing of the early rise for trials with rightward orientations of the handle was identical for both grip types (left and right panel), before the curves separated shortly afterward.

Figure 2.

Three example neurons with different tuning onsets. For each neuron, precision grip trials are shown on the left panel and power grips on the right panel. Different colors indicate various handle orientations, for which spike rasters (on top) and averaged firing rates (at bottom) are shown individually. The dotted line within the movement epoch indicates the release of the hand rest button (movement start). All trials are aligned to both the end of the cue epoch and the start of the movement (arrow heads below); gaps in the curves (at ∼0.6 s) indicate the realignment. A, Neuron that exhibits tuning for the handle orientation and the instructed grip type starting in the cue period and extending until movement execution. B, Neuron with tuning for the handle orientation starting in cue, but with significant grip type modulation only during movement execution. C, Neuron showing no response during cue presentation and movement planning, but with a strong selectivity for precision grips during movement execution without significant orientation tuning. D, Population firing rate across all 571 neurons for each combination of the cells' preferred and nonpreferred grip type and orientation.

A second type of neuron is depicted in Figure 2B. It showed a clear modulation of its firing rate with object orientation immediately after cue presentation, while throughout the cue and planning epoch its activity was identical for power and precision trials. However, during movement execution (starting immediately after handrest release) the firing rate of precision trials increased with respect to power trials. Therefore this neuron represented the object orientation from the object presentation onward, while grip type modulated the neuron only during movement execution.

Finally, neuron C in Figure 2 did not respond at all after cue presentation, neither for the grip type nor for the object orientation. However, it responded vigorously during movement execution with a strong peak for precision grips while being indifferent to object orientation.

These examples illustrate the variety in our dataset. As a summary, Figure 2D shows the population firing rate across all 571 neurons for each neuron's preferred and nonpreferred grip type and orientation. Both grip type and object orientation were well represented in the population during cue presentation and remained so until the movement was finished. Importantly, this was true even if the definition of the preferred and nonpreferred condition was based on the activity during the movement epoch alone, indicating that this finding is not a selection artifact.

To quantify the number of cells with a particular tuning in each task epoch, we performed, for each cell, a two-way ANOVA with factors grip type and orientation on the firing rates within each task epoch (Table 1). We found that in the course of the trial, these two variables behaved distinctively (Fig. 3A). During the cue period the fraction of neurons showing specificity for the object-cued factor, i.e., the object orientation, accounted for 55% of all cells, and this ratio stayed approximately constant throughout the planning and movement epochs. In contrast, only ∼25% of the cells showed selectivity for the context-cued variable (grip type) during the cue period; however this value increased to 37% during the planning epoch and 58% during movement execution, reaching a level that was eventually similar to the number of orientation-tuned cells.

Table 1.

Cell classification by tuning in task epochs

Orientation tuning				Grip type tuning
Cue	Plan	Move	Percentage	Cue	Plan	Move	Percentage
+	+	+	30%	+	+	+	12%
+	+	−	11%	+	+	−	4%
+	−	+	6%	+	−	+	3%
+	−	−	8%	+	−	−	6%
−	+	+	7%	−	+	+	15%
−	+	−	3%	−	+	−	6%
−	−	+	12%	−	−	+	28%
−	−	−	23%	−	−	−	26%

Left, List of cell classes (different rows) according to the presence (+) or absence (−) of significant orientation tuning in the task epochs: cue, planning, and movement (2-way ANOVA, see Materials and Methods). Percentages indicate the fractional size of each class in our population (n = 571). Right, Corresponding classification for grip type tuning.

Figure 3.

Orientation and grip type tuning in the neuronal population (n = 571). A, Fraction of cells showing tuning for grip type (black) and handle orientation (gray) in the different task epochs (two-way ANOVA; see Materials and Methods). Tuning for object orientation was constant from cue to movement, while grip type tuning increased over time. B, Percentage of tuned cells in a sliding window (width: 200 ms, centered on each data point). During cue, tuning for orientation started ∼150 ms earlier than for grip type. Grip type tuning increased in two steps: one during cue and one during the movement epoch. Trials are aligned on cue offset and movement onset (as in Fig. 2).

Of the cells which did not show tuning for either grip type or orientation, more than half (cue: 57%, planning: 62%, movement: 68%) displayed significant rate variations (increase or decrease) when compared with the baseline activity in the fixation epoch (2-tailed t test, p = 0.01). Presumably, some of these neurons could be tuned for other objects or grip types than the ones we tested in this study.

To further investigate grip type and orientation tuning over time, without constraining the analysis to predefined epochs, we extended the 2-way ANOVA on a sliding window (window width: 200 ms, step size: 50 ms), which revealed marked differences between the two variables (Fig. 3B). First, the number of orientation-selective cells after cue presentation rose considerably earlier than for grip type-selective cells. By measuring the time at half height of this increase during the cue epoch, this time difference was found to be ∼150 ms in the population (p ≪ 10⁻³, Monte Carlo procedure). This time difference was also observed in individual neurons (e.g., see: Fig. 2A), perhaps indicating that the processing of an abstract cue took longer than processing of an object cue. During the planning epoch, both fractions of tuned neurons stayed on a plateau, with the orientation fraction slightly larger than type; while during movement execution, the number of grip type specific neurons further increased and actually exceeded the number of orientation-selective cells, which was due to neurons that became type specific only during the movement epoch (e.g., as in Fig. 2B,C). However, no significant time difference was found between the two fractions in this second increase (p > 0.3).

In summary, the population analysis showed that the representation of grip type, while already present in the cue period shortly after the instruction was given, strongly increased toward movement execution, both in absolute terms (number of tuned cells) and in relation to the number of cells coding for orientation. These different roles of AIP for the encoding of an object-cued factor (orientation) and a context-cued factor (grip type) are further analyzed in the next section.

Tuning onset

As we have seen in the example neurons (Fig. 2A–C), some cells were grip type modulated already in the cue period, while others were tuned only during movement execution. Similarly, cells exhibited different onset times for the tuning of orientation. To quantify this effect, we determined each cell's tuning onset for grip type and object orientation, respectively (see Materials and Methods). Figure 4 (top row) shows, separately for grip type (left panel) and object orientation (right panel), the time periods when neurons were significantly tuned (black lines). Each neuron is represented by one row, and neurons were sorted (along the y-axis) according to their tuning onset. The graph emerging from the white–black transition depicts the cumulative distribution of the tuning onset, which is shown below in its derivative (histogram) form (bottom panels). For grip type (left panels), the tuning onset distribution was clearly multimodal with a first peak during cue presentation followed by a second peak of similar size after the go signal, and only a small fraction of cells located in between. For object orientation, the majority of cells had a tuning onset during cue presentation and only a few cells became tuned late in the task.

Figure 4.

Times with significant tuning in the neuronal population. A, Sliding window analysis (two-way ANOVA) for each cell (y-axis) at each time step (x-axis). Significant grip type (left) and orientation tuning (right) is indicated by black squares (p < 0.01). Cells are ordered by tuning onset (first occurrence of five consecutive significant steps). B, Histogram of tuning onset for grip type (left) and orientation (right) across the population.

To describe the relationship between the onset of grip type and orientation tuning, we classified neurons into four groups according to their tuning onset (early, middle, late, and none), separately for grip type and orientation tuning. Early, middle, and late onset corresponded to the cue, planning, and movement epochs. Table 2 shows a 4 × 4 contingency table of the combined tuning onset for grip type and object orientation. As can be readily seen, this contingency table is not statistically independent (Pearson's χ² = 88.5, df = 9, p ≪ 10⁻³).

Table 2.

Cell classification by tuning onset

Object orientation	Grip type				Sum
	Cue	Planning	Movement	None
Cue	108A	40	53B	77	278 (49%)
Planning	11	14	9	8	42 (7%)
Movement	9	7	37	12	65 (11%)
None	24	18	75C	69	186 (33%)
Sum	152 (27%)	79 (14%)	174 (30%)	166 (29%)	571 (100%)

Contingency table of tuning onset for type (columns) and orientation (rows) for all neurons in our population (n = 571). Example neurons for the marked classes (A–C) are shown in Figure 2A–C.

During the cue period, neural responses were dominated by the object feature orientation. While approximately half of the recorded cells (278/571, 49%) showed an early onset of orientation tuning, only 152 cells (27%) signaled the grip type. A considerable part of the orientation-sensitive group was also modulated by the instructed grip (108/278, 39%), leading to the largest class of cells (for an example, see Fig. 2A). However, neurons that were orientation tuned during the cue period were not always grip type tuned at the same time. In fact, many of these cells only became tuned for the grip type during planning (40) or movement (53) (e.g., Fig. 2B), and others not at all (77). In contrast, cells that were grip type tuned in the cue period were very likely also orientation tuned (108/152, 71%), while the other groups with early type tuning were small. Together, these results suggest that during the cue period, grip type coding is only modulating the primary coding of object features.

Only a few neurons had their tuning onset during the planning period. Some cells that were already orientation tuned in the cue became additionally grip type tuned in the planning epoch (40). All other groups were very small. Therefore, most of the activity during the planning period (Fig. 3A,B) was a continuation of the activity already present during the cue epoch, which is consistent with a role of AIP, and of the parietal cortex in general, in working memory (Mountcastle et al., 1975; Sakata et al., 1995; Murata et al., 2000; Andersen and Buneo, 2002).

Neurons whose tuning began during the movement epoch behaved quite differently from those with early tuning onset. Late onset tuning was directed primarily to the grip type rather than the object orientation. The largest group of these neurons was selective for the grip type but lacked orientation tuning (75) (see also Fig. 2C). A second large group consisted of cells that had developed orientation selectivity during the cue period and now additionally expressed a late onset tuning to grip type (53) (see Fig. 2B). A third group of neurons developed tuning to both the grip type and the orientation during the movement period (37). These latter neurons accounted for the majority of cells with late orientation tuning. In contrast, neurons that became orientation tuned during the movement period, without tuning to grip type, were very rare (12).

Note that cutaneous tactile information could not have been the major source of input for late-onset cells, because the movement epoch ended when the hand had grasped the object. Also, previous studies found little or no neurons in AIP with somatosensory responses (tactile or joint-related) (Taira et al., 1990; Murata et al., 2000). Therefore, these cells are most likely related to motor output. Together, our findings indicate that neurons with a late tuning onset, in contrast to early-tuned cells, primarily encode the grip type, and only secondarily (and optionally) the object orientation.

Cue separation task

Given these asymmetries between the coding of grip type and orientation in AIP, we also tested a subset of 120 neurons in the cue separation task, in which the two task instructions were presented in two cue epochs that were separated by an additional planning period (Fig. 1C). This cue separation task was run in two versions, with either the object orientation presented in the first cue epoch and the type instruction in the second (OT task), or the type instruction in the first cue epoch and the orientation in the second (TO task). In addition, all neurons were also tested in the standard delayed grasping task, where similar results were obtained as in the full dataset. This indicated that our subset was representative. An example neuron tested in the cue separation task is shown in Figure 5. For clarity of presentation, only 4 task conditions are shown: the preferred (−25°) and the nonpreferred orientation (50°) for the two grip types. In the standard task (Fig. 5A), the neuron was tuned for orientation and grip type in all three epochs, with the highest activity for power grips and the object tilted to −25°. When only the object orientation was revealed during the first cue (OT task) (Fig. 5B), the cell showed a clear response to the preferred orientation, independent of grip type. Then later, starting with the second cue, the firing rate was additionally modulated with respect to the instructed grip type. In contrast, when the grip type was instructed first (TO task) (Fig. 5C), the cell's firing rate did not reflect this information, but stayed low for all conditions. Only when the object orientation was revealed during the second cue did the neuron respond vigorously and with preference for power grips in the preferred orientation, obviously combining the newly presented orientation information with the grip type information that was given before.

Figure 5.

Neural activity in the cue separation task. Panels show one example neuron during the delayed grasping task (A) and the cue separation task: OT task (B), TO task (C). Different grip types are shown in red (precision) and blue (power), while the two grip orientations are shown in light and dark color. A, The neuron was early tuned for both parameters, showing highest activity for power grips at −25° orientation. B, In the OT task, presentation of the object in the −25° orientation evoked a strong response, which was then differentially modulated for the two grip types after the second cue. C, In the TO task, the cell did not respond to the type cue when presented in the absence of the object. However, the cell responded vigorously after the orientation cue with a preference for power grips in the preferred orientation.

Such a response pattern, with a strong modulation for orientation when presented first, and a delayed modulation for grip type that kicked in only after the orientation cue was revealed, was typical for many cells, as shown in our population analysis (Fig. 6). The top panels show the firing rate of the population in 4 conditions (preferred and nonpreferred type and orientation). In the first cue of the OT task (left panel), the population activity increased strongly for both conditions in which the handle was presented in the preferred orientation. This orientation modulation persisted despite some decrease in activity during the first planning period. When the grip type information was subsequently provided in the second cue, the population response decreased for the nonpreferred grip type, but remained constant for the preferred grip type instruction. This activity pattern suggests that the neural population in AIP encodes movement plans for both types of actions simultaneously, until the ambiguity between the two grip types is resolved by the type instruction.

Figure 6.

Population analysis of the cue separation task. A, Population firing rates in the cue separation task (N = 120) with OT task on the left and TO task on the right panel. For each cell, its preferred type and orientation was established in the delayed grasping task (data not shown). B, Fraction of cells that were significantly tuned by grip type and orientation in the course of the OT and TO task (sliding window ANOVA as in Fig. 3B).

The activity pattern in the OT task also became apparent in the sliding window ANOVA (Fig. 6B, left). Object orientation was maximally encoded at the end of the first cue. After the second cue, grip type was represented in ∼35% of all cells, similar to the planning phase of the delayed grasping task. A second increase in grip type selectivity then occurred during movement execution.

The population activity in the TO task showed a quite different pattern (Fig. 6A, right panel). Neurons responded weakly to the grip type instruction (first cue), but when the object orientation was presented in the second cue, the population activity became tuned for the object orientation and the grip type at once, similar to the population response in the delayed grasping task (Fig. 2D). The sliding window analysis (Fig. 6B, right) showed a reduced number of cells, ∼20%, that displayed grip type selectivity before the object presentation. This selectivity is reflected in the slight increase in population activity of the preferred grip type conditions (red and blue curves) during the first planning epoch (Fig. 6A, right). Overall however, this modulation was weak; only after the presentation of orientation information in the second cue was there an increase in the number of grip type-tuned neurons to a similar level to that observed in the OT task.

Given the weak modulation in the population activity during the first planning of the TO task, the amount of grip type selectivity in the sliding window ANOVA seems surprisingly high, especially when compared with the level found in the second planning epoch. This could in part be explained by an increased sensitivity of the ANOVA for grip type effects in the absence of orientation information before the second cue, due to a reduction of within-group variance in the power and precision groups.

To compare grip type effects at different task phases of the TO task without being influenced by the presence or absence of orientation information, we performed ROC analyses separately for each orientation and averaged the five results (see Materials and Methods). This allowed us to compute, for each individual cell and any task epoch, how well the firing rates during precision grip trials could be discriminated from those during power grip trials. The result of this analysis showed that only 24 cells (20%) significantly discriminated between power and precision trials before the object cue, while this value rose to 62 (52%) after object presentation, confirming that many more cells displayed a grip type effect after object presentation. These findings indicate that the representation of grip type is strongly reduced in AIP in the absence of object information, which corresponds well with our cell classification in the full dataset (Table 2), where grip type-selective neurons during cue were usually orientation tuned as well.

Coding schemes

As we have shown, many neurons in AIP represent the object orientation or the grip type in one or several trial epochs. Here we explore, which grip types and object orientations are preferred in the population, and how these representations change over time. Figure 7A shows the ratio of cells that prefer precision vs power grip. During the cue period, half of the grip type specific cells preferred precision grip and the other half power grip. However, later on in the trial, this ratio shifted in favor of precision-preferring cells, such that during movement execution, the ratio of precision- to power-grip-preferring cells was ∼60 to 40.

Figure 7.

Distribution of preferred grip type and orientation in various task epochs. A, Ratio of cells preferring precision (white) or power grip (black). From cue to movement, the fraction of cells encoding precision grip increased substantially. B, Number of cells preferring each of the five orientations. In the movement epoch, the distribution shifts in favor of terminal orientations (±50°).

A somewhat similar development became apparent when looking at the preferred orientation (Fig. 7B). During cue presentation, the preferred orientations were fairly evenly distributed with only a slight overrepresentation of the extreme (±50°) orientations (47% vs 40% expected from uniform distribution), while during movement execution, the fraction of neurons preferring extreme orientations increased to 59%.

These shifts of preference in the population were not caused by preference changes of individual neurons. Figure 8A illustrates the consistency of grip type preference between consecutive epochs. Between adjacent task epochs, only 2% (cue to planning) and 4% (planning to movement) of the cells tuned in one epoch changed their grip type preference between power and precision (gray bars), while the overwhelming majority of cells remained either tuned for the same grip (black bars; 64% and 70%, respectively) or were no longer significantly tuned (white bars; 34% and 26%). This indicates that in general, the preferred grip type did not change across task epochs but remained constant throughout the task. Similarly, the cell's preferred orientation usually did not change systematically between task epochs, but stayed the same or shifted by one step at most (Fig. 8B,C). Note that a shift by one is usually not meaningful, since the cell's firing rate was often not significantly different between neighboring orientations.

Figure 8.

Tuning consistency across task epochs. A, Grip type tuning. Histogram bars indicate the number of cells that stay tuned for the same grip (black), change preference to the opposite grip (gray), or lose their tuning (white) when transitioning between consecutive task epochs (cue–planning and planning–movement). B, C, Change of orientation tuning in consecutive task epochs: cue–planning (B) and planning–movement (C). Histograms show the fraction of cells for which the preferred orientations in the two epochs were the same (0° shift), neighboring orientations (25°), or further apart (50–100°), and of cells that lost their tuning (white bars). Preferred orientation shifts of >25° were rare. In general, cells were tuned consistently over time.

We demonstrated that the tuning of individual neurons remained largely constant during the task whereas the population tuning shifted at later task epochs toward an overrepresentation of precision grips and extreme orientations. This apparent discrepancy suggests that different populations of cells with diverse coding schemes might be active at different task epochs. To explore this further, we determined the preferred grip type and preferred orientation separately for the three cell groups of early, intermediate, and late tuning onset. We found, in fact, that the cell group with early-onset grip type tuning preferred precision grips and power grips equally likely throughout the task (in the cue, planning, and movement epoch); in contrast, the cell group with a late tuning onset for grip type (in the movement epoch) had a preference ratio for precision and power grips of 70 to 30 (Fig. 9A). Similarly, the cell group with the orientation tuning onset during the cue epoch had a fairly constant rate of neurons preferring terminal orientations (±50°) during the course of the task (cue: 49%, planning: 52%, movement: 54% of cells), while neurons with a late onset of orientation tuning mainly preferred extreme orientations (76% of cells) (see Fig. 9B). Moreover, for both grip type and orientation, the middle group behaved similarly to the early group, suggesting that they followed the same scheme as early-tuned neurons. Such tuning differences between cells with early and late tuning onset suggest that these cell groups encode different entities earlier in the task during movement instruction, compared with later during movement execution. During movement instruction, similar numbers of neurons are allocated for the representation of the two grip types and for the various object orientations. However, later in the task, this coding scheme seems to change in favor of a motor representation, in which the precision grip (being more difficult) requires more cortical resources than the power grip while the preponderance of neurons preferring extreme orientations could indicate a push–pull representation for hand rotation in the pronation–supination direction.

Figure 9.

Distribution of preferred grip type and orientation in different cell classes. A, Ratio of precision and power preference in cell groups with early (top row), intermediate (middle), and late (bottom) tuning onset for grip type. In all three task epochs, early-tuned cells preferred power grips and precision grips approximately equally often. In contrast, ∼70% of late tuning cells preferred precision grips. B, Number of cells with a particular orientation preference for the three cell classes. In early orientation-tuned cells, the portion of cells that preferred extreme orientations (±50°) changed little from cue (49%) to movement (53%), while 78% of cells with a late onset of orientation tuning preferred extreme orientations.

Anatomical organization

Finally, we analyzed whether there is a correlation between the reported functional classification (Table 2) and the location along the intraparietal sulcus (IPS) where the cells were recorded. For this we projected the recording coordinates of each neuron onto an axis parallel to the IPS and then split the cell population into eight bins, according to the cells' posterior-anterior position on that axis, such that each bin contained the same number of cells. This allowed us to calculate, separately for each bin along the IPS, the fraction of cells that belonged to a particular cell class (e.g., early onset orientation-tuned cells). Figure 10 displays the result for orientation and grip type-tuned cells with early and late tuning onset, respectively. Although all cell classes were present in all bins, the distributions were clearly nonuniform, but instead showed steady gradients. Cells with an early tuning onset (orientation or grip type) were found with a higher probability in the posterior recording sites. In contrast, cells with late tuning onset (orientation or grip type) occurred more frequently in the more anterior segments. To assess the significance of these effects, we compared the occurrence of cell classes in the anterior half of the recordings to those in the posterior half (i.e., to the left and the right of the dashed lines in Fig. 10; for the anatomical location of the median, see Fig. 1E,F). For a cell class that was evenly distributed along the anterior–posterior axis, one would expect to see an even distribution of neurons between the anterior and posterior halves. Instead we found that 60% of all cells with early onset of orientation tuning and 61% of the cells with early onset of grip type tuning were located in the posterior half of the neural population, while 66% of the cells with late onset of orientation tuning and 67% with late onset of type tuning were located in the anterior half. All of these findings were significantly different from the null hypothesis of a uniform distribution (binomial test, p < 0.01). Additionally, cells that displayed early orientation tuning but late tuning for grip type (as the example cell in Fig. 2B) did not show such a gradient but were evenly distributed among the different bins. These results suggest the presence of a visuomotor gradient along the IPS, with cells that show strong responses during the cue epoch being more frequently found in the posterior part of AIP, and cells with predominantly motor responses occurring more frequently toward the tip of the IPS. This fits well with anatomical data of neurons projecting from AIP to F5 that seem to be more frequently located in the anterior part of AIP, as Figure 1 of Borra et al. (2008) suggests. Finally, the presence of cells with sensory and motor representations in one area might facilitate sensorimotor transformation (Cisek and Kalaska, 2005; Optican, 2005; Buneo and Andersen, 2006).

Figure 10.

Anatomical distribution of different cell classes. Cells (N = 571) were distributed into eight bins according to their location along the intraparietal sulcus, such that each bin contained the same number of cells. Bin 1 contained the most posterior and bin 8 the most anterior cells (x-axis). Individual panels show the distribution along the intraparietal sulcus (IPS) for a particular cell class (left: orientation tuned, right: grip type tuned, top: early onset, bottom: late onset). Histograms display the fraction of cells in each bin that belonged to the respective cell class. Early onset cells showed a decreasing, late onset cells an increasing gradient from posterior to anterior. Dashed line, Median of the population.

Discussion

We investigated neural activity in AIP during a delayed grasping task, in which monkeys grasped a single object in various orientations with one of two possible grip types (Fig. 1). AIP neurons represented the object orientation and the instructed grip type immediately after cue presentation, indicating that AIP integrates 3D features of graspable objects with contextual information (Fig. 2, Table 1). The representation of grip type was stronger during movement execution than in the cue and planning epochs (Fig. 3) due to grip type-selective cells that became specifically active during movement (Fig. 4). Furthermore, grip type selectivity in the cue epoch was mainly found in cells that were also orientation selective, while the opposite was true in the movement epoch (Table 2).

In the TO task, the grip type was only weakly encoded in response to the grip type cue alone. In contrast, in the OT task, neurons preferring either grip type were activated simultaneously, after object presentation but before the type was instructed (Figs. 5, 6).

Individual cells generally kept the same preference for grip type and object orientation across task epochs (Figs. 7, 8). However, at the population level, neurons with early and late tuning onset had different distributions of preferred conditions (Fig. 9) as well as different anatomical distributions within AIP (Fig. 10).

Anatomical connectivity of AIP

Our findings are compatible with known anatomical connections of AIP. AIP receives input from parietal visual areas (in particular LIP, CIP, and V6a) and from the inferior temporal cortex (TEa, TEm) (Nakamura et al., 2001; Borra et al., 2008). These areas represent spatial and object orientation information of visible objects (Sakata et al., 1997; Tsutsui et al., 2001, 2002; Galletti et al., 2003). Also, AIP receives connections from the prefrontal cortex (areas 46v and 12l) (Borra et al., 2008), which might convey contextual information, as we have observed in AIP. Furthermore, AIP is reciprocally connected to area F5 in the ventral premotor cortex (Luppino et al., 1999) which exhibits similar activity related to hand movements (Rizzolatti et al., 1988; Murata et al., 1997; Raos et al., 2006; Stark et al., 2007) and is considered to be part of the cortical output structures for controlling the hand due to its projections to primary motor cortex and the spinal cord (Rizzolatti et al., 1988; Luppino et al., 1999; Lemon, 2008). Together these previous studies locate AIP at the interface between sensory and motor areas related to hand movement control.

Functional classification of AIP neurons

The group of Sakata described three cell classes in AIP based on their activity during grasping in the light and in the dark (Taira et al., 1990; Sakata et al., 1995, 1997): visual-dominant cells were only active when grasping in the light, visuomotor cells were preferentially active in the light, and motor-dominant cells were equally active for grasping in the light or dark. Furthermore, visually responsive cells were labeled “object type” if they were active in a separate object fixation task, and otherwise classified as “non-object type” (Sakata et al., 1995; Murata et al., 2000).

Our results confirm and extend this classification. Neurons active in the cue period of the delayed grasping task correspond to Sakata's object-type cells (visual dominant and visuomotor). These neurons fall in two subcategories: some are tuned to object orientation without selectivity for grip type, while others are modulated by grip type instruction. Most of these cells remain active during the planning epoch, suggesting a role for working memory or movement planning. Neurons active exclusively during the movement epoch correspond to Sakata's motor-dominant or non-object type/visuomotor classes.

In contrast to the previous categorization, our classification is based on the tuning onset for object orientation and grasp type during the entire course of the task, not just during the movement execution, and therefore quantifies the temporal appearance of object features and actions. This might allow us to draw inferences about the functional role of these cells during sensorimotor transformation.

Sensorimotor transformation and context dependency

The parietal cortex has long been known for its role in sensory–motor transformation (Mountcastle et al., 1975; Andersen, 1997; Scherberger and Andersen, 2003). Different subregions are specialized for particular types of actions, like the lateral intraparietal area for eye movements and the parietal reach region for arm reaching. Neurons in these areas are continuously active from stimulus presentation to action execution (Barash et al., 1991; Snyder et al., 1997). Furthermore, they represent not only the target object, but also context information, to select an appropriate action for that target (Gottlieb and Goldberg, 1999; Kalaska et al., 2003; Gail and Andersen, 2006; Scherberger and Andersen, 2007).

AIP fits well into this scheme. It is specialized for hand grasping, and its function can be well described within the framework of sensorimotor transformation (Taira et al., 1990; Sakata et al., 1995, 1997; Murata et al., 2000). Using a delayed grasping task, we found a strong visual component of AIP activity, with 55% of the cells distinguishing a spatial property of the grasp target—its orientation—already in the cue epoch (Fig. 3A). The activity of the majority of these cells extended to planning and execution (Table 1). Furthermore, 25% of the cells discriminated between power and precision grips already in the cue epoch, although the applicable grip type was not provided by the grasp target but by context information from the LEDs. This demonstrates that AIP represents not only the target object but also context information for action selection.

Our results suggest that upcoming hand movements are initially encoded as an object representation that is modulated by the action context, rather than a representation of a particular hand and finger configuration in purely motor terms. Such a context-dependent enhancement of motor-relevant object features has previously been described as a crucial step in visuomotor transformation: the remapping from a visual object description onto a representation that is more meaningful in motor terms (Allport, 1987; Rizzolatti et al., 1987; Gail and Andersen, 2006). This view is compatible with several aspects of our findings.

First of all, the neural response during cue was dominated by the spatial object feature. Orientation-selective neurons outnumbered the grip type-selective ones more than twofold during cue. In addition, 71% of all grip type-selective neurons were also selective for the object feature orientation during cue.

Second, in the cue separation task we found no increased activity for an abstract grip type instruction in the absence of an object to be grasped (TO task), while neural activity in the OT task was increased immediately after the object orientation cue for all neurons preferring either grip type. AIP neurons therefore seem to represent visual object features together with the ambiguities of the grip type until they are resolved by further instructions.

Finally, neural activity during the cue epoch was consistent with a coding scheme that is possibly more suitable for a uniform representation of object features, whereas late onset cells are probably more motor related, as we discuss in the following section.

Possible coding schemes

It has been argued that activity in cortical areas related to sensorimotor transformation reflects the sensory stimuli and context cues during the instruction phase of the task, while during movement execution these areas represent the movement plan independent of the sensory stimuli. This becomes evident in decision experiments for eye and arm movements (Platt and Glimcher, 1999; Gold and Shadlen, 2000; Scherberger and Andersen, 2007) and in anti-saccade and anti-reaching tasks (Everling et al., 1999; Zhang and Barash, 2000; Gail and Andersen, 2006).

Our study supports this view. Neurons with a tuning onset during cue were stimulus-driven and represented the different object features and potential movement plans roughly in a uniformly distributed manner. In contrast, neurons with tuning onset during movement execution encoded the grasp type independently of the object orientation, and were more frequently tuned for precision grips and for extreme orientations. These neurons therefore seemed to use a different coding scheme than the visually responsive cells.

We consider late-onset neurons to be closely related to movement execution based on the following arguments: the overrepresentation of precision grips could be explained by the need of increased neural resources for controlling fine precision grips as opposed to power grips, as observed in other cortical areas (e.g., M1 and F5) (Muir and Lemon, 1983; Lemon et al., 2004; Umilta et al., 2007). Likewise, the overrepresentation of extreme object orientations could be explained by a motor-related encoding, namely a push–pull representation in pronation/supination coordinates. In contrast, visually responsive cells seem to use a coding which is closer to the visual input, as discussed above.

In summary, AIP neurons are modulated by contextual information about upcoming grasp movements when multiple grip types are possible. The encoding of a motor plan in AIP depends on the presence of knowledge about a target object, suggesting that hand movements are initially encoded by a goal-dependent modulation of the object representation, while during movement execution neurons seem to represent the grip type as such, independent of the target object.