Neural Representation of Hand Kinematics During Prehension in Posterior Parietal Cortex of the Macaque Monkey

Abstract

Studies of hand manipulation neurons in posterior parietal cortex of monkeys suggest that their spike trains represent objects by the hand postures needed for grasping or by the underlying patterns of muscle activation. To analyze the role of hand kinematics and object properties in a trained prehension task, we correlated the firing rates of neurons in anterior area 5 with hand behaviors as monkeys grasped and lifted knobs of different shapes and locations in the workspace. Trials were divided into four classes depending on the approach trajectory: forward, lateral, and local approaches, and regrasps. The task factors controlled by the animal—how and when he used the hand—appeared to play the principal roles in modulating firing rates of area 5 neurons. In all, 77% of neurons studied (58/75) showed significant effects of approach style on firing rates; 80% of the population responded at higher rates and for longer durations on forward or lateral approaches that included reaching, wrist rotation, and hand preshaping prior to contact, but only 13% distinguished the direction of reach. The higher firing rates in reach trials reflected not only the arm movements needed to direct the hand to the target before contact, but persisted through the contact, grasp, and lift stages. Moreover, the approach style exerted a stronger effect on firing rates than object features, such as shape and location, which were distinguished by half of the population. Forty-three percent of the neurons signaled both the object properties and the hand actions used to acquire them. However, the spread in firing rates evoked by each knob on reach and no-reach trials was greater than distinctions between different objects grasped with the same approach style. Our data provide clear evidence for synergies between reaching and grasping that may facilitate smooth, coordinated actions of the arm and hand.

INTRODUCTION

When humans or monkeys reach to grasp an object, the hand is preshaped to match the size and shape of that object. Neurons in the anterior posterior parietal cortex (PPC) have been shown to play a key role in these preshaping mechanisms (Begliomini et al. 2007; Binkofski et al. 1998; Brochier and Umiltà 2008; Castiello 2005; Culham et al. 2003; Ehrsson et al. 2000; Fogassi and Luppino 2005; Frey et al. 2005; Gallese et al. 1994; Gardner 2008; Jeannerod et al. 1995; Milner and Goodale 1995, 2006; Rizzolatti and Luppino 2001; Tunik et al. 2007). Human patients with lesions of PPC fail to shape and orient the hand properly to grasp objects and misdirect the arm during reaching.

Hand manipulation neurons (HMs) in areas 5 and 7 were first described by Mountcastle and colleagues (1975) as cells that responded intensely to grasp of various objects and to purposeful exploratory movements of the hand. HMs were located more laterally in PPC than arm projection neurons that responded to goal-directed reach to objects in peripersonal space. Subsequent studies suggested that HMs represented objects either by the hand postures needed to grasp them or by the underlying patterns of muscle activation. HMs and their projection targets in ventral premotor cortex (PMv) have been postulated to transform relevant visual features of objects—size, shape, and spatial location—into a motor signal that permits the animals to grasp objects efficiently and perform intended manipulatory actions (Brochier and Umiltà 2007; Fogassi and Luppino 2005; Gardner et al. 2007a,c; Jeannerod et al. 1995; Murata et al. 1996, 1997, 2000; Raos et al. 2004, 2006; Rizzolatti and Luppino 2001; Sakata et al. 1995, 1997; Taira et al. 1990; Umiltà et al. 2007).

In our earlier studies of prehension in monkeys (Gardner et al. 2007c), we reported that about one third of HMs in area 5 distinguished the shape of objects grasped during a trained task. It was unclear whether the neural responses encoded the particular hand postures used to grasp each object, the intrinsic object properties sensed by vision and/or touch, the area of tactile contact on the hand during grasping, or simply the task goal.

In this report, we analyze the role of hand actions and visual guidance on the responses of these same neurons during performance of the prehension task. The data to be presented reveal that task trials in which the animal reached to the target and preshaped the hand evoked stronger responses than those without a reach in which the hand was guided by tactile contact with the object surface to secure grasp. Moreover, the approach style used seemed to exert a stronger effect on firing rates than that of the shape or location of the grasped object. These findings are further supported by studies of a smaller subpopulation of neurons in which view of the workspace was occluded by an opaque panel, requiring the animal to remember the shape of the objects tested at each position in the workspace.

METHODS

Neurophysiological and behavioral data were obtained from two adult rhesus monkeys (Macaca mulatta) trained to perform a prehension task; these animals were also used in our earlier studies of PPC (Gardner et al. 2007a,b,c). Experimental protocols were reviewed and approved by the New York University Medical Center Institutional Animal Care and Use Committee and are in accord with the guiding principles for the care and use of experimental animals approved by the American Physiological Society, the National Research Council, and the Society for Neuroscience.

Prehension task

The animals were trained to grasp and lift one of four objects designated by visual cues displayed on a computer monitor. The objects constituted a set of round and rectangular knobs arrayed on a box placed at a comfortable reaching distance from the shoulder. Two round and two rectangular knobs were tested in each session. The round knobs were 15- and 30-mm-diameter spheres. The two rectangular knobs were identical in size and shape, measuring 20 × 20 × 40 mm. To compensate for differences in object mass (brass vs. aluminum), additional weights were placed inside the shape box at the base of the aluminum knob shafts; the total load lifted ranged from 108 g (small round), to 137 g (rectangles), and to 242 g (large round).

In sessions testing the right hand (Fig. 1), the knobs were arrayed from left to right at positions 1) slightly medial to the left shoulder, 2) at the midline, 3) aligned to the right shoulder, and 4) lateral to the right shoulder. During recordings from the right hemisphere, when the left hand was tested, the shape box was shifted to the left side of the chair so that knob 1 was located lateral to the left shoulder, knob 2 was aligned to the left shoulder, knob 3 was at the midline, and knob 4 was medial to the right shoulder (Fig. 2). The large round knob was tested at the shoulder or lateral position; the small sphere occupied either the medial or the midline location. The rectangles were tested at the remaining empty positions on the shape box. The most common orders tested were flanking arrangements with the two rectangles at the outer margins (positions 1 and 4) and the spheres central, or alternating round and rectangular knobs, in both cases with the large round knob lateral to the small one.

Fig. 1.

Kinematic drawings of forward (A), lateral (B), and local (C) approach to the large round knob by monkey H17094 using the right hand. Outlines of the hand and test objects were traced from sequential video frames and superimposed to provide a time series of actions performed on individual trials of the prehension task. Numbers to the right in each panel indicate the time codes of the traced images (min:s:frame) recorded at 30 frames per second (fps). Each trial began with the animal viewing cues on the computer monitor (dashed green lines); the cued knob is denoted in the figure by an asterisk. The hand trajectory toward the knob was smooth and direct regardless of the reach direction. Note the common hand preshaping, contact, and grasp postures used in these trials. Unit H17094-129-2, tactile receptive fields on the tips of digits 2–5 and the thumb.

Fig. 2.

Hand kinematics used by monkey N18588 during task performance with the left hand. Same format as that in Fig. 1. The forward (A), lateral (B), and local (C) approaches used by this animal were similar to those for the other animal, but the grasp postures differed. Monkey N18588, Track 313, clip 7, right hemisphere; passive receptive fields were not determined in this session.

The knob locations in the workspace were represented as four identical black icons displayed on a computer monitor placed directly behind the shape box and raised to eye level. One icon was flashed in red to indicate which knob should be lifted on each trial. The animal had to grasp the knob at the specified position, lift it until an upper stop was contacted, and hold it in place until the computer signaled the accuracy of performance on that trial. If the correct object was lifted and held in place, the animal received a juice reward; if it chose the wrong object, no reward was delivered on that trial. Individual knobs were cued in blocks of 2–10 sequential trials. The order of testing was determined by the computer's random-number generator and the number of repeats by the length of training of the animal.

Task actions were self-paced, without external time constraints on trial initiation or stage duration beyond a 1.5-s pause between cues after the knob was lowered. Our goal was to allow the action to proceed in as natural a fashion as possible using standardized objects located at specific positions in the workspace. The animal was free to select the particular grasp posture for each object that was natural, comfortable, and fluid. Each monkey developed its own grasp strategy that was used repeatedly during the period of study. Likewise, we did not place any constraints on the start position of the hand on each trial and animals were not required to remove the hand from the workspace between trials.

Because there was no required “home” position, the animals began their reach from a variety of start locations. This allowed us to divide the trials into four classes depending on the approach trajectory: 1) forward approach from outside of the immediate workspace of the shape box, 2) lateral approach from one knob to another, 3) local approach from the immediate vicinity of the test knob, and 4) regrasp, in which the same object was grasped repeatedly and lifted without breaking hand contact. The same object could therefore be approached from a variety of directions and objects of the same or similar contours could be grasped at different workspace locations.

In most trials, the animals could view the workspace and all four objects. They could therefore use visual guidance at will to locate specific objects and position their hand on them. We did not require them to fixate the objects or the computer screen and did not specifically monitor gaze or eye movements. In some blocks of trials we occluded view of the workspace by inserting an opaque plate at shoulder level below the animal's chin. This visual block protocol prevented the animal from seeing its hand or the objects, but did not change the environmental light level or interfere with reaching or grasping movements. These trials were used to assess the role of visual guidance on hand kinematics and unit responses in area 5.

Hand kinematics during specific task stages

The actions of the hand and arm during the task were monitored using digital video (DV) recordings of the animal's behavior synchronized to neuronal spike trains, as previously described (Debowy et al. 2001, 2002; Gardner et al. 1999, 2002, 2007a,b,c; Ro et al. 1998). Eye movements and gaze were also visible on the video clips, but were not specifically quantified.

Hand behaviors viewed in frame-by-frame mode allowed us to compile event logs of the start times of the task stages on each trial and visualize how the hand postures evolved over time. The event time codes were subsequently used to align neural responses in rasters and peristimulus time histograms (PSTHs) and to bracket the task stages for statistical analyses of firing rates. The timing of hand actions was measured in increments of the 29.97 frames per second (fps) capture rate of the DV recording system; measurements of the stage duration are therefore accurate to 33.34 ms.

The task timeline consisted of four major epochs: a pretrial interval (stage 0), a movement period in which an object was acquired and manipulated (stages 1–4), a hold interval (stage 5), and a lower-relax period (stages 6–7). The movement period was further subdivided into four stages—approach, contact, grasp, and lift—that reflected the actions performed by the animal to obtain the reward. Each stage reflected the actions of distinct muscle groups in the hand and arm (Brochier et al. 2004; Espinoza and Smith 1990; Picard and Smith 1992).

Stage 1: Approach.

This stage, which marks the start of a trial, begins at the video frame preceding the onset of hand movement toward or on the selected object, and ends when the fingers first contact the target object or start to move over its surface.

Stage 2: Contact.

This stage, which comprises the hand positioning actions whereby the object is secured in the hand, spans the period of hand motion over the knob surface and its full enclosure in the hand. The contact stage ends when tangential motion ceases at the onset of grasping. These actions correspond to the preload stage of prehension described by Johansson and colleagues in human subjects (Johansson 1998; Westling and Johansson 1984, 1987).

Stage 3: Grasp.

This stage describes the period of static enclosure of the knob in the hand when grip and load forces are applied by the hand prior to lift. This period corresponds to the load stage of prehension (Johansson 1988). Although the knob was sometimes rotated during the grasp stage, there was no further tangential motion of the hand over its surface. The grasp stage ends at the start of lift.

Stage 4: Lift.

This stage encompasses the period of upward displacement of the knob 3–5 mm from rest to the top position.

Stage 5: Hold.

This stage marks the end of the movement period. Here, the knob is maintained at the raised position above the shape box and the animal receives a juice reward if the correct object is lifted.

Stage 6: Lower.

This stage begins as the knob is moved downward to the start position and ends when the animal relaxes the grasp.

Stage 7: Relax.

This stage is the period of maintained partial hand contact on the knob in a relaxed posture at the end of the trial. The relax stage ends when hand movement is resumed or when the 4-s sample interval concludes. Subsequent movements include hand withdrawal from the knob to start another trial, local grasps, regrasps of the same object, or spontaneous reaches to other targets in the workspace.

Recording techniques

The neurons described in this report comprise a subset of a larger population recorded in the postcentral gyrus and adjacent cortical areas described in previous reports from our laboratory (Gardner et al. 2007a,b,c). Using techniques for chronic single-unit recordings detailed in Gardner et al. (2007a): a stainless steel chamber was permanently implanted over the postcentral gyrus hand area in an aseptic surgical procedure under general anesthesia; the dura was left intact to prevent infection and contain brain swelling. The recording chamber provided access to a 25-mm-diameter region of cortex centered 2–4 mm posterior and 18–20 mm lateral to the bregma. The rostral end was situated 2–5 mm anterior to the central sulcus and the caudal end was located over the inferior parietal lobule. The recording sites spanned the most caudal region of primary motor cortex (area 4), the postcentral gyrus hand area of S-I, the cortex caudal to S-I bordering the intraparietal sulcus (IPS; area 5 hand region and AIP), and the convexity of the inferior parietal lobule (area 7b).

Extracellular recordings of spike trains were made in the left hemisphere of both animals with epoxylite-insulated tungsten microelectrodes advanced through the intact dura and into the brain by a remotely operated miniature stepping hydraulic motor (Model 607W; David Kopf Instruments). Microelectrode recordings in the right hemisphere of monkey N18588 used a computer-controlled multiple electrode positioning system (Alpha Omega EPS-MT) that allowed simultaneous recordings from four independently mobile tungsten microelectrodes. Calibrated positioning guides placed within the chamber lumen specified the site of microelectrode insertion. The position of each penetration site was marked on photographs of the brain made during surgery and following intracardiac perfusion with 10% buffered formalin.

Task performance was used as a search stimulus to isolate neurons. To maximize data collection, we refrained from mapping receptive fields until the animal took a spontaneous break from the task. Tactile sensitivity was tested with gentle stroking and palpation of the hand, wrist, arm, and shoulder using cotton swabs, soft brushes, or the experimenter's fingers. Light pressure was applied to the skin and joints to distinguish rapidly and slowly adapting neural responses. Passive flexion and extension of the fingers, wrist, elbow, and shoulder joints were used to measure proprioceptive activity.

Data analysis procedures

Spike rasters and peristimulus time histograms (PSTHs), aligned to the video frame onset times of hand contact with the knob, were used to measure the consistency and reliability of neural responses. PSTHs spanning a 4-s interval were smoothed to create spike density functions of firing rate as a function of time using a 50-ms window displaced every 10 ms.

The spike trains from each trial were subdivided into stages linked to the actions recorded in the video clips. Mean firing rates per stage were computed on each trial by dividing the spike count between stage markers by the stage duration. These single trial values provided the input data for statistical analyses of the task variables.

Trials for each neuron were grouped by approach style, knob shape, knob location, and/or visual guidance type. The individual firing rates per stage were averaged across trials for each task condition to compute average firing rate graphs that were independent of the action used to align the PSTHs. The values obtained were used to determine the onset of task-linked activity and the stage of peak firing.

Neural responses during the seven task stages and the pretrial interval were assessed using a repeated-measures ANOVA model (one- or two-way ANOVA, StatView, SAS Institute, and/or SPSS). Mean firing rates measured during each task stage served as within-subject factors in the repeated-measures model and the experimental variables tested in that trial (e.g., forward, lateral, local, and regrasp for approach style) were between-subject factors. The factor was considered to have a significant main effect on firing rate throughout the task if F-tests yielded P < 0.05. The same significance criteria were used in post hoc tests (Bonferroni–Dunn and Tukey honestly significant difference) to assess interactions of these factors during some task stages but not others. Two-way repeated-measures ANOVAs measured the interaction of approach styles and object features on firing rates.

Neurons were considered “task-related” if their firing rates demonstrated significant stage-dependent modulation across the seven task stages and the pretrial interval (F-test, P < 0.001). Task-related neurons were also required to show significantly increased or decreased firing rates during at least one task stage compared with the pretrial rate in paired means comparisons (P < 0.05).

To calculate population responses to the various task variables, we compiled normalized response profiles for each neuron by dividing the mean firing rate per stage for each condition by the peak mean response of the neuron averaged across all trials of the session. Normalized mean firing rates for each approach style, knob shape, knob position, and visual condition were averaged across the population and plotted as a function of task stage in Figs. 7 and 10.

Fig. 7.

Population mean normalized firing rates per stage plotted as a function of approach style (A), object shape (B), and object location (C) for the 75 neurons studied in the 2 animals. P values in each panel indicate the selectivity for style, shape, or location as determined by repeated-measures ANOVA; the significance of interaction effects on firing rates across the task stages is also noted. Top panels: trials grouped by approach style (A) showed greater disparity than those grouped by object shape (B) or location (C). Bottom panels: approach styles in the population were grouped into reach trials (forward and lateral approach, solid lines) and no-reach trials (local approach and regrasp, dashed lines). Reach trials evoked significantly greater responses than no-reach trials during the approach, contact, grasp, and lift stages, as well as in the pretrial interval. The distinction between reach and no-reach trials was greater than that of the object shape (B) or location (C) within each style.

Fig. 10.

A: population mean normalized firing rates per stage for the 25 neurons studied as a function of visual condition during reach (R) and no-reach (NR) trials. Firing rates were slightly higher during sighted trials than when vision of the workspace was blocked, but the difference was not significant. B: the mean stage duration in sighted and blocked trials differed only during the contact stage as the hand was positioned for grasp. For clarity, timing data are shown only for the reach trials.

RESULTS

This report describes the effect of approach style on the prehension responses of 75 task-related neurons recorded in the hand representation of area 5 in the posterior parietal cortex of two monkeys (left hemisphere, n = 46; right hemisphere, n = 29). The recording sites lie caudal to the S-I hand area and are highlighted in light blue in Fig. 3 of Gardner et al. (2007c). Each of these cells was tested with four objects over a minimum total of 50 trials. Although the grasp posture and hand used differed between the two animals, both of them performed the movements consistently in a continuous sequence across sessions and days once the task was learned.

Fig. 3.

Spike rasters comparing responses of an area 5 neuron in monkey H17094 to the four approach styles. Each of the raster panels is aligned to contact, subdivided by knob location and shape, and ordered chronologically within each style. Colored markers in the rasters show the onset times of the task stages on each trial: 1. Approach (gold); 2. Contact (red); 3. Grasp (magenta); 4. Lift (dark blue); 5. Hold (light blue); 6. Lower (dark green); 7. Relax (green); 8. Release (cyan). Forward (A) and lateral (B) approach trials evoked higher firing rates and longer duration responses than local grasps (C) and regrasps (D). Spike trains evoked by individual knobs acquired with the same style resembled each other. Regrasp trials in which the animal used the “ulnar grasp” posture are marked with a cyan bar to the left of the raster in D. Unit H17094-131-3.2, tactile receptive field located on the glabrous and hairy skin of the hand and digits.

Kinematics of the prehension task

Four approach styles were used by each of the two animals to perform the task. Forward and lateral approaches included a distinct reach and a period of hand preshaping; local approach and regrasp trials had no reach component. These styles are illustrated in Figs. 1 and 2 by kinematic drawings of individual trials made by tracing outlines of the hand, fingers, and objects in single video frames. By superimposing successive images we constructed a time series of kinematic drawings delineating the task stages of each trial (Gardner et al. 2007a; Reitzen et al. 2004).

During forward approach trials the animal reached toward the object from outside the immediate environment of the shape box. When the hand started from the chair frame (Fig. 1A), the initial movement was an elevation of the wrist (green hand), followed by lift of the fingertips from its surface. As the animal raised the hand, the fingers straightened and were aimed toward the target in an upwardly curved path (turquoise). As the hand neared the knob, its forward rate slowed and the wrist was rotated to bring the ulnar side of the palm downward. The fingers were curved to conform to the shape of the large sphere or maintained in the extended posture to match the flat face of the rectangular knob. The tips of the index and middle fingers were the first to make contact with the knob (pink), followed by the other fingers and the palm. The fingers traced the surface of the sphere, or slid smoothly into place along the face of the rectangular knob, and came to a stop when the object was fully enclosed by the hand and grasped (yellow). Grasping was succeeded almost immediately by lift. The same hand preshaping and orientation actions were performed when approach began from above the shape box (Fig. 2A).

During lateral approach trials, the animal reached to the target object from another location on the shape box, usually the previously cued object (Figs. 1B and 2B). At the start of approach, the hand was retracted toward the body then lifted above the knobs and moved with the fingers extended. Although the reaching movement was directed to the left or right, parallel to the frontal plane, rather than forward in space, the same general pattern of wrist rotation and hand preshaping occurred during lateral and forward approach. Likewise, the contact site on the object and the grasp and lift postures were the same regardless of the approach path.

During local approach trials, there was no reach because the hand remained in loose contact with the knob at the end of the previous trial (Fig. 1C) or was rested on the shape box below the knob or adjacent to it (Fig. 2C). The animal appeared to use tactile cues to guide its hand over the knob surface into the preferred position for grasping. The video frame preceding the start of forward hand movement was designated as the approach stage in Fig. 1C. The approach stage in Fig. 2C spanned the interval from one frame prior to the start of upward hand movement to contact of the knob by the glabrous fingertips.

On regrasp trials, the animal maintained full hand contact with the knob after lowering it and then lifted the same knob again. Consequently, there was no reach or active hand positioning movements on the knob during regrasp trials, but only changes in the applied grip and/or load forces. To compare neural firing rates on trials with and without an active hand positioning interval, we designated the video frame preceding lift as the “grasp” stage and the two preceding frames as the “approach” and “contact” stages during regrasp trials.

One of the animals studied developed a unique variant on the regrasp behavior in which the knob was not fully grasped, but instead propped up with the fingertips (see Fig. 6 in Gardner et al. 2007a). These trials began with the hand placed on the side of a knob. If that same object was cued on the next trial, the animal supinated the hand and placed the two ulnar digits below the knob to prop it up, with the palm facing up and digits extended. The period of hand supination was designated as the “contact” stage and the frame preceding lift as “grasp.” We called this behavior “ulnar grasp” because lift was accomplished by upward rotation of the ulnar part of the hand.

Fig. 6.

Mean stage durations (±SE) for the 4 approach styles are similar in the 2 animals. The reach style modified the duration of the approach, contact, and hold stages but the grasp and lift stages lasted the same time regardless of how the knob was acquired.

The distribution of the four trial types was similar in the two animals (Table 1). Lateral and local approaches were the most common types averaging about 30% for each style. This reflected the tendency of both animals to leave the hand in the workspace close to the rewarded objects. Regrasps were more common in monkey H17094, as a result of “ulnar grasps” used during some trials.

Table 1.

Distribution of trial types in the population studied

	Both Animals			H17094			N18588
Approach Style	Mean	SE	% Total	Mean	SE	% Total	Mean	SE	% Total
Forward	20.5	1.5	18.8	22.9	2.5	19.0	17.4	1.3	18.4
Lateral	32.9	1.6	30.0	35.3	2.6	28.4	29.8	1.4	32.0
Local	39.1	2.5	34.4	41.5	4.0	35.8	35.9	2.3	37.7
Regrasp	18.3	1.5	16.8	23.9	2.2	20.7	11.3	1.0	11.9

Spike trains of area 5 neurons reflect approach styles used in the task

The contact-aligned rasters of Fig. 3 illustrate the effect of the four approach styles on neural spike trains. Neural responses were strongest during forward (Fig. 3A) and lateral approach trials (Fig. 3B) and were similar in time course and firing patterns. Firing rates increased abruptly coincident with the onset of reach (gold marker) and peaked at contact (red). High firing rates were maintained as grasp was secured (magenta), but decreased as lift began (dark blue). The spike trains returned to or below the baseline level at the start of the hold stage (light blue) and remained at low levels until the knob was lowered and the grasp relaxed (green).

Responses were weaker and shorter in duration during local approach trials (Fig. 3C). Firing rates were relatively low prior to contact and did not rise to the same high levels during hand positioning and grasping as on trials that included a reach. Regrasps of the same object (Fig. 3D) or ulnar grasps evoked the lowest firing rates and the briefest spike burst. These data suggest that the early actions of the task—reaching, hand preshaping, and positioning on the knob—enhanced the subsequent responses to grasp and lift.

The rasters showed little difference in spike trains evoked by the four knobs when approached with the same reach style. This may reflect the fact that three of the knobs were grasped with similar hand postures.

The superimposed spike density plots of Fig. 4 A quantify these raster data. In each trial type, the task-evoked spike burst seemed to track the movement interval from the start of approach (gold marker) to the end of lift (light blue marker). Forward (green trace) and lateral (blue trace) approach styles evoked the strongest and earliest onset responses, beginning about 250 ms before contact. Their firing rate profiles were closely matched, suggesting that responses of this neuron were independent of the direction of reach. Neural activity peaked at contact and as grasp was secured and declined rapidly toward baseline after lift began.

Fig. 4.

Superimposed spike density plots (left) and average firing rates per task stage ± SE (right) compiled for the neuron in Fig. 3. Responses in the spike density plots are aligned to contact and grouped by approach style (A) and object shape (B). Markers above the spike density graphs show the mean onset times of the task stages averaged across all trials of the stated class; same color codes per stage as those in Fig. 3. The top-to-bottom listing of the knobs in the key corresponds to their medial-to-lateral positions on the shape box. P values in the average firing rate graphs indicate the selectivity of the neuron for approach style (A) and object shape (B) as determined by repeated-measures ANOVA; distinctions between approach styles were stronger than those between the various objects.

Responses on local approach trials (gold trace) rose later in the trial and were weaker in intensity. Firing rates peaked during the grasp stage and decayed in parallel with responses to the other trial types. The weakest responses occurred on regrasp trials (red trace) in which the firing rate increased primarily during lift. Responses in all four trial types converged during the lift and hold stages and appeared suppressed during the remainder of the trial.

As a result of differences in response onset, spike trains evoked during forward and lateral approach lasted longer than on trials without a distinct reach and the peak response amplitude was also higher. Consequently, the area under the spike density plots was greatest during forward and lateral approaches and lowest during regrasps.

In contrast, when the same data were replotted as a function of the knob tested at each position (Fig. 4B), the PSTHs did not differ in time course. The two rectangular knobs evoked nearly identical responses even though they were located at different positions on the shape box. The small round knob evoked the strongest responses from this neuron during grasping and the large round knob the weakest activity. However, differences in spike density evoked by the four knobs were weaker than when spike trains were grouped by approach styles.

The influence of hand actions was also quantified by statistical comparison of the average firing rates per stage for the four approach styles (Fig. 4A, right). We found a significant main effect of approach style on firing rates throughout the task [F(3,190) = 17.13; P < 0.0001, one-way repeated-measures ANOVA] as well as significant interactions between the approach style and firing rates during specific task stages [F(21,190) = 6.95; P < 0.0001]. Firing rates did not differ during forward and lateral approaches [F(1,99) = 2.18; P = 0.14] and were highest during the contact stage. Post hoc tests indicated that mean firing rates during the approach and contact stages were significantly higher in the forward and lateral approaches trials than on local approach, but not in later stages. All three styles evoked significantly higher firing rates than regrasps during object acquisition, but not in later stages.

There was no significant main effect of object shape or location on firing rates [F(3,190) = 1.65; P = 0.18, one-way repeated-measures ANOVA]. The two rectangular knobs evoked nearly identical responses even though they were located at different positions on the shape box (Fig. 4B, right panels). The small round knob evoked the strongest responses during grasping and the large round knob the weakest activity, but the interactions between the knob tested and the actions performed during specific task stages were not significant [F(21,190) = 1.24; P = 0.21].

Similar effects of approach style on firing rates were observed in the other animal studied even though it handled the knobs with a less-precise grasp posture. The neural responses illustrated in Fig. 5 displayed a significant main effect of approach style on firing rates [F(3,98) = 10.43; P < 0.0001]. Forward and lateral approach trials evoked the strongest responses in this cell (Fig. 5, A and B). Firing rates increased at the onset of reaching, peaked during the contact stage, and fell abruptly as the knob was grasped and lifted. The neuron remained nearly silent for the remainder of the trial.

Fig. 5.

Selectivity of an area 5 neuron recorded in monkey N18588 for approach style and object properties. A–D: rasters of all 107 trials of this neuron are grouped by approach style, and subdivided by knob shape and location. Same format as Fig. 3. Forward and lateral approach styles evoked the highest firing rates and longest duration responses. Regrasps (D) occurred most frequently at the site lateral to the active arm (knob 1) and on only one trial at the medial and midline sites (knobs 3 and 4). E: spike density plots for the 4 approach styles. F: average rate graphs for approach style (left) and knob shape (right).

Significant interactions between approach style and task actions occurred early in these trials [F(21,98) = 2.85; P < 0.0001]. Firing rates were significantly lower prior to contact during local approach (Fig. 5C), but were indistinguishable from forward or lateral approach trials in subsequent stages, reflecting the common hand positioning and grasping actions used. Responses to regrasps were weaker than the other trial types, signaling only the onset of lift (Fig. 5D). During the hold stage, the weight of the object had a dominant effect on neural responses. Post hoc tests showed that the large round knob, which was the heaviest object, evoked significantly higher firing rates than the other knobs in the hold stage (medial rectangle P = 0.006; small round P = 0.024; lateral rectangle P = 0.01).

The task timeline data shown in Figs. 3–5 are typical of the temporal characteristics of prehension behaviors in the population analyzed. The mean stage duration for each animal is illustrated in Fig. 6 as a function of the approach style. The approach stage duration was roughly proportional to the reach path length, with the longest times for forward approach. Hand positioning at contact was faster during forward and lateral approach than in local approach trials, perhaps because the momentum of reaching enabled the animal to position the hand rapidly at the grasp sites on the object. The grasp stage was the briefest task action because movements shifted rapidly from hand positioning to lift. Lifting typically lasted three to four video frames; the lift stage duration was the most consistent period across trials, recording sessions, and animals. The hold stage—which marked the end of the movement period—was the most variable in duration from trial to trial. Here, the animal received a juice reward if the correct knob was lifted. On some trials the animal drank the juice with the knob maintained at the raised position; on others, it lowered the knob and relaxed the grip while licking the juice tube. As a result, the duration of the hold stage depended primarily on when the animal consumed its reward.

Population representation of style selectivity in area 5

The trends observed in individual neurons were clearly evident in the population activity when responses of all of the neurons studied were pooled. To compare the effect of hand kinematics and object properties across the population studied, we normalized the average firing rate graphs for each neuron by dividing the mean firing rates per stage by the peak mean response of the neuron averaged across all trials during the session. The normalized firing rate profiles were averaged across the population after grouping the trials by specific approach styles, knob shapes, or locations and plotted as a function of task stage in the top panels of Fig. 7.

The population responses displayed the same characteristics as the sample neurons in Figs. 3–5. The greatest spread in firing rates occurred when the population responses were grouped by approach style (Fig. 7A). Neural firing rates in forward and lateral approach trials were ≥40% higher during stages 1–3 than on regrasp trials. Local approach responses were intermediate in intensity and regrasps evoked the weakest mean firing rates throughout the movement period. Moreover, peak activity in local approach and regrasp trials was delayed compared with trials in which the animal reached to the object.

Statistical analyses of the firing rates of the total population studied (one-way repeated-measures ANOVA) demonstrated a significant main effect of approach style on firing rates throughout the task [F(3,296) = 11.22; P < 0.0001]. There were also significant interactions between the four approach styles and the actions performed during different stages of the task [F(21,296) = 16.91; P < 0.0001].

Post hoc comparisons of approach styles across the task stages indicated clear differences between trials that included a distinct reach (forward and lateral approach) and those that did not (local approach and regrasps). During the approach stage, mean firing rates showed clear rank-order preferences between trial types.The strongest responses were recorded on lateral approach trials, where mean firing rates were significantly higher than those on forward approach trials (P = 0.025) and each of the no-reach types (P < 0.0001). Forward approach trials, in turn, evoked significantly higher firing rates than those on local approach (P = 0.048) and regrasps (P < 0.0001) and local approach responses were stronger than regrasps (P < 0.0001).

The greatest spread in firing rates occurred at contact, where forward and lateral approaches evoked significantly higher rates than those of local approach and regrasp trials (P < 0.0001), but did not differ from each other (P = 1.0). In the grasp stage, population responses to forward and lateral approaches remained stronger than local approaches, but the difference in mean firing rates was not significant (forward: P = 0.062; lateral: P = 0.14). Regrasps evoked significantly weaker responses than those of forward or lateral approaches during all of the movement stages (approach through grasp, P < 0.0001; lift, P = 0.002), but not in the later stages when firing rates had returned to baseline (P = 1.0). Local approach responses remained significantly higher than those on regrasp trials during contact and grasping (P = 0.0004 and 0.020), but not during lift (P = 0.058).

Interestingly, firing rates during the pretrial period also differed somewhat, paralleling those observed during approach, but only lateral approach responses were significantly higher than those of the other three types (P < 0.0001 for local and regrasp; P = 0.009 for forward); forward approach was significantly higher than regrasps (P = 0.015) but not local approach (P = 1.0). These findings suggest that neural activity in area 5 increased in the period prior to the onset of visible motion of the hand or arm.

Statistical tests were also used to quantify approach style preferences among the individual members of the 75-cell population. Mean firing rates per stage were computed for each trial and analyzed using one-way repeated-measures ANOVA with the four approach styles as between-subjects factors. Fifty-five percent of the neurons analyzed (41/75) showed significant main effects of approach style on firing rates (P < 0.05) and 69% (52/75) showed significant interactions between style and task stage. Because there was only partial overlap between the groups showing significant main and interaction effects (Table 2A), 58 of 75 neurons studied (77%) distinguished the approach style used to perform the task by modulating their firing rates during all or part of the task.

Table 2.

Statistical analyses summary

	Four Styles (One-Way ANOVA)		Reach–No Reach (Two-Way ANOVA)
A. Approach Style	Total Cells	% Total	Total Cells	% Total
Significant Main and Interaction	35	46.7	32	42.7
Significant Main only	6	8.0	4	5.3
Significant Interaction only	17	22.7	18	24.0
Total Significant style	58	77.3	54	72.0

	Four Knobs (One-Way ANOVA)		Four Knobs (Two-Way ANOVA)
B. Object Features	Total Cells	% Total	Total Cells	% Total
Significant Main and Interaction	12	16.0	11	14.7
Significant Main only	14	18.7	11	14.7
Significant Interaction only	12	16.0	13	17.3
Total Significant knob	38	50.7	35	46.7

Forward and lateral approach trials evoked the strongest mean responses in 80% (60/75) of neurons studied. The percentage of neurons preferring forward and lateral approaches was even higher (91%, 53/58) when the population analysis was restricted to cells showing significant approach style–related activity. Only 10 of 75 cells (13%) distinguished forward and lateral approach styles; eight neurons fired at significantly higher rates in lateral approach trials and two responded more vigorously to the forward approach. Seventeen cells showed significant interactions between approach style and task stage in comparisons of forward and lateral approaches.

Neurons firing favoring local approach or regrasp trials were rare and generally did not show significant effects of approach style on firing rates. They comprised 9 of 17 cells that did not distinguish approach styles and generally showed peak responses during the grasp stage.

Taken together, these findings suggest that actions related to hand placement on the object during the approach and contact stage enhanced subsequent neural responses and hand behaviors as objects were grasped and manipulated. The higher firing rates in forward and lateral approach trials reflected not only the need to direct, orient, and preshape the hand to the target during approach, but persisted through the contact and grasp stages. The population responses converged in the hold stage when the task goals had been achieved, and movement ceased, and in the remainder of the task.

Neural coding of object properties

The influence of approach style on firing rates was particularly striking compared with the differences in firing rates evoked by the four knob shapes or the four object positions on the shape box. When the population responses were plotted as a function of object shape (Fig. 7B), the small round knob produced the highest firing rates and the weakest responses were evoked by the lateral rectangle; responses to the medial rectangle were nearly as high as those produced by the small sphere. However, the peak mean firing rates evoked by these objects differed by at most 15%, much less than the spread by approach style. Firing rates in the other stages overlapped.

Statistical analyses using one-way repeated-measures ANOVA indicated no significant main effect of object shape on firing rates [F(3,296) = 0.74; P = 0.53]. However, there were significant interactions between object properties and task actions [F(21,296) = 2.39; P = 0.0004]. Post hoc tests showed significant differences between the objects only during the contact stage. The population responses to the lateral rectangle were significantly weaker than those to the medial rectangle (P = 0.023) and to the small round knob (P = 0.005). None of the other knob pairings was significant.

Because the two rectangular knobs differed only in location on the shape box—and not in size, shape, or weight—we also examined whether these neurons distinguished the knob positions on the shape box rather than their intrinsic features (Fig. 7C). Although these neurons responded more vigorously to objects placed at or across the midline, than to objects lateral to the test shoulder, the differences in the population mean firing rates were not significant [F(3,296) = 1.92; P = 0.13]. However, ANOVA analyses revealed significant interactions between the knob locations and hand actions during the task stages that were similar to the effects of knob shape [F(21,296) = 2.44; P = 0.0003]. Post hoc tests showed that these effects were significant only during the contact stage. Objects placed at the lateral position evoked significantly weaker responses than those at the midline (P = 0.003) or the medial location (P = 0.011).

The relative importance of approach style and object properties is also apparent in the spike trains of the individual neurons studied. Whereas 58/75 neurons analyzed (77%) showed significant effects of approach style on their firing rates (Table 2A), only half of the population (38/75) showed significant object selectivity when their firing rates were grouped by knob shape and/or location (Table 2B). Moreover, 84% of the object selective population (32/38) was also selective for the approach style used to perform the task. As a result, 43% of the neurons analyzed (32/75) signaled both the object properties and the hand actions used to acquire them. These findings were similar across the two animals despite the differences in hand postures used for grasping or the hand performing the task.

Interaction between approach style and object properties

In light of the strong correlation between neurons showing object selectivity and approach style preferences, we reanalyzed the population responses using two-way repeated-measures ANOVA to measure interactions between the knob tested and the reach style used. To have sufficient statistical power for these analyses we first subdivided each neuron's responses to the four objects into trials that included a reach (forward and lateral approaches) and those that did not (local approach and regrasp).

The population normalized firing rate profiles of the reach and no-reach trials (Fig. 7A, bottom) showed an even cleaner separation of these two performance styles than the four-style analyses shown in Fig. 7A, top. The spread between the neural responses was greatest at contact but persisted throughout the movement period from approach through lift. Firing rates during reach trials were significantly higher than those on no-reach trials during the approach, contact, and grasp stages (P < 0.0001), but not during lift (P = 0.11) or later stages (P = 0.65, 0.58, 0.44). Even the background firing rate shown in the pretrial period was significantly higher on reach trials (P < 0.0001), suggesting that neural firing rates started to rise before overt hand or arm movements were visible in the video images.

From a single-cell perspective, 84% of neurons studied (63/75) showed a preference for trials including a reach; the preference for reach trials was significant in 53 of the 63 neurons (P-style <0.05). All but four of the cells showing significant selectivity in the four-style analyses maintained their style selectivity in the reach–no reach pairing. As in the population responses, the effect of reaching was most pronounced in the contact stage for individual neurons as the hand moved over the object, but was also strong during the approach and grasp stages.

When neural responses were sorted by both object properties and reach style, the normalized population profiles formed two distinct bands during the approach, contact, and grasp stages. The spread in firing rates for each knob between reach trials and no-reach trials during these acquisition stages was far greater than the distinctions between different objects grasped with the same reach style (Fig. 7B, bottom) or between the various knob locations on the shape box (Fig. 7C).

Statistical analyses confirmed that the reach style used during individual trials had the most important modulatory effect on firing rates, second only to the task actions within each trial. The population response profiles showed significant main effects of reach style on firing rates [F(1,584) = 60.30, P-style < 0.0001, two-way repeated-measures ANOVA] but not of knob shape [F(3,584) = 0.083, P-shape = 0.97] or location [F(3,584) = 1.12, P-location = 0.34]. Likewise, there was no significant interaction of reach style and knob shape [F(3,584) = 0.27, P-shape × style = 0.85] or reach style and knob location [F(3,584) = 0.48, P-location × style = 0.69]. As a result, 67.4% of the total variance in firing rates was attributed to the progression of hand actions from stage to stage, 26.6% was contributed by the approach style used to bring the hand to the object, and 4.9% to the interaction of approach style and task stage. The knob shape contributed only 0.04% of the total variance and the interactions of approach style and knob, task stage and knob, and those of all three factors together contributed 0.53% of the variance. Likewise, the object location contributed 0.47% of the variance and the interaction terms another 0.59%.

The mean normalized firing rates evoked by the various objects were more tightly grouped on reach trials than on no-reach trials, particularly in tests of the knob location. Objects at the midline or medial sites evoked stronger mean responses in the population than those located at the shoulder or lateral positions. The positional effects were similar in reach and no-reach trials. The medial rectangle evoked stronger responses than those of the lateral rectangle in all trial types.

The same object and location preferences demonstrated in the population at large were also seen in the individual neurons analyzed. The small round knob evoked the highest mean firing rates in 41% of the neurons tested (31/75), the medial rectangle evoked the strongest responses in 27% of neurons, and the large round knob in only 13% of the cells (10/75). Knobs located at the midline or the medial positions were equally likely to evoke strong responses; these positions were each favored by one third of the cells tested. The least preferred position was located lateral to the test arm; this site was favored by only 9% (7/75) of neurons.

The small round knob was the overwhelming favorite of 13 of 17 neurons (76%) that distinguished the knob shape in the reach or no-reach conditions. This object was normally grasped with the most flexed hand posture. The preferred location in the cross-parameter analyses was split between the medial and midline locations, reflecting the shift in position of the small sphere in those sessions. Although the large round knob was heavier than the other three objects, and therefore required greater load forces during lifting, the mean normalized firing rates were nearly identical for both spheres in the lift stage (small round = 72.5 spikes/s; large round = 73.2 spikes/s, P = 1.0). Object weight influenced neural firing rates only during the hold stage when the large round knob evoked significantly higher firing rates (44.1 spikes/s) than the small round knob (36.9 spikes/s, P = 0.001) or the two rectangles (medial = 38.6, P = 0.037; lateral = 37.0, P = 0.004). However, the responses to the large sphere during holding were less than half the rates obtained in the contact stage (111.5 spikes/s) and were lower than those in the pretrial period (50.1 spikes/s).

The effects of reach style and object properties were most pronounced during the contact and grasp stages when firing rates peaked. Responses to the various objects, locations, and reach styles merged during lift as firing rates fell toward baseline. As a result, the two-way ANOVA analyses revealed significant interactions between the task stage actions and the reach style, object shape, and knob location, similar to those demonstrated in the one-way analyses of these parameters [F(7,584) = 44.47, P-style < 0.0001; F(21,584) = 1.87, P-shape = 0.0094; F(21,584) = 2.12; P-location = 0.0021]. The dominant object properties shifted during the various task stages from the reach location, to the grasp posture, to the load force needed to match the object weight following lift.

Visual guidance of prehension

We also measured the effects of visual guidance on task performance in 25 of the neurons described earlier. These sessions compared sighted trials, in which the animal had full view of all four test objects, to visual block trials, in which view of the shape box and the animal's hands was occluded by an opaque plate placed under the animal's chin and secured to the chair frame (Fig. 8). The environmental light level remained constant in the sighted and blocked trials and the animal had full view of the computer monitor that provided cues about the rewarded object on each trial.

Fig. 8.

Kinematic drawings demonstrating clumsy grasps during visual block trials; same format as that in Fig. 1. An opaque plate placed below the chin blocked view of the workspace and the hand, but did not physically impede reaching or grasping movements. A: lateral approach trial in which the animal palpated the top surface and fumbled with the knob before grasping it. B: forward approach trial in which the animal slid the ulnar margin of the hand along the top of the shape box and used an abnormally large grip aperture during approach. The left hand rested on the medial rectangle, thereby providing proprioceptive cues about the relative locations of the knobs in the workspace. Unit H17094-131-3, clip 7; neural responses from this cell are shown in Fig. 9A.

During sighted trials, the animals typically looked at the computer screen to obtain the cue, then looked at the objects, maintaining fixation until the hand approached or touched the target. On contact, the gaze shifted back to the computer screen for visual feedback concerning the accuracy of object selection. Although the animals could locate the objects with their hand when vision of the shape box was occluded by the plate, they clearly found the task more difficult than on sighted trials. They performed fewer trials during visual block and when working tended to maintain the active hand in contact with the last object tested to establish a body- or hand-centered reference frame. This strategy was further aided by placement of the other hand on the medial edge of the shape box (Fig. 8B) and using it on some trials that tested the most medial knob.

Acquisition movements were often clumsy on blocked trials and errors more frequent than on sighted ones. In contrast to the rapid, smooth, and accurate hand placement on the knob during sighted trials (Fig. 1), hand contact on the knobs during visual block trials was sometimes tentative, as the animal used the fingertips to probe the workspace (Fig. 8A) or slid the ulnar margin of the hand along the top of the shape box to locate the objects (Fig. 8B). Palpation of the knob surface, use of an abnormally large grip aperture during approach, or poor orientation of the hand often dislodged the object from the optimum grasp position, prolonging its acquisition time. As a result, the contact stage generally lasted roughly 50 ms longer in the blocked condition. However, once the object was secured in the hand, the remaining manipulations were executed normally.

Errors in knob selection were also more common on visual block trials. The most common mistakes involved missed grasps in which the animal reached too far across the midline beyond the medial knob. Although the neural responses increased normally during approach, they terminated abruptly when the animal grasped the air, sensing only the impact of his fingers on the palm rather than the target object. The animal then shifted his hand laterally and successfully grasped the cued knob. The corrective movement was signaled by a second spike burst similar to activity generated during lateral approach to a neighboring knob. Likewise, when the animal touched an apparently incorrect object, the tactile signal at contact led it to abort the trial and redirect the reach to another object. The corrective maneuvers were signaled by a brief drop in firing, followed by another spike burst as he reached to a different object. In this manner, the neural responses appeared to reflect both the animal's intentions during approach and the motor actions performed during the task.

The impaired performance during visual block conditions was observed in both of the animals studied. We quantified the frequency of clumsy or impaired trials from 23 2-min clips from five randomly selected sessions in monkey H17094 and 18 clips from four sessions in monkey N18588 testing the right hand. Monkey H17094 showed clumsy behavior in 3 of 233 sighted trials (1.3%) and in 55 of 282 visual block trials (19.5%). Monkey N18588 showed clumsy actions in 6 of 286 sighted trials (2.1%) and in 21 of 203 visual block trials (10.3%).

Neural responses on sighted and blocked trials are compared in Fig. 9 for three representative hand manipulation neurons. The effects of visual guidance on firing rates were not dramatic. Distinctions between sighted and blocked trials, when present, occurred primarily in the contact and grasp stages when task performance was most likely to be impaired by clumsy behaviors. PSTHs and average firing rate graphs overlapped in the other task stages. Nearly half (12/25) of the neurons tested showed significant sight-dependent modulation of firing rates (two-way repeated-measures ANOVA): 5 fired at highest rates on sighted reach trials (Fig. 9A), 5 were most responsive on blocked reach trials (Fig. 9B), 2 preferred sighted no reach trials, and the remainder showed no significant difference between visual conditions (Fig. 9C).

Fig. 9.

Spike density plots (left), average firing rate graphs (center), and mean stage duration (right) comparing sighted and blocked reach trials for 3 neurons in monkey H17094. A: neuron favoring sighted trials; same neuron as that in Figs. 3, 4, and 8. B: neuron favoring blocked trials. It also responded to passive stroking of the dorsum of digit 3 (proximal phalanx) and passive flexion of the metacarpophalangeal joints of digits 2–5. C: neuron that did not distinguish the 2 trial types; receptive field not identified. The mean duration of stage 2 (contact) was significantly longer on blocked trials in all 3 neurons.

The population mean normalized firing rates (Fig. 10A) reflected the weak effect of visual guidance on neural responses. Firing rates did not differ in the pretrial and approach stages, but diverged at contact, where they were higher on sighted trials. These higher firing rates persisted during grasp, perhaps reflecting the shorter duration of the contact stage (Fig. 10B) and earlier onset of grasping when visual guidance was available. During no-reach trials, there was little difference in firing rates between sighted and blocked conditions. Visual guidance was apparently unnecessary on no-reach trials because the animal's hand was already on or near the cued object.

Statistical analyses of the population firing rates demonstrated significant main effects of reach style on the population response profiles [F(1,96) = 9.49, P = 0.0027, two-way repeated-measures ANOVA], but not of vision [F(1,96) = 0.57, P = 0.45] nor of reach × vision [F(1,96) = 0.98, P = 0.32]. The interaction of reach style and task actions strongly influenced the subgroup studied in the visual block trials [F(7,96) = 9.99, P < 0.0001]. However, interaction of visual guidance and actions performed during specific task stages was not significant [F(7,96) = 1.79, P = 0.086].

These results suggest that visual guidance improves the proficiency of object acquisition, but is not crucial to task completion. We conclude that visual information aids precise and smooth hand–object interactions during acquisition in a trained task, but has weak or no effect on firing rates of area 5 neurons during prehension.

DISCUSSION

In this series of studies, we used a prehension task to analyze factors influencing the firing rates of hand-manipulation neurons in posterior parietal cortex. The task integrated components of trained behavioral paradigms with natural hand movements used by monkeys to grasp and manipulate objects. The data presented in this report indicate that the factors controlled by the animal—how and when he used the hand—played the principal roles in modulating firing rates of neurons recorded in rostral area 5. Two thirds of the total variance in the population responses of Fig. 7 was attributed to the progression of hand actions from stage to stage and, presumably, to the underlying muscle activation patterns. The approach style used to bring the hand to the object contributed another 31% of the total variance. Forward or lateral approach trials—those that combined reaching, wrist rotation, and hand preshaping prior to contact—evoked the most vigorous responses in 80% of the population studied. The high firing rates that began during approach carried over into the subsequent contact and grasp stages and were associated with fluid movement of the hand as the object was acquired and lifted. Moreover, spike trains evoked during these reach trials were longer in duration than those in the no-reach condition in which the animal simply grasped an object in the immediate environment (local approach) or lifted objects repeatedly without releasing the grasp (regrasps). In this manner, actions of the hand and arm before contact enhanced the neural responses in subsequent stages.

The high firing rates observed during reach trials were only weakly modulated by the direction of reach, the location of the designated object in the workspace, or its intrinsic features (size, shape, or weight). Although half of the individual cells analyzed (38/75) showed object selectivity in their spike trains, particularly in the contact and grasp stages, the disparity in firing rates for each object between reach trials and no-reach trials was far greater than the distinctions between different objects grasped with the same approach style. The reach trajectory had relatively minor effects on firing rates because 87% of the neurons tested (65/75) did not distinguish forward and lateral reaches. However, objects placed at or medial to the midline evoked the strongest responses in two thids of the cells studied, whereas only 9% of neurons responded at maximum rates to objects placed lateral to the active shoulder.

Taken together, these findings suggest that the firing rates of hand manipulation neurons in rostral area 5, like those in area AIP, are modulated more vigorously by actions of the hand and arm than by object features. The neural spike trains seem to reflect the time course of hand actions regardless of where it is aimed or the precise shape of the object grasped, as long as these actions achieve a common goal.

Does the sensitivity to reaching reflect actions of the arm or the hand?

The principal finding of our study was that grasping a particular object in trials with or without an accompanying reach produced larger differences in firing rates during acquisition than the distinctions between different object shapes or workspace locations. The context in which an object was acquired clearly modulated the responses to grasping because firing rates were higher when preceded by a reach than when the hand was simply slid over the object surface. The question then arises whether the enhanced neural activity reflects actions of the arm, wrist, or hand during reaching?

When a monkey or a person reaches to grasp an object, several coordinated actions occur along the forelimb. The arm and hand are transported in a smooth path toward the object. During the reach, the wrist is rotated to best orient the hand for grasping and the fingers are opened and preshaped to clear the object and enable efficient contact. This requires coordination of many different muscle groups in the limb, starting proximally and progressing distally over time.

Although our data provide clear evidence for synergistic effects of reaching and grasping in the prehension task, we cannot rule out the possibility that these responses reflect actions of both the arm and hand. We propose that neural activity during the approach stage and the subsequent enhancement of contact and grasp responses may reflect the coordination of the arm and hand, rather than the reach per se. There are anatomical and physiological reasons for postulating indirect effects of reaching on the responses of hand manipulation neurons described herein.

There is considerable evidence from the literature that reaching is strongly represented by neurons in the superior parietal lobule (SPL) that have been collectively named the “parietal reach region” (PRR) (Andersen and Buneo 2002; Andersen et al. 1997; Batista and Andersen 2001; Battaglia-Mayer et al. 2000, 2003; Buneo and Andersen 2006; Caminiti et al. 1998; Crammond and Kalaska 1989; Cui and Andersen 2007; Fattori et al. 2004; Ferraina et al. 2001; Fogassi and Luppino 2005; Galletti et al. 1993, 1997, 2003; Hamel-Pâquet et al. 2006; Kalaska and Crammond 1995; Kalaska et al. 1983, 1990, 1997; Lacquiniti et al. 1995; Pesaran et al. 2008). PRR neurons are postulated to play major roles in sensorimotor transformations in which visual, proprioceptive, and tactile signals are combined with central motor commands to generate a plan of action directing the arm toward particular targets in space and subsequently monitoring its execution.

Anatomically, the PRR comprises the medial regions of area 5, including the medial intraparietal area (MIP), the caudal portion of the superior parietal gyrus (area PEc) posterior to the postcentral dimple and extending onto the medial wall of the hemisphere, the parieto-occipital area (called area PO or V6/V6A), and cortex of the medial wall ventral and caudal to the cingulate sulcus (called area 7m or PGm).

Our recording chambers were not located over PRR, but were situated more laterally and rostrally over the SPL (see Fig. 3 in Gardner et al. 2007c). The electrode tracks in area 5 entered the cortex near the anterior end of the IPS, between the hand area of S-I rostrally and area AIP and 7b caudally; physiological responses of neurons in these neighboring areas were described previously (Gardner et al. 2007a,b). Nearly all of the recording sites in which passive somatosensory responses were tested had receptive fields located on the hand, not the arm or shoulder. Furthermore, many of the neurons in our study were recorded in the medial bank of the IPS rostral to area MIP (area PEip); this region projects to the same portion of the ventral premotor cortex (area PMvr or F5) as neurons in area AIP (Borra et al. 2008; Petrides and Pandya 1984; Tanné-Gariépy et al. 2002). Proprioceptive information from the hand is also transmitted from rostral area 5 to area AIP and neighboring regions of area 7b on the convexity of the inferior parietal lobule (IPL) (Neal et al. 1986). The anatomical projection targets of our recording sites are therefore directed primarily to cortical areas devoted to grasping, not reaching (Fogassi and Luppino 2005; Rizzolatti and Luppino 2001).

The physiological responses of hand manipulation neurons in our study also differed in time course and tuning properties from those described for reach-related neurons in PRR. Neurons recorded in medial area 5 displayed strong directional tuning that typically spanned about one third of the movement space and were maintained throughout the trial; reaches in the nonpreferred direction evoked weak or no responses at all (Crammond and Kalaska 1989; Hamel-Pâquet et al. 2006; Kalaska et al. 1983, 1990, 1997). Forty-two percent of reach-related neurons in those studies showed phasic responses during the movement stage, but ceased firing at or before attainment of the target by the hand; another 30% were described as tonic, firing at constant rates throughout the movement and hold periods.

In contrast, hand manipulation neurons in our study were not strongly directionally tuned. They did not fire exclusively during the reach period, nor did they show stepwise increases in tonic firing during the reach, grasp, lift, and hold periods. Instead, we found that firing rates rose steadily during reach and peaked at contact or during grasp; neural activity ceased or returned to baseline when the hold stage began. Moreover, hand manipulation neurons in area 5 responded on trials when there was no reach (local approach or regrasp), albeit at lower rates than those during forward or lateral reach trials. The actions that transpired as the hand was transported to the selected object enhanced the subsequent responses to grasping, but were not mandatory for activating these neurons.

The neural responses recorded during local approach trials in our study are similar to those reported by Salimi et al. (1990a,b) for dynamic cells in areas 5 and 7b. In a precision-grip lifting task, they found that peak firing occurred at the onset of grip force application when the fingers first contacted the manipulandum. Neural firing rates in the Salimi et al. reports did not track the grip force amplitude and showed weak, nonmonotonic effects of object weight on peak firing rates. Their findings and ours are consistent with the notion first proposed by Kalaska and colleagues (1990) that area 5 neurons encode the invariant spatial parameters of hand and arm movements (kinematics) and are only weakly affected by the load moved (movement dynamics).

The coordination of reach and grasp in our task may be implemented by anatomical connections between the hand and arm representations in area 5. Horizontal connections linking the caudal and rostral subregions of SPL have been documented by Pandya and Selzer (1982), Selzer and Pandya (1980), and Lewis and Van Essen (2000b). In our laboratory, microinjection of Fast blue dye at the crown of the SPL in monkey H17094 confirmed anatomical projections between physiologically identified hand manipulation neurons and more medial regions of area 5 (unpublished observations).

Our findings suggest that the concept of motor schemas originally proposed by Arbib and colleagues (Hoff and Arbib 1993; Jeannerod et al. 1995) may be somewhat oversimplified. They proposed that complex motor sequences are built from simple behaviors governed by specific groups of muscles (motor schemas). Our data suggest that successive actions during complex motor sequences are not simply conjoined, but instead mutually reinforce each other.

In a recent study of grasping, Fogassi and colleagues (2005) demonstrated that neurons recorded on the convexity of the inferior parietal lobule (IPL) encoded the same motor act differently depending on the final action goal: grasp to eat or grasp to place. They noted “the fluidity with which the different motor acts follow one another,” proposing a “kinetic melody” composed of “prewired intentional chains in which each motor act is facilitated by the previously executed one.” Our data support and extend these ideas. The strongest responses occurred on trials that required the animal to reach, select appropriate grasp sites prior to contact, orient, and preshape the hand, grasp the object, and lift it. Although we have subdivided these actions into stages, the fluidity of hand movement from one stage to the next is reflected in the continuous, relatively uniform spike train that spans the approach through lift actions. The cascading of the spike train through the entire series of actions produces stronger responses to grasp and lift when they are preceded by a reach and period of hand preshaping. These findings suggest that the complex visuomotor and somatosensory actions preceding contact reinforce the subsequent neural activity accompanying grasp and lift, allowing the animal to complete the task in a continuous, smooth sequence.

It might be argued that the distinction between reach and no-reach trials might be attributed to target switching. However, when we regrouped the data into trials in which the sequence of knobs tested was repeated or not, we found that “switch” trials yielded responses nearly identical to those of reach trials. The “stay” trials included repeated forward reaches to the knob cued on the previous trial. They evoked stronger mean firing rates than no-reach trials and reduced the number of neurons showing significant effects of kinematic style from 72 to 63%. We believe that classification of responses by the actions of the arm and hand is preferable to schemas based on the inferred mental states required to plan acquisition of novel objects.

Our measurements of hand kinematics with digital video were limited by the viewing angle of the DV camera and its 33.3-ms frame rate. This monitoring system did not provide precise signals of joint angle or the grip aperture as in studies using cybergloves or integrated motion analysis systems (Jeannerod et al. 1995; Mason et al. 2001, 2004; Roy et al. 2000, 2002, 2006; Santello et al. 1998, 2002; Thakur et al. 2008; Theverapperuma et al. 2006; Winges et al. 2003). However, most of these detailed kinematic studies did not include simultaneous neural recordings. Studies of cortical responses to prehension in monkeys used video techniques similar to ours to monitor hand postures during task performance (Fattori et al. 2004, 2009; Galletti et al. 2003; Murata et al. 1996, 1997, 2000; Raos et al. 2004, 2006; Umiltà et al. 2007), but did not distinguish subcomponents of the “movement” stage, or subdivided it into quartiles that approximated the approach, contact, grasp, and pull actions performed by the animal. Although we recognize that the brief duration of the grasp stage (one to two video frames) may provide a less than ideal interval for measuring mean firing rates, we believe that it is important to distinguish hand–object interactions in which the hand moved or was stationary on the object. The fact that responses in rostral area 5 typically declined during the grasp and lift stages suggests that tactile inputs from rapidly adapting Meissner's afferents during tangential hand motion on objects provide important contributions to the activity measured in these neurons (Gentilucci et al. 1997; Jenmalm et al. 2000; Monzée et al. 2003; Witney et al. 2004).

Finally, we note that to compare regrasp trials to the other types, we designated the three video frames preceding lift as the approach, contact, and grasp stages. Although this binning procedure may seem somewhat arbitrary, it reflected the parcellation by the 33.3-ms frame rate of all seven task stages in individual trials; one video frame was the minimum time increment allowed. We feel that this procedure provides an appropriate basis for comparing responses on trials with and without an accompanying reach. The rasters and spike density plots show that the duration of the spike train reflected the time period in which the animal moved his hand, whether during reaching, positioning the hand on the object, altering grip and load forces, and lifting the knob. Once these actions were completed, the spike train dropped back to baseline or firing was inhibited. Mean firing rates were not uniform during these stages, but reflected both the current and preceding hand actions.

We also note that the detailed firing patterns within individual spike trains reflected the same characteristics as the average rates determined by our binning protocols. In an independent analysis of data from this study, Goldberg et al. (2009) used the metric space technique (Victor 2005) to analyze spike trains from a sampling of neurons illustrated in this report. They measured how much information about the task kinematics and the properties of the grasped object was conveyed by the internal temporal structure of spike trains aligned to contact and whether partitioning the spike trains into categories determined by the clustering procedure matched the parameters of the experimental trials. They found that the greatest information about kinematics was conveyed in the 200-ms interval prior to contact. For example, the metric space analyses showed that the neuron in Figs. 3 and 9A did not convey information about the knob shape in its spike train, but transmitted significantly more information about approach style (0.7 bit) during sighted trials than during blocked conditions (0.4 bit). Likewise, the neuron in Fig. 9B conveyed 0.4 bit of information about the approach style, but not the knob identity nor visual guidance. Shuffling of the spike time stamps reduced the information about trial parameters. These conclusions about selectivity for reach style, object parameters, and visual guidance match our findings derived from statistical analyses of averaged responses, while eliminating the need for binning.

Visual guidance of prehension

The overall effect of visual guidance on firing rates in our studies was less robust in area 5 than that described by Murata et al. (1996, 2000) for area AIP. Visuomotor cells in their studies fired at higher rates during task performance in the light than in darkness and visual dominant neurons failed to respond on trials conducted in the dark. Neurons in our population were not tested in total darkness. Instead, environmental light levels were maintained throughout the period of study, but direct views of the objects and the hands were blocked with an opaque panel. None of the neurons we studied in rostral area 5 failed to respond to grasping during visual block trials. Moreover, only 12 of 25 neurons tested in our study showed significant changes in firing rates between the sighted and blocked conditions and these were evenly divided between the two conditions. Spike trains evoked during approach overlapped during sighted and blocked trials, reflecting the common duration of reaching, but diverged as the hand contacted, grasped, and lifted the object. Firing rates were slightly higher on sighted trials that were performed skillfully and smoothly.

The principal effect of visual guidance on task performance occurred at the onset of hand–object interaction. Visual block trials were characterized by tentative acquisition kinematics, clumsy grasps, missed grasps, erroneous object selection, and reaches beyond the most medial object location in the workspace. When the animal directed his hand toward an incorrect object, the tactile signal of an apparently unexpected shape at contact caused him to abort the trial and redirect the reach elsewhere. In this manner, the neural responses appeared to reflect both the animal's predictions during approach and the tactile information gleaned when the hand initiated the grasping actions (Jenmalm and Johansson 1997; Schettino et al. 2003; Winges et al. 2003).

We cannot rule out contributions from visual stimuli or eye movements in this task because we did not require the animal to fixate the objects or the on-screen task cues, nor did we explicitly monitor gaze or eye movements beyond the images of the animal's face and eyes in the DV clips. The fact that the visual block experiments yielded only minor contributions of vision to the task suggests that the principal sensory events in area 5 are derived from somatosensory receptors in the hand or arm and from postulated efference copy of motor signals transmitted from the frontal lobe (Kalaska et al. 1997; Mountcastle 1995; Mountcastle et al. 1975). Likewise, the absence of overt neural responses to the cues or gaze-related responses when the animal looked at the objects, but did not reach, suggests a weak role at best for visual modulation of neural activity in rostral area 5.

The greater skill required for hand placement on objects following reach appears to strengthen neural responses to tactile feedback in the contact stage as the hand is positioned for grasp. It was in precisely these same intervals in which hand manipulation neurons in area 5 were most likely to distinguish features of individual knobs (Gardner et al. 2007c). These findings suggest that somatosensory input from the hand following contact provides powerful signals of object properties and grasp actions. Sensory feedback from the hand signals the accomplishment of grasping and manipulatory actions when successful and is used for error correction when they fail to achieve the desired goals. Monkeys and humans are thus able to adapt their hand behavior to accomplish the task goals.

GRANTS

This work was supported by National Institute of Neurological Disorders and Stroke (NINDS) Grants R01 NS-011862 and R56 NS-011862 and Human Brain Project Research Grant R01 NS-044820, funded jointly by NINDS, National Institute of Mental Health, and National Institute on Aging.