Integration of Visuospatial and Effector Information during Symbolically Cued Limb Movements in Monkey Lateral Intraparietal Area

Abstract

Natural behavior requires close but flexible coordination between attention, defined as selection for perception, and action. In recent years a distributed network including the lateral intraparietal area (LIP) has been implicated in visuospatial selection for attention and rapid eye movements (saccades), but the relation between the attentional and motor functions of this area remains unclear. Here we tested LIP neurons in a task that involved not an ocular but a manual operant response. Monkeys viewed a display containing one cue and several distractors and reported the orientation of the cue (right- or left-facing) by releasing one of two bars grasped, respectively, with the right or left hand. The movement in this task thus was associated with (cued by), but not directed toward, the visual stimulus. A large majority of neurons responded more when the cue rather than when a distractor was in their receptive field, suggesting that they contribute to the attentional selection of the cue. A fraction of these neurons also was modulated by limb release, thus simultaneously encoding cue location and the active limb. The results suggest that the LIP links behaviorally relevant visual information with motor variables relevant for solving a task in a wide range of circumstances involving goal-directed or symbolically cued movements and eye as well as limb movements. A central function of the LIP may be to coordinate visual and motor selection during a wide variety of behaviors.

Introduction

The posterior parietal cortex long has been implicated in the control of attention (Mesulam, 1999), and in recent years a specific subdivision of the parietal lobe, the lateral intraparietal area (LIP), has been investigated intensively in this regard (Goldberg et al., 2002). LIP neurons have visual responses with circumscribed receptive fields (RFs) and respond preferentially for physically conspicuous or behaviorally relevant objects (Gottlieb et al., 1998, 2004). LIP activity correlates with psychophysically measured attention (Bisley and Goldberg, 2003), and reversible inactivation of this area produces deficits in target selection during visual search (Wardak et al., 2002, 2004). The LIP thus is proposed to encode a salient representation of the environment, which specifies the momentary locus of attention (Gottlieb et al., 1998; Goldberg et al., 2002). Two areas with which the LIP is interconnected strongly, the frontal eye field (FEF) and the superior colliculus (SC), are thought to play similar roles in attention (Cavanaugh and Wurtz, 2004; Thompson and Bichot, 2005).

In addition to its role in attention, the LIP, like the FEF and SC, has been implicated in the planning of saccadic eye movements (Snyder et al., 2000b). This strong relationship with saccades has generated a long-standing debate as to whether LIP indeed has a general contribution to attention, defined as a modulation of perception independent of motor output, or whether it has a specific oculomotor function. In support of an oculomotor interpretation, some studies have shown stronger LIP responses in tasks involving saccadic rather than limb operant responses (Snyder et al., 1997; Quian Quiroga et al., 2006).

To shed light on this question, we recorded LIP activity in a novel task involving an arbitrary association between a visual stimulus and a nontargeting manual report. Monkeys viewed a display containing one cue and several distractors and, while maintaining their gaze straight ahead, reported the orientation of the cue (right- or left-facing) by releasing their grasp with, respectively, the right or left hand. The task differed from those typically used in the parietal cortex in that it required a manual report that was not spatially related to the locus of the visual discriminandum. Tasks using “arbitrary” or symbolically cued motor responses have been used in studies of the premotor cortex and basal ganglia, but not in those of the parietal cortex (Wise and Murray, 2000). Neurons responded much more strongly if the cue rather than if a distractor was in the RF, consistent with the idea that they contributed to the attentional selection of the cue. In addition, in a fraction of neurons the responses to the cue were modulated strongly by choice of limb. The results suggest that the LIP links sensory and motor variables relevant for solving a task in a variety of behaviors, including goal-directed and symbolically cued movements and eye as well as limb movements. Thus a specific function of the LIP may be to coordinate attentional and motor selection in a wide range of behaviors.

Materials and Methods

General methods and behavioral tasks.

Data were collected with standard behavioral and neurophysiological techniques, using the Tempo software (Reflective Computing, St. Louis, MO) for behavioral control and stimulus presentation, the scleral search coil (DNI, Newark, DE) for eye position recording, the APM digital processing module (FHC, Bowdoinham, ME) for neural signal recording, and MatLab (MathWorks, Natick, MA) for off-line data analysis. All methods were approved by the Animal Care and Use Committees of Columbia University and New York State Psychiatric Institute as complying with the guidelines within the Public Health Service Guide for the Care and Use of Laboratory Animals. During experimental sessions the monkeys sat in a primate chair that enclosed their bodies from the neck down while their heads were fixed in the straight-ahead position. Visual stimuli were presented on a Sony (Tokyo, Japan) GDM-FW9000 Trinitron monitor (30.8 × 48.2 cm viewing area) located 57 cm in front of the monkeys’ eyes. Structural magnetic resonance imaging (MRI) was used to verify that electrode tracks coursed through the lateral bank of the intraparietal sulcus.

Isolation and receptive field mapping: memory saccade task.

Neurons were recorded in the right and left hemispheres in monkeys 1 and 3 and in the right hemisphere in monkey 2. Each neuron was isolated and characterized with the memory saccade task on which, after the monkey fixated on a central fixation point, a small annulus (2° diameter) was flashed for 100 ms at a peripheral location. After a delay of 800–1000 ms the fixation point was extinguished, and the monkey was rewarded for making a saccade to the remembered location of the annulus. The location eliciting the best response was determined manually first, and responses then were examined at 8–12 locations uniformly distributed around the fixation point at a constant eccentricity. Neurons were tested additionally only if they had significant spatial selectivity in the visual (50–250 ms after target onset), delay (400–900 ms after target onset), or presaccadic epoch (200 ms before saccade onset) on the memory saccade task as determined by a one-way Kruskal–Wallis ANOVA (p < 0.05). Each location tested during the search task was designated as falling within the RF if it was associated with either excitatory or inhibitory responses relative to pretarget activity (200 ms before target onset) in any epoch of the memory saccade task.

Covert search task.

For the search task an array of four figure-eight placeholders was present throughout the intertrial interval (see Fig. 1a). Individual stimuli were scaled with retinal eccentricity and ranged from 1.5 to 3.0° in height and 1.0–2.0° in width. To begin a trial, the monkeys fixated on a central fixation spot (presented anew on each trial) and grabbed two response bars. Two line elements then were removed from each placeholder, yielding a display with one cue (a right- or left-facing letter “E”) and several unique distractors. Monkeys were rewarded for reporting cue orientation by releasing the right bar for a right-facing cue (bottom right panel) or the left bar for a left-facing cue (top right panel) within 100–1000 ms of the display change. While holding the bars, the monkeys completed an electrical circuit, which was broken at bar release, providing instantaneous TTL (transistor–transistor logic) signals of bar grasp and release. A correct response was rewarded with a drop of juice, after which the fixation point was removed and the placeholder display was restored. Fixation was enforced continuously to within 1.5–2° of the fixation point, and trials were aborted without reward if the eye exited this fixation window.

Figure 1.

Behavioral task. a, Each panel illustrates the visual display at different stages of the task. Four placeholders remained stably on the screen throughout the intertrial interval (left). To begin a trial, the monkeys fixated on a central spot and grabbed two bars at stomach level (center). At ∼500 ms after fixation achievement, two line segments were removed from each placeholder, revealing one cue, the letter E, and three distractors (right). Unpredictably and with uniform probabilities, the E appeared at any display location and could be forward-facing (bottom right) or backward-facing (top right). Monkeys received a juice reward if they indicated the orientation of the E by releasing the right hand if the E was forward-facing or the left hand if it was backward-facing. Trials ended with a reward (if correct), extinction of the fixation point, and restoration of the placeholder display. b, Average population activity (81 neurons in monkeys 1 and 3) on the memory saccade task. The thick trace represents activity associated with saccades toward the RF center. The thin trace is the averaged activity for the other three locations tested with the search task for each neuron (90, 180, and −90° from the RF center). Responses are aligned on cue onset (left) and saccade beginning (right).

Data analysis.

Unless it otherwise has been stated, firing rates were measured from the raw spike times in the 200 ms epoch preceding bar release. Statistical tests were based on the Wilcoxon rank test or paired rank test or on nonparametric ANOVA, evaluated at p = 0.05. For population analyses the average firing rates were calculated for each neuron, and the distributions of average firing rates were compared. Only trials with a manual release within the proscribed latency limits are included in this report. Trials that were erroneous because of fixation breaks or early or late bar releases constituted <10% of trials and were discarded from the analysis.

To test for possible drifts of eye position, we submitted average eye position during the reaction time in correct trials to a two-way nonparametric ANOVA with cue location and cue orientation as factors (Friedman test). Significant effects were found in only 3 of 100 sessions, less than the fraction expected by chance. So that we could test for set size effects, monkeys 1 and 3 performed the task in blocks of trials with two, four, or six display elements. In monkey 1 the median reaction times were 411, 435, and 445 ms with accuracy of 98, 96, and 93%; in monkey 3 the median reaction times were 471, 492, and 501 ms with accuracy of 96, 88, and 82% (in all cases p < 10⁻³; Kruskal–Wallis ANOVA for set size effect). To test activity related to limb movement in the absence of visual stimulation as well as for the crossed hand condition, we used an array size of two elements in monkeys 1 and 3. In the former condition the absence of visual stimulation to the RF explains the low baseline firing rates relative to the standard condition. To calculate correlations between firing rates and reaction times, we first computed trial-by-trial firing rates and reaction times for each neuron and trial group (cue location and manual release) and pooled all of the data after normalizing each group by subtracting its mean. To test for a possible partial segregation of limb and saccade activity, we searched for negative correlations between limb receiver-operating characteristic (ROC) values and firing rates (after baseline subtraction) on the memory saccade task. We found no significant correlation between limb ROC and firing rates in the visual, delay, or presaccadic epochs of the saccade task. Average delay period activity for neurons with and without significant limb ROC was 31 versus 36 spikes/s (sp/s; p > 0.05 for all monkeys combined).

ROC indices (Green and Swets, 1968) were calculated for each neuron in the time windows indicated in the text. Spikes that occurred after bar release were excluded from the analysis. Confidence intervals were obtained by a permutation test with 1000 repetitions, and a value was deemed significant if its 95% confidence interval did not include 0.5. Note that ROC values for cue location and limb necessarily are related inversely. This is because pooling trials across one variable (cue location or limb) increases the firing rate variance related to other variable (limb or cue location). Because of this the ROC values for limb calculated at a single cue location are larger than those obtained after pooling across locations (see Figs. 6 vs 4). For the same reason monkey 3, who had stronger limb effects across the population, had somewhat lower selectivity for cue location than did monkeys 1 and 2 (see Figs. 4, 5).

Figure 6.

Limb selectivity as a function of cue location. a, Scatter plots of absolute limb ROC (absolute difference between limb ROC and 0.5) obtained for trials in which the cue was inside or opposite the RF. Filled symbols show neurons with significant selectivity for both cue and distractor in the RF. The diagonal line is the identity line. In monkeys 1–3 16, 6, and 22 neurons showed significant limb effects only when the cue was in the RF, and 5, 3, and 11 neurons showed limb selectivity at both cue locations. b, Same as a but for raw (not absolute) limb ROC. ROC values above and below 0.5 indicate, respectively, preference for the right and left limbs.

Figure 4.

Distribution of ROC indices for cue location (top) and limb (bottom) in the 200 ms before bar release. Filled triangles show the median index, with black denoting a significant difference from 0.5 (p < 0.05; permutation test). For cue location ROC the values above and below 0.5 indicate higher firing rates for, respectively, a cue or a distractor in the RF. For limb ROC the values above and below 0.5 indicate higher firing rates for the right and left limb.

Figure 5.

ROC indices for cue location and limb. Each color map shows the ROC values in one subject, indicating sensitivity for cue location (top) and limb (bottom). Each row represents one neuron, and each column represents a 10 ms time bin aligned on cue onset (left) and bar release (right). Within each panel, the neurons are sorted by time of onset of significant selectivity (gray dots) and by the direction of this selectivity, with nonselective neurons at the bottom, followed by those with ROC values below 0.5 and above 0.5. For clarity of presentation, the sorting order in the bottom left panel was reversed between neurons with opposite selectivity. Sorting was independent within each panel, and corresponding rows do not indicate the same neurons in the top and bottom panels. The few neurons with very early limb or location selectivity (< 50 ms) most likely reflect preexisting motor bias rather than true accumulation of information within a trial.

Differences among monkeys.

Data did not differ among monkeys with respect to the anatomical range of locations within the LIP that were sampled as determined by MRI. However, monkey 3 had lower delay and saccade-related activity than monkey 1. In monkeys 1, 2, and 3 median delay activity was 36, 30, and 27 sp/s, respectively (p < 0.01 for monkey 1 vs monkey 3; all other comparisons, not significant). Similarly, presaccadic activity was 46, 35, and 37 sp/s in monkeys 1–3. To see whether these differences may have accounted for differences in the strength of limb effects, we compared limb selectivity in monkeys 1 and 3 for a subset of neurons with equivalent delay and presaccadic responses (neurons with delay activity higher than the lowest value in monkey 1 and lower than the highest value in monkey 3). For this subset the median delay activity for monkey 3 versus monkey 1 was 30 versus 35 sp/s (p > 0.1; n = 26 and 46), and median presaccadic activity was 42 sp/s for both monkeys. Median absolute ROC values (absolute difference between the ROC index and 0.5) for hand selectivity were higher in monkey 3 than in monkey 1 for this subset (0.12 vs 0.08; p = 0.0166), replicating the differences in the entire sample. We also considered whether our results may have been attributable to the fact that we sampled slightly more neurons with RFs near the midline in monkey 3 than in monkey 1, and these neurons tended to have smaller hand effects. Thus we compared hand preference indices for the subset of neurons for which the receptive fields were centered >20° away from the vertical meridian (28 neurons in monkey 3 and 27 in monkey 1). In these neurons the normalized ROC values were 0.18 in monkey 3 and 0.08 in monkey 1 (p = 0.0084), again replicating those in the entire sample. Thus the differences in the strength and prevalence of the limb-related modulations could not be explained by sampling differences among subjects.

Results

Task and behavioral performance

Three monkeys performed a covert visual discrimination task in which they reported the orientation of a cue embedded in a distractor array (Fig. 1a). A stable visual display of several figure-eight placeholders remained on the screen in the intertrial interval. To begin each trial, the monkeys shifted their gaze to a central fixation point and grabbed two response bars (middle panel). After a brief delay the display changed to reveal a search array containing one cue, a letter E, and three unique distractors. The letter E could appear at any display location and could face forward or backward (to the right or to the left). Monkeys were rewarded for reporting cue orientation by releasing the right bar if the E was right-facing (bottom right panel) or the left bar if it was left-facing (top right). Trials were aborted without reward if central fixation was broken at any time before bar release. Thus the task required monkeys to find the visual cue by using peripheral attention and to indicate cue orientation with a nondirectional manual response.

All monkeys achieved a high level of performance, with the median percentage correct of 96% (monkey 1), 92% (monkey 2), and 88% (monkey 3) and reaction times of 435, 438, and 492 ms. Separate testing with display sizes of two and six elements showed that accuracy declined and reaction times increased with increasing numbers of distractors, suggesting that visual search remained effortful despite extensive practice (see Materials and Methods). Eye position during the reaction time was not influenced by cue location or responding limb, revealing no tendency for small ocular drifts related to task performance (see Materials and Methods). The hand movement was performed without direct visual guidance, because the bars were outside the monkey’s view during central fixation. Typically, monkeys released the bar for very brief periods (< 50 ms), quickly grasped it again in preparation for the next trial, and made no directed limb movements toward the cue or visual display. Success rates did not differ between right and left hand release in any subject.

Neuronal database

Electrode penetrations were directed to the lateral bank of the intraparietal sulcus, as confirmed by structural MRI. Neurons initially were identified by using the memory saccade task and were tested additionally only if they had significant spatially tuned activity on this task (see Materials and Methods). Of the 100 neurons described here (46 from monkey 1, 19 from monkey 2, and 35 from monkey 3) 95% showed significant tuning during the visual epoch, 97% during the delay epoch, and 99% during the presaccadic epoch of the memory saccade task (Kruskal–Wallis ANOVA; p < 0.05). Receptive fields were contralateral to the recording hemisphere in 80% of neurons, with a median eccentricity of 11.7° (range, 2.6–20.0°; see Materials and Methods). Figure 1b shows population activity on the memory saccade task with saccades directed toward the RF center (thick trace) and toward the other locations tested during the search task (thin traces). These responses unambiguously identify our neurons as belonging to the LIP, because in no other parietal area can one find this strength and prevalence of delay and presaccadic activity (Barash et al., 1991; Murata et al., 2000; Snyder et al., 2000a).

Visuospatial and limb motor responses during covert search

During covert search the placeholder array was scaled and rotated so that one placeholder fell in the center of the RF. Typically, the remaining placeholders were outside the RF (Fig. 1b). The majority of neurons encoded the location of the cue, responding strongly and selectively when the cue was in the RF (Fig. 2a). In contrast, neurons responded only weakly when a distractor was in the RF, although distractors were visually similar to the cue (Fig. 2a). Note that, because the search display was unmasked by a relatively subtle visual offset, neurons lacked the early transient response that is characteristic of the LIP for flashed stimuli (Fig. 1b) but developed cue-selective responses gradually during the reaction time. This selectivity for the location of the task-relevant cue is consistent with previous work showing that LIP neurons reliably encode the location of behaviorally relevant objects in multi-stimulus displays (Gottlieb et al., 2004).

Figure 2.

Representative neurons. a, Neuron selective only for cue location. Neural activity is represented in raster plots and averaged spike density histograms from trials in which the cue was either in the center of the RF or at the opposite location and was facing either to the right (red) or to the left (blue). In the raster plots each line is one trial; each tick represents the time of one action potential relative to cue presentation. Spike density histograms were derived by convolving individual spike times with a Gaussian kernel with SD of 15 ms. Activity is aligned on cue presentation (time 0), and black dots represent the bar release. Only correct trials are shown, ordered off-line according to reaction time. Diagrams indicate the location of the cue, the RF (gray oval), and the limb released in each configuration. This neuron responded when the cue was in its RF (top right quadrant) regardless of manual release. b, Neuron selective for cue location and manual release. c, Neuron selective for left-hand release on both E and U search (trials in which the cue was in the RF). Neurons a and b were recorded in monkey 1 and neuron c in monkey 3.

In addition to their sensitivity to cue location, many neurons were also sensitive to the type of the cue. An example is shown in Figure 2b. This neuron responded strongly when the cue was in the RF and was left-facing, prompting release of the left hand, but responded much more weakly when the cue was in the RF and was right-facing, prompting right-hand release. Because in our task there was a fixed association between cue orientation and responding limb, sensitivity to cue type may have reflected either the orientation of the E or the planning of the associated limb movement. We therefore tested a subset of neurons (n = 16) during discrimination of a second set of cues: an upward- or downward-facing “U” mapped, respectively, to right and left bar release (see Materials and Methods). Of 12 neurons with significant modulation for both E and U cues, 11 showed preference for the same limb during E and U search. This fraction is much higher than would be expected by chance under the assumption of independently tuned shape-selective responses for the two cue sets (p < 10⁻¹⁰; χ² test; three neurons had no limb preference and one had preference on the E, but not U, search). A representative neuron preferring left-hand release for both E and U cues is shown in Figure 2c. The consistent preference for manual report argues that the modulation by cue type reflected selectivity for the manual response rather than for the shape associated with that response. Firing rates were slightly higher overall for the E relative to the U search (200 ms before bar release; 41 vs 31 sp/s; p < 0.05), with five neurons showing significantly higher responses during the E search. This suggests that, in addition to their sensitivity to the manual report, neurons also showed some selectivity for E-shaped cues, or, more likely, it was an effect of training, because monkeys had much more experience with E- than U-shaped cues.

The finding that LIP neurons are sensitive to a nondirectional manual response (which does not involve selecting a target in the RF) is unexpected, based on known properties of LIP neurons, and constitutes the major focus of this report. In the following text we present control experiments characterizing the limb effect, followed by quantitative analysis of the cue location and limb signals and the interactions between them.

Effector specificity

Because in the standard version of the task the monkeys held each arm straight in front of them on the corresponding side of the body, sensitivity to limb release may have represented either the active limb itself or the position of the limb relative to the body. To answer this question, we tested 17 neurons (11 in monkey 1 and 6 in monkey 3) in the standard version of the task as well as in blocks of trials in which the monkeys performed the task with arms crossed (Fig. 3). In the crossed condition the monkeys continued to release the right hand to indicate a right-facing cue and the left for a left-facing cue, but the two hands were on opposite sides of the body. All neurons that were tested had significant limb effects in the standard condition, and none changed limb preference on crossed limb trials. Population firing rates for the preferred limb (the limb associated with the higher firing rates in the standard condition) were, in the crossed versus uncrossed conditions, 70 versus 64 sp/s in monkey 1 and 56 versus 51 sp/s in monkey 3 (both p > 0.1). For the nonpreferred limb the firing rates also did not differ between crossed and uncrossed conditions (monkey 1, 44 vs 51 sp/s; monkey 3, 9 vs 11 sp/s; both p > 0.4). Thus neurons reflected the consistent mapping between a cue and a specific limb independently of limb position.

Figure 3.

Responses in the standard and crossed hand conditions. Population activity from trials in which the cue appeared in the RF is aligned on the time of the bar release for responses with the preferred (black) and nonpreferred (gray) limb. Testing was conducted with an array size of two elements in blocks of trials performed with hands in uncrossed (top) and crossed (bottom) positions.

Visual dependence of limb effects

To determine whether limb modulations were free-standing manual responses or whether they appeared only as modulations of a visual response (Fig. 2b), we tested whether neurons would respond to the grasp release in the absence of visual stimulation to the RF. We tested a subset of neurons (10 in monkey 1 and 15 in monkey 3) in blocks of trials in which the monkeys performed the search task with all stimuli (cue and distractors) outside the RF (see Materials and Methods). In this case the LIP neurons remained unresponsive. Median firing rates before release of the preferred hand were 4 sp/s relative to a baseline of 5 sp/s in each monkey (p > 0.5), compared with robust responses in the same neurons when the search cue was in the RF (monkey 1, 60 sp/s vs baseline of 18 sp/s; monkey 3, 53 vs 16 sp/s; both p < 10⁻⁴). Therefore, limb modulations were not free-standing motor responses but were expressed in the LIP only as modulation of a visual/attentional response.

Quantitative analysis of cue and manual selectivity

We used ROC analysis to measure the magnitude and reliability of the cue location and limb signals (Green and Swets, 1968). ROC indices close to 0 or 1 indicate that an ideal observer can decide reliably in favor of one or the other alternative (cue or distractor in the RF or right or left limb release) on the basis of the firing rate distributions associated with each. Indices close to 0.5 show no reliable discrimination. To calculate selectivity for cue location, we compared the distribution of firing rates associated with the cue (“signal”) with that associated with a distractor (“noise”) in the RF, pooling across limb release. In the 200 ms before bar release 87, 84, and 71% of the neurons in monkeys 1, 2, and 3 showed significant selectivity for cue location (permutation test; p < 0.05), with median population indices of 0.82, 0.68, and 0.71, respectively (all p < 10⁻¹² relative to 0.5) (Fig. 4, top row). To calculate ROC values for limb, we adopted the arbitrary convention of assigning the right limb to the signal and the left limb to the noise distribution so that ROC values above and below 0.5 indicate preference for right and left limb release, respectively. A substantial fraction of neurons showed significant limb preference in the 200 ms before bar release (Fig. 4, bottom) (43% in monkey 1, 42% in monkey 2, and 77% in monkey 3). Median ROC values were not significantly different from 0.5, indicating no bias across the sample for the right or left limb (0.54, 0.46, and 0.57 in monkeys 1–3; all p > 0.05 relative to 0.5). Thus the strongest signal in the LIP population was that of cue location, and a significant fraction of neurons also was modulated by limb release.

To estimate the time course of cue location and limb selectivity, we calculated ROC indices in 10 ms bins aligned on the time of cue onset and estimated the onset of significant selectivity as the start of the first four consecutive 10 ms bins with significant ROCs (Fig. 5) (see Materials and Methods). Cue location selectivity arose gradually across the neuronal population with median times of onset of 150, 160, and 160 ms after cue presentation (monkeys 1, 2, and 3, respectively). Limb selectivity arose with a similar time course and median times of onset of 200, 175, and 200 ms. The slight delay in the limb signal (which reached statistical significance in monkey 1 only) may have been explained by the smaller magnitude of limb relative to cue selectivity, because differences in time of onset were correlated negatively with differences in the magnitude of the two signals across the neuronal sample (r = −0.51; p = 0.0019; n = 100).

Nonlinear interactions

Examination of Figure 2b suggests that the signals of cue location and active limb interacted nonadditively, because limb effects appeared to be stronger when the cue rather than when a distractor was in the RF. Figure 6 shows a comparison of limb effects (limb ROC values before bar release) when the cue was inside versus opposite the RF. This comparison is restricted to neurons for which the opposite RF location was clearly outside the visual RF as determined by the lack of excitatory or inhibitory responses on the memory saccade task (∼70% of the sample). Comparison of absolute ROCs (absolute difference between limb ROC and 0.5) (Fig. 6a) shows that limb selectivity was much higher when the cue was in the RF, with median indices of 0.33 versus 0.12, 0.29 versus 009, and 0.49 versus 0.17 in monkeys 1, 2, and 3, respectively (each p < 10⁻⁴). We considered the possibility that the limb effect was a gain change, because this could have resulted in modulations proportional to search-related activity. However, there was no correlation in the gain of the limb effect (the ratio of firing rates associated with the two limbs) when the cue or a distractor was in the RF, either across the entire population (monkeys 1 and 2, r = 0.11 and p = 0.218; monkey 3, r = −0.18 and p > 0.1) or for the neurons with significant selectivity at both locations (r = 0.13, all monkeys combined). This suggests that neurons showed a more complex nonlinear interaction between spatial and effector selectivity. For neurons that showed statistically significant limb effects even when the cue was opposite the RF (black symbols; ∼30%), the preferred limb remained constant at both cue locations. This is shown in Figure 6b, which plots the raw limb ROC values, and demonstrates that all but one neuron selective at both locations (black) fell either in the top right or bottom left quadrants (indicating preference for the right or left limb). The fact that some neurons had limb effects even when the cue was outside the visual RF provides additional evidence against the hypothesis that these effects represent a shape-selective visual response. The fact that the preferred limb remained constant confirms that the neurons encoded a constant limb regardless of the location of the visual cue.

Cue–limb congruence and intersubject differences

Although limb effects were fundamentally similar in all three monkeys, monkey 3 differed from monkeys 1 and 2 in two respects. First, it showed a higher fraction of limb-selective neurons (Figs. 4, 5). In addition, limb preference in this monkey showed a stark bias toward the limb contralateral to the recording site. Although neurons in all monkeys tended to prefer the limb contralateral to the recording site, this bias was most pronounced in monkey 3 (59, 62, and 85% of neurons in monkeys 1, 2, and 3 preferred the contralateral limb, respectively). These differences could not be explained by differences in the neuronal population sampled in each subject as determined by the locations of recording sites, strength of activity on the memory saccade task, or location and size of the RF (see Materials and Methods).

The large bias for the contralateral limb in monkey 3 implies that in this subject the neurons were most responsive when the cue was in the RF, typically contralateral to the recording site, and the monkey released the contralateral limb, that is, when there was congruence between the hemifield of the cue and the active limb. Figure 7a shows the breakdown of trials according to cue–limb congruence. Congruent trials are those in which the cue was in the right hemifield and prompted right-hand release or was in the left hemifield and prompted left-hand release. Incongruent trials are those in which the cue was in the right hemifield but prompted left-hand release or was in the left hemifield, prompting right-hand release. Figure 7b shows the population response from trials in which the cue was inside or opposite the RF (solid vs dashed traces) and the manual response was either congruent or incongruent with the hemifield of the cue (black vs gray traces). In monkey 3 the population responses were very strong when the cue was in the RF and the monkey released the congruent limb (black solid traces) but were much weaker if the cue was in the RF and the monkey released the incongruent (typically the ipsilateral, nonpreferred limb; gray solid traces). This resulted in much stronger selectivity for cue location (the difference between cue- and distractor-evoked responses; solid vs dashed traces) on congruent than on incongruent trials. In contrast, in monkeys 1 and 2 cue-related responses were only slightly higher on congruent than on incongruent trials, reflecting the more balanced representation of the contralateral and ipsilateral limb and the higher fraction of neurons with selectivity for cue location, but not limb. ROC analysis confirmed these differences. In monkey 3 the population ROC values for cue location, although significant on both trial types, were much stronger on congruent than on incongruent trials (median ROC index of 0.73 vs 0.53) (both p < 0.05 relative to 0.5 and p < 10⁻⁴ relative to each other; 100–200 ms after cue onset). In monkeys 1 and 2 median ROC values were 0.69 and 0.66 on congruent versus incongruent trials (both p < 10⁻⁴ relative to 0.5 and p = 0.1358 relative to each other; 100–200 ms after cue onset).

Figure 7.

Congruence. a, Diagrams illustrating congruent and incongruent trials with cues in the right and left hemifield. b, Average population activity from trials in which the cue was either in the RF (solid) or at the opposite location (dashed) and the monkey released the limb congruent or incongruent with the hemifield of the cue (black vs gray). c, Average activity for neurons with significant limb effects from trials in which the cue was either in the RF (solid line) or at the opposite location (dashed line) and the monkey released the preferred or nonpreferred limb for each neuron (black vs gray).

Consistent with the neural responses in the LIP, monkey 3, but not monkeys 1 and 2, showed a strong behavioral effect of cue–limb congruence. Monkey 3 had median latencies of 423 versus 512 ms and accuracy of 98 versus 82% (both p < 10⁻⁴) on congruent versus incongruent trials. In contrast, in monkey 1 the median reaction times were 434 versus 439 ms, with accuracy of 98 versus 96%; in monkey 2 the corresponding values were 432 versus 440 ms, with accuracy of 93 versus 94% (all p > 0.05). To confirm the relationship between LIP activity and congruence, we additionally examined the crossed hand condition in monkey 3. Like the neural effects, behavioral congruence effects remained linked to the responding limb, with median latencies for the congruent and incongruent limbs of 429 versus 500 ms in the standard condition and 404 versus 485 ms in the crossed condition (both p < 10⁻⁴). [Because the crossed hand condition was tested with an array size of 2 (see Materials and Methods), accuracy was near ceiling on all trial types. Monkey 1 had no congruence effect in the crossed hand condition.] Thus in both intra-individual and inter-individual comparisons the congruence effects in LIP activity paralleled those in the monkeys’ performance.

It is important to note that the large individual differences in the congruence effect reflected only the contralateral bias in limb preference in monkey 3, and not fundamental differences in the limb phenomenon itself. As shown above (Figs. 3–6), limb effects were similar in all three monkeys with respect to their strength, temporal properties, dependence on visual stimulation, and independence of limb position. To reinforce this point, Figure 7c shows average responses of limb-selective neurons sorted according to cue location (solid vs dashed traces) and limb preference (black vs gray) regardless of whether the preferred limb was ipsilateral or contralateral. This sorting (which illustrates the limb effect itself independent of laterality and congruence) shows that limb selectivity of comparable magnitude and time course was found in all three monkeys, consistent with the quantitative analyses in Figures 4 and 5. The traces also show the weak limb preference when the cue was opposite the RF (dashed traces) (compare with Fig. 6). Note that, when grouped by congruence (Fig. 7b, dashed traces), this limb preference translates into slightly higher responses for incongruent than for congruent trials in monkey 3 (gray vs black dashed traces; 28 vs 24 sp/s; 200–300 ms after cue onset; p < 10⁻⁴). This reflects the fact that in monkey 3 the preferred limb was typically congruent with the hemifield of the RF and was thus incongruent with the cue when the cue was opposite the RF. These observations highlight two important points. First, fundamentally similar limb effects were found in all subjects. Second, LIP neurons did not encode congruence explicitly between cue location and limb but showed congruence effects only as an indirect consequence of the limb modulation and its bias toward the contralateral limb.

Relationship with reaction time

The combined attentional/motor influences raise the question of whether neural activity was related more strongly to cue selection or to motor planning. To explore this question, we examined correlations between firing rates and behavioral reaction times for different combinations of cue location and limb (Fig. 8). For each comparison we divided trials into those with short and long reaction times based on the median (Hanes and Schall, 1996) and compared the activity associated with each group. We also calculated correlations between firing rates and reaction times in individual trials, pooling data across neurons (after normalizing the within-neuron data by subtracting the mean).

Figure 8.

Neural activity and reaction time. a, Average population response from trials in which the cue was in the RF, the monkey released the preferred limb of the neurons, and reaction times were either faster (thick) or slower (thin) than the median. b, Same as a, but for release of the nonpreferred limb. c, Average population response trials in which a distractor was in the RF and the animal released the preferred limb with fast (thick) or slow (thin) reaction times.

If the LIP contributed to the attentional selection of the cue, we might expect neural responses associated with the cue in the RF to be correlated with reaction time regardless of the responding limb. This was indeed the case for monkeys 1 and 2, who had significant correlations between firing rates and reaction time if the cue was in the RF whether the monkeys released the preferred or nonpreferred limb (Fig. 8a vs b) [for the interval of 100–200 ms after cue onset; monkey 1, r = −0.19 and −0.25 (for the preferred and nonpreferred limb) and monkey 2, r = −0.16 and −0.20; all p < 10⁻³]. In monkey 3 the correlations reached statistical significance for release of the preferred, but not the nonpreferred, limb (r = −0.16 and p = 0.0235; r = −0.10 and p = 0.0565, respectively). This reflects the weak cue-related response in this monkey during release of the nonpreferred limb (compare with Fig. 7b). If LIP neurons contributed to the motor response, we might expect correlations between LIP activity and reaction time whenever monkeys released the preferred limb, regardless of cue location. However, although activity correlated with the reaction time when the cue was in the RF (Fig. 8a), there was no such correlation when a distractor was in the RF (Fig. 8c) (monkey 1, r = 0.063; monkey 2, r = 0.058; monkey 3, r = 0.013; all p > 0.1). The results suggest that LIP activity is correlated more strongly with the selection of the behaviorally relevant cue than with driving the limb response, although its contributions to spatial selection can be strongly dependent on the active limb.

Discussion

A large number of studies have investigated the contribution of the parietal lobe to goal-directed movements, such as saccades or reach movements, which target specific visual stimuli. In contrast, the behavioral paradigm we adopted in this study required an arbitrary, nonspatial report of a stimulus attribute: monkeys reported the orientation of a visual cue with a nontargeting grasp release. Such symbolically cued acts, or “arbitrary visuomotor associations,” have been shown to depend on a distributed network including the premotor cortex and basal ganglia (Wise and Murray, 2000). We found that LIP neurons also responded robustly in this task, simultaneously encoding the location of the cue and the active limb. This suggests that parietal association areas are instrumental for linking relevant visuospatial and motor variables, not only for goal-directed but also for symbolically cued behaviors (Toth and Assad, 2002; Grol et al., 2006).

Selectivity for cue location was the primary and most prevalent signal in the LIP, because strong cue-evoked activity was found in the vast majority of neurons and correlated with behavioral reaction time mainly independently of the active limb. This selectivity resembles the LIP response before visually guided saccades and confirms the idea that the LIP contributes to attentional selection regardless of the modality of the behavioral report (Goldberg et al., 2002). However, this visuospatial signal was sensitive not only to visual but also to motor variables; the signal encoding cue location was modulated by the limb reporting the orientation of the cue. Because limb effects arose only as modulations of a visual response (which described a cue that was not the motor target), they are not easily interpretable as driving the limb movement per se. Thus our data suggest that, although LIP is not critical for driving a limb movement (Snyder et al., 1997; Quian Quiroga et al., 2006), it receives feedback about a limb motor plan, which can modulate its visuospatial activity. This motor feedback may have originated in adjacent limb-related areas including the parietal reach area (Calton et al., 2002) and the anterior intraparietal area, which contains neurons with limb-specific activity and has been implicated in hand manipulation movements (Kermadi et al., 2000; Lewis and Van Essen, 2000; Nakamura et al., 2001).

Limb selectivity

In a previous experiment, Snyder et al. (1997) showed that a minority (∼20%) of LIP neurons had sustained, delay period activity preceding an arm reach movement. In contrast with this reach-related activity, the responses we report encoded the effector (limb) itself and not the spatial goal of the movement. Dickinson et al. (2003) reported that LIP neurons had no limb-specific activity in a task requiring choice between an ocular and a manual response. However, these investigators used a task in which monkeys first were notified of the effector to use and then were shown the spatial goal of the movement. The lack of limb effects in their study is consistent with our finding that limb selectivity is not expressed in the absence of visual stimulation to the RF. Our results show that limb-specific information does reach the LIP, but its expression is gated strongly by a visual/attentional response.

We ruled out several potential confounds for the limb phenomenon. The robust saccade-related activity and location of recording sites rule out the possibility that we inadvertently recorded from adjacent limb-related areas, because these have very little presaccadic activity (Barash et al., 1991; Murata et al., 2000; Snyder et al., 2000a). Limb modulations remained consistent for two sets of cues and also were found when the search cue was outside the RF, showing that they were distinct from any shape selectivity that may have existed in our neurons (Sereno and Maunsell, 1998; Toth and Assad, 2002). Limb selectivity was maintained when the arms were crossed, ruling out the possibility that it represents a generalized spatial selection signal covering both cue and limb. The preferred limb remained constant regardless of cue location, ruling out the possibility that neurons explicitly encoded the congruence between cue location and active limb. Finally, limb modulations could not be explained by differential reward rates, which were equivalent for both limbs in all subjects.

A noteworthy feature of our data is that limb selectivity appeared early and shaped cue location responses throughout the reaction time (Fig. 5). This finding is consistent with parallel processing models in which information about visuospatial, perceptual, and motor variables accumulates in parallel during the reaction time (Roitman and Shadlen, 2002). Our data suggest that attention-related areas simultaneously interact with feature-selective visual areas (Itti and Koch, 2001; Navalpakkam and Itti, 2005), as well as with areas related to motor planning. Consequently, their output is a hybrid signal reflecting an interaction between visual and motor task demands. The task design we adopted likely has maximized such parallel processing because it did not separate visual analysis and motor planning temporally (i.e., did not include a delay period) and relied on a highly trained, stereotyped association between a visual stimulus and a motor response.

Congruence, attention, and motor planning

In all three subjects LIP neurons showed a bias for the contralateral limb, which, in conjunction with the mostly contralateral visual representation, resulted in highest activity for congruent cue–limb configurations when both cue and limb were contralateral to the recording hemisphere. This bias differed quantitatively among subjects, being very pronounced in monkey 3 but only slight in monkeys 1 and 2. Consistent with this, monkey 3 showed an idiosyncratic congruence effect in behavior, performing much better on congruent than on incongruent trials. Thus our results suggest that the hybrid attentional/motor signal in the LIP may be part of the substrate of the manual congruence effect previously described in humans as well as monkeys (Riehle et al., 1994, 1997; Requin and Riehle, 1995; Zhang et al., 1999). It is important to note, however, that our results do not clarify fully the root cause of the congruence effect. It is not clear from our data whether an initial (possibly anatomical) lateralization of limb input to the LIP was the primary cause of the behavioral effect or whether the behavioral effect originated elsewhere, giving rise to a behavioral strategy that indirectly shaped LIP activity. Thus more work is required to determine whether the LIP is part of the early causes of the behavioral congruence effect.

Whether or not this turns out to be the case, our data do show strong correlations between the LIP and congruence in the steady state. It is interesting to consider the implications of these neural congruence effects with respect to the role of the LIP in attention and motor planning. To the extent that the LIP contributes to attentional selection, our results suggest that attention may have been deployed more efficiently to the cue on congruent than on incongruent trials, at least in subjects with a large behavioral congruence effect. It is well known that the planning of goal-directed movements such as a reach or saccade facilitates the attentional selection of the motor target (Kowler et al., 1995; Deubel et al., 1998). Our results suggest that even a simple nontargeting hand movement may bias attention toward the active hand (Gilchrist et al., 2003; Eimer et al., 2005; Hannus et al., 2005). Moreover, because the neurons receiving limb efferent information also had saccade-related activity, LIP may also help direct gaze toward the active limb, thus potentially contributing to eye-hand coordination during nontargeting hand movements (Nanayakkara and Shadmehr, 2003). With respect to motor planning, although we have argued above that LIP neurons were not likely to have driven the limb movement per se, it remains possible that they may have had higher level, task-dependent contributions to motor selection. For example, neurons with a specific limb preference may have contributed to selection of the appropriate movement on trials in which the cue was in the RF. If so, the motor contributions of LIP also would be stronger on congruent than on incongruent trials (at least in subjects with strong congruence effects). Thus whether LIP is considered to contribute to attention or motor planning, its contributions depend jointly on visual and motor task demands.

Task dependence and relation to previous studies

The idea that the LIP is a highly associative visuomotor representation implies that its neural activity may have few stereotyped relationships with either sensory input or motor output, but it may depend strongly on the visuomotor links in each particular task. This consideration may help to understand our results in the context of previous experiments. In an earlier experiment Colby et al. (1996) reported only modest attentional enhancement in the LIP in a luminance detection task with a manual response. However, the discriminandum used by Colby et al. (1996) was a highly salient, flashed object that evoked strong responses in both attended and passive viewing conditions, whereas the cue in our task was much less salient because it was not flashed on and was surrounded by distractors. As a consequence, may have unmasked stronger top-down modulations. Thus an important determinant of LIP responses may be the relative weight of bottom-up and top-down attentional components. A second important parameter may be the consistent mapping of a visual stimulus onto a motor act. If, indeed, a key feature of the LIP is the extraction of relevant visual and motor variables, our use of a stereotyped, overlearned visuomanual association could explain the stronger limb effects we found relative to experiments that used goal-directed reach movements (Snyder et al., 1997; Quian Quiroga et al., 2006). The ways in which responses in association areas are shaped by task demands and training history will be fruitful venues for additional research.