Simultaneous Reconstruction of Continuous Hand Movements from Primary Motor and Posterior Parietal Cortex

Abstract

Primary motor cortex (MI) and parietal area PE both participate in cortical control of reaching actions, but few studies have been able to directly compare the form of kinematic encoding in the two areas simultaneously during hand tracking movements. To directly compare kinematic coding properties in these two areas under identical behavioral conditions, we recorded simultaneously from two chronically implanted multielectrode arrays in areas MI and PE (or areas 2/5) during performance of a continuous manual tracking task (CMTT). Monkeys manually pursued a continuously moving target that followed a series of straight-line movement segments, arranged in a sequence where the direction (but not length) of the upcoming segment varied unpredictably as each new segment appeared. Based on recordings from populations of MI (31–143 units) and PE (22–87 units), we compared hand position and velocity reconstructions based on linear filters. We successfully reconstructed hand position and velocity from area PE (mean r² = 0.751 for position reconstruction, r² = 0.614 for velocity), demonstrating trajectory reconstruction from each area. Combing these populations provided no reconstruction improvements, suggesting that kinematic representations in MI and PE encode overlapping hand movement information, rather than complementary or unique representations. These overlapping representations may reflect the areas’ common engagement in a sensorimotor feedback loop for error signals and movement goals, as required by a task with continuous, time-evolving demands and feedback. The similarity of information in both areas suggests that either area might provide a suitable target to obtain control signals for brain computer interface applications.

Introduction

Reaching movements involve the coordination across a broad cortical network involving multiple areas (Kalaska and Crammond 1992), but it remains unclear how these areas function together to coordinate time-evolving goal-directed behavior. Among the cortical areas involved in goal directed reaching, primary motor cortex (MI) and area 5 (including the area known as 5d or PE) in the posterior parietal cortex (PPC) play a key role in guiding and updating the direction of movements. Traditional views of MI consider it as a motor output area, containing neurons with firing rates correlated with muscle activations and forces, joint positions, movement kinematics, or complex movements (e.g. Georgopoulos et al. 1982; Schwartz 1993; Kakei et al. 1999; Sanes and Donoghue 2000; Graziano et al. 2002), though more recent studies have also accumulated evidence for MI involvement in motor learning and imagery (see Sanes and Donoghue 2000 for review), as well as action observation or rehearsal (Tkach et al. 2007; Dushanova and Donoghue 2010). Parietal-frontal networks comprise essential processing stages that help shape MI activity (Burnod et al. 1999; Buneo and Andersen 2006), with PE providing a major reciprocal projection to MI (Strick and Kim 1978; Johnson et al. 1993). Like MI, PPC areas (including area 5 and the parietal reach region) play a broad role that includes encoding movement kinematics (e.g. Kalaska et al. 1990; Mulliken et al. 2008a). This role encompasses pre-movement activity including movement planning, goal, and context (Kalaska et al. 1990; Gail and Andersen 2006; Quian Quiroga et al. 2006); mental simulation (Evangeliou et al. 2009); motor commands, efference copy, and proprioceptive feedback arising from passive motion (Seal et al. 1982; Mulliken et al. 2008b); and error correction during ongoing movements (Tunik et al. 2005). Overall, MI and PPC (including PE) share many functional characteristics, as both areas show a wide range of behavioral, sensory, and preparatory activity related to multiple aspects of motor control and observation. However, much remains unknown about how kinematic encoding in PE interacts with encoding in MI.

Since area 5 appears to involve sensory and pre-movement activity, a task requiring continuous feedback and control should provide the best way to study the interaction between areas PE and MI. We developed a continuous manual tracking task (CMTT), based loosely on a random pursuit tracking task (PTT; Paninski et al. 2004). CMTT entails a continuously moving target stimulus that moves through a consecutive series of straight lines, with each line approximating the natural characteristics of a linear point-to-point movement (Morasso 1981). Traditional point-to-point movement tasks (e.g. Georgopoulos et al. 1982; Buneo and Andersen 2006; Mulliken et al. 2008a) require sensory feedback only at the endpoint, but successful CMTT performance required constant contact with a moving target. Therefore, CMTT provides a superior method for investigating representations and reconstructions of hand trajectories, because it forces the subject to continuously control and optimize their hand position or velocity. Furthermore, manual tracking movements of externally fixed extent with task-relevant time-evolving characteristics appear in many natural situations, such as driving a car or printing letters with a pen.

Recording from neural units in one area at a time does not allow direct comparison between the two areas, because the two recordings cannot involve identical behavioral conditions, let alone match motivational or physiological variables. Despite that, relatively few studies have compared simultaneously recorded neural populations in multiple areas (Wessberg et al. 2000; Carmena et al. 2003; Carmena et al. 2005; Bansal et al. 2011). Of these, only one directly compared MI and PPC (Wessberg et al. 2000), but it only recorded from PPC in a single owl monkey; nevertheless, it suggested that MI and PPC may contribute similarly to reconstruction on a per-unit basis. Lee et al. (1998) compared MI and PE, and found similar neuronal noise properties in each area, but did not record from both areas simultaneously. To directly compare kinematic coding properties between MI and PE, we recorded simultaneously from two chronically implanted multielectrode arrays, one in each area. By recording from both areas during identical behavioral and physiological conditions, we sought to understand how and when position and velocity encoding differ in areas MI and PE.

In the present study, our primary goal was to determine whether and when neurons in MI or PE encode different information (or different amounts of information) about hand position and velocity. (Here we use “information” to refer to neural modulation correlated with a behavioral variable.) We asked two specific questions about hand movement representations in MI and PE. First, can PE population activity reconstruct ongoing movement trajectories? One previous study attempted such reconstruction (Mulliken et al. 2008a), but achieved only moderate success (average r² < .53) via a model that determined hand trajectory from both hand kinematics and goal-related parameters. We expected to reconstruct movements more accurately, despite simplifying our model to use only kinematic parameters, because we reconstructed trajectories using a continuous movement task instead of the point-to-point task used by Mulliken et al. (2008a). Second, do neurons in MI or PE encode different (or different amounts of) hand position or velocity information? If the two areas encode non-overlapping kinematic information, then the two areas might act separately or in series; e.g. if MI activity fully included the information in “upstream” PE, but not vice versa. If one area encodes more kinematic information than the other, that would necessarily entail the two areas encoding different information (as one area must encode information that the other does not). Alternatively, we expected that MI and PE could operate jointly in integration of sensory feedback, motor command, and efference copy. If so, both areas would encode the same movement-related information, and pooled neurons from MI and PE together would not provide more accurate reconstruction than either area alone. These studies form a base for further studies to understand how cortico-cortical networks integrate internal plans and sensory feedback to produce voluntary reaching movements.

Materials and Methods

Behavioral Task

Two macaque monkeys (Macaca mulatta) were operantly trained to move a cursor that matched the monkeys’ hand location to targets projected onto a horizontal reflective surface in front of the monkey. Animal research procedures were approved by the Brown University Institutional Animal Care and Use Committee. The monkey sat in a primate chair with the right arm placed on individualized, cushioned arm troughs secured to links of a two-joint exoskeletal robotic arm (KINARM system; BKIN technologies, Kingston ON, Canada) (Scott 1999) underneath an image projection surface that reflected a computer monitor display. The shoulder joint was abducted 85° such that shoulder and elbow movements were made in an approximately horizontal plane. The shoulder and elbow joint angles were digitized at 500 Hz by the motor encoders at the joints of the robotic arm. The x and y position of the hand were computed using the standard forward kinematic equations, and sampled at 200 Hz.

Continuous manual tracking task

The continuous manual tracking task (CMTT) comprised a single target cursor that moved under computer control through the workspace along a new trajectory on each trial (Fig. 1). The target’s trajectory within a given trial comprised a series of straight movements (“lines”), each of which had the same length and velocity profiles but different directions. The monkey was rewarded for keeping a hand position cursor in continuous contact with the target, via scaling the reward volume to the monkey’s performance across the entire trial.

Fig. 1.

Schematic of trial time course. A: Target appears on screen. B: Target acquired by cursor for 300 ms. C: Target begins moving continuously along indicated trajectory. Arrows displayed for schematic purposes only. Line length 48.2 mm, bell-shaped speed profile, mean speed 30 mm/s.

At the start of each trial, the workspace contained the hand cursor (“cursor”), and a single target cursor (“target”) at a random location in the workspace (Fig. 1a). The target was a circle 22.5 mm in diameter, covering a visual angle of 4.4°. All visual angle measurements reflect visual angle with the object at maximum possible distance from the monkey’s eye, since objects could appear at many locations across a horizontal workspace. Once the monkey’s hand cursor (8.0 mm, 1.6°) maintained contact with the target for 300 ms (Fig. 1b), a trial began.

After trial start, the target moved smoothly through 3 (monkey PF) or 4 (monkey AB) consecutive lines (Fig. 1c). Each line comprised a 48.2 mm (9.4°) extent, during which the target followed a bell-shaped velocity profile of mean speed 30 mm/s (5.85°/s), producing a trajectory of 1.99 s duration. The target had a minimum speed of 1.7 mm/s (0.34°/s), occurring at the times of direction change; the target had a maximum speed of 51.7 mm/s (9.8°/s), occurring in the middle of each line. The direction of each successive line in the trial was chosen randomly from 16 possible directions equally spaced on a circle, excluding directions that would move the target outside the workspace.

At one point during each 7.9 s trial, the target cursor disappeared for 260 or 400 ms, as if moving behind an object that could only be distinguished from the background by the way it occluded the target; lines including these occlusions were excluded from the current analyses.

A juice reward was given at the end of each trial, followed by a 2 s delay before the next trial, during which a black screen (no stimulus) was presented. The juice volume was scaled with position performance; a theoretical maximum reward (duration 0.45 s; volume 0.65 ml) was multiplied by the “accuracy” on each trial, with accuracy defined as the proportion of trial time spent with the cursor overlapping the target by at least 1.6 mm (0.3°). If the accuracy was below 0.37, no reward was given. A trial was aborted without reward under the following conditions: (1) the root mean squared (RMS) position error between target and hand cursor ever exceeded 41.0 mm (8.0°); (2) the target did not overlap the cursor for 3 consecutive seconds; (3) if the monkey’s non-task (left) hand released a compression switch. All analyses and trial counts used only rewarded trials.

Electrophysiology

Silicon-based electrode arrays (Blackrock Microsystems, Salt Lake City UT) composed of 100 electrodes (1.0 mm electrode length; 400 μm inter-electrode separation) were implanted in the left hemisphere arm area of the primary motor cortex (MI) and parietal area PE of each monkey, as diagrammed in Fig. 2. We attempted to center the parietal array on area PE (Pandya and Seltzer 1982); however, because of the unclear (and, here, un-measured) cytoarchitectonic boundary between areas 2 and 5, our recording may also include units from area 2. Therefore, our PE array might be more accurately as an “area 2/5” array; however, we retain the label “PE” for terminological consistency.

Fig. 2.

Location of multielectrode arrays. Lateral view of cortical surface. Grey squares indicate array locations (2.4 mm²). Abbreviations: arc = arcuate sulcus; cs = central sulcus; ips = intraparietal sulcus; pcd = postcentral dimple. Diagrams not to same scale. A: Monkey PF. B: Monkey AB.

Each monkey had both arrays. The MI array was located on the precentral gyrus medial to a line extending from the genu of the arcuate sulcus posteriorly to the central sulcus, and parallel to the sagittal fissure. The 2/5 array was located on the superior parietal lobule, immediately parallel to the intraparietal sulcus and lateral to the postcentral dimple. All procedures were in accordance with Brown University Institutional Animal Care and Use Committee approved protocols and the Guide for the Care and Use of Laboratory Animals (National Institutes of Health publication no. 85–23, revised 1985).

During a recording session, signals from up to 96 electrodes were amplified (gain 5000), bandpass filtered between 0.3 kHz and 7.5 kHz, and recorded digitally at 30 kHz per channel using a Cerebus acquisition system (Blackrock Microsystems, Salt Lake City UT). Waveforms were defined as 1.6 ms data windows starting 0.33 ms before the voltage crossed a threshold of at least −5 times the channel root mean square variance. These waveforms were then sorted using a density clustering algorithm (Vargas-Irwin and Donoghue 2007), the results of which were reviewed using Offline Sorter (Plexon, Dallas TX) to eliminate any putative units with multi-unit signals (defined by interspike intervals (ISI) < 1 ms) or signal to noise ratios (SNR) less than 1. Multi-unit signals were only retained if they could be clearly differentiated into single-unit components in the first two dimensions of principal component analysis space.

A “recording session” or “session” refers to all data collected from one monkey on one date, while a “dataset” refers to all data collected from a single array during one recording session, along with accompanying kinematic data. A total of 4 sessions (8 datasets) were recorded for monkey AB, and 3 sessions (6 datasets) for monkey PF.

Eye position was recorded continuously at 120 Hz with an infrared pupil detection system (Iscan, Inc., Burlington MA), during sessions with monkey PF. Monkeys were allowed to direct their gaze freely throughout the experiment.

Kinematics

Using the Kinarm device described above, endpoint (hand) position in Cartesian x and y coordinates was recorded at 500 Hz, smoothed using an acausal Gaussian filter with sigma of 5 samples (40 ms), and resampled to 50 ms bins. RMS position error between the target and the hand cursor was calculated for each sample point. X and Y velocity values were calculated by taking the appropriate derivatives, and speed error was determined as the RMS error between the target and hand cursor speeds. All target and cursor characteristics were measured using the center of the target or cursor.

Time lag between cursor and target was calculated based on the distance between the cursor and target in each Cartesian direction, for each time bin. Time lag was defined as the amount of time the hand cursor is behind the target cursor, determined by (1/[target speed] * position error).

For kinematic and all other analyses, distribution values represent mean ± standard deviation unless otherwise specified. If the text specifies median as the measure of central tendency, the measure of variability remains standard deviation.

Single-Unit Neural Data

For single-unit analyses, neural spike timestamps were binned in 50 ms bins and smoothed with an acausal Gaussian filter with sigma of 3 bins (150 ms). Spike times were offset unit-by-unit by the amount of time within ± 500 ms that maximized the correlation between firing rate and hand position. This reflects a simple, computationally efficient measure of spike timing (unlike OLT, below), since single-unit analyses were only used for data characterization, not movement reconstruction.

Preferred directions (PD) of individual units were calculated to confirm that the arrays recorded from populations that uniformly sampled the workspace of possible movement directions. PDs were calculated by rounding actual movement direction to the nearest 5° (72 total directions). Neural activity was averaged over all kinematic points in the same movement direction during successful trials, and a cosine function was fit to these 72 points. A least squares method was used to fit the directions and firing rate to a cosine function. Rayleigh tests were performed to calculate whether the population of units on an array uniformly sampled the workspace. A dataset was considered uniform if the distribution of cosine-fit PDs was not significantly different from chance (α = .05).

Kinematic Reconstruction from Neural Data

Arm endpoint (hand) position and velocity were independently reconstructed from neural data using linear regression, in order to provide a measure of how well the neural code encoded movement kinematics. We used a linear filter instead of a Kalman filter, because linear filters better handle discontinuities in data (such as those produced by our removal of visually occluded data from analysis), because Kalman filters rely on a Markov chain of consecutive states (Wu et al. 2003). Details of the linear regression method and algorithms are available elsewhere (Hatsopoulos et al. 2004). A filter length of 1 s (± 500 ms from the time of the kinematics) and bin size of 50 ms was used; i.e., each 50 ms filter was reconstructed using 20 bins of data. Neural data were smoothed as described above; the Gaussian filter eliminates the need to use overlapping bins (equivalent to a box filter). The reconstructed position or velocity values were smoothed using an acausal Gaussian filter with sigma of 5 bins (250 ms). 10-fold cross-validation was used across all data, to prevent overfitting.

Reconstruction assessment

Correlation between hand position and reconstructed position (reconstruction correlation, RC) was calculated by determining the correlation (as measured by the coefficient of determination, r²) between the hand trajectory and the reconstructed trajectory, separately for each trial. Position RC represents the average of the x position correlation and y position correlation, while velocity RC represents the average of the direction correlation and speed correlation. Reported RC values are median ± standard deviation across trials.

The analyses were repeated using an alternative measurement of reconstruction quality: reconstruction error (RE). Position RE was measured as the RMS position distance between the reconstructed and hand cursor positions at each time bin, and velocity RE was measured as the magnitude of the velocity difference vector between the reconstructed and hand cursor velocities at each time bin. RE and RC measurements produced very similar patterns of results, which never pointed towards different conclusions. Therefore, for brevity, only RC values are reported.

Balanced-units comparison

In order to determine the amount of reconstruction information available per unit, the number of units recorded was normalized across sessions. The data from each dataset was normalized to 22 units, the number of units available from the dataset with the fewest units. Within each dataset, 22 units were selected randomly (creating one “draw”), and reconstruction quality was evaluated as described above, except using only 5 cross-validation folds to make the problem computationally tractable. This was repeated 50 times for each dataset, with the units chosen randomly from all the available units on each draw. Distributions of RC values for each dataset were created from the distributions of these 50 draws, to eliminate any effects of which specific units constituted each draw.

Combined-areas analysis

To evaluate the redundancy of neural coding in MI and PE, units from the two areas were merged for a combined-areas analysis. In this analysis, all units from a single recording session were treated as if they had been recorded from a single array or cortical area (labeled “combined”).

Other influences on reconstruction

Because MI and PE might differ in which movement-related characteristics they encode, an analysis was performed to measure the potential contribution of 4 possible influences (factors) on reconstruction quality: hand position in workspace, hand velocity, kinematic error (Cartesian behavioral error at each time point), and eye position (relative to target). We measured the correlation between reconstruction quality and each factor, separately for each factor and dataset (eye position was not recorded or analyzed for monkey AB). The correlation between a factor and reconstruction quality provides an estimated upper limit for that factor’s contribution to reconstruction quality for a given dataset; for example, if PE but not MI reconstructions show an influence of kinematic error, then there would be a higher correlation between kinematic error and reconstruction quality in PE datasets.

Timing Analysis

The timing of individual units with respect to kinematics was determined by identifying the optimal lag time (OLT) within ± 500 ms that maximized the mutual information between the spiking and hand kinematics, for a given measure of hand kinematics (position or velocity). Mutual information quantifies how much the variability of a particular parameter (e.g. neural firing rate) depends on the variability of another (e.g. hand position). Mutual information was calculated following Mulliken et al. (2008b, formulae 4 and 5); normalization was omitted, since mutual information values were only used to calculate OLT for each unit, and thus mutual information was never directly compared between variables or units. OLT was determined separately for each neuron and each trial, and group means are presented across trials for each unit, divided into 30 ms bins for plotting. OLT values indicate the lag of the kinematics behind the neural activity, such that positive OLT represents pre-movement neural activity.

Results

We compared data across 7 sessions total from two monkeys. In each session, monkeys performed 110–148 successful CMTT trials (out of 202–335 initiated trials; success rate 52 ± 9%) during recording from both MI and PE. We compared reconstruction quality for hand position and velocity between the two recorded areas. Subsequently, we tested for overlapping versus independent encoding in the two areas, after which we reviewed the possible contributions of confounding factors to movement reconstruction.

Characterization of Neural Data

Each session comprised 45–70 min of tracking behavior, containing 110–148 rewarded CMTT trials, with 22–139 units recorded from each of two arrays. Table 1 lists the number of units and rewarded CMTT trials in each dataset, as well as a measure of population PD uniformity, the Rayleigh test. 13 of the 14 datasets showed uniformly distributed PDs, as shown by non-significant Rayleigh p-values (α = .05); a significant value reflects a non-circular distribution. Therefore, our neural recordings generally included a full sampling of possible tuning directions.

Table 1.

Overview of recording sessions. The table shows the number of rewarded CMTT trials, counts of simultaneously recorded MI and PE units, and mean ± std rating of unit quality (1–4 scale) on each day of recording.

Recording Sessions
Monkey	Session	Trials	MI units	MI Rayleigh	MI unit rating	PE units	PE Rayleigh	PE rating
AB	1	148	139	.418	2.33 ± .1.09	25	.891	1.96 ± .74
	2	111	126	.036*	2.44 ± .1.04	23	.819	2.09 ± .85
	3	119	141	.202	2.57 ± .91	22	.964	2.14 ± .64
	4	129	143	.733	2.56 ± .90	30	.175	1.93 ± .69
PF	1	110	31	.850	2.03 ± .84	80	.337	2.44 ± .92
	2	113	48	.059	2.08 ± .92	87	.980	2.35 ± .79
	3	134	50	.182	2.18 ± .80	82	.240	2.31 ± .86

Also shows p-value for Rayleigh tests;

^*indicates population PDs significantly different from circular (α = .05).

We recorded populations of directionally tuned neurons simultaneously from MI and PE, on each session for each of two monkeys. As shown in Table 1, we rated individual units on a subjective 1–4 scale, based on waveform reliability, waveform amplitude, SNR, and distinctness from noise. Because reconstruction could depend on any combination of unit count and unit quality, we evaluated the relationship between the number of recorded units in each dataset, and the mean unit rating. We found a strong linear relation (r² = 0.815), suggesting that these two measures largely reflect a single variable of “array recording quality”

Behavioral Performance

Monkeys were able to track the moving target, as illustrated in representative movement trajectories from each monkey (Fig. 3). While speed profiles varied substantially between trials, hand cursor speed generally lagged behind target speed, with increased variability during deceleration. Across trials, both monkeys showed a mix of pursuit (high lag) and predictive (low/negative lag) hand movement. 34.8 ± 2.2% of trials for monkey AB, and 18.1 ± 6.3% of trials for monkey PF, showed mean time lag below 90 ms, too short for feedback-driven pursuit. This mix of strategies conforms with findings by Paninski et al. (2004) that monkeys use a mix of pursuit and predictive strategies when tracking a continuously moving target.

Fig. 3.

Behavioral performance. Hand cursor (grey) and target (black) trajectories, reflecting individual movements during representative trials (median kinematic error). Positions translated so all plots start at origin. Dots represent bin centers (50 ms each); large circles represent target/hand cursor size and initial position. Movement angles 203° (A, session AB2) and 135° (B, session PF2) shown.

Kinematic reconstruction from PE and MI units

Fig. 4 shows representative samples of position (4a–4b) and velocity (4c–4d) reconstruction from the PE dataset of median reconstruction quality (AB4_PE). The samples present the trial of median reconstruction correlation (4a, 4c), and the trial of best reconstruction quality (4b, 4d). We found generally successful reconstruction, with RC (r² between hand kinematics and reconstructed kinematics) of 0.38–0.87 for these examples. Trajectory reconstruction from MI reached similar levels: samples chosen in the same pattern would show RC of 0.37–0.93. (Not shown for brevity; see e.g. Wu et al. (2006) for previous solutions of MI trajectory reconstruction from similar microelectrode arrays).

Fig. 4.

Samples of PE reconstruction, from dataset of median quality (AB1_PE), showing hand cursor (grey) and target (black) trajectories. Top row (A–B): Position reconstruction. Bottom row (C–D): Velocity reconstruction. Left column (A, C): median-quality trial. Right column (B, D): best trial. Reconstruction quality determined by correlation between hand kinematics and reconstructed kinematics (r²).

Fig. 5a–b summarizes the distribution of reconstruction quality results for all PE datasets, revealing generally superior reconstruction of position compared to velocity, across monkeys. Position RC (0.571 ± .05, group mean ± SEM across datasets) was significantly better than velocity RC, which reached 0.382 ± .04 (t-test p < .01)

Fig. 5.

Distributions of reconstruction quality for all datasets. Bars show mean ± std across trials. * indicates session with significant difference between arrays (t-test p < .05). (A) PE position reconstruction. (B) PE velocity reconstruction. (C) MI position reconstruction. (D) MI velocity reconstruction.

Fig. 5c–d illustrated the distribution of RC for MI reconstruction. As with PE, MI units led to superior reconstruction of position compared to velocity, across monkeys; position RC reached 0.669 ± .05, significantly better than velocity RC (0.497 ± .03; t-test p < .05).

For most sessions (12/14), we found no significant difference in RC between arrays (p > .15). We found two exceptions: in session PF1, PE led to better position reconstruction (RC difference 0.25, p < .05), and better velocity reconstruction (RC difference 0.119, p < .05) compared to MI; in session AB4, area MI led to better velocity reconstruction than PE (RC difference 0.269, p < .05). Comparing across sessions, we found no systematic difference between MI and PE in the ability to reconstruct position or velocity (Wilcoxon signed-rank p > .2).

The above analysis did not control for the number of neurons in each dataset, which may bias our results. Indeed, Fig. 6 reveals a strong linear relationship between the number of recorded units and the reconstruction quality. Despite the lack of between-session significant differences, the session averages showed a clear linear trend. Even grouping across arrays and monkeys, r² between unit-count and reconstruction-quality ranged from 0.723 (velocity) to 0.874 (position). We grouped data across monkeys in this case to highlight the robustness of this effect; doing so introduces many un-measurable sources of variance, yet the effect of unit count can still account for up to 87% of the variation between datasets (as demonstrated by r² = 0.874 between unit count and RC during position reconstruction, Fig. 6a). As such, the reconstruction measures summarized in Figs. 5–6 may depend substantially on the neuron count, occluding any differences in movement-related information between the two cortical areas. Therefore, we compared reconstruction from MI and PE using a method that balanced the number of neurons in each dataset.

Fig. 6.

Relationship between reconstruction quality and unit count. Lines and r² values indicate linear fit across all datasets, grouping both monkeys and both areas. A: Position. B: Velocity.

Balanced-Units Reconstruction Quality

In order to disentangle the effect of unit count from the available information in different cortical areas, we performed a balanced-units analysis. We randomly selected 22 units (the size of the smallest dataset) from each dataset, reconstructed hand movements based on these units, and repeated the process 50 times. In this way, we created distributions of reconstruction quality for each dataset, all balanced at 22 units per dataset.

The balanced-unit analysis did not substantially change results. As shown in Fig. 7, RC values have decreased for many datasets (due to the use of fewer units in many datasets, e.g. AB1_M1) but not all (for datasets that originally had few units, e.g. AB3_PE); however, the overall pattern remains the same. For area PE, position RC reached 0.480 ± .03, and velocity RC reached .328 ± .02. For area MI, position RC reached 0.535 ± .03, and velocity RC reached 0.370 ± .01. We found no significant differences between the RC distributions of the two arrays on any session (t-test p > .3). We also detected no significant difference between RC means across sessions (Wilcoxon signed-rank p > .11). Therefore, we found no consistent differences between arrays, when controlling for unit count.

Fig. 7.

Fixed-unit-count analysis for both arrays. Bars show mean ± std across 50 draws, 22 units per draw. No sessions have significant differences between arrays. (A) PE position reconstruction. (B) PE velocity reconstruction. (C) MI position reconstruction. (D) MI velocity reconstruction.

Note that Fig. 5 and Fig. 7 have different error bars, reflecting different measures of variability. For the balanced-unit analysis, we tested the averages and variation across draws, to average out any effects that might arise from

We next tested whether different areas provided better performance for different monkeys, even when controlling for the number of units. To detect any interactions between monkey and area, we performed a 2 (monkey) * 2 (array: PE, MI) ANOVA for each variable. We found no significant main effects (p > .1), but a significant interaction effect, both for position (F[1,10] = 25.36, p < .001), and for velocity (F[1,10] = 6.81 p < .05), indicating a different array provided higher RC for each monkey, for either position or velocity reconstruction. This could arise from technical factors rather than anatomical factors; the array with more units also provided units of higher quality (correlation between unit count and quality: r² = .815, see Table 1), so the array with more units may retain a recording-quality advantage even when controlling for unit count.

Timing Analysis

Even though both MI and PE encode similar types of information, this information may evolve differently in time. We measured this via OLT, the time shift that optimized the mutual information between neural firing and hand kinematics. Recall that positive OLT reflects kinematics occurring after neural activity, or pre-movement (early) neural activity.

Fig. 8 demonstrates highly overlapping distributions of OLT for position reconstruction in two representative sessions (one from each monkey). Both areas encoded kinematic variables at multiple time scales, including ≥90 ms lags consistent with motor command activity, −30 to 90 ms lags consistent with forward estimates of current movement state, and ≤30 ms lags consistent with sensory feedback (Miall et al. 1993, Mulliken et al. 2008a). In these examples, the timing distributions between the two areas overlapped heavily; the two areas differed significantly for velocity (t-test p < .05 for each monkey) but not for position (p > .6 for each monkey). Overall we found significant differences between the OLT means of the two areas (α = .05) for position reconstruction in 1/7 datasets, and for velocity reconstruction in 5/7 datasets. Across the 6 sessions with significant differences between areas, MI always preceded PE (higher mean OLT for MI), but the difference reached only 29.7 ± 7.5 ms. For comparison, in those same datasets, within-dataset standard deviations averaged 43.9 ± 10.0 ms. Therefore, even when mean timing differed between the two areas, the timing distributions remained heavily overlapped. Overall, our data show that while PE activity never precedes MI activity (for position and velocity), the two areas show only minor differences in reconstruction timing during a continuous task.

Fig. 8.

Histograms of optimal lag time (OLT); lag time that maximized mutual information between individual unit activity and hand kinematics. Data from two representative sessions, one from each monkey, in 30 ms bins. Positive OLT indicates kinematics after neural (i.e., neural first). P-values indicate difference in mean (t-test) between MI and PE. A: OLT from position reconstruction, session AB2. B: OLT from velocity reconstruction, session AB2. C: Position reconstruction, session PF2. D: Velocity reconstruction, session PF2.

Distinct Encoding: Combined-Areas Analysis

We next sought to evaluate whether areas PE and MI encode distinct vs. overlapping kinematic information. To do so, we pooled the units across arrays for each session, creating a single “combined” dataset from each session. If the two areas encode distinct (complementary) information, then pooling their units should produce a better reconstruction than either area’s units alone. Conversely, if the areas encode overlapping information, the combined dataset should produce a reconstruction no better than (or, likely, worse than) either constituent dataset.

Fig. 9 demonstrates that the combined datasets showed no improvement in position or velocity RC, compared to the datasets from MI or PE alone. Fig. 9 shows the distribution of reconstruction quality across datasets, modeled after Fig. 6 but now also including the combined datasets (grey points). Monkey AB reached position RC of 0.777 ± .04 with the combined dataset, compared to 0.778 ± .02 for MI (the individual array with higher RC); for velocity, monkey AB’s combined dataset reached RC = 0.535 ± .03, compared to 0.553 ± .02 for MI. Monkey PF showed a similar pattern: combined dataset position RC = 0.630 ± .05, compared to 0.701 ± .04 for the better individual dataset (PE); for velocity, monkey PF had combined dataset RC = 0.396 ± .06, but 0.481 ± .04 for PE. Across sessions, we detected no significant differences between any possible pair of arrays (MI, PE, combined) for either position or velocity (Wilcoxon signed-rank p > .21). To summarize, combined datasets produced reconstructions of equivalent quality to individual-area datasets; some sessions showed a small trend towards worse reconstruction with the combined dataset, but these differences never reached statistical significance.

Fig. 9.

Combined-arrays relationship between reconstruction quality and unit count. Same format as Fig. 6, with combined datasets added. Lines indicate original linear fit to single-area datasets; r² values indicate fit of all points (combined and single-area) to original line from Fig. 6 (single-area only). Combined datasets always show equal or worse reconstruction compared to single-area datasets. A: Position. B: Velocity.

The combined datasets fall below the RC-per-unit relationship of the single areas. Fig. 9 illustrates this by retaining the original linear fits from Fig. 6, showing that the combined datasets all fall below the line fitted to the single-area datasets. Adding the combined areas reduced r²-values for the plots from .867 to .711 for position, and .711 to .448 for velocity. Overall, the combined datasets produced position and velocity reconstructions equal to or worse than any best superior single-area reconstruction, despite higher unit counts in the combined datasets. This held for any sample of neurons, as confirmed by a bootstrap analysis drawing 22 units from each dataset (i.e, a balanced-units, combined-areas analysis), which produced the same results (not shown); this also illustrates that our combined-areas results did not arise primarily from changed unit counts or degrees of freedom. In other words, we acquired no additional movement-related information by combining recordings from the two arrays, whether or not the combination process entailed an increased unit count. This supports the hypothesis that the two areas contain largely overlapping information about hand position and velocity during CMTT movements.

Other Influences on Reconstruction Quality

MI and PE could potentially encode information about different characteristics of movement: for example, one area could encode more information arising from task-related error signals, while the other contains activity that more accurately reflects movements in a certain spatial area. We calculated the influence of kinematic state (hand position, velocity), kinematic error (Cartesian behavioral error at each time point), and eye position (relative to target) on reconstruction, and found no evidence that these characteristics distinguish between MI and PE.

Most possible influences (position error, hand position, hand velocity, and eye position) showed no substantial impact on reconstruction quality (r² < 0.06). However, kinematic error accounted for up to 25% of the variation in velocity reconstruction quality (r² = .256), but it did not affect the two areas differently (p > .1). Therefore, our comparisons between MI and PE do not primarily depend on the aforementioned influences; however, a velocity error signal may contribute to movement encoding in both areas.

Discussion

We presented simultaneous multielectrode recordings from MI and parietal area PE (or areas 2/5) in multiple macaque monkeys, and demonstrated reconstruction of continuous hand movements from both populations. We showed that a population of neural units sampled from PE can reconstruct hand kinematics, demonstrating a robust representation of hand position and velocity in PE. We found no systematic difference in trajectory reconstruction quality between the two areas; across monkeys, the two areas showed equally good representation of position and velocity. Most interestingly, we found that combining the two areas provided no improvement in movement reconstruction, implying that the two areas encode overlapping (rather than complementary) information about hand kinematics.

Trajectory Reconstruction from PE

Our data demonstrate successful reconstruction of trajectory kinematics from PE neural activity. We used a linear filter to reconstruct hand kinematics due to discontinuities in our data, but under most conditions, Kalman filters provide better reconstruction (Wu et al. 2003; Mulliken et al. 2008a). While the linear filter proved adequate to reconstruct hand kinematics, alternative algorithms could presumably produce even better reconstructions than reported here.

Previously, Mulliken et al. (2008a) successfully reconstructed trajectories from activity in a parietal area including PE, but did so with reconstruction algorithms that included goal-based decoding as well as kinematic decoding. When they used a linear filter similar to our methods, their reconstructions were much less successful; our average reconstruction accuracy (r² = 0.56) exceeded their best single session reconstruction accuracy (r² approaching 0.5). Instead, our average accuracy matched their average accuracy for their best method (G-Kalman filters, r² ~ 0.55). Our superior linear filter reconstruction results likely arise from our choice of task. A continuous feedback and control task such as CMTT should engage goal, sensory, and feedback mechanisms throughout the movement, rather than just with respect to the final goal. Since all of these processes involve area PE (Gail and Andersen 2006; Mulliken et al. 2008b), PE activity should capture the full trajectory of hand movement during CMTT, but not necessarily during point-to-point movement tasks.

Recall that our PE array may include units from outside area PE proper. We did not perform histological analyses because the monkeys remain implanted and behaving. As such, our PE array may also be considered an “area 2/5” array, due to the possible presence of somatosensory units in adjacent area 2. Somatosensory units could encode information about hand position, and thus contribute to reconstruction. However, as discussed in the following section, our PE array recorded a similar distribution of sensory-like and non-sensory-like units, compared to other studies of PPC (Mulliken et al. 2008b). Therefore, the evidence suggests that our reconstruction performance was not driven by the contributions of somatosensory units in area 2.

Reconstruction Timing

Our reconstructions from both MI and PE involved units with a wide distribution of timing, including units with sensory-like timing, motor-like timing, and forward-estimate-like timing (Fig. 8). Recent studies have shown similar distributions in PPC neural populations (Mulliken et al. 2008b), which suggests that the sensory-like component of our PE (or areas 2/5) recordings does not arise wholly from any electrodes that may have ended up in area 2. However, previous studies of MI populations have shown only motor-like activity during continuous tracing tasks (Schwartz 1993; Moran and Schwartz 1999). MI has no adjacent sensory areas to account for this, unlike PE, but the difference in task may account for this discrepancy. Predictable or overtrained trajectory such as sinusoids or spirals could reduce the role of sensory information during motor execution, compared to the unpredictable trajectory of the CMTT. Indeed, one study that involved continuous tracking of an unpredictable target (Paninski et al. 2004) found an average time lag near zero across units, similar to our findings here. To fully resolve this question, future studies might directly compare predictable vs. unpredictable continuous tracking tasks within a single recording session.

Comparison of Simultaneously Recorded Activity from MI and PE

Here, we reconstructed hand trajectories from simultaneously recorded MI and PE populations. Previous studies have reconstructed hand movement kinematics from neural populations in MI (e.g. Serruya et al. 2002; Taylor et al. 2002; Hochberg et al. 2006; Velliste et al. 2008) and neural populations in PE (e.g. Kalaska et al. 1990; Averbeck et al. 2005; Mulliken et al. 2008a; Archambault et al. 2009) during a variety of tasks, but never from both areas simultaneously. Simultaneous recording allows for robust comparison of the two areas, because it ensures complete control of behavioral and external factors between the two recordings. We demonstrated that, during continuous movements, neither area holds a consistent advantage in reconstruction quality. This overt similarity is consistent with older studies that found similar movement encoding in MI and PE during non-simultaneous recordings (Kalaska et al. 1983; Ashe and Georgopoulos 1994).

While neither area provided better reconstruction across monkeys, we did see between-areas differences in reconstruction quality within each monkey. Because the two monkeys showed opposite areas of better reconstruction (MI for monkey AB, PE for monkey PF), the area effect may arise from a sampling artifact. Our data support this interpretation, because in both monkeys, the array with more units also provided better reconstruction-per-unit (i.e. better reconstruction with unit-balanced analysis), and both “unit count” and “unit quality” seem to reflect the same underlying variable. In addition, unit count served as a strong predictor of reconstruction quality (r² > .8 for position reconstruction). Taken together, this indicates that arrays with better unit quality generally allowed better reconstruction, even though unit quality arises primarily from sampling artifacts such as electrode placement and depth. Across monkeys, any potential difference between areas remained sufficiently modest to remain concealed by the sampling artifact, consistent with our conclusion that we found no evidence for overt differences.

Note that our finding of no overt differences applies only to the current task. For example, one area might provide better reconstruction during feedback-driven movement, while the other area provides for better reconstruction during prediction-driven movement. If so, our task would not have distinguished between these two possibilities, because monkeys used a mix of pursuit and predictive strategies to track the continuously moving target.

Overlapping Information in MI and PE

In order to determine how movement representations change as information proceeds from MI to PE, we evaluated whether the two areas encoded the same representation of hand kinematics, as opposed to encoding different information about ongoing hand movements. We found that combining neurons from the two areas never provided better reconstruction than a single area, whether or not the number of units increased. Therefore, MI and PE contained overlapping, non-complementary information about movement trajectories during the CMTT. If the two areas encoded distinct movement-related information, then combining MI and PE together would have allowed a better reconstruction than either area alone, but this never occurred for any session. Instead, our results suggest that the two areas’ movement encodings contain largely the same information about hand movements; to simplify, whatever hand position or velocity information one area encodes, the other encodes also.

One alternative explanation of our data might hold that, in our monkeys, the sampling artifacts could have led to noise in the worse-reconstructing area that interfered with (rather than contributing to) across-areas reconstruction. A second possible alternative explanation might hold that the two areas encoded distinct information that our algorithms failed to effectively combine into a single model. However, neither of these alternatives adequately explains our data, for the same reason. Since both areas contained unit populations with uniformly distributed directional tuning that allowed successful reconstruction of hand kinematics, it seems implausible for that same information to interfere, or fail to be captured by the model, when combining areas. However, we cannot reject the possibility that the two areas could uniquely encode alternative movement characteristics and variables, other than hand position and velocity. Such alternative characteristics lie beyond the scope of this experiment; we claim only that MI and PE encode overlapping information about hand position and velocity.

This raises an obvious question: why would the brain encode the same information in two different cortical areas? However, this phrasing reflects a misunderstanding of our findings; we do not claim that MI and PE are redundant. Each area plays a different role in sensorimotor processing, even though both areas encode movement kinematics. MI is primarily involved in motor output, learning, and observation (e.g. Ashe and Georgopoulos 1994; Sanes and Donoghue 2000), while PE plays a major role in adjusting movements based on comparisons between expected and actual sensory feedback (e.g. Shadmehr and Krakauer 2008). We propose that our results reflect a situation wherein motor command signals and goal/error signals each capture the same information about the kinematics of an ongoing CMTT movement. For example, if MI sends goal information to PE, area PE could then interpret incoming somatosensory information as an error signal. In a continuous feedback task, immediately upcoming goals and ongoing sensory information both involve representations of current hand kinematics. In other words, as goal and error signals constantly update each other during a time-evolving movement, a situation arises wherein two areas encode overlapping information.

In conclusion, our results suggest that neural encoding in MI and PE is best understood not as two distinct representations, but as both areas functioning as part of the unitary closed planning-and-feedback loop that drives ongoing time-evolving feedback-driven behavior.

Summary

This work investigated simultaneous multiunit recordings from MI and area PE (or areas 2/5) in the macaque monkey, and demonstrated two major points about continuous hand movement encoding in these two areas. First, we showed here that neural population activity in PE during a continuous movement task can accurately reconstruct hand trajectories, even using a simple linear decoding model.

Second, we found that MI and PE populations encode overlapping information about hand movements, at least for a task with continuous, time-evolving demands and error feedback. Combining units from the two areas provided no advantage for reconstruction of either position or velocity, demonstrating that the two areas did not encode complementary information about hand position and velocity. This overlapping information may reflect both areas’ participation in an ongoing sensorimotor feedback loop for the generation of motor goals and error signals during a continuous task.

Advancing our understanding of how the cortex computes and processes motor commands provides multiple benefits. It not only provides new knowledge about how movement plans become actions, but also provides practical information to improve patient care through developing and driving neural prosthetic devices.