Decoding 3-D Reach and Grasp Kinematics from High-Frequency Local Field Potentials in Primate Primary Motor Cortex

Abstract

Intracortical microelectrode array recordings generate a variety of neural signals with potential application as control signals in neural interface systems. Previous studies have focused on single and multiunit activity, as well as low frequency local field potentials (LFPs), but have not explored higher frequency (>200 Hz) LFPs. In addition, the potential to decode three dimensional (3-D) reach and grasp kinematics based on LFPs has not been demonstrated. Here, we use mutual information and decoding analyses to probe the information content about 3-D reaching and grasping of 7 different LFP frequency bands in the range of 0.3 Hz – 400 Hz. LFPs were recorded via 96-microelectrode arrays in primary motor cortex (M1) of two monkeys performing free reaching to grasp moving objects. Mutual information analyses revealed that higher frequency bands (e.g. 100 – 200 Hz and 200 – 400 Hz) carried the most information about the examined kinematics. Furthermore, Kalman filter decoding revealed that broadband high frequency LFPs, likely reflecting multiunit activity, provided the best decoding performance as well as substantial accuracy in reconstructing reach kinematics, grasp aperture and aperture velocity. These results indicate that LFPs, especially high frequency bands, could be useful signals for neural interfaces controlling 3-D reach and grasp kinematics.

I. Introduction

Intracortical microelectrode recordings in motor cortex, especially primary motor cortex, generate a variety of neural signals with potential applications in neural interface systems for motor prostheses: extracellular action potentials or single unit activity (SUA), multiunit activity (MUA) and LFPs. The use of SUA for decoding of intended and/or executed movements has been extensively studied in monkeys [1], [2], [3], [4], [5] and recently demonstrated in humans with tetraplegia [6], [7]. Several groups have also examined neural decoding of movement parameters based on MUA and intracortical LFPs [8], [9], [10], [11], [12], [13], [14]. In particular, Mehring et al. [10] using four simultaneously recording electrodes in each hemisphere, showed that time-domain low-frequency features of LFPs in monkey motor cortex carried information about movement direction and two-dimensional (2-D) hand position in a 8-directions center-out task. Using the same data, Rickert et al. [11] demonstrated that tuning to movement direction was prominent in the frequency bands 0 – 4Hz, 6 – 13 Hz and 63 – 200 Hz. Stark and Abeles [14], using 8 electrodes implanted in monkey M1, showed that 2-D hand velocity could be decoded from MUA and LFP signals. However, the use of higher frequency band field potentials in neural decoding has not been systematically examined in intracortical recordings. With the exception of [15] and [16], this has also been the case in most studies based on extracortical recordings such as subdural electrocorticograms (ECoGs) [17], [18], [19], [20]. Furthermore, none of the above mentioned work has systematically studied decoding of 3-D reach and grasp kinematics based on LFPs. The addition of signals that increase the dimensionality of control signals, especially to achieve 3-D reach and grasp is important for corticomotor prostheses that aim at restoring movement in people with paralysis.

In this paper, we examined how well endpoint hand kinematics and grasp aperture can be decoded from LFPs recorded from M1 in monkeys performing free reaching and grasping movements in a 3-D workspace. We chose the power spectrum in seven different frequency bands as the main LFP features. These frequency bands corresponded to: δ band (0.3 – 5 Hz), θ-α (5 – 15 Hz), β (15 – 30 Hz), γ₁ (30 – 50 Hz), γ₂ (50 – 100 Hz), γ₃ (100 – 200 Hz) and a broad high frequency band (200 – 400 Hz). This 200–400 Hz broadband high frequency LFP is henceforth referred to as the bhfLFP. The information content about 3-D reach kinematics and grasp aperture in these seven different LFP frequency bands was assessed via mutual information analyses and also evaluated by comparing the performance of a Kalman filter decoding algorithm based on each of these bands. Our results show that power in high-frequency LFPs (i.e. γ and bhfLFPs) carried the most information about reach and grasp kinematics, and that these bands can be used to decode hand 3-D position and grasp aperture with substantial accuracy. Overall, our findings indicate that local field potentials, used alone or combined with SUA and MUA, might provide useful control signals for 3-D reach and grasp control in neural interfaces that aim at restoring motor function in humans with paralysis.

II. Methods

A. Data acquisition and signal preprocessing

Two male macaque monkeys (C and G) were used in this study. Neural activity was recorded using 10 × 10 microelectrode arrays (Cyberkinetics Neurotechnolgy Systems, Inc., Foxborough, MA) chronically and intracortically implanted in the arm region of the primary motor cortex (M1, contralateral to the hand used for the task). The array recorded from a 4 × 4 mm patch of cortex, with minimal inter-electrode distance of 400 μm. Additional details of array structure and surgical implantation procedures are described in [21]. Neural activity was recorded and stored using a Cerebus multi-channel data acquisition system (CKI). LFPs were originally recorded after analog filtering (1st-order Butterworth, high pass with cutoff edges at 0.3 Hz; 3rd-order Butterworth, low pass with cutoff edge at 7.5 kHz), digital low pass filtering (4th-order Butterworth, low pass with cutoff edge at 500 Hz), and sampling at 2,000 samples per second. 47 and 96 simultaneous LFP channels were recorded from monkeys C and G, respectively. Hand kinematics, including 3-D position of the hand (measured at the wrist) across a natural reach workspace in front of the monkey and the grasp aperture (the distance between the distal-most joints of the thumb and index finger), were recorded. We used a Vicon motion capture system (Vicon Motion Systems, Oxford Metrics Group, plc.) with 29 markers placed to provide position of the entire arm, from the shoulder to each of the finger tips. The setup of the motion capture system followed that described in [22]. For the purpose of the present study, the marker system was used to identify only the location of the hand in space and grasp aperture (see Fig. 1). Position and grasp aperture were recorded at a rate of 240 samples per second. 3-D hand velocity and aperture velocity were derived from 3-D hand position and aperture through approximate time derivative calculations. For computational convenience, all kinematics data were resampled to 20 samples per second after they were low passed by a Kaiser window FIR filter with passband cutoff of 2 Hz, stopband cutoff of 8 Hz, passband ripple of 1% and a stopband attenuation of 20 dB. The spatial resolution of the kinematics data was 1 mm. All decoding analyses employed in this paper were performed with normalized kinematics variables, that is, the corresponding mean was subtracted and the resulting value was then divided by the corresponding standard deviation.

Fig. 1.

Free reaching and grasping task. (a) Monkeys reached and grasped moving objects in a 3-D workspace. A motion capture system recorded the hand endpoint position (measured at the wrist) and the positions of the tip of the thumb and index fingers (see markers). Grasp aperture was defined as the distance between these two fingers tips. (b) Distribution of hand endpoint positions in the 3-D space over the whole session time. (c,d,e) Projection of hand endpoint trajectories on 2-D subspaces. Marginal distributions of hand position, obtained during one entire experimental session, are shown along each axis. (f) Histogram of grasp aperture over the session time. (Data from monkey C, session 1. Session 2 showed qualitatively similar statistics.)

The aim of the behavioral task was to elicit a wide range of 3-D reaching and grasping movements. Animals were trained to perform a continuous single-handed reaching and grasping task in which one of six to nine different objects attached to a string swung from above repeatedly. After successfully grasping the object for about one second, monkeys were rewarded with fruit juice. The object rotated freely and was swung on each trial by an investigator through an arbitrary path within an approximately natural reach workspace for a monkey (25 cm × 20 cm × 25 cm).

After each reach/grasp action, the monkey would typically bring the arm/hand closer to the body, but there was no specific action to ‘end’ or ‘reset’ a trial. The movements were always ongoing. In other words, each trial had different epochs or stages, but these were not as clearly separated as in typical tasks with discrete and fixed sequences such as object presentation, movement preparation, go cue, etc. Endpoint velocity profiles were commonly bell shaped, but there were variations, such as multiple peaks. Aperture velocity was typically ‘bimodal’. Note also that the monkey could grasp the same object in slightly different manners, and that different trial blocks contained different objects to be grasped. Correct trials also required the monkey to close a hand-switch using the opposite hand during the entire duration of the trial (See Fig. 1). Short temporary breaks were given to the monkeys during the task. Data recorded during the breaks were not included in the analyses. Datasets from two experimental sessions for each monkey were used in the analyses reported here: COS080319 and COS071212 (sessions 1 and 2, respectively), for monkey C, and GAR080702 and GAR080710 (sessions 1 and 2, respectively), for monkey G. All procedures complied with protocols approved by Brown University Institutional Animal Care and Use Committee.

B. Spectral analysis and coefficient of variation

We used the power in different frequencies as the LFP features. In the computation of the LFP spectrogram and coefficient of variation (Fig. 2a, b, c, d), the LFP time series from an entire session (which lasted close to one hour for each monkey) were segmented into non-overlapping time windows of 250 ms. Power is given in dB. (We arbitrarily chose 1, with the appropriate units, as the power reference level.) The multitaper spectral estimation approach [23] was employed to obtain the power spectrum in each window for all the channels. The bandwidth in the multitaper spectral estimation (not to be confounded with the different frequency bands used in the decoding and mutual information analyses) was set to 10 Hz.

Fig. 2.

LFP spectrogram, CV, and mutual information. (a) The spectrogram of a single LFP channel from monkey C shows variations in spectral power across time in various frequency bands. Color bars show spectral power in dB. (b) The CV of the power at each frequency was computed across the experimental session (based on the spectrogram in (a)). High variability is observed for very low frequencies (<50 Hz) and for a broad high-frequency (200 – 400 Hz) band. (c, d) Same as in (a) and (b), but for monkey G. (e, f) To examine whether power variability was related to hand kinematics during reaching and grasping movements, we computed the mutual information between each kinematic variable and the integrated power in each of 7 non-overlapping frequency bands. For both monkeys, power in high γ and bhfLFPs achieved the highest mutual information regardless of the kinematic variable. Colored bars represent the 95% interval for mutual information values computed over the ensemble of 47 (monkey C) or 96 (monkey G) channels. The black square gives the median and the top/bottom triangles the extremum values. Results refer to session 1. Results from the same analyses on datasets from session 2 were consistent with those shown in this figure.

We assessed the power variability across the non-overlapping time windows during the session for each frequency. Because the LFP power values across different frequencies can differ by orders of magnitude, we used a relative quantity of variability given by the coefficient of variation (CV) of the power at each frequency defined as

$${CV}_i(p_{i,f})=\frac{\text{std}(p_{i,f})}{\text{mean}(p_{i,f})},$$ (1)

where p_i,f denotes the power for channel i at the frequency f, and std denotes the standard deviation.

C. Frequency bands

The frequency spectrum (0.3 Hz – 400 Hz) was further partitioned into seven different frequency bands. The upper limit was set, based on a conservative choice, to be lower than the low pass cutoff of 500 Hz in the original digital filtering to avoid any potential artifacts. The seven frequency bands corresponded to: δ (0.3 – 5 Hz), θ-α (5 – 15 Hz), β (15 – 30 Hz), γ₁ (30 – 50 Hz), γ₂ (50 – 100 Hz), γ₃ (100 – 200 Hz) and bhfLFP (200 – 400 Hz).

D. Mutual information

We compared the information content about kinematics in each of the seven frequency bands by computing the mutual information between a single kinematic variable (e.g. hand position in one coordinate) and the integrated power in a given frequency band in a single LFP channel. The power spectrum was computed for windows of 250 ms immediately preceding the corresponding kinematic state, with the exception when computing power for the δ band (0.3 – 5 Hz), where a 350 ms time window was chosen instead to improve sampling at this low frequency band. Therefore, for the mutual information analyses, as well as for decoding, the windows overlapped in time. Mutual information was computed as [24]

$$I_i(K;P_{i,f})=H(K)+H(P_{i,f})-H(K,P_{i,f}),$$ (2)

where P_i,f denotes the integrated power in a given frequency band f for channel i, K denotes the kinematics in one single coordinate (e.g. hand x-velocity or grasp aperture), and H (·) and H (·,·) are the marginal and joint Shannon entropies, respectively. The plug-in method [24] was used to estimate the mutual information, i.e. marginal and joint probability distributions were derived from the empirical histograms. We tried different quantizations of the normalized kinematic variables and normalized integrated LFP-power by partitioning the range of these variables into different number of subintervals (20, 40, 100 and 150 for kinematic and 20, 40, 100 for power). Our results did not qualitatively change with different choices of quantization. In this paper, we focus on the relative differences in mutual information across different bands, rather than on the absolute magnitudes.

E. Decoding of reach and grasp kinematics

We decoded each kinematic variable separately and based on power in a single frequency band at a time. These kinematic variables corresponded to: (a) hand position and hand velocity in the 3-D Cartesian space with x, y and z denoting horizontal (left-right), horizontal (back-front) and vertical coordinate, respectively; and (b) grasp aperture and aperture velocity. A linear Gaussian state-space model (Kalman filter) was employed to decode reach and grasp kinematics from LFP observations. The state-space model was given by

$$\begin{array}{l}{x}_k={Ax}_{k-1}+{\epsilon }_k\\ P_{i,k}={Hx}_k+{\eta }_k\end{array}$$ (3)

where x_k is a given (mean subtracted) kinematics state (here horizontal hand position for concreteness) and k indexes discrete time (50 ms time-steps); P_i,k corresponds to the integrated power for the ith LFP channel in a chosen single frequency band and computed in a time window (e.g. 250 ms) immediately preceding the kinematic state (as in the mutual information analysis, a larger time window was used for the δ band); A and H are the state and observation matrices, respectively, and ε_k ~ (0,Q) and η_k ~ (0,W) are the state and observation Gaussian noise, respectively, with covariance matrices Q and W. Parameters in the state-space model and Kalman filter solutions were estimated and computed according to [25].

F. Assessment of decoding performance

Decoding analyses were performed under a 12-fold cross-validation scheme. To evaluate the decoding performance for continuous reach and grasp kinematics, we report both the Coefficient of Correlation (CC) and Root Mean Square Error (RMSE). The CC was defined here as the maximum of the normalized cross-correlation function between the actual and decoded kinematics, computed for a range of time lags. The cross-correlation at each time lag was defined as the Pearson correlation coefficient. RMSE values for velocity decoding were restricted to velocities within the 95% confidence interval of the entire data, to minimize the contribution of outliers due to the ‘noise’ in the computation of velocity. We prefer the CC and RMSE, instead of the coefficient of determination (r²), because we are interested in more informative quantities about shape similarity (CC) and mean distance (RMSE) rather than simply the amount of explained variance (r²). (As a simple example, consider two exactly equal sinusoidal signals, except that one is a scaled version of the other. The coefficient of determination goes to zero as the scaling diverges from 1, while the Pearson correlation coefficient is always 1, providing thus a measure of shape similarity.)

We also estimated the chance level CC for the Kalman filter performance. The state matrix in the Kalman filter model already incorporates knowledge about temporal correlations in hand kinematics and aperture. This by itself could potentially lead to significant CCs even in the case of non-informative LFP features. A phase-randomization approach was used to estimate the CC’s chance level. We randomized the phases of the FFT of each P_i,k sequence. An inverse FFT was then used to obtain a time domain phase-randomized sequence. This sequence had the same autocorrelation function (power spectrum) as the original sequence, but had its time relationship to the kinematics randomized. State-space parameters and the solutions of the Kalman filter were estimated as before. Chance level CCs were computed under a 12-fold cross-validation scheme.

G. Selection of an optimal LFP-channel subset

We attempted to identify which LFP channel and channel subsets provided best decoding. An exhaustive search of best LFP-channel subsets for decoding was not computationally feasible in our case. Instead, we used a greedy selection algorithm to find the optimal subset out of 47 (monkey C) or 96 (monkey G) channels. The optimal subset was the subset that gave the best decoding performance measured in terms of the CC between actual and decoded kinematics as defined above. This greedy (forward) selection algorithm consisted of choosing first the best one-channel subset, then the best two-channel subset that included the best found one-channel, and so forth for n-channel subsets.

III. Results

The reach and grasp task resulted in a wide range of reaches to targets broadly distributed across the natural workspace. Distributions of these kinematic variables and other details of the task are shown in Fig. 1. When opening or closing their hands, monkeys tended to move the four fingers opposite to the thumb in the same manner. Therefore, the aperture variable, given this large amount of redundancy in finger movement, captured most of the variability of grasping motion. The range, mean and standard deviation of all kinematic variables are presented in Table I.

TABLE I.

Statistics of 3-D Reach and Grasp kinematics

Kinematics	Mean ± SD	Max	Min
x	−6.2 ± 4.8 −7.4 ± 6.5	6.4 9.6	−20.7 −24.9
y	14.2 ± 2.5 7.7 ± 2.6	27.9 42.9	6.3 0.6
z	7.0 ± 3.3 6.0 ± 5.4	27.4 25.5	1.4 −4.7
x-velocity	0.0 ± 10.5 −0.1 ± 18.7	54.1 140.6	−58.9 −105.1
y-velocity	0.0 ± 7.6 0.0 ± 11.8	40.4 270.7	−49.7 −311.7
z-velocity	0.0 ± 12.1 0.0 ± 21.0	83.1 123.0	−70.0 −184.5
aperture	4.2 ± 1.0 4.1 ± 1.1	6.2 8.7	0.9 0.9
aperture velocity	0.0 ± 4.9 0.0 ± 8.6	29.4 35.9	−36.4 −36.9

Numbers in the top and bottom row of each cell in the table correspond to values obtained from experimental session 1 for monkeys C and G, respectively. Positions are given in cm and velocities in cm/second. (The second sessions showed qualitatively similar statistics.)

A. Information about reaching and grasping movements in LFP power spectrum features

We examined the variability of the single-channel LFP power in different frequencies during these reaching and grasping movements. In this case, power was based on spectrograms computed on 250 ms nonoverlapping time windows (see Methods).

The coefficient of variation (CV) was used to assess the variability of the power in each band across the entire experimental session. Since the CV normalizes the standard deviation of a variable by its mean, the CV allowed us to assess power variability across the session while taking into account the large power differences between low and higher frequencies. (These large differences reflected an approximately power-law scaling of power with frequency.) In addition to high CVs at low frequencies (< 50Hz), we observed large CVs for power in a broad high-frequency band for both monkeys (Fig. 2b, d). For most channels, the CV in this broadband peaked around 300 Hz. Exceptions included cases where the CV spectrum showed increasing CVs, but no defined peaks, or cases were the CV spectrum was flat at these higher frequencies. The latter cases were found even in channels in which well isolated single unit activity was present, indicating that these properties were not related to unusual characteristics of the electrode.

Although variability is a requirement for LFP power to convey information about kinematics, the observed high CVs for certain frequency bands could simply reflect intrinsic variability of neural processes unrelated to reaching and grasping movements. We therefore assessed the information content about kinematics, for integrated LFP power in each of the 7 different frequency bands, by computing the mutual information (see Methods). Power was computed from a 250 ms time window immediately preceding the kinematics variables, and kinematics were sampled every 50 ms. (For this reason, time windows partially overlapped in time.) The integrated power in each frequency band is, henceforth, simply referred to as the spectral power in a given band. We note that our selection of a 200 – 400 Hz band was motivated by the combination of three main factors: previous studies have indicated that this high frequency (> 200 Hz) band is likely to also contain multiunit activity [14], [26], in addition to other commonly considered sources of field potentials; the observed high CVs for frequencies in this band in the current data; and, as mentioned in the Methods section, the fact that our maximum frequency was limited by the low pass filtering (500 Hz cutoff) in the original recorded data. To avoid filtering artifacts close to the cutoff frequency, we set the boundary of bhfLFP band to 400 Hz. Thus, the upper limit of this band may be artificially bounded by signal processing.

Power in γ and bhfLFP bands tended to show the highest mutual information for all of the kinematic variables (hand position and velocity, and grasp aperture) in both monkeys (Fig. 2e, f). This result was qualitatively insensitive to the choice of bin sizes in the computation of histogram used in the estimation of marginal and joint distributions (see Methods for details). Despite this similarity between the two monkeys’ datasets, mutual information increased with frequency in monkey C, while this relationship was not monotonic (a U-shaped pattern) for monkey G. The finding of highest mutual information between kinematics and bhfLFPs suggests that at least some of the variation in the bhfLFPs (Fig. 2b,d) was related to reach and grasp kinematics, and that this band could potentially be a better source to decode upper limb kinematics compared to other low frequency bands that have been previously examined.

B. Decoding 3-D hand endpoint kinematics from LFPs

Decoding provides additional means to evaluate the information available in power fluctuations in different LFP frequency bands. A linear Gaussian state-space model (Kalman filter) was used to decode a single kinematic variable at a time from observations consisting of LFP power in a single frequency band in multiple channels. All decoding analyses and results described below are based on decoding under a 12-fold cross-validation scheme. A greedy algorithm was employed to select the channel subset (out of 47 or 96 channels, for monkeys C and G, respectively) that achieved the best decoding performance based on a CC criterion (see Methods).

Based on the mutual information results, we start by focusing our analysis on decoding bhfLFPs. 3-D reach kinematics could be decoded with substantial accuracy. This was suggested by the strength of the CC and low RMSE values, as well as by the visual inspection of plots of reconstructed trajectory paths (Fig. 3). Table II summarizes the analysis of decoding performance including hand position and velocity. Overall, decoding of hand position was better than velocity, with z-coordinate (against gravity) providing the best decoding in both monkeys. A chance level decoding performance was obtained via phase randomization (see Methods). Fig. 3c shows a single ‘chance level’ decoded trajectory example for vertical position (monkey C). In this case, the 99.9% confidence interval for the chance level CC corresponded to [0.08 – 0.18], while the CC based on the actual bhfLFP power was 0.73 (17-channel best subset). The largest upper limit of the 99.9% confidence intervals for chance level CC (including all variables, monkeys and sessions) was about 0.2, which is much lower than the obtained CCs in the actual decoding results (Table II). Similar statistical significance results were obtained for the RMSE. These results show that information in the bhfLFP power was the main contributor to decoding performance, rather than prior knowledge about a kinematic variable’s temporal correlation structure incorporated in the Kalman filter state transition matrix. Accuracy of hand position decoding was also substantial when considering reconstruction of full 3-D hand trajectories, as demonstrated in Fig. 4. In sum, these results reveal that the bhfLFP power provides substantial information for the decoding of kinematics of 3-D reaching movements.

Fig. 3.

Kalman filter decoding of hand kinematics based on power in bhfLFPs. (a,b) For illustration, we show the actual (black) and decoded (red) vertical hand position (z) and corresponding velocities. (Kinematics have been normalized; see Methods.) These sample segments (computed on test data; 12-fold cross-validation) demonstrate the ability to recover kinematics with significant accuracy from bhfLFP power. (c) For comparison, these two plots show a sample trajectory (blue) obtained from a ‘chance level’ Kalman filter decoder for session 1 of each monkey. In this case, a phase randomization approach was used to obtain surrogate data (see Methods). As can be seen, the good decoding performance (red curves) depended primarily on the information about kinematics carried by the bhfLFP power, not the prior knowledge incorporated in the Kalman filter’s state transition matrix. (d) Performance of vertical hand position (z) decoding, based on the CC and RMSE (in percent), varied slightly with the number of channels in a best selected channel subset. This finding was common also for all the other kinematic variables. Best channel subsets were identified based on a greedy selection algorithm applied to test data (see Methods). For example, the best channel subset for vertical position (monkey C) included 17 channels. Larger subsets tended to decrease CC performance in this case. Because this greedy selection algorithm optimized only the CC value, addition of new channels could actually increase the RMSE for certain subsets of intermediate size. For larger subset sizes (e.g. >15), changes in RMSEs were consistent with the increase in CC. (Examples are from session 1. See Table II for summary across sessions.)

TABLE II.

Decoding performance based on power in bhfLFPs

Kinematics	Best channel subset						Best single channel
	# of channels		CC		RMSE (%)		CC		RMSE (%)
	C	G	C	G	C	G	C	G	C	G
x	18 14	19 20	0.68 0.76	0.59 0.61	13.6 9.5	16.8 16.2	0.46 0.37	0.21 0.28	17.4 16.7	25.9 22.1
y	13 18	20 20	0.67 0.49	0.63 0.63	11.6 10.4	6.3 11.3	0.55 0.40	0.45 0.49	13.7 11.8	8.1 11.7
z	17 20	20 20	0.73 0.84	0.67 0.72	11.2 10.8	12.0 14.1	0.63 0.81	0.55 0.64	13.1 10.9	12.4 14.3
x-velocity	20 20	20 5	0.55 0.55	0.41 0.29	19.5 19.9	24.6 19.9	0.22 0.18	0.23 0.28	14.5 23.2	18.4 19.6
y-velocity	20 19	20 2	0.60 0.45	0.44 0.47	17.4 21.2	22.5 19.9	0.36 0.24	0.35 0.45	19.9 20.3	22.0 19.6
z-velocity	20 20	20 20	0.61 0.76	0.52 0.73	17.0 14.5	24.5 18.7	0.37 0.60	0.51 0.64	18.7 25.9	25.7 31.5
aperture	20 20	20 20	0.54 0.68	0.59 0.67	15.9 12.4	12.4 12.5	0.33 0.60	0.30 0.37	22.7 14.4	15.5 20.4
aperture velocity	20 20	20 20	0.47 0.55	0.36 0.47	17.3 14.8	18.7 17.0	0.29 0.30	0.31 0.43	16.7 14.6	16.7 17.8

Numbers in the top and bottom row of each cell in the table correspond to values obtained from sessions 1 and 2, respectively. Because of the high computational demand, in this study we only considered best subset sizes up to 20 channels. Higher decoding performance for some of the kinematics could potentially have been achieved with larger channel subsets. (Average coefficient of determination, r², for vertical hand position (z) across the 2 monkeys and 2 sessions was 0.37; we focused on CC and RMSE measures, rather than on r² values, for the reasons explained in the Methods section.)

Fig. 4.

3-D hand position reconstruction based on Kalman filter decoding. (a)A single selected sample segment (~2 s, decoded on test data) from monkey C (session 1) illustrates the effectiveness of bhfLFP power decoding. Position in each coordinate was decoded separately via a Kalman filter. (b) The 3 plots show two-dimensional projections of the actual (black) and decoded (red) trajectories in (a). (c) Five different segments provide additional illustration of 3-D hand position reconstruction. All these sample segments were extracted from a much longer continuously decoded segment. (e,f,g) Same as in a, b and c, but for monkey G (session 1). (Kinematics have been normalized. See Table II for summary across sessions).

In many cases, best channel subsets contained a much smaller number of channels than the full recorded set (Table II). We did not observe any spatial pattern in the location of these best channel subsets on the microelectrode array. Impedances of selected channels varied broadly in the range of a few hundred K Ohms to about 2M Ohms. Furthermore, the improvement in the CC with the addition of new LFP channels was relatively small, especially for vertical position (Fig. 3d). For example, decoded vertical hand positions based on the best single channel and best channel subset were very similar (Fig. 5a). This small gain in decoding performance could be due to redundancy in information about kinematics across different LFP channels. In addition, this observation could potentially mean that the choice of channel matters little for decoding. To test this possibility, we computed the CC and RMSE for the decoded kinematics from each single channel in the 47 (96) channels from monkey C (G). As seen in Fig. 5b, the CCs and RMSEs varied broadly: for example the CC corresponding to vertical position in monkey C could reach values smaller than 0.1, a value within the chance level decoding as described above. Therefore, it matters which channels are selected. We also compared decoding performances based on power computed from different time window lengths (50, 100, 250 and 500 ms) and frequency bands (Fig. 5c and 6). Based on CC and RMSE, longer window seemed to provide slightly better decoding performance. CC and RMSE values for monkey C corresponded to: 0.68 and 11.1% (50 ms), 0.68 and 11.4% (100 ms), 0.73 and 11.2% (250 ms), and 0.74 and 11.3% (500 ms), respectively; and for monkey G: 0.45 and 12.2% (50 ms), 0.62 and 6.0% (100 ms), 0.67 and 12.0% (250 ms), and 0.69 and 12.1% (500 ms), respectively.

Fig. 5.

Decoding performance of vertical hand position (z) based on single versus multichannel bhfLFP power, and dependence on time window length. (a) A sample segment (session 1) of decoded vertical hand position based on bhfLFP power is shown. Best channel subsets consisted of 17 and 20 channels for monkeys C and G, respectively. (b) Histograms for CC and RMSE (in %) summarize the decoding performance for vertical hand position based on each of the recorded single LFP channels. CC and RMSE could vary considerably depending on the choice of channel. (c) Sample segments of vertical position decoding based on different time window lengths. Kinematics have been normalized.

Fig. 6.

Vertical hand position (z) decoding based on different frequency bands. Each plot shows an example of Kalman filter z-decoding (green curve) based on LFP power observations computed in a different frequency band (δ, θ, β, γ₁, γ₂ and γ₃). For comparison, grey and light red curves show the actual and the decoded position based on bhfLFPs (200–400 Hz), respectively. Examples refer to session 1. (See Table III for summary including results for both monkeys and sessions.)

We examined how the decoding performance based on bhfLFP compared to decoding performances based on the other frequency bands. Decoding examples of vertical hand position based on each of the 7 frequency bands selected here are shown in Fig. 6. We chose vertical hand position as the reference since that was the best decoded kinematic variable based on the bhfLFPs. Similar results were obtained for decoding of horizontal hand positions (x, y). Consistent with our mutual information analysis, decoding based on power in the bhfLFPs achieved the best performance (Table III). Nevertheless, decoding performance based on power in other high-frequency bands, such as γ₂ (50 – 100 Hz) and γ₃ (100 – 200 Hz), was also substantial.

TABLE III.

Decoding performance based on different frequency bands

Band	Decoding reach kinematics (vertical hand position)						Decoding grasp aperture
	CC		CC (mean)	RMSE (%)		RMSE (mean)	CC		CC (mean)	RMSE (%)		RMSE (mean)
	C	G		C	G		C	G		C	G
δ	0.47 0.36	0.63 0.75	0.55	16.9 20.6	12.9 13.8	16.1	0.38 0.30	0.56 0.59	0.46	16.0 15.1	13.4 14.8	14.8
θ-α	0.18 0.40	0.58 0.65	0.45	15.7 19.1	12.5 15.3	15.7	0.15 0.41	0.32 0.48	0.34	16.5 15.9	14.8 16.1	15.8
β	0.35 0.35	0.55 0.60	0.46	15.2 19.9	13.8 17.0	16.5	0.31 0.45	0.38 0.45	0.40	19.5 15.2	15.1 18.2	17.0
γ1	0.20 0.36	0.53 0.57	0.42	16.8 19.1	13.3 16.2	16.4	0.18 0.41	0.29 0.41	0.32	17.1 15.3	15.1 17.3	16.2
γ2	0.45 0.66	0.56 0.59	0.57	15.8 14.4	13.8 16.6	15.2	0.28 0.58	0.40 0.32	0.40	19.2 14.0	14.4 17.7	16.3
γ3	0.67 0.83	0.62 0.68	0.70	12.4 10.1	13.1 14.9	12.6	0.36 0.66	0.50 0.47	0.50	19.0 12.5	13.4 16.7	15.4
bhfLFP	0.73 0.84	0.67 0.72	0.74	11.2 10.8	12.0 14.1	12.0	0.54 0.68	0.59 0.67	0.62	15.9 12.4	12.4 12.5	13.3

Numbers in the top and bottom row of each two-rowed cell in the table correspond to values obtained from sessions 1 and 2, respectively. Mean values correspond to the average value across monkeys and sessions.

C. Decoding grasp aperture from LFPs

Beyond decoding hand 3-D endpoint kinematics during reaching movements, we also investigated whether features of grasping could be reconstructed based on LFP power in these different frequency bands. The same state-space decoding approach used above was applied to decode grasp aperture and aperture velocity. Although with smaller accuracy in comparison with hand position decoding, the decoded grasp aperture and aperture velocity, based on bhfLFP power, still captured most of the qualitative features of the actual trajectories (Fig. 7).

Fig. 7.

Kalman filter decoding of grasp aperture and aperture velocity based on bhfLFP power observations. Sample segments (session 1), decoded from test data, are shown for each monkey. Grasp aperture kinematics have been normalized. (See Table II for summary including results for both sessions.)

Summary statistics regarding decoding performance (CC, RMSE) are given in Table II. Qualitatively similar results to hand position decoding were also obtained when considering single versus multi-channel decoding (not shown), different time window lengths for power spectrum estimation (not shown), variation in CC and RMSE obtained from single channel decoding (Fig. 8), and different frequency bands (Table III). Overall our findings suggest that substantial information about grasp aperture is also available in LFP power in high-frequency bands and that this neural signal can potentially provide an important signal for grasp control in cortico-motor prostheses.

Fig. 8.

Grasp aperture: single channel LFP decoding. Grasp aperture was decoded via Kalman filter based on bhfLFP power observations from a single channel. The performances based on the individual decoding of each of 47 and 96 channels, for monkeys C and G, session 1, are summarized by the histograms. CCs and RMSEs varied broadly, emphasizing the importance of single channel selection.

IV. Discussion

Intracortical recordings provide a variety of signals generated by neural processes occurring at different spatial and temporal scales. These include single unit activity, multiunit activity and low and high frequency local field potentials. The encoding and decoding properties of these signals, and their application in neuromotor prostheses, is an important and recent topic of research in neuroscience and neural engineering. Here, we have demonstrated that intracortical field potentials recorded from primary motor cortex carry significant information about 3-D reach and grasp kinematics in able-bodied monkeys. Among the studied frequency bands, power in high γ and bhfLFP bands yielded the highest mutual information. Importantly, 3-D hand position and grasp aperture could be decoded with substantial accuracy from the power in these high-frequency bands. Neural decoding of kinematics based on the bhfLFP band outperformed decoding based on other, commonly defined, LFP frequency bands. Compared with SUA-based decoding performance on the same datasets [27], bhfLFP decoding performance is lower but close to that based on SUA. Overall, our findings indicate that high frequency LFP activity can potentially provide a useful neural control signal for applications in neural interfaces that aim at restoring movement in humans with tetraplegia or other motor disorders.

In a recent study, [15] explored decoding of 2-D hand position from different frequency bands in the range of 1 Hz – 6K Hz in human ECoGs. Similar to our findings, better decoding performance was obtained for a high-frequency band (300 Hz – 6K Hz). In contrast, computed average correlation coefficients [15, Table 1], as well differences across bands, were smaller than the ones obtained in our study. However, a direct comparison of these two signal sources (ECoGs vs. intracortical recordings) is difficult in this case due to additional differences in tasks and decoded kinematics (2-D vs. free 3-D reach and grasp as done here), and differences in signal processing and decoding algorithms.

Previous studies [26], [28], [29], [30] suggest that fluctuations in > 200 Hz LFPs observed in our study are likely to reflect multiunit activity and possibly spike after potentials, rather than potentials related to transmembrane currents directly produced by synaptic inputs. In particular, [14] defined MUA by a series of processing steps that result in a signal similar to a smoothed version of power fluctuations in bhfLFPs. In that case, band-pass filtering (300 Hz – 6K Hz) of recorded potentials was followed by the clipping of potentials smaller or larger than the mean ±2SD, therefore largely attenuating the contribution of both SUA and thresholded but unsorted spikes. The resulting potentials were then squared and lowpass filtered, resulting in a signal similar to smoothed bhfLFPs. Interestingly, [14] found that the MUA signal defined in this way provided better decoding of 2-D velocities than either sorted single-unit spikes or unsorted spikes. Their finding seems to indicate that, once the contribution of neurons very close to the electrode tip are attenuated, the resulting high-frequency broadband signal better represents the activity of a larger cluster of functionally related neurons and therefore achieve higher signal-to-noise ratio. Although we believe power in high-frequency LFPs in our study might share similar features with MUA as defined in [14], our data cannot (because of the original lowpass filtering at 500 Hz) distinguish the relative contribution of various types of potentials to power in high-frequency LFP bands [28], [29]. We hope to address this problem with new recordings and analyses in a future study. Regardless of the underlying sources, our results indicate that power in high-frequency LFPs (γ and bhfLFPs) provides control signals that carry significant information for the decoding of free 3-D reaching and grasping movements and are easy to obtain. For example, a simpler implementation (without the need of FFTs) that would allow for low power consumption and little on board signal processing, important for miniaturized wireless neural interfaces [31], [32], could consist of analog bandpass filtering of recorded potentials followed by averaging (analog lowpass filtering) of squared values.

It remains an open question how the stability of both recording and tuning properties of high-frequency LFPs compare to that of SUA. If, despite the lowpass filtering used here, the activity in these bands still primarily reflects spiking of large units near the tip of the electrode, we would expect this activity to have similar recording and tuning stability as that of SUA. According to several studies, a subset of isolated single units can be stably recorded over weeks even several months after array implantation [33], [34]. Using the same type of microelectrode array as the one used here, Dickey et al. [34] reported that 57% (43%) of the original units in their study remained stable over a period of 7 (10) days. On the other hand, if these bands reflect primarily MUA as defined in [14], we would expect their recording to be stable over longer periods. However, even in this case, the issue of how fast the tuning properties of activities in high-frequency LFPs change with time remains open. We also note that, because nonstationarities in the brain are likely to happen at many different spatial and temporal scales, the issue of tuning stability and the need for adaptive decoders in neural prostheses is unlikely to be restricted to a particular type of neural signal, but should be common to all types (SUA, MUA, slow and fast LFPs, ECoGs, EEGs, etc.).

Our analyses show that the decoding performance can be significantly affected by the choice of channel in the microelectrode array in either single or multichannel decoding. Therefore, the development of computationally efficient algorithms for channel selection will also be an important issue for real-time applications. Further, we also note that LFP features in time domain, involving slow and motor related potentials, might also carry significant information about 3-D reach and grasp kinematics. The ability to decode kinematics based on power in the δ band supports this possibility, consistent with previous findings in LFPs [10] and ECoGs [19]. LFP decoding algorithms based on mixed time and frequency domain as well as multiple (low and high frequency) bands are therefore also an important topic for future studies.

We believe high-frequency local field potentials will be important for functional electrical stimulation systems [35], and prosthetic or robotic arms [36], since this signal carries enough information to specify hand endpoint position and grasp aperture. Also, one might expect that real-time closed-loop control based on these signals might achieve higher performance than that demonstrated here. Our assessment of decoding performance was based on off-line analyses, i.e. without feedback correction. In real-time closed-loop applications, the user will actively correct for errors and, therefore, potentially achieve higher decoding performance than suggested by our offline analyses.

Biographies

Jun Zhuang (S’10) was born in Shanghai, China, in 1981. He received the B.S. degree in biomedical engineering from Shanghai Jiao Tong University, Shanghai, China, in 2004. He is currently working toward the Ph.D. degree at the Department of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.

From 2008 to 2009, he was a trainee at the Donoghue Laboratory, Neuroscience Department, Brown University. His current research interests include biomedical signal processing, neural prosthesis and pattern classification. Mr. Zhuang is a Student Member of the Society for Neuroscience.

Wilson Truccolo received the Ph.D. degree in complex systems from Florida Atlantic University, Boca Raton, FL, and the postdoctoral training from the Department for Neuroscience, Brown University, Providence, RI.

He is currently an Assistant Professor (Research) at the Department of Neuroscience, Brown University, where he is also with the Brown Institute for Brain Science, and a Research Fellow at the Department of Neurology, Massachusetts General Hospital, Boston. He is also a Principal Investigator in a Career Award from the National Institutes of Health - National Institute of Neurological Disorders and Stroke. His research interests include stochastic models of cortical dynamics and computation, statistical analysis of neural systems, and brain–machine interfaces.

CarlosVargas-Irwin was born in Cali, Colombia, on September 24, 1979. He received the B.S. (in neuroscience) as well as B.A. (in computer science) degrees from Brown University, Providence, RI, in 2002. He was a Laboratory Technician in Brown University, where he is currently a Graduate Student at the Department of Neuroscience. His research interests include cortical representations of movement, neuroprosthetics, and automated spike sorting.

John P. Donoghue (M’03) was born in Cambridge, MA, on March 22, 1949. He received the Ph.D. degree from Brown University, Providence, RI, in 1979, and the postdoctoral training from the National Institute of Mental Health. He is currently the Wriston Professor of Neuroscience and Engineering and the Director of the Brown Institute for Brain Science, Brown University. He is also with the Rehabilitation R&D Service, Department of Veterans Affairs, Washington, DC. His research interests include understanding of how the brain turns thought into movement and the development of the BrainGate neural interface system to restore control and independence for people with paralysis.

Prof. Donoghue is a Fellow of the American Institute for Medical and Biological Engineering and the American Association for the Advancement of Science.