Continuous decoding of human grasp kinematics using epidural and subdural signals

Abstract

Objective

Restoring or replacing function in paralyzed individuals will one day be achieved through the use of brain-machine interfaces (BMIs). Regaining hand function is a major goal for paralyzed patients. Two competing prerequisites for the widespread adoption of any hand neuroprosthesis are: accurate control over the fine details of movement, and minimized invasiveness. Here, we explore the interplay between these two goals by comparing our ability to decode hand movements with subdural and epidural field potentials.

Approach

We measured the accuracy of decoding continuous hand and finger kinematics during naturalistic grasping motions in five human subjects. We recorded subdural surface potentials (electrocorticography; ECoG) as well as with epidural field potentials (EFPs), with both standard- and high-resolution electrode arrays.

Main results

In all five subjects, decoding of continuous kinematics significantly exceeded chance, using either EGoG or EFPs. ECoG decoding accuracy compared favorably with prior investigations of grasp kinematics (mean± SD grasp aperture variance accounted for was 0.54± 0.05 across all subjects, 0.75± 0.09 for the best subject). In general, EFP decoding performed comparably to ECoG decoding. The 7–20 Hz and 70–115 Hz spectral bands contained the most information about grasp kinematics, with the 70–115 Hz band containing greater information about more subtle movements. Higher-resolution recording arrays provided clearly superior performance compared to standard-resolution arrays.

Significance

To approach the fine motor control achieved by an intact brain-body system, it will be necessary to execute motor intent on a continuous basis with high accuracy. The current results demonstrate that this level of accuracy might be achievable not just with ECoG, but with EFPs as well. Epidural placement of electrodes is less invasive, and therefore may incur less risk of encephalitis or stroke than subdural placement of electrodes. Accurately decoding motor commands at the epidural level may be an important step towards a clinically viable brain-machine interface.

Introduction

Hand function is critical to an individual’s ability to maintain independence and carry out activities of daily living. This function can be impaired, sometimes permanently, by disorders of the brain, spinal cord, and peripheral nerves, as well as limb amputation. Brain-machine interfaces (BMIs)—devices that establish a direct connection from the cortical signals that control movement to an end effector—represent one of the most promising means of restoring lost functionality to patients with these disorders. Function could be replaced, e.g., by controlling a prosthesis, or rehabilitated by using the BMI to drive cortical plasticity (as reviewed by Soekadar et al. 2015). Rehabilitative BMIs could be used for much larger target populations, such as patients with stroke or traumatic brain injury.

The earliest use of invasive BMIs relied upon the action potentials, or spikes, recorded from ensembles of single neurons (Kennedy and Bakay 1998, Chapin et al. 1999, Serruya et al. 2002, Taylor et al. 2002, Carmena et al. 2003). This technique has been applied in disabled human subjects (Hochberg et al. 2012, Wodlinger et al. 2014, Aflalo et al. 2015, Gilja et al. 2015) to grant them control over a computer cursor or robotic arm. However, it is difficult to record single unit spikes over long time periods (Simeral et al. 2011, Perge et al. 2014), due at least in part to electrode failure, shear injury, and inflammatory response (Barrese et al. 2013). Recent studies have shown that local field potentials (LFPs) can also be used to decode movement kinematics (Mehring et al. 2003, Stark and Abeles 2007, Zhuang et al. 2010, Bansal et al. 2011, Flint et al. 2012), or control an online BMI (Flint et al. 2013, So et al. 2014, Stavisky et al. 2015). LFPs have been shown to exhibit greater stability than spikes in nonhuman primates (Wang et al. 2014, Flint et al. 2016), and can provide information about movement intent even in the absence of spikes on the same electrodes (Flint et al. 2012). However, LFPs are recorded using the same intracortical (penetrating) arrays as spikes. This invasiveness, as well as the limited spatial extent of intracortical arrays, may reduce the utility of intracortical BMIs over long periods, particularly for rehabilitative applications.

Subdural field potentials, also called electrocorticograms (ECoG), are recorded from the surface of the pia mater using flat disc electrodes or polymer-based arrays (Chao et al. 2010, Viventi et al. 2011, Ledochowitsch et al. 2013, Chen et al. 2014). Because the cortex is not penetrated, ECoG arrays may provoke less immune response, which could prolong recording viability. In addition, ECoG is generated by tens to hundreds of thousands of neurons. Both of these factors could enhance signal longevity and stability. The ability to classify discrete hand postures using ECoG has been demonstrated in human subjects (Pistohl et al. 2012, Yanagisawa et al. 2012, Chestek et al. 2013). ECoG has been shown to provide a stable signal source for continuous, end-point kinematic decoding over long time periods in nonhuman primates (Chao et al. 2010). Continuous decoding of finger flexion-extension in humans using ECoG has been demonstrated (Kubanek et al. 2009, Acharya et al. 2010, Liang and Bougrain 2012, Nakanishi et al. 2014), while Wissel et al. (2013) and Hotson et al. (2016) successfully classified individual finger-tapping movements. However, continuous decoding of multidimensional hand and finger kinematics during naturalistic grasping has not yet been demonstrated with these signals (but see Agashe and Contreras-Vidal 2011 for a comparable EEG study).

Hand grasp involves the coordinated movement of a large number of degrees of freedom, and involves a widespread spatial representation on the cortical surface (Schieber 2002). ECoG arrays cover more surface area than most designs of penetrating electrodes. This allows the sampling of multiple cortical areas, though it can come with a tradeoff of lower spatial resolution, especially with standard clinical grid spacing (10 mm between recording sites).

Epidural field potentials (EFPs) are recorded from the outer surface of the dura mater, leaving the dura intact. This should correspond to a lower risk of bacterial encephalitis should the device become infected—about 5–6% risk over all craniotomies, with increased risk with device implants (Erman et al. 2005, Korinek et al. 2005, Abu Hamdeh et al. 2014). It also may reduce the risk of stroke or subdural hemorrhage (Slutzky et al. 2010). For these reasons, EFPs may be preferable to subdural ECoG if similar decoding accuracy can be achieved. EFPs have been used to decode arm movement kinematics in nonhuman primates (Flint et al. 2012, Shimoda et al. 2012, Marathe and Taylor 2013, Rouse et al. 2013) and in rodents (Slutzky et al. 2011). To date, no one has demonstrated the ability to decode grasp kinematics using EFPs, though Gharabaghi et al. (2014) demonstrated the ability to classify open vs. close of a virtual hand with EFPs.

In this study, we decoded continuous finger and hand joint kinematics of human subjects during grasping, using either ECoG or EFPs recorded from the same individuals. Previous studies that examined the effect of the dura and CSF on neural signals concentrated primarily on the relative power spectra or amplitude of the signals (Slutzky et al. 2010, Bundy et al. 2014, Olson et al. 2015). Here, we evaluated how useful these signals would be for BMIs by using them to decode hand movements.

Methods

Subjects and Electrode Placement

This study included 5 human participants (4 male, 1 female, ages 26–58, referred to in chronological order as S1 through S5). S1, S2, and S3 were undergoing extraoperative, intracranial seizure monitoring prior to resective surgery for treatment of medication-refractory epilepsy. S4 and S5 required intraoperative mapping during awake resection of low-grade gliomas. In both intraoperative subjects, the tumors were located remotely from areas related to hand grasp (S4: R prefrontal lobe; S5: R inferior parietal lobule), and no upper extremity deficits were observed in their neurological exams. All experiments were performed under protocols approved by the institutional review board of Northwestern University. All subjects gave written informed consent to participate in the study.

Electrode placement for S1–S3 was determined by clinical need. Subjects were recruited for the study if their monitoring arrays were expected to cover the hand area of primary motor cortex. During surgery, arrays were placed in reference to anatomical landmarks (“hand knob” in primary motor cortex), using intraoperative MRI navigation (Figure 1, Curve, BrainLab, Inc., Munich, Germany). In all subjects, the recording arrays covered parts of MI, premotor cortex and some S1 as well. We used arrays with 2.3-mm exposed contact area, 10-mm interelectrode spacing with holes in the silicone midway between contact sites (PMT, Inc). Epidural arrays were placed after the dura was closed, and were placed such that the contact sites were located over the holes in the subdural arrays. This arrangement minimized signal interference or attenuation in the epidural array due to the electrodes or silicone of the subdural array; however, the recording sites could not be placed exactly atop one another. Array locations were confirmed postoperatively using co-registration of pre-operative 1.5 T MRI and post-operative CT images. As in our previous work (Flint et al. 2014), we performed cortical surface reconstruction and electrode co-localization using a technique similar to Hermes et al. (2010). S2 was undergoing a repeat resection and had some subdural adhesions that prevented the placement of a subdural array.

Figure 1.

Locations of electrode arrays. Each cortical surface reconstruction was made from pre-operative MRI, co-registered with post-operative CT scans to localize the electrodes on the surface (see Methods). For S1 and S3, epidural (blue) and subdural (red) signals were simultaneously recorded; the arrays were offset to minimize attenuation of the epidural signals by the subdural arrays. No subdural array was placed for S2, who had subdural adhesions from a previous surgery. S4 and S5 were recorded intraoperatively; epidural and subdural signals were recorded sequentially not simultaneously. For S4, the position of the subdural array (red) was optimized after opening of the dura (see Methods). For S5, the subdural array was exactly aligned directly beneath the position that had been occupied by the epidural array. Arrays covered the hand area of M1, with some coverage of premotor and primary somatosensory cortex as well.

For S4 and S5, we used 8×8 higher density ECoG arrays containing 1.5 mm discs spaced 4 mm apart (Integra, Inc.). For these subjects, the hand area of the primary motor cortex was identified on preoperative MRI from both anatomic features (using BrainLab) and functional measures (fMRI during finger tapping, or mapping with transcranial magnetic stimulation). Intraoperatively, we placed the arrays epidurally by centering them over the hand region identified preoperatively. All epidural recordings for these subjects were completed prior to the start of subdural recordings. For S4, following dural opening the array was placed over the hand motor cortex, which was identified using navigation, direct visualization of the cortical surface, and stimulation (done for clinical mapping purposes). These optimizations were not possible at the epidural level, so the placement of the two arrays is slightly different (Figure 1). For S5, it is important to note that the subdural array was placed so that it occupied the space directly beneath the epidural array’s position (using intraoperative MRI-based guidance). In other words, the recording sites of both epidural and subdural arrays occupied the same positions over cortex in S5. As a consequence, only one array is shown for S5 in Figure 1.

Experimental Protocol

During each experimental session, photos on a monitor cued subjects to perform one of 3 distinct grasps: lateral, palmar, and tripod pinch (Figure 2). Each cue was presented for 3 seconds, followed by an inter-trial interval, which varied randomly between 3 and 3.5 seconds. During the inter-trial interval, the subjects were instructed to open their hands. The three cues were presented in random order (uniformly distributed), for 5 minutes per recording block (about 50 total trials per block). We recorded between 1 and 4 blocks per subject.

Figure 2.

Experimental protocol. Subjects were cued which grasp to perform by an image appearing on a monitor in front of them. The cue (onset at dashed line) lasted 3 seconds, and was followed by a randomly varying inter-trial interval (ITI) between 3 and 3.5 seconds in duration. Subjects were instructed to relax their hand to an open resting state during the inter-trial interval. Displayed are the three grasp types used: lateral or key grasp (left); tripod pinch (middle), palmar or cylinder grasp (right), and. During the experiment, the order of the cue presentations was varied randomly.

Signal Acquisition

For subject S1, ECoG and EFPs were recorded at 500 Hz (Nihon Kohden EEG-1100). For S2 and S3, ECoG and EFPs were recorded at 1000 Hz using a RZ2 Bioamp (Tucker Davis Technologies, Inc.). For S4 and S5, ECoG and EFPs were recorded at 2000 Hz using a Neuroport Neural Signal Processor (Blackrock Microsystems, Inc.). For all subjects, finger joint angles were recorded using a 22-sensor CyberGlove (Immersion) at the same sampling rate as the ECoG or EFP.

Decoding Continuous Grasp Kinematics

We decoded hand and finger kinematics from ECoG and EFPs, using techniques similar to those we have previously used to decode isometric grasp force (Flint et al. 2014). The CyberGlove recorded metacarpophalangeal joint angles for all 5 digits of the hand, as well as proximal and distal interphalangeal joint angles for the fingers, and the interphalangeal joint angle for the thumb. In addition, it recorded abduction/adduction angles between each pair of fingers, thumb rotation and palm arch. To reduce the dimensionality of the kinematic space, we first performed principal components analysis (PCA) on the 22 dimensional CyberGlove data. We then performed continuous decoding on the principal components in a similar way to a previous study of continuous isometric force (Flint et al. 2014). Briefly, we calculated time- and frequency-domain features from the ECoG and EFPs, including the time domain smoothed local motor potential (LMP; Mehring et al. 2004, Schalk et al. 2007) and the average spectral power in 5 frequency bands: 0–4 Hz, 7–20 Hz, 70–115 Hz, 130–200 Hz, and 200–300 Hz (power line frequency was 60 Hz). We used a 512-point discrete Fourier transform (DFT) to calculate spectral features. Power was calculated as the log-normalization (baseline subtracted log) of the square of the DFT amplitude. For subject S1, we did not use the power in the 200–300 Hz band, due to the 500 Hz sampling rate. Together, the LMP and spectral power bands comprised 6 features per channel (except for S1). We ranked the features from all electrodes by the absolute value of their mean correlation coefficient with the principal components, and selected the top 90% of features. Feature selection was performed on training data, not test data. We decoded the principal components (PCs) of the joint angles using a Wiener cascade decoder (Flint et al. 2014), which has been shown to improve decoding accuracy compared to a simple linear (Wiener) filter (Pohlmeyer et al. 2007). Decoding accuracy was measured using the fraction of variance accounted for (FVAF) between predicted and actual PCs. We calculated the FVAF using 11-fold cross-validation. To obtain the FVAF for each fold, we used 9 of the 10 folds to train the Wiener cascade decoder. The 10^th non-test fold was used to optimize parameter selection (Fagg et al. 2009). The 11^th fold was used to test the decoder performance. We selected values for the other parameters, such as the Fourier window length (256 points), bin size (0.1 s), and bin overlap (156 points), consistent with previous offline and online decoding of movement (Flint et al. 2012, Flint et al. 2013).

We calculated chance level decoding performance by applying the Wiener cascade decoder to phase-randomized versions of the input signals (Bansal et al. 2011). We repeated this procedure 1000 times for each recording, then calculated the mean decoding performance over all iterations. Unless otherwise noted, we used nonparametric statistics throughout our analysis, due to the non-Gaussian nature of our data. In each case, individual comparisons were adjusted for multiple-comparisons using the Bonferroni correction (Dunn 1961).

Results

In four human subjects, we decoded PCs of continuous hand and finger kinematics during grasp behaviors, using subdural (ECoG) and epidural (EFP) signals. In one additional subject (S2), we decoded these kinematics using EFPs alone. Table 1 lists details of the recordings for each subject.

Table 1.

Recorded sessions and trials for each subject. S1, S2, and S3 were recorded in the epilepsy monitoring unit. The subdural and epidural recordings for S1 and S3 occurred simultaneously (S2 was only recorded epidurally). S4 and S5 were recorded intra-operatively, sharply limiting the amount of time we were able to collect data. The epidural (EP) and subdural (S) records occurred sequentially in S4 and S5. The rightmost column shows the number of principal components that were necessary to account for 90% of the variance in the joint position data. In each subject, fewer than 1/3 of the 22 available principal components were necessary to account for >90% of the variance in the full data set. To facilitate comparison across subjects, we chose to include 3 PCs per subject in our decoding analysis

Subject	Blocks	Trials	# of PCs
S1	4	200	5
S2	4	199	6
S3	4	205	3
S4 - EP	1	51	4
S4 - S	1	51	4
S5 - EP	1	51	7
S5 - S	1	51	7

Representing Grasp with Principal Components

We used PCA to reduce the dimensionality of our joint position data while retaining most of the information about hand shape (Ingram et al. 2008). We found that 3 to 7 PCs were needed to account for 90% of the variance in the full joint position data set (Table 1). Similar to Santello et al. (1998), we found that decoding all joint angles from increasing numbers of PC projections yielded increasing accuracy (Figure 3, triangles). However, when using neural signals to decode PCs and then reconstruct joint angles, the decoding accuracy plateaued after 2 to 3 PCs (Figure 3, squares). Importantly, the accuracy using just 3 PCs was very close to that of the mean accuracy obtained by decoding all angles directly (Figure 3, filled circle). In the remainder of our analyses, we decoded the first 3 PCs only.

Figure 3.

Decoding performance vs. number of PCs. Decoding joint angle positions directly (filled circle, mean±SE over all joints) with ECoG yielded highly similar accuracy compared to decoding principal components with ECoG and then using an inverse PCA to recover joint angles (squares, mean±SE over joints). As expected, reconstructing the joint angles directly from the principal components was highly accurate when using 3–4 PCs (triangles, mean±SE over joints). Data shown is from S4’s ECoG, which enabled high decoding accuracy for PC2. We concluded that including 3 PCs offered a desirable compromise between low dimensionality and high accuracy of decoding.

Spectral Changes During Grasp Execution

In motor regions, there was an increase in high-frequency power during the time between cue presentation and movement onset (measured by a deflection in PC1; see Figure 4). This frequency range (approximately 70–300 Hz) is generally known as the “high gamma” band. High gamma power largely modulated in a phasic manner – increasing just prior to both flexor and extensor movement onset as the grasp closed and opened (Figure 4D). These patterns were also present on a single-trial basis (Figure 4A and B), though more subtly. The time course of changes in the spectral dynamics of EFPs closely resembled that of ECoG in most respects; notably, we did not observe any frequency-dependent attenuation of the signals.

Figure 4.

Spectral activity during grasp in single EFP (A,C) and ECoG (B,D) electrodes over precentral gyrus (subject S4). In A and B, the lower traces show examples of actual (blue line) and predicted (green line) first principal component (PC1) of joint angle. Upward deflections of PC1 denote closure of the hand. In C and D, the average (± SD) of PC1 across trials is superimposed on the averaged spectrogram, from −0.5 s to 4.5 s relative to cue presentation (vertical dashed line). High-frequency power increased just prior to the onset of grasp in both EFP (C) and ECoG (D). Low-frequency power, in contrast, increased at the release of the grasp and stayed elevated for 1 to 2 seconds.

At lower frequencies (~4–30 Hz), grasp onset was associated with a decrease in power in both ECoG and EFPs. Conversely, there was an increase in power (event-related synchronization; Pfurtscheller et al. 2003) at these low frequencies just after the grasp was released. This increase lasted approximately 1 to 2 seconds, and returned to baseline power level prior to the onset of the next grasp.

We measured the continuous decoding accuracy of ECoG and EFP by calculating the fraction of variance accounted for (FVAF) between actual and predicted PCs. As expected, PC1 (representing grasp aperture) decoding accuracy (FVAF) was higher than the FVAF for PC2 or PC3. Averaged across subjects, all three components were decoded at above chance level using both ECoG (p=10⁻⁷ for PC2, p=10⁻⁴ for PC3, Wilcoxon signed-rank test) and EFPs (p=10⁻⁴ for PC2, p=0.002 for PC3). See Figure 5. Individually, each subject’s PC1 decoding was significantly higher than chance. S1, S4, and S5 ECoG decoding was higher than chance for all three PCs. In addition, S1 and S2 had EFP decoding that was significantly higher than chance for all 3 PCs. Results of significance testing are presented in Table 2.

Figure 5.

Decoding accuracy for joint kinematics. Shown are the mean±SD of the first 3 principal components of the hand and finger kinematics. Red bars show subdural (ECoG) decoding, blue bars show epidural (EFP) decoding. The grand mean across subjects (±SE) is shown at the right. Downward pointing triangles at the top of the plot indicate prediction accuracies that were significantly different from chance (black solid and dashed lines, mean±SD) at the p=0.05 level. All subjects’ grasp aperture (PC1) was decoded well above chance level, and 4 of 5 subjects had above chance level decoding for all three PCs, for either ECoG or EFP.

Table 2.

Significance testing for kinematic decoding, comparison to chance. P-values are calculated with the Wilcoxon rank sum test (non-parametric t-test). P-values above 0.05 have been replaced with a hyphen

Subject	PC1	PC2	PC3
S1 - EFP	2*10−20	0.002	0.004
S1 - ECoG	4*10−26	8*10−11	5*10−5
S2 - EFP	5*10−23	4*10−6	1*10−8
S3 - EFP	2*10−21	-	-
S3 - ECoG	9*10−23	-	-
S4 - EFP	1*10−6	-	-
S4 - ECoG	1*10−6	2*10−6	0.01
S5 - EFP	1*10−6	0.006	-
S5 - ECoG	2*10−6	0.02	0.003
All - EFP	4*10−77	5*10−10	2*10−9
All - ECoG	2*10−61	2*10−11	2*10−9

Subdural vs. Epidural Decoding Performance

Considering all principal components, ECoG decoding was significantly better than EFP decoding (p=0.001, Kruskal-Wallis test). This difference, however, was largely attributable to the difference when decoding grasp aperture (PC1): ECoG decoding accuracy, averaged across subjects for PC1, was 0.54±0.05 (mean±SD). EFP mean (±SD) decoding accuracy for PC1 was 0.4±0.04. The difference for PC1 decoding was significant (p=10⁻⁶, Wilcoxon test). However, the higher order kinematic components, (PC2 and PC3), were decoded with similar accuracy for ECoG vs EFP (p=0.23 for PC2, p=0.62 for PC3, Wilcoxon test), indicating that ECoG signals were not necessarily better than EFP for carrying this information.

Given the heterogeneity of our subject pool, it was important to examine the differences in decoding accuracy on a subject-by-subject basis. We compared the ECoG decoding performance to the EFP decoding performance for each PC that was significantly different from chance in S1 and S3–S5 (7 total comparisons). Of this group, only S1’s PC1 showed significantly higher decoding accuracy for ECoG compared to EFP (p=10⁻⁸, Wilcoxon test). We note that S1 had substantial subdural hemorrhage on top of the subdural array, which likely interfered with signal quality in the epidural array.

In S1, S4 and S5, the agreement between the actual and predicted kinematics was greater than chance level for the first three PCs using ECoG, implying that both the grasp aperture (PC1) and the details of finger joint angles (PC2 & PC3) were represented. EFP-based decoding exceeded chance level for PC1 in all subjects. In addition, all three PCs were decoded above chance level for S1 and S2 EFPs. The grand mean over all subjects was greater than chance for all PCs (Figure 5, right).

Subdural vs. Epidural Spectral Information About Grasping

To explore how joint angle information was represented in different motor cortical frequencies, we calculated the decoding accuracy for each feature type separately (LMP, 0–4 Hz, 7–20 Hz, 70–115 Hz, 130–200 Hz, 200–300 Hz). For both ECoG and EFPs, the 7–20 Hz frequency band (μ/β) provided the best decoding, when considering all PCs, or only PC1 (Figure 6). However, when predicting PC2, the highest decoding accuracy was found in the 70–115 Hz (γ1) range for both ECoG and EFPs (Figure 6). For PC3, the decoding accuracy in γ1 was also higher than in μ/β (Figure 6). Successful decoding of continuous grasp aperture in μ/β suggests that there is more information represented in this spectral region than simple binary indication of movement state; the amount of hand openness is represented to some degree, as well. Increased accuracy of decoding PC2 and PC3 in γ1 might indicate that the higher order joint angle “details” of movement are better represented within these high frequency signals.

Figure 6.

Decoding accuracy within individual frequency bands. Plots show the mean±SE across subjects averaged over all PCs (left panel), and for individual PCs (right panels), for accuracy using ECoG (red) and EFPs (blue). High prediction accuracy for γ1 (70–115 Hz) is consistent with our previous findings in ECoG; here, EFP signals also performed similarly. For PC2 and PC3, there were no significant differences between ECoG and EFP decoding in any band.

In analyzing the frequency-band specific representation of movement, S5 merits particular consideration as an example of interest. With S5, we placed the recording array over the same cortical location epidurally and subdurally (see Methods); therefore, it is instructive to make direct comparisons of the ECoG and EFP recorded from this subject (Figure 7). By examining individual frequency bands, we found in S5 that γ1 provided the most accurate decoding, regardless of whether we measured decoding over all PCs, or considered each PC separately (Figure 7). The other high gamma bands (γ2:130–200 and γ3:200–300 Hz) also show higher performance for PC2. PC3 EFP decoding was not significantly above chance level for this subject, and so those data were excluded from this analysis.

Figure 7.

Frequency-band specific decoding accuracy in subject S5. Plots show the mean±SD averaged over all PCs (left panel), and for individual PCs (right panels), for accuracy using ECoG (red) and EFPs (blue). For PC3, EFP data was not decoded above chance, so it is not shown. Consistent with our results across subjects (Figure 6), this subject’s difference between ECoG and EFP decoding accuracy was mainly attributable to a difference in decoding PC1. Notably, the highest performance was seen when decoding with the γ1 band, which is consistent with previous studies.

Discussion

We decoded continuous hand and finger joint kinematics during natural grasping movements, using both subdural (ECoG) and epidural (EFP) signals. Overall, we found that both ECoG and EFPs enabled decoding performance exceeding chance level decoding for the first three principal components of movement (Figure 5). This indicates that both grasp aperture (PC1) as well as the higher-order joint angle details represented by PC2 and PC3 can be extracted from either ECoG or EFPs. Our ability to accurately decode kinematics with ECoG compares favorably with prior investigations that examined similar movements (Kubanek et al. 2009, Acharya et al. 2010, Liang and Bougrain 2012, Nakanishi et al. 2014). Further, we compared the decoding performance using ECoG vs. that using EFPs. The use of EFPs to decode continuous grasp kinematics has not previously been demonstrated. In most cases, EFPs enabled decoding accuracy that was close to, or statistically indistinguishable from, decoding accuracy using ECoG. One exception (for PC1 in subject S1) was likely caused by the subdural bleeding on top of the subdural array. Statistically, any superiority of ECoG over EFPs in decoding grasp kinematics appeared to be due to better decoding of grasp aperture (PC1). When decoding finer details of movement (PC2 and PC3), there were no significant differences between ECoG and EFPs.

Our decoding results support and extend a body of research (reviewed in the Introduction) that has investigated the cortical representation of hand movements (Acharya et al. 2010, Hotson et al. 2016), postures (Yanagisawa et al. 2012, Chestek et al. 2013), and grasp types (Pistohl et al. 2012). Here, we decoded continuous whole hand grasping movements and linked to motor control literature (Santello et al. 1998, Ingram et al. 2008) showing that the many degrees of freedom inherent to finger movements can be represented by a few principal component representations. These results can also be seen as validating prior modeling studies, which have not necessarily predicted a worsening of decoding accuracy due to the dura’s presence (Slutzky et al. 2010, Bundy et al. 2014), even despite lower amplitude signals (Ramon et al. 2014).

Analyzing continuous-valued potentials such as ECoG (and EFPs) involves decomposing the neural signals into features calculated from either the continuous time series (here, the LMP), or the power in various spectral domains. Crone et al. (1998) are largely responsible for the concept that motor behavior is encoded in ECoG by synchronization of high-frequency oscillations. Many subsequent studies have corroborated the idea that low-frequency (μ/β: here, 7–20 Hz) power encodes movement onset while higher frequency modulations correlate with more precise details of movement (Rickert et al. 2005, Hwang and Andersen 2009, Aggarwal et al. 2013, Pistohl et al. 2013, Williams et al. 2013, Perel et al. 2015). Our findings are largely consistent with this idea. We found that the μ/β and γ1 (70–115 Hz) spectral bands decoded overall movement with the highest accuracy (Figure 6). For decoding PC1—which largely represents grasp aperture—the μ/β band had superior performance compared to other bands, when considering all subjects together. This difference was most pronounced at the epidural level. For PC2 and PC3, the γ1 band generally performed best for both ECoG and EFPs. The high performance of the γ1 band is consistent with earlier results demonstrating the ability to decode isometric grip force and EMGs using ECoG (Shin et al. 2012, Chen et al. 2014, Flint et al. 2014), and the high amount of information in high gamma regarding movement in general (Stark and Abeles 2007, Zhuang et al. 2010, Flint et al. 2012, Flint et al. 2012, Liang and Bougrain 2012).

One way of minimizing clinical complications, and possibly speeding adoption of BMI technology, is utilizing epidural implants. Decoding even two or three degrees of freedom, as we demonstrate here, may provide enough control to be useful, if the control strategy properly exploits redundancies in the movements of the fingers during grasp. This may especially hold true for low DOF and rehabilitative BMI applications. Alternatively, it may be possible to utilize existing clinically approved devices, such as the Neuropace Responsive Neurostimulation system (Morrell 2011), to obtain long-term implantation of a surface-electrode device that is capable of providing functional gains for paralyzed individuals.

In the two subjects with intraoperative recordings, we observed generally higher decoding accuracies. These differences may have arisen due to more advantageous array position. Moreover, we also used higher density arrays with these two subjects, and high-density arrays have been shown to provide superior decoding performance compared to traditional clinical array spacing (Wang et al. 2016). This may account in part for the generally higher performance in decoding PC2 and PC3 in these subjects (Figure 5). We previously found that higher-density arrays enabled better distinction between isometric pinch force produced by different fingers (Flint et al. 2014). The electrode spacing on these arrays still does not achieve the optimal spatial resolution of surface arrays, which is likely on the order of 1–2 mm (Slutzky et al, 2010; Ledochowitsch et al. 2013). As array technology matures and even higher-density, multiplexed arrays become available, further enhancements in decoding performance may be achievable with surface recordings.

To construct a neuroprosthetic that provides maximum utility for performing activities of daily living, it will be necessary to decode multiple types of grasps and other hand postures. The first principal component is thought to largely represent whole-hand grasp aperture, regardless of grasp type (Santello et al. 1998). This means that PC1, while it is a single degree of freedom (DOF) in PC space, summarizes a complex combination of three-dimensional finger joint movements in Cartesian space. Accurate decoding of PC1 is therefore a necessary, but not sufficient, requirement of a successful hand neuroprosthesis. By contrast, decoding individual joint angles for all degrees of freedom in the hand and fingers would require a BMI user to master an output space with approximately twenty dimensions. When considering how to train BMI decoders in paralyzed patients, this is likely to be extremely unwieldy, if not impossible. Further, cortical representations of individual finger movements, though widespread, are thought to overlap (Schieber 2002). Therefore, controlling all aspects of all the fingers’ movements may be prohibitive. Instead, using our current data, we estimate that accurate decoding of two or three principal components of movement would offer a more acceptable tradeoff between precision of control and ease of use. In such an approach, it is crucial to decode higher order PCs in addition to PC1. In three of the four subjects with ECoG recordings, we were able to decode both PC2 and PC3 at above chance level. In four of five subjects with EFP recordings, we were able to decode PC2 above chance, and in two of those subjects we could also decode PC3 above chance with EFPs. These higher order PCs were decoded more effectively with higher-density grids as well. These results offer promise that the details of grasp, and not only its aperture, can be controlled reliably in a BMI using cortical surface recordings.

The next steps forward to continue these investigations will include a demonstration of online control. In recent years, it has been demonstrated that discrete classification techniques can be used with ECoG to provide command signals to a robotic controller (Yanagisawa et al. 2012, Fifer et al. 2014, Hotson et al. 2016). It remains to be seen whether control of this type will increase BMI users’ sense of embodiment and agency with the prosthetic. Ultimately, the design requirements for a clinically viable neuroprosthetic are likely to include continuous proportional control over finger position, using a decoding scheme similar to the one demonstrated in this study. Other probable design criteria include the ability to specify isometric force, and a capacity for somatosensory feedback in order to manipulate objects.