Simultaneous Neural Control of Simple Reaching and Grasping with the Modular Prosthetic Limb using Intracranial EEG

Abstract

Intracranial electroencephalographic (iEEG) signals from two human subjects were used to achieve simultaneous neural control of reaching and grasping movements with the Johns Hopkins University Applied Physics Lab (JHU/APL) Modular Prosthetic Limb (MPL), a dexterous robotic prosthetic arm. We performed functional mapping of high gamma activity while the subject made reaching and grasping movements to identify task-selective electrodes. Independent, online control of reaching and grasping was then achieved using high gamma activity from a small subset of electrodes with a model trained on short blocks of reaching and grasping with no further adaptation. Classification accuracy did not decline (p<0.05, one-way ANOVA) over three blocks of testing in either subject. Mean classification accuracy during independently executed overt reach and grasp movements for (Subject 1, Subject 2) were (0.85, 0.81) and (0.80, 0.96) respectively, and during simultaneous execution they were (0.83, 0.88) and (0.58, 0.88) respectively. Our models leveraged knowledge of the subject's individual functional neuroanatomy for reaching and grasping movements, allowing rapid acquisition of control in a time-sensitive clinical setting. We demonstrate the potential feasibility of verifying functionally meaningful iEEG-based control of the MPL prior to chronic implantation, during which additional capabilities of the MPL might be exploited with further training.

I. Introduction

Reaching to and grasping objects is an important skill that forms the basis for many activities of daily living (ADLs). It is thus an important target for brain-machine interfaces (BMIs) being developed for patients with impaired limb function due to neurological lesions of motor pathways (e.g., spinal cord injury, amyotrophic lateral sclerosis, stroke, etc.). Recent work has demonstrated that grasp types [1], grasp timing [2], hand postures [3], and reach parameters [4] can be decoded from spectral changes in human intracranial electroencephalographic (iEEG) signals, and that movement-related spectral modulation of iEEG can be used for online control of BMIs, for example during dexterous grasping [5], when selecting between grasp types and elbow movement [6], or for three dimensional cursor control [7]. We therefore sought to determine whether human iEEG could be used to provide simultaneous and independent online control of reaching and grasping movements, thus demonstrating segregation of these two movement types at the spatial scale of iEEG macroelectrodes. iEEG is an attractive platform for the development of BMIs because of the potential for better long-term signal stability than multi-unit recordings, as well as the availability of subjects who have accepted the risks of electrode implantation for the mapping of their seizure onset zones prior to epileptic resection surgery [8].

Relative to scalp EEG, iEEG provides better spatial resolution and better signal quality for high frequency activity [9, 10]. There is substantial empirical evidence from local field potential studies in humans [11] and nonhuman primates [12] that this high frequency activity closely tracks population firing rates. The degree of control that can be achieved with the large-scale population activity recorded with iEEG [13, 14] is unknown, however, especially with chronic training beyond the time constraints of seizure monitoring. To ensure that the risk of long-term electrode implantation is offset by the benefit of stable long-term BMI use, it would be advantageous to confirm at least basic control of the intended prosthetic at the time of implantation. Previous work in scalp EEG and iEEG has demonstrated two- and three- dimensional cursor control where at least one dimension is controlled by behavior unrelated to the task at hand (e.g., vocalization or tongue movement) [13-16]. Although it has been demonstrated that training and “operant conditioning” can be used to learn BMI control on the time-scale of months [17], it is unclear to what extent an unnatural mapping will scale up to more complex tasks in more complex environments. We therefore sought to determine whether the command signals for forward reaching and grasping of the Johns Hopkins University Applied Physics Lab (JHU/APL) Modular Prosthetic Limb (MPL) could be derived from high frequency (70-110 Hz) neural population activity associated with naturalistic reaching and grasping movements, respectively. These commands were interpreted by the hardware in the MPL and converted to multi-axial anthropomorphic movements spanning two controllable joints for forward reaching and 10 controllable joints for grasping.

II. Methods

A. Subject Info

The subjects for this study were 55 year old (Subject 1) and 30 year old (Subject 2) right-handed males implanted with intracranial electrodes to map the ictal onset zone of medically resistant seizures prior to surgical resection. In Subject 1, an 8x8 grid of subdural platinum-iridium electrodes (Adtech, Racine, WI, 2.3 mm diameter exposed surface, 1-cm spacing) were surgically implanted over right frontal-parietal regions (see Fig. 1), in addition to a 4×5 electrode grid over right lateral occipital cortex and a 1×8 electrode strip stretching from right mid-temporal regions to dorsolateral prefrontal cortex (both not shown). Subject 2 was implanted with a 1×8 electrode strip (Adtech; Racine, WI; as above) across right frontoparietal cortex, six depth electrodes with eight platinum macrocontacts each (Adtech; 2.41-mm long, 6.5 mm center-to-center spacing) placed medially from the right premotor area to the posterior parietal lobe, and one hybrid depth with eight platinum macrocontacts (Adtech; 1.57 mm long, 5 mm center-to-center) and sixteen microcontacts (Adtech; 75 micron diameter). Neuronavigation via the Cranial Navigation Application (BrainLab; Westchester, IL) was used during placement of the depth electrodes in Subject 2. Anatomical reconstructions of the subjects' brains with the location of implanted electrodes were generated by volumetrically co-registering the pre-surgical MRI with a post-surgical CT using BioImage (Fig. 1A) [18]. Subject 1's seizures began after a bout of viral encephalitis with coma at 33 years of age. His complex partial seizures were typically preceded by a somatosensory aura in his left hand with spread to the face and subsequent shaking of the left hand, and were sometimes followed by secondary generalization. Subject 2 had previously undergone chronic recording with partial resection of his right post-central gyrus and superior parietal lobule. Both patients gave informed consent for research testing, which was done in accordance with a protocol approved by the Institutional Review Board of the Johns Hopkins Medical Institutions.

Fig. 1.

Functional mapping of cue-averaged task-related high gamma activity in training set. (A) Reconstruction of the implanted grid location for Subject 1 is depicted; the electrode used for reaching (number 25) is highlighted in red and corresponds to the channel circled in red in the activation maps below, while the electrode used for grasping (number 11) is highlighted in blue and similarly corresponds to the electrode circled in blue below; the central sulcus is highlighted in green. (B) Reconstruction of the depth electrodes implanted in right hemisphere of Subject 2; electrodes used for reaching highlighted in red, electrodes used for grasping highlighted in blue (transparent medial view in inset). (C, D) Each task map displays the spatiotemporal distribution of significant increases (red spectrum) or decreases (blue spectrum) in high gamma energy relative to precue baseline in 16 ms windows for Subject 1. Each row corresponds to a different iEEG electrode in the frontoparietal grid displayed in (A). All times are relative to cue onset. (E) A differential map is shown for Subject 1, which is the result of a Wilcoxon test between two conditions for each (channel, time) pair with FDR correction for comparisons across multiple time points within each channel. Channel and time pairs are in the red spectrum if forward reach is more activated than grasp, and in the blue spectrum if grasp is more activated than forward reach. The average times of relevant behavioral events are marked with vertical lines and labeled (movement onset, MO; pressed target button, PT; released target button, RT; returned arm to home position, Home; released pressure bulb, Rest).

B. Neural Signal Acquisition

Using a NeuroPort system (BlackRock Microsystems; Salt Lake City, UT), iEEG signals were initially sampled at 30 KHz with an analog bandpass filter with cutoffs of 0.3 Hz and 7500 Hz. The NeuroPort system then applied a digital 4th order Butterworth lowpass filter with a 250 Hz cutoff and downsampled to 1000 Hz. Artifactual channels were visually identified and excluded from all further analysis. Acquired iEEG signals were broadcast over UDP to an experimental workstation, where they could be accessed for online spectral feature extraction and model evaluation to drive the MPL.

C. Experimental Procedures

Short offline data sets of 30 (Subject 1) or 50 (Subject 2) auditorily cued trials were collected each for forward reaches and grasping movements. Audio cues of “reach” or “grasp” were delivered via external speakers by E-Prime software (PST, Inc.; Sharpsburg, PA). For Subject 2 only, the reach and grasp trials were interspersed with “Reach and Grasp” trials, of which there were also 50. The onset of each trial was manually initiated by the experimenter to ensure that the preceding reach was completed and an additional varying delay had passed before a cue was given. Behavioral states were detected using analog sensors sampled at 1000 Hz on the same hardware as the neural data: 1) the onset and offset of each reaching movement were detected using a pushbutton embedded in a wooden lap desk, 2) the termination of each reach on a distal target was detected using a pushbutton, and 3) the onset and offset of each grasp were detected using a pneumatic squeeze bulb connected via flexible tubing to an electronic pressure sensor. A detailed schematic of the experimental setup is included in Fig. 2, and a video of the training procedure is shown for Subject 1 in Supplemental Video 1. Reaches by Subject 1 ranged in duration from 1.3 to 1.8 seconds (median = 1.4 seconds) with response latencies ranging from 330 to 500 ms (median = 410 ms), while grasps (i.e., as detected by the squeeze bulb) ranged in duration from 0.6 to 2.7 seconds (median = 0.9 seconds) with response latencies ranging from 380 to 930 ms (median = 460 ms). Reaches by Subject 2 ranged in duration from 1.9 to 4.8 seconds (median = 2.7 seconds), with response latencies ranging from 450 to 1450 ms (median = 790 ms), while grasps ranged in duration from 0.8 to 3.5 seconds (median = 2.0 seconds) with response latencies ranging from 640 to 2070 ms (median = 1010 ms).

Fig. 2.

Schematics and photographs of experimental setup with MPL. (top) A schematic of the experimental setup is shown, with Subject 1 seated and interacting with three behavioral sensors. The MPL is to the front and right of the subject, in the same room as and in full view of the subject. Traces of the behavioral sensors, high gamma power, and MPL commands during a three trial segment are shown as an example. (A-C) The subject is seated on his hospital bed (not pictured, right of view), with his arm at rest on a lap desk with inset pushbutton or “home switch.” The subject is holding but not actively grasping the squeeze bulb used to query grasp status. On the subject's hospital tray are a pushbutton for reach offset detection, and a laptop displaying a red bar indicating pressure exerted on the squeeze bulb. (A) In the background, the MPL is at its baseline state (rest posture). (B) The subject is executing a grasp movement, and (C) the subject is executing a reach movement.

D. iEEG Electrode Evaluation

Following collection of the reach and grasp datasets, event-related high gamma activations were analyzed. The audio cue played to the subject was split and fed into the BlackRock system; the beginning of this cue was detected and used as a stimulus onset (SO) marker. The 1024 ms prior to SO was pooled into baseline distributions for each channel, while the 3072 ms following the onset of the audio cue was used as a post-stimulus epoch. The 1024 ms prior to SO and 3072 ms following SO were segmented into 128 ms windows with 112 ms overlap. Each window was reduced to a single estimate of the high gamma analytic amplitude in a 16 ms bin using a Hilbert transform with an embedded, flat-top Gaussian bandpass filter with cutoffs of 72 and 110 Hz. Separate distributions were created for each post-stimulus 16 ms time bin and channel and referenced to the channel baseline distributions using two-sample t tests with significance threshold p < 0.05. The thresholds for p-value significance of these tests were corrected for multiple comparisons within each channel using the false discovery rate (FDR) correction [19]. Any resulting significant p-values were then log₁₀ transformed, and any significant modulation was labeled as an increase or a decrease. This resulting matrix of statistical significance measures therefore contained timing information about activation that was used to exclude channels which displayed modulation in response to the audio cue. This entire analysis was performed with custom MATLAB (MathWorks, Inc.; Natick, MA) software, from which the results were available within the experimental session (see Fig. 1).

E. BMI Model Training

For Subject 1, a final training set was recorded in which the verbal commands “reach,” “grasp,” and “reach and grasp” were pseudo-randomly chosen and played to the subject via external speakers with E-Prime; this training set contained 46 trials and lasted approximately five minutes. For Subject 2, the 150 trials spanning approximately sixteen minutes collected for electrode evaluation were used as a training set. Also for Subject 2, the initially trained model was used to drive a virtual version of the MPL as visual feedback during an additional 120 trials (i.e., 40 each of “reach,” “grasp,” and “reach and grasp”). The iEEG and behavioral data recorded during this block were used as the training set for online testing.

Signals in each training set were first spatially filtered with a common average reference [20] of all channels not excluded by visual inspection because of artifact or noise. Autoregressive power was extracted from the streamed signals using the Burg algorithm with model order 16 on a 400 ms window. The logarithm of the spectral power from components between 72.5 and 110 Hz were then averaged to yield an estimate of the broadband high gamma power. In offline data collection for model training purposes, feature extraction windows were overlapped by 300 ms.

In Subject 1, one electrode each was chosen for reach and grasp using information from the functional maps of post-stimulus activation. The high gamma log-powers during movement and rest movement were compared to manually establish a threshold for movement classification. In Subject 2, four channels each were selected as model inputs to separate binary linear discriminant analysis (LDA) classifiers for reach and grasp. In addition, transition probabilities were adjusted manually before the testing session to smooth the output from the classifier. For this study, we used a probability of 0.95 for the probability of a rest classification if currently at rest (i.e., 0.05 for a movement classification), and 0.8 for the probability of a movement classification if currently in the movement state (i.e., 0.2 for a rest classification).

F. JHU/APL Modular Prosthetic Limb

Developed by JHU/APL under the Defense Advanced Research Project Agency (DARPA) Revolutionizing Prosthetics Program, the MPL (Supplemental Fig. 1) is an advanced upper-body extremity prosthetic and human rehabilitation device [21]. The MPL has 17 controllable degrees of freedom (DoF) and 26 articulating DoF in total (Supplemental Fig. 1, with specifications and architecture details in Supplementary Methods). To facilitate control from neural decoded motion intent, the MPL has a custom software interface, VulcanX, that receives movement/motion commands locally and sends them over a controller area network (CAN) bus to a limb controller (LC) board in the hand of the MPL [22]. Three types of high-level control commands, passed through VulcanX, are fused together to form individual actuator commands by the LC: 1) Degree of Motion Control (DOM) commands, which allow each degree of motion to be controlled individually with position and/or velocity commands; 2) Endpoint Control (EP) commands, which allow the hand's position and orientation to be controlled in Cartesian space using a Jacobian-based algorithm for computing inverse kinematics; and, 3) Reduced Order Control (ROC) commands, which allow pre-programmed hand grasp patterns to be actuated in a coordinated fashion as a single degree of freedom [23, 24]. EP velocity and ROC commands were utilized to control reach and grasp, respectively, in this study.

G. Online Testing

Once the high gamma thresholds for movement were established, classification outputs from the trained models of reach and grasp movements were simultaneously used to actuate the MPL via the VulcanX interface. Whenever the classifiers predicted that the subject was reaching and/or grasping, the MPL was commanded to reach and/or grasp, respectively, at a set rate. If the either reach or grasp classifier predicted that the subject was resting, the limb was commanded to return to its rest arm or hand posture, respectively, at an equal rate. For Subject 2 only, the return rate for reaching was adjusted to be 50% higher than the forward rate. High gamma log-power calculations were performed in 400 ms windows (i.e., as in training) computed as quickly as possible on the streaming iEEG signals to provide inputs to the trained model (i.e., 11 ms for Subject 1, slowed to 32 ms for Subject 2 purposefully to avoid inundation of the MPL). Both subjects completed three blocks of online trials by performing the same overt movements with their native limbs as during the training set. In Subject 1 only, the second and third blocks were separated by a battery of physical and imagined movements that were not analyzed as a part of this study.

H. Quantitative Evaluation of Control

The physical movement blocks lasted approximately 4, 11, and 13 minutes for Subject 1 and 11, 15, and 10 minutes for Subject 2 (respectively). The MPL VulcanX control software created a log of commands sent to the limb with timestamps, which was compared offline to the timestamps of salient cues and behavioral events recorded by the BlackRock system (e.g., subject leaves the home switch, subject grasps the squeeze bulb, etc.). Trials were designated as starting 500 ms prior to the earliest of the reach and/or grasp onsets and ending 500 ms prior to the onset of the next trial. For each trial, we recorded the proportion of correct commands (e.g., the percentage of ‘grasp’ commands with a positive velocity when a physical grasp was performed) in a window of equal length to the corresponding physical movement duration for that trial. To account for variable response latencies by the subject and an inconsistent system latency, the start of the window relative to the onset of the trial was selected individually for each trial to maximize the accuracy. For reach-and-grasp trials, durations and latencies were selected separately for the reach and grasp components. As a control, a window whose length equaled the average duration of the reaches or grasps was used to compute the peak reach or grasp command accuracy in grasp-only and reach-only trials, respectively. Accuracy for each trial was computed as the average of the single trial sensitivity (i.e., proportion of reach or grasp commands within the selected movement window) and the single trial specificity (i.e., proportion of rest commands outside of the selected movement window). The median reach command accuracies for reach-only vs. grasp-only and reach-and-grasp vs. grasp-only and the grasp command accuracies for grasp-only vs. reach-only and grasp-only vs. reach-and-grasp were compared using a nonparametric two-sided Wilcoxon rank sum test.

III. Results

Both subjects were able to attain a high degree of subjective control over reaching and grasping with the MPL across the experimental session with no model adaptation while moving their native limbs (Supplemental Video 2 and 3). Furthermore, both subjects were able to achieve a level of performance throughout the experimental session that was qualitatively similar to the first block.

We investigated the spectrogram of modulation time-locked to salient stimuli and behavioral events to validate our choice of the high gamma band for online control. As shown in the functional mapping results (Fig. 1), the electrodes used for control of the MPL exhibited robust high gamma modulation. Fig. 3 shows the time-frequency response of the reaching electrode during reach-only and reach-and-grasp trials of the online task, as well as the grasping electrode during grasp-only and reach-and-grasp trials. High gamma modulation in the reaching electrode occurred within the frequency range of 80-160 Hz for Subject 1, in contrast with the more spectrally restricted 60-120 Hz modulation in the grasping electrode. Subject 2 displayed activation at a lower frequency range, centralized around 40-90 Hz. These frequency ranges of power modulation show that while our choice of 72.5-110 Hz for control may not have exactly matched the neurophysiological response to the task, it did capture a substantial amount of the power modulation for both tasks. The temporal envelope of activation was relatively restricted in the reaching electrode for both subjects, with mean power modulation peaking roughly 200 ms before the onset of movement. Subject 2 had similarly tight timing in grasp-related cortical activation. In contrast, power modulation in the grasping electrode of Subject 1 began an average of 300 ms prior to movement onset and peaked more than 300 ms after movement onset. The reach-related high gamma power modulation also differed from grasp-related power modulation in the presence of two distinct temporal peaks, time-locked to outward reach and the subsequent return to rest. Fig. 3 (bottom row) provides verification that gamma power modulations in the grasp and reach electrodes were markedly lower during execution of reach and grasp, respectively.

Fig. 3.

Average change of power spectral densities (PSD) relative to baseline, aligned to movement onset. (A) Reach and grasp electrodes are shown for Subject 1, and (B) two representative electrodes are shown for Subject 2. The first vertical dashed line in all plots corresponds to the average time the audio cue began. For each trial, the baseline was chosen from before the onset of the cue (leftmost dashed line). The solid line denotes movement onset (MO). In reach trials, the dashed lines after the solid line correspond to the average time of the reach completion (pressing target button, PT), release of the target button (RT), and return to home (resting on the home switch), from left to right. The rightmost dashed line in the grasp trials corresponds to the average time of grasp completion. The PSD's were computed via autoregressive spectral analysis. Window size did not allow for accurate calculations at 0-7.5Hz, so these frequencies are not displayed.

During online control in Subject 1, we observed that control of grasping was less reliable for reach-and-grasp trials than grasp-only trials. Fig. 3 (middle row) shows that high gamma modulation in the reaching and grasping electrodes during reach-and-grasp trials was qualitatively reduced relative to reach-only and grasp-only trials. To evaluate this effect, log high gamma power was extracted in 300 ms around the onset of movement. Statistical analysis revealed that log power in the grasping electrode around the onset of grasp was significantly higher in grasp-only than in reach-and-grasp trials (p < 0.05, Wilcoxon test); the log power in the reaching electrode around the onset of reach was not significantly different in reach-only and reach-and-grasp trials, however. Identical analyses performed in Subject 2 did not reveal any significant differences in movement-related power modulation between reach-and-grasp trials and either the reach-only or grasp-only trials in any of the electrodes used for control (p > 0.05, Wilcoxon test, Bonferroni-corrected).

To evaluate the high gamma power modulation associated with movement state in reach-only trials, log power was also extracted in time windows around the onset of stable hold and the onset of return, in addition to a baseline window preceding the cue. For Subject 1, log-power in the reach electrode was significantly higher in the reach window and return window than in the hold window, all of which were significantly higher in the baseline window (p < 0.05, oneway ANOVA with Tukey's honestly significant difference post-hoc). In all four electrodes used for reaching control in Subject 2, median hold activity was lower than median reaching and returning activity; the difference was significant in three out of four electrodes (p < 0.05, one-way ANOVA, with Tukey's honestly significant difference post-hoc). Reaching, returning, and intermediate hold windows similarly exhibited higher levels of high gamma activity than baseline windows in Subject 2.

Classification accuracy for both reaching and grasping started and remained high throughout all three blocks of the online task. The mean reach classification accuracy across all trials was 86% (Subject 1) and 82% (Subject 2) for reach-only trials; the reach accuracy across reach-and-grasp trials was 83% (Subject 1) and 89% (Subject 2). The mean grasp classification accuracy across all grasp-only trials was 81% (Subject 1) and 96% (Subject 2); the grasp accuracy across reach-and-grasp trials was 55% (Subject 1) and 88% (Subject 2). The evolution of classification accuracies showed no significant effect of block (p > 0.05, one-way ANOVA) in either subject. The trial-by-trial reach and grasp accuracies are depicted in Fig. 4. Reach accuracies were significantly higher than chance for both reach-only trials and reach-and-grasp trials (p < 0.05, Wilcoxon test with Bonferroni correction), while grasp accuracies were significantly higher than chance for grasp-only trials (p < 0.05, Wilcoxon test with Bonferroni correction), but not reach-and-grasp trials in Subject 1 only (p=0.078, Wilcoxon test). Grasp accuracies were significantly higher in grasp-only trials than in reach-and-grasp trials in both subjects (p < 0.05, Wilcoxon test). Reach accuracies were not significantly higher in reach-only trials than in reach-and-grasp trials for Subject 1 (p > 0.05, Wilcoxon test), although reach accuracies were higher in reach-and-grasp trials than in reach-only trials for Subject 2 (p > 0.05, Wilcoxon test).

Fig. 4.

Limb performance accuracy metrics. (A, B) The accuracies are shown for reaching and grasping during trials where reach and grasp were executed simultaneously. (C, D) The reach and grasp accuracies are shown for reach and grasp only trials, respectively. The vertical dashed lines in A-D denote separate blocks. Distributions are shown and summarized with boxplots of the peak sensitivities for grasps in Subject 1 (E), reaches in Subject 1 (F), grasps in Subject 2 (G), and reaches in Subject 2 (H). Each distribution is comprised of the peak sensitivities from each trial. Bars above the boxplots with asterisks mark distributions with significantly different medians (p < 0.05, Wilcoxon test).

To investigate whether reaching and grasping were indeed independent, sham sensitivities were calculated as a control; reach sensitivities were calculated during grasp-only trials and grasp sensitivities were calculated during reach-only trials (Fig. 4). Since no physical reaches took place in grasp-only trials, nor physical grasps during reach-only trials, the average reach and grasp durations were used as surrogates. Peak reach sensitivities were significantly higher in cued reach-only and reach-and-grasp trials than in cued grasp-only trials for both Subjects (p < 0.05, Wilcoxon test); reach sensitivities were significantly higher in reach-only trials than in reach-and-grasp trials for Subject 2 (p < 0.05, Wilcoxon test) but the difference was not significant in Subject 1 (p = 0.16, Wilcoxon test). Peak grasp sensitivities were higher in cued grasp-only and reach-and-grasp trials than in cued reach-only trials for both subjects (p < 0.05, Wilcoxon test); grasp sensitivities were significantly higher in grasp-only trials than in reach-and-grasp trials for Subject 1 (p < 0.05, Wilcoxon test) but the difference was not significant in Subject 2 (p = 0.15, Wilcoxon test).

IV. Discussion

We were able to provide two human subjects with control of the MPL using a control scheme that exploited individual functional anatomy, i.e., the population responses in cortical regions used for control of each subject's native arm. This allowed our subjects to achieve control without extensive training. To identify iEEG control sites and characterize their response selectivity, we used iEEG functional mapping during reaching and grasping. By using electrodes over cortical areas that were differentially activated during reaching and/or grasping, we were able to afford the patient independent control over the reaching and grasping functionalities of the MPL arm. We showed that these two movements, when executed individually, elicited cortical responses in the gamma band that generalized to their simultaneous execution, although the same responses occurred with a reduced magnitude.

Additionally, the subject's control over the arm did not wane over the course of three separate blocks using thresholds derived from a short training block. Models were equally effective across blocks with no adaptation or retraining, providing evidence that control was achieved by accurately detecting the naturalistic circuits for reaching and grasping, not via adaptation or operant conditioning. Reach and grasp commands were controlled independently, suggesting functional segregation of these movements at the spatial scale of clinically routine iEEG electrodes. There is abundant evidence from experiments in non-human primates that reaching and grasping engage different networks of cortical areas [25]. As in non-human primates, human premotor cortices engaged by reaching are likely dorsal to those engaged by grasping [26]. As expected, the iEEG site activated by and used for control of reaching was dorsal to the site activated by and used for control of grasping.

Our BMI used event-related high gamma power augmentation as an index of task-related neural activity during physical movements. This choice was based on a body of literature which demonstrates that high gamma band modulation is an index of cortical processing in humans [27-30] and recent experimental evidence that high gamma power changes are strongly and positively correlated with the firing rates of neuronal populations in close proximity to recording electrodes [11, 31-33]. Our findings are consistent with empirical evidence that compared with power changes in other frequencies, high gamma power augmentation has high spatial selectivity with respect to task-related cortical activation, such that adjacent iEEG electrodes can yield signals with greater independence at higher frequencies [34, 35]. High gamma responses are also robust enough to be detected in single trials [36], a necessary requirement for BMI applications. Furthermore, several studies have shown that high gamma features extracted from human iEEG outperform corresponding lower frequency features for offline motor decoding [1, 37] and online BMI control [6, 14, 38].

This study focused on the control of reach and grasp in the MPL since they are fundamental to upper limb use, which provides a proof of concept for the systems-level integration groundwork necessary for more complicated and dexterous tasks. Reaching and grasping movements were decoded for actuation of the MPL with high accuracy and stability; furthermore, this was achieved in a clinical epilepsy monitoring setting under time constraints that did not allow for long-term training or testing. Although this prohibited testing the long-term stability of MPL control, it did demonstrate the feasibility of obtaining MPL control within a compressed timeframe, which could have important clinical benefits. Specifically, it would be highly advantageous to demonstrate acceptable brain control of a neuroprosthetic at the time of surgical implantation, in order to verify the placement of electrodes and troubleshoot any technical difficulties at the time of the operation. Noninvasive methods of functional mapping (e.g., fMRI) can be used to perform gross surgical planning, but intraoperative verification of control with iEEG would be extremely useful to refine the final implantation site. This would help to avoid the need for re-implantation because the patient is unable to control the neuroprosthetic. This would be both costly and increase surgical risk. Although the total time for our experiment was longer than that of an awake craniotomy, most of this time was due to experimental setup and troubleshooting, and thus could be reduced with additional practice.

We observed during online testing with the subject that it was fairly common for the MPL to exhibit a secondary reach or grasp as the subject returned to the resting position. This corresponded to a burst of high gamma activity as the subject initiated return of his limb to the home switch or as the subject relaxed his hand after squeezing the bulb. This was best demonstrated in the reaching trials by post-hoc offline analysis of the high gamma power in windows associated with reaching to, holding at, and returning from the distal reach target, which demonstrated a higher degree of modulation for reaching and returning than for the intermediate holding in the reaching electrode for Subject 1 and for a subset of the reaching electrodes in Subject 2.

This report provides additional evidence for the potential utility of iEEG as a source of control signals for BMIs. Although the participants in this study did not suffer from upper limb paralysis, we believe that the technique of rapid trial-averaged spatiotemporal mapping of high gamma modulation can be used to identify sites that are activated when subjects with motor impairments attempt to perform movements. These patients often have residual motor function and could attempt to move with assistance, be moved passively, or observe upper limb movements in a trial-based framework.

A large amount of decoding and BMI success has been achieved using command signals derived from iEEG [6, 7, 13, 14, 38]. Although iEEG macroelectrodes [13, 14], iEEG microelectrodes [39], and multi-electrode arrays [40, 41] have all been used to demonstrate effective BMI control in small populations, no large-scale longitudinal studies have compared the tissue response and control performance between these classes of implants. Much previous work has illustrated a significant redundancy of motor encoding at the single neuron level [42], suggesting that population activity could be useful for prosthetic control. Nevertheless, there is evidence from studies in motor, perceptual, and cognitive systems that the richness of encoding increases with improvements in spatial resolution (i.e., iEEG macroelectrodes exhibit coarser encoding than iEEG microelectrodes, and iEEG microelectrodes exhibit coarser encoding than local field potentials from multi-electrode arrays) [43]. It is possible that as the spatial resolution of iEEG implants improves and more comparative studies are done between iEEG and multi-electrode arrays, that iEEG implants for BMI control will be an attractive option for some patients [44, 45]. In the meantime, iEEG recordings in patients undergoing epilepsy surgery will continue to serve as a platform for demonstrating the degree of useful control that can be achieved without extensive training, prior to chronic implantation of iEEG electrodes for BMIs.