Decoding movement intent from human premotor cortex neurons for neural prosthetic applications

Summary

Primary motor cortex (MI), a key region for voluntary motor control, has been considered a first choice as the source of neural signals to control prosthetic devices for humans with paralysis. Less is known about the potential for other areas of frontal cortex as prosthesis signal sources. The frontal cortex is widely engaged in voluntary behavior. Single neuron recordings in monkey frontal cortex beyond MI have readily identified activity related to planning and initiating movement direction, remembering movement instructions over delays, or mixtures of these features (Kurata & Wise, 1988; Boussaoud & Wise, 1993; Crammond & Kalaska, 1994, 2000). Human functional imaging and lesion studies also support this role (Toni et al., 1999; Simon et al., 2002). Intraoperative mapping during deep brain stimulator placement in humans (Benabid et al., 1989) provides a unique opportunity to evaluate potential prosthesis control signals derived from non-primary areas and to expand our understanding of frontal lobe function and its role in movement disorders. Here we show that recordings from small groups of human prefrontal/premotor cortex neurons can provide information about movement planning, production and decision making sufficient to decode the planned direction of movement. Thus, additional frontal areas, beyond M1, may be valuable signal sources for human neuromotor prostheses.

INTRODUCTION

A neuromotor prosthesis (NMP) is a device intended to provide movement signals from the brain so that neurologically impaired humans can interact with their environment. Several studies have shown that neurons in primary motor cortex (MI) of monkeys (Serruya et al., 2002; Taylor et al., 2002; Carmena et al., 2003) and humans (Goldring & Ratcheson, 1971; Kennedy & Bakay, 1998; Kennedy et al., 2000) could provide movement-related signals to control assistive devices for paralyzed humans. However, other motor areas may provide alternative or additional information. Andersen and colleagues demonstrated that signals in parietal cortex of monkeys could provide a command that signaled upcoming movement intent for specific movement directions and expected reward value (Musallam et al., 2004). Large extents of frontal cortex outside of MI are active in movement planning (Kurata & Wise, 1988; Fu et al., 1995; Toni et al., 1999; Crammond & Kalaska, 2000; Harrington et al., 2000) and movement intent (di Pellegrino & Wise, 1993; Rao, et al., 1997; Kalaska & Crammond, 1995; Crammond & Kalaska, 2000), suggesting that useful movement signals may be available in these areas as well. Indeed, premotor and primary motor areas in the monkey provide different types of movement information (Hatsopoulos et al., 2004). While monkey and human frontal cortex appear to be functionally similar, the movement-related properties of neurons in human frontal cortex have not been extensively studied.

The use of single neuron mapping in conscious humans during neurosurgical procedures provides a valuable opportunity to record single cortical neurons while humans perform motor tasks. The majority of human neuronal recording studies have been carried out in temporal (Ojemann & Schoenfield-McNeill, 1999), ventral prefrontal (Kawasaki et al., 2001) and cingulate cortex (Hutchison et al., 1999; Williams, et al., 2004; Davis, et al., 2005). The few recordings of human MI support the existence of movement-related activity in this area (Goldring & Ratcheson, 1971; Kennedy et al., 1998), but the behavioral correlates of neurons in human frontal cortex have not been examined. Thus, with this study we assessed the movement-related information within small groups of neurons randomly recorded from non-primary motor cortical areas as a means to judge the suitability of these areas for use in neuroprosthetics. Because our sample size was small and recording sites somewhat heterogeneous, our attempt was not to fully assess the fundamental function of this cortical region in humans but to examine the movement information contained within such imperfect samples.

METHODS

Participants

Neurophysiological cortical recordings were performed in three patients undergoing elective Deep Brain Stimulator (DBS) electrode implantation surgery at Rhode Island Hospital, Providence, RI. These subjects had movement disorders that were non-responsive to medication; two suffered from Parkinson's disease (P1,P2) and one from essential tremor (P3). Deep brain stimulator targets were the subthalamic nucleus and the VIM thalamic nucleus, respectively (Benebid et al., 1989). The experimental paradigm occurred at the beginning of the DBS implantation surgery, before neurophysiological mapping of the basal ganglia began. Participants were alert during this procedure and off any Parkinson's medications, which is the standard protocol for DBS implant procedures. This study was submitted and approved by the Institutional Review Boards of both Brown University and Rhode Island Hospital, and every effort was made to ensure the participants comfort. Table 1 details the clinical description of the patients and the cortical recording coordinates.

Table 1.

Details of the three patients and the recording locations, tasks and analysis.¹

Patient ID	Patient 1	Patient 2	Patient 3
Diagnosis/Age/Sex	PD/63 yrs/male	Essential Tremor/74yrs/female	PD/ 51 yrs/male
# Cells	6	4	5
Entry point, re: AC: Lateral Anterior Superior; Brodman's Area; Gross anatomical location	38 mm 17.8 mm 60.4 mm BA 6/8 Middle Frontal Gyrus	32.2 mm 15.6 mm 59.9 mm BA 6 Middle Frontal Gyrus	29.0 mm 5.0 mm 57.9 mm BA 6 Middle Frontal Gyrus
Tasks	4-Direction	4-Direction	Go/NoGo Direction
Able to classify: (maximum % classification)	Yes (46%)	Yes( 43%)	Yes-83% (Direction) Yes-63% (Go/NoGo)

¹PD ‘Parkinson's Disease’; AC ‘Anterior Commissure’.

Neurophysiological Recording

Recordings of simultaneous single- and multi-neuron activity were made at the premotor or prefrontal cortical entry point of a standard trajectory planned for the DBS surgery through a 14 mm burrhole with the dura retracted. Five yoked tungsten microelectrodes (Frederick Haer) were inserted using the Alpha Omega microdrive and recording system (Alpha Omega Engineering, Nazareth, Israel), and recordings began within a few millimeters of the cortical surface. Impedances measured with the AO system once the electrodes were recording in cortex were between 0.5 and 1.5 MOhms. The electrode insertion proceeded at 5 to 50 micron steps depending on visible cellular activity. All five electrodes moved simultaneously and were not independently adjustable. Individual neurons were discriminated online whenever possible; however multi-unit activity was accepted, as well. Usually, one to three neurons were discriminated per channel, and between 4 and 6 total single- and multi-unit cells were discriminated per participant. Neuronal responsiveness to tactile stimulation and passive and active limb movements was tested, yet responses were not robust. In an effort to be timely, neurons were discriminated quickly, yet as accurately as possible. Once discriminated, the electrodes were not moved for the duration of the behavioral experiment (10 – 20 minutes).

When possible, the trajectory for the implanted DBS electrode (and, hence, the trajectory of the recording electrodes) was planned to traverse a gyrus parallel to a sulcus to maximize the depth of cortical tissue traversed by the recording electrodes. Reconstructions of the entry points (Figure 1a) indicated that recordings were near the Area 6/8 border (P1) and within Area 6 (P2 and P3), as depicted schematically in Figure 1b (Talaraich & Tournoux, 1988). Figure 1c depicts the location of the recording sites in each patient on a sagittal MR scan (round circles in cortex). All recording sites were lateral to the superior frontal sulcus, immediately rostral to the region of the M1 hand representation area (Yousry, et al., 1997; Matsumoto et al., 2003) where imaging studies indicate arm activation (Matsumoto et al., 2003). Table 1 describes the neuronal yield and location for each patient.

Figure 1.

Intraoperative recording sites. a, Left, The entry points of the three participants plotted on the rendered brain of P1. Red circle: P1; Yellow circle: P2, Blue Circle: P3. P2's coordinates are 9 mm more medial and 12.8 mm more posterior than P1's entry point. P3's cortical entry point was 9 mm more medial and 12.8 mm more posterior than that of P1. CS, central sulcus; PCS, precentral sulcus, SFS, superior frontal sulcus, IFS, inferior frontal sulcus; “The Knob” area as described by Yousry and colleagues (Yousry et al., 1997). b, Approximate locations of 3 Brodman's areas in frontal cortex, as estimated by Talaraich atlas (Talaraich and Tournoux, 1988) and surface anatomy. c, Coronal slices from Turbo Inversion Recovery sequences used for neurosurgical planning, detailing projected trajectory between entry point and basal ganglia target structure (subthalamic nucleus or ventralis intermedius nucleus of the thalamus) for each participant.

Behavioral System and Tasks

A custom system consisting of a graphics tablet and pen and computer displays was used for behavioral tasks. Subjects held a digitizing pen and moved it on a horizontal tablet positioned under the right hand next to the body. The movement of the pen moved a cursor displayed on a vertical computer screen approximately one meter in front of the patient.

Two similar behavioral paradigms were implemented. The first was the classic radial ‘Center Out’ task using four targets, in which a cursor is moved from a center target to a radially-displaced target 10 cm away and back, after a variable delay (Figure 2a). This task was used with the first two participants, and typically 10 to 15 trials were obtained to each target. Figure 2b displays typical trajectories obtained from one participant during the experiment. The second task was a simple ‘delayed response’ task, in which the participant performed a paradigm similar to the first, although was asked to pay attention to and remember for a short delay a colored instruction cue which signaled either ‘Go’ or ‘No Go’ for the upcoming trial (Figure 2c). Direction of movement was first signaled at the cue to move; thus, memory during the delay was not for direction, but for whether the trial was one requiring movement or not. This task was implemented with the third participant (P3) in an effort to increase neuronal responses in the more prefrontal area. Total experiment time was strictly limited to one hour, including searching for and discriminating units, with task performance lasting between 10-20 minutes.

Figure 2.

Behavioral tasks a, Four-direction Center-Out task used by Patients 1 and 2. b, Superimposed hand trajectories of one patient for movements in all four directions during one experiment. c, Go/NoGo task used by Patient 3. Left, Behavioral ‘Go’ task, indicated by the green dot. Right, ‘No Go’ indicated by red dot. Note the subject must remember if the trial is a ‘go’ or ‘no go’ trial, but has no indication of direction during the delay. IS1: Instructional Stimulus 1; IS2: Instructional Stimulus 2.

Data Analysis

To determine the ability to decode movement intent (direction or ‘Go – No Go’), the number of spikes in a specific time window was analyzed using a maximum-likelihood (ML) classifier. We modelled the likelihood of observing a spike count (U) given a condition (C), P(U|C), as a Gaussian distribution with mean and variance calculated from the observed spike counts of the neuron in a time window under the appropriate condition. In cases in which more than one unit was used, the activities of the units were assumed to be independent. The ML estimator is optimal in the sense that it is unbiased and has minimum variance (Deneve et al., 1999) if the prior probability of direction or go vs. no go are uniform. Moreover, it is a particularly attractive approach to decoding when the number of recorded cells is small. Although our choice of an ML classifier was motivated by our desire to extract as much information about movement intent as possible from the small number of recorded neurons, it should be noted there are biologically realizable neural networks that could implement such a decoder (Sanger, 1996, 1998; Zhang et al., 1998; Deneve et al., 1999).

To evaluate the performance of the decoder, a cross validation approach was used. First, one trial was singled out as the test trial, and the rest of the trials were used to calculate the mean and variance of the Gaussian likelihood functions for all the units. The estimator was then applied to the isolated trial to predict the condition that maximized the probability of the observed spike count:

$$\mathrm{C}={\mathrm{argmax}}_{\mathrm{c}}\left({\Pi }_{\mathrm{i}}\mathrm{P}({\mathrm{U}}_{\mathrm{i}}\mid \mathrm{C})\right)$$

This procedure was repeated for all trials and the number of correct classifications noted. This ultimately yielded a percentage of correct predictions for each cell or cell grouping for direction and/or movement intent, which could be compared to a binomial distribution to obtain a significance level. For example, for the 4-direction classification, the parameters of the binomial distribution (p, N) were set to 0.25 and the total number of trials, respectively. The same analysis was performed for a control period in which the relevant cue was not yet revealed to the participant. The test time periods were as follows: P1: 750 ms post instruction cue, P2: 600 ms post-instruction cue, P3: 500 ms post-instruction cue for the 4-direction task and 2 s post instruction for the Go/NoGo task. Control period epochs were of the same size but preceding the instruction cue, except for P3 (Go/NoGo) in which the control period was the 1s preceding the instruction, due to the experimental design. We chose test periods to optimize results – our rationale being that in trying to glean as much information as possible from the data, it was valid and desirable to choose the windows based on performance.

All possible neuronal combinations for each subject were examined, from single neurons alone to the entire neuronal ensemble together. As a second test of significance the expected distribution of classifications due to chance alone was calculated using a random shuffle procedure. The trials' event labels associated with the firing rates of each cell or cell combination were shuffled randomly and the ML analysis run again. This procedure was repeated 1000 times for each cell combination, and the results were used to construct a 1000-point histogram of the correct classification percentages due to chance. Each cell and cell combination's observed prediction was compared to its own 1000-point shuffle histogram, and only if the observed data prediction was equal to or greater than 95% of the shuffled data points (p<0.05) was it deemed ‘significant.’

RESULTS

Time limitation and inability to independently move electrodes contributed to the low number and isolation quality for recorded neurons. The majority (10/15) of the recorded waveforms discussed here appeared to be single units based upon shape consistency and refractory period, but we also included waveforms likely to be a mixture of units (Figure 3a). Table 1 details the number of individual units discriminated for each participant. Peri-movement aligned histograms of neural activity suggested task-related activity, although the depth of modulation was modest (see Figure 3c,d). Approximately 50% of neurons weakly modulated with direction whether aligned on instruction (53%, 8/15) or movement onset (47%, 7/15). One pair of waveforms recorded on the same electrode appeared to be functionally connected based on cross correlation analysis (Figure 3b). Cell 1 from this pair was rather weakly directionally tuned exhibiting decreased firing rate modulation for upward movements (p< 0.05, Student's t-test, see Figure 3c). Cell 2 did not modulate with direction and was multi-unit. Figure 3d shows example rasters and histograms for a neuron from P3 recorded during the Go/No Go task, aligned on the instruction cue.

Figure 3.

Human cortical activity. a, Examples of neural activity. Left, oscilloscope-like trace (vertical bar=50 mV, horizontal bar= 50 ms). Right, corresponding discriminated neurons from those traces (bar=1 ms). 1,2 = two discriminated units (one single, one multi-unit) recorded from the same electrode in Patient 2. b, Autocorrelograms from the same two units (above) and cross-correlogram (below). These two were recorded on the same electrode and appear to be functionally connected based on cross correlation. c, Rasters and histograms from Cell 1 in a, aligned on IS1 during 4-direction task. Arrows indicate direction moved, red bar indicates 500-ms window tested for differences with direction. * indicates significant difference when tested with other directions (p< 0.05, Student's t-test). d, Perievent rasters and histograms from one neuron (Patient 3) during ‘Go’ and ‘No Go’ trials aligned on IS1 (green triangles). Clear arrows point to visible differences in firing rates in the two conditions.

Activity patterns of neuron groups were tested to see if these modulations were sufficient to predict movement direction (P1-3) or intent (P3 only). We used a maximum likelihood classifier to compute the probability of the observed spike count conditioned on each outcome; the class with the highest probability was compared to the actual behavior (Kleinbaum et al., 1988; Sanger, 1996, 2003) and correct predictions tallied. For both the directional and the intent to move (Go/NoGo) tasks, we compared spike counts in windows after the instruction cues when information was known, to a control window before the instruction was given. The results were examined for each individual neuron as well as for all possible subsets of neuron ensembles. The percentage correct value for each cell combination was compared first to the binomial probability distribution and counted as significant if it was greater than or equal to the 95% criterion. Secondly, it was tested for significance using a bootstrapping procedure comparing prediction performance to random shuffles of the data (See Methods).

Decoding of Movement Direction and Intent

Both direction and intent to move were predicted from the spiking of premotor neurons (Figure 4a, green diamonds). In all three participants, single cells and multi-cell ensembles predicted movement intent significantly (p < 0.05, binomial distribution and shuffle procedure). Significant direction predictions in P1 ranged from 35% to 46% (mean 38%, ±SD 3.4%, p<0.05, binomial distribution). In this case a single neuron successfully provided the highest prediction; in fact, only one of the ten additional neuron ensembles with significant predictions did not contain this neuron. In P2 significant direction predictions for cell ensembles ranged from 35% to 43% accuracy (mean 38%, ± SD 2.8%) with the highest prediction from a 3-cell combination (p<0.05, binomial distribution); likewise, in P3 the mean significant prediction was 47% ±SD 7.4% and ranged from 42% to 63% with the highest prediction from a 2-cell combination (p<0.05, binomial distribution).

Figure 4.

Maximum likelihood analysis results for all patients and tasks. a, Classification percentages for all possible combinations of neurons in periods after presentation of the relevant cue (green diamonds) and in control periods before the presentation of the cues (red circles). The graph is split into columns according to the number of cells that make up the combination. Significant cell combination predictions fall in the yellow-shaded area (binomial distribution, p<0.05). Significant predictions with the shuffle procedure (p< 0.05) have an additional dark outline. Black line connects mean percentage correct prediction (o) for each column. b, Histograms of distribution of the 1000-shuffle classification results for one cell combination from each participant and task, as noted in a. Red line denotes actual percentage correct prediction for cell combination (not shuffled). Blue line= p< 0.05 criterion (shuffled histogram), c, Predictions for the same 4 cell combinations as in b in the test epoch (red and yellow) and for the control periods (blue). Note: P1 had no control period. The red, dashed line indicates expected chance level for each task.

Classification was further validated by evaluating its success during a time window before the subject was provided with the task parameter (P2 and P3 only). In a random 4-direction task, one would predict 25% correct classification by chance. As shown in Figure 4a during the hold period before the direction task with P2 and P3 classification yielded predictions (red circles) which were considerably lower, and, except for three combinations, non-significant (p>0.05, binomial distribution). For both P2 and P3 the mean prediction accuracy was significantly higher after the instruction was known (38±3% and 47±7%, respectively) than before (30 ±8.2% and 32 ±4%, respectively; p<0.05, paired t-tests). The task design was not appropriate to make this comparison in P1.

Movement intent (Go/NoGo) could also be predicted from the neural activity in the one participant tested (P3). Whether the upcoming intent was to move or not move was predicted with significantly greater than chance levels during the hold period after the instruction was given in 20/31 possible cell combinations with an average accuracy level of 73%±6% (p<0.05, binomial distribution), and one three-cell combination predicted correctly in 83% of trials tested. During the control period when the participants had no knowledge of the upcoming cue, trials were correctly classified by the same cell ensembles with a mean accuracy level of 51% (± SD 6%) which was significantly lower than during the test period (p<0.05, paired t-test) and not significantly different from 50% chance level (p>0.05, t-test). Although two control period predictions successfully predicted the outcome for 64% of trials, the probability of observing 2 such predictions by chance out of the total 31 cell combinations with the significance level of p=0.05 is 26% (binomial distribution).

Small clusters of neurons as well as certain single cells provided better than chance predictions of upcoming behavior. Two of the three neurons which yielded best predictions in isolation, also improved prediction when included with other neurons, whether the additional neurons were significant alone or not. If no single neurons classified significantly (e.g., P2 and P3 for the direction task), then groups of neurons improved classification greatly, suggesting that the ensembles provide useful information when constituent neurons cannot. The black circles with connecting lines in Figure 4a show the means of all predictions for each neuron or neuron group, thus showing the trend in predictive value when the number of neurons in the groups was increased.

The expected distribution of classifications due to chance alone was calculated using a bootstrapping shuffle procedure. Here, each significant data point was tested against its own 1000-trial shuffle histogram created by shuffling trial events randomly with respect to the firing rates for each cell ensemble followed by a recalculation of the ML analysis. Figure 4b shows examples of predictions (using the optimal cell selection) from each participant (red line) compared to the results one would expect if predictions were made by chance alone (blue shuffle histogram). The blue line shows the p<0.05 significance level for each histogram. The shuffle procedure was a more stringent significance criterion than the binomial distribution, although the two criteria generally yielded similar results (see Figure 4a).

Figure 4c shows data from four separate cell combinations (one from each experiment) which had the highest prediction and/or the largest difference between the control and test periods. The test periods for both the direction (red bars) and Go/NoGo task (gold bar) are significantly higher than one would expect by chance alone (dotted red lines). Predictions for the control periods for P2 and P3 when no movement information was known fall near chance level (blue bars). For the P1 data set no control period was available.

DISCUSSION

Our results suggest that neurons in human premotor cortex, like those of the monkey, contain information about movement direction and intent (Fu et al., 1993; Messier and Kalaska, 2000). During the preparatory phase, the discrete classifier was able to predict with many individual cells and cell groups which of the four directions was forthcoming and whether the intention was to move or withhold movement. The three recording sites were in the middle frontal gyrus and appeared to lie from the area 6/8 border to more central regions of area 6, as predicted from Talaraich coordinates. Although it can be difficult to differentiate the boundaries of frontal agranular architectonic fields even with direct histological analysis, the recordings were anterior enough to be clearly from premotor not primary motor cortex. The recording region was immediately anterior to the hand/arm area of the primary motor cortex, consistent with areas where arm related neurons are found in monkeys and arm activated regions can be identified in human fMRI studies (Fink et al., 1997; Matsumoto et al., 2003). When coupled with the observation that these neurons contained information about hand movement, we can conclude that the posterior part of the middle frontal gyrus in humans commonly contains neurons related to hand motion. The data set is too small to know whether any classification differences relate to differences in areal location.

These findings may have important implications for the development of neuromotor prostheses in humans. These devices attempt to provide a substitute motor output from the cortex when movement output is blocked, as in spinal cord injury or degenerative nerve or muscle diseases, such as muscular dystrophy or amyotropic lateral sclerosis. We have shown it is possible to predict intended actions from a very small set of nearly randomly sampled neurons in the premotor cortex. Primary motor cortex can provide control signals suitable to perform two and three dimensional visuomotor tasks in monkeys (Serruya et al., 2002; Taylor et al., 2002;Carmena et al., 2003) and more recently in a preliminary report in humans (Serruya et al., 2004). Our work extends the areas of frontal cortex which could be useful sources of cortical control signals to premotor areas and shows that useful information about the decision to move may also be obtained.

It may be advantageous to utilize neurons in non-primary motor areas for control signals for prosthetic applications. Neurons in premotor and prefrontal areas more frequently exhibit early discrete aspects of planning, such as the desire to respond, as well as the direction of upcoming motion when compared to MI neurons, which are superior at coding continuous position (Hatsopoulos et al., 2004). Signals from the parietal cortex of monkeys code earlier, more cognitive information about a movement, such as the goal of an upcoming reaching movement and expected reward-value of a movement (Musallam et al., 2004). Thus, planning signals from prefrontal, premotor or parietal areas might be useful in place of MI if MI is damaged or in combination with MI cortex activity for neuromotor prostheses.

In this study, prediction rates from decoding were less than would be desired for a practical human device. However, the current sampling of neurons was limited by the small number of electrodes used. Chronically implanted arrays capable of recording dozens of cells (Paninski et al., 2004) have been tested in monkeys (Serruya et al., 2002; Suner et al., 2005) and in a human (Mukand et al., 2004). Thus, our results would be improved if similarly coding cells were recorded in larger numbers, but this is challenging during intraoperative sessions. The detection of movement signals with such small samples suggest that they are abundant in premotor cortex. However, all of our participants had movement disorders and were off medication at the time of recording which may affect the frequency or form of the signals we recorded.

We did not observe qualitative indication of disease effects in the firing of cells, suggesting that premotor cortex is relatively unaffected by these disorders, although our sample size was not large enough to perform a comprehensive analysis. These neurons were quite similar to those recorded in macaque monkey premotor cortex, however, in firing rate and response to preparatory cues suggesting that these non-human primates are excellent models of normal human premotor function.