Decoding movement-related cortical potentials from electrocorticography

Abstract

Object

Control signals for brain-machine interfaces may be obtained from a variety of sources, each with their own relative merits. Electrocorticography (ECoG) provides better spatial and spectral resolution than scalp electroencephalography and does not include the risks attendant upon penetration of the brain parenchyma associated with single and multiunit recordings. For these reasons, subdural electrode recordings have been proposed as useful primary or adjunctive control signals for brain-machine interfaces. The goal of the present study was to determine if 2D control signals could be decoded from ECoG.

Methods

Six patients undergoing invasive monitoring for medically intractable epilepsy using subdural grid electrodes were asked to perform a motor task involving moving a joystick in 1 of 4 cardinal directions (up, down, left, or right) and a fifth condition (“trigger”). Evoked activity was synchronized to joystick movement and analyzed in the theta, alpha, beta, gamma, and high-gamma frequency bands.

Results

Movement-related cortical potentials could be accurately differentiated from rest with very high accuracy (83–96%). Further distinguishing the movement direction (up, down, left, or right) could also be resolved with high accuracy (58–86%) using information only from the high-gamma range, whereas distinguishing the trigger condition from the remaining directions provided better accuracy.

Conclusions

Two-dimensional control signals can be derived from ECoG. Local field potentials as measured by ECoG from subdural grids will be useful as control signals for a brain-machine interface.

Control signals for brain-machine interfaces may be obtained from a variety of sources, each with its own respective advantages and disadvantages. Whereas scalp EEG provides a noninvasive method that may provide up to 2 or 3 dimensions of control, using features such as the mu rhythm²⁰ and P300, EEG is limited in both spatial resolution (due to its distance from the brain surface) and temporal resolution (due to the filtering properties of the scalp, which markedly attenuates spectral content in the gamma range [> 30 Hz]).^4,16,29,30

With regard to invasive control signals, several options are available. The most invasive methods, which involve single or multiunit recordings obtained from implanted high-impedance electrodes such as the Utah array, have been used in both humans and monkeys with remarkable results.^{3,8,12,21,26,27} Advantages of these methods are high signal-to-noise ratios and the ability to detect single and multiunit activity, whose firing rates may be precisely modulated by motor activity. Combining 100 or more channels of multiunit activity allows for powerful population coding algorithms, which at present have been demonstrated to control at least 4 degrees of freedom, but have the potential to do much more.^2,6,7,15,28 For the same reasons that multiunit approaches are attractive, there are some disadvantages. The small size and high density of the multiunit electrodes allow coverage of only a small area of cortex; for example, the standard Utah array measures 0.4 × 0.4 cm.¹⁸ To implant these devices, they must penetrate the cortex, and the act of penetration may cause some damage to the cortex. A glial scar may develop, which decreases the number of units recorded over time.^10,24,25 Although the quality or total number of recordable units may decline over time, groups have recorded useful control signals > 3 years after implantation.⁹

With the aim of developing a fully implantable, widely used brain-machine interface, one can understand the concern of either damaging healthy cortex or repeatedly needing to replace electrodes due to a decline in signal over time. At present, most implantable technologies also have bulky wires that cross through the skin surface, allowing a conduit for potential infection, although wireless technologies are undergoing testing and development. Multichannel data obtained at high sampling rates presents a problem of bandwidth and power for fully implanted systems, but several groups are currently actively engaged in addressing these challenges.^13,26

Keeping in mind the criticisms of both the most invasive of interfaces (the penetrating electrodes) and the least invasive (scalp EEG), some groups have proposed ECoG as a potential intermediary. Electrocorticography involves the use of subdural low-impedance surface electrodes, typically spaced between 0.5 and 1.0 cm apart, which are in common use for invasive monitoring in patients with epilepsy. These electrodes do not measure single or multiunit activity, but rather local field potentials, representing the summed activity of many action potentials originating from approximately 50–350 μm from the tip of the electrode or slower ionic membrane fluctuations from as far away as 3.0 mm.²² Because of their closer proximity to the surface of the brain, higher frequency signals are available in ECoG than in scalp EEG.¹¹ In patients with epilepsy, large sheets of subdural electrodes are typically used to cover large areas of frontal, temporal, or parietal cortex, depending on the location of seizure focus. The long-term safety and reliability of these electrodes is currently undergoing evaluation in some patients who have undergone subdural motor cortex stimulation using similar electrodes.⁵ The efficacy of these electrodes in controlling a brain-machine interface has been demonstrated by a number of groups.^{1,14,19,23,31} The goal of the present study was to determine whether 2D directional selectivity could be achieved with ECoG surface recordings.

Methods

The 6 participants in this study were patients with medically intractable epilepsy who were undergoing invasive subdural electrode monitoring before resection. There were 3 males and 3 females in the study, with an age range of 22 to 40 years, and a median age of 30 years. All patients underwent unilateral, frontotemporal, subdural grid electrode coverage using a standard 32-contact frontal grid with 1.0-cm interelectrode spacing, and a high-density 96-contact temporal grid with 0.5-cm interelectrode spacing on the side of suspected seizure onset (Fig. 1). There were 3 patients with left-sided coverage and 3 with right-sided coverage. Patients did not incur additional risk by participating in these studies. Research protocols were approved by the University of Iowa Human Subjects Review Board.

Fig. 1.

Representative grid schematic of the electrode contact locations for participant 153. On electrical stimulation, the red contact (105) was noted to induce left-arm tingling, whereas stimulating the green contact (113) caused left-hand tingling.

Participants were presented with a visual display of an arrow pointing in 1 of 4 cardinal directions (up, down, left, or right) or a fifth display of a square, indicating the “trigger” condition. These stimuli were displayed on a computer screen using Simulink (The Mathworks, Inc.), and were randomly interleaved and equally weighted. The majority of experiments were performed with the hand contralateral to the grid electrode coverage. When a cue appeared, each participant was asked to move the joystick in the direction of the arrow cue, or to click the trigger at the top of the joystick with the tip of the index finger in response to the trigger cue. When the visual cue disappeared from the screen, the patient was asked to return the joystick to the neutral position or release the trigger. Each trial lasted approximately 4 seconds, with the first 70% of the trial (2.8 seconds) used for stimulus presentation and the remaining 30% of the trial (1.2 seconds) consisting of a blank screen, during which time the participant returned the joystick to the neutral position. In control trials, participants were given the same visual cues and asked to hold the joystick but not to move, only to imagine the movement they would have performed during the movement trials. All participants performed at least 90 trials each for movement and control, with a range of 95–210 control trials and 90–250 movement trials, depending on time constraints and each patient's willingness and ability to participate. In a subset of patients, additional motor comparison trials were performed using the hand ipsilateral to the grid electrode coverage. Electrical stimulation mapping and somatosensory evoked potentials were performed in a subset of patients to physiologically locate the motor-sensory cortex.

All signal acquisition was performed using TDT Acquisition Systems (Tucker Davis Technologies) and all analysis was performed using Matlab (The Mathworks, Inc.). All signals used in this analysis were acquired at 2034.5 Hz. Time-frequency analysis was performed using either the complex Morlet wavelet transform¹⁷ or short-time discrete FFT with a sliding window.

Time-frequency analysis was performed on each of the trials using the discrete FFT method. Sliding windows of 250 msec width with 12.5 msec overlap were used to compute a baseline spectrogram from 0.5 seconds before movement onset to 1.0 seconds after movement onset, in the 0–200 Hz range. For each channel, the time-frequency plane was divided into 12.5 msec × 10 Hz windows, in which the energy in each tile was computed by averaging across nonoverlapping frequency bands from 0 to 200 Hz (for example, 0–10 Hz, 10–20 Hz, up to 190–200 Hz). Each trial was associated with 1 of 5 movement trajectories. The 32 channels across the frontal grid were used to divide the data for each trial into a time-frequency-channel voxel. A 5-way ANOVA was performed for each time-frequency-channel voxel across the 5 movement conditions. Any voxel that differed significantly across the movement conditions (p < 0.001) was selected. These significant voxels were then passed into a naïve Bayes classifier to classify each trial according to 1 of the 5 movement conditions. Repeated 10-fold cross-validation was used to check the accuracy of predictions.

Results

Stimulation Mapping

Results of stimulation mapping in a typical patient (Fig. 1) indicate that 2 contacts were particularly relevant. Contact 105 (Fig. 1, red) was associated with left-arm tingling and contact 113 (Fig. 1, green) was associated with left-hand tingling.

Visual Evoked Potential in Motor Cortex and Power Inversion During Movement

All movement trials from a typical participant and motor channel were averaged (Fig. 2, lower). When synchronized to the onset of the visual cue (arrow or trigger), there was a clear evoked potential in the motor cortex, with initial negative deflection at 200 msec, followed by a positive deflection at 270 msec. This evoked potential was associated with an increase in low-frequency energy (< 10 Hz), as indicated by the burst of low-frequency energy between 0.0 and 0.5 seconds on the spectrogram (Fig. 2, upper). Movements typically occurred 0.5 seconds after the onset of the visual stimulus, and were associated with a drop in lower-frequency energy (10–40 Hz) and an increase in higher frequency energy (50–160 Hz), as indicated on the spectrogram. This increase in high-frequency power and decrease in low-frequency power occurred for the duration of the movement, and was subsequently replaced by a broadband increase in low-frequency energy after 1.5 seconds, when the patient was holding the joystick in the desired target position.

Fig. 2.

Time-frequency analysis (upper) and averaged evoked potential (lower) for contact 105, with all contralateral movement conditions collapsed together. Zero corresponds to visual stimulus onset. Note the visual evoked potential synchronized to the visual stimulus between 0 and 0.5 seconds and associated low frequency increase in power. A later increase in high-frequency power and drop in low-frequency power associated with actual movement occurred between 0.5 seconds and 1.5 seconds. Finally, there was a persistent low-frequency increase in power associated with the steady-state movement response after 1.5 seconds.

It should be noted that when the trials were aligned at the beginning of movement rather than at visual stimulus onset, as shown in Fig. 3, the response noted in Fig. 2 was eliminated from the evoked potential. This result was due to the variable reaction time on a trial-by-trial basis that causes the precisely phase-locked visual evoked potential recorded over the motor cortex to be eliminated in the average across trials. The drop in low-frequency power and increase in high-frequency power still persisted and was centered around the zero time-point, coincident with actual movement. Figure 4 shows the results of motor experiments performed with the ipsilateral hand, synchronized to visual stimulus onset. In this case, it should be noted that a visual evoked potential was also observed in the motor cortex of the ipsilateral side, and the similar power inversion during movement on the contralateral side was also visible on the ipsilateral side, although to a lesser extent.

Fig. 3.

Time-frequency analysis (upper) and averaged evoked potential (lower) for contact 105, with all contralateral movement conditions collapsed together. Zero corresponds to movement onset. Note the disappearance of the visual evoked potential but the persistence of the high-frequency/low-frequency inversion that occurs during movement (at the zero time point).

Fig. 4.

Time-frequency analysis (upper) and averaged evoked potential (lower) for contact 105, with all ipsilateral movement conditions collapsed together. Zero corresponds to visual stimulus onset. Note the persistence of the visual evoked potential, even in the hemisphere ipsilateral to movement. Note also the persistence of the high-frequency/low-frequency inversion associated with movement.

Decoding Direction

When trials were separated by movement direction, and a repeat time-frequency analysis was performed using the sliding discrete FFT method, it became apparent that different channels had varying ability to distinguish the different movements. The contact associated with arm tingling (105) did not reveal any significant differences between movement directions (Fig. 5). The more lateral contact (113), which was associated with hand tingling, did allow a distinct separation of the trigger condition compared with the other conditions (Fig. 6). This separation was likely due to the differing nature of the trigger condition (as an index finger movement) compared with the other 4 joystick movement conditions, which tended to involve the same amount of motor forearm movement. Figure 7 shows the graph of the averaged (70–130 Hz) energy for each of the 5 movement conditions (4 directions and 1 trigger) demonstrating this clear separation for the trigger condition relative to the other movement directions.

Fig. 5.

Time-frequency analysis performed for channel 105 (marked in red on the grid schematic), separating the 4 movement directions and the trigger condition. Although there is a power inversion associated with movement, there is no clear separation between the movement conditions. Trials synchronized to joystick movement onset at 0 seconds.

Fig. 6.

Time-frequency analysis performed for channel 113 (marked in green on the grid schematic), separating the 4 movement directions and the trigger condition. With this channel, the trigger condition separates clearly from the other conditions, with a marked increase in high-frequency power relative to the other 4 directional movement conditions. Trials synchronized to joystick movement onset at 0 seconds.

Fig. 7.

Graph of normalized power for channel 113 between 70 and 130 Hz over time showing clear separation of the trigger condition (blue) relative to the other 4 movement conditions (down [green], right [red], up [cyan], left [magenta]). Trials synchronized to joystick movement onset at 0 seconds.

Statistical Decoding

Decoding accuracies for each of the 6 patients are shown in Table 1. The first column of values indicates decoding accuracies when attempting to classify movement from rest, with accuracies ranging from 83–96%. The second column indicates the accuracy involved in predicting 1 of the 5 movement conditions, with accuracies ranging from 58 to 86%. The third column indicates the accuracy achieved when predicting the trigger condition separately from the other 4 directional categories (which have been collapsed together), with accuracies ranging from 72 to 93%. Detection accuracies were generally greater for resolving trigger from nontrigger movement, as compared with resolving direction.

TABLE 1.

Decoding accuracy percentages in 6 patients using statistical selection and a naive Bayes classifier with 10-fold cross-validation

Case No.	Movement/Rest	Direction	Trigger/Nontrigger
146	96	86	91
147	90	71	88
149	94	76	72
153	97	74	93
154	83	58	83
156	87	76	90

Discussion

Decoding accuracy shown in this experiment is comparable to that reported in the literature for 2D motor tasks.^1,14,25 The procedures for decoding used in this paper are time-intensive with current computing power, and have only been performed in an offline manner, although other groups have achieved online 2D control.^14,31

The power inversion noted during movement is the electrocorticographic extension of the mu rhythm, which has been successfully used by a number of groups in both the EEG-based literature³⁰ and the ECoG literature for movement control.^{1,14,19,23,31} Mu rhythm has been reported in both the ipsilateral and contralateral hemispheres, although weaker in the hemisphere ipsilateral to movement,²² consistent with the findings in this experiment. From the standpoint of a cortical interface, it would be important not only to separate ipsilateral from contralateral signals, but also useful to be able to control bilateral neuroprosthesis from a single hemisphere.

A number of other methods for decoding movement from cortical parameters have been proposed, including regularized least-squares regression, principle components analysis, independent components analysis, and support-vector machines, to name a few. These methods may be more promising in terms of the efficiency of processing that would be needed for an online decoder.

Although detection accuracies in the 80% range for direction are better than chance, they would be problematic when applied to a real-world application such as driving a car. Therein lies a debate about the degree of information that can be derived from ECoG. Much work remains to be performed on the decoding side while analyzing the information present in recording channels before the statement can be made that the maximum amount of information made available by ECoG has been usefully extracted.

A number of channels, including far-frontal channels, were used in this analysis. Some of these channels are more relevant to decoding movement intention prior to the onset of movement, whereas others may be linked to somatosensory feedback. Disambiguating these channels and devising decoding procedures limited to motor, pre-motor, or sensory cortices might be of value in limiting the size of coverage needed for a decoding interface. Conversely, a broad area of coverage may potentially facilitate areas involved with different stages of the motor process, ranging from planning, to execution, to feedback and error-correction. Perhaps the ideal brain-machine interface will combine both multiunit activity available from carefully targeted invasive arrays and broad predictive cognitive processing information attainable from ECoG.

Conclusions

Two dimensions of control can be decoded from the ECoG signal recorded over the motor cortex and frontal lobes.

Abbreviations used in this paper

ECoG

electrocorticography

EEG

electroencephalography

FFT

fast Fourier transformation