Cortical Adaptation to a Chronic Micro-Electrocorticographic Brain Computer Interface

Abstract

Brain–computer interface (BCI) technology decodes neural signals in real time to control external devices. In this study, chronic epidural micro-electrocorticographic recordings were performed over primary motor (M1) and dorsal premotor (PMd) cortex of three macaque monkeys. The differential gamma-band amplitude (75–105 Hz) from two arbitrarily chosen 300 μm electrodes (one located over each cortical area) was used for closed-loop control of a one-dimensional BCI device. Each monkey rapidly learned over a period of days to successfully control the velocity of a computer cursor. While both cortical areas contributed to success on the BCI task, the control signals from M1 were consistently modulated more strongly than those from PMd. Additionally, we observe that gamma-band power during active BCI control is always above resting brain activity. This suggests that purposeful gamma-band modulation is an active process that is obtained through increased cortical activation.

Introduction

Primary motor cortex (M1) has long been understood to play a key role in voluntary motor control (Jasper and Penfield, 1949). Adjacent premotor cortical areas are also involved in this process, though arguably less directly-higher levels of microstimulation are needed to trigger movements and neural activity relating to motor planning, in addition to execution of movement, are seen in a variety of experimental settings (Weinrich and Wise, 1982). The natural functions of these motor areas have been coopted to create brain–computer interfaces (BCIs) that use intracortical recordings of small subpopulations of neurons to directly control external devices, in nonhuman primates (Taylor et al., 2002) and in humans (Donoghue et al., 2007). These systems are generally based on the underlying principle that individual neurons in motor cortex are approximately linearly “cosine tuned” to a number of different movement parameters, including velocity and position of the hand during translational and rotational movements (Georgopoulos et al., 1986; Wang et al., 2010). The relationship between firing rate and movement kinematics seen in these classic open-loop neurophysiology studies can be strengthened and modified when closed-loop BCI tasks are used to provide biofeedback, allowing subjects to enhance their performance by learning to modulate neural activity in specific ways (Fetz and Baker, 1973; Moritz et al., 2008).

As an alternative to single-unit recordings, both local field potentials (Heldman et al., 2006) and electrocorticography (ECoG) (Crone et al., 1998; Schalk et al., 2007; Chao et al., 2010) provide useful information about intended motor movements. These studies and others have found various frequencies in the high gamma range to be particularly informative. When using ECoG for BCI applications, a common approach is to screen a set of large-scale, topographically distinct motor movements to identify specific actions that cause features of the neural recordings to differ significantly from the resting state. A particular feature is then mapped onto a single degree of freedom in the BCI, and subjects are instructed to imagine performing the corresponding movement to control the device (Leuthardt et al., 2004). For example, a subject might imagine a tongue movement, causing modulation of gamma band activity over tongue area of M1. This change in neural activity is detected by the BCI device and a cursor on a computer screen moves as a result. Performance can be improved with practice once the subject receives closed-loop feedback (Leuthardt et al., 2004), but the full potential of this adaptation is not well quantified. What would happen if an electrode were naively chosen without prescreening and gamma band activity on that electrode caused a cursor to move in a particular direction? Could subjects learn to use such an arbitrary signal to exert purposeful control of a BCI device?

In this study, we implement just such a system. By eliminating prescreening and motor mapping, the work presented here accomplishes two main objectives. First, we test whether subjects can learn to perform a novel μECoG BCI task through biofeedback alone, and characterize the adaptation that occurs during the learning process. Second, since we are not limited to using electrodes identified via prescreening, we are able to select pairs of electrodes such that one electrode is located over M1 and the other over dorsal premotor (PMd) cortex. By assigning gamma-band activity on these electrodes to opposing directions along a single degree of freedom in a BCI task, we can quantify the differences in modulation of neural signals from these cortical areas in a well controlled, closed-loop environment. The findings have practical implications for further development of BCI technology.

Materials and Methods

Three male 6–10 kg macaque monkeys (M, J, and N) were used in this study. Once each monkey was familiarized with the task (described below) using a joystick for training, a custom-built chronic epidural μECoG recording grid was implanted through a 22 mm craniotomy centered over proximal arm area of M1 and PMd. The electrodes were 300 μm diameter platinum-iridium (90-10 Pt-Ir) wires arranged in a 28-electrode hexagonal pattern with a 3 mm interelectrode spacing (Fig. 1b). Signals were referenced to an identical electrode (i.e., differential recordings) located at the dorsal edge of the recording chamber. The recording chamber was a large titanium ring encircling the craniotomy serving as the common ground for the recordings. Analog signals were band limited between 3 and 500 Hz and digitized at 6 kHz using a multichannel neurophysiology recording system (Tucker-Davis Technologies).

Figure 1.

A, The two-target radial choice task. 1, At the start of the trial, the cursor is centered by the computer and one of two targets appears. 2, The monkey then has 5 s to move the cursor to the correct target. 3, Once the cursor touches the outer circle or the maximum movement time has been exceeded, the trial is over, a 1 s intertrial interval occurs, and the monkey is rewarded if the correct target was chosen. B, Electrode placement of epidural μECoG array shown on brain surface of monkey J. The two electrodes (green) used for horizontal control are shown along with the reference electrode (R). The central sulcus (CS), arcuate sulcus (AS), and superior precentral dimple (SPD) are labeled.

One-dimensional radial choice task.

In these experiments, a one-dimensional (1D) radial choice task was implemented (Fig. 1a). The scene of the task was displayed on an LCD monitor in front of the monkey. The monkey interacted with the task by controlling the velocity of a spherical cursor, via joystick or brain control signal (described below). A large circular ring served as the target for the task. To begin each trial, the cursor was placed in the center of the ring. A 180° arc segment of the ring changed color, indicating the rewarded direction for that trial. The monkey had 5 s to move the cursor from the center of the workspace through the instructed target to receive a liquid reward. Moving the cursor through the nonhighlighted ring segment was considered an error.

Control signal.

For each experiment, the signals recorded from two micro-ECoG (μECoG) electrodes were used for brain control. The recordings were bandpassed between 75 and 105 Hz using a 16th order digital Butterworth filter, then full-wave rectified and low-pass filtered at 3 Hz to generate a gamma-band amplitude estimate. Next, each amplitude estimate was normalized by subtracting the mean and dividing by the SD of the previous 100 s of the signal (Rouse et al., 2011). The difference in the normalized amplitudes of the two signals was mapped to cursor velocity such that modulation of both signals in the same direction (e.g., above the mean) would lead to low cursor velocities, while modulation in opposite directions would generate large velocities.

Using the two electrodes in this “push-pull” control setup has several key advantages. First, it minimizes the effect of modulation at the reference recording site by subtracting away this common signal to ensure that the two chosen electrode sites are responsible for controlling the cursor. Likewise, having the two control signal electrodes relatively close to one another (<1.5 cm) ensures that subjects cannot simply increase gamma activity across the whole array to move the cursor. Rather, the subjects need to learn how to modulate the activity differentially under the two electrodes assigned for brain control.

Experimental timeline.

Each monkey learned two distinct closed-loop configurations for controlling a BCI cursor moving along an axis. Each configuration used a different pair of electrodes, requiring each monkey to modulate two sets of cortical sites underneath the recording grid. The axis of the 1D task was oriented such that the cursor moved horizontally (x) during the first control configuration and vertically (y) during the second set of experiments. This change provided a cue to the monkey that a new BCI control approach was required.

Each set of experiments used two push-pull electrodes separated by 9 or 15 mm depending on the orientation of the recording pair. Electrode locations were verified when the animals were killed by identifying the electrode grid location relative to the major gyri and sulci landmarks of each brain (Fig. 1b). During the initial training phase, a constant velocity bias in the direction of the target was added to the decoded velocity control signal to aid cursor movement in the correct direction. The strength of this bias was adjusted to maintain an overall success rate of ∼75%. As the monkey's performance improved, the bias was gradually reduced to zero at which point cursor movement resulted solely from brain control.

Analysis.

Performance of the subjects during the closed-loop radial choice task was characterized by the percentage correct and trial time. These metrics were combined into a single measure of bit rate based on the percentage of targets correctly hit within a given amount of time. The per trial bit rate of data transfer for a discrete target task can be determined with the following equation (Pierce, 1980):

where N represents the number of targets and P is the percentage correct expressed as a fraction. The bits per trial, B, can then be multiplied by the trial rate (trials/s) to produce the bit rate per unit time.

Results

All three monkeys successfully learned to control the cursor using both arbitrarily assigned configurations for horizontal and vertical control. For each configuration's best week (5 d) of pure brain control (no bias), the percentage of targets correctly selected ranged from 85 to 95% (chance = 50%). Less than 10% of the trials resulted in the monkey reaching the 5 s time limit without selecting a target. The mean movement time to select a target was ∼2–3 s for each monkey. This corresponds to a bit rate range of 10–25 bits/min.

Performing the brain-control task required the monkeys to differentially modulate the amplitude of the gamma band signal. An example of the trial-averaged time course of gamma band modulation following the presentation of a target is shown in Figure 2a. To convert the amplitude estimate between 75 and 105 Hz for each trial from a time series to a single value, the power spectral density was calculated from the raw signal during the movement time of each trial using the Thomson multitaper method (Thomson, 1982). The area under the curve of the spectral density was calculated between 75 and 105 Hz, and the square root was taken to give an amplitude estimate in units of microvolts. Histograms of these single-trial amplitudes from the channel shown in Figure 2a is plotted in Figure 2b. To obtain a single measure of the amount of amplitude modulation that occurred on a given channel for the two different target conditions, a d′ statistic was used. d′, or the discriminability index, is the difference of the mean of the signals for the two targets divided by their pooled SD:

where μ_R and μ_L represent the mean μECoG root mean square (RMS) in the 75–105 Hz band for right and left targets, respectively. The difference between μ_R and μ_L is normalized by dividing by the pooled SD (σ_R, σ_L) of the right and left target distributions. n_R and n_L are the number of trials to right and left desired targets, respectively. The d′ value is in units of SDs and represents the amount of separation in observed signal amplitude during the two target conditions. d′ is a classic metric of signal detection theory and provides a direct measure of how well the amplitude of the signal can be used to predict the correct target (Green and Swets, 1966).

Figure 2.

Epidural μECoG mean control signals and histogram. Data are from a single week (5 d) of recording once bias had been completely turned off with a consecutive block of both 400 correct plus any incorrect trials from each day for a total of 2000+ trials for each monkey–dimension combination. A, Mean control signals between 75 and 105 Hz for monkey M on the electrode over M1 for both leftward (red trace) and rightward (blue trace) movements. The shaded area represents the 95% confidence interval of the mean. After an initial symmetric rise, the amplitude for the left trials continues to rise significantly more to properly control the cursor. B, Histogram showing separation in spectral amplitudes between 75 and 105 Hz for trials of right versus left targets. C, The d′ statistic for each control channel for the six different monkey–dimension combinations was calculated using the separation of the mean amplitudes for the two targets. The more caudal recording electrode is shown on the left for each control pair. Error bars represent 95% confidence intervals. D, Cortical location of control electrodes for all three monkeys. The color coding in C maps directly to the colors in D. The label “S” represents the strongly modulated electrode, while the “W” represents the weaker modulated electrode in the push-pull pair. The central sulcus (CS), arcuate sulcus (AS), and superior precentral dimple (SPD) are labeled.

The d′ values for all channels used for brain control are shown in Figure 2c. These d′ values are calculated using the best week of performance (with no bias) and are calculated from 5 consecutive days of recording for each of the six monkey–dimension combinations. The labeled sign of each electrode indicates whether modulation above baseline on that channel generated a positive or negative velocity. For optimal performance on the task, “positive electrodes” should have large positive d′ values while “negative electrodes” should have large negative d′ values. Figure 2c shows that there were four clear examples (M-x, J-x, J-y, and N-y) where one electrode was being strongly modulated while the other electrode stayed relatively constant for both conditions or even modulated slightly in the nonoptimal direction for push-pull control. Conversely, in the two other cases (M-y and N-x) neither electrode by itself had a large and distinct separation between the two conditions; however, both electrodes were modulated in the correct direction.

Each of the six electrode pairs used for control was chosen such that one electrode was over M1 and the other was over PMd. For each pair in Figure 2c, the left bar is the d′ value from the electrode over M1 while the right bar is from PMd. In all cases, the electrode located over M1 was modulated more strongly (i.e., had a larger magnitude d′) than the electrode over PMd. Figure 2d shows the locations of electrodes mapped onto a standard macaque brain atlas. For each pair of electrodes, the one with the larger d′ statistic was labeled with “S,” and the other with “W.” Anatomical segregation of the strongest modulated channels over M1 is readily apparent.

In addition to examining the electrodes assigned for control, the other electrodes of the recording array were also analyzed to quantify the spatial scale of modulation that occurred. The analysis divided electrodes based on their distance from the previously identified strongly and weakly modulated control electrodes. In addition to the control electrodes (0 mm), the modulation observed on the electrodes 3 and 6 mm away from the actual control electrodes was also quantified in Figure 3a. The largest modulation occurred on the assigned control electrode that was strongly modulated and decreased on surrounding electrodes.

Figure 3.

A, d′ as a function of distance from the control electrode. Electrodes are grouped based on distance from a strongly modulated or weakly modulated control electrode as shown in Figure 2c. For the strongly modulated electrodes, the mean d′ values significantly decreased as a function of distance (ANCOVA, p < 0.001) from a mean d′ of 2.40 on the control electrodes to 0.97 on the electrodes 6 mm away. B, Time course of learning as measured by Δd′. The increase in the difference between positive and negative control channel (Δd′) across recording days as the subject adapts to the task. The endpoint (large dot) represents full brain control (no bias). Each point represents the first time that a monkey for a given dimension reached a given Δd′ value.

Initial learning time course

In addition to looking at the d′ metric during peak performance, the initial time course of control signal modulation was also examined. When first learning to control each new electrode configuration, the subjects had no information on which cortical site or what type of cortical modulation (i.e., above or below running mean) would correctly move the cursor. Since initial training was aided by a computer-controlled bias (see Materials and Methods), success rate is a poor measure of learning. However, even in the presence of bias, the d′ statistic provides a way to quantify cortical modulation during learning.

Successful performance of the brain-control task requires properly modulating the activity under at least one electrode (ideally both). In the push-pull configuration used here, the critical quantity is the difference between the two d′ values, or Δd′. A larger Δd′ value between the two channels in the push-pull decoding scheme leads directly to increased performance. Figure 3b shows the cumulative maximum performance across days, quantified as the best Δd′ observed over a 100-trial interval. (To focus on the learning trend, only increasing changes in Δd′ are illustrated such that each plot is monotonically increasing.) All subjects showed a clear rise in performance while learning the task.

BCI control versus rest

The control signals obtained from each electrode were normalized by subtracting the running average and dividing by the SD. How does the running average used for normalization in this study compare to baseline activity? At the start of each day, a 5 min segment of baseline activity was recorded before the beginning of the task. A power spectrum was calculated from these baseline recordings using one second nonoverlapping sliding windows. For comparison, in this analysis only, a movement power spectrum was performed on the first second of data after the target appeared. For each control signal electrode, two distributions of the RMS of the signal between 75 and 105 Hz were compared with the baseline distribution: (1) the distribution for trials where an increase in amplitude caused the cursor to move toward the displayed target (“increased amplitude”); and (2) the distribution for trials where the presented target could be reached by decreasing the amplitude on the electrode (“decreased amplitude”). The mean RMS for both the increased amplitude targets (Fig. 4a) and decreased amplitude targets (Fig. 4b) was compared with the mean RMS during the baseline recording period. Individually, eight of eight control electrodes showed significant differences from baseline when increased amplitude was optimal, while only two of eight electrodes were significantly different when decreased amplitude was optimal (one-sided paired t test, p < 0.05). Overall, combined two-way ANOVA was significantly different for the increased activity case (p < 0.001) and not significantly different for the decreased activity case (p = 0.48). The observation that subjects did not modulate the RMS to below-baseline levels, even though it would be beneficial to do so, suggests that this type of modulation is more difficult than modulating activity above baseline levels.

Figure 4.

The mean RMS values for each day. A, Increased amplitude targets. B, Decreased amplitude targets compared with baseline RMS at the beginning of the day. The data suggest that increasing the signal amplitude is the active process used by the brain to gain control.

Discussion

This study describes adaptation to a BCI using chronic epidural μECoG recordings, and illustrates several features that have practical importance for the design of future systems. Perhaps the most clear and promising finding is the fact that all three monkeys were able to learn to control two different electrode configurations that were naively chosen, not selected based on screening tasks. While there was some variation in the level of peak performance, every configuration that was used in these experiments provided control that was substantially above chance. Additionally, the electrodes assigned for control were selectively more modulated than surrounding electrodes. This suggests that BCI control with ECoG signals is not limited to observable changes during screening tasks but rather can also include randomly assigned cortical locations and/or frequencies. Additionally, the time course of adaptation to a novel configuration occurred on the order of days. One potential reason for the rapid learning is the novel “brain + bias” training scheme used in this study. By initially adding bias to control signals, the subject is kept more engaged with a reasonable reward level. As true brain control performance improves, it is straightforward to eliminate the bias.

While the subjects were all able to modulate cortical signals to move the cursor, our push-pull decoding scheme resulted in a mixture of observed BCI behavior. When conducting these experiments, we determined that it was necessary to normalize the two control signals to make sure that overall differences in signal amplitudes did not enable the subject to use global amplitude changes to create a differential signal that would move the cursor. While this did result in what we believe to be more selective cortical areas of modulation, it did not always require the true push-pull modulation that we initially expected. For instance, we often observed that task-related modulation was largely restricted to the signal on a single electrode, while the amplitude of the other signal was held relatively constant.

Stronger modulation consistently occurred under the more caudal electrode of the pair (i.e., the one closer to the central sulcus, in M1), while the rostral sites in PMd were more weakly modulated. Several explanations for this finding are possible. First, the stronger modulation of M1 seen in this study could be a reflection of the velocity-based decoding algorithm chosen for the study. The decoding algorithm mapped the high gamma band activity to cursor velocity. The dominant kinematic parameter encoded in M1 is velocity (Moran and Schwartz, 1999), and while dorsal premotor activity does contain some velocity encoding, it also contains strong hand position and target encoding as well (Pesaran et al., 2006). Second, neuronal differences between cortical areas could affect the strength of the modulated signal relative to the signal-to-noise ratio. M1 is defined by large layer V output cells known as Betz cells that descend to the spinal cord through the corticospinal tract and form the major motor output pathway of the brain. When firing action potentials, these pyramidal cells generate large extracellular currents, which likely comprise a significant portion of the high gamma ECoG signal. Premotor cortex also contributes significant efferent pyramidal neurons to the corticospinal tract; however, these cells comprise a smaller percentage per unit volume in PMd (Weinrich and Wise, 1982; Dum and Strick, 1991).

The modulation differences between cortical areas have practical implications in ECoG BCI design. While primary motor cortex may be one of the best areas for ECoG BCI control, it would be beneficial to recruit other areas like dorsal premotor cortex to take an active role in control. Training paradigms that better facilitate recruitment of more weakly modulated cortical sites should be further explored. For example, when the activity under one electrode is observed to be stronger than the other electrode, the relative weighting between the two push-pull electrodes could be adjusted to increase the relative contribution of the more weakly modulated electrode (i.e., much like an eye patch that covers the dominant eye allows the lazy eye to improve). This could strengthen the modulation occurring on the weaker channel and create a more balanced push-pull control.

Also of practical concern is that active modulation could be used to increase high gamma power above resting levels, but decreasing from rest appeared difficult even when it would be beneficial to do so. Much debate exists about the neuronal processes that contribute to observed high gamma signals in ECoG recordings, but there is strong evidence that high gamma activity is related to single-unit activity (Heldman et al., 2006). Previous single-unit studies have shown that when subjects are required to modulate a differential control signal using two neurons, the subjects will increase the firing rate in one neuron while maintaining baseline activity in the other neuron (Fetz and Baker, 1973). Thus, it appears that increases in high gamma activity specifically correspond to mental effort in BCI tasks. This is consistent with previously observed increases in high gamma activity with actual motor movements (Pfurtscheller et al., 2003; Leuthardt et al., 2004; Miller et al., 2007; Yanagisawa et al., 2012).

The observed increased activity of high gamma ECoG signals when engaged in a BCI task has a practical implication to BCI design. First, the design of decoding algorithms should take into account the expected asymmetry in the range of modulation and not expect a linearly balanced modulation of signal amplitude above and below baseline. Second, it appears that increases in high gamma activity are a good predictor of whether the subject is actively involved in the task. Measuring this level relative to a known baseline could potentially serve as a state detector that determines when the user wants to operate the BCI controlled device.