Multi-modal Decoding: Longitudinal Coherency Changes between Spike Trains, Local Field Potentials and Electrocorticogram Signals

Abstract

Neural information degeneracy in chronic implants due to signal instabilities affects optimal performance of brain-machine interfaces (BMIs). Spike-decoders are more vulnerable compared to those using LFPs and ECoG signals. In order for BMIs to perform reliably across years, decoders should be able to use neural information contained in various signal modalities. Hence, it is important to identify information redundancy among signal types. In this work, spikes, LFPs and ECoGs were recorded simultaneously from motor cortex of a rhesus monkey, while the animal was learning to control a multi-DOF robot with a spike-decoder. As the behavioral performance increased, the linear association among the signal types increased. Coherency of these signals increased in specific frequency bands as learning occurred. These results suggest the possibility of substituting the information lost in one modality by another.

I. Introduction

Cortically controlled BMIs use extracellular recordings towards controlling devices. Decoders use temporal and spectral features of spike trains [1][2][3], local field potentials (LFPs) [4][5][6][7] or ECoG signals [8][9] and map these measured neural activity to a kinetic or kinematic variable. The longevity and stability of these signals, however, vary largely. Clinically viable BMIs should be able to compensate for information loss that might be incurred due to signal instability, in order for them to function optimally for several years. In the case of spike-decoding, following chronic implantation of an array, the neural yield (spiking units) typically increases for a few weeks and remains relatively stable over a few months. Decoders are typically built based on this stable set of spiking units. Subsequently, due to several reactive factors in the region of implantation, the number of available signal sources sometimes start to diminish thereby affecting the BMI performance. Attempts to compensate for the loss of spiking units include using (i) multi-unit (MUA)-based decoders [10][11], which map neural activity from a channel ignoring the source identity or (ii) reconstructed signal from another modality [12]. MUA decoders are relatively more stable than their single-unit counterparts. Furthermore, LFPs recorded from the same electrodes tend to have higher longevity compared to spikes [13]. Likewise, ECoGs have comparable stability to LFPs.

We show that as BMI motor learning occurs with a behavioral task, different signal modalities increase in their synchronized oscillations. Longitudinally, the coherence between signals emanating from the motor cortex is shown to increase. Specific frequency bands were identified that shows increase in their synchrony. Since the region of recording was deafferented for years and was not involved in any fine motor behavior, the changes observed can be attributed primarily to the BMI learning. Together, the results point to the possibility of substituting one signal modality with other along the process of BMI learning.

II. Experimental Methods

A. Neural Implants

The chosen Rhesus macaque (Macaca Mulatta) was a chronic transradial amputee for over 7 years, and the implantation was made on the side contralateral to the amputation. A 100-channel multi-electrode array (MEA) (1.0 mm electrode length; 400 m interelectrode separation from Blackrock Microsystems, Inc., Salt Lake City, UT) was chronically implanted on the primary motor cortex (M1) region. A micro-ECoG grid with 32 recording sites (4×8) was also implanted sub-durally lateral to the MEA. The surgical and behavioral procedures involved in this study were approved by the University of Chicago Institutional Animal Care and Use Committee and conform to the principles outlined in the Guide for the Care and Use of Laboratory Animals.

B. Brain-machine Interface

The animal was operantly conditioned to control a multiple DOF robot, details of which can be found in [14]. Briefly, the behavioral task involved generating (a) reach velocities, to move the robotic arm towards and away from the target object and (b) grasp velocities, to control the opening and closing of the aperture formed by the three digits of the robotic hand. Two distinct subset of neural units controlled the reach and the grasp dimensions independently. Action potential spikes recorded by the MEA were sorted and binned into 50 ms bins. A 20-tap static Wiener-filter was used to estimate the control velocity from the binned spikes. Subsequently, the decoded output was translated into joint-space velocities. The LFPs and ECoG signals were passively sampled at 2 kS/S, as the animal performed the BMI task (see Figure 1).

Fig. 1.

Schematic of the BMI setup. Single unit activity from the implanted MEA was sorted, binned and decoded to control a multiple DOF robot. The local field voltages from the MEA and the micro-ECoG grid were sampled concurrently with the behavioral task execution.

III. Neural Signal Analysis

Periods of decoded output with 500ms of near-zero velocity followed by transitions that were larger than 1.2σ within a 50ms window and sustained for another 500ms was considered an epoch. As the animal learned to control the velocity of the robot effectively, the movements became more volitional. The decoded reach velocity was used in determining the epochs of transition regions from low to higher velocities.

A. Neural Synchrony

Linear time invariant association of bivariate time series (u(t) and v(t)) is often estimated using coherency. It determines the frequency domain correlations between the signals such that change in synchrony at specific bands can be assessed. Mathematically, coherence (γ_uv) is defined as,

$${\gamma }_{uv}\left(f\right)=\frac{S_{uv}}{\sqrt{S_{uu}}\sqrt{S_{vv}}}$$ (1)

where, S_uv is the cross-spectral density of the two time series and S_uu and S_vv are individual power spectral densities of the signals. Coherence measures were estimated longitudinally under two categories, (a) spikes vs LFP and (b) LFP vs ECoG. Both binned spikes and the field voltages were sampled with ΔT=0.5ms, to keep the sampling consistent. A multi-taper Fourier transform was used to determine the coherency with a bandwidth of 10Hz and a 300ms window. The number of tapers was set as 5. The Chronux toolbox was used to estimate the coherencies [15].

IV. Results

Neural recordings from early and late sessions of behavioral learning (three of each) were analyzed. The selected sessions were separated by 30 days (with 15 days of actual training). Coherence between single unit activity from ten neural channels used for decoding the reach velocity and the corresponding LFP on the same electrodes is shown in Figure 2 (top). Similarly, the change in coherence between LFPs and ECoGs is shown in Figure 2 (middle). It shows a shift in the peak coherent frequency as behavioral learning progresses. Synchrony within a single modality i.e., LFP also increased as shown in Figure 2 (bottom).

Fig. 2.

Comparison of coherence between earlier (blue) and later (red) training sessions. Changes in coherency between spikes and field potentials (top) occurred at two bands (0-50Hz and 100-150Hz). The shaded region shows the 95% confidence interval. In the case of LFP and ECoG signals (middle), the synchrony increased and shifted in frequency as learning progressed (inset in middle panel). Synchrony between LFPs become distinctive in two frequency bands peaking at ~20 and 38Hz (bottom)

In the case of LFPs, the coherency in the β range appeared to be bi-modal. The temporal relation between these modes as a function of the behavioral task is plotted as a coherogram (see Figure 3). Mean coherence of different frequency bands between LFPs from MEA and ECoG grid from early and late sessions of training is shown in Figure 4.

Fig. 3.

Coherogram between LFPs in the later session showing the temporal distribution of various frequency bands w.r.t. the task execution. Bands in 30-40Hz was active before the movement onset and bands in 12-25Hz shows higher coherence after the movement onset

Fig. 4.

Coherogram between field potentials of a selected channel in the MEA and the ECoG grid in the early(top) and late(bottom) sessions showing the temporal distribution of various frequency bands w.r.t. the task execution. In the early sessions, synchrony occurred after movement onset, while in the late sessions, there is a shift in the band of synchrony towards 15-25Hz and is persistent through the entirety of the task

V. Discussions

Spike-based decoders were predominantly used in BMIs in motor control applications until recently [16][17][18][19]. Experiments using LFPs [13][20][21] and decoders comparing both LFPs and spiking activities [22][23] have shown successful decoding using multiple frequency bands ranging between 0 and 150 Hz. Combined decoding of β frequency and power has been used in a reaching task [24]. ECoG signals have been used in reaching kinematics decoding for ALS patients [25] and in grasp related classification tasks [26]. However, there was not much literature on how neural correlates of learning in BMI context had changed over the course of BMI learning. Our results show that increase in coherency occur among three signal modalities used in the study, unit spiking activities, LFPs, and ECoG over β and low and high γ bands. Coherence changes between spikes and LFPs are relatively small, perhaps, as the task was quite complex and learning may still not have consolidated. Analyzing these changes over a larger longitudinal window might establish a stronger association between these modalities. Nevertheless, the coherencies were significant in several frequency bands. Thus, there is a potential to use this frequency redundancy to design a hybrid decoder that would be stable for a much longer duration. Coherence among LFPs and between LFPs and ECoG showed clear peaks over a β range, the prominent β frequency peaks decreased in LFPs, ECoG, and their coherence. Kilavik et. al. [24] showed that even when monkeys were well trained, depending on behavioral context, β frequency peaks changed. Therefore, during our learning task, the monkey may have organized behavioral contexts in relation to the decoded speed onset. Furthermore, a low gamma band coherence only among LFPs and between LFPs and unit spiking used to decode the robot kinematics increased as the monkey had been operantly conditioned as seen in a recent study using the same type of an intracortical array [27]. However, there was no γ peaks present in ECoG signals, which may indicate that this operant conditioning induced γ oscillation may be specific to deeper layers as opposed to aggregate signals from more superficial signals such as ECoG signals. Our future work includes offline hybrid adaptive decoder designs to include unit spiking activities as well as multiple frequency bands in LFPs and ECoG so that decoding performance would remain high over a sustained duration of learning. Such a novel decoder will bring us a step closer to realizing a robust and clinically viable BMI.