Physiological properties of brain-machine interface input signals

Abstract

Brain-machine interfaces (BMIs), also called brain-computer interfaces (BCIs), decode neural signals and use them to control some type of external device. Despite many experimental successes and terrific demonstrations in animals and humans, a high-performance, clinically viable device has not yet been developed for widespread usage. There are many factors that impact clinical viability and BMI performance. Arguably, the first of these is the selection of brain signals used to control BMIs. In this review, we summarize the physiological characteristics and performance—including movement-related information, longevity, and stability—of multiple types of input signals that have been used in invasive BMIs to date. These include intracortical spikes as well as field potentials obtained inside the cortex, at the surface of the cortex (electrocorticography), and at the surface of the dura mater (epidural signals). We also discuss the potential for future enhancements in input signal performance, both by improving hardware and by leveraging the knowledge of the physiological characteristics of these signals to improve decoding and stability.

brain-machine interfaces (BMIs), also called brain-computer interfaces (BCIs), decode neural signals and use them to control various external devices. The BMI field has expanded swiftly in the past two decades. Motor BMIs are perhaps the best known and constitute the focus of this review, but the field also includes “reactive” BMIs using evoked potentials (Cheng et al. 2002; Donchin et al. 2000; Wang et al. 2006), “passive” BMIs (Zander and Kothe 2011) that do not attempt direct control, and sensory/afferent BMIs (Bensmaia and Miller 2014; London et al. 2008; O’Doherty et al. 2011; see also Lebedev 2014 for a review of BMI categories). Motor BMIs have the potential to restore or replace motor function to people with central nervous system disorders such as spinal cord injury, motor neuron disease, stroke, and traumatic brain injury. At the moment, however, a high-performance BMI device that is completely clinically viable (i.e., ready for usage as a mainstream medical device) has not been demonstrated. Many factors affect clinical viability and BMI performance. The first of these is the selection of brain signals used to control BMIs. In this review, we concentrate on the physiology, movement-related information, and other properties of the various types of input signals that have been used to date in BMIs.

The first invasive motor BMI was demonstrated almost 50 years ago, by Eberhard Fetz (Fetz 1969). This BMI was a biofeedback BMI (Fetz 2007): monkeys were operantly conditioned to modulate the firing rates of motor cortical single-unit action potentials to control an external display. Similar approaches have also enabled direct control of cursors using noninvasive electroencephalographic (EEG) signals (Birbaumer et al. 2000; Pfurtscheller and Neuper 2001; Wolpaw et al. 1991). More recently, the ability to record many signals simultaneously from inside the cortex has led to the development of biomimetic BMIs, that is, BMIs that are based more closely on the brain’s functioning during normal movements (Carmena et al. 2003; Fetz 2007; Serruya et al. 2002). In biomimetic BMIs, a decoder is built to translate the cortical activity and control the external device, which can range from cursors (Gilja et al. 2012, 2015; Hochberg et al. 2006) to robotic limbs (Collinger et al. 2013; Hochberg et al. 2012; Velliste et al. 2008) to paralyzed limb muscles (Bouton et al. 2016; Ethier et al. 2012; Pohlmeyer et al. 2009).

Key Issues in Designing Clinically Viable BMIs

There are several key issues to consider when designing motor BMIs to be clinically viable, starting with the choice of signal source. This choice is influenced by input signal longevity (the duration that a signal can provide information), signal stability (here defined as the stationarity of the movement-related information), the amount of input signal information about movement (and the amount required to control the desired output), digital sampling frequency, and the level of invasiveness and risk involved in recording the signals. Longevity is clearly paramount. If the BMI cannot successfully acquire brain signals for a very long time, it will cease to be a viable medical device and will need to be replaced, requiring additional brain surgery. This is not a desirable option for clinical practice. Signal stability is important for two reasons. First, if signals are changing too quickly, BMI users will face more difficulty in achieving proficient control. Second, unstable signals will require some period of recalibration, which could be frustrating to the user if it occurs frequently, such as daily. Power consumption is proportional to sampling frequency; both should ideally be minimized to enable wireless transmission with minimal heating of tissue and reduce the need for battery replacement/recharging (Jackson and Hall 2016; Kipke et al. 2008). The amount of information required to control the desired output might change depending on the output. Proportionally controlling two degrees of freedom (for example, a cursor control task) requires less information than proportionally controlling a high degree-of-freedom output, for example, controlling multiple joints of a robotic limb, or functional electrical stimulation (FES) of many arm and hand muscles (Bouton et al. 2016; Ethier et al. 2012). Thus, from a clinical perspective, it would be wise to design a BMI based on the desired application, minimizing invasiveness if possible without sacrificing BMI function.

Types of Input Signals

Many types of signals have been used to control BMIs. These include electrical signals, which can be recorded at multiple levels: action potentials (spikes), intracortical local field potentials (LFPs), subdural signals (electrocorticography, or ECoG), epidural field potentials (EFPs), and EEG (Fig. 1). Other signals have also been used to control BMIs, including blood flow [near-infrared spectroscopy (NIRS) and functional magnetic resonance imaging (fMRI), which have poor spatial or temporal resolution, respectively (Sitaram et al. 2009)], calcium imaging (Clancy et al. 2014), which is only suitable for animals, and magnetoencephalography (MEG; Buch et al. 2008; Mellinger et al. 2007), which is impractical for widespread use due to extremely high cost and large size. Electrical signals are thus by far the most common inputs for motor BMI. Because electrical sources are attenuated by the cerebrospinal fluid (CSF), skull, and scalp, there is in general a correlation between the level of invasiveness of the recording and the signal quality as well as the spatial, spectral, and temporal resolution (Bundy et al. 2014; Markowitz et al. 2011; Nunez and Srinivasan 2006; Slutzky et al. 2010). In this review, we focus on invasive neural signals, including intracortical signals, ECoG, and EFPs. Noninvasive BMIs, such as those based on EEG or neuroimaging technologies, may be most suitable for biofeedback training or higher level tasks such as goal selection. Very few noninvasive BMIs have been demonstrated to be capable of high degree-of-freedom online control, and we do not focus on them (but see McFarland et al. 2010 for an example). For purposes of this review, we define invasiveness with respect to the disruption of brain, meningeal, and vascular tissue, rather than with respect to disruption of the skull or skin. We note that in this perspective, greater invasiveness does not necessarily imply greater risk to the patient, although it may seem that way intuitively. For example, deep brain stimulators are very successful invasive devices with large electrodes (1.3 mm in diameter) that are nevertheless safe in terms of clinically measured outcomes (Isaias et al. 2009; Kennedy et al. 2007, 2011). All of the BMI recording arrays discussed herein are less invasive than deep brain stimulation in terms of volume of brain tissue disrupted.

Fig. 1.

Multiple levels of electrical recordings that have been used as BMI inputs. A: schematic showing the locations of each type of recording array used to acquire the signals discussed in this review. ICS, intracortical LFPs and spikes (recorded with electrodes such as those in B); ECoG, subdural electrocorticography (recorded with electrodes such as those shown in C); Epi, epidural recordings (also recorded with electrodes shown in C). D: data of each type recorded in our laboratory. Spike signals (bottom trace) can either be single units (SU) if waveform-sorted or multiunit spikes (MSP) if detected solely by a threshold (for example, gray dashed line).

Intracortical signals.

Intracortical recordings were the first input signals to be used for BMIs. These signals are obtained with the use of multielectrode arrays, most often either silicon shanks (Rousche and Normann 1998; Subbaroyan et al. 2005) or assembled from metal microwires (Nicolelis et al. 2003; Schwarz et al. 2014). These recordings can be bandpass filtered at around 0.5–300 Hz, to extract local field potentials (LFPs), or filtered in a higher band (300 Hz−12 kHz) to extract action potentials, or spikes. Spikes can be sorted into single units or simply thresholded, also called multiunit spikes (MSPs). Single-unit spikes (or single-unit activity, SUA) have been the gold standard of neuroscientific studies since the mid-20th century because of their clear origin from a single neuron. Because many of the early BMI studies were pioneered by groups with motor cortical neurophysiology backgrounds, most of them used ensembles of SUA. More recent BMI studies have started using MSPs and LFPs. LFPs are the extracellular potentials that are hypothesized to be produced by the summation of local, largely postsynaptic potentials (Buzsáki et al. 2012; Mitzdorf 1985).

Subdural (ECoG) signals.

Electrodes placed directly on the cortex (or more accurately, on the pia mater), known clinically as ECoG, have been used for many years to record brain signals in patients with epilepsy (Engel 1996). These electrodes are placed in people who have intractable seizures and may require resection of the seizure focus, both to more accurately localize the seizure focus and to assist with mapping of “eloquent” cortical functions (mainly hand and face movements and language). The fact that such electrodes are left in place for over a week and sometimes cover hand and arm areas of the motor cortex provides an opportunity to study motor control and BMIs in humans without incurring extra risk to the patients.

Around the turn of the millennium, researchers started to investigate ECoG properties in the motor cortex (Crone et al. 1998, 2001), and in 2004 the first closed-loop, biofeedback BMI for cursor control was implemented (Leuthardt et al. 2004). Since that time, a number of BMI studies have used ECoG, most often for offline decoding, but also in online control studies. As with LFPs, the largest proportion of the motor-related information in ECoG is contained in the high-gamma band (or “broadband”), as well as the local motor potential (LMP; Ball et al. 2009; Chao et al. 2010; Flint et al. 2014; Kubánek et al. 2009; Manning et al. 2009; Miller et al. 2007; Nakanishi et al. 2013). LMP is typically calculated by smoothing the continuous time-domain signal, for example, with a moving average filter (Mehring et al. 2004; Schalk et al. 2007). Some computational models have suggested that ECoG high-gamma activity is a proxy for the mean ensemble spiking activity in the cortex underlying the ECoG electrode (Miller et al. 2009; Ray et al. 2008). Certainly, it seems likely that ECoG is the summed activity of the underlying LFPs in that area of cortex (Nunez and Srinivasan 2006; Ray et al. 2008).

Epidural signals (EFPs).

Electrodes that are placed outside the dura may be capable of controlling a motor BMI and would have several clinical advantages over ECoG arrays in terms of invasiveness. By leaving the meninges intact, the risk of a device infection—incidence of 7–9% in two long-term epilepsy studies of fully implanted devices (Bergey et al. 2015; Cook et al. 2013)—spreading to the brain itself should be markedly lowered. The risk of stroke and subdural or subarachnoid hemorrhage should also be reduced. Finally, if only a small array would be required, then it is conceivable that such an array could be placed using a burr hole, which is a substantially less invasive procedure than a full craniotomy. Because electrodes are not routinely placed epidurally for clinical purposes, there is a dearth of evidence to support these hypotheses. The only relevant recent trial is the EVEREST trial of an epidural stimulation device, which reported a 7% infection rate through a 6-mo trial period, although distinction was not made between local wound infection and those involving the brain (Levy et al. 2016). There are also bridging veins to be concerned with, when placing anything between the dura and the skull. Nevertheless, epidural signals pose an intriguing option for certain applications.

Movement-Related Information

Intracortical signals.

Motor BMI performance relies on the amount of movement-related information contained in its input signals. SUA from primary motor cortex (M1) has been used extensively for offline decoding of arm movements, inspired by the work of Humphrey et al. (1970) in decoding kinematics using a linear filter and that of Georgopoulos et al. (1986) in developing the population vector approach. These two approaches are closely related, but distinct (Chase et al. 2009). In a population vector, the tuning curves (or preferred directions; Georgopoulos et al. 1982) of multiple single units are combined to derive an ensemble preferred direction at a given time. During reaching and grasping, SUA from motor cortex provides a great deal of information about both the intended movement direction (Georgopoulos et al. 1986), arm and hand kinematics (Moran and Schwartz 1999; Vargas-Irwin et al. 2010; Wessberg et al. 2000), joint torques (Evarts 1968; Fagg et al. 2009), and arm muscle activity (Morrow and Miller 2003; Pohlmeyer et al. 2007). Multiunit spikes have been shown to provide essentially equivalent decoding performance to single units (Chestek et al. 2011; Fraser et al. 2009; Stark and Abeles 2007), with one study showing a small improvement from sorting (Todorova et al. 2014). Multiunit activity could also be represented as a continuous, rather than discrete, signal by bandpass filtering the raw signal and computing the envelope (root-mean-square amplitude) of the signal (Stark and Abeles 2007). Spike ensembles (both single and multiunit) have been used in many BMI experiments in the laboratory with intact animals: to control cursors (Carmena et al. 2003; Gilja et al. 2012; Serruya et al. 2002; Taylor et al. 2002), robotic limbs (Velliste et al. 2008; Wessberg et al. 2000), and functional electrical stimulation of (temporarily) paralyzed muscles (Ethier et al. 2012; Pohlmeyer et al. 2009). Likewise, BMI control with spike signals has also been translated into applications that can be used by paralyzed humans: such systems have been employed for binary control (Kennedy et al. 2000) as well as control over cursors (Gilja et al. 2015; Hochberg et al. 2006), robotic limbs (Hochberg et al. 2012; Wodlinger et al. 2015), and functional electrical stimulation of paralyzed muscles (Ajiboye et al. 2017; Bouton et al. 2016). Substantial effort has been exerted to maximize the degrees of freedom (DOF) that can be controlled with spike-based BMIs. To date, the highest number of BMI-controlled DOF reported is 10 (Wodlinger et al. 2015). Despite these impressive demonstrations, engineering a fully implantable BMI to use spike signals carries an added challenge of high sampling rate and computational demands on the device, which translate into high power requirements.

LFPs could provide an additional (or alternate) signal source to spikes for controlling a motor BMI. LFPs were first used for decoding movement in posterior parietal areas, where they were found to have equal or superior information for distinguishing the direction and state, respectively, of oculomotor saccades (Pesaran et al. 2002). LFPs from the motor cortex were long thought to have less movement-related information than spikes (Stark and Abeles 2007). However, more recent studies have shown that LFPs are very informative (Bansal et al. 2011), with nearly as much information as spikes about kinematics (Flint et al. 2012b) and muscle activity during reaching and grasping (Flint et al. 2012a). Prominent examples of offline decoding performance are shown in Supplemental Table S1. (Supplemental material for this article is available online at the Journal of Neurophysiology website.) The largest amount of this information is contained within the power of the high-gamma band (~70–300 Hz) of the LFP (Flint et al. 2012a; Flint et al. 2012b; Rickert et al. 2005; Zhuang et al. 2010). A considerable amount is also contained in the LMP, which is dominated by the lowest frequency (0–4 Hz) delta band (Flint et al. 2012b; Rickert et al. 2005), because power in LFPs decreases with frequency according to a power law (Miller et al. 2009). There is a considerable amount of information in the phase of the delta band (Saleh et al. 2010), which likely explains why we found more information in the LMP (which retains time domain information) than in delta-band power (Flint et al. 2012a, 2012b). The mu (8–12 Hz) and beta (12–30 Hz) bands also contain substantial movement-related information (Murthy and Fetz 1996; Sanes and Donoghue 1993), but this tends to be more useful in discriminating movement vs. rest (Williams et al. 2013) than in predicting directional information. Information in LFPs (at least in prefrontal cortex) varies with the depth of recording, with considerably greater information within the shallow cortical layers (<1 mm) than in layers deeper than 1 mm or on the cortical surface (Markowitz et al. 2011). Deeper cortical layers were shown to have more information in frequencies >300 Hz in that study, which would agree with a study showing greater SUA information in deeper layers (Parikh et al. 2009).

Motor cortical LFPs have also been used for online control of cursors in two dimensions, with performance that again nears that seen with spikes (Clancy et al. 2014; Flint et al. 2013; So et al. 2014). Furthermore, delta LFP power provides additional information to that of spikes in online control (Stavisky et al. 2015). LFP-controlled BMIs have been used less extensively than BMIs controlled with spikes; however, there have been demonstrations in laboratory animals (Clancy et al. 2014; Flint et al. 2013; So et al. 2014; Stavisky et al. 2015). A recent study combined high-gamma LFPs and spikes for online BMI control in paralyzed humans (Gilja et al. 2015). To date, there have been no concerted efforts to maximize the number of DOF in an LFP-based BMI; multiple studies have reported 2 DOF of BMI control (Flint et al. 2013; So et al. 2014; Stavisky et al. 2015). Instead, these BMIs are often developed with the intent of maintaining BMI performance when spike signals are poor or absent (Flint et al. 2012b; Stavisky et al. 2015; Wang et al. 2014). A combination of high movement-related information and lower bandwidth (and hence power) requirements compared with spikes makes LFPs an attractive candidate for a BMI input signal. However, design and implementation of a fully implantable LFP-based BMI will require clearing many of the same regulatory hurdles as a BMI based on spikes. Fortunately, this process is already underway, and future years may see delivery of a clinically viable BMI using intracortical signals.

ECoG signals.

ECoG signals furnish a potential BMI signal source that does not disrupt the cortical surface, which may prove desirable in long-term applications. ECoG has been used to decode reach direction (Ball et al. 2009) and arm kinematic trajectories in monkeys (Chao et al. 2010; Eliseyev and Aksenova 2014) and humans (Bundy et al. 2016; Ganguly et al. 2009; Gunduz et al. 2009; Hotson et al. 2014; Nakanishi et al. 2013; Pistohl et al. 2008; Schalk et al. 2007). It has also been used to decode discrete hand postures (Chestek et al. 2013), discrete finger movements, and finger kinematics (Acharya et al. 2010; Flint et al. 2017; Kubánek et al. 2009). Grasping objects involves both movement and isometric muscle activations. ECoG has been used to decode two discrete force levels in a reach-and-grasp task (Milekovic et al. 2012), as well as continuous grasp force from an isometric task with very high accuracy (Flint et al. 2014). EMG activity can also be decoded with moderate accuracy using higher density ECoG (Flint et al. 2014; Shin et al. 2012). Motor cortical ECoG activity has been used to control cursors, using biofeedback (Bundy et al. 2012; Leuthardt et al. 2004; Miller et al. 2010; Schalk et al. 2008), biomimetic (Milekovic et al. 2012), and hybrid/adaptive BMIs (Wang et al. 2013), and robotic hands and arms (Hotson et al. 2016; Yanagisawa et al. 2011, 2012). As with LFP-based BMIs, few efforts have been made to maximize DOF controlled by ECoG BMIs.

Evaluation of performance among studies is complicated by the facts that a wide range of electrode numbers, locations, and frequency bands have been used and that most early studies used low-density electrode arrays (10-mm interelectrode spacing), whereas more recent studies have used higher density arrays (3- to 4-mm interelectrode spacing) or sometimes much smaller microwire arrays (Bundy et al. 2014; Flint et al. 2014; Kellis et al. 2010; Leuthardt et al. 2009; Rouse et al. 2013). In addition, most online BMI studies have been conducted with human subjects undergoing invasive monitoring for epilepsy, over the course of a week or less. Exceptions include the studies of Wang et al. (2013) and Vansteensel et al. (2016). With limited time for practice, subjects are much less likely to become proficient in using the BMI, compared with animals or paralyzed humans with a longer term implant (e.g., as in Collinger et al. 2013; Hochberg et al. 2006, 2012). Even with these caveats in mind, ECoG BMI performance overall lags behind that of intracortical BMIs, and offline decoding information is somewhat lower in ECoG (using depth electrodes) than in spikes, or even than in intracortical LFPs (Markowitz et al. 2011). See Supplementary Table S1 for offline decoding performance results from exemplary studies.

One application for which ECoG may be particularly well suited is a speech BMI. Ventral motor cortical ECoG high gamma can distinguish different speech articulator activities (Bouchard et al. 2013) and can be used to decode individual speech sounds, or phonemes (Mugler et al. 2014; Pei et al. 2011). By combining these sounds, it should be possible to decode intended words for locked-in subjects. Indeed, ECoG has been shown to classify small sets of words (Herff et al. 2015; Kellis et al. 2010) with modest accuracy. Because the area of cortex devoted to speech and language production is large, and ECoG arrays easily cover a wide area, ECoG may be uniquely suited to this desired output.

A benefit of designing a clinically viable BMI using ECoG signals is that the recording devices and arrays have been in use for some time, in other clinical applications. A fully implantable ECoG BMI will likely benefit from these predicate devices, as early demonstrations have shown (Khanna et al. 2016; Vansteensel et al. 2016).

Epidural signals.

Given the success of motor BMIs controlled by LFP and ECoG signals, it is appealing to consider using epidural field potentials as BMI inputs. Modeling studies have shown that the dura has little effect on signal quality; rather, it is the CSF layer that causes some dispersion of the signal (Slutzky et al. 2010; Torres Valderrama et al. 2010). Signals recorded with epidural electrodes were shown to have similar mean spectral amplitudes in the high-gamma band to that of subdurally recorded signals when macro-sized (several millimeters in diameter) contacts were used (Bundy et al. 2014), which are the size used for clinical purposes. When microwire electrodes are used, there may be a substantial reduction in mean high-gamma power compared with subdural locations (Bundy et al. 2014). EFPs have been used to decode arm and hand movements in animals (Flint et al. 2012b; Gharabaghi et al. 2014a; Marathe and Taylor 2013; Shimoda et al. 2012; Slutzky et al. 2011). In humans, EFPs (recorded using macroelectrodes) contain nearly as much information about hand kinematics as ECoG (Flint et al. 2017), which suggests that they can indeed provide a useful signal source for BMIs. EFPs have been used to control online BMIs, both in monkeys (Marathe and Taylor 2013; Rouse et al. 2013) and in humans with epilepsy (Leuthardt et al. 2006), limb amputation (Gharabaghi et al. 2014b), and amyotrophic lateral sclerosis (ALS; Birbaumer et al. 2000).

One particular application for which EFPs seem particularly well suited is rehabilitation of function after stroke (Gharabaghi et al. 2014a), which has thus far been mainly performed using EEG (Ang et al. 2011; Ramos-Murguialday et al. 2013) or MEG (Buch et al. 2008), with one study using perilesional spikes, as well (Gulati et al. 2015). The combination of low invasiveness, good signal quality, and wide spatial coverage could make epidural BMIs attractive for a short-term implant designed to last a few months and designed to drive plasticity in the cortex (Soekadar et al. 2015). For example, the BMI could control a device providing somatosensory feedback to the cortex to simultaneously activate primary motor and somatosensory cortices (Daly and Wolpaw 2008; Ramos-Murguialday et al. 2013).

Clinical BMIs based on epidural potentials may be suitable for applications requiring less information bandwidth than is provided by other invasive signals. As with ECoG, a benefit of designing an EFP-based BMI is that it may be possible to facilitate regulatory approval by starting with already approved ECoG devices.

Signal Longevity

Intracortical signals.

Over time, the movement-related information content of single- or multiunit spike sources degrades, which is of primary concern in designing BMIs. The biggest reason for the drop off in spike signal recording quality is likely the electrodes themselves: the most widely used electrodes for recording intracortical signals for BMIs are typically microwires or silicon, both of which are mechanically stiff. The difference in mechanical stiffness between electrodes and the brain tissue can cause shearing, which unleashes an inflammatory cytokine cascade leading to electrode encapsulation (Griffith and Humphrey 2006; Polikov et al. 2005), with subsequent increase in effective electrode impedances (McConnell et al. 2009; Roitbak and Syková 1999; Turner et al. 1999). Some evidence suggests that this impedance increase may not be due to glial scarring, but rather to other inflammatory proteins adhering to electrical contacts (Malaga et al. 2016). In addition, the implantation itself causes chronic breaching of the blood brain barrier (Kozai et al. 2015; Potter et al. 2012; Saxena et al. 2013), which can allow influx of neurotoxic serum proteins, cytokines, and pro-inflammatory cells that lead to neuronal cell death (Abbott et al. 2006; Banks 2005; Biran et al. 2005; Hassel et al. 1994; Liu et al. 1999; Matz et al. 2001; Winslow and Tresco 2010). This reduces the ability to record spikes in the long term. Spike recording longevity is also impaired by neuronal loss from the inflammatory cascade (Biran et al. 2005), as well as mechanical degradation such as insulation breakdown (Barrese et al. 2013) and electrode breakage (Prasad et al. 2012). Thus spike recordings with current technologies typically have a half-life (i.e., the time at which half of spikes on an array remain) of months to a few years (Simeral et al. 2011). This may change with future technologies, including carbon fiber electrodes and special coatings (Patel et al. 2016).

Because LFPs are thought to represent summed activity from many thousands of neurons, it has long been hypothesized that they would exhibit greater longevity (that is, be observable for a longer time) than spikes. Limited evidence exists to support this hypothesis, in part because studies lasting many years are expensive and not often performed in laboratory animals. Simeral et al. (2011) found evidence in humans that significant mu (8–12 Hz)- and beta (12–30 Hz)-band power modulation remains on electrodes that have lost spiking activity in human subjects. Flint et al. (2012b). showed that the high-gamma band (70–300 Hz), as well as lower frequencies, still contains the same amount of information about movement when spikes are present or absent on a given set of electrodes. Wang et al. (2014). showed that decoding performance using LFPs was higher than that using MSPs or single units in monkeys after the number of spike channels began to wane, at around 300 days postimplantation. Furthermore, experiments in humans have successfully used high-gamma activity plus higher frequency activity (over 300 Hz, what some refer to as “hash”) to control BMIs with high performance even after spikes could no longer be recorded (Gilja et al. 2015). Finally, Hall et al. (2014) devised an approach to estimating spiking rates from LFPs recorded in the area, which could provide more stable “inferred” spiking to use in BMIs.

ECoG.

ECoG electrodes are typically low impedance, which means they have a large listening sphere and record from hundreds of thousands of neurons. Thus, in theory, ECoG recordings should last a long time: as long as, or possibly longer than, LFPs. Some studies have used microcontact ECoG electrodes (Bundy et al. 2014; Flint et al. 2014; Kellis et al. 2016; Rouse et al. 2013) with higher impedances, which could possibly alter their longevity; newer polymer-based methods have enabled lower impedances in smaller contacts (Schendel et al. 2014a; Thongpang et al. 2011). In practice, most ECoG BMI studies in humans have lasted less than a week, because that is the typical duration of clinical ECoG monitoring. Some human ECoG studies for epilepsy have shown that ECoG recordings can provide epilepsy-related information for at least 2 yr during passive recording (Cook et al. 2013) and up to a mean of 5.4 yr in a system using responsive neurostimulation (Bergey et al. 2015; Heck et al. 2014). These studies did not describe the quality of the ECoG information that was recorded, however. Furthermore, clinical recording systems typically low-pass filter ECoG at ~30 Hz, so it is unclear from these studies how high-gamma activity, in particular, behaves in the long term.

Studies in monkeys using BMIs have demonstrated that ECoG can be used to decode arm movements with good accuracy for at least 6 mo (Chao et al. 2010) to more than a year (Wang et al. 2014). Most recently, a wireless ECoG BMI was implanted into a person with ALS with locked-in syndrome. This BMI was used by the patient at home with little assistance and enabled use for 262 days (Vansteensel et al. 2016). The amount of information obtained from ECoG was limited in this study, because the tasks were binary and used a biofeedback BMI. The overall signal quality in terms of signal to noise was not discussed in the report, but the subject was still using the system at the end of the study with similar accuracy to that at the beginning of the study. Another experiment using the same investigational device allowed subjects with Parkinson’s disease to modulate their cortical beta-band power using a neurofeedback paradigm; these sessions occurred from 11 to 18 mo following implantation (Khanna et al. 2016). Thus the limited evidence available supports high longevity of ECoG, but further study of this property is warranted.

Epidural signals.

The potential for using EFPs as BMI signal inputs remains largely untapped. Given that EFPs are very similar to ECoG signals in origin (postsynaptic potentials derived from hundreds of thousands of neurons) and quality, it seems likely that EFPs will have very similar longevity, as well. However, there exists at the moment extremely little evidence to bear on these properties of EFPs, and more study is warranted.

Signal Stability

A clinically viable BMI should exhibit stable performance properties over years. To a large extent, stability in BMI performance depends on stability of the movement-related information in the signals themselves. As stated earlier, stability is vital for two reasons: 1) enabling users to achieve proficient control and 2) reducing the need for decoder recalibration. Although recent techniques have been shown to improve the response of BMIs to signal variability over the course of days to months using longer data sets and statistical properties of spikes (Jarosiewicz et al. 2015; Sussillo et al. 2016), stability is still an important design criteria in an implanted BMI that is in use for decades.

Intracortical signals.

Signal stability can be defined in multiple ways. For spikes, one definition is the waveform characteristics of the spikes themselves (Chestek et al. 2011; Dickey et al. 2009). A more relevant measure for BMI use is the stability of a signal’s movement-related information over time, which does not correlate highly with spike waveform characteristics (Chestek et al. 2011). Spikes were initially thought to be relatively unstable in their information about reaching over a few hours (Carmena et al. 2005; Cohen and Nicolelis 2004; Rokni et al. 2007). However, subsequent investigations showed that spikes were highly stable in the short term. Chestek et al. (2007) built linear decoders between M1 single-unit firing rates and reach velocity, and found that correlations between decoded and actual velocity remained stable for 1–2 days (Fig. 2A; see also Nicolelis and Lebedev 2009 for discussion of this issue). Stevenson et al. (2011) showed that prior estimates of variability in signal preferred directions over time (Rokni et al. 2007) were likely largely due to measurement noise. Indeed, Ganguly and Carmena (2009) examined preferred directions of a highly selected set of single units in M1 during BMI use and found that they remained stable for up to 19 days when the BMI decoder was held constant (Fig. 2B). It was not clear until recently, however, that BMI performance or signals could remain stable for much longer than that.

Fig. 2.

Short-term stability of single-unit spikes. A: single-unit decoder performance (correlation coefficient ρ between decoded and actual hand velocity) in 10-min epochs during reaching movements, sorted by value in the first epoch. [Modified from Chestek et al. (2007) with permission.] Units were highly stable over 2–3 days. B, top: color maps represent the cortical ensemble tuning curves on 3 different days. Each row is the tuning curve of one neuron. Bottom, color map of pairwise correlations of ensemble tuning for each pair of days. Warmer colors denote higher correlations (R). [Modified from Ganguly and Carmena (2009) with permission.]

Flint et al. (2013) recently showed that monkeys could use a biomimetic BMI decoder based on LFPs for over a year without a need for recalibration or even daily use, while maintaining high performance in that time. Performance using multiunit spikes was also largely stable for almost 6 mo. Since then, another study has shown that spike control can be stable, while keeping the decoder constant with extremely high performance, for up to 22 mo in one monkey and a few weeks in another monkey (Nuyujukian et al. 2014). That study hypothesized that more stable performance in one monkey was due to having more spikes that contributed highly to BMI control (which they termed “equipollence”) so that if a few spikes became unstable, they would have less influence on overall decoding. Neither of these studies specified whether the high stability was due to stable signals or adaptations that the brain made to adjust to variations. By contrast, others have measured a gradual decrease in offline decoding accuracy over a span of months (Perge et al. 2014) to years (Wang et al. 2014). Our group recently analyzed BMI input signal stability in more detail, both during BMI control with stable performance and during manual two-dimensional (2D) planar reaching (Flint et al. 2016). We used both the preferred directions and single unit (or LFP feature) decoders to examine the stability of spikes and LFPs. We built a decoder for each recording epoch of BMI control and used it to decode the cursor velocity during the final five epochs of BMI control and then computed the R² between decoded and actual velocities. Movement information stability was higher at the mesoscopic (LFP) scale than at the MSP scale for up to 3 yr (Fig. 3), which could suggest that the brain is capable of different levels of control on LFP and single-neuron scales (Engelhard et al. 2013; Ganguly and Carmena 2009), although it could also be due to signal averaging at the mesoscopic scale.

Fig. 3.

Long-term stability of multiunit spikes (MSPs) and LFPs. Single-feature decoders were used to decode the final 5 epochs of actual cursor velocity during hand control, and the performance is shown as color for LFPs (A) and MSPs (B). C: the stability index for single-feature decoding (SI_SFD), the average of the correlation map as in Fig. 2B over time, shows that both LFPs and MSPs are stable (close to 1) for almost 3 yr, although MSPs are somewhat less stable. D and E: ensemble tuning patterns, similar to that shown in Fig. 2B, are shown for each feature in LFPs and MSPs. Color represents the band power, LMP amplitude, or spike rate in each direction of movement. F: the stability index for ensemble tuning (SI_ET) shows extremely stable LFP tuning and moderately stable MSP tuning. G–I: similar findings to A–C but for brain control data for 200 days. J–L: same as D–F but during brain control. [Modified from Flint et al. (2016) with permission.]

In addition, a subset of MSPs was highly variable during BMI use (Flint et al. 2016). To investigate this finding further, we used singular value decomposition to partition the Wiener filter into a task-relevant space (the 2D cursor control space) and a task-null space (the subspace of variation not in the task-relevant space). We projected each neural vector onto each of the singular vectors. Similarly to the single feature decoder analysis, for each BMI control epoch we built two decoders of the BMI-controlled cursor velocity, one based on the task-relevant projections and one based on the null-space projections.

The performance of the null-space decoders was much more variable over time than that of the task-relevant decoders. This was true for both spikes and LFPs. These findings could provide an explanation for the conflicting evidence of neuronal stability presented in some prior studies. Moreover, these results support the minimal intervention principle (MIP) of optimal feedback control, a computational theory of how the brain controls movement (Todorov and Jordan 2002). The MIP hypothesizes that the brain minimizes effort by controlling only those movement components that are relevant to the task goals while allowing variability in those movement components that are unrelated to the task goals. This principle accounts for the simultaneous observation of highly accurate control of movements in a particular task with substantial trial-to-trial variability in the way the limb or body moves while achieving the task goal (Todorov 2004). The principle predicts low variability in the task-relevant space and higher variability in the task-null space. Although substantial evidence supporting this principle existed from behavioral studies (Latash et al. 2001; Scholz and Schöner 1999; Scott 2004; Todorov and Jordan 2002), these results were the first neural evidence of the MIP in the cortex itself (Flint et al. 2016). From a BMI viewpoint, these results indicate that projection into the task-relevant space will maximize signal stability.

ECoG and epidural signals.

Because ECoG and EFPs are effectively the sum of many LFP currents, they should exhibit high stability, as well. However, there have been very few studies addressing ECoG/EFP stability, largely due to the limited time window for recording in the vast majority of studies. One ECoG study in humans showed that a biofeedback BMI using the high-gamma power from one motor cortex site with the decoder held constant enabled stable 1D cursor performance over 5 days (Blakely et al. 2009). In monkeys, offline decoding of 2D reaching kinematics using 32-channel ECoG grids demonstrated stable decoding performance over 5–6 mo (Chao et al. 2010). The BMI demonstrated by Vansteensel et al. (2016) provides the most convincing human results suggesting stability to date. In that study, the subject used fixed decoders from 94 to 247 days (on 3 different binary tasks) and maintained high performance during that time. The high-gamma signal used in that BMI also remained highly correlated with movement vs. rest imagery for the duration of the study (Vansteensel et al. 2016). Note, however, that signals were calibrated for z scoring on a daily basis; small drifts in the baseline signals would not have been detected with this method. Epidural signals should have similar stability properties to ECoG, but as yet there have been no studies to provide evidence to prove this assertion.

Discussion and Future Directions

The field of brain machine interfaces has exploded in the last 10–15 yr. This review has attempted to summarize some of the important factors in choosing input signals and designing the BMI according to the desired application. We have reviewed both offline decoding and online (closed loop) control experiments. Although there is not necessarily a perfect correlation between offline and online BMI performance, well-constructed offline decoding experiments can provide valuable insight about the operation of the unimpaired motor system. With the use of current recording technologies, intracortical signals contain the most information, whereas surface potentials can sample larger areas more easily. This makes surface potentials attractive for applications that may require access to large areas of the cortex, such as speech and rehabilitative BMIs. For control of high-DOF applications such as prosthetic limbs and FES, intracortical signals offer higher information, but spikes currently are limited in terms of longevity, as well as stability (Fig. 4).

Fig. 4.

Schematic of the trade-offs between signal type (invasiveness), movement-related information, and stability. Dashed lines denote limited or missing evidence and are based on hypotheses.

Many research groups across the globe are working to improve the longevity, stability, and quality of intracortical and surface electrodes. To improve intracortical spike longevity, multiple approaches are being tested. These include reducing the mechanical stiffness of the electrodes including using carbon nanofibers (Kozai et al. 2012; Patel et al. 2016) or polymers (Hara et al. 2016; Ware et al. 2014); reducing shear forces by eliminating the tethering of the array to the skull (i.e., making the device completely implanted and wireless (Borton et al. 2013; Kennedy et al. 2000; Sodagar et al. 2009); reducing inflammation using anti-inflammatory drugs (Shain et al. 2003; Zhong and Bellamkonda 2007), electrical stimulation (Otto et al. 2006), or changes to the nanostructure or coatings of the electrode shanks, which promise to reduce the foreign body response (Grill et al. 2009; He et al. 2006; Kotov et al. 2009; Kozai et al. 2012; Sommakia et al. 2014); and using neurotrophic factors to promote growth near the electrodes (Kennedy et al. 1992; Lee et al. 2016).

Multiple approaches are being used to improve surface electrode recordings, as well. The simplest is improving the spatial resolution—optimal ECoG spatial resolution has been estimated at ~600 μm in rats (Ledochowitsch et al. 2013; Slutzky et al. 2010) and 1–2 mm in humans (Slutzky et al. 2010)—which should also improve information (Flint et al. 2014; Wang et al. 2016). Increasing the number of electrodes into the hundreds or thousands introduces practical issues of many leads coming out from the arrays, which can increase the risk of infection. This issue may require active electronics on the array to multiplex the signals (Castagnola et al. 2015; Viventi et al. 2011). An alternative to avoid this issue and potentially enable much larger scale recordings is a modular system of “dust mote”-sized electrodes that can sense and transmit wirelessly to a transceiver (Seo et al. 2016). Electrodes with greater flexibility or moldability will enable closer approximation to the cortex and likely better information (Kim et al. 2010; Morris et al. 2015). Use of a mesh microgrid can reduce meningeal growth under the array (Schendel et al. 2014b). Another promising direction is reducing impedance on smaller contacts by using different materials or coatings (Cui and Martin 2003; Fattahi et al. 2014; Green et al. 2008; Khodagholy et al. 2011, 2013; Schendel et al. 2014a), which has even enabled spike recording from the cortical surface (Khodagholy et al. 2015, 2016).

In addition to hardware innovations, software innovations offer the promise of improved ability to extract information from cortical signals that is informative, stable, and long lasting. In particular, transforming the signals into different feature spaces can provide important insights into the neurophysiology. Chase et al. (2010) showed that using latent inputs (mathematical estimates of the inputs to motor cortical neurons) instead of spikes with a BMI enabled higher performance. As mentioned above, relying more on task-relevant projections could improve stability (Flint et al. 2016). Hall et al. (2014) showed that low-frequency LFPs can be used to predict local spiking activity by projecting the LFP information into principal components space. These predictions of firing rates were stable for just over a month. In addition, monkeys were able to use these predicted firing rates to control a BMI. This could enable stable performance to continue even after spikes cannot be recorded.

Stability could also be affected by processes such as neural plasticity and learning in general. Plasticity could have either a destabilizing or stabilizing effect on neural signals, depending on the rate of change in learning and whether the learning was only transient (e.g., Gilja et al. 2012) or ongoing throughout a BMI experiment (e.g., Orsborn et al. 2014; Taylor et al. 2002). There is evidence that learning to use a BMI employs strategies (Hwang et al. 2013) and brain networks (Koralek et al. 2012; Wander et al. 2013) similar to those used in learning other motor tasks (namely, prefrontal, premotor, and posterior parietal cortices and striatum). In addition, plastic changes are seen during sleep after BMI use (Gulati et al. 2014). Thus applying principles from the decades of motor learning research, and effects from sleep, to BMIs could also help us to improve BMI performance (Orsborn and Carmena 2013). Examining the neural activity changes during BMI learning can also guide BMI design. For example, gradually adapting decoders appear to enable smooth learning profiles and skill learning (Li et al. 2011; Orsborn et al. 2014). This corroborates findings from learning a human-machine interface task (Danziger et al. 2009). Sadtler et al. (2014) also studied neural behavior during BMI learning. By projecting the neural activity into a smaller dimensional space using factor analysis, they found that there was a manifold within which the neural data were completely constrained during baseline BMI use. They then perturbed the BMI decoder and let the monkeys attempt to relearn the new decoder. The monkeys were quickly able to learn perturbations that still enabled neural activity to remain within the manifold. However, they had much more difficulty learning perturbations that required neural activity outside of the manifold. This implied that the motor cortex has intrinsic connectivity (whether anatomic or functional) that constrains its ability to learn new behaviors. Thus knowledge of the underlying neurophysiology of cortical signals is critical to designing clinically viable neural interfaces.

GRANTS

This work was supported by National Institutes of Health Grants K08NS060223 and R01NS094748, Defense Advanced Research Projects Agency Grant N66001121-4023, Brain Research Foundation Grant BRF SG 2009-14, the Northwestern Memorial Foundation Dixon Translational Research Grant Program (supported in part by NIH Grant UL1RR025741), Paralyzed Veterans of America Grant 2728, Doris Duke Charitable Foundation Clinical Scientist Development Award 2011039, and the Craig H. Neilsen Foundation.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

M.W.S. and R.D.F.I. conceived and designed research; M.W.S. and R.D.F.I. performed experiments; M.W.S. and R.D.F.I. analyzed data; M.W.S. and R.D.F.I. interpreted results of experiments; M.W.S. and R.D.F.I. prepared figures; M.W.S. and R.D.F.I. drafted manuscript; M.W.S. and R.D.F.I. edited and revised manuscript; M.W.S. and R.D.F.I. approved final version of manuscript.