Brain machine interfaces: powerful tools for clinical treatment and neuroscientific investigations

Abstract

Brain machine interfaces (BMIs) have exploded in popularity in the past decade. BMIs, also called brain computer interfaces, provide a direct link between the brain and a computer, usually to control an external device. BMIs have a wide array of potential clinical applications, ranging from restoring communication to people unable to speak due to amyotrophic lateral sclerosis or a stroke, to restoring movement to people with paralysis from spinal cord injury or motor neuron disease, to restoring memory to people with cognitive impairment. Because BMIs are controlled directly by the activity of pre-specified neurons or cortical areas, they also provide a powerful paradigm with which to investigate fundamental questions about brain physiology, including neuronal behavior, learning, and the role of oscillations. This article reviews the clinical and neuroscientific applications of BMIs, with a primary focus on motor BMIs.

Introduction

Brain machine interfaces (BMIs), also called brain computer interfaces (BCIs), are neural prostheses that allow devices to communicate directly with parts of the brain, usually the cerebral cortex. The most common types of BMIs, those based on event-related potentials, evoked potentials, or motor-related potentials, enable users to communicate by spelling words, control computer cursors, control prosthetic limbs or even reanimate paralyzed muscles. BMIs thus have the exciting potential to restore substantial function to people with paralysis from neurologic disorders, such as spinal cord injury, motor neuron disease, or stroke, that have minimal meaningful treatments at present. In addition, BMIs provide a powerful way to investigate brain function, for a variety of reasons. This article reviews both the clinical and neuroscientific applications of BMIs, with a focus on motor BMIs.

Development of BMIs

BMIs have roots in neurofeedback, i.e., operant conditioning of brain signals. Experiments in animals and humans demonstrated the ability to operantly condition components of visual and auditory evoked responses (Fox and Rudell, 1968; Rosenfeld and others, 1969). Around the same time, Eberhardt Fetz operantly conditioned a monkey to move a gauge based on the firing rate of a single neuron in primary motor cortex (M1) in order to receive food reward (Fetz, 1969). Inspired by these demonstrations of the ability to modulate brain activity, Vidal proposed the idea of using operant conditioning for the purposes of controlling a computer, though he did not implement a BMI (Vidal, 1973). In 1988, Farwell & Donchin showed that an event-related potential, the P300 response, could be used to communicate with a computer (Farwell and Donchin, 1988). The P300 can be recorded broadly from scalp EEG in response to an unexpected stimulus. This response was used in a BMI to spell out letters by displaying all letters on a computer screen and flashing groups of them at a time; when the user saw the letter he was concentrating on light up, his brain produced a P300 response. A similar BMI paradigm used steady-state visual evoked potentials (SSVEPs) recorded with EEG to type numbers (Cheng and others, 2002).

Later BMIs used signals from the motor cortex to control computer cursors. This concept was inspired by advances in understanding of the basic neurophysiology of the cortical control of movement. For example, Humphrey showed that the activity of a monkey’s motor cortical neurons could be used to estimate the movement of the monkey’s arm in a reaching task (Humphrey and others, 1970), and later, Georgopoulos et al. showed that these neurons each had a “preferred direction” of reaching in which they were more active than in other reach directions (Georgopoulos and others, 1982). Pfurtscheller and colleagues demonstrated that decreases and increases in EEG mu (8–12 Hz) and beta (12–25 Hz) band power from the motor areas could indicate the onset and offset of movement, respectively (Pfurtscheller and Aranibar, 1977). This translated into the first human biofeedback BMI, in which participants learned to increase or decrease power in different frequency bands by moving, or imagining moving, a body part (Wolpaw and others, 1991; Pfurtscheller and Neuper, 2001). Many similar EEG-based BMIs have been implemented using other frequency bands, including very low-frequency signals in patients with amyotrophic lateral sclerosis (ALS; Birbaumer and others, 2000).

The development of intracortical multielectrode arrays in the 1990s (Rousche and Normann, 1998; Nicolelis and others, 2003) enabled scientists to record signals simultaneously from many individual neurons, and this in turn enabled more sophisticated intracortical BMIs to be developed, first in animals (Serruya and others, 2002; Taylor and others, 2002; Carmena and others, 2003; Musallam and others, 2004) and then in humans (Hochberg and others, 2006). These BMIs are considered “biomimetic,” in that they are based on building a decoder of normal movement or muscle activity to translate the cortical activity to control of an external device.

Thus far I have discussed mainly efferent BMIs, i.e., BMIs that translate brain activity to control some output. In addition to these, there are several other types of BMIs that have been developed. These include “passive” BMIs (Zander and Kothe, 2011) that do not attempt direct control, sensory BMIs (London and others, 2008; O’Doherty and others, 2011a; Bensmaia and Miller, 2014b) and BMIs that aim to replace memory function (Ezzyat and others, 2018; Hampson and others, 2018). This review will focus on efferent BMIs.

Signal sources

As was mentioned above, BMIs can be controlled using a variety of signals from the brain. These signals can be obtained noninvasively, using measures of electrical potentials at the scalp (EEG), magnetoencephalography (MEG), and blood flow (fMRI and functional near infrared spectroscopy, or fNIRS). Control signals can also be obtained invasively (Figure 1), using signals on the surface of the pia mater (subdural signals, also called electrocorticography, or ECoG), the surface of the dura (epidural field potentials, or EFPs), or intracortically (spikes or local field potentials, LFPs). Here I briefly review each of these modalities; for greater detail, refer to the reviews in (Wolpaw and others, 2002; Waldert and others, 2009; Jackson and Hall, 2017; Slutzky and Flint, 2017).

Figure 1.

Electrical signals from the brain can be obtained at multiple levels of invasiveness (A), from EEG to intracortical signals (ICS). B) Examples of time-domain signals obtained invasively. (Modified from Slutzky and Flint (2017) with permission from publisher.)

Blood-flow signals

The measurement of blood-flow related signals can give an approximation to neuronal activity, and indeed, the BOLD (Blood Oxygenation Level Dependent) signal of fMRI has been shown to correlate with both low- and high-frequency field potentials (Logothetis, 2003). Recent advances in MRI technology has enabled subjects to control the BOLD signal in a given area of the brain (Sulzer and others, 2013); changing responses in visual cortex has even been shown to modify perception (Shibata and others, 2011). However, real-time fMRI has poor temporal resolution, is expensive and limited in availability, and therefore not practical to use for applications such as motor BMIs. fNIRS is much less expensive and is portable, and has been used to control communication devices, including for locked-in patients (Gallegos-Ayala and others, 2014; Chaudhary and others, 2017). However it suffers from limited signal-to-noise and spatiotemporal resolution.

Noninvasive electromagnetic signals

Electrical signals are by far the most common BMI signal sources, and the most common of these is scalp EEG. A comprehensive review of EEG-based BMIs is beyond the scope of this review, but I attempt to summarize a few of the more prominent studies. BMIs have used multiple features of EEG for control. These include evoked responses such as P300 (Farwell and Donchin, 1988; Birbaumer and others, 2000; Guan and others, 2004), SSVEPs (Wang and others, 2006; Gangadhar and others, 2009) or contingent negative variation (Chen and others, 2015). These responses have typically been used for binary switches or for spelling words by selecting one letter at a time. Time domain responses, called slow cortical potentials, have also been used for controlling spelling applications (Birbaumer and others, 2000). As mentioned above, the frequency domain signals of EEG can also be used for control signals. For efferent BMIs, the mu (8–12 Hz) and beta (12–25 Hz) frequency bands are used most often, as they correlate with motor activity (Pfurtscheller and Aranibar, 1977). For example, modulating the power in these bands can be used to move a cursor in 2 or even 3 dimensions (Wolpaw and McFarland, 1994; Wolpaw and McFarland, 2004; McFarland and others, 2010). Due to the limited spatial and spectral resolution of EEG (Nunez and Srinivasan, 2006; Slutzky and others, 2010), applications using spectral power are usually of the biofeedback variety, requiring multiple sessions of training to achieve good performance. EEG also requires donning and doffing caps for each session, usually with application of gel as well.

MEG has multiple advantages over EEG, including the ability to record signals from the cortical sulci and higher bandwidth, enabling some gamma band (~30–60 Hz) range to be recorded (Nowak and others, 2009). It has been used in several studies to control BMIs with mu, beta, and even gamma band power (Mellinger and others, 2007; Buch and others, 2008). However, it has the major disadvantage of being extremely expensive due to the use of superconducting materials, and very few such devices are available worldwide, although a recent advance may make them a bit more practical (Boto and others, 2018).

Invasively recorded electrical signals

Intracortical recordings can be separated into local field potentials (LFPs), by bandpass filtering from 0.5–300 Hz, or action potentials (spikes) by filtering in higher band (300 Hz-12 kHz) and thresholding. Spikes can be sorted into single units (single neurons) or simply thresholded, also called multiunit spikes (MSPs). Since many of the early intracortical BMI studies grew out of motor cortical neurophysiology experiments that used single-unit spikes (or single-unit activity, SUA), most of these early BMIs also used ensembles of SUA for control. More recent BMI studies have started using MSPs and LFPs. LFPs are extracellular potentials that are hypothesized to be produced by the summation of many thousands of local, mostly postsynaptic, potentials (Mitzdorf, 1985; Buzsáki and others, 2012).

Single-unit spikes in general contain the most information about multiple motor parameters, including reach target (Shen and Alexander, 1997), reaching and grasp kinematics (Moran and Schwartz, 1999; Wessberg and others, 2000; Vargas-Irwin and others, 2010; Aggarwal and others, 2013), arm joint torques (Evarts, 1968; Fagg and others, 2009), and arm muscle activity (Morrow and Miller, 2003; Pohlmeyer and others, 2007). Multiunit spikes (also called threshold crossings) provide essentially equivalent information to single units about reaching (Stark and Abeles, 2007; Fraser and others, 2009; Chestek and others, 2011), with one study showing a small improvement from spike sorting (Todorova and others, 2014). Multiunit activity has also been represented as a continuous, rather than discrete, signal by using the power in the 300–6000 Hz band (Stark and Abeles, 2007). The vast majority of intracortical BMIs have used spikes. Spike-based BMIs face substantial challenges for clinical translation, including limited recording longevity (typically, limited to a few years for the majority of electrodes in an array {Barrese 2013) and high power requirements due to high sampling rates and processing needs.

LFPs have long been known to contain information related to movement and muscle activity {Sanes, 1993 #746;Murthy, 1996 #749}. LFPs in posterior parietal areas have equal or superior information to spikes for distinguishing the direction and state, respectively, of oculomotor saccades (Pesaran and others, 2002). LFPs from the motor cortex were long thought to have less movement-related information than spikes (Stark and Abeles 2007). However, recent studies demonstrated that LFPs are nearly as informative as spikes about kinematics (Bansal and others, 2011; Flint and others, 2012b) and muscle activity (Flint and others, 2012a) during reaching and grasping. The high gamma band (~70–300 Hz) of LFPs is the most informative (Rickert and others, 2005; Zhuang and others, 2010; Flint and others, 2012a; Flint and others, 2012b), while the local motor potential (LMP; smoothed time domain signal) of the LFP also is informative. The LMP is dominated by the lowest frequency (0–4 Hz) delta band (Rickert and others, 2005; Flint and others, 2012a), and there is a considerable amount of information in the phase of the delta band as well (Saleh and others, 2010). The mu (8–12 Hz) and beta (12–30 Hz) bands also contain substantial movement-related information (Sanes and Donoghue, 1993; Murthy and Fetz, 1996), but this tends to be more useful in discriminating movement vs. rest (Williams and others, 2013) than in predicting directional information. LFPs have some potential advantages over spikes for clinical translation, including greater longevity and stability (see below), since they are summed from many thousands of neurons. They also have lower bandwidth requirements than spikes, which translates to lower power needs.

Subdural signals (ECoG) are field potentials recorded with electrodes placed on top of the pia mater. Such recordings are used in some people with epilepsy who require invasive monitoring to identify their seizure focus prior to surgical resection. ECoG has increasingly been investigated for BMI use as well as other neuroscientific investigations (Crone and others, 1998), starting with Leuthardt et al. demonstrating a 2D biofeedback BMI (Leuthardt and others, 2004). ECoG high gamma band has been found to contain substantial information about reaching (Schalk and others, 2007; Pistohl and others, 2008; Ganguly and others, 2009; Chao and others, 2010; Hotson and others, 2014; Bundy and others, 2016), finger movements (Kubanek and others, 2009; Flint and others, 2017), grasp force (Flint and others, 2014), gait (McCrimmon and others, 2017), as well as various aspects of speech and language (Pei and others, 2011; Bouchard and others, 2013; Mugler and others, 2014; Herff and others, 2015). Higher resolution ECoG electrode arrays (< 5 mm interelectrode spacing) provide higher accuracy than the standard 10 mm resolution used for clinical recordings (Flint and others, 2014) (Wang and others, 2016), yet can still cover a broad area of the cortex, which makes ECoG particularly well-suited for BMIs decoding speech movements, which have broad representation in motor cortices.

Epidural field potentials (EFPs) have also been investigated as potential BMI inputs. EFPs have been used to decode arm and hand movements in animals (Slutzky and others, 2011; Flint and others, 2012b; Shimoda and others, 2012). When using standard macro-ECoG contacts, there is little difference in high-gamma power (Bundy and others, 2014) and movement-related information (Flint and others, 2017) between subdural and epidural recordings. EFPs have been used to control online BMIs in monkeys (Marathe and Taylor, 2013; Rouse and others, 2013) and humans (Birbaumer and others, 2000; Leuthardt and others, 2006; Gharabaghi and others, 2014). EFPs share similar properties with ECoG, with additional advantages of being less invasive and therefore, potentially safer.

BMIs for functional restoration

The goal of most BMIs is to restore function of people with impairments from neurologic conditions. These can range from the locked-in syndrome (complete paralysis, sometimes sparing the eyes) from brainstem stroke or motor neuron disease, to tetraplegia from spinal cord injury or cerebral palsy, to hemiplegia from stroke or traumatic brain injury, to memory impairment from neurodegenerative diseases like Alzheimer disease. I briefly summarize studies that have shown promise for treatment of these conditions.

Restoring communication to people without articulatory or limb movement

BMIs were first developed to restore communication, which is lost or severely impaired in people with locked-in syndrome (LIS). LIS can be caused by amyotrophic lateral sclerosis (ALS), other motor neuron diseases, brainstem stroke, or rarely by muscular dystrophy and cerebral palsy. BMIs that use EEG-based evoked responses such as P300 or SSVEPs, or sensorimotor rhythm BMI (Kübler and others, 2005) have been used in people with severe tetraparesis and dysarthria from cerebral palsy (Pires and others, 2011; Daly and others, 2014) or with LIS (Birbaumer and others, 1999; Kubler and others, 2001; Sellers and others, 2014). In general, these studies provided a rate of communication of only a few letters per minute, and patients with even minimal muscle control still preferred communicating with more conventional assistive devices such as eye gaze trackers or binary switches, that can provide higher communication rates. Further, the need for a caregiver to support the donning and doffing of the EEG cap and setting up the system on a daily basis presents substantial challenges to adherence to the protocol. fNIRS has recently been used in patients with LIS, even those completely locked-in; however, the communication rate was even lower than that of EEG-based BMIs (Chaudhary and others, 2017), and accuracy was about 70% on binary (yes/no) classification. Intracortical BMIs have been used in LIS as well, starting with a cone electrode with two contacts in two patients to provide simple but slow spelling (<3 letters/min; Kennedy and others, 2000). More recent work has used multielectrode intracortical arrays in patients with ALS or pontine stroke, who had partial LIS or were nearly locked-in, to provide communication by controlling a computer cursor to point and click among displayed letters (Bacher and others, 2015; Jarosiewicz and others, 2015). This provided substantially higher communication rates, up to 32 letters/min in one recent study (Pandarinath and others, 2017). Importantly, this rate was better than one of the subjects’ assistive device performed and moreover, was better than what 72% of ALS patients said was acceptable in a survey (Huggins and others, 2011); it could also be improved using predictive spelling enhancements. Still, this performance falls far short of texting (12–19 words/min; (Hoggan and others, 2008), let alone speaking (90–170 words/min; (Venkatagiri, 1999). However, almost all intracortical human studies to date (except (Guenther and others, 2009) have used a percutaneous implant and the BMIs could not be used unless lab members were present. One recent study demonstrated ad-lib home use of a simple but fully-implanted ECoG-based biofeedback BMI to allow typing and playing simple games (Vansteensel and others, 2016). Performance was modest, but it provided the first example of 24-hour availability in an implanted BMI.

To increase communication rates, several groups are working toward decoding speech directly from the cortex, and several studies have examined the ability to decode speech offline. One early study examined the potential to decode words from ECoG in Wernicke’s area with modest accuracy (Kellis and others, 2010), while another used spikes from multiple areas in the temporal lobe to classify isolated vowels. More recent studies have focused on decoding the smallest speech sounds, called phonemes, as components of words. The ventral motor cortex contains substantial information about phonemes (Bouchard and others, 2013). ECoG from this area has been used to classify among vowels within words (Pei and others, 2011) and among all English phonemes within words (Mugler and others, 2014), the type or manner of phoneme production (Lotte and others, 2015), and sentence production (Herff and others, 2015). Spikes from ventral M1 were also used to decode among all 38 isolated English phonemes with accuracy up to 21% (Brumberg and others, 2011). Two studies have demonstrated closed-loop control of speech BMIs; one with cursor control representing 2 different vowels (Leuthardt and others, 2011), and one using decoding of the first and second formants to artificially synthesize up to 8 vowel sounds with synthesized feedback (Guenther and others, 2009). A recent study also demonstrated real-time decoding during listening to sentences in superior temporal gyrus (Moses and others, 2018). These results demonstrate the promise of speech decoding for BMIs, and several companies have been formed with this promise in mind (Strickland, 2017).

Restoring limb function to people with tetraplegia or limb amputation

Efferent BMIs have been used to control not only communication devices, but also robotic limbs and FES. This could help patients with amputated limbs, or tetraplegic patients. The first robotic prostheses controlled by M1 spikes with feedback were one-dimensional: a rat moving a sipper tube (Chapin and others, 1999), opening and closing of a prosthetic hand by a man with a cervical spinal cord injury (Hochberg and others, 2006), and a monkey that controlled reaching of a robotic gripper (Velliste and others, 2008). Monkeys with arm amputations have also used BMIs to control robotic reach and grasping (Balasubramanian and others, 2017; Vaidya and others, 2018). In humans, BMIs have been used to control robot arms (Figure 2; (Hochberg and others, 2012) and prosthetic arms (Collinger and others, 2012) with up to 10 degrees of freedom (Wodlinger and others, 2014). Such control could benefit people with limb amputations. In addition, a hybrid BMI using EEG and electro-oculograms was used to control a robotic hand exoskeleton that restored simple but functional grasping to people with low tetraplegia (Soekadar and others, 2016). Noninvasive BMI-controlled leg exoskeletons are also being studied to restore walking to paraplegic patients (Do and others, 2013; López-Larraz and others, 2016; He and others, 2018). One study in SCI patients suggested that BMI-controlled exoskeleton training might help patients improve walking (Donati and others, 2016). Paired with advances in exoskeleton technology, BMIs could prove an intriguing option for restoring function.

Figure 2.

BMIs have been used to replace or rehabilitate function. Patients with tetraplegia have used BMIs to control (A) robotic/prosthetic arms and (B) functional electrical stimulation of reach and grasp. Images modified from Hochberg et al. (2012) and Ajiboye et al. (2017) with permission from publisher. C) BMIs can help to rehabilitate function after stroke. Here, stroke subjects used biofeedback of the mu-beta sensorimotor rhythm to move the hand using an exoskeleton. (Modified from Ramos-Murguialday et al. (2013) with permission from publisher.)

EEG-based BMIs have also been used to control binary FES (open/close) of the hand for grasping in humans (Lauer and others, 1999; Pfurtscheller and others, 2003; Muller-Putz and others, 2005). Invasive BMIs have been used to control proportional FES of temporarily paralyzed muscles in monkeys, starting with biofeedback-based (Moritz and others, 2008) and biomimetic wrist flexion and extension (Pohlmeyer and others, 2009) and continuous grasping (Ethier and others, 2012). In two tetraplegic people, invasive BMIs have reanimated paralyzed muscles to restore 3 different types of grasp (Bouton and others, 2016) as well as supported reaching plus grasping of up to 3 joints (Ajiboye and others, 2017) (Figure 2). BMI-controlled FES of leg muscles has been used to restore walking, with weight support (McCrimmon and others, 2015). BMI-controlled spinal epidural stimulation has also been shown to restore walking function to partially paralyzed monkeys (Capogrosso and others, 2016). While many technological and pathophysiologic hurdles remain, FES remains a promising application for BMIs to restore function to the paralyzed.

BMIs for rehabilitation of stroke

In addition to replacing function (i.e., acting as a “bypass” of the injured part of the nervous system), several groups are investigating the use of BMIs to rehabilitate function by driving plasticity in the brain. This could greatly broaden the potential patient populations that could benefit to those with hemiparetic stroke or traumatic brain injury. Several studies of BMIs controlling haptic feedback via robots (Ang and others, 2015) or exoskeletons (Buch and others, 2008; Ramos-Murguialday and others, 2013), or BMI-controlled FES (Daly and others, 2009; Takahashi and others, 2012; McCrimmon and others, 2015) have been performed. One randomized controlled study paired BMI-controlled exoskeleton use with physical therapy immediately following BMI use and showed improved upper extremity function compared with physical therapy alone (Ramos-Murguialday and others, 2013; Figure 2C). While the precise mechanisms behind this improvement are unclear, there are two general hypotheses. The first is that the BMI use enables a more precise version of mental imagery (Prasad and others, 2010), that helps generate more “normal” activity in the ipsilesional hemisphere (Daly and Wolpaw, 2008). The second is that combining motor intent plus haptic feedback can drive plasticity in a Hebbian-like manner (Soekadar and others, 2015). To date, almost all of these BMIs have been noninvasive (largely EEG or MEG); however, it is possible that driving such plasticity may be improved by using signals with higher temporal and spatial resolution, such as high-gamma signals, and thus could benefit from subdural or epidural recordings. Another way to drive plasticity that can be considered a simple form of a BMI is to stimulate an area of cortex based on spiking activity in another cortical area (Jackson and others, 2006). Stimulating somatosensory cortex in response to premotor cortical activity can improve function after stroke-like lesions in rats (Guggenmos and others, 2013). For a more extensive review on BMIs for rehabilitation, see (Soekadar and others, 2015).

Other BMI applications

One important factor in trying to replace motor function in patients with paralysis or amputation is sensory feedback. Normal movements are highly dependent on somatosensory feedback, especially proprioceptive feedback, and addition of proprioception to visual feedback can improve BMI control (Suminski and others, 2010). Therefore, several groups are investigating how best to restore somatosensation with afferent BMIs. Essentially, such BMIs have stimulated different parts of primary or secondary somatosensory cortex (London and others, 2008; O’Doherty and others, 2011b) to provide information about sensation; this is usually perceived as tactile rather than proprioceptive (Tabot and others, 2013) and has recently been demonstrated in humans (Flesher and others, 2016). For more details on sensory BMIs, see the review by Bensmaia and Miller (Bensmaia and Miller, 2014a).

Finally, BMIs are starting to be investigated for replacing cognitive function, such as memory. Berger’s group has demonstrated that a hippocampal BMI that takes inputs and stimulates CA1 can restore memory to rats that have induced memory deficits (Berger and others, 2011) and improve on normal memory function in monkeys (Deadwyler and others, 2017) and humans (Hampson and others, 2018). In addition, recent studies are using stimulation of multiple temporal lobe locations to improve memory performance in epilepsy subjects undergoing invasive monitoring prior to surgery (Titiz and others, 2017; Ezzyat and others, 2018).

BMIs for scientific investigation

In addition to their clinical applications, BMIs provide a powerful and unique method to examine fundamental neurophysiology. The neuromuscular system is highly redundant: many neurons control the movements of many muscles, which in turn create movements in a few degrees of freedom. Thus, it can be difficult to draw direct conclusions about motor learning and other aspects of physiology. In contrast, BMIs explicitly define the output neurons in the cortex; thus there are no intervening neurons in spinal cord and muscles, nor sensory feedback, to complicate the picture (Golub and others, 2016). These properties have enabled, for example, confirmation that the primary motor cortex in paralyzed humans is tuned to imagined hand movement kinematics (Truccolo and others, 2008). In addition, BMIs provide a way to test more than simple correlations of a small sample of neural activity with behavior; they provide direct causal relationships with some external output. The ability to examine conscious vs. subconscious feedback of brain-behavior relationships can be a powerful tool to study such causal relationships (Moxon and Foffani, 2015).

Signal stability and neural control of movement

The stability of brain signals is important to designing high performing BMIs. Conversely, BMIs have enabled us to evaluate stability as well. Single unit spikes can be difficult to record for long durations, and were initially thought to be relatively unstable in their information about reaching over a few hours (Cohen and Nicolelis, 2004; Carmena and others, 2005; Rokni and others, 2007). However, other studies showed that single units that are long-lasting can exhibit stable tuning with movement both within a day (Chestek and others, 2007) and up to at least 19 days (Ganguly and Carmena, 2009; Figure 3A). Multiunit spikes (MSPs) may be more stable, and likely have greater longevity. It has long been hypothesized that LFPs are more stable than spikes, since LFPs are composites derived from many thousands of neurons. It was not clear until recently, however, that either BMI performance or signals could remain stable for longer than a few days. A biomimetic BMI using MSPs was shown to provide largely stable performance for about 6 months without recalibrating the decoder, while one using LFPs remained stable over a year (Flint and others, 2013). Another study showed multiunit spike BMI performance remained stable for up to 22 months in one monkey and a few weeks in another monkey (Nuyujukian and others, 2014). Our group analyzed the stability of movement information in LFPs and MSPs during BMI control with a fixed decoder and during manual 2D planar reaching (Flint and others, 2016). The stability of movement information (measured using ensemble tuning and single unit or single LFP feature decoding) was higher in LFPs than in MSPs for more than 6 months of BMI use (Figure 3B–G). This could suggest that the brain is capable of different levels of control on LFP and single neuron scales (Ganguly and Carmena, 2009; Engelhard and others, 2013).

Figure 3.

BMIs reveal stability and mechanisms of cortical control of movement. A) Ganguly and Carmena (2009) demonstrated that preferred directions of single neuron ensembles (left panel, each row shows one neuron’s tuning curve; ensemble tuning for 3 days shown) showed high stability over a few weeks, as measured by the correlation of ensemble tuning curves between sessions (right panel). (Modified from Ganguly and Carmena, 2009 with permission from publisher.) Similar measures were used to examine stability of multi-unit spikes (MSPs) and LFPs over more than 6 months during BMI use (Flint et al, 2016). Single-feature decoders were used to decode the final 5 epochs of actual cursor velocity during hand control, and the performance of each day’s decoder is shown as color for (B) LFPs and (C) MSPs. D) The stability index (SISFD), the average of the correlation map (as in A) over time, shows that both LFPs and MSPs were stable (close to 1) for more than 200 days, though MSPs were somewhat less stable. E,F) Ensemble tuning patterns, similar to that shown in A, are shown for each feature in LFPs and MSPs. Color represents the band power, LMP amplitude, or spike rate in each direction of movement. G) SIET, the stability index for ensemble tuning, shows extremely stable LFP tuning and moderately stable MSP tuning over time. H) Decoding performance of neural data projected into the task-relevant (solid line) and task-null (dashed-line) spaces for two monkeys. The task-null space performance was much more variable than the task-relevant space performance, which supports the minimum intervention principle of optimal feedback control. (Modified from Flint et al., 2016 with permission from publisher.)

Our 2016 study also used BMIs to provide insight into the cortical control of movement. In particular, the study provided results supporting the minimal intervention principle (MIP) of optimal feedback control (Todorov and Jordan, 2002). The MIP hypothesizes that the brain minimizes effort by controlling only those movement components that are relevant to task goals (the task-relevant space), while allowing variability in those movement components that are unrelated to task goals (the task-null space). This principle accounts for the simultaneous observation of highly accurate control of movements in a particular task with substantial trial-to-trial variability in the way the limb or body moves while achieving the task goal (Todorov, 2004). The principle predicts low variability in the task-relevant space and higher variability in the task-null space. Although substantial evidence supporting this principle existed from behavioral studies (Scholz and Schöner, 1999; Todorov and Jordan, 2002; Scott, 2004), no evidence from cortical recordings existed. To investigate this principle in our BMI experiment, we partitioned the BMI decoder into a task-relevant space (the 2D cursor control space) and a task-null space (the subspace of variation not in the task-relevant space), and we projected the neural activity into these spaces (Flint and others, 2016). For each BMI control epoch we built two decoders of the BMI-controlled cursor velocity—one based on the task-relevant projections and one based on the null-space projections. The performance of the null-space decoders, both for spikes and LFPs, was much more variable over time than that of the task-relevant decoders (Figure 3H). These results were the first neural evidence supporting the MIP in the cortex itself. The direct link of M1 neurons to behavioral output using the BMI enabled us to partition the neural space and perform this analysis in a unique way.

Motor learning

Motor cortical neurons can learn highly specific and arbitrary mappings between their firing patterns and cursor movements. For example, Jarosciewicz et al. showed that a subset of M1 neurons controlling a BMI can relearn mapping directions that have been rotated, similar to a visuomotor rotation experiments (Jarosiewicz and others, 2008). The neurons did this by making both global (all neurons in the decoder) and local (just the rotated neurons) adjustments to their firing patterns. Further, the cortex can learn to remap M1 neuron firing patterns to control a BMI quickly (within a few days), even with randomly-remapped values (Ganguly and Carmena, 2009). These M1 neurons increased their modulation depths over time, while neurons not included in the BMI decoder decreased their modulation depths (Ganguly and others, 2011). This change in modulation also occurred when the decoder was re-computed (adapted) using the activity during BMI control, as long as the adaptation was not repeated too quickly (Orsborn and others, 2014). This increased modulation depth with BMI use was also demonstrated with both ECoG mu/beta activity (Miller and others, 2010) and in ensembles of layer II/III neurons using calcium imaging (Clancy and others, 2014), the layers which are thought to produce most of the local ECoG activity (Oke and others, 2010).

Changes in primary motor cortical activity within a few hours appear to be limited to a low-dimensional subspace (the intrinsic manifold) of the total neural space. In a study using M1 spikes, monkeys were trained to use a BMI and were then subjected to perturbations in the tuning directions of some neurons (Sadtler and others, 2014; Figure 4A). The tuning directions were perturbed such that they pointed either within or outside of the low-dimensional intrinsic manifold that captured the natural patterns of co-modulation among the recorded neural activity (Fig. 4 A). The monkeys easily adapted to perturbations within the intrinsic manifold, but had trouble adapting to perturbations outside the manifold (Figure 4B,C). This suggested that, at least on a timescale of hours, it is difficult to learn neural activity patterns that are much different from the existing neural network structure. Indeed, a follow-up study showed that the monkeys used largely the same patterns of neural firing and learned to reassociate them with new behavioral outputs during BMI use (Golub and others, 2018). This could explain why we can learn new skills more quickly if they are similar to skills we already possess.

Figure 4.

BMIs shed light on learning mechanisms within and among sessions. A) Within a single day, M1 neural activity during BMI use was largely confined to a low-dimensional, intrinsic neural manifold (yellow plane) within the total neural space (represented here schematically as three-dimensions of single unit firing rates, FR). The intuitive control space is represented as the black line. BMI decoders were then perturbed such that the control space still lay within the neural manifold (red arrow) or outside the neural manifold (blue arrow). B) Task performance during one representative within-manifold perturbation session shows rapid, successful learning of the new BMI decoder. Black line, success rate; green line, time to acquire the target; both are relative to the performance with intuitive BMI mapping. Dashed lines represent onset and offset of the perturbation. Grey vertical lines represent 50-trial bins used to determine initial and best performance with perturbed mapping. C) Task performance during one representative outside-manifold perturbation shows little learning during the session. D) Athalye et al. (2017) explored learning of new BMI decoders across days. As BMI performance increased over days, private variance decreased, while shared variance increased (E) and the intrinsic manifold increased its alignment and became more stable (F). (Figures A,B and C modified with permission from publisher from Sadtler et al., 2014.; Figures D and E modified from Athalye et al. 2017 with permission from publisher.)

A more nuanced analysis of de novo BMI learning showed that this intrinsic manifold actually can adjust and align to the decoder over multiple days (Athalye and others, 2017). This study examined the private variance and shared variance (within the intrinsic manifold) among neurons in the decoder, and showed that the private variance decreased (Figure 4D) and shared variance increased (Figure 4E) as BMI performance increased. The intrinsic manifold also increased its alignment with the task-relevant space and increased stability over time (Figure 4F), corroborating the findings of (Flint and others, 2016). These results suggest that the M1 activity is stabilized by the involvement of the broader neural network, and BMI experiments have indeed demonstrated that learning requires brain changes that extend beyond just the motor cortex. Learning a biofeedback BMI task using only one ECoG electrode in M1 correlated with broad changes in the cortical network activity: activity in the prefrontal, premotor, and posterior parietal cortices decreased with improving function, which suggested that the task execution was being shifted from “cognitive” to “automatic,” which is often reported anecdotally by participants in human BCI studies (Wander and others, 2013). Further, BMI learning was shown to be dependent upon corticostriatal activity. Mice learning a biofeedback BMI exhibited increased coherence between M1 and dorsal striatal neurons, and disrupting NMDA receptor function in striatal medium spiny neurons prevented BMI learning as well as corticostriatal coherence (Koralek and others, 2012). In addition, BMI learning was shown to incorporate elements of internal models (Golub and others, 2015), which are believed to be dependent on cerebellar activity (Shadmehr and others, 2010). This suggested that BMI learning uses similar neural circuitry to natural motor learning, and therefore may have similar properties to motor learning.

Finally, BMI learning has provided the opportunity to evaluate mechanisms of consolidation of learning (that is, learning that occurs “offline”) during sleep. Successful learning of a 1-D BMI in rats was accompanied by increased spike-field coherence between motor cortical neurons in the BMI decoder and delta waves (0.3–3 Hz) during slow-wave sleep (Gulati and others, 2014). Coupling of these neuronal ensembles, but not of neurons unrelated to the BMI task, was “reactivated” during slow-wave sleep at the trough of the delta wave. This paradigm also demonstrated that neurons not in the BMI decoder were “downscaled” after non-REM sleep, and that this rescaling was dependent upon spike-field coherence during the UP phases of slow wave sleep, since rescaling did not occur when firing was blocked during these phases (Gulati and others, 2017).

BMI effects on the state of the motor system

BMIs can also modify the state of the brain directly. For example, modulating the power in the beta band using a BMI modulated the time to movement onset of monkeys performing a subsequent reaching task. Increasing beta power moved the neural state farther from the movement onset “threshold” state (Khanna and Carmena, 2017). Another study showed that adaptations made in using a BMI can transfer to arm movements. That is, learning visuomotor rotations when using a BMI can improve performance on the same rotation during reaching movements (Vyas and others, 2018). The neural states during BMI and movement adaptations were similar. Both of these studies show that BMI use can not only elucidate mechanisms of motor learning, but also influence it. This concept relates to the use of motor imagery and BMIs for rehabilitation, as described above. Finally, the use of biofeedback BMIs have been shown to increase visual attention (in frontal eye fields; (Schafer and Moore, 2011) and visual perception (Shibata and others, 2011).

Conclusion and Future directions

BMIs have just begun to demonstrate their potential for clinical benefit. While intracortical BMIs display the most promise for replacing function, they also face several barriers to clinical translation. These include development of completely-implantable and wireless devices, power consumption and storage, and signal longevity and stability. The recent approval in the U.S. of completely-implanted subdural recording and stimulation devices for epilepsy as well as movement disorders may make subdural and epidural BMIs attractive options for translation in the nearer term.

By providing direct knowledge and control of the neurons controlling a behavior, BMIs represent a powerful way to investigate brain function. Since they do not observe neurons upstream (premotor, posterior parietal, supplementary motor, prefrontal), downstream (spinal cord and brainstem) and parallel (basal ganglia and cerebellum) to motor cortex, there are some limitations to what we can conclude about motor learning from BMIs. Thus, we should interpret findings with caution. Nevertheless, many more insights into brain function are expected to be gleaned from BMI studies.