Brain-computer interfaces: a powerful tool for scientific inquiry

Abstract

Brain-computer interfaces (BCIs) are devices that record from the nervous system, provide input directly to the nervous system, or do both. Sensory BCIs such as cochlear implants have already had notable clinical success and motor BCIs have shown great promise for helping patients with severe motor deficits. Clinical and engineering outcomes aside, BCIs can also be tremendously powerful tools for scientific inquiry into the workings of the nervous system. They allow researchers to inject and record information at various stages of the system, permitting investigation of the brain in vivo and facilitating the reverse engineering of brain function. Most notably, BCIs are emerging as a novel experimental tool for investigating the tremendous adaptive capacity of the nervous system.

Introduction

In the spring of 1965, an article titled “‘Matador’ With a Radio Stops Wired Bull” appeared in the New York Times [1]. The article garnered considerable public attention, given that the ‘matador’ in question was not a matador at all but actually a neuroscientist. The article was reporting on experiments by Yale neuroscientist Jose Delgado, who in the previous year had demonstrated that animals like the charging bull could be stopped in their tracks by using a radio transmitter; the transmitter sent signals to an implant called the “stimoceiver” which stimulated the caudate nucleus in the animal’s brain [2]. Interestingly, however, the initial purpose of this set of experiments was not to create a remote-controlled bull, but to test hypotheses about the role of subcortical structures in an animal’s drive to eat. At roughly the same time, Eberhard Fetz, who was in the process of attempting to discover the motor system analog to sensory receptive fields [3], demonstrated that when given visual feedback in the form of a neurally controlled needle on an analog meter, monkeys could learn to volitionally modulate the firing rates of individual neurons [4]. These two experiments were among the first in a long series of increasingly impressive demonstrations in a field now known as brain-computer interfacing [5], [6]. In both studies, the brain-computer interface was primarily a tool being used to make scientific discoveries about the nervous system.

The term brain-computer interface (BCI) refers to a range of techniques and technologies that all involve a direct interface to the nervous system; such interfaces can be made at nearly any level, limited mostly by technical constraints and surgical techniques. Due in part to portrayal of BCIs by the mass media, the model of a BCI that often comes to mind is the recording of electrical activity from the motor regions of the brain and the mapping of this activity to an output device, either a cursor on a screen or a robotic arm. Recent studies have demonstrated that these activity patterns can be used for reasonably dexterous control of sophisticated robotic limbs [7]–[11]. In reality, however, BCI technology spans a much larger space. For example, the cochlear implant [12] is another form of BCI that provides artificial sensory inputs directly to the auditory system. More recently, recording and stimulation technologies have been linked together to build bidirectional BCIs that are capable of bridging injured portions of the motor pathway, effectively reanimating paralyzed limbs [13], or even biasing the underlying mechanisms of neural plasticity to favor some circuits over others [14]. Because BCIs permit recording and injection of information at effectively arbitrary points in the nervous system, they are a versatile tool for investigating computation and adaptation in individual brain structures.

From population vectors to prosthetic control: new insights into neural plasticity

The link between neuronal firing rates in motor cortex and movement parameters has been established for nearly a half century [15]. Georgopoulos et al. expanded on this by demonstrating that the population vector, a simple linear sum of the preferred movement directions of neurons weighted by their firing rates, quite accurately predicts the actual arm movement direction [16]. This discovery fueled a rapid expansion in BCI research by demonstrating that it is possible to extract movement parameters from the population activity of motor cortical neurons. BCI researchers could then train mimetic decoders during overt (e.g. joystick-based) cursor control and then transition to direct brain control. This method has been used extensively in non-human primate BCIs [10], [11], [17]–[21] as an effective means to rapidly train both the BCI subject and the decoder in the absence of verbal instruction.

Motor BCIs have been successfully implemented using a variety of signal types, ranging from the firing rates of individual neurons [4], [7]–[11], [13], [17]–[22], to the aggregate activity of thousands to millions of neurons [23]–[26], and even to hemodynamic surrogate indicators of neural activity [27], [28]. Interestingly, though the vast majority of motor BCIs involve mapping motor-correlated neural activity to the movements of an end effector, it has also been shown that through the performance of non-motor tasks such as mental arithmetic [29] or mental object rotation [30], subjects can exert control over a BCI. Even more impressively, the brain can learn to modulate the firing rates of arbitrary neurons or populations of neurons to achieve BCI control [4], [26], even dissociating the activity of individual neurons that are normally correlated [22]. These findings demonstrate not only that motor BCIs may be clinically applicable when cortical motor areas are no longer intact, but also that the brain is capable of dynamically repurposing areas natively associated with non-motor computation to exert closed-loop, motor-like control. Given the brain’s ability to develop control over such a wide range of output signals, BCIs provide an excellent platform for probing the limits of neural plasticity.

In the early motor BCI experiments, it was theorized that as long as the animal continued to carry out whatever motor action was correlated with the changes in neuronal activity, it would maintain reasonable performance when the experimental paradigm was changed from manual control to brain control. The astounding observation was that overt movements lessened and in some cases ceased entirely while the animals continued to maintain control of the cursor [11], [17]. This finding suggested that the brain was dynamically modifying internal networks to dissociate changes in neural activity from the motor movements with which they were originally correlated. This hypothesis was verified by Ganguly et al. who demonstrated not only that extended BCI training resulted in the generation of a stable cortical map but multiple maps associated with different decoders could be stored simultaneously, recalled when necessary [18], and reverted from when not in use [19]. Such a finding was suggestive of network-scale plasticity which has been demonstrated both at local [19], [31] and distributed spatial scales [27], [32], [33]. Even when effective control of the BCI only explicitly requires modulation of activity in a small cortical region, frontal and parietal cortical regions are strongly task modulated during initial performance of the task and less so after extensive training [32] (see figure 2). Finally, given that changes in the neural activity controlling a BCI are inevitable (e.g., due to plasticity, impedance changes, fatigue, etc.), significant work has gone into the development of co-adaptive BCI frameworks that respect that there will be both learning-based and incidental changes in cortical activity that can be accounted for with incremental changes to BCI decoders [20], [34]–[38].

Figure 2.

Summary of findings from a 1-D electrocorticographic (ECoG) BCI learning task. (a) Across seven subjects, performance increased significantly between the first and last runs (each run consists of approx. 20 trials). (b) Trends in the ECoG activity feature (70–100 Hz) that was being used for BCI control. Up targets (red) required volitional increases in activity and down targets (blue) required maintaining baseline levels. With experience, subjects develop the ability to better separate activity for up and down targets. Black vertical line represents a trial that best separates early from late trials. Horizontal red and blue lines represent mean power values for corresponding combinations of early/late and up/down trials. Black dotted line depicts a running difference in the subject’s ability to differentially modulate activity for up and down targets. (c) Group average of distributed high frequency (70–200 Hz) activity changes over the course of learning to control a BCI. Cortical areas PFC, PMd, and PPC exhibited notable decreases in activity after learning. Adapted from [32].

An important open question is whether the brain can develop control over a BCI in such a way that it is nondestructive of pre-existing motor function. As discussed above, neuroprosthetic control is accompanied by a stable cortical representation of task parameters [18]; can these cortical maps be generated in such a way that they can effectively share underlying neural tissue with pre-existing functionality? The ability to actively control a BCI while simultaneously performing other cognitive or motor tasks is beginning to be studied in BCI research [39]. The nature of the feedback being given will likely play a dominant role in enabling simultaneous BCI and motor or cognitive control. For example, such BCIs may need to utilize proprioception and somatosensation rather than rely solely on visual feedback to allow multitasking. In cases such as amputation where afferent sensory information from the distal limb is no longer available, artificial sensory inputs from tactile, position, or force sensors will be necessary.

BCIs and sensory coding

The need to provide biologically meaningful inputs to the brain from artificial sensors has added BCI as an additional tool for studying neural coding of sensory information. Considerable advances have already been made in the area of sensory BCIs, which seek to replace lost sensory functions such as hearing or sight. The most notable example of a sensory BCI is the cochlear implant [12], which exploits the tonotopic (frequency-to-place) representation of sound in the cochlea to effectively convert sound from an external microphone to electrical stimulation of nerve fibers at different locations along the cochlea. Though cochlear implants are the most prevalent in clinical practice, sensory BCIs are under development for other sensory modalities. These include visual cortex stimulators [40], [41], retinal implants [42], [43], vestibular prostheses [44], and somatosensory prostheses [21], [45], [46]. Somatosensory prostheses are of particular interest to the field of motor BCIs because not only are tactile sensation and proprioception necessary for our ability to effectively carry out motor tasks [47], they are also crucial for motor skill learning [48]. Cortical somatosensory prostheses have, to date, been limited to momentary (not continuous) stimulation of sensory cortex that alter perceptual effects by varying stimulus intensity [45], [46], frequency [21], [45], and location [46].

Like the cochlear implant, the majority of sensory prostheses utilize biomimetic encoding schemes, providing sensory inputs to the nervous system that attempt to match the spatial, temporal and spectral characteristics of the natural stimuli and functioning sensory system. The effectiveness of these encoding schemes relies greatly on our understanding of how these sensory systems encode naturally occurring stimuli, which is far from complete. Sensory BCIs provide an excellent opportunity to evaluate our hypotheses about sensory coding by providing synthetic sensory inputs and observing behavioral capability to utilize these inputs.

However, an advantage of the BCI sensory interface is that encoding transforms are not necessarily constrained to being biomimetic. The plasticity of the nervous system allows it to learn to parse these novel inputs [49] just as it learns to control the novel outputs of a motor BCI. It has been observed that even inputs from biomimetic cochlear implants can be unintelligible in the early days after implantation; it is only with time that the brain learns effective interpretation of inputs from the implant [50]. These observations open the door to a variety of compelling questions surrounding both the coding of sensory information in the intact brain as well as how sensory neuronal circuits adapt to novel inputs; additional investigation will be necessary to better understand the limits of these adaptive processes.

Studying closed-loop behavior and plasticity using bidirectional BCIs

Combining a motor BCI and a sensory BCI results in a bidirectional BCI that both records from and stimulates the nervous system. Bidirectional BCIs are multifarious in type, with architectures that depend greatly on the specific question or questions they are being used to address. Some examples of bidirectional BCIs include direct stimulation of sensory cortex as an additional sensory feedback modality during motor BCI use [21], electrical stimulation-based activation of distal muscles of a paralyzed limb through volitional modulation of firing rates in motor cortex [13], and recording and stimulation of hippocampal neurons to encode and then “play back” memory traces necessary for successful task execution [51]. A separate thread in bidirectional BCI research focuses on establishing an artificial connection between brain regions during natural behavior and strengthening connectivity by using endogenous activity-dependent stimulation to encourage mechanisms of Hebbian plasticity [14]. Besides clinical applications in neurorehabilitation for restoration of connections after stroke or injury, such bidirectional BCIs also represent a novel experimental technique for studying synaptic plasticity.

Bidirectional BCIs provide a tremendous opportunity to test hypotheses about the computation taking place within and the transformations between various components in the nervous system, because the loop from recording to stimulation is completely under experimental control. This principle was demonstrated by Karniel et al. in an excellent study where they electrically connected the sensors and actuators from a simple robot to neurons in a lamprey brainstem [52]. Because the plant dynamics of the robot and its environment were well characterized, they could be removed from the overall system dynamics, which then allowed the authors to directly extract the dynamics of computation taking place in the lamprey nervous system (see [53] for a review).

Bidirectional BCIs permit direct investigation of the sensorimotor transformations taking place in the nervous system of an individual organism, but they are not limited to this. Because the neural activity recorded by a bidirectional BCI is digitized and processed upon acquisition, the distance that this information can travel before it is utilized extends far beyond the laboratory setting. This permits teleoperation of disembodied robotic limbs, but also, direct neural interaction between organisms: to date, brain-to-brain interaction has been demonstrated between two rats [54], a human and an anesthetized rat [55], and two humans (Direct Brain-to-Brain Communication in Humans: A Pilot Study; URL: http://homes.cs.washington.edu/~rao/brain2brain/). Though such studies have focused largely on proving the concept is feasible, the idea of direct neural interaction between two or more organisms offers some unique and unprecedented experimental opportunities for investigating brain function, such as probing neural coding by transferring different types of information from one brain to another, and exploring whether connected brains can achieve greater sensorimotor, cognitive, and creative capabilities than single brains. It is important to note that direct brain-to-brain interfacing also opens up a Pandora’s box of ethical and moral issues that neuroscientists and society as a whole must start to address.

Conclusions

The recent surge of interest in brain-computer interface technology is not undeserved. Advances in neuroscience coupled with the now ubiquitous access to embedded computational power, improvements in wireless sensing and improvements in wireless power transfer are dramatically increasing the capability of BCIs. To fully realize the potential of these devices, it is becoming increasingly important to thoroughly understand the system with which the BCI is trying to interface. Advances must be made in the neuroscience of motor representations, sensorimotor transforms, sensory coding, cognitive processing, and plasticity. Fortunately, the two can advance in tandem; BCIs can be a powerful tool for scientific inquiry into the very system with which they interface. BCIs afford the experimentalist opportunities not only to observe sensorimotor transformations as information travels through the brain, but also to modify the nature of these transformations in real-time. Most importantly, BCI technology provides a scaffold for scientific experimentation that enables investigation of the nervous system doing what it does best: incorporating new information and rapidly adapting to new constraints.

During the last decade of his life, neuroscientist-matador Jose Delgado was remorseful that no useful application came out of his research with the stimoceiver, saying “We knew too little about the brain” [56]. It would have pleased him to know that the BCI-successors of his stimoceiver today are emerging as a new scientific tool for achieving a deeper and more nuanced understanding of the brain.

Figure 1.

Recent scientific discovery through BCI research. Motor, sensory, and bidirectional BCIs have all been leveraged in studies that have made significant scientific contributions to our understanding of brain function. Abbreviations: Intra-cortical microstimulation (ICMS), primary motor cortex (M1), pre-frontal cortex (PFC), dorsal pre-motor cortex (PMd), posterior parietal cortex (PPC). Subcortical regions of interest not shown.

Highlights.

• Motor BCIs lead to discovery about sensorimotor processing loops

• Sensory BCIs provide opportunities to better understand neural coding mechanisms

• The brain possesses tremendous capacity to adapt to novel inputs and outputs

• Simultaneous recording and stimulation permits closed-loop study of the brain