The Science of Neural Interface Systems

Abstract

The ultimate goal of neural interface research is to create links between the nervous system and the outside world either by stimulating or by recording from neural tissue to treat or assist people with sensory, motor, or other disabilities of neural function. Although electrical stimulation systems have already reached widespread clinical application, neural interfaces that record neural signals to decipher movement intentions are only now beginning to develop into clinically viable systems to help paralyzed people. We begin by reviewing state-of-the-art research and early-stage clinical recording systems and focus on systems that record single-unit action potentials. We then address the potential for neural interface research to enhance basic scientific understanding of brain function by offering unique insights in neural coding and representation, plasticity, brain-behavior relations, and the neurobiology of disease. Finally, we discuss technical and scientific challenges faced by these systems before they are widely adopted by severely motor-disabled patients.

INTRODUCTION

Research to develop systems that can help restore sensory function, communication, and control to impaired humans is coalescing into a new branch of experimental neuroscience, variously named brain-machine interfaces (BMIs), brain-computer interfaces (BCIs), neural prostheses, or neural interface systems (NISs). This emergence is evidenced by the dramatic increase in the number of publications and presentations related to NISs in neuroscientific journals and conferences. NISs have captured the broader public imagination by providing the possibility of freeing ourselves from the limitations of our bodies by forming a direct interaction between the brain and the outside world. More importantly, NIS research offers the possibility of helping people with severe sensory and motor disabilities better interact with their world, thereby improving their quality of life. However, an important question that has often been raised by neuroscientists is whether this emerging area of research and technology can also provide fundamental scientific knowledge about nervous system function, especially in humans. Does NIS research legitimately constitute a scientific discipline within the neurosciences, or does it belong within a subfield of biomedical engineering such as neural engineering, which is focused mainly on producing useful technology? In this review, we begin with recent clinically relevant advances in NIS research, dealing mainly with neuromotor prosthesis studies that tie together basic and translational research using single neuron recordings. We then critically evaluate neuroscientific contributions that this emerging field is providing. We finish with our view of the future challenges in NIS research and its ability to advance neuroscience. Although we briefly discuss recent, promising developments in noninvasive, electroencephalogram (EEG)-based NISs, our emphasis in this review is invasive systems that rely on extracellular microelectrode recordings of action potentials (spikes) from neuronal populations.

BACKGROUND

The field of NISs is actually quite broad and could include any form of connection between the brain and the outside world beyond the natural interfaces provided by the sensory and motor systems. Input NISs apply electrical stimulation to some part of the central or peripheral nervous system to help restore sensory processing or to improve function by modulating neural activity in certain brain diseases. The most successful neural interfaces today fall into the stimulation category because they are already available to humans. They include the cochlear implant (Gifford et al. 2008, Wilson & Dorman 2008) to restore audition in the hearing impaired and the deep brain stimulator (DBS) (Arle & Alterman 1999) to relieve the symptoms of Parkinson disease and dystonia. Output NISs, which are the focus of this review and are referred to simply as NISs throughout this review, record electrical potentials from the brain to read out ongoing neural activity. Most commonly, this readout is used to predict cognitive intentions, motor plans, or actions of an organism to replace a lost connection to the outside world, but this information may also aid in diagnosis of disease or injury or to guide therapy.

The scientific origins of NISs go back at least to the emergence of behavioral neuro-physiology research in the 1960s when Evarts performed his pioneering electrophysiological experiments from primary motor cortex (MI) of awake, behaving primates (Evarts 1968). Evarts found that the firing rates of individual MI neurons in monkeys were strongly correlated with the force or torque generated by the joints during arm movements. A landmark paper published by Humphrey and colleagues (1970) demonstrated that the kinematic and kinetic time course of a monkey's single-joint wrist movement could be predicted quite accurately using a small population of simultaneously recorded MI neurons, further elucidating the type of population or ensemble processing that occurs in the cortex to produce movement. Another landmark study, by Fetz (1969), demonstrated that monkeys could volitionally control the activity of single MI neurons using visual biofeedback and reward. Monkeys were reinforced for moving an analog dial with a meter arm whose position was controlled by the firing rate of a neuron being recorded. This experiment provided initial evidence that primates could learn feedback control of neural activity without intervening movements.

These early studies also clarify an important distinction between two forms of this neural interface research: (a) open-loop prediction or decoding from multisite recordings, of which the Humphrey study is an excellent example, and (b) closed-loop control, of which the Fetz study is one of the first examples. In a more recent example of NIS research showing open-loop prediction, Wessberg and colleagues (2000) demonstrated that it was possible to predict the 2D and 3D position of a monkey's hand in real time using neuronal ensembles from various cortical areas. This prediction signal was then used to move a robotic arm in a remote location, demonstrating that sufficient information was present in the sample of neurons to reconstruct arm movements and that this motion could be performed in nearly real time. This study is an example of open-loop prediction but not of closed-loop control because the animal did not receive any form of sensory feedback from the robot and thus was unaware that it was moving the robotic arm. In contrast, a recent study demonstrated that a monkey could control the 3D position of a robot's end effector as well as its 1D gripper to feed itself (Velliste et al. 2008). This finding is qualitatively different from the Wessberg study because the animal's goal was to control the robot, and not its own arm, to grab the food and bring it to its mouth. This study is an example of a closed-loop control NIS. One might argue that prediction by itself as manifested by the first example is actually just part of behavioral electrophysiology and is not a constituent of the NIS paradigm. However, this study as well as numerous other studies involving open-loop prediction/ decoding were essential in forming the foundation for future NIS research. Especially significant are the studies of Georgopoulos and colleagues (Georgopoulos et al. 1986, 1988), who demonstrated how populations of spiking neurons can predict arm kinematics in space. These studies were not necessarily motivated by the desire to build NISs, but they certainly advanced knowledge necessary to build a neural interface. As we later argue, these efforts also seek to establish principles of neural function and therefore also make fundamental contributions to neuroscientific knowledge.

After initial developments in EEG-based NISs pioneered by W. Gray Walter in the 1960s (Bladin 2006), neural interface research experienced a marked resurgence in the late 1990s with a number of demonstrations of closed-loop control using neuronal spikes. Chapin and colleagues (1999) demonstrated a rat's ability to control a one-dimensional feeder using multielectrode recordings from sensorimotor cortex. This was followed by a number of studies (Carmena et al. 2003, Musallam et al. 2004, Santhanam et al. 2006, Serruya et al. 2002, Taylor et al. 2002) showing that closed-loop control in primates is possible. That is, monkeys could replace actions of its hand with a command derived directly from neural populations in the cortex to perform actions. Importantly, the monkeys in these studies were observing the consequences of their control and could use that visual information to guide behavior. All these studies describe a common architecture for closed-loop NISs, consisting of four components: (a) a multielectrode recording array, (b) a mapping or decoding algorithm, (c) an output device, and (d ) sensory feedback (Figure 1).

Figure 1.

The four components of a closed-loop, neural interface system: (1) a recording array that extracts neural signals, (2) a decoding algorithm that translates these neural signals into a set of command signals, (3)an output device that is controlled by these command signals, and (4) sensory feedback in the form of vision and potentially other sensory modalities. Transparent head image is courtesy of ©iStockphoto.com/Kiyoshi Takahase Segundo.

CLINICAL APPLICATIONS

NIS research using multineuron recording sensors has now been translated from preclinical proof of concept to early-stage human clinical trials (Hochberg et al. 2006, Kennedy & Bakay 1998). The motivation for these trials is to create a device that can be used by individuals with movement impairment to regain control, communication, and independence. This advance also provides a new opportunity for long-term neurophysiological investigation in human cortex at the level of single and multiple neurons and local field potentials (LFPs), which has previously never been possible.

NISs are being created to help the large number of people who have limited movement abilities owing to damage or disease of the motor system. A number of disorders disconnect an otherwise healthy cerebral motor system from the muscles, leading to various degrees of paralysis but with retention of the ability to plan and imagine making movements. These conditions include trauma, such as spinal cord injury, which renders ~100,000 Americans tetraplegic; strokes, which interrupt descending motor pathways, with the most devastating being a pontine stroke, which can disconnect all descending control to produce a locked-in state; and degenerative disorders such as amyotrophic lateral sclerosis (ALS, or Lou Gehrig's disease), in which alpha motor neurons (and likely cerebral neurons) progressively die. Cerebral palsy, muscular dystrophy, and limb amputation may also lead to the inability to execute voluntary actions. In each case, muscle control by the brain is lost, while cerebral mechanisms to generate movement intentions could remain relatively intact (Enzinger et al. 2008, Shoham et al. 2001). Thus, an NIS may be a means to directly deliver motor commands from the brain to assistive technologies, bypassing the biological lesion to restore control and independence to those with paralysis. Assistive technologies are aids to function; they can include any device from a simple switch, a computer cursor, a robot, or an artificial limb. A neural command with sufficiently rich and stable information could be used to operate any of these devices. Neural signals could also be used to recouple the brain to the muscles through functional electrical stimulation (FES). An FES system converts command signals into stimulation trains, which the FES system uses to activate muscles (Peckham et al. 2002). With a neural command source, an FES system could allow movement intentions to produce arm movements once again, although the complexity of artificially coordinating the myriad of arm muscles used for normal human arm movements is daunting.

Like their preclinical counterparts, the clinical NIS must have a sensor for recording volitional signals, a decoding algorithm, an effector, and some form of feedback. Here, we emphasize those recently tested systems that are based on intracortical signals to derive commands because they establish a new link between human and nonhuman primate behavioral neurophysiology experiments. They provide insights into human neural processes not available from surface field potentials, which is the other main control signal source being tested for NISs (Birbaumer 2006). Kennedy (Kennedy et al. 2000, 2004; Kennedy & Bakay 1998) developed the first spike-based approach for NISs using a sensor of individually implanted microwires encased in glass cones so that neurites would grow into the cone and establish long-lasting connections to the nervous system. Using a few channels, humans with tetraplegia from stroke or degenerative disease showed that they could activate spiking or LFPs from cortical neurons to move a cursor.

An ongoing pilot study of the first human chronically implanted multielectrode array-based NIS has made several advances in recording, decoding, and demonstrating potential use of a pilot NIS system. This system, termed BrainGate, is based on spiking signals recorded from a 4 × 4 mm array of 100 Si microelectrodes in a fixed 10-by-10 arrangement placed within the motor cortex arm area. Studies to date utilizing four patients with tetraplegia have demonstrated participants’ ability to produce two-dimensional cursor control that can be used to operate a computer and control a robot arm to perform simple grip and transport actions (Hochberg et al. 2006). These studies demonstrated that both spiking and LFPs remain in motor cortex and can be volitionally modulated in the absence of movement, years after spinal cord injury (Hochberg et al. 2006) or stroke (Kim et al. 2007b). Patterns of activity appear to be similar to those observed in intact monkeys when movement is actually performed. This demonstration, that MI displays similar activity when movement is imagined and performed, raises fundamental questions about the nature of neural activity in MI and provides a key finding necessary for any further development of spike-based NISs.

Control achieved by human participants using continuous control commands derived from linear decoders was quite similar to that achieved with the same algorithms in preclinical studies using able-bodied monkeys, discussed above (Carmena et al. 2003, Serruya et al. 2002, Taylor et al. 2002, Velliste et al. 2008). Improvements in the decoders using Kalman filters showed not only the ability to make smooth point-to-point movements, but also the ability to stop cursor motion and select targets by clicking on the basis of decoding an intended hand squeeze (Kim et al. 2007b). Using this improvement, a participant in three test sessions was able to point and click to one of eight screen targets, never selecting the wrong target and rarely (<4%) failing to make correct target selections. However, to achieve this success, movements were slow, taking 6.4 s, on average, to move from the middle of the screen to a target at the screen's edge and click on it. These initial findings suggest that practical NISs for humans with paralysis are feasible and that this field is on a path to improving independence, control, and in the case of locked-in persons, communication, which is otherwise severely limited.

SCIENTIFIC IMPLICATIONS

Whereas the clinical potential of an NIS is clearly evident, it may be less obvious how neural interface research has added to our understanding of functional properties of the brain and, therefore, whether this research constitutes a fundamental endeavor in neuroscience. To support this proposition, we demonstrate that both open- and closed-loop forms of neural interface research have advanced understanding in neural coding, distributed representations, parametric multiplexing, spike-field potential relations, plasticity, cortical mirror responses, brain-body uncoupling, and recording in chronically injured patients.

New Approaches to Understanding Neural Coding

The NIS paradigm has become a standard test bed for evaluating particular encoding models that have been postulated to relate movement features to neuronal responses. Traditional approaches to studying encoding have relied on building peri-event time histograms, tuning curves, and regression models (Ashe & Georgopoulos 1994; Fu et al. 1993, 1995; Georgopoulos et al. 1982). NIS research has motivated the development of decoding models that invert the problem to predict movement from the responses of neuronal populations, thus providing a direct test of the information available from a simultaneously active group of neurons. In this regard, the population vector method has demonstrated that a population of MI neurons contains sufficient information for animals to achieve closed-loop control of a virtual object in 3D space (Taylor et al. 2002) as well as the end point arm position and gripper aperture velocity of a robot arm (Velliste et al. 2008). This approach begins with a linear velocity encoding model that relates the instantaneous velocity (i.e., direction and speed) of the hand to the neuron's firing rate (Georgopoulos et al. 1982, Moran & Schwartz 1999) to estimate the preferred velocity of the neuron. To decode instantaneous velocity, the preferred directions of the neuronal population are then vectorally added, weighted by the firing rate of each neuron.

A statistically principled approach begins with an encoding model that estimates the conditional probability of a neuron's response, given the motor behavior of the animal (Brockwell et al. 2004, Kemere et al. 2004, Sanger 1996, Shoham et al. 2005, Wu et al. 2006, Wu & Hatsopoulos 2008). Assuming conditional independence, the conditional probability of a population response can be estimated by multiplying the single neuron encoding models, and maximum likelihood estimation of the motor behavior can be performed. Alternatively, by invoking a prior distribution on the behavior and applying Bayes’ rule, this model can be inverted to estimate the probability of the behavior conditioned on the population response. Similar approaches for decoding sensory stimuli as well as motor behaviors have been used by a number of groups (Barbieri et al. 2004a,b; Paninski et al. 2007; Sanger 2003; Wiener & Richmond 2002).

DISTRIBUTED REPRESENTATIONS

Results from NIS studies have provided further support for highly distributed arm movement representations in the frontal and parietal cortices and have extended this knowledge to the human motor cortex. Despite clear evidence for functional specialization of movement planning and performance in motor, premotor, and parietal cortical areas (Andersen & Buneo 2002, Buneo & Andersen 2006, Luppino & Rizzolatti 2000), it has become evident from NIS experiments that large networks of neurons are active to various degrees even during simple movements. For example, neural interface studies have directly compared decoding performance from simultaneously recorded neuronal populations in various cortical areas including the ipsi- and contralateral primary motor (MI), dorsal premotor (PMd), supplementary motor (SMA), primary somatosensory (SI), and posterior parietal cortices (Carmena et al. 2003, Hatsopoulos et al. 2004, Wessberg et al. 2000). Investigators observed significant differences in decoding performance across different cortical areas that could help determine which area to implant for optimal NIS performance, but what is particularly surprising is the fact that all tested areas exhibited a certain degree of accurate arm-movement decoding performance. These results strongly support the view that ordinary forms of voluntary limb motor control recruit large numbers of neurons in distributed areas encompassing many regions of the frontal and parietal cortices.

Multiplexing of Movement Parameters

Another related and somewhat surprising result from NIS research is that different movement components, parameters, and features may be multiplexed in the same distributed network. Recent studies have shown that both transport and grip components can be controlled from a single, simultaneously recorded population of cortical neurons in monkeys (Carmena et al. 2003, Velliste et al. 2008) and in tetraplegic humans (Hochberg et al. 2006, Kim et al. 2007b). In the animal studies, the position of a device as well as the grip force/grasp aperture were simultaneously controlled by using two different linear mappings from a single neuronal population, whereas in the human study, grip state was decoded in a binary fashion at the same time that two-dimensional cursor motion was controlled. These findings provide direct evidence for the idea that mixed control of hand and arm exists within small regions of primate cortex, a view that influences theories of motor coding but conflicts with some studies of functional segregation using indirect imaging methods (Beisteiner et al. 2001).

Relationship between Spikes and Field Potentials

Developments in NISs have rekindled scientific inquiry into the relationship between electrical field potentials [EEG, electrocorticogram (ECoG), and LFP] and spiking. Research paths into these two fundamental electrical signals of the nervous system have largely drifted apart when considered as signals that code information in neural systems (Bullock 1997). Most inquiries related to information coding at the systems level have focused on spiking patterns. Field potentials are a major signal used for diagnostic aids in human disease (as in epilepsy), although they are also of great interest for use as control signals for NISs, especially because they can operate using external, noninvasive sensors (Wolpaw & McFarland 2004). Studies have begun to investigate how information in the field potential compares with that in spikes (Belitski et al. 2008, Buneo et al. 2003, Donoghue et al. 1998, Murthy & Fetz 1996, Pesaran et al. 2002). This work promises to help clarify the nature and origin of the wide spectrum of field potentials, their roles as information-carrying signals, and their relationships to spiking in single neurons.

Learning and Cortical Plasticity

A closed-loop NIS offers an important setting for studying the relationship between cortical plasticity and learning. A number of experiments have reported short-term (over the course of minutes within a single recording session) and long-term (over the course of multiple days) improvements in device control. Neural interface experiments using recordings from primary motor cortex (Taylor et al. 2002) and posterior parietal cortex (Musallam et al. 2004) have documented modest but significant linear increases in percent correct performance ranging from 0.08 to 0.9 percentage points per recording session over the course of 30–70 daily sessions. Carmena and colleagues (2003) have directly compared the chronic improvements in online control from SMA, PMd, MI, and SI and found the largest linear increase in performance from SMA. At the same time, a number of studies have documented systematic variations in neural modulation and directional tuning at the single neuron level during short-term exposure to an NIS (Carmena et al. 2003, Taylor et al. 2002, Zacksenhouse et al. 2007). Thus, this research provides detailed assessment of neural tuning dynamics.

An important question is whether these adaptation phenomena offer something unique to our understanding of neural plasticity that we cannot or have not learned from traditional learning and plasticity experiments. The relationship between learning and neural plasticity is often studied in behavioral electrophysiological experiments by examining changes in single-unit activity in a particular brain region after an animal has been exposed to some learning paradigm. This approach has been fruitful, but it has its limitations. First, this approach is inherently correlational, so one cannot determine whether the changes in neural activity have any causal relationship with the behavioral changes (Figure 2A1). Second, even if there is evidence to suggest an indirect causal relationship between the recorded brain area and behavior through downstream areas (Figure 2A2), it is still unclear to what degree the behavioral changes associated with learning are due to plasticity downstream or upstream from the area being studied.

Figure 2.

(a) A schematic of a typical electrophysiological experiment investigating the cortical correlates of motor learning. An electrode records changes in neural activity from a cortical area that (A1) may not be causally related to behavioral changes observed in behavioral output or (A2) may not be directly connected with the behavioral output. (b) A schematic of a typical neural interface experiment in which a multielectrode array records signals from the same cortical areas as in a. In this case, changes in neural activity are causally related to the output behavior of the device being controlled by the neural interface.

The NIS paradigm forms a direct, causal link between the recorded brain area and behavior via the decoding algorithm so any behavioral changes can be attributed to changes in neural modulation from the recorded brain area, although these neural changes are not necessarily due to synaptic modification. In particular, a neural interface using recordings from MI can rule out the possibility that plasticity is occurring in the spinal cord, neuromuscular junction, or muscles because they have been removed from the sensorimotor loop (Figure 2b). Although there are afferent projections from the muscles and spinal cord back to the cortex, they do not likely contribute to the observed behavioral changes because the animal is often not significantly moving its arm (Carmena et al. 2003, Lebedev et al. 2005, Velliste et al. 2008). Of course, the NIS paradigm cannot necessarily localize exactly where any synaptic plasticity may be occurring. That is, the modifications in neural activity observed within the recorded brain area could be a consequence of synaptic plasticity occurring in upstream brain areas that provide altered inputs to the recorded brain area.

Brain-Body Uncoupling

In a closed-loop NIS paradigm, it remains a mystery as to how a population of recorded neurons can be co-opted to control an artificial device without moving the limb, which was, under normal circumstances, moved by the activation of the same population. Several studies have documented the fact that monkeys that are exposed to an NIS paradigm will eventually stop moving or minimally move their own limbs when guiding the artificial device through cortical control (Carmena et al. 2003, Lebedev et al. 2005, Serruya et al. 2002, Taylor et al. 2002, Velliste et al. 2008). In all these cases, tiny movements or subthreshold activation of motor neurons could not be ruled out. However, what is impressive is that the original gross action that was used for control was substituted for another behavior, despite the involvement of a similar pattern of MI activity. Whether this change in behavior is a result of short-term plasticity, a newly learned motor pattern, or active inhibition of downstream cortico-spinal circuits remains untested. A lucid explanation of this phenomenon would have far-reaching implications for mechanisms underlying adaptation to novel control situations.

Mirror-Like Responses in MI

An effort to solve a problem inherent to NISs for motor-disabled patients led to advances in understanding fundamental response properties of neurons in motor cortex. Building a decoding mapping between neural ensemble activity and motor output requires collecting data in which neural activity and motor behavior are concurrently recorded, which is not possible for persons who are paralyzed. We recently discovered that passive visual observation of a familiar task can elicit responses in the primary motor cortex that closely resemble those observed when the animal actually performs the task (Tkach et al. 2007). Indeed, decoding filters were built for paralyzed persons while they observed computer-controlled motion of a cursor (Hochberg et al. 2006). However, unlike in our animal studies, the human participants were asked to imagine they were producing the cursor motion. Our recent discovery of mirror-like single neuron responses in the primary motor cortex was directly motivated by a desire to elicit motor cortical activity to build a neural interface mapping between neural activity and observed motion without explicit movement.

Recording in Chronically Injured Human Patients

Finally, clinical NISs have provided the opportunity to engage in cellular physiology in persons with injury and degenerative diseases. They have shown that neural spiking and LFP activity survive years after damage including transection of layer V pyramidal cell axons in the cord or the pons (Hochberg et al. 2006, Kennedy & Bakay 1998). Moreover, these studies have shown that tetraplegic persons can volitionally modulate MI activity in the absence of any ability to move. This result is surprising because a variety of animal and indirect human studies suggest that the motor cortex can reorganize rapidly in response to injury (Dancause et al. 2005, Donoghue et al. 1990, Giraux et al. 2001, Nudo 2006, Sanes et al. 1990), which apparently has not occurred in similar form in those studied so far. Thus, the results of these human studies have raised a number of issues related to the function of MI and the nature of cortical plasticity, including whether different mechanisms operate in humans and in other species or whether the types of injury or disease result in different types of functional responses.

FUTURE CHALLENGES AND OPPORTUNITIES

Technical Challenges

We believe important technical issues must be addressed as NISs become useful technologies for disabled patients. First, no multielectrode recording arrays have currently been fully verified to stably and reliably record action potentials from multiple single units for extended periods of time (i.e., over many years). Reliable chronic recording has been an area of considerable concern, although both cone electrodes (Kennedy et al. 1992a,b) and micro-electrode arrays can record for many months in monkeys (Suner et al. 2005) and humans (Kim et al. 2007b). We have sometimes encountered loss of recordings associated with complex connectors and faulty insulating materials in chronically implanted electrode arrays. Ongoing testing of advanced and improved versions of these sensors will be required to achieve long-term viability of implants.

A second important technical challenge for a long-term human NIS is the creation of a fully implantable system that can provide high-bandwidth information. A fully implantable sensor is necessary to eliminate cabling, which limits mobility, as well as the need for percutaneous connectors, which can present an ongoing infection concern. In addition, implantable systems have the advantage that they are hidden from view, improving the cosmetic appeal of such systems. Creation of high-channel count, high-bandwidth implantable systems, which are required for spiking signals, is complex particularly because initial signal processing must now be completed inside the body. Active electronics of this complexity are difficult to seal fully, can induce excess heat, and require power and wireless communication. Despite these formidable challenges, several groups are now developing these systems (Kim et al. 2007a, Patterson et al. 2004, Rizk et al. 2007, Song et al. 2005). This type of technology could also be a boon to animal researchers because this would allow long-term, continuous recording of multiple neurons, potentially in multiple areas, which in turn would reveal new experimental paradigms. Early work with this approach has already begun (Mavoori et al. 2005, Santhanam et al. 2007).

Controlling More Degrees of Freedom

As neural interface technologies are extended to control the numerous degrees of freedom (DOF) of the hand as well as the arm, this challenge begs the scientific question of how many independent control signals can be extracted from a neural ensemble. The arm has seven DOF, whereas the hand has more than 20 DOF. Several psychophysical studies, however, have indicated that natural grasp postures and reaching-to-grasp movements reside in a much smaller subspace of physically possible movements (Ingram et al. 2008, Mason et al. 2001, Santello 2002). Using principal components analysis, these studies have shown that a large proportion (i.e., >75%) of the kinematic variance of natural grasping behaviors can be accounted for with two or three principal components. With restricted goals, state-of-the-art neural interface technology that currently can recover four DOF (3 spatial dimensions plus 1 grasp DOF) is not that far from controlling the apparent complexities of certain natural motor behaviors.

However, reach-to-grasp is only one, albeit important, category of neuroethological movement. Moreover, more encephalized vertebrates, particularly those with developed neocortex and rich cortico-spinal and corticomotoneuronal projections, can learn new arm and hand movements that include fractionated finger movements (Nudo et al. 1995; Nudo & Masterton 1990a,b; Schieber 1991). The NIS paradigm may provide a unique opportunity to study the neuronal limits of independent control systematically. In particular, given a neuronal ensemble of size N, is it possible to derive a number of independent control signals that approaches N or even surpasses that number?

Controlling a Dynamical System

An important issue that has received little or no attention is whether an NIS can provide control signals to an output device that has physically realistic dynamics. Most previous studies have developed decoding algorithms that map neural ensemble activity to a kinematic signal such as position or velocity, which guides a massless, virtual object or controls a physical device via a controller, thereby negating any inherent dynamics that the device may possess (Figure 3a) [see, however, Carmena et al. (2003), who demonstrate that grip force can be controlled in a closed-loop NIS]. The fact that NISs have largely ignored dynamics is somewhat surprising given the evidence that the motor cortex may be encoding kinetic (Cabel et al. 2001, Cheney & Fetz 1980, Evarts 1968, Hepp-Reymond et al. 1978, Kalaska et al. 1989, Sergio et al. 2005, Smith et al. 1975, Taira et al. 1996) or muscle-like (Morrow & Miller 2003, Pohlmeyer et al. 2007, Santucci et al. 2005, Westwick et al. 2006) parameters of motion.

Figure 3.

(a) An NIS paradigm in which kinematic parameters are decoded and sent to either a virtual object or a physical plant, both of which passively follow the kinematic commands. In the case of a physical plant, a proportional-derivative (PD) controller parameterized by a stiffness (K) and viscosity (B) generates a force or torque signal such that the plant is forced to follow the desired kinematics. (b) An NIS paradigm in which kinetic parameters are decoded and act as control variables on a dynamic system.

An alternative decoding approach is to map neural ensemble activity directly to torque and force signals, which guide the dynamics of the physical device (Figure 3b). We have recently shown that motor cortical ensembles can quite faithfully reconstruct (i.e., in an open-loop, off-line setting) the joint torque profiles of the shoulder and elbow, despite their relatively larger bandwidth (Fagg et al. 2007, 2009). It will be interesting to determine whether closed-loop control of a dynamical system is possible and, perhaps, is even superior to closed-loop systems that decode position and velocity signals.

Other Forms of Sensory Feedback

Researchers are currently interested in augmenting neural interfaces with other forms of sensory feedback in addition to vision to achieve more realistic motor control. It is well-documented that somatosensory feedback in the form of tactile and proprioceptive sensation is critical for normal motor control (Ghez et al. 1990). This finding is evident particularly in patients with large-fiber sensory neuropathy who manifest slow and uncoordinated movements, particularly those movements involving multiple joints (Ghez et al. 1995; Gordon et al. 1995; Sainburg et al. 1993, 1995; Sanes et al. 1985). The effects of the proprioceptive system on movement are generally extremely fast (~70 ms) (Crago et al. 1976), whereas visual feedback is often delayed, resulting in movement instabilities. In principle, somatosensory feedback can be implemented either by artificially stimulating peripheral and central sensory systems or by naturalistically providing tactile and proprioceptive feedback using an approach referred to as extended physiological proprioception (Simpson 1973).

Artificial Feedback

For several decades, researchers have tried to provide amputees with sensory feedback to aid in control of mechanical prostheses (Phillips 1988; Shannon 1976, 1979). This feedback has come about in the form of either mechanical stimulation of peripheral tissue or electrical stimulation of residual nerves (Dhillon & Horch 2005). We term these efforts as examples of associative, artificial feedback because, in most cases, there is an artificial transfer function between the state of the prosthesis and the features of the stimulation, which must be associated through learning by the user. That is, the relationship between the position of the pros-thesis and the mechanical or electrical stimulation parameter is not innately identified by the patient. An example of associative, artificial feedback in the context of a cortically controlled NIS is described in a recent study that showed how vibrotactile stimulation of the arm of normal human subjects could be used as a sensory feedback signal for controlling a one-dimensional EEG-based NIS (Chatterjee et al. 2007). In this example, subjects voluntarily modulated EEG power in the mu frequency-band to move a virtual device toward the left or right. In addition to visual feedback, the subjects received vibratory stimulation, the frequency of which was proportional to the one-dimensional position of the virtual device. This linear mapping between position and stimulation frequency is not inherently understood by the patient but can be learned, which suggests that paralyzed persons operating prosthetic systems could incorporate sensory feedback to improve control. It would also be of great interest to learn how the functional properties of neural populations in humans evolve as these new mappings are learned.

Extended Physiological Proprioception

Tactile and proprioceptive feedback can be provided in a more natural way such that the link between the subject's biological sensors and the state of the device can be directly understood. One important form of natural feedback was termed extended physiological proprioception (EPP) by Simpson, who proposed an approach to improving prosthetic control in amputees in which residual proprioception and tactile sensation could be used in a natural way to “feel” and guide prostheses (Doubler & Childress 1984a,b; Simpson 1973, 1974). The idea is to link the joints or degrees of freedom of a prosthesis directly with intact physiological degrees of freedom that possess residual proprioception. An example of EPP in amputees was proposed recently in which a magnet surgically implanted in the distal end of the residual humerus of an arm amputee could be used to control powered rotation of a pros-thesis attached to the residual humerus (Li & Kuiken 2008). A magnetic sensor placed on the external arm stump could sense the shifts in magnetic field strength as voluntary rotation of the residual humerus occurs. This mechanism would drive the rotation of the motorized lower-arm prosthesis. At the same time, natural proprioception in the residual muscles, ten-dons, and joint capsule of the humerus could assist the patient in “sensing” the motion of the prosthesis.

EPP is a simple but powerful approach that could be used with a cortically driven neural interface. This approach could be implemented in several ways: through the use of an exoskeletal robot or through functional electrical stimulation (FES). Control signals from the cortical implant could drive the motion of the exoskeleton, which a patient “wears” and which would, in turn, passively move the arm and hand of the paralyzed patient. Residual proprioception in certain user populations could be recruited to improve control of the exoskeletal robot. Alternatively, FES could be used to stimulate sets of limb muscles, and residual proprioception could sense the position and motion of the limb. A range of feedback sensors are also being used for FES applications and could deliver feedback to implanted cortical stimulation arrays (Inmann et al. 2001, Inmann & Haugland 2004, Sinkjaer et al. 2003). Although it is not yet tested, electrical stimulation of Brodmann's areas 3a and 2 may be directly interpreted as proprioceptive state information much like Romo has shown that electrical stimulation of the somatosensory cortex can be used to make tactile perceptual judgments (Romo et al. 1998).

CONCLUSION

The creation of the NIS paradigm has been motivated by a desire to create devices to treat and recover functioning in patients with severe sensory and motor disabilities. Output NISs are now at the cusp of moving from research proofs of concept and pilot human clinical trials to useful devices as input NISs already have. We have demonstrated that this paradigm has also offered a unique framework for studying basic scientific problems in coding, representation, and plasticity in neuronal ensembles. As current and future challenges are addressed to create useful systems for patients, this field may further expand our understanding of how large systems of neurons compute, adapt, and interact with the outside world.

Glossary

BCI

brain-computer interface

NIS

neural interface system

Input NIS

stimulates the central or peripheral nervous system to restore sensory function or improve function by altering neural activity associated with brain disease

Output NIS

records signals from the central or peripheral nervous system to decipher the cognitive or motor intentions of an organism

MI

primary motor cortex

Open-loop prediction

records neural activity to predict the kinematic, kinetic, or goal state of an organism's peripheral motor apparatus

Closed-loop control

records neural activity that acts to guide a device controlled by an organism. Organism receives sensory feedback

LFP

local field potential

Extended physiological proprioception (EPP)

a method for using the natural kinesthetic system to guide the motion of a device