Evolution of brain-computer interfaces: going beyond classic motor physiology

Abstract

The notion that a computer can decode brain signals to infer the intentions of a human and then enact those intentions directly through a machine is becoming a realistic technical possibility. These types of devices are known as brain-computer interfaces (BCIs). The evolution of these neuroprosthetic technologies could have significant implications for patients with motor disabilities by enhancing their ability to interact and communicate with their environment. The cortical physiology most investigated and used for device control has been brain signals from the primary motor cortex. To date, this classic motor physiology has been an effective substrate for demonstrating the potential efficacy of BCI-based control. However, emerging research now stands to further enhance our understanding of the cortical physiology underpinning human intent and provide further signals for more complex brain-derived control. In this review, the authors report the current status of BCIs and detail the emerging research trends that stand to augment clinical applications in the future.

The notion that the brain can be directly accessed to allow a human being to control an external device with his or her thoughts alone is emerging as a real option in patients with motor disabilities. This area of study, known as neuroprosthetics, has sought to create devices, known as “brain-computer interfaces” (BCIs), that acquire brain signals and translate them into machine commands such that they reflect the intentions of the user. In the past 20 years, the field has rapidly progressed from fundamental neuroscientific discovery to initial translational applications. Examples are seen in the seminal discoveries by Georgopoulus and colleagues^26,27 indicating that neurons in the motor cortex, taken as a population, can predict the direction and speed of arm movements in monkeys.⁵³ In subsequent decades, these findings have been translated to increasing levels of brain-derived control in monkeys and preliminary clinical trials in humans. ^32,88 Another example is seen in Pfurtscheller and colleagues’ work analyzing EEG.^57–59 This group was one of the first to describe the changes in amplitudes in the sensorimotor rhythms associated with motor movement. As a result, researchers have used these signals to achieve basic levels of control in humans with ALS and SCI.^7,38,104 Early findings have sparked a growing interest within the neurosurgical community for novel technical approaches in aiding patient populations for whom treatment options thus far have been limited.

Fundamental to the evolution of neuroprosthetic applications, this brain-derived control is dependent on our emerging understanding of cortical physiology as it encodes information about intentions. For “output BCIs,” which allow a user to control an external device (rather than artificially creating an internal perception), the majority of devices have been based on the brain signal changes associated with movements of a contralateral limb. As a result, the most likely candidates to benefit from current BCI platforms are patients whose motor cortex is intact. Thus, patients with SCIs, neuromuscular disorders, and amputations all stand to potentially benefit from current approaches.

In recent years, an emerging understanding of the role that cortical plasticity plays in device control and how cortex encodes motor and nonmotor intentions as well as sensory perception has led to new insights in brain function and BCI applications. These new discoveries stand to expand the potential of neuroprosthetics in regard to both control capability and the patient populations that will be served. In this review, we provided an overview of current BCI modalities and emerging research on the use of nonmotor areas for BCI applications, and we assessed their potential clinical impact.

Brain-Computer Interface: Definition and Essential Features

A BCI is a device that can decode human intent from brain activity alone to create an alternate communication channel for people with severe motor impairments. More explicitly, a BCI does not require the “brain’s normal output pathways of peripheral nerves and muscles” to facilitate interaction with one’s environment.^102,103 A real-world example would entail a quadriplegic person controlling a cursor on a screen with signals derived from individual neurons recorded in primary motor cortex (M1) without the need for overt motor activity. It is important to emphasize this point: a true BCI creates a completely new output pathway for the brain.

As a new output pathway, the user must have feedback to improve how they alter their electrophysiological signals. Similar to the development of a new motor skill (for example, learning to play tennis), there must be continuous alteration of a person’s neuronal output. The output should be matched against feedback from the intended actions such that the person’s output (swinging a tennis racket or altering a brain signal) can be tuned to optimize his or her performance toward the intended goal (getting the ball over the net or moving a cursor toward a target). Thus, the brain must change its signals to improve performance, but the BCI may also be able to adapt to the changing milieu of the user’s brain to further optimize functioning. This dual adaptation requires a certain level of training and a learning curve—both for the user and the computer. The better the computer and the user are able to adapt, the shorter the training required for control.

There are 4 essential elements to the practical functioning of a BCI platform (Fig. 1): 1) signal acquisition, the BCI system’s recorded brain signal or information input; 2) signal processing, the conversion of raw information into a useful device command; 3) device output, the overt command or control functions administered by the BCI system; and 4) operating protocol, the manner in which the system is altered and turned on and off. All of these elements act in concert to manifest the user’s intention to his or her environment.

Fig. 1.

Schematic showing essential features and components of a BCI. There are 4 essential elements to the practical functioning of a BCI platform: 1) signal acquisition, the BCI system’s recorded brain signal or information input; 2) signal processing, the conversion of raw information into a useful device command; 3) device output, the overt command or control functions administered by the BCI system; and 4) operating protocol, the manner in which the system is turned on and off and the way in which the user or a technical assistant adjusts the parameters of the previous 3 steps in converting intentions to machine commands. All of these elements act in concert to manifest the user’s intention to his or her environment.

Signal acquisition is some real-time measurement of the electrophysiological state of the brain. This measurement of brain activity is usually recorded via electrodes. These electrodes can be either invasive or noninvasive. The most common types of signals include EEG, electrical brain activity recorded from the scalp;^{20,25,60,85,95} ECoG, electrical brain activity recorded beneath the skull;^40,42,73 field potentials, electrodes monitoring brain activity from within the parenchyma;² and “single units,” microelectrodes monitoring individual neuron action potential firing.^27,37,39,88 Figure 2 shows the relationship of the various signal platforms in terms of anatomy and the population sampled. Once acquired, the signals are then digitized and sent to the BCI system for further interrogation.

Fig. 2.

Drawing depicting the signals for BCI and their locations relative to the brain. Three general categories of signals are used for BCI applications.

In the signal-processing portion of the BCI operation, there are 2 essential functions: feature extraction and signal translation. The first function extracts significant identifiable information from the gross signal; the second converts that identifiable information into device commands. The process of converting raw signal into one that is meaningful requires a complex array of analyses. These techniques can vary from the assessment of frequency power spectra, event-related potentials, and cross-correlation coefficients for analysis of EEG and/or ECoG signals to directional cosine tuning of individual neuron action potentials.^44,53,61 The impetus for these methods is to determine the relationship between an electrophysiological event and a given cognitive or motor task. For example, after recordings are made from an ECoG signal, the BCI system must recognize that a signal alteration has occurred in the electrical rhythm (feature extraction) and then associates that change with a specific cursor movement (translation). As mentioned above, it is important for the signal processing to be dynamic such that it can adjust to the changing internal signal environment of the user. In terms of the actual device output, this overt action is accomplished by the BCI. As in the previous example, this action can result in moving a cursor on a screen; other possibilities are choosing letters for communication, controlling a robotic arm, driving a wheelchair, or controlling some other intrinsic physiological process such as moving one’s own limb or controlling the bowel and bladder sphincters.⁴¹

An important consideration for practical applications is the overall operating protocol, which refers to the manner in which the user controls how the system functions. The “how” includes such things as turning the system on or off, controlling what kind of feedback is provided and how fast, the speed with which the system implements commands, and switching between various device outputs. These elements are critical for BCI functioning in the real world application of these devices. In most current research protocols, the investigator sets these parameters; in other words, the researcher turns the system on and off, adjusts the speed of interaction, and defines very limited goals and tasks. The user must be able to do all of these things by her- or himself in an unstructured applied environment.

Current BCI Platforms

Currently, 3 general categories of BCI platforms have been put forward as candidates for clinical application. These categories are primarily determined by the source from which the controlling brain signal is derived. The first category utilizes EEG, which measures brain signals acquired from the scalp. The second category, referred to as “single-unit systems,” utilizes intraparenchymal microelectrodes that detect action potential firings of individual neurons. The third is an intermediate modality in which electrodes acquire signals from the cortical surface directly (either above or below the dura).

Electroencephalography-Based Systems

Electroencephalography-based BCIs use electrical activity recorded from the scalp.^{7,10,20,38,47,49,51,54,62,93,104–106} Most BCI studies in humans have utilized EEG, probably because this recording method is convenient, safe, and inexpensive. Nonetheless, it has relatively poor spatial resolution given that a large brain area must be involved to generate the necessary detectable signals.⁸⁴ Despite this limitation, signals relevant to BCI research can still be found in EEG recordings, including modulations of mu (8–12 Hz) or beta (18–25 Hz) rhythms produced by sensorimotor cortex. These rhythms show nonspecific changes (typically decreases in amplitude) related to movements and movement imagery. They do not contain specific information about the details of movements such as the position or velocity of hand movements, which may be an important limitation because signals associated with specific movement parameters are typically used in BCI systems based on action potential firing rates. Another issue with EEG recordings is the size of the detected amplitudes; they are very small, which makes them susceptible to artifacts created by sources outside the brain such as electromyographic signals produced by muscle contractions. Despite these potentially limiting issues, EEG-based BCIs have been shown to support a higher performance than is often assumed, including accurate 2D^47,104 and even 3D control of a computer cursor.⁵⁰ To date, the majority of the clinical applications of BCI technologies in people with severe motor disabilities have been demonstrated using EEG.^38,56,93 Ultimately, this intrinsic lack of signal robustness may have significant implications for the chronic application of BCI systems in real-world environments. Brain-computer interface systems based on EEG typically require substantial training^6,104 to achieve accurate 1D or 2D device control (~ 20 and 50 respective 30-minute training sessions), although some authors have reported shorter training requirements.¹⁰ The shortcomings of noise sensitivity and prolonged training are fundamental limitations in the scalability of the widespread clinical application of EEG-based BCIs.

In summary, EEG has been shown to support much higher performance than previously assumed and is currently the only modality shown to actually help people with paralysis. However, because of its important limitations, it is not clear to what extent EEG-based BCI performance, in the laboratory and clinical settings, can be further enhanced.

Single Neuron–Based Systems

From a purely engineering point of view, the optimal method of extracting electrical information from the brain would be to place a series of small recording electrodes directly into the cortical layers (1.5–3 mm) to record signals from individual neurons. This procedure, in essence, is what single-unit action potential BCI systems do, and they have been very successful for limited time periods in both monkeys^14,78,88,94 and humans.^32,37 To extract single-unit activity, small microelectrodes having ~ 20-μm-diameter tips are inserted in the brain parenchyma where relatively large (for example, 300-μV) extracellular action potentials are recorded from individual neurons from 10 to 100 μm away. These signals are usually band pass–filtered from 300 to 10,000 Hz and then passed through a spike discriminator to measure spike time occurrences. The firing rates of individual neurons are computed in 10- to 20-msec bins and “decoded” to provide a high fidelity prediction to control either a computer cursor or robot end point kinematics.^27,52,97 Given its high spatial resolution (100 μm) as well as its high temporal resolution (50–100 Hz), this modality arguably provides the highest level of control in BCI applications.

Unfortunately, there are 2 major problems with single-unit BCIs. First, the electrodes must penetrate into the parenchyma where they not only cause local neural and vascular damage, but also increase the chances for CNS infections.⁸ Second, single-unit action potential microelectrodes are very sensitive to encapsulation. The insertion of penetrating devices into brain parenchyma damages neurons and vasculature, which can initiate a cascade of reactive cell responses typically characterized by the activation and migration of microglia and astrocytes toward the implant site.⁸ The continued presence of devices promotes the formation of a sheath composed partly of these reactive astrocytes and microglia.^64,86 This reactive sheath can have numerous deleterious effects, including neural cell death and increased tissue resistance that electrically isolates the device from surrounding neural tissue. ^5,99 Research into novel biomaterial coatings and/or local drug delivery systems that can reduce the foreign body response to implanted electrodes is ongoing, but we are far from clinical application as yet.^1,79,82 Until these issues are solved, there is little hope of building a long-term BCI system based on single-unit activity.

Electrocorticography-Based Systems

Over the past 5 years, the use of ECoG as a signal platform for BCI has gained mounting enthusiasm as a more practical and robust platform for clinical application. As detailed above, both EEG and single unit–based systems are not optimal for large-scale clinical application. This limited utilization is due either to prolonged user training and poor signal-to-noise limitations for EEG, or to an inability to maintain a consistent signal for current single-unit constructs.^8,86,104 Thus, the use of ECoG has been posited to be an ideal tradeoff for practical implementation.⁴² Compared with EEG, the signal in ECoG is substantially more robust. Its magnitude is typically 5 times larger, its spatial resolution is much greater (0.125 vs 3.0 cm for EEG), and its frequency bandwidth is significantly higher (0–500 vs 0–40 Hz for EEG).^11,25 Of particular note, the access to higher frequency bandwidths carries particularly useful information amenable to BCI operation. Data from many studies have demonstrated that different frequency bands carry specific and anatomically distinct information about cortical processing. The lower-frequency bands known as mu (8–12 Hz) and beta (18–26 Hz), which are detectable with EEG, are thought to be produced by thalamocortical circuits and show broad anatomical decreases in amplitude in association with actual or imagined movements.^34,43,61,68 The higher frequencies appreciable only with ECoG, also known as gamma band activity, are thought to be produced by smaller cortical assemblies. Gamma activity shows a close correlation with the action potential firing of tuned cortical neurons in M1 in monkey models.³¹ Additionally, these high-frequency changes have been associated with numerous aspects of speech and motor function in humans.^{15–17,42,72} Beyond the greater information content, because the ECoG signal is recorded from larger electrodes that do not penetrate the brain, these constructs should have a greater likelihood for long-term clinical durability. This expectation of good long-term stability of ECoG sensors is supported by pathological and clinical evidence. For example, in cat and dog models, long-term subdural implants have shown minimal cortical or leptomeningeal tissue reactions while maintaining prolonged electrophysiological recording.^13,45,46,107 In addition, preliminary work in humans has shown stability in using the implantable NeuroPace device for the purpose of long-term subdural electrode monitoring for seizure identification and abortion.⁹⁶

The use of ECoG for BCI applications has been studied primarily in motor-intact patients with intractable epilepsy requiring invasive monitoring. Similar to EEG-based BCI systems, the ECoG approach has generally focused on changes in sensorimotor rhythms from motor cortex. What has been distinct, however, is the access to the higher-frequency gamma rhythms with ECoG. The utilization of these higher-frequency rhythms has provided a significant advantage with regard to training requirements and multidimensional control. In 2004 Leuthardt et al.⁴² demonstrated the first use of ECoG in closed-loop control in a 1D cursor control task with minimal training requirements (< 30 minutes). In additional experiments, the same group of authors and others have demonstrated that specific frequency alterations encode very specific information about hand and arm movements.^42,63,70,72 In 2006 Leuthardt et al.⁴⁰ further demonstrated that ECoG control using signals from the epidural space was also possible. Recently, Schalk et al.⁷⁴ have demonstrated that ECoG signals can be used for 2D control. The results showed that the ECoG-based BCI system’s performance was within the range of that shown before using invasive single-unit systems. Because the electrode arrays cover broad regions of cortex, several groups have begun to explore alternate cognitive modalities and their relevant cortical physiologies to expand BCI device control. Felton et al.²¹ have shown that sensory imagery, in addition to motor imagery, can be used for device control. The same group also demonstrated that auditory cortex could be trained to acquire simple control of a cursor.¹⁰⁰ Ramsey et al.⁶⁷ have shown that higher cognitive functions, such as working memory in dorsolateral prefrontal cortex, can also be used for effective device operation.

Taken together, these studies show that ECoG signals carry a high level of specific cortical information and can allow a user to gain control rapidly and effectively. It is worth noting that these control paradigms have not been extended to motor-impaired patients thus far. How these cortical signals will be affected in the setting of an SCI or ALS has not been explicitly tested.

Emerging Research in Cortical Physiology and BCI Implications

As detailed above, from single-cell to macroscopic cortical population recordings, the majority of brain signals used for BCI control have been based on primary motor cortical physiology. From an output BCI standpoint, this physiology most clearly and directly reflects a user’s intentions to interact with the world, namely, to use his or her limbs and hands to interact with the environment. In several preliminary trials these various approaches have shown promise for enhancing the lives of patients with SCI and neuromuscular disorders. Although a superb starting point, there are other neurophysiological substrates that could further improve or complement existing control paradigms, allow new forms of control, and expand the range of patient populations that might benefit from a BCI (Fig. 3).

Fig. 3.

Schematic showing a summary of the cortical sites and modalities used for BCI. The 3 fundamental signal modalities currently being explored include EEG, ECoG, and single-unit systems that record action potential firing from single neurons.

Ipsilateral Motor Physiology: Potential Applications for Additional Control Signals and Hemispheric Stroke

To date, BCIs have been put forth as novel engineering approaches to enhance communication and control in patients who have intact cortex but lack motor control because of brainstem stroke, SCI, or peripheral neuromuscular dysfunction. With requisite improvements in performance and robustness, these systems have the potential to help tens of thousands of motor-disabled patients; however, these approaches offer little hope to the 4.6 million suffering from hemispheric stroke or traumatic brain injury. Hemiparesis is one of the most common reasons for their disability, and often it is hand function that is left chronically impaired.^29,92 The US health care costs attributable to hemispheric stroke exceed $29 billion.⁸⁹ This source of disability is a vastly underserved population in regard to restorative interventions. The majority of BCI methodologies are based on a functioning motor cortex capable of controlling the contralateral limb. This exact situation does not exist in unilateral stroke. As a result, current BCI techniques have a limited potential to impact functional improvements for the largest motor-disabled population. In recent years, there has been an evolving appreciation of how ipsilateral motor and motor-related areas participate in ipsilateral movements in both healthy and stroke-affected patients. These findings have prompted further exploration into the underlying cortical physiology as a possible substrate for neuroprosthetic application.

Currently, how the motor cortex is involved with ipsilateral limb movements in humans varies depending on whether primary or premotor cortex is considered. For M1, early evidence has supported the notion that ipsilateral activity is the result of transcallosal inhibition. Transcranial magnetic stimulation of M1 has shown reduced electromyographic responses when preceded by a conditioning pulse to the opposite hemisphere.²² Additionally, Newton et al.⁵⁵ have demonstrated that there is a negative baseline change in the fMR imaging bold response in M1 associated with ipsilateral movements, and they have postulated that it represents increased inhibition. Distinct from the inhibition associated with ipsilateral M1, the premotor cortex appears more involved in the active planning of a given movement. In healthy persons, fMR imaging shows that there is more robust bilateral activation of the dorsal premotor cortex with either contralateral or ipsilateral hand movements.³⁶ Using MEG, Huang et al.³³ have demonstrated that ipsilateral premotor areas have dipole peak latencies that significantly precede contralateral M1 sensorimotor cortex in performing unilateral finger movements. These findings were posited to support more of a motor planning role in ipsilateral finger actions.

Until recently, definitive electrophysiological studies to parse out the manner and extent to which ipsilateral cortex physiologically encodes hand movements have been limited in humans. Most researchers to date have relied on EEG. Consistent with fMR imaging and MEG findings that active changes appear to be associated with bilaterally represented motor planning, ipsilateral hand movements have been shown to induce alterations in cortical potentials prior to movement, which have been referred to as the “premotor positivity.”^80,87 Additionally, spectral analyses of EEG signals have shown low-frequency amplitude changes associated with various hand and finger movements that are bifrontally located.⁴⁸ Data in these studies further support the notion that the ipsilateral cortex is likely involved with the planning of both ipsilateral and contralateral movements. The EEG modality, however, is limited by poor spatial resolution and restricted spectral bandwidth. Wisneski et al.¹⁰¹ have used ECoG to more definitively define this physiology in 6 motor-intact patients undergoing invasive monitoring for seizure localization. Electrocorticographic signals were recorded while the patients engaged in specific ipsilateral or contralateral hand motor tasks, including opening and closing the hand and operating a joystick. Spectral changes were identified with regard to frequency, location, and timing. Data in this study have shown that ipsilateral hand movements are associated with electrophysiological changes occurring in lower-frequency spectra (average 37.5 Hz), at distinct anatomical locations (most notably in premotor cortex), and earlier (by 160 msec) than changes associated with contralateral hand movements. Given that these cortical changes occurred earlier and were localized preferentially in premotor cortex as compared with those associated with contralateral movements, the authors postulated that ipsilateral cortex is more associated with motor planning than its execution. Additionally, these changes were quite distinct from those associated with contralateral motor movements, which were more dominantly associated with higher gamma rhythms (average 106.9 Hz). In a more recent analysis, these low frequencies associated with joystick movements appeared to have a higher information content in decoding velocities of joystick movements.³ Thus, the cortical changes associated with ipsilateral motor movements are not simply nonspecific movement-related alterations but rather represent specific motor kinematics (that is, the direction of the joystick movement). Taken together, in healthy motorintact humans, there appears to be cortical activity ipsilateral to the hand activity that is distinct from activity associated with contralateral hand movements, associated with planning rather than execution, and encodes specific information about the motor movement.

The separable cortical physiology and anatomy that distinguishes ipsilateral and contralateral hand movements can provide additional signal features for device control. Thus, in the clinical situations in which the brain is intact and motor output is impaired (for example, SCI and ALS), these ipsilateral-derived signals can provide an additional dimension of control that supplements current BCI paradigms. This ipsilateral physiology may allow a doubling of the degrees of freedom that a user can achieve from an implanted prosthetic. For example, if one can achieve 2 dimensions of control with contralateral related physiology (as has been shown⁷⁴), further control may be supplemented by the addition of control signals derived from independent ipsilateral signals. These dual physiologies could allow an additional “click function” superimposed on the 2D movements or could summate to allow multidimensional control of a more complex machine such as a robotic arm (3D movement) with a grasper (1D movement). The earliest demonstration of ipsilateral-derived control was recently documented by Wisneski et al.¹⁰¹ Using ECoG, these authors showed that control comparable to contralateral-derived control could be achieved by using the anatomical sites or the lower-frequency amplitude changes distinctive to ipsilateral movements.

In the setting of stroke, premotor cortex appears to play a role in patients with poor functional recovery. Functional imaging has shown these severely affected patients to have increased activity in the premotor regions of their unaffected hemispheres.^77,98 The exact role this activity plays is unclear. It may simply be an indicator of a more severe outcome⁹¹ or an adaptive mechanism to optimize an already poor situation.^28,36 Note, however, that transcranial magnetic stimulation suggests that interference with this activity will worsen the already compromised or negligible function.³⁶ Thus, incomplete recovery and its association with heightened ipsilateral activation may reflect the upregulation of motor planning with an inability to execute or actuate the selected motor choice.

Consequently, in the scenario of hemispheric stroke with contralesional premotor upregulation, a BCI may provide a unique opportunity to help actuate the nascent premotor commands. By detecting the brain signals associated with these motor choices, the BCI may convert these signals into machine commands that can control a robotic assist device that will allow improved hand function (that is, a robotic glove that opens and closes the hand). The BCI would allow the ipsilateral premotor cortex to bypass the physiological bottleneck determined by the small and variable percentage of uncrossed motor fibers. This new methodology would allow the restoration of function in chronically and severely affected patients for whom methods of rehabilitation have not provided a sufficiently functional recovery (Fig. 4). In the past year, the first demonstration of this principle of utilizing ipsilateral motor signals for device control has been demonstrated with MEG in patients with stroke.^12,36

Fig. 4.

Diagram showing ipsilateral BCI for hemispheric stroke. In the normal physiological scenario (A) motor planning is represented in both hemispheres and motor execution is accomplished by the contralateral M1. In the setting of stroke (B), contralateral primary motor and motor-associated cortex is lost. Premotor cortex ipsilateral to the affected limb is left unaffected. There is an increase in ipsilateral premotor activity following hemispheric stroke. Thus, in the scenario of hemispheric stroke with contralesional premotor upregulation, a BCI can provide a unique opportunity to aid in actuating the nascent premotor commands. A BCI detects the brain signals associated with these premotor commands (C) and converts these signals into machine commands that can control a robotic assist device, which would in turn allow improved hand function (that is, a robotic glove that opens and closes the hand). The BCI allows the ipsilateral premotor cortex to bypass the physiological bottleneck determined by the injured contralateral M1 and small, variable percentage of ipsilateral uncrossed motor fibers.

In the future the extent to which ipsilateral cortex encodes complex motor kinematics such as finger and limb movements remains to be resolved. Additionally, how this ipsilateral motor encoding is affected by a stroke on the opposite side and whether it will impact brain-derived device control requires further study. Despite current unknowns, the recent evolution in the understanding and application of neuroprosthetic methods toward ipsilateral motor processing provides hope for an alternative nonbiological approach to ameliorating one of the largest causes of disability in the world. If successful, these techniques could provide a mechanism by which a BCI could achieve “bisomatic” control—a neuroprosthetic that can enable a single hemisphere to facilitate control of both sides of the body.

Utilization of Speech for BCI Control

Brain activity associated with speech and/or auditory processing is another possible candidate for use in BCI applications. Speech processing has been extensively studied using different types of neuroimaging (for example, using PET or fMR imaging), neurophysiological functional mapping (for example, MEG or ECoG), lesional models, or behavioral studies.^{15,18,23,65,66,81,90} These and other studies have shown that speech processing involves a widely distributed network of cortical areas located predominantly in the temporal perisylvian regions.^76,83 In particular, these regions include the Wernicke center and Broca center. The Wernicke center is located in the posterosuperior temporal lobe and is thought to be responsible for receptive language; the Broca center is located in the anterior frontal gyrus and is associated with expressive language.^4,23,90 Other findings have suggested that left premotor cortex also plays a major role in language tasks, in particular for the planning of articulation and speech production.^19,30 Cumulatively, these studies have revealed at least 4 different components involved in different aspects of speech processing. The first component consists of the M1 and medial cerebellum, which are involved in motoric aspects of speech production. The second component consists of anterosuperior temporal regions, which are activated bilaterally by the auditory stimulation provided by one’s own vocalizations. The third component, the left posterior temporal regions, contributes to acoustically based phonological analysis. The fourth and final component, the left inferior frontal and anterior insular regions, contributes to articulatory-based phonological analysis.

Given the numerous cortical networks associated with speech and the intuitive nature by which people regularly imagine speech, both the separable physiology and the cognitive task of utilizing speech may provide ideal control features that can be used either independently or as an adjunct to motor-derived control. Some authors have begun to explore the value of these speech or auditory networks for the purpose of neuroprosthetic applications. A recent study utilizing fMR imaging has shown that signals from different brain areas allow categorization into 1 of 2 phonemes.²⁴ Other authors have begun using ECoG. Wilson et al.¹⁰⁰ first demonstrated that auditory cortex could be used for real-time control of a cursor. A more recent study has shown initial evidence that some phonemes are separable during actual speech.⁹ In ongoing studies, we are determining whether groups of spoken or imagined words—those that contain different phonemes—can be differentiated using ECoG signals.^71,75 In this paradigm, patients are presented with either auditory or visual cues and are asked to repeat or imagine repeating the presented word. Each word belongs to 1 of 4 phonemic categories. The results of our comprehensive study, which includes 9 participants, demonstrate that ECoG signals, those in the gamma band in particular, differentiate between speech or imagined speech and rest. Our data also show that ECoG carries substantial information about the category of the words, either when the patient is speaking the words or simply imagining the word.

In summary, researchers are beginning to explore the value of ECoG signals in speech or auditory cortices for the purpose of BCI applications. The results to date have revealed preliminary but encouraging evidence that information in these areas could provide signals that might complement traditional motor-based signals.

Nascent Cortical Plasticity as an Alternative Approach for Acquiring Control Signals

Most authors using cortical activity for BCI control first have the users perform a series of “open-loop” activities to identify a cognitive action (for example, motor or speech intent) that maximally affects a given cortical region. For instance, electrodes over the hand area of motor cortex are best modulated with contralateral finger movements, whereas the Broca center is best controlled by jaw movements. When applying these techniques in paralyzed individuals who cannot actively move their limbs, researchers have the patients imagine moving their affected limb to elicit cortical activations. Once a differential activation is found, the researchers “close the loop” using this signal and allow the patients to directly control an artificial device through imagined movements. Through biofeedback the patients quickly adapt their neural activity—putatively through neural plasticity—to increase accuracy and independence among the control channels.^42,74

In studies involving nonhuman primates, researchers have not been able to directly instruct a monkey to imagine moving a limb; thus, to reliably control a BCI without prescreening requires online adaptation of the decoding algorithm, cortical activity, or both (that is, coadaptation). For instance, Jarosiewicz and colleagues³⁵ have used bio-feedback to enhance single-unit BCI control of 3D computer cursors.⁸⁸ In a 2002 study, Taylor and colleagues⁸⁸ first randomized the decoding vectors (that is, preferred directions) and then allowed the animal to attempt to control the 3D cursor via single-unit activity. By tracking the errors, the decoding algorithm updated the weights over time to increase accuracy. Furthermore, the animal modified the firing rates of the cortical neurons controlling the cursor such that their responses matched the underlying tuning model (that is, they became more cosine tuned). In a 2008 study, Jarosiewicz and associates³⁵ expanded on this research by showing that neurons that have had their decoding vectors specifically modified to yield errors in control will eventually reduce their firing rates to eliminate errors in the population response (only a subset of the neurons had its decoding vectors modified to yield errors). Recently, Rouse and colleagues⁶⁹ have used epidural ECoG recordings in monkeys to control 2D computer cursors for both point-to-point movements and circular drawing. By randomly choosing 2 epidural cortical locations spaced several millimeters apart, monkeys were able to modulate the high gamma band power under these 2 electrodes to control the horizontal and vertical velocity of the cursor. Using biofeedback solely (that is, no adaptation in the decoding algorithm), the monkeys were able to train the population of neurons under each 380-μm micro-ECoG electrode to accurately modulate its high gamma band activity to control the 2D cursor. Given that the horizontal and vertical dimensions were controlled by separate electrodes spaced only a few millimeters apart, the initial cursor movements were highly correlated because of the gamma band correlations in that area. After a week of training, however, the monkeys learned to independently control the gamma band activity under each electrode such that the activity completely decorrelated in the circular drawing task (Fig. 5).

Fig. 5.

Utilizing cortical plasticity for device control. To achieve 2D control, the amplitude of the signal between 65–100 Hz from one epidural ECoG electrode was used as the control for the horizontal velocity of the cursor, and a separate electrode was used for the vertical velocity of the cursor. The 2 sites were ~ 1 cm apart. For the monkey to improve his performance in a circle-drawing task, it must gain independent control of the 2 signals being used for control. For a perfectly drawn circle, the overall correlation between the 2 signals will be 0. This decorrelation could be done either indiscriminately across all frequencies or only within the frequency band being used for control. To examine what actually occurred during the experiment, the power spectrum was calculated for the 2 recorded signals in 300-msec nonoverlapping time bins. The correlation between the powers at each given frequency for the 2 different channels was then calculated for all points in time. Graph showing that the correlation between the recording sites decreased across most frequencies but most dramatically between 65–100 Hz. Therefore, these data clearly show that through biofeedback, motor cortex is quite adaptable to learning and improving BCI control.

These findings support another route for acquiring signals that can be used for BCI device control. Instead of identifying and decoding a cortical physiology that reflects an existing cognitive intention (for example, speech and motor intentions), one can capitalize on the nascent plastic ability of the brain to alter its own activity based on feedback. This plasticity allows for any region of the brain to be a potential substrate for signals for neuroprosthetic control. Preliminary studies utilizing ECoG in invasively monitored humans have shown that various sites of nonsensorimotor regions can be trained to alter their cortical physiology to achieve rapid control.¹⁰⁰ This notion of direct feedback to a given area of the brain to create the needed signal output could greatly expand the number of features available for more complex control. Additionally, because any region of cortex could be a source of signal for control, this method might have applications in more heterogeneous populations (for example, brain trauma) in which the implant may need to be tailored to their respective region of functional and dysfunctional cortex.

Conclusions

The field of neuroprosthetics is growing rapidly. The cortical physiology that underpins the manner in which a human brain encodes intentions is beginning to be understood and will have significant impact in augmenting function in those with various forms of motor disabilities. As research stretches beyond motor physiology, the field of neuroprosthetics stands to further expand its applications and the diversity of the clinical population served. Our evolving understanding of cortical physiology—as it relates to ipsilateral limb movements, language function, and plasticity—could provide higher levels of complexity in brain-derived control and extend to populations such as those with hemispheric stroke or brain trauma, who are not currently helped by existing BCI methodologies. Given the rapid progression of these technologies over the past 5 years and the concomitant swift ascent of computer processing speeds, signal analysis techniques, and emerging ideas for novel biomaterials, neuroprosthetic implants will in the near future be as common as deep brain stimulators are today. The clinical advent of this technology will usher in a new era of restorative neurosurgery.

Abbreviations used in this paper

ALS

amyotrophic lateral sclerosis

BCI

brain-computer interface

ECoG

electrocorticography

EEG

electroencephalography

fMR

functional MR

MEG

magnetoencephalography

SCI

spinal cord injury