Brain–computer interfaces: communication and restoration of movement in paralysis

Abstract

The review describes the status of brain–computer or brain–machine interface research. We focus on non-invasive brain–computer interfaces (BCIs) and their clinical utility for direct brain communication in paralysis and motor restoration in stroke. A large gap between the promises of invasive animal and human BCI preparations and the clinical reality characterizes the literature: while intact monkeys learn to execute more or less complex upper limb movements with spike patterns from motor brain regions alone without concomitant peripheral motor activity usually after extensive training, clinical applications in human diseases such as amyotrophic lateral sclerosis and paralysis from stroke or spinal cord lesions show only limited success, with the exception of verbal communication in paralysed and locked-in patients. BCIs based on electroencephalographic potentials or oscillations are ready to undergo large clinical studies and commercial production as an adjunct or a major assisted communication device for paralysed and locked-in patients. However, attempts to train completely locked-in patients with BCI communication after entering the complete locked-in state with no remaining eye movement failed. We propose that a lack of contingencies between goal directed thoughts and intentions may be at the heart of this problem. Experiments with chronically curarized rats support our hypothesis; operant conditioning and voluntary control of autonomic physiological functions turned out to be impossible in this preparation. In addition to assisted communication, BCIs consisting of operant learning of EEG slow cortical potentials and sensorimotor rhythm were demonstrated to be successful in drug resistant focal epilepsy and attention deficit disorder. First studies of non-invasive BCIs using sensorimotor rhythm of the EEG and MEG in restoration of paralysed hand movements in chronic stroke and single cases of high spinal cord lesions show some promise, but need extensive evaluation in well-controlled experiments. Invasive BMIs based on neuronal spike patterns, local field potentials or electrocorticogram may constitute the strategy of choice in severe cases of stroke and spinal cord paralysis. Future directions of BCI research should include the regulation of brain metabolism and blood flow and electrical and magnetic stimulation of the human brain (invasive and non-invasive). A series of studies using BOLD response regulation with functional magnetic resonance imaging (fMRI) and near infrared spectroscopy demonstrated a tight correlation between voluntary changes in brain metabolism and behaviour.

Non-invasive brain–computer interfaces

The brain–computer interface (BCI) or brain–machine interface (BMI) utilizes neurophysiological signals originating in the brain to activate or deactivate external devices or computers. These neurophysiological rhythms can be recorded from electrode sites inside (invasive BCIs) or outside (non-invasive BCIs) the brain. Since the pioneering studies of operant training of single neuron spike trains by Fetz (1969) and EEG alpha waves by Kamiya (1971), different signals have been proposed to control external devices (for a comprehensive review see Birbaumer, 2006b), including EEG oscillations ranging from 4 to 200 Hz, primarily the mu rhythm or sensorimotor rhythm (SMR) and its harmonics (8–30 Hz from sensorimotor cortex), electrocorticogram (ECoG) from implanted macroelectrodes using frequencies from 1 to 200 Hz, event-related brain potentials (ERPs), primarily the P300 and slow cortical potentials (SCPs) (see Elbert et al. 1984 and Birbaumer et al. 1990 for review), short latency subcortical potentials and visual evoked potentials, action potential spike trains from implanted microelectrodes (for a review see Nicolelis et al. 2004), extracellular synaptic field potentials from implanted electrodes, metabolic brain activity, and the BOLD response recorded through functional magnetic resonance imaging (fMRI) and blood oxygenation with near infrared spectroscopy (NIRS) (Weiskopf et al. 2007; Sitaram et al. 2007a). The non-invasive BCI, the focus of this review, does not involve tissue-penetrating medical procedures but recording of brain signals from surface electrodes, sensors or electromagnetic fields.

The idea of ‘reading thoughts’ has been mentioned since Berger et al. (1929), with the possibility of processing EEG waveforms using sophisticated mathematical analyses. Grey Walter, the brilliant EEG pioneer who described the contingent negative variation (CNV) often called the ‘expectancy wave’, built the first automatic frequency analyser and the computer of ‘average transients’ with the intention of discriminating covert thoughts and language in the human EEG (Walter, 1964). Fetz (1969) published the first paper on invasive operant conditioning of cortical spike trains in monkeys. Only the recent development of BCIs, however, has brought us closer to the dreams of these pioneers of EEG research.

The historic roots of non-invasive BCI research lie in work done in the fields of neurofeedback and operant conditioning of neuroelectric brain activity. Most of the clinical BCI studies in human patients use biofeedback of EEG oscillations or ERPs (see Elbert et al. 1984). In biofeedback the subject receives visual or auditory on-line feedback of his or her brain activity and tries to voluntarily modify a particular type of brain wave. The feedback signal contains both the information on the degree of success in controlling the signal and it indicates the reward. Self-regulation of brain waves as described in the biofeedback literature was reported to have therapeutic effects on many psychiatric and neurological conditions but only a few indications passed rigorous clinical–experimental testing as described below (see Barber et al. 1971–1978, Birbaumer & Kimmel, 1979).

Neuroelectric and neuromagnetic BCIs

Invasive and non-invasive BCIs

Generation of movement from single cell firing activity within the motor cortex (Nicolelis, 2003) or from parietal neuronal pools (Scherberger et al. 2005) in animals and human patients (Hochberg et al. 2006) has been accomplished using densely packed microelectrode arrays of up to several hundred microelectrodes (Suner et al. 2005).

Monkeys learned to move cursors onto computer screen targets in a predetermined sequence by successively activating neuronal pools. In one particularly successful preparation (Nicolelis, 2003), activity from 32 cortical cells in the motor hand cortex was sufficient to implement skillful reaching movements of an artificial arm after extensive training. Plasticity of the cortical circuits involved allowed control of these movements directly from cellular activity even outside the primary or secondary homuncular representations of the motor cortex (Taylor et al. 2002). In human patients, a multielectrode array recording spike and field potentials was implanted in the hand motor representation areas of two quadriplegic patients (Hochberg et al. 2006). Within a few training sessions, patients learned to use neuronal activity from spikes to move a computer cursor in several directions, comparable to the tasks used for multidimensional cursor movements in the non-invasive BCI models controlled by sensorimotor rhythms recorded from surface scalp electrodes (Wolpaw & McFarland, 2004). The advances in the invasive BCI field have used intact animals who learned to move an artificial device or cursor for food reward without moving their intact arm in highly artificial laboratory situations (Nicolelis, 2003). While impressive and intriguing for its multiple scientific implications, any generalization from the invasive animal BCI approach to paralysed people appeared premature. In contrast to the non-invasive approach none of the invasive procedures allowed restoration of skillful movement in paralysed animals or people in everyday-life situations. None of the experimental animals used were paralysed and the three human patients did not move a paralysed limb.

Four types of signals, three based on EEG activity and one on MEG activity, have been more thoroughly tested in non-invasive BCI research (Fig. 1). A slow cortical potential (SCP)-based spelling BCI has been developed to allow severely paralysed patients to communicate (Birbaumer et al. 1999), a system originally tested in patients with intractable epilepsy (Kotchoubey et al. 2001). Sensorimotor rhythms (SMRs) have been used as the critical EEG oscillation for control of a BCI device (Wolpaw et al. 2002; Wolpaw & McFarland, 2004). Another extensively tested BCI controller is the P300 event-related brain potential (ERP) BCI developed by Donchin (Farwell & Donchin, 1988). SCP control and SMR (often called mu rhythm) control are learned through visual and auditory feedback and reward and need 5–20 training sessions before significant production of SCPs or mu rhythms is achieved, while the P300 BCI needs no training at all. For P300, as depicted in Fig. 1C, rows and columns of a matrix consisting of the letters of the alphabet are lightened in rapid succession. The subject is instructed to concentrate on the letter he or she wants to spell. Whenever the desired letter (P on Fig. 1C) is among the lightened string, a P300 ERP appears in the EEG selecting the desired letter (Sellers & Donchin, 2006). This visual version of the P300 BCI appears to be more easily controlled and stable in performance as shown in a study by Hinterberger et al. (2006).

Figure 1.

Three different types of non-invasive brain–computer interfaces (BCI) A, an ALS patient select letters from a letter string (below on screen) with slow cortical potentials (SCPs, top row); the spelled words appear at the upper part of the screen. B, sensorimotor rhythm (SMR) training. Imagery of movement desynchronizes the SMR (lower EEG trace) from sensorimotor cortex (coloured gyri below). Feedback appears as a moving cursor (red rectangles) towards the right lower bar on the screen (i.e. desynchronization) or the (non-visible) upper bar on the screen (synchronization). C, P300 brain–computer interface (BCI). The rows and columns of the letter matrix (right) are randomly illuminated horizontally and vertically. The patient concentrates on the letter he or she wants to select (p). The P300 potentials to the desired letter in comparison to the non-desired letters (green) are depicted on the left in red.

Slow cortical potential BCIs

Beginning in 1979 Birbaumer and coworkers published an extensive series of experiments demonstrating operant control of SCPs. These demonstrations differed from previous work on biofeedback in that they documented in well controlled experimental protocols (1) strong and anatomically specific effects of self-induced changes in cortical activity on behaviour and cognition, and (2) solid neurophysiological evidence about the anatomical sources and physiological function of SCPs (see Birbaumer et al. 1990, 1992, 1995; and Birbaumer, 1999 for a review).

Of particular interest in the context of central nervous system motor mediation of voluntary control of brain activity was the fact that SCPs originating from posterior parietal and occipital sources were resistant to operant learning while central and frontal SCPs of both hemispheres could be brought under voluntary, differential operant control after one to five training sessions of 1 h each (Lutzenberger et al. 1993). Several clinical studies confirmed the critical importance of the anterior brain systems for physiological regulation of these functions. Lutzenberger et al. (1980) showed that patients with extended prefrontal lobe lesions were unable to learn SCP control despite intact intellectual functioning. Disorders with prefrontal dysfunction such as attention deficit disorder (ADD; Birbaumer et al. 1986) and schizophrenia (Schneider et al. 1992) exhibited extreme difficulties in acquiring SCP control. Attentional improvement in these disorders may require much longer training periods (Strehl et al. 2006). In this type of BCI, peripheral motor mediation played no particular role in SCP conditioning while intact prefrontal systems seemed to be a prerequisite for successful brain control (Birbaumer & Kimmel, 1979). Figure 2 shows the results of a study where healthy subjects learned SCP control from a central vertex electrode and fMRI (BOLD response) was recorded simultaneously during training indicating that voluntary control of SCP is associated with regulation of activity in the anterior basal ganglia. Subjects with superior performance in neuroelectric regulation showed more anterior basal ganglia activation.

Figure 2.

BOLD response during cortical negativity and positivity A, BOLD responses during self-generated cortical negativity (first vertical column of three brain views, red indicates increase in BOLD, green decrease) and cortical positivity (second vertical column of left figure). B, activation sites during successful voluntary positive SCP regulation in anterior basal ganglia top row, and premotor cortex (below). Adapted from Hinterberger et al. (2003) with kind permission of Springer Science and Business Media.

In this study, subjects received visual feedback of positive and negative SCPs in 6 s trials and were rewarded for the production of target SCP amplitudes (Hinterberger et al. 2004, 2005a, b). Successful voluntary brain control correlated with changes in activity in premotor areas and the anterior parts of the basal ganglia. These findings were consistent with the previous proposal that physiological regulation of SCPs and selective attention and preparation depend critically on anterior basal ganglia activity regulating local cortical activation thresholds and SCPs (Birbaumer et al. 1990. Braitenberg (in Braitenberg & Schüz, 1991) created the term ‘thought pump’ (‘Gedankenpumpe’ in German) for this basal ganglia–thalamus–cortex loop. Taken together, the extensive literature on the neurophysiology of SCPs suggests that operant voluntary control of local cortical excitation thresholds underlying goal directed thinking and preparation depends on an intact motor and/or premotor cortical and subcortical system.

The sensorimotor rhythm BCI

The Wolpaw group in Albany, New York and the Pfurtscheller lab in Graz, Austria demonstrated in an extensive series of experiments that healthy subjects and paralysed patients achieve voluntary control of right and left hemispheric SMR at the rolandic cortex by imagining movements. Sterman (Sterman & Friar, 1972; Sterman, 1981) was the first to propose self-control of epileptic seizures (Elbert et al. 1984) by an augmentation of the SMR. SMRs in human subjects are recorded exclusively over sensorimotor areas with frequencies of 10–20 Hz and variable amplitudes. Pfurtscheller et al. (2005) localized the source of human SMRs in the sensorimotor regions following the homuncular organization of the motor and somatosensory cortical strip. Imagery of hand movement abolishes SMRs over the hand region, and imagery or actual movement of the legs blocks SMRs in the interhemispheric sulcus. With MEG BCIs (see below) even single finger sources can be localized and regulated using SMR desynchronization. Pfutscheller called these phenomena event-related desynchronization and event-related synchronization (Pfurtscheller et al. 2005).

On the basis of careful animal experiments (Sterman & Clemente, 1962a, b), Sterman demonstrated incompatibility of seizures in motor and premotor areas in the presence of SMRs. Cats exhibited maximum SMRs during motor inhibition and various sleep stages. The presence of spindles during different sleep stages, particularly during rapid eye movement (REM) sleep indicated recruitment of inhibitory thalamocortical circuits and blocked experimentally induced seizures. Sleep spindles and SMRs share identical physiological mechanisms. However, –it is not clear whether the neurophysiological bases of the two phenomena are really comparable, and therefore we recommend that the term ‘SMR’ instead of ‘mu rhythm’ as used by Sterman and colleagues be retained because of its well-defined theoretical and experimental background.

It is not accidental that SMR operant control is achieved through activation and deactivation of the central motor loops. Again, successful voluntary regulation of a physiological variable is tied to the regulation of the central motor system. The results of SMR control in animals and patients seem to demonstrate that manipulation (mediation) of the peripheral motor efferents is not a necessary requirement of SMR control, at least on the basis of EMG recordings of the arm muscles showing no measurable variation during motor imagery with central nervous system event-related desynchronization (Pfurtscheller, Neuper & Birbaumer, 2005). The successful brain regulation of SMRs in completely paralysed patients reported below confirms that changes of the peripheral motor system do not mediate central nervous system (CNS) activity responsible for SMR origin. The notion of the critical role of central motor activity in voluntary action and thought remains.

The P300 BCI

The P300-ERP is the best studied event-related brain potential. It is evoked by the random and surprising presentation of target stimuli that require updating of current memory traces (Donchin, 1981; Verleger, 1988). Depending on the complexity of the stimulus its latency varies from 300 ms to 1 s. It can be recorded best over anterior parietal areas but its anatomical sources are often hippocampal, dependent upon the content of the memory trace and the stimulus material. The positive polarity of the P300 indicates an inhibitory function, probably blocking competing information processing in the presence of new and challenging material. The P300 is widely used in the clinic to evaluate deficits in attentional processing in cognitive disorders and ageing, and recently in lie detection research. Donchin exploited in his BCI system the reliability and validity of the P300: even after long time periods the P300 amplitude to the desired letter during BCI training does not habituate and very few subjects lack a P300 (Nijbor et al. submitted).

In contrast to all other existing BCIs, learning of self-regulation of the brain response and feedback is not necessary and the short latency of the P300 (300 ms instead of seconds in the SCP and SMR BCIs) allows much faster selection of letters than any other BCI system. It requires, however, the ability to internally spell at high speed, an intact visual system (for the more reliable visual P300) and intact attention, not always present in completely paralysed patients. Auditory P300 BCIs are slower and their efficacy has not been thoroughly evaluated yet.

The magnetoencephalography BCI

For the purpose of restoration of hand function in stroke (see below) a BCI using a 270-channel magnetoencephalography (MEG) device was developed in parallel at the Human Cortical Physiology Section–Stroke Neurorehabilitation Clinic at the NIH in the USA and at the MEG Center at the University of Tuebingen, Germany. The MEG BCI uses the same program modules of BCI 2000 for EEG and electrocorticogram (ECoG) developed by the BCI groups at the Wadsworth Laboratory in Albany, New York and the University of Tuebingen, available (free of charge) on the net (http://www.bciresearch.org). The system selects the most responsive MEG channels during imagery or actual movements and uses these channels for the learning of self-control of a prosthetic hand affixed to the paralysed hand. Usually frequencies in the SMR or beta–gamma range are classified and rewarded, dependent upon the individual patient's frequency profile. The exquisite spatial and time resolution of the MEG allows anatomically specific and graded activation and deactivation of circumscribed electric sources (i.e. single fingers) and a much wider frequency range than EEG-based BCIs.

Learning and neural plasticity involved in BCI control

Acquisition of operant learning control over autonomic functions and brain activity without an intact somatic central or peripheral system was demonstrated first in the curarized rat, but turned out to be impossible to replicate (see below). In humans, of seven patients with ALS who started training after they had entered the completely locked-in state (CLIS) without any muscular control (no eye movement, no external sphincter control), none acquired sufficient brain control to communicate, and only one communicated for a limited time period of 3 h with a ph-recording imagery technique developed in our lab (Wilhelm et al. 2006). We hypothesized (Birbaumer, 2006a, b) that loss of the contingency between a voluntary response and its feedback or subsequent reward in individuals who are completely paralysed would prevent learning even if afferent input and cognitive processing (attention, memory, verbal imagery) remains intact. If the voluntary response is only cognitive, such as in non-overt goal orientated imagery in the locked-in state, the feedback or reward does not follow reliably an environmental or internal change and consequently extinguishes. Psychophysical studies (Haggard et al. 2002) demonstrate that if the behavioural response is elicited independently of a conscious decision and intention, the conscious awareness of the contingency and the conscious experience of the decision (‘will’) vanish. In the CLIS all contingencies between goal-directed thinking and intentions are lost because nobody responds to the particular intention. We termed this process ‘extinction of thought’, and it may reduce synaptic plasticity at the molecular level leading to a generalized deficit in learning (Birbaumer & Schmidt, 2005; Birbaumer, 2006a). A single case of a completely locked-in patient able to communicate with a BCI after entering that state without a history of BCI control would disprove this speculation. In the BCI literature no such case has been reported yet.

A possible way to address the ‘extinction of thought’ problem may consist in the creation of artificial contingencies. For example, it would be possible to utilize transcranial magnetic stimulation (TMS) pulses or peripheral nerve stimulation (PNS) applied to a specific body part or to different frontal or motor brain areas contingent on the elicited brain or peripheral nerve responses of the BCI. We believe that the absence of extensive contingent stimulation and reward during waking hours may lead to the lost of associative connections between intention and response or consequence. A particularly original and Hebbian solution to this problem was recently proposed by Jackson et al. (2006) (see below). Transmission of external currents and magnetic fields into the brain to induce or accelerate learning of BCI control may represent an interesting future extension of existing BCI systems (see Karim et al. 2004 for a first attempt).

Another possibility to address these problems may include metabolic BCIs using fMRI or near infrared spectroscopy (NIRS) (see Sitaram et al. 2007a, b). Metabolic changes and vascular variations are sensed by arterial receptors and may be utilized to control contingent perceptual responses (Adam, 1998). However, evidence so far indicates that even metabolic brain activity needs a history of learned contingencies in order to be evoked voluntarily and become useful for communication in the completely locked-in patient. It is still unclear whether successful BCI training results can be transferred from the partially locked-in state to the completely locked-in state.

Clinical applications of BCIs

Epilepsy and attention regulation

SCP control allows voluntary regulation of activity in different brain areas with specific behavioural and cognitive consequences (for an overview see Rockstroh et al. 1989; Kotchoubey et al. 2001). Sterman (Sterman & Friar, 1972; Sterman, 1981) was the first to propose the use of SCPs, particularly augmentation of SMRs, to control epileptic seizures (Elbert et al. 1984). On the basis of a series of elegant animal experiments (Sterman & Clemente, 1962a, b), Sterman demonstrated incompatibility of seizures in motor and premotor areas during augmented SMRs. Cats exhibited maximum SMRs during motor inhibition and spindle sleep stages. The presence of spindles particularly during REM sleep indicated recruitment of inhibitory thalamocortical circuits and blocked experimentally induced seizures. Sleep spindles and SMRs share common physiological substrates. In a series of experiments, epileptic cats and humans were trained to increase SMRs, and after extensive training ranging from 20 to more than 100 sessions, Sterman (1977) was able to demonstrate seizure reduction and remission in some patients with drug-resistant epilepsy. It is important to note that while the SMR is often called the mu rhythm following a suggestion of Gastaut (1952a) and Gastaut et al. (1952), who noted its abolition in some types of seizures, it is not clear whether the neurophysiological bases of the two phenomena are comparable. Sterman's interesting discovery never evolved into the design of well controlled clinical trials and as a consequence, its applicability as a therapeutic tool remains in the realm of a well founded hypothesis.

More recent studies on SCP regulation of intractable drug resistent epilepsy showed additional support for this approach (Rockstroh et al. 1989, 1993; Kotchoubey et al. 2001). Patients with focal epileptic seizures were trained to down-regulate cortical excitation by rewarding them for positive cortical potentials and perception of SCP changes. After very long training periods, some of these patients gained close to 100% control of their SCPs and consequently there was a strong decrease of seizure activity. Clinically the BCI training with focal intractable epilepsies was highly successful in reducing seizure frequencies on average to half the baseline and controls. One to two years of follow-ups demonstrated stable improvements and some patients remained seizure free. Only patients with very high negative SCP amplitudes before training did not profit from this SCP BCI (Kotchoubey et al. 2001; Strehl et al. 2006). In addition, a significant gain in IQ and cognitive functioning was achieved.

The fact that some patients were able to control their SCPs in 100% of the trials tempted Birbaumer and colleagues to apply cortical regulation as a BCI for paralysed patients: given that epileptic patients suffering from a dysregulation of cortical excitation and inhibition learn to control their brain responses both within the laboratory and in the outside world, it is not unreasonable to ask whether a paralysed patient could learn to activate an external device or computer in order to move a prosthetic arm or to convey messages to a voice-system.

A similar procedure was applied in children with attention deficit disorder (ADD) (Fuchs et al. 2003; Strehl et al. 2006). It was shown that both learning to induce an increase of central–frontal negativity of SCPs and increase of SMRs and beta-1 frequencies improves the symptoms. These EEG changes were associated with an improvement of ADD symptoms comparable to medication (Ritalin). No difference in efficacy was found between SCP, beta frequency or SMR training, suggesting that the behavioural effects rely on the convergence of control of different EEG activities on a final common therapeutic pathway, possibly an improvement in the general capacity to regulate attention via brain regulation.

ALS and verbal communication with BCI

Over the past 15 years, 28 patients with ALS and five patients with other severe brain disorders were trained with BCIs, most of them in their homes (Birbaumer et al. 1999; Kübler et al. 2001a, b, Birbaumer, 2006b; Kübler & Birbaumer, submitted). Stages of impairment ranged from completely locked-in (no eye and sphincter movements or other form of motor activity present) to paralysis of legs or arms at the beginning stage of the disease (see Table 1). Most patients were trained with the SCP BCI system described above. More recently the P300 and the SMR systems were used. Most of the patients achieved significant control of their brain activity and were able to select letters and write words with one of the three BCI systems. Six of them achieved complete independence of communication with the BCI. The most notable exception, as discussed above, was patients (n = 7) who started the training when they were already in a complete locked-in state. None of them were able to communicate with a BCI; one patient communicated with a ph-based device but lost control after three sessions without regaining it (Wilhelm et al. 2006).

Table 1.

Background and disease related data of ALS patients in BCI experiments (n = 33)

	Sex		Diagnosis		Degree of physical impairment					Ventilation
Mean age ±s.d.	M	F	ALS	other	minor	moderate	major	LIS	CLIS	no	yes	night
47.6 ± 11.6	23	10	28	5	5	7	7	7	7	17	14	2

LIS, locked-in state; CLIS, completely locked-in state.

From the studies summarized in the cited work it is not clear which of the three non-invasive BCIs is the most promising. In a recently completed study (Nijboer et al. submitted) eight severely paralysed patients with ALS were trained with SCP, P300 and SMR BCIs in a balanced crossover-within-subject design. Each training block of each BCI type lasted 20 sessions. The results were clear cut: SCP BCI improved control of brain activity within this short training period but was not good enough to select letters (70% minimum success rate). The fastest acquisition and fastest spelling rates were achieved with the P300 BCI; SMR was also successful but not as fast as the P300 system. In the locked-in state, eye control is lost and it is unclear from this study whether an auditory P300 BCI with only a few letters presented simultaneously can achieve better results. Open to investigation is whether a patient successfully trained when the locked-in state was still not complete is able to carry the improved control over the completely locked-in state (see Birbaumer, 2006b).

Paralysis, curarization and operant control of physiological responses

BCI research is rooted in the tradition of biofeedback and instrumental operant learning of autonomic functions. During the late 1960s and early 1970s, Neal E. Miller and collaborators opposed the traditional wisdom of the autonomic nervous system (ANS) as autonomous and independent of voluntary control of the somatic central nervous system (CNS). Miller (1969), in a landmark paper in Science, challenged the view that voluntary control is acquired through operant (instrumental) conditioning while modification of involuntary ANS functions is learned through classical (Pavlovian) conditioning, a distinction first emphasized by Skinner (Skinner, 1953; Holland & Skinner, 1961). Miller presented experimental evidence in curarized and artificially ventilated rats showing that even after long-term curarization of several weeks the animals learned to increase and decrease heart rate, renal blood flow, and dilatation and constriction of peripheral arteries in an operant conditioning protocol rewarding the animals for increases and decreases of these physiological functions. The studies stirred an enormous interest in the scientific and clinical community, particularly in the fields of psychosomatic medicine and behaviour modification. The results suggested that instrumental (‘voluntary’) control of autonomic functions is possible without any mediation of the somatic muscular system. Operant training of any internal body functions seemed possible, opening the door for psychological and learning treatment of many medical diseases, such as high blood pressure, cardiac arrhythmias, vascular pathologies, renal failure and gastrointestinal disorders. In the clinic, biofeedback of these functions replaced the operant conditioning in rats; the feedback from the specific physiological variable constituted the reward (for an overview of these years of enthusiasm see the Aldine series Biofeedback and Self-Control, Barber et al. 1971–1978). During the next two decades, Miller and his students at Rockefeller University tried to replicate their own findings. However, the steady decline of the size of the conditioning effect with each replication created a severe credibility problem for operant learning of autonomic functions and biofeedback. Finally, by the mid-eighties, it was impossible to replicate the effects in curarized rats. Barry Dworkin, Neal Miller's last and most prolific student, continued to try and built a sophisticated ‘intensive care unit’ for curarized rats, but again, operant training of autonomic function or nerves in the curarized rat was impossible.

In contrast, classical conditioning succeeded even in single facial nerve fibres (Dworkin & Miller, 1986; Dworkin, 1993). Dworkin attributed the failure of operant techniques to the missing homeostatic effect of the reward: the reward acquires its positive effect through homeostasis-restoring effects (i.e. ingestion of food restores glucostatic and fluid balance). In the curarized rat (and the completely paralysed intubated and fed patient), where all body functions are kept artificially constant, the homeostatic function of the reward is no longer present because imbalances of the equilibrium do not occur. As we have stated above, without a rewarding contingency, learning and voluntary control and goal orientated consciousness are likely to be extinguished.

The chronically curarized rat and the completely paralysed, artificially ventilated and fed locked-in patient share many similarities; difficulties in communicating with these patients may conceivably have some common origins.

The difficulties in replicating the operant learning of autonomic variables were accompanied by an ‘awakening’ in the clinical arena of biofeedback applications: the most impressive clinical results were achieved with electromyographic feedback in chronic neuromuscular pain (Flor & Birbaumer, 1993), neuromuscular rehabilitation of various neurological conditions (Birbaumer & Kimmel, 1979), particularly external spincter control in enuresis end encopresis (Hoelzl & Whitehead, 1983), and posture control in kyphosis and scoliosis (Dworkin et al. 1985; Birbaumer et al. 1994), but clinically unimpressive or negligible results in essential hypertension (Engel, 1981; McGrady et al. 1995), heart rate control (Cuthbert et al. 1981), and gastric motility (Hoelzl & Whitehead, 1983). It became painfully clear that only very limited positive effects of biofeedback on visceral pathology with clinically and statistically relevant changes occur. There was one notable exception, however: neurofeedback of brain activity (Elbert et al. 1984).

Paralysis from spinal cord lesions

Pfurtscheller et al. (2005) were the first to apply the SMR BCI to a patient with a high spinal cord lesion. The patient learned to control delivery of electrical stimulation to hand and arm muscles using SMR increase or decrease. This application allowed him to grab a glass and lift it to his mouth with successive activation of electrostimulation of arm and hand muscles. However, he was using this BCI for demonstration purposes only and not for activities of daily living. Donoghue (Hochberg et al. 2006) implanted an array of 100 microelectrodes in the primary motor cortex of several high spinal cord patients and one patient with a pontine infarct. All patients had preserved motor control of face and other muscles above the lesion level. Using this invasive BCI system, patients were able to move a computer curser on a screen in circular fashion with neuronal spike activation of single cells, but no restoration of movement was possible (see Hochberg et al. 2006).

Several studies (see Nicolelis, 2003 for a summary) trained healthy monkeys and rats to perform skilful movements with spike patters from single motor or parietal neurons controlling BCI devices. These highly encouraging results led to substantial optimism. It is not clear why the human preparations (see also Kennedy et al. 2004 for a single electrode preparation) have achieved less satisfying results so far in terms of application to activities of daily living (Hochberg et al. 2006).

Restoration of movement in chronic stroke

Figure 3 shows a BCI based on magnetoencephalography (MEG) for chronic stroke patients developed by the groups of L. Cohen, E. Buch, C. Braun and N. Birbaumer (Birbaumer, 2006b).

Figure 3.

Brain–orthosis interface MEG BCI for chronic stroke. A, patients observe their SMR activity represented by the red curser on a screen (right part) after instructed to increase (top goal bar) or decrease (lower goal bar) SMR. Decrease closes and increase opens the hand (right) in 5 steps depending on SMR 2 s before. B, stroke patient with hand in orthosis in the MEG during training. Note that patients not only receive visual feedback from the feedback screen but also from watching and feeling their own paralysed hand moving.

MEG oscillations and evoked magnetic fields offer a better spatial resolution than, and the same exquisite time resolution as EEG, SCPs and P300. The whole-cortex magnetic activity is recorded in patients with subcortical stroke without remaining hand function from 270 sensors. Three sensors detecting activity over sensorimotor regions of the lesioned hemisphere, most sensitive to kinesthetic imagery of movement of the paralysed hand, are selected for the BCI training. In most cases a 10–15 Hz SMR or one of its high frequency harmonics is used for training. An artificial, flexible hand prosthesis is attached to the paralysed hand (see Fig. 3) and the patient learns with visual feedback to open the hand by increasing SMR over the lesioned hemisphere and to close the hand by decreasing it. The choice of brain activity from the lesioned hemisphere originates in reports that stroke patients that recover motor function rely on activity in a motor network that highly resembles that in normal individuals (predominantly contralateral) and studies indicating that up-regulation of activity in ipsilesional regions and down-regulation in contralesional areas, at least in some patients, result in clinical improvements (Murase et al. 2004).

Preliminary data on five patients demonstrate clearly that voluntary control of the prosthesis attached to the completely paralysed hand is possible using activity originating in the lesioned hemisphere in 20 sessions. Figure 4 shows the learning curves of two paradigmatic cases: opening and closing of the completely paralysed hand was achieved in all but one patient with direct brain commands. Patients increased neuromagnetic activity in the relevant frequencies with training, mainly over the central brain sites, less so over the lesioned area as predicted. These preliminary data show that control of a prosthesis applied to a completely paralysed limb without remaining peripheral function is possible from ipsilesional brain regions.

Figure 4.

Learning to move a paralysed hand in chronic stroke with the MEG BCI Performance of two chronic stroke patients with no residual hand movement over 20 sessions (170 runs). The light blue line depicts the online performance, the red line the significant linear trend (see text).

Ethical and quality of life issues in BCI research and application

Most ALS patients opt against artificial respiration and feeding and die of respiratory and other clinical complications. In many countries, doctors are allowed to assist the transition with sedating medication to ease respiration related symptoms. If doctor-assisted suicide or euthanasia is legal, as it is in the Netherlands and Belgium, very few patients vote for continuation of life. The vast majority of family members and doctors believe that the quality of life in total paralysis is poor, that continuation of life constitutes a burden for the patient and that it is unethical to use emergency measures such as tracheostomy to continue life. The pressure on the patient to discontinue life is high. However, the evidence does not support hastened death decisions in ALS (Quill, 2005; Albert et al. 2005). Most instruments measuring depression and quality of life, such as the widely used Beck or Hamilton depression scales, are invalid for paralysed people living in protected environments because most of the questions do not apply to the life of a paralysed person (‘I usually enjoy a good meal’, ‘I like to see a beautiful sunset’). Special instruments have had to be developed for this population (Kübler et al. 2005). In studies by Breitbart et al. (2000) and at the University of Tuebingen by Kübler et al. (2005) only 9% of ALS patients showed long episodes of depression, most of them in the time period following the diagnosis and a period of weeks after tracheostomy. The results for depression and for quality of life rated by patients and family members and caretakers demonstrated that ALS patients are not clinically depressed but in better mood than psychiatrically depressed individuals without any life threatening disease. Likewise, patients rate their quality of life as much better than their caretakers and family members do, even when these patients are completely paralysed and intubated. None of the patients of our sample (some of them in the locked-in state; LIS) requested hastened death, and life expectancy with adequate communication through BCI or other means is much longer than the average of 5 years.

It could be argued that questionnaires and interviews reflect more social desirability and social pressure than the ‘real’ behavioural–emotional state of the patient. The social pressure in ALS, however, directs the patient toward death and interruption of life support. The data therefore may underestimate the positive attitude in these groups. This hypothesis is strongly supported by a series of experiments with ALS patients at all stages of their disease using the International Affective Picture System (IAPS; Lang, et al. 1999). Luléet al. (2005). Using a selection of IAPS pictures with social content, more positive emotions to positive pictures and less negative ratings to negative pictures were found in ALS than in matched healthy controls. Even more surprising are the brain responses to the IAPS slides (Fig. 15). fMRI measurement in 13 patients with ALS and controls demonstrated increased activation in the supramarginal gyrus and other areas responsible for empathic emotional responses to others, comparable to the ‘mirror neuron network’ identified first by Rizolatti and colleagues (Gallese et al. 2004) in patients. Furthermore, brain areas related to the processing of negative emotional information such as the anterior insulae and amygdala show less activation in ALS. These differences become stronger with progression of the disease six months later.

Figure 5.

Brain activations during social emotional slide viewing A, only areas with larger BOLD responses in paralysed ALS patients than controls are shown. B, the replication of the experiment 6 months after the first session shown in A when patients progressed toward complete paralysis. (From Luléet al. 2005.)

This evidence suggests that with progression of this fatal disease, emotional responses improve toward positively valenced social cues resulting in a more positive emotional state than in healthy controls. The positive responding and positive interaction of the social environment and caretakers to a fatally ill, paralysed person may in part be responsible for the prosocial emotional behaviour and for the modified brain representation of the ALS patients depicted in Fig. 15 as predicted by social learning theory (Bandura, 1969).

Taken together, the results on emotional responding and quality of life in paralysed ALS patients suggest a more cautious and ethically more responsive approach toward hastened death decisions and last-will orders of patients and their families. The data reported here also speak pervasively for the usefulness of BCIs in ALS and other neurological conditions leading to complete paralysis.

Future directions

The metabolic BCI: BOLD control as a model system

fMRI measures increases and decreases of paramagnetic load of blood flow to activated pools of neurons, particularly to apical dendrites (Logothetis et al. 2001). Paramagnetic charge is determined by blood oxygenation level dependent (BOLD) flow, which reflects local metabolic deficiencies of the vascular bed supplying the neurons. Logothetis et al. (2001) have shown that the correlation of local blood flow change and the BOLD signal is particularly high for the neuronal inflow to the apical dendrites reflecting primarily intracortical activity (‘thinking’). SCPs seem to be the closest electrophysiological relative of the BOLD signal and simultaneous recording of BCI-regulated SCPs and BOLD responses has demonstrated a strong relationship between self-regulated increases and decreases of SCP at central electrodes and BOLD variations in the anterior basal ganglia and premotor cortex as predicted by neurophysiological considerations of SCP sources (Birbaumer et al. 1990; Hinterberger et al. 2005; see Fig. 12)

All BCI systems need some form of learned voluntary control or cognitive voluntary attentional modulation such as the P300 BCI, and fMRI with its exquisite anatomical resolution allows study of the neuronal processes necessary for BCI control. Figure 16 shows the first fMRI BCI system developed in Birbaumer's laboratory (Weiskopf et al. 2003) using extremely fast echo-planar gradient sequences in a 1.5–3 T MR scanner (Weiskopf et al. 2005, 2007). Subjects observe the visual feedback reflecting the movement corrected BOLD changes of a circumscribed cortical or subcortical brain area; Fig. 7 gives an example of the anterior insular region implicated in the regulation of negative emotions and affective pain. The latency of the BOLD change to the neural discharge is 3 s but the feedback to the subject or patient is instantaneous and constant to the physiological response and therefore functions as an effective reward for voluntary BOLD changes. Subjects are instructed to use emotional or motor or cognitive imagery to influence the feedback signal in the required direction. Areas with unknown function or subjects without cognitive abilities to imagine are treated identically, but subjects receive only the instruction to influence the feedback signal in the required direction and subjects are rewarded for successful attempts to modify the metabolic flow in the particular brain region.

Figure 7.

Voluntary increase of brain activity in the insula A, increase in percentage BOLD during operant feedback training of the left insula averaged over 9 healthy subjects and 3 sessions of 20 min each. B, horizontal section at the insular region; BOLD responses averaged over 9 healthy subjects and three training sessions. C, control group of 3 subjects instructed to use emotional imagery without contingent feedback of the BOLD responses over three sessions.

Figure 6.

FMRI BCI system for on-line training of BOLD regulation as described in Weiskopf et al. (2005) See text for explanation.

Weiskopf et al. (2005, 2007) provide an overview of recent results of fMRI BCI. DeCharms et al. (2005) reported its effects on pain perception. Apart from several cortical areas, amygdalae, and anterior cingulate, anterior insulae and parahippocampal gyrus were shown to respond differentially with increased and decreased BOLD response to self-regulation within 2–10 sessions of 20 min each. What is even more important, area-specific effects on behaviour were demonstrated: voluntary increases of insular activation intensified negative emotional valence ratings of emotional slides, anterior cingulate affects arousal, and parahippocampal gyrus BOLD increase reduced memory performance (probably through overactivation of the region).

These studies on fMRI BCI and behaviour illustrate some important properties of BCI control:

1. voluntary control of circumscribed brain areas is possible within several hours of training in healthy subjects

2. the learned regulation is specific and does not coactivate unspecifically other brain sites

3. the behavioural effects of self-induced local brain changes are functionally specific for the respective brain region

4. the central motor systems need not be coactivated during brain self-regulation indicating that motor mediation of the brain is not a necessary prerequisite for its regulation and learned brain control should be possible in completely paralysed people and animals unable to use motor projections for peripheral physiological regulation (Dworkin & Miller, 1986)

5. cognitive activity such as imagery may assist the acquisition of brain control but is not a necessary condition to manipulate brain regions locally.

Psychopathy, anxiety and the fMRI BCI

In an ongoing series of experiments, Birbaumer et al. (2005) demonstrated that criminal psychopaths show a dramatic deficit in metabolic activity of the fear circuit: lateral orbital prefrontal cortex, amygdalae, and anterior cingulate, anterior insula and superior parietal cortex are not activated during aversive classical conditioning. Learning to re-activate these areas in fearevoking situations should reduce the central deficit of psychopathic traits, a complete lack of anticipatory fear.

The fMRI BCI described in the first and second paragraph was applied in preliminary pilot studies to increase emotions of anxiety and fear in healthy subjects and criminal psychopaths (Sitaram et al. 2007a). Figure 7 shows the effects of training to increase BOLD responses in the left anterior insula involved in the fear circuit in nine healthy subjects. After three sessions only, within 60–90 min there was a significant increase of BOLD responses in this area compared to a sham control and imagery control. The imagery control group only is depicted in Fig. 7C.

The experimental condition resulted in a more negative emotional valence rating of fear-evoking pictures of the IAPS series (Lang et al. 1999) during insula BOLD increase only. Note: the percentage BOLD change in the target ROI – left insula – increased with training while the change in the right insula and the reference ROI was kept comparatively constant. The effect is not only area specific but also valence specific: aversive slides only change their valence; other types of emotional slides are not affected. In contrast to the results of Birbaumer et al. (2005) finding a complete lack of activation of the fear circuit (amygdala, anterior insula and cingulate, orbitofrontal cortex) in criminal psychopaths during aversive Pavlovian conditioning the first three psychopaths investigated were able to learn to increase activation of the anterior insula. The behavioural effects of the training are currently being investigated.

Near infrared spectroscopy BCI

Near infrared spectroscopy (NIRS) is a non-invasive psychophysiological technique which utilizes light in the near infrared range (700–1000 nm) to determine cerebral oxygenation, blood flow and metabolic status of localized regions of the brain. The degree of increases in regional cerebral blood flow (rCBF) exceeds that of increases in regional cerebral oxygen metabolic rate (rCMRO₂) resulting in a decrease in deoxygenated haemoglobin in venous blood. Thus, increase in total haemoglobin and oxygenated haemoglobin with a decrease in deoxygenated haemoglobin is expected to be observed in activated areas during NIRS measurement. NIRS uses multiple pairs or channels of light sources and light detectors operating at two or more discrete wavelengths. The light source is usually a light emitting diode (LED). The optical parameter measured is attenuation of light intensity due to absorption by the intermediate tissue. The concentration changes of oxygenated haemoglobin and deoxygenated haemoglobin are computed from the changes in the light intensity at different wavelengths, using the modified Beer–Lambert equation (Villringer & Obrig, 2002). The advantage of the NIRS approach is its simplicity, flexibility and high signal to noise ratio. The depth of brain tissue which can be measured is typically 1–3 cm.

Subcortical regions are not accessible with the present instrumentation.

Sitaram et al. 2007a) published the fist controlled evaluation of a NIRS BCI. Using motor imagery with a 20 channel NIRS system over sensorimotor cortex they reported 89% correct classification of right and left hand imagery without any training and the use of a hidden Markov model as a classifier.

At present the first clinical application with training of children with attention deficit disorders following the approach by Strehl et al. (2006) is underway.

Transcranial magnetic stimulation, direct transcranial cortical DC stimulation (tDCS) and neurochips in BCI training

Transcranial magnetic stimulation (TMS) and tDCS were shown to affect cortical plasticity and learning in healthy persons as well as stroke patients. The combination of these stimulation techniques with BCIs is a largely unexplored field. Only one study (Karim et al. 2004) manipulated operant training of SCP control with anodal and cathodal polarization using tDCS. Anodal polarization before trials with required positivity substantially increased performance of SCP control by augmenting baseline negativity (depolarization) and thus increasing the amplitude of the self-produced positivity afterwards.

A highly relevant contribution for BCI research and application was recently published by Jackson et al. (2006). Based on a strict Hebbian concept, monkeys received intracranial electric stimulation of the motor hand area contingent upon a specific spike pattern related to a torque movement of the hand. The stimulated area, however, was distant from the recording area and was responsible for hand movements in the opposite direction. Repetitive electric stimulation instigated automatically at that distant region in close time proximity to a specific spike pattern (50 ms later) led to a conditioned reproduction of the spike pattern at the recording sites and a consequent change in the hand movement. This study demonstrates that the brain can be induced to reroute neural information responsible for a particular behaviour if Hebbian connectivity between two arbitrary brain areas is built through artificial but time-contingent stimulation.

Translated into the BCI situation, electric or magnetic stimulation of specific brain areas contingent upon a classified brain-signal pattern (EEG, MEG, BOLD, etc.) may circumvent interrupted neuronal pathways or structures with forced reorganization using this paired electrical stimulation technique. Stroke and other disorders caused by brain lesions and local degeneration may profit from the ‘neurochip’.