Brain-Computer Interfaces Using Sensorimotor Rhythms: Current State and Future Perspectives

Abstract

Many studies over the past two decades have shown that people can use brain signals to convey their intent to a computer using brain-computer interfaces (BCIs). BCI systems extract specific features of brain activity and translate them into control signals that drive an output. Recently, a category of BCIs that are built on the rhythmic activity recorded over the sensorimotor cortex, i.e. the sensorimotor rhythm (SMR), has attracted considerable attention among the BCIs that use noninvasive neural recordings, e.g. electroencephalography (EEG), and have demonstrated the capability of multi-dimensional prosthesis control. This article reviews the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. The characteristic features of SMR from the human brain are described and their underlying neural sources are discussed. The functional components of SMR-based BCI, together with its current clinical applications are reviewed. Lastly, limitations of SMR-BCIs and future outlooks are also discussed.

I. Introduction

Over the last twenty years, neural engineering has emerged as a new field that merges systems neuroscience and engineering and has resulted in neurotechnology to link brain activity with man-made devices. Such technology, called the brain-computer interface (BCI), provides a new output channel for brain signals to communicate or control external devices without using neuromuscular pathways [1-3]. A BCI recognizes the intent of the user through electrophysiological or other signals of the brain. In real time, neural recordings are used to decode ongoing activity and translate into output commands that accomplish the desire of the user. BCI technology holds promise of restoring motor ability or communication to people severely disabled by a wide variety of devastating neuromuscular disorders, and to enhance functions in healthy individuals. Capturing the intention and communicating with or executing the desired device form the basis of brain-controlled interface. Motor intention has been naturally adopted mainly for the purpose of restoring motor control function. In a BCI system, the number of independent control parameters derived from the brain signals, i.e. the degree of freedom (DOF), is a key attribute and to a large extent determines the functions that the BCI can execute. Studies have shown that BCIs can achieve independent multi-dimensional control, which can be utilized in many rehabilitation applications, such as modulating environment [75, 94], communication [25-27, 31], orthosis control [77] and navigation in two-dimensional (2D) [78] or three-dimensional (3D) space [19], etc. (Fig. 1).

Fig. 1.

A schematic diagram of the essential components of a Brain-Computer Interface system.

Various forms of brain signals have been harnessed to fulfill the goal of movement control. Currently the three major recording modalities for BCI are electrophysiological signals acquired over the scalp (electroencephalography (EEG)), over the cortical surface (electrocorticography (ECoG)), and within the brain (single-neuron action potentials (single units) and local field potentials (LFPs)). All of these methods record microvolt-level extracellular potentials generated by neurons in the cortical layers, but they are sampled at different field distances and at different spatial resolutions. Intracortical recordings of single neuron action potentials are of the highest resolution yet represent the most invasive BCI methods since they record electrical activity from electrodes implanted in the parenchyma. Research by systems neurophysiologists studying motor systems has uncovered how kinematic parameters of movement control are encoded in neuronal firing rates [4, 5]. Capitalizing on these neuroscience findings, several groups were able to develop real-time, closed-loop, BCI systems capable of multi-dimensional control [6-8]. Initially these systems were tested on non-human primates [6-8] but electrode arrays have also been implanted in several severely disabled individuals for multi-dimensional control of a computer cursor [9, 10] or a robotic arm [11, 12]. Although invasive BCIs using intracortical recordings (mostly single units) achieve a high level of DOF, there still remain significant and unresolved questions regarding the long-term stability of intracortical electrodes, particularly for recording action potentials from individual neurons [13], which would significantly limit its clinical applications [14]. LFP is usually acquired in an equally invasive way as single units yet does not offer the same high signal resolvability as single units, which is not an optimal solution for real-time BCI systems. In contrast, EEG is noninvasive and has supported many important BCI applications, including two- and three-dimensional BCI control [15-19]. Because EEG records the extracellular field potentials over the scalp, the signals is limited in spatial resolution (at cm level) and frequency range (mostly below 70 Hz) [20], and it is more susceptible to environmental interference and other artifacts like electromyographic (EMG) signals from cranial muscles or electrooculographic (EOG) activity. Nonetheless, EEG is the simplest, safest recording method and perhaps has the most clinical applications. ECoG signals are recorded from electrodes surgically placed on the surface of the cortex and are considered less invasive than intracortical recordings as ECoG electrodes do not penetrate the brain tissue. Compared with EEG, ECoG shares the same electrophysiological sources with EEG, i.e. the underlying field potentials, but are measured at a closer distance to the cortex, yielding a finer spatial resolution on the order of mm as well as the ability to record higher-frequency content in the signal (up to 200 Hz) [21, 22]. Nonetheless, placing ECoG electrodes still requires surgery and issues such as risk of tissue damage and infection and long-term recording stability arise. To date the ECoG-based BCI systems are only tested in patients with intractable epilepsy who are candidates for invasive monitoring to localize their seizure foci and to identify eloquent cortex [22-24].

In these studies, a key phenomenon observed across different recording modalities is that the neurophysiological rhythmic activities recorded over the sensorimotor cortex are modulated by actual movement, motor intention, or motor imagery. The modulation manifests as decreases in the alpha (8-13 Hz, also known as mu rhythm) and beta (14-26 Hz) frequency bands accompanied by increase in the gamma frequency band (>30 Hz), which is repeatedly observed in EEG, ECoG, LFP as well as electromagnetic recordings (magnetoencephalography (MEG)). Such rhythmic brain activities measured by EEG or MEG over the sensorimotor cortex are collectively referred to as the sensorimotor rhythms (SMR), which is in general applicable to the counterparts in ECoG and LFP. Motor intention or motor imagery can be decoded from the sensorimotor rhythms, which forms the basis of neural control in SMR-based BCIs. Studies have demonstrated that people can learn to increase and decrease the amplitude of sensorimotor rhythm using mental strategy of motor imagery, and thereby control physical or virtual devices (e.g. [15, 17, 22, 23, 25-27]).

SMR-based BCIs have been widely investigated in healthy human subjects, as well as in people with amyotrophic lateral sclerosis (ALS) [28] and in those with severe CNS damage from spinal cord injuries [15] and stroke resulting in substantial deficits in communication and motor function [29]. By modulating their SMR signals, users were able to acquire 2D or 3D movement control [15-19], with DOF and performance comparable to studies using intracortical single units recordings [9, 10]. An example of fine control by EEG sensorimotor rhythms was demonstrated by He’s research group who showed that human subjects could fly a model helicopter to any point in a 3D space using control of EEG signals recorded from scalp [17-19]. In these studies, subjects were given the opportunity for continuous 2D or 3D control to fully explore an unconstrained space; they learned to fly the helicopter to any target point in the 3D space. Sensorimotor rhythms are also being utilized in ECoG-based BCIs. So far the report of highest DOF by ECoG-based BCI (2D control) and the only report of 2D control was built based on using SMR from the upper arm region of motor cortex for one dimension and SMR from the hand region for the other dimension [23].

Up to date, in BCIs using EEG or ECoG signals, sensorimotor rhythms offer the highest level of control in terms of DOF among all other signal components, such as stimuli evoked potentials and slow cortical potentials. In addition, SMR are readily detectable in healthy as well as disabled individuals by neuromuscular diseases or injuries, including spinal-cord injury, amyotrophic lateral sclerosis (ALS), and stroke. Over the past decade, SMR-based BCI has been one of the fastest growing areas. In this paper, we discuss the current state and future perspectives of SMR-based BCI and its clinical applications, in particular focusing on the EEG SMR. We will describe the characteristic features of SMR from the human brain and discuss the electrophysiological sources of SMR. We will also describe the functional components of SMR-based BCI, review the current clinical applications, and identify potential users and potential applications. Lastly, we will discuss current limitations and expectations for the future.

II. Characteristic Features of Sensorimotor Rhythms

A. Time-frequency Modulation of Sensorimotor Rhythms

EEG as well as ECoG measures the extracellular field potentials associated with neural activity and have been used to study the basic mechanisms of cortical process. They exhibit endogenous oscillation that is widespread across the entire brain, and have been found to be related to important aspects of motor function, sensory perception, or cognition. Task-related modulation in sensorimotor rhythms is usually manifested as amplitude (or power) decrease in the low-frequency components (alpha/beta band) (also known as event-related desynchronization (ERD) [30]. In contrast, an amplitude increase in a frequency band is known as event-related synchronization (ERS) [30].

Planning and execution of movement has been found to lead to predictable decreases in the alpha and beta frequency bands [30]. Also, many studies have demonstrated that motor imagery can cause ERD (and often ERS) in primary sensorimotor areas [30-34]. Discriminant information can be extracted from the spatial patterns of sensorimotor rhythmic modulations [33, 35-38]. More importantly, the modulation of alpha- and beta-band SMR have been found to be organized in a somatotopic manner. Source imaging studies of SMR [39, 40] have revealed that movement or motor imagery of different body parts were associated with decrease in SMR from regions along the primary sensorimotor cortex corresponding to different body parts, known as the Homunculus. Such characteristic changes in EEG sensorimotor rhythms can be used to classify brain states relating to the planning/imagining of different types of limb movement, which forms the basis of neural control in SMR-based BCIs [27, 41].

The findings of EEG SMR modulations have basically been corroborated in ECoG studies. Motor actions as well as motor imagery, are usually associated with a decrease in alpha- and beta-band spectral amplitude across the corresponding area of motor cortex [42]. In addition, as ECoG more readily detects activity in the higher frequency band, many ECoG studies have demonstrated that spatially focused gamma activity (40 – 200 Hz) correlates closely with specific aspects of motor, language, or cognitive function [43, 44]. In particular, increase of amplitude in the gamma-band activity has been found to accompany the decrease in alpha and beta frequency bands during motor execution, motor imagery, and imagery-based online feedback [34]. Notably, the gamma-band SMR tends to be more spatially focused than the low-frequency activities [34, 44].

In addition to modulation in the alpha/beta/gamma frequency bands, SMR signals in the very low frequency band (<1 Hz) have also been explored for studying the kinematic information. Researchers have been able to decode 2D and 3D velocity of hand movement from the very low frequency sensorimotor rhythms [45-47]. Online BCI systems based on such slow SMR have been demonstrated to allow users to acquire 2D movement control with a relatively short training time [48]. Interestingly, similar counterpart of very low frequency oscillations in ECoG, referred to as local motor potentials, have also been found to show close correlation with hand and finger kinematics [49].

In addition to amplitude and time domain features, several recent studies have also begun to explore the cross-frequency coupling in SMR. It has been demonstrated that gamma activity is modulated by the phase of low-frequency brain rhythms (e.g., in the theta, mu, and beta ranges) [50, 51]. To what extent these coupling mechanisms could be useful in a BCI context remains unclear.

In summary, SMR can detect several physiological processes and their interactions within and across sites. SMR in a wide range of frequency bands contains substantial and complementary information on motor function. Kinematic information of motor control as well as motor imagery is found to be distributedly encoded in SMR from the cerebral cortex. Through decoding, such functional neurophysiological information can be and has in part been used for BCI control.

B. Electrophysiological Sources of Sensorimotor Rhythms

The endogenous oscillations in the lower frequencies occurring continually during idling or resting state are thought to be generated by complex thalamocortical networks of neurons that create feedback loops [52]. Alpha rhythms recorded in the human brain at resting state have been found to be correlated with blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) signals at the primary visual cortex as well as the thalamus [53, 54], corroborating the thalamocortical origins for the endogenous rhythms. Furthermore, source analysis studies have shown that the electrophysiological sources for the idling or resting-state alpha rhythms are located in the primary visual cortex [55, 56].

During non-idling periods, however, these oscillations change in amplitude and/or frequency, and these changes are evident in the EEG or MEG. The sources of sensorimotor rhythms induced by movements or imagined movements of various body parts have been located in the primary sensorimotor cortex in a somatotopic manner [39]. They are also found to be co-localized with fMRI activations following the somatotopic organization, as shown in Fig. 2(c). In addition, the amplitudes of SMR decreases are found proportional to the amplitudes of positive BOLD fMRI responses [40, 57] but not to the negative BOLD responses [40]. Furthermore, the temporal modulations of sensorimotor rhythms are correlated with the BOLD fMRI, primarily in the contralateral sensorimotor cortex [40, 58]. These findings suggest that there is tight coupling between SMR and BOLD fMRI, suggesting the spectral-spatial encoding of SMR over the cortex.

Fig. 2.

Time, frequen cy, and spatial characteristics of sensorimotor rhythms. (a) Steps of feature extraction for sensorimotor rhythms [33]. It is difficult to detect a coherent component in the raw EEG signal depicted in the top frame because there is a lot of noise in the signal. The second frame shows the signal after being processed through a surface Laplacian filter that focuses on EEG components in a specific spatial frequency range. As shown in the third frame, the signal is then band-pass filtered to isolate the frequencies of interest. The features become evident in the fourth frame as they are extracted by using a grand averaging method over a fixed bin or window size (b) An example of the time-frequency representation of SMR dynamics [33]. (c) Source localization for decreases of alpha (blue ball) and beta rhythms (green ball) induced by motor imagery of right hand, co-localized with BOLD fMRI activations (white arrow) [39]. Figures are adapted with permission.

In contrast to the lower frequencies, gamma activity has been shown to be strongly correlated to the firing rate of individual neurons and has also been closely linked to the BOLD signals detected by fMRI, which has been observed in LFPs [59], ECoG [60], MEG [61], and EEG [62]. Many studies have co-localized the simultaneous decrease in alpha/beta frequency bands and increase in gamma band induced by movement and motor imagery [34, 42-44]. These findings suggest that common neuronal events, likely neural activation, underlie the task-related hemodynamic response and multiband SMR responses. Nonetheless, the exact mechanism of SMR modulations remains elusive. For example, it is unclear to what extent the amplitude of SMR modulations depends on the respective contributions of neuronal firing rates and synaptic potentials, and of their relative phases. Better understanding of the mechanism of sensorimotor rhythms, including their generation and modulation, would provide more informative features of SMR, which in turn can be used for decoding in SMR-based BCI.

III. Current Bcis Using Sensorimotor Rhythms

A. Signal Acquisition

EEG signals can be noninvasively measured using commercially available amplifier systems with electrodes positioned in elastic caps according to international standards. More recently, the more user-friendly electrodes, such as dry, capacitance-based electrodes, have also been developed which could reduce the demands on BCI-users for long term usability [63, 64]. For targeting low frequency sensorimotor rhythms other than gamma band activity, a typical sampling frequency of 200 Hz or 100 Hz would suit the needs. Otherwise, a higher sampling frequency would be required to satisfy Shannon’s sampling theorem if the gamma band activity is intended.

B. Feature Extraction

In order to define the neural modulation of SMR, the EEG signals are usually subjected to time/frequency analysis. Frequency-based features have been widely used in SMR-based BCIs because of their ease of application, computational efficiency and straightforward interpretation. Because these features do not provide time domain information, they are not sensitive to the non-stationary nature of EEG SMR. Thus, mixed time-frequency representations (TFRs) that map a one-dimensional (1D) signal into a 2D function of time and frequency are used to extract the time-varying spectral content of the signals [65, 68]. Parametric approaches are also commonly used to estimate the time/frequency features, such as autoregressive (AR) modeling for stationary signals and adaptive autoregressive modeling for non-stationary signals [85, 91], which are widely implemented in online BCI systems due to their computational efficiency [120]. However, it is worth noting that such parametric modeling approaches usually require pre-determined parameters, such as the model order, which can influence BCI performance [91].

In order to extract discriminant information in SMR modulations, defining features by spatial location is as important as defining them by temporal/spectral characteristics, as the spatial resolution and specificity of EEG/MEG signal is relatively low and the acquired signal usually reflects activity in large regions of the brain. Thus, in order to optimize the spatial information, the channels used for BCI control are usually a selected subset of a few channels. These can be selected by comparing discriminative features [17], by subspace decomposition method [121, 123], or based on a priori knowledge of the functional organization of the relevant cortical area(s) [27, 39].

In addition, methods are developed to integrate the spatial and time/frequency features for quantifying the SMR modulations. He and his colleagues have developed methods to extract the so-called time-frequency synthesized spatial patterns [33, 35, 36], as shown in Fig. 2(b). In these methods, the EEG signals are decomposed into a series of frequency bands, and the instantaneous power is represented by the envelope of oscillatory activity, which forms the spatial patterns for a given electrode montage at a time-frequency grid. Time-frequency weights determined by training process are used to synthesize the contributions from the time-frequency domains. In this space-time-frequency synthesis approach, individual differences are accommodated; it does not contain a priori subject-dependent parameters, and is computationally efficient, thus making it suitable for robust online classification.

Recently electrophysiological source imaging methods have also been proposed as a spatial deconvolution approach to extracting spatial information about the features used in a BCI [27, 65-67]. He and colleagues proposed to use such EEG-based source signals to classify motor imagery states for BCI purposes [68]. Several groups have reported promising results from source analyses as compared to results from the scalp EEG data [27, 57, 66, 67, 69-71]. The use of source estimation in BCI applications involves increased computational cost due to the need to solve the inverse problem. On the other hand, such source analysis transforms signals from sensor space back to source space with improved spatial resolution and specificity, and may lead to enhanced performance for SMR based BCI. Numerous studies on EEG source imaging have demonstrated substantially enhanced spatial resolution and resolvability of brain processes in both healthy subjects and subjects with brain disorders (for a review, see [72, 73]). Based on the co-localization between sensorimotor rhythms and fMRI activations, information from complementary imaging techniques such as fMRI can help determine potential target areas for a specific subject [39]. FMRI measurement of the BOLD response has facilitated determination of cortical areas useful for recording of brain activity and has also been shown to provide reliable BCI control across several cortical areas using different cognitive tasks [74].

C. Feature Translation and Effector

Translation techniques are algorithms developed with the goal of converting the input features (independent variable) into device control commands (dependent variables) that achieve the user’s intent [75]. Ideally, the translation algorithm will convert the chosen features into output commands that achieve the user’s intent accurately and reliably. Furthermore, an effective translation algorithm will adapt so as to adjust for spontaneous changes in the features and will also encourage and facilitate the user’s acquisition of better control over the features.

Thus far there are two major types of translation: continuous and discrete. In continuous feature translation, consecutive output commands are generated continually based on the features. Examples of this translation are the kinematic parameters (e.g. arm position, velocity and etc.) that control a prosthetic arm. The features are usually derived from short-time windowed signals and are then continuously fed into the translation algorithm so that dynamic outcomes are obtained for BCI control [120]. Continuous translation allows the users to adjust their strategies in the course of control, which can result in very fine movement control but can also be more mentally demanding on the users. Discrete feature translation produces periodic commands at fixed intervals. An example of this type of translation is a BCI that classifies the mental states of various types of motor imageries [41]. Thus, it is particularly suited for applications such as word-processing which requires discrete selections, and less suited for applications such as multi-dimensional robotic arm control, which is best implemented by a continuous series of output commands.

The outputs of translation are commands to operate an external device. As illustration in Fig. 1, the output might be used to operate a spelling program on a computer screen through letter selection [76], to move a cursor on a computer screen [25-27, 31], to manipulate a robotic arm [77], to drive a wheelchair [78], to control a functional electrical stimulation device [79], or even to control a flying robot in the three-dimensional physical space [19]. At present, the most commonly used output device is the computer screen, and it is used for communication.

D. Sensorimotor-rhythm-based BCI Systems

Over the past decades, the SMR-based BCI applications have come a long way from the very beginning of a 1D ‘Ping-Pong’-like game [25, 80] to the state-of-the-art, a computer cursor [16] or virtual helicopter [17, 18] in the 3D virtual world, or even an unmanned helicopter in the real world [19], all controlled by thoughts alone (Fig. 3).

Fig. 3.

Examples of SMR-based BCI that can achieve 1D (a), 2D (b), 3D (c) control of cursor movement, or navigate in 3D space (d), adapted from [16,19] with permission.

Early work by Pfurtscheller and coworkers has developed BCI systems that used mu-rhythm EEG recordings measured over sensorimotor cortex [77, 80-84]. The raw EEG signals were filtered to yield the mu band (8-12 Hz) and then squared to estimate the instantaneous mu power. Alternatively, the power modulation can be extracted using adaptive autoregressive models [85]. The spectral [86] and spatial features [87, 88] of SMR have been extensively studied, and can be used for classifying the states of motor imageries, e.g. left hand vs. right hand, using various pattern classification techniques [89, 122]. The outputs of classification can be quantified and translated into the executive functions of BCI applications. More degrees of independent control can be achieved by classifying multiple mental states of imageries [41, 90].

Another approach of using the sensorimotor rhythms is to allow continuous control through linear combination of sensorimotor rhythms, instead of discrete pattern classifications. Wolpaw and coworkers developed BCI system that allows users to control to move a computer cursor in 1D [25], 2D [15, 26, 91], or 3D [16], and many other applications [92-94] based on linear combination of sensorimotor rhythms. The EEG is recorded as the users actively controlled mu and/or beta rhythm power (amplitude squared) at one or several specific electrode locations over sensorimotor cortex. The EEG power spectra are calculated by an autoregressive method to generate the feature vector [15, 91, 95]. This methodology provides multi-dimensional control that is comparable in speed and accuracy to that achieved to date in humans with microelectrodes implanted in cortex [15, 16].

More recently, highly dexterous control by SMR-based BCIs for continuous navigation in a virtual or real three-dimensional world has been demonstrated by He et al. [17-19]. Control signals were derived from motor imagery tasks and intelligent control strategies were used to improve the performance of navigation. By using a constant forward flying velocity, three-dimensional navigation was reduced to two-dimensional navigation, which allowed human subjects to fly a virtual helicopter to any point in the three-dimensional space [17]. Further studies have enabled human subjects to perform fast, accurate and continuous control of a virtual helicopter in three-dimensional space [17, 18] or quadcopter in a three-dimensional physical space [19], as shown in Fig. 3(d). In these BCI systems, the helicopter’s forward-backward translation and elevation controls were actuated through the modulation of sensorimotor rhythms that were converted to forces applied to the helicopter at every simulation time step, and the helicopter’s angle of left or right rotation was linearly mapped, with higher resolution, from sensorimotor rhythms associated with other motor imaginations. These different resolutions of control allow for interplay between general intent actuation and fine control as is seen in the gross and fine movements of the arm and hand. Subjects controlled the helicopter with the goal of flying through rings (targets) randomly positioned and oriented in a three-dimensional space. Such work suggests the potential of noninvasive EEG based BCI systems to accomplish complex control in freely exploring and interacting with the world, which is a crucial element of autonomy that is lost in the context of neurodegenerative disease.

E. Clinical Applications

The most important clinical application for SMR-based BCI is to restore or replace the lost motor function [1-3]. It has been demonstrated in patients disabled by injuries or diseases. Thus, via the use of BCI, someone who cannot speak could use a BCI to spell words that are then spoken by a speech synthesizer. Or one who has lost limb control could use a BCI to operate a powered wheelchair. Someone with a spinal cord injury whose arms and hands are paralyzed could use a BCI to control stimulation of the paralyzed muscles with implanted electrodes so that the muscles move the limbs. Or one who has lost bladder function from multiple sclerosis could use a BCI to stimulate the peripheral nerves controlling the bladder so as to produce urination.

Other than the application of restoring or replacing movement related functions, BCI, specifically SMR-based BCI, has received increasing attention for rehabilitation purpose by improving natural output of central neural system. For example, a person whose arm movements have been compromised by a stroke damaging sensorimotor cortex might employ a BCI that measures signals from the damaged areas and then excites muscles or controls an orthosis that improves arm movement. A recent pilot study using virtual hands showed promise in stroke patients for rehabilitation [117]. Because this BCI application enables the production of more normal movements, its continued use might induce activity-dependent CNS plasticity that improves the natural CNS output and thus helps to restore more normal arm control.

Many studies have shown that training for and using BCIs can lead to changes in neural activity that facilitate use of prosthetic devices, especially when combined with functional electric stimulation (FES) [96, 97]. Such learning-related changes are especially important for people with brain injuries, such as those who have suffered strokes. In a study using MEG recordings, patients with chronic hand hemiplegia after stroke successfully learned to use motor imagery to control their sensorimotor rhythms, and they were able to use a BCI to control an orthotic device that opened and closed their paralyzed hands [29]. Over a period of three weeks, subjects’ performances steadily improved as they learned to use the device. Comparison between the early and late training stages revealed enhanced sensorimotor rhythms in the ipsilesional hemisphere, which was the hemisphere used to control the device. Several randomized controlled studies have indicated that assisting movement with FES coupled to BCI use can substantially improve upper-limb function in individuals who have been mildly to moderately [98] or severely [99] impaired by stroke. Studies with both invasive and noninvasive BCIs also indicate that learning-related changes can occur over days to months [100]. Interestingly, once users have learned to operate a neuroprosthesis with a BCI they retain this skill months later without intervening use [15], suggesting a long-term learning-related change in neural circuits. Thus, BCIs might be used to help actually restore motor function by promoting beneficial neuroplasticity in neuromuscular pathways.

It is worthwhile to point out that other than clinical applications, there are also nonmedical applications of SMR-based BCIs. For example, the cursor movement [15], virtual helicopter control [17,18] and real helicopter control [19] are not directly related to a clinical application, albeit with potential for clinical application. Other applications include those for entertainment purpose [119], which however, is not the focus of the current review.

IV. Important Questions and Areas for Future Research

A. Towards a Kinematic BCI

Motor imagery has been the most widely employed mental strategy to solicit sensorimotor rhythm modulation in SMR-based BCI. Many noninvasive BCI systems using EEG or MEG signal have been built based on classification of different mental states. In these systems, typically imagination of a certain body movement, e.g., imagination of moving left hand or right hand, corresponds to one mental state that will be translated into one direction of control, and four independent mental states are generally required for full two-dimensional control, which is also utilized in BCI systems using ECoG [22, 23] and in some studies using intracranial recordings in human brain [9]. Therefore, this strategy requires users to develop the skill to maintain and manipulate various mental states to enable the control throughout each attempt (usually more than a few seconds). The mental load of maintaining and manipulating various motor imagery can be quite demanding, especially in disabled users [77], and a substantial period of training is typically needed for users to develop such kind of skill to control their sensorimotor rhythms [15, 100].

One approach towards addressing this challenge for SMR-based BCI is to build a BCI by decoding kinematic movement parameters, which is commonly utilized in BCI/BMI systems using intracranial recordings. Notably different from many SMR-based BCIs, BCI/BMI studies using single unit activity from intracranial recordings have been focusing on decoding kinematic parameters associated with motor control and directly apply those for prosthetic manipulation. Kinematic parameters that have been decoded include the position, direction, speed, and acceleration of the arm or wrist movements [101]. Many of these parameters have been implemented for real-time neural decoding to achieve multi-dimensional, dexterous control [6-8].

More recently, capitalizing by the findings of systems neurophysiology, investigators began to examine how the kinematic information related to movement or motor imagery are represented from non-spiking neural recordings, including LFP [102], ECoG [49], EEG [46, 57] and MEG [46, 47, 103]. Several studies have reported success in decoding information about the (imagined) movement direction and speed from the spatiotemporal profiles of EEG signals [40, 46, 47, 57, 103]. Interestingly, linear models are commonly utilized and have succeeded in explaining the relationship between kinematic parameters and electrophysiological features. For example, studies have revealed that the speeding of arm movement can be modeled as a continuous gain factor in the linear encoding model [8, 104]. Yuan et al. has also found a consistent speed encoding model in the EEG sensorimotor rhythms where the speed acts as multiplicative gain factor. Although the type of hand movement was different (clenching vs. reaching), the study suggested that speed can be a common gain factor to the cortical recordings at both microscale and macroscale levels across various hand tasks, even including motor imageries. Such investigations focusing on kinematic decoding in sensorimotor rhythm can lead to the future of kinematic, finely-controlled, multi-dimensional and fast-attained BCI control [18-19, 48, 57].

B. Towards a Versatile BCI

As discussed above, most previous SMR-based BCI studies have been focused on motor imagery of large body part, e.g. upper or lower limb. In order to further advance the SMR-based BCIs, recent studies have begun to explore whether or how complicated dexterous movement, such as individual finger movement, multiple hand postures/synergies, can be decoded from sensorimotor rhythms, other than from neural spiking activity. Studies using single unit recordings [105, 124] or functional MRI [106] have shown that while a relative large portion of the sensorimotor cortex is associated with dexterous movement of finger or joint, and that organization of such fine movement is highly spatial specific and tends to be mixed and overlapping. This is encouraging for researchers to attempt decoding fine movements of the finger/hand in more macroscale neural recordings, because finger/hand representations are more spread out in the cortex than, for example, upper arm muscles, even though the signal resolvability would be compromised. Indeed, several studies have achieved notable success in decoding individual movement based on ECoG recordings. Interestingly, Miller et al. showed that the most discriminative feature for individual finger encoding is the broadband modulation of sensorimotor rhythms [109], which corroborates another finding that the broadband modulation of sensorimotor rhythm in LFP is tightly coupled to the underlying neuronal action potentials [118]. Meanwhile, studies using EEG have also reported certain success in decoding individual fingers, and even imagined individual finger movement [107, 108], from the scalp recorded sensorimotor rhythms. In their studies phenomena of broadband power increase and low-frequency-band power decrease were observed in EEG. These movement-related spectral structures and their changes caused by finger movements in EEG are consistent with observations from ECoG recordings [108-110]. The average decoding accuracy of 77.11% was obtained from EEG in classifying each pair of fingers [108], whereas the average decoding accuracy using ECoG data was 91.28% [108]. These findings again demonstrate the common electrophysiological phenomenon across different recording modalities and suggest their common sources for the sensorimotor rhythms. These findings also suggest the future generation of a more versatile SMR-based BCI that can achieve complex, dexterous prosthetic control.

C. Towards a Hybrid BCI

As mentioned above, it takes substantial training and effort for users to develop the SMR skill for multi-dimensional control, which may discourages the users by fatigue and mental loads, especially in the disabled users. Nonetheless, it has been shown that it is relative easy and robust for many users to develop reasonable 1D control by motor imagery with relatively little training [100], which would offer the advantage of SMR-based control, such as continuous, accurate and can be of both direction and speed control. Such advantage makes SMR-based BCI especially suitable for candidate of a hybrid BCI [111], which is composed of more than one BCI. Hybrid BCIs can either process their inputs simultaneously, or operate two systems sequentially, where the first system can act as a “brain switch”. Hybrid BCIs are proposed to combine the advantage of individual component BCI and therefore offer joint and perhaps better usability. So far many of the reported hybrid BCIs have at least one of them is the SMR-based BCI. By combing the SMR-BCI with a BCI based on another type of brain signal, the hybrid BCI is reported to offer the highest classification accuracy than any of its individual component alone [111]. Moreover, a multi-dimensional (2D) BCI can be achieved by combining a 1D SMR-based BCI with another BCI based on SSVEP [112] or P300 [113], with comparable performance to that reported in a purely SMR-based 2D control [15]. Importantly, since only one dimension of control is required for SMR-BCI and the other is based on another type of EEG signal that requires little training for modulation, the total training is substantially reduced compared to a SMR-based 2D BCI. This avenue of research in hybrid BCI would greatly extend the possibility of clinical and practical usage for SMR-based BCI systems.

D. Learning and Rehabilitation

It is now widely recognized that training is essential for using sensorimotor rhythms based BCI systems and substantial training can lead to improvement in the skill of modulating sensorimotor rhythms [1-3]. However, there are still important questions about the basic mechanism of SMR learning. Little is known about how the brain instantiates SMR-based BCI control and the accompanying plastic changes in motor cortical areas. Answers to the question of SMR learning will also bear significant importance to further the clinical application of SMR-BCI in rehabilitation.

Notable progress has been made recently in investigating the mechanism of learning in SMR-BCI, by using multimodal imaging approach. Pichiorri et al. [114] has combined transcranial magnetic stimulation (TMS) mapping and EEG connectivity imaging to assess if and how SMR-based BCI training would induce persistent functional changes in motor cortex. A group of naïve participants learned to use SMR-based BCI in a series of sessions and TMS mapping was applied before and after their training. When TMS was applied, peak amplitude and volume of the motor evoked potentials recorded from the opponens pollicis muscle were significantly higher only in those subjects who develop a MI strategy based on imagination of hand grasping to successfully control a computer cursor. Their results demonstrated that SMR-based BCI training led to a significant increase in motor cortical excitability. Their findings are also corroborated in a recent study that combined SMR-BCI and TMS in a closed-loop stimulation design [115], in which TMS was applied to the right hand M1 when the motor-imagery-associated decrease of SMR magnitude exceeded predetermined thresholds during online BCI feedback. Their results showed that the large ERD during wrist motor imagery was associated with significantly increased MEP amplitudes. These studies provide electrophysiological evidence that ERD magnitude during motor imagery represents M1 excitability and SMR learning enhances the excitability. The finding that a motor imagery task involving ERD may induce changes in corticospinal excitability similar to changes accompanying actual movements has important clinical implications. It suggests the important merit of motor imagery and motor-imagery-based BCI in rehabilitation, e.g. in the stroke patients. In this regard, a recent study compared the rehabilitation effects of upper-extremity robot-assisted rehabilitation versus an electroencephalography-based brain computer interface setup with motor imagery (i.e., SMR-based BCI) in hemiparetic patients after stroke. The pre- and post-treatment assessments including Fugl-Meyer upper-extremity motor score and resting-state fMRI showed that patients with SMR-BCI rehabilitation obtained as good performance (even numerically higher) as those with the traditional physical robot-assisted rehabilitation. After BCI training, increased functional connectivity were observed in the supplementary motor area, the contralesional and ipsilesional motor cortex, and parts of the visuospatial system with mostly association cortex regions and the cerebellum, which were correlated with individual upper-extremity function improvement. These network changes corroborates with neuroimaging observations of the SMR during online feedback [27, 34]. Such initial evidence strongly encourages the usage of SMR-BCI in clinical rehabilitation applications [116].

V. Summary

Among various strategies for EEG-based brain-computer interface (BCI), sensorimotor rhythms have thus far offered control of the highest degrees of freedom and demonstrated a versatile spectrum of useful applications. Future research are needed in areas including 1) developing better methods to characterize the spatiotemporal dynamics of sensorimotor rhythm (SMR) modulations; 2) decoding more information from SMR based on better understanding of the mechanisms of SMR modulations; and 3) developing BCI applications that provide multi-dimensional and high-performance neuro-prosthetic control to allow individuals in needs to perform activities of daily living.