Progress in Brain Computer Interface: Challenges and Opportunities

Abstract

Brain computer interfaces (BCI) provide a direct communication link between the brain and a computer or other external devices. They offer an extended degree of freedom either by strengthening or by substituting human peripheral working capacity and have potential applications in various fields such as rehabilitation, affective computing, robotics, gaming, and neuroscience. Significant research efforts on a global scale have delivered common platforms for technology standardization and help tackle highly complex and non-linear brain dynamics and related feature extraction and classification challenges. Time-variant psycho-neurophysiological fluctuations and their impact on brain signals impose another challenge for BCI researchers to transform the technology from laboratory experiments to plug-and-play daily life. This review summarizes state-of-the-art progress in the BCI field over the last decades and highlights critical challenges.

1. Introduction

The brain computer interface (BCI) is a direct and sometimes bidirectional communication tie-up between the brain and a computer or an external device, which involves no muscular stimulation. It has shown promise for rehabilitating subjects with motor impairments as well as for augmenting human working capacity either physically or cognitively (Lebedev and Nicolelis, 2017; Saha and Baumert, 2020). BCI was historically envisioned as a potential technology for augmenting/replacing existing neural rehabilitations or serving assistive devices controlled directly by the brain (Vidal, 1973; Birbaumer et al., 1999; Alcaide-Aguirre et al., 2017; Shahriari et al., 2019). The first systematic attempt to implement an electroencephalogram (EEG)-based BCI was made by J. J. Vidal in 1973, who recorded the evoked electrical activity of the cerebral cortex from the intact skull using EEG (Vidal, 1973), a non-invasive technique first studied in humans invented by Berger (1929). Another early endeavor to establish direct communication between a computer and the brain of people with severe motor impairments had utilized P300, an event related brain potential (Farwell and Donchin, 1988). As an alternative to conventional therapeutic rehabilitation for motor impairments, BCI technology helps to artificially augment or re-excite synaptic plasticity in affected neural circuits. By exploiting undamaged cognitive and emotional functions, BCI aims at re-establishing the link between the brain and an impaired peripheral site (Vansteensel et al., 2016). However, the research applications of BCI technology evolved significantly over the years, including brain fingerprinting for lie detection (Farwell et al., 2014), detecting drowsiness for improving human working performances (Aricò et al., 2016; Wei et al., 2018), estimating reaction time (Wu et al., 2017b), controlling virtual reality (Vourvopoulos et al., 2019), quadcopters (LaFleur et al., 2013) and video games (Singh et al., 2020), and driving humanoid robots (Choi and Jo, 2013; Spataro et al., 2017). Figure 1 demonstrates the progression of BCI in various application fields since its conception.

Figure 1.

The number of publications over the years: The statistics was based on a search on PubMed in which “brain computer interface” was the search keyword. The publications those were listed until 4th December 2020 have been accounted only. A significant increase in the number of publications in this decade as compared to the last decade implicates the engagement of a greater community in this field and, thus the importance of BCI technology.

According to the Brain/Neural Computer Interaction Horizon 2020 project, an initiative by the European Commission for coordinating BCI research, six major application themes, i.e., restore (e.g., unlocking the completely locked-in), replace (e.g., BCI-controlled neuroprosthesis), enhance (e.g., enhanced user experience in computer games), supplement (e.g., augmented reality glasses), improve (e.g., upper limb rehabilitation after stroke), and research tool (e.g., decoding brain activity with real-time feedback) have been outlined as feasible and promising fields (Brunner et al., 2015). This overview encompasses a wide range of challenges and trends in BCI field. For specialized reviews on particular BCI topics we refer to the recent literature (McFarland et al., 2006; Schwartz et al., 2006; Bashashati et al., 2007; Lotte et al., 2007, 2018; Matthews et al., 2007; Sitaram et al., 2007; Mak and Wolpaw, 2009; Menon et al., 2009; Nicolelis and Lebedev, 2009; Summerer et al., 2009; Vaadia and Birbaumer, 2009; Milan and Carmena, 2010; Min et al., 2010; Clausen, 2011; Krusienski et al., 2011; Liao et al., 2012; Nicolas-Alonso and Gomez-Gil, 2012; Shih et al., 2012; Jebari, 2013; McCullagh et al., 2014; Ahn and Jun, 2015; Jayaram et al., 2016; Lebedev and Nicolelis, 2017; Mudgal et al., 2020; Rashid et al., 2020; Saha and Baumert, 2020).

1.1. Characterization of BCI Systems

BCI systems can be categorized by the way they use the brain: Passive BCI decode unintentional affective/cognitive states of the brain (Zander et al., 2009), while active BCI directly involve the user's voluntary intention-induced brain activity. Reactive BCI use brain waves generated as response to external stimuli. Detecting driver's drowsiness to prevent road accidents is an example of passive BCI (Lin et al., 2008; Gao et al., 2019). BCI systems driven by users' intentional motor imagery (MI) (Marchesotti et al., 2016; Saha et al., 2019a; Saha and Baumert, 2020) and visually evoked P300 produced by external stimulation (Farwell et al., 2014) can be considered active BCI and reactive BCI, respectively.

The modality of signal acquisition has been used to divide systems into invasive and non-invasive BCI (Min et al., 2010; Rosenfeld and Wong, 2017). Non-invasive BCI exploiting EEG are most common, although more recently, functional near infrared spectroscopy (fNIRS) (Matthews et al., 2007), magnetoencephalography (MEG) (Fukuma et al., 2016), functional magnetic resonance imaging (fMRI) (Kaas et al., 2019) and functional transcranial Doppler ultrasonography (Faress and Chau, 2013; Lu et al., 2015; Khalaf et al., 2019) have been exploited. In contrast, invasive intracortical electrodes (Pandarinath et al., 2017) and electrocorticography (ECoG) (Kaiju et al., 2017) have been used, providing a superior signal-to-noise ratio and better localization of brain activity. Table 1 summarizes signal acquisition modalities and their suitability for BCI applications.

Table 1.

A list of neuroimaging techniques and their suitability in brain computer interface (BCI) applications.

Feature	EEG	MEG	ECoG	Intracortical Recording	fMRI	fNIRS	PET
Activity type	Electrical	Magnetic	Electrical	Electrical	Metabolic	Metabolic	Metabolic
Measurement type	Direct	Direct	Direct	Direct	Indirect	Indirect	Indirect
Invasiveness	Non-invasive	Non-invasive	Invasive	Invasive	Non-invasive	Non-invasive	Invasive
Portability	Yes	No	Yes	Yes	No	Yes	No
Temporal resolution	~0.05 s	~0.05 s	~0.003 s	~0.003 s	~1 s	~1 s	1–2 min
Spatial resolution	~10 mm	~5 mm	~1 mm	~0.5 mm (LFP) ~0.1 mm (MUA) ~0.05 mm (SUA)	~1 mm	~5 mm	~4 mm
BCI applicability	Acceptable spatio-temporal resolution with high-density electrodes	Mobility constraint	Unfavorable for healthy BCI users	Unfavorable for healthy BCI users	Slow and mobility constraint	Slow, but mobile and a potential alternative to fMRI	Limited potentiality

EEG, electroencephalography; MEG, magnetoencephalography; ECoG, electrocorticography; fMRI, functional magnetic resonance imaging; fNIRS, functional near infrared spectroscopy; PET, positron emission tomography.

Recent technological advancements allow both the decoding of neural activities and the delivery of external signals into targeted brain areas to induce plasticity, i.e., remodeling of neurosynaptic organization (Lajoie et al., 2017). Plasticity is an inherent characteristic of the brain and peripheral nervous system underpinning BCI-based rehabilitation and other neuroscientific applications. While most of the BCI systems translate brain signals to computer commands, some systems utilize external stimulation modalities such as transcranial magnetic stimulation (Grau et al., 2014; Rao et al., 2014; Schaworonkow et al., 2019) and transcranial direct current stimulation (Baxter et al., 2017) to stimulate specific brain areas. The bidirectional framework of BCI comprises either one brain with feedback modality or two brains. Transcranial direct current stimulation directed by MI-related EEG signals alters the connectivity in sensorimotor networks of healthy individuals (Baxter et al., 2017). Another possible application of bidirectional BCI framework is direct brain-to-brain communication (Grau et al., 2014; Rao et al., 2014). Moreover, some BCI applications require auxiliary modalities, e.g., proprioceptive feedback and functional electrical stimulation driven by brain signals as feedback for augmenting or regaining peripheral motor actions (Darvishi et al., 2017; Bhattacharyya et al., 2019; Bockbrader et al., 2019; Murovec et al., 2020).

1.2. Factors Influencing BCI Performance

For medical applications of BCI, three criteria are essential: (1) a comfortable and convenient signal acquisition device, (2) system validation and dissemination, and (3) reliability and potentiality of BCI (Shih et al., 2012). For the restoration of mobility in patients with motor impairments, invasive intracortical recordings show better BCI performance (Hochberg, 2013) than non-invasive methods such as EEG (Milan and Carmena, 2010). The performance determines how efficiently a patient can perform an impaired motor task or communicate with an external device. Invasive modalities are also suitable for locked-in patients, because the benefits (significantly improved quality-of-life) outweigh the risks associated with implantation (Gilja et al., 2011). A pilot study found no adverse effects pertaining to surgery or tissue reaction at 1 year follow-up (Friehs et al., 2006). Invasive BCI should generally not be considered for neurologically intact people due to risks associated with surgery. However, invasive recordings may enable the utilization of localized inner cortex activities and a better interpretation of surface recordings from a non-invasive modality (Schalk, 2010; Lina et al., 2012).

Many factors influence BCI performance; taking the underlying cortical-subcortical networks into consideration is of crucial importance. For example, MI-induced signals are best recorded from premotor and motor areas, because premotor cortex, primary motor cortex and supplementary motor area along with basal ganglia and thalamus of the subcortical areas are the mostly activated areas during MI (Marchesotti et al., 2017). While EEG can capture premotor and motor area activation (Edelman et al., 2015; Saha et al., 2019a), intracortical electrodes can record signals from basal ganglia and thalamus (Sand et al., 2017).

Several issues can significantly impede BCI performance. Maintaining an acceptable signal-to-noise ratio in non-invasive long-term recordings is critical. Event-induced brain waves or oscillations are dynamic and affected by unstable resting state networks (RSNs) (Mantini et al., 2007). Time-variant psychophysiological (Gonçalves et al., 2006; Zhang et al., 2015; Acqualagna et al., 2016; Saha and Baumert, 2020), neuroanatomical (Kasahara et al., 2015) factors and users' fundamental traits (Ahn and Jun, 2015) cause unreliable estimates of RSNs, causing short and long-term signal variation within and across individuals (Saha and Baumert, 2020). Due to these intrinsic signal variations, BCI systems require subject-specific training, during which subjects attend a calibration session that is tedious and often frustrating. To eliminate subject-specific training, the concept of inter-subject associativity, demonstrated in previous works in case of natural vision (Hasson et al., 2004) and natural music listening (Abrams et al., 2013), could be exploited toward inter-subject operable BCI. Recent studies suggest that inter-subject operable sensorimotor rhythm-based BCI might become feasible for subjects who share common brain dynamics (Saha et al., 2017, 2018, 2019a; Saha and Baumert, 2020). Inter-subject BCI holds promise predominantly for healthy people and in applications such as gaming, drowsiness and lie detection, because rehabilitative BCI must consider the characteristics and severity of individual impairment (Park et al., 2016). Transfer learning can also reduce the effects of session-to-session and subject-to-subject variabilities, by using systems that were trained on data from different people exploiting commonalties and reducing training requirements (Jayaram et al., 2016; Saha et al., 2017, 2018, 2019a; He and Wu, 2019; Wu et al., 2020).

2. Challenges

2.1. Psychophysiological and Neurological Challenges

Emotional and mental processes, neurophysiology associated with cognition and neurological factors, i.e., functions, anatomy, play crucial roles in BCI performance and give rise to significant intra- and inter-individual variability (Saha and Baumert, 2020). Psychological factors such as attention, memory load, fatigue and competing cognitive processes (Gonçalves et al., 2006; Käthner et al., 2014; Calhoun and Adali, 2016) as well as users' basic characteristics such as lifestyle, gender, and age, (Kasahara et al., 2015) influence instantaneous brain dynamics. For example, individuals with lower empathy participate less emotionally in a P300-BCI paradigm and can produce higher amplitudes of P300 waves than subjects with greater empathetic involvement (Kleih and Kübler, 2013). Motivation is also related to P300-BCI performance (Nijboer et al., 2010).

Besides psychological traits, resting state physiological parameters, for example, frequency domain features of resting state heart rate variability are associated with BCI performance (Kaufmann et al., 2012). In addition, the baselines of RSNs are dynamic and modify any cortical signature instantaneously (Mantini et al., 2007). Age alters RSNs and associated cognitive responses (Wang et al., 2016b). Adapting to such time-variant RSNs is more demanding when the effects of RSNs mask event-related cortical responses (Jensen et al., 2011). Moreover, the inherent complexity and diversity in the formation of human brains (Sporns, 2013) that influence the functional neural networks (Honey et al., 2010), construct highly volatile neuronal connectivity over time and across subjects (Honey et al., 2009). An efficient BCI system must be robust to such inherent physiological fluctuations over time to enable more generalized systems (Saha and Baumert, 2020).

Experiments correlating BCI performance with neuroanatomical, neurophysiological and psychological parameters have provided fascinating results: gray matter volume in sensorimotor cortical areas is associated with BCI success (Kasahara et al., 2015). Sensorimotor rhythm-based BCI has implicated that physiological predictors such as spectral entropy and power spectral density, derived from resting state EEG recordings are correlated with BCI performance (Zhang et al., 2015; Acqualagna et al., 2016). Psychological predictors such as attention and motivation, are also associated with sensorimotor rhythm-based BCI performance (Hammer et al., 2012). Corticospinal excitability could be used as another reliable marker for BCI performance (Vasilyev et al., 2017). Taking head anatomy into consideration augments BCI performance (Wronkiewicz et al., 2015; Saha et al., 2019a).

Around 15–30% of individuals are inherently not able to produce brain signals robust enough to operate a BCI (Blankertz et al., 2009; Halder et al., 2019; Cecotti, 2020). Considering neurophysiological phenomena may reduce BCI illiteracy. An adaptive machine learning approach incorporating neurophysiological and psychological traits has been proposed to reduce BCI illiteracy (Vidaurre and Blankertz, 2010). The causes of BCI illiteracy do not exclusively rely on users' ability to produce signals. Sometimes technological limitations may hinder essential features extraction for a successful BCI operation for an individual. For example, measurements of scalp EEG/MEG may not show good task-specific signals due to the folding of the cortex or scalp-to-cortex distance for that individual (Andersen et al., 2020).

Other case-specific investigations on neuro-psycho-physiological parameters contributing to BCI performance are essential. For the rehabilitation of stroke survivors, affected neural circuits, i.e., lesions are to be identified carefully, because brain responses fluctuate according to the spatial location of the stroke lesion (Park et al., 2016). Although current neuroimaging methods are effective in capturing stroke lesion sites, a case-specific BCI design that incorporates residual brain function is required for rehabilitative interventions. Highly individualized design impedes wide dissemination of BCI-driven rehabilitation of neurological conditions.

2.2. Technological Challenges

Event related potential (ERP) (McCane et al., 2015), steady-state visual evoked potential (SSVEP) (Chen et al., 2015; Abu-Alqumsan and Peer, 2016), auditory evoked potential (AEP) (Schreuder et al., 2010), steady-state somatosensory evoked potential (SSSEP) (Muller-Putz et al., 2006; Oxley et al., 2017), and motor imagery (MI) (Marchesotti et al., 2016; Saha et al., 2019a; Saha and Baumert, 2020), have been proposed to detect cognitive signatures although none of the approaches performs well for all BCI applications. For example, ERPs and SSVEPs are target-specific and elicited by external stimuli; however, if ERPs depend on visual stimuli, they cannot be used for communication by locked-in patients with impaired visual processing. In that case, auditory-based ERP (e.g., AEP) could be used if auditory processing remains intact. The SSVEP method provides the highest information transfer rate of a non-invasive EEG-based BCI (Chen et al., 2015; Abu-Alqumsan and Peer, 2016). Limitations of the SSVEP technique include visual fatigue caused by looking at a flickering display for a long time. When using this technique, the control signal could be arbitrary and counter-intuitive, although it might depend mostly on the experimental context. For example, when using a BCI speller based on SSVEP, an individual looks at the letter “A”, which flickers at 10 Hz. It is generally not given importance to any inherent relationship between “A” and 10 Hz, instead the control signal is arbitrary mapped to interface with a computer. An advantage of an MI-based BCI is the use of explicit mapping of task-related brain signals to operate (Saha and Baumert, 2020). However, MI seems too slow for action control, thus they are not suitable for controlling virtual reality environments or videogames (Lécuyer et al., 2008). Recently proposed hybrid BCIs which utilize more than one signature, i.e., SSVEP/ERP (Combaz and Van Hulle, 2015; Yin et al., 2015) and SSVEP/MI (Pfurtscheller et al., 2010; Horki et al., 2011), seem to offer more robust features. Considering asynchronous BCI where the user decides to activate a command when necessary, the performance is still unsatisfactory (Han et al., 2020).

The intrinsic neurophysiological instability of brain dynamics poses critical challenges for making BCI systems efficient. The major components of a BCI system are signal acquisition, signal processing and effector device (Schwartz et al., 2006). Various neuroimaging techniques have been used to explore cortical activities through either electrical or hemodynamic signatures (Min et al., 2010), but none of the methods shows any advantage for a lucrative BCI design meeting the four important criteria: cost efficiency, portability, easy maintenance, and little or no involvement of surgery. EEG-based BCI are relatively more compliant with the abovementioned criteria as compared to other signal acquisition modalities. Tables 2, 3 list a diverse range of BCI applications exploiting EEG. Both invasive and non-invasive signal acquisitions have recently shown that reliable long-term (i.e., for at least several months) use of BCI systems is quite feasible (Saeedi et al., 2016; Sauter-Starce et al., 2019; Shahriari et al., 2019; Oxley et al., 2020).

Table 2.

A summary of sensorimotor rhythms and electroencephalography-based brain computer interface (BCI) studies.

References	Modality	Wave/Task	Data analysis	Context	Application
Romero-Laiseca et al., 2020	EEG	MI	Riemannian geometry, LDA	Motor learning & plasticity induction	Lower limb rehabilitation
Song and Kim, 2019	EEG	MI	Spectral averaging	Visuo-Tactile stimulation	MI enhancement
Saha et al., 2019a	EEG	MI	wMEM	Transfer learning	Inter-subject BCI
Toriyama et al., 2018	EEG	MI, ME	LDA	MI & ME resemblance	Corticospinal excitability
Yu et al., 2018	EEG	MI	CSP, LDA	Assistive technology	Wheelchair navigation
Darvishi et al., 2017	EEG	MI	LF	Motor training by proprioceptive feedback	Upper limb rehabilitation
Park et al., 2016	EEG	MI, ME	ICA, CAR	Variable lesion characteristics	Stroke rehabilitation
Kasahara et al., 2015	EEG	MI	LF, AR	Neuroanatomical predictor	Motor rehabilitation
Edelman et al., 2015	EEG	MI	WMNE	Decoding complex MI	Higher degrees of freedom
Leamy et al., 2014	EEG	ME, MI	Filter bank CSP	Plasticity induction	Stroke rehabilitation
Ramos-Murguialday et al., 2013	EEG	ME, MI	AR	Stroke rehabilitation, assitive technology	Hand orthosis control
LaFleur et al., 2013	EEG	MI	AR	Assistive technology, robotics	Quadcopter control

EEG, electroencephalography; ME, motor execution; MI, motor imagery; wMEM, wavelet-based maximum entropy on the mean; LF, Laplacian filter; AR, autoregressive model; CSP, common spatial pattern; ICA, independent component analysis; CAR, common average reference; WMNE, weighted minimum norm estimate; LDA, linear discriminant analysis.

Table 3.

A summary of non-sensorimotor rhythms and electroencephalography-based brain computer interface (BCI) studies.

References	Modality	Wave\Task	Data analysis	Context	Application
Jin et al., 2020	EEG	P300	LDA	Vibrotactile stimuli	Spinal cord injury rehabilitation
Cecotti, 2020	EEG	SSVEP	CCA	Generalized framework	BCI literacy
Halder et al., 2019	EEG	P300	LDA	Auditory stimuli	BCI literacy
Wei et al., 2018	EEG	Continuous (alert vs. drowsy)	PCA, LDA, SVM	Driving scenario in virtual reality	Drowsiness detection
Guy et al., 2018	EEG	P300	LDA	Amyotrophic lateral sclerosis	Visual P300 speller
Alcaide-Aguirre et al., 2017	EEG	P300	LDA	Cerebral palsy	Cognitive assessment
Waytowich et al., 2016	EEG	P300	Information geometry	Transfer learning	Inter-subject BCI
Norton et al., 2015	EEG	SSVEP, P300	Averaging	Soft, curved electrode systems	Text speller
Botrel et al., 2015	EEG	P300	LDA	Amyotrophic lateral sclerosis	Brain painting
Farwell et al., 2014	EEG	P300	Bootstrapping	Brain fingerprinting	Lie detection
Petrov et al., 2014	EEG	VEP	Naive Bayes	High-density EEG systems	Epilepsy or other BCI applications
Chi et al., 2011	EEG	SSVEP	CCA	Dry and non-contact sensors	Typical BCI application
Iturrate et al., 2009	EEG	P300	LDA	Assistive Technology	Wheelchair control

EEG, electroencephalography; P300, an event-related potential; SSVEP, steady-state visual evoked potential; VEP, visual evoked potential; LDA, linear discriminant analysis; CCA, canonical correlation analysis; PCA, principal component analysis; SVM, support vector machine.

EEG provides relatively poor spatial resolution due to non-invasive scalp recordings compared to fMRI, but finer temporal resolution (Lystad and Pollard, 2009; Min et al., 2010; He et al., 2011; Nicolas-Alonso and Gomez-Gil, 2012). Employing high density EEG mapping increases spatial resolution but results in high computational cost and efforts to maintain a reasonable signal-to-noise ratio across all channels (Chen et al., 2015). Since EEG captures only the electrical field associated cognitive processes, concomitant assessment of blood-oxygen level-dependent (BOLD) activity may improve BCI performance. BOLD activity is typically captured with fMRI (Sitaram et al., 2007), which is not feasible for most BCI applications, due to unmanageable size and cost of the device. fNIRS provides a safe, non-invasive, relatively inexpensive and portable neuroimaging alternative for recording BOLD activity (Matthews et al., 2007). Integrating fNIRS with EEG can significantly enhance classification performances regardless of low information transfer rate caused by inherent delays in hemodynamics (Fazli et al., 2012). A recent study has suggested that fNIRS is unable to adequately offer acceptable performances on its own, but can be combined with EEG to boost the performances (Ge et al., 2017). However, continuous technological advances could promote fNIRS as an exclusive tool for neuroscience research, including the development of BCI (Scholkmann et al., 2014; Naseer and Hong, 2015).

Probing sources in cortico-subcortical networks is another important limitation of scalp-based sensors such as EEG. Reconstructing task-induced networks while resolving the so-called inverse problem imposes a significant challenge. A two-equivalent-dipole model was applied on EEG data to discern the anatomical nature of the MI induced sources and to aid the classification performances (Kamousi et al., 2005). Saha et al. proposed a wavelet-based source localization approach to investigate MI-related sources and their impact on BCI performance (Saha et al., 2019a). The neuronal potentials attenuate through several tissue layers of complex geometry and diverse electrical properties; however, the magnetic permeability in the cerebrospinal fluid, skull, and skin, is consistent (da Silva, 2013). Thus, MEG can capture signal with less distortion than EEG. Although MEG provides better spatiotemporal resolution as compared to EEG, the magnetic field created by the brain is very small, requiring costly, stationary recording equipment (Mellinger et al., 2007; Corsi et al., 2019).

The BCI classifier design has to address two issues (Bashashati et al., 2007; Lotte et al., 2007, 2018). First, the dimensionality of the features set used for estimating the model parameters should be chosen for optimal performance based on the nature of the classifier. Second, the trade-off between bias and variance has to be considered and may involve regularizing the parameter estimation.

Covariate shift occurs when the features extracted from the training differ from those of test data impacting the classification performance (Krusienski et al., 2011). Covariate shift is an important issue requiring the application of adaptive methods for compensating feature space transitions (Jayaram et al., 2016; Saha and Baumert, 2020). The unsupervised subspace learning method enables session-to-session and subject-to-subject information transfers, augmenting BCI performance (Samek et al., 2013; Jayaram et al., 2016; Saha et al., 2018). The common spatial pattern, a supervised method, has been extensively used in EEG-based online and offline BCI settings (Ramoser et al., 2000; Wu et al., 2017a). A common problem with such a data-driven technique is over-fitting of the model parameters based on training sets, causing unreliable prediction on the test data (Sannelli et al., 2016). Recent studies integrated diverse methods into potential transfer learning frameworks for BCI including spatial filters (e.g., common spatial pattern), Riemannian geometry, Euclidean alignment and subspace adaptation and deep learning-based techniques (Barachant et al., 2011; Congedo et al., 2013, 2017; Marathe et al., 2015; Wu et al., 2016, 2017b; Wu et al., 2020; He and Wu, 2019, 2020; Kwon et al., 2019; Zhang and Wu, 2020).

3. Neuroplasticity, Sensors, Signal Processing, Modeling, and Applications

Exploiting neuroplasticity, designing hi-fidelity and customized neural sensors, applying advanced signal processing, and machine learning techniques are the key aspects of an effective BCI design. Tables 2–5 highlight diverse characteristics of BCI components and applications including signal acquisition modality, experimental paradigm, data analysis and pattern recognition, application field, and significance. Notably, there are no specific selection criteria for studies in Tables 2–5 due to the broad spectrum of topics covered in this review; however, they are summarized such that critical advances over the last several years can be appreciated.

Table 5.

A summary of brain computer interface (BCI) studies involving invasive procedures.

References	Modality	Wave\Task	Data analysis	Context	Application
Oxley et al., 2020 Oxley et al., 2017 Oxley et al., 2016	Stentrode	SSSEP	Spectral analysis	Catheter angiography- guided implantation	Minimally invasive BCI: human clinical trial in progress
Sauter-Starce et al., 2019 Mestais et al., 2014	ECoG	SSEP	Spectral analysis	WIMAGINE (Wireless Implantable Multi-channel Acquisition system for Generic Interface with Neurons)	Intracranial BCI: validation on in vivo sheep model
Sand et al., 2017	EEG	DBS response	Averaging	Parkinson's disease	Motor rehabilitation
Kaiju et al., 2017	ECoG	SEP	Wavelet transform	Finger stimulation	Motor learning/ rehabilitation
Vansteensel et al., 2016	iMEA	Visuomotor	Autoregression filter	Late-stage amyotrophic lateral sclerosis	Motor rehabilitation
Downey et al., 2016	iMEA	ME	Firing rate estimator	Vision-guided assistive technology	Robotic prosthetics control
Pahwa et al., 2015	ECoG	Sleep-Wake	Welch's periodogram, logistic regression	Assistive technology	Neuroprosthetic control
Yin et al., 2014	iMEA	ME, Visuomotor, Sleep-Wake	PCA	Full-spectrum electrophysiological recording	BCI, diagnostics, therapeutic treatments
Keefer et al., 2008	iMEA	Video watching	Spectrogram	Carbon nanotube- coated electrodes	Neural decoding and stimulation
Friehs et al., 2006	iMEA	MI	Not specified	Assistive technology/ clinical use of BCI	Cursor control/ epilepsy monitoring

ECoG, electrocorticography; iMEA, intracortical microelectrode array; Stentrode, stent-electrode recording array; EEG, electroencephelography; ME, motor execution; SEP, somatosensory evoked potential; MI, motor imagery; SSSEP, steady-state somatosensory evoked potential; DBS, deep brain stimulation; PCA, principal component analysis.

3.1. Neuroplasticity and Cognitive Rehabilitation

The time-variant behavior of synapses within complex neural networks underpins the plastic characteristics of the brain and was first illustrated by Donald O. Hebb in 1949 (Brown and Milner, 2003). Neuroplasticity not only helps to assist cognitive and perceptual learning but also is the main ingredient for neurorehabilitation. How plastic a particular brain area is, may define the effectiveness of a neurofeedback strategy to induce specific activity patterns. Studies have shown visual cortices are plastic enough to produce robust neural signals for post-neurofeedback perceptual learning (Shibata et al., 2011; Amano et al., 2016). Another study has demonstrated if right/left hemispheric differences in neurofeedback-induced alpha activities are associated with visual information processing and motor behaviors, and, thus, control spatial attention (Jones and Sliva, 2020). fMRI-based neurofeedback training sessions induce the plasticity of attention-related behavior. Implications suggest that neurofeedback can offer rehabilitation of attentional deficit (Megan et al., 2015). A recent study has used neurofeedback to generate robust somatosensory oscillations associated with human perception (Brickwedde et al., 2019).

Closed-loop BCI with neurofeedback is assumed to contribute to the reorganization of cortical-subcortical neural networks and assist subjects in self-regulating specific brain rhythms; notwithstanding, the underlying mechanisms that alter neural substrates are still not fully-understood (Sitaram et al., 2017). For example, BCI-based covert visuomotor training modulates associated neural substrates, where the effects of modulated neural substrates are observed while performing that particular movement-related task (Vyas et al., 2018). Substantial changes in overt movement-related task following BCI-driven training induced learning suggest a critical role of BCI in enhanced motor learning for proficiently controlling neuroprosthetics (Orsborn et al., 2014), i.e., devices that can enhance or repair the output of the nervous system. For example, intracortical electrodes may be used to stimulate specific brain regions to regain motor control (Oxley et al., 2016; Sand et al., 2017). BCI may augment training-induced plasticity during therapeutic motor rehabilitation and, thus, re-excite corresponding neural substrates to regain control by means of neuroprosthetics or upper limb functions (Dobkin, 2007). Other examples include BCI-driven exoskeletons to enhance human working capacity (Benabid et al., 2019).

The extent of BCI-induced plasticity entails several factors, including (1) the selection of the signal acquisition modality, which plays an important role in diagnosing neural states, (2) the design of feedback modality that has explicit association with the neural signal classification performance, (3) the consideration of application-specific feedback delays, and (4) the utilization of a suitable feedback modality (Grosse-Wentrup et al., 2011). Neural ensemble recordings using signal acquisition modalities such as EEG, MEG, fNIRS, and fMRI have become dominant over single unit recordings. Behavioral activities are likely to be distributed across three-dimensional cortical-subcortical networks and that cannot be captured within single unit recordings (Nicolelis and Lebedev, 2009).

Rehabilitative BCI can be designed either by attaching neural prostheses to the impaired body parts or by re-stimulating the damaged synaptic networks; in any of the cases, the idea is to exploit and promote neuroplasticity (Wang et al., 2010; Ramos-Murguialday et al., 2013; Park et al., 2016; Darvishi et al., 2017; Toriyama et al., 2018; Song and Kim, 2019; Romero-Laiseca et al., 2020). In stroke patients with paretic muscles without residual finger movement, increased electromyographic activity post rehabilitation by BCI-driven orthoses exhibits increased neuromuscular coherence that is essential for restoring movement control (Pfurtscheller et al., 2000; Ramos-Murguialday et al., 2013). Explicit application of functional electrical stimulation regulated by EEG-based movement-related signatures further suggests a role of BCI in rehabilitation (Zhao et al., 2016). Increased electromyographic activity in paretic muscles is indicative of plasticity induced by electrical stimulation (De Marchis et al., 2016). For BCI-based rehabilitation in a real-life environment, differentiating between task-induced activities and resting state activities is a key factor for controlling the prosthesis or stimulation modality (Pahwa et al., 2015).

Externally stimulating the affected brain areas by electric or magnetic fields holds promise for stroke rehabilitation. A recent study demonstrated the induction of neuroplasticity in white matter and cortical functions in chronic stroke patients by motor imagery-based BCI and transcranial direct current stimulation applied to targeted brain areas (Hong et al., 2017). Magnetic stimulation of brain areas driven by BCI increases cortical activation in stroke patients (Johnson et al., 2018). The level of neuroplasticity achieved post-rehabilitation varies across subjects and, thus, an individual-specific training session is necessary (Leamy et al., 2014). The use of BCI-based motor rehabilitation for locked-in patients is limited because they are unable to fully interact with the system (Birbaumer and Cohen, 2007). Other examples of BCI-driven rehabilitations include optimizing the parameters for deep brain stimulation applied into the subthalamic nucleus in patients with Parkinson's disease (Sand et al., 2017) and treating major depressive disorder by BCI-driven transcranial magnetic stimulation (Ray et al., 2015).

Either by providing direct control of assistive technologies or by direct neurostimulation, BCI can help patients who may suffer from amyotrophic lateral sclerosis, cerebral palsy, brainstem stroke, spinal cord injuries, muscular dystrophies, or chronic peripheral neuropathies (Kauhanen et al., 2006; Iturrate et al., 2009; Mak and Wolpaw, 2009; Allison et al., 2010; Ramos-Murguialday et al., 2013; Leamy et al., 2014; Botrel et al., 2015; Combaz and Van Hulle, 2015; Edelman et al., 2015; Park et al., 2016; Zhao et al., 2016; Alcaide-Aguirre et al., 2017; Ge et al., 2017; Chiarelli et al., 2018; Guy et al., 2018; Yu et al., 2018; Rezazadeh Sereshkeh et al., 2019; Jin et al., 2020; Zuo et al., 2020). Providing auxiliary degrees of freedom improves the quality of life of people with disabilities significantly. Brain signals can be translated to drive wheelchairs (Galán et al., 2008; Iturrate et al., 2009; Perdikis et al., 2017; Tonin and Millán, 2020). Integration of BCI with a vision-guided autonomous system was shown to effectively perform the grasping task using a prosthetic arm in a tetraplegic patient (Downey et al., 2016). An implanted microelectrode array has been proposed to operate a three-dimensional neuroprosthetic device (Taylor et al., 2002).

3.2. Signal Acquisition, Signal Processing, and Modeling

A significant number of studies are now involved in combining multimodal signal acquisition modalities to augment current BCI systems. For example, simultaneous EEG and fMRI yield complementary features by exploiting good temporal resolution of EEG and good spatial resolution of fMRI (Debener et al., 2006). Enhanced multiclass sensorimotor tasks classification performance using hybrid EEG and fNIRS signals implicates the importance of features extracted from both hemodynamic and electrical activities (Buccino et al., 2016). MEG is another potential tool to combine with EEG, as it captures radially/tangentially dipole sources in cortical-subcortical networks and adds complementary information to EEG signals (Kauhanen et al., 2006). Skepticism might still present about the detection of brain activities originated from subcortical areas; however, an increasing number of studies argue that EEG and MEG could capture subcortical activities (Andersen et al., 2019; Min et al., 2020; Piastra et al., 2020). A recent trend is to combine different signal acquisition modalities together to improve BCI efficiency. Table 4 highlights multimodal and hybrid BCI applications.

Table 4.

A summary of multimodal and hybrid brain computer interface (BCI) studies.

References	Modality	Wave\Task	Data analysis	Context	Application
Zuo et al., 2020	EEG	MI+P300	CSP, LDA	Hybrid BCI	Post-stroke rehabilitation
Rezazadeh Sereshkeh et al., 2019	EEG+fNIRS	Imagined speech	WT, LDA	Multimodal BCI	Robotic control
Corsi et al., 2019	EEG+MEG	MI	LDA	Multimodal BCI	Motor rehabilitation
Chiarelli et al., 2018	EEG+fNIRS	MI	Deep learning	Multimodal BCI	Motor rehabilitation
Ge et al., 2017	EEG+fNIRS	MI	CSP, SVM	Multimodal BCI	Motor rehabilitation
Zhao et al., 2016	EEG+FES	SSVEP	CSP	Motor plasticity	Paretic limb rehabilitation
Combaz and Van Hulle, 2015	EEG	P300+SSVEP	SVM	Hybrid BCI	Assistive control
Rao et al., 2014	EEG+TMS	Visuomotor	LF	Hyperinteraction	Brain-to-brain interface
Grau et al., 2014	EEG+TMS	MI	Spatial filter, re-referencing	Hyperinteraction	Brain-to-brain interface
Choi and Jo, 2013	EEG	P300+SSVEP+MI	CSP, CCA	Hybrid BCI-based assistive technology	Humanoid robot control
	EEG	SSVEP+MI	LDA	Hybrid BCI	Motor rehabilitation, assistive technology
Kauhanen et al., 2006	EEG+MEG	ME	Particle filters	Multimodal BCI	Spinal cord injury

EEG, electroencephalography; fNIRS, functional near infrared spectroscopy; MEG, magnetoencephelography; FES, functional electrical stimulation; TMS, transcranial magnetic stimulation; P300, an event-related potential; SSVEP, steady-state visual evoked potential; MI, motor imagery; ME, motor execution; WT, wavelet transform; CSP, common spatial pattern; CCA, canonical correlation analysis; SVM, support vector machine; LDA, linear discriminant analysis; LF, Laplacian filter.

The combination of signal processing and machine learning approaches plays critical role in translating any brain signal to a command for a computer or other external devices. Tables 2–5 highlight different signal processing and machine learning techniques. Representing signals in the time-frequency-space is necessary to obtain physiological correlates of BCI outcomes (McFarland et al., 2006; Bashashati et al., 2007). Fourier transform (FT) and autoregressive models are examples of time domain representations of brain signals while short time FT and wavelet transform are examples of time-frequency representations (McFarland et al., 2006; Bashashati et al., 2007). In case of spatial filtering, the most popular filtering approaches are common spatial pattern, independent component analysis and the Laplacian filter. A diverse range of inverse models allow to discern the actual sources projected on three-dimensional cortical-subcortical networks (Wronkiewicz et al., 2015; Saha et al., 2019a). Extracted features can be translated using various linear and non-linear classification algorithms. Examples of linear and non-linear classifier models are linear discriminant analysis and non-linear kernel-based support vector machines (Lotte et al., 2007, 2018).

Since the first publication in 2000, common spatial pattern is still one of the most popular methods to represent multichannel EEG signals by corresponding spatial contents (Ramoser et al., 2000). As a data-driven method, it requires a significant number of training samples to model the filtering parameters. In case of small training trials, regularizing the covariance estimation works better than the traditional algorithm (Lotte and Guan, 2010). Other modifications in spatial filtering include projecting EEG by using sparse representation and filter bank spectral division of raw signals (Arvaneh et al., 2014). Generally, spatial filtering is applicable in subject-specific BCI development although recent studies have proposed estimating the filter coefficients from a subject and applied that filter to another subject, which contributed no training sample (Saha et al., 2017, 2018, 2019a). Other popular data-driven methods include linear discriminant analysis, support vector machine and principal component analysis (Lotte et al., 2007, 2018). With the exceptional advancements in computational facilities in the last decade, deep learning-based BCI paradigms by allowing the evaluation of large datasets could soon become a trend in the community (Chiarelli et al., 2018; Kwon et al., 2019; Nagel and Spüler, 2019).

On the other hand, independent component analysis is a blind source separation method requiring no training. The estimation of independent components is based on statistical properties of the signals (Bell and Sejnowski, 1995). However, modeling the actual cortical sources as dipoles in the complex brain anatomy from the scalp EEG recordings seeks to solve the so-called inverse problem (Qin et al., 2004; Kamousi et al., 2005; Wronkiewicz et al., 2015; Saha et al., 2019a). More recent source localization methods such as wavelet-based maximum entropy on the mean represent EEG/MEG signals as relevant time-frequency contents and finally transform them into spatial representations (Lina et al., 2012; Saha et al., 2019a). Notably, different inverse methods and toolboxes demonstrate considerable variability in localized sources (Mahjoory et al., 2017). Even it is not very straightforward to know the exact sources, which are to be modeled using EEG/MEG. For example, the ground truth defined by implanted electrodes might not be 100% reliable because of sparse (spatial) sampling. In the case of fMRI, the measurement of neural activity is indirect. Notwithstanding, inverse methods have shown promise for designing various BCI models (Qin et al., 2004; Kamousi et al., 2005; Wronkiewicz et al., 2015; Saha et al., 2019a).

3.3. Neurosensors: The-State-of-the-Art

Deeper regions of the brain, e.g., subcortical and cerebellar regions, contribute to various neuronal activities (Müller et al., 2002; Wardman et al., 2014). Interpreting the genesis of cortical sources from cellular to scalp levels and RSNs spanned throughout the three-dimensional brain space can guide BCI development (Donoghue, 2008). Sensors with customized design are developed to advance brain signal acquisition modalities. Neurosensors can be constructed in different forms like electrical, optical, chemical and biological (Deisseroth and Schnitzer, 2013). Dry EEG electrodes are convenient, but assumed to provide lower signal-to-noise ratio compared to conventional wet electrodes. Wet electrodes may cause inconvenience to users as they use conductive gel and require proper skin preparation for minimizing the skin-electrode impedance (Liao et al., 2012). However, a study on dry electrodes-based BCI suggested that dry electrode could be used to collect good quality signals by designing the circuits carefully (Chi et al., 2011). Further studies support dry electrodes with wireless systems that could offer comparable signal quality as of wet electrodes, but with more convenience (Di Flumeri et al., 2019; Marini et al., 2019; Hinrichs et al., 2020). While utilizing the advantages of both dry and wet electrodes, quasi-dry electrodes exploiting the mechanical properties of polymer can capture signals as comparable to commercial Ag/AgCl electrodes (Mota et al., 2013). To increase the spatial resolution of EEG, Petrov et al. have proposed an ultra-dense sensor array of 700–800 electrodes (Petrov et al., 2014). The signal-to-noise ratio was twice as high as for high-density EEG that has up to 256 gold-coated electrodes. An auricle electrode with stretchable connector was proposed that not only can increase portability but also can offer a comfortable alternative for long term recordings (Norton et al., 2015). The electrode is flexible with the alterations of electrical and mechanical properties of skin.

Invasive sensors must be biocompatible. A novel organic electrochemical transistor-based sensor enables to collect neural signals directly from the brain surface (Khodagholy et al., 2013). This sensor is biocompatible and mechanically flexible, and the transistor-based design amplifies captured signals locally, thus providing much better signal-to-noise ratio than conventional ECoG. To enhance the signal quality, carbon nanotube coating can decrease the electrode impedance and, thus, increase the charge transfer (Keefer et al., 2008). Another invasive biocompatible sensor, designed for recording previously inaccessible spectra of large neuron populations, includes data transmission for use in natural environments (Yin et al., 2014). With the outstanding progress of nanotechnology, nanowire Field Effect Transistor and other p/n junction devices have potential for neuro-sensing modalities for intracellular recordings, even in the deep brain regions (Kruskal et al., 2015). Oxley et al. have proposed stent-electrode array (stentrode) that involves minimal invasiveness (Oxley et al., 2016, 2017). Using computer-guided catheter angiography, the stentrode can be placed within arteries or veins located inside the brain anatomy. Capturing high-fidelity cortical signals, this technology will significantly reduce the risk factors of craniotomy. A follow-up study has recently demonstrated successful implantation of the strentrode in humans for long-term neural signal recording (Oxley et al., 2020). The information transfer rate for strentrode-based BCI was comparable to the landmark study by Vansteensel et al. with implanted electrodes (Vansteensel et al., 2016). Another implanted ECoG recorder called as WIMAGINE (Wireless Implantable Multi-channel Acquisition system for Generic Interface with Neurons) allows wireless neural data access (Mestais et al., 2014). The WIMAGINE has recently been tested for long-term reliability of data acquisition and any risk associated with craniotomy (Kruskal et al., 2015; Sauter-Starce et al., 2019).

Besides large-scale recording modalities like EEG and MEG, very small-scale recordings of neuronal activities are crucial for understanding brain circuits' functions and intra- and inter-neuron interactions. Representation of any cognitive task as functions of both small-scale and large-scale neuronal interactions is crucial for the advancement of neuroengineering and BCI. In this regard, a high-density neurosensor array made from silicon probes combined with optogenetics enables single unit recordings (Buzsáki et al., 2015). Yang et al. proposed a novel multi-plane two-photon microscope that can be used to capture multi-layer neuronal structure and mechanism with cellular resolution (Yang et al., 2016). Other potential imaging methods for investigating cell signaling include calcium imaging (Grienberger and Konnerth, 2012) and advanced microscope with chronically implanted lenses (Resendez et al., 2016). Designer receptor exclusively activated by designer drugs, provides a chemogenetic tool to understand cell-signaling including electrical activities in molecularly clustered cell groups (Sternson and Roth, 2014; Roth, 2016). A new ultrasonic-based wireless system, called neural dust, enables the recording of electromyogram and electroneurogram on the millimeter scale (Seo et al., 2016).

3.4. Affective Computing, Gaming, Robotics, and Miscellaneous Applications

Future computers are assumed to have emotional and perceptual capabilities, which could extend the use not only to assisting humans but also to making decisions (Picard, 2000). Computers might able to recognize and interpret underlying affective states based on physiological and behavioral variables. Recent studies demonstrated BCI is a potential tool to investigate affective states, expanding the applications into psychology (Piho and Tjahjadi, 2018; Song et al., 2018; Huang et al., 2019). Huang et al. have proposed an EEG-based BCI to detect positive and negative emotions induced by video stimulus (Huang et al., 2019).

Integration of arts into BCI is referred to as artistic BCI (Andujar et al., 2015). In the late 1960s, David Rosenboom began experimenting with ways to link brain functions with musical production, perception of musical forms and musical proprioception (Rosenboom, 2014). Other examples of artistic BCI include affective states detection, playing video games and controlling virtual/augmented reality environment. Studies have demonstrated that a user can fully operate video games by SSVEP-BCI (van Vliet et al., 2012; Filiz and Arslan, 2020). Other studies have proposed how multiple users can participate in a collaborative game, in which joint decision making is required to control the gaming environment (Nijholt and Poel, 2016; Sekhavat, 2020). Another study previously suggested the aggregation of information from two intelligence analysts' brain signals may lead to better decision making than one's brain signals (Stoica, 2012). The underlying cause could be explained by inter-individual differences in human cognitive and perceptual skills (Kleinschmidt et al., 2012). Collaboration between users might assist an individual's decision making by diversity inclusion. A modified setup could investigate how people interact in different social contexts, extending BCI applications in sociology (Amaral et al., 2017).

Virtual/augmented reality (V/AR) technologies together with BCI could offer immersive experiences and have many potential applications including arts and neurofeedback (Andujar et al., 2015; Tremmel et al., 2019; Putze et al., 2020). Brain painting allows a user to draw lines in a virtual canvas by brain signals, which gives an alternative communication channel for people with paretic motor functions (Botrel et al., 2015). McClinton et al. have developed a brain painting application using VR environment (McClinton et al., 2019). Another work has evinced VR-BCI to measure cognitive workload that can contribute to neuroergonomics (Tremmel et al., 2019). Studies have also used immersive VR as a better neurofeedback option as compared to the computer screen leading to increased BCI accuracy (Luu et al., 2016; Škola et al., 2019; Vourvopoulos et al., 2019; Juliano et al., 2020). Vourvopoulos et al. have integrated the principles of VR and BCI into a platform called REINVENT for motor rehabilitation (Vourvopoulos et al., 2019). Likewise, BCI with AR can be used to remotely control a robot for rehabilitating children with attention-deficit/hyperactivity disorder (Arpaia et al., 2020).

While BCI-driven robotic controllers can offer advanced assistive technology for people with mobility constraint, it may also augment human ergonomic performance for healthy subjects (Millan et al., 2004; Gandhi et al., 2014; Tidoni et al., 2016; Perdikis et al., 2017; Spataro et al., 2017; Yuan and Li, 2018; Deng et al., 2019; Tonin et al., 2019; Tonin and Millán, 2020). EEG-based BCI-driven controller of mobile robot or wheelchair has demonstrated the possibility of this technology in robotics industry (Millan et al., 2004; Tidoni et al., 2016; Perdikis et al., 2017; Yuan and Li, 2018; Deng et al., 2019; Tonin et al., 2019). BCI can also be used for controlling humanoid robots remotely using EEG (Spataro et al., 2017), suitable in hazardous environments, for example by sending a robot in a coal mine for executing a task that is potentially unsafe for a human. In space, BCI can be used to monitor astronauts' working capacity and to drive an exoskeleton (Menon et al., 2009; de Negueruela et al., 2011). In the absence of gravity, working becomes tedious and inconvenient. Furthermore, astronauts' working time is precious. BCI-driven systems could be practical for improving astronauts' functionality, efficiency and safety (Summerer et al., 2009; Farwell et al., 2014; Botrel et al., 2015; Ortiz et al., 2016; Wang et al., 2016a; Vourvopoulos et al., 2019; Singh et al., 2020).

Recently, brain-to-brain interface (BBI) experiments that involve decoding sender's cognitive intentions, translate them into commands for stimulating receiver's brain, have been explored (Pais-Vieira et al., 2013; Rao et al., 2014; Jiang et al., 2019). In 2013, researchers implemented a direct BBI system in which one rat was able to share sensorimotor information to another rat (Pais-Vieira et al., 2013). Intracortical microstimulation was used to stimulate the receiver's target brain areas. An early attempt to develop sensorimotor rhythm-based BBI between two human subjects used non-invasive EEG and transcranial magnetic stimulation has been proposed by Rao et al. (2014). Other total non-invasive BBI experiments have proposed sharing pseudo-random binary streams encoded words between human subjects (Grau et al., 2014) and playing collaborative games (Stocco et al., 2015). Figure 2 illustrates a timeline for current advances of BCI in diverse applications.

Figure 2.

A schematic illustration of the evolution of the brain computer interface (BCI) applications: Cognitive & Perceptual Learning/Rehabilitation (McMillan et al., 1995); Orthosis Control (Pfurtscheller et al., 2000); Music BCI (Rosenboom, 2014); Robotics (Millan et al., 2004); Wheelchair Control (Iturrate et al., 2009); Drowsiness Detection (Lin et al., 2008); Affective Computing (Zander et al., 2009); Brain Racers (Perdikis et al., 2017); Multiplayer Gaming (Nijholt and Poel, 2016); Brain-to-Brain Interface (Rao et al., 2014).

4. Ethical Concerns and Socioeconomic Contexts

Irrespective of the scientific breakthroughs in BCI field, there are key factors pertaining to safety, ethics, privacy protection and data confidentiality, community acceptance and socioeconomic aspects that should be considered with adequate precautions to maximize users' benefits and social impacts (Illes and Bird, 2006; Bostrom and Sandberg, 2009; Jebari, 2013; McCullagh et al., 2014). Obtaining an ethically sound informed consent from a BCI worn patient may be challenging for BCI researchers due to difficulty in communicating and the lack of alternatives. However, more awareness and attention to ethics policies are recommended to improve the chance for patients to get adequate information.

Physical and mental safety of BCI users is important. Invasive procedures such as deep brain stimulation and intracortical microelectrode array may cause postoperative psychological and neurological side effects (Jotterand and Giordano, 2011; Gilbert, 2015; Maslen et al., 2015). Additionally, bleeding and infections are infrequent but do occur and may require removal or further maintenance of the implanted electrodes. Guidelines are required to safely advance neurotechnologies (Goering and Yuste, 2016), because BCI devices can alter behavior and, thus, introduce potential threats to one's emotions, personality and memories; more generally one's mind. For human brain-to-brain interface applications (Rao et al., 2014; Stocco et al., 2015), one may define an upper bound for research depths keeping in mind the necessity of ethical utilization of this technology. Because both sender and receiver play complicated roles, more specifically, sender's intentional manipulative control over neural signals might alter the anticipated outcome. Altering human cognitive and possibly moral capacity raise a serious ethical question and it is not predictable if the cognitive changes reversible and efficacious (Nakazawa et al., 2016).

A user's expectations of achieving extended or auxiliary degree of freedom may not be fulfilled, and even the unfamiliar risk factors can diminish the accomplished advantage of using BCI (Clausen, 2011; Schicktanz et al., 2015). Creating broad awareness of BCI technology and its pros and cons would educate people, who fear unnecessary technological dependency (Hobson et al., 2017). However, successful clinical trials of sophisticated devices such as strentrode or WIMAGINE are essential to demonstrate potential advantages, especially for people suffering from any form of cognitive disability (Sauter-Starce et al., 2019; Oxley et al., 2020). In the case of healthy users, it should not be too difficult to create acceptance to a broader community when dry electrodes could offer the long-term operation of a BCI application with little maintenance effort (Di Flumeri et al., 2019; Marini et al., 2019; Hinrichs et al., 2020).

It is critical to introduce a suitable act for lawful utilization of BCI and preservation of privacy and confidentiality of stored data. Recent studies have demonstrated decoding of password or recognizing faces utilizing consumer-grade BCI successfully, prompting a potential concern of any illegitimate access to users' raw data and their further exploitation (Martin et al., 2016; Alomari et al., 2019). For example, affective states define users' moral judgment and emotional traits. Thus, it is critical to limit the applications of affective BCI while preserving sensitive information (Steinert and Friedrich, 2020). Necessary precursor initiatives should propose application-specific BCI frameworks, which can restrict unauthorized access to stored data or the system (Ienca and Haselager, 2016). For example, illicit access to a wireless BCI-driven limb and manipulative reprogramming of a computer-guided neuro-stimulation have demonstrated the importance of establishing resilient safeguards to BCI use (Denning et al., 2009). Agarwal et al. have proposed cryptographic protocols as integrated parts of BCI to preserve the privacy of a user by keeping confidential information obscure to others (Agarwal et al., 2019). Without evaluating socioeconomic, ethical and policy issues, the commercialization of BCI would hinder the progress in this field (Eaton and Illes, 2007).

By creating a common networking platform for BCI researchers worldwide, the immediate proposition of a comprehensive list of universal guidelines is key to sustainable advancements of the field (Vaadia and Birbaumer, 2009). Various alliance-based projects are running as common platforms for advancing the knowledge of neuroscience, for example, by strengthening efforts to fund neuroscience research projects (Grillner et al., 2016). The European Union along with its partner universities have initiated the Human Brain Project. In addition, the Brain Initiative has been announced by the White House. In our opinion, advanced understanding of basic neuroscientific phenomena will determine the structure, efficacy and applications of futuristic BCI.

5. Conclusion

Numerous groundbreaking advances in neurosensors and computational tools herald great promise for more sophisticated and user friendly BCI systems requiring no or little maintenance. In addition to hi-fidelity signal acquisition, significant progress in signal processing and machine learning tools, their complementary roles, and high computation power and increased mobility of computers have significantly contributed in the emergence of BCI technologies. The future of BCI technology will rely greatly on addressing the following key aspects:

• Elucidating the underlying psychophysiological and neurological factors that potentially influence BCI performance.

• Designing less invasive sensors with reliable signal acquisition and resolution, while considering portability, easy maintenance, and affordability.

• Modeling session-to-session and subject-to-subject information transfer for the proposition of more generalized BCI models with insignificant or no calibration requirement.

• Establishing broad consensus on ethical issues and beneficial socioeconomic application of this technology.

Author Contributions

SS conceived the initial idea, wrote the first draft, and generated all figures and tables. KM, KA, RM, GN, and SD participated in the discussion and commented on the draft. AHK and MB provided further insight and helped SS to finalize the structure and materials. All authors read and approved the final paper.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.