A review on the performance of brain-computer interface systems used for patients with locked-in and completely locked-in syndrome

Abstract

Patients with locked-in syndrome (LIS) and complete locked-in syndrome (CLIS) own a fully functional brain restricted within a non-functional body. In order to help LIS patients stay connected with their surroundings, brain-computer interfaces (BCIs) and related technologies have emerged. BCIs translate brain activity into actions that can be performed by external devices enabling LIS patients to communicate, leading to an increase in their quality of life. The past decade has seen the rapid development of BCIs that have the potential to be used for patients with locked-in syndrome, from which a great deal is tested only on healthy subjects and not on actual patients. This study aims to (1) provide the readers with a comprehensive study that contributes to this growing area of research by exploring the performance of BCIs tested specifically on LIS and CLIS patients, (2) give an overview of different modalities and paradigms used in different stages of the locked-in syndrome, and (3) discuss the contributions and limitations of BCIs introduced for the LIS and CLIS patients in the state-of-the-art and lay a groundwork for researchers interested in this field.

Introduction

The term ‘Locked-in syndrome’ was first introduced in 1966 by Plum and Posner (Posner et al. 2008). According to their definition, locked-in syndrome or LIS “is a specific neurobehavioral diagnosis that refers to patients who are alert, cognitively aware of their environment, and capable of communication but cannot move or speak” (Posner et al. 2008). In this devastating condition, the brain is fully functional while restricted within a non-functional (quadriplegic and anarthric) body (Ohry 1990) that cause this condition to be interpreted as “buried alive” (Khanna et al. 2011) or the disease, in which patients are imprisoned or trapped within their own body (Sledz et al. 2007; Chisholm and Gillett 2005). In fact, the first description of this condition, without recognizing it as what is now called locked-in syndrome, can be traced back to describing the situation of a character in the book ‘The Count of Monte Cristo’ (Dumas and Sante 2004) written by the writer Alexander Dumas, in 1844, almost a century before introducing the condition by Plum and Posner: “Sight and hearing were the only senses remaining, and they, like two solitary sparks, remained to animate the miserable body which seemed fit for nothing but the grave; it was only, however, by means of one of these senses that he could reveal the thoughts and feelings that still occupied his mind, and the look by which he gave expression to his inner life was like the distant gleam of a candle which a traveller sees by night across some desert place, and knows that a living being dwells beyond the silence and obscurity” (Dumas and Sante 2004).

Stephen Hawking, and the French journalist and editor in chief of Elle magazine, Jean-Dominique Bauby, are well-known people diagnosed with LIS and have played a significant role in raising public awareness about this condition. Bauby describes his experience in his book ‘The Diving-Bell and the Butterfly (Bauby 1998)’ as “something like a giant invisible diving-bell holds my whole body prisoner.” It may be assumed that the rare condition of being trapped in one’s own body forces these patients to have a poor Quality of Life (QoL), bringing up the assumption that this severe motor disability is unbearable and may lead these patients to euthanasia. But surprisingly, having a fully functional brain makes the majority of these patients want to have a meaningful life as much as a healthy person does (Lulé et al. 2009; Bruno et al. 2011; Rousseau et al. 2015), only if they can find a way to unlock this devastating condition and communicate with the outer world.

Before the development of advanced means of communication, the only way for these patients to connect with others was via gaze and facial movements in some patients whose condition had not yet progressed (Patterson and Grabois 1986). In recent decades, with technological advancements, in order to provide LIS patients with a device for communication and control without the need for any muscular input, brain-computer interfaces or BCIs have been developed and are continuously being improved (Lulé et al. 2009). Brain-computer interfaces (BCIs), among numerous applications that have proven to be game-changers, have significantly impacted the lives of individuals with locked-in syndrome, helped them restore communication, and improve their quality of life.

This paper provides a review of the state-of-the-art studies that have investigated the use of brain-computer interfaces specifically for LIS and CLIS patients. This comprehensive study aims to contribute to this growing area of research by exploring the impacts of BCIs on the quality of life of LIS patients and probing the limitations to lay a groundwork for researchers interested in this field. Different review articles in the literature discuss the applications of BCIs for LIS patients and review the methods used to develop them. Still, some studies discussed in these review papers are not tested on actual LIS or CLIS patients despite introducing locked-in and complete locked-in patients as their main potential users and are only tested on healthy subjects (Xiaoxiao et al. 2020; S. Aziz, S. Ibraheem, A. Malik, F. Aamir, M.U. Khan, U. Shehzad, Electrooculugram based Communication System for People with Locked-in-Syndrome, in 2020; Rozado et al. 2015). What makes the current review study stand out amongst other such studies in this field is our inclusion criteria which made us discuss only studies in which the proposed BCI is tested on at least one LIS or CLIS patient. This approach ensures that researchers interested in this field can gain more practical insights on this matter by reading the current review paper, allowing them to get to know the challenges faced by LIS patients while using these BCIs more realistically.

This study aims to provide the readers with a comprehensive study that contributes to this growing research area by exploring the performance of BCIs tested specifically on LIS and CLIS patients. Furthermore, an overview of different modalities and paradigms used in different stages of the locked-in syndrome is provided, and last but not least, the contributions and limitations of BCIs introduced for LIS and CLIS patients in the state-of-the-art are discussed.

Locked-in syndrome and its clinical features

The term ‘locked-in syndrome’ has been used to refer to a condition in which awareness fully exists, but the patient is tetraplegic, aphonic, and anarthric (Ohry 1990). According to the reports which have hitherto been published, the most likely causes of this syndrome are motor neuron diseases (MNDs) such as amyotrophic lateral sclerosis (ALS), pontine lesions in the brainstem caused by stroke, traumatic and anoxic brain injury, and cerebrovascular diseases, which involves progressive degeneration of all the motor neurons of the somatic motor system that might lead to a completely locked-in state (Birbaumer et al. 1999; Casanova et al. 2003). In some cases, even bacteria such as Orientia tsutsugamushi bacterium (Sharma et al. 2020) or can be the underlying etiology of locked-in syndrome. With more precise definition of the locked state, as well as the enhancement of the quality and efficiency of the devices, the number of misdiagnoses between LIS patients and other similar conditions such as COMA has decreased, and the number of cases diagnosed as LIS has increased, which is drawing the attention of the experts in this field.

In acute conditions, there is a likely chance of misdiagnosis of LIS with coma, vegetative state, catatonia, akinetic mutism, or unresponsive wakefulness syndrome (Saito et al. 2019; Phillips et al. 2011; León-Carrión et al. 2005; Adams and Victor 1981). In order to diminish controversies and confusion on the use of LIS and other diagnostic and clinical terms assigned to patients with severe alterations in consciousness, the American Congress of Rehabilitation Medicine (ACRM) presented the following neurobehavioral criteria as the biomarker for diagnosing LIS:

“1-Eye opening is well sustained (bilateral ptosis should be ruled out as a complicating factor in patients who do not open their eyes but demonstrate eye movement to command when the eyes are opened manually),

2- Basic cognitive abilities are evident on examination,

3- There is clinical evidence of severe hypophonia or aphonia, 4-There is clinical evidence of quadriparesis or quadriplegia,

5- The primary mode of communication is through vertical or lateral eye movement or blinking of the upper eyelid (American Congress of Rehabilitation Medicine 1995).”

Researchers and neurologists working on nonresponsive patients should be vigilant while working with patients who are tetraplegic and anarthric because lack of voluntary eye movements does not necessarily indicate that these patients are unconscious, and the possibility of being in the locked-in state should be highly regarded (Kondziella 2017).

A system of classification of locked-in syndrome proposed by Bauer et al. (Bauer et al. 1979) in 1979 is the most widely used. In this system, LIS is divided into three main categories. Classical LIS is defined as the condition of total immobility except for vertical eye movements and blinking. The presence of any other movements in addition to the criteria stated in the classical form leads to Incomplete LIS. The condition of total immobility, where there is not even eye movement, but there are signs of intact cortical function in the EEG, introduces the concept of Total LIS or Complete LIS (CLIS). Later in an article presented by Kübler et al. (Kübler and Birbaumer 2008), this classification system has been broadened to include additional sub-groups for the case of ALS. They introduced Minor Impaired conditions, where only slight limb movement and normal speech are present. Moderate Impaired condition as patients have restricted limb movement with unaffected speech and are just wheelchair-bound, or they have intact limb movement without speech, and Major Impaired condition, in which patients are almost tetraplegic with restricted speech. Categories 4 and 5, which are our concerns, would consist of LIS and CLIS patients.

LIS and CLIS are terms frequently used in the literature, but to date, there is no consensus about how to separate them from each other in some cases precisely. In a recent study by Chaudhary et al. in 2020 (Chaudhary et al. 2020), despite the absence of a quantitative and reliable method to discriminate LIS and CLIS, an effort has been made to circumscribe the differences between them and decrease the ambiguities in their definition. Hence, they have pointed out the necessity of performing repeated eye movement measurements with electrooculography (EOG) to indicate permanent CLIS. They also reported that in some cases, performing electromyography (EMG) on face muscles and the external sphincter could effectively define the existing state.

Data from several studies have suggested that responses in LIS patients and healthy individuals' brain signals are not significantly different (Rosanova et al. 2012; Casali et al. 2013). Still, this claim has recently been challenged by Freudenburg et al. (Freudenburg et al. 2019) in a study conducted in 2019. They have noticed some differences between the spectral power changes of the LIS patients generated by their hand movement. They observed that a portion of the Beta range (12–22 Hz) which plays a vital role in motor imagery (MI) tasks, does not contain clear oscillations in these patients, which is a new claim added to what was earlier believed that there are just slight differences between the alpha (Hawkes and Bryan-Smyth 1974; Kotchoubey and Lotze 2013) and delta (Babiloni et al. 2010) ranges of these patients and healthy subjects. Therefore, they have concluded that oscillations in the low frequencies might be considerably different in these patients compared to non-disabled people. These alterations can also exist between different LIS patients due to the underlying etiology of their motor impairment. In another study conducted by Maruyama et al. (Maruyama et al. 2020) in 2020, the results showed a significant power reduction in high alpha, beta, and gamma bands in CLIS patients, indicating the dominance of slower EEG frequencies in their oscillatory activity. It is worth noting that after studying two CLIS ALS patients, Hohmann et al. (Hohmann et al. 2018) suggested that alpha peak frequency (APF) in these patients is shifted towards the lower end of the EEG spectrum and the alpha rhythm is slowed in late stage ALS patients.

Life expectancy and quality of life of LIS patients

Health-related quality of life (HRQOL) is a reflection of how individuals perceive and respond to their health conditions and non-medical aspects of their lives, including factors related to health and well-being. Health-related, such as physical, functional, emotional, and mental health, as well as non-health related items, such as work, family, friends, and other life situations (Gill 1994). QOL is a broad concept encompassing all aspects of human life, while HRQOL focuses on the effects of disease and, more specifically, the impact of treatment on quality of life (Guyatt et al. 2007). Despite the difference between these two concepts, QOL and HRQOL are often used interchangeably to refer to the same concept.

Previous studies on life expectancy showed that after medical stabilization of a patient in LIS for more than a year, 10-year and 20-year survival rates were 83% and 40%, respectively (Doble et al. 2003; Smith and Delargy 2005). Undoubtedly, the early months of being in locked-in state for patients and what they experience is almost beyond imagination for healthy individuals. As Nick Chisholm (Chisholm and Gillett 2005), who was diagnosed with LIS after an accident on a rugby field, states about his early months: “If dying is as painless and peaceful as just drifting off to sleep, then there’s plenty of really very frustrating times that at a particular point I wished I wasn’t here anymore. It felt like I was in a really bad nightmare constantly for about the first three months. I could only just hear (I couldn’t even open my eyes or breathe by myself); without them even knowing that I still could hear, the doctors and specialists in front of me said to my mum that I would die. They even asked my mum if she wanted them to turn the life support machine off after a few days.”

It has been demonstrated that in well over 50% of the cases, it is the family and not the physician who first notices the signs of awareness. The diagnosis of LIS usually takes over 2.5 months. Unfortunately, in some cases, patients locked in their bodies were recognized as being conscious after 4 to 6 years (Laureys et al. 2005). As time passes, LIS patients may either recover to some extent in some post-stroke cases or adapt to the situation psychologically. The case of Chisholm confirms this adaptability as he adds after a couple of years passing through his condition: “The incredibly immense frustration levels at times have eased slightly over the years because of physical and health gains I have made.” (Chisholm and Gillett 2005). Many root causes of LIS have been reported in the literature (Table 1).

Table 1.

Causes and mechanisms of locked-in syndrome

Cause	Mechanism
Ischaemic	Basilar artery occlusion, hypotensive or hypoxic events
Haemorrhage	Haemorrhage originating within or infiltrating into the pons
Traumatic	Direct brain stem contusion or vertebrobasilar axis dissection
Tumour	Primary or secondary infiltration of the ventral pons
Metabolic	Central Pontine Myelinolysis
Demyelination	Multiple sclerosis affecting the ventral pons
Infectious	Abscess infiltrating the ventral pons, brain stem encephalitis

One of the well-known tools for measuring the effects of assistive technology on Quality of Life (QoL) is the Psychosocial Impact of Assistive Device Scale (PIADS), in which the perception of the three dimensions of competence, adaptability, and self-esteem is being used (Jutai and Day 2002). Contrary to expectations, there has not been a significant difference between the perception of general personal health and mental well-being reported by LIS patients compared to that of matched control individuals (Ware et al. 1993). Another unanticipated finding is that despite the confirmation of the patients about the existence of suicidal thoughts (Anderson et al. 1993), they would want life-sustaining treatments, and there are rare demands of euthanasia among them (Doble et al. 2003). It is now well established from various studies (Khanna et al. 2011; Lulé et al. 2009; Kuzma-Kozakiewicz et al. 2019) that the QoL in LIS patients is often significantly underestimated by the patient’s primary caregivers and spouses compared to what the patients claim about their condition. It should be noted that differences in opinions about the end of life in LIS might be affected by personal characteristics and cultural backgrounds. A recent study by Yan et al. (Yan et al. 2019) suggests that the inclination to be kept alive decreases in elderly and non-religious people. In a study by Corallo et al. (Corallo et al. 2017), the impact of the augmentative and alternative communication (AAC) on the QoL of 15 LIS patients was evaluated, and they concluded that AAC could decrease depression and anxiety symptoms both in patients and their primary caregivers and improve their QoL significantly. They believed that this improvement stems from the fact that reducing anxiety levels and having a better emotional condition can change the perception of pain and hence, ease the experience of pain. Back then, when communication with CLIS patients was not possible, family members reported that their loved one was communicating by changing pupil size, pulse rate, or just looking in the eye (Hochberg and Cudkowicz 2014). This encouraged passionate physicians not to lose hope in these patients and try to find a way to communicate with them.

Brain-computer interfaces have brought LIS patients out of the prison

As much as we hope for motor recovery in LIS patients, it has been reported by studying some post-stroke LIS patients (either by performing rehabilitative physical exercises (Law et al. 2018; Høyer et al. 2010) or spinal cord stimulation(SCS) (Huang et al. 2019) or transcranial direct current stimulation (tDCS) (Satow et al. 2019)) that it does not usually happen. Unfortunately, for patients with progressive neurodegenerative diseases such as amyotrophic lateral sclerosis (ALS) or cerebral palsy, available rehabilitation methods are not able to restore normal motor function (Daly and Wolpaw 2008). A LIS patient having difficulty using his/her muscular system or unable to communicate effectively due to the absence of means to do so can benefit from Brain-Computer Interfaces and substitute brain activity for muscular activity. BCIs are developed to provide a muscle-independent communication pathway between the brain and a computer or machine (Kübler et al. 2001). A brain-computer interface, also known as brain-machine interface (BMI) or neural interface system (NIS), uses control signals generated from brain signal activity and helps humans to command external devices and interact with their surroundings through a new non-muscular channel (Nicolas-Alonso and Gomez-Gil 2012). Differences in the ability to use a BCI vary from person to person and from session to session. A reliable aptitude assessment would allow for the selection of suitable BCI paradigms. BCI research is frequently conducted by multidisciplinary experts, including engineers, computer scientists, neurologists, and allied health professionals such as occupational and physical therapists, speech-language pathologists, etc.

BCIs may be classified based on the signal acquisition method into three categories:

• Non-Invasive methods, in which electrodes are placed on the scalp,

• Semi-Invasive method, where electrodes are placed under the skull and on the surface of the cortex, or the electrodes are injected into the major blood vessel of the brain

• Invasive method, in which electrodes penetrate the brain tissue.

Invasive and semi-invasive methods require a surgical procedure called craniotomy, in which the skull is opened to acquire signals from the brain. Signals recorded from the semi-invasive and invasive techniques are called Electrocorticogram (ECoG) and Intracortical microelectrode Recoding, respectively (Graimann et al. 2010). In another system of classification, BCI systems can be divided into synchronous, which can only generate commands or messages at a specified time, and asynchronous, which can generate commands at user discretion (Lim et al. 2017). Researchers working in the field of brain-computer interfaces usually take advantage of different machine learning techniques to improve the Information Transfer rate (ITR) and the classification accuracy of their proposed system leading to enhancing the overall performance of their BCI. Figure 1 presents a further breakdown of the aforementioned categories and provides an overview of the proportion of each signal acquisition method in the existing BCIs developed for LIS and CLIS patients.

Fig. 1.

The proportion of each signal acquisition method in the existing brain-computer interfaces developed for locked-in and completely locked-in patients

Non-invasive BCIs: The primary means of communication with LIS patients

Non-invasive BCIs provide us a chance with an effective means of communication with LIS patients. Electroencephalography (EEG), Near-Infrared spectroscopy (NIRS), Functional Magnetic Resonance Imaging (fMRI), and Magnetoence-phalography (MEG) are among the neuroimaging methods that work in a non-invasive manner. EEG and NIRS being portable have made them the most suitable signal acquisition methods to be used in BCIs developed for everyday use. Some cases of using fMRI and MEG are observed in studies related to BCIs for LIS patients. Still, they mostly serve as methods that provide complementary information that is not directly related to the application of the BCI systems.

EEG-based BCIs

The discovery of electroencephalography by Hans Berger (Berger 1929) in the 1920s was the very first step toward the realization of the human dream of communicating via brain signals. About half a century later, since the coining of the term “Brain-Computer Interface” by Jacques Vidal on the EEG-based technology that he introduced in 1973 (Vidal 1973), BCI technology has evolved in so many ways by taking advantage of different signal acquisition methods; but yet the EEG-based BCI is the most frequently used system by the researchers working in this field. This can be due to the inexpensiveness, portability, and high temporal resolution of electroencephalography (Minguillon et al. 2017; Cruse et al. 2011).

EEG-based BCIs can be classified into two broad types: exogenous BCIs and endogenous BCIs. In this system of classification, the BCIs fall into one of these two categories based on whether there exists any external stimulation to trigger the system or not. Figure 2 shows different paradigms and neurophysiologic signals used in each category. In the following subsections, each paradigm or signal and its corresponding role in developing BCIs specifically used for LIS and CLIS patients will be discussed in greater detail. In addition, the number of electrodes in each study and some details on the machine learning algorithms used in the EEG-based BCIs reported in this article are presented in the form of tables.

Fig. 2.

Different paradigms and neurophysiologic signals used in electroencephalography-based brain-computer interfaces developed for locked-in and completely locked-in patients

Exogenous BCIs

Event-related potential (ERP) and evoked potential (EP)

Exogenous BCIs consist of systems based on Event-Related Potentials (ERPs) or Evoked Potentials (EPs). The term ‘Event-Related Potential’ has come to be used to refer to an electrical brain response recorded via electroencephalography, time-locked to a particular stimulus or event. (Charman et al. 2013) The main difference between ERP and EP is that ERPs are responses to individual stimuli, whereas EPs are responses to the whole stimulation period (Kotlewska et al. 2017). The precise definition of these two terms varies in the literature, and there exists terminological confusion; thus, it can be seen that they are often used interchangeably (Panoulas et al. 2010). Stimuli of either Visual, Auditory, or Tactile nature can trigger exogenous BCIs. P300 and Steady-State Visual Evoked Potential (SSVEP) are amongst the most common paradigms used for communication with locked-in patients.

P300 A very common and practical ERP utilized in brain-computer interfaces is P300, which is a positive deviation in voltage that can be observed around 300 ms after the triggering event (Arvaneh et al. 2019). P300 can be used as a reliable marker of conscious processing in patients with locked-in syndrome (Lugo et al. 2016; Morlet et al. 2017). This method has been used for activities such as writing E-mails, speech generation, environmental control, and pursuing a science career (Sellers et al. 2010). The P300-based BCIs are founded on the concept of the “oddball paradigm” (Farwell and Donchin 1988). In the oddball paradigm, two types of stimuli, target and standard, are randomly presented to subjects. The presentation of the target can evoke a P300 event-related potential. The less anticipated a target stimulus is, the greater is the amplitude of generated P300 component (Li et al. 2019).

The first brain-computer interface developed using the P300 component of the ERP was proposed by Farwell and Donchin in 1988 (Farwell and Donchin 1988). Seven years later, in 1995, Gil et al. (Gil et al. 1995) were amongst the first to conduct an experiment to study the P300 wave in a group of 20 patients with Amyotrophic Lateral Sclerosis (ALS). Event-related potentials and their P300 component are often used as an effective means of assessing the cognitive functions in severely paralyzed patients or complete locked-in patients, even at their homes and their bedside (Chatelle et al. 2018; Schnakers et al. 2009; Kotchoubey et al. 2003; Neumann and Kotchoubey 2004; Onofrj et al. 1997). Although these studies have shown that there can be seen significantly longer P300 latency and a subtle cortical cognitive dysfunction in patients with ALS, P300 is still one of the most commonly used paradigms in BCIs developed for communication with LIS patients, and it has been proved to be beneficial in improving the quality of life of LIS patients. This is almost certainly due to the preserved cognitive abilities, including adequate language comprehension and signs of having normal to high information processing capacity in most of the locked-in patients (Kotchoubey et al. 2003). To date, several studies have confirmed the effectiveness of the P300 paradigm in communication with LIS patients.

In 2010, Sellers et al. (Sellers et al. 2010) developed a BCI for independent home use of severely motor-disabled patients and tested the system on a patient with advanced ALS. The subject participating in this study had been using a gaze-controlled device for communication and also running his NIH-funded research laboratory before his eye movement was weakened. As the disease progressed and the device he was using became unreliable, he entered this experiment and went through 2.5 years of using the P300-based BCI proposed in this study. In this experiment, the BCI user had to face a video monitor in which a 72-item matrix was presented. In order to make a selection from this matrix, the patient had to attend to the desired item and count the number of flashes. In this study, the authors reported that the patient and his caregivers were satisfied with the system, they could interact with each other well, and it provided the patient with the means to continue his successful career in scientific research.

In 2012, Mak et al. (Mak et al. 2012) conducted research to identify the EEG features that best correlate with P300-based BCI performance in ALS patients. In their study, 20 ALS subjects used a P300 speller (Farwell and Donchin 1988; Rezeika et al. 2018), and it was reported that the root-mean-square amplitude, the negative peak amplitude of the target ERP and theta rhythm of EEG (4.5–8 Hz) are good predictors of P300 BCI.

In 2014, Oken et al. (Oken et al. 2014) used the RSVP keyboard, which is a paradigm for rapid serial visual presentation, proposed in an earlier study by themselves (Orhan et al. 2012). It is a presentation technique in which visual stimuli are displayed as a temporal sequence at a fixed location on the screen and with arbitrarily large sizes if needed. They used the RSVP keyboard to fuse a real-time statistical language prediction model with an EEG classifier for spelling and expressive communication by the 9 LIS patients participating in this study. Six out of nine patients were in incomplete LIS, two were in classical LIS stage, and one was in complete locked-in state. It was reported that subjects with incomplete LIS might benefit more from their proposed system.

In a study conducted by Holz et al. (Holz et al. 2015) in 2015, a visual P300-based BCI was used for a woman with ALS in a locked-in state with only residual eye movement. They used a BCI with Brain Painting application introduced earlier by Zickler et al. (Zickler et al. 2013), with which the user can choose different tools for painting from a six by eight P300 matrix, and the resulting digital painting is displayed on a second monitor. The LIS patient participated in this study and painted with the BCI several times weekly, resulting in 200 sessions within 14 months. In this study, the effect of the BCI-controlled Brain Painting on the patient’s QoL was assessed, and she expressed her satisfaction because she had a result “in front of her eyes” and it could make her “happy and free.” She just complained about being dependent on her family and caregivers to set up the BCI for her, and she stated that “I am using Brain Painting 1 to 3 times per week, but if I could, I would use it every day.”

In a study conducted by Guy et al. (Guy et al. 2018) in 2018, 20 ALS patients went through an experiment to use a P300 speller BCI system. This research aimed not to test the BCI as an augmentative and alternative communication (AAC) system but to test whether ALS patients with multiple deficiencies can use the BCI to communicate in a non-predefined environment. Each patient participated in two P300 speller sessions which lasted 60 to 90 min and consisted of three blocks with incremental complexity: copy spelling (block 1), free spelling (block 2), and free use (block 3). In the first block, participants tried to type two 10-letter words they overtly chose from a list while they were provided with cues and feedback. In the free spelling block, the participants tried to type two 5-letter words or one 10-letter word they covertly chose from a list.

Contrary to the previous block, they received no cues or feedback. In the first and second blocks, the participants were instructed not to correct possible errors to help the accuracy metrics stay homogenous. The third block, the free-use task, which was optional, and the patients could choose whether they wanted to perform the task or not depending on their fatigue level and motivation, the participants could use the P300 speller freely to type what they wished, and it was possible for them to correct their errors. The authors have claimed that the effectiveness of the system was 100% because all patients accomplished blocks 1 and 2 in the two sessions; for efficiency, 95% of participants achieved up to 75% correct symbol selection (up to 95% for 65% of participants), and the mean satisfaction score was 8.7/10 for usefulness, ease of use and comfort of use.

Wolpaw et al. (Wolpaw et al. 2018) in 2018 presented a study on a BCI device with the P300 paradigm, which could be used at home independently. In this study, 42 ALS patients participated at the beginning. Still, they got out of the test due to some problems such as death, disease exacerbation, or simply not being interested in continuing the experiment. Finally, 14 patients finished their training. They indicated and claimed that their “Wadsworth BCI system” system can be effective, as patients use them at home.

Lack of voluntary eye movement in unresponsive patients, such as those suffering from a disorder of consciousness (DOC) and CLIS, urged researchers to design BCI systems that do not rely on visual stimuli. To this end, Vibrotactile stimulators were introduced to elicit a P300 response. These stimulators are also useful for loud environments. One of the successful BCIs that use vibrotactile stimulation to communicate with LIS and CLIS patients is the mindBEAGLE system proposed by Guger et al. in 2017 (Guger et al. 2017a) that g.tec Medical Engineering GmbH later commercialized. This system consists of three vibrotactile stimulators and 16 EEG electrodes and works with two paradigms that can be selected one at a time: P300 Evoked Potential (EP) and Motor Imagery. This section covers the details of the mindBEAGLE system with the P300 evoked potential paradigm. The motor imagery paradigm of this system will be covered later in this article in its corresponding section. The mindBEAGLE system performance has been investigated by several research groups (Chatelle et al. 2018; Guger et al. 2017b; Heilinger et al. 2018). In a study conducted by Guger et al. (Guger et al. 2017b), to evaluate the performance of the mindBEAGLE system while working with its P300 paradigm, its command following and communication functions were investigated with 9 LIS and 3 CLIS patients (all 12 diagnosed with ALS) and three healthy controls (Guger et al. 2017b). In the vibrotactile EP paradigm with two tactors called VT2, the left and right wrists are randomly stimulated with these tactors, and the participant has the task of counting the rare stimuli presented to the target wrist. In the vibrotactile EP paradigm with three tactors called VT3, in addition to those stimulators in VT2 mode, an additional stimulator is placed as the distracter on the back or shoulder, and the participant counts stimuli on either the right or left hand. The latter mode allows the participant to answer yes/no questions. In this study, two out of the three CLIS patients managed to use the system to communicate with VT3 with 90% and 70% accuracy. In another study performed by Heilinger et al. (Heilinger et al. 2018) in 2018, the performance of the mindBEAGLE system while using it in EP paradigm mode with 15 LIS patients was evaluated (6 were diagnosed with stroke, and nine were diagnosed with ALS). In this study, the number of questions was ten, and the system was reliable only if the correct answers were more than 7. All patients except one performed above chance level in at least one run in the VT2 paradigm. In the VT3 paradigm, all 6-stroke patients and 8 out of 9 ALS patients showed at least one run above chance level. Overall, patients achieved higher accuracies in VT2 compared to VT3 paradigm. They also claimed that LIS patients due to ALS attained better performance with the BCI compared to those with stroke. The authors concluded that achieving a better performance while working with ALS patients compared to those patients with stroke etiology could be explained by the lesion of the sensory pathways in patients with LIS due to brainstem stroke. They also stated that the improvement of one of the ALS patients while working in VT3 mode shows the effect of the short-term learning. Guger et al. has claimed that mindBEAGLE platform is the first successful EEG-based BCI introduced for the purpose of communication with CLIS patients.

Another paradigm that can be used for late-stage locked-in patients who have lost their gaze control is auditory P300. The authors working with auditory systems have reported that the spelling accuracy is significantly lower in comparison with a similar visual system (Kübler et al. 2009). In a study by Lugo et al. (Lugo et al. 2016) in which 11 LIS patients who were presented with a complex-tone auditory oddball paradigm in passive (listening to the sound) and active (counting the deviant tones) conditions participated, it was reported that locked-in patients show less reliable results when being tested with event-related potentials as compared to healthy subjects, specifically in the passive condition.

Table 2 shows a summary of works about Electroencephalography-based brain-computer interfaces using the P300 paradigm for communication with locked-in and completely locked-in patients.

Table 2.

Electroencephalography-based brain-computer interfaces using P300 paradigm for communication with locked-in and completely locked-in patients (in chronological Order)

Reference No. and year	subjects	Etiology	# Channels	Paradigm/BCI	Feature extraction/Selection method	Classification method	Performance
Sellers et al. (2010)	1 LIS	Advanced ALS	8	Visual P300	–	SWLDA	BCI accuracy: 83% for over 2.5 years
Mak et al. (2012)	20 LIS	Advanced ALS	16	P300/BCI speller	–	SWLDA	Classification Accuracy: 60%
Orhan et al. (2012)	1 LIS		14	RSVP paradigm P300	RDA	–	symbol selection accuracy: 85%
Oken et al. (2014)	9 LIS (6I, 2C, 1 T)	ALS	16	P300/RSVP keyboard	RDA	Naive Bayesian	AUC: 0.71
Holz et al. (2015)	1 LIS	ALS	8	visual P300 BCI/Brain painting	Stepwise fit approach	SWLDA	Psychosocial Impact on patient: competence: 1.50; adaptability: 2.17; self-esteem: 1.50 (scores out of 3)
Lugo et al. )2016)	11 LIS	Stroke	3	Visual P300	Filtered using ICA	–	Three out of seven patients (42.8%) showed the P3 waveform with significant difference target/non target stimuli in the passive condition and five of seven patients (71.4%) in the active condition
Guger et al. (2017a)	3 LIS	ALS	16	Vibrotactile and auditory evoked potentials P300/mindBEAGLE	–	LDA	P4: VT2: 100%, VT3: 90% P5: VT2: 95%, VT3:100% P7: VT2: 90%, VT3: 80%
Guger et al. (2017b)	9 LIS, 3 CLIS	ALS	8	Vibrotactile P300/mindBEAGLE	–	LDA	mean accuracy of 76.6% in VT2 and 63.1% in VT3 2 out of the 3 CLIS patients managed to use the system to communicate with VT3 with 90% and 70% accuracy
Heilinger et al. (2018)	15 LIS	6 stroke, 9 ALS	8	Vibrotactile P300/mindBEAGLE	–	LDA	VT2: median classification accuracy of 98% for ALS and 32.8% for stroke VT3: median classification accuracy of 82% for ALS and 22% for stroke
Guy et al. (2018)	20 LIS	ALS	12	P300/BCI speller	Spatial filters	LDA	Effectiveness: 100% Efficiency: 95% mean satisfaction score: 8.7/10
Chatelle et al. (2018)	1 LIS	Brainstm ischemic stroke	8	Vibrotactile and auditory evoked potentials	–	LDA	AEP: 80% VT2: 100% VT3: 100%

SWLDA stepwise linear discriminant analysis, I incomplete LIS, C classical LIS, T total LIS, RSVP rapid serial visual presentation, RDA regularized discriminant analysis, PCA principal component analysis, AUC area under the curve, CSP common spatial pattern, AEP auditory evoked potentials

2SSVEP

Steady-state visual evoked potential or SSVEP is a phenomenon that can be observed when a subject looks at a light source flickering at a specific frequency (Friman et al. 2007), which leads to an increase in the amplitude of the EEG at flickering frequencies and their harmonics (İşcan and Nikulin 2018). Because SSVEP is an inherent response of the brain, it has some advantages over other EEG-based BCI systems, including low susceptibility to eye movements and blink artifacts, as well as electromyography artifacts. Given that the SSVEP response is at its maximum over medial occipital electrode regions (typically Oz), it is concluded by researchers that its origin is mainly from the primary visual cortex (Russo et al. 2007). Although some researchers have suggested that lateral occipital cortex (LOC) is also involved in the SSVEP task (Norcia et al. 2015), and some have used this conclusion to mount their electrodes on a rather hairless area such as the occipitotemporal area to acquire signals with higher SNR (Floriano et al. 2018). Thus, mounting electrodes on the back of the LIS patient’s head while they are bedridden and depend on the mechanical ventilation system through a tracheostomy might be challenging. To solve this issue, some researchers have come up with a solution to ease the electrode mounting procedure to the occipital area of the patients by using a doughnut/ring-shaped cushion to secure the space between the electrodes and the bed (Lim et al. 2017; Hwang et al. 2017). Table 3 shows a summary of works about Electroencephalography-based brain-computer interfaces using SSVEP paradigm for communication with locked-in and completely locked-in patients.

Table 3.

Electroencephalography-based brain-computer interfaces using SSVEP paradigm for communication with locked-in and completely locked-in patients (in chronological Order)

Reference No. and year	subjects	Etiology	# Channels	Paradigm/BCI	Feature Extraction/Selection Method	Classification Method	Performance
Combaz et al. (2013)	7LIS	6 Brainstem stroke 1 TBI	8	SSVEP-based BCI	Spatial filter then minimum energy combination	Linear SVM	All of the subjects had at least 70% accuracy
Lesenfants et al. (2014)	6LIS	Brainstem stroke	12	Covert SSVEP-based BCI	DFT- PMTM- CCA- LAS/ DSLVQ	LDA—linear SVM	(LIS) 2 out of 6 above chance level 1 out of 6 communicated online
Lim et al. (2017)	3LIS	ALS	3	SSVEP-based brain switch	Extraction of power values at the stimulation frequency	SVM	LIS patients were able to call their guardian in an average of 6.56 s
Hwang et al. (2017)	5ALS	ALS	3	Four-class SSVEP-based BCI	–	–	Mean classification accuracy was 76.99%

TBI traumatic brain injury, DFT discrete-time fourier transform, PMTM multi-tapers spectral analysis, CCA canonical correlation analysis, LAS locked-in analyzer system, DSLVQ distinction sensitive learning vector quantization, LDA, linear discriminant analysis, SVM support vector machine

In a study conducted by Combaz et al. (Combaz et al. 2013), the performance of a visual P300-based and an SSVEP-based BCI for mental text spelling on seven incomplete LIS patients were compared. In their proposed SSVEP-based BCI, 64 symbols were divided into four quadrants (4 by 4 characters each) located in the corners of the screen. Each quadrant was flickering with a specific frequency, allowing the subject to select one group of characters by focusing on the quadrant containing the target symbol. Then the selected quadrant was also divided into four new quadrants so that the subject could narrow down the selection. After three successful identification of the subject’s SSVEP response, just one symbol was selected. In their study, all the patients managed to achieve an accuracy of 70% or more with the SSVEP-based BCI, while just three out of seven patients were able to achieve acceptable performance with P300-based BCI. The superiority of SSVEP-based BCI was also due to a lower mental workload and higher overall satisfaction.

Lesenfants et al. (Lesenfants et al. 2011) proposed an SSVEP-BCI based on covert attention, which was independent of neuromuscular function and gaze control. In a follow-up study (Lesenfants et al. 2014), they investigated the feasibility of their proposed BCI, which provides synchronous communication without ocular motor control in a group of LIS patients. They claimed their system could be used as an offline diagnostic tool and/or online communication system for severely brain-injured patients. The BCI was first used in offline mode on a group of healthy subjects to determine the best feature extraction and channel selection algorithms. Then, with the parameters defined in this group, the BCI was applied online on a second group with six LIS patients. In this study, they utilized an interlaced square made of red and yellow LED squares with a white fixation cross in the middle. The flickering frequencies of the yellow and red squares were set to 10 and 14 Hz, respectively. During the training session, LIS patients were exposed to the aforementioned interlaced square pattern, which was continuously flashing, and an equal number of both stimuli was presented in random order. The subjects were instructed to fix his/her gaze on the white cross in the middle and to focus attention on one of the flashing colors. After training the classifier, the subjects underwent an online communication session in which 33 yes/no questions were asked synchronously to the subject and the subjects had to focus their attention over 7 s on the yellow flashes to answer ‘yes’ or on the red for ‘no’. Two LIS patients obtained accuracies above chance level in the training session and one was able to functionally communicate online. Two patients stopped the test due to fatigue.

Lim et al. (Lim et al. 2017) proposed an SSVEP-based brain switch system for LIS patients with severe ALS. Their intention was to help these patients to make an emergency call to their caregiver via a skype call. They claimed that their system could also be used as a switch to help LIS patients turn on or off external devices. For signal acquisition, three EEG electrodes were attached to the patient’s scalp (Oz, O1, and O2), and the chromatic pattern was used as the visual stimulus. The BCI system proposed in this study can be referred to as the simplest form of an asynchronous BCI system. The patients had to concentrate on a target stimulus to activate the turn-on command on the system. The authors have reported that the LIS patients were able to call their guardians in an average of 6.56 s, and they have shown that the proposed system can be used for up to at least four weeks without altering the initial calibration data.

ERP techniques are accompanied by disadvantages, such as their dependence on arousal fluctuations and the necessity of averaging, which assesses ERP components as being dependent on latency jitter. ERP-based studies usually suffer from false-negative responses. Fortunately, they do not lead to false-positive responses, meaning that the ERP assessment can underestimate but not overestimate the cognitive abilities of locked-in patients (Kotchoubey et al. 2003). ERP-based BCIs that are dependent on gaze control prevent their use with severely disabled patients. Although visual ERP-based BCIs are being used more frequently in BCI studies and also are more accurate than the auditory and tactile types, they are not feasible for LIS and CLIS patients who have impaired vision or are unable to control their gaze movement (Guger et al. 2017b; Sellers and Donchin 2006; Riccio et al. 2012; Simon and I. Käthner, C.A. Ruf, E. Pasqualotto, A. Kübler, S. Halder 2015; Pokorny et al. 2013; McCane et al. 2014). It is also important to note that BCIs based on the SSVEP technique can evoke seizures in some patients (Lesenfants et al. 2014).

Endogenous BCIs

Motor imagery

Motor imagery (MI) is a cognitive process in which the subject imagines performing a movement without actually involving any muscles. In this state, during which the sensorimotor rhythms (SMR) are modulated, no external motor action or output is present (Mulder 2007; Kubler et al. 2005). MI-based BCIs mostly use power changes in the 8 to 30 Hz frequency bands (mu and beta rhythms) in accordance with the preparation and start and stop of imagined or executed body movements. These phenomena are called event-related desynchronization (ERD) for movement and event-related synchronization (ERS) for stopping movements and are commonly observed in EEG signals recorded from the central area (Maruyama et al. 2020). Common MI tasks are the kinesthetic imagination of left or right hand movements or both feet (Scherer et al. 2015). Table 4 shows a summary of works focused on Electroencephalography-based brain-computer interfaces using Motor Imagery paradigm for communication with locked-in and completely locked-in patients.

Table 4.

Electroencephalography-based brain-computer interfaces using Motor Imagery paradigm for communication with locked-in and completely locked-in patients (in chronological order)

Reference No. and year	Subjects	Etiology	# Channels	Paradigm/BCI	Feature extraction/Selection method	Classification method	Performance
Scherer et al. (2015)	2 LIS	Brainstem stroke	30	Motor imagery	CSP	LDA	MI leads to accuracy > 70%
Guger et al. (2017b)	9 LIS, 3 CLIS	ALS	8	Motor imagery/Mindbeagle	CSP	LDA	mean MI assessment accuracy was about 71%, and 82.5% of questions were answered correctly
Chatelle et al. (2018)	1 LIS		8	Motor imagery Motor action	–	LDA	Motor Imagery = 41.7% Motor Action = 48.3%
Lugo et al. (2019)	5 LIS	Brainstem stroke	32	Motor imagery	ICA	Fisher’s LDA	Participant 1 achieved significant levels of accuracy in the passive (71%) and attempted (64%) feet movements in the theta frequency band Participant 2 showed 64% accuracy in the theta frequency during the sport task
Han et al. (2019)	1 CLIS	ALS	19	Motor imagery	SFFS	RG LDA SVM	Offline: RG, LDA, and SVM. An average classification accuracy of 95% Online: average accuracy of 87.5%

CSP common spatial pattern, LDA linear discriminant analysis, SVM support vector machine, ICA Independent component analysis, SFFS sequential forward feature selection, RG Riemannian geometry

Höhne et al. (Höhne et al. 2014) performed an investigation for two LIS patients among four subjects (patients #3 and #4), where the obtained results showed that the considered cases did not have alpha or beta rhythms in eye-opening and eye-closing conditions. In contrast, they had very atypical EEG signatures. In this study, the third patient could not achieve control over the BCI system. In addition, for their online framework, none of the Meta nor LRP classifiers could have reliability above the chance level for this patient. Meanwhile, the fourth patient could achieve up to 90% control over the BCI system in less than four sessions. He showed very typical EEG activity during the right hand and foot tasks of motor performance attempts, despite being unable to move his legs for over nine years. This patient could achieve these results while having very atypical EEG signatures in the absence of alpha and beta rhythms.

In 2015, Scherer et al. (Scherer et al. 2015) applied a pair-wise approach for combining mental tasks in their proposed BCI, and they claimed that individual selection of cognitive task pairs significantly boosts binary classification of induced EEG patterns and therefore enhances the BCI performance. In this study, as two LIS patients were present amongst other participants, the combination of ‘brain-teaser’ tasks which require problem specific mental work (e.g., mental subtraction or word association), and ‘dynamic imagery’ tasks (e.g., MI or spatial navigation) was used. Mental tasks included word association (WORD), which was generating as many words as possible, beginning with the presented letter in Spanish. Mental subtraction (SUB) was the task of subtracting a random 1-digit number from a randomly selected number between 15 and 30. Spatial navigation (NAV) was the imagination of navigating through a familiar house (flat), focusing on orientation. MI of the right hand (HAND) was the kinaesthetic imagination of repetitively squeezing a hand-sized ball with the right hand. MI of both feet (FEET) was the kinesthetic imagination of repetitive self-paced movements of both feet without actual movement.

The authors reported that a statistically significant difference in performance could be observed depending on the mental task pairs involved. They found that both SUB versus FEET and WORD versus HAND performed significantly better than HAND versus FEET, as well as that WORD versus FEET performed better than NAV versus FEET. They believed that one of the reasons that the use of MI tasks solely is not as efficient as using the pair-wise approach is that users with functional disabilities enjoy motor tasks less than non-motor tasks due to frustration coming from the inability to move and the lack of sensation of body ownership makes the process more difficult for them.

In 2017, Guger et al. (Guger et al. 2017a) proposed the mindBEAGLE system for the purpose of communicating with individuals with Disorder of Consciousness (DOC), LIS, and CLIS. This system consists of three vibrotactile stimulators that work with two paradigms that can be selected one at a time: P300 Evoked Potential (EP), which is covered thoroughly in this paper in the P300 section, and Motor Imagery. In this section, we cover the details of the mindBEAGLE system with motor imagery paradigm reported in a study consisting of 12 LIS and CLIS patients (Guger et al. 2017b). In this study, for training the classifier of the MI paradigm, the patients were taught to randomly imagine a left- or right-hand movement for 4 s, and a single run consisted of 30 imagined movements of each hand. The MI paradigm provided the patients with YES/NO questions that had to be answered by imagining either left- or right-hand movement for 8 s. The staff attempted to communicate only if the classification accuracy was higher than 63.19%, which was the 95% confidence interval for 60 trials using a binomial test. In this study, only 3 out of 12 LIS patients managed to communicate successfully with the MI paradigm. Still, the important point is that this approach allowed 2 out of 3 CLIS patients to communicate. These three patients were able to answer 14 out of 15 questions correctly. One of the superiorities of the MI paradigm of mindBEAGLE over its EP paradigm is that the MI method is able to provide faster communication with LIS patients than VT3 (see P300 section of this article for further explanation on VT3). The time needed to get yes or no question answers for VT3 was 38 s by counting 120 stimuli, while in the MI method only 8 s is needed by imagining either left- or right-hand movement.

In a study conducted by Lugo et al. (Lugo et al. 2019) in 2019, five LIS patients participated in the experiment. The participants were asked to perform different motor imagery tasks to induce ERD and ERS patterns. Three MI tasks were announced by verbal cues, which were “sport,” “navigation,” and “feet,” and the participants had to imagine performing their favorite sport, navigating through their house, and attempting to perform feet dorsiflexion, respectively. A passive condition was also added to make it easier to establish an initial classifier, which was the manual dorsiflexion of the participant’s feet executed by the examiner in an identical protocol to the MI task. The authors reported that only 2 out of 5 patients achieved significant accuracy in the task. However, 4 participants mentioned they were highly uncomfortable with passive feet dorsiflexion. They also evaluated setting up the device negatively due to a large amount of gel, which gave them a “wet head” sensation.

In a study conducted in 2019 by Han et al. (Han et al. 2019) the performance of their proposed BCI for online binary communication with a patient in CLIS was investigated. After the complete loss of motor function, she had not communicated even with her family for more than a year. Her auditory and cognitive functions were evaluated, and after confirming that these abilities were not completely impaired, the motor imagery tasks were performed offline and online. In the offline mode, the patient was asked to perform three different mental tasks: left motor imagery (LMI), tongue motor imagery (TMI), and mental subtraction (MS) that needed imagination of left-hand and tongue movement and subtracting a small number from a three-digit number, respectively. The highest accuracy from the offline mode, which was the combination of LMI and MS, was selected in the online mode. In this study, the target frequency bands for each mental task were different: alpha (8–13 Hz) and low-beta (13–20 Hz) for LMI and TMI, and theta (4–7 Hz) and alpha (8–13 Hz) for MS. The results show that combining motor and non-motor imagery tasks leads to higher classification accuracy than combining two motor tasks.

CSP common spatial pattern, LDA linear discriminant analysis, SVM support vector machine, ICA independent component analysis, SFFS sequential forward feature selection, RG riemannian geometry.

It is worth mentioning that the authors also used the ERP data recorded to assess the cognition of the patient for testing the feasibility of a P300-based BCI and observed that MI-based BCI, with an accuracy of 90% and 5 s time for decision making, outperformed P300-based BCI with the accuracy of 80% and 24 s time for decision making.

Although motor imagery can be helpful for LIS patients' communication and neurorehabilitation purposes, some limitations should be noted. Due to the fact that locked-in patients may have impaired or absent sensory feedback, it might be difficult for them to accurately imagine movement, affecting the quality and effectiveness of motor imagery.

SCP

Slow cortical potentials (SCPs) are represented by the gradual changes in the membrane potentials of the cortical region, which usually last from several hundred milliseconds to several seconds. SCP might be externally triggered or self-induced. Positive SCPs are related to the decreased activity in neurons, whereas negative SCPs are associated with neuronal activity (Khan et al. 2020; Gevensleben et al. 2014; Birbaumer et al. 1990). In SCP-based studies, feedback from the central Cz electrode is usually provided (Kübler and Birbaumer 2008).

Various studies have assessed the efficacy of SCP in brain-computer interfaces developed for LIS patients. Birbaumer et al. (Birbaumer et al. 1999) were the first to develop a BCI based on slow cortical potentials of the EEG to be tested with locked-in patients. They proposed an electronic spelling device to be voluntarily controlled by 2 LIS patients with amyotrophic lateral sclerosis. The subjects were required to produce either negativity or positivity greater than a specific criterion amplitude in random order. Both patients performed better in producing positivity; thus, training for negativity was discontinued. In this study, one of the subjects managed to communicate and write a message at a rate of about two characters per minute, which was an achievement that had not previously existed for such severely paralyzed patients. In a follow-up study (Birbaumer et al. 2000), they conducted the experiment at the patients’ homes. They proposed a Thought Translation Device (TTD) enabled by self-control of SCP of the 5 LIS patients participating in this study. With feedback training, patients learned to voluntarily regulate their SCP and consequently select different letters. The proposed TTD was then used by an ALS patient in an experiment conducted by Hinterberger et al. (Hinterberger et al. 2003), in which other EEG analysis methods were applied to improve spelling accuracy. In this study, perception of the feedback signal produced a positive ERP to facilitate the patient’s ability to enhance an SCP shift with identical positive polarity. Discriminant analysis classification was applied to separate negative and positive SCP amplitude shifts super-positioned with task-specific ERP. The mixed filtering method (MF), which was introduced as using two low-pass filter settings alternately for calculating the feedback of both SCP and ERP components, happened to cause an increase in the patients’ performance online compared to the SCP filter alone.

In a study by Neumann et al. (Neumann et al. 2003), a locked-in patient managed to work with a binary spelling device by learning to produce slow cortical potentials. Karim et al. (Karim et al. 2006) introduced an SCP-based web browser called Descartes naming it after the famous quotation from the philosopher Descartes: “I think, therefore I am!” which bears a special meaning for locked-in patients. This Neural Internet system was the first BCI that enabled a LIS patient diagnosed with ALS to operate a web browser and surf the internet solely by regulating his electrical brain activity.

While SCPs have shown potential for communication and control for locked-in patients, they might not function perfectly in real-world tasks. In other words, SCP modulation during biofeedback training sessions may not directly translate into effective control of external devices or communication systems in real-world scenarios. Therefore, there can be a gap between the learned SCP modulation skills and their practical applications, requiring additional training and adaptation to specific tasks or interfaces.

NIRS

Near-Infrared Spectroscopy or NIRS is a neuroimaging technique that has been used as the second most common modality after EEG to be employed in brain-computer interfaces due to its non-invasive nature and low cost. In the NIRS technique, the near-infrared range (700 to 1000 nm) of the light spectrum harmlessly penetrate several centimeters beneath the scalp into the tissue before being absorbed or reflected. Measurement of the wavelengths of the reflected light provides information on the degree to which various chromophores or molecular units that absorb light are present in the tissue. Oxy/De-oxy haemoglobin is among the chromophores that help us investigate cerebral oxygenation changes and thus measure functional activation in cortical areas (McGrath 2011). Reasonable spatial resolution (about 1 cm), good temporal resolution (about one millisecond), being robust to motion artifacts and, being somehow portable make NIRS-based BCIs suitable for use at the bedside of LIS patients and help with their everyday tasks (Strangman et al. 2002).

In a study conducted by Naito et al. (Naito et al. 2007), the report on using NIRS with the purpose of detecting the intentions of patients with CLIS was presented. The method used in their study utilized changes in blood volume at the frontal lobe accompanied by changes in brain activity. In their experiment, when a patient was asked a question and the answer was 'yes,' the patient made his or her brain active by calculating mentally or by singing in his/her head as fast as possible. The patient kept relaxing when the answer was ‘no.’ The authors stated that their method applied only to 40% of CLIS patients.

In a study conducted by Gallegos-Ayala et al. (Gallegos-Ayala et al. 2014) a CLIS patient participated in a NIRS-based experiment. This module was successfully used to investigate the functional activations in the cortex of a CLIS patient in response to auditorily presented stimuli containing correct or incorrect statements and open questions. The hemodynamic change in the motor cortex of the CLIS patient was recorded across many sessions spread over more than a year and was used to train a classifier to predict the “yes” and “no” answering pattern of the CLIS patient. They claimed that they had obtained significantly above chance-level answers in a CLIS patient over an extended period of time and the overall performance of 76.30% in the last training period and the 100% correct performance in some sessions suggests that Metabolic BCIs might thus break the unbearable “silence” of CLIS.

In spite of the useful applications of NIRS for LIS patients, it should be noted that the quite low spatial resolution and the limited channel coverage of this neuroimaging technique may affect the accuracy and specificity of these systems for precise communication or control that can be solved by combining this technique with other modalities such as EEG. Furthermore, NIRS signals are susceptible to contamination from various sources, including movement artifacts or external light sources, limiting this technique's use in everyday applications.

fMRI and MEG

Functional magnetic resonance imaging or fMRI is a neuroimaging technique that demonstrates regional, time-varying changes in brain metabolism resulting from task-induced cognitive state changes or unregulated processes in the resting state (Glover 2011). fMRI technique is mostly utilized as a tool for clinical diagnostic purposes such as distinguishing LIS from other severe conditions like vegetative state (Roquet et al. 2016), as an additional tool for studying volitional brain activity (Boly et al. 2007) in BCI-related studies or for localization of target areas for implanted electrodes for BCI purposes. Monti et al. (Monti et al. 2010) did research in 2010 with 54 patients having consciousness disorders (none of them were LIS) and their results showed only five cases could deliberately modulate their brain activity. Only one of their patients was able to answer “yes” and “no” questions. However, that patient was not able to start a communication. Magnetoencephalography or MEG is the measurement of the magnetic field generated by the electrical activity of neurons (Singh 2014). After reviewing articles with LIS patients, MEG appears to be served as a means of evaluating functional recovery following the locked-in syndrome caused by stroke (Silver et al. 2006). With the emergence of portable BCIs that can be used at the bedside of the patients, both fMRI and MEG are less welcomed by LIS patients due to their bulky equipment (Chaudhary et al. 2015).

Semi-invasive BCIs based on ECoG

Recently, subdural electrocorticography or ECoG is getting more attention as a signal acquisition method for BCI applications. In this method, the signals are recorded from the surface of the brain and the sensorimotor cortex is usually used as the source of signal for BCI control. The high spatial, spectral, and temporal fidelity of ECoG signal makes it a promising platform for real-time BCI communication (Brunner et al. 2011a; Moses et al. 2019). In ECoG signal acquisition, no electrode penetrates the brain and consequently minimal damage to the cortex beneath the implant and to the neurons occurs (Graimann et al. 2010; Degenhart et al. 2016).

In 2015, a woman with LIS from late-stage ALS underwent through the procedure of implantation of subdural electrodes over the hand region of her left motor cortex and then participated in an experiment designed by Vansteensel et al. (Vansteensel et al. 2016) in which a communication system for the home use of LIS patients was proposed. In this procedure, four subdural electrode strips on which there were four electrodes on each was implanted in her. For 28 weeks after implantation, several computer tasks were used to control the BCI and each setting that led to acceptable control was chosen to be used for the rest of the study. First, the patient practiced activating the motor cortex with the ‘target task’ in which she tried to move her right hand (the hand on the opposite side of the implanted electrodes) in order to move a cursor upward and then relax her hand to move the cursor downward. Then, by performing the ‘ball task’ she tried to regulate the magnitude and timing of her brain signal by moving an image of a ball up and down on the computer screen at specific moments indicated by the computer. After that in the ‘click task’, she learned to select specific items, shown in rows and columns on the screen, by generating “brain clicks,” which are identical to mouse clicks. She managed to make a brain click by trying to move her hand for approximately 1 s and then had to withhold brain clicks until the correct item was highlighted. After 197 days, she started using the entire system without any assistance. The patient achieved the speed of two letters per minute while working with the proposed computer-typing program.

Pels et al. (Pels et al. 2019) investigated the stability of a fully implanted ECoG-based BCI for home use and communication on the same patient that had participated in the experiment conducted by Vansteensel et al. (Vansteensel et al. 2016). In this study, the data was acquired from the cortical surface of the motor and prefrontal cortex and was evaluated for 36 months after implantation. Three tasks were performed on a daily basis by the participant, while her ECoG data was being recorded to evaluate the stability of the BCI from different aspects. The tasks were: (1) a ‘Localizer Task’ with 15 s on and off intervals of rest and attempted hand movement to determine and trace the signal features, (2) a ‘Target Task’ to move a cursor up and down by rest and attempted hand movement, respectively in order to train the participant and also to study BCI performance accuracy and signal features in presence of visual feedback (Vansteensel et al. 2016) and (3) a ‘Baseline Task’ with 3-min recordings of both motor and dorsolateral prefrontal cortex during rest with eyes open to study fluctuations of the base-line signal features over time. They have concluded that despite the small changes in impedance and high-frequency band power, accurate and stable long-ECoG-based BCI control can be achieved in a patient with LIS due to ALS.

In an experiment conducted recently by Freudenburg et al. (Freudenburg et al. 2019) two LIS patients with different etiologies (ALS and brainstem stroke) were implanted with subdural ECoG electrodes over their sensorimotor area. They have demonstrated that baseline low-frequency band (LFB) oscillatory components and changes generated in the LFB power of the sensorimotor cortex by attempted hand movement is different between participants, thus they concluded that the etiology of LIS may affect LFB spectral components in the sensorimotor cortex and change the baseline spectral characteristics significantly. In a research performed by Hill et al. in 2005 after repeating some experiments on healthy and unhealthy subjects using EEG and ECoG methods, the conducted certain procedures that worked good on healthy subjects, did not necessarily indicate successful achievements with unhealthy users (Hill et al. 2006).

Oxley et al. (Oxley et al. 2016) proposed endovascular electrocorticography for the first time in which the brain activity is recorded from within a vein by using a passive stent-electrode recording array (stentrode). In this study, the implantation was performed into a jagular vein and placed adjacent to the brain over the motor cortex. Later in 2023, Mitchell et al. (Mitchell et al. 2023) assessed the safety of this technology on 4 patients with ALS and 1 patient with primary lateral sclerosis (PLS). All the patients had preserved motor cortex activity. The authors have reported that at least 5 attempted movement types were decoded offline and all the patients managed to control a computer with the BCI successfully.

In general, it can be inferred that in contrast to using the traditional arrays which need an open craniotomy procedure which might lead to inflammatory tissue responses, using the endovascular electrocorticography would help to ease the procedure leading to a more efficient approach.

Invasive BCIs based on intracortical microelectrode recording

Leigh R. Hochberg was the first to show in his Ph.D. dissertation (Hochberg 1998) in 1998 that intracortical recordings acquired from a monkey could be utilized for the offline, real-time control of a robot wrist. After years of utilizing intracortical BCIs (iBCIs) for non-human primates (NHPs), the first case that was implanted into humans (Kennedy and Bakay 1998; Kennedy et al. 2000) was the electrode with an insulated gold wire fixed inside a hollow glass cone earlier introduced (Kennedy 1989) by Philip R. Kennedy in 1989. The patient who participated in this clinical study was a locked-in patient with severe ALS and hopefully, she was able to control the neural signals in an on/off manner, which was a promising step towards enabling such patients to communicate with their environment.

In this context, some fundamental researches could be referred that dealt with analysis of intracortical microelectrode recordings for the application of this method in patients. Cristina Marin et al. (Marin 2010) in 2010 and Mehdi Jorfi et al. (Jorfi et al. 2015) in 2014 mentioned some exciting challenges in the biocompatibility of intracortical microelectrodes lied in this intersection of neuroscience research, which might affect intracortical microelectrode performance. Scott F Lempka et al. discussed factors that affected recording quality using theoretical assessment of intracortical microelectrode recordings (Lempka et al. 2011).

In 2012, Hochberg et al. (Hochberg et al. 2012) proposed a BCI, in which brain signals were recorded using a 4 mm × 4 mm, 96-channel microelectrode array. The electrodes were implanted in the arm/hand area of the motor cortex of two tetraplegic and anarthric (unable to speak) patients suffered from a brainstem stroke. These two patients were enrolled in a pilot clinical trial of the BrainGate Neural Interface System (BrainGate2: Feasibility Study of an Intracortical Neural Interface System for Persons With Tetraplegia 2020). Both participants that had no functional arm control, used the neuronal ensemble activity generated by the intended arm and hand movements to control a robotic arm for performing three-dimensional reach and grasp movements. The sensors were implanted in one of the patients with incomplete locked-in syndrome, almost 64 months before the beginning of this study. By the aid of the robotic arm, it became possible for her to drink coffee from a bottle. Although, their study did not show a satisfactory result in terms of pace and accuracy compared to the performing of the robotic reach and grasp activity by the able-bodied participants; but it indicated the possibility of recreation of control of the complex devices years after the central nervous system was injured. The aforementioned incomplete LIS patient has also participated in a study conducted by Bacher et al. (Bacher et al. 2015), which was the first use of an intracortical BCI for neural point-and-click communication. She used the interface to communicate face-to-face with research staff via text-to-speech conversion with both QWERTY and radial virtual keyboards where the latter one yielded a remarkable improvement in typing accuracy and speed and was easier to use for the participant. The system also enabled the participant to use an internet chat application, remotely. Moreover, a similar research by J D Simeral et al. (Simeral et al. 1000) revealed that a neural interface system based on an intracortical microelectrode array could provide repeatable, accurate point-and-click control of a computer interface to an individual with tetraplegia 1000 days after implantation of this sensor.

In a study conducted by Milekovic et al. (Milekovic et al. 2018), two LIS patients with tetraplegia participated and received a 96-channel intracortical multielectrode array. The patients used a BCI system that interfaced with the FlashSpeller text-entry application, which was temporarily relocated to their residence. This system identified their attempts to perform a particular movement based on the patients’ neuronal activity. In this system, local field potentials (LFPs) are used to decode participants’ commands, which are believed to be more stable than neuronal action potentials. As this BCI had an unchanged decoder without any performance drop for 76 and 138 days for the first and second patients participated in this study, respectively, the authors have reported that this system can help LIS patients to write messages over long periods without a decoder recalibration and with minimal technical intervention. In the proposed system in this study, the correct character per minute (CCPM) was 3.07 ± 0.20 CCPM and 6.88 ± 0.33 CCPM for the first and second patient, respectively which means they managed to spell full sentences in a few minutes.

Intracortical microelectrode arrays (MEA) could be used as part of a brain–machine interface system to provide sensory feedback control of an artificial limb to help patients with quadriplegia and locked-in state. According to recent research performed in (2021) by Linda J Szymanski et al. (Szymanski et al. 2021), records of MEAs were implanted into the anterior intraparietal area and Brodmann's area 5 (BA5) of the posterior parietal cortex and the records and stimulation array was implanted in BA1 of the primary somatosensory cortex (S1) in a tetraplegic person.

Recordings of neuronal activity were obtained from all three MEAs despite meningeal encapsulation. However, the S1 array had a greater encapsulation, yielded lower signal quality than the other arrays and failed to elicit somatosensory precepts with electrical stimulation. Histological examination of tissues underlying S1 and BA5 implant sites revealed localized leptomeningeal proliferation and fibrosis, lymphocytic infiltrates, astrogliosis, and unknown body reaction around the electrodes. The BA5 recording site showed focal cerebral microhemorrhages and leptomeningeal vascular ectasia. The S1 site showed focal tissue damage including vascular recanalization, neuronal loss, and extensive subcortical white matter necrosis. The tissue response at the S1 site included haemorrhagic-induced injury suggesting a likely mechanism for reduced function of the S1 implant. Accordingly, the findings are similar to those from animal studies with chronic intracortical implants and suggest that vascular disruption and micro-haemorrhage during device implantation are important contributors to overall array and individual electrode performance and should be a topic for future device development to mitigate tissue responses.

Although intracortical microelectrode arrays have numerous advantages namely high signal-to-noise ratio (SNR) and accurate recordings which reduces the time needed to learn the strategies of controlling external devices, this technique requires surgical operations and there will be potential risks associated with it, inevitably (Hochberg and Donoghue 2006).

Eye tracking/EOG

Eye tracking systems measure gaze which is defined as the point the observer is looking at on a screen or in the world through video recording of eye position (Franchak 2020). The EOG method measures the differences in potential changes induced by eye movements between two electrodes placed either horizontally or vertically around the eyes (Belkhiria and Peysakhovich 2020). Assessing the vertical eye movement and upper eyelid movement in unresponsive individuals is of great importance. Furthermore, pupil size plays a significant role in assessing the cognitive abilities of CLIS patients (Laeng et al. 2012; Granholm et al. 1996; B. J 1977). It is shown that variations in the size of the pupil can indicate subject’s occupation in mental arithmetic (Pedrotti et al. 2014), working memory load (Peavler 1974; Alnaes et al. 2014) and, sexual arousal (Garrett et al. 1989).

In 2013, Stoll et al. (Stoll et al. 2013) reported that pupil size that can be measured by a bedside camera is a possible method to communicate with patients with locked-in syndrome without any training. They claimed that processing pupil dilation provides the severely motor-impaired patients with a means of communication via their pupil size and consequently helps the LIS patients to be understood. In this study, the experimenter read a yes/no question of which the answer was known. A “yes” or “no” answer was read out for each question. Because mental arithmetic changes the pupil size, the participants had to perform the calculation presented in the interval that accompanied the correct answer and ignore the calculation that came with the incorrect answer. The authors reported that with their proposed BCI, up to 90% decoding performance was achieved. They also claimed that this system could even be used for CLIS patients or as an additional diagnosis approach to assessing the state of consciousness in these patients.

In a study conducted by Käthner et al. (Käthner et al. 2015) in 2015, an ALS patient who was in locked-in state underwent an experiment that consisted of using an EOG system, an eye tracker, and an auditory BCI. When the patient entered the experiment, he was able to communicate with slow residual eye movement. For all the systems, the parameters were optimized in a stepwise approach to allow the patients to gain control over them. After comparing the three methods, the authors reported that in terms of ease of use, the patient rated the auditory BCI as the easiest and the EOG-based system as the most difficult. In addition, in terms of fatigue caused by using the systems, the patient rated the auditory BCI as the most tiring while he rated the eye tracking system as the least tiring.

In a recent study in 2020, Tonin et al. (Tonin et al. 2020) developed an auditory speller system based on EOG for ALS patients who are in locked-in state or are transitioning to complete locked-in state. This system can be used for patients that are unable to use eye tracker systems, have lost the ability of gaze fixation, and have low eye-movement amplitude range. In this study, 4 ALS patients in the transition from LIS to CLIS utilized the BCI system to select letters to form words and then full sentences. The patients were asked 20 questions with known answers, of which half of them had “yes” answers and half of them had “no” answers. Patients were trained to move their eyes when the answer was “yes” and not to move their eyes when the answer was “no.” Features of the EOG signals were then extracted to train a binary classifier. This whole process then helped the patients to use the BCI system auditorily to select letters in order to form letters and sentences. In this system, to increase the speed of sentence completion, the speller predicts and proposes words based on the letters previously chosen. The authors reported that the patients with eye-movement amplitude between the range of ± 200 μV and ± 40 μV were able to form complete sentences and communicate independently and freely. However, a follow-up of one year with one of the patients shows the feasibility of the proposed system in long-term use and the correlation between speller performance and eye-movement decay.

Hybrid BCIs

In hybrid BCIs, different modalities are used simultaneously for brain signal acquisition, providing the researchers with the data that might not be achieved in the case of using only a single modality.

The main purpose behind using more than one type of brain signal as an input to the BCI system is to achieve more reliability and avoid the shortcomings of each type of signal. Table 5 is an outline of the prior art of the hybrid system. See also a paper by Amiri et al. (Amiri et al. 2013) for more information on these systems. Table 5 summarizes some of the works done using Hybrid Signals.

Table 5.

Hybrid brain-computer interfaces (in chronological order)

Reference No. and year	Signals type	Purpose
Punsawad et al. (2010)	ERD & EOG	Enhancing the classification accuracy and reducing the training time
Brunner et al. (2011b)	ERD & SSVEP	Adding feedback to BCI applications
Edlinger et al. (2011)	P300 & SSVEP	Used for smart home control
Su et al. (2011)	P300 & ERD	Expand the control function for virtual environment
Riechmann et al. (2011)	P300 & ERD	Enhancing reliability
Fazli et al. (2012)	ERD & NIRS	Enhancing the classification accuracy
Yin et al. (2013)	SSVEP & P300	Enhances the classification accuracy and increases the transfer rate
Li et al. (2013)	SSVEP & P300	Improving the performance of the BCI system in terms of detection accuracy and response time. A wheelchair control system is used for testing
Punsawad et al. (2013)	SSVEP & alpha rhythm	Studying the accuracy of the BCI system during fatigue state. A wheelchair control system is used for testing
Koo et al. (2014)	EOG and P300	Enhancing the classification accuracy and the system response time

More on complete locked-in state

Roald Dahl, the internationally acclaimed children's book author, described complete locked-in syndrome and his communication approach via a BCI-like system in one of his short stories, William and Mary in 1959 (Dahl 1960). In this book which was written almost half a century before the medical community became fully aware of this devastating condition, a conscious man who was devoid of all motor output was connected to the external world by a fictional system in which the brain resides in a basin of artificial cerebrospinal fluid and is still functioning.

In a study conducted by Murguialday et al. (Murguialday et al. 2011), they documented the transition of a patient with ALS from a locked-in state to the completely locked-in state. They clarified the physiological and behavioral boundaries between them. They observed that their subject's last remaining controllable muscles were the eye muscles. This finding was contrary to previous studies, which had suggested that the external anal sphincter is the last remaining controllable muscle in ALS. In the case of CLIS, due to unpredictable circadian rhythms that result in spontaneous sleep and dozing during the day, communication with the patients occurs with a higher error rate (Soekadar et al. 2013). In an experiment performed by Wilhelm et al. (Wilhelm et al. 2006), a CLIS patient was taught to think of lemon or milk and declare what he was thinking about with YES or NO, respectively. A considerable difference in the level of salivary pH between these two conditions was seen, which strongly indicates that verbal perception is still present in CLIS patients. According to Action Perception Theory (Pulvermüller et al. 2014), brain areas responsible for linguistic tasks overlap with those related to movement and motor actions. Neural substrates for thought, language, and movement are intrinsically interwoven and functionally interdependent (Moseley and Pulvermüller 2018). The fact that language-related tasks activate the sensorimotor circuits proposes the hypothesis that motor activation is involved in understanding action words (Rizzolatti and Rozzi 2016). Following this purport, Khalili et al. (Khalili Ardali et al. 2019) hypothesized that the performance of the BCI to questions containing action words in comparison to object words is decreased in CLIS patients. Due to the absence of the majority of communication means, the ideal system to use with CLIS patients is the one that can perfectly discriminate between two different mental states, such as a binary switch that can enable a speller system (Khalili Ardali et al. 2019). In a study conducted by Bensch et al. (Bensch et al. 2014) it was reported that high-frequency band (HFB) power was present during LIS, but the transition to CLIS was accompanied by a sharp drop in this feature.

Conclusion and future directions

The technological advancements in the field of brain-computer interfaces have provided LIS and CLIS patients with the means to restore communication with their surroundings more effectively and improve their quality of life. In this paper, by reporting the successful approaches and pointing out the limitations, the performance of the BCIs applied and tested on LIS and CLIS patients is investigated to lay the groundwork for researchers interested in this field and provide them with a realistic insight into the challenges of designing an efficient BCI for LIS and CLIS patients. The result of this research supports the idea that in the early stages of the emergence of the locked-in syndrome, using gaze related paradigms would result more successfully. As the disease progresses and the patient enters the complete locked-in phase, due to the fact that the eye muscles are gradually losing functionality, auditory or tactile paradigms would perform better. Another implication of this study is that in the final stages of the disease when the patients approach the complete locked-in phase, they are more likely to be willing to use semi-invasive or invasive modalities. Considering the higher spatial resolution of the more invasive approaches, if the patients choose to undergo surgery, they would benefit from a higher SNR leading to more successful applications of BCIs. Although the feasibility of BCI technology for LIS patients is proven, a wide gap should be closed to achieve BCIs that are more efficient for CLIS patients. The emergence of new technologies that involve implanted endovascular BCIs or flexible threads that can be implanted in the brain in a minimally invasive manner shows that the future of BCI technology seems very promising. It is anticipated that these novel technologies can offer propitious solutions to provide CLIS patients with high SNR signals and improve their communication quality while helping to minimize the risk of inflammation of brain tissues by using minimally invasive approaches.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declarations

Competing interests

The authors declare that they have no competing interests.