Abstract
Many studies reported that ERP-based BCIs can provide communication for some people with amyotrophic lateral sclerosis (ALS). ERP-based BCIs often present characters within a matrix that occupies the center of the visual field. However, several studies have identified some concerns with the matrix-based approach. This approach may lead to fatigue and errors resulting from flashing adjacent stimuli, and is impractical for users who might want to use the BCI in tandem with other software or feedback in the center of the monitor. In this paper, we introduce and validate an alternate ERP-based BCI display approach. By presenting stimuli near the periphery of the display, we reduce the adjacency problem and leave the center of the display available for feedback or other applications. Two ERP-based display approaches were tested on 18 ALS patients to: (1) compare performance between a conventional matrix speller paradigm (Matrix-P, mean visual angle 6°) and a new speller paradigm with peripherally distributed stimuli (Peripheral-P, mean visual angle 8.8°); and (2) assess performance while spelling 42 characters online continuously, without a break. In the Peripheral-P condition, 12 subjects attained higher than 80% feedback accuracy during online performance, and 7 of these subjects obtained higher than 90% accuracy. The experimental results showed that the Peripheral-P condition yielded performance comparable to the conventional Matrix-P condition (p > 0.05) in accuracy and information transfer rate. This paper introduces a new display approach that leaves the center of the monitor open for feedback and/or other display elements, such as movies, games, art, or displays from other AAC software or conventional software tools.
Keywords: BCI, ERP, ALS, Accuracy, ITR
Introduction
Brain–computer interfaces (BCIs) have been used to establish a communication pathway directly between human brains and external devices by recognizing voluntary changes in the users’ brain activities. This approach can help severely disabled patients regain some functional independence and freedom to interact with people and their environments (Li et al. 2016; Puanhvuan et al. 2017; Wang et al. 2014; Wolpaw et al. 1991; Yadav and Nicolelis 2017). BCI systems have most often relied on motor imagery (Feng et al. 2018; Zhang et al. 2015), steady state visual evoked potential (SSVEP) (Xu et al. 2016; Yin et al. 2015; Zhang et al. 2014), slow cortical potentials (SCP) (Birbaumer et al. 1999), and the P300 with other event-related potentials (ERP) (Jin et al. 2015; Zhang et al. 2016). The ERP-based BCI was first presented by Farwell and Donchin (1988), and has become one of the most prominent BCI approaches. In last three decades, many methods were explored to improve the ERP-based BCI in terms of classification accuracy and information transfer rate (ITR) (Jin et al. 2011), convenience, and comfort (Jin et al. 2014b). Optimized paradigms and algorithms have been presented to enhance the difference between attended and ignored events, which could entail more recognizable differences in the ERP and/or other components and increase the classification accuracy and ITR (Acqualagna and Blankertz 2013; Jin et al. 2014a; Kaufmann et al. 2013b; Xu et al. 2018). Many other paradigms were presented to meet the requirement of some special groups (Huang et al. 2016, 2018). For example, gaze-independent BCIs were designed for patients with difficulty controlling gaze (Kaufmann et al. 2014).
Amyotrophic lateral sclerosis (ALS) can cause the progressive loss of motor and speech functions due to declining motor control, eventually leaving patients with no or few ways to move and thereby communicate (Lewis and Rushanan 2007). BCIs have become a well-studied technique that could help these patients by providing communication that does not require any movement (Cipresso et al. 2012; Hsu et al. 2016; Kubler et al. 2001, 2005; McCane et al. 2014; Riccio et al. 2013). Sellers and Donchin (2006) tested a four selection ERP-based BCI on three ALS patients and reported that an ERP-based BCI can serve as a non-muscular communication device for ALS patients. Sellers et al. (2010) studied extensive P300 speller use on ALS patients (6–8 h a day for several months). Later work surveyed the performance of a P300 speller using a 36-character spelling system with ALS patients, and demonstrated that P300 spellers could work on ALS patients (Mak et al. 2012; McCane et al. 2015). Mainsah et al. (2015) tested ERP-based BCIs on ALS patients using a dynamic stopping strategy, in which the ALS patients spelled 36 characters online in three sessions. In 2015, a ERP-based BCI that adopted the T9 speller approach was tested on one locked-in ALS patient, who could spell a 23-character long sentence with only one mistake using the T9 speller in 500 s (Ron-Angevin et al. 2015). Holz et al. (2015) presented a brain painting BCI system for long term use with ALS patients. Moreover, face stimuli can yield higher performance with both healthy and patient subjects compared to flashing characters (Jin et al. 2014b; Kaufmann et al. 2013b). Thus, many published studies have sought to validate and improve ERP-based BCIs with ALS patients, although the extent to which these BCIs require control of eye gaze is controversial (Allison et al. 2008; Brunner et al. 2010; Severens et al. 2014).
Some work has addressed sustained use (Holz et al. 2015; Kaufmann et al. 2013a) and gaze independence (Brunner et al. 2010; Severens et al. 2014). However, most studies have explored these issues with matrices or other display elements presented in the center of the visual field. Prior work has shown that displays can rely on peripheral stimuli for ERP-based target selection, but has focused on goals other than spelling. For example, one early study presented an ERP-based BCI with four targets in the four sides of the computer screen and a control object in the middle of the screen (Piccione et al. 2006). Bai et al. (2015) also used several targets around the computer screen to control an explorer. Other work explored a two-monitor approach, with one monitor devoted to the BCI interaction and a second monitor for the application it controls (Martinez-Cagigal et al. 2017).
This paper introduced a new peripheral speller approach to ERP-based BCIs. In this paradigm, the targets were the same size, with the same minimum distance between targets, as a conventional matrix speller paradigm. We explored the performance of the peripheral distribution speller paradigm (Peripheral-P) and the conventional matrix speller paradigm (Matrix-P) when ALS patients participated in continuous spelling tasks. The patients spelled 42 characters online continuously, without a break. As we known, ALS may result in eye movement abnormalities and impair eye gaze (Brunner et al. 2010; Donaghy et al. 2011), but the effect of ALS on visual angle has not been studied yet. Since the visual angle of these two paradigms was different, we also surveyed the effects of the characters’ visual angle.
Materials and methods
Subjects
Eighteen ALS subjects (14 male, 4 female, aged 30 to 70 years, mean 55.1 ± 12.9 years) participated in this study (see Table 1). It has been reported that about 30% of patients with ALS exhibit cognitive impairments (Keller et al. 2015). In our study, we communicated with all ALS patients before the experiment, and all subjects could understand our BCI task and perform the task correctly.
Table 1.
Background information for participants in this study. “CS” is Clinical stage (2A, 2B or 3). “ALSFRS-R” is ALS Functional Rating Scale; “m” reflects the time from disease onset in months. In column EMG, “B” is Bulbar, “C” is cervical, “T” is thoracic, and “L” is lumbar. (The Gender and exact age are not listed, as required by the Ethics and Integrity Guidelines of Frontiers.)
| No | Age range | CS | Type of onset | ALSFRS-R | Duration(m) | EMG |
|---|---|---|---|---|---|---|
| S1 | 50–55 | 2B | Upper limb(C) | 34 | 45 | C,T(2) |
| S2 | 65–70 | 2B | Upper limb(C) | 38 | 21 | C,T(2) |
| S3 | 40–45 | 3 | Lower limb(L) | 20 | 48 | B,L(2) |
| S4 | 30–35 | 3 | Upper limb(C) | 42 | 26 | B,C,T,L(4) |
| S5 | 40–45 | 2A | Upper limb(C) | 47 | 33 | B,C,T,L(4) |
| S6 | 45–50 | 3 | Upper limb(C) | 19 | 56 | B,C,T,L(4) |
| S7 | 30–35 | 3 | Upper limb(C) | 42 | 49 | B,C,T,L(4) |
| S8 | 55–60 | 3 | Upper limb(C) | 27 | 20 | B,C,L(3) |
| S9 | 70–75 | 2B | Upper limb(C) | 30 | 22 | B,C,T,L(4) |
| S10 | 70-75 | 2B | Upper limb(C) | 27 | 12 | B,C,T,L(4) |
| S11 | 60-65 | 2A | Lower limb(L) | 44 | 12 | T,L(2) |
| S12 | 55-60 | 2B | Lower limb(L) | 46 | 12 | T,L(2) |
| S13 | 65-70 | 2B | Upper limb(C) | 33 | 43 | B,C,L(3) |
| S14 | 60-65 | 2A | Upper limb(C) | 37 | 36 | B,C,L(3) |
| S15 | 55-60 | 2B | Bulbar | 39 | 18 | B,C,L(3) |
| S16 | 55–60 | 2B | Upper limb(C) | 40 | 11 | B,C,L(3) |
| S17 | 65–70 | 3 | Lower limb(L) | 25 | 10 | B,C,T,L(4) |
| S18 | 45–50 | 3 | Upper limb(C) | 29 | 24 | B,C,T,L(4) |
| AVG | 55.1 ± 12.9 | 34.4 ± 8.6 | 27.7 ± 15.1 |
Four stages are used to classify ALS patients (Roche et al. 2012). ALS symptoms are mild in Stage 1, whereas ALS patients in Stage 4A need gastrostomy and ALS patients in Stage 4B even require non-invasive ventilation. We selected ALS patients in Stages 2 and 3 (Stage 2A: diagnosis; Stage 2B: involvement of second region; Stage 3: involvement of third region (Roche et al. 2012)). S3, S6, S17 and S18 could not speak clearly and could not walk. Only their families could get some information from them, and we communicated with these patients with the help of their families. We talked directly with the remaining patients. Doctors tested the EMG of bulbar, cervical, thoracic and lumbar to further detail each patient’s remaining abilities, shown in Table 1.
Only one of the subjects (S1) had experience with a BCI. Each subject participated in all sessions within 1 day. All subjects signed a written consent form prior to the experiment. The ethics committee of Huashan hospital approved the consent form and experimental procedure before any of the subjects participated (Ethical Committee Approval #298). In the experiment, the subjects were seated approximately 80 cm away from an LED monitor, which was adjusted to make sure the stimulus matrix from both paradigms was displayed in the center of subjects’ visual field.
Stimuli
We use the term “flash groups” to refer to the method of grouping icons within each flash. There were 12 flash groups in total, and each flash group flashed a different subset of icons. Figure 1a shows the Matrix-P display, a 6 × 7 matrix, presented to all subjects. The cells in the matrix with same number were grouped within one flash group. For example, Fig. 1a shows that the cells in the first flash group are flashing. All the cells contain number “1”.
Fig. 1.
The displays used in the Matrix-P (a) and Peripheral-P (b) conditions. The white numbers, circles and characters in Fig. 1a and b have been added to show when each corresponding stimulus flashed, but these elements were not shown to the subjects. Similarly, the red lines and text have been added to depict the locations of different screen elements, and were not part of the display shown to the subjects. The illustration of the eye and the words “visual angle” on the bottom of b help depict how we estimated the visual angle toward one of the characters. (Color figure online)
Furthermore, the feedback in the Matrix-P display was presented on the right side of the screen. In the Peripheral-P display, feedback was instead presented in the center of the screen, indicated by the white circle in Fig. 1b. This white circle has been added to the display in Fig. 1b; the subject did not see it and thus the center of the screen was available to present feedback. Aside from these differences in the positions of characters and feedback, the two displays were otherwise identical in terms of characters within each flash group, timing of flashes and delays between them, and other details.
Sixteen subjects participated in the experiment in their homes or hospitals, while the remaining two subjects participated in the experiment in our lab. Stimuli were presented to the patients using a laptop computer. The size of the display was 26 cm × 19.5 cm. The target visual angle of Matrix-P was from 1.07° to 9.58° (6.0° ± 2.3°) and the target visual angle of Peripheral-P was from 4.43° to 12.34° (8.8° ± 2.5°). “DH” is “,”, “JH.” is “.”, “SP” is “space”, “BS” is “backspace”, and “No” is “cancel”. We noted that all the visual angles were defined as the angles between the central fixation point (the central white dot in Fig. 1b) and the center of each stimulus.
Experimental protocols
EEG signals were recorded using a g.USBamp and a g.EEGcap (Guger Technologies, Graz, Austria) and active electrodes, sampled at 256 Hz. The g.USBamp uses wide-range DC-coupled amplifier technology in combination with 24-bit sampling, with an input voltage of ± 250 mV and a resolution of < 60 nV. The impedance of the active electrodes was kept below 30 kΩ prior to recording. Data were recorded and analyzed using the BCI platform software package developed by the East China University of Science and Technology. Using the extended international 10–20 system, sixteen EEG electrode positions were placed at F3, Fz, F4, FC1, FC2, C3, Cz, C4, P7, P3, Pz, P4, P8, O1, Oz, and O2, referenced to the right ear, and grounded to FPz. The recorded data were filtered using a high pass of 0.5 Hz and a low pass of 30 Hz, notch-filtered at 50 Hz (Jin et al. 2010; Mainsah et al. 2015).
Each subject participated in six offline runs, then six online runs. Each of these two groups of six runs contained three runs of each of the two conditions (Matrix-P and Peripheral-P), with these two conditions counterbalanced. Both conditions used a “copy spelling task” in the offline and online testing described below, meaning that subjects were told which item was the target before each run. We asked the subjects to spell the letters one by one (from A to the last one). Before each spelling task, a white block will appear around the target to guide the subjects.
The stimulus “on” and “off” time was 100 ms, respectively, which yields an SOA of 200 ms. Each flash reflected each time a stimulus changed from its background to a face. One trial lasted 2.4 s and contained all flashes within one of the 12 flash groups. A trial block referred to a group of trials with the same target. During offline testing, there were 16 trials per trial block and each run consisted of five trial blocks, each of which involved a different target (see Fig. 2). The number of trials averaged in the online runs for each trial block was selected adaptively in each run (Jin et al. 2011). The system stops presenting additional trials if the classifier outputs the same target twice in succession.
Fig. 2.

One run of the online and offline experiments
Each subject first participated in three offline runs, with a 3 min break after each offline run. After all offline runs of the two conditions, subjects were tasked with attempting to select 42 targets (i.e. 42 trial blocks) without interruption during each condition in the online experiment (called one online run). Before each online run, we would ask each subject if he/she had enough rest and was ready for the online run. All the subjects were ready for the online run after the 3 min break. Before each trial block began, a 4 s delay was used as target selection time so that all the subjects had enough time to find the target on the display. After the first trial block, feedback from the preceding trial block (specifically, the target that the classifier selected) was also presented during this 4 s delay.
Feature extraction
Epochs of 800 ms were created (− 100 ms to 700 ms relative to stimulus onset). The pre-stimulus interval of 100 ms was used for baseline correction for single trial. A third-order Butterworth band pass filter was used to filter the EEG signals between 1 and 30 Hz (Jin et al. 2017). The EEG signals were then down-sampled from 256 to 36.6 Hz by selecting every seventh sample from the filtered EEG signals.
Classification scheme
Bayesian linear discriminant analysis (BLDA) is an extension of Fisher’s linear discriminant analysis (FLDA) that avoids overfitting. BLDA was selected because of its demonstrated classification performance in ERP-based BCI applications (Hoffmann et al. 2008). Data acquired offline were used to train the classifier using by BLDA and obtained the classifier model. This model was then used in the online system. The size of the feature vector was Nc × Nt, with Nc denoting the number of electrodes and Nt denoting the number of sample points in one channel, which was 29 points here. The features were extracted from 16 electrodes. There were 2880 offline samples used to train the classifier for each subject and each pattern. The hyper parameters were selected automatically by the toolbox (Hoffmann et al. 2008).
Statistical analyses
We conducted paired-samples t-tests with the independent factor “paradigm” with two levels (Matrix-P and Peripheral-P) and the dependent variable “class” for the ERP peaks, classification accuracy and ITR. The ITR is the most commonly applied BCI performance measure in synchronous BCIs, and the ITR formula considers accuracy, speed, and number of available selections (such as characters) (Wolpaw et al. 2000). All the dependent variables were statistically tested for normal distribution (One-Sample Kolmogorov–Smirnov test). The alpha level was α = 0.05 (significant). In this paper, the online and offline accuracies, online ITRs, online trials for average all satisfy a normal distribution. Since the number of samples was less than 30, bivariate correlation analyses with Spearman coefficients was used to show the correlation between the visual angle and the classification accuracy, between the visual angle and the information transfer rate, between the number of the characters and the classification accuracy, and between the number of the characters and the information transfer rate.
Results
We assessed two conditions with 18 ALS patients to explore the new peripheral speller paradigm in comparison with a conventional matrix speller paradigm. The mean visual angle of Matrix-P is 6.0° and the mean visual angle of Peripheral-P is 8.8°. Figure 3 shows the average ERP amplitudes elicited by the Matrix-P and Peripheral-P conditions. Both paradigms elicited clear N200, P300 and N400 components.
Fig. 3.
The average amplitude of event-related potentials for the Matrix-P (red) and Peripheral-P (gray) conditions. In all panels of this figure, the blue line reflects non-target stimuli, the red or black lines show target stimuli and the red or gray shaded areas are the standard deviation. (Color figure online)
Figure 4 shows the offline classification accuracy of Peripheral-P and Matrix-P after 15-fold cross validation. Table 2 shows the online feedback accuracy, number of trials required to spell all 42 characters, and ITR for both Matrix-P and Peripheral-P. “Feedback accuracy” was calculated based on the online output of the BCI system. There were no significant differences between the two paradigms in accuracy (p > 0.05) and ITR (p > 0.05).
Fig. 4.
Offline classification accuracy. a Is the classification accuracy for each subject based on single trial, b is the average classification accuracy for all subjects across averages of 1–16 trials, and panel c is the average ITR for all subjects across averages of 1–16 trials
Table 2.
Online feedback accuracy, number of trials used for spelling 42 characters and ITR
| Subject | Feedback accuracy (%) | Trials used for 42 targets | ITR (bits/min) | |||
|---|---|---|---|---|---|---|
| Matrix-P | Peripheral-P | Matrix-P | Peripheral -P | Matrix-P | Peripheral -P | |
| S1 | 88.1 | 76.2 | 112 | 127 | 39.6 | 27.5 |
| S2 | 81.0 | 92.9 | 104 | 105 | 37.1 | 46.4 |
| S3 | 81.0 | 85.7 | 110 | 103 | 35.1 | 41.1 |
| S4 | 59.5 | 88.1 | 121 | 120 | 19.5 | 37.0 |
| S5 | 61.9 | 71.4 | 131 | 136 | 19.2 | 23.1 |
| S6 | 64.3 | 47.6 | 133 | 138 | 20.0 | 12.1 |
| S7 | 78.6 | 83.3 | 110 | 119 | 33.4 | 33.9 |
| S8 | 100.0 | 97.6 | 94 | 91 | 60.2 | 58.9 |
| S9 | 90.5 | 97.6 | 95 | 95 | 49.0 | 56.4 |
| S10 | 85.7 | 100.0 | 104 | 99 | 40.7 | 57.2 |
| S11 | 81.0 | 69.0 | 107 | 121 | 36.0 | 24.6 |
| S12 | 90.5 | 92.9 | 95 | 102 | 49.0 | 47.8 |
| S13 | 92.9 | 81.0 | 100 | 112 | 48.7 | 34.4 |
| S14 | 83.2 | 92.9 | 98 | 99 | 41.1 | 49.2 |
| S15 | 81.0 | 85.7 | 125 | 119 | 30.9 | 35.6 |
| S16 | 95.2 | 100 | 107 | 91 | 47.7 | 62.2 |
| S17 | 71.4 | 61.9 | 115 | 128 | 27.4 | 19.6 |
| S18 | 90.5 | 78.6 | 105 | 103 | 44.3 | 35.6 |
| AVG | 82.0 ± 11.5 | 83.5 ± 14.2 | 109.2 ± 11.9 | 111.6 ± 14.9 | 37.7 ± 11.4 | 39.0 ± 12.1 |
| p > 0.05 | p > 0.05 | p > 0.05 | ||||
There were 11 different visual angles in Matrix-P and 12 different visual angles in Peripheral-P, relative to the center of the display for each paradigm. Table 3 shows the number of targets for each visual angle in each paradigm. Figure 5 shows the online feedback accuracy and ITR across different visual angles. Figure 5 indicates that the changes in visual angle did not affect BCI accuracy or ITR in our study. Since the number of samples was less than 30, we used bivariate correlation analyses with Spearman coefficients to explore the correlation between online feedback accuracy and visual angle. There were no significant correlations between online feedback accuracy and visual angle (r = − 0.446, p > 0.05 for Matrix-P, r = − 0.067, p > 0.05 for Peripheral-P), and between ITR and visual angle (r = − 0.455, p > 0.05 for Matrix-P, r = − 0.119, p > 0.05 for Peripheral-P).
Table 3.
The number of targets for across different visual angles in each paradigm
| Matrix-P | Peripheral-P | ||
|---|---|---|---|
| Visual angle (°) | Number of targets | Visual angle (°) | Number of targets |
| 1.07 | 2 | 4.43 | 2 |
| 2.72 | 4 | 5.28 | 4 |
| 3.36 | 2 | 6.21 | 4 |
| 4.08 | 4 | 7.2 | 4 |
| 5.28 | 4 | 7.41 | 4 |
| 5.5 | 2 | 9.37 | 4 |
| 6.21 | 8 | 9.51 | 4 |
| 7.62 | 4 | 10 | 2 |
| 7.69 | 4 | 10.2 | 4 |
| 8.2 | 4 | 11.59 | 2 |
| 9.58 | 4 | 11.72 | 4 |
| 12.34 | 4 | ||
Fig. 5.
Online feedback accuracy and ITR for different visual angles for the Matrix-P and Peripheral-P conditions
We also explored ERP activity and classification accuracy over sustained use. Figure 6 shows the offline ERP differences between the first five characters and the last five characters of each run for both paradigms. Based on visual inspection, major differences between these ERPs are not apparent in either paradigm.
Fig. 6.
The offline average ERP amplitude difference between first five target characters and last five target characters for the Matrix-P and Peripheral-P conditions
Online BCI performance did decline slightly with continuous use in this study. Figure 7 shows the online feedback accuracy and ITR across the 42 targets in the chronological order, presented to subjects. In Matrix-P, four of the first 21 characters were classified below 80% (86.0 ± 7.8%), while 11 of the last 21 characters were classified below 80% (78.0 ± 12.5%). In Peripheral-P, six of the first 21 characters were classified below 80% (86.5 ± 7.4%), whereas ten of the last 21 characters were classified below 80% (80.4 ± 10.8%). In the Matrix-P paradigm, the ITR was at least 30 bits/min across all of the first 21 characters (39.1 ± 6.1 bits/min), whereas nine of the last 21 characters were below 30 bits/min (33.0 ± 8.7 bits/min). In the Peripheral-P paradigm, two of the first 21 characters were below 30 bits/min (40.6 ± 6.3 bits/min), and 6 of the last 21 fell below 30 bits/min (34.8 ± 7.6 bits/min).
Fig. 7.
Online feedback accuracy and ITR for 42 targets in chronological order
We conducted bivariate correlation analyses using Pearson coefficients between both measures of online performance (accuracy and ITR) and character order (the order in which the characters were presented to the patients). There were significant negative correlations between accuracy and character order (r = − 0.373, p < 0.05 for Matrix-P and r = − 0.395, p < 0.01 for Peripheral-P), and between ITR and character order (r = − 0.385, p < 0.05 for Matrix-P and r = − 0.499, p < 0.01 for Peripheral-P). The results indicate that sustained use did impair BCI performance.
Discussion
While early BCI work with ALS patients did not use ERP-based BCIs (Kuebler et al. 1998), ERP-based BCIs have since become more prominent tools for ALS patients, due to their high classification accuracy and ITR, reliability with most users, and very low training requirements. As of 2015, 56% of BCI studies on ALS patients used ERP-based BCIs (Marchetti and Priftis 2015).
In this paper, a new ERP-based BCI system with peripherally distributed stimuli was introduced, in which feedback was presented in the middle of the screen. Results showed that this Peripheral-P approach was comparable to the Matrix-P in accuracy and information transfer rate. With additional work, this peripheral approach could be incorporated with other display elements in the center to create a more practical interface for some applications and goals.
Visual angle effect
We considered the impact of the difference in visual angle between the two ERP-based BCI paradigms, which might influence gaze dependence and fatigue. The mean visual angle of Peripheral-P was 2° larger than that of Matrix-P. Table 2 showed that there were no significant differences (all p > 0.05) between these two paradigms in feedback accuracy or ITR. Figures 4 and 5 concordantly showed no significant differences in offline classification accuracy, online classification accuracy, and ITR, as well as no clear feedback accuracy and ITR trends that correspond to any difference between the paradigms. These results indicate that the visual angle did not affect the performance of this ERP-based BCI for these ALS patients when the visual angle of the stimulus was within 12.34° of the center, which was the largest visual angle used in our study.
Continuous use of BCI
We were not aware of work that assessed the performance of an ERP-based BCI system when more than 30 characters were tested without any pause in this fashion. In this study, ALS patients could spell 42 characters online, requiring more than 7 min (420 s). Figure 7 and the associated correlation analyses show that accuracy and ITR did decline slightly during the second half of the run. Our results with continuous use indicate that spelling for about seven minutes with this approach is feasible. In future studies, to further improve the performance of our BCI speller, we will perform additional studies that adopt adaptive algorithms to generalize the classifiers over time, and customize the duration of time breaks between runs according to each subject’s mental state.
Patient variability
The patients in this study had some residual motor control. This may seem to reduce the relevance of this paper, since these patients might be able to use other augmentative or assistive communication (AAC) to communicate instead of a BCI. However, ALS patients in Stages 2 and especially 3 may find speech, eye movements, or other muscle activity fatiguing, and might switch to a BCI when they are tired (Allison et al. 2012; Leeb et al. 2011; Millan et al. 2010). Other AAC systems may be impractical for various reasons; notably, several of our subjects did not consistently use AAC to communicate. Thus, even for ALS patients like those we studied here, BCIs could be helpful, especially if BCIs continue to improve in terms of portability, cost, speed, accuracy, flexibility, and ease of use. Patients in Stage 4 may have difficulty with this approach, and we are considering modifications that might reduce gaze dependence or fatigue such as a gaze-independent BCI.
Subjective report
We wish to note a possible flaw in our study, as well as a suggestion for future research. We were concerned that patients may be fatigued at the end of the recording session. Thus, we did not administer a short questionnaire immediately after data recording, as we have done in some earlier studies with healthy patients. We did attempt to contact most patients after data recording to ask about their preferences, but only a few were available.
Conclusion
Two ERP-based BCI systems were tested on eighteen ALS patients with moderate disabilities to explore the performance of Peripheral distribution speller paradigm. Results showed that the ALS patients in this study were able to use the system effectively, with performance roughly comparable to a conventional matrix display. This approach may be a viable option for users who prefer to leave the center of the monitor for the BCI’s feedback and/or other display elements. This is an initial validation of this new approach with patients, and many issues need additional research. In future work, we will further assess continuous use as well as users’ needs and preferences; some users may not like long sessions or certain display approaches. We will also explore improvements to the display, signal processing, and adaptive user interaction.
Acknowledgements
This work was supported by the National key research and development program 2017YFB13003002. This work was also supported in part by the Grant National Natural Science Foundation of China, under Grant Nos. 61573142, 61773164 and 91420302, and the programme of Introducing Talents of Discipline to Universities (the 111 Project) under Grant B17017.
Footnotes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Yangyang Miao and Erwei Yin are contributed equally to this work.
Contributor Information
Yan Chen, Email: chhyann@163.com.
Jing Jin, Email: jinjingat@gmail.com.
References
- Acqualagna L, Blankertz B. Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP) Clin Neurophysiol. 2013;124:901–908. doi: 10.1016/j.clinph.2012.12.050. [DOI] [PubMed] [Google Scholar]
- Allison BZ, McFarland DJ, Schalk G, Zheng SD, Jackson MM, Wolpaw JR. Towards an independent brain-computer interface using steady state visual evoked potentials. Clin Neurophysiol. 2008;119:399–408. doi: 10.1016/j.clinph.2007.09.121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Allison BZ, Leeb R, Brunner C, Muller-Putz GR, Bauernfeind G, Kelly JW, Neuper C. Toward smarter BCIs: extending BCIs through hybridization and intelligent control. J Neural Eng. 2012;9:013001. doi: 10.1088/1741-2560/9/1/013001. [DOI] [PubMed] [Google Scholar]
- Bai LJ, Yu TY, Li YQ. A brain computer interface-based explorer. J Neurosci Methods. 2015;244:2–7. doi: 10.1016/j.jneumeth.2014.06.015. [DOI] [PubMed] [Google Scholar]
- Birbaumer N, et al. A spelling device for the paralysed. Nature. 1999;398:297–298. doi: 10.1038/18581. [DOI] [PubMed] [Google Scholar]
- Brunner P, Joshi S, Briskin S, Wolpaw JR, Bischof H, Schalk G. Does the ‘P300’ speller depend on eye gaze? J Neural Eng. 2010;7:056013. doi: 10.1088/1741-2560/7/5/056013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cipresso P, et al. The use of P300-based BCIs in amyotrophic lateral sclerosis: from augmentative and alternative communication to cognitive assessment. Brain Behav. 2012;2:479–498. doi: 10.1002/brb3.57. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Donaghy C, Thurtell MJ, Pioro EP, Gibson JM, Leigh RJ. Eye movements in amyotrophic lateral sclerosis and its mimics: a review with illustrative cases. J Neurol Neurosurg Psychiatry. 2011;82:110–116. doi: 10.1136/jnnp.2010.212407. [DOI] [PubMed] [Google Scholar]
- Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol. 1988;70:510–523. doi: 10.1016/0013-4694(88)90149-6. [DOI] [PubMed] [Google Scholar]
- Feng JK, et al. Towards correlation-based time window selection method for motor imagery BCIs. Neural Netw. 2018;102:87–95. doi: 10.1016/j.neunet.2018.02.011. [DOI] [PubMed] [Google Scholar]
- Hoffmann U, Vesin JM, Ebrahimi T, Diserens K. An efficient P300-based brain-computer interface for disabled subjects. J Neurosci Methods. 2008;167:115–125. doi: 10.1016/j.jneumeth.2007.03.005. [DOI] [PubMed] [Google Scholar]
- Holz EM, Botrel L, Kübler A. Independent home use of Brain Painting improves quality of life of two artists in the locked-in state diagnosed with amyotrophic lateral sclerosis. Brain Comput Interfaces. 2015;2:117–134. [Google Scholar]
- Hsu HT, et al. Evaluate the feasibility of using frontal SSVEP to implement an SSVEP-based BCI in young, elderly and ALS groups. IEEE Trans Neural Syst Rehabil Eng. 2016;24:603–615. doi: 10.1109/TNSRE.2015.2496184. [DOI] [PubMed] [Google Scholar]
- Huang MQ, Daly I, Jin J, Zhang Y, Wang XY, Cichocki A. An exploration of spatial auditory BCI paradigms with different sounds: music notes versus beeps. Cogn Neurodyn. 2016;10:201–209. doi: 10.1007/s11571-016-9377-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Huang MQ, Jin J, Zhang Y, Hu DW, Wang XY. Usage of drip drops as stimuli in an auditory P300 BCI paradigm. Cogn Neurodyn. 2018;12:85–94. doi: 10.1007/s11571-017-9456-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jin J, et al. P300 Chinese input system based on Bayesian LDA. Biomed Tech. 2010;55:5–18. doi: 10.1515/BMT.2010.003. [DOI] [PubMed] [Google Scholar]
- Jin J, Allison BZ, Sellers EW, Brunner C, Horki P, Wang XY, Neuper C. An adaptive P300-based control system. J Neural Eng. 2011;8:036006. doi: 10.1088/1741-2560/8/3/036006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jin J, Allison BZ, Zhang Y, Wang XY, Cichocki A. An ERP-based BCI using an oddball paradigm with different faces and reduced errors in critical functions. Int J Neural Syst. 2014;24:1450027. doi: 10.1142/S0129065714500270. [DOI] [PubMed] [Google Scholar]
- Jin J, Daly I, Zhang Y, Wang XY, Cichocki A. An optimized ERP brain-computer interface based on facial expression changes. J Neural Eng. 2014;11:036004. doi: 10.1088/1741-2560/11/3/036004. [DOI] [PubMed] [Google Scholar]
- Jin J, Sellers EW, Zhou S, Zhang Y, Wang X, Cichocki A. A P300 brain-computer interface based on a modification of the mismatch negativity paradigm. Int J Neural Syst. 2015;25:1550011. doi: 10.1142/S0129065715500112. [DOI] [PubMed] [Google Scholar]
- Jin J, Zhang HH, Daly I, Wang XY, Cichocki A. An improved P300 pattern in BCI to catch user’s attention. J Neural Eng. 2017;14:036001. doi: 10.1088/1741-2552/aa6213. [DOI] [PubMed] [Google Scholar]
- Kaufmann T, Holz EM, Kubler A. Comparison of tactile, auditory, and visual modality for brain-computer interface use: a case study with a patient in the locked-in state. Front Neurosci. 2013;7:129. doi: 10.3389/fnins.2013.00129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kaufmann T, Schulz SM, Koblitz A, Renner G, Wessig C, Kubler A. Face stimuli effectively prevent brain-computer interface inefficiency in patients with neurodegenerative disease. Clin Neurophysiol. 2013;124:893–900. doi: 10.1016/j.clinph.2012.11.006. [DOI] [PubMed] [Google Scholar]
- Kaufmann T, Herweg A, Kubler A. Toward brain-computer interface based wheelchair control utilizing tactually-evoked event-related potentials. J Neuroeng Rehabil. 2014;11:7. doi: 10.1186/1743-0003-11-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Keller J, et al. Eye-tracking controlled cognitive function tests in patients with amyotrophic lateral sclerosis: a controlled proof-of-principle study. J Neurol. 2015;262:1918–1926. doi: 10.1007/s00415-015-7795-3. [DOI] [PubMed] [Google Scholar]
- Kubler A, Kotchoubey B, Kaiser J, Wolpaw JR, Birbaumer N. Brain-computer communication: unlocking the locked in. Psychol Bull. 2001;127:358–375. doi: 10.1037/0033-2909.127.3.358. [DOI] [PubMed] [Google Scholar]
- Kubler A, et al. Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology. 2005;64:1775–1777. doi: 10.1212/01.WNL.0000158616.43002.6D. [DOI] [PubMed] [Google Scholar]
- Kuebler A, Kotchoubey B, Salzmann HP, Ghanayim N, Perelmouter J, Homberg V, Birbaumer N. Self-regulation of slow cortical potentials in completely paralyzed human patients. Neurosci Lett. 1998;252:171–174. doi: 10.1016/s0304-3940(98)00570-9. [DOI] [PubMed] [Google Scholar]
- Leeb R, Sagha H, Chavarriaga R, Millan Jdel R. A hybrid brain-computer interface based on the fusion of electroencephalographic and electromyographic activities. J Neural Eng. 2011;8:025011. doi: 10.1088/1741-2560/8/2/025011. [DOI] [PubMed] [Google Scholar]
- Lewis M, Rushanan S. The role of physical therapy and occupational therapy in the treatment of amyotrophic lateral sclerosis. Neurorehabilitation. 2007;22:451–461. [PubMed] [Google Scholar]
- Li Y, Pan J, Long J, Yu T, Wang F, Yu Z, Wu W. Multimodal BCIs: target detection, multidimensional control, and awareness evaluation in patients with disorder of consciousness. Proc IEEE. 2016;104:332–352. [Google Scholar]
- Mainsah BO, Collins LM, Colwell KA, Sellers EW, Ryan DB, Caves K, Throckmorton CS. Increasing BCI communication rates with dynamic stopping towards more practical use: an ALS study. J Neural Eng. 2015;12:016013. doi: 10.1088/1741-2560/12/1/016013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mak JN, et al. EEG correlates of P300-based brain-computer interface (BCI) performance in people with amyotrophic lateral sclerosis. J Neural Eng. 2012;9:026014. doi: 10.1088/1741-2560/9/2/026014. [DOI] [PubMed] [Google Scholar]
- Marchetti M, Priftis K. Brain-computer interfaces in amyotrophic lateral sclerosis: a metanalysis. Clin Neurophysiol. 2015;126:1255–1263. doi: 10.1016/j.clinph.2014.09.017. [DOI] [PubMed] [Google Scholar]
- Martinez-Cagigal V, Gomez-Pilar J, Alvarez D, Hornero R. An asynchronous P300-based brain-computer interface web browser for severely disabled people. IEEE Trans Neural Syst Rehab Eng. 2017;25:1332–1342. doi: 10.1109/TNSRE.2016.2623381. [DOI] [PubMed] [Google Scholar]
- McCane LM, et al. Brain-computer interface (BCI) evaluation in people with amyotrophic lateral sclerosis. Amyotroph Lateral Scler Front Degener. 2014;15:207–215. doi: 10.3109/21678421.2013.865750. [DOI] [PMC free article] [PubMed] [Google Scholar]
- McCane LM, et al. P300-based brain-computer interface (BCI) event-related potentials (ERPs): people with amyotrophic lateral sclerosis (ALS) vs. age-matched controls. Clin Neurophysiol. 2015;126:2124–2131. doi: 10.1016/j.clinph.2015.01.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Millan JDR, et al. Combining brain-computer interfaces and assistive technologies: state-of-the-art and challenges. Front Neurosci. 2010;4:161. doi: 10.3389/fnins.2010.00161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Piccione F, et al. P300-based brain computer interface: reliability and performance in healthy and paralysed participants. Clin Neurophysiol. 2006;117:531–537. doi: 10.1016/j.clinph.2005.07.024. [DOI] [PubMed] [Google Scholar]
- Puanhvuan D, Khemmachotikun S, Wechakarn P, Wijarn B, Wongsawat Y. Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities. Cogn Neurodyn. 2017;11:117–134. doi: 10.1007/s11571-017-9424-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Riccio A, et al. Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis. Front Hum Neurosci. 2013;7:732. doi: 10.3389/fnhum.2013.00732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Roche JC, et al. A proposed staging system for amyotrophic lateral sclerosis. Brain. 2012;135:847–852. doi: 10.1093/brain/awr351. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ron-Angevin R, Varona-Moya S, da Silva-Sauer L. Initial test of a T9-like P300-based speller by an ALS patient. J Neural Eng. 2015;12:046023. doi: 10.1088/1741-2560/12/4/046023. [DOI] [PubMed] [Google Scholar]
- Sellers EW, Donchin E. A P300-based brain-computer interface: initial tests by ALS patients. Clin Neurophysiol. 2006;117:538–548. doi: 10.1016/j.clinph.2005.06.027. [DOI] [PubMed] [Google Scholar]
- Sellers EW, Vaughan TM, Wolpaw JR. A brain-computer interface for long-term independent home use. Amyotrophic Lateral Scler. 2010;11:449–455. doi: 10.3109/17482961003777470. [DOI] [PubMed] [Google Scholar]
- Severens M, Van der Waal M, Farquhar J, Desain P. Comparing tactile and visual gaze-independent brain-computer interfaces in patients with amyotrophic lateral sclerosis and healthy users. Clin Neurophysiol. 2014;125:2297–2304. doi: 10.1016/j.clinph.2014.03.005. [DOI] [PubMed] [Google Scholar]
- Wang HT, Li YQ, Long JY, Yu TY, Gu ZH. An asynchronous wheelchair control by hybrid EEG-EOG brain-computer interface. Cogn Neurodyn. 2014;8:399–409. doi: 10.1007/s11571-014-9296-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. An EEG-based brain-computer interface for cursor control. Electroencephalogr Clin Neurophysiol. 1991;78:252–259. doi: 10.1016/0013-4694(91)90040-b. [DOI] [PubMed] [Google Scholar]
- Wolpaw JR, et al. Brain-computer interface technology: a review of the first international meeting. IEEE Trans Rehabil Eng. 2000;8:164–173. doi: 10.1109/tre.2000.847807. [DOI] [PubMed] [Google Scholar]
- Xu MP, et al. Use of a steady-state baseline to address evoked vs. oscillation models of visual evoked potential origin. NeuroImage. 2016;134:204–212. doi: 10.1016/j.neuroimage.2016.03.073. [DOI] [PubMed] [Google Scholar]
- Xu MP, Xiao XL, Wang YJ, Qi HZ, Jung TP, Ming D. A brain-computer interface based on miniature-event-related potentials induced by very small lateral visual stimuli. IEEE Trans Biomed Eng. 2018;65:1166–1175. doi: 10.1109/TBME.2018.2799661. [DOI] [PubMed] [Google Scholar]
- Yadav AP, Nicolelis MAL. Electrical stimulation of the dorsal columns of the spinal cord for Parkinson’s disease. Mov Disord. 2017;32:820–832. doi: 10.1002/mds.27033. [DOI] [PubMed] [Google Scholar]
- Yin EW, Zhou ZT, Jiang J, Yu Y, Hu DW. A dynamically optimized SSVEP brain-computer interface (BCI) speller. IEEE Trans Biomed Eng. 2015;62:1447–1456. doi: 10.1109/TBME.2014.2320948. [DOI] [PubMed] [Google Scholar]
- Zhang Y, Zhou GX, Jin J, Wang XY, Cichocki A. Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis. Int J Neural Syst. 2014;24:1450013. doi: 10.1142/S0129065714500130. [DOI] [PubMed] [Google Scholar]
- Zhang Y, Zhou GX, Jin J, Wang XY, Cichocki A. Optimizing spatial patterns with sparse filter bands for motor-imagery based brain-computer interface. J Neurosci Methods. 2015;255:85–91. doi: 10.1016/j.jneumeth.2015.08.004. [DOI] [PubMed] [Google Scholar]
- Zhang YS, Guo DQ, Xu P, Zhang Y, Yao DZ. Robust frequency recognition for SSVEP-based BCI with temporally local multivariate synchronization index. Cogn Neurodyn. 2016;10:505–511. doi: 10.1007/s11571-016-9398-9. [DOI] [PMC free article] [PubMed] [Google Scholar]






