Performance Assessment of a Custom, Portable, and Low-Cost Brain-Computer Interface Platform

Abstract

Objective

Conventional brain-computer interfaces (BCIs) are often expensive, complex to operate, and lack portability, which confines their use to laboratory settings. Portable, inexpensive BCIs can mitigate these problems, but it remains unclear whether their low-cost design compromises their performance. Therefore, we developed a portable, low-cost BCI and compared its performance to that of a conventional BCI.

Methods

The BCI was assembled by integrating a custom electroencephalogram (EEG) amplifier with an open-source microcontroller and a touchscreen. The function of the amplifier was first validated against a commercial bioamplifier, followed by a head-to-head comparison between the custom BCI (using 4 EEG channels) and a conventional 32-channel BCI. Specifically, 5 able-bodied subjects were cued to alternate between hand opening/closing and remaining motionless while the BCI decoded their movement state in real-time and provided visual feedback through a light emitting diode. Subjects repeated the above task for a total of 10 trials, and were unaware of which system was being used. The performance in each trial was defined as the temporal correlation between the cues and the decoded states.

Results

The EEG data simultaneously acquired with the custom and commercial amplifiers were visually similar and highly correlated (ρ=0.79). The decoding performances of the custom and conventional BCIs averaged across trials and subjects were 0.70±0.12 and 0.68±0.10, respectively, and were not significantly different.

Conclusion

The performance of our portable, low-cost BCI is comparable to that of conventional BCIs.

Significance

Platforms, such as the one developed here, are suitable for BCI applications outside of a laboratory.

I. Introduction

Brain-computer interface (BCI) systems have been designed for diverse applications, such as smart living, entertainment, and neuroprostheses. Recent studies have also examined whether BCIs can facilitate neurorehabilitation after neurological injuries by improving residual motor function. However, these studies often employ conventional BCIs that rely on expensive commercial amplifier arrays and bulky computers (e.g. [1], [2], [3], [4], [5]). These factors inevitably drive up the cost, complexity, and setup time of BCI systems, while reducing their portability. Consequently, these BCI systems are not ideal for at-home use by the community.

One way to decrease the setup time associated with conventional BCIs is to reduce the number of EEG channels. Prior studies have demonstrated that EEG-based motor BCIs could be successfully operated with as few as 1 channel [6], although some applications may require at least 8 channels [7]. Reducing the number of channels in a cost-effective way requires the replacement of commercial bioamplifiers (typically with dozens of channels) with custom, low-channel-count amplifier arrays. Similarly, further enhancement of portability and cost reduction could be achieved by replacing full-size computers in conventional BCIs with low-cost embedded systems. These strategies have been employed in several studies, where custom portable BCIs were developed for applications ranging from drowsiness detection [8], [9], smart living environments [10], and multimedia navigation [11], to prosthesis control [12] and motor rehabilitation [13]. However, reducing a BCI’s bulkiness, cost, and complexity in this manner may consequently decrease its decoding performance. Many of the above studies compared their decoding performance to previous work, but, to date, no head-to-head performance comparison between portable, cost effective BCIs and conventional BCIs has been reported in the literature. Maintaining a high decoding accuracy is critical in applications such as drowsiness detection and prosthesis control.

In this study, we developed a portable, low-cost BCI system based on [13], and then performed a head-to-head comparison of its decoding capability against that of a conventional BCI system. Our findings demonstrate that there need not be a trade-off between decoding performance and portability, cost, and simplicity. This suggests that portable and low-cost custom systems, such as the one developed here, may be ideally suited for BCI applications outside of a laboratory setting.

II. Methods

A. Overview

A low-cost, embedded BCI system was developed by integrating a custom EEG amplifier and a commercial microcontroller unit (MCU) with a touchscreen (see Fig. 1). Custom software was developed and uploaded to the MCU to control all facets of the system’s operation. The real-time decoding performance of the custom BCI was compared to that of a conventional BCI system in able-bodied subjects. Both BCI systems were trained to recognize, from EEG, when a subject was opening/closing their right hand or remaining motionless. The subject received feedback in the form of a red light-emitting diode (LED) that was turned on when hand movement was decoded, and turned off when idling was decoded. The correlation between cues and decoded states for each trial was calculated and used to determine whether the custom BCI’s performance was significantly inferior to that of the conventional BCI.

Fig. 1.

Top Left: Exploded view of the individual components of the custom BCI system. Top right: The fully assembled custom BCI system connected to a handheld battery and EEG cap. Bottom: Graphical user interface navigation map for operating the custom BCI system. Note the simple and straightforward interface design.

B. Hardware

The custom BCI system consisted of 3 main hardware components: an 8-channel EEG amplifier array (details below), an open-source Arduino Due MCU (Arduino, Ivrea, Italy), and an LED touchscreen with integrated micro SD card slot (Adafruit Industries, New York, NY). The entire system was ~13×9×3 cm³ in size, and consumed 1 W of power during normal operation. This enabled it to be powered by a rechargeable 5V battery. Each channel of the EEG amplifier array (see Fig. 2) consisted of a cascade of one instrumentation amplifier (Texas Instrument INA128, Dallas, TX) followed by two operational amplifiers (Texas Instrument OPA 4241) to achieve a total of gain of >89 dB with >80 dB common mode rejection ratio (CMRR). Active low-pass and high-pass filters provided a banded response between 1.6–32.9 Hz. The amplifier array circuit was implemented on a printed circuit board that interfaced with the MCU and touchscreen as well as with the EEG electrodes. The MCU’s ADC unit had a resolution of 12 bits.

Fig. 2.

Circuit diagram for each channel of the custom amplifier array. The mid-level V_CC/2 is connected to a bias electrode as well as to all the electrodes’ active shielding.

The amplifier array was empirically validated by comparing its output to that of a commercial amplifier system (EEG100C, BIOPAC Systems, Goleta, CA) with a 1–35 Hz banded response. Specifically, one EEG channel derived by referencing electrode Cz to AFz (nomenclature consistent with the international 10–10 EEG standard [14]) was simultaneously amplified by both the custom and commercial amplifiers. The output of each amplifier was acquired simultaneously at 250 Hz by a commercial data acquisition system (MP150, BIOPAC Systems, Goleta, CA) over the course of 1 min. The gain of EEG100C was ~86 dB with 110 dB CMRR, and the MP150’s ADC resolution was 12 bits. Different software filters were applied to the data from the custom and commercial amplifiers to account for their different hardware filter settings. Finally, the lag-optimized correlation coefficient (Pearson) between the signals was calculated.

The conventional BCI system has been used extensively in previous studies [15], [16], and consisted of a commercial 32-channel EEG amplifier (NeXus-32, Mind Media, Netherlands), a desktop computer, and the MP150 data acquisition system for aligning the EEG and cue signals. The gain of the NeXus-32 amplifier was ~26 dB with >90 dB CMRR, and its ADC resolution was 22 bits.

A cost breakdown of both BCI systems (excluding the EEG cap) is shown in Table I. The cost of the custom BCI was <½0th of the cost of an equivalent 8-channel version of the conventional system (using per channel costs). The conventional system’s amplifier, however, has medical CE and FDA certifications, which may account for its high cost.

Table I.

Cost breakdown of the custom and conventional BCI systems. The cost of the custom BCI’s 8-channel EEG amplifier includes PCB manufacturing, assembly, and components. The cost of the custom BCI’s computer includes the cost of the MCU, battery, and microSD card. The cost of the conventional BCI system does not include the cost of the separate data acquisition system for aligning the EEG and cues.

Component	Custom BCI	Conventional BCI
EEG Amplifier	~$210	~$22,500
	(~$26.25/channel)	(~$703.13/channel)
Computer	~$65	~$1,500
Display/Human Interface	~$35	~$200
Total	~$310	~$24,200

C. Software

Specialized software was written in C++ and uploaded to the custom BCI’s MCU to render the graphical user interface (GUI) and perform the following BCI functions: 1. EEG training data acquisition, 2. generation of the BCI decoding model, 3. real-time decoding to control an output device. The simple GUI is depicted in the bottom panel of Fig. 1. The effector output can be manually controlled on the home screen. In training mode, the screen alternates between displaying “GO” (during movement epochs) and a blank screen (during idling epochs), and then displays the accuracy of the generated BCI decoding model. Lastly, before the end of training, a small number of calibration cues (“GO”/blank screen) are presented to the user. Back at the home screen, the user can enter calibration mode to manually select thresholds for the decoding model (based on histograms from data collected during the calibration cues). During real-time BCI decoding, the user is presented with the same “GO”/blank screen cues as before and their decoded brain state is used to control the effector output. The software developed to operate the BCI, including the GUI, is publicly available at https://github.com/cbmspc/ucibci.

The conventional BCI system utilized custom Matlab scripts to perform the same functions as the custom BCI system. These were originally described in [15].

D. Subject Recruitment

The use of human subjects was approved by the University of California, Irvine Institutional Review Board. Able-bodied individuals with no history of neurological disease were recruited for the study.

E. Setup

The general experimental procedure for each subject is depicted in Fig. 3. Subjects were first fitted with and EEG cap (Waveguard, ANT-Neuro, Germany) with 64 actively-shielded electrodes. Only a subset of 33 electrodes was used (see Fig. 4), and their impedances were reduced to <10 kΩ using conductive gel. The conventional BCI utilized 32 channels (32 electrodes all referenced to AFz), while the custom BCI used only 4 channels (C1, C3, C5, and CP3, all referenced to AFz). Specifically, AFz was the V- electrode in Fig. 2 for every channel of the custom BCI. In addition, the custom BCI used a bias electrode (Fz) during testing. For subject S3, FC3 was used instead of C5 due to excessive noise in that channel. The 4 channels used by the custom BCI were chosen based on their proximity to the expected hand representation area of the primary motor cortex. Although the custom BCI could accommodate up to 8 channels, preliminary post-hoc analysis of foot movement data from a previous BCI study [17] demonstrated no significant loss of decoding accuracy when only ~4 (albeit well chosen) EEG channels were used instead of all 32. In addition, our results from [13] suggested that high decoding performance was attainable with only 4 EEG channels. Therefore, we used only 4 of the 8 channels for this study.

Fig. 3.

Experimental procedure for the head-to-head comparison of the custom and conventional BCI, depicting the order of each system’s training, decoding model generation (Dec. Mod.), binary state machine calibration (Cal.), and real-time decoding trials. The entire procedure lasted around 1.5 h.

Fig. 4.

Electrode locations for the international 10–10 EEG system. The electrodes used by the conventional BCI are colored grey, while those used by the custom BCI are outlined in red.

F. BCI Training

In order to train the BCI systems to distinguish the presence/absence of hand movements, users followed verbal cues to alternate between repetitively opening/closing their right hand for 6 s (“move” epochs) and remaining motionless for 6 s (“idle” epochs). EEG data from 4 (custom BCI) or 32 (conventional BCI) channels were acquired at 240 Hz (custom BCI) or 256 Hz (conventional BCI) per channel. The sampling rate for the custom BCI was chosen simply because it was close to 256 and produced many software parameters that were divisible by 10, and changing it to 256 Hz did not affect decoding performance. Each channel’s EEG data were digitally filtered either into the α (8–13 Hz) and β (13–30 Hz) physiological bands by the custom BCI or into 2 Hz bands covering the same 8–30 Hz range by the conventional BCI. The custom BCI utilized the entire α and β bands, instead of smaller frequency bands, due to its limited memory space (96 kB) and to simplify the subsequent decoding steps. The average power at each channel and frequency band was calculated for every 6-s-long “move” and “idle” epoch. To prevent movement state transitions from affecting the subsequent decoding models, the custom and conventional BCIs discarded the first 1-s of EEG data from each epoch. The conventional BCI also discarded the last 1-s of EEG data from each epoch. However, doing the same for the custom BCI had no impact on its decoding performance, and therefore, it was not implemented in this study.

For each subject, the custom BCI was trained first, followed by the conventional BCI (see Fig. 3). To minimize the total time that each subject spent training, the training sessions for the custom BCI lasted only 5 min. However, the training sessions for the conventional BCI lasted 10 min and could not be reasonably reduced further because of the high dimensionality of its data (32 EEG channels × 11 frequency bands). The custom BCI was trained for 5 min instead of 10 min because it made no difference in its decoding capability during preliminary tests. During training, subjects were positioned facing away from the experimenters/BCI systems and were not told of the training time discrepancy in order to blind them to which BCI was being used. The BCI cues were relayed verbally to the subjects by the experimenters, who also performed mock typing and mouse clicking (to mimic the sounds of operating the conventional system) before the use of the custom system.

G. Decoding Model

The custom BCI extracted hand movement features from its 8-dimensional EEG training data using linear discriminant analysis (LDA) [18], while the conventional BCI first reduced its training data’s dimensionality (down from 352) using classwise principal component analysis (CPCA) [19] before extracting hand movement features with either LDA or approximate information discriminant analysis (AIDA) [20]. The conventional BCI’s initial CPCA step was necessary to perform LDA/AIDA. Next, both BCI systems generated a Bayesian classifier to calculate the probability of the movement state (hand opening/closing) from extracted features (f), denoted as P(M | f). Each system also performed leave-one-out cross-validation to predict the accuracy of the decoding model. If the cross-validation accuracy was <85%, the subject repeated the training for that system. If the accuracy was ≥85%, the subject performed an additional 2-min calibration session of cued hand opening/closing and idling (in alternating 6-s epochs) with that BCI system to provide data for calibrating a binary state machine.

H. State Machine Calibration

For each BCI system, histograms of P(M | f) from “move” and “idle” epochs of the 2-min calibration session were generated to calibrate a binary state machine that classified users’ underlying movement states (“move” or “idle”) from P(M | f). Specifically, for each BCI, the values of two thresholds, T_M and T_I (where T_M > T_I), were manually selected by the experimenters to be used by its state machine as follows. When P(M | f) < T_I, the state machine entered the “idle” state; when P(M | f) > T_M, the state machine entered the “move” state; when T_I < P(M | f) < T_M, the state machine remained in its previous state. This binary state machine design reduces noisy state transitions and alleviates users’ mental workload, and has been successfully used before [15], [16]. If a BCI system’s histograms from “move” and “idle” calibration epochs appeared highly similar, the training session for that BCI was repeated.

I. Real-Time Decoding

During real-time operation, both the custom and conventional BCI systems employed a 0.75 s sliding analysis window (0.25 s overlap) for determining P(M | f) from the users’ EEG. To further prevent noisy state transitions, the posterior probabilities over the most recent 1.5 s of EEG data (6 values) were averaged to generate $\overline{\text{P}}(M|f)$. $\overline{\text{P}}(M|f)$ was used by the systems’ state machine to decode users’ underlying movement state every 0.25s. This decoded state was used by each system to control an LED which turned on during decoded “move” states and turned off during decoded “idle” states.

Subjects participated in five, 2-min-long trials for each BCI system (total of 10 trials). During each trial, subjects followed alternating 6-s cues to open/close their right hand or remain motionless. Subjects were positioned facing away from the experimenters/BCI systems and towards the single LED light that provided real-time visual feedback from both systems. Experimenters provided verbal cues for subjects to “move” and “idle” based on the computerized cues displayed by each system. In addition, the experimenters performed mock typing and mouse clicking during use of the custom BCI. Subjects were told that the order of the 10 trials was randomized, although the custom and commercial systems were actually used in an alternating fashion (starting with the custom system). The alternating utilization of the BCI systems was intended to avoid subject learning or fatigue. For each trial, the performance of the system was assessed as the lag-optimized correlation (Pearson) between the cues and the decoded state. Then, for each subject, a left-sided Mann-Whitney U test (α=0.05) was performed between the decoding correlations of the custom and conventional BCI.

III. Results

A. Custom Amplifier Validation

EEG (Cz referenced to AFz) from one human subject was simultaneously passed to both the custom and commercial amplifiers. The correlation between the 1-min-long signals acquired from both amplifiers was 0.79. Moreover, both signals appeared visually similar. See Fig 5 for a representative 3-s example of each amplifier’s output.

Fig. 5.

3-s example from the 1 min of human EEG data simultaneously acquired by the custom and commercial amplifiers. Note the high degree of similarity between the signals.

B. Decoding Performance

Five able-bodied subjects (S1–5) gave their informed consent to participate in this study. Three of the subjects had prior BCI experience. Anecdotally, the setup time for the custom BCI system required ~10 minutes, as opposed to ~30–40 minutes for the conventional BCI system, due to its lower number of channels. All subjects successfully operated both the custom and conventional BCI systems. The overall cross-validation accuracy across all subjects was 93.6±4.3 and 96.2±1.8 for the custom and conventional BCI systems, respectively. In the meantime, the custom BCI’s processor was still able to generate the decoding model and perform cross-validation in a timely manner (<1 min for each subject). For each subject, the conventional BCI utilized features around C3 in the α and/or β bands, so the 4 channels used by the custom BCI may have been an appropriate choice in these subjects. For example, the average of all S2’s β band features is shown in Fig. 6.

Fig. 6.

The average β band features used by the conventional BCI for decoding S2’s hand movements. Areas in red represent highly weighted features, while those in blue are less important. As expected, the region around C3 was important for decoding.

The average lag-optimized correlation between cues and decoded states across all subjects and trials was 0.70±0.12 (average lag of 2.22±0.27 s) for the custom BCI and 0.68±0.10 (average lag of 2.23±0.37 s) for the conventional BCI. Training cross-validation accuracies and decoding correlations for both systems are provided for each subject in Table II and Fig. 7, respectively. No subject demonstrated a significantly lower BCI performance with the custom system compared to the conventional system.

Table II.

Subject demographics and cross-validation accuracy for each BCI system.

Subject	Age/Sex	Prior BCI Experience	Custom BCI Training Accuracy	Conventional BCI Training Accuracy
S1	23/M	N	90%	96%
S2	46/M	Y	96%	99%
S3	21/M	N	96%	96%
S4	28/M	Y	98%	97%
S5	35/M	Y	88%	95%

Fig. 7.

The correlation between cues and the decoded state for each real-time decoding trial using the custom and conventional (conv.) BCI systems. For each subject, trials 1–5 are represented by a cross, circle, square, diamond, and plus sign, respectively. In addition, p-values from the Mann-Whitney U tests are provided. The performance of the custom BCI was not significantly inferior (p<0.05) to the conventional system in any subject.

IV. Discussion

This study demonstrates that low-cost, embedded EEG-based BCI platforms, such as the one tested here, can achieve similar performance to a conventional BCI system with substantially more channels and computational resources. Low-cost, easy-to-use, standalone systems make BCIs more accessible to researchers, clinicians, and patients, and increase the feasibility of large clinical trials involving BCI use. The small profile and minimal power requirements of embedded EEG systems make them highly portable, increasing the number of applications in which BCIs can be used. Some of these include smart environment control, gaming/entertainment, and mobile solutions to neurological deficits, such as BCI-controlled neuroprostheses, wheelchairs, and robotic exoskeletons. It may even be possible in the future to develop fully implantable BCI systems with onboard processing.

Although the custom EEG amplifier did not perform identically to a commercial system (0.79 correlation), the custom BCI still achieved high decoding performance. In fact, the decoding performance of both systems was generally higher than what we have previously reported for motor execution tasks in able-bodied [15], [21] and stroke subjects [17] using an equivalent conventional BCI. We believe that the different hardware and software filters used with the custom and commercial amplifiers may have reduced the correlation between the output signals. In particular, the custom amplifier’s output was observed to be contaminated with environmental noise, possibly because its 60 Hz notch filter was of lower order than that of the commercial amplifier.

Our finding that a low-cost, embedded BCI using only 4 EEG channels can achieve a high decoding performance and does not perform significantly worse than a conventional system is encouraging, but not wholly unexpected. For example, high BCI decoding performance with few channels has been observed previously [13] and is consistent with previous channel-dropping studies [6], [7]. Although a moderately long decoding delay (~2 s) was observed for both BCIs in this study, a significant fraction of this delay in both systems may have been caused by the experimenters’ translation of visual computer cues into verbal cues for the subjects.

Custom, embedded BCI platforms, such as the one developed in this study, can be highly modifiable. Not only are the software libraries readily customizable, but even the system hardware can be adapted by community users for a variety of applications. For example, with this BCI platform, the bandwidth and gain of the custom amplifier array can be changed by adjusting its resistive and capacitive components. In addition, surface-mount components can replace the large dual-inline packages to further reduce the system’s size. Based on the software execution time, the current Arduino Due MCU can tolerate an increase in channel number and sampling rate without causing delays during its operation. Therefore, this system is even practical for applications where higher frequencies (beyond the β band) are desired. Lastly, an expensive (~$2500) EEG cap was used in this study out of convenience, but this may not be appropriate for community users. Instead, dry electrodes, which offer shorter setup time, could be used. However, dry electrodes may still be inferior to wet electrodes [22], and in preliminary testing, we observed them to be highly sensitive to movement artifacts. A great alternative is high quality, individual EEG cup electrodes (wet) that are inexpensive (~$50 each).

Many portable, reasonably low-cost BCI systems have already been developed academically ([23], [24], [25], [26], [27], [28]) and commercially (OpenBCI, Emotiv, and NeuroSky). However, these BCI systems do not perform onboard signal analysis and decoding. Yet, if these devices are modified (e.g. paired with a microcontroller for decoding), the results of this study suggest that they may be suitable for mobile BCI applications and could demonstrate similar decoding performance to conventional BCIs. Wang et al. [29] developed a portable, 4-channel BCI that transmitted EEG data to a smartphone for signal analysis and decoding. While the system was specifically designed to decode occipital steady-state visually evoked potentials (SSVEPs) and is unlikely to work for sensorimotor rhythm modulation, its performance may not be inferior to SSVEP-based conventional BCIs. Likewise, the BCIs that utilize embedded processing units for signal analysis in [8], [9], [10], [11] may perform similarly to expensive, full-size, conventional BCIs. However, these BCIs rely on commercial DSPs or FPGAs without user-friendly open-source development tools, so it may be hard for community users to modify them for other BCI applications.

A. Limitations

While many BCI systems are intended for use by individuals with neuromotor deficits, such as those resulting from stroke or spinal cord injury (SCI), only able-bodied subjects participated in this study. Thus it is unclear how low-cost, embedded BCI systems with few channels will fare against conventional BCIs in subjects with neurological disease. In the future, we intend to test the functionality of our custom BCI platform against a conventional system in stroke and SCI populations. We envision that systems like this one could be applied for BCI-based at-home physiotherapy or mobile neuroprosthetics. In addition, we did not explicitly assess the system’s feasibility for use outside of a laboratory setting (e.g. at-home) and further studies are required. Lastly, the decoding performance in this study focused on a simple motor paradigm, i.e. the presence or absence of hand movements. However, it is unclear whether these results will generalize to more elaborate movement tasks where a higher number of EEG channels and/or complex decoding algorithms may be necessary to maintain sufficiently high BCI performance.

V. Conclusion

Current BCI systems are not practical for use outside research laboratories due to their complicated setup/operation, prohibitive costs, and lack of portability. The custom BCI system tested here utilized 4 EEG channels as well as a low-cost, open-source MCU for decoding, but still performed similarly to a conventional BCI system. The findings of this study indicate that a high number of EEG channels and extensive computational resources are not always necessary for BCI systems to operate with high accuracy, and many of the portable, inexpensive academic or hobby-level commercial BCIs may perform similarly to conventional systems. In addition, these platforms are more practical and cost-effective than conventional BCIs for large scale studies, as well as for motor rehabilitation or hobby applications outside of a laboratory setting.