Operation of a brain-computer interface walking simulator for individuals with spinal cord injury

Abstract

Background

Spinal cord injury (SCI) can leave the affected individuals with paraparesis or paraplegia, thus rendering them unable to ambulate. Since there are currently no restorative treatments for this population, novel approaches such as brain-controlled prostheses have been sought. Our recent studies show that a brain-computer interface (BCI) can be used to control ambulation within a virtual reality environment (VRE), suggesting that a BCI-controlled lower extremity prosthesis for ambulation may be feasible. However, the operability of our BCI has not yet been tested in a SCI population.

Methods

Five participants with paraplegia or tetraplegia due to SCI underwent a 10-min training session in which they alternated between kinesthetic motor imagery (KMI) of idling and walking while their electroencephalogram (EEG) were recorded. Participants then performed a goal-oriented online task, where they utilized KMI to control the linear ambulation of an avatar while making 10 sequential stops at designated points within the VRE. Multiple online trials were performed in a single day, and this procedure was repeated across 5 experimental days.

Results

Classification accuracy of idling and walking was estimated offline and ranged from 60.5% (p = 0.0176) to 92.3% (p = 1.36×10⁻²⁰) across participants and days. Offline analysis revealed that the activation of mid-frontal areas mostly in the μ and low β bands was the most consistent feature for differentiating between idling and walking KMI. In the online task, participants achieved an average performance of 7.4±2.3 successful stops in 273±51 sec. These performances were purposeful, i.e. significantly different from the random walk Monte Carlo simulations (p<0.01), and all but one participant achieved purposeful control within the first day of the experiments. Finally, all participants were able to maintain purposeful control throughout the study, and their online performances improved over time.

Conclusions

The results of this study demonstrate that SCI participants can purposefully operate a self-paced BCI walking simulator to complete a goal-oriented ambulation task. The operation of the proposed BCI system requires short training, is intuitive, and robust against participant-to-participant and day-to-day neurophysiological variations. These findings indicate that BCI-controlled lower extremity prostheses for gait rehabilitation or restoration after SCI may be feasible in the future.

Introduction

Spinal cord injury (SCI) can leave the affected individuals with severe paraparesis or paraplegia, thus rendering them unable to ambulate. Since there are currently no restorative treatments for this population, technological approaches have been sought to substitute for the lost motor functions. Examples include robotic exoskeletons [1], functional electrical stimulation (FES) systems [2], and spinal cord stimulators [3]. However, these systems lack intuitive able-body-like supraspinal control, so the ambulation function of these devices is controlled manually. As a result, they monopolize the upper extremities, which interferes with the user’s hand function. Due to these and other limitations, wheelchairs remain the primary means of mobility after SCI. Unfortunately, the extended reliance on wheelchairs typically lead to a wide variety of comorbidities that constitute the bulk of chronic SCI-related medical care costs [4-7]. Consequently, to address the above issues associated with the treatment of severe paraparesis and paraplegia after SCI, novel brain-controlled prostheses are currently being pursued [8,9].

Recent results by our group [8,9] suggest that an electroencephalogram (EEG) based brain-computer interface (BCI) controlled lower extremity prosthesis may be feasible. More specifically, these studies demonstrated the successful implementation of a BCI system that controls the ambulation of an avatar (a stand-in for a lower extremity prosthesis) within a virtual reality environment (VRE). By using a data-driven machine learning approach to decode the users’ kinesthetic motor imageries (KMIs), this BCI-controlled walking simulator enabled a small group of participants (one with paraplegia due to SCI [9]) to achieve intuitive and purposeful BCI control after a short training session. While the single SCI participant outperformed most able-bodied participants in this study, the operability of this system has not yet been tested in a SCI population. The successful implementation of the BCI-controlled walking simulator in a population of participants with SCI will establish the feasibility of future BCI-controlled lower extremity prostheses and will represent an important step toward developing novel approaches to gait rehabilitation after SCI.

Extending the application of the BCI-controlled walking simulator to a SCI population is faced with several problems. First, cortical reorganization, which is common after SCI [10-13], may cause the cortical representation of walking KMI to vary vastly from one SCI participant to another. Second, this representation may dramatically evolve over time when SCI participants are engaged in KMI training [14]. Finally, participants with SCI may interpret walking KMI either as motor imagery or as attempted walking, which in turn may result in multiple patterns of cortical activation across these individuals. Therefore, intuitive BCI operation under these conditions requires a system that can accommodate for the variations of brain physiology across SCI individuals, time, and strategies. To address these problems, we used a data-driven machine learning method to decode walking KMIs in a small population of SCI individuals. This approach enabled 5 participants to achieve intuitive and self-paced operation of the BCI-controlled walking simulator after only minimal training. Furthermore, they were able to maintain this level of control over the course of several weeks.

Methods

Overview

The goal of this study was to determine if individuals with complete motor SCI can use intuitive control strategies to purposefully operate a BCI-controlled walking simulator. To achieve this goal, 5 participants with SCI underwent a short training procedure where they performed alternating epochs of idling and walking KMI while their EEG were recorded. These training EEG data were then analyzed to build decoding models for subsequent online BCI operation. To ascertain purposeful BCI control, participants then performed 5 sessions of an online BCI goal-oriented virtual walking task [9]. This entire procedure was performed 5 times over the course of several weeks to determine if participants’ performances improved with additional practice.

Participant recruitment

This study was approved by the University of California, Irvine Institutional Review Board. Four participants with paraplegia and one with tetraplegia due to SCI were recruited via physician referral from the Long Beach Veterans Affairs Spinal Cord Injury Center and other SCI outreach programs. The participants (see Table 1) gave their informed consent to participate in the study. Note that all participants were BCI naïve and most of them performed the experimental procedures at a rate of once per week.

Table 1.

Demographic data

Participant	Gender	Age	SCI status
1	M	34	T11, ASIA A, 8 yr. post injury
2	M	46	T1, ASIA B, 4 yr. post injury
3	M	43	C5, Syringomyelia, 14 yr. post onset
4	M	59	T1, ASIA B, 2 yr. post injury
5	M	21	T11, ASIA B, 1 yr. post injury

List of participants with demographic data and level of SCI. ASIA = American Spinal Injury Association impairment scale.

Data acquisition

Each participant was positioned ∼0.8 to 1 m from a computer screen that displayed textual cues or the VRE. EEG were recorded using a 63-channel cap (Medi Factory, Heerlen, The Netherlands) with Ag-AgCl electrodes arranged according to the extended 10-20 International Standard. Conductive gel (Compumedics USA, Charlotte, NC) was applied to all electrodes and the 30-Hz impedances between each electrode and the reference electrode were maintained at < 10 K Ω by abrading the scalp with a blunt needle. The EEG signals were amplified and digitized (sampling rate: 256 Hz, resolution: 22 bits) using two linked NeXus-32 EEG bioamplifiers (MindMedia, Roermond-Herten, The Netherlands). These signals were then streamed in real-time to a computer, re-referenced in a common average mode, band-pass filtered (0.01–40 Hz), and subsequently analyzed. The above procedures were performed using custom-written MATLAB (MathWorks, Natick, MA) programs.

Training procedure

To facilitate intuitive control of ambulation within in a VRE, a EEG decoding model was developed that differentiates between idling and walking KMI. To this end, training EEG data were acquired while participants underwent 30-sec alternating epochs of idling and walking KMI over a 10-min session. Participants were instructed to perform walking KMI and idling via automated textual cues displayed on the screen. For the walking KMI task, participants were instructed to vividly imagine themselves walking or to attempt to perform the cyclic leg movements of walking (note that these attempted movements did not produce any motor output). During this entire procedure, a computer labeling signal (to identify idling and walking KMI epochs) was recorded by an auxiliary data acquisition system (MP150, Biopac Systems, Goleta, CA), and a common pulse train sent to both EEG and auxiliary data systems was used to synchronize the EEG and labeling signals. Also, the participants were asked to remain still during the entire procedure, and their movement was monitored by the experimenter. If consistent movements were observed, the participant was asked to repeat the entire training procedure without making the undesired movements.

Decoding model generation

Offline signal analysis to generate EEG decoding models was performed in a manner similar to Wang et al. [9]. Briefly, EEG and labeling signals were first aligned and merged using the synchronization signal, and an iterative artifact rejection algorithm [15] was used to exclude EEG channels with excessive electromyogram (EMG) artifacts from further analysis. This was done by removing those channels whose EEG amplitude exceeded an outlier voltage threshold. The EEG data from the remaining channels were then split into ∼30 sec segments of idling and walking KMI states using the labeling signal. Each EEG segment was then divided into 5 randomly spaced, non-overlapping 4-sec trials for a total of 100 trials, which were then transformed into the frequency domain using the Fast Fourier Transform (FFT). Note that the remaining data in each segment were discarded. Finally, the power spectral density of each trial was integrated in 2-Hz bins, resulting in 20 binned power spectral values per channel.

Once the EEG segments were transformed into the frequency domain, a systematic search was performed to find the best contiguous frequency range for classification [9]. The dimension of the input data was then reduced using classwise principal component analysis (CPCA) [16,17], and discriminating features were extracted using either Fisher’s linear discriminant analysis (LDA) [18] or approximate information discriminant analysis (AIDA) [19,20]. This resulted in the extraction of one-dimensional (1D) spatio-spectral features:

$$f=\mathbf{T}{\mathrm{\Phi }}_{\mathrm{C}}\left(\mathbf{d}\right)$$ (1)

where $\mathbf{d}\in {ℝ}^{B\times C}$ is a single-trial of EEG data with B frequency bins per channel and C retained EEG channels, ${\mathrm{\Phi }}_{\mathrm{C}}:{ℝ}^{B\times C}\to {ℝ}^m$ is a piecewise linear mapping from the data space to an m-dimensional CPCA-subspace, and $\mathbf{T}:{ℝ}^m\to ℝ$ is a LDA or AIDA transformation matrix.

Once the single-trial data were reduced to 1D spatio-spectral features, a linear Bayesian classifier:

$$f^{\star }\in \left\{\begin{array}{cc}\mathcal{I},& \text{if}P\left(\mathcal{I}\right|f^{\star })>P(\mathcal{W}\left|f^{\star }\right)\hfill \\ \mathcal{W},& \text{otherwise}\hfill \end{array}\right)$$ (2)

was designed in the feature domain, where $P\left(\mathcal{I}\right|f^{\star })$ and $P\left(\mathcal{W}\right|f^{\star })$ are the posterior probabilities of idling and walking classes given the observed feature, f^⋆, respectively. Eq. (2) can be read as: “classify f^⋆ as idle class if the posterior probability of idling is greater than the posterior probability of walking given the feature, f^⋆, and vice versa.” The offline performance of the Bayesian classifier (2), expressed as a classification accuracy, was then assessed by performing 10 runs of stratified 10-fold cross-validation (CV) [21]. To determine the optimal parameters for classification, the lower bound of the frequency range was then increased in 2-Hz steps, and this procedure was repeated until the classifier performance stopped improving, allowing the optimal lower frequency bound to be determined. The upper bound of the optimal frequency range was determined in a similar manner. The classification accuracy was also used to decide whether to use AIDA or LDA as a feature extraction technique. Finally, the parameters of the decoding model, including the optimal frequency range, the feature extraction mapping, and the classifier parameters, were saved for real-time EEG analysis.

Online signal analysis

During online operation, 0.5-sec-long blocks of EEG data were acquired in real time (at a rate of two blocks per second). This rate was limited by the computer processing speed and was empirically found to ensure data acquisition without dropping packets [9]. Of these 0.5 sec blocks, 0.75-sec-long segments (i.e. the most recent 0.5 sec block plus 50% of the previous block) were assembled and processed using the methods described in the Decoding model generation section. Briefly, the EEG signals were first band-pass filtered, then transformed into the frequency domain using FFT, and the power spectral densities over the optimal frequency range were integrated into 2-Hz bins. These spectral data were used as an input for the feature extraction algorithm (1). Finally, using Bayes’ rule, the posterior probabilities of idling and walking classes given the observed EEG feature, f^⋆, were calculated.

Online calibration

Since the BCI system is a binary state machine with an idling and a walking state, a brief calibration procedure was required to identify the posterior probability thresholds to transition between these two states [9]. Using the decoding model developed during the training procedure, the BCI system was set to run in the online mode while participants were prompted to alternate between performing idling and walking KMI for ∼2 min. The posterior probabilities were recorded and their histograms were plotted to determine two thresholds, one to initiate the ambulation of the avatar, $T_{\mathcal{W}}$, and the other to stop the avatar, $T_{\mathcal{I}}$. These two thresholds were initially set as $T_{\mathcal{W}}=\text{median}\left\{P\left(\mathcal{W}\right|f^{\star }\in \mathcal{W})\right\}$ and $T_{\mathcal{I}}=\text{median}\left\{P\left(\mathcal{W}\right|f^{\star }\in \mathcal{I})\right\}$. Then, a brief online test was performed and based on the participant’s feedback, the thresholds were adjusted as necessary. During online BCI operation, the posterior probabilities $P\left(\mathcal{W}\right|f^{\star })$ corresponding to the most recent 1.5 sec of EEG data were averaged and compared to the thresholds $T_{\mathcal{W}}$ and $T_{\mathcal{I}}$, and the state transitions were executed based on the binary state machine diagram given in Figure 1. This step was implemented in order to smooth the sequence of posterior probabilities and reduce noisy state transitions.

Figure 1.

Finite State Machine. The BCI system as a binary state machine with walking and idling states represented by circles. The state transitions are represented by arrows, with transitions triggered by the conditions shown next to the arrows. Self-pointing arrows denote that the system remains in the present state.

BCI and virtual reality environment integration

Using a simulated physics environment (Half-Life 2 Garry’s Mod, Valve Corporation, Bellevue, WA), a VRE was constructed to represent a flat grassland with 10 non-player characters (NPCs) arranged in a linear path in front of an avatar (see Figure 2). During online evaluation, the participants were assigned the task of utilizing idling and walking KMI to control the avatar’s standing and walking, similar to [8,9]. A virtual joystick program [22] was used to interface the BCI software (MATLAB) and the VRE to enable the BCI’s control of the avatar’s walking. Full details on this integration process can be found in Wang et al. [9].

Figure 2.

Avatar within the VRE. The avatar controlled by the BCI within the VRE. The simulator is operated in 3 ^rd person view. The participant uses walking KMI to move the avatar to each NPC and idling KMI to dwell there for at least 2 sec.

Online performance and assessment

To assess the online BCI performance, participants were tasked to move the avatar within two body lengths of each NPC and remain idle at each location for at least 2 sec. On each experimental day, participants repeated this task for 2–8 online sessions. This depended upon the participant’s willingness and availability. In total, each participant underwent between 19 and 29 online sessions performed over 5 experimental days. Depending on the participant’s availability, each experimental day was separated by approximately one week. Two performance measures were recorded for each session: the number of successful stops at the NPCs and the time taken to complete the course. The successful stops were defined as follows. Participants were given one point for dwelling at the designated stop for at least 2 sec. Only a fraction of a point was given for dwelling at the designated stop between 0.5 sec and 2 sec (increasing proportionally with the dwell time duration). Also, no point was given for dwelling less than 0.5 sec, and a full point was awarded for dwelling longer than 2 sec, but this inherently increased the time to completion. Thus, the maximum successful stop score was 10 points. In addition, participants were given a 20-min time limit to complete the course. If the participant could not finish the course within 20 min, the trial was interrupted and the number of successful stops achieved thus far was recorded.

Control experiments

The online performances (number of successful stops and completion time) were compared to random walk to determine if purposeful control was attained. The random walk performance was simulated by uniformly sampling the posterior probabilities between 0 and 1 and following the state transition rules with threshold values presented in Online calibration of the Methods section (details can be found in [9]). To ensure statistical significance, 1000 Monte Carlo (MC) simulation runs were performed and the number of successful stops and completion times were logged (the 20-min completion time limit was enforced as above). In addition, the same task was performed by an able-bodied participant using a manually controlled physical joystick [9].

Statistical tests

Purposeful control was defined as the ability to complete an online session within the allotted 20 min, with performances significantly different from random walk (p<0.01). This was ascertained by comparing the participants’ online performances to MC simulations using a multivariate analysis. To this end, the 2D probability density function (PDF) of each participant’s simulated random walk performance (consisting of two variables: number of successful stops and completion time) was estimated using the Parzen-Rosenblatt window method [18,23]. A constant-value PDF contour was then drawn through each participant’s observed performance point and the corresponding p-value was determined as the fraction of the volume under the random walk PDF outside of this contour. These p-values were used to ascertain a statistically significant difference between the online performances and the simulated random walk. Finally, participants are deemed to have attained purposeful control once they have a single online session that is purposeful based on the definition above.

Composite score

Given the difficulty of interpreting multivariate performance measures across participants and sessions, a single composite score was defined as the following geometric mean:

$$\begin{array}{ll}c=\sqrt{c_s{c}_t}& \end{array}$$ (3)

where c_s and c_t are the normalized performance measures for the number of successful stops and completion times, respectively, i.e.

$$\begin{array}{lll}{c}_s& =\frac{s}{s_{\text{max}}}& \\ c_t& =\frac{t_{\text{max}}-t}{t_{\text{max}}-t_{\text{min}}}\end{array}$$ (4)

Here, s is the participant’s number of successful stops, s_max = 10 is the maximum number of successful stops, t is the participant’s completion time, t_max = 1200 sec is the maximum allowed time, and t_min = 201.52 sec is the minimum time required to complete the course while achieving 10 successful stops. The values of c_s and c_t, and consequently c, range from 0 to 100%, where 100% corresponds to a perfect performance. Note that the use of the geometric mean favors a performance that is balanced over a performance that sacrifices one performance measure over the other (e.g. when a participant finishes the course in a short time while failing to make stops). Also note that the normalization of c_s and c_t ensures that the performance measures are unitless.

Results

Offline performances

After participants underwent the training data collection for each experimental day (see Training procedure section), participant-specific EEG decoding models were generated as described in the Decoding model generation section. Cross-validation of these models resulted in offline classification accuracies that ranged from 60.5% to 92.3% (Table 2), with corresponding p-values between 0.018 and 10⁻²⁰, thus indicating that the classifier performances were well above the chance level (50%). In addition, the participants’ average offline performances over 5 experimental days were 82.3%, 71.8%, 82.3%, 82.5%, and 82.2%, respectively. Finally, the overall average for all 5 participants across all 5 experimental days was 80.2% ± 8.62 (n = 25).

Table 2.

Offline performances

Offline performances
Participant	Day	P (correct | f⋆)(%)	p-value
1	1	71.9 ± 2.2	6.29×10−6
	2	89.4 ± 1.2	1.53×10−17
	3	83.9 ± 2.0	1.30×10−12
	4	84.0 ± 1.9	1.30×10−12
	5	82.2 ± 1.7	6.55×10−12
	Avg.	82.3 ± 6.4	1.26×10−6
2	1	62.2 ± 1.8	6.00×10−3
	2	62.0 ± 1.8	1.05×10−2
	3	60.5 ± 2.0	1.76×10−2
	4	91.6 ± 1.7	1.60×10−19
	5	82.5 ± 1.6	6.55×10−12
	Avg.	71.8 ± 14.3	6.82×10−3
3	1	90.3 ± 1.3	1.66×10−18
	2	83.9 ± 1.1	1.30×10−12
	3	72.8 ± 2.9	2.35×10−6
	4	81.0 ± 2.1	1.35×10−10
	5	83.3 ± 1.2	1.30×10−12
	Avg.	82.3 ± 6.3	4.69×10−7
4	1	74.7 ± 1.7	2.82×10−7
	2	92.3 ± 1.6	1.36×10−20
	3	81.5 ± 1.3	3.07×10−11
	4	80.5 ± 2.0	1.35×10−10
	5	83.5 ± 2.2	1.30×10−12
	Avg.	82.5 ± 6.4	5.64×10−8
5	1	82.7 ± 1.3	6.55×10−12
	2	86.3 ± 1.3	6.56×10−15
	3	78.9 ± 1.4	2.17×10−9
	4	82.2 ± 1.5	6.55×10−12
	5	81.0 ± 1.3	1.35×10−10
	Avg.	82.2 ± 2.7	4.30×10−10

Offline performances represented as classification accuracies estimated using 10 runs of stratified 10-fold CV. The classification accuracy is defined as the probability of correctly classifying a trial given the feature, f^⋆, i.e. $P\left(\text{correct}\right|f^{\star })=P(\mathcal{I}|f^{\star }\in \mathcal{I})P\left(\mathcal{I}\right)+P\left(\mathcal{W}\right|f^{\star }\in \mathcal{W}\left)P\right(\mathcal{W})$, where $P\left(\mathcal{I}\right|f^{\star }\in \mathcal{I})$ and $P\left(\mathcal{W}\right|f^{\star }\in \mathcal{W})$ are defined in Online calibration in the Methods section, and $P\left(\mathcal{I}\right)$ and $P\left(\mathcal{W}\right)$ are the prior probabilities of idling and walking class, respectively.

Spatio-spectral feature extraction maps generated from offline analysis, as described in the Decoding model generation section, revealed salient brain areas and frequency bands underlying idling and walking KMI for each participant (see the supplementary material [Additional file 1] for all feature extraction images from 8–14 Hz). Qualitative analysis of these topographic maps revealed significant variations in the brain areas utilized by the participants while performing this task. For example, Participant 2 used mostly the Cz area (Figure 3), whereas Participant 5 used areas C3 and C4 (Figure 4). Furthermore, the utilized brain areas and frequency bands evolved for each participant over the 5 experimental days. For example, Participant 2 had growing activation of mid-frontal areas up to experimental day 4, followed by a slight shrinkage on day 5. However, one consistent feature among all participants was the activation of mid-frontal areas (over Cz or FCz and adjacent electrodes) mostly in the μ (8–12 Hz) and low β (12–16 Hz) bands.

Figure 3.

Feature Extraction Images of Participant 2. Feature extraction images of Participant 2 for all experimental sessions. Dark colors (red and blue) represent the areas that were responsible for encoding the differences between idling and walking KMI. The EEG power in the 9–13 Hz bins in the mid-frontal (FCz), central (Cz) and central-parietal (CPz) areas were the most salient.

Figure 4.

Feature Extraction Images of Participant 5. Feature extraction images of Participant 5 for all experimental sessions. The EEG power in the 9–13 Hz bins in the mid-frontal (Fz), lateral central (C3 and C4) and lateral central-parietal (CP3 and CP4) areas were the most informative for encoding the differences between idling and walking KMI.

Online calibration

After the short calibration procedure (see Online calibration in the Methods section), the state transition thresholds, $T_{\mathcal{I}}$ and $T_{\mathcal{W}}$, were determined from the distributions of the posterior probabilities, $P\left(\mathcal{W}\right|f^{\star })$ (see Figure 5 for an example), and their values are presented in Table 3. Note that in an ideal situation, $P\left(\mathcal{W}\right|f^{\star }\in \mathcal{W})$ and $P\left(\mathcal{W}\right|f^{\star }\in \mathcal{I})$ should cluster around 1 and 0, respectively, but as long as these probabilities are separable, online BCI control should be achievable. The values of $T_{\mathcal{I}}$ and $T_{\mathcal{W}}$ ranged from 0.07 to 0.70 and from 0.09 to 0.90, respectively, and the average across all participants on all experimental days was 0.40 and 0.62 for $T_{\mathcal{I}}$ and $T_{\mathcal{W}}$, respectively.

Figure 5.

Calibration Histograms. Histograms of the posterior probability of walking KMI given idling, $P\left(\mathcal{W}\right|f^{\star }\in \mathcal{I})$, and walking KMI given walking, $P\left(\mathcal{W}\right|f^{\star }\in \mathcal{W})$, for Participant 3. Note that $P\left(\mathcal{W}\right|f^{\star }\in \mathcal{I})=1-P(\mathcal{I}|f^{\star }\in \mathcal{I})$. Dashed lines indicate the 25%, 50%, and 75% quantiles.

Table 3.

Online parameters

Online parameters
Participant	Day	$$T_{\mathcal{I}}$$	$$T_{\mathcal{W}}$$
1	1	0.37	0.47
	2	0.42	0.67
	3	0.45	0.55
	4	0.22	0.39
	5	0.35	0.44
2	1	0.5	0.5
	2	0.07	0.09
	3	0.42	0.45
	4	0.6	0.7
	5	0.6	0.66
3	1	0.2	0.3
	2	0.57	0.62
	3	0.26	0.75
	4	0.7	0.9
	5	0.4	0.9
4	1	0.62	0.8
	2	0.61	0.8
	3	0.38	0.45
	4	0.4	0.6
	5	0.58	0.65
5	1	0.3	0.85
	2	0.4	0.7
	3	0.3	0.7
	4	0.4	0.8
	5	0.3	0.85

The stop (and start) parameters,$T_{\mathcal{I}}$ (and$T_{\mathcal{W}}$), for online operation as determined by the calibration session.

Online performances

All participants were able to achieve purposeful online control immediately (on day 1), with the exception of Participant 2 (on day 2). The online BCI performances upon achieving purposeful control are summarized in Table 4. The average stop score was 7.4 ± 2.3, and the average completion time was 273 ± 51 sec across all online sessions (n=124). Note that ideally, the course should be completed within 211 sec with a fixed walking time of 191 sec and idling time of 20 sec, so the additional time spent to complete the course was due to errors (e.g. false stops, difficulties transitioning to the walk state, overestimating the 2 sec dwelling time at each character). Also, only minimal lapses in BCI control occurred, as only 5 out of 124 online sessions (∼4%) were non-purposeful. Examples of the best online performances for each participant are shown in Figure 6 along with the simulated random walk PDFs. For additional comparison, the average performance achieved by a manually controlled physical joystick was 9.38 ± 0.95, and on their best day, Participants 3, 4, and 5 achieved a similar number of successful stops; however, no participants were able to complete the course as fast as manual control. Two time-space courses of the best online sessions (see Figure 7) show only a single false alarm and a single omission for Participant 3, and no false alarms and only two omissions for Participant 5. Note that the online performances of the other subjects were not recorded with this level of detail (the computer code was modified in the late stage of the study to accommodate for this function), and it is likely that other subjects’ false alarms and omissions were higher than the those reported in Figure 7. Therefore, while not universally applicable, these results illustrate the level of control achievable by this BCI system. Finally, a video showing a representative online session from Participant 3 is provided as a supplementary material [see Additional file 2].

Table 4.

Online performances

Participant		Completion time	Successful stops	Session
		mean ± std (sec)	mean ± std	breakdown
1	n=29	275 ± 45	6.2 ± 1.8	(26, 3)
	Best: Day 5	298 ± 77	6.8 ± 2.3
	Random Walk	227 ± 7	5.0 ± 1.3
2	n=25	271 ± 66	5.7 ± 2.3	(24, 1)
	Best: Day 5	293 ± 26	8.1 ± 1.2
	Random Walk	1445 ± 118	9.0 ± 0.9
3	n=24	277 ± 65	9.4 ± 1.3	(24, 0)
	Best: Day 4	231 ± 8	10.0 ± 0.0
	Random Walk	>1200	0.1 ± 0.25
4	n=19	289 ± 43	8.3 ± 1.8	(18, 1)
	Best: Day 1	264 ± 12	8.9 ± 0.3
	Random Walk	>1200	4.3 ± 0.7
5	n=27	258 ± 31	7.7 ± 2.1	(27, 0)
	Best: Day 4	260 ± 17	10.0 ± 0.0
	Random Walk	>1200	5.1 ± 1.4
All participants		273 ± 51	7.4 ± 2.3	(119, 5)
Physical joystick		205.72 ± 4.2	9.38 ± 0.95

Average online performances, including completion time, and number of successful stops for each participant, upon achieving purposeful control, which was on day 2 for Participant 2, and on day 1 for all other participants. Note that the completion time consists of a fixed walking time (191 sec) and a variable amount of idling time. Also presented are the average online performances of each participant’s best experimental day. Performances of random walk are shown for comparison. Sessions are also broken down by number of purposeful and non-purposeful performances, i.e. (p<0.01, p≥0.01).

Figure 6.

Best Online Performances. Best online performances (completion times and stop scores) of each participant. Performances are marked by crosses with associated composite scores shown in parenthesis. PDF of the random walk simulations (contour lines) are also shown for Participant 1, but are absent for other participants as the contours lie outside of the alloted 20-min limit. Note that all performances are purposeful (p<0.01).

Figure 7.

Online BCI Operation Time-Space Courses. Two representative time courses of the best online sessions for (top) Participant 3 and (bottom) Participant 5, where both subjects were able to achieve a high number of successful stops with a short completion time. False positives (i.e. the avatar walked when the participant intended to stop) are marked by orange segments, and false negatives (i.e. the avatar stopped when the participant intended to walk) are marked by red segments.

The composite scores were calculated from the online performance measures and are summarized in Table 5. In general, the performances improved significantly over time; the average on day 1 was 77.8% and the average on day 5 was 85.7% (p = 0.0302). For comparison, the composite score of the joystick task is also shown. On their best days, Participants 3, 4 and 5 achieved performances similar to those of the manually controlled joystick, reaching nearly perfect performances (100%).

Table 5.

Composite online performance score

		Composite score (%)
Participant		Day 1	Day 2	Day 3	Day 4	Day 5
1		66.2 ± 3.1	76.8 ± 9.1	75.0 ± 11.3	74.4 ± 11.6	76.9 ± 11.9
	Best	69.7	90	85.6	80.5	89.9
2			69.9 ± 11.2	69.3 ± 13.5	59.9 ± 12.3	85.4 ± 5.4
	Best		86.8	89.3	68.9	90.4
3		86.4 ± 3.2	87.5 ± 14.2	93.0 ± 5.6	98.4 ± 0.4	97.3 ± 2.2
	Best	89.9	95.2	98.5	99	98.6
4		91.5 ± 1.7	90.1 ± 5.3	89.7 ± 4.1	79.4 ± 16.3	80.7 ± 2.3
	Best	93.2	95.5	93.1	96.1	82.3
5		66.4 ± 9.0	80.7 ± 5.1	93.5 ± 3.5	96.9 ± 0.9	88.7 ± 6.1
	Best	76.7	86.8	97.3	98.2	98.3
Average		77.8 ± 13.0	79.8 ± 11.5	81.7 ± 13.9	80.7 ± 17.7	85.7 ± 10.2
Joystick				96.5 ± 3.8

Average and best composite online performance score for all participants for each experimental day as calculated using Eqs. 3 and 4. The composite scores for random walk and the physical joystick are also shown for comparison. Note that Participant 2 did not achieve purposeful control on day 1 and was therefore unable to participate in online sessions on this day.

Discussion

The results of this study show that participants with paraplegia or tetraplegia due to SCI can operate a non-invasive BCI-controlled avatar within a VRE to accomplish a goal-oriented ambulation task. All participants gained purposeful online BCI control on the first day after undergoing a 10-min training session, with the exception of Participant 2, who did not attain control until day 2. In addition, BCI control was maintained over the course of the study, and performance continued to improve in 4 out of 5 participants. These findings suggest that a BCI-controlled lower extremity prosthesis for either gait rehabilitation or restoration may be feasible.

Offline performances

The offline classification accuracies varied across participants and experimental days, but were significantly above the chance level performance (50%). Similar to able-bodied participants engaged in the same task [8,9], a short 10-min training session was sufficient for the data-driven machine learning algorithm to generate accurate participant-specific EEG decoding models for this population. The topographic maps of these models (e.g. Figure 3 and Figure 4) showed that the spatio-spectral features underlying the differences between walking and idling KMIs varied across participants and evolved over experimental days. The differences in the brain areas and EEG frequencies across participants may be due to variations in cortical reorganization following SCI [10-13], or due to differences in imageries employed by each participant (e.g. the KMI of walking instructions may have been interpreted differently by each participant). Nevertheless, all participants showed activation of mid-frontal areas, which likely overlay the pre-motor and supplementary motor areas, as well as the pre-frontal cortex. Their activation during walking KMI is consistent with functional imaging findings, such as those in [24]. Another common pattern was the presence of activity near bilateral, lateral central-parietal electrodes, which likely represents the arm sensorimotor areas. A similar pattern was observed in able-bodied individuals [9], and is hypothesized to originate from arm swing imagery. Finally, the evolution of the feature extraction maps over the 5 experimental days may be indicative of a neuro-plasticity process associated with practice and learning [25,26].

The spatio-temporal variations of walking KMI activation patterns demonstrate the necessity of a data-driven machine learning approach for rapid acquisition of intuitive BCI control. First, our approach accommodates for the variations of these activity patterns across participants, as well as their evolution over time. Second, it facilitates rapid acquisition of online BCI control, presumably by enabling participants to utilize intuitive mental strategies. The user training time necessary to acquire purposeful BCI control in this study is significantly shorter than those of other BCI studies where users must learn a completely new cognitive skill to modulate pre-selected EEG features, such as the μ-rhythm over lateral central areas [27]. Finally, this approach carries a significant potential value in the future practical implementation of BCI-prostheses, as it may drastically reduce the training time needed to attain purposeful and useful control of self-paced BCIs from a timescale of weeks to months to one of minutes to days. This in turn may significantly reduce the cost of training users to operate future BCI-prostheses.

Online performances

The results presented in Table 4 show that once purposeful control was achieved, it was maintained in 96% of all online sessions. In addition, 3 out of 5 participants achieved successful stop scores similar to those obtained using a manually controlled joystick. Even though no participants were able to complete the course as fast as manual control, it is encouraging that the average composite scores increased significantly over the course of the study. Furthermore, for Participants 3 and 5, the average composite scores (Table 5) approached 100% by the toward of the study. Therefore, not only was online control significantly different from random walk, but it was also meaningful. Given this trend, additional training and practice may help further improve performance, possibly to the point of approaching that of the manually controlled joystick.

Comparison with other studies

The similarities and differences between the present study and other related studies in the field are shown in Table 6. While this study represents a continuation of our prior work [8,9], it demonstrates the first successful application of a BCI-controlled walking simulator in a SCI population. The present approach also uses KMI of walking/idling as a control strategy, which is intuitive because it matches the walking task. In contrast, related studies have used idling/foot imagery [28] or foot/hand imagery [29] to achieve control within a VRE. Furthermore, the data-driven machine learning approach to generate EEG decoding models in this study resulted in a very short training time, while those presented in [28,29] resulted in a significantly longer training time (see Table 6). Finally, both our prior work [9] and [28] report on the BCI performances of a single SCI participant, whereas the current study includes a larger SCI population. This suggests that our BCI-VRE system may be generalizable in SCI individuals.

Table 6.

Comparison with other studies

Study	Mental strategy	Training time	Sample size
Present	KMI of walking/idling	10 min	5 SCI
Wang et al. [9]	KMI of walking/idling	10 min	8 AB, 1 SCI
Wang et al. [8]	KMI of walking/idling	12 min	3 AB
Pfurtscheller et al. [29]	Foot/hand motor imagery	3–5 mo	3 AB
Leeb et al. [28]	Foot/idling motor imagery	4 mo	1 SCI

A comparison of the present study with similar studies in the field.

Future work

The success of this study suggests that restoring BCI-controlled ambulation after SCI may be feasible. To formally test this concept, integration of our BCI system with a physical ambulation prosthesis, such as a robotic gait orthosis (e.g. Lokomat, Hocoma, Volketswil, Switzerland) or an FES system (e.g. Parastep, Sigmedics, Fairborn, OH) is necessary. If such a system is successfully tested in a small SCI population, clinical trials to develop a commercial BCI-prosthesis would be warranted. Finally, these research efforts may ultimately lead to the development of a permanent BCI prosthesis for ambulation; for example, the integration of an invasive BCI system with an invasive FES system for ambulation. This type of system would eliminate the need for any preparation or mounting of EEG, while potentially achieving better performances.

Conclusions

This study shows that SCI participants can purposefully operate a self-paced BCI-VRE system in real time, and that the current BCI design approach is able to overcome the potential problems associated with variations in neurophysiology due to cortical reorganization after SCI, learning and plasticity processes, and differences in KMI strategies. Furthermore, the system satisfies the requirements of an ideal BCI-lower extremity prosthesis [9], namely: intuitiveness, robustness, and short training time. The operation of the system is intuitive as it enabled participants to use walking KMI to control the ambulation of the avatar. The system is robust in that the data-driven decoding methodology successfully accommodated for person-to-person and day-to-day variations in the neurophysiological underpinnings of idling and walking KMI behaviors. In addition, participants were able to maintain purposeful online control over the course of several weeks, further underscoring the system’s robustness over time. Finally, the system required only a short training time, as BCI control was generally attained after only a 10-min training data collection procedure followed by a 2-min calibration session on the 1^st experimental day (for 4 out of the 5 participants).

In summary, the high level of control achieved by SCI subjects over the course of the study indicates that BCI-controlled lower extremity prostheses for gait rehabilitation or restoration after SCI may be feasible. The proposed BCI-VRE system may also serve as a training platform for BCI-controlled lower extremity prostheses once they become widely available.

Abbreviations

SCI: Spinal cord injury; BCI: Brain-computer interface; VRE: Virtual reality environment; KMI: Kinesthetic motor imagery; EEG: Electroencephalogram; FES: Functional electrical stimulation; EMG: Electromyogram; FFT: Fast Fourier Transform; CPCA: Classwise principal component analysis; LDA: Fisher’s linear discriminant analysis; AIDA: Approximate information discriminant analysis; 1D: One-dimensional; CV: Cross-validation; NPC: Non-player character; MC: Monte Carlo; PDF: Probability density function; ASIA: American Spinal Injury Association.

Competing interests

CEK received salary from HRL Laboratories, LLC (Malibu, CA). The remaining authors declare that they have no competing interests.

Authors’ contributions

CEK carried out the experiments, collected and analyzed the data, and wrote the article. PTW programmed the brain-computer interface software, assisted with carrying out the experiments, collecting the data, and analyzing the data. LAC contributed to conception of the study. AHD conceived and designed the study, implemented the VRE, recruited and consented participants, supervised the experiments, and co-wrote and proofread the article. ZN conceived and designed the study, designed the signal processing, pattern recognition, and classification algorithms, and co-wrote and proofread the article. All authors read and approved the final manuscript.

Supplementary Material

Additional file 1

Feature extraction images of all participants for all experimental sessions over the 8-14 Hz frequency band. Dark colors (red and blue) represent the areas that were responsible for encoding the differences between idling and walking KMI.

Additional file 2

Subject with tetraplegia due to syringomyelia uses the BCI to control the ambulation of a virtual reality avatar. To move the avatar, the subject imagines walking, and to stop, the subject relaxes. The goal is to move the avatar across the field and make 10 stops within 2 body lengths of non-player characters. Each stop should be at least 2 seconds long. This subject can complete the task while making all 10 stops with a completion time that is only 15 seconds longer than that of a manually controlled joystick.

Acknowledgements

This study was funded by the Roman Reed Spinal Cord Injury Research Fund of California (RR 08-258 and RR 10-281), and was partially funded by the Long Beach VA Advanced Fellowship Research Award.