EEG-based classification for elbow versus shoulder torque intentions involving stroke subjects

Abstract

The ultimate aim for classifying elbow versus shoulder torque intentions is to develop robust brain–computer interface (BCI) devices for patients who suffer from movement disorders following brain injury such as stroke. In this paper, we investigate the advanced classification approach classifier-enhanced time–frequency synthesized spatial pattern algorithm (classifier-enhanced TFSP) in classifying a subject’s intent of generating an isometric shoulder abduction (SABD) or elbow flexion (EF) torque using signals obtained from 163 scalp electroencephalographic (EEG) electrodes. Two classifiers, the support vector classifier (SVC) and the classification and regression tree (CART), are integrated in the TFSP algorithm that decomposes the signal into a weighted time, frequency and spatial feature space. The resulting high-performing methods (SVC-TFSP and CART-TFSP) are then applied to experimental data collected in four healthy subjects and two stroke subjects. Results are compared with the original TFSP, and significantly higher reliability in both healthy subjects (92% averaged over four healthy subjects) and stroke subjects (75% averaged over two subjects) are achieved. The accuracies of classifier-enhanced TFSP methods are further improved after a rejection scheme is applied (~100% in healthy subjects and > 80% in stroke subjects). The results are among the highest reliability reported in literature for tasks with spatial representations on the motor cortex as close as shoulder and elbow. The paper also discusses the impact of applying rejection strategy in detail and reports the existence of an optimal rejection rate on a stroke subject. The results indicate that the proposed algorithms are promising for future use of rehabilitative BCI applications in neurologically impaired patients.

1. Introduction

Brain–computer interface (BCI) has been gaining much attention as an approach to detect an individual’s intents and to convert brain signals into usable control commands [1–4]. The rapid development of this technique demonstrated that scalp EEG-based BCI is a feasible way to provide a non-muscular control of a cursor on the screen [2,5]. Application of these studies in rehabilitation engineering has been targeting ‘locked-in’ patients. Differently, we aim at exploring the ability to use a scalp EEG-based BCI to control a neural prosthesis or other devices to help patients who suffer from movement discoordination (such as patients with moderate to severe motor impairment due to stroke or cerebral palsy). An example of this discoordination is the abnormal coupling that exists between the shoulder abduction (SABD) and elbow flexion (EF) following stroke. We hope to help these patients overcome this abnormal coupling between shoulder and elbow by a well-designed BCI approach that can classify the individual’s intent of a shoulder or elbow motor task with high accuracy. This is a challenging task for the following reasons: First, shoulder and elbow are closely represented at the motor cortex (about 10 mm between their respective centers of activity [6]. Second, the cortex has been shown to reorganize following stroke: Yao and Dewald [7,8] have shown that there is a significant increase in the overlap of cortical activity following stroke, which is likely to make the classification between shoulder and elbow tasks considerably more difficult than in able-bodied subjects.

Recently, we experimented using a time–frequency synthesized spatial patterns (TFSP) BCI algorithm [9] to classify the intent of the generation of isometric SABD and EF torques with high recognition rate in four healthy subjects. The TFSP algorithm provides detailed analysis of event-related desynchronization (ERD) in spontaneous EEG rhythmic activity by decomposing the signal into a time, frequency and spatial feature space. Time–frequency segments are weighed based on their corresponding contribution and the weights are synthesized during final classification. We found that the TFSP algorithm is an effective EEG-based BCI approach for classifying SABD and EF intentions [10]. However, its prediction power is constrained (especially for stroke subjects) since a simple comparison with minimal training was used for the classification in the algorithm.

In order to achieve the full potential of the TFSP algorithm and investigate its possible use in classifying SABD–EF torque intentions involving stroke subjects, in this paper we propose an improved version of TFSP called classifier-enhanced TFSP and present detailed experimental results in four healthy and two stroke subjects. In the classifier-enhanced TFSP, classifiers with explicit training process and high recognition reliability are incorporated into the algorithm, which are used to derive weight distribution of time–frequency grid with the help of a validation process; the trained classifiers are used to obtain predicted labels on each time–frequency segment for a new EEG trial. In addition, a rejection module is added to the algorithm to further improve accuracy that may have important implications for the use of practical rehabilitative devices.

Various classifiers have been employed in BCI technique including linear classifiers such as Fisher’s linear discriminant [11], signal space projection [12], and nonlinear classifiers such as distinction sensitive learning vector quantization (DSLV) [13], and the hidden Markov model [14,15]. Following the consensus on choosing BCI classifiers based on low error rate and fast response time [16,17], we chose to introduce support vector classifier (SVC) [18,19] and classification and regression tree (CART) [20,21] into the TFSP. To our knowledge, these two high performing classifiers have never been explored in the context of BCI for a challenging task such as separating shoulder versus elbow motor intents and deserve to be studied in this context. The first classifier SVC has been widely regarded as one of the most reliable classifiers with good generalization capability [21,22]: when most classifiers are based on empirical risk minimization that minimizes the summed error on training samples which may lead to overfitting, SVC follows the principle of structural risk minimization which targets better prediction of unknown testing samples and thus increases the reliability during applications. The second classifier CART is a multi-stage classification scheme that has a fast testing phase. It follows one path of a trained tree, goes through a set of linear decisions (bounded by the level of the tree) and stops when reaching any leaf node with a classification target. In this paper, the TFSP algorithm with SVC as the classifier is called SVC-TFSP, and similarly the TFSP algorithm with CART is named CART-TFSP.

In this study, using our classifier-enhanced TFSP algorithms (SVC-TFSP and CART-TFSP), we are able to achieve a very high recognition rate of 92% averaged in four healthy subjects when classifying a shoulder versus elbow task. In addition, two stroke subjects are included in the experiments of this paper, and an average recognition rate of 75% is achieved, which is better than the results observed with original TFSP. The rejection scheme reduces the error rate and improves the accuracy even further: an accuracy rate higher than 80% is achieved in stroke subjects while an accuracy rate close to 100% in every healthy subject. The results suggest that, the classifier-enhanced TFSP BCI algorithm can reliably detect a subject’s intent of generating a shoulder or elbow torque; and thus may provide a reliable control for neural prostheses using non-invasive high-density scalp EEG.

2. Experimental setup and signal preprocessing

2.1. Experimental protocol

Scalp EEG signals were collected from four able-bodied and two stroke subjects. Table 1 lists the information of these subjects.

Table 1.

Subject information.

Subject	Age	Sex	Affected hand	Dominant hand	Site of lesion	Fugl–Meyer score
N1	46	M	N/A	R	N/A	N/A
N2	31	M	N/A	R	N/A	N/A
N3	31	F	N/A	R	N/A	N/A
N4	35	M	N/A	R	N/A	N/A
S1	60	M	L	L	R. Posterior limb of internal capsule	26/66
S2	51	F	R	R	L. Dorsal lateral SMA and PM. Subcortical white matter	35/66

N* are able-bodied subjects; S* are stroke subjects; M = male; F = female; R = right limb; N/A = not available or applicable; SMA = supplementary motor area; PM = primary motor cortex.

Each subject learned to self-initiate the generation of isometric SABD or EF at a level of 25% of his/her maximum voluntary torques (MVTs). EEG and torques were collected during the generation of isometric elbow/shoulder torque.

Subjects were cast at the wrist and secured to a six degree of freedom (DOF) load cell with shoulder at 70° abduction angle. The tip of the hand was aligned with the median sagittal plane of the subject and located at a distance from the body which yields an elbow angle of 90°, with 0° representing full extension of the elbow, and the shoulder at approximately a 40° flexion angle. In order to minimize the effect of trunk muscle activation, subjects were seated in a Biodex chair with the trunk secured and the shoulders strapped to the back of the chair. A computer monitor was placed in front of the subject to provide visual feedback of the torque generation during the training protocol.

Scalp recordings were made using a 163-channel EEG system with active electrodes. The electrodes are mounted on a stretchable fabric based on a $\frac{10}{20}$ system positioned as illustrated by Fig. 1. The cap was fitted on the head of the subject lining the Cz electrode with the intersection of the planes defined by the nasion, inion, and pre-auricular points. The skin under each electrode site was prepared and conductive gel was injected to achieve electrode impedances lower than 5 kΩ throughout the experiment. EEG data were collected at 1000 Hz sampling rate. Antialiasing filtering (100 Hz) was provided before data acquisition. The system was equipped active electrodes that provide a first amplification stage, allowing for the recording of EEG signals with a higher SNR and quicker preparation.

Fig. 1.

The electrode montage. For EEG signal recording, 163 electrodes were used: scattered regions A to F. Two additional eye movement detection electrodes were also placed on the zygomatic positions for horizontal eye movement and on the supra- and infra-orbital margins for detection of vertical eye movement.

2.2. Signal preprocessing

The recorded 163-channel EEG signals were preprocessed using Brain Vision Analyzer V1.04 software (Brain Products GmbH, München, Germany). Using this software, we first manually removed the noisy channels, bad intervals of eye artifacts and drifting signals. The remaining data were segmented from −1800 to −100 ms pre-torque onset, and then were baseline corrected and down sampled to 256 Hz. In order to reduce the smearing effects on EEG measurements due to the head volume conductor, a spatial high-pass filter, i.e., finite difference surface Laplacian (SL) [23], was applied to signal from each of the channels before exporting the data for further analysis. As a result, peripheral electrodes were removed and preprocessed EEG signals from the remaining inner 131 electrodes were exported for BCI classification. Fig. 2 shows some typical time traces of the EEG signals.

Fig. 2.

Time traces of EEG signals in different channels for subject S1 shoulder abduction. The traces are for the first training trial. Due to space limit, only signals from channels A3-A22 are shown.

3. Methodology—classifier-enhanced TFSP algorithm

In this paper, two classifier-enhanced TFSP algorithms, namely SVC-TFSP and CART-TFSP, are developed based on the TFSP algorithm by Wang et al. [9] and Deng et al. [10]. Fig. 3 shows the complete flow chart of classifier-enhanced TFSP BCI algorithm including the training and testing processes. In an effort to remain brief, we explain each of the modules following the data flow sequence while focusing on modules newly developed in this paper.

Fig. 3.

Flow chart of the classifier-enhanced TFSP algorithm.

3.1. Presorting and time–frequency feature decomposition (modules 1, 2 and Te1)

First of all, a presorting procedure divides EEG trials for each motor task into two subsets of different levels of covariance complexity, using the average complexity value of all trials as the dividing line. The purpose of this step is to consider the possibility that subjects may use different strategies for the same motor task [10].

Time–frequency feature decomposition is then conducted on the presorted data using the TFSP algorithm [9,10]. EEG signals are decomposed ranging from 5 to 34 Hz into a series of frequency bands (totally 13 bands) using a group of constant Q value (Q = 4) band-pass filters. We then use Hilbert transformation to extract the profiles of the oscillatory activities in the temporal domain, and the resulting profiles were divided into equal-length intervals (55 ms of each interval) with a 50% overlap. Following the divisions in both frequency and time domains, we calculated the instantaneous power (i.e. integration of the profile) in each of the time–frequency grid in the spatial domain. Accordingly, a single-trial EEG signal was represented in a new spatial, time and frequency space by a time frequency spatial pattern p_131*61*13. (vectors and matrices are represented by bold font with dimensions specified using subscripts.) For details of above two procedures, see [10].

3.2. Classifier training (module 3)

After the features are extracted, a training data set is used to train the classifier. This training process was not included in the original TFSP algorithm. Instead, in original TFSP algorithm, classification was done by a similarity comparison of the trial with different types of average spatial patterns in training data [9]. In this study, we introduce the classifiers of high recognition abilities—SVC and CART—into the algorithm. An explicit classifier training process (the module 3 in Fig. 3) is now included for each of these two classifiers.

(1) Support vector classifier (SVC)

The basic idea of SVC is to find an optimal separating hyperplane for different classes [19,22]. When there are only two classes involved, like in this study, the hyperplane is a line. The training process of SVC obtains the weight vector w_s and offset b of the hyperplane by solving the optimization problem that maximizes the margin of the two classes as follows (the margin is the distance between the bounding planes of the two data sets and equivalent to 2/||w_s ||₂):

$$\begin{array}{l}\underset{ws,b}{min}\left\{{\left|\right|w_s\left|\right|}_2+C\sum _{i=1}^I{\xi }_i^2\right\}\\ \text{subject}\text{to}{l}_i(w_s·\mathrm{\Phi }({\mathbf{p}}_i(t,f))+b)\ge 1-{\xi }_i,i=1\dots I\end{array}$$ (1)

where ξ_i are called slack variables which ensure that the problem has a solution when the data are not linearly separable; l_i is the label of the ith training trial, I is the size of the training set, C is the regularization parameter, and Φ(·) is the implicit mapping of the sample from the raw data space to a feature space, which enables SVC to work with nonlinear problems. It satisfies the property 〈Φ(x₁),Φ(x₂)〉 = k(x₁,x₂), with 〈·, ·〉 as inner product of two vectors and k as a potentially nonlinear kernel function.

The optimization problem can be solved using Lagrange theory, which will give us the weight vector w_s and offset b as below:

$$w_s=\sum _{i=1}^I{\alpha }_i{l}_i\mathrm{\Phi }({\mathbf{p}}_i(t,f)),$$

where α_i are the Lagrange coefficients.

$$b=\frac{1}{\mid SV\mid }\sum _{j\in SV}\left(l_j-\sum _{i=1}^I{\alpha }_i{l}_{i}k({\mathbf{p}}_j(t,f),{\mathbf{p}}_i(t,f))\right)$$ (2)

where SV is the set of support vectors (a subset of training samples that participate in defining the optimal hyperplane, and hence the name of the classifier).

(2) Classification and regression tree (CART) [20]

For CART, the training process is to build a decision tree based on training data. At each node of the tree, a split of branches is made during training such that the decrease in Gini criterion is maximized. Gini criterion for a given node t is defined as

$$G(t)=\sum _{i\ne j}p({\omega }_i\mid t)p({\omega }_j\mid t)$$ (3)

where class labels ω_i, ω_j ∈{abdI, abdII, efI, efII}, p(ω_i|t) is the proportion of training samples in node t with class label ω_i. Once the tree is built, each branch is associated with a rule generated by splitting (which indicates the value range of spatial pattern) and each leaf node of the tree is associated with one category (which is the majority label of samples in the leaf node).

3.3. Weighting process (module 4)

Once the training is complete, we apply the trained classifiers on a validation data set, which is distinct from the training and testing data sets. When using SVC as the classifier, the class label for a validation trial p(t,f) is obtained as

$${\widehat{d}}_{(\mathit{SVC})}(t,f)=\text{sign}\left(\sum _{j\in SV}{\alpha }_j{l}_{j}k({\mathbf{p}}_j(t,f),\mathbf{p}(t,f))+b\right)$$ (4)

where p_j(t,f) is the spatial pattern of jth trial in the set of support vectors SV; l_j is the desired label of the jth trial in SV.

When a CART is used as the classifier, the validation trial goes down the branches following the rules until reaching a leaf node, and we have the class label for a validation trial as

$${\widehat{d}}_{(\mathit{CART})}(t,f)=\{\begin{array}{ll}+1\hfill & \text{if}\text{leaf}\text{node}\text{is}\text{associated}\text{with}\mathit{abd}\text{I}\text{or}\mathit{abd}\text{II}\hfill \\ -1\hfill & \text{if}\text{leaf}\text{node}\text{is}\text{associated}\text{with}ef\text{I}\text{or}ef\text{II}\hfill \end{array}$$ (5)

In this study, LibSVM package [24] with Matlab interface is used for the implementation of SVC and Matlab statistics package is used for CART. The size ratio of training and validation sets is set to 9:1 for stroke subjects and 7:3 for healthy subjects based on a rough search for optimal ratios (increased difficulty in classification may cause having more training samples necessary for stroke subjects, which offsets the disadvantage of a smaller test set.) The same classification process is applied to module Te2 for a testing trial to obtain class labels on each time–frequency segment.

After a class label is assigned to a time–frequency grid for a validation trial, a weight w(t,f) ⊂ [0,1] is determined based on the recognition rate of going through all single trials in the validation dataset as follows:

$$w(t,f)=\{\begin{array}{ll}{[(r(t,f)-Th)/(1-Th)]}^4,\hfill & r(t,f)>Th\hfill \\ 0,\hfill & r(t,f)\le Th\hfill \end{array}$$ (6)

where Th is the threshold and grids with recognition rates lower than Th are discarded (set to 0.4 in our experiment). r(t,f) is the recognition rate achieved on the specific time–frequency grid (the left box in the “weighting process” of Fig. 3). With the introduction of validation set, r(t,f) is calculated as below:

$$r(t,f)=\frac{{\sum }_{i=1}^{N_v}\varphi ({\widehat{d}}_i(t,f),l_i)}{N_v},\varphi (\theta ,\gamma )=\{\begin{array}{ll}+1,\hfill & \theta =\gamma \hfill \\ 0\hfill & \text{otherwise}\hfill \end{array}$$ (7)

where N_v is the number of samples in the validation set; l_i is the desired label for the ith trial in the validation set; d̂_i(t, f) is the class label of the ith trial in the validation set assigned by the trained classifier SVC or CART. The outcome of module 4 is the weight matrix on time–frequency grid, W(t,f)_61*13.

3.4. A weighted synthesis (module Te3)

The weight matrix W(t,f)_61*13 is used to derive a judging decision R(p) regarding the class of a single-trial EEG in the testing dataset by synthesizing the weighted results over the entire time–frequency grid:

$$R(\mathbf{p})=\text{sign}\left(\sum _{t=1}^T\sum _{f=1}^{F}w(t,f)\ast d(t,f)]\right)$$ (8)

where p represents the spatial pattern of a testing trial. d(t,f) is the label assigned to the testing trial by the classifier SVC or CART on the specific time–frequency domain. A positive R(p) indicates that the synthesized decision for the test trial is a SABD-related event, while a negative one indicates a EF-related event.

3.5. Rejection scheme for classifier-enhanced TFSP algorithm (module Te4)

A rejection scheme is introduced in the classifier-enhanced TSFP algorithm by setting a threshold on the absolute value of the synthesized sum. With the rejection scheme, the synthesized decision of Eq. (8) is now changed to the following decision making process, with the threshold denoted as Th_REJ:

If $\mathit{abs}\left({\sum }_{t=1}^T{\sum }_{f=1}^{F}w(t,f)\ast d(t,f)]\right)<Th\_\mathit{REJ}$

then test trial i is rejected.

else if $\text{sign}\left({\sum }_{t=1}^T{\sum }_{f=1}^{F}w(t,f)\ast d(t,f)]\right)=+1$

then test trial i is a SABD-related event.

else

test trial i is an EF-related event.

End

In the above rejection scheme, w(t,f) is calculated using Eq. (6); d(t,f) ∈ {−1,+1}: +1 if the classifier considers the trial as SABD on the specific time–frequency segment and −1 otherwise.

With rejection scheme introduced, it is common to evaluate the accuracy of an algorithm which is defined as

$$\mathit{Acc}=\frac{N_c}{N_t-N_r}$$ (9)

where N_c is the number of correctly predicted test trials, N_t is the total number of test trials, and N_r is the number of rejected trials. Accuracy indicates the reliability of an algorithm (i.e., how reliable is the algorithm’s decision on the label of a test trial if it is not rejected). When no rejection is generated, accuracy is equivalent to recognition rate.

The value of Th_REJ is lower bounded by 0 and upper bounded by either of the two cases: (1) when all test trials in a test set are rejected and we can not further increase the Th_REJ; or (2) the accuracy has reached 100% and it is no longer necessary to increase the Th_REJ.

4. Experimental results

We used the leave-five-out cross-validation method (i.e., dividing the whole dataset into multi-folds with five trials per fold for each of the motor tasks) to train and test the BCI algorithm. Using this method, one of the folds was taken iteratively as the testing data, while the rest were used as the training and validation data. The result for a subject is the average of testing performances over all folds.

In order to compare with the original TFSP, we reported the recognition rates of CART-TFSP and SVC-TFSP in the same four healthy subjects as compared with our previous work [10] and list the results in Table 2. Table 2 shows that, in four healthy subjects, the results of CART-TFSP and SVC-TFSP are comparably high with the average recognition rate of 92%; both types of classifier-enhanced TFSP reported average recognition rates of 3% higher than the original TFSP. The results we achieved are the highest recognition rates reported for tasks with such close spatial representations on the motor cortex.

Table 2.

Recognition rates (without rejection scheme) for four able-bodied subjects.

Subject #	Original TFSP (%)	CART-TFSP (%)	SVC-TFSP (%)
N1	90	91	92
N2	86	88	85
N3	91	94	96
N4	89	93	95
Mean	89	92	92

We further tested classifier-enhanced TFSP in two stroke subjects with results listed in Table 3, again with comparison to the original TFSP algorithm. The results show that improvements on recognition rates can again be obtained by using classifier-enhanced TFSP. In this case, SVC-TFSP reported the highest average recognition rate of 75%. A paired t-test was used to compare the overall performance of TFSP combined with different classifiers across all subjects (control and stroke). Results of the paired t-test showed that compared to the original TFSP method, both the CART-TFSP (p = 0.046) and SVC-TFSP (p = 0.042) methods achieved significant levels of improvement in recognition rates in the tested six subjects. However, there is no significant difference (p = 0.17) between the performance obtained by the CART-TFSP and the SVC-TFSP methods.

Table 3.

Recognition rates (without rejection scheme) for two stroke subjects.

Subject #	Original TFSP (%)	CART-TFSP (%)	SVC-TFSP (%)
S1	64	69	74
S2	73	72	76
Mean	69	71	75

For SVC-TFSP, we did a search of suitable kernel functions. We found that linear and third degree polynomial kernels work the best for our purpose. In particular, these two kernels achieved the same average accuracy (92%) in healthy subjects, while in stroke subjects, the averaged accuracy of SVC-TFSP linear kernel is 73%, which is 2% lower than what we obtained with the polynomial kernel, but still better than the original TFSP and CART-TFSP (the results of SVC-TFSP in Tables 2 and 3 are based on linear and polynomial kernels, respectively). We also found that a radial basis (Gaussian) kernel appears not suitable for our purpose. These observations indicate that there may be a slightly increased polynomial nonlinearity in stroke subjects along the classification boundary of different motor tasks.

The weight distribution in stroke and control subjects obtained by CART-TFSP and SVC-TFSP are shown in Fig. 4. We can see from the figure that weights assigned to time–frequency segments by SVC-TFSP and CART-TFSP are very different. For the SVC approach (the right column), there are a few time–frequency segments that carry very high weights compared with other segments and they are visually highlighted as “islands” on the time–frequency grid. These “islands” correspond to important contributions from lower α (5–8 Hz), α (8–12 Hz) and β (12–24.5 Hz) bands.

Fig. 4.

The time–frequency weight distribution for subjects with CART-TFSP and SVC-TFSP. Brighter colors indicate bigger weights. Frequency band was divided into 13 frequency bins (1: 5.3–6.8 Hz; 2: 6.0–7.7 Hz; 3: 6.9–8.8 Hz; 4: 7.8–10.0 Hz; 5: 9.0–11.5 Hz; 6: 10.2–13.2 Hz; 7: 11.7–15.0 Hz; 8: 13.4–17.2 Hz; 9: 15.3–19.6 Hz; 10: 17.5–22.5 Hz; 11: 20.0–25.7 Hz; 12: 22.8–29.3 Hz; 13: 26.0–33.5 Hz). Time interval from 1800 to 60 ms prior to the onset of torque (denoted as time point 0) was divided into 61 segments (1: −1800 to −1745 ms; …; 10: −1554 to −1449 ms; …; 20: −1280 to −1226 ms; …; 30: −1007 to −952 ms; …; 40: −734 to −679 ms; …; 50: −460 to −405 ms; …; 60: −159 to −105 ms). (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 5 shows the changes of accuracy rates with the increase in rejection rates for CART-TFSP and SVC-TFSP methods. We can see from Fig. 5 that the impact of applying the rejection scheme on improving the accuracy is quite apparent. Very high accuracy rates (close to 100%) can be achieved in healthy subjects with reasonable rejection rates lower than 40%. In stroke subjects, maximum increase in accuracy rates is higher when compared with able-bodied subjects. For stroke subject S1, whose data present the most difficult classification problem in this study, there exists an optimal rejection rate as marked with dashed ovals: with CART-TFSP, this occurs when the rejection rate is around 50%, when the highest accuracy (> 80%) can be obtained; with SVC-TFSP, the optimal rejection rate is higher, which helps to boost the accuracy to an even higher rate. Further increasing rejection rates beyond the optimal point would not help increasing accuracy. For stroke subject S2, whose data is easier to classify, we do not observe such optimal rejection rate, instead, we can boost the accuracy to almost 100% using both SVC-TFSP and CART-TFSP, but with much higher rejection rates compared with control subjects.

Fig. 5.

Changes in accuracy rates with increase in rejection rates for SVC-TFSP and CART-TFSP methods. Dashed ovals indicate the optimal accuracy rates observed on stroke subject S1.

As a comparison, we also tested the changes of accuracy rates with the increase in rejection rates for the original TFSP, shown in Fig. 6. We can see that the introduction of rejection mechanism also increases the accuracy of the original TFSP; however, the cost on rejection rates is higher in general (the lines stretch to the right side of the graph). In addition, SVC-TFSP and CART-TFSP deliver higher overall accuracy.

Fig. 6.

Changes of accuracy rates with increase in rejection rates for the original TFSP.

In order to evaluate the potential of using TFSP algorithms with signals from as many as 163 electrodes in a real-time BCI application, we estimated the response times of the original TFSP, CART-TFSP and SVC-TFSP as listed in Table 4. The measurements were testing time per trial in milliseconds. The experiments were conducted on Matlab 7.0.1 on a 3 GHz Pentium 4 PC with 1 G RAM.¹ The operating system was Windows XP. Only the duration of the testing process was measured since that is when the predictions of the new trials were performed.

Table 4.

Testing time based on Matlab platform.

Subject #	Original TFSP	CART-TFSP	SVC-TFSP
N1	132	66	108
N2	118	55	95
N3	118	53	96
N4	144	70	110
S1	119	53	94
S2	116	51	82
Mean	125	58	98

Per test trial in milliseconds.

5. Discussion

Compared to our previous report on the performance of a time–frequency spatial patterns (TFSP) algorithm that uses a primitive classifier [10], the classifier-enhanced TFSP BCI algorithms in this study applied classifiers with potentially higher recognition capabilities into the synthesis process. This led to a better recognition rate of intended motor tasks in both healthy subjects (92% averaged over four healthy subjects) and stroke subjects (75% averaged over two subjects). The percentage of accurate estimates is further improved by employing a rejection scheme (~100% in healthy subjects and > 80% in stroke subjects).

5.1. Impact of using enhanced classifiers on TFSP BCI algorithm (weights change and rate change)

The recognition rates achieved by the SVC-TSFP and CART-TFSP (see Table 2) have clearly shown that combining the enhanced-classifiers with TFSP BCI algorithm can improve the classification performance. Such an improvement is expected mainly due to the high recognition abilities of the two enhanced classifiers, which increase the reliability of weights of time–frequency feature segments and the synthesized classification. In addition, a different weight assignment method is used in this paper: weights of the time–frequency feature segments were obtained based on the recognition rate r(t,f) on a validation set. In the original TFSP, the calculation of the average patterns p_abd(t,f)_131*1 and p_ef(t,f)_131*1 (used as the base for comparison) involves all training samples including the trial p(t,f)_131*1 with which the spatial correlation is compared and the label is determined. It was a reasonable strategy for a simple classification based on similarity comparison. However, since SVC and CART are algorithms with distinct training and testing processes, we assign the weight of the feature segment based on the generalization capability (i.e. prediction of the label of unseen data) instead of the performance on seen data. We expect the strategy of using a validation set to have a greater impact when larger data sets are being used.

The weight distribution on time–frequency grids reflects the property of different classifiers. Comparing weight distributions for CART-TFSP, SVC-TFSP and the original TFSP (see Fig. 3 of [10]), we find that the two linear classifiers (i.e., CART and the classifier of the original TFSP) have more widely distributed weights, while SVC-TFSP, which can solve nonlinear problems with the help of a kernel function, has a more focalized distribution especially in stroke subjects. This may explain why a better recognition rate is obtained using SVC-TFSP in the two stroke subjects. This phenomenon suggests that for the separation of upper arm motor tasks in brain-injured patients, a nonlinear classifier such as SVC with polynomial kernel may provide more discriminating power than a linear classifier; while for healthy subjects, both types of classifiers work well. It implies that SVC-TFSP may be preferred in future clinical applications for neurological impaired subjects for the purpose of reducing error rates, given that both methods satisfy the speed requirement.

For a real-time BCI system, the ideal response time from acquiring to processing and then to responding to the brain signal should be on the order of milliseconds [34]. As an example in literature, Millan et al. [35] reported a response time of 500 ms for an online EEG-based BCI system. We measured the processing time of the two enhanced algorithms as well as the original TFSP. As listed in Table 4, all three methods, especially the two classifier-enhanced algorithms, satisfy the speed requirement of a potential real-time BCI system. Building algorithms into digital signal processing (DSP) hardware will further increase the speed of the algorithms in the future. We also observe that the original TFSP algorithm is relatively slow, which may be explained by the nearest neighbor strategy it employs that does not produce a trained model—nearest neighbor classification belongs to the so called “lazy learning” models which need a relatively longer testing time [36].

5.2. Impact of applying rejection scheme

Applying rejection scheme reduces the error rates and increases the accuracy on testing data. Meanwhile, more rejected trials typically occur. The choice of Th_REJ (0–100%) determines the trade-off between the rejection rate and error rate. If Th_REJ is set to 0.0, no rejection is generated. On the other hand, if Th_REJ is set too high, then all the trials would be rejected and the recognition rate will become zero. As seen in Fig. 5, rejection scheme can improve the accuracy of the algorithm both on healthy and stroke subjects while increasing the number of rejected trials. More importantly, there exists a subject-specific optimal rejection rate. Such optimal rejection rate defines the limit of the boost in accuracy and corresponds to the highest accuracy rate that can be obtained by rejecting trials. In our results, this happens for the most difficult subject, stroke subject S1. This suggests that a search of the optimal rejection rate for subjects may be necessary, especially for stroke subjects. Despite the rejected trials, we think that rejection scheme will be an important component for future online clinical BCI applications. For example, during adaptive training for rehabilitation device, when a trial is rejected due to unclear motor intent, several choices can be employed; the intent can be ignored and the patient can be asked to retry, or a guess is presented back to the patient without actual muscle stimulation, based on which s/he can adaptively re-generate the signal to help shorten the learning curve of using the rehabilitative device. On the other hand, a wrongly judged intent may lead to unintended stimulation with undesirable effects.

5.3. Frequency band difference between healthy and stroke subjects

Using the two enhanced classifiers, high weights are assigned to α and β bands in the four healthy subjects. This observation is more obvious in the weight distribution obtained by SVC-TFSP method. Contributions from α (8–12 Hz) and β (12–24.5 Hz) are consistent with widely accepted results that the Rolandic μ (10–13 Hz) and central β (14.5–18 Hz) rhythms are two important rhythmic activities showing characteristic spatiotemporal patterns during the imagination, planning and execution of movements [4,28–33]. In the two stroke subjects, the contribution from lower α (5–8 Hz) band are stronger compared with that in healthy subjects. This may indicate that the difference associated with cognitive functions of temporal awareness, anticipation and motor memory between the two tasks may be important for the classification in stroke subjects.

5.4. Application of our BCI algorithm for rehabilitation of neurologically impaired subjects

The reported results, especially by SVC-TFSP, are among the highest for EEG-based BCI motor tasks reported in literature, including the separation of left versus right hand movement (67–95%) [11,16,17,37,38], despite that the task of separating SABD versus EF is harder than studies aiming to separate left and right hand movement intents (whose cortex representation are on opposite hemisphere and about 10 cm apart while shoulder and elbow movements correspond to two closely neighboring areas in the cortex of only about 10 mm apart).

In addition, research in the area of brain reorganization has found that the brain injury following stroke causes an increased overlap between cortical activities for different motor tasks in the sensorimotor cortical (SMC) regions[7,8]. This increased cortical overlap could make predicting torque directions generated by brain-injured subjects even more difficult which could provide an explanation of the relatively low recognition rates compared with able-bodied subjects [25–27].

We need to make a note that the stroke subjects evaluated in this study show feasibility of our TFSP approach in the case of a moderate to large size white matter subcortical stroke (subject 1, Table 1) and following a moderate to large size cortical combined with a white matter stroke (subject 2). We are currently expanding our work to a larger group of moderately to severely affected stroke survivors who are most likely going to benefit from future development in brain machine technologies to regain upper limb function.

Another issue of importance for any future clinical application of our high density EEG approach is limiting the number of electrodes. This is important for several reasons: first of all, limiting the number of electrodes could reduce the annoyance of applying the montage to the patient and secondly, optimally numbered and positioned electrodes could potentially provide signals of a smaller redundancy and may improve overall accuracy of our classifier-enhanced TFSP approach. Determination of the optimal number of electrodes will be examined as part of our future work.

In the long term, by estimating motor intent of a subject via EEG signals, rehabilitation techniques using real-time interaction with neural prostheses or other devices may be developed for the treatment of abnormal movement disturbances in muscular coordination including the presence of abnormal upper limb synergies following hemiparetic stroke [39]. Being among the earliest experiments conducted in stroke subjects, this study provides a potentially effective EEG-based classification approach towards non-muscular BCI-controlled neural prosthesis for stroke rehabilitation. This paper also reported that the response time of the TFSP algorithm is promising for the application in real-time BCI systems.

6. Conclusions and future work

The improvement on recognition rates confirms that by using classifiers of high recognition reliability, the classifier-enhanced TFSP can improve the prediction power of the original TFSP for challenging BCI problems.

Currently the TFSP method weights the time and frequency information, but the spatial information is treated equally. In the future, spatial information can also be weighted based on the contribution of different EEG electrode channels. Classifier-enhanced TFSP brings another potential advantage of channel selection to facilitate the research toward this direction. For example, it has been observed that in CART-TFSP, an inherent electrode selection is conducted and only limited number of electrodes are involved in the decision making process. Similarly, SVC can also be used to do channel selection [16]. We plan to incorporate channel selection and spatial information weighting into the TFSP algorithm, which may lead to deeper understanding of spatial information of EEG signals corresponding to elbow versus shoulder torques and further improvement of the prediction reliability.

7. Summary

The ultimate aim for classifying elbow versus shoulder torque intentions is to develop robust BCI devices for patients who suffer from movement disorders following brain injury such as stroke. In this paper, we investigate the advanced classification approach classifier-enhanced TFSP in classifying a subject’s intent of generating an isometric SABD or EF torque using signals obtained from 163 scalp EEG electrodes. Two classifiers, the SVC and the CART, are integrated in the TFSP algorithm that decomposes the signal into a weighted time, frequency and spatial feature space. The resulting high-performing methods (SVC-TFSP and CART-TFSP) are then applied to experimental data collected in four healthy subjects and two stroke subjects. Results are compared with the original TFSP, and significantly higher reliability in both healthy subjects (92% averaged over four healthy subjects) and stroke subjects (75% averaged over two subjects) are achieved. The accuracies of classifier-enhanced TFSP methods are further improved after a rejection scheme is applied (~100% in healthy subjects and > 80% in stroke subjects). The results are among the highest reliability reported in literature for tasks with spatial representations on the motor cortex as close as shoulder and elbow. The paper also discusses the impact of applying rejection strategy in detail and reports the existence of an optimal rejection rate on a stroke subject. The results indicate that the proposed algorithms are promising for future use of rehabilitative BCI applications of neurologically impaired patients. In the future, different EEG electrode channels can also be weighted to further improve the reliability of the algorithm. Introducing online learning mechanisms are also important for applying the algorithm to stroke rehabilitative applications.

Biographies

Jie Zhou received her BS and MS degrees in Biomedical Engineering from Southeast University, Nanjing, China, in 1993 and 1996, respectively, and her PhD degree in Computer Science from Concordia University, Montreal, Canada, in 2000. She has been an Assistant Professor since 2002 and an Associate Professor since 2008 in the Department of Computer Science at Northern Illinois University, USA. Her research interests include pattern recognition, machine intelligence and applications of computational methods in medicine and biology. Prof. Zhou was a recipient of FONDS F.C.A.R. (Fonds pour la Formation de Chercheurs et l’Aide a la Recherche) of Quebec, Canada. She has also been a recipient of Northern Illinois University Graduate School Research and Artistry Grants. Prof. Zhou is an Associate Editor of Pattern Recognition Journal.

Jun Yao received her BS, MS and PhD degrees from Chongqing University, Southeast University and the Chinese University of Hong Kong, China, respectively, all in Biomedical Engineering. Dr. Yao is a Research Assistant Professor in the Department of Physical Therapy and Human Movement Sciences at Northwestern University, USA. Her current research interests focus on the changes of brain electrical activities after hemispheric stroke based on non-invasive EEG during isometrical upper limb movements.

Jie Deng received her BSc in Biomedical Engineering from Southeast University, China, in 2001. She obtained her MS in Bioengineering from the University of Illinois at Chicago in 2004. She entered the PhD program of the Department of Biomedical Engineering at Northwestern University in September 2004. Currently, Jie is a Research Assistant in the Cardiovascular MRI Research Lab of the Department of Radiology at Northwestern University. Her research is focused on the development and application of diffusion weighted MR imaging.

Julius P.A. Dewald received his BS and MS degrees in Physical Therapy and Rehab Medicine from Vrije Universiteit Brussel, Brussels, Belgium, and his PhD degree in Neurophysiology from Loma Linda University, Loma Linda, California. Dr. Dewald is currently the Chair and an Associate Professor of Department of Physical Therapy and Human Movement Sciences at Northwestern University, USA. His research interests are on upper extremity discoordination following stroke and associated brain imaging.