A space-frequency localized approach of spatial filtering for motor imagery classification

Abstract

Classification of Motor Imagery (MI) signals is the heart of Brain-Computer Interface (BCI) based applications. Spatial filtering is an important step in this process that produce new set of signals for better discrimination of two classes of EEG signals. In this work, a new approach of spatial filtering called Space-Frequency Localized Spatial Filtering (SFLSF) is proposed to enhance the performances of MI classification. The SFLSF method initially divides the scalp-EEG channels into local overlapping spatial windows. Then a filter bank is used to divide the signals into local frequency bands. The group of channels, localized in space and frequency, are then processed with spatial filter, and features are subsequently extracted for classification task. Experimental results corroborate that the proposed space localization helps to increase the classification accuracy when compared to the existing methods using spatial filters. The classification performance is further improved when frequency localization is incorporated. Thus, the proposed space-frequency localized approach of spatial filtering helps to deliver better classification result which is consistently 3–5% higher than traditional methods.

Introduction

The communication system that connects human thoughts to external world through computer is usually called Brain Computer Interface (BCI). There are numerous examples of BCI such as controlling a robotic arm, wheel chair or playing computer games by just using our thoughts. Some of these applications are very helpful for disabled people. The brain signals are recorded using either invasive (such as ECoG) or non-invasive (such as EEG) methods, which interprets the different brain activities. EEG is more preferable as this method is non-invasive and portable. A number of techniques have been developed using EEG [1–5] for different BCI applications.

Our brains generate specific patterns or rhythms when we move our limbs. It is found that similar rhythms can also be generated from sensorimotor area of our brains when imagining the limb-movement. Thus without actual movement, one can generate voluntary modulation in sensorimotor rhythm called Motor Imagery (MI) signal. In a BCI system, this MI signal can be captured from the scalp and be used to control devices such as artificial limb. However, recognizing patterns in MI signal is complex and faces many difficulties such as proper training of a user, variation of signal patterns across subjects and sessions etc. A user can imagine the movement visually to produce an MI signal. To make the MI signal robust, user can also add kinesthetic experience on top of visual imagination [6].

To capture the MI signal, Electro-Encephalogram (EEG) is popular among the non-invasive methods that collect electrical signals from the scalp. When dealing with MI patterns, EEG signals have some issues like artifacts, specifically ocular, muscular and cardiac artifacts. Moreover, as EEG signals have a very low signal-to-noise ratio (SNR) [7], it is not very suitable to use these signals directly in BCI applications. Spatial filters are common techniques in BCI to partially overcome these problems by enhancing signal-to-noise ratio. These filters increase inter-class distances in the newly generated signal sets.

Different spatial filtering methods proposed by the researchers are Common Average Referencing (CAR) [8], Independent Component Analysis (ICA) [9], Laplacian (LAP) [10], Principal Component Analysis (PCA) [11], Common Spatial Pattern (CSP) [12] and variants of CSP [13–15]. Among these, CSP is a very popular method. CSP was first introduced for the classification of single-trial EEG during imagined hand movement [16]. Novi et al. [13] proposed a new method called Sub-band CSP (SBCSP) for BCI. Dai et al. [17] proposed Transfer Kernel Common Spatial Patterns (TKCSP) for classification of MI signals. Regularized Common Spatial Pattern (Reg-CSP) [14] was developed to make CSP more robust against noise and to avoid over-fitting. Ang et al. [15] proposed new algorithm named Filter bank CSP (FBCSP) for the classification of MI-based EEG signals. In FBCSP, signals are decomposed into different frequency bands before applying CSP that results in better classification of signals. After applying spatial filters on raw EEG signals, a new set of signals is created. Like any pattern recognition problem, signals from spatial filter are then processed to extract discriminative features and a classifier is used to detect MI patterns in EEG signals [18, 19, 28–31].

EEG signals are collected from different locations of the scalp and each signal is a mixture of signals produced from different areas of the brain. As EEG is a blurred version of local signals inside the brain, it has a poor spatial resolution. This leads researchers to use patterns mostly in time and frequency domains. Researchers tried to include spatial information from other modalities of signals [32] to improve the BCI performance. However, involving other modalities such as fNRI and MRI is not very practical considering cost-effectiveness and portability. Thus it is essential to find more spatial information from captured EEG signal which is dominated by the local field. Emphasizing on the local field we can extract more discriminative information for the MI classification, because different MI signals are originated from different spatial locations of the brain. However, most techniques use all the spatial channels from whole scalp at a time and ignore the potential in focusing local fields. Some other techniques use channels from selected scalp region such as motor-area and may overlook information hidden in other areas. We propose a new method to address these limitations.

Our proposed method is based on signal localization in both space and frequency domains. It considers all the channels of scalp-space, but handles fewer channels (space-localized) at a time. Each of these channels is further localized in frequency domain. Such space-frequency localization helps to capture local patterns of brain activities. The proposed method, named Space-Frequency Localized Spatial Filter (SFLSF), divides the scalp region into a number of local spatial windows. Instead of using all the channels together, this method uses space localized channels, each of which is further divided into different frequency bands. Other traditional steps such as spatial filtering, feature extraction and classification are then employed to classify the MI signals. Compared with other existing methods, the proposed space-frequency localized approach helps to extract more discriminative information which is consistently reflected in the superior classification results.

Methodology

The proposed method is divided into five steps. First step is to divide the signals into some spatial windows. The second step is to apply filter bank by which the signals of a spatial window are decomposed into different frequency bands. The third and fourth steps are applying spatial filter and feature extraction respectively. Features from different sets of signals are then combined into a single feature vector to be used in the classifier. Figure 1 shows the block diagram of the proposed method.

Fig. 1.

Block diagram of proposed method

Spatial windows

This method initially divides the 22 EEG channels of an international 10–20 system used in the dataset [20] into a number of spatial windows. Each spatial window contains a number of channels from a specific spatial area. These spatial windows are mutually overlapping which means that few channels are common in two or more windows. Instead of using all the channels at the same time, we have used the channels within a window at a time for further processing towards feature extraction. We examined the performance of different sets of spatial windows to obtain the optimum window set. Figure 2 shows some spatial window sets used in this experiment.

Fig. 2.

Some sets of spatial windows used in the experiment

Filter bank

A filter bank decomposes the EEG signals into multiple frequency bands. In this work, we have used Butterworth filter of order 5. We have examined the performance of different frequency bands to obtain the most suitable ones for classifying the MI signals. We also examined six different sets of frequency bands which differ in numbers and widths of bands. We have used both overlapping and non-overlapping frequency bands. Table 1 shows different frequency band sets examined in our experiment. For example, signals from a particular spatial window passes through the filter bank and are divided into 9 different frequency-localized signals (for set-a, Table 1). Thus, a set of 2 spatial windows (Fig. 2b) generates 18 (2 × 9) sets of signals localized in space-frequency domain, where each set contains a number of EEG signals localized in scalp as illustrated by Fig. 2.

Table 1.

Different frequency-band sets

Set	Frequency bands
Set-a	4–8,8–12,12–16,16–20,20–24,24–28,28–32,32–36,36–40
Set-b	3–9,7–13,11–17,15–21,19–25,23–29,27–33,31–37,35–41
Set-c	4–10,10–16,16–22,22–28,28–34,34–40
Set-d	3–11,9–17,15–23,21–29,27–35,33–41
Set-e	8–20,16–28,24–36
Set-f	8–16,16–24,24–32

Spatial filter

EEG signals are like a blurred signals generated from numerous signals of brain and other noises. Spatial filtering techniques try to generate new sets of signals from the projection of original EEG signals. This results in less signal-to-noise ratio and maximizes the discriminability of the two classes. Some popular spatial filters are already mentioned in the previous section. Among them, CSP, Reg-CSP, FBCSP and ICA are used in this experiment.

ICA is a spatial filter which is unsupervised in nature and used to solve cocktail-party type problems [21]. ICA linearly projects the data into new axis where mutual independences are maximized. To improve signal-to-noise ratio of EEG signal, ICA is commonly used to project given set of signals into a new set of signals that are functionally independent [22]. It is a kind of “blind source separation” technique that is capable of extracting information from a noisy background. ICA can be briefly described as follows and further details can be referred to [21, 23].

Suppose a signal x(t) is a linear projection of some source signals s(t) which are generated from independent sources. These two signals can be related by a mixing matrix A as follows.

$$x(t)=As(t)$$ 1

The matrix A contains spatial information such that each observed data in x(t) can be calculated from the Independent Components in s(t). Now we can think about a de-mixing matrix $\left(W=A^{-1}\right)$ to get back the source signal s(t) from x(t) such that

$$s(t)=Wx(t)$$ 2

CSP is a supervised method to implement the spatial filter and it has become very popular in recent years specially for classifying MI signals. In CSP, the de-mixing matrix W in (2) is calculated in supervised way where class information is used. For a binary classification problem, CSP projects different channels of EEG signals into a new set of channels where it achieves maximum variances for one class while maintaining minimum variance for other class. In this way, two classes of MI signals become more discriminatory. The details of the calculation of W in CSP can be referred to [12].

In spite of popularity of traditional CSP, it has few limitations such as non-robustness, sensitivity to noise and tendency for over-fitting to small training data. Regularized Common Spatial Pattern (Reg-CSP) was developed to overcome these limitations, details of which can be referred to [14].

In our method, spatial filter is applied on every window obtained from space-frequency localization. For example, after applying a CSP with two output pairs, each window creates a new set containing four channels of signals. Thus a choice of three spatial windows and 9 frequency bands finally produce a total of 2 × 9 × 4 channels of signals. This is illustrated in Fig. 3.

Fig. 3.

Illustration of applying spatial filter on space & frequency localized channels

Feature extraction

Features are higher level descriptions of raw data that helps to discriminate different classes. A number of useful features for classification of MI signal can be found in the literature [24, 25]. Here, we have included three features that appeared to be more effective in our experiments compared with others. They are Mean Absolute Value (MAV), Energy (Eng) and Log Variance (LV).

The LV feature is calculated using the variances of signals that are spatially filtered from original EEG signals. To make the LV features, the variance of each signal z is calculated. It is then normalized by the total variance of all signals and a log-transformation is applied. The Log Variance of pth channel from a total of N channels in ith trial is calculated as

$$f_p^i=log\left(\frac{\text{var}(z_p^i)}{{\sum }_{p=1}^N\text{var}(z_p^i)}\right)$$ 3

The final feature vector is created by combining the log-variances of all the signals:

$$F_{\mathit{LV}}^i=[f_1^i{f}_2^i\dots f_N^i].$$

In literature, wavelet decomposition is found to be effective in extracting features that better helps the classification of MI signals [25, 26]. Wavelet transform allows us to capture information from both time and frequency domains. In our experiment, we have chosen two features to be extracted from wavelet-transformed signal that are found to be more effective compared with others.

Suppose $D_i(n)$ is the nth sample of a wavelet decomposed detail at level i. The MAV feature can be calculated as follows:

$$g_i=\frac{1}{N}\sum _{n=1}^N\left|D_i\right|$$ 4

The final feature vector is formed by combining features from all the wavelet levels of all the N channels of signals:

$$F_{\mathit{MAV}}^{}=[g_1^1{g}_2^1\dots g_L^1\dots \dots g_{L-1}^N{g}_L^N].$$

Energy feature can be calculated as follows:

$$h_i=\sum _{n=1}^N{\left|D_i\right|}^2$$ 5

Similar to MAV features, the final feature vector is formed:

$$F_E^{}=[h_1^1{h}_2^1\dots h_L^1\dots \dots h_{L-1}^N{h}_L^N].$$

Classification

Classification is an important step of MI based BCI where user’s intension is matched against some known classes of activities. In classification process, the high-level information of feature-vector is analysed to predict the class of data. Two phases are involved with a classifier: training and testing. During the training phase, some data are presented to the classifier together with their class labels. The machine learns the patterns from the feature vectors to discriminate the different classes. After the training is complete, classifier is ready to recognize any new data by comparing its feature vector. Among different classification methods, some popular methods are Linear Discriminant Analysis (LDA), Neural Network (NN) and Support Vector Machine (SVM).

Support Vector Machine (SVM) [27] is one of the most popular classifiers for EEG-based pattern recognition problems. It is used for all the experimental results of this paper. For a two class problem, SVM generates a hyper plane that separates the features vectors from two classes. The distances of the training samples from the hyper plane are maximized during the training phase of SVM [27].

Results and discussion

Dataset description

In this study, we have used the dataset “2a of BCI Competition IV” [20] which is publicly available on the internet. In this dataset, 22 channels are used during the recording of EEG signals. A total of nine persons participated in making this dataset. They were asked to perform one of four MI at a time which are related to movement of left hand, right hand, foot and tongue. In this study, only left hand and right hand MIs have been used. The training and testing set were recorded in different sessions (session independent), each consists of 72 trials for each class. Each trial consists of a 3 s window of a signal, which contains any one of the MI tasks.

We have summarized the comparative results from our proposed method along with the existing methods. In our experiment, we have compared the performance of different features, spatial window sets and frequency bands.

Comparison of features

At first, we like to compare the performance of different features and select the best one, so that rest of the experiment continues on the selected feature. Table 2 shows the comparative classification results from three types of features when using the traditional CSP as spatial filter without applying spatial window and filter bank in Fig. 1.

Table 2.

Classification results for three features without spatial window and filter bank

Subject	MAV	Eng	LV
Sub-1	77.7	77	79.1
Sub-2	54.1	52	56.9
Sub-3	91.6	87.5	90.2
Sub-4	71.5	71.5	70.1
Sub-5	50.6	51.3	50
Sub-6	63.1	67.3	67.3
Sub-7	67.3	66.6	64.5
Sub-8	97.2	97.9	97.2
Sub-9	82.6	82.6	77
Mean	72.9	72.6	72.5

Considering the results from individual subjects, it is clear that classification performance varies for different combinations of subject and features. For instance, subject-8 performs best and Subject-5 delivers the worst. Again, each subject got different winner from features. Considering the mean results, MAV feature is slightly better than the others and it is used in the next part of the experiment.

In the next part, we will investigate the performance of our proposed SFLSF method using MAV features for different selection of spatial windows and frequency bands.

Comparison of spatial windows in SFLSF

Table 3 shows the comparative results for using different spatial window-sets in our proposed SFLSF method. We have experimented with a number of window-sets, out of which four sets are included here as shown in Fig. 2. For most of the subjects, window-set ‘a’ and 'c' performed better than others. Both of these sets contain three spatial windows. For set ‘c’, the middle window is shifted downward compared to set ‘a’. Considering mean results, it can be said that window-set ‘c’ provides the best classification accuracy in comparison to others.

Table 3.

Classification results for different spatial window-sets

Subject	Window set-a	Window set-b	Window set-c	Window set-d
Sub-1	86.1	85.4	87.5	86.1
Sub-2	56.2	58.3	58.3	62.5
Sub-3	93.7	80.5	92.3	93
Sub-4	63.1	59	65.9	65.2
Sub-5	75.6	70.1	72.9	56.2
Sub-6	63.1	57.6	62.5	61.1
Sub-7	86.8	91.3	90.2	91.6
Sub-8	93.7	90.2	92.3	94.4
Sub-9	83.3	72.2	81.1	82.6
Mean	78	73.9	78.1	77

Comparison of frequency bands in SFLSF

We now like to investigate the impact of selecting frequency bands on the proposed SFLSF method. Different frequency band-sets are used in our experiment as listed in Table 1. Table 4 shows the comparative results of different frequency band-sets for nine subjects. Frequency set ‘a’ provides maximum classification results in the case of subjects 2 and 7. In the case of subject 8 and 9, maximum classification results are obtained by using the frequency set-b. Frequency set-e provides the maximum classification result for subject-5. Subject 1, 3, 4 and 6 provide maximum classification accuracy by using the frequency set-f. Considering mean results, it is clear that the frequency set ‘a’ is the best choice for our proposed MI classification.

Table 4.

Classification results for different frequency bands using spatial window-set ‘c’

Sub	Frequency-bands
	Set-a	Set-b	Set-c	Set-d	Set-e	Set-f
Sub-1	87.5	86.8	84	88.1	92.3	93.7
Sub-2	58.3	54.1	54.8	57.6	57.6	56.2
Sub-3	92.3	91.6	86.1	87.5	88.1	93
Sub-4	65.9	58.3	70.1	54.8	69.4	70.8
Sub-5	72.9	75	65.9	72.2	76.3	63.1
Sub-6	62.5	56.9	56.9	56.2	59	64.5
Sub-7	90.2	86.8	88.8	86.8	80.5	87.5
Sub-8	92.3	95.1	92.3	88.8	92.3	90.2
Sub-9	81.2	81.9	72.9	73.6	78.4	80.5
Mean	78.1	76.3	74.6	73.9	77.1	77.7

Comparison of methods

We have compared our SFLSF method with three other methods: CSP, SW-CSP and FBCSP. Among these, CSP [12] and FBCSP [15] are well-known. The SW-CSP is a Spatial window based CSP in which we employed our spatial windows together with CSP without using filter-bank. Figure 4 presents the classification results for all 4 techniques. It is interesting to note that SW-CSP is performing better than traditional CSP for most of the subjects. Then, we employed FBCSP [15], which is a combination of filter-banks with traditional CSP technique. This method is then compared with our proposed SFLSF technique where spatial windows are used in addition to filter-banks and CSP technique. If we compare FBCSP with SFLSF for each subject, SFLSF wins 7 out of 9. Thus, the above results clearly indicate the merit of incorporating spatial windows.

Fig. 4.

Comparative results from different methods across nine subjects

The average results of classification from four different techniques are presented in the Fig. 5. The lowest result is achieved from the traditional CSP method, which is 72.9%. This result is improved by 2.1% when SW-CSP is used. FBCSP delivers better mean result (improved by 2.7%) than existing CSP. It also delivers slightly better result (improved by 0.6%) than SW-CSP. Finally, when SFLSF is used, the maximum result of 78.1% accuracy is obtained which is 5.2% higher than that of the traditional CSP.

Fig. 5.

Mean classification accuracies of four techniques

To make it fair, we should compare SW-CSP with traditional CSP as both are without using any frequency band localization. Then, we should compare SFLF with FBCSP as both use frequency localization. In both the cases, the inclusion of space localization enhances the mean classification accuracy. Another important observation is that when we compare all four methods for each subject, SFLSF wins only for one subject among nine. However, SFLSF beat other three methods by the mean result. This clearly indicates that SFLSF has the least variance of classification accuracies across different subjects.

Table 5 shows mean performance indicators for six different methods including Regularized-CSP [14] and ICA [9]. In addition to the classification accuracy, we have included the performance parameter Kappa. Kohen’s Kappa is a useful statistical measure, specially in multi-class or imbalance class problem, where accuracy do not provide full view of the classification performance. It basically tells us how good is the performance of a given classifier compared to a classifier that simply guesses the class randomly according to frequency of classes. A higher the value of kappa indicate a better the performance of classification. We can see from the table that our proposed method SFLSF beats all other methods by both accuracy and kappa that are good indicators of overall classification performance. This demonstrates superiority of the proposed SFLSF over other methods.

Table 5.

Mean performance indicators over nine subjects for different methods

Methods	Accuracy	Kappa
CSP	72.9	0.45
SW-CSP	75	0.48
FBCSP	75.6	0.53
Reg-CSP	70.9	0.43
ICA	70.5	0.40
SFLSF	78.1	0.55

Conclusion

In this work, we have proposed a new approach of spatial filtering to enhance the classification of MI signals. Unlike the traditional method, our proposed method divides the whole scalp space into smaller windows upon which other techniques like filter-banks and spatial filters are applied. Our experimental results corroborate that such space localization helps to increase the classification accuracy of the proposed method when compared to the existing methods based on spatial filters. The classification performance further improves when frequency localization is added to the process by applying filter banks in-between space-localization and spatial-filtering. The proposed SFLSF method has the least variance of classification accuracy across different subjects and beats the other methods by a good margin of classification accuracy.

Funding

This research was funded by United International University [Grant No. Grant-UIU-RG-161013].