Time–frequency feature extraction based on multivariable synchronization index for training-free SSVEP-based BCI

Abstract

Multivariate synchronization index (MSI), as an effective recognition algorithm for steady-state visual evoked potential (SSVEP) brain-computer interface (BCI), can accurately decode target frequencies without training. To further consider temporal features or extract harmonic components, extended MSI (EMSI), temporally local MSI (TMSI), and filter bank MSI (FBMSI) have been proposed. However, the promotion effects of the above three strategies on MSI have not been compared in detail. In this paper, the performance of EMSI, TMSI, and FBMSI under different time windows was analyzed with the same dataset. The results indicated that the improvement effect of the temporally local method on MSI was better than that of the other two methods under the short time window, and the effect of the filter bank method was better when the time window was greater than 0.8 s. Based on the idea of simultaneously extracting time–frequency features, FBEMSI and FBTMSI were proposed by integrating time delay embedding and temporally local method into FBMSI respectively. The two improved methods, which has no significant difference, can improve the recognition effect of FBMSI. But the computing time of FBEMSI was shorter, which can be a potential method for SSVEP-BCI.

Introduction

As one of the current research hotspots, the brain-computer interface (BCI) can establish a new channel between the brain and the external device, so as to help patients with stroke and amputation return to normal life (Gao et al. 2021). Electroencephalogram (EEG) has become the most commonly used modality for signal acquirement in non-invasive BCI because of its lower cost, higher temporal resolution, and portability (Abiri et al. 2019). The EEG-based BCI can be realized by different paradigms, including steady-state visual evoked potentials (SSVEP) (Wang et al. 2008), motor imagination (MI) (He et al. 2015), P300 (Powers et al. 2015), etc. Among them, SSVEP has been widely used with the merits of higher information transfer rate (ITR) and lower training requirements, in which the visual stimuli are encoded with different frequencies or phases and the selected targets can be detected by analyzing the EEG signals in the visual cortex of the brain (Vialatte et al. 2010).

Based on the characteristics of SSVEP-based BCI, many feature extraction and target recognition methods have been proposed (Hong and Qin 2021; Wong et al. 2020). In the beginning, the fast Fourier transform (FFT) was mainly used to complete the power spectral density analysis (PSDA), and the frequency with the largest amplitude was selected as the recognition target (Cheng et al. 2002; Wang et al. 2006). However, this method is sensitive to noise and requires electrode selection. For this reason, many multi-channel recognition algorithms have emerged, such as minimum energy combination (MEC) (Friman et al. 2007), canonical correlation analysis (CCA) (Lin et al. 2006) multivariate synchronization index (MSI) (Zhang et al. 2014), and Task-related component analysis (TRCA) (Nakanishi et al. 2018).

According to the training requirement and feature, multi-channel recognition algorithms are typically categorized into three types, i.e. training-free, subject-specific training, and subject-independent training (Zerafa et al. 2018). For subject-specific training algorithms, special spatial filters and individual templates are obtained from each subject's training data, such as TRCA and its extended algorithms (Nakanishi et al. 2018; Sun et al. 2021). However, subject-independent training algorithms use training data from existing subjects to train general models and reference templates for new subjects, such as cross-subject learning algorithms based on CCA (Yuan et al. 2015) and TRCA (Tanaka 2020).

Although the above two kinds of methods can achieve higher ITR, they need to be calibrated in advance, which increases the difficulty of practical application (Chen et al. 2015). Training-free algorithms can effectively avoid visual fatigue caused by the calibration process. By calculating the maximum correlation coefficient between EEG data and sine–cosine standard template, the target recognition algorithm based on CCA has gradually matured (Lin et al. 2006; Chen et al. 2021; Poryzala and Materka 2014). In addition, the MSI algorithm was proved to be more effective than CCA (Zhang et al. 2014).

Many researchers have discussed the improvement effect of the temporal feature extraction on target recognition, usually including temporally local covariance and time delay embedding. The temporally local information was considered in the variance modeling of CSP, so that the more distinguishing feature information can be extracted (Wang and Zheng 2008; Zhang et al. 2013). Through the time delay embedding method, the features extracted by CSP can be improved (Lemm et al. 2005; Qi et al. 2015). Inspired by the ideas of the above paper, Zhang et al. put forward TMSI and EMSI respectively (Zhang et al. 2016, 2017). Recently, The temporally local covariance method has been applied to CCA (Shao and Lin 2020) and TRCA (Jin et al. 2021) for the first time. Liu et al. (2021a) proposed the TDCA algorithm, in which the time delay embedding method was used to extract time features. These again demonstrate the potential of the two strategies.

In addition to the fundamental frequency of EEG, the features of harmonic components also play a role in target recognition. In order to make effective use of the harmonic frequency, the filter bank method has been proposed. Ang et al. (2008) applied the filter bank method in the CSP algorithm for the first time. Chen et al. (2015) combined this method with CCA. Qin et al. (2021) discussed the application effect of the filter bank method in MSI and proposed filter bank MSI (FBMSI).

Among the current extension methods of MSI, the most classic TMSI, EMSI, and FBMSI can all improve their performance. However, there is no literature to compare the above three methods in detail. For this reason, this paper first compared the lifting effects of the three algorithms with the same dataset, so as to obtain the advantages and disadvantages of several improvement strategies. In order to realize synchronous extraction of time–frequency features, two kinds of temporal information utilization methods were integrated into FBMSI respectively, and filter bank extended MSI (FBEMSI) and filter bank temporally local MSI (FBTMSI) were proposed. Next, the two methods were analyzed together with FBMSI.

Before comparing the algorithms, the grid search method was used to optimize the parameters. The parameter optimization in this paper selects the first seven subjects in the Benchmark dataset, while the algorithm verification selects the data of all 35 subjects. Group analysis of the algorithms was carried out under different time windows based on recognition accuracy and ITR. Meanwhile, the computational complexity of the algorithm was evaluated based on the computation time.

Materials and methods

Multivariate synchronization index (MSI)

Let $X\in {\mathbb{R}}^{N\times T}$ denote a multichannel EEG signal, N represents the number of channels, and T is the number of recording samples. The reference signal $Y_k\in {\mathbb{R}}^{2N_h\times T}$ at each stimulus frequency is constructed as following:

$$Y_k=\left(\begin{array}{c}sin\left(2\pi f_{k}t\right)\\ cos\left(2\pi f_{k}t\right)\\ ⋮\\ sin\left(2\pi N_h{f}_{k}t\right)\\ cos\left(2\pi N_h{f}_{k}t\right)\end{array}\right),t=\frac{1}{F_s},\frac{2}{F_s},\cdots \frac{T}{F_s}$$ 1

where N_h denotes the number of harmonics, F_s is the sampling rate, and f_k is the kth stimulus frequency.

X and Y are normalized to have a zero mean and unitary variance. Then, a covariance matrix of the concatenation of X and Y can be calculated as

$$C=\left[\begin{array}{cc}{C}_{11}& C_{12}\\ C_{21}& C_{22}\end{array}\right]$$ 2

$$C_{11}=\frac{1}{T}X{X}^T,C_{22}=\frac{1}{T}Y{Y}^T,C_{12}=C_{21}=\frac{1}{M}X{Y}^T$$

To reduce the influences of the autocorrelation of X and Y on the future synchronization computing, the transformation matrix is constructed and the linear transformation is adopted. The new correlation matrix is:

$$R=UC{U}^T,U=\left[\begin{array}{cc}{C}_{11}^{-1/2}& 0\\ 0& C_{22}^{-1/2}\end{array}\right]$$ 3

Let λ₁, λ₂,..., λ_P be the eigenvalues of matrix R. Then, the normalized eigenvalues are calculated as follows:

$${\lambda }_i^{\prime }=\frac{{\lambda }_i}{\sum _{i=1}^P{\lambda }_i}=\frac{{\lambda }_i}{tr\left(R\right)},P=N+2N_h$$ 4

Then, the synchronization index between two multichannel signals can be obtained by

$$S=1+\frac{\sum _{i=1}^P{\lambda }_i^{\prime }log\left({\lambda }_i^{\prime }\right)}{log\left(P\right)}$$ 5

Then, the target frequency could be recognized by the formula:

$$f_{\text{target}}=\underset{f_k}{max}{S}_k,k=1,2,\cdots ,K$$ 6

Temporally local structure

Previous studies have shown that the performance of MSI can be effectively improved by considering the temporally local information of the signal (Zhang et al. 2016). A new method, termed temporally local MSI (TMSI), was presented. The core operation is to construct the temporally local covariance matrix.

Denote the weight matrix by W ∈ ${\mathbb{R}}^{T\times T}$ and multivariate signals by Z = $[{\mathbf{z}}_1,{\mathbf{z}}_2\cdots {\mathbf{z}}_T]$ ∈ ${\mathbb{R}}^{C\times T}$ respectively. C is the number of variables or channels. The formula for the temporally local covariance matrix is as follows (Zhang et al. 2016; Wang 2010):

$$\overline{\mathbf{C}}=\frac{1}{2T}\sum _{i=1}^T\sum _{j=1}^T\mathbf{W}\left(i,j\right)\left({\mathbf{z}}_i-{\mathbf{z}}_j\right){\left({\mathbf{z}}_i-{\mathbf{z}}_j\right)}^T$$ 7

The formula (7) can be further expressed as:

$$\begin{array}{cc}\hfill \overline{\mathbf{C}}=& \frac{1}{T}\sum _{i=1}^T{\mathbf{z}}_i{\mathbf{z}}_i^T\sum _{j=1}^T\mathbf{W}\left(i,j\right)-\sum _{i=1}^T\sum _{j=1}^T\mathbf{W}\left(i,j\right){\mathbf{z}}_i{\mathbf{z}}_j^T=\frac{1}{T}\left(\mathbf{Z}\left(\mathbf{D}-\mathbf{W}\right){\mathbf{Z}}^T\right)\hfill \\ \hfill =\frac{1}{T}\left({\mathbf{ZLZ}}^T\right)\\ \hfill \end{array}$$ 8

where L is the Laplacian matrix and L = D – W. D is a diagonal matrix, and D(i,i) = ${\sum }_{j=1}^T\mathbf{W}\left(i,j\right)$ for i = 1, 2,…, T.

The value of W is determined by Tukey’s tricube weighting function (Cleveland 1979):

$$\mathbf{W}\left(i,j\right)=K\left(\frac{j-i}{\tau }\right),K\left(v\right)=\left\{\begin{array}{cc}{\left(1-{\left|v\right|}^r\right)}^r,\hfill & ,\left|v\right|<1\\ 0,\hfill & \mathit{else}\end{array}\right)$$ 9

Here, τ defines the temporally local range. Similar to the study by Cleveland (Cleveland 1979), we set r = 3 in the current study.

When $\overline{\mathbf{C}}$ was generated, we can use the Formula (2)–(5) to calculate the synchronization index, and also use the Formula (6) to implement the frequency recognition.

Time delay embedding

In the literature (Zhang et al. 2017), the authors extended the MSI by concatenating the first-order delayed version of EEG data (termed as EMSI). The delayed version was appended to X as:

$$\tilde{\mathbf{X}}={\left[{\mathbf{X}}^T,{\mathbf{X}}_d^T\right]}^T$$ 10

where $\tilde{\mathbf{X}}\in {\mathbb{R}}^{2N\times T}$ is the augmented EEG data. ${\mathbf{X}}_d\in {\mathbb{R}}^{N\times T}$ denotes the first-order time delay with d number of samples.

$${\mathbf{X}}_d=\left[{{\mathbf{X}}^{\prime }}_d,{\mathrm{O}}^{N\times d}\right]$$ 11

where ${{\mathbf{X}}^{\prime }}_d\in$ ${\mathbb{R}}^{N\times (T-d)}$ denotes the data copy from time d + 1 to time N (in data points). ${\mathrm{O}}^{N\times d}$ denotes zero matrix.

After the $\tilde{\mathbf{X}}$ was obtained, we could use the formula (2)-(5) to calculate the new synchronization index and implement the frequency recognition by the rule defined by formula (6).

Application of filter bank decomposition

The filter bank strategy has been used to take full advantage of harmonic components in the SSVEP signal (Chen et al. 2015; Qin et al. 2021). Firstly, the sub-band decomposition was performed with multiple Chebyshev zero-phase filters that have different pass-bands. Then, the synchronization indices or correlation coefficients between each sub-band component and the reference signal were calculated respectively.

For FBMSI, MSI was used to calculate the synchronization index corresponding to all subbands. In this paper, time delay embedding and temporally local method are incorporated into FBMSI respectively to obtain FBEMSI and FBTMSI, i.e., the calculation of the first sub-band is replaced by TMSI or EMSI. According to our previous research results, if TMSI or EMSI is applied to all sub-bands, the amount of computation will be increased, but the recognition accuracy will not necessarily be improved. The principle of the above filter bank methods is shown in Fig. 1.

Fig. 1.

Flowchart of FB(T/E)MSI

A weighted sum of squares of the synchronization indices corresponding to all sub-band components is calculated as the feature for target identification:

$${\tilde{S}}_n=\sum _{l=1}^{N_{\mathit{fb}}}q\left(l\right)·{\left(S_n^l\right)}^2$$ 12

where l is the index of the sub-band, N_fb is the number of filter banks. q(l) is the weight of the lth sub-band component, expressed as:

$$q\left(l\right)=l^{-a}+bl=1,2,\dots ,N_{\mathit{fb}}$$ 13

where a and b are constants that maximized the classification performance.

Experimental dataset

The data used in this study are from the benchmark dataset, which consists of 64-channel Electroencephalogram (EEG) data from 35 healthy subjects. The virtual keyboard of the speller was composed of 40 visual flickers, which were coded using a joint frequency and phase modulation (JFPM) approach. The stimulation frequencies ranged from 8 to 15.8 Hz with an interval of 0.2 Hz. The phase difference between two adjacent frequencies was 0.5π.

The data was collected by SynAmps2 (Neuroscan Inc.) at a sampling rate of 1000 Hz, which was then downsampled to 250 Hz. Data from nine electrodes (Pz, PO5, PO3, POz, PO4, PO6, O1, Oz, and O2) were used as the input to the analysis. The reference electrode was placed at the vertex (Cz). For more details, please refer to the literature (Wang et al. 2017).

Evaluation

According to the section “Experimental dataset”, the number of channels for EEG data is nine. In data analysis, a time delay of ∼ 140 ms should be added in the extraction of SSVEP epochs. The time window was set to 0.5 s to 1.5 s with an interval of 0.1 s. Meanwhile, this study set N_h as five for all standard MSI processes.

One-way repeated measurement analysis of variance (ANOVA) was used for statistical analysis. The Greenhouse–Geisser method was used to correct Mauchly’s Test of Sphericity. When the main effect was significant (p < 0.05), post hoc pairwise comparisons of t-tests were performed with Bonferroni correction. The analyses were conducted in SPSS Statistics 26 (IBM, Armonk, NY, USA).

Accuracy and ITR were used as evaluation indicators. The accuracy was defined as the ratio of the number of correctly identified targets to the total number of targets. The ITR in bits per min(bpm) was defined as:

$$ITR=60·\left({log}_{2}M+P{log}_{2}P+\left(1-P\right){log}_2\frac{1-P}{M-1}\right)/H$$ 14

where M denotes the number of classes, P denotes the accuracy, and H (in s) denotes the selection time including gaze time and gaze shift time. A gaze shift time of 0.5 s was used for analysis.

Results

Before verifying and comparing the performance of the methods, the parameters of each method need to be determined. The optimization used the grid search method based on the average classification accuracy of the first seven subjects, and the data length was 1.0 s. Figure 2 shows the classification accuracy of TMSI corresponding to different temporally local ranges (from 2 to 15 with a step of 1). When the temporally local range was five, the classification accuracy reached the maximum.

Fig. 2.

Classification accuracy of TMSI method with differentτ values, when the time window is 1.0 s

For FBMSI, the main parameters are the number of sub-bands and weight parameters a and b. In the previous literature (Qin et al. 2021), when the data length was 1.0 s, it was determined that the number of sub-band decomposition (SDN) is nine, and the weight parameters a and b are 2 and 0.1 respectively. Based on the benchmark dataset, this paper reoptimizes the number of sub-bands under the same data length and combination of weight parameters. As shown in Fig. 3, the classification accuracy basically reached the maximum when the number of sub-bands was seven. After the number of sub-bands was determined, the grid search was carried out on the weight parameters a and b of FBMSI. The optimal values were 1.25 and 0.1, as shown in Fig. 4.

Fig. 3.

Classification accuracy of the FBMSI method using different number of sub-bands from 2 to 9. The data length of SSVEPs was 1.0 s

Fig. 4.

Grid search for optimizing the parameters a,b of FBMSI

According to the previous literature (Zhang et al. 2017) and research results, the delay parameter d of the EMSI should be set to one. In FBEMSI, the synchronization index corresponding to the first sub-band is calculated by EMSI, and the delay parameter remains the same. For the weight parameters of FBEMSI, the optimal values of a and b obtained by grid search were 1.5 and 0, when the number of sub-bands was also set to seven, as shown in Fig. 5.

Fig. 5.

Grid search for optimizing the parameters a,b of FBEMSI

The parameter optimization process of FBTMSI still uses the data of the first seven subjects, and the data length is also 1.0 s. Figure 6 shows the classification accuracy of FBTMSI at different temporally local ranges, and the values of weight parameters a and b were temporarily set as 1.25 and 0.1. When the temporally local range equaled 12, the classification accuracy reached the maximum. Then, the temporally local range was fixed as 12, and the grid search for weight parameters a and b was carried out. Figure 7 discusses this process. The best combination obtained by the search was [1.25, 0.25].

Fig. 6.

Classification accuracy of FBTMSI method with differentτ values, when the time window is 1.0 s

Fig. 7.

Grid search for optimizing the parameters a,b of FBTMSI

Figure 8 compares the classification accuracy of MSI and its extensions with different time windows. One-way repeated measures ANOVA showed there were significant differences between these methods for all time windows (p < 0.001). Post-hoc paired t-tests showed that there were significant differences between MSI and the other three extension methods. When the time window was greater than 0.8 s, there existed MSI < EMSI < TMSI < FBMSI. When the time window was less than 0.8 s, TMSI has the highest accuracy compared with the other three methods. For all methods, the classification accuracy tended to increase with the increasing length of the time windows. Figure 9 shows the ITR of different methods. The results of ANOVA and post-hoc comparison were the same as those of accuracy. However, when the time window reached a certain length, ITR no longer increased.

Fig. 8.

The average classification accuracy obtained by the four methods with different time windows across all subjects. The error bars indicate standard errors. The asterisks symbolize the significant difference between the four methods according to one-way repeated measures ANOVA (***P < 0.001)

Fig. 9.

The average ITR obtained by the four methods with different time windows across all subjects. The error bars indicate standard errors. The asterisks symbolize the significant difference between the four methods according to one-way repeated measures ANOVA (***P < 0.001)

Figure 10 shows the classification accuracy of FBMSI and its extensions under different time windows. One-way repeated measures ANOVA showed there were significant differences between these methods for all time windows. Post-hoc paired t-tests showed that there were significant differences between FBMSI and FTMSI, except for 1.5 s. When the time window was greater than 0.8 s, there were significant differences between FBMSI and FBEMSI, while for FBEMSI and FBTMSI, the differences were only significant at 0.6 s and 0.7 s. For all methods, the classification accuracy tended to increase with the increasing length of the time windows.

Fig. 10.

The average classification accuracy obtained by the three methods with different time windows across all subjects. The error bars indicate standard errors. The asterisks symbolize the significant difference between the three methods according to one-way repeated measures ANOVA (*P < 0.05, **P < 0.01, ***P < 0.001)

The ITR of the three filter bank methods is shown in Fig. 11. One-way repeated measures ANOVA showed there were significant differences between these methods, except the time window was 0.5 s. Post-hoc paired t-tests showed that there were significant differences between FBMSI and FTMSI, except for 0.5 s, 1.2 s, and 1.5 s. When the time window was greater than 0.8 s, there were significant differences between FBMSI and FBEMSI. While for FBEMSI and FBTMSI, the differences were only significant at 0.7 s. When the length of the time window increased to a certain value, ITR began to decrease.

Fig. 11.

The average ITR obtained by the three methods with different time windows across all subjects. The error bars indicate standard errors. The asterisks symbolize the significant difference between the three methods according to one-way repeated measures ANOVA (*P < 0.05, **P < 0.01, ***P < 0.001)

Discussion

Based on idea of temporally local covariance and time delay embedding, TMSI (Zhang et al. 2016) and EMSI (Zhang et al. 2017) methods were proposed respectively. At the same time, combined with the idea of filter bank decomposition, the FBMSI algorithm was developed (Qin et al. 2021). The literature showed that the performance of these three algorithms was better than that of MSI, and Figs. 8 and 9 again verify this conclusion. However, the differences between the three algorithms have not been compared in detail. In this paper, we illustrate the problem under the unified framework. The classification accuracy and ITR results agree that TMSI has a better effect in a short time window, and FBMSI is more suitable for longer data length.

In order to extract time–frequency features simultaneously, this paper combines FBMSI with the two methods of time information mining respectively to obtain FBEMSI and FBTMSI. Under most time windows, FBEMSI and FBTMSI are significantly different from FBMSI. It indicates that by adding temporal information to the calculation of the first sub-band, the performance of the FBMSI algorithm was significantly improved, regardless of time delay embedding or temporally local information. However, there are significant differences between FBEMSI and FBTMSI only under a few time windows, and there are accidental factors, manifesting that the improvement of FBMSI by the two methods is similar.

In general, the improvement of recognition accuracy would lead to an increase in calculation consumption. This study estimated the computational time of MSI and its extension methods, which include preprocessing, feature extraction, and classification. Table 1 lists the total computational time required in 40-trials analysis for each method under 1.0 s data length. In order to ensure the accuracy of the results, each method was run three times. The evaluation process was based on MATLAB R2019b on Microsoft Windows 11 (with an Intel i7-12700H 2.3G processor). From the evaluation results, it can be seen that the calculation time of the six methods in the table could meet the needs of online experiments. The recognition effect of TMSI under the 1.0 s window is lower than that of FBMSI, but the computational cost is higher. For the two extension methods of FBMSI, the computational cost of FBTMSI is significantly increased, but the recognition accuracy is almost the same as that of FBEMSI.

Table 1.

The total computational time of 40 trials using six methods under 1.0 s data length

Method	Calculation order
	First (s)	Second (s)	Third (s)
MSI	0.42	0.43	0.43
EMSI	0.68	0.71	0.69
TMSI	5.02	4.89	4.78
FBMSI	2.21	2.29	2.19
FBEMSI	2.49	2.47	2.44
FBTMSI	6.74	6.78	6.65

The temporally local range is very important to the performance of the algorithm. If the value is too high, the algorithm is similar to the traditional MSI, but if the value is too small, the effective features will be suppressed. Figures 2 and 6 show the search graphs for the temporally local range of TMSI and FBTMSI respectively, and the changing trends are similar to those of TCCA and FBTCCA (Shao and Lin 2020). However, the optimal value of Fig. 2 is different from that in the previous literature (Zhang et al. 2016), which may be due to the different experimental designs. The temporally local range of this paper is determined by grid search, and the fixed value is used in the subsequent calculation. How to use variable parameters to achieve independent parameter optimization is the future research direction.

After the filter bank decomposition, a weighted sum of squares of the synchronization indices corresponding to all sub-band components was calculated. The weight parameters a and b influence the final feature value. Figures 4, 5 and 7 show the grid search results for the weight parameters of FBMSI, FBEMSI, and FBTMSI, respectively. It can be seen from the figure that the optimal parameter combination of different algorithms is different, indicating the instability of the parameter. The weight formula used in this paper has been used in many previous studies, and the design of a better weight formula may improve the performance of the filter bank algorithm.

At the same time, the number of sub-bands is also an important parameter. Figure 3 searches this value of FBMSI, and the results show that the number required is about seven. Increasing the number of sub-bands could not improve the recognition accuracy, but increase the computational consumption. However, according to the previous research, the number of minimum sub-bands required by different data lengths was different (Qin et al. 2021). This paper only focuses on the results of FBMSI when the data length is 1.0 s, while other data length and recognition algorithms need to be further studied.

Recently, deep learning methods have developed rapidly and been applied to multiple disciplines to solve practical problems (Altan and Karasu 2020; Yag and Altan 2022). There have been several papers describing deep learning methods for decoding BCI (Liu et al. 2021b; Zhang et al. 2021). In the future, we can explore whether the deep learning method can be combined with the method in this paper to improve the system performance.

Conclusion

This paper makes a detailed comparison of the performance of EMSI, TMSI, and FBMSI, under a unified framework for the first time. Then, two kinds of temporal information utilization strategies are integrated into FBMSI respectively, and two new time–frequency feature extraction algorithms are obtained. The analysis results indicate that the recognition effect of FBEMSI, in which the calculation time is shorter, is similar to that of FBTMSI. The research of this paper provides a meaningful direction for the improvement of the recognition algorithm based on MSI, and the proposed FBEMSI has great potential.

Data availability

This study is based on benchmark open dataset. The datasets are available in the http://bci.med.tsinghua.edu.cn/download.html.

Declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.