A dynamic center and multi threshold point based stable feature extraction network for driver fatigue detection utilizing EEG signals

Abstract

Driver fatigue is the one of the main reasons of the traffic accidents. The human brain is a complex structure, whose function can be evaluated with electroencephalogram (EEG). Automated driver fatigue detection utilizing EEG decreases the incidence probability of related traffic accidents. Therefore, devising an appropriate feature extraction technique and selecting a competent classification method can be considered as the crucial part of the effective driver fatigue detection. Therefore, in this study, an EEG-based intelligent system was devised for driver fatigue detection. The proposed framework includes a new feature generation network, which is implemented by using texture descriptors, for fatigue detection. The proposed scheme contains pre-processing, feature generation, informative features selection and classification with shallow classifiers phases. In the pre-processing, discrete cosine transform and fast Fourier transform are used together. Moreover, dynamic center based binary pattern and multi threshold ternary pattern are utilized together to create a new feature generation network. To improve the detection performance, we utilized discrete wavelet transform as a pooling method, in which the functional brain network-based feature describing the relationship between fatigue and brain network organization. In the feature selection phase, a hybrid three layered feature selection method is presented, and benchmark classifiers are used in the classification phase to demonstrate the strength of the proposed method. In the experiments, the proposed framework achieved 97.29% classification accuracy for fatigue detection using EEG signals. This result reveals that the proposed framework can be utilized effectively for driver fatigue detection.

Introduction

Background

Driver fatigue is a common driving problem, which is the one of the main reasons of traffic accidents. Almost each driver becomes fatigue after a while, particularly when driving for a long time or monotonous driving that become the main reason of driver fatigue. Fatigue produces attention deficiency, slow response and failure to conquer unpredicted circumstances, which is defined in numerous high-profile accidents (Hu and Min 2018; Oprea et al. 2020; Zeng et al. 2018). Electroencephalogram (EEG) is utilized to analyse the electrical activity of the brain. EEG signals contains different frequency bands, which are ranked from high to small as gamma, beta, alpha theta and delta. The state of the brain is represented by each frequency band (Wang et al. 2014, 2018). The EEG signals are the activation of the electrical potential of the brain. Electrical potential is produced by the activation of millions of neurons in the nervous system in an asynchronous structure (Halim and Rehan 2020; Hu and Min 2018). The information obtained from the patterns of EEG signals are utilized for the diagnosis of the brain disorders. The use of EEG signals is helpful for researchers and healthcare professionals in the early diagnosis of Alzheimer's disease (Al Ghayabet al. 2016; Houmani et al. 2018; Liu et al. 2014), in the detection of brain damage (Li et al. 2015), in tumor detection (Selvam and Devi 2015), in the diagnosis of epilepsy (Noachtar and Rémi 2009; Seneviratne et al. 2013), and in the detection of many other neurological disorders. Utilizing an appropriate signal processing method and applying an effective classifier, abnormal EEG signals can be easily distinguished from the normal ones (Chen et al. 2018; Manshouri et al. 2020). Moreover EEG signals can be utilized to determine the status of the brain such as stress, sleep, awake and fatigue (Li et al. 2019; Luo et al. 2019). Stress, fatigue, and sleep deprivation have all been shown to affect human performance.

Research motivation

Being stressed or tired means that a person is over- or under-stimulated and will not do his best (Lin et al. 2018). In particular, the failure to ensure safe vehicle driving and accidents are commonly caused by the weary of the user (Luo et al. 2019) With the help of artificial intelligence and machine learning (ML) methods, EEG signals can be interpreted in a beneficial way so that fatigue detection can be done quickly and problems can be solved. Fatigue is the act of extreme tiredness, weariness, or exhaustion as a result of mental or physical exertion or illness. Driving is among the most sensitive tasks and operations and it requires high levels of driver attention, focus, and concentration. It is only when the driver is alert, focused, and concentrating that he/she can remain in full control of the vehicle resulting to accident free journeys. Fatigue is one of driving consequences that a driver is likely to experience. This is especially after driving for a long time or distance without taking a rest or in the case the driver is undergoing stressing personal and environmental situations or even illness. Driver fatigue has been said to cause approximately 75% of road carnages or other machine-driven accidents. It is hence said that managing to understand and control driver fatigue can go along way reducing the number of accidents that emanates from drivers’ mistakes as a result of fatigue. Accidents have caused so many lives both young and aged and it is high time that somebody has to arise and take the right action which is to save lives through reducing accidents.

Many studies have been carried out in the literature using machine learning (ML) techniques and these studies show that ML techniques can be used effectively in different areas (Abdar et al. 2019; Hammad et al. 2020; Pławiak et al. 2019; Pławiak and Acharya, 2019; Tuncer et al. 2020). However, there are studies such as driver fatigue, mental fatigue, driver drowsiness estimation, path planning, which are focused on in this article and presented with ML techniques. Some of the EEG based fatigue related works are listed in Table 1.

Table 1.

Literature review

	Goal	Method	Database	Performance criteria
Li et al. (2019)	Pre-service fatigue screening for construction workers	Regression model	Collected data	Power spectral entropy, gravity frequency, time, averaged mental fatigue level, mental fatigue value
Wang et al. (2015)	Driving fatigue detection	Back propagation neural network	Collected data	Accuracy
Kar et al. (2010)	The assessment and quantification of driver’s fatigue	Wavelet entropy	Collected data	Fatigue level
Luo et al. (2019)	Fatigue driving detection	Adaptive scaling factor, multi-scale entropy	Bonn dataset (Andrzejak et al. 2001)	Accuracy
Yang and Ren (2019)	Detection exercise-induced fatigue	Hilbert–Huang transform, multivariate empirical mode decomposition	Collected data	Correct rate
AlZu'bi et al. (2013)	Driver fatigue detection	IIR band-pass filter, power density estimation	Collected data	Time, fatigue alert level
Azarnoosh et al. (2011)	Mental fatigue	Symbolic dynamics	Collected data	Commission error, false alarm, sensitivity index, error percentage
Yao et al. (2009)	Systematic brain signal adaptations	Chaos	Collected data	Force, average mutual information, false nearest neighbors
Lin et al. (2018)	EEG and HRV entropy analysis	Wiener entropy	Data collected from twenty-two students attending National Chiao Tung University (Hsinchu, Taiwan)	Effectiveness scores
Charbonnier et al. (2016)	Control operators’ mental fatigue monitoring	Spatial covariance	Collected data	Time, boxplot analysis
Sengupta et al. (2017)	Fatigue analysis	Discrete wavelet transform	Collected data	Energy
Zhang et al. (2014)	Driver fatigue	Transfer learning, mean power frequency, median frequency, energy	Collected data	Recognition rate, energy
Gao et al. (2018)	Fatigue driving	Relative wavelet entropy	Data collected from Complex Network Laboratory, Tianjin University	Receiver operating characteristic curve, true positive rate, false positive rate
Halim et al. (2019)	Optimum commuting path	A* algorithm, Dijkstra's and Bellman–Ford algorithms	Created data	Length
Haider et al. (2020)	Collision avoidance	Hamming distance	Collected data	Warning message generation rate, unwanted warning messages generation
Gao et al. (2019)	Driver fatigue evaluation	Convolutional neural network	Collected data	Accuracy
Cui et al. (2019)	Driver drowsiness estimation	Feature weighting	Collected data	Root mean squared error, Pearson correlation coefficient
Halim et al. (2016)	Driving safety	Artificial intelligent techniques	Collected data	Frames, attributes
Halim et al. (2016)	Road safety	Artificial neural networks	Collected data	Variance of the principal components, iterations, objective function, accuracy

Contribution

Driver fatigue detection can be realized by utilizing EEG signals. Nevertheless, a robust driver fatigue detection by employing the EEG signals produces different complications. To eliminate these complications and reach high classification performance, an appropriate combination of the feature generation techniques (the used feature generators must extract low, medium and high levels features), and the dimension reduction (informative feature selection methods) methods must be employed. Besides, a proper classifier (we used four traditional classifiers to illustrate discriminative ability of the generated and reduced features) to enhance the classification perfromance should be employed. Since the brain signals are of multi-dimensional nature and they cannot be modelled mathematically, it is not easy to propose a robust feature generation, dimension reduction and classification techniques. Hence the aim of this study is to develop framework with a good combination of feature generation, dimension reduction and classification techniques. The developed model has 5 layered feature extraction network. Firstly, discrete cosine transform (DCT) (Ahmed et al. 1974; Rao and Yip 2014) and Fast Fourier transform (FFT) (Van Loan 1992) are used together in the pre-processing phase. In this network, dynamic center based binary pattern (DCBP) and multi threshold ternary pattern (MTTP) are used together. To create layers, discrete wavelet transform (DWT) (Shensa 1992) is used with “sym4” filter. Sym4 filter is very effective for both decomposition and noise reduction. The proposed DCBP–MTTP based feature generator extract 34,560 features. To select discriminative features a 3-layered hybrid feature reduction scheme is used, and 108 the most informative/meaningful features are obtained. In the classification phase, four shallow classifiers are used to illustrate success of the proposed DCBP–MTTP based feature generation network and 3-layered feature reduction methods. The major contributions of the proposed framework are listed in below.

• A center pixel is used for feature extraction in the 1D-BP. Thus, in order to comprehensively extract features, dynamic center based binary pattern (DCBP) is proposed for feature extraction.

• The main problem of the ternary pattern (TP) is how to set the threshold value. Therefore, a multiple threshold value-based TP is utilized as one of the feature extractors.

• By using a multi threshold ternary pattern (MTTP) and DCBP, novel textural feature extractor module is developed. Also, 1D-DWT with Symlets 4 (sym4) filter is used for pooling since the sym4 filter is very effective method in noise reduction. By using 1D-DWT (Ramírez et al. 2000) with sym4 filter, both feature extraction layers are created and noise reduction is realized.

• Neighbourhood component analysis (NCA) (Yang et al. 2012), ReliefF (China 2015), and Principle Component Analysis (PCA) (Chen and Zhu 2004) are widely used feature selection techniques. In this article, a novel hybrid feature selection method which is called as RFNCAPCA is proposed and 34,560 features are reduced to 108 features.

Organization

In this study, a dynamic center and multi threshold point based stable feature extraction method is proposed for fatigue detection. This work consists of 5 sections. Theoretical background is given in second section. "Theoretical background" section also contains detailed steps of dynamic center based binary pattern and multi threshold based ternary pattern. The proposed fatigue detection method is explained in third section. Results and discussion are presented in fourth section and conclusion is given in last section.

Theoretical background

In this article, two textural feature extraction methods namely DCBP and MTTP are proposed. These are improved versions of the 1D binary pattern (Kaya et al. 2014) and ternary pattern (Ren et al. 2013) respectively. Textural descriptors are very effective feature extractors and they use local relationships to generate global optimal features of the signal or image. They have less computational complexities and can extract discriminative features. They can also be used in both images and signals.

Dynamic center based binary pattern (DCBP)

In the local binary pattern (LBP) (Ojala et al. 2002) and 1D-BP (Kaya et al. 2014), a center pixel is used to extract features. DCBP uses all of the values in the block as center pixel. Therefore, it extracts 256 × 9 = 2304 features from a signal. Because, DCBP extracts 8-bits features by using each center value. It is deeper feature extractor than 1D-BP because all possibilities are considered in the DCBP. Equation 1 denotes mathematical description of the signum function (kernel of the LBP).

$$Sig\left(V_1,,,V_2\right)=\left\{\begin{array}{c}0,V_1<V_2\\ 1,V_1\ge V_2\end{array}\right)$$ 1

where $Sig(.,.)$ is signum function, $V_1$ and $V_2$ are parameters of this function.

As can be seen from Eq. 1, signum function extracts 0 or 1. Since the DCBP is an LBP like descriptor, it is considered as kernel (binary feature generator) in most of the LBP like descriptors. The steps of the proposed DCBP are given as below.

• Step 1 Divide signal into 9 sized non-overlapping blocks.

• Step 2 Create parametrical BP function to use each value as center value. Procedure of the parametrical center based BP is shown in Algorithm 1.

• Step 3 Extract features by using P1D-BP and concatenate extracted features. Both feature generation and feature fusion processes are performed using Algorithm 2.

The given steps demonstrate that the DCBP is a very simple feature extractor, and graphical presentation of DCBP is shown in Fig. 1.

Fig. 1.

A graphical example about DCBP

Figure 1 clearly shows that the DCBP extracts nine 8-bits feature values and each value in a block is used as a center value respectively.

Multi threshold based ternary pattern (MTTP)

Local ternary pattern (LTP) is one of the widely used LBP like descriptor proposed by Tan and Triggs (2007). It is similar to LBP because it uses 3 × 3 sized block. However, it uses ternary function as kernel and it is defined in Eq. 2.

$$Ter\left(V_1,,,V_2\right)=\left\{\begin{array}{c}-1,V_1-V_2<-thr\hfill \\ 0,-thr\le V_1-V_2\le thr\hfill \\ 1,V_1-V_2>thr\hfill \end{array}\right)$$ 2

where $Ter(.,.)$ is ternary function and $\mathit{thr}$ describes threshold value. As seen from Eq. 1, ternary function generates three values and these values are − 1, 0 and 1. Upper and lower bit values are created as shown in Eqs. 3 and 4.

$$lowe{r}^{\mathit{bit}}=\left\{\begin{array}{cc}0,\hfill & Ter\left(V_1,,,V_2\right)>-1\hfill \\ 1,\hfill & Ter\left(V_1,,,V_2\right)=-1\hfill \end{array}\right)$$ 3

$$uppe{r}^{\mathit{bit}}=\left\{\begin{array}{c}0,Ter\left(V_1,,,V_2\right)<-1\hfill \\ 1,Ter\left(V_1,,,V_2\right)=1\hfill \end{array}\right)$$ 4

where $lowe{r}^{\mathit{bit}}$ and $uppe{r}^{\mathit{bit}}$ define lower and upper bits respectively.

This equation clearly shows that the main problem of the TP, which is how to determine the threshold value. Therefore, a multi-threshold values based TP is proposed. A standard deviation based automatic threshold value determination strategy is used. The mathematical explanation of this is given as Eq. 5.

$$thr=\frac{SD\left(s,i,g,n,a,l\right)\ast i}{10},i=\{\text{1,2},\cdots ,9\}$$ 5

where $SD(.)$ is standard deviation function, $\mathit{thr}$ defines threshold value and $i$ is multiplier. 1D-TP extracts 512 features. The MTTP uses 10 threshold values, hence it extracts $512\times 9=4608$ features. The procedure of the MTTP is given in Algorithm 3.

Moreover, we can use parametric TP. Hence, DCBP and MTTP are utilized as feature generation module.

The proposed fatigue detection method

In this study, a new multilevel learning framework is developed to detect fatigue with high classification accuracy by using the proposed dynamic center and multi threshold point based stable feature extraction network. Therefore, 5 layered feature generation network is proposed to generate low, medium and high levels features. The proposed DCBP–MTTP feature generation network and RFNCAPCA feature selector based method contains DCT–DFT based pre-processing, feature generation using DCBP–MTTP network, feature selection with RFNCAPCA selector and classification phases. Graphical representation of this method is shown as Fig. 2.

Fig. 2.

Graphical overview of the proposed DCBP and MTTP based feature generation for fatigue detection

Figure 2 indicates that the proposed DCBP–MTTP based feature generation network has 5 layers and 1D-DWT is utilized as pooling. Steps of our DCBP–MTTP feature generation network and RFNCAPCA feature selector based method are explained as follows.

Pre-processing

DCT and FFT are widely used for signal transformation. Both DCT and FFT generates frequency coefficients. But DCT uses cosine and it generates real frequency coefficients, on the other hand, FFT generates both imaginary and real frequency coefficients. We utilized both of them in the pre-processing phase. Steps of this phase are given below.

• Step 0 Load raw EEG signal.

• Step 1 Apply DCT to raw EEG signal.

  $$C^{\mathit{DCT}}=DCT\left(s,i,g,n,a,l\right)$$ 6

  where $C^{\mathit{DCT}}$ coefficients of the DCT, $DCT(.)$ is DCT function.
• Step 2 Apply FFT to coefficients of the DCT by using Eq. 7.

  $$C^{\mathit{FFT}}=FFT\left(C^{\mathit{DCT}}\right)$$ 7

  where $C^{\mathit{FFT}}$ is coefficients of the FFT, $FFT(.)$ is FFT coefficients calculation function.
• Step 3 Divide imaginary and real part of the $C^{\mathit{FFT}}$ and concatenate these parts.

  $$signa{l}^P=imaginar\left(C^{\mathit{FFT}}\right)|real\left(C^{\mathit{FFT}}\right)$$ 8

  where $signa{l}^P$ is preprocessed signal and $|$ is concatenation operator.

Feature generation

In the feature generation phase, a 5 layered network is proposed and DCBP and MTTP are utilized for feature extraction. In this section, pre-processed signal is utilized as input. Also, DWT is used as a pooling method. In this phase, 34,560 features are extracted because DCBP and MTTP generate 2304 and 4608 features respectively. Totally, $2304+4608=6912$ features are extracted in each layer. Since all of the features are concatenated, $6912\times 5=34,560$ features are extracted in total. Details of the proposed DCBP–MTTP feature generation network is shown in Algorithm 4.

In this phase, DWT with Sym4 filter is utilized as a pooling method. Sym4 is a widely used filter for signal processing because its noise reduction capability is very high and effective.

Feature selection

Three widely used feature selection techniques namely NCA, ReliefF and PCA are used together. Each of them generates weights for every feature and these features are used to find discriminative features. In this step, 108 features are selected from 34,560 features. The steps of the proposed 3 layered RFNCAPCA method are given as below.

• Step 1 Apply ReliefF to feature vectors extracted in the feature extraction step. Since the ReliefF generates both negative and positive weights, in order to eliminate negative weighted features, we have chosen ReliefF for feature selection. The redundant features elimination process is shown in Algorithm 5.

• Step 2 Apply NCA to $\mathit{featR}$ and select 1000 most distinctive features. The most important characteristic of the NCA is to generate non-negative features for each feature. It uses distance-based metrics to generate weights and uses stochastic gradient descent optimization. As we know from the literature, most of the deep network for instance AlexNet (Krizhevsky et al. 2012), GoogleNet (Szegedy et al. 2015), ResNet (He et al. 2016) extracts 1000 features in the last fully connected layer. Therefore, we selected 1000 most distinctive features of the $\mathit{featR}$. Algorithm 6 explains NCA based feature selection.

• Step 3 Calculate PCA weights and select positive weighted features. PCA is well known dimension reduction method. It uses Eigen values to reduce data dimension. After applying the proposed feature selection scheme, the size of final feature is found as 108. The proposed dynamic center and multi threshold point based stable feature extraction network and the 3-layered feature selection method aim to generate and select informative or discriminative features. Therefore, we used shallow classifiers in the classification phase to illustrate strength of these methods.

Classification

The last phase of the proposed method is classification. We used traditional classifiers to demonstrate strength of the proposed feature generation network. Therefore, k-nearest neighborhood (k-NN) (Keller et al. 1985), artificial neural network (ANN) (Haykin 2009), random forest (Díaz-Uriarte and De Andres 2006) and support vector machine (SVM) (Suykens and Vandewalle 1999) are utilized as classifiers. In the testing and training phases, tenfold cross validation is used. The attributes of the used classifiers are listed in Table 2. These parameters are selected by trial and error to achieve the highest classification accuracy.

Table 2.

Attributes (parameters) of the used four classifiers.

Classifier	Attribute
k-NN	k-NN is one of the widely preferred distance based classifier and it uses many parameters such as k value and distance metric. We set these parameters as follows. k and distance metric are selected as 1 and City Block (Manhattan) distance
ANN	ANN is one of the traditional classifiers and variable parameters can be used to set ANN. ANN consists of input, hidden and output layers. Backpropagation method is selected as scaled conjugate gradient (trainscg). Number of hidden layers is chosen as 50
RF	RF is one of improved version of DT. Hyper-parameters of it are; bootstrap is true, 200 is selected as maximum depth, minimum sample of leaf is chosen as 4 and number of estimators is set as 107
SVM	SVM has many kernel and is an optimization based classifier. Therefore, time cost of the SVM is $O\left(n^3\right)$. We used Cubic (3rd degree polynomial) SVM. Box constraint level (C) value is selected as 1 and auto is selected as kernel scale mode. These are default setting of the Cubic SVM in the MATLAB classification learner

Results and discussion

In this section, information about the used publicly available EEG signal dataset, the obtained results and discussions of the obtained results are given.

Dataset

In this study, the database of Luo et al.’s study was used (Luo et al. 2019). A static simulator (ZY-31D vehicle driving simulator) is used to create this database. This system has a software teaching system for driving simulations called as ZM-601 V9.2. 24-in. monitor was used to monitor the system. The data was collected via an EEG collecting cap with 32 electrodes. It has windows operating system and Neuroscan3.2 software was used for pre-processing. Data in this system was processed with MATLAB. This data was collected from 16 subjects. These subjects are between the ages of 17–25. They can be grouped as fatigue and rested individuals. Figures 3 and 4 show an example for fatigue and rested individuals EEG signals (Luo et al. 2019).

Fig. 3.

A fatigue EEG signal example of the used dataset

Fig. 4.

A rest EEG signal example of the used dataset

Experimental results

To test the proposed DCBP–MTTP feature generator and RFNCAPCA feature selector based method, a personal computer (PC) and MATLAB 2018a programming environment were used. This PC has 16 gigabytes (GB) RAM and Intel Core i7 7th generation microprocessor with 3.6 GHz. To evaluate the proposed DCBP–MTTP feature generator and RFNCAPCA feature selector based fatigue detection method, accuracy, sensitivity and specificity are used (Tuncer et al. 2019). To calculate these evaluation metrics, number of true positives ($\mathit{tp}$), true negative ($\mathit{tn}$), false positives ($\mathit{fp}$) and false negatives ($\mathit{fn}$) are used. Eqs. 9–11 define the used classification metrics mathematically.

$$Accuracy=\frac{tp+tn}{fp+fn+tp+tn}$$ 9

$$Sensitivity=\frac{\mathit{tp}}{tp+fn}$$ 10

$$Specificity=\frac{\mathit{tn}}{fp+tn}$$ 11

The calculated results according to the classifiers are listed in Table 3.

Table 3.

Performances results of the used classifiers.

Evaluation metric	k-NN	ANN	RF	SVM
Accuracy	97.29	96.56	94.27	95.83
Sensitivity	97.08	96.66	95.00	96.04
Specificity	97.50	96.45	93.54	95.63

Table 3 clearly denotes that k-NN is the best classifier among the used classifiers because it achieved 97.29% classification accuracy, 97.08% sensitivity, 97.50% specificity rates respectively. The confusion matrix of the best results (for k-NN) is also shown in Fig. 5.

Fig. 5.

Confusion matrix of the k-NN classifier.

The computational (time) complexity of the proposed DCBP–MTTP feature generator and RFNCAPCA feature selector based method is also calculated by using big O notation. Time complexity of the pre-processing phase is calculated as $O\left(n\right)$. The feature extraction stage uses 2 loops and DWT based pooling. Therefore, time complexity of the proposed DCBP–MTTP feature generation network is calculated as $O\left(n^2,l,o,g,n\right)$. Then, we used 3-layered feature selection method. It is $O\left(3,n\right)$. In the classification phase, tenfold cross validation (CV) was used. The space complexity of the classification stage is calculated as $O\left(10,n\right)$.

We also measured CPU time of the proposed pre-processing and dynamic center and multi threshold point based stable feature extraction network for an EEG sample. This experiments were implemented on a PC. They were calculated as 0.019 and 0.82 s for DFT-DCT based pre-processing and dynamic center and multi threshold point based stable feature extraction network respectively. Execution times of the classifiers (these calculation was implemented for 960 × 108 sized matrix with tenfold CV) were also calculated as 6.06, 10.51, 6.95 and 5.08 s for k-NN, SVM, RF and ANN respectively. According to these results, execution time of testing a sample was calculated approximately 1 ms. These clearly implied that the proposed DCBP–MTTP feature generation network and RFNCAPCA feature selector based fatigue detection method is lightweight.

Discussion

This work aimed to implement a framework to automatically detect the driver fatigue by utilizing EEG signals. Nevertheless, a robust driver fatigue detection by employing the EEG signals produce different complications. To eliminate these complications and reach high classification performance, an appropriate combination of the feature generation techniques (the used feature generators must extract low, medium and high levels features), and the dimension reduction (informative feature selection methods) methods must be employed. Besides, a proper classifier (we used four traditional classifiers to illustrate discriminative ability of the generated and reduced features) to enhance the classification performance should be employed. Since the brain signals are of multi-dimensional nature and they cannot be modelled mathematically, it is not easy to propose a robust feature generation, dimension reduction and classification techniques. Therefore, in this work, a novel EEG signal recognition method for driver fatigue detection is proposed with different layers of feature extraction and dimension reduction. DCT and FFT are used together in the pre-processing phase. Two novel textural feature extraction method namely DCBP and MTTP is used for feature extraction. By using DCBP and MTTP together, a novel 5 layered feature extraction network is developed. A new hybrid feature selection module is also developed by using ReliefF, NCA and PCA together. The generated and selected informative features were classified with different benchmark classifiers and the results were compared for the driver fatigue classification/detection works. The calculated results of this study have shown that utilizing the developed feature extraction approach has improved the success rate significantly. The study of the classification of EEG signals in the literature has revealed that there was no study on utilizing the DCBP and MTTP together in this regard. The results achieved in this study has been found to accomplish satisfactorily with a success rate of 97.29%, compared with the literature examples.

This result clearly demonstrates the strength of the extracted and selected features by DCBP–MTTP feature generation network and RFNCAPCA feature selector. To prove the success of the developed framework features, boxplot analysis was used and results of it are shown in Fig. 6.

Fig. 6.

Statistical characteristics of the extracted features according to rest and fatigue classes

Since these features are normalized, the feature range is from 0 to 1. Boxplot analysis shows mean, median, minimum and maximum values of the features. Also, 3rd and 1st quartiles (Q3–Q1) range is shown in boxplot using blue boxes. Figure 6 indicates that the extracted and selected 108 features have separable statistical characteristics and this situation proves the achieved high classification performance.

Moreover, the classification ability of the proposed DCBP–MTTP and RFNCAPCA based framework is compared with the previous fatigue studies. The classification performances of these studies are presented in Table 4 and it demonstrates that the classification performance of the developed DCBP and MTTP based framework is more successful than the other state-of-the-art for driver fatigue detection. The total classification accuracy of the proposed method is 97.29% with tenfold cross validation.

Table 4.

Comparison of the proposed DCBP–MTTP based method with the state of art fatigue detection methods.

Method	Classification accuracy (%)
Mu et al. (2017)	85.0
Wang et al. (2018)	90.7
Li et al. (2012)	91.5
Luo et al. (2019)	95.37
Shalash (2019)	91
Wu et al. (2020)	83
Ma et al. (2019)	95
Liu et al. (2019a)	72.7
Chaudhuri and Routray (2019)	86
Dong et al. (2019)	95.81
Liu et al. (2019b)	96.5
Our method	97.29

Table 4 denotes the comparison of the selected 11 state-of-the-art driver fatigue detection methods with the proposed method and it indicates that the proposed DCBP–MTTP and RFNCAPCA based method reached 0.79% higher classification accuracy than Liu et al.’s method which is the best of other. By using ANN and SVM classifiers, the proposed DCBP and MTTP based feature extraction method also achieved better result than some of the previous studies.

The benefits of the proposed DCBP–MTTP and RFNCAPCA based framework are given as follows:

1. A new stable DCBP and MTTP based feature generation framework is proposed. Time cost of it is calculated as $O\left(n^2,l,o,g,n\right)$. It clearly denotes that the proposed DCBP and MTTP based network is lightweight.

2. Since any optimization algorithm is not used in this work to improve classification capability, the proposed DCBP–MTTP and RFNCAPCA based framework is cognitive.

3. The developed DCBP–MTTP and RFNCAPCA based framework is highly accurate and is outperformed.

4. 1D-BP and 1D-TP are very effective feature generators for signal analysis, the modified version of the BP and TP, namely DCBP and MTTP, are utilized for the feature generation to improve classification ability.

5. Our DCBP–MTTP and RFNCAPCA based framework can be applied for other biomedical signals classification.

The limitation of our framework is to use small dataset produced in a simulation environment. Since, there is no publicly available EEG dataset for fatigue detection, we tested the developed DCBP–MTTP and RFNCAPCA framework on this dataset.

Conclusions

A novel EEG signal recognition framework is proposed for driver fatigue detection in this work. The proposed framework uses DCT and FFT for pre-processing, DCBP and MTTP for feature generation, ReliefF, NCA and PCA as feature selector and k-NN, ANN, RF and SVM as classifiers. The proposed DCBP and MTTP based feature generation network is improved versions of the BP and TP respectively. This feature generation network has low time cost and high classification representative features. To indicate high classification capability of the proposed DCBP–MTTP feature generation and RFNCAPCA feature selection methods, benchmark classifiers such as k-NN, ANN, SVM and RF were chosen for classification. k-NN with a k value of 1 and with distance metric Manhattan achieved the best results among the benchmark classifiers. Accuracy, sensitivity and specificity rates were calculated as 97.29%, 97.08% and 97.50% respectively for k-NN using tenfold CV. The time complexity of the proposed DCBP–MTTP feature generation network was $O\left(n^2,l,o,g,n\right)$. These results clearly show that the proposed DCBP–MTTP feature generation and RFNCAPCA feature selection based fatigue detection method has low time cost and high accuracy.

The proposed DCBP and MTTP based feature extraction can be used to analyze other types of signals such as voice, speech, ECG, and EMG. By using the proposed framework, a novel healthcare monitoring system can be developed in the future studies. The proposed DCBP and MTTP can be used in deep learning networks instead of convolutional layers.

Funding

This project is funded by Effat University with the Decision Number of UC#7/28 Feb. 2018/10.2-44i, Jeddah, Saudi Arabia.

Compliance with ethical standards

Conflict of interest

The authors declare no conflict of interest.