Design of electroencephalogram authentication access control to smart car

Abstract

In recent years, with the development of intelligent vehicles, the demand for security will be bigger and bigger. One of the most important solutions is the use of new biometric technology. At present, there are still some areas to be improved on biometric technology. For example, diseases will destroy some biological characteristics, some detection methods are too slow, many detection methods do not need living detection, and so on. Electroencephalogram (EEG) is a new biometric tool for living identification. In this Letter, a kind of identity authentication system based on the EEG signal is presented. The overall goal of this research is to design a new authentication method and develop the corresponding application. Therefore, the authors carried out a series of EEG experiments, and analysed and discussed the experimental results. Based on these results, they build and present an access control system based on the uniqueness of their EEG signals to be capable of authenticating access control to the car. The accuracy of the authentication system is >87.3%.

1. Introduction

The authentication system of the smart car is an urgent part to be installed within smart car's compounds, which focused on security aspects of entry access. Conventional authentication systems use key and password, however, they are deemed inconvenient and insecure. Biometrics appears to solve these problems related to the conventional system [1]. Currently, several relatively mature biometric technologies including fingerprint [2], face [3], iris [4], signature [5], and speech [6] recognition are well studied. These methods have their disadvantages and advantages depending on the performance, usability, and capability [1]. Therefore, there is a need to explore and find another method of a biometric system.

Electroencephalogram (EEG) is a very good biometric feature, which can be used as a person authentication [7–12]. EEG signal has recently received substantial attention as a potential biometric. EEG signal is stable enough for biometric analysis, it is convenient to record it for a relatively long duration to obtain sufficient data to first train and then test a classifier. Owing to the difference between morphology and function, the brain needs to organise and coordinate the specific cognitive function or mental state. Recent studies have shown that some of the specific information about EEG can be used to automatically identify the subject's identity. Owing to the very high universality of the EEG signal, and it also has a strong ability to resist deception, EEG is very suitable for identification as a biometric signal. Since it is impossible for an attacker to imitate an EEG signal of the tested person and transmit it to the corresponding electrode.

When the subjects were collected in the resting state, the brain waves produced by brain activity were mainly distributed among the 0.5–40 Hz frequency bands. There are five main types of rhythms that can be found in EEG signals. It is assumed that the brain is the slowest in the inactive state, while the fastest rhythm indicates that the brain is currently processing information. Different EEG waveforms can be decomposed using different mathematical methods. Usually, the EEG signal is a combination of many different waveforms. We can classify it according to their frequency, amplitude, wave shape, and spatial distribution.

EEG has been used in many different fields for clinical diagnosis. For example, verification of brain death or coma, identification of epileptic seizures, movement disorders, migraine, deep anaesthesia testing etc. Each person's EEG is as unique as his fingerprints. It has a very strong uniqueness. When a person receives visual stimulation, the visual cortex of the brain on the back of the head produces a strong change in the EEG signal. Therefore, this region is the best position to measure the visual EEG signal.

On the management of system security, we must first distinguish between three completely different concepts: identification, authentication, and authorisation. The main content of this paper is the system authentication.

2. Literature survey

As a non-invasive brain–computer interface (BCI) application, the EEG-based biometric recognition system has received more and more attention in the scientific community. The identification of EEG signals is a relatively new method of biometric identification, which has benefited from the development of human genetics and clinical neurophysiology.

In 2002, Palaniappan and Raveendran [11] used visual evoked potentials (VEPs) for biometric identification [13, 14]. The system extracted the absolute spectral power (ASP) of the gamma band. These ASP features were extracted from 61 electrodes and classified by the neural network (NN), achieving on average 90.95% accuracy on ten subjects. Nguyen et al. developed a model that relies on a feature vector consisting of Mel-frequency cepstral coefficients, energy and pith measurements, spectral features, zero-crossing rate, and their statistical functions. The model achieved 92.8% classification accuracy on 20 subjects and 61.7% classification accuracy on 122 subjects [15]. Marcel and Millan developed a system, which included the imagined task of producing words, moving the right hand and the left hand. The system was based on a Gaussian mixture model classifier and power spectral density (PSD) features. In the left-hand moving task, they obtain the best performance of 6.60% of equal error rate [16]. Thorpe et al. presented the idea for one type of authentication system by pass-thoughts and described the design of this system. Since the human brain signal is unique, when we carry out the same thinking activities, different people produce EEG signals that are completely different. So they think such a certification system is feasible [17]. Singhal and RamKumar have designed a system that can measure VEP by a single electrode (Oz). The classification accuracy of this system achieved 78% on ten subjects [18]. Das et al. studied the spatiotemporal pattern responsible for encoding personal discriminant data. After EEG was classified by a linear support vector machine (SVM), the system achieved performances between 90 and 95% for 20 subjects [19]. Ferreira et al. developed a system based solely on a SVM. Their best effort achieved 84.33% accuracy on 13 subjects [20]. Sun et al. proposed a multi-task learning approach where an NN was trained with features extracted from multiple tasks EEGs, obtaining 95.6 and 94.81% of classification on imagining left- and right-hand moving task, respectively [21]. Su et al. used PSD from 5 to 32 Hz as features and auto regression coefficients, when classifying with a combination of linear discriminant classifier and k-nearest neighbour the system achieved 97.5% accuracy when classifying with a NN and a SVM, the system achieved 81.90 and 79.6% accuracy [22]. In the work of Bao et al., autoregressive mode, phase synchronisation, energy spectral density, and linear-complexity value were used as EEG features. The NN was employed for the identification of individual differences [23]. Our previous research is mainly based on the personal identification of EEG signals, and the classification method based on multi-feature fusion [24–28].

Some studies have shown that EEG signals can be recorded by mobile devices and sent to the server after signal preprocessing [29]. Kumar et al. have collected EEG signals of 50 users while drawing different patterns. The system identified 2400 unauthorised attempts. Their experiments reveal that the method is promising and can be a possible alternative to develop robust authentication protocols for hand-held devices [30]. Saini et al. proposed a novel multimodal user identification and verification scheme combining two inter-linked biometric traits, i.e. signature and EEG. User's identification is performed with individuals’ signature and EEG signals as well as their combined traits. The identification accuracy of the proposed multimodal approach has been achieved up to 98.24% [31]. Zúquete et al. processed VEP for authentication and used the image of the black and white line map by Snodgrass and Vanderwart to derive an EEG authentication system [32]. This is similar to the way we use it because we also used visual stimulation to establish the authentication system.

3. Method

The design of the new key for implementing the access control system (ACS) is developed using a model that actually works with various parts. These parts are summarised as shown in Fig. 1. Users through the interface of ACS control the authentication application software. Software directly calls BCI middleware to achieve the core functions. The user's EEG signals are also sent to the middleware through the electrode cap. The middleware also exchanges data with the algorithm library and the pattern library.

Fig. 1.

Conceptual components for ACS

3.1. EEG acquisition

Based on our previous research, a 40-channel neuroscan amplifier was used to collect EEG signals, and scan 4.3 software is adopted. Data from all electrodes were referenced to two electrically linked mastoids at A1 and A2, digitised at 1000 Hz from a 32-channel electrode cap (including 30 effective channels and two reference channels) based on the international 10–20 system (Fig. 2). The sampling frequency is set to 1000 Hz: 200 Hz low-pass and 0.05 Hz high-pass.

Fig. 2.

Electrodes position according to International 10–20 System Standards

3.2. Scenario

The experiments are carried out in the Collaborative Innovation Center of Automobile Service Engineering and Industrial Upgrading, Jiangxi University of Technology. Subjects are students and young teachers from the Jiangxi University of Technology. In the tests, subjects look towards the computer screen and perform operations according to experimental requirements.

In the tests, the stimulation programme will display different pictures on the computer screen. For each subject, five different pictures are displayed in each experiment. These pictures consist of one self-photo and any four non-self-photos in the photo gallery. A self-photo denotes a target subject's own photo, while the non-self-photo includes other familiar or unfamiliar photos. Each picture is randomly displayed on the screen for 1000 ms, which is followed by a 250 ms black screen, totally taking 1250 ms. Each picture uses the head part of the subject with the same background.

3.3. Data analysis

Feature extraction mainly includes the establishment of different types of data sets, the calculation of fuzzy entropy, the analysis of features, and the calculation of Fisher distance. We performed a signal analysis of the two types of EEG data from both self- and non-self-photographed responses. To realise the identification of human identity, we calculate the fuzzy entropy of the EEG signal. The formula of fuzzy entropy is shown as follows.

For a time series of EEG data, $\{a(i):i\in [1,\dots ,N]\}$, we can reconstruct an m-dimensional vector as follows:

$$\begin{array}{rl}{U}_i^m& =\left\{u\left(i\right),u\left(i+1\right),\dots ,u\left(i+m-1\right)\right\}\\ & =\left\{a\left(i\right),a\left(i+1\right),\dots ,a\left(i+m-1\right)\right\}\left(i=1,\dots ,N-m\right)\end{array}$$ (1)

where $\left\{u\left(i\right),u\left(i+1\right),...,u\left(i+m-1\right)\right\}$ and $\left\{a\left(i\right),a\left(i+1\right),...,a\left(i+m-1\right)\right\}$ represent m consecutive values from the ith point, and $a_0\left(i\right)$ represents the average of m values. The reconstructed vector can produce a maximum distance and we define it as follows:

$$d_{ij}^m=d\left[U_i^m,U_j^m\right]=max\left\{|u(i+k)-u(j+k)|\right\}k\in (0,m-1),i\ne j$$ (2)

and then the similarity degree, $D_{ij}^m$, of $U_i^m,U_j^m$ can be defined as follows:

$$D_{ij}^m=\mu \left(d_{ij}^m,n,r\right)=\exp {\left(d_{ij}^m\right)}^n/r$$ (3)

R and N are exponential functions of gradient and width. In this Letter, R = 0.3×SD (SD of the time series) and N = 2. Therefore, we can define the following functions:

$${\varphi }^m\left(n,r\right)=\frac{1}{N-m}\sum _{i=1}^{N-m}\left(\frac{1}{N-m-1}\sum _{j=1j\ne i}^{N-m}{D}_{ij}^m\right)$$ (4)

When the fuzzy entropy of the time series N is finite, its sequence length can be defined as

$$\mathrm{FuzzyEn}=\left[\ln {\phi }^m\left(n,r\right)-\ln {\phi }^{m+1}\left(n,r\right)\right]$$ (5)

The Fisher distance method is used to analyse the characteristics of EEG signals with different electrodes. Fisher distance is often used in classification studies to represent differences between classes. In this study, the Fisher distance is proportional to the distance between classes. The greater the difference, the greater the Fisher distance, and vice versa. The formula for calculating the Fisher distance is as follows:

$$F=\frac{{({\mu }_1-{\mu }_2)}^2}{{\sigma }_1^2+{\sigma }_1^2}$$ (6)

where F is the Fisher distance, $\sigma$ and $\mu$ are variance and mean, respectively, the subscripts 1 and 2 represent two types.

To provide a more intuitive and easier to understand method to measure the prediction quality. In this Letter, the following equations are used for performance testing:

$$\left\{\begin{array}{c}\mathrm{FRR}=\frac{\mathrm{FN}}{\mathrm{TP}+\mathrm{FN}}\\ \mathrm{FAR}=\frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}}\end{array}\right.$$ (7)

True positive (TP) represents the number of EEG signals of the EEG signal, which is identified as a self-portrait photograph. True negative (TN) represents the number of EEG signals for non-self-portrait photographs that are identified as non-self-portrait photographs of EEG. False positive (FP) is the number of EEG signals that are identified as self-time photos of EEG signals of non-self-portrait. False negative (FN) is the number of EEG signals of the self-portrait EEG signal identified as non-self-portrait photos. FRR is the false rejection rate, FAR is the false acceptance rate.

3.4. Technical front-end setup

The core modules of the control system are encapsulated in the middleware, which shields a large number of technical details so that non-professional developers can develop a variety of application software based on EEG. The system middleware architecture diagram is shown in Fig. 3, including the Android operating system, the device abstraction layer (Bluetooth interface, WiFi interface, ZigBee interface etc.), the data management layer (containing some basic data operations such as data storage, data query, data reception, and so on), and some entities such as database, model library, algorithms library, EEG library, and so on. In the core service layer, where all of the analysis and processing of EEG signals is completed, including the signal preprocessing, feature extraction, feature classification, and other aspects of the complex process. The upper layer is the application interface, these applications include identity EEG, BCI, detection EEG, polygraph EEG, and so on. Each layer is called a layer of content, also a call, end users only need to call the application interface to complete the development work, without understanding how the EEG signal acquisition, processing, and classification process is.

Fig. 3.

Middleware architecture based on Android platform

Prior to recording the user's EEG to authenticate him or her, several steps must be carried out. The entire system flows from the perspective of the user is illustrated in Fig. 4. After the EEG is collected through the signal preprocessing, feature extraction, and other steps, the system will produce a series of characteristic values. If the current operation type is the feature collection, these features will be saved to the feature database through several procedures, such as feature selection, feature classification, feature storage, and so on. Once the identity of the operating system begins to match the characteristics and features of library data. If the match is successful, the engine can be started, otherwise, the system displays ‘login failed’, then the system restarts the identification process. If failed five times, the system access will be denied.

Fig. 4.

System flow chart

3.5. System structure

The main function of the system is realised by the vehicle-mounted mobile device.

As shown in Fig. 5, the system is composed of driver, mobile terminal equipment, EEG acquisition, analysis module, engine control unit (ECU), and engine. Mobile terminal devices will show real-time EEG signals and stimulation photos. The EEG signals of drivers are evoked by the photos stimuli, EEG signals are collected and analysed after being sent to the mobile terminal equipment and ECU, ECU will identify the results into a signal to control the start of the engine.

Fig. 5.

System structure diagram

4. Results

As can be seen from Table 1, we use our proposed identification method to identify EEG data, the average success rate in the electrodes FP1 and FP2 fuzzy entropy is about 87.3%. Among the ten subjects, the highest classification accuracy was 92%, while the lowest was about 84%. The method has a slightly lower error acceptance rate (FAR) 5.5% and a lower FRR of 5.6%. In addition, the average FAR and FRR of this method is relatively low, which means that our proposed method has better stability. The remarkable improved success rate indicates that using fuzzy entropy in the electrodes FP1 and FP2 to design identify authentication access control is feasible.

Table 1.

Performance of the authentication access system

Subjects	Accuracy, %	FAR, %	FRR, %
1	85.4	8.0	6.4
2	92.1	3.1	4.6
3	84.0	6.8	9.0
4	83.9	7.1	8.9
5	90.0	6.6	3.3
6	84.6	3.8	5.7
7	86.5	5.7	7.6
8	88.2	4.5	5.2
9	87.3	6.2	3.1
10	90.7	3.6	2.4
mean (std)	87.3 (2.9)	5.5 (1.7)	5.6 (2.4)

It can be seen from Fig. 6 that there are two major data entities in this entity relationship (ER) diagram, the first entity is EEG features, the second entity is feature vector, between both of them, is 1: N relationship, a feature entity corresponding to N eigenvectors. EEG features data entity includes subject, experimental mode, algorithm, acquisition mode, and other related attributes. Feature vector data entity includes a vector matrix, feature value, number, and other attributes. According to this, ER diagram can infer the basic data framework EEG database.

Fig. 6.

Feature library ER diagram

Fig. 7 shows the user interface (UI) of driver's identity recognition programme, the upper part of UI is the stimulation photos to induce specific EEG signals, the lower part is the change of EEG signals. Also, the progress bar is the time of single recognition tips.

Fig. 7.

User interface of driver identification

5. Discussion and conclusion

At this point, one can confirm that the problem of identity authentication ACS based on the EEG signal is sensitive to several factors. such as frequencies, the location of electrodes, stimulation paradigm, and subject condition. Space and frequency may be two decisive factors of the problem. It seems clear that the spectral distribution of EEG signals contains subject-specific information that can be used in an authentication system. These biometric systems can extract several characteristic behaviours. We can expect that multiple electrodes, rhythms, and mission data can be used to increase the system's recognition performance.

The main disadvantage of EEG-based personal identification is that the acquisition equipment is inconvenient to use. Nevertheless, many researchers are now working to design a variety of new EEG recording devices for users.

At present, people are trying to use some EEG-based systems in our daily life. The problem of mobile acquisition of BCI system based on single-channel EEG signal has been solved. Using the mobile client as the foundation of the system platform, a simple EEG-based authentication system can be established by using a low-cost acquisition device. The monitoring of the impedance of the dry electrode and the integrated skin to electrode as well as the reliable high-quality wearable EEG monitoring system has also been developing continuously. With the rapid development of these technologies, the acquisition equipment becomes more and more practical and reasonable. We believe that in the near future there will be a cheap and practical EEG-based authentication system.