A method of VR-EEG scene cognitive rehabilitation training

Abstract

Virtual reality technology can intuitively provide patients of neuropsychological diseases with an almost real environment for cognitive rehabilitation training . In this paper, virtual reality technology is used to construct specific scenes that are universal and related to MCI patients to restore and train patients’ scene memory cognitive ability to help patients strengthen or gradually restore scene memory cognitive ability. The construction of virtual reality scenes with different contents such as life, environment, transportation and tourism, real-time detection is carried out in combination with EEG signals of patients in different scenes. The experimental results of the analysis of EEG signals of patients shows that memory rehabilitation training is strengthened by using specific stimulation scenes.

Introduction

At present, VR has been proposed to treat some neuropsychological diseases, such as anxiety and fear. Virtual reality technology can intuitively provide patients with an almost real environment, so that patients can be placed in the middle, and get better treatment effect. In this regard, we have carried out some research before [1–4].At the same time, VR can also be used in surgical training, providing a recyclable and ecologically efficient way for physicians to gain more training experience. VR can also be used for other medical purposes such as post-stroke intervention and treatment.

Carlos et al. Proposed a novel P300 Brain-Computer Interface (BCI) paradigm based on VR technology [5] , which uses soc7izial clues to guide the focus of attention and combines VR technology with the characteristics of P300 signals into a training tool, which can be used to train social attention disorders. Alexander optimizes the current BCI [6, 7]; Ioannis et al. Proposed a computer intervention based on virtual reality, which can be used for cognitive stimulation and can evaluate the condition of cognitive diseases such as MCI [8]. Dong Wen et al. Proposed a research on evaluation and rehabilitation of cognitive impairment at different stages based on virtual reality and EEG [9]. Yang and Tan et al. Pointed out that although some achievements have been made in VR-based EAR for MCI and AD patients, VR-based EAR for SCD patients has not yet begun. SCD patients are preclinical states of MCI and AD, and the evaluation of SCD patients is mainly limited to cerebrospinal fluid and EEG methods [10, 11]. Innes proposed that SCD rehabilitation is limited to meditation, music therapy [12], and mindfulness training [13] proposed by Smart et al. A lot of work has been done on the research of cognitive algorithms using computers [14–20]. These studies have achieved remarkable results. However, some subjects in these studies cannot complete the experiment. The sample size of these studies is quite small, and their follow-up period is very short. For EAR of SCD, MCI and AD patients, cognitive training should be combined with full immersion VR environment to attract patients and achieve the effect of rehabilitation training. In addition, Anguera and de Tommaso proposed that brain activity of patients can be measured during training [21] to objectively and quantitatively evaluate the progress of rehabilitation, thus encouraging patients to continue the rehabilitation process. EAR of PDCIP (Patients With Differential Cognitive Impairment Phases) can be performed using VR, while EEG signals of patients can be collected to evaluate the rehabilitation effect.

Through literature analysis and clinical investigation, although the current cognitive rehabilitation training has made considerable research progress, there are still obvious shortcomings in some aspects. On the one hand, due to differences in environment, educational background, living habits, and the degree of disease development, each MCI patient has differences in areas of cognitive impairment. The current computer-based cognitive rehabilitation training system mainly uses chess and card games, digital Games, etc., did not perform functional segmentation and targeted cognitive function rehabilitation training for specific cognitive domains. On the other hand, cognitive rehabilitation training using virtual reality technology has shown good rehabilitation training effects and clinical application prospects, but there is still no better solution to how to monitor the effect of cognitive rehabilitation training of patients on specific scenes. In addition, there is a lack of recording feedback on the patient’s rehabilitation training process, the training content cannot be adjusted in real time, and the personalized rehabilitation treatment cannot be performed according to the patient’s cognitive domain damage.

In response to the above problems, this article uses VR and EEG technology to carry out cognitive rehabilitation training for MCI patients. In this paper, virtual reality scenes with different contents such as life, environment, transportation and tourism are constructed, and EEG signals of patients in different scenes are combined for real-time detection. Through analysis of EEG signals of patients, memory rehabilitation training is strengthened by using specific stimulation scenes.

Method

The research method described in this article is mainly divided into four stages, including VR scene construction, EEG data acquisition, EEG signal preprocessing, and EEG feature analysis. The specific process is shown in Fig. 1.

Fig. 1.

Frame diagram of VR-EEG scene cognitive rehabilitation training

VR scene construction

In this paper, virtual reality scenes with five different contents such as life, environment, transportation and tourism are constructed. In order to ensure that the stimulation time of each video is the same and the total duration is not easy to be too long, combined with the patient’s physical condition, the duration of each full-motion video is strictly set to 15 seconds, the number of videos played at one time is 10, and some full-motion video contents are shown in Fig. 2.

Fig. 2.

Partial panoramic video screenshot

The flow of EEG acquisition method is as follows:

(1) First, wear EEG cap and VR head display for patients; (2) Collect EEG first, then play full-motion video, and record timestamp when full-motion video plays; (3) Quit automatically after full-motion video plays, and then stop collecting EEG; (4) According to the playback and stop time of full-motion video, intercept the corresponding EEG fragments to complete EEG acquisition.

EEG preprcessed

The EEG data collected by the EEG acquisition device is in. Cdt format, and the original collected EEG is shown in Fig. 3. Since the EEG acquisition time is strictly consistent with the full-motion video playback time during the experiment, the acquisition is processed by default. In order to remove the interference signals in EEG and make the analysis results more accurate, EEG preprocessing is required.

Fig. 3.

Preprocessed EEG

Firstly, the sampling rate is set to 500 Hz in order to compress the data amount. The sampling rate of 500 Hz has reached the experimental accuracy requirements. Selecting reference electrodes A1, A2; After that, filtering is carried out, which is divided into high-pass filtering and low-pass filtering. High-pass filtering means that high-frequency signals can pass normally, while low-frequency signals below the set threshold are blocked and weakened. Low-pass filtering refers to the normal passage of low-frequency signals, while high-frequency signals exceeding the set threshold are blocked and weakened. Generally speaking, if time-frequency analysis is to be done in the later period, the filtering range can be selected to be wider, 0.1–100. If only traditional ERP analysis is carried out, about 1–30 can be selected.

In addition, if filtering is performed at 0.1–100 Hz, depression filtering at 50 Hz can be performed to eliminate interference. As shown in Fig. 4, filtering is performed and 50 Hz sag filtering is removed.

Fig. 4.

Filter and remove the 50 Hz notch filter

The following steps are segmentation and baseline correction, either before or after the electrooculography is removed. In fact, it is best to remove the electrooculography, because continuous data is better when running ICA, but the amount of data is relatively large and the running speed is relatively slow. However, if there is noise and body movement in the experimental design, resulting in more artifacts and messy data, it can be segmented first, but it can be segmented as long as possible. The ICA step is carried out, which is mainly to remove artifacts. It takes about 300–400 steps to run ICA, and the time will be increased according to the segmentation situation and the processing speed of the computer. The 30 independent components obtained after ICA are shown in Fig.5.

Fig. 5.

30 independent components obtained after ICA

Then the vertical ophthalmogram is removed, and the effect after treatment is shown in Fig.6.

Fig. 6.

Remove EEG after vertical eye electricity

After the vertical ophthalmogram is removed, the horizontal ophthalmogram is then removed. The effect after treatment is shown in Fig.7. The preprocessed EEG data is saved in. Mat format for subsequent EEG feature extraction.

Fig. 7.

Removal of EEG after horizontal eye electricity

EEG analysis algorithm

Based on the characteristics of EEG signals, this paper analyzes whether there are significant changes in EEG signal patterns synchronously recorded under the stimulation of different VR scenes. The EEG signal features used include power spectral density (PSD) and differential entropy (DE). The original feature distribution and statistical variables such as mean and variance are used for feature significance analysis.

Research on PSD Algorithm Signals are usually expressed in the form of waves, such as sound waves, electromagnetic waves, etc. When the functional spectral density of the wave is multiplied by an appropriate coefficient, the power carried by the wave per unit frequency will be obtained. The PSD of the signal exists only if and only if the signal is a generalized stationary process. The spectral density of f (t) and the autocorrelation of f (t) form a Fourier transform pair, and Fourier transform is usually used for PSD estimation.

One of the results of Fourier analysis is Parseval theorem, which shows that the sum of squares of functions is equal to the sum of squares of its Fourier transformations.

$${\int }_{-\infty }^{+\infty }{|x(t)|}^{2}dt={\int }_{-\infty }^{+\infty }{|X(f)|}^{2}df$$ 1

where X(f)=F.T.x(t) is the continuous Fourier transform of x (t) and f is the frequency component of x.

EEG characteristics are mainly the amplitude, frequency, variance, mean and other statistical characteristics of electrical signals. These methods generally have poor effect on EEG signals with low signal-to-noise ratio, so time domain features cannot be used as the final classification features alone. The basic method of frequency domain feature analysis is to convert sequential EEG signals into frequency domain through Fourier transform. Among them, power spectral density is a commonly used frequency domain feature.

Power spectral density reflects the relationship between power and frequency of EEG signals and can be used to observe the changes of signals in various frequency bands. It is known that the autocorrelation function of the discrete random signal x (n) is r (k). According to the discrete Fourier transform, the power density of the signal can be calculated by the following formula:

$$P(w)=\sum _{k=-\infty }^{+\infty }r(k)e^{-j\omega k}$$ 2

here $r(k)=E[x(n)x^{\ast }(n+k)]$ , E denotes the mathematical expectation of the signal, * is a complex conjugate.

Research on DE Algorithm: Differential entropy is a new characteristic of EEG signals. It is defined as follows:

$$h(X)=-{\int }_{-\infty }^{+\infty }f(x)lgf(x)dx$$ 3

where the timing signal is represented by X, f(x) is the probability density function of X. Because in the specific frequency band after band-pass filtering, the signal basically meets Gaussian probability distribution.

To calculate the differential entropy, only the variance of the sequence needs to be known. In order to simplify the calculation of variance, we can use another method to calculate the differential entropy. For a discrete signal sequence x(n), its Short-time Fourier Transform (STFT) is represented as:

$$STFTx[n](m,{\omega }_n)=X(m,{\omega }_n)=\sum _{n=1}^{N}x[n]\omega [n-m]e^{(-j{\omega }_{k}n)}$$ 4

where $\omega [n]$ is the window function, $X(m,{\omega }_k)$ is the Fourier transform of $X(m,{\omega }_k)$, ${\omega }_k$ is the angular frequency, k=0,1...N − 1 ,N is the total number of sampling points.

Experimental results and discussion

In this paper, the EEG signals collected in a single experiment are divided into a single sample by a sliding window with a window length of 2s and an overlapping length of 1s. 58 samples under a single stimulus segment were obtained (when the number of samples under a single stimulus segment exceeded 58, only the first 58 samples were taken to ensure that the number of samples corresponding to each stimulus segment was consistent), and finally 580 samples under a single experiment were obtained. After obtaining the sample data, this paper further uses PSD estimation and DE to extract features, and compares the original feature distribution, and analyzes the feature significance through statistical variables such as the mean and variance of features.

Characteristic analysis of PSD

The first subject: The change trend of mean and variance of PSD characteristics under the three experiments is basically the same. Under three experiments, the experimental results are shown in the left column of Fig. 8. The average value of the corresponding PSD features reaches the maximum value. According to the classification strategy mentioned above, the two videos belong to the same category of stimulus materials. That is to say, subject 1 has a larger physiological response to such video stimulation materials, while the response to fragments 2, 3, 4 and 6 is weaker. In the third experiment, the mean and variance of PSD features corresponding to segment 5 are not as obvious as those in the first and second experiments, and their responses to videos 1, 7 and 10 are relatively significant.

Fig. 8.

PSD feature analysis chart.The left column is the results of subject 1, and the right column is the result of subject 2

The second subject: Under three experiments, the experimental results are shown in the right column of Fig.8. The mean and variance distribution of PSD characteristics of Subject 2. Based on the data of the three experiments, it can be preliminarily determined that there may be problems in the original EEG signals of the first experiment to the stimulus segment 6 and the second experiment to the stimulus segment 10. As can be seen from the figure, the physiological response of subject 2 to fragments 1, 2, 3, 4 and 8 is not very significant, but the response to fragments 5, 7 and 9 is relatively significant. This result is basically the same as that of subject 1, but there are certain differences, that is, different subjects have different sensitivity to stimulation materials. Therefore, we need to carry out experiments for each subject to find stimulation materials that can cause significant changes in physiological signals of each subject.

Characteristic analysis of DE

The first subject: Similar to PSD characteristics, under three experiments, the experimental results are shown in the left column of Fig.9. The mean value of DE features corresponding to stimulus fragments 10, 5 and 10 respectively reached the maximum, while the response to fragments 2, 3, 4 and 6 was weak.

Fig. 9.

DE feature analysis chart.The left column is the results of subject 1, and the right column is the result of subject 2

The second subject: the same PSD characteristics, the experimental results are shown in the right column of Fig.9. The subjects were not very sensitive to fragments 1, 2, 3, 4 and 8, and the responses to fragments 5, 7 and 9 were significant.

DE characteristic distribution of the third experiment

The experimental results of subject 1 are shown in Fig.10. The standard deviation of the first and second experiments of the subject one under stimulus fragments 5 and 10 is large, which is consistent with the previous conclusion, but the standard deviation of the third experiment does not have this phenomenon. In addition, there is no special rule for the mean distribution of the three experiments.

Fig. 10.

The mean and variance of direct feature analysis of subject 1

The experimental results of subject 2 are shown in Fig.11. From the standard deviation distribution, it can be found that subjects 2 responded well to fragments 5, 6 and 7 in the first experiment, and responded well to fragments 5, 7, 10 and 5, 7 and 9 in the second and third experiments respectively. The reaction to fragments 2 and 4 was poor under three experiments.

Fig. 11.

The mean and variance of direct feature analysis of subject 2

Conclusion

In this paper, the key algorithms of EEG analysis in VR-EEG scene cognitive training module are studied and implemented. The experimental results show that PSD and DE feature analysis are obviously better than direct feature analysis. At the same time, each type of scene in VR full-motion video has different EEG stimulation to human beings. The research results of this paper provide a new idea and method for scene-based memory rehabilitation training.The combination of VR and EEG technology described in this article provides cognitive rehabilitation training for MCI patients, which can effectively prevent the development of mild cognitive impairment to deeper cognitive impairment, as well as prevent various types of senile dementia. This provides new and more effective technical means for non-drug treatment, which can reduce the burden on medical staff and reduce the burden on families and society.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.