Highlights
-
•
The unpredictable nature of seizures is a factor affecting the patients quality of life.
-
•
Recognition of interictal from the preictal state is the basis for early seizure detection.
-
•
Machine Learning-based Detection algorithms pick up patterns of each individual.
-
•
System analyzes 1500 statistical parameters that measure different EEG signal characteristics.
-
•
Each trained model was tested to maximize sensitivity with minimal false detections.
Keywords: Seizure early detection, EEG analysis, Epilepsy, Epileptic seizures, Machine learning, Artificial intelligence
Abstract
Around one-third of epilepsy patients develop drug-resistant seizures; early detection of seizures could help improve safety, reduce patient anxiety, increase independence, and enable acute treatment. In recent years, the use of artificial intelligence techniques and machine learning algorithms in different diseases, including epilepsy, has increased significantly. The main objective of this study is to determine whether the mjn-SERAS artificial intelligence algorithm developed by MJN Neuroserveis, can detect seizures early using patient-specific data to create a personalized mathematical model based on EEG training, defined as the programmed recognition of oncoming seizures before they are primarily initiated, usually within a period of a few minutes, in patients diagnosed of epilepsy. Retrospective, cross-sectional, observational, multicenter study to determine the sensitivity and specificity of the artificial intelligence algorithm. We searched the database of the Epilepsy Units of three Spanish medical centers and selected 50 patients evaluated between January 2017 and February 2021, diagnosed with refractory focal epilepsy who underwent video-EEG monitoring recordings between 3 and 5 days, a minimum of 3 seizures per patient, lasting more than 5 s and the interval between each seizure was greater than 1 h. Exclusion criteria included age <18 years, intracranial EEG monitoring, and severe psychiatric, neurological, or systemic disorders. The algorithm identified pre-ictal and interictal patterns from EEG data using our learning algorithm and was compared to a senior epileptologist’s evaluation as a gold standard. Individual mathematical models of each patient were trained using this feature dataset. A total of 1963 h of 49 video-EEG recordings were reviewed, with an average of 39.26 h per patient. The video-EEG monitoring recorded 309 seizures as subsequently analyzed by the epileptologists. The mjn-SERAS algorithm was trained on 119 seizures and split testing was performed on 188 seizures. The statistical analysis includes the data from each model and reports 10 false negatives (no detection of episodes recorded by video-EEG) and 22 false positives (alert detected without clinical correlation or abnormal EEG signal within 30 min). Specifically, the automated mjn-SERAS AI algorithm achieved a sensitivity of 94.7% (95 %; CI 94.67–94.73), and an F-Score representing specificity of 92.2% (95 %; CI 92.17–92.23) compared to the reference performance represented by a mean (harmonic mean or average) and a positive predictive value of 91%, with a false positive rate of 0.55 per 24 h in the patient-independent model.
This patient-specific AI algorithm for early seizure detection shows promising results in terms of sensitivity and false positive rate. Although the algorithm requires high computational requirements on specialized servers cloud for training and computing, its computational load in real-time is low, allowing its implementation on embedded devices for online seizure detection.
Introduction
The International League Against Epilepsy (ILAE) describes a seizure as the transient occurrence of signs and/or symptoms due to excessive and/or synchronous abnormal neuronal activity in the brain; epilepsy is further defined as a disorder of the brain characterized by an enduring predisposition to seizures and the neurobiological, cognitive, psychological and social consequences of this condition; a diagnosis of epilepsy requires the occurrence of at least one epileptic seizure in a patient whose brain, for whatever reason, demonstrates a pathological and enduring tendency to have recurrent seizures [1]. The World Health Organization estimated in 2019 that around 50 million people worldwide were diagnosed with epilepsy, it develops at any age with a higher incidence in the very young and the elderly, and carries an increased risk of premature death of up to three times in patients compared to the general population [2], studies suggest that each year there are about 1.16 cases of Sudden Unexpected Death in Epilepsy (SUDEP) for every 1000 people with epilepsy, although estimates vary.
Epilepsy is considered a multifactorial disease with a wide spectrum of characteristics and different predisposing factors for its development, the consequences include not only the impact on the patient's health, but also all aspects (cultural, interpersonal, and social) of a person's life [3]. About 70% of patients with epilepsy could live seizure-free despite accurate diagnosis and treatment. Therefore, the main purpose of epilepsy treatment is to diminish seizures to a minimum level that allows patients to achieve the best possible quality of life [4]. Although most patients remain seizure-free with anti-seizure medication (ASM), more than 30% continue to have seizures despite treatment [5], [6], this situation is known as drug-resistant epilepsy or refractory epilepsy and is associated with a greater socioeconomic and psychosocial burden [7], [8]. The random and unpredictable nature of seizures is one of the principal factors affecting the patient’s quality of life [9], [10], along with associated comorbidities, neuropsychiatric disorders, cognitive deficits, and side effects of ASM [2]. Therefore, there is an urgent need to develop reliable tools for accurate seizure prediction and early detection, which is defined as the programmed recognition of oncoming seizures, before they initiate primarily, usually within a period of a few minutes [11]. In this regard, it is known that EEG is a valuable non-invasive method of diagnosis in epilepsy that can display abnormalities in brain activity, disrupted EEG features enable classification and localization, and rise the likelihood of seizure detection, therefore, it could potentially assess seizure recurrence risk [12]. However, the diagnosis of epilepsy is also centered on clinical information and the EEG should be considered as support but not an exclusive diagnostic test [13],
On the other hand, prolonged video-EEG monitoring, by analyzing both EEG and ictal semiology of seizures, often yields the confirmatory diagnosis [14], [15]. Video-EEG monitoring remains the gold standard of epilepsy for the detection and diagnostic evaluation of seizures in clinical practice, however, this method has limited sensitivity, requires a long time for analysis, and is subject to different interobserver and intraobserver biases [16]. Therefore, there is a need to develop robust methods that allow for greater sensitivity and prompt detection of seizures.
New computer-based technology has improved the quality of EEG assessments [17] and opened a road for EEG machine learning-based prediction. These algorithms can be trained to learn patterns from a big database by processing it throughout a multi-layer hierarchical architecture, allowing seizure detection and might provide warnings to patients, allowing for acute treatment at the time of seizure onset [18], [19]. The main aim of our study was to determine if the mjn-SERAS artificial intelligence algorithm (developed by MJN Neuroserveis) shows an accurate detection of seizures in previously diagnosed drug-resistant epilepsy patients using a video-EEG training set and whether it is appropriate for real-time use. For this purpose, we create a system that integrates artificial intelligence (AI) correlations, frequency analysis with spectral segments, and other EEG features. Ultimately, this could allow us to develop an effective medical device to help patients to be warned of an impending seizure, consequently, decreasing uncertainty about seizure onset, providing acute treatment, and improving the patient’s quality of life.
Methods
Human subjects and ethical issues
We conducted a retrospective cross-sectional, observational, multicenter study to determine the sensitivity and specificity of the artificial intelligence algorithm. We performed a search in the database of the Epilepsy Units of three Spanish medical centers and selected 50 patients evaluated between January 2017 and February 2021 diagnosed with refractory focal epilepsy with video-EEG monitoring of 3 to 5 days duration, recording a minimum of 3 seizures per patient that lasted more than 5 s and the interval between each seizure was greater than 1 h. The exclusion criteria included age <18 years, intracranial EEG monitoring, and severe psychiatric, neurological, or systemic disorders (due to their possible interference with the result interpretation) (Fig. 1).Approval of the used protocols was obtained through the ethics committees of the Regional Ethical Committee of Madrid.
Fig. 1.
After searching for the 50 patients in the databases of the 3 centers, each patient was analyzed.
Video-EEG acquisition and electrode positioning
Video-EEG data were recorded using the Natus Seizure Advisory System obtained with full scalp montage. All EEG data were 24 channel recordings using the 10–20 system. Portable EEG machines were used, with gold-plated disc electrodes and collodion adhesive. EKG and video were also recorded. All electrode impedances were maintained at <10 kilo-ohms at the start of the recording. The EEG was acquired referentially at a 500-Hz sampling rate, and visualization and inspection for neurologist seizure detection were made with high-pass and low-pass filters set at 1 Hz and 35 Hz or 70 Hz, respectively. Sensitivity was set between 5 and 10 µV/mm. A 50-Hz notch filter was used in most of the recordings. All patients received photic stimulation but not hyperventilation during the first 30 min of the recording. Epilepsy monitoring unit (EMU) nurses were trained to press the alarm button and write notes for clinical seizures or other significant events. The technicians kept the subjects alert to prevent drowsiness. The selected channels have been using the approximation to the mjn-SERAS device, so we have used the F7-T3 and T3-T5 or the F8-T4 and T4-T6, depending on the laterally of focus.
Video-EEG analysis (EEG Pre-processing)
Before analysis, all EEG data were reviewed the day following the recording by epilepsy-specialized neurologists and neurophysiologists with experience as electroencephalographers (JRU, GD) using a Natus viewer and standard visual inspection, 10 s per page. Clinical features included in this study were: the number of seizures (determining the onset of seizure according to medical criteria), duration, identification of lateralization of the epileptic focus, and drug resistance. Specifically, the EEG recordings showed a minimum of 3 seizures per patient, with each seizure lasting at least 5 s and an inter-seizure interval of more than 1 h.
Algorithm
The seizure early detection method used during the study is based on artificial intelligence techniques, specifically supervised machine learning. The artificial intelligence algorithm developed by MJN Neuroserveis, derived from the comprehensive analysis of the data collected in this trial, analyzes the signals recorded from each patient to create a customized mathematical model that captures the complexity of individual brain dynamics. The information was collected using the EEG technique and processed in a sequential procedure, starting with the raw data and ending with an alarm in case of seizure detection. We explain the process below, the algorithm has three main operational blocks:
Pre-processing block: artifact removal
Raw data obtained from video-EEG recordings contain noise and electrical interferences, which distort the recorded signals. Artifacts in EEG recordings are disturbances in brain signals, which are outlier measurements generated within the brain. External artifacts are caused frequently by aberrant technology such as electromagnetic interference and disconnection of the electrode box (for instance, displacement in the skin-electrode contact). Internal artifacts can occur due to changes in the potential between electrodes because of eye movement or muscular activity [20]. We implemented a set of filters that allowed us to remove the electrical interference at 50 Hz and the baseline drift at 0.5 Hz detecting only the frequencies that are from brain activity. Furthermore, EEG experts performed visual detection of these artifacts, and these were automatically detected and rejected. Several methods have been described for the automation of this process; we use the amplitude thresholding method, which is calculated as three to five times the standard deviation of the signal. Additionally, upon completion of the pre-processing block, the remaining signal contains brain waves, namely delta, theta, alpha, beta, and gamma.
Data formatting block: EEG feature extraction
We used powerful data mining techniques to extract information from individual artifact-free time series of the 24-hour video-EEG recordings, the signals were transformed into descriptive statistical values and named features, and the data were segmented into fixed-sized windows, which have a certain degree of overlap between contiguous segments. Each of the windows was labeled using a binary method, interictal segments were associated with a zero value and preictal segments were labeled with a unitary value. The windows were 60 s with a 50% overlap, the splits were not done at the sample level, and they were designed as ictal-interictal segment packets, which ensured that no training information was sent to the test due to overlap. Every window needed to be transformed in a single statistical value to be able to enter a mathematical model but with this model, we can find class imbalance, so we use undersampling and weighted scores appropriate in each case. In total, over 150 features have been implemented in the algorithm for different subbands. The implemented features can be divided into the following main groups:
-
●
Energies: They measure the degree of electrical activity for a given range of frequencies. Such values are scaled to different magnitudes to exploit the non-linear properties of the energies.
-
●
Complexity measurements: These values describe the variability of the signal, finding trends and calculating the degree of predictability of such trends.
-
●
Centrality measurements: Describe the values distribution of the signal.
-
●
Connectivity measurements: Calculate the degree of synchronization between the two channels, analyzing the similarity of the patterns.
Once this procedure was completed, the statistical descriptors of each series of features (mean, quartiles, variability) were extracted, leaving only the tendency values of all the features for each patient. In total, approximately 5000 statistical EEG parameters were extracted from video-EEG recording.
Training block
It is known that 4 distinctive states of brain activity exist in epileptic patients: preictal (immediately prior to seizure), ictal (during a seizure), postictal (immediately following a seizure), and interictal (between seizures). The ictal state is easily identified in EEG recording, thus, the important fact in seizure prediction studies is to distinguish between interictal and preictal state [21]. For that reason, it is important to have reliable features and automated systems that correlate and allow for discrimination between preictal and interictal states, respectively [22]. Additionally, feature extraction techniques usually generate a lot of features that could be redundant and difficult to apply due to an excess of computational requests [23]. Hence, we labeled each EEG data segment as interictal or preictal one and we categorized these parameters of features to identify which of them better represents the differences between interictal or preictal segments. Only relevant information was collected and redundancies or information that could affect the yield of the model were discarded. This identification of preictal and interictal patterns from EEG data was performed using our learning algorithm. Finally, mathematical models of each patient were trained using these features’ data sets. Each trained model was evaluated to maximize the sensitivity of seizure detection with minimal false detections.
The data is split into train and test groups. The first one was used to train the algorithms, while the second one was used to assess the performance of the model against unknown data. The features that showed the greatest difference between interictal and preictal segments were selected based on maximizing the separation between populations and minimizing the correlation between them. This analysis and feature selection was performed on the training data only, and the features were “fixed” for the rest of the algorithm. This way the mathematical models were trained with the top-ranked features. This operation was performed only using the training dataset to avoid data leaking, which may lead to overly optimistic performance.
The performance of the algorithm was assessed using the test group and improved through fine-tuning of each of the hyperparameters of the model. This optimization technique allowed the regulation of internal parameters of the model until finding the subset that offered the greatest performance of the test split. We have applied two different methods depending on the number of seizures, less than or equal to 5 seizures, and we have applied a leave-one-out method so that 100% of the seizures have been trained and tested. For patients with more than 5 seizures, we have applied a split ratio of 70–30 to 50–50, depending on the size of the raw cost-effective seizure data, excluding artifacts, device disconnections, and other noise.
The system analyses 1500 statistical parameters that measure different EEG features without visualizing or interpreting the different graphical elements and it can generate an alert of this event (Table 1). As mentioned previously, EEG data were extracted, and input data were obtained from the extraction machine learning algorithm.
Table 1.
The system analyses 1500 statistical parameters that measure different EEG features without visualizing or interpreting the different graphical elements and it can generate an alert of this event, here are some of these.
| Mean Delta band power |
| Normalized FFT Kurtosis |
| Covariance |
| Spectral Centroid |
| Zero Crossing |
| Cross-Correlation |
| Escort Tsallis Entropy |
| Mutual information |
| Relative Entropy |
| Tsallis Entropy |
| Shannon Entropy |
| Geometrical Mean |
| Harmonical Mean |
| Kolmogorov Complexity |
| Kurtosis |
| Skewness |
| Spectral Flatness |
| Epoch Based Entropy |
| Higuchi Fractal Dimension |
| Hurst Fractal Dimension |
After the training and subsequent analysis by the mjn-SERAS artificial intelligence algorithm, the number of seizures detected by the algorithm was calculated and the corresponding alerts were recorded. Subsequently, the calculation of false negatives (seizures not detected by the algorithm) and false positives (alarms that do not correspond to clinical seizures or registered in the video-EEG recording) was performed (Table 2).
Table 2.
Results per patient.
| Register VideoEEG |
Total Test |
False Negative |
False Positive |
Test Split |
Prediction Time (minutes) |
||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Number | Sex | Focus | Hours | Seizures | Hours | Seizures | Alarms | Pre-alarm <30 min | TPR Sensitivity |
F-Score | TNR Specificity |
PPV | False Alarm Rate | Average | Min. | Max. | |
| 011900001 | F | R | 91 | 8 | 45,5 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 3 | 6 |
| 011900002 | M | L | 108 | 9 | 60,0 | 5 | 0 | 1 | 1 | 100 % | 91 % | 92 % | 83 % | 0,4 | 9 | 2 | 15 |
| 011900003 | M | L | 83 | 5 | 83,0 | 5 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 2 | 12 |
| 011900004 | F | R | 66 | 7 | 18,9 | 2 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 14 | 12 | 15 |
| 011900005 | M | R | 91 | 17 | 48,2 | 9 | 0 | 3 | 1 | 100 % | 86 % | 80 % | 75 % | 1,5 | 6 | 1 | 15 |
| 011900006 | F | R | 85 | 9 | 28,3 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 11 | 4 | 15 |
| 011900007 | F | L | 22 | 7 | 12,6 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 6 | 1 | 15 |
| 011900008 | F | L | 45 | 16 | 22,5 | 8 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 2 | 1 | 4 |
| 011900009 | M | R | 89 | 8 | 44,5 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 3 | 7 |
| 011900010 | F | L | 65 | 8 | 32,5 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 12 | 8 | 15 |
| 011900011 | F | L | 94 | 5 | 94,0 | 5 | 0 | 1 | 0 | 100 % | 91 % | 83 % | 83 % | 0,3 | 14 | 12 | 15 |
| 011900012 | M | R | 19 | 6 | 9,5 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 6 | 4 | 10 |
| 011900013 | F | L | 59 | 3 | 59,0 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 3 | 7 |
| 011900014 | M | R | 88 | 4 | 88,0 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 11 | 3 | 15 |
| 011900015 | M | L | 9 | 6 | 4,5 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 3 | 12 |
| 011900016 | F | L | 20 | 4 | 20,0 | 4 | 1 | 1 | 2 | 75 % | 75 % | 79 % | 75 % | 1,2 | 13 | 9 | 15 |
| 011900017 | M | R | 8 | 6 | 4,0 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 1 | 9 |
| 011900018 | M | R | 19 | 4 | 19,0 | 4 | 1 | 1 | 1 | 75 % | 75 % | 86 % | 75 % | 1,3 | 11 | 6 | 15 |
| 011900019 | M | L | 21 | 6 | 10,5 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 2 | 1 | 4 |
| 011900020 | M | R | 51 | 6 | 25,5 | 3 | 0 | 0 | 1 | 100 % | 100 % | 99 % | 100 % | 0,0 | 9 | 4 | 15 |
| 011900021 | M | R | 26 | 4 | 26,0 | 4 | 0 | 0 | 3 | 100 % | 100 % | 80 % | 100 % | 0,0 | 15 | 14 | 15 |
| 011900022 | M | L | 22 | 4 | 22,0 | 4 | 1 | 2 | 0 | 75 % | 67 % | 99 % | 60 % | 2,2 | 5 | 1 | 10 |
| 011900023 | M | L | 19 | 3 | 19,0 | 3 | 1 | 0 | 0 | 67 % | 80 % | 100 % | 100 % | 0,0 | 1 | 1 | 1 |
| 011900024 | M | L | 20 | 3 | 20,0 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 2 | 1 | 4 |
| 011900025 | M | L | 20 | 4 | 20,0 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 2 | 7 |
| 011900026 | M | L | 18 | 3 | 18,0 | 3 | 0 | 2 | 0 | 100 % | 75 % | 94 % | 60 % | 2,7 | 12 | 3 | 15 |
| 011900027 | M | L | 20 | 5 | 20,0 | 5 | 0 | 1 | 2 | 100 % | 91 % | 93 % | 83 % | 1,2 | 12 | 6 | 15 |
| 011900028 | M | R | 47 | 5 | 47,0 | 5 | 1 | 1 | 2 | 80 % | 80 % | 95 % | 80 % | 0,5 | 9 | 3 | 12 |
| 011900029 | F | R | 11 | 6 | 5,5 | 3 | 0 | 0 | 1 | 100 % | 100 % | 98 % | 100 % | 0,0 | 11 | 7 | 15 |
| 011900030 | F | R | 58 | 4 | 58,0 | 4 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 10 | 3 | 15 |
| 012000031 | F | L | 24 | 5 | 2,3 | 3 | 0 | 0 | 3 | 100 % | 100 % | 100 % | 100 % | 0,0 | 10 | 15 | 15 |
| 012000032 | M | R | 40 | 8 | 11,0 | 4 | 0 | 1 | 1 | 100 % | 89 % | 99 % | 80 % | 2,2 | 15 | 15 | 15 |
| 012000033 | M | L | 18 | 5 | 9,1 | 3 | 0 | 1 | 1 | 100 % | 86 % | 95 % | 75 % | 2,6 | 10 | 1 | 15 |
| 012000034 | F | R | 15 | 3 | 15,0 | 3 | 0 | 0 | 1 | 100 % | 100 % | 100 % | 100 % | 0,0 | 9 | 6 | 15 |
| 012000035 | M | L | 13 | 4 | 4,0 | 2 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 10 | 5 | 15 |
| 012000036 | M | L | 16 | 7 | 6,9 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 1 | 7 |
| 012000037 | F | L | 16 | 7 | 8,0 | 4 | 0 | 0 | 2 | 100 % | 100 % | 100 % | 100 % | 0,0 | 14 | 11 | 15 |
| 012000038 | M | R | 21 | 3 | 21,0 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 10 | 2 | 9 |
| 012000039 | F | L | 29 | 5 | 6,0 | 2 | 0 | 0 | 2 | 100 % | 100 % | 100 % | 100 % | 0,0 | 15 | 15 | 15 |
| 012000040 | F | L | 8 | 3 | 8,0 | 3 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 5 | 2 | 7 |
| 012000041 | F | R | 9 | 15 | 5,5 | 8 | 0 | 0 | 0 | 100 % | 100 % | 100 % | 100 % | 0,0 | 15 | 15 | 15 |
| 012000042 | F | R | 38 | 14 | 8,6 | 7 | 1 | 1 | 0 | 86 % | 86 % | 98 % | 86 % | 2,8 | 12 | 2 | 15 |
| 012000043 | M | R | 61 | 4 | 20,5 | 2 | 0 | 3 | 0 | 100 % | 57 % | 91 % | 40 % | 3,5 | 8 | 2 | 15 |
| 012000044 | F | L | 63 | 6 | 20,5 | 3 | 0 | 2 | 0 | 100 % | 75 % | 95 % | 60 % | 2,3 | 7 | 2 | 15 |
| 012000045 | M | R | 15 | 2 | – | – | – | – | – | – | – | – | – | – | – | – | – |
| 012000046 | M | R | 41 | 5 | 22,0 | 3 | 1 | 0 | 2 | 67 % | 80 % | 100 % | 100 % | 0,0 | 15 | 15 | 15 |
| 012000047 | F | L | 50 | 5 | 20,0 | 3 | 1 | 0 | 0 | 67 % | 80 % | 100 % | 100 % | 0,0 | 12 | 10 | 15 |
| 012000048 | F | L | 46 | 6 | 17,4 | 3 | 0 | 0 | 1 | 100 % | 100 % | 100 % | 100 % | 0,0 | 12 | 7 | 15 |
| 012000049 | M | R | 12 | 7 | 4,0 | 4 | 0 | 0 | 2 | 100 % | 100 % | 100 % | 100 % | 0,0 | 11 | 5 | 15 |
| 012000050 | M | R | 34 | 10 | 10,3 | 4 | 2 | 1 | 2 | 50 % | 57 % | 97 % | 67 % | 2,3 | 8 | 2 | 15 |
| TOTAL | 1963 | 309 | 1205,5 | 188 | 10 | 22 | 31 | 94,7 % | 92,2 % | 97 % | 91,5 % | 0,55 | 9,0 | ||||
Abbreviations: TPR, true positive rate; TNR, true negative rate; PPV, predictive positive value.
Statistical analysis
All statistical analyses were performed using SPSS version 23 (IBM SPSS Statistics). The study data were analyzed using descriptive statistics. The chi-square test and/or the Yates and Fisher correction were used to compare proportions and study relationships. Comparisons between two quantitative variables were made using the t-Student or Mann-Whitney U test for normal distribution or non-normal distribution data, respectively. The statistical analysis was conducted using the scores of the results for each patient; no statistical analysis of the features of the training set was done. We used techniques to select those features that best separated the classes and, at the same time, were not correlated between them. We mainly used network component analysis (NCA) and relief-based algorithms (RBA)”.
The dataset was determined to be sufficient for a proof-of-concept study due to a small size as the size limited the machine learning techniques that were able to be applied to this study.
Results
Fifty patients were included, and a total of 1963 h of 49 video-EEG monitoring were recorded, with a mean per patient of 39.26 h. One patient was excluded due to insufficient seizures. Video-EEG monitoring registered 309 seizures according to the epileptologists' analysis of each patient's data. The mjn-SERAS algorithm used 100% of the seizures collected by video-EEG monitoring, where 119 seizures were used only to train the algorithms and 188 seizures were used for the split test. The statistical analysis includes the data from each model and reports 10 false negatives (no detection of episodes recorded by video-EEG) and 22 false positives (alert detected without clinical correlation or abnormal EEG signal within 30 min). Specifically, the automated mjn-SERAS algorithm achieved 94.7% (95 %; CI 94.67–94.73) sensitivity and 92.2% (95 %; CI 92.17–92.23) specificity compared to the reference performance represented by a mean (harmonic mean or average) F-Score and a positive predictive value of 91% with 0.55 false positive (FP/24-hour) detections in the patient-independent model. Table 1 shows a summary of the findings for each patient. Cases with high false alarm rates were analyzed, most of which correlated with inter-critical activity and some with electrical seizures that did not correspond to selected clinical seizures in the video-EEG recording, thus having no impact on the patient's quality of life. In a theoretical sub-analysis without this subgroup of 9 patients, the PPV increased by 5.5% to 97%, and reduced the FAR from 0.55 to 0.12 false alarms per 24 h.
The early detection system based on the Artificial Intelligence algorithm developed by MJN Neuroserveis enabled learning of detection models targeting recognizing patterns in EEG activity that were in the precise time window of the seizure. Fig. 2 shows three cases of seizure early detection, one of them lasting up to 13 min. Next, it used advanced machine learning along with the design and selection of accurate features from EEG signals to elucidate probability and class assignment of seizures.
Fig. 2.
Selected spectrograms of three study subjects. The dotted line shows the time of alarm identified by AI and the solid line the onset of the seizure with the current standard.
Discussion
We found that for a validation set (test split) of 188 pre-recorded epileptic seizures in 49 patients, our method was sensitive and specific in detecting EEG segments to predict preictal and interictal patterns. Epilepsy seizure AI detection has become one of the most appealing fields that applies original solutions, the seizure detection systems based on video-EEG recording are characterized by the generation of high-dimensional spatiotemporal features, which increasingly need to be processed using state-of-the-art methods such as artificial intelligence and machine learning to better forecast ongoing seizures. In 2019 Troung et al published their analysis of a group of 56 patients, with 6.8 seizures per patient with early 5-minute seizure detection, including patients with intracranial electrodes. Haider et al. in 2017 included 47 patients to report a sensitivity of 87.8% with 0.142 false positives per hour. They reported up to 8 min warning without inclusion of implanted patients.
Using patient-independent algorithms, it is more difficult to build detection models due to EEG variability between individuals [25]. The results for seizure detection are better and more accurate than those previously reported because of detecting an upcoming seizure is easier than predicting a seizure in advance [23], [26]. However, compared to non-patient-specific classifiers [27], the accuracy we achieved with our early detection model was higher. Although we used a small sample cohort, our results show high quality results with high accuracy and sensitivity of 95%, and a reduced false positive trend of 0.023 false alarms per hour. Our model showed high performance, both in terms of sensitivity and specificity.
Our study sheds light on the use of machine learning algorithms to detect drug-resistant seizures from patient-specific EEG recordings. We have used a patient-specific early seizure detection algorithm, which is similar to most feature extraction algorithms, as each patient has different EEG signals [24].
Each patient has unique brain activity and seizures, hence the need for patient-specific models, as generic a global model is not specific. This ML model has not been developed as a generic model because with all the patient data there are 50 different personalized models. The software architecture of the algorithm is unique, the same for all patients, but we can select various parameters such as features, weights, thresholds, and ML hyperparameters, so it is a customized model, a patient-specific algorithm for early seizure detection. Although to train the algorithm it needs to be taught how to record a seizure, in the case of an out-of-hospital algorithm, the digital patient record would be used.
However, these results should not be considered the best possible, as training of the algorithm leads to better results and shorter warning times. In addition, our study had some other limitations, the main one being the small number of previous studies focusing on the performance of this mathematical algorithm as a predictive method for seizure detection. Although we improved feature selection and subsequent analysis to improve seizure classification with our system, another problem detected was the large inter-patient variability. Therefore, we opted for individualization to improve sensitivity and specificity. Because of this, there is still a need to improve the overall detection models by choosing more accurate features, the most advanced and appropriate classifier, and selecting a high-performance algorithm for non-specific patients [28]. Cross-validation is difficult in a customized model without regrouping all seizures. As we have few seizures in several patients, we have implemented a leave-one-out method with a version that is a k-fold with k = number of seizures. However, we were unable to do a k = 10 because many patients did not have 10 seizures in the record to do the 10 folds. A single receiver operating characteristic (ROC) curve for all models, useful in this type of analysis because it represents the optimal compromise between sensitivity and specificity, is not feasible as it is a customized mathematical model. Thus, each individual model was optimized. Fig. 3 shows an example of a ROC curve for two of the patients studied.
Fig. 3.
Mjn-SERAS algorithm individualized roc curve for seizure detection in two patients. The red point represents the optimal compromise between sensitivity vs specificity. roc, receiver operating characteristic; tnr, true negative rate.
Therefore, seizure prediction remains a challenging task in epilepsy research and requires further investigation. In this regard, a recent study reveals that clinical seizure prediction is feasible in a wider range of patients than previously thought, thanks to the crowdsourcing of more than 10,000 algorithms worldwide [18].
In addition, algorithm training and new sample calculation are computationally demanding on specialized cloud servers but have a light real-time computational load allowing deployment on embedded devices for early online seizure detection with sufficient accuracy to minimize disruption to the patient's life and with minimal false alarms [26].
Conclusions
In the present study, we demonstrate an artificial intelligence (AI)-based system for early seizure detection using the mjn-SERAS algorithm, which documents an accurate detection of pre-ictal and interictal EEG segments in drug-resistant epilepsy patients. Furthermore, the successful validation of our study in a cohort of 49 patients allows for future research on the feasibility of employing video-EEG monitoring and artificial intelligence to detect seizures. Overall, we demonstrate that the use of artificial intelligence together with engineering features can improve clinical practice in epilepsy.
Furthermore, training the algorithm and computing new samples has a high computational requirement on specialized cloud servers, but a light computational load in real-time, allowing its deployment on embedded devices for online seizure detection. This technology could be useful to develop customized hardware (device) using this algorithm, which may be able to help patients with epilepsy to identify seizures and warn relatives; or perhaps anticipate their seizures, prevent the risk of injuries, accidents, SUDEP or take appropriate safety precautions including rescue medication. This would help this group of patients who have a type of epilepsy that is not controllable with the resources we have today.
Funding
This work was supported by Grants from European Commission, Horizon2020 SME Instrument phase 2, SERAS_v4.0, Seizure Risk Assessment for Epilepsy, grant number 849781.
Authors’ contributions
D.B. and A.T. conceived and organized this review. G.T. and A.T. provided the first draft and prepared figures. D.B. and A.G. edited the draft and provided the final version. D.B. and A.T. provided financial support for the project that led to this publication. D.B., J.V., B.C., L.M., and B.L., have developed studies and research on the algorithm. J.U. and G.D. have reviewed the v-EEG data. D.B., A.T., and F.A. have carried out all the bureaucratic procedures related to the project. The authors approved the latest revised version of the manuscript and accept responsibility for their contributions.
Ethics approval and consent to participate
Approval of the used protocols was obtained through the ethics regional committee of Madrid, ref: 47/695620.9/19.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References
- 1.Fisher R.S., Cross J.H., French J.A., Higurashi N., Hirsch E., Jansen F.E., et al. Operational classification of seizure types by the international league against epilepsy: position paper of the ILAE commission for classification and terminology. Epilepsia. 2017 Apr;58(4):522–530. doi: 10.1111/epi.13670.50. [DOI] [PubMed] [Google Scholar]
- 2.Devinsky O., Vezzani A., O’Brien T.J., Jette N., Scheffer I.E., Perucca P., et al. Epilepsy Nat Rev Dis Prim. 2018;4:18024. doi: 10.1038/nrdp.2018.24. [DOI] [PubMed] [Google Scholar]
- 3.Sirven J.I. Epilepsy: A spectrum disorder. Cold Spring Harb Perspect Med. 2015;5(9) doi: 10.1101/cshperspect.a022848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Thijs R.D., Surges R., O’Brien T.J., Sander J.W. Epilepsy in adults. Lancet. 2019;393:689–701. doi: 10.1016/S0140-6736(18)32596-0. [DOI] [PubMed] [Google Scholar]
- 5.Shheng J, Liu S, Qin H, Li B, Zhang X, 2017. Drug-Resistant Epilepsy and surgery. Curr Neuropharmacol 2018;16(1):17-28. 10.2174/1570159X15666170504123316. [DOI] [PMC free article] [PubMed]
- 6.Kalilani L., Sun X., Pelgrims B., Noack-Rink M., Villanueva V. The epidemiology of drug-resistant epilepsy: A systematic review and meta-analysis. Epilepsia. 2018;59:2179–2193. doi: 10.1111/epi.14596. [DOI] [PubMed] [Google Scholar]
- 7.Laxer K.D., Trinka E., Hirsch L.J., Cendes F., Langfitt J., Delanty N., et al. The consequences of refractory epilepsy and its treatment. Epilepsy Behav. 2014;37:59–70. doi: 10.1016/j.yebeh.2014.05.031. [DOI] [PubMed] [Google Scholar]
- 8.Xue-Ping W., Hai-Jiao W., Li-Na Z., Xu D., Ling L. Risk factors for drug-resistant epilepsy. Medicine (Baltimore) 2019;98(30):e16402. doi: 10.1097/MD.0000000000016402. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Schulze-Bonhage A. Unpredictability of seizures and the burden of epilepsy. Seizure Prediction in Epilepsy 2008; Chapter 3: 1–10. 10.1002/9783527625192.ch1.
- 10.Téllez-Zenteno J.F., Hunter G., Wiebe S. Injuries in people with self-reported epilepsy: A population-based study. Epilepsia. 2008;49:954–961. doi: 10.1111/j.1528-1167.2007.01499.x. [DOI] [PubMed] [Google Scholar]
- 11.Moghim N., Corne D.W. Predicting epileptic seizures in advance. PLoS One. 2014;9:e99334. doi: 10.1371/journal.pone.0099334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Radhakrishnan K. The role of EEG in the diagnosis and classification of epilepsy syndromes. J Clin Neurophysiol. 2020;37:87. doi: 10.1097/WNP.0000000000000555. [DOI] [Google Scholar]
- 13.Stafstrom C.E., Carmant L. Seizures and epilepsy: an overview for neuroscientists. Cold Spring Harb Perspect Med. 2015;5(6) doi: 10.1101/cshperspect.a022426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Geertsema E.E., Thijs R.D., Gutter T., Vledder B., Arends J.B., Leijten F.S., et al. Automated video-based detection of nocturnal convulsive seizures in a residential care setting. Epilepsia. 2018;59(Suppl 1):53–60. doi: 10.1111/epi.14050. [DOI] [PubMed] [Google Scholar]
- 15.Gedzelman E., LaRoche S. Long-term video EEG monitoring for diagnosis of psychogenic nonepileptic seizures. Neuropsychiatr Dis Treat. 2014;10:1979–1986. doi: 10.2147/NDT.S49531. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Elger C.E., Hoppe C. Diagnostic challenges in epilepsy: seizure under-reporting and seizure detection. Lancet Neurol. 2018;17(3):279–288. doi: 10.1016/S1474-4422(18)30038-3. [DOI] [PubMed] [Google Scholar]
- 17.Beniczky S., Aurlien H., Brøgger J.C., Hirsch L.J., Schomer D.L., Trinka E., et al. Standardized computer-based organized reporting of EEG: SCORE – Second version. Clin Neurophysiol. 2017 Nov;128(11):2334–2346. doi: 10.1016/j.clinph.2017.07.418. [DOI] [PubMed] [Google Scholar]
- 18.Kuhlmann L., Karoly P., Freestone D.R., Brinkmann B.H., Temko A., Barachant A., et al. Epilepsyecosystem.org: crowd-sourcing reproducible seizure prediction with long-term human intracranial EEG. Brain. 2018;141(9):2619–2630. doi: 10.1093/brain/awy210. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Usman S.M., Usman M., Fong S. Epileptic seizures prediction using machine learning methods. Comput Math Methods Med. 2017;2017:9074759. doi: 10.1155/2017/9074759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Jang X., Bian G., Tian Z. Removal of Artifacts from EEG Signals: A review. Sensors. 2019;19(5):987. doi: 10.3390/s19050987. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Cherkassky V, Veber B, Lee J, Shiao H-T, Patterson N, Worrell GA, et al. Reliable seizure prediction from EEG data, in 2015 International Joint Conference on Neural Networks (IJCNN). Ireland, 2015; pp. 1-8. 10.1109/IJCNN.2015.7280327.
- 22.Page A., Sagedy C., Smith E., Attaran N., Oates T., Mohsenin T. A Flexible multichannel EEG feature extractor and classifier for seizure detection. IEEE Trans Circuits Syst II Express Briefs. 2015;62:109–113. doi: 10.1109/TCSII.2014.2385211. [DOI] [Google Scholar]
- 23.Raghavendra U., Acharya U.R., Adeli H. Artificial intelligence techniques for automated diagnosis of neurological disorders. Eur Neurol. 2019;82:41–64. doi: 10.1159/000504292. [DOI] [PubMed] [Google Scholar]
- 24.Bozhkov L., Georgieva P., Santos I., Pereira A., Silva C. EEG-based subject independent affective computing models. Procedia Comput Sci. 2015;53:375–382. doi: 10.1016/j.procs.2015.07.314. [DOI] [Google Scholar]
- 25.Rasheed K., Qayyum A., Qadir J., Sivathamboo S., Kwan P., Kuhlmann L., et al. Machine learning for predicting epileptic seizures using EEG signals: A review. IEEE Rev Biomed Eng. 2021;14:139–155. doi: 10.1109/RBME.2020.3008792. https://10.1109/RBME.2020.3008792 [DOI] [PubMed] [Google Scholar]
- 26.Kuhlmann L., Lehnertz K., Richardson M.P., Schelter B., Zaveri H.P. Seizure prediction - ready for a new era. Nat Rev Neurol. 2018;14:618–630. doi: 10.1038/s41582-018-0055-2. [DOI] [PubMed] [Google Scholar]
- 27.Orosco L., Correa A.G., Diez P., Laciar E. Patient non-specific algorithm for seizures detection in scalp EEG. Comput Biol Med. 2016;71:128–134. doi: 10.1016/j.compbiomed.2016.02.016. [DOI] [PubMed] [Google Scholar]
- 28.Sridevi V., Ramasubba Reddy M., Srinivasan K., Radhakrishnan K., Rathore C., Nayak D.S. Improved patient-independent system for detection of electrical onset of seizures. J Clin Neurophysiol. 2019;36:14–24. doi: 10.1097/WNP.0000000000000533. [DOI] [PMC free article] [PubMed] [Google Scholar]