Ictal autonomic activity recorded via wearable-sensors plus machine learning can discriminate epileptic and psychogenic nonepileptic seizures

Abstract

Differentiating epileptic seizures (ES) and psychogenic nonepileptic seizures (PNES) is commonly based on electroencephalogram and concurrent video recordings (vEEG). Here, we demonstrate that these two types of seizures can be discriminated based on signals related to autonomic nervous system activity recorded via wearable sensors. We used Empatica E4 Wristband sensors worn on both arms in vEEG confirmed seizures, and machine learning methods to train classifiers, specifically, extreme gradient boosting (XGBoost). Classification performance achieved a predictive accuracy of 78 ± 1.5% on previously unseen data for whether a seizure was epileptic or psychogenic, which is 6 standard deviations above the baseline of 68% accuracy. Our dataset contained altogether 35 seizures from 18 patients out of which 8 patients had 13 convulsive seizures. Prediction of seizure type was based on simple features derived from the segments of autonomic activity measurements (electrodermal activity, body temperature, blood volume pulse, and heart rate) and forearm acceleration. Features related to heart rate and electrodermal activity were ranked as the top predictors in XGBoost classifiers. We found that patients with PNES had a higher ictal heart rate and electrodermal activity than patients with ES. In contrast to existing published studies of mainly convulsive seizures, our classifier focuses on autonomic signals to differentiate convulsive or nonconvulsive semiology ES from PNES. Our results show that autonomic activity recorded via wearable sensors provides promising signals for detection and discrimination of psychogenic and epileptic seizures, but more work is necessary to improve the predictive power of the model.

I. Introduction

Psychogenic nonepileptic seizures (PNES) consist of paroxysmal events with or without alterations in consciousness that resemble epileptic seizures (ES) without the common associated signatures in neocortical, mesial and subcortical epileptiform electrophysiological activity [1], [2]. For this reason, PNES are considered functional neurological or conversion disorders. Misdiagnosis of seizures is a common problem. About one quarter of the patients referred to comprehensive epilepsy centers, for apparent drug-resistant epilepsy, are found to have nonepileptic seizures [3], [4], [5]. Furthermore, approximately 10% of individuals with PNES also have ES [6].

Commonly, laboratory differentiation between ES and PNES is based on examination of concurrent video and electroencephalogram (vEEG) recordings, which is the gold standard for seizure diagnosis [7]. However, vEEG can be costly, time consuming, not always available, and inconclusive in some cases. More recently, there have been attempts to examine the contribution of autonomic nervous system (ANS) activity to both understanding PNES, as well as using it for diagnosis, detection and discrimination [8]. Changes in autonomic system physiology are mediated by the spread of paroxysmal activity to centers involved in autonomic function such as amygdala, hypothalamus, hippocampus, insula, and spinal cord [9]. Autonomic dysfunction is known to be associated with both ES and PNES. For example, autonomic dysfunction can lead to sudden death in patients with epilepsy [10], [11], [12].

Despite this common association, autonomic dysfunction may manifest itself in different ways in ES and PNES. In this study, we examine ANS signals recorded by Empatica E4 Wristband © sensors and use machine learning methods to attempt to discriminate between ES and PNES recorded in the epilepsy monitoring unit. Previous studies examining ANS signals recorded via wearable sensors have reported mostly kinematic biomarkers (e.g. wrist acceleration or surface EMG) limiting analyses to only convulsive seizures (e.g. [13], [14], [15]) or have focused only on epileptic seizure identification [16]. Likewise, studies of heart rate variability (HRV) have distinguished only convulsive ES and PNES [17]. In contrast, our dataset included a mix of convulsive and non-convulsive seizures. We assessed the discrimination power of several ANS related signals, including electrodermal activity (EDA), heart rate (HR), blood volume pressure (BVP) and body temperature. We hypothesized that EDA and HR would distinguish between ES and PNES.

II. Methods

A. Data acquisition and preprocessing

Of 30 participants with inpatient video EEG long-term monitoring, 18 had at least one seizure during the monitoring period of the data collection. The 18 patients had altogether 35 seizures: 10 patients had 23 ES, and 8 patients had 12 PNES. Most of the seizures were measured by two devices - one device worn on each wrist. 13 out of the 35 seizures are convulsive. Seizure type (ES or PNES) and semiology (convulsive or non-convulsive) was determined by a board-certified neurologist/neurophysiologist (AB). An example of on participant’s recordings of a 24 hours epoch is illustrated in Fig. 1.

Fig. 1.

Example of the Empatica data of one patients session covering roughly 24 hours. The skin temperature (TEMP), heart rate (HR), the net acceleration (ACC), electrodermal activity (EDA) and blood volume pulse (BVP) were measured by the wrist-worn sensor. This patient wore two sensors (one on each wrist) and the left/right side shows the measurements of the left/right wrist device, respectively. The shaded red regions show the time of a seizure, the ictal period. The shaded grey regions show time intervals when the sensor was not in proper contact with the skin of the patient based on the BVP measurements and were not included in the machine learning analysis. Only the ictal period is used in our analysis.

Data preprocessing included dividing the variable-length seizure periods into fixed-length segments and removing segments if the Empatica device was not in contact with the patients skin. The 35 seizures were divided into 1-minute segments with no temporal overlap. Data from periods when the wrist sensor was not in contact with the patients skin were excluded from our model. Excluded periods were identified when the standard deviation of the BVP was less than 20 or larger than 600 in a segment. If the standard deviation of the BVP was outside the range, we observed the HR to be above 100–120 beats per minute, which is inconsistent with sedentary activity for patients lying in bed, and the HR changed erratically (see the grey region on Fig. 1). The BVP limits were determined by visually inspecting the data.

The final dataset contains 86 ES and 190 PNES segments, each of which are 1 minute long from 16 patients. A larger number of PNES segments were present with the smaller number of PNES because of the longer duration of the PNES. Some PNES lasted more than 10 minutes, while ES were maximum 4–5 minutes long, which is consistent with their typical ictal types.

B. The machine learning pipeline

We generated a set of simple features to illustrate that it is possible to differentiate ES from PNES based on autonomic activity signals. Each 1-minute segment was described by nine summary features: the mean, standard deviation, skewness, minimum, maximum, 25th, 50th, 75th percentiles, and the linear trend of the measurement (slope). As we have five measurements (net acceleration, blood volume pulse, electrodermal activity, heart rate, and body temperature - see Fig. 1), each segment was described by 45 features (nine features calculated for the five measured quantities).

Extra care was taken to ensure that the trained classification model generalizes to data from new patients. We performed three-fold patient-level cross validation (CV). The dataset was divided into training, test and holdout sets based on anonymous patient IDs. The classification model learns on the training set, the test set is used to select the model parameters that maximize the accuracy of the test set, and the holdout set is used to report the accuracy of the final trained model on previously unseen data. We used two CV methods. CV1 collects all segments of one patient in the test set, all segments of another patient are in the holdout set, and the remaining segments from the other patients are in the training set. CV2 collects segments from one patient with psychogenic and one with epileptic seizures in the test set, and segments of another patient with psychogenic and another with epileptic seizures are in the holdout set. The remaining segments of the other patients are in the training set.

XGBoost was used to train the classification model [18]. XGBoost (eXtreme Gradient boosting) is a tree-based classification/regression algorithm. We chose a tree-based method because they generally perform well on small datasets. XGBoost is the most advanced tree-based method to our knowledge and it generates models that are easy to interpret. Several parameters can be optimized to improve the accuracy of model (learning rate, subsampling parameters, regularization parameters, etc). We tuned the depth of the trees using grid search and explored tree depth values between 2 and 10. The number of trees is another parameter which is tuned by early stopping. In early stopping, the number of trees is increased until the accuracy of the resulting model on the cross-validation set does not improve for some time (a parameter described by early_stopping_rounds in XGBoost). Other parameters were unexplored: learning_rate = 0.03, subsample = 0.66, colsample_by_tree = 0.9.

We looped through all possible ways the dataset can be split into the three folds and repeated the process 10 times with different random seeds to calculate uncertainty estimates. There were 240 different combinations to split our dataset into three folds with CV1, and 3024 combinations with CV2. We looped through each possible combination and accumulated the predicted and true labels of the holdout set. The accumulated information was used to calculate the accuracy score and the confusion matrix. The uncertainty estimate was calculated by repeating this whole process 10 times. Even-though the exact same dataset was used in each iteration, tree-based methods have an inherent randomness and this uncertainty is what we assessed with our pipeline. We developed the machine learning pipeline in python 3.6 using packages like scikit-learn, numpy, pandas, and xgboost.

III. Results

A. Classification accuracy

The predictive power of a classifier was assessed based on the confusion matrix and derived metrics. The confusion matrix (C) is defined such that C_i,j is equal to the observations that are known to be in group i but are predicted to be in group j. Thus the i = j elements are the correctly predicted observations (true positives and true negatives, TP and TN respectively) and the i ≠ j elements are the misclassified observations. It is not obvious in our case what should be called a false positive (FP or false alarm) and a false negative (FN or miss). The reason is that we do not distinguish between a positive and a negative condition (i.e., no seizure or seizure) rather differentiate between two seizure types. As FP and FN cannot be unambiguously defined, we report the accuracy metric of the classifier ((TP + TN)/(n_samples)) and the confusion matrix.

The classification accuracy was 78% which is at least 6 standard deviations above the baseline accuracy (see Table I). The baseline accuracy (the accuracy if all segments in the holdout set are predicted to be PNES) is 68% with CV1 and 74% with CV2. The standard deviation of the accuracy is 1.5% with CV1 and 0.5% with CV2. The difference is due to the larger number of combinations and more accumulated data in the CV2 pipeline where we select data from two patients in the test set and data from two patients in the holdout set.

TABLE I.

Confusion matrices of the holdout set. The values are in percentage points. The complete dataset was analysed with the CV1 and CV2 patient-level cross validation approaches described in Sect. II-B. All possible combinations were explored for the uncertainty estimates.

CV1, 240 combinations, 78 ± 1.5% accuracy with a balance of 68%.
		Predicted label
		ES	PNES
True label	ES	23.1 ± 0.4%	8.0 ± 0.4%
	PNES	14.4 ± 1.3%	54.5 ± 1.3%
CV2, 3024 combinations, 78.6 ± 0.5% accuracy with a balance of 74%.
		Predicted label
		ES	PNES
True label	ES	19.3 ± 0.1 %	6.7 ± 0.1 %
	PNES	14.6 ± 0.05 %	59.3 ± 0.05 %

B. Predictive features

Feature importance measures helped us identify that the most predictive features are derived from the HR and EDA measurements. Feature importance measures how useful/valuable a feature is in constructing the decision trees. We calculated the feature importances in both models and the HR and EDA emerged as the two most predictive features of differentiating ES from PNES.

We plot the histograms of the two most predictive features during interictal and ictal periods (see Figs. 2 and 3) to understand their clinical relevance, which indicate how the autonomic functions differ during ES and PNES. As the interictal periods are much longer than the ictal periods, the interictal distributions are much better constrained (≈ 10⁴ interictal data points vs 276 ictal data points.) We find that the minimum HR measured within the 1-minute segments is the most predictive feature. The interictal HR of ES and PNES patients are 68 ± 11 BPM and 72 ± 10 BPM, respectively. The ictal HR of ES and PNES patients are 73 ± 14 BPM and 94 ± 12 BPM, respectively. We performed two two-sample two-sided K-S tests on the HR data. First, we compared the interictal vs ictal HRs of ES patients and found that the null hypothesis that the distributions of the two samples are the same cannot be rejected at a 1% level (p = 0.018). However, when we compared the interictal vs ictal HRs of PNES patients (p = 3 × 10⁻⁸²), we can confidently reject the null hypothesis. Thus the heart rates of ES patents during the ictal and interictal periods are similar, while PNES patients have a higher ictal heart rate. The second most important feature is the the minimum EDA. The interictal EDA of ES and PNES patients are 0.71 ± 1.5 and 1.3 ± 2.4, respectively. The ictal EDA of ES and PNES patients are 0.33 ± 0.6 and 3.5 ± 3.6, respectively. The K-S test of the interictal vs ictal EDA of ES patients was not significant (p = 0.09, null hypothesis cannot be rejected at a 5% level), while the interictal vs ictal EDA of PNES patients is significant (p = 2 × 10⁻³², the null hypothesis is confidently rejected). These findings indicate that the EDA of ES patients are similar during the interictal and ictal periods, but PNES patients have a high EDA during seizures.

Fig. 2.

The most predictive feature in our classification model is the minimum heart rate. The left figure shows the ES and PNES HR histograms during the interictal period. The figure on the right shows the same histograms during the ictal periods. The ES histogram is shown in blue, the PNES histogram is red, the purple area is overlap between the two distributions. The y axis is the fraction of segments present in a bin (1 equals 100%). The heart rate of PNES patients is higher during the ictal period compared to the interictal period and the ictal HR of ES patients.

Fig. 3.

The second most predictive feature is the minimum electrodermal activity measured during the segments. Similarly to Fig. 2, the left(right) side shows the interictal(ictal) histograms. The EDA of PNES patients is higher during the ictal period compared to the interictal period and the ictal EDA of ES patients.

Limitations of this pilot study are the sample size, and collection from a single site. Novel aspects of the study include comparing multiple ANS signals to differentiate convulsive and nonconvulsive ES from PNES. The preliminary findings reflect a robust signal for differentiating ES from PNES, and can be strengthened with a multi-site large sample trial.

IV. Summary and outlook

Our results indicate that ANS signals recorded via wrist-worn wearable sensors provide promising biomarkers for differentiating vEEG confirmed ES and PNES, both convulsive and nonconvulsive. Among the ANS signals, statistical features related to HR and the EDA were ranked as the top discriminating features. In particular, the ictal HR during PNES appears to be higher than in ictal ES. Other studies have previously examined HR differences in ES and PNES using different sensors and statistical methods [19], [17], [20]. In contrast to our findings, [19] has found higher HR changes in ES ictal events than in PNES, using HR measures relative to baseline references, while we have used absolute rates. Other studies have found prominent peri-ictal increases in EDA in association with ES [21] but comparative studies in PNES versus ES have not been previously reported. We find that EDA responses are higher in PNES than in ES in our cohort. The results of this pilot ictal physiology biomarker study are promising and warrant further data collection and more sophisticated feature-engineering in future studies.