EDA-Gram: Designing Electrodermal Activity Fingerprints for Visualization and Feature Extraction

Abstract

Wearable technology permeates every aspect of our daily life increasing the need of reliable and interpretable models for processing the large amount of biomedical data. We propose the EDA-Gram, a multidimensional fingerprint of the electrodermal activity (EDA) signal, inspired by the widely-used notion of spectrogram. The EDA-Gram is based on the sparse decomposition of EDA from a knowledge-driven set of dictionary atoms. The time axis reflects the analysis frames, the spectral dimension depicts the width of selected dictionary atoms, while intensity values are computed from the atom coefficients. In this way, EDA-Gram incorporates the amplitude and shape of Skin Conductance Responses (SCR), which comprise an essential part of the signal. EDA-Gram is further used as a foundation for signal-specific feature design. Our results indicate that the proposed representation can accentuate fine-grain signal fluctuations, which might not always be apparent through simple visual inspection. Statistical analysis and classification/regression experiments further suggest that the derived features can differentiate between multiple arousal levels and stress-eliciting environments for two datasets.

I. INTRODUCTION

Wearable devices are increasingly being embedded into our everyday life spanning a wide range of applications from health and well-being, to education, security and human-computer interaction [1]. This results in a tremendous amount of physiological data, for which reliable processing and interpretation are essential [2]. Along with data-driven methods, knowledge-based algorithms are equally important towards representing and interpreting physiological signals in a meaningful way. Visualization and informative signal measures can further assist with analysis and exploration.

Electrodermal activity (EDA) is one of the most commonly used physiological signals [3]. It represents the activation of the sympathetic “fight or flight” nervous system, increasing due to stress, emotional arousal, and/or attention [3], [4]. The association of EDA to various behavioral and pathological indices and its relatively non-intrusive data capture ability underscore its usefulness to several mobile sensing applications [5]. EDA is decomposed into tonic and phasic parts. The former captures the signal levels and is called skin conductance level (SCL), while the latter reflects the fluctuations superimposed onto the signal trend, known as skin conductance responses (SCR). SCRs depict sharp increase and slow recovery and are commonly studied in several psychophysiological settings [3], [4].

Previous approaches on EDA representation have used least-squares fit methods [6], generative causal models [7], [8], as well as convex optimization techniques [9]. Other studies have designed EDA features considering signal variations within parameterized temporal windows [10] and time-series approximations [11]. Most of these models impose restrictive constraints on the SCR shape, or take the SCR variability implicitly into account, highlighting the need to explore the finer details of the EDA structure.

Taking these limitations into account, we propose an interpretable model that considers the typical EDA shape and incorporates a wide range of signal variations. We build signal-specific dictionaries of tonic and phasic atoms from which we decompose the signal with sparse representation techniques [12], [13]. Based on this representation, we design a multidimensional EDA fingerprint for visualization and feature extraction pusposes. The “EDA-Gram” is created similarly to a spectrogram in signal processing, in which the horizontal axis captures the signal frames, the vertical dimension depicts the selected atom width, while the intensity values are computed from the corresponding atom amplitude. Thus it incorporates the SCR width and amplitude variations. Measures of the EDA-Gram intensity are used as features to describe the underlying signals. Results indicate that the proposed features are able to discriminate between multiple arousal and stress conditions in two datasets, signifying their ability at capturing the fine-level variations of the EDA.

II. EDA-GRAM

A. Knowledge-Driven EDA Representation

Let $x_n\in {\Re }^P$ be an analysis frame of length P from signal $x={\left[x_1^T\dots x_N^T\right]}^T\in {\Re }^{PN}$. According to sparse representation theory, x_n can be expressed as a small number of atoms from the dictionary $D\in {\Re }^P{}^{\times K}$, such that x_n = Dc_n, where $c_n\in {\Re }^K$, ∥c_n∥₀ = L ≪ K, are the atom coefficients. The dictionary contains K₁ tonic and K₂ phasic atoms (K=K₁+K₂) for the corresponding EDA components:

$$D=\left[{\varphi }_1\left(t,{\mathit{\zeta }}_1\right),\dots ,{\varphi }_1\left(t,{\mathit{\zeta }}_{{\mathit{K}}_1}\right),\dots {\varphi }_2\left(t,{\mathit{\theta }}_1\right),\dots ,{\varphi }_2\left(t,{\theta }_{{\mathit{K}}_2}\right)\right]\in {\Re }^{P\times \left(K_1+K_2\right)}$$ (1)

Tonic atoms ${\varphi }_1\left(t,{\mathit{\zeta }}_k\right)\in {\Re }^2\to {\Re }^P$, ζ_k = [Γ_k, Δ_k]^T are modelled with straight lines, where Γ_k, Δ_k denote the off-set and slope. Phasic atoms ${\varphi }_2\left(t,{\mathit{\theta }}_k\right)\in {\Re }^4\to {\Re }^P$, θ_k = [a_k b_k s_k μ_k]^T are appropriately designed through Bateman functions [7] in order to match the SCR shape. Their parameters include the steepness of onset and recovery a_k, b_k, as well as the time shift and scale μ_k, s_k. The final dictionary contains 63 tonic and 2790 phasic atoms to account for the large variability of EDA fluctuations (Table I). Sparse decomposition is performed with orthogonal matching pursuit (OMP) because of its simplicity and effectiveness [14]. More details on this EDA representation can be found in [12], [13].

TABLE I.

Description of EDA-specific dictionary atoms and initial parameters.

Tonic Atoms
ϕ1(t, ζ) = Γ + Δ · t	Γ ∈ {−20, −10, 1} Δ ∈ {−0.01, −0.009,…, 0, 0.01, 0.02,…, 0.1}
Phasic Atoms
ϕ2(t, θ) = (e−b(st−μ) − e−a(st−μ)) · u(t – μ)	a ∈ {8, 14, 18} b ∈ {10, 15, 20} μ ∈ {0, 10, 20,…, 150} s ∈ {0.100, 0.105,…, 0.140}

u(t) = 1, t ≥ 0 and u(t) = 0, otherwise

B. EDA-Gram Design

Since EDA is represented from a set of parametric functions, we can recover its structure in a knowledge-driven way. We focus on SCR shape and occurrence, as these are related to various psychophysiological conditions [3], [4] and can be measured from the phasic atoms. An intuitive characteristic of phasic atoms is their width ω_k, k = 1,…, K₂, denoted as phasic width (PW), and incorporating SCR rise and recovery (Fig. 1a). Because of the large number of phasic atoms in the dictionary (Section II-A), we group the PW values through a histogram, such that each ${\left\{{\omega }_k\right\}}_{k=1}^{K_2}$ is assigned to a band ${\left\{{\Omega }_m\right\}}_{m=1}^M$, M ≪ K₂, called phasic width band (PWB). Therefore PWBs correspond to the centers of the histogram generated from the PW values (Fig. 1b).

Fig. 1.

EDA-Gram design. (a) Example of phasic dictionary atoms with phasic width (PW). (b) Histogram of PW values with the resulting phasic width bands (PWB). (c) Input EDA. (d) Signal analysis frames (in solid blue line) and selected phasic atoms (in non-solid lines). (e) EDA-Gram.

An analysis frame is decomposed as $x_n=x_{n_{\text{tonic }}}+{\sum }_{k\in {\mathcal{I}}_n}{c}_k\varphi \left(t,{\theta }_k\right)$, where $x_{n_{\text{tonic }}}$ is the reconstructed signal from the tonic atoms, ${\mathcal{I}}_n=\left\{i_1,\dots ,i_{L_2}\right\}$ and $c_{i_l}\ne 0\left(l\in {\mathcal{I}}_n\right)$ are the L₂<L indices and coefficients of phasic atoms. Each atom from set ${\mathcal{I}}_n$ is associated with coefficient $c_{i_l}$ and PWB ${\Omega }_{i_l}$. Figs. 1c–d show an EDA signal, indicative analysis frames, and the corresponding selected phasic atoms.

Inspired by the spectrogram, which is a signal representation over time and frequency, the EDA-Gram can be expressed as the time evolution of the EDA over the PWBs ${\left\{{\Omega }_m\right\}}_{m=1}^M$. The EDA-Gram S_x(n, m) for signal x_n is:

$$\begin{gathered} S_x(n,m)=\sum _{i\in {\mathcal{I}}_{nm}}{c}_{i^{*}},n=1,\dots ,N,m=1,\dots ,M \\ {\mathcal{I}}_{nm}=\left\{i_l:i_l\in {\mathcal{I}}_n\wedge \exists m\text{ s}.\text{t}.{\Omega }_m={\Omega }_{i_l}\right\} \end{gathered}$$ (2)

where ${\mathcal{I}}_{nm}$ includes the indices of the selected phasic atoms for the n^th time frame that belong to the m^th PWB. The horizontal axis of EDA-Gram corresponds to time (i.e. the n^th frame), the vertical axis to PWB (i.e. Ω_m), while intensity values are computed from the sum of coefficients of selected atoms over each PWB (Fig. 1e).

C. EDA-Gram Feature Extraction

EDA-Gram is a compressed multidimensional representation of EDA that offers a foundation for deriving appropriate features. We extract two types of features that capture the mean intensity and the variability of intensity across PWBs:

$$S_x^{Int}=\frac{1}{NM}\sum _{n=1}^N\sum _{m=1}^M{S}_x(n,m)$$ (3)

$$S_x^{Var}=\frac{1}{N}\sum _{n=1}^N\sqrt{\frac{1}{M}\sum _{m^{\prime }=1}^M{\left(S_x(n,m)-{\overline{S}}_x(n)\right)}^2}$$ (4)

where ${\overline{S}}_x(n)=\frac{1}{M}{\sum }_{m=1}^M{S}_x(n,m)$ is the mean intensity over all PWBs for the n^th frame. Large values of $S_x^{Int}$ denote frequent SCRs of high amplitude, while large $S_x^{Var}$ indicates high SCR variability across the signal. We will explore how the proposed features vary in relation to different arousal levels and environmental conditions.

III. EXPERIMENTS

In the following experiments, EDA representation was derived with analysis window of 10 sec and L=5 selected atoms. EDA-Gram was designed with M=8 PWBs.

A. Data Description

We validate our approach with two datasets. The first is the publicly available dataset for emotion analysis using physiological signals (DEAP) [15], which investigates the relation of human physiology to multimedia. DEAP includes 40 one-minute recordings from 32 individuals. We use the self-reported arousal measures, which take continuous values between 13–9, as these are closely related to EDA [3], [4]. EDA was collected from the index and middle finger and was downsampled to F_s=32Hz, which consists a typical practice.

Motivated by the challenges children with special health-care needs face during oral care [16], the second dataset was collected in order to examine the impact of a sensory adapted dental environment (SADE) on the behavioral distress, physiological stress, pain, and sensory discomfort of children with autism spectrum disorders (ASD). Specifically, the SADE modified the dental room with respect to visual, auditory, and tactile stimuli and was compared to a regular dental environment (RDE) in children with ASD and their typically developing (TD) peers [17]. Our data (DENTISTRY) include EDA recordings from children with ASD (n=22) and their TD peers (n=22), 6–12 years of age, during the dental prophylaxis (cleaning). Each child underwent a dental cleaning in both the RDE and SADE, which will be compared in terms of EDA measures. EDA was collected with a sampling frequency F_s=31.25Hz from the index and middle finger.

B. Baseline Features

We compare the EDA-Gram features with two baselines. The first includes the commonly used mean SCL and SCR frequency [3], [4] (SCL_M and SCR_F), that capture the average signal level and number of SCRs over a given time. For the second baseline, we compute the power spectral density (PSD) 16 frequency bands (0–2F_s Hz). We further extract the average PSD over time and frequency (PSD^Int), as well as its mean standard deviation over frequency bands (PSD^Var), similarly to EDA-Gram features (Section II-C).

C. Visualization

EDA-Grams show increased intensity over regions of large tonic levels and high phasic activity (Fig. 2). RDE samples from the DENTISTRY data depict larger SCL and SCR frequency compared to SADE (Fig. 2a), which is reflected in higher EDA-Gram intensity values (Fig. 2a,b). It is noteworthy that although Participant 2 depicts EDA signals within the same range for the two environments, there are differences in the corresponding waveforms, that are also captured by the EDA-Grams. Similar observations were made for DEAP.

Fig. 2.

Example of EDA signals and EDA-Grams for two subjects during the regular (RDE) and sensory adapted (SADE) dental environments.

D. Statistical Analysis

We explore the relation between arousal and EDA features in DEAP through a linear mixed-effects model (LME) [18]. LME addresses independence assumption violations that could occur from multiple samples per participant. It models the self-reported arousal score A_ij from the i^th individual during the j^th trial based on EDA feature F_ij, a grand-mean γ₀₀, an individual-specific mean u_0j, and a residual r_ij, such that A_ij=γ₀₀+u_0j+γ₁₀F_ij+r_ij, where γ₁₀ reflects the relation A_ij-F_ij. Results indicate significant association between all considered EDA measures and the arousal scores, which is increased for the EDA-Gram features (Table II).

TABLE II.

Linear mixed-effects model (LME) for predicting self-reported arousal (DEAP) from EDA features and Two-way ANOVA comparing differences on EDA features between dental environments and cleaning tasks (DENTISTRY).

	DEAP		DENTISTRY
Feature	LME Estimate		ANOVA P-value
	Intercept γ00	Independent γ10	Environment (RDE/SADE)	Outcome (ASD/TD)
SCLM	0.94*	0.05*	<0.01	<0.05
SCRF	0.42*	0.23*	<0.11	<0.05
PSDInt	0.94*	0.05*	<0.01	0.05
PSDVar	0.59*	0.08	0.60	0.80
$S_x^{Int}$	0.25†	0.33*	<0.05	0.06
$S_x^{Var}$	0.25†	0.33*	<0.05	0.31

^†p < 0.05,

^*p < 0.01

For the DENTISTRY data, we examine the effect of dental environment on EDA features using a two-way ANOVA, whose independent variables include the dental environment (RDE/SADE) and outcome (ASD/TD). Results suggest significant differences between RDE and SADE (Table II).

E. Regression and Classification

Regression and classification were performed with a leave-one-subject-out cross-validation in order to see how features are generalized on unseen data. Linear regression was used for the DEAP dataset to predict the continuous arousal levels from EDA features, while K-nearest neighbor (K-NN, K=13) classification was performed for the DENTISTRY data to classify between RDE and SADE. Feature dimensionality is one for all cases and chance classification accuracy is 50%. Results suggest better discriminability of the EDA-Gram features compared to the considered baselines (Table III).

TABLE III.

Pearson’s correlation between real and predicted arousal values (DEAP) and classification accuracy between regular and sensory adapted dental environments (DENTISTRY).

Feature	DEAP - Correlation	DENTISTRY - UA (%)
SCLM	0.090*	52.3
SCRF	0.088*	47.2
PSDInt	0.080†	52.3
PSDVar	0.042†	44.3
$S_x^{Int}$	0.152*	59.1
$S_x^{Var}$	0.127*	65.3

^†p < 0.05,

^*p < 0.01

IV. DISCUSSION

This paper introduces the EDA-Gram, a multidimensional representation for fingerprinting the EDA. Results indicate that the proposed approach can capture and accentuate fine-grain signal fluctuations, not always apparent from traditional signal inspection. Such visualizations can assist in clinical applications, therefore future work will evaluate those with human annotators for usefulness and easiness of operation.

Experiments suggest that the proposed features can be more informative than the baseline. This might be due to the fact that EDA-Gram is able to better incorporate signal changes, that are concealed from simple averaging and might not be as effectively captured by the PSD. Also the proposed EDA-specific representation can group them in a meaningful way, i.e. based on PWB values. Future work will examine the association of each PWB to psychophysiological predictors.

V. CONCLUSIONS

We propose the EDA-Gram for visualizing EDA signals and extracting meaningful features. It is created based on the sparse decomposition of EDA with knowledge-driven dictionaries that capture the tonic and phasic signal components. EDA-Gram is a multidimensional representation, in which the x-axis denotes time, the y-axis reflects the phasic EDA characteristics through the PWB space, while the intensity is measured from the amplitude of the selected atoms. Visualization and analysis indicate the ability of EDA-Grams and the derived features to differentiate between multiple arousal conditions and environmental effects in two datasets outperforming the considered baseline metrics.