Real-Time Modeling and 3D Visualization of Source Dynamics and Connectivity Using Wearable EEG

Abstract

This report summarizes our recent efforts to deliver real-time data extraction, preprocessing, artifact rejection, source reconstruction, multivariate dynamical system analysis (including spectral Granger causality) and 3D visualization as well as classification within the open-source SIFT and BCILAB toolboxes. We report the application of such a pipeline to simulated data and real EEG data obtained from a novel wearable high-density (64-channel) dry EEG system.

I. Introduction

Dynamic cortico-cortical interactions are central to neuronal information processing. The ability to monitor these interactions in real time may prove useful for Brain-Computer Interface (BCI) and other applications, providing information not obtainable from univariate measures, such as bandpower and evoked potentials. Wearable (mobile, unobtrusive) EEG systems likewise play an important role in BCI applications, affording data collection in a wider range of environments. However, reliable real-time modeling of neuronal source dynamics using data collected in mobile settings faces challenges, including mitigating artifacts and maintaining fast computation and good modeling performance with limited amount of data. Here we describe some of the wearable hardware and signal processing we are developing that attempt to address these challenges, contributing to the development of EEG as a mobile brain imaging modality.

II. Material and Methods

A. Wearable EEG Hardware

For mobile EEG applications, we have developed a 64-channel wireless dry headset (Fig. 1). A padded sensor is used on the forehead locations for contact with bare skin. For through-hair recordings, a novel dry electrode is used, consisting of a flexible plastic substrate coated with a conductive surface. The legs of the sensor push through hair to achieve contact with scalp, flattening with pressure to minimize discomfort and injury. Typical contact impedances are 100kW–1MΩ, and high input impedance amplifiers are used to ensure minimal signal degradation. Typical correlation between simultaneously recorded averaged evoked potentials (AEP, SSVEP, P300) for dry and standard wet electrodes is r > 0.9 indicating comparable signal measurement. Nonetheless, the dry electrodes may exhibit a higher level of drift and low frequency noise due to the lack of gel and skin-abrasion.

Fig. 1.

Real-time data processing pipeline. A Cognionics 64-channel system is depicted above with flexible active dry electrodes.

All electronics, including preamplifiers, digitization, battery (6–7 hour capacity), onboard impedance monitoring, micro-controller and Bluetooth transceiver are contained in a miniature box at the rear of the headset. Signals are sampled at 300Hz with 24-bit precision. The total noise of the data acquisition circuitry, within EEG bandwidth, is less than 1 µV RMS. Event markers are transmitted from the PC to the headset via a special low-latency wireless link specifically optimized to minimize jitter (< 500 microseconds).

B. Preprocessing and Artifact Rejection

EEG data are streamed into MATLAB (The Mathworks, Natick, MA), and an online-capable pre-processing pipeline is applied using BCILAB [3]. Elements of this pipeline may be intialized on a short segment of calibration data. These include rejection (and optional re-interpolation) of corrupted data samples or channels. Short-time high-amplitude artifacts in the continuous data may be removed online, using a new method we call Artifact Subspace Reconstruction (ASR). This relies on a sliding-window Principal Component Analysis, which statistically interpolates any high-variance signal components exceeding a threshold relative to the covariance of the calibration dataset. Each affected time point of EEG is then linearly reconstructed from the retained signal subspace based on the correlation structure observed in the calibration data. Artifacts may also be removed by rejecting a subspace of ICA components pre-computed using an (overcomplete) decomposition [14] on calibration data or adaptively estimated using Online Recursive ICA [1]. Artifactual components may be identified automatically by fitting dipoles to components and selecting a subspace of components to retain based on heuristic criteria such as residual variance of dipole fit or dipole anatomical coordinates and labels.

C. Source Reconstruction

Following pre-processing, one may estimate the primary current source density (CSD) over a medium- to high-resolution cortical mesh (3751–12000 vertices). For this report we used a 3751-vertex mesh. Our default forward model consists of a four-layer (skull, scalp, csf, and cortex) Boundary Element Method (BEM) model derived from the MNI Colin 27 brain and computed using OpenMEEG [5]. For inverse modeling, we rely on anatomically constrained LORETA with Bayesian hyperparameter estimation [12]. This approach is well suited for real-time adaptive estimation and automatically controls the level of regularization for each measurement vector. For processing speed we follow the SVD-based reformulation of [9]. Additionally, we segment the source space into 90 regions of interest (ROIs) using Automated Anatomical Labeling (AAL) [13]. The user can compute spatially averaged, integrated or maximal CSD for any subset of these ROIs. We make use of routines from our MoBILAB toolbox freely available online [8].

D. Dynamical Systems Analysis

Preprocessed channel or source time-series are forwarded to SIFT [3] and an order-p sparse vector autoregressive (VAR[p]) model is fit to a short chunk of recent data (e.g. 0.5–2 sec). The VAR coefficients are estimated using Alternating Direction Method of Multipliers (ADMM) with a Group Lasso penalty [2]. Model estimation is warm-started using the solution for the previous data chunk. The regularization parameter can be initialized by cross-validation on the calibration data and/or adapted online using a heuristic based on two-point estimates of the gradients of the primal and dual norms. Model order can be selected offline, by minimizing information criteria (e.g. AIC or BIC) on calibration data. Following model fitting and tests of stability and residual whiteness (autocorrelation function or Portmanteau), we obtain the spectral density matrix and any of the frequency-domain functional and effective connectivity measures implemented in SIFT. Graph-reductive metrics such as degree, flow, and asymmetry ratio can be applied to connectivity matrices. These measures may be piped to BCILAB as features for subsequent classification or visualized in real time. Available graphs include current density, power spectra, connectivity, outflow, etc. in 2-D plots as well interactively within a three-dimensional model of the head and brain.

E. Connectivity Classification

To learn robust predictive models on the high-dimensional connectivity feature space (d > 7000) from few trials strong prior assumptions need to be employed. We applied a regularized logistic regression, implemented via ADMM, to log-transformed time/frequency (T/F) Direct Directed Transfer Function [6] (dDTF) measures (yielding a 4-dimensional feature tensor across pairwise connectivity, time and frequency). The regularization simultaneously employed a sparsifying l₁/l₂ + l₁ norm with one group for each connectivity edge, containing its associated T/F weights, plus three trace norm terms to couple the T/F weights for all out-edges of a node, all in-edges of a node, and all auto-edges across nodes, respectively, plus an l₂ smoothness term across time and frequency, respectively. The regularization parameter for the data term was searched via (nested) 5-fold blockwise cross-validation over the range 2⁰,2^−0.25, …, 2⁻¹⁰. The relative weights of the regularization terms were searched over 10 predefined joint assignments, although setting all weights simply to 1 yielded comparable results.

F. Data Collection

We have conducted preliminary evaluation of our pipeline on both simulated data and real (64-channel) data collected in mobile settings using Cognionics hardware.

1) Simulated Data

To test the ability of our pipeline to accurately reconstruct source dynamics and connectivity in real-time, we generated a five-dimensional VAR[3] system of coupled oscillators as described in Eq. 3.1 of [10]. This comprised the CSD time-series of 5 sources postioned on a 3571-vertex cortical mesh. Each source had a Gaussian spatial distribution (σ = 5 cm) with mean equal to the centroid of each of the following AAL ROIs (respectively): x₁: Left Middle Cingulate Gyrus, x₂: Left Middle Occipital Gyrus, x₃: Right Medial Superior Frontal Gyrus, x₄: Right Precentral Gyrus, x₅: Left Precentral Gyrus. The system is depicted in the inset in Fig. 3. We generated two minutes of source time-series data (Fs=300 Hz) and projected this through the realistic forward model described in Section II-C to produce 64-channel EEG data. Gaussian i.i.d sensor noise was added (SNR = σ_data/σ_noise = 5).

Fig. 3.

Temporal snapshot of reconstructed source networks (PDC estimator) displayed within the BrainMovie3D visualizer. Here edge color denotes preferred coupling frequency while edge size and tapering respectively denote coupling strength and directionality at that frequency. Node size indicates outflow (net influence of a source on all other sources). The graph is thresholded at the commonly used PDC heuristic level of 0.1. Cortical surface regions are colored according to their AAL atlas label (90 regions).

2) Real Data

One session of EEG data was collected from a 22 year-old right-handed male performing a modified Eriksen Flanker task with a 133 ms delay between flanker and target presentation [7]. Flanker tasks have been extensively studied and are known to produce error-related negativity (ERN, Ne) and error-related positivity (P300, Pe) event-related potentials (ERPs) following error commission as seen on Fig. 5 bottom. The Ne is an early negativity in the EEG with middle/anterior cingulate generators which peaks 40–80 ms following commission of a response error. This is often followed by a Pe peaking at 200–500 ms.

Fig. 5.

ERPImage of single-trial EEG potentials (no smoothing) at FCz for response-locked error trials, sorted by latency of response to target onset (red sigmoidal trace). Responses occur at 0 ms (vertical line). The bottom panel shows the averaged ERP without ASR in blue, and the ERP with ASR enabled in red.

III. Results

A. Simulation Data

The simulated EEG data (Section II-F.1) was piped through our pre-processing pipeline (filtering and ASR disabled here) and median CSD was computed for each of the 5 ROIs. Fig. 2 shows a 1-sec segment of reconstructed CSD superimposed on the true CSD. Superficial sources were accurately recovered, while the deep, tangential source (X₁; mid-cingulum) was somewhat more poorly reconstructed. Spectral density, dDTF and partial directed coherence (PDC) between ROI median-CSD time-series was estimated using a 1-sec sliding window over 1–65 Hz. The max operator was applied to collapse across frequency producing a 2D connectivity matrix (directed graph). Fig. 3 shows the estimated source network for a representative time window, using our BrainMovie3D visualizer. Ground truth is displayed in the inset. Over all time windows, the connectivity graph was recovered with high accuracy – the average area under curve (AUC) was 0.97 +/− 0.021. Peak coupling frequency and relative strength were also correctly recovered.

Fig. 2.

Comparison of true (red, dashed) vs. reconstructed (blue, solid) current source density for a 1-sec segment of our 5 simulated ROIs. Sources and data are scaled to unit variance

B. Real Data

1) Data Quality and Artifact Rejection

For this paper, our online pipeline included the following pre-processing elements (in order of application): downsampling to 128 Hz, sub-1 Hz drift correction (Butterworth IIR filter), bad channel removal and interpolation, ASR, average referencing and 50 Hz low-pass minimum-phase FIR filtering. Four channels were automatically removed (Afz,T7,T8,P09), which were those also identified by eye as corrupted during data collection. Fig. 4 shows a segment of EEG data contaminated by blink and muscle artifacts, before and after ASR artifact removal.

Fig. 4.

10 sec of EEG data following ASR data cleaning (blue trace) superimposed on original data (red trace)

Single-trial EEG data for response-locked error trials are shown for electrode FCz in Fig. 5. Trials are sorted by reaction time. Although acausal filters cannot be used online, for this plot alone, in order to accurately assess ERP latencies, all filters were zero phase (acausal). We ran the analysis with and without ASR (the latter shown here) and confirmed that ASR did not distort ERPs (Fig. 5, red trace). Note that nearly every trial shows a visual evoked response to the stimulus as well as prominent Ne and Pe following the erroneous button press. The scalp topography of the Ne (upper left) has a frontocentral distribution centered at FCz, as expected for a mid/anterior cingulate or frontal midline generator. Encouragingly, the quality of the evoked responses is comparable to that reported using research-grade gel-based EEG systems.

2) Source Reconstruction

Preliminary source analysis showed good reconstruction of early visual evoked potentials from occipital and parietal regions as well as frontal localization of the Pe following button presses. However, the Ne was not reliably recovered. As noted above, single-trial estimates of current density may be less reliable for deep, medial sources (i.e., cingulate regions) than for superficial sources. While this issue has remedies, we defer a more complete analysis to subsequent work and report connectivity analysis and classification on channel data.

3) Connectivity Analysis

For a moderate number of channels (or sources) (8–15) with model order in the 10–15 range, we generally obtained good VAR model fit (stable with uncorrelated residuals, ACF test: p < 0.05). The VAR process spectrum exhibits characteristic EEG 1/f shape with theta, alpha, and beta peaks, including prominent occipital alpha gain and occipital-frontal Granger-causality at rest with eyes closed.

4) Classification

We tested our classification stage on the problem of detecting human response error commission from EEG data for which univariate source processes such as event-related potentials (ERPs) have been employed in the past [4]. However, to our knowledge, effective connectivity features have not been used in this context. To this end we used dDTF estimates between channels FP1, FP2, FCz, C3, C4, PO3 and PO4 selected after 64-channel data cleaning. Analysis was performed on epochs at −0.5 to 1.5 seconds relative to button press events and time-frequency dDTF was computed in a 1-sec sliding window within each epoch. We performed a 5-fold blockwise cross-validation with clean separation of testing and training data to measure AUC on 172 trials, yielding a mean AUC of 0.895 +/− 0.047. In order to compare with more conventional features, we also performed 5-fold classification using a state-of-the-art first-order ERP method [11]. Here we obtained a mean AUC of 0.97 +/− 0.02. Given the saliency of error-related ERP features in this dataset, it is not surprising that the ERP-based method performs extremely well, outperforming higher-dimensional connectivity features.

IV. Conclusions and Future Work

The presented system is capable of real-time analysis. On an Intel i7 4-core (2.3 Ghz) laptop, preprocessing and source reconstruction (3751 vertex mesh) typically takes 50–80 ms, model fitting 50–70 ms, classification under 5 ms and (optional) MATLAB-based visualization 200–300 ms. While it is promising that channel connectivity features contain information relevant to error detection, it will be fruitful to examine source connectivity as well attempt prediction of error commission from pre-stimulus dynamics where time-domain ERP features are absent.