Cognitive behavior classification from scalp EEG signals

Abstract

The electroencephalography (EEG) has become increasingly valuable outside of its traditional use in neurology for neuropsychiatric diagnosis, neurological evaluation of traumatic brain injury, neurotherapy, gaming, neurofeedback, mindfulness and cognitive enhancement training. The trend to increase the number of EEG electrodes, the development of novel analytical methods and the availability of large datasets has created a data analysis challenge to find the “signal of interest” that conveys the most information about ongoing cognitive effort. Accordingly, we compare three common types of neural synchrony measures that are applied to EEG - power analysis, phase locking and phase-amplitude coupling to assess which analytical measure provides the best separation between EEG signals that were recorded while healthy subjects performed eight cognitive tasks - Hopkins Verbal Learning Test and its delayed version, Stroop Test, Symbol Digit Modality Test, Controlled Oral Word Association Test, Trail Marking Test, Digit Span Test and Benton Visual Retention Test. We find that of the three analytical methods, phase-amplitude coupling, specifically theta (4–7 Hz) – high gamma (70–90 Hz) obtained from frontal and parietal EEG electrodes provides both the largest separation between EEG during cognitive tasks and also the highest classification accuracy between pairs of tasks. We also find that phase-locking analysis provides the most distinct clustering of tasks based on their utilization of long-term memory. Finally, we show that phase-amplitude coupling is the least sensitive to contamination by intense jaw-clenching muscle artifact.

I. Introduction

Electroencephalography (EEG) is receiving increased interest because it is a readily accessible non-invasive tool for assessing brain function with extraordinary temporal resolution that is practically on the scale of milliseconds [1]. The use of EEG is now extending from the traditional application in neurological diagnosis to psychiatric diagnosis [2], neurological evaluation of traumatic brain injury [3], neurotherapy [4] and brain-machine interface applications [5]. During the last two decades, EEG devices have reduced substantially in their size and cost, mainly due to the availability of commercial multichannel analog front-end integrated circuits, which have significantly reduced the skill, time, and cost needed to develop a new EEG device [6]. New headsets with new types of electrodes, coupled with innovations in analog and digital electronic design have also converged to substantially simplify the EEG setup which has also contributed to making high quality EEG technology more accessible for both clinical and experimental applications [7], [8]. Indeed, there has been a recent proliferation of inexpensive EEG devices and systems designed for non-clinical use and a simple internet search will turn up many claims that EEG is capable of assessing and tracking brain function for widely-varying applications that include gaming, neurofeedback for pain, trauma, and wellness management, also management of cannabis use, mindfulness and cognitive enhancement training, to cite a few [9]–[11].

Advances in EEG hardware also require advancements in EEG analysis tools and algorithms, which too is experiencing substantial growth, much of it pioneered in the use of novel analyses of intracranial local field potential signals that underlie the scalp EEG [12]–[14]. Despite the advances, answers to one of the most fundamental and crucial questions remain uncertain – what are the signals of interest in the EEG for classifying diverse cognitive efforts; should classification and decoding focus on information theory measures, spectral power, synchrony, or some combination? Whatever the answer, it will dictate what EEG electrodes, active electronic hardware, and operating characteristics are to be used for EEG data collection as well as dictate which EEG analyses convey the most information about the functional state of the brain. Intuitively, one might argue that more is better and therefore the highest number of electrodes combined with the most sophisticated data analysis will provide the most information. Despite the intuitive appeal of “more is better,” it is not generally true as there are also several drawbacks with this approach. First, a large number of recording channels increases the dimensionality a.k.a. the “curse of dimensionality” [15] of the data and therefore introduces problems with dimensionality reduction and feature selection [16]. There is no point in increasing the number of data samples if the information is redundant. Second, more electrodes require more time to setup, as well as might require more complex data fidelity monitoring such as impedance checking. Third, portable systems and personal computing systems rely on battery power and therefore heavy processing can significantly reduce battery life, and the practical duration of EEG recordings if data logging and the bandwidth of data collection capacity are issues, when real-time analysis and/or portable applications are the target. Finally, EEG has traditionally ignored fast frequencies in the 30–100 Hz gamma band and in fact signals above 70 Hz are standardly filtered out by the hardware and so not collected because muscle activity generates contaminating signals in this frequency range [17], whereas intrinsic gamma band brain signals carry substantial information about fundamental neural dynamics [18] and internally generated cognitive events [19], [20]. It is consequently of substantial importance to investigate gamma frequency components of the EEG that are decisively not of muscle or other extra-cerebral origin.

What indeed are the signals of interest in the EEG? Here we investigate this question as it pertains to the brain states that generate distinct cognitive abilities. Previously, EEG classification of brain states using spectral properties was used to identify alertness [21] and mental fatigue [22]. Theta and alpha oscillations recorded from scalp have been positively correlated with ability to encode new information and long-term semantic memory performance, respectively [23]. Furthermore, activity in the theta band was associated with working memory maintenance, where theta power parametrically increases with number of items held in the working memory [24], [for review see 25]. Increased gamma band activity compared to baseline was observed in expectancy, learning, reading and subtraction tasks [26].

Instead of blindly searching for the best analytical method for EEG classification from the myriads of available choices, we decided to follow a theory-driven approach and instead focused on specific measures of synchrony, which are directly linked to theories of brain function and cortical computation. The “communication by coherence” hypothesis [27] proposes that neural synchrony allows flexible cortical organization and communication by precise temporal synchronization of spiking of excitatory neurons. This form of synchronization can be observed both locally as increased oscillatory activity and globally as an increase in long-range phase synchronization of neural oscillations [28]. A special case of synchrony –cross-frequency coupling, which occurs between slow and fast neural oscillations has been proposed to play a role in the segmentation of information where fast oscillations, modulated by the phase of slow oscillations provide “time slots” for different types of neuronal representations involved in neural computation [29]. Following those theoretical frameworks, we investigated the discriminative ability of three classes of synchrony measures: 1) the power in the traditionally-defined frequency bands, corresponding to local synchronization, 2) the band-specific phase synchronization between electrode sites, corresponding to long-range synchronization and 3) the cross-frequency coupling between the phases of slower oscillations and the amplitude of faster oscillations at each electrode site. The goal of the current investigation is to determine what signals in the EEG best discriminate the different tasks, presumed to depend on different states of brain function.

II. Materials and Methods

A. Subjects and Experimental Design

The studies were approved by the NYU Institutional Review Board, and written informed consent was obtained from all the volunteer subjects. There were 92 subjects recruited from the NYU student population. The average age was 22.2 ± 4.0 years and 56 subjects were female. The EEGs were acquired as a part of a study of the effect of exercise on cognitive functions and the behavioral results have been published [30]. The present study examined EEG recordings from the subjects in the pre-intervention condition. Each subject was asked to perform standard neuropsychological tests while EEG was recorded in otherwise constant conditions, generating a database of EEGs collected during distinctive cognitive tasks. The key assumption was that because the different cognitive tasks require different brain functions the EEG features during the different tasks would differ. The extent to which information in the EEG can distinguish which EEG was from which task provides a basis upon which to judge what information from the EEG is most useful for discriminating the task. The beginning and end of each cognitive task was recorded as an key-press event and stored together with the EEG data. EEG was acquired using a commercial, easy-to-use FDA-approved wireless, wearable, 24-channel EEG system (microEEG®, Bio-Signal Group Corp., MA, USA; [7], [8] connected to the Electro-cap (Eaton, Ohio, USA) headset with 16 recorded electrodes in standard 10–20 electrode placement, without extraocular electrodes (Fig. 1A). EEG was sampled at 250 Hz, transmitted to a laptop computer using a Bluetooth wireless protocol and stored for offline analysis. All signals were referenced to the Cz electrode with system ground connected to the Fpz electrode. Impedance was continuously monitored during data acquisition, and lowered as necessary. Each recording lasted 50 minutes on average.

Fig. 1.

Fundamental data collection and analysis. (A) Example EEG traces from 16 scalp electrodes while a subject was sitting still with eyes closed (BASE, black) or performing the Hopkins Verbal Learning Test (HVLT, red). (B) Example power spectrum (O1 electrode) is shown for BASE (black) and HVLT (red) with the five bands of interest outlined in blue (theta, alpha, beta, low gamma and high gamma). (C) Example PLV (between O1–O2 electrode pair) with five bands of interest outlined in blue. (D) Example PAC comodulogram (O1 electrode) computed for the full range of frequencies with the five combinations of slow and fast frequencies used for PAC estimation (left: alpha-low gamma, alpha-high gamma; right: theta-beta, theta-low gamma, theta-high gamma; white rectangles). Note the overlap in the beta and low gamma frequency ranges for PAC analysis, which occur because the amplitude filters are 20 Hz wide and beta is defined as 15–35 Hz and low gamma as 30–50 Hz.

EEG was preprocessed using a custom artifact rejection algorithm, which excluded segments of the signal with extremely low variance (slow-changing signals during impedance measurements) or extremely high variance (high amplitude artifacts caused by movement or excessive muscle activity). The variance threshold was set manually based on the visual observation of EEG signals. Ocular artifacts were identified using two methods based on independent component analysis (ICA). The component rejection method used manual identification of ICA activations with the strongest ocular artifacts and excluded these components before projecting the data back to the original space. The epoch rejection method used ICA activations only to identify moments of ocular artifacts using a manually set threshold, we then excluded those time segments from the analyzed data. For all analyses, EEG features were independently z-score normalized, to exclude amplitude differences due to variation between individuals and electrode placement.

B. Neuropsychological Tests

We recorded the scalp EEG from healthy undergraduate volunteers as they performed a series of neuropsychological tests, selected to assess a range of functions believed to be critically reliant on activity of either the prefrontal cortex or the hippocampus. The EEG of each subject was recorded in either a standard conference room or classroom, while the subject performed eight cognitive tasks as well as during a baseline recording. Each subject performed the tasks in the same order. Standard precautions to limit external noise sources and disturbances were taken although the goal was to simulate a real-world environment, rather than tightly controlled laboratory settings. During the baseline recording, the subjects were sitting still with eyes closed. In the Hopkins Verbal Learning Test [HVLT; 31] of verbal short-term memory, subjects had to recall and say as many words they can after the list of words was read to them. In the delayed version of the HVLT task (DHVLT), to assess short-term verbal memory recall across a delay, approximately 20 min later, at the end of the testing session, the subjects had to recall as many words as possible from the original HVLT list. During the Stroop Test [ST; 32] of cognitive control, subjects first had to read a list of words printed on paper in black ink. Next, the subjects went through a list of “XXXX’s” and reported each color. Finally, they went through a list of words printed in colored ink but were only to report the color of the letters and ignore the meaning of the word. In the Symbol Digit Modality Test [SDT; 33] to assess overall cognitive function including divided visual attention, the subject has to rapidly learn and match symbol-number pairs by drawing on paper. In the Controlled Oral Word Association Test [COWAT; 34] to assess verbal fluency, the subject is given a letter and told to say as many words as possible that start with the letter. In the Trail Marking Test [TMT; 35] that assesses mental flexibility and executive function, the subject uses a pen to connect randomly placed numbers on a paper with a line so they would form a monotonically increasing sequence (numeric and alphanumeric versions were used). In the Digit Span Test [DST; 36] that assesses short-term memory, the subject is told to repeat lists of numbers both in a forward and then backward direction. In the Benton Visual Retention Test [MODBENT; 37] to assess visual perception and memory, the subject must select using a computer mouse which out of four figure designs best matches the one s/he had seen before. All neuropsychological tests were administered during comparable periods of time (HVLT: 153 ± 92s, ST: 241 ± 39s, SDT: 192 ± 37s, COWAT: 251 ± 25s, TMT: 203 ± 63s, DST: 242 ± 60s, DHVLT: 97 ± 22, MODBENT 967 ± 107s) with baseline being recorded for 678 ± 384s.

C. Measures of EEG Synchrony for Classification

EEG was quantified using three measures of synchrony. The power spectrum (Fig. 1B), is perhaps the most standard EEG analysis and measures local amplitude synchrony resulting from local synchronized post-synaptic potentials. Instantaneous power was computed by convolving the EEG signal with a group of complex Morlet wavelets [38] in the range of 2–100 Hz with step of 1 Hz and squaring the result. The Morlet wavelet is defined as $w(t,f_0)=\mathit{Aexp}\left(-\frac{t^2}{2{\sigma }_t^2}\right)\mathit{exp}(2i\pi f_{0}t)$, with f₀ denoting the center frequency and σ_t and σ _f denoting standard deviation in the time and frequency domains, respectively, where σ_f = 1/2πσ_t. Normalization factor $A={({\sigma }_t\sqrt{\pi })}^{-1/2}$ ensures that the total wavelet energy is 1, and $\left(\frac{f_0}{{\sigma }_f}\right)~7$ practically [38]. POWER estimates were then averaged for each of 16 EEG channels in 5 bands of interest – theta (4–7 Hz) [23], alpha (9–12 Hz) [23], beta (15–25 Hz) [39], low gamma (30–50 Hz) and high gamma (70–90 Hz) [40] leading to a total of 80 power estimates for each subject and cognitive task (Fig. 2A).

Fig. 2.

The task-specific EEG dataset. Across-subject averages of activity vectors during eight cognitive tasks and baseline. (A) Power was computed at 16 electrode sites and 5 EEG bands (theta, alpha, beta, low gamma and high gamma). (B) PLV was calculated for 16 pairs of electrodes and 5 EEG bands (theta, alpha, beta, low gamma and high gamma). (C) PAC was computed for 16 electrodes and 5 cross-frequency interactions (theta-beta, theta-low gamma, theta-high gamma, alpha-low gamma and alpha-high gamma).

Phase locking value (PLV; Fig. 1C) [41] is a measure of non-local phase synchrony independent of amplitude synchrony. PLV was computed between a pair of electrodes to estimate inter-area phase synchrony. Here we estimated PLV between three classes of electrode pairs – intra-hemispheric synchrony (F7–T5, F3–P3, F4–P4, F8–T6, Fp1-O1, Fp2-O2), symmetric inter-hemispheric synchrony (Fp1-Fp2, F7–F8, F3–F4, T3–T4, T5–T6, O1–O2) and asymmetric inter-hemispheric synchrony (Fp1-O2, Fp2-O1, F7–T6, F8–T5). Instantaneous phase was computed by filtering a pair of EEG signals using band-pass FIR filters designed using Matlab ‘fdesign’ function using Kaiser window with 3-Hz bandwidth, 40 dB attenuation and 2 Hz transition band in the range 4–91 Hz and taking an argument of a Hilbert-transformed band-pass filtered signal. The PLV value was calculated as $\mathit{PLV}=\frac{1}{T}\left|{\sum }_{t=1}^T\mathit{exp}(j({\phi }_1(t)-{\phi }_2(t))\right|$, where t corresponds to time samples, j is the imaginary unit, T is the total length of the signal in samples, and φ₁ (t) – φ₂ (t) corresponds to the instantaneous phase difference between the two signals. The band-specific phase locking value was then extracted for theta using the 4–7 Hz band, alpha using the 9–12 Hz band, beta by averaging PLV values of 3 Hz wide bands within the range 15–30 Hz, low gamma by averaging PLV values in the 30–48 Hz range and high gamma by averaging PLV values in the 70–91 Hz range. The combination of 16 measures of synchrony and 5 bands of interest also lead to 80 PLV estimates for each subject and cognitive task (Fig. 2B).

Weighted phase lag index (WPLI) [13] was used as an alternative measure of phase synchrony because of its lower sensitivity to volume conduction. We used FieldTrip [42] implementation of debiased WPLI using 5-second long window and 9 tapers. Obtained WPLI estimates were averaged across same bands as used for PLV.

Phase-amplitude coupling (PAC; Fig. 1D) [12], [43] is a measure of synchronization between the phase of low frequency oscillations (theta or alpha) and the amplitude of high frequency oscillations (beta, low gamma or high gamma). PAC provides an estimate of local cross-frequency synchrony between frequency bands that are believed to have distinctive biophysical origins [18]. Phases of low frequency oscillations were extracted using two band-pass FIR filters with 3-Hz bandwidth either at the theta (4–7 Hz) or alpha (9–12 Hz) bands followed by the Hilbert transform. Amplitudes of high frequency oscillations were computed using three band-pass FIR filters with 20-Hz bandwidth either in beta (15–35 Hz), low gamma (30–50 Hz) or high gamma (70–90 Hz) followed by the Hilbert transform. All FIR filters were designed using Matlab ‘fdesign’ function using Kaiser window with 40 dB attenuation and 2 Hz transition band. The use of narrow-band filters for estimating modulatory phase and wide-band filters for estimating the modulated amplitude is crucial for unbiased PAC estimation [14]. The measure of PAC, the modulation index (MI), was computed as the Kullback-Leibler divergence (KLD) between the phase-amplitude distribution obtained from band-pass filtered signals and a uniform distribution [12]. PAC was estimated for all 16 electrodes and 5 combinations of bands (theta-beta, theta-low gamma, theta-high gamma, alpha-low gamma, alpha-high gamma; Fig. 1D), leading to the total of 80 PAC estimates for each subject and cognitive task (Fig. 2C).

Support vector machine classification was performed using the LIBSVM library [44]. In order to set parameters for the support vector machine, we used a parametric grid search combined with 5-fold cross-validation applied to the entire dataset (POWER + PLV + PAC features). The accuracy of classification was then tested separately for each pair of tasks and for each individual analysis type (POWER, PLV and PAC) as well as for their combination (POWER+PLV+PAC). Each classifier was first trained on a randomly selected 75% sample of the data and then tested on the remaining 25% of the data in order to estimate classification accuracy. Training and classification was repeated 10 times and the accuracy results were averaged.

Principal component and clustering analyses were performed using JMP (Cary, NC, USA) statistical software. All other analyses were performed using custom written Matlab (Natick, MA, USA) software.

III. Results

A. PAC Provides the Largest Discrimination Between Tasks

We first asked which combinations of electrodes or electrode pairs, EEG bands or x-bands (cross-frequency bands used in PAC estimation, e.g. theta-beta) and analytical measures provide the largest overall discrimination of all tasks. Because numerous commercial applications are limited to the use of minimal EEG electrode montages and limited computing power for real-time feedback, we examined the discriminative power of small numbers of electrodes or electrode pairs (used in PLV estimates, e.g. O1–O2) and frequency bands or x-bands. We selected all possible combinations of 1 electrode/electrode pair and 1 band/x-band, 1 electrode/electrode pair and 5 bands/x-bands, 4 electrodes/electrode pairs and 1 band/x-band and 4 electrodes/electrode pairs and 5 bands/x-bands and computed the average Euclidean distance between all tasks for the power (Fig. 2A), PLV (Fig. 2B) and PAC (Fig. 2C) analytical measures. These results are presented as a heat map interpolated over a head schematic for power and PAC measures and color connections of electrode pairs in order to demonstrate which electrodes/electrode pairs provide for the largest discrimination. For combinations where only 1 band/x-band was selected, we also computed the average Euclidean distance for each band to see which band provides the largest discrimination. Additionally, we present distances between tasks as histograms for all electrode and bands/x-bands combinations (Fig. 3D). A summary of the electrodes/electrode pairs and bands that provide the best discrimination for a given analysis is provided in Table 1.

Fig. 3.

Discrimination using minimal montages and frequency bands. (A) Average Euclidean distance computed between the EEGs during all pairs of tasks using only the power data for all combinations of 1 electrode and 1 band (top, left), 1 electrode and 5 bands (bottom, left), 4 electrodes and 1 band (top, right) and 4 electrodes and 5 bands (bottom, right). The average distance for 5 bands of interest is shown for both options where only 1 band is selected. (B) Same as A, but for PLV (electrode pairs are used in PLV analysis instead of individual electrodes). (C) Same as B, but for PAC (cross-frequency bands x-bands are used in PAC analysis instead of regular frequency bands). (D) Histograms of distances between pairs of tasks for all combinations of electrodes and bands/x-bands.

TABLE I.

Summary of Electrodes/Electrode Pairs and Bands/x-Bands With the Best Discrimination. Summary of the Best 4 Electrodes/Electrode Pairs That Provide the Best Overall Discrimination of All Tasks When 1 Electrode and 1 Band/x-Band, 1 Electrode/Electrode Pair and 5 Bands/x-Bands, 4 Electrodes/Electrode Pairs and 1 Band/x-Band and 4 Electrodes/Electrode Pairs and 5 Bands/x-Bands Are Selected Using the Power, PLV and PAC Measures. For Combinations Where Only a Single Band/x-Band Was Used we Also Provided in Parenthesis the Frequency Band/x-Band, Which Provided the Largest Average Discrimination

	1 electrode 1 band/x-band	1 electrode 5 bands/x-bands	4 electrodes 1 band/x-band	4 electrodes 5 bands/x-bands
POWER	T6, T4, T5, 01 (α)	T6, P4, T5, P3	T6, T5, T4, P4 (α)	T6, P4, T5, P3
PLV	Fp2-02, F7–F8, Fp2-01, Fp1-Fp2 (α)	Fp2-02, F7–F8, Fp2-01, Fp1-Fp2	Fp2-02, F7–F8, Fp2-01, Fp1-Fp2 (α)	Fp2-02, F7–F8, Fp2-01, Fp1-Fp2
PAC	F8, F4, P4, F7 (θ-γH)	F8, F4, P4, F7	F8, F4, P4, F7 (θ-γH]	F8, F4, P4, F7

POWER (Fig. 2A, 3A) provides the lowest inter-task discrimination of the three analytical measures. When one electrode and one band are selected for the power analysis (Fig. 3A, top left), the largest discrimination is obtained with the temporal (T6, T4 and T5) and occipital (O1) electrodes in the alpha band (Fig. 3A middle, left). When a single electrode but all five bands are selected for the power analysis (Fig. 3A bottom, left), the best discrimination is provided by a symmetric combination of temporal (T6 and T5) and parietal (P4 and P3) electrodes. When four electrodes and one band are selected (Fig. 3A top, right), the largest discrimination is provided by the temporal (T6, T5 and T4) and parietal (P4) electrodes in the alpha band (Fig. 3A middle, right). For four electrodes and all five bands, the largest discrimination is provided by the symmetric combinations of the temporal (T6 and T5) and parietal (P4 and P3) electrodes.

Of the three analytical measures, PLV analysis (Fig. 2B, 3B) provides for medium discrimination. When one pair of electrodes and one band are selected for the PLV analysis (Fig. 3B top, left), the best discrimination is provided by the intrahemispheric Fp2-O2 and inter-hemispheric F7–F8, Fp2-O1 and Fp1-Fp2 electrode pairs in the alpha band (Fig. 3B middle, left). This pattern was the same for 1 electrode pair and 5 bands (Fig. 3B bottom, left) as well as for 4 electrode pairs and 1 band (Fig. 3B top, right) and 4 electrode pairs and 5 bands (Fig. 3B bottom, right) although dominance of the alpha band for the selection of 4 electrode pairs and 1 band was less apparent (Fig. 3B middle, right).

Of the three analytical measures, PAC (Fig. 2C, 3C) provides for the maximal discrimination. For 1 electrode and 1 x-band (Fig. 3C top, left), the best discrimination is provided by the F8, F4, P4 and F7 electrodes in the theta-high gamma x-band (Fig. 3C middle, left). The same pattern can be seen when 1 electrode and all five x-bands are selected (Fig. 3C bottom, left) as well as when 4 electrodes and one x-band (Fig. 3C top, right) and four electrodes and all five x-bands (Fig. 3C bottom, right) are selected although dominance of the theta-high gamma band for the selection of 4 electrodes and 1 x-band was less apparent (Fig. 3C middle, right).

B. PAC Provides the Largest Overall Task Classification Accuracy

We next tested how well the individual tasks can be classified based on a selected set of measures (POWER, PLV or PAC) or their aggregate (Fig. 4). For this analysis, we used the full 80-D datasets with all bands of interests and the full set of electrodes. The best overall prediction accuracy (82.3 %) was obtained by the aggregate dataset where all three measures (POWER+PLV+PAC) were combined into a 240-D dataset (Fig. 4A bottom, right). PAC showed the second highest overall accuracy (78.8%; Fig. 4A bottom, left). In contrast, POWER showed the lowest overall accuracy (65.3%; Fig. 4A top, left) with PLV somewhere in the middle (75.4%; Fig. 4A top, right). Not surprisingly, the accuracy of classification of the baseline task from other tasks was highest for all datasets, reaching 95%. The PLV dataset provided for better classification accuracy than PAC in the ST, SDMT and TMT tasks (Fig. 4B).

Fig. 4.

Task classification accuracy. (A) Classification accuracy between pairs of tasks when using only the POWER dataset (top, left), PLV dataset (top, right), PAC dataset (bottom, left) and the combined POWER + PLV + PAC dataset (bottom, right). (B) The average classification accuracy for all tasks for POWER dataset (green), PLV dataset (blue), PAC dataset (red) and combined dataset (gray).

C. Tasks Form Distinct Clusters in Principal Component Space

Next we asked if the tasks could be organized into subgroups based on the features we extracted from the EEG. We started by taking the full 240-D dataset (POWER + PLV + PAC) and performed principal component analysis (PCA). Averages of task-specific vectors formed three distinct clusters when plot in the principal component space (Fig. 5A). Baseline appeared isolated from the rest of tasks, which formed two clusters. The first cluster was formed from the TMS, SDMT and ST tasks and the second cluster was formed by the MODBENT, DST, HVLT, COWAT and DHVLT tasks. While distinctiveness of the Baseline task is not surprising, separation into two other clusters is intriguing because all tasks in the cluster formed by the MODBENT, DST, HVLT, COWAT and DHVLT procedures are mnemonic tasks, while the other three tasks are not. We then computed PCA for separate POWER, PLV and PCA datasets and plotted averages of the task-specific activity vectors in the principal component space separately for each dataset (Fig. 5B–D). While clustering of tasks in the POWER (Fig. 5B) and PAC (Fig. 5D) dataset was not very obvious, tasks formed very clear clustering in the PLV (Fig. 5C) dataset while the PAC dataset provided the largest discrimination of individual tasks.

Fig. 5.

Task-specific EEG measures form clusters in PCA space. (A) Clustering of tasks in the PCA space of the combined (POWER + PLV+ PAC) dataset with dendrogram in the inset. Explained variance of first (horizontal axis) and second (vertical axis) components shown in axis labels. (B) Clustering of tasks in the PCA space for POWER dataset. (C) Clustering of tasks in the PCA space for PLV dataset. (D) Clustering of tasks in the PCA space for PAC dataset.

D. Rejection of Ocular Artifacts and Improved Phase Synchrony Measure Does Not Improve Classification

We next tested the possible impact of ocular artifacts on the classification accuracy (Fig. 6). We used two ICA-based methods for rejection of ocular artifacts. The component rejection method (see Materials and methods for more details), provided lower classification accuracy; it was lower for every task in all three datasets (27/27 test of proportions z = 5.19, p = 10⁻⁷). By inspection, the largest impact was on PLV dataset (Fig. 6B). This was likely caused by the disruption of phase synchronization that occurred when asymmetrical amounts of components were removed from pairs of signals used in PLV analysis. The epoch rejection method led to comparable classification accuracy as without applying any ocular artifact rejection.

Fig. 6.

Ocular artifacts and phase estimation methods. (A) Classification accuracy of POWER dataset without ocular artifact rejection (yellow), with ICA-based component rejection method (blue) and ICA-based epoch rejection method (green). (B) Same as A but for PLV dataset. (C) Same as A but for PAC dataset. (D) Classification accuracy using phase locking value (PLV, blue) and weighted phase lag index (WPLI, purple).

We then tested a potentially improved measure for phase synchrony with lower sensitivity to volume conduction, namely the weighted phase lag index (WPLI, see Materials and methods for more details). The classification accuracy using WPLI was significantly lower than using PLV; it was lower for every task (9/9 test of proportions z = 3.0, p = 0.004; Fig. 6D).

E. PAC Is the Least Contaminated With Muscle Artifact

Because the frequency content of muscle artifacts overlaps with high-frequency EEG bands (β, γ) and because muscle activity is expected to be the highest in tasks that especially involve speech, we decided to explore the effect of exaggerated muscle activity on POWER, PLV and PAC estimates. We took advantage of a published EEG dataset [45] recorded in two conditions, first when the subject was sitting relaxed with his or her eyes closed and second, when the subject was actively clenching his or her jaws with eyes closed. The power and the modulation index, respectively, were estimated from individual electrodes while PLV estimates were computed between contralateral electrode pairs (Fp1-Fp2, O1–O2, F7–F8, T7–T8, P7–P8, C3–C4, F3–F4, P3–P4). All estimates were computed from 30-s periods of rest or jaw clenching and the ratio between jaw clenching and rest were computed. The power ratio was log-transformed. The same algorithms were used as in the first part of the study (Fig. 1).

In the case of POWER (Fig. 7A), the ratios were progressively more biased by jaw clenching with increasing frequency. The largest bias was present at the frontal and temporal electrodes, which lie directly above the temporalis muscle. Signal power at the fronto-polar, parietal and occipital electrodes were mostly in the beta and gamma bands. In the case of PLV (Fig. 7B), which was estimated between symmetric inter-hemispheric pairs of electrodes, all estimates except those estimated from parietal and occipital electrodes were strongly biased in the gamma band. Finally, PAC (Fig. 7C), which was estimated as the modulation index between the theta-low gamma, theta-high gamma, alpha-low gamma and alpha-high gamma bands showed strong bias at frontal and temporal electrodes, with low bias present at fronto-polar, parietal and occipital electrodes. Stronger than anticipated bias in the modulation index at the frontal and temporal electrodes was most likely caused by the broadband profile of the EMG activity, which extended from the gamma band well into the alpha and even the theta bands (see Fig. 7A).

Fig. 7.

Sensitivity to muscle artifact. (A) Log ratio of power estimated during jaw clenching to power estimated during rest for five bands of interest. (B) Ratio of PLV estimated between contralateral electrode pairs during jaw clenching and rest for five bands of interest. (C) Ratio of MI estimated during jaw clenching and rest for theta-low gamma, theta-high gamma, alpha-low gamma and alpha-high gamma.

IV. Discussion

We compared three types of synchrony measures (Fig. 1, 2) in their ability to discriminate EEG signals recorded during eight cognitive tasks and rest. The band-specific power estimate provided the lowest discrimination between tasks (Fig. 3) while PAC provided the highest discrimination with PLV between the two. It should be noted that PAC and PLV measures aggregate more information than the power measure, because they analyze signal relationships between two electrodes (PLV) or between two frequency bands (PAC). The best discrimination using power estimates was obtained from the temporal and parietal electrodes with a clear dominance of the alpha band (Table 1). PLV provided the best discrimination with combinations of fronto-polar and occipital, frontal and fronto-polar electrode pairs with noticeable alpha band dominance. PAC provided the largest discrimination using the frontal and parietal electrodes with theta-high gamma dominance. In summary, these findings suggest that each synchrony measure points to different electrode locations for highest discrimination although some overlaps can be found for frontal (F7, F8; PLV and PAC), parietal (P4; POWER and PAC) and occipital (O1, POWER and PLV) electrodes. Notably, when modifying the number of selected electrodes/electrode pairs and bands/x-bands, PLV and PAC measures showed an extremely consistent selection of these features pointing to the robustness of the signal providing the discrimination at these electrodes. Finally, we demonstrate that by using relationships to oscillations that are comfortably in the frequency range of brain oscillations such as theta without contamination by muscle artifact, it is possible to utilize information from gamma frequency oscillations for characterizing and discriminating between states of EEG.

Similar results were obtained when we estimated task discrimination by computing the classification accuracy between pairs of tasks (Fig. 4). Here, the combined dataset (POWER + PLV + PAC) provided the highest overall classification accuracy, followed by PAC and PLV with the POWER dataset providing the lowest accuracy. Interestingly, PLV provided higher classification accuracy in executive function tasks that do not require long-term memory (ST – Stroop test, SDMT – symbol digit modalities, TMT – trail marking test) but presumably especially requires coordination between distant brain areas.

The tasks that showed the best classification accuracy from EEG signals (Fig. 4) also appeared as distinct clusters when high-dimensional EEG feature vectors were plotted in the PCA space (Fig. 5). The baseline condition appeared as a clearly separated cluster with a second cluster formed by the EEGs during the Stroop (ST), symbol digit modalities (SDMT) and trail marking tests (TMT). A third cluster was formed by the EEGs during the remaining tasks, which all require active use of long-term memory. When considering each of the synchrony measures separately, PLV provided the highest intergroup discrimination (Fig. 5D) while PAC provided the highest inter-task discrimination (Fig. 5E). While the distinctiveness of the baseline condition is not surprising, the separation of the task-associated EEG patterns into two other clusters is intriguing because they separate based on the requirement for memory rather than on the basis of functional anatomy. Specifically, all tasks in the MODBENT, DST, HVLT, COWAT and DHVLT cluster make demands on memory but components of only two of these tasks (MODBENT, DHVLT) are known to depend on hippocampus function [46]. In contrast, all the tasks in the other cluster tax executive functions including attention and cognitive flexibility (TMT), visuomotor processing (SDMT) and attention and cognitive control/response inhibition (ST). It is also notable that the two task clusters were most clearly separated by the PLV measure, specifically by the second principal component, which might point to the long-range inter-areal brain synchrony that may be required for performing these tasks effectively. Consistent with this view, in a study that analyzed processing speed in the symbol-digit modalities test it was found that increased speed resulted in an increased level and number of functional networks as well as increased connectivity within the fronto-parietal and fronto-occipital networks that are presumably required for effective information transfer between the distant frontal and parietal cortices [47]. A different study demonstrated a relationship between performance in the trail making test (TMT) and anatomical coupling between prefrontal cortex and distributed cortical regions [48]. Several other studies reported lower performance on the symbol digit modalities test (SDMT) is associated with decreased long-range synchrony in the sensorimotor and dorsal attention networks in Huntington disease and multiple sclerosis [49], [50]. On the other hand, PAC analysis appeared to distinguish the hippocampus-dependent tasks (MODBENT and DHVLT) from the remaining tasks, which might point to the role of cross-frequency coupling in memory processes [51], [52] and the mechanistic role of PAC in hippocampus information processing, although these signals in the scalp EEG cannot be localized to anatomical sources.

The present goal was to determine whether and to what extent it is possible to use signals in the EEG to classify what task the subject was doing during the EEG. While the results suggest that, at least in principle, it may be possible to use the EEG itself to define and characterize states or modes of brain function, it remains to be demonstrated that the brain signals in the EEG are alone sufficient for the discrimination, because signals of other origins like ocular and muscular signals that are differentially biased across the tasks could have contributed signals to the EEG that were used for the discriminations.

One shortcoming of the present study was the lack of control over the duration of each task as well as a precise timing of the individual presentations of cognitive tasks synchronized with EEG recording. It is therefore possible that some of the observed effects can in principle be explained by differences in stimulus properties during each cognitive task. While we acknowledge this limitation, we also want to stress that the major conclusion of the study was that PAC appears most informative in task separation and classification accuracy and it is least sensitive to intense muscle artifacts. In those comparisons, PAC was compared against PLV and POWER measures using the same data, therefore with the same differences between tasks. Related to the previous limitation, are differences in responding to different task by means of speaking (HVLT, dHVLT, DST, COWAT, ST) or writing (SDT, TMT) as well as no output (Baseline). Indeed, SDT and TMT tasks formed similar clusters (Fig. 5), although the same cluster was also shared with the ST task, which was performed without written output, indicating that the clustering cannot be merely due to the writing component.

Finally, we analyzed each synchrony measure’s sensitivity to muscle artifacts because this is especially important for EEG mobile applications that are impractical in an EEG laboratory. Estimates of structural power showed a strong positive bias with muscle activity >10 Hz at all electrodes with PLV showing a positive bias >20 Hz at all sites except the parietal electrodes. PAC showed very low bias in the fronto-polar, parietal and occipital electrodes and wide-band bias at frontal and temporal electrodes. While as expected, the lowest impact of muscle activity on the PAC measure was demonstrated here, we did not expect such a strong impact on PAC estimated from the frontal and temporal electrodes (electrodes closest to the temporalis muscle). This contamination is clearly caused by biasing not only the high-frequency amplitude but also the low-frequency phase (compare Fig. 7C and Fig. 7A). One possible explanation for the strong bias of PAC in the high frequency range might be sharp EEG waveshapes, which might be decomposed into broad-band spectral power appearing as PAC in the high frequency range [53]. It is important to note that while muscle activity is generally considered to be ‘noise’ in the EEG recordings, it along with ocular and other artifacts can still potentially provide important information for task discrimination.

Because it was possible to discriminate amongst the tasks that were being performed during the EEG recordings, the findings and analyses presented here point to the possibility that cognitive states can themselves be discriminated from information in the standard scalp EEG, and that significant discriminative information is in the inter-relations between EEG signals at distinct electrode sites and distinct frequency bands, that include the often excluded gamma band. Based on this information, future efforts to design applications and diagnostics based on EEG should be motivated in their search for optimal discriminative measures of brain functional states, to explore these and additional inter-relations such as cross-site PAC where the frequency for phase would be obtained from one electrode and the frequency for amplitude obtained from another electrode.

V. Conclusions

We conclude that standard scalp EEG, in application settings outside of the laboratory, can provide sufficient information to discriminate amongst internally-distinct cognitive behaviors, and that estimates of neural synchrony in cross-frequency and cross-electrode measures such as PAC and PLV provide more discriminative information than estimates of power.

Biographies

Dino Dvorak received a M.Sc. degree in Biomedical engineering from the Czech Technical University in Prague, Czech Republic in 2001 and a Ph.D. degree in Biomedical engineering from State University of New York, Downstate Medical Center in Brooklyn, New York in 2015. Dino then joined Dr. Fenton’s Neurobiology of Cognition Laboratory at New York University as a postdoctoral associate. He investigates the role of brain rhythms in normal brain function and cognitive disorders such as schizophrenia and autism using advanced computational methods for functional brain imaging, data processing and biomedical instrumentation.

Andrea Shang received a B.Sc. degree in Psychology, History and German Studies at Smith College in Northampton, Massachusetts in 2010. Andrea is currently enrolled in a Ph.D. program in Behavioral and Systems Neuroscience at Rutgers University in New Brunswick, New Jersey.

Samah Baki received his MD at the American University of Beirut in 2006 and his internship in emergency medicine at State University of New York (SUNY) in 2008. Samah leads NIH-funded clinical trials that utilize neurologists at SUNY to interpret EEGs recorded from patients in multiple New York area neonatal intensive care units. With colleagues at SUNY, he discovered a drug pair that was unusually potent in treating cognitive deficits after traumatic brain injury. The US Army recognized it as one of the most outstanding research projects funded by the Congressionally Directed Medical Research Programs (CDMRP) in fiscal year 2007.

Wendy Suzuki received her B.Sc. in physiology and human anatomy at the U.C., Berkeley in 1987 and her Ph.D. in Neuroscience from U.C., San Diego in 1993.

Wendy Suzuki is a Professor of Neural Science and Psychology at New York University. She is best known for her studies on the anatomy, physiology and function of the brain areas critical for long-term memory. More recently she has started to examine the neuro-behavioral, physiological and neurochemical effects of physical aerobic activity on a wide range of brain and cognitive functions in people. She is also an award-winning teacher, scholar and author.

André A. Fenton received a B.Sc. degree in Biology from McGill University, Montréal, Canada in 1990 and a Ph.D. degree in Neural and Behavioral Science from the State University of New York, Downstate Medical Center, Brooklyn, New York in 1998. André is a Professor of Neural Science at New York University. His research focus is molecular, neural, behavioral, and computational aspects of memory. He studies how brains store experiences as memories, and how expressing knowledge activates relevant information without activating what is irrelevant. André founded Bio-Signal Group Corp., which commercialized an FDA-cleared platform for EEGs anywhere, anytime, for everyone.