Electrophysiological Decoding of Spatial and Color Processing in Human Prefrontal Cortex

Abstract

The prefrontal cortex (PFC) plays a pivotal role in goal-directed cognition, yet its representational code remains an open problem with decoding techniques ineffective in disentangling task-relevant variables from PFC. Here we applied regularized linear discriminant analysis to human scalp EEG data and were able to distinguish a mental-rotation task versus a color-perception task with 87% decoding accuracy. Dorsal and ventral areas in lateral PFC provided the dominant features dissociating the two tasks. Our findings show that EEG can reliably decode two independent task states from PFC and emphasize the PFC dorsal/ventral functional specificity in processing the where rotation task versus the what color task.

1. Introduction

The prefrontal cortex (PFC) is central to goal-directed cognition and has been implicated in encoding both stable and dynamic representations of task-relevant variables such as goals, rules, and rewards (Duncan, 2001; Fuster, 2013; Miller and Cohen, 2001; Rougier et al., 2005). Therefore, characterizing the representational code of PFC holds promise to open important new avenues in the study of decision-making, cognitive control, planning, and reasoning. Multivariate pattern analysis (MVPA) methods are crucial to achieving this goal (Norman et al., 2006). These methods, instead of focusing on individual signals (e.g. voxels), search for reproducible spatial patterns of activity that differentiate across experimental conditions. This is accomplished through the use of powerful machine learning classifiers that decode the information that is represented in activity patterns.

A growing number of studies with machine learning classifiers report reliable decoding of neural representations from multiple areas in the brain (Brouwer and Heeger, 2009; Harrison and Tong, 2009; Horikawa et al., 2013; Kamitani and Tong, 2005; Lemm et al., 2011; Muller et al., 2008; Wolpaw and Wolpaw, 2012). However, decoding PFC signals remains a challenge for both fMRI and electrophysiological non-invasive EEG measurements (Bhandari et al., 2018). Here, we assessed whether EEG can reliably measure representational information in PFC. Specifically, we investigated whether engagement in two distinct tasks, a mental-rotation and a color-perception task, could be predicted by EEG PFC features.

The mental-rotation and color-perception tasks used the same set of stimulus images, which displayed objects of varying spatial orientation and color, to control for sensory effects. We hypothesized that visual information from the primary visual cortex would propagate through the dorsal pathway processing “where” information preferentially for the rotation task (spatial manipulation) (Goodale and Milner, 1992; Hesse et al., 2014) and the ventral pathway processing “what” information preferentially for the color task (visual perception) (Kravitz et al., 2013; Mishkin et al., 1983), with subsequent representational patterns reflecting this dorsal/ventral organization decodable in PFC. To assess whether EEG signals could resolve these patterns, we localized EEG signals across major cortical areas spanning both PFC and non-PFC regions. We then extracted EEG features across canonical frequency bands and trained a regularized linear discriminant analysis classifier to discriminate between the two tasks. A systematic relationship between PFC signals and task performance would provide evidence for the role of PFC EEG features in encoding task-relevant information. In addition, the activation patterns detected by the classifier would provide insights into the representational format of the PFC.

2. Materials and methods

2.1. Participants

Twenty healthy individuals (age: 22.8 ± 2.6; 10 men and 10 women) participated in this study. The study was conducted in accordance with the ethical guidelines established by the Institutional Review Board of Korea University (No. 1040548-KU-IRB-14–28-A-2) and the Declaration of Helsinki (World Medical Association, 2013). Participants had normal or corrected-to-normal vision, and none were color-blind, as determined by the Ishihara color test. Participants provided informed consent prior to the study.

2.2. Materials and procedure

Pairs of stimuli randomly drawn from a set of red or green objects (Fig. 1) were presented bilaterally on a black background at an eccentricity of 5 ° visual angle on a computer monitor, which was placed in front of the participant at a distance of 65 cm. In the center of the monitor, a small gray fixation cross was presented. Each stimulus spanned 5 ° visual angle and was presented for 5 sec. Stimulus presentation was followed by a variable inter-stimulus interval ranging from 2.2 to 2.8 sec with a mean value of 2.5 sec. All types of stimuli appeared pseudo-randomly with equal probability.

Fig. 1. Mental-rotation and color-perception tasks.

Upper panel: Example stimuli of the mental-rotation task, with degrees of object-rotation ranging 0 °, 45 °, 90 °, and 135 ° clockwise, which progressively increase task-difficulty (difficulty 1 to 4, respectively). Lower panel: Example stimuli of the color-perception task, which varied saturation values of red and green to 100%, 50%, 10% and 5% (difficulty 1 to 4, respectively). Participants performed a two-alternate forced-choice task reporting whether the shapes of the two objects are identical when rotated (mental-rotation task) or the colors of the two objects are the same (color-perception task).

Using the same stimulus set, the present study consisted of two tasks, a mental-rotation and color-perception task, which were counterbalanced in their presented order and were designed to activate different PFC areas. An essential function of the lateral PFC [LPFC; including both dorsolateral PFC (DLPFC) and ventrolateral PFC (VLPFC)] is executive control functions such as volitional responses to achieve an intended goal (Fuster, 2008; Royall et al., 2002; Sarazin et al., 1998). Dorsal route activation (DLPFC; processing “where” information) was predicted during the mental-rotation task (Shepard and Metzler, 1971). In contrast, ventral route activation (VLPFC; processing “what” information) was predicted during the color-perception task. In the mental-rotation task, participants were instructed to determine whether the shapes of two images were identically matched when rotated (see Fig. 1). In the color-perception task, participants were instructed to determine whether the colors of two presented objects were the same. During these two tasks, participants were instructed to fix their eyes on a fixation dot presented on the center of monitor. Participants were also instructed to respond by pressing a button with one hand as quickly as possible whenever the two images were identical in the task-relevant feature of the images (shape or color, respectively) and to otherwise press another button with the opposite hand. Response hands were counterbalanced across participants. Each task comprised 4 blocks with a short break in between; each block included 80 trials. In each block, 4 levels of task-difficulty and 2 types of object-pair (identical vs. mirror-reflected) of 10 different shapes of objects were presented as stimuli in a random order. Participants underwent a training session to become familiar with the task before the experimental session.

Four different levels of the task difficulty were generated for each task. For the rotation task, the degrees of object-rotation were 0 °, 45 °, 90 °, and 135 ° clockwise, which is the order of the increasing task-difficulty (difficulty 1 to 4, respectively; see the top panel in Fig. 1 as examples). When the object is rotated more, its identification becomes more difficult. For the color-perception task, the saturation values of red and green varied from 100%, 50%, 10% and 5% (difficulty 1 to 4, respectively; see the bottom panel in Fig. 1 as examples). The isoluminant RGB values for red and green were used for the experiment. As the difference in saturation values between two presented objects decreased, identification became more difficult. Since an easy task would not effectively induce differences in brain activity between these two tasks, the easiest task-difficulty (i.e. difficulty 1) was not included for further analysis. Reaction times and accuracies of task-performance in each task-difficulty were measured for the behavioral analysis. Reaction times values outside of mean ± 1.98 SD for each individual were considered outliers and were discarded from further analyses.

2.3. EEG Acquisition

The EEG was measured using a BrainAmp DC amplifier (Brain Products, Germany) with 64 Ag/AgCl electrodes in an actiCAP (Brain Products, Germany) in accordance with the international 10–10 system. An electrode was placed on the tip of the nose as reference, and a ground electrode was placed at electrode AFz. Electrode impedances were maintained below 5 k Ω prior to data acquisition. The EEG was recorded at 500 Hz. For further analyses, EEG data were epoched from 500 ms prestimulus to 7000 ms poststimulus. Eye movement activity was monitored with an EOG electrode placed sub-orbitally to the left eye, and vertical and horizontal electro-ocular activity was computed using two pairs of electrodes placed vertically and horizontally with respect to both eyes (i.e. Fp1 and EOG for the vertical EOG, F7 and F8 for the horizontal EOG). All epochs were visually inspected for artifacts, and epochs containing eye movements or other artifacts (maximum amplitude ± 100 μV and maximal gradient voltage step 50 μV/ms) were automatically rejected from further analyses. Only the trials with correct responses were collected for further analysis. Three participants were excluded from further analyses because of poor data quality.

2.4. Data Analysis

We assessed whether a decoder could detect task engagement in humans using source-level EEG signals from 15 Brodmann areas (BAs) comprising cortical areas in both dorsal and ventral visual routes up to PFC. Figure 2 depicts a flowchart of the analytic procedure.

Fig. 2. Flowchart of analyses procedures.

Following source reconstruction, a sequence of processing steps extracted signals from 15 BAs by IDF-bandpass filtering, aggregated across voxels, applied Hilbert transform, and aggregated across time. Last, an rLDA classifier was used to compute decoding accuracies and activation patterns. BA: Brodmann area; IDF: individual dominant frequencies. rLDA: regularized linear discriminant analysis.

2.4.1. Source reconstruction

First, source-level cortical activity was estimated from the scalp EEG signals measured using exact low-resolution tomography analysis (eLORETA) (Pascual-Marqui, 1999; Pascual-Marqui, 2007; Pascual-Marqui et al., 2002; van der Loo et al., 2011). eLORETA outputs were modeled as a set of 2,394 voxels of 7 × 7 × 7 mm forming a 3D description of the cortex. The transformation criteria for source localization take advantage of a brain model based on the Talairach coordinate system included in the anatomical brain atlas (MNI-305) (Collins et al., 1994; Evans et al., 1993; Oakes et al., 2004; Talairach and Tournoux, 1988) and the international 10–10 system data. eLORETA provides more precise and accurate information because it has higher spatial resolution than sLORETA (Jatoi et al., 2014).

2.4.2. Extraction of signals in brain regions of interest

To investigate which brain areas encode task-related information, we selected the following 12 regions of interest (ROIs), which spanned the area of 15 BAs (Fig. 3). For lateral and anterior PFC, the DLPFC (BA 9 and 46), VLPFC (BA 45 and 47), and anterior prefrontal cortex (BA 10) were chosen. The orbitofrontal cortex (BA 11) and anterior cingulate cortex (BA 24) were selected for the orbital and medial frontal cortex, respectively. BA 5 and 40 were selected for the posterior parietal cortex (PPC) (Aflalo et al., 2015). The primary motor cortex (BA 4) and premotor area (BA 6) were selected to assess motor activity, while the primary visual cortex (BA 17), secondary visual cortex (BA 18), associative visual cortex (BA 19), and inferior temporal gyrus (BA 20) were analyzed for primary and extrastriate visual processing. All of these anatomical areas were delineated based on the atlas embedded in LORETA software (Pascual-Marqui et al., 2002).

Fig. 3. Cortical locations of 15 Brodmann areas.

The viewpoints of the cortical maps are left-lateral (A) and left-medial (B) (asterisks indicate the anterior direction). For the right hemisphere, the corresponding areas are symmetrically placed.

The following five canonical frequency bands were used in the present study: delta (0.5 to 4 Hz), theta (4 to 8 Hz), alpha (8 to 13 Hz), beta (13 to 30 Hz), and gamma (30 to 50 Hz) (Buzsaki and Draguhn, 2004). However, since the dominant peak frequency within each frequency band varied between participants, we estimated individual dominant frequencies (IDF) for each participant and frequency band separately, based on the power spectrum average across all training trials and voxels belonging to the 15 selected BAs. It was difficult to clearly identify peak frequency in every frequency band for most individuals. In the present study, we aimed to fine tune the selection of frequency bands to improve discrimination between them and assess their contribution to decoding performance. The IDF for each frequency band was selected at the maximum power spectral density within each frequency band during the entire 5-sec poststimulus period.

Following the determination of IDFs, the source-level cortical activities were bandpass filtered within a range of IDF ± 1 Hz for each frequency band. The source-level IDF-bandpass-filtered time series were then averaged across the corresponding voxels of each BA. Signals were averaged across voxels because the limited spatial resolution of non-invasive EEG recordings did not allow inferences at a voxel level, and we focused on the differences in information encoded between BAs. Last, the envelopes of the source-level IDF-bandpass-filtered EEG time series of the 15 BAs for each frequency band were computed using the Hilbert transform. To reduce feature dimensionality, the extracted envelopes of the time series were averaged within 50-ms non-overlapping sliding windows, yielding down-sampled time series over the entire 5-sec poststimulus period.

2.4.3. Classification using regularized LDA

The averaged envelope of the source-level IDF-bandpass-filtered EEG time series in each ROI over the entire 5-sec poststimulus period was used as input features to the classifier. With this feature representation, we then applied a linear discriminant analysis (LDA) (Duda et al., 2001) to extract a class-discriminative feature f_n and built a Gaussian classifier. Given a test sample $\tilde{X}$, we used the following decision rule:

$$\tilde{C}=\{\begin{array}{cc}{\text{C}}_1,& \text{if}p\left(\tilde{f}\mid C_1\right)>p\left(\tilde{f}\mid C_2\right)\\ {\text{C}}_2,& otherwise\end{array}$$ (1)

where $p\left(\tilde{f}\mid C_i\right)$ is a likelihood of a feature $\tilde{f}$ conditioned on the class i.

Classical| LDA is optimal in the sense that it minimizes the risk of misclassification for new samples drawn from known Gaussian distributions (Duda et al., 2001). Particularly, regularized LDA (rLDA) is a powerful machine learning technique that yields excellent results for single-trial event-related potential classification, which are superior to classical LDA when the ratio of features to trials is high (Blankertz et al., 2011; Lemm et al., 2011; Tomioka and Müller, 2010). Thus, in the present study, rLDA with shrinkage was used as a classification algorithm. In order to handle a singular sample covariance matrix Ŝ, estimated with EEG signal features, we used a regularized covariance $\tilde{S}=(1-\lambda )\widehat{S}+\mu \lambda I_p$, where I_p denotes an identity matrix with p diagonal entries, μ is mean of the eigenvalues of $\widehat{S}$ (i.e. $\sum _{i=1}^p{\widehat{S}}_{ii}/p$) and λ ∈ [ 0, 1] is a ‘shrinkage’ hyperparameter. The shrinkage parameter λ was computed analytically using the Ledoit-Wolf estimator (Ledoit and Wolf, 2004). The rLDA uses this regularized sample covariance, instead of the original sample covariance Ŝ (Friedman, 1989).

After fixing the parameters of the rLDA with shrinkage on the training data, the resulting calibrated classifier was used for out-of-sample prediction, i.e. novel unseen EEG trials could be decoded. We performed a 5-fold cross-validation (Lemm et al., 2011) to obtain the performance of out-of-sample classification. Thus, with trials remaining after artefact-rejection, we designated 512 trials for training and the remaining 128 trials for testing, out of all 640 trials (320 trials per each task) per participant. Model (hyper-)parameters were chosen during the cross-validation process, and this procedure was iterated 5 times to provide different combinations of training and test data sets. The resulting decoding accuracies were averaged. This decoding procedure was performed for each participant separately. The decoded signals were evaluated in terms of whether the information encoded in the task could be successfully reconstructed (i.e. whether the task that the participant performed was correctly decoded). For classification, the rates of successful classification of the test data were compared for evaluating the decoding performance. The decoding accuracy was computed based on the estimated activities of all 15 BAs (with activities averaged across right and left hemispheres). Given that PFC signals are weak and we had no prior hypothesis on hemispheric laterality, we averaged signals across hemispheres to increase the signal-to-noise ratio. To enhance performance, we applied a filter-bank method (Ang et al., 2008) that concatenates all features of all frequency bands (delta, theta, alpha, beta, and gamma bands) as input features and trained rLDA.

2.4.4. Computation of activation pattern

Regarding the direction type of classification models (i.e. forward or backward), our classification model corresponds to a backward model. Given observed EEG signals, our model finds the source or task label information that possibly induces the observations. In order to gain better understanding of the classifier with respect to the neurophysiological basis of the extracted task-relevant signal, an ‘activation pattern’ approach (Haufe et al., 2014) was adopted in the present study. Due to the linear property and independence among vectors in the weight matrix Ŵ, it is straightforward to obtain its counterpart A = (Ŵ ⁻¹)^⊤ in a forward model, where each column is considered as an ‘activation pattern’. The observed EEG signals are then understood as a linear combination of the activation patterns in A. The learned parameters of linear classifiers such as rLDA (i.e. their weight vectors) cannot be interpreted with respect to the origin of the signal of interest because the parameters of the models are a function of the task-relevant signal and the task-uninformative signals (i.e. noise signals) (Blankertz et al., 2011; Haufe et al., 2014; Lemm et al., 2011). Therefore, in order to visualize how the extracted signal is encoded in the features that are used by the classifier, a so-called ‘activation pattern’ has to be computed (Dähne et al., 2015; Haufe et al., 2014). To assess the impact of different cortical areas, we converted the weight vectors of the rLDA classifier to activation patterns, which provide neurophysiologically interpretable values (Haufe et al., 2014).

Assuming that the task-relevant and task-uninformative signals are uncorrelated, the activation pattern is given by the covariance between the classifier output and the input features to the classifier (Haufe et al., 2014). To compare classification contributions across 15 BAs, input features were normalized by dividing with its maximum value in each trial. In practice, we estimated an activation pattern involving all 15 BAs by multiplying the covariance in the normalized input features of 15 BAs with the classifier weight as follows:

$$\widehat{A}={\text{Σ}}_X{(\widehat{W})}^{\top }$$ (2)

where Σ_X and Ŵ denote the covariance matrices of normalized input features of 15 BAs and classifier weight vectors, respectively.

2.4.5. Interpretation of activation pattern

The activation patterns based on the rLDA provide neurophysiological insights related to the class label (i.e. task types in this study). The sign of the activation pattern was directly related to the direction of the classification. As a linear classifier was used in the present study, the sign of the pattern depended on how the classes were coded, here + 1 for the mental-rotation task and −1 for the color-perception task. Thus, a positive sign in the activation pattern means that the corresponding feature has larger values for the class coded as + 1, which is the mental-rotation task. Similarly, a negative sign in the activation pattern represents the contribution of the class coded as −1, which is the color-perception task. To extract neurophysiologically interpretable brain–machine interfacing (BMI) features decisive in decoding performance, the activation pattern values of DLPFC were compared with those of VLPFC.

2.4.6. Comparison of PFC contribution to decoding performance

To compare the amount of contribution (i.e. feature strength) of each BA to the decoding accuracy, the activation patterns (normalized by the mean value of 15 BA absolute signals in each participant) of each BA were compared. As both positive DLPFC and negative VLPFC activities were dominantly and contrastively observed over the entire 5 sec in the grand average, the activation patterns of all 15 BAs were compared based on the time window when these most pronounced activation patterns (i.e. when a maximal gap between positive DLPFC and negative VLPFC activities was detected) were individually observed for each participant within the shared time window between stimulus onset and button-pressing for both the rotation and the color task. To compute the LPFC contribution in the activation pattern to decoding performance, the mean values of the absolute LPFC (i.e. BAs 9, 45, 46, and 47) activation pattern values were compared with those of the remaining non-LPFC region. Similarly, the contribution comparison between PFC and non-PFC was computed using the ratio of the absolute activation pattern values of PFC and non-PFC.

In order to arrive at a comparable single activation pattern for each condition that could be visualized as a 3D topographical map, the grand-averaged activation patterns of each frequency band were normalized by the mean value of 15 BA absolute signals in each participant. Hence, based on the activation patterns, it allows us to gain a neurophysiological insight of the input channels, i.e. Brodmann areas, associated with the classification task. This is an important aspect of activation pattern analysis; thus, the contribution of PFC to the decoding of EEG signals for BMIs, reflected in the values of activation patterns is interpretable in terms of neurophysiological concepts.

2.4.7. Statistical analyses

All of the measures were analyzed using two-tailed paired-sample t-tests, and one-sample t-test was used to statistically examine whether the classification accuracies were significantly higher than by chance (50%). A false discovery rate (FDR) of q < 0.05 (Benjamini and Hochberg, 1995) was used to correct for multiple comparisons. To statistically assess whether DLPFC activity fluctuated primarily in the positive domain of the activation pattern and VLPFC activity oscillated mostly in the negative domain of the activation pattern, we computed the 95% confidence intervals of each mean throughout the entire 5 sec of the task performance in each frequency band. All analyses were performed using MATLAB (ver. R2018b, MathWorks, USA) or Python (Python Software Foundation, https://www.python.org).

3. Results

3.1. Behavioral performance in mental-rotation and color-perception tasks

We observed significant differences in behavioral responses for the different levels of task difficulty (Fig. 4). Both tasks had consistently higher reaction times with increasing task difficulties (mental-rotation task: difficulty 1: 1136.1 ms, difficulty 2: 1520.8 ms, difficulty 3: 1793.2 ms, difficulty 4: 1964.6 ms; color-perception task: difficulty 1: 733.4 ms, difficulty 2: 777.8 ms, difficulty 3: 1032.6 ms, difficulty 4: 1416.7 ms). Similarly, performance accuracies declined with higher task difficulty, though not all differences were statistically significant (mental-rotation task: difficulty 1: 98.8%, difficulty 2: 98.2%, difficulty 3: 96.0%, difficulty 4: 92.1%; color-perception task: difficulty 1: 98.5%, difficulty 2: 98.5%, difficulty 3: 97.7%, difficulty 4: 89.0%).

Fig. 4. Behavioral performance.

Reaction times and performance accuracy in the mental-rotation task (A) and color-perception task (B). Error bars indicate standard errors of the mean. Note a significant systematic increase in reaction times at higher task difficulties. *q < 0.05 and NS = non-significant, FDR-corrected.

3.2. Decoding task engagement from EEG time courses of 15 Brodmann areas

The beta band provided the highest decoding accuracy of 83.72% (t(16) = 18.711, q < 0.05, FDR-corrected) among all frequency bands (Fig. 5), but this did not differ between bands as all other frequency bands yielded decoding accuracies well above chance level (delta 74.26%: t(16) = 11.009, q < 0.05, FDR-corrected; theta 80.03%: t(16) = 16.868, q < 0.05, FDR-corrected; alpha 80.50%: t(16) = 17.943, q < 0.05, FDR-corrected; gamma 83.45%: t(16) = 20.756, q < 0.05, FDR-corrected). A filter-bank method (Ang et al., 2008) using the combinational features of all frequency bands increased decoding accuracy to 86.99% (t(16) = 23.347, q < 0.05, FDR-corrected).

Fig. 5. Decoding task engagement (mental-rotation vs. color-perception) using spectral features of 15 BAs across different frequency bands.

Decoding accuracy is significant in all frequency bands. A filter-bank method combining features from all frequency bands yielded the highest decoding accuracy of 87%. * q < 0.05, FDR-corrected. Error bars indicate standard errors of the mean.

3.3. Contribution of subregions of LPFC to decoding performance

Different behavior dependent activation patterns were observed between DLPFC and VLPFC (Fig. 6). DLPFC activity was prominently observed in a positive domain of activation patterns (95% confidence intervals) supporting the mental-rotation task (delta [0.200–0.517], theta [0.011–0.368], alpha [0.082–0.438], beta [0.024–0.350], gamma [0.153–0.481]), whereas VLPFC activity was predominantly detected in a negative domain of activation patterns indicating more feature strength for the color-perception task (delta [−0.132–0.098], theta [−0.364–−0.118], alpha [−0.420–−0.185], beta [−0.369–−0.139], gamma [−0.339–−0.089]) throughout the entire 5 sec of the task performance.

Fig. 6. Dissociative activation patterns between DLPFC and VLPFC.

Time courses of activation patterns (feature strength) between DLPFC (red curves) and VLPFC(blue curves) in the (A) delta, (B) theta, (C) alpha, (D) beta, and (E) gamma bands. The scales were normalized by the mean value of 15 BA absolute signals within each individual. Error bands indicate standard errors of the mean; the vertical black error bars represent 95% confidence intervals of mean activities throughout the entire 5 sec of the task performance. DLPFC patterns are primarily in the positive domain, whereas VLPFC are in the negative domain. Black lines below curves represent the stimulation period; the vertical blue dotted line represents the grand-averaged response onset for the color-perception task, and the vertical red dotted line represents the grand-averaged response onset for the mental-rotation task.

Activation patterns within each BA averaged across participants are shown in Figure 7. Overall, the activation pattern values of the DLPFC were significantly higher than those of the VLPFC in all frequency bands: delta (t(16) = 3.763, q < 0.05, FDR-corrected; DLPFC: 0.642 vs. VLPFC: −0.447), theta (t(16) = 6.654, q < 0.05, FDR-corrected; DLPFC: 0.671 vs. VLPFC: −0.544), alpha (t(16) = 9.410, q < 0.05, FDR-corrected; DLPFC: 0.923 vs. VLPFC: −0.655), beta (t(16) = 2.345, q < 0.05, FDR-corrected; DLPFC: 0.902 vs. VLPFC: −0.064), and gamma bands (t(16) = 4.545, q < 0.05, FDR-corrected; DLPFC: 0.922 vs. VLPFC: −0.600). The sign of the activation pattern (feature strength) is indicative of the class label (i.e. task types in this study), with positive values relevant to the mental-rotation task and negative values relevant to the color-perception task. Since the DLPFC consistently exhibited a positive feature strength while the VLPFC demonstrated a negative feature strength based on 95% confidence intervals of their mean activities during the task performance, the DLPFC features reflected the mental-rotation task and the VLPFC features were linked to the color-perception task. These observations are consistent with the dorsal (for rotation task) and ventral (for color task) stream processing model. Furthermore, the averages of the absolute LPFC activation patterns were higher than those of the non-LPFC in all frequency bands: delta (t(16) = 2.284, q < 0.05, FDR-corrected; LPFC: 1.205 vs. non-LPFC: 0.926), theta (t(16) = 2.803, q < 0.05, FDR-corrected; LPFC: 1.221 vs. non-LPFC: 0.920), alpha (t(16) = 2.135, q < 0.05, FDR-corrected; LPFC: 1.247 vs. non-LPFC: 0.910), beta (t(16) = 3.917, q < 0.05, FDR-corrected; LPFC: 1.420 vs. non-LPFC: 0.847), and gamma bands (t(16) = 4.144, q < 0.05, FDR-corrected; LPFC: 1.374 vs. non-LPFC: 0.864). These findings demonstrated that the most dominant feature for distinguishing two tasks was observed in the LPFC.

Fig. 7. Grand-averaged activation patterns in the 15 BAs.

Activation patterns across 15 different BAs were averaged across participants in the (A) delta, (B) theta, (C) alpha, (D) beta, and (E) gamma bands. The scales were normalized by the mean value of 15 BA absolute signals within each individual. An upward direction (positive domain) of the activation pattern (red bars) represents a classification feature relevant to the mental-rotation task while a downward direction (negative domain) of the activation patterns (blue bars) indicates relevancy to the color-perception task. Error bars indicate standard errors of the mean. The 15 BAs in the x-axis are order based on their relative anatomical locations starting from frontal and ending to occipital regions. The mean values of the absolute LPFC (BA 9, 46, 45, and 47) activation pattern values were statistically compared with those in the remaining non-LPFC regions. Note, the LPFC region yielded significant contributionsin the decoding accuracy as compared with the non-LPFC region. * q < 0.05 and NS = non-significant, FDR-corrected two-tailed paired t-tests between LPFC and non-LPFC regions as well as across DLPFC (i.e. BA 9 and 46) and VLPFC (i.e. BA 45 and 47).

The grand-averaged activation patterns of all frequency bands are displayed on the cortex (Fig. 8). It is noteworthy that the DLPFC-centered dominant positive value was observed across all frequency bands. In contrast, the color-task-relevant features of negative activation patterns were evident in the VLPFC and occipital visual processing areas.

Fig. 8. Cortical maps of activation patterns.

Cortical maps of grand-averaged activation patterns of 15 BAs are shown in the (A) delta, (B) theta, (C) alpha, (D) beta, and (E) gamma bands. In each set of cortical maps, the viewpoints are left-lateral, superior, right-lateral, left-medial, right-medial, anterior, inferior, and posterior (asterisks indicate the anterior direction). The scales were normalized by the mean value of 15 BA absolute signals within each individual. The maximum of the color-coding scale is set to the maximum value across all frequency bands. Red regions represent the most dominant features for classifying the rotation task, whereas blue regions indicate the most dominant features for classifying the color-perception task. Within the PFC, DLPFC exhibits red activation, and VLPFC displays blue activation.

The LPFC contribution to the decoding accuracy in the activation patterns was enhanced compared to non-LPFC contribution in the theta, beta, and gamma bands (Fig. 9); delta (t(16) = 1.984, n.s.), theta (t(16) = 2.597, q < 0.05, FDR-corrected; LPFC: 56.16% vs. non-LPFC: 43.84%), alpha (t(16) = 1.872, n.s.), beta (t(16) = 3.898, q < 0.05, FDR-corrected; LPFC: 61.26% vs. non-LPFC: 38.74%), and gamma bands (t(16) = 3.978, q < 0.05, FDR-corrected; LPFC: 60.33% vs. non-LPFC: 39.67%). The averaged LPFC contribution to decoding accuracy was 57.8% across all frequency bands, and 59.2% for the theta, beta, and gamma bands.

Fig. 9. LPFC versus non-LPFC contributions (%) in activation patterns of each frequency band.

*q < 0.05, FDR-corrected two-tailed paired t-tests between LPFC (red) and non-LPFC (blue) regions. Error bars indicate standard errors of the mean.

4. Discussion

EEG activation pattern analysis using source reconstructed data distinguished mental-rotation vs. color-perception tasks. The most salient features selected by the rLDA classifier were concentrated in LPFC known to be central to virtually all cognitive tasks, including working memory and decision-making among others (Fuster, 2008; Royall et al., 2002; Sarazin et al., 1998). The mental-rotation task required continuous spatial information processing and utilization of mental manipulation, and the color-perception task required holding information online to make a successful match; both processes are known to engage LPFC. We observed that EEG signals distinguishing the two tasks were maximal in LPFC.

Differential activation patterns between DLPFC and VLPFC demonstrated that LPFC signals can categorize each task performance. The sign of the activation pattern predicts the direction of classification (positive for the mental-rotation task and negative for the color-perception task) (Haufe et al., 2014). Positive classification features for mental rotation were reliably observed in DLPFC regions while negative classification features for color perception were detected in VLPFC regions. These findings are in accord with proposals that DLPFC is the end point for the dorsal stream (i.e. the “where” pathway for the mental-rotation task) (Goodale and Milner, 1992; Hesse et al., 2014) and VLPFC is the end point for the ventral stream (i.e. the “what” pathway for the color-perception task) (Kravitz et al., 2013; Mishkin et al., 1983). We observed that spatial information processing and mental manipulation during object rotation engaged more DLPFC resources, as compared to the color-perception task, which was weighted toward VLPFC and posterior visual cortices. Notably, beta and gamma bands (Fig. 7 DE) had positive activation patterns in the mental-rotation task in BA 5 and 40, which belong to the dorsal stream. In contrast, negative values of activation patterns indicating relevancy to the color-perception task were observed particu larly along with occipito-temporal visual cortices such as BA 17, 18, 19, and 20, encompassing the ventral stream (Fig. 7). These observations provided a neurophysiological signature for the dissociation of the dorsal “where” route for the mental-rotation task and the ventral “what” route for the color-perception task.

The selective contribution of LPFC to decoding accuracy was most pronounced in the theta, beta, and gamma bands (Fig. 9). We identified frequency-specific effects contributing to LPFC decoding accuracy, but we did nor observe differential decoder performance across frequency bands. All five canonical bands supported task discrimination (Fig. 5) and their activation patterns shared numerous spatial similarities (Fig. 8). Thus, it is reasonable to expect that a corresponding analysis with a broadband signal would achieve similar discrimination results as shown in Figure 5. PFC-based decoding performance could be improved using different combinations of analytic approaches (e.g. wide-band filters), and optimal methods for decoding should be explored in future work.

The entire PFC (including not only LPFC but also BA 10 and 11) provided a further enhanced contribution to decoding accuracy compared to the non-PFC region in the alpha, beta, and gamma bands (Fig. S1). The role of neural oscillations in PFC-mediated mental processing has been highlighted in recent studies with humans (Johnson et al., 2018) and monkeys (Miller et al., 2018). PFC-dependent top-down processing integrates and controls bottom-up visual information from occipital cortices (Barcelo et al., 2000). Reciprocal connections between the PFC and the occipital cortices (Hwang and Luna, 2013; Wakana et al., 2004) through the fronto-occipital fasciculi provides a neuroanatomical substrate for control. The extensive and reciprocal connections between the PFC and other brain regions provide further neuroanatomical substrates for PFC control of diverse cognitive processes (Barbas, 2000). Functional interactions between the PFC and posterior cortical regions have been consistently reported during executive control (Gazzaley et al., 2004; McIntosh et al., 1996). For instance, electrocorticography studies report phase coupling between fronto-parietal cortices during spatial attention control (Szczepanski et al., 2014) and over the posterior cortex during visual tasks (Voytek et al., 2010).

Taken together, our observations provide evidence that PFC activity reliably decodes the implementation of intended goals in the mental-rotation/color-perception paradigm. Both PFC and PPC have been linked to goal-directed movement planning (Brandi et al., 2014; Fincham et al., 2002; Gremel and Costa, 2013; Lindner et al., 2010; Niedermeyer, 1998; Pereira et al., 2017; Petzschner and Kruger, 2012; Rosenberg-Katz et al., 2012). PFC signals provided enhanced decoding performance in the present tasks compared to sensorimotor and parietal regions. For instance, LPFC features were more robust than the sensorimotor features in the mental-rotation paradigm (Figs. 7 and 8). DLPFC signals exhibited dominant features of mental-rotation processing over these motor-related cortical areas across all frequency bands. Presumably, the mental-rotation task did not require sensorimotor activity for efficient performance. This may be due to the fact that pure mental-manipulation of a visualized object does not need to induce efferent commands to move peripheral muscles, which are engaged in motor imagery BMI paradigms typically requiring imagining movement of peripheral limbs. The current findings reveal that higher-order PFC-based cognitive brain signals, which encode goal-directed intentions instead of details on how to perform reaching, enlarge the repertoire of BMI control signals.

Activation patterns for the color-perception task were enhanced in both VLPFC and occipital areas. This is likely due to the dominant visual processing of color features in occipital cortices (Hadjikhani et al., 1998; Tootell et al., 2004). These results indicate that neural features are dependent on the characteristics of the assigned cognitive task. This suggests that depending on task type, the brain regions inducing the most dominant control features can be selectively manipulated overcoming limitation of classical BMI task paradigms.

PFC provides rich signal space for cognitive BMI applications yet despite the prominent role in goal-directed cognition, PFC has largely been ignored in BMI research (Min et al., 2017). Advances in BMI technology have been applied to scenarios ranging from therapeutic approaches to consumer applications (Dornhege et al., 2007; Min et al., 2010; Wolpaw and Wolpaw, 2012). For example, BMI has been used to assist patients with motor deficits, such as amyotrophic lateral sclerosis (McCane et al., 2014), as well as by healthy individuals for lifestyle enhancement including biofeedback and other relaxation metrics (Blankertz et al., 2010; Millán and Carmena, 2010; Millán et al., 2010; Muller et al., 2008; Tan and Nijholt, 2010). The majority of non-invasive BMI brain signals utilize primary sensorimotor cortex activity including mu and beta rhythms focused on motor control (Muller-Putz et al., 2005; Pfurtscheller and Neuper, 1997), posterior parietal scalp potentials including the P300 (Kleih and Kubler, 2013; Sellers and Donchin, 2006), or SSVEP paradigms (Muller-Putz and Pfurtscheller, 2008). Recent studies report that direct invasive recordings from the human posterior parietal cortex can decode higher-level aspects of movement and these signals can effectively control a robotic arm for a tetraplegic patient with implanted electrodes and can be used as potent BMI control signals for decoding higher-level aspects of movement (Aflalo et al., 2015; Hauschild et al., 2012; Lindner et al., 2010), suggesting that the association cortex provides reliable control signals. However, it is noteworthy that LPFC contributed more to decoding performance than the posterior parietal cortex (BA 5 and 40) in the present study. This suggests that PFC provides a novel signal source for goal-directed intention applicable to BMI (Carlson and Millan, 2013; Iturrate et al., 2015). The current findings provide evidence for use of prefrontal brain activity for non-invasive cognitive BMI-control signals that are goal-directed and task-specific, including attention, memory implementation, and domain-general decision making (Fuster, 1997). This technology can be applied to various higher-order PFC-dependent cognitive functions, such as planning and performance-monitoring independent of how actions are executed through button pressing, speech, or eye movement.

PFC-based decoding approaches have recently received growing attention in both EEG and MEG studies. For example, using a deep-learning technique, more than 85% accuracy was observed when decoding imaginary vowels using frontal EEG signals (Parhi and Tewfik, 2021). In a MEG study, classifiers were trained to distinguish the MEG field patterns during the presentation of two probabilistic outcomes (reward, loss), and then applied to decode such patterns during deliberation (Castegnetti et al., 2020). Decodable outcome representations predominantly captured in the PFC during probabilistic decision-making predicted subsequent action with a classification accuracy of 70%. Taken together, these findings support the use for decoding of cognitive signatures of goal-directed tasks for development of reliable and robust PFC-mediated EEG/MEG-based BMIs.