Cortical Imaging of Event-Related (de)Synchronization during Online Control of Brain-Computer Interface Using Minimum-Norm Estimates in Frequency Domain

Abstract

It is of wide interest to study the brain activity that correlates to the control of Brain-Computer Interface (BCI). In the present study, we have developed an approach to image the cortical rhythmic modulation associated with motor imagery using minimum-norm estimates in the frequency domain (MNEFD). The distribution of cortical sources of mu activity during online control of BCI was obtained with the MNEFD. Contralateral decrease (event-related desynchronization, ERD) and ipsilateral increase (event-related synchronization, ERS) are localized in the sensorimotor cortex during online control of BCI in a group of human subjects. Statistical source analysis revealed that maximum correlation with movement imagination is localized in sensorimotor cortex.

I. INTRODUCTION

There has been a great deal of interest in frequency-specific rhythmic activity in relation to motor, sensory and cognitive functions. Early reports measuring cortical [1] and scalp [2] recorded brain activity described changes in EEG rhythms accompanying the preparation and performance of voluntary movement. More recently, a number of neuroelectric (EEG) and neuromagnetic (MEG) experiments have demonstrated that during planning and execution of hand and/or finger movement the power of rhythmic activity in the mu (8–12 Hz, also known as sensorimotor rhythm) and beta (13–26 Hz) bands in the central region modulates [3]. Upon movement termination, mu and beta power recovers or exceeds baseline levels [4, 5]. These observations are further supported from experiments that measured synchrony in primate local field potentials (LFP) from sensorimotor cortex and reported that on-going (resting-state) synchronous cortical oscillations are interrupted by activating neurons involved in preparing and performing movement [6, 7]. This phenomenon has been utilized as the basis of noninvasive Brain-Computer Interface (BCI), which provides communication and control to people who are totally paralyzed [8]. Studies have demonstrated that people can learn to increase and decrease sensorimotor rhythm amplitude over one hemisphere using the mental strategy of motor imagery to control physical or virtual devices [9, 10].

The neurophysiological mechanisms producing these oscillations are poorly understood [11] and little is known about their functional significance [12]. The low frequency bands have been associated with thalamocortical circuits and they reflect phase coherence of cortical circuit [13–15]. With this interpretation, the spectral decrease is quantitatively defined as event-related desynchronization (ERD) and the increase as event-related synchronization (ERS) [3]. It is generally postulated that spectral shifts in these oscillations do not necessarily reflect differences in overall cortical computational activity but rather that they reflect changes in population coherence [13–15].

Using recently developed EEG/MEG source imaging techniques, the movement-related rhythmic activities have been investigated with enhanced spatial resolution. Sources of mu rhythm during offline motor imagery were previously studied using dipole localization method [16, 17] or distributed source imaging [16, 18, 36]. Different from the above studies in which source estimates are obtained from every sample point in the temporal domain, Jensen and Vanni [19] have developed a new computationally efficient approach to estimate the minimum current in the frequency domain. However, as a minimum L1-norm approach is employed in [19], an over-focused solution with a few distinct source points is favored, which is not suitable for spread mu rhythm reconstruction.

In the present study, we have developed an approach to image the cortical rhythmic modulation using a Minimum-Norm Estimate in the Frequency Domain (MNEFD). We applied the MNEFD analysis to study the rhythmic activities in a BCI experiment in which subjects used imagination of hand movement as a mental strategy to achieve one-dimensional (1D) cursor control.

II. METHODS

A. Subjects and Experimental Setup

Four healthy subjects (male, ages 19–21 years, two left-handed and two right-handed as measured by the Edinburgh Handedness Inventory [38]) participated in the study with written consent according to a protocol approved by the Institutional Review Board of the University of Minnesota. They sat in a comfortable armchair in an electrically shielded room facing a virtual computer screen from a distance about 2 m. EEG activity was recorded from 64 electrode locations distributed over the entire scalp (Fig. 1A). The signals were acquired with a BrainAmp amplifier (BrainProducts, Germany) at sampling frequency of 1000 Hz.

Fig. 1.

Electrode Positions (A) and experimental paradigm (B). The red diamond in (A) illustrates the subset of channels used for controlling cursor movement.

The subject was instructed to move the cursor to hit the left/right target within 6 s by imagining left/right hand movement respectively (Fig. 1B). A trial started when a target appeared at one of two locations on the periphery of the screen at 0 s, with a fixation cross at the center till inter-trial interval. One second later, the cursor appeared in the middle of the screen and began to move horizontally with its movement controlled by the user’s EEG activity. If the cursor reached the target within 6 s, the target flashed as a reward. If it reached the wrong target within 6 s or failed to reach the target within 6 s, the cursor and the target disappeared. In each case, the screen was blank for 3 s, and then the next trial began. The experiment consisted of eight 5-min runs separated by 2-min breaks, and each run had 30–40 trials. Within each run, the tasks were performed in a random order to avoid adaptation. Using the general-purpose system BCI2000 [20], the horizontal cursor movement was controlled by a linear equation of a weighted combination of the amplitude in mu band from EEG channels over the left and right hemisphere. EEG activity from controlling channels were spatially filtered and fitted into an Autoregressive (AR) model to extract power spectra features. The power in the mu rhythm frequency band of EEG signals is then linearly mapped to the rightward/leftward position of a moving cursor.

B. Anatomical MRI and Electrode Digitization

The individual anatomical MRI data set consisted of 256 contiguous sagittal slices with 1 mm slice thickness (matrix size: 256 × 256, FOV: 256 mm × 256 mm). The images were acquired using a Turboflash sequence (TR/TE=20 ms/5 ms) [21] on a 3T MRI system (Siemens Trio, Siemens, Erlangen, Germany). The physical landmarks (nasion and left, right preauricular points) and electrode positions were digitized using a Polhemus Fastrak digitizer (Polhemus, Colchester, VT) and 3DSpace software from the SCAN (Compumedics, Inc., El Paso, TX) software package.

C. Minimum-Norm Estimate in Frequency Domain

Assuming a cortically constrained distributed source model, the relationship between source amplitudes and scalp potentials can be expressed by the following linear model [16, 18, 22]:

$$\mathrm{\Phi }(t)=AS(t)+N(t),$$ (1)

where Φ is a matrix of the measured EEG. S is the unknown matrix of amplitudes of the dipoles along the time. A is the transfer matrix. Data are corrupted by an additive noise N. Although the measured data Φ do not give the source strengths S unambiguously as the number of discretized sources is larger than the number of sensors, a minimum-norm estimate (MNE) in the sense of L2-norm can be obtained by applying a linear inverse operator to the measured signals:

$$\hat{S}=W\mathrm{\Phi }.$$ (2)

.

An expression for W is obtained in closed form by minimizing:

$${\Vert C^{-1/2}(\mathrm{\Phi }-AS)\Vert }_2^2+{\lambda }^2{\Vert R^{-1/2}S\Vert }_2^2,$$ (3)

where C and R are covariance matrices of the noise and sources, respectively. λ² is a regularization parameter [23, 24], and ‖·‖₂ indicates the L2 norm. Minimization of (3) over Φ yields

$$W=R{A}^T{(AR{A}^T+{\lambda }^{2}C)}^{-1}.$$ (4)

And the regularization parameter is

$${\lambda }^2=\frac{\mathit{\text{tr}}(AR{A}^T)}{\mathit{\text{tr}}(C)SN{R}^2.}$$ (5)

As no prior knowledge of source activity is assumed, R is an identity matrix here. The data with 15% lowest global field power are selected for noise estimation. The noise covariance matrix C is constructed as a diagonal matrix with diagonal elements proportional to the average noise power over all channels. In order to compensate the tendency of the minimum-norm solution to favor superficial sources, depth-weighting method was also used.

Using the Fourier transform, both S(t)and Φ(t) are transformed to S′(f) and Φ′(f) respectively in the frequency domain [19]. Thus, (1) becomes:

$$\mathrm{\Phi }\text{'}(f)=AS\text{'}(f)+N\text{'}(f).$$ (6)

And (6) holds with both the real ${\mathrm{\Phi }}_{\text{Re}}^{\text{'}}(f)$ and imaginary part ${\mathrm{\Phi }}_{\text{Im}}^{\text{'}}(f)$:

$$\{\begin{array}{c}{\mathrm{\Phi }}_{\text{Re}}^{\text{'}}(f)=A{S}_{\text{Re}}^{\text{'}}(f)+N_{\text{Re}}^{\text{'}}(f)\hfill \\ {\mathrm{\Phi }}_{\text{Im}}^{\text{'}}(f)=A{S}_{\text{Im}}^{\text{'}}(f)+N_{\text{Im}}^{\text{'}}(f).\hfill \end{array}$$ (7)

Then ${\mathrm{\Phi }}_{\text{Re}}^{\text{'}}(f)$ and ${\mathrm{\Phi }}_{\text{Im}}^{\text{'}}(f)$ of the Fourier transformed signal were applied to the MNE method resulting in the current distributions $S_{\text{Re}}^{\text{'}}(f)$ and $S_{\text{Im}}^{\text{'}}(f)$. The source strength estimates of single trial data are then estimated by summing up the real and imaginary parts corresponding to each trial. The average source distributions over multiple-trials are shown in the figures.

Specifically, the inverse estimates of real and imagery parts after Fourier transformation were calculated using BESA (MEGIS Software GmbH, Graefelfing). A realistic geometry head model was applied when calculating the transfer matrix. The conductivity ratio used for the forward solution computation is 1:0.05:1 for scalp:skull:brain [25, 26]. Individual frequency band and time window of imagination for the source estimates was selected with the aid of wavelet time-frequency representation (TFR) (see below).

D. EEG Data Analysis

EEG recordings from trials that ended with hits are subject to data analysis. The signals are band-pass filtered from 1 Hz to 30 Hz using a zero-phase FIR filter. After artifacts are visually rejected, EEG data are segmented into epochs from 2 s before the beginning of the trial to 1 s after the cursor hit the target. Then, the epochs are baseline corrected, and detrended. Epochs with eye movements were visually identified and excluded from further analysis. Finally, the artifact-free signals are down-sampled to 200 Hz.

Each trial lasts from 5 s to 9 s, but not all time points of each trial carry information about the oscillatory modulation by motor imagery; so it is not efficient to use the whole time range for source analysis. In addition, the desynchronization phenomenon during motor imagery tasks is highly frequency related. In the present study, the time-frequency analysis was used to select the appropriate time window and frequency band for source analysis [16, 27]. TFRs of these single-trial EEG data were computed individually using a Morlet wavelet-based technique over the 6–30 Hz frequency range, with center frequencies at 1 Hz intervals. Since the period of cursor movement varies from trial to trial, the movement time was normalized to 200 equally spaced time points having an average spacing of approximately 5 ms (i.e., 1-s movement time).

E. EEG Source Analysis

For reconstructed source activities, the negative (ERD) or positive (ERS) spectral change is calculated by comparing the distributions of mu powers for each imagery type with the pooled rest distributions, as detailed in (8). We calculated the p values associated with power in the mu band using an unpaired t test with the power during imagination compared with baseline. For each subject, the calculated p value was Bonferroni corrected [28].

$$D^i=\frac{{\overline{p}}_I^i-{\overline{p}}_0^{\mathrm{L}\cup \mathrm{R}}}{{\overline{p}}_0^{\mathrm{L}\cup \mathrm{R}},}$$ (8)

where p_I and p₀ are source power during motor imagery and baseline respectively. i denotes left (L) or right (R) imagery. ¯ is the average operation over multiple trials for each imagery type. The decrease (ERD) and increase (ERS) of cortical rhythm relative to baseline are illustrated in Figure 3.

Fig. 3.

Cortical distribution of mu ERD and ERS during cursor movement for four subjects.

We assessed the EEG control by topographical analysis of the correlation between movement imagination and the mu source activities (measured as R², the coefficient of determination) [9, 35]. Specifically, R² is calculated to be the proportion of the explained variance of mu activity as two types of motor imagery based on single-trial source estimates. The topographies are for R rather than R² to show the opposite (i.e. positive and negative, respectively) correlations of right and left hemispheres with imagination types corresponding to the two target positions. We also calculated the R from scalp recordings (scalp R) to compare with the R from source estimates (source R). After transforming the signal traces from electrodes into the spectral domain, the power in the mu frequency band are extracted and subject to R² analysis.

III. RESULTS

Four subjects achieved reliable 1D control over the cursor movement, as shown in Table 1. The average accuracy of target hits out of all the trials from the four subjects is 90.89 ± 3.39% and the average hitting time is at 2.57 ± 0.30 s. Fig. 2 shows a typical example of spectral modulation of motor imagery. Fig. 2C and Fig. 2D indicate that such modulation is a dynamic process in which the instant spectrum varies from point to point. In fact, by averaging the TFR for each imagery type, a characteristic modulation is identified in the normalized period 0.7 s to 1.9 s and single trial data lying in such time window are subject to source analysis. The baseline of rest condition is selected as −1.9 s to −0.2 s. The frequency bands for source analysis are displayed in Table 2. During imagination, a blocking (ERD) of mu rhythm over the contralateral scalp and enhancement (ERS) over the ipsilateral area is shown in Fig. 2B. And the mu source activity is localized at the sensorimotor cortex (Fig. 2A).

TABLE I.

Online Cursor Control in Healthy Human Subjects.

Subject	Accuracy (%)	Target Time (s)
#1	88.24 ± 1.66	2.15 ± 0.73
#2	94.85 ± 2.82	2.87 ± 1.15
#3	92.57 ± 4.01	2.71 ± 0.85
#4	87.90 ± 5.12	2.56 ± 0.87
Average	90.89 ± 3.39	2.57 ± 0.30

Accuracy is calculated as the percentage of target hits out of all trials.

Fig. 2.

Topologies of power in mu band during cursor movement are shown on the cortex (A) and scalp (B) for subject #1. Average TFR (C) and a typical single trial data (D) show imagery-related modulation. TFRs were realigned at time = 0 s (dashed line) and the target time were normalized to be 2 s (solid line).

TABLE II.

Spectral Changes of Mu Source and the Selected Frequency Bands for All Subjects.

	Left Hand		Right Hand
Subject	ERD (%)	ERS (%)	ERD (%)	ERS (%)	Frequency Band (Hz)
#1	−57	45	−52	76	9 – 12
#2	−34	46	−42	34	9 – 12
#3	−40	28	−43	32	9 – 12
#4	−35	13	−30	30	8 – 11
Mean	−41.50 ± 10.66	33.00 ± 15.68	−41.75 ± 9.03	43.00 ± 22.06	—

Minimum values of ERD and maximum values of ERS are listed to show the largest absolute changes.

Maps of spectral change of cortical rhythms during movement imagination are shown in Fig. 3 for all individuals (only changes significant at the 0.05 Bonferroni-corrected level are shown). Maximum values for source spectral changes and the frequency band selected for source analysis are listed in Table 2. A highly generalized response is found for left and right hand imagery across individual subjects. Contralateral ERD and ipsilateral ERS of mu rhythm in the hand region of sensorimotor cortex is a consistent, significant feature in all subjects. These changes occur typically across sensorimotor cortex for the mu ERD and are somewhat more spatially specific for the ERS. Some subjects displayed desynchronization in the parietal and/or parietal-occipital cortex.

The correlation maps with two types of imagination are shown in Fig. 4 for all individual subjects. A highly consistent and significant R map was found in all subjects. The mu sources in two hemispheres show opposite activities with the two target positions, i.e. ERD in one type of imagery and ERS in the other type. The peak of correlation is localized at the hand area of sensorimotor cortex. Parietal/occipital areas which show ERD in both types of imagery do not show strong correlation with movement imagination.

Fig. 4.

Cortex topographies of the correlations of the mu rhythm with movement imagination for four subjects. The topographies are for R to show the opposite correlations of right and left sides (see text for details).

Maximum values of source R are listed in Table 3. The maximum values of scalp R and R at electrodes C3/C4 are also listed in Table 3 in comparison with source R estimates. As shown in Table 3, the absolute values for the source R index are always larger than those obtained for the same subject by using the scalp potentials during the imagination.

TABLE III.

Comparison between the Source R (Estimated from Source Signals of Mu Band) and the Scalp R (Estimated from Scalp Signals).

Subject	Source R (Min/Max)		Scalp R (Min/Max)		Scalp R
	Negative	Positive	Negative	Positive	C3	C4
#1	−0.86	0.91	−0.80	0.77	−0.58	0.76
#2	−0.82	0.69	−0.68	0.53	−0.68	0.53
#3	−0.86	0.81	−0.75	0.74	−0.75	0.74
#4	−0.57	0.63	−0.60	0.52	−0.38	0.52
Mean	−0.78 ± 0.14	0.76 ± 0.12	−0.71 ± 0.08	0.64 ± 0.13	−0.6 ± 0.16	0.64 ± 0.13

Minimum of negative R and maximum of positive R from two hemispheres are listed to show the largest absolute R. The scalp R from C3 and C4 electrodes are also listed.

IV. DISCUSSION

The movement-related rhythmic activities during voluntary [37] and imagined movement [16–18, 36] have been studied using advanced imaging techniques with high spatial resolution. Although rhythmic activities from scalp recordings have been widely exploited for BCI control, it is still unclear how the cortical rhythmic activities are distributed in the brain and how they correlate to the control during the online interactive process. The present study aimed at addressing this basic question and our results for the first time report the cortical distribution of rhythmic activity during online control of BCI. The present results also showed the contralateral decrease of cortical rhythms and ipsilateral increase at the sensorimotor cortex in a group of human subjects.

Tremendous efforts have been made to improve the spatial resolution of EEG. Among them equivalent dipole fitting [29] and current density reconstruction methods [22, 30, 31] produced numerous valuable results. Previous EEG source reconstruction studies of movement-related activity used spatiotemporal multi-dipole modeling, which estimates the dipole locations and waveforms that best explain the EEG measurements. Due to this approach a constraint on the number of sources, i.e., current dipoles, had to be used and usually only a very few number of dipoles are assumed, as the maximum number of moving dipoles which can be estimated reliably is quite limited. Nevertheless, the cortical MNE provides a distributed source activity reconstruction over the entire cortex surface, rather than a few isolated sources produced by equivalent current dipole analysis or an over-focused solution by the L1-norm method. Based on the reconstructed activity, the changes of sources can be analyzed to produce the activation map and source correlation analysis can also be performed to evaluate the quality of BCI control, which makes the approach particularly suitable to investigate the rhythmic activity during online process of motor imagery.

In the present approach, Fourier transformation converts EEG signals in the temporal domain into concrete representation in the frequency domain; this enables one to directly image the source activities in the targeted frequency band, avoiding laboriously searching each time-sample over the whole segment of oscillatory signals. Particularly, the rhythmic signal during an on-line process demonstrates prominent dynamics, which reflect the evolving brain states (Fig. 2C and Fig. 2D). The mu activity corresponding to imagination and baseline states were selected from averaged time-frequency representation which approximates two distinct stationary processes and thus the Fourier transform is applied. Through source imaging in the spectral domain, the method overcomes the obstacle of designating one time-sample as the representative point for the desynchronization or synchronization state. Furthermore, this method may be improved by considering time and frequency concurrently and characterizing the source activity at each time-frequency point.

The present study for the first time reports the source imaging results during online control of BCI. Fig. 3 illustrates the activations associated with hand imagery across four subjects. The topographical activations displayed significant concurrence to known topography of hand regions in the sensorimotor cortices. The locations line up with the expected peri-rolandic sensorimotor cortical sites. Subjects reported that they imagined voluntary movement of left or right hand, e.g. lifting bicep curl, during cursor movement as instructed. Therefore, it is not surprising that the activations span the central sulcus, as the imagination involves both motor movement and sensory feedback. The contralateral decrease in mu source activity during online feedback presented here is consistent with previous literature which reported ERD with contralateral dominance accompanying imagination of hand movement without feedback [15, 3]. Some subjects (Fig. 3 Subject #1 left and right, Subject #3 left) showed ERD in the parietal cortex but such activation is not of statistically significant correlation with the target positions. This desynchronization located in parietal cortex can be explained by the general alerting when the subject was given a cue as to when a target will occur. It also suggests that the other subjects maintain an alert mental state during the baseline period and therefore no ERD was found in parietal cortex.

Our results support the notion that spectral shifts in low frequency reflect the coherence changes of cortical circuits. Basically, hand motor imagery activates neural networks in the cortical hand area which is here manifested as blocking of mu rhythm in the contralateral hand area (Fig. 2A). Functional magnetic resonance imaging (fMRI) studies of offline motor imagery have co-localized the activation in the contralateral primary motor (M1) cortex [32]. Less clear is the enhancement of mu rhythm in the ipsilateral hand area (also known as event-related synchronization), which is consistent with our previous findings [16–18]. A rebound in power in EEG mu and beta band after movement, at the contralateral side, has been well documented by Pfurtscheller and colleagues [4, 33]. It is interesting that fMRI study by Stefanovic et al. [34] reported the decrease of blood oxygenation level dependent (BOLD) signal predominantly in ipsilateral MI accompanying neuronal inhibition in a motor task. As the cortical basic rhythms are considered to be generated by thalamocortical input [11] and the inhibitory effect of the basal ganglia on the thalamus is considered essential for alpha and beta rhythm modulation, the mu activities suggest that the pyramidal neurons in contralateral sensorimotor cortex are released from the idling state and the ipsilateral neurons are deactivated by enhancing the idling state.

As the source imaging technique proposed in the present study can substantially enhance the spatial resolution of EEG on a single-trial basis, it can be applied to BCI processing in order to improve the performance of control. Particularly, the MNEFD approach targets the source imaging at the time-frequency region of interest, which suits the characteristic modulation associated with motor imagery. As shown in Table 3, the values for the source R index are always larger than those obtained for the same subject by using the raw scalp signals during the execution of the movement imagination. Previous studies using offline analysis [16–18] and online study [36] also showed that source imaging with enhanced spatial resolution facilitates the discrimination of movement imagination. The present results suggest the possibility to use the MNEFD approach to improve classification accuracy of the BCI based on noninvasive EEG recordings. The performance of the present MNEFD method in online BCI control is beyond the scope of the present study and will be addressed in future investigations.

In summary, we have developed a new oscillatory source imaging method and applied it to the study of the rhythmic activity of motor imagery during online BCI control. The present results in a group of human subjects are promising and suggest the MNEFD method merits further investigation and may provide a solution to oscillatory source imaging of brain activity.