Electrophysiological Source Imaging: a Noninvasive Window to Brain Dynamics

Abstract

Brain activity and connectivity are distributed over the 3-dimensional volume and evolves in time. It is of significance to image brain dynamics with high spatial and temporal resolution. Electroencephalography (EEG) and magnetoencephalography (MEG) are noninvasive measurements associated with complex neural activations and interactions that encode brain functions. Electrophysiological source imaging estimates the underlying brain electrical sources from EEG and MEG measurements. It offers increasingly improved spatial resolution and intrinsically high temporal resolution for imaging large-scale brain activity and connectivity in a wide range of timescales. Integration of electrophysiological source imaging and functional magnetic resonance imaging holds the potential to further enhance the spatiotemporal resolution and specificity that are unattainable with either technique alone. Here we review the methodological development in electrophysiological source imaging in the past three decades and envision its future advancement into a powerful functional neuroimaging technology for basic and clinical neuroscience applications.

1. INTRODUCTION

The human brain is comprised of roughly hundred billions of neurons (1). Each of these building blocks of the brain typically forms 10³-10⁴ synapses on average, forming a huge interconnected network with quadrillions of connections, which enables our brains to function as they do (2). Although, a great deal is known about neurons in a microscopic scale, little is known as to how these huge number of neurons (and synapses) work collectively to give rise to macroscopic brain signals and human behaviors.

Human brain functions are carried out by complex neural activations and interactions, which elevate electromagnetic signal changes (the primary effects) accompanied by hemodynamic and metabolic changes (the secondary effects). These changes are the basic sources for all noninvasive neuroimaging techniques. Depending on the signal sources, these imaging techniques can be divided into two categories. The first category is directly imaging neural electrical sources by detecting the induced electromagnetic signal changes using electric or magnetic sensors over the scalp. The common methods in this category are electroencephalography (EEG) and magnetoencephalography (MEG). The second category is indirect imaging approaches based on hemodynamic (cerebral blood flow, cerebral blood volume) and/or metabolic (glucose and oxygen utilization) changes associated with neural activity. The common methods in this category are functional magnetic resonance imaging (fMRI), positron emission tomography (PET), single photon emission computed tomography (SPECT), and near infrared spectroscopy.

Brain activation is accompanied by induced electrical activity due to excitation of neurons. The electrical activity of the brain can be analyzed in a variety of scales depending on the aim and focus, including the levels of ion channels, synapses, neurons, neuronal ensembles, lamina, columns, regions, and networks. Invasive electrophysiological recordings such as spike trains and local field potentials (LFP), as well as intracranial EEG, have contributed to our understanding of neuronal activities at microscopic or mesoscopic scales. Understanding of human brain dynamics at macroscopic scales, however, relies on noninvasive measurements such as EEG and MEG (3, 4). An important notion is that activation of single or multiple neurons does not lead to detectable EEG/MEG. Instead it requires the synchronous excitation of a large number of neurons to generate detectable EEG/MEG. Thus, EEG/MEG reports rich information about brain function (or dysfunction) encoded by dynamics of large-scale brain networks. This makes EEG and MEG studies highly relevant for clinical, cognitive, and behavioral brain research.

EEG/MEG has intrinsically high temporal resolution about underlying brain activity. The measured electrical or magnetic field changes reflect instantaneous neuronal currents, since electromagnetism in the brain can be treated in a quasi-static condition (5). Among existing neuroimaging modalities, EEG/MEG offers uniquely high temporal resolution. The challenge however is the limited spatial resolution of EEG/MEG. Significant efforts over the past three decades have greatly advanced electrophysiological source imaging based on EEG/MEG, resulting in high temporal and good spatial resolution that places EEG/MEG among the most widely used tools in neuroscience. This article reviews the principles of electrophysiological source imaging, the state of the art, and remaining challenges.

2. SOURCE IMAGING CONCEPTS

The electrophysiological source imaging (ESI) is the process of estimating neural electrical activity underlying non-invasive electromagnetic measurements such as EEG and MEG (4, 6). The principle of ESI is to counter the effect of volume conduction or field propagation for reconstructing brain sources from EEG/MEG. Solving this ill-posed problem encounters severe challenges if it is only treated mathematically. But significant progress has been made as anatomical and physiological constraints can be utilized in source estimation. In the following, we first describe the origins of EEG and MEG to lay the neurophysiological foundation for EEG/MEG based functional neuroimaging.

Given an imbalance between the ions inside and outside a neuron, a change occurs in the potential difference between the cell interior and exterior. When this change exceeds a threshold, the cell produces a sharp deflection of the transmembrane potential - the action potential (7). These action potentials propagate through neuronal axons and travel from one physical point to another, allowing neurons to communicate over a variety of spatial scales (3).

Electrical currents are produced by the movement of charges inside, outside, and along the neuronal cells. The electrical/magnetic fields due to these microscopic currents, when added constructively, can produce observable electrical or magnetic signals at the macroscopic level. The spatial distribution of such currents determines the overall effect at the scalp (3, 7, 8).

The organization of cortical pyramidal cells makes them most detectable with EEG/MEG among all types of cells in the brain. These cells are spatially aligned in such a way that when excited in synchrony, they produce detectable effects on the scalp. EEG/MEG signals primarily arise from post-synaptic currents. Currents that drive action potentials are generally negligible in the scale of EEG/MEG (9).

The elongated shape of the pyramidal neuron separates the current inflow and outflow by a short distance. In the extracellular space, such currents are viewed as a pair of current sink and source (monopoles), respectively. For far-field measurements like EEG/MEG, such a pair of current monopoles can be represented as a current dipole. Currents in extended cell populations may also be modeled as quadrapoles (10–12). Compared to monopole or multipole models, dipole models are most widely used. A dipole can be viewed as a discrete representation of current density, and has a clearer physical and physiological interpretation.

Given neuroelectric currents, finding the resulting electromagnetic signals on the scalp is called solving the forward problem of EEG/MEG. The electric/magnetic fields are generated by the currents that propagate through brain tissue and produce an effect at scalp sensors. When the average current density in each volumetric or areal element is modeled as a dipole (or less frequently a monopole or multipole), the forward problem can be solved with the superposition principle as the head is considered to be a linear system that generates additive effects of neuronal currents.

In the quasi-static condition, the forward problem can be expressed as solving Poisson’s equation (5). The solution depends on the tissue conductivity or permeability and the boundary conditions of electromagnetic fields. When the head geometry is modeled as one or multiple spheres with homogeneous conductivity, analytical solutions exist for the forward problem (7). An important development in ESI is to perform “constrained” source imaging by using anatomical information (13). Such anatomical constraint can be obtained from structural MRI, which offers soft-tissue contrast to segment the brain, skull, cerebrospinal fluid and scalp. For realistic-geometry head models, numeric solutions are attainable with either the boundary element method (BEM) by assuming isotropic conductivity (13, 14) or the finite element method (FEM) models by assuming anisotropic conductivity (15). In principle, FEM allows for more accurate head modeling than BEM; however, accurate information about tissue conductivity is not readily available. Since the introduction to brain source imaging in late 1980s (13, 14), the BEM has been most widely used for anatomically constrained ESI with EEG/MEG.

Unlike the deterministic forward problem, the inverse problem (i.e. estimating source distribution given scalp measurements) is known to be under-determined (16). The number of current sources is significantly greater than the number of measurements, despite high-density EEG/MEG. Inferring source distribution from measurements is ill-posed without applying constraint or regularization based on a priori information about the desired source characteristics or physiological assumptions. Regularization also helps to stabilize the solution against noise (17).

An important issue in ESI is the adequate spatial sampling. While MEG uses ~150+ channels of recordings, clinical EEG often uses less channels (e.g. 19–32 electrodes). Studies indicate that higher spatial sampling helps improve the precision of ESI (18, 19). A recent guideline of the International Federation of Clinical Neurophysiology suggests at least 64 channels of EEG should be used for ESI (20).

Fig. 1 illustrates the relationship of neuronal excitation, volume conduction, and scalp EEG/MEG, and the concept of the forward and inverse problem.

Fig. 1. Physiological Basis of EEG/MEG and the Biophysical Modeling of Forward and Inverse Problem.

The electrical activity of the brain is due to the ions (charges) that enter and exit the selective membrane of neurons. EEG and MEG signals represent scalp manifestation of the underlying activation of synchrony neuronal ensembles, which encode brain function or dysfunction. Current dipoles can be used to model neuronal currents. Maxwell’s equations can be solved to obtain the electric potential (EEG) and the magnetic field (MEG) – the so-called forward problem. Various numerical techniques, such as the boundary element method (BEM) and the finite element model (FEM), can be used to model head volume conductor linking neuronal current dipoles to EEG/MEG. The current density distribution of brain activity can be estimated from scalp EEG and MEG by means of signal processing algorithms – the so-called inverse problem.

3. SOURCE IMAGING ALGORITHMS

The ESI, as an inverse problem, can be solved by minimizing the difference between the measured signals and those generated by source estimates through the head volume conductor model. Depending on the source models adopted, various techniques have been developed to accomplish this goal. This section includes discussions about some of these techniques to illustrate the general strategies for ESI, but it is not intended to be a fully inclusive survey.

3.1. Equivalent Current Dipole (ECD) Models

In one of the earliest approaches for brain source imaging, the whole-brain activity is modeled with a limited number of dipoles (13, 21). Many events or tasks, such as epileptiform activity, involve electrical activity at a focal region, which may be modeled by an ECD located at the center of this region. Estimating one or two ECDs is an over-determined inverse problem that has a unique solution, since there are six parameters for each dipole while clinical EEG recordings normally use at least 19 electrodes.

There are multiple approaches for dipole source localization (76). One is to fit dipoles to scalp recordings separately for each moment; this approach is known as moving dipole solution since both the dipole location and orientation are allowed to vary in time. Another approach is to fix the dipole location but allow the orientation to vary freely, or to fix both the dipole location and orientation but only allow the dipole magnitude to vary. All these approaches are effective in different situations, if 1) the number of unknown parameters is smaller than the number of sensors, 2) source activity can be properly modeled as a single or few dipoles, and 3) the number of dipoles is known a priori.

A major challenge for dipole source localization is the nonlinear relationship between EEG/MEG and dipole locations. When the location of dipole is fixed and known, finding its orientation and amplitude becomes a linear problem (13). For moving dipole solution, the estimated locations of ECDs provide useful physiological or clinical information.

However, localization of more than two moving dipoles is challenging. The assumption of focal sources is not always valid and knowing the number of dipoles a priori is also not feasible for many applications. The nonlinearity of the moving dipole solution escalates as the number of dipoles increases. These all limit the scope of application for ECD localization, yet it is still being used in clinical neuroscience applications especially in MEG source localization.

3.2. Distributed Source Models

It is more realistic to describe EEG/MEG sources as a current density distribution, which can be modeled by a large number of current dipoles distributed over the 3-dimensional brain volume or 2-dimensional cortical surface. The latter is more preferable since the sources of EEG/MEG are primarily currents through pyramidal cells aligned perpendicular to the cortical surface, as proposed in the cortical current density (CCD) model (22). Since the dipole locations are fixed, the inverse problem of estimating dipole moments from EEG/MEG is linear but under-determined. A priori information is required to regularize the solutions to this inverse problem. Depending on the physiological plausibility of the regularization introduced, the distributed source estimates bear different characteristics and affinity to neural activity in a mesoscopic scale.

Minimum Norm (MN) Family

The minimum norm approach was the earliest solution to the bio-electromagnetic inverse problem with distributed source models (23). In this approach, the Euclidean norm (or L₂ norm) of distributed currents is used to regularize the least squares estimation of the inverse solution. This approach solves the under-determined inverse problem by choosing a unique source configuration with the minimum energy among all that equally fit the data. This approach has established characteristics in statistics and mathematics; however, its physiological plausibility is not fully understood despite its wide use in EEG/MEG source imaging.

MN solutions are biased for superficial sources, as superficial sources generate stronger fields with less energy due to their spatial vicinity to sensors. To mitigate this bias, one strategy is to weight current sources by the norm of the EEG/MEG signals that can be generated by each of them with a unitary magnitude. Introducing this weighting to MN regularization leads to the so-called weighted minimum norm (WMN) solution (24–27). A variation of WMN is the low resolution electromagnetic tomography (LORETA) in which the norm of the second-order spatial derivative of the current source distribution is minimized to ensure spatial coherence and smoothness (28). Post-hoc non-linear normalization can be further applied to the inverse solution obtained by MN or its variations. Using this strategy, standardized LORETA (sLORETA) (29) has shown improved spatial accuracy.

The MN approach and its variations have progressively evolved to become one of the most successful ESIs for EEG/MEG or their combination (30). A main drawback of the MN approach is that the resulting solutions tend to be overly smooth and widespread beyond the extent of underlying sources.

Beamforming and Scanning Methods

Alternatively, beamforming, originally established for radar and sonar applications (31), is increasingly used for ESI (32). The central idea is to refocus scalp EEG/MEG to their originating locations, by spatially filtering EEG/MEG specifically for each source location such that the output of the filter has minimal variance for every source location. As such, this approach is sometimes referred to as linearly constrained minimum variance (LCMV) localization (32, 33). The source estimates can be obtained by applying the optimized filters to the measured EEG/MEG. Extensions or variations of this idea have led to various beamforming approaches (34–36). However, the accuracy of beamforming solutions is of concern when underlying sources are correlated (36). Beamforming is sensitive to inaccuracy of the forward model, e.g. due to approximation in tissue conductivity. Perhaps for this reason, beamforming is more widely used for MEG than EEG, since the forward problem is relatively simpler for MEG.

A related but distinct technique uses a scanning strategy. That is, all source locations are scanned one by one to quantify how well the scalp map forwardly computed from assuming a dipole at each location, can be classified as being in the signal sub-space as opposed to being in the noise sub-space, while the signal and noise sub-spaces are orthogonal to each other and estimated from EEG or MEG (37, 38). An implementation of this approach is MUltiple SIgnal Classification (MUSIC) (37, 38), where the nonlinear optimization process of finding the dipole location is avoided. However, MUSIC also suffers if sources to be localized are correlated especially if they are close to each other (39). Variations of MUSIC, such as the recursively applied and projected MUSIC (RAP-MUSIC) (39) and the first principle vector (FINE) localization method (40), mitigate this problem to a certain degree.

Bayesian Methods

Bayesian inference provides a general framework in which many source imaging algorithms can be derived and interpreted. Given the Bayes’ theorem, Bayesian algorithms maximize the posterior distribution of sources given measurements while assuming a prior probabilistic distribution of the sources. If the prior distributions are not known, non-informative priors may be used (such as assigning equal probability to all possible outcomes), or the prior distribution itself may be marginalized by a set of hyper-parameters that can be estimated from data (41). The latter is known as empirical Bayes methods, which successively apply the Bayes rule to iteratively update and jointly estimate the prior and posterior distributions through expectation maximization (41, 42).

Minimum-norm and its variations can be viewed as special cases of Bayesian inference algorithms with different prior distributions. For instance, the MN is the maximum a posteriori (MAP) estimation when the prior distribution is assumed to be a Gaussian distribution (41). More importantly, Bayesian inference allows to incorporate physiological knowledge in the multivariate prior distribution to constrain the locations and connectivity of the unknown sources (43). Bayesian algorithms are more efficient if the prior distribution can be described with fewer hyper-parameters, e.g. by assuming independent sources (or a diagonal covariance matrix). Selecting and parameterizing the prior distribution is not straightforward, bringing both challenges and opportunities. Bayesian methods have many variants (43–46), such as Sparse Bayesian learning (SBL) algorithms (47), coherent maximum entropy on the mean (cMEM) (48), and dynamic Maximum a Posteriori Expectation-Maximization (dMAP-EM) (49), and remain an area of active research.

Sparsity-constrained Source Reconstruction

Sparsity is often assumed to avoid over-fitting in inverse problems, and has increasing applications in signal and image processing (43, 50–54). In the context of ESI, the sparsity in the source space means that it has much fewer non-zeros than zeros at specific moments, periods, or frequency bands. The sparsity assumption is not necessarily limited to the spatial domain, but other domains to which the source distribution may be transformed. For instance, sources may be piece-wise continuous; while it is not sparse, the edges in the source distribution or its spatial gradient is sparse (55). In general, sources should be described with most compressed (i.e. the least redundant) representations in order to be estimated from limited data – a notion also underlying compressed sensing (56).

FOCal Underdetermined System Solution (FOCUSS) (50), is an early example of using sparsity for ESI. However, the resulting solutions tend to be overly focused. Selective minimum norm method (57), minimum current estimate (58) and sparse source imaging (51) are other examples of sparse methods where the sparsity is imposed on distributed current density. L₁-norm regularization favors sparse source distribution (57–60); it assumes that the sources follow a Laplace prior distribution in Bayesian inference. In more recent studies, sparsity is imposed in other domains such as the gradient domain (55), wavelet coefficients domain (61), Laplacian domain (52) and even multiple domains combined together (54, 62, 63). Recent development in sparse ESI algorithms suggests the capability of estimating both the source locations and extents (54), reflecting the spatio-temporal characteristics of underlying brain sources and dynamics.

Fig. 2 depicts the main classes and families of source imaging algorithms, and illustrates an example of ESI using various algorithms.

Fig. 2. Electrophysiological Source Imaging (ESI) In a Glimpse.

Different classes and families of source imaging algorithms are depicted in this figure. In the center, an underlying brain activity with two separate sources and corresponding time-course of activity is simulated; the forward problem is solved and the scalp potential distribution is calculated (simulated EEG). On the upper left the solution of the dipole localization method for the given example is depicted. In the rest of the figure, major families of inverse algorithms are shown and some of the well-known algorithms in each family are listed as examples. For each case, the algorithm used to solve and produce the result is marked with an asterisk under the result. The mathematical formulation for the algorithm marked by the asterisk is provided under each solution for the reader’s interest. The interested reader should refer to the cited papers in the manuscript for more information regarding these inverse algorithms. The lead-field matrix from the current dipole distribution and the scalp potential is denoted by $\mathcal{K}$, the scalp potential φ, the current density distribution $\mathcal{j}$ and in the dipole model ${\mathcal{d}}_{\mathcal{j}}$. The inverse imaging operator (for the MN family) is denoted by $\mathcal{T}$ and λ and α are regularization parameters. In the beamforming family, the data covariance is denoted by ${\mathcal{R}}_{\varphi }$ and the spatial filter weights ${\mathcal{w}}_{\mathcal{r}}$. For the IRES algorithm $\mathcal{V}$ is the discrete gradient operator, Σ the estimated noise covariance and ε estimated noise power. MN: Minimum Norm, WMN: Weighted MN, LORETA: Low Resolution Electromagnetic Tomography, sLORETA: Standardized LORETA, dSPM: dynamic Statistical Parametric Mapping, LCMV: Linearly Constrained Minimum Variance, DICS: Dynamic Imaging of Coherent Sources, VBB: Vector-Based Beamformer, MUSIC: Multiple Classification Algorithm, FINE: First Principle Vector, VB-SSI: Variation-Based Sparse Source Imaging, IRES: Iteratively Reweighting Edge Sparsity, SBL: Sparse Bayesian Learning, FOCUSS: Focal Undetermined System Solution.

4. SOURCE IMAGING APPROACHES AND APPLICATIONS

Neural activity of interest to EEG and MEG includes ongoing activity in the absence of any task, or the responses evoked or induced by various events. Events such as inter-ictal spikes (IIS), seizures and motor imagination (MI) are endogenous events spontaneously emerging during cognitive or pathological processes. Evoked potentials (EP) and event related potentials (ERP) are related to external stimuli or tasks. Despite their differences, similar ESI techniques may be used.

4.1. Imaging External Event-Related Brain Activity

External events trigger rapid brain responses. EEG/MEG is uniquely suited to resolve the full response dynamics due to its high temporal resolution. ESI further enhances the spatial resolution and specificity towards comprehensive spatiotemporal imaging of the brain in action. Applications of ESI have been shown in numerous basic and clinical neuroscience studies. To name a few, ESI has been used to map the spatiotemporal responses underlying sensory processing, object recognition (64), disrupted visual processing in autistic patients (65), attention and consciousness (66), visual rivalry (67–69), visuomotor coordination (11), and speech recognition (70). The balanced temporal and spatial resolution of EEG/MEG earns themselves indispensable positions among existing tools for studying the spatiotemporal dynamics of large brain networks in humans.

4.2. Imaging Endogenous Event-Related and Spontaneous Brain Activity

ESI is also applicable to endogenous brain activity in normal and abnormal conditions. For motor imagery, a paradigm for brain computer interface (BCI) (71, 72), ESI enhances the capability of decoding subjects’ intent, relative to directly using sensor space signals (73, 74). Combining ESI and fMRI data has also shed light on the nature of event related de/synchronization (75).

ESI has been widely used in epilepsy source localization, including inter-ictal discharges (76–81). It has been shown in a study with over 150 patients that source imaging using high density EEG recordings, has higher sensitivity and specificity compared to other imaging modalities such as MRI and PET (78). When accurate forward models and high density EEG caps are utilized, the location and dynamics of the epileptiform activities can be estimated to potentially inform clinical evaluation. Furthermore, ESI has been used successfully to localize seizures (82–84). Seizures are more difficult to localize due to patients’ head movement and consequently low signal-to-noise ratio. The oscillatory nature of seizure-related activity requires spatiotemporal ESI (83). The theta oscillation and its role in consciousness have also been characterized with ESI (85). Fig. 3 presents an example of seizure source imaging in partial epilepsy patients, validated against intracranial recordings or surgical outcome.

Fig. 3. Electrophysiological Source Imaging of Seizure.

Seizure onset zones (SOZs) and the source time-frequency representations (TFRs) estimated from a typical seizure in patient 1 with frontal lobe epilepsy (top) and patient 2 with temporal lobe epilepsy (bottom). The estimated SOZ (left and middle panels, yellow to orange colorbar) is co-localized with surgically resected zones (shown in green) in patient 1, and SPECT foci (green) in patient 2. The TFR (right panels) shows the time-frequency evolution at the maximal estimated SOZ point. Intracranial electrodes were implanted in patient 2 (spherical dots) and SOZs determined from intracranial seizure recordings (Red Spherical dots). (Adapted with permission from (83))

Imaging on-going and spontaneous brain activity is another domain of applications for ESI. In the resting state, ESI has uncovered frequency-dependent functional networks and their dynamics (86–88). In disease states, ESI has revealed epileptic network characteristics in patients (89, 90), even in the absence of epileptic activity.

5. IMAGING FUNCTIONAL CONNECTIVITY

Two notions are essential to the brain’s functional organization: functional segregation and integration (91). While some brain regions or networks are specialized in performing particular tasks, others integrate information (91), leading to complex cognition, behavior, or pathology. ESI can help to understand and disentangle these processes (6).

5.1. Functional vs. Effective Connectivity

Functional connectivity concerns whether neural activity in a pair of regions (denoted as A and B) is correlated or coherent (92, 93). As correlation or coherence does not imply causation, functional connectivity does not report the directionality of the interaction between A and B. This sets functional connectivity apart from directional connectivity or effective connectivity (94), which measures the causal relationship between A and B, e.g. Granger causality (95). Causality or directionality is often inferred from time series: if the past activity in A predicts the current or future activity in B, A causes or drives B through a connectivity pointing from A to B (96).

5.2. Electrophysiological Connectome (eConnectome)

ESI allows to map functional/effective connectivity in source space – a notion of the electrophysiological connectome (eConnectome) (6, 97). Algorithms have been developed for estimating functional connectivity from electrophysiological source signals derived from EEG (or MEG), including directed transfer function (DTF) algorithm (98), the adaptive DTF (ADTF) (99–101), direct DTF (dDTF) (102), and partial directed coherence (PDC) (103). The eConnectome approach (104, 105) has stirred interest in studying pathological networks such as epilepsy, where determining the source that drives seizures is of particular importance (106). MEG is also an effective tool for studying resting network functional connectivity and has been shown to correlate well with fMRI studies with the added value of high temporal resolution (107). Such approaches have been applied to determine epileptic networks with positive results indicating the merits of integrating ESI with functional connectivity in mapping brain networks (108, 109).

Fig. 4 schematically illustrates the eConnectome approach for mapping functional brain networks with ESI.

Fig. 4. The Concept of Electrophysiological Connectome (eConnectome).

Electrophysiological source imaging can not only image brain activity but also functional connectivity of the brain. The eConnectome approach is to estimate brain network dynamics from noninvasive surface measurements such as EEG and MEG. The location of activity (nodes), the time-course of activity at such nodes, the dynamics/connectivity among these nodes (links) can be estimated from EEG/MEG to reveal the underlying brain networks. ESI is a key element in realizing this goal. eConnectome has been shown to be effective and accurate in imaging brain network dynamics in the source domain. In the figure, arrows show information flows and directional functional connectivity, or causality.

5.3. Dynamic Causal Modeling

The aforementioned methods for directional connectivity are all data-driven. Model-based connectivity measures such as the dynamic causal modeling (DCM) has also drawn much attention (110). While there are some similarities between DCM and Granger causality algorithms, they are fundamentally different (111). Model-based methods depend on the choice of the model and its parameters; thus many different parameters need to be tested to ensure unbiased results (112). This brings higher computational demand but may lead to valuable insights if the model and parameters can be appropriately selected.

6. MULTIMODAL NEUROIMAGING

Where EEG and MEG fall short in neural imaging is opposite to where fMRI excels, and vice versa. EEG and MEG can resolve neural events with high temporal resolution but limited spatial resolution (113); fMRI localizes brain activity with millimeter precision but cannot probe the rapidly changing neuronal dynamics (114). Their complementary strengths and limitations have motivated researchers to integrate EEG/MEG and fMRI towards a multimodal imaging tool that is more powerful than each of them alone (115–117). The feasibility of concurrent EEG and fMRI acquisition makes EEG more preferable than MEG, especially for functional imaging in task-free or self-paced states that are hard to precisely replicate in separate experiments.

6.1. Synaptic Activity: the Common Origin of fMRI and EEG/MEG

The origin of fMRI is rather complex and incompletely understood (118). The most widely used fMRI signal is blood oxygenation level dependent (BOLD) (119) and observable with rapid pulse sequences sensitized to the T₂*-weighted contrast (120–122). What causes the BOLD effect is the varying concentration of deoxygenated hemoglobin relative to the total hemoglobin that is either deoxygenated or oxygenated (123). As neural activity elevates, local oxygen consumption and demand increases, triggering vasodilation of arterioles and capillaries to increase cerebral blood flow (124), which supplies oxygenated hemoglobin in excess of the metabolic rate of oxygen consumption (125). As a result, the concentration of deoxygenated hemoglobin decreases in capillary and venous vessels, reducing the paramagnetic susceptibility effect and thus increasing the T₂*-weighted signal (123). Such a cascade of metabolic and vascular events, known as the neurovascular coupling, involves complex signaling among neurons, astrocytes, and the local vasculature (124, 126, 127), and remains a topic of active research enabled by emerging tools (128).

The mechanism of neurovascular coupling is incompletely understood but essential to interpretation of fMRI (114, 118). Increasing evidence suggests that the primary source of fMRI is synaptic activity, rather than spiking activity, in the gray matter (129, 130). In primates, synaptic activity consumes much more energy than action potentials (131); the gray matter is more densely vascularized than the white matter (132). As such, synaptic activity is a stronger driver of energy demand, and the gray-matter vasculature further amplifies the metabolic fluctuation into an even greater vascular effect. Importantly, the synaptic contribution to the BOLD signal implies that fMRI and EEG/MEG may reflect highly distinct manifestations of the same physiological origin. Of synaptic activity in the gray matter, the metabolic and vascular effects give rise to fMRI through neurovascular coupling with a severe loss in temporal specificity; the electromagnetic effect gives rise to EEG/MEG through head volume conduction with a severe loss in spatial specificity.

Studies that have directly compared fMRI and neural signals lend strong support to the above notion (129, 130, 133–135). In neural signals, LFP reflects the synaptic input to a neuronal ensemble, and multi-unit activity (MUA) reflects its spiking output. When recorded simultaneously during sensory stimulation, the BOLD response is more correlated with LFP than MUA (129, 130). After dissociating neuronal input and output, the BOLD response is still correlated with LFP but not with MUA (133, 134). Although it is perhaps most notable in the gamma range, the LFP-BOLD correlation is not limited to any single frequency, but spans a broadband (135). In the absence of any sensory input, spontaneous BOLD fluctuations are still correlated with underlying synaptic activity observed with LFP (136, 137), ECoG (138), and EEG (139, 140). Given these findings, it is reasonable to state that fMRI and EEG signals, to a large extent, share a common origin of cortical synaptic activity, but reflect its distribution and dynamics in highly different spatial and temporal scales with distinctive sensitivity, resolution, and specificity (141).

6.2. Joint Solutions to Two Inverse Problems

Most fMRI-EEG/MEG combined imaging methods are being developed in the context of two inverse problems. One is a spatially inverse problem, i.e. spatial localization of temporally resolved EEG/MEG signals (8, 142). The other is a temporally inverse problem, i.e. temporal decomposition of spatially resolved fMRI signals (143, 144). When fMRI is utilized to help solve the EEG/MEG inverse problem (i.e. fMRI-constrained EEG/MEG ESI), the solution benefits from the spatial precision and resolution of fMRI while inheriting the intrinsic temporal resolution from EEG/MEG (115, 117). When EEG is utilized to solve the fMRI inverse problem (i.e. EEG-informed fMRI), the solution benefits from the use of EEG to separate neural components or events in time or frequency while inheriting the intrinsic spatial resolution of fMRI (116, 140). What is more preferable is a “symmetric” strategy to arrive at the estimation of a common set of unknown sources that simultaneously fit both fMRI and EEG measurements. Such a strategy is lacking despite initial progress (145), and awaits further quantitative understanding and modeling of the basis of and coupling between fMRI and EEG.

Fig. 5 schematically illustrates the joint inverse problem of fMRI and EEG/MEG.

Fig. 5. Illustration of the Common Origin and the Joint Inverse Solutions of fMRI and EEG/MEG.

The spatially inverse problem refers to spatial localization and imaging of temporally resolved EEG/MEG signals. The temporally inverse problem refers to temporal decomposition of spatially resolved fMRI signals. Both fMRI and EEG (or MEG) have synaptic origins.

6.3. Using fMRI for the EEG/MEG Inverse Solution

The fMRI-constrained EEG/MEG analysis has evolved from being mostly empirical to being progressively more principled. Perhaps, the earliest and simplest method is to use fMRI activation foci to place multiple current dipoles, and then to fit the dipole moments to event-related potentials (146). The estimated time series of each dipole reports the response dynamics at the corresponding fMRI hotspot. While overly simplified, this method is valuable in revealing the temporal sequence of task-evoked neural responses underlying perception or cognition (146, 147).

An alternative method is to improve the EEG/MEG-based estimation of cortical current density by using algorithms with fMRI-biased regularization (97, 148–150). Although they may be implemented in different theoretical frameworks, e.g. weighted minimum norm (97), Wiener filter (149, 150), and Bayesian inference (43), such algorithms all share and utilize a common assumption that EEG/MEG sources are more likely to occur at where fMRI views as activated. Often critical is the choice of the hyper-parameter that controls the fMRI bias. This choice is mostly empirical for the lack of biophysical and quantitative interpretation of fMRI activation. In fMRI, a voxel being activated means that the voxel’s signal is significantly different from noise and it is predictable by the stimuli and tasks of interest. Note that this statistical meaning does not inform any physical characteristic about neuroelectric activity. A more rational choice requires further understanding and modeling of the relationships among EEG/MEG, fMRI, and stimuli.

It is feasible to model the relationship between fMRI and EEG/MEG given relatively simple stimuli that repeat as discrete events and/or in alternating blocks (115). For event-related or block-design paradigms, it has been shown that the BOLD effect size given repeated stimuli in a long time-scale is approximately proportional to the integral of the power of stimulus-evoked synaptic activity in a short period following each stimulus (151). This relationship yields a quantitative model to relate fMRI to EEG/MEG, enabling a more principled way for using fMRI to constrain the EEG/MEG inverse solution for stimulus-evoked responses (150–152). However, this model fails to account for negative BOLD responses (153), or changes in spontaneous activity (154). The limitation is worth noting, since spontaneous activity consumes most energy (155), drives BOLD fluctuations (143), and interacts with tasks or stimuli (156, 157).

6.4. Using EEG for the fMRI Inverse Solution

Compared to fMRI-constrained EEG/MEG inverse solutions, using EEG to inform fMRI mapping is more straight-forward (140, 158). The central idea is to extract features from EEG, and then relate them to the voxel-wise fMRI signal. With this idea, one may generate high-resolution activity maps presumably underlying the EEG features of interest (158), or address the EEG correlates to fMRI activity at specific voxels, regions, or networks (144, 159).

Features of EEG are often defined and extracted by frequencies or frequency bands (e.g. delta, theta, alpha, beta, and gamma). Different frequency components can be extracted by simply filtering EEG within narrow bands. Such band-pass filtered signals, often coined as oscillations or rhythms, have been thought to bear important functional roles (160) and indicate brain states (161). Although the oscillation itself is too fast for fMRI to follow, its power or amplitude fluctuation falls in a similar time range as the fMRI signal (162). Correlation between the fMRI signal and the power fluctuation of an oscillation at a given frequency has been used to map the network that generates brain rhythms (140, 143). An excellent example is the alpha rhythm – a hallmark phenomenon in EEG. Studies have shown that the alpha-band power is correlated to the fMRI signal from visual and sensorimotor cortices (163), attention network (164, 165), and thalamic nuclei (166). The EEG-fMRI correlation in the thalamus demonstrates the synergistic merit of fMRI-EEG, since the thalamus is too far away to generate reliable EEG and localizing such deep sources with EEG alone is challenging. Beyond the alpha band, other frequency bands have also been studied in a similar fashion, e.g. showing the negative coupling between frontal theta rhythm and default-mode network (167).

Caution should be taken when correlating the fMRI signal to a single frequency component of EEG. This is because EEG, or its underlying neural activity, nearly never manifests itself as a single-frequency rhythm; instead, it always contains a mixture of rhythms, and an arrhythmic component that follows a power-law (1/f) distribution across a broadband (168). Different frequency components are often related to each other such that the magnitude of one frequency is coupled to the magnitude or phase of another frequency (169). Therefore, it is important to consider all frequency components together as a collective account of the fMRI signal, while disentangling the differential contributions from broadband vs. narrowband components (139, 170), as well as oscillations at different frequencies (171). In this regard, recent findings suggest that (broadband) arrhythmic and (narrowband) rhythmic processes account for global and modular patterns of functional connectivity observed with resting state fMRI, respectively (170). The EEG correlates of resting state networks exhibit their distinct spectral signatures, characterizing the way by which neural oscillations support inter-regional interactions within networks (144, 159).

From EEG, features can also be extracted as characteristic spatial patterns or microstates that reoccur over time (8, 172). The EEG microstates have been found to correlate with the fMRI signal in the resting state (173–175). Moreover, for event-like tasks or stimuli, EEG features may also be extracted from temporal variations in single-trial potentials, and their correlations with fMRI reveal rich temporal dynamics of information processing in task-evoked neural networks (176, 177).

6.5. Challenges and Opportunities

A critical challenge for integrating fMRI and EEG is the strong electromagnetic interference that causes artifacts in simultaneously acquired fMRI and EEG signals (178). The artifacts are more of concern for EEG than for fMRI, and tend to deteriorate in higher fields (179). Existing ways of removing such recording artifacts are mostly in post-processing algorithms (180). Hardware solutions that reduce or eliminate the artifacts are needed for broader and more routine applications of fMRI-EEG. Initial progress is encouraging and merits further development (181).

Combining fMRI and EEG serves to bridge brain signals across spatial and temporal scales. Questions remain as to how functional information reported with fMRI-EEG relates to the brain’s structural characteristics, the roles of different neuronal circuits, and the excitation-inhibition balance. Such questions are approachable from multiple perspectives, some of which are speculated as below.

For example, the relationships between fMRI and EEG in distinct frequencies likely reflect the spectral signatures of regional activity and inter-regional interaction (144, 182, 183). Such spectral signatures may be closely related to the topological properties of structural connectivity. As illustrated in Fig. 6, neuronal communication takes time, depending on the distance and the velocity of signaling between neurons (Fig. 6.a). The frequency of neuronal oscillation is inversely related to the cumulative time delay for information to travel out of a region and back to itself through an indirect poly-synaptic pathway (Fig. 6.b). As a region is involved in many such pathways with various time delays (Fig. 6.c), the spectrum of regional activity should indicate the histogram of the time delays of all structural circuits in which this region is actively involved for a given period of time (Fig. 6.d). It is plausible to predict that when a region (or a pair of regions) is engaged in relatively local-scale processing, it tends to involve shorter pathways and less synapses, giving rise to shorter time periods and thus higher frequency components will be more evident. Such interpretation and prediction remain to be tested, but represent a plausible scenario about the fundamental relationships between circuit structures and dynamics.

Fig. 6. The Structure-Function Relationship in Spectral Signatures of Brain Activity and Connectivity Observable with fMRI-EEG/MEG.

(a) The time delay between two interconnected regions depends on the axonal length (l), conduction velocity (v), and synaptic delay (δ). (b) The cumulative delay of a circular path entails the sum of delays via every path, and contributes to network dynamics at one specific frequency. (c) A region can be involved in multiple circuits with distinct frequencies. (d) The spectrum at this region indicates its relative involvements in all circuits for a given period.

Spectral information in EEG/MEG or EEG-fMRI may be interpreted in relation to feedforward and feedback pathways. Recent studies suggest that the frequency of neural oscillations marks the directionality of network interactions: low-frequency oscillations (e.g. alpha) reflect feedback processes and high-frequency oscillations (e.g. gamma) reflect feedforward processes (184–186). Such findings are consistent with the observable laminar profile of fMRI (187), LFP-based phase relationship (184), and effects of neuromodulation (188). Although the findings need further replication and validation, conceivable implications include using the frequency information to infer directional networks, and to investigate the dynamics and roles of feedforward vs. feedback pathways against theories in computational neuroscience, e.g. predictive coding (189, 190) and free-energy principles (191).

7. CONCLUSIONS AND FUTURE TRENDS

EEG and MEG are non-invasive measurements, which record brain electromagnetic activity with high temporal resolution. Once ESI is applied to high density EEG/MEG recordings, brain electrical activity can be imaged with sub-lobar resolution in the order of ~ 5mm, or the level of cortical gyri and sulci (192). ESI also disentangles the sensor-level signals to reveal electrophysiological dynamics in regional activity or inter-regional connectivity, yielding new insights to brain functions in health (193) and disease (194–196).

Given its current advances and continuing development, ESI will be increasingly used for clinical applications. The inexpensive EEG setups available in most clinical settings, the availability of computers, and the accessible open-source ESI analysis tools will enable broad applications of EEG source imaging. Furthermore, the capability of imaging dynamic brain activity from the whole brain, makes ESI a desirable means for studying large brain networks in human.

While ESI can provide highly valuable information about brain networks and dynamics, it also has the advantage of being integrated with other modalities such as the fMRI to combine the high temporal resolution of the EEG with the high spatial resolution of fMRI (see Section 6). The need for increasing the spatiotemporal resolution in imaging brain function is ongoing (197). It is foreseeable that developments in designing better ESI algorithms, and combining EEG/MEG with other neural imaging or modulation techniques, such as transcranial focus ultrasound (198), will be at the frontiers in advancing functional neuroimaging.

8. Box 1 – Open Source ESI Software.

ESI algorithms are widely available to public via free and open source software. Some of these toolboxes are capable of performing source imaging as well as connectivity analysis, like eConnectome (104), FieldTrip (199) and MNE (200) and Nutmeg (201), while some mainly focus on source imaging, such as the BrainStorm (202). There are toolboxes which are more specialized for time-series and component analysis such as the EEGLAB (203) toolbox and some are more specialized with scalp topographical analysis and clustering such as CARTOOL (204). The statistical parameter mapping (SPM) toolbox (205, 206) is a widely used toolbox which was developed initially for fMRI and DCM analysis and now includes EEG/MEG analysis. There are some toolboxes such as the partial directed coherence (PDC) (207) and the source information flow (SIFT) (208) toolboxes, which are specialized for connectivity analysis without much work on source imaging.

Subject specific head models derived usually from subject’s MRI is necessary to solve the forward problem and is ultimately used in ESI; such toolboxes include the FreeSurfer (209), BrainSuite (210) and BrainVISA Anatomist (211). The OpenMEEG (212) can be used to make subject specific BEM models.

There are many more freely available open source toolboxes online as well as commercial software. We intended to provide a few examples of the more well-known toolboxes to guide the readers. All of the introduced toolboxes above have extensive online tutorial and user guides and are relatively easy to set up and work with.