Inference of Whole Brain Electrophysiological Networks Through Multimodal Integration of Simultaneous Scalp and Intracranial EEG

Abstract

Brain imaging research has transitioned over the past decades from identifying isolated regions of task-evoked activation to characterizing the spatiotemporal dynamics of large-scale brain networks. Electrophysiological signals are the direct manifestation of brain activity; thus, characterizing whole-brain electrophysiological networks (WBEN) can serve as a fundamental tool for neuroscience studies and clinical applications. In this work, we introduce a framework for integrating scalp EEG and intracranial EEG (iEEG) for WBEN estimation through a principled state-space modeling approach, where an Expectation-Maximization (EM) algorithm is designed to infer the state va riables and brain connectivity simultaneously. We validated the proposed method on synthetic data, and the results revealed improved performance compared to traditional two-step methods using scalp EEG only, demonstrating the importance of including iEEG signals for WBEN estimation. For real data with simultaneous EEG and iEEG, we applied the developed framework to understand the information flows during encoding and maintenance phases of a working memory task. The information flows between subcortical and cortical regions are delineated, highlighting more significant information flows from cortical to subcortical regions during encoding than during maintenance. The results are consistent with previous research findings, but from a whole-brain perspective, which underscores the unique utility of the proposed framework.

1. Introduction

Brain networks represent the intricate and dynamic connectivity of neurons that facilitates communication across different brain regions. These networks are essential for supporting cognitive functions, from basic sensory processing to complex decision-making [1, 2]. Existing studies have suggested that accurately inferred brain connectivity patterns can help gain insights into the coordination and interactions between different brain regions [3, 4], reveal the brain network underpinnings of cognitive processes, and uncover the mechanisms and biomarkers of neuropsychiatric diseases [5, 6]. Over the past decades, functional Magnetic Resonance Imaging (fMRI) has been the most widely used brain imaging modality for functional brain network modeling and analysis [7]. By analyzing fMRI data, researchers can explore the connectomes of the human brain across different cognitive tasks [8, 9], consciousness states [10], degenerative diseases [11], and mental disorders [12]. However, fMRI has limitations, including non-portability and low temporal resolution, which restrict its use in applications that require characterization of instantaneous brain activity at sub-millisecond timescales [13].

Neuroimaging techniques with high temporal resolution, such as Magnetoencephalography (MEG) and Electroencephalography (EEG), can be used to measure electrical and magnetic brain signals, offering a direct measurement of brain activity rather than metabolic signals. To establish the electrophysiological connectome of the human brain, inferring whole-brain electrophysiological networks (WBEN) is important since it provides a direct network-level delineation of brain connectivity. Existing studies that use MEG/EEG to reconstruct brain electrophysiological networks have typically adopted a two-step procedure, with EEG/MEG source imaging (ESI; Step 1) followed by brain connectivity measures (Step 2), such as phase–amplitude coupling [14], coherence [15], phase synchronization [16], and Granger causality [17].

Existing ESI frameworks based on brain source localization suffer from low accuracy in estimating whole-brain activity across regions due to the ill-posedness of the inverse problem and are theoretically limited by the Restricted Isometry Property (RIP, the definition provided in Appendix A.1) [18]. While a number of recent works have sought to alleviate these limitations through refined spatial priors and filtering strategies [19–21], the fundamental challenges remain significant. To address the limitations of two-step approaches, Yang et al. proposed a one-step state-space model that jointly estimates source localization and dynamic connectivity by modeling ROI mean activities with time-varying autoregression [22]. Pirondini et al. developed computationally efficient algorithms combining spatial covariance estimation, linear state-space dynamics, and sparsity constraints, achieving improved source localization performance with significant reduction in computation time through steady-state Kalman filtering [23]. However, estimating whole-brain networks poses additional challenges for the ESI problem, as it further relaxes sparsity constraints [24]. More recently, Soleimani et al. demonstrated that Granger causal links can be directly inferred from MEG measurements without an intermediate source localization step, achieving superior performance with low false alarm rates through integrated parameter estimation and statistical analysis [25]. Additionally, Sanchez-Bornot et al. introduced multiple penalized state-space models with novel algorithms based on backpropagation, gradient descent, and alternating least squares, enabling simultaneous solution of source localization and functional connectivity problems for thousands of cortical sources using data-driven regularization [26].

In addition to non-invasive modalities, invasive neuroimaging technologies such as intracranial Electroencephalography (iEEG), including Electrocorticography (ECoG) and stereoelectroencephalography (sEEG), which place subdural electrodes on the brain surface (ECoG) or penetrating electrodes in subcortical regions (sEEG), can achieve more accurate connectivity mapping among different regions of interest with high temporal resolution [27, 28]. Recent studies have leveraged simultaneous scalp EEG and iEEG recordings to improve the reliability of electrophysiological source imaging. For example, Jiao et al. proposed an explainable deep learning framework (XDL-ESI) that unrolls optimization algorithms into neural networks and achieves accurate, interpretable source localization validated on simultaneous EEG-iEEG data [29]. Despite these advances, iEEG recordings remain limited in spatial coverage due to their invasive nature, making the brain only partially observable through iEEG measurement. As iEEG electrodes can only cover part of the brain, important neural activity or connectivity patterns might be undetected in other regions. For example, in the analysis of seizure onset zones in epilepsy, Shu et al. used MEG/EEG source imaging and identified interictal spikes that are missed by iEEG [30], highlighting the value of synergy between scalp EEG and invasive iEEG. The partial observability of iEEG and the challenges of localizing networked brain sources from EEG motivate the integration of scalp EEG and iEEG as complementary modalities. To address the above issues, we propose to integrate scalp EEG and iEEG to provide a accurate delineation of electrophysiological activation and connectivity at the whole-brain scale. Integrating iEEG and scalp EEG can yield a faithful reconstruction of WBEN, however, a principled multimodal integration modeling and inference framework has not been explored before. Early work leveraging state-space models solving the ESI and brain networks demonstrated significant potential [22, 25, 31]. In this paper, we propose a new inference framework based on state-space dynamical systems and Bayesian inference that leverages multimodal fusion of scalp EEG and iEEG for WBEN estimation. This work represents a new and unified computational paradigm that integrates scalp EEG and intracranial recordings for whole-brain network inference, treating both modalities as complementary observations of shared underlying neural dynamics within a rigorous Bayesian state-space paradigm. This framework enables comprehensive characterization of whole-brain electrophysiological networks when simultaneous recordings are available. The pipeline of the proposed approach is illustrated in Fig. 1.

Figure 1:

The overall pipeline of integration of EEG and iEEG for brain network reconstruction

2. Method

2.1. Basic problem definition

The linear discrete dynamic system of brain sources, as well as the linear model of EEG and iEEG observations, can be defined as:

$$\begin{gathered} x_t=\sum _{k=1}^K{\mathbf{\Phi }}_k{x}_{t-k}+n_t,t=1,\dots ,T,x_t\in {\mathbf{R}}^N, \\ {y}_t=L{x}_t+w_t,y_t\in {\mathbf{R}}^M, \\ {z}_t=C{x}_t+e_t,z_t\in {\mathbf{R}}^O, \end{gathered}$$ (1)

where $N,M$, and $O$ are the number of the source regions, EEG electrodes, and iEEG electrodes, respectively. ${\mathbf{\Phi }}_k\in {\mathbf{R}}^{N\times N}$ is the state transition matrix that delineates the impact of the source state at time $t-k$ to $t.n_t\in {\mathbf{R}}^N,n_t~𝒩(\mathbf{0},\mathbf{Q})$ is the noise in source state space which is assumed to be a multivariate Gaussian distribution with mean 0 and diagonal covariance matrix $\mathbf{Q}.L\in {\mathbf{R}}^{M\times N}$ is the lead field matrix. $w_t\in {\mathbf{R}}^M,w_t~𝒩(0,\mathbf{P})$ is the measurement noise in EEG observation which is also assumed to be a multivariate Gaussian distribution with mean 0 and a covariance matrix $\mathbf{P}$ that is assumed to be known by measuring on a realistic head model. And $C\in {\mathbf{R}}^{O\times N}$ is a full-row rank transformation matrix that selects the source signal where its region can be observed by iEEG directly. $e_t\in {\mathbf{R}}^O,e_t~𝒩(\mathbf{0},\mathbf{S})$ is the iEEG observational noise which is assumed to follow multivariate Gaussian distribution with mean 0 and a covariance matrix $\mathbf{S}$ that is also can be measured in a similar manner as $\mathbf{P}$. Thus, according to the model definition, the parameters that need to be estimated are $\mathbf{\Phi }$ and $\mathbf{Q}$. Define the unknown parameters as $\theta =\{\mathbf{\Phi },\mathbf{Q}\}$. Then, the log-likelihood can be written in the form:

$$\text{log}p\left(y_{1:T},z_{1:T}\mid \theta \right),$$ (2)

The log-likelihood of the model can be defined as

$$\mathit{\text{log}}p\left(y_{1:T},z_{1:T}\mid \theta \right),\theta =\left\{\mathbf{\Phi },\mathbf{Q}\right\}.$$ (3)

Since the number of observations substantially exceeds the number of sources, the inverse estimation problem is highly ill-posed. To alleviate this phenomenon and simplify the problem, a regularization term was added based on the assumption that the connection from all other regions to a given region is sparse. In this case, the regularized maximum log-likelihood of parameters for the model can be defined as:

$${\theta }^{*}=\underset{\theta }{\text{arg max}}\mathit{\text{logp}}\left(y_{1:T},z_{1:T}\mid \theta \right)-\lambda \sum _{n=1}^N{‖{\mathbf{\Phi }}_{n,:}‖}_1,$$ (4)

where ${\Phi }_{n,\text{:}}$ is the $n^{\mathit{\text{th}}}$ row of the state transition matrix. $\lambda$ is a regularization weight for model estimation that can be decided manually according to the experience or by grid search [25].

2.2. An Expectation-Maximization estimation framework

In the domain of statistical methodology, the Expectation–Maximization (EM) algorithm serves as an iterative computational approach employed to determine the maximum likelihood, whether it pertains to local optima or the maximum of posterior estimations (MAP) of parameters in the context of a statistical model. These models, in particular, depend on latent variables hidden from direct observations, underscoring the importance of this sophisticated approach. The EM is characterized by iterative execution, with two pivotal steps: firstly, the Expectation (E) step, where a Q-function is calculated to encapsulate the expected value of the log-likelihood. Secondly, the Maximization (M) step, during which optimal parameters are derived to maximize the anticipated log-likelihood obtained during the E step. Consequently, these derived parameters play an important role in illustrating the distributional characteristics of the latent variables to prepare for the subsequent E step within the iterative loop. Since the data distribution in the source domain is unknown, the Expectation-Maximization framework takes the source state as the latent variable, which can be applied to find the optimal estimation of the log-likelihood and then obtain the approximated state in the source domain with the problem defined above.

We first illustrate the E-step by starting from Eq.(2) and using the facts that the observations of EEG and iEEG are conditional independent on the source state $x$. The log-likelihood can be rewritten and derived by introducing source state $x_{1:T}$ as

$$\begin{gathered} \text{log}p\left(y_{1:T},z_{1:T},x_{1:T}\mid \theta \right)=\text{log}p\left(y_{1:T},z_{1:T}\mid x_{1:T},\theta \right)+\text{log}p\left(x_{1:T}\mid \theta \right) \\ =\text{log}p\left(y_{1:T}\mid x_{1:T},\theta \right)+\text{log}p\left(z_{1:T}\mid x_{1:T},\theta \right)+\text{log}p\left(x_{1:T}\mid \theta \right). \end{gathered}$$ (5)

The first and second terms of the right-hand side of the second row can be easily obtained based on the Gaussian noise assumption while source $x_t$ is given which are

$$\text{log}p\left(y_{1:T}\mid x_{1:T},\theta \right)=\sum _{t=1}^T\text{log}p\left(y_t\mid x_t,\theta \right)=-\frac{T}{2}\mathit{\text{log}}\left(2\pi \left|\mathbf{P}\right|\right)-\frac{1}{2}\sum _{t=1}^T{‖y_t-L{x}_t‖}_{{\mathbf{P}}^{-1}},$$ (6)

and

$$\text{log}p\left(z_{1:T}\mid x_{1:T},\theta \right)=\sum _{t=1}^T\text{log}p\left(z_t\mid x_t,\theta \right)=-\frac{T}{2}\mathit{\text{log}}\left(2\pi \left|\mathbf{S}\right|\right)-\frac{1}{2}\sum _{t=1}^T{‖z_t-C{x}_t‖}_{{\mathbf{S}}^{-1}},$$ (7)

where $|\cdot |$ is the matrix determinant, and $\Vert v{\Vert }_W=v^{T}Wv$ is the quadratic form in the exponential term of multivariate Gaussian distribution. The last term can be obtained based on the linear dynamical model from (1). and utilize the presumption that $\mathbf{Q}$ is a diagonal covariance matrix with the items $\left\{{\sigma }_i^2,i=1,\dots ,N\right\}$ on its diagonal, then we have

$$\text{log}p\left(x_{1:T}\mid \theta \right)=\sum _{t=1}^T\text{log}p\left(x_t\mid \theta \right)=-\frac{T}{2}\mathit{\text{log}}\left(2\pi \right|\mathbf{Q}\left|\right)-\frac{1}{2}\sum _{i=1}^M\frac{1}{{\sigma }_i^2}{‖{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i-\underset{\_}{\mathbf{X}}{\hat{\varphi }}_i‖}_2^2,$$ (8)

where ${\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i={\left[x_{i,K+1},x_{i,K+2},\dots ,x_{i,T}\right]}^T,\underset{\_}{\mathbf{X}}=\left[{\left[x_{1,K+1},x_{1,K+2},\dots ,x_{1,T}\right]}^T,\dots ,{\left[x_{\mathrm{1,1}},x_{\mathrm{1,2}},\dots ,x_{1,T-K}\right]}^T,{\left[x_{N,K+1},x_{N,K+2},\dots ,x_{N,T}\right]}^T,\dots \right.,\left.{\left[x_{N,1},x_{N,2},\dots ,x_{N,T-K}\right]}^T\right]$, and ${\hat{\varphi }}_i={\left[{\left[{\mathbf{\Phi }}_k\right]}_{i,j}\right]}^T,k=1,\dots ,K,j=1,\dots ,N$. By substituting Eqs. (6)–(8) into Eq.(5), the formula can be reformulated, and then take the expectation to get the Q-function for EM as

$$\begin{gathered} \text{log}p\left(y_{1:T},z_{1:T},x_{1:T}\mid \theta \right)=-\frac{T}{2}\mathit{\text{log}}\left(2\pi \right|\mathbf{P}\left|\right)-\frac{1}{2}\sum _{t=1}^T{‖y_t-L{x}_t‖}_{{\mathbf{P}}^{-1}} \\ -\frac{T}{2}\mathit{\text{log}}\left(2\pi \right|\mathbf{S}\left|\right)-\frac{1}{2}\sum _{t=1}^T{‖z_t-C{x}_t‖}_{{\mathbf{S}}^{-1}} \\ -\frac{T}{2}\mathit{\text{log}}\left(2\pi \right)-\frac{T}{2}\sum _{i=1}^N\mathit{\text{log}}\left({\sigma }_i^2\right)-\frac{1}{2}\sum _{i=1}^N\frac{1}{{\sigma }_i^2}{‖{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i-\underset{\_}{\mathbf{X}}{\hat{\varphi }}_i‖}_2^2 \\ ={𝒞}_{\theta }-\frac{T}{2}\sum _{i=1}^N\mathit{\text{log}}\left({\sigma }_i^2\right)-\frac{1}{2}\sum _{i=1}^N\frac{1}{{\sigma }_i^2}{‖{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i-\underset{\_}{\mathbf{X}}{\hat{\varphi }}_i‖}_2^2, \end{gathered}$$ (9)

$$\begin{gathered} 𝒬\left(\theta ;{\theta }^{\left(j\right)}\right)=ℰ\left[\text{log}p\left(y_{1:T},z_{1:T},x_{1:T}\mid \theta \right)\mid y_{1:T},z_{1:T},{\theta }^{\left(j\right)}\right]= \\ {𝒞}_{{\theta }^{\left(j\right)}}-\frac{T}{2}\sum _{i=1}^N\text{log}\left({\sigma }_i^{(j{)}^2}\right)-\frac{1}{2}\sum _{i=1}^N\frac{1}{{\sigma }_i^{(j{)}^2}}ℰ\left[{‖{{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i}^{\left(j\right)}-{\underset{\_}{\mathbf{X}}}^{\left(j\right)}{\hat{\varphi }}_i^{\left(j\right)}‖}_2^2\right] \\ ={𝒞}_{{\theta }^{\left(j\right)}}-\frac{T}{2}\sum _{i=1}^N\text{log}\left({\sigma }_i^{(j{)}^2}\right)-\frac{1}{2}\sum _{i=1}^N\frac{1}{{\sigma }_i^{(j{)}^2}}\left\{ℰ\left[{{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i}^{(j{)}^T}{{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i}^{\left(j\right)}\mid y_{1:T},z_{1:T},{\theta }^{\left(j\right)}\right]-2ℰ\left[{{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i}^{(j{)}^T}{\underset{\_}{\mathbf{X}}}^{\left(j\right)}\mid y_{1:T},z_{1:T},{\theta }^{\left(j\right)}\right]{{\hat{\varphi }}_i}^{\left(j\right)}+{{\hat{\varphi }}_i}^{(j{)}^T}ℰ\left[{\underset{\_}{\mathbf{X}}}^{(j{)}^T}{\underset{\_}{\mathbf{X}}}^{\left(j\right)}\mid y_{1:T},z_{1:T},{\theta }^{\left(j\right)}\right]{{\hat{\varphi }}_i}^{\left(j\right)}\right\}, \end{gathered}$$ (10)

$${𝒞}_{{\theta }^{\left(j\right)}}=-\frac{T}{2}log\left(2\pi \left|\mathbf{P}\right|\right)-\frac{1}{2}\sum _{t=1}^T{‖y_t-L{x}_t^{\left(j\right)}‖}_{{\mathbf{P}}^{-1}}-\frac{T}{2}log\left(2\pi \left|\mathbf{S}\right|\right)-\frac{1}{2}\sum _{t=1}^T{‖z_t-C{x}_t^{\left(j\right)}‖}_{{\mathbf{S}}^{-1}}-\frac{T}{2}\mathit{\text{log}}\left(2\pi \right),$$ (11)

where the bracket superscript represents the $j^{\mathit{\text{th}}}$ iteration in EM, and ${𝒞}_{{\theta }^{\left(j\right)}}$ is a constant term when $\theta$ is given at $j^{\mathit{\text{th}}}$ iteration. We can find that the $p\left(x_{1:T}\mid y_{1:T},z_{1:T},\theta \right)$ is also Gaussian due to the Gaussian property on $x,y$, and $z$ given $\theta$ [32]. To find the Q-function, we can permute the equation and notice that the first and third expectation terms in the last row of the Eq.(10) consist of the second-order moment of the density $p\left(x_{1:T}\mid y_{1:T},z_{1:T},\theta \right)$ while the second expectation term can be expressed by the first-order moment of $p\left(x_{1:T}\mid y_{1:T},z_{1:T},\theta \right)$ whose mean as well as the covariance matrix can be estimated via Fixed Interval Smoothing (FIS), and the details of it will be listed in the next section. In the M-step, the optimal $\theta =\{\mathbf{\Phi },\mathbf{Q}\}$ that maximizes the Q-function defined above should be found. Since the number of observations is far less than that of the source regions, the inverse problem is ill-posed. The $l_1$ regularization on $\mathbf{\Phi }$ is introduced based on the premise that the functional connection among a given region and others possesses sparse properties to reduce problem-solving difficulty. Thus, the equation of the maximization can be described with the form

$${\theta }^{(j+1)}=\underset{\theta }{\text{arg max}}𝒬\left(\theta ;{\theta }^{\left(j\right)}\right)+\lambda \sum _{i=1}^N{‖{\mathbf{\Phi }}_{i,:\mid \theta }‖}_1,$$ (12)

which can be efficiently addressed through the implementation of the Fast Adaptive Shrinkage/Thresholding Algorithm (FASTA). At this juncture, the EM)framework for source estimation has been established, providing a statistically rigorous foundation for the subsequent analytical procedures.

2.3. Source density estimation with FIS

Fixed interval smoothing is a statistical technique used in time series analysis and signal processing. It involves retrospectively estimating and improving the values of a time series over fixed time intervals, considering both past and future observations. This method is particularly useful for reducing noise or uncertainty in historical data and obtaining more accurate, smoothed estimates of the underlying trends or states within the time series. The principle of FIS consists of two parts: forward filtering and backward smoothing. The forward filter is executed to derive posterior estimates and covariances up to the given time $t$. Subsequently, the backward filter is applied to yield prior estimates and covariances, effectively extending the timeline backward to time $t$ or providing a prior perspective in reverse chronology, in other words. Finally, the estimates and covariances derived from both forward and backward filtering at time $t$ are integrated to produce the ultimate estimation of the state and covariance matrix. Recall the main problem in Eq. (1). Considering computing performance factors, the fusion observation can be described by combining $y$ and $z$. The idea is that EEG and iEEG can be viewed as components of a unified electrophysiological system. Still, there is no correlation between the measuring noise of the two modalities, and the conditional density of merged observation is also a Gaussian. Thus, observation is redefined as

$$y_t^{+}=L^{+}{x}_t+w_t^{+},$$ (13)

where

$$\begin{gathered} L^{+}=\left(\genfrac{}{}{0pt}{}{L}{C}\right),L^{+}\in {\mathbf{R}}^{(M+O)\times N}, \\ {y}_t^{+}=\left(\genfrac{}{}{0pt}{}{y_t}{z_t}\right),y_t^{+}\in {\mathbf{R}}^{M+O}, \\ {w}_t^{+}=\left(\genfrac{}{}{0pt}{}{w_t}{e_t}\right),w_t^{+}~𝒩\left(0,{\mathbf{P}}^{+}\right),w_t^{+}\in {\mathbf{R}}^{(M+O)}, \\ {\mathbf{P}}^{+}=\left(\begin{array}{cc}\mathbf{P}& {\mathbf{0}}_{(M\times O)}\\ {\mathbf{0}}_{(O\times M)}& \mathbf{S}\end{array}\right),{\mathbf{P}}^{+}\in {\mathbf{R}}^{(M+O)\times (M+O)}, \end{gathered}$$ (14)

$$\begin{gathered} \text{log}p\left(y_{1:T}^{+},x_{1:T}\mid \theta \right)=\text{log}p\left(y_{1:T}^{+}\mid x_{1:T},\theta \right)+\text{log}p\left(x_{1:T}\mid \theta \right) \\ ={𝒞}_{\theta }^{+}-\frac{T}{2}\sum _{i=1}^N\mathit{\text{log}}\left({\sigma }_i^2\right)-\frac{1}{2}\sum _{i=1}^N\frac{1}{{\sigma }_i^2}{‖{\stackrel{\mathbf{ˆ}}{\mathbf{x}}}_i-\underset{\_}{\mathbf{X}}{\hat{\varphi }}_i‖}_2^2, \\ {𝒞}_{\theta }^{+}=-\frac{T}{2}\mathit{\text{log}}\left(2\pi \left|{\mathbf{P}}^{+}\right|\right)-\frac{1}{2}\sum _{t=1}^T{‖y_t^{+}-L^{+}{x}_t^{\left(j\right)}‖}_{{\mathbf{P}}^{+-1}}-\frac{T}{2}\mathit{\text{log}}\left(2\pi \right). \end{gathered}$$ (15)

Then the log-likelihood problem can be transformed to Eq. (15), and the same transformation can be applied to Q-function. The estimation of the mean and the covariance matrix of $p\left(x_{1:T}\mid y_{1:T},z_{1:T},\theta \right)$ can be redefined as that of the $p\left(x_{1:T}\mid y_{1:T}^{+},\theta \right)$. And one can easily find that $p\left(x_{1:T}\mid y_{1:T}^{+},\theta \right)$ is a Gaussian, since for two jointly Gaussians, the conditional distribution is also a Gaussian [32]. Then, the FIS can be applied to find the mean and covariance matrix for $p\left(x_{1:T}\mid y_{1:T}^{+},\theta \right)$. The forward Kalman filtering can give the estimation on $p\left(x_t\mid y_{1:t}^{+},\theta \right),\forall t$, while the backward Kalman smoothing can calculate the $p\left(x_t\mid y_{t:T}^{+},\theta \right),\forall t$. The merging of two estimates can generate the final estimate on $p\left(x_t\mid y_{1:T}^{+},\theta \right),\forall t$. Next, referring to the estimation framework [25], we start with Vector Auto Regressor (VAR) to generate the initial value for estimation. Since the source state at time $t$ depends on the state of the former $K$ time points, it is necessary to redefine the augmented source state as ${\underset{\_}{\mathbf{x}}}_t=\left[x_t;x_{t-1};\dots ;x_{t-K+1}\right],{\underset{\_}{\mathbf{x}}}_t\in {\mathbf{R}}^{\left(KN\right)}$ and the augmented dynamic model as (22) to transform $\text{VAR}\left(K\right)$ problem into a VAR(1) one.

$${\underset{\_}{\mathbf{x}}}_t=\underset{\_}{\mathbf{\Phi }}{\underset{\_}{\mathbf{x}}}_{t-1}+{\underset{\_}{\mathbf{n}}}_t,\forall t,$$ (16)

where

$$\begin{gathered} \underset{\_}{\mathbf{\Phi }}=\left(\begin{array}{ccccc}{\mathbf{\Phi }}_1& {\mathbf{\Phi }}_2& \dots & {\mathbf{\Phi }}_{(K-1)}& {\mathbf{\Phi }}_K\\ {\mathbf{I}}_{(M\times M)}& \mathbf{0}& \dots & \mathbf{0}& \mathbf{0}\\ \mathbf{0}& {\mathbf{I}}_{(M\times M)}& \dots & \mathbf{0}& \mathbf{0}\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ \mathbf{0}& \mathbf{0}& \dots & {\mathbf{I}}_{(M\times M)}& \mathbf{0}\end{array}\right),\underset{\_}{\mathbf{\Phi }}\in {\mathbf{R}}^{KN\times KN}, \\ {\underset{\_}{\mathbf{n}}}_t=\left(\begin{array}{c}{n}_t\\ 0\\ ⋮\\ 0\end{array}\right),{\underset{\_}{\mathbf{n}}}_t\in {\mathbf{R}}^{KN},{\underset{\_}{\mathbf{n}}}_t~𝒩(0,\underset{\_}{\mathbf{Q}}), \\ \underset{\_}{\mathbf{Q}}=\left(\begin{array}{cc}\mathbf{Q}& \mathbf{0}\\ \mathbf{0}& \mathbf{0},\end{array}\right) \end{gathered}$$ (17)

are the augmented state transition matrix and disturbance in source, respectively. And $\underset{\_}{\mathbf{Q}}$ is the covariance matrix for the augmented disturbance, which is also diagonal, whose diagonal values are the same variance as $\mathbf{Q}$ and 0 elsewhere. Then, the augmented version of merged observations is

$$y_t^{+}={\underset{\_}{\mathbf{L}}}^{+}{\underset{\_}{\mathbf{x}}}_t+w_t^{+},\forall t,$$ (18)

where

$${\underset{\_}{\mathbf{L}}}^{+}=\left(\begin{array}{ll}{\mathbf{L}}^{+}& 0\end{array}\right),{\underset{\_}{\mathbf{L}}}^{+}\in {\mathbf{R}}^{M\times KN}.$$ (19)

In this way, the $p\left({\underset{\_}{\mathbf{x}}}_t\mid y_{1:T}^{+},\theta \right)$ can be calculated via FIS if the observations as well as $\theta$ are known. Next, according to [33] one can define

$$\begin{gathered} {\underset{\_}{\mathbf{x}}}_{t_1\mid t_2}=ℰ\left[{\underset{\_}{\mathbf{x}}}_{t_1}\mid y_{1:t_2}^{+}\right], \\ {\mathbf{\Sigma }}_{t_1\mid t_2}=\mathit{\text{Cov}}\left({\underset{\_}{\mathbf{x}}}_{t_1}\mid y_{1:t_2}^{+}\right), \\ {\mathbf{B}}_{t_1,t_2\mid T}=\mathit{\text{Cov}}\left({\underset{\_}{\mathbf{x}}}_{t_1},{\underset{\_}{\mathbf{x}}}_{t_2}\mid y_{1:T}^{+}\right)=\left\{\begin{array}{r}{\mathbf{B}}_{t_2,t_1\mid T}^T,\mathit{\text{if}}{t}_1>t_2\\ {\mathbf{\Sigma }}_{t_1\mid t_2},\mathit{\text{if}}{t}_1=t_2\\ {\mathbf{K}}_{t_1}^b{\mathbf{B}}_{t_1+1,t_2\mid T}^T,\mathit{\text{if}}{t}_1<t_2\end{array}\right., \\ {\mathbf{K}}_{t_1}^b={\mathbf{\Sigma }}_{t_1\mid t_1}{\underset{\_}{\mathbf{\Phi }}}^T{\mathbf{\Sigma }}_{t_1+1\mid t_1}^{-1}, \end{gathered}$$ (20)

as the mean, covariance matrix as well as the cross-covariance matrix for any given $t_1$ and $t_2$. Then, the model can be fitted with the FIS framework. We started with the initial value obtained via VAR in the forward filtering step. Then, for $t=\mathrm{0,1},2,\dots ,T-1$ can have

$$\begin{gathered} {\underset{\_}{\mathbf{x}}}_{t+1\mid t}=\underset{\_}{\mathbf{\Phi }}{\underset{\_}{\mathbf{x}}}_{t\mid t}, \\ {\mathbf{\Sigma }}_{t+1\mid t}=\underset{\_}{\mathbf{\Phi }}{\mathbf{\Sigma }}_{t\mid t}{\underset{\_}{\mathbf{\Phi }}}^T+\underset{\_}{\mathbf{Q}}, \\ {\mathbf{K}}_{t+1}^f={\mathbf{\Sigma }}_{t+1\mid t}{\underset{\_}{\mathbf{L}}}^{+T}{\left({\underset{\_}{\mathbf{L}}}^{+}{\mathbf{\Sigma }}_{t+1\mid t}{\underset{\_}{\mathbf{L}}}^{+T}+{\mathbf{P}}^{+}\right)}^{-1}, \\ {\underset{\_}{\mathbf{x}}}_{t+1\mid t+1}={\underset{\_}{\mathbf{x}}}_{t+1\mid t}+{\mathbf{K}}_{t+1}^f\left(y_{t+1}^{+}-{\underset{\_}{\mathbf{L}}}^{+}{\underset{\_}{\mathbf{x}}}_{t+1\mid t}\right), \\ {\mathbf{\Sigma }}_{t+1\mid t+1}={\mathbf{\Sigma }}_{t+1\mid t}-{\mathbf{K}}_{t+1}^f\left({\underset{\_}{\mathbf{L}}}^{+}{\mathbf{\Sigma }}_{t+1\mid t}{\underset{\_}{\mathbf{L}}}^{+T}+{\mathbf{P}}^{+}\right){{\mathbf{K}}_{t+1}^f}^T. \end{gathered}$$ (21)

Taking the results from the filtering step, we can further do backward smoothing for $t=T-1,T-2,\dots ,0$ as

$$\begin{gathered} {\mathbf{K}}_t^b={\mathbf{\Sigma }}_{t\mid t}{\underset{\_}{\mathbf{\Phi }}}^T{\mathbf{\Sigma }}_{t+1\mid t}^{-1}, \\ {\underset{\_}{\mathbf{x}}}_{t\mid T}={\underset{\_}{\mathbf{x}}}_{t\mid t}+{\mathbf{K}}_t^b\left({\underset{\_}{\mathbf{x}}}_{t+1\mid T}-{\underset{\_}{\mathbf{x}}}_{t+1\mid t}\right), \\ {\mathbf{\Sigma }}_{t\mid T}={\mathbf{\Sigma }}_{t\mid t}-{\mathbf{K}}_t^b\left({\mathbf{\Sigma }}_{t+1\mid T}-{\mathbf{\Sigma }}_{t+1\mid t}\right){\mathbf{K}}_t^{b^T}. \end{gathered}$$ (22)

Then, the cross-covariance matrix can be obtained according to (20), and finally, simply extract the first $N$ rows of ${\underset{\_}{\mathbf{x}}}_{t\mid T}$ and the $n^{\mathit{\text{th}}}$ order submatrix of the upper left corner of the matrix ${\mathbf{B}}_{t_1+1,t_2\mid T}$ for $\forall t_1,t_2=\mathrm{1,2},\dots ,T$, which are exactly the first- and second-order moment of $p\left(x_t\mid y_{1:T}^{+},\theta \right),\forall t$, to finalize the E-step. The algorithmic pipeline is described in Appendix A.2.

3. Results

To validate the added value of the integration of scalp EEG and iEEG in estimating the WBEN, we first conducted experiments with simulated data, and then we tested the proposed inference framework on the Sternberg verbal working memory task to explore the connectivity maps between cortical and subcortical regions during the encoding and maintenance phases [34].

3.1. Numerical experiment on synthetic data

Realistic simulated data are generated and several baseline methods are used for comparison. Detailed experimental configurations are provided in the Appendix A.3.

Validation on the added value using simultaneous scalp EEG and iEEG:

Firstly, the impact of iEEG coverages of the partially observable brain regions (state variables) on the estimation of brain connectivity was evaluated. In the experiment, the source space state variables are partial observable with a prescribed portion rendered by the iEEG electrodes. The coverage ratio was set to range from 0% to 50% with a stepsize of 10%. The SNR levels for EEG and iEEG observations were set to be −5dB and 30dB, respectively. The network complexity was set with the number of activations as 10 and in-degree as 2. In the full factorial design of experiments, 10 repetitive simulations were conducted.

We use Acc and Sen (definitions provided in Appendix A.3) to evaluate the network estimation performance. The impact of the percentages of observable variables is given in Fig. 2, accompanied with the numerical results given in Table 1. Not surprisingly, as the proportion of observable state space variables increases, both Sen and Acc score are increasing. It is worth noting that when the percentage of observable state space variables reaches 30%, there is a significant improvement in both Acc and Sen, as is shown in Fig. 3. The result shows that the performance of WBEN estimation can be significantly improved when appropriate amount of activated areas are observable from iEEG electrodes, which highlights the value of using simultaneous recordings of scalp EEG and iEEG to characterize the whole brain network dynamics. An example with 30% partially observable brain nodes is shown in Fig. 4. When using two-step methods, due to inaccuracy caused by classical ESI approaches, the over-diffused source estimation results in highly dense brain networks. When using one-step approach with 0% of observable brain regions (without iEEG electrodes) [25], the performance is 0.461 for sensitivity and 0.189 for accuracy respectively, which a significant amount of false positive predictions.

Figure 2:

Evaluation of the Sen and Acc score varied with the percentage of observable brain regions on the WBEN estimation. The curve plots show the mean value of Acc and Sen.

Table 1:

Evaluation of the percentage of observable brain regions on the WBEN estimation.

Metrics	0%	10%	20%	30%	40%	50%	60%
Sen	0.461 ± 0.155	0.517 ± 0.042	0.556 ± 0.070	0.639 ± 0.120	0.690 ± 0.147	0.721 ± 0.153	0.852 ± 0.131
Acc	0.189 ± 0.176	0.179 ± 0.161	0.180 ± 0.174	0.545 ± 0.224	0.657 ± 0.194	0.687 ± 0.184	0.735 ± 0.201

Figure 3:

The significance of difference on accuracy and sensitivity between 0% and 30% source observation using group t-test.

Figure 4:

Visualization for network estimation in different methods, where the activated patch centers are highlighted with yellow color, and the iEEG observed patch centered are highlighted with green color. For two-step methods, the eLORETA that has the best performance is selected for comparison.

Impact of SNR on WBEN estimation:

The scalp EEG SNR is a key factor for WBEN estimation. Since the SNR in the iEEG signal is known to be significant greater than that of the EEG signal [35], we mainly evaluate the impact of scalp EEG SNR on the WBEN estimation. In the experiment, the SNR of iEEG signal was set to 30dB with different levels of SNR for the scalp EEG signal. It is unknown how the performance of WBEN estimation will be improved when integrating the iEEG into the WBEN inference framework with a high level of noise presence in scalp EEG recordings. The SNR for the EEG signal was set from −10dB to 10dB [35], meanwhile, the percentage of observable brain regions from iEEG was set to be 30%, and the network complexity configuration is the same as in Sec.3.1. Table 2 shows the statistical result for the experiment. The pairwise comparison of 0% and 30% observable state variables is illustrated in Fig. 3. The group level t-test shows the significant difference when making 30% of state variables (brain regions) directly observable from iEEG electrodes, with a pronounced improvement in Acc score by reducing the false positive rates.

Table 2:

Evaluation of performance with different levels of scalp EEG SNR.

Method	Metrics	SNR=5	SNR=0	SNR=−5	SNR=−10
MNE	Sen	0.328 ± 0.131	0.253 ± 0.149	0.162 ± 0.120	0.170 ± 0.192
	Acc	0.013 ± 0.006	0.011 ± 0.005	0.008 ± 0.004	0.006 ± 0.003
dSPM	Sen	0.199 ± 0.144	0.203 ± 0.144	0.152 ± 0.134	0.149 ± 0.159
	Acc	0.010 ± 0.008	0.011 ± 0.007	0.007 ± 0.003	0.006 ± 0.003
sLORETA	Sen	0.255 ± 0.160	0.269 ± 0.159	0.165 ± 0.123	0.176 ± 0.189
	Acc	0.011 ± 0.006	0.011 ± 0.007	0.008 ± 0.004	0.006 ± 0.004
eLORETA	Sen	0.442 ± 0.101	0.412 ± 0.119	0.383 ± 0.135	0.363 ± 0.143
	Acc	0.013 ± 0.006	0.012 ± 0.006	0.010 ± 0.005	0.010 ± 0.005
ALCMV [36]	Sen	0.427 ± 0.134	0.390 ± 0.145	0.412 ± 0.161	0.384 ± 0.139
	Acc	0.002 ± 0.001	0.002 ± 0.001	0.002 ± 0.001	0.002 ± 0.001
ASTAR [37]	Sen	0.113 ± 0.011	0.107 ± 0.013	0.125 ± 0.022	0.095 ± 0.013
	Acc	0.003 ± 0.001	0.002 ± 0.001	0.003 ± 0.001	0.003 ± 0.002
VSSI-ARD [38]	Sen	0.448 ± 0.117	0.421 ± 0.128	0.361 ± 0.124	0.338 ± 0.148
	Acc	0.003 ± 0.002	0.003 ± 0.001	0.003 ± 0.003	0.003 ± 0.001
EEG with 0% of iEEG obs.	Sen	0.707 ± 0.194	0.592 ± 0.169	0.461 ± 0.155	0.402 ± 0.132
	Acc	0.429 ± 0.292	0.253 ± 0.245	0.189 ± 0.176	0.034 ± 0.023
EEG with 30% of iEEG obs.	Sen	0.780 ± 0.136	0.716 ± 0.115	0.639 ± 0.120	0.491 ± 0.103
	Acc	0.404 ± 0.270	0.501 ± 0.269	0.545 ± 0.224	0.595 ± 0.167

From Table 2, the classical two-step methods do not achieve satisfactory results even under high SNR conditions. In contrast, when employing the one-step state space modeling approach, performance remains comparable with either 0% or 30% iEEG observations given high SNR scalp EEG, highlighting the advantages of the one-step approach with state-space modeling paradigm for WBEN estimation. As SNR decreases with increased noise, Acc and Sen metrics deteriorate across all methods. However, performance using the one-step state space dynamic model degrades significantly, producing numerous incorrectly predicted connections at lower SNR values, whereas integrating iEEG measurements yields a relatively stable performance curve. This finding further confirms the value of incorporating iEEG for more accurate WBEN estimation, particularly when scalp EEG channels are contaminated with high noise levels.

Further analysis on the false positive rate, the different causal analysis baselines, and the impact of brain network complexity on the WBEN estimation are given in the Appendix A.6, A.5 and A.7, respectively.

3.2. Cortical-subcortical network analysis of working memory task

In this section, we analyzed the networks between cortical and subcortical brain regions during the Working Memory (WM) tasks. WM is commonly associated with learning, understanding, executive functioning, information processing, intelligence, and problem-solving in humans and various animals from infancy to old age [39]. Maintaining content in WM requires communication between an extensive network of brain regions [40]. In this section, the proposed method was applied to estimate and analyze the brain connectivity between cortical and subcortical regions at different WM phases, which are the encoding phase and maintenance phase, in a verbal WM task conducted by patients with epilepsy where simultaneous scalp EEG and iEEG were recorded. The data description and preprocessing are detailed in Appendix A.4.

The experiment used a second-order vector auto-regressor model for the estimation framework stated in section 2.2. The regional connectivity of brain activity during the encoding and maintenance phases was estimated separately for each human participant’s tasks. Lastly, the final estimation was obtained by first taking the sum over the estimated state transition matrices for each task and then averaging all task-leveled state transition matrices according to different phases. To ensure the stability of the result, only the connections from region $i$ to $j$ that contains signal power no less than 10% of the signal power from region $j$ to itself were kept, i.e., $\underset{\_}{\mathbf{\Phi }}(i,j)>0.1*\underset{\_}{\mathbf{\Phi }}(j,j)$. Moreover, the HO atlas regions in the cortical area were further merged into lobe-level granularity, while caudate, putamen, and pallidum were aggregated as basal ganglia to further pursue robust macro-scale results. The estimated dynamic networks (Appendix A.9) revealed significant phase-dependent differences. During encoding, predominant information flow occurred from cortical to subcortical regions, specifically from frontal, temporal (including auditory processing areas), and parietal lobes to thalamus and basal ganglia. Meanwhile, subcortical-originating connections primarily targeted other subcortical structures, with directed pathways from basal ganglia and hippocampus to thalamus, and from amygdala to basal ganglia and thalamus. Conversely, maintenance phase exhibited a reversed directionality pattern, with prominent connections from thalamus and basal ganglia to frontal and parietal lobes, and from hippocampus to temporal regions (including auditory processing areas). Statistical analyses in Fig. 5 confirmed significant differences in both directional connectivity and distribution patterns between subcortical and cortical regions across phases. We further analyzed the connectivity strength and the direction of information flow between cortical and subcortical regions during the encoding and maintenance phases. As shown in Fig. 5, the overall connectivity strength during the encoding phase is stronger than that of the maintenance phase. Besides, the amount of connection that flows start from cortical regions is generally greater during the encoding phase than the maintenance phase, whereas brain connections during the maintenance phase are more active in the information flow that is directed from subcortical regions to elsewhere.

Figure 5:

Cortical-subcortical connectivity estimation analyses across encoding and maintenance phases in verbal working memory. A depicts the distribution of absolute connectivity strength between cortical and subcortical regions during both phases based on averaged results, with significant differences confirmed by group t-test (t = 3.359, p < 0.001). B illustrates the distribution of connection frequencies across phases derived from event-level analyses: left panel represents outgoing information flows from cortical regions (t = 5.487, p < 0.001), while right panel shows outgoing information flows from subcortical regions (t = −13.336, p < 0.001).

4. Conclusion

We proposed the first unified computational framework for integrating simultaneously recorded scalp EEG and iEEG data to estimate the whole-brain electrophysiological networks. This framework enables the delineation of neurophysiological networks at a whole brain scale and with millisecond-level temporal resolution. Results validate the complementary value of both modalities, demonstrating that strategic multimodal scalp EEG and iEEG deployment can significantly improve network reconstruction accuracy even with high volume of measurement noise. Numerical experiments confirm the robustness under broad experiment configurations, while application to Sternberg verbal working memory task yielded insightful findings consistent with previous studies [40–47], particularly the characterization of cortical-subcortical information flows during encoding and maintenance phase [40], which is the first analysis of this type at the whole brain scale. The proposed framework can serve as a fundamental computational tool for electrophysiological brain network analysis with applications to clinical studies and neuroscience research.

A. Appendix

A.1. Definition of restricted isometry property

The definition of Restricted Isometry Property (RIP) is that for a lead field matrix $\Phi \in {\mathbb{R}}^{m\times n}$ and sparsity level $s$, the RIP is mathematically defined as:

$$\left(1-{\delta }_s\right)\Vert x{\Vert }_2^2\le \Vert \Phi x{\Vert }_2^2\le \left(1+{\delta }_s\right)\Vert x{\Vert }_2^2,$$ (23)

where ${\delta }_s\in (0,1)$ is the Restricted Isometry Constant (RIC) and the inequality holds for all $s$-sparse vectors $x$.

The RIP can be equivalently expressed using operator norms:

$${\delta }_s=\underset{\left|S\right|\le s}{\text{max}}{‖{\Phi }_S^T{\Phi }_S-I‖}_{2\to 2},$$ (24)

where ${\Phi }_S$ denotes the submatrix of $\Phi$ with columns indexed by set $S$, and $I$ is the identity matrix.

In terms of eigenvalues, RIP ensures:

$$1-{\delta }_s\le {\lambda }_{\text{min}}\left({\Phi }_S^T{\Phi }_S\right)\le {\lambda }_{\text{max}}\left({\Phi }_S^T{\Phi }_S\right)\le 1+{\delta }_s.$$ (25)

Cai et al. proved that as sparsity $s$ increases, even optimal measurement matrices cannot keep ${\delta }_s$ sufficiently small, making recovery unreliable [48]. Since EEG leadfields often violate RIP at moderate–high sparsity, ESI accuracy degrades under multi-source conditions, motivating the use of strong priors or data-driven constraints.

A.2. Algorithmic framework and procedure

Algorithm 1.

EM Framework for Parameter Estimation

Require: EEG measurement ${\left\{y_t\right\}}_{t=1}^T$, iEEG measurement ${\left\{z_t\right\}}_{t=1}^T$, leadfield matrix for EEG $L$ and iEEG $C$, estimated noise covariance matrix of EEG $P$ and iEEG $S$, other parameters for VAR model, regularization, and halting condition.
1:	Integrate the two modalities to get an augmented measurement ${\left\{y_t^{+}\right\}}_{t=1}^T$, leadfield matrix $L^{+}$, and noise covariance matrix $P^{+}$. Initialize parameter ${\theta }^{(0)}$ according to any conventional source localization method, e.g., MNE.
2:	while the convergence or halting condition is not met do
3:	E-step: Compute $𝒬\left(\theta ;{\theta }^{(j)}\right)$ based on the conditional density $p\left(x_{1:T}\mid y_{1:T}^{+},{\theta }^{(j)}\right)$ estimated via FIS.
4:	M-step: Maximize ${\theta }^{(j+1)}=\underset{\theta }{\text{argmax}}\left\{𝒬\left(\theta ;{\theta }^{(j)}\right)+\lambda {\sum }_{i=1}^N{‖{\Phi }_{i,:\mid \theta }^{(j)}‖}_1\right\}$ using FASTA.
5:	$j=j+1$
6:	end while
Ensure: Optimal parameter ${\theta }^{*}$

A.3. Simulated experiment settings

Brain Forward Model:

To generate synthetic EEG data, we used a realistic head model to compute the leadfield matrix based on the T1-MRI images from FreeSurfer [49]. The brain tissue segmentation and tissue surface generation were conducted using FreeSurfer. A 128-channel BioSemi EEG cap layout was used, and the EEG channels were co-registered with the scalp surface and further validated on the MNE-Python toolbox [50]. The source space was parcelled and resampled. Then a three-layer boundary element method (BEM) was built based on the reconstructed surfaces, resulting in a leadfield matrix denoted as $L$.

Realistic data generation:

To generate a causal time series, the Berlin Brain Connectivity Benchmark (BBCB) [51] is used with randomly generated state transition matrices ${\mathbf{\Phi }}_{\mathbf{k}}$ with $k\in \{1,\cdots ,K\}$. To ensure the convergence of source signals, each eigenvalue in $\mathbf{\Phi }$ is further validated to be less or equal to 1. Then we add an independent random Gaussian noise to each source signal at every time step. Lastly, an acausal third-order Butterworth filter with zero phase delay was applied with band-pass frequency being [0.1Hz, 40Hz] [52]. With the generated source signal, the observation time series can be derived by multiplying the leadfield matrix with the source signal and adding the channel-wise correlated random Gaussian noise with a given signal-to-noise ratio (SNR) level. The iEEG noise is generated separately with a relatively higher SNR level [35].

Evaluation Metrics:

In the simulated experiment, the ground truth of the connectivity network is defined based on the generated state transition matrix $\mathbf{\Phi }$. If the $i,j$-th element ${\mathbf{\Phi }}_{(i,j)}$ is zero, there is no link from $i$ to $j$, otherwise there is a link from $i$ to $j$. In this experiment, $K$ is set to be 1 so that the state at time $t$ is only dependent on the state at time $t-1$. The Sensitivity (Sen) and Accuracy (Acc) metrics for connectivity estimation are defined to evaluate the performance given as:

$$\mathit{\text{Sen}}=\frac{N_c}{N_{tot}},Acc=\frac{N_c}{N_c+N_w},$$ (26)

where $N_c$ denotes the number of correct predictions, and $N_{\text{tot}}$ is the total number of connectivities, and $N_w$ is the number of wrong predictions (number of false positives).

Benchmark algorithms:

The baseline methods include traditional two-step methods, which conducted brain source localization first with existing ESI algorithms, such as MNE [53], sLORETA [54], dSPM [55], eLORETA [56] etc., followed by applying the Granger causality analysis on the estimated source signals for source space connectivity estimation. The observation signals were set with sampling rate 100Hz and a group of 10s time series were generated. In order to analyze the impact of the integration of iEEG, several hyperparameters, i.e., the number of partially observable brain regions, different levels of SNR, and complexity of brain networks (which includes the number of activated source regions and the maximum in-degree of each region), were evaluated comprehensively.

Computational resources:

All experiments are run on a Windows 11 pro desktop with 32G memory, an Intel i9-12900KF CPU and an A6000 GPU of 48G memory.

A.4. Real Data Description and Preprocessing

The dataset comprises intracranial recordings from 15 epilepsy patients undergoing clinical monitoring for seizure localization while performing a modified Sternberg verbal working memory task. This paradigm temporally segregated encoding, maintenance, and recall phases. The comprehensive dataset includes simultaneous scalp EEG recordings following the 10–20 system, depth electrode iEEG recordings, and the corresponding MNI coordinates with anatomical labels for all intracranial electrodes [34]. Each participant completed multiple experimental sessions, with 50 distinct events per session. Each event epoch consisted of an 8-second recording (sampled at 200 Hz for EEG and 2000 Hz for iEEG), structured as follows: fixation (0–1s), working memory encoding (1–3s), maintenance (3–6s), and response (6–8s). Our analysis specifically targeted the encoding and maintenance phases. To account for potential temporal extension of the auditory encoding process beyond the visual stimulus presentation, we analyzed only the final 2 seconds of the maintenance phase, consistent with methodological considerations outlined in prior research [40].

All iEEG and EEG electrodes were co-registered to the standard ‘fsaverage’ template [57]. The forward model was calculated using MNE-Python with source spacing set to oct-5, generating over 2000 patches per hemisphere. Forward solutions were computed independently for each subject according to the standard 10–20 montage of their specific electrode configurations. For regional connectivity analysis, source locations were aggregated into anatomical regions defined by the Harvard-Oxford (HO) atlas (48 cortical and 21 subcortical areas) using nearest-neighbor mapping in MNI space, thereby reducing the high-dimensional source space to a more tractable atlas-based representation. iEEG channels were mapped to HO atlas areas based on electrode positions, with signals averaged across channels assigned to identical atlas regions. The iEEG leadfield matrix was constructed as a binary mapping, with unit values representing measured atlas areas and zeros elsewhere. In the absence of empty room recordings, we modeled EEG noise as a multivariate Gaussian distribution (mean=0, standard deviation=1) and iEEG noise as a multivariate Gaussian with reduced variance (mean=0, standard deviation=0.1). Both EEG and iEEG signals were subsequently downsampled to 50 Hz.

A.5. Assessment of model performance with respect to different causal analysis under different SNR

We further use transfer entropy and partial directed coherence for two-stage causal analysis baselines. As can be seen from Table 3 and 4. The proposed method is consistently better than baselines.

Table 3:

Evaluation of performance with different levels of scalp EEG SNR. The connectivity estimation for two-step methods is based on transfer entropy.

Method	Metrics	SNR=5	SNR=0	SNR=−5	SNR=−10
MNE	Sen	0.255 ± 0.093	0.255 ± 0.093	0.255 ± 0.093	0.255 ± 0.093
	Acc	0.002 ± 0.001	0.002 ± 0.001	0.002 ± 0.001	0.002 ± 0.001
DSPM	Sen	0.000 ± 0.000	0.000 ± 0.000	0.008 ± 0.024	0.008 ± 0.024
	Acc	0.000 ± 0.000	0.000 ± 0.000	0.033 ± 0.100	0.000 ± 0.001
SLORETA	Sen	0.262 ± 0.074	0.277 ± 0.043	0.292 ± 0.046	0.296 ± 0.053
	Acc	0.002 ± 0.000	0.002 ± 0.000	0.002 ± 0.000	0.002 ± 0.001
ELORETA	Sen	0.296 ± 0.053	0.296 ± 0.053	0.296 ± 0.053	0.291 ± 0.058
	Acc	0.004 ± 0.001	0.003 ± 0.001	0.003 ± 0.000	0.002 ± 0.000
ALCMV [36]	Sen	0.140 ± 0.104	0.166 ± 0.095	0.212 ± 0.113	0.232 ± 0.103
	Acc	0.096 ± 0.064	0.056 ± 0.033	0.021 ± 0.013	0.007 ± 0.005
ASTAR [37]	Sen	0.009 ± 0.018	0.009 ± 0.018	0.004 ± 0.013	0.004 ± 0.013
	Acc	0.014 ± 0.031	0.014 ± 0.031	0.005 ± 0.014	0.004 ± 0.011
VSSI-ARD [38]	Sen	0.087 ± 0.065	0.067 ± 0.058	0.066 ± 0.053	0.062 ± 0.056
	Acc	0.120 ± 0.085	0.088 ± 0.121	0.062 ± 0.089	0.035 ± 0.060
EEG with 0% of iEEG obs.	Sen	0.707 ± 0.194	0.592 ± 0.169	0.461 ± 0.155	0.402 ± 0.132
	Acc	0.429 ± 0.292	0.253 ± 0.245	0.189 ± 0.176	0.034 ± 0.023
EEG with 30% of iEEG obs.	Sen	0.780 ± 0.136	0.716 ± 0.115	0.639 ± 0.120	0.491 ± 0.103
	Acc	0.404 ± 0.270	0.501 ± 0.269	0.545 ± 0.224	0.595 ± 0.167

Table 4:

Evaluation of performance with different levels of scalp EEG SNR. The connectivity estimation for two-step methods is based on partial directed coherence.

Method	Metrics	SNR=5	SNR=0	SNR=−5	SNR=−10
MNE	Sen	0.306 ± 0.151	0.217 ± 0.157	0.150 ± 0.178	0.115 ± 0.190
	Acc	0.011 ± 0.009	0.011 ± 0.007	0.012 ± 0.014	0.011 ± 0.017
DSPM	Sen	0.111 ± 0.210	0.147 ± 0.197	0.126 ± 0.200	0.105 ± 0.204
	Acc	0.018 ± 0.030	0.151 ± 0.136	0.155 ± 0.122	0.129 ± 0.178
SLORETA	Sen	0.270 ± 0.194	0.214 ± 0.181	0.175 ± 0.191	0.121 ± 0.213
	Acc	0.007 ± 0.006	0.006 ± 0.003	0.007 ± 0.004	0.005 ± 0.006
ELORETA	Sen	0.467 ± 0.170	0.431 ± 0.143	0.366 ± 0.163	0.417 ± 0.168
	Acc	0.019 ± 0.011	0.013 ± 0.006	0.007 ± 0.002	0.006 ± 0.002
ALCMV [36]	Sen	0.643 ± 0.069	0.699 ± 0.104	0.716 ± 0.113	0.699 ± 0.077
	Acc	0.004 ± 0.001	0.005 ± 0.000	0.005 ± 0.001	0.005 ± 0.001
ASTAR [37]	Sen	0.078 ± 0.042	0.068 ± 0.060	0.055 ± 0.039	0.064 ± 0.048
	Acc	0.038 ± 0.023	0.034 ± 0.033	0.026 ± 0.018	0.029 ± 0.022
VSSI-ARD [38]	Sen	0.648 ± 0.215	0.658 ± 0.221	0.624 ± 0.214	0.515 ± 0.218
	Acc	0.005 ± 0.001	0.005 ± 0.001	0.005 ± 0.001	0.004 ± 0.001
EEG with 0% of iEEG obs.	Sen	0.707 ± 0.194	0.592 ± 0.169	0.461 ± 0.155	0.402 ± 0.132
	Acc	0.429 ± 0.292	0.253 ± 0.245	0.189 ± 0.176	0.034 ± 0.023
EEG with 30% of iEEG obs.	Sen	0.780 ± 0.136	0.716 ± 0.115	0.639 ± 0.120	0.491 ± 0.103
	Acc	0.404 ± 0.270	0.501 ± 0.269	0.545 ± 0.224	0.595 ± 0.167

A.6. Assessment of model performance using false positive rate

To address ghost interactions, which mainly represent false positive connections in brain network analysis, we calculated the false positive rate (FPR) of different methods. As shown in Table 5, the proposed method achieved significantly lower FPR compared to other benchmark algorithms, demonstrating superior specificity in identifying true brain network connections while effectively suppressing spurious interactions caused by signal leakage.

Table 5:

False positive rate of different methods under varying observation noise levels. The connectivity estimation for two-step methods is based on Granger causality.

Method	Metric	${\text{SNR}}_{\text{obs}}=5$	${\text{SNR}}_{\text{obs}}=0$	${\text{SNR}}_{\text{obs}}=-5$	${\text{SNR}}_{\text{obs}}=-10$
MNE	FPR	0.154 ± 0.273	0.156 ± 0.260	0.145 ± 0.263	0.151 ± 0.262
dSPM	FPR	0.129 ± 0.276	0.137 ± 0.266	0.139 ± 0.265	0.139 ± 0.263
eLORETA	FPR	0.256 ± 0.278	0.265 ± 0.272	0.272 ± 0.275	0.258 ± 0.271
sLORETA	FPR	0.146 ± 0.271	0.147 ± 0.263	0.146 ± 0.263	0.148 ± 0.262
ALCMV [36]	FPR	0.643 ± 0.225	0.700 ± 0.236	0.732 ± 0.242	0.736 ± 0.249
ASTAR [37]	FPR	0.172 ± 0.071	0.228 ± 0.101	0.188 ± 0.085	0.165 ± 0.092
VSSI-ARD [38]	FPR	0.548 ± 0.176	0.522 ± 0.179	0.468 ± 0.197	0.371 ± 0.194
EEG with 0% of iEEG obs.	FPR	0.037 ± 0.091	0.038 ± 0.089	0.028 ± 0.061	0.071 ± 0.085
EEG with 30% of iEEG obs.	FPR	0.007 ± 0.005	0.008 ± 0.005	0.003 ± 0.003	0.002 ± 0.002

A.7. Assessment of model performance with respect to network architectural complexity

Impact of brain network complexity on the WBEN estimation:

The benefit of integrating iEEG and scalp EEG is further validated with different network complexity levels. The network complexity level is designed with varied number of activated source regions, and varied value of in-degree of node that effects the number of edges in the brain networks. The number of activated source regions is set to be 10, 15 and 20, while the in-degree was varied from 1 to 4 with other parameters being the same as the previous experiments.

Table 6:

Evaluation of performance with different number of activated source regions

Method	Metrics	#Brain Regions=10	#Brain Regions=15	#Brain Regions=20
MNE	Sen	0.162 ± 0.120	0.251 ± 0.191	0.154 ± 0.156
	Acc	0.008 ± 0.004	0.013 ± 0.008	0.011 ± 0.006
dSPM	Sen	0.152 ± 0.134	0.254 ± 0.195	0.208 ± 0.212
	Acc	0.007 ± 0.003	0.012 ± 0.009	0.010 ± 0.005
sLORETA	Sen	0.165 ± 0.123	0.272 ± 0.189	0.206 ± 0.214
	Acc	0.008 ± 0.004	0.013 ± 0.008	0.010 ± 0.005
eLORETA	Sen	0.383 ± 0.135	0.390 ± 0.169	0.389 ± 0.146
	Acc	0.010 ± 0.005	0.009 ± 0.005	0.011 ± 0.005
EEG with 0% of iEEG observation	Sen	0.461 ± 0.155	0.350 ± 0.087	0.398 ± 0.096
	Acc	0.189 ± 0.176	0.118 ± 0.095	0.191 ± 0.091
EEG with 30% of iEEG observation	Sen	0.639 ± 0.120	0.518 ± 0.110	0.547 ± 0.093
	Acc	0.545 ± 0.224	0.500 ± 0.172	0.552 ± 0.182

Table 7:

Evaluation of performance with different values of node in-degree

Method	Metrics	In-degree=1	In-degree=2	In-degree=3	In-degree=4
MNE	Sen	0.117 ± 0.043	0.162 ± 0.120	0.230 ± 0.197	0.304 ± 0.259
	Acc	0.005 ± 0.002	0.008 ± 0.004	0.014 ± 0.011	0.016 ± 0.011
dSPM	Sen	0.121 ± 0.037	0.152 ± 0.134	0.423 ± 0.294	0.504 ± 0.296
	Acc	0.005 ± 0.002	0.007 ± 0.003	0.011 ± 0.010	0.011 ± 0.009
sLORETA	Sen	0.131 ± 0.058	0.165 ± 0.123	0.428 ± 0.292	0.511 ± 0.289
	Acc	0.005 ± 0.002	0.008 ± 0.004	0.011 ± 0.010	0.011 ± 0.010
eLORETA	Sen	0.196 ± 0.098	0.383 ± 0.135	0.505 ± 0.189	0.630 ± 0.139
	Acc	0.005 ± 0.003	0.010 ± 0.005	0.010 ± 0.007	0.009 ± 0.006
EEG with 0% of iEEG observation	Sen	0.570 ± 0.194	0.461 ± 0.155	0.401 ± 0.198	0.422 ± 0.241
	Acc	0.228 ± 0.234	0.189 ± 0.176	0.149 ± 0.201	0.109 ± 0.168
EEG with 30% of iEEG observation	Sen	0.797 ± 0.129	0.639 ± 0.120	0.546 ± 0.143	0.422 ± 0.203
	Acc	0.697 ± 0.248	0.545 ± 0.224	0.498 ± 0.159	0.410 ± 0.215

Based on performance summarized in Table 6 and Table 7, the performance of the two-step methods is usually limited, with high false positive predictions. By integrating iEEG and scalp EEG in the WBEN estimation, both Acc and Sen showed improved performance compared with the comparison methods. The experiments further highlight the benefits of leveraging simultaneously recorded scalp EEG and iEEG for WBEN estimation.

A.8. Intra- and Inter-subject analysis

We conducted a comprehensive analysis of connectivity pattern consistency to validate the reliability of our derived brain connections across different subjects and experimental sessions using Pearson correlations.

Intra-subject consistency:

Encoding phase: 0.539 ± 0.062, Maintenance phase: 0.511 ± 0.043.

Inter-subject consistency:

Encoding phase: 0.355 ± 0.037, Maintenance phase: 0.353 ± 0.059.

The results reflect the expected balance between network stability and task-specific adaptability based on EEG data. The moderate intra-subject correlations are consistent with the functional flexibility of brain networks during working n-back memory tasks (each time different words are presented to the participants). Individual connectivity patterns are preserved across sessions while allowing for task-related adaptations to different verbal stimuli. The inter-subject consistency reflects a biologically meaningful balance [58]. It demonstrates the presence of shared functional networks underlying verbal working memory while preserving the individual neural diversity that characterizes brain function [59]. This moderate correlation (compared to random correlation (≈ 0) between two vectors of 4761 dimensions (directed edges from a 69 by 69 matrix)) indicates our method successfully captures both the common neural mechanisms required for the task and the individual network signatures that contribute to cognitive variability across subjects.

A.9. Visualization of connectivity estimation in real data experiment

Figure 6:

A visualization example of cortical-subcortical connectivity estimation in working memory. Sub-figure A shows the brain connectivity network in two WM phases according to the HO atlas, while B is the network of aggregated lobe-leveled regions used for macroscopic expression. C is the circular brain connectivity graph in HO atlas parcellation for encoding and maintenance phases, respectively.

A.10. Approximated computational complexity

The computational complexity of the proposed method is dominated by the FIS in the E-step. The forward Kalman filtering requires matrix operations at each time step, including the covariance update and matrix inversion, both involving $O\left(N^3\right)$ operations, where $N$ is the number of source regions.

To align with established work on brain network analysis using fMRI [60], we also adopted an atlas-based approach where we estimate region-to-region connectivity. In this work, we used the Harvard-Oxford atlas regions $(N=69)$. The computational cost for the atlas-based analysis will remain stable regardless of the different dimensions of the source space. Practically, the brain network estimation is computationally feasible (about 3 minutes wall clock for 1000 time points).

A.11. Limitations

Although this method is a one-step approach to modeling source-space connectivity, false-positive connections may still occur near true connections due to ‘signal leakage’ during source reconstruction [61, 62]. This problem occurs because estimating thousands of sources from data collected by only a hundred electrodes is inherently under-determined. As a result, residual signal leakage between locations will inevitably influence the source data, causing false connectivities to appear as unintended artifacts of true interactions between source pairs. Moreover, the exact locations of these sources may be inaccurately determined. This issue, known as “ghost interaction,” can be mitigated by a novel method that organizes connections into hyperedges based on their adjacency in signal mixing [63].

A.12. Safeguards

The code and result produced from our developed framework can give a delineation of electrophysiological brain networks which can be used for surgical decision. However, epileptologists and neurosurgeons should use this as a augmented tool for decision making and should double check the result with other brain imaging modalities.