Scalable and Robust Tensor Decomposition of Spontaneous Stereotactic EEG Data

Abstract

Objective:

Identification of networks from resting brain signals is an important step in understanding the dynamics of spontaneous brain activity. We approach this problem using a tensor-based model.

Methods:

We develop a rank-recursive Scalable and Robust Sequential Canonical Polyadic Decomposition (SRSCPD) framework to decompose a tensor into several rank-1 components. Robustness and scalability are achieved using a warm start for each rank based on the results from the previous rank.

Results:

In simulations we show that SRSCPD consistently outperforms the multi-start alternating least square (ALS) algorithm over a range of ranks and signal-to-noise ratios (SNRs), with lower computation cost. When applying SRSCPD to resting in-vivo stereotactic EEG (SEEG) data from two subjects with epilepsy, we found components corresponding to default mode and motor networks in both subjects. These components were also highly consistent within subject between two sessions recorded several hours apart. Similar components were not obtained using the conventional ALS algorithm.

Conclusion:

Consistent brain networks and their dynamic behaviors were identified from resting SEEG data using SRSCPD.

Significance:

SRSCPD is scalable to large datasets and therefore a promising tool for identification of brain networks in long recordings from single subjects.

I. Introduction

EXPLORING functional connectivity (FC) in resting brain signals is a rich approach to studying brain networks [1]. Of particular recent interest is the dynamic nature of FC [2], The most commonly used strategy for decoding dynamic FC (DFC) is to compute correlation or coherence using a sliding window [3], [4], However, using a long temporal window to obtain robust FC estimates inevitably leads to over-smoothing of dynamic changes [5], [6], To overcome this difficulty, principal component analysis (PCA)-based and independent component analysis (ICA)-based approaches have been proposed. Although they do not introduce temporal smoothing, a limitation of those methods is that either the time series of each network is required to be independent (temporal ICA) [7] or the spatial modes of the networks are disjoint (spatial ICA) [8], [9] or orthogonal (PCA) [10], whereas real networks can overlap and be correlated in both space and time [11].

Tensors are a generalization of matrices in which we can represent data with more than two indices. For example, here we use a third-order tensor to represent SEEG data in terms of space, time and frequency (in the following “tensors” refers to tensors of order 3 or higher). As with matrices, tensors can be represented as a sum of rank-1 components. Structured data often then admit to low rank models with respect to this tensor representation. One of the two popular tensor models is the canonical polyadic (CP) form [12], which has been shown [13]–[15] to also be equivalent to parallel factors analysis (PARAFAC) [16] and the canonical decomposition (CANDECOMP) [17].

CP is a model that can capture structure inherent in multidimensional data when represented in a low rank tensor of order 3 or higher. Conversely, standard 2D matrix decomposition methods, such as PCA or ICA, applied to matricized or unfolded tensors [15] are typically not able to capture this structure using a similar low-rank model. Moreover, CP decomposition has a unique solution under less restrictive conditions than the orthogonality or independence assumptions implicit in PCA or ICA [18], [19], This latter property is particularly appealing when analyzing SEEG or other brain data that can be represented as a third or higher-order tensor, since we can avoid the restrictive assumptions of orthogonality or independence between components.

Among the many algorithms for computing the CP decomposition, alternating least squares (ALS), [16][17] is the most widely used because of its simplicity relative to alternatives [14], [15], [20], [21], Comparisons in these papers show that, in general, ALS provides solutions of similar quality to other algorithms. While some are more robust to over-factoring than ALS, especially in ill-conditioned cases, the expense is higher computational complexity in both memory and time [15].

ALS-based CP decomposition has been widely used in EEG analysis by transforming the raw EEG recordings to a time-frequency representation using short-time Fourier or Morlet wavelet transforms and applying a 3-way CP decomposition (channel by time by frequency) [22], Möck [23] applied CP to event-related potentials (ERP). Miwakeichi et al. [24] analyzed both spontaneous and evoked EEG recordings and showed that theta activity was predominant during a task condition, while alpha activity was observed continuously during both rest and task conditions. CP decomposition has also been applied to ictal EEG recordings from patients with epilepsy. The extracted components have been used to localize the seizure onset zone [25]–[27] as well as to remove artifacts [25], [26], When applied to group data analysis, CP decomposition and the extracted components/features can be used for a variety of purposes: classification [28], cross-modality comparison [29], and hypothesis testing [30].

The application of CP decomposition to functional MRI (fMRI) data has also been explored, usually at the group-level with the goal of finding common factors among subjects [14], Individual analysis using CP is rarely performed because fMRI data lacks the rich spectral information present in EEG data. Andersen et al. [31] applied CP decomposition to finger-tapping fMRI data. Beckmann et al. [32] proposed a variation of CP which extends probabilistic independent component analysis (PICA) to higher orders by adding an independence constraint in the spatial dimension. While originally applied to task fMRI data, Damoiseaux et al. [33] applied tensor-PICA to resting fMRI data and identified 10 networks consistent with previous findings.

In previous studies where CP decomposition was applied to either EEG or fMRI data [24]–[29], [31], [32], [34], [35], the size of the datasets is relatively small (the largest dimension has an order of 10⁴ elements or less). To explore dynamic functional connectivity, especially large-scale dynamics, we need to explore much larger datasets. There are two issues that have limited studies of this kind: scalability and robustness.

Scalability:

The majority of the studies cited above either truncate the data into short temporal segments (e.g. in EEG ictal and event related potential recordings) or heavily down-sample the data in the spatial domain (e.g. in fMRI studies) or sometimes both. The degrees of freedom (DOF), approximately reflected by the largest dimension in the CP model for a typical 10-minute SEEG recording used in our experiments is larger (≥ 10⁵) than in these previous studies. Moreover, as we will show in Section II below, the computational complexity increases approximately quadratically as the DOF increase when using ALS to estimate rank. Additionally, to achieve similar quality of solutions relative to our algorithm, ALS has to be applied with multi-start as shown in Section III, resulting in an even higher complexity. Therefore, in order to compute CP decompositions on data of this size, a fast and efficient algorithm is required.

Robustness (against local minima):

It is well known that the ALS algorithm on a CP model is not guaranteed to converge to a global minimum or a stationary point, even when multi-start is applied during the optimization [14], [15], The local minimum problem becomes more severe as the number of components increases. Performance is further compromised when a larger number of components than necessary are fit to the data (over-factoring), resulting in splitting rank-1 components into two or more factors.

Several techniques have been explored to improve the robustness and efficiency of the ALS algorithm. For example, Rajih et al. [36] added a line search after each major ALS iteration. Navasca et al. [37] applied Tikhonov regularization on each sub-problem in ALS iteration. However, similar to the ALS alternatives reviewed above, these modifications result in significantly higher computation cost limiting their practical utility, particularly for large scale problems.

In 2D matrix scenarios, Haidar et al. [38] proposed an incremented-rank PowerFactorization (IRPF) approach to solve minimum-rank matrix recovery problems, where higher-rank solutions were obtained recursively using lower-rank results as warm initializations, resulting in a substantially improved performance compared to the standard convex optimization approach. Performance was also theoretically characterized later in [39], In this work, we extend IRPF to higher-order tensors, with the goal of resolving the scalability and robustness issues discussed above. We refer to our approach as “scalable and robust sequential CP decomposition” (SRSCPD). As we show below, this algorithm is more robust than ALS and can be extended to large-scale problems. An outline of the remainder of the paper is as follows. Section II will briefly summarize the notation that will be used in later sections. Section III will describe the SRSCPD framework and the experimental design for both simulation and in-vivo data. Section IV will present the experiment results and conclusions follow in Section V.

II. Notation and Preliminaries

We first define some necessary notation and review the ALS algorithm which we use as part of our SRSCPD framework in section III. We largely follow the notational conventions and definitions ofKolda and Bader [14].

A. Scalar, Vector, Matrix and Tensor

A scalar is denoted by a lowercase letter, e.g. x; a vector by a bold lowercase letter, e.g. $\mathcal{x}$; a matrix by a bold uppercase letter, e.g. X; and a tensor by a bold script letter, e.g. $\mathcal{X}$. The number of dimensions is called the order and each dimension is referred as a mode. We use a third-order tensor $X\in {ℝ}^{I\times J\times K}$ in the following with individual elements denoted by x_i,j,k The notation and algorithms extend naturally to higher-order tensors.

A rank-1 tensor can be expressed as the outer product of vectors, i.e. $X=\mathit{a}\circ \mathit{b}\circ \mathit{c}$ where “°” represents the vector outer product. We use the norm:

$$\Vert X\Vert =\sqrt{\sum _{i=1}^I\sum _{j=1}^J\sum _{k=1}^K{x}_{i,j,k}^2}$$ (1)

B. Matricization

Tensors can be unfolded or “matricized” into matrix form [14], Matricization along dimension n is denoted by X_(n) so that a third-order tensor $X\in {ℝ}^{I\times J\times K}$ can be matricized into ${\mathit{X}}_{(1)}\in {ℝ}^{I\times JK}\text{ or }{\mathit{X}}_{(2)}\in {ℝ}^{J\times IK}\text{ or }{\mathit{X}}_{(3)}\in {ℝ}^{K\times IJ}$.

C. Kronecker, Khatri-Rao and Hadamard Product

We use the definitions of the following matrix products defined in [14] and repeated here for convenience. The Kronecker product X ⊗ Y of matrix $\mathit{X}\in {ℝ}^{I\times J}$ and $\mathit{Y}\in {ℝ}^{K\times L}$ is defined as

$$\mathit{X}\otimes \mathit{Y}=\left[\begin{array}{cccc}{x}_{11}\mathit{Y}& x_{12}\mathit{Y}& \cdots & x_{1J}\mathit{Y}\\ x_{21}\mathit{Y}& x_{22}\mathit{Y}& \cdots & x_{2J}\mathit{Y}\\ ⋮& ⋮& \ddots & ⋮\\ x_{I1}\mathit{Y}& x_{I2}\mathit{Y}& \cdots & x_{IJ}\mathit{Y}\end{array}\right]$$ (2)

where x_ij is the (i,j)-th element of X.

The Khatri-Rao product X ⊙ Y of matrix $\mathit{X}\in {ℝ}^{I\times K}$ and $\mathit{Y}\in {ℝ}^{J\times K}$ is the column-wise Kronecker product of X and Y

$$\mathit{X}\odot \mathbf{Y}=\left[{\mathit{x}}_1\otimes {\mathit{y}}_1{\mathit{x}}_2\otimes {\mathit{y}}_2\cdots {\mathit{x}}_K\otimes {\mathit{y}}_K\right]$$ (3)

where x_i is the i-th column of X.

The Hadamard product X * Y of matrix $\mathit{X}\in {ℝ}^{I\times J}$ and $\mathit{Y}\in {ℝ}^{J\times J}$ is the element-wise matrix product

$$\mathit{X}*\mathit{Y}=\left[\begin{array}{cccc}{x}_{11}{y}_{11}& x_{12}{y}_{12}& \cdots & x_{1J}{y}_{1J}\\ x_{21}{y}_{21}& x_{22}{y}_{22}& \dots & x_{2J}{y}_{2J}\\ ⋮& ⋮& \ddots & ⋮\\ x_{11}{y}_{I1}& x_{I2}{y}_{I2}& \cdots & x_{IJ}{y}_{IJ}\end{array}\right].$$ (4)

In the following we use the property of the Khatri-Rao product [14]:

$${(\mathit{X}\odot \mathit{Y})}^{†}={\left({\mathit{X}}^T\mathit{X}*{\mathit{Y}}^T\mathit{Y}\right)}^{†}{(\mathit{X}\odot \mathit{Y})}^T$$ (5)

where X^† represents the Moore-Penrose pseudo-inverse of X

D. CP Decomposition

CP decomposes a tensor into a sum of rank-1 tensors or components. For a third-order tensor $\mathcal{X}\in {ℝ}^{I\times J\times K}$

$$\mathcal{X}=\sum _{r=1}^R{\mathit{a}}_r\circ {\mathit{b}}_r\circ {\mathit{c}}_r+\mathcal{E}$$ (6)

where ${\mathit{a}}_r\in {ℝ}^I$, $b_r\in {ℝ}^J$, $c_r\in {ℝ}^K$, R is the rank or the number of components and $\mathcal{E}$ is the error tensor. If we group the components in each mode into a matrix, i.e. let $\mathit{A}=\left[{\mathit{a}}_1{\mathit{a}}_2\cdots {\mathit{a}}_R\right]\in {ℝ}^{I\times R}$ and similarly for $\mathit{B}\in {ℝ}^{J\times R}$ and $\mathit{C}\in {ℝ}^{K\times R}$, then the CP decomposition can be expressed as [14]

$${\mathit{X}}_{(1)}=\mathit{A}{(\mathit{C}\odot \mathit{B})}^T+{\mathit{E}}_{(1)}$$ (7)

or

$${\mathit{X}}_{(2)}=\mathit{B}{(\mathit{C}\odot \mathit{A})}^T+{\mathit{E}}_{(2)}$$ (8)

or

$${\mathit{X}}_{(3)}=\mathit{C}{(\mathit{B}\odot \mathit{A})}^T+{\mathit{E}}_{(3)}$$ (9)

where A, B, C are called the loading matrices for the three modes respectively.

E. Computation of CP decomposition and the ALS algorithm

Suppose we want to find the best rank R approximation of $\mathcal{X}\in {ℝ}^{I\times J\times K}$ via

$$\underset{\widehat{\mathcal{X}}}{\text{min}}\Vert \mathcal{X}-\widehat{\mathcal{X}}\Vert +g(\widehat{\mathcal{X}})$$ (10)

Where $\widehat{\mathcal{X}}={\sum }_{r=1}^R{\lambda }_r{\mathit{a}}_r\circ {\mathit{b}}_r\circ {\mathit{c}}_r,{\lambda }_r$ represents the scale of where component r and a_r, b_r and c_r have unit norm. $g(\widehat{\mathcal{X}})={\mu }_1{g}_1(\mathit{A})+{\mu }_2{g}_2(\mathit{B})+{\mu }_3{g}_3(\mathit{C})$ is a regularizing function with (μ₁,μ₂,μ₃) the corresponding regularization parameters. The ALS algorithm solves this problem in an alternating fashion. We first solve for A with B and C fixed, then solve for B with A and C fixed, and so on. This procedure is repeated until some convergence criterion is satisfied. Note that, for quadratic regularizes, each sub-problem reduces to ordinary least square. Specifically, assume B and C are fixed and we are solving for A. Using the equivalent matrix expression discussed above, we can write the optimization problem as

$$\widehat{\mathit{A}}=\underset{\mathbf{\text{A}}}{ \text{argmin} }{\Vert {\mathit{X}}_{(1)}-\mathit{A}{(\mathit{C}\odot \mathit{B})}^T\Vert }_F+{\mu }_1{g}_1(\mathit{A})$$ (11)

The solution with μ₁ = 0 (without regularization) reduces to a regular least square solution:

$$\widehat{\mathit{A}}={\mathit{X}}_{(1)}{\left[{(\mathit{C}\odot \mathit{B})}^T\right]}^{†}$$ (12)

Using the property in Eq. (5), we can rewrite as

$$\widehat{\mathit{A}}={\mathit{X}}_{(1)}(\mathit{C}\odot \mathit{B}){\left({\mathit{C}}^T\mathit{C}*{\mathit{B}}^T\mathit{B}\right)}^{†}$$ (13)

This expression is almost always preferable to (12) because it achieves a much lower computational complexity by only calculating the pseudo-inverse of an R×R matrix. Finally, we normalize each component and set λ_r equal to the normalization factor for the r^th component, r = 1,…, R. For the case μ₁ ≠ 0, the solution in (12) is replaced by the solution to (11), which will be closed form if g₁(A) is quadratic but may require an iterative solution in other cases. The full ALS algorithm is shown in Algorithm I.

III. Materials and Methods

A. SRSCPD Framework

The best rank-r approximation of a matrix with respect to the Frobenius norm is given by the leading r factors of the SVD. This is not the case for CP decomposition of a higher-order tensor. Kolda [40] showed an example where the best rank-1 approximation is not part of the best rank-2 approximation of a tensor. As a result, components in the CP decomposition for a given desired rank should be found simultaneously. Smilde et al. [41] (example 4.3) showed that the naïve sequential CP, in which a rank-1 tensor is fit to the residue at each iteration, failed to extract the correct components even when the data are known to be perfectly trilinear. Interestingly, this greedy sequential approach is still frequently used, simply because it is the most tractable approach to fitting tensor models to large datasets [42], [43].

The determination of tensor rank is NP-hard [44], Many metrics have been proposed to help find the correct rank, e.g. the core consistency diagnostic (CORCONDIA) [45], difference in fit (DIFFIT) [46] and automatic relevance determination (ARD) [47], All these metrics require a set of decomposition results for all ranks up to the maximum rank R. Obtaining such a set of solutions using CP decomposition is quadratically more complex than finding a rank-1 approximation, as we need to compute the decompositions for each rank r = 1,2…, R separately (1 + 2 + ⋯ + R = 0(R²)). This represents a significant challenge to use of higher rank tensor models and was the primary motivation for our development of the SRSCPD framework.

The SRSCPD framework is built on the original ALS algorithm. Our goal is to compute a rank-recursive set of decompositions from rank 1 to rank R. Our approach uses the result for rank r to initialize the decomposition for rank r + 1. Initialization for the additional component is found by fitting a rank-1 tensor to the residual from the rank r fit. In contrast, the original ALS algorithm does not use any information from rank r when fitting a model of rank r + 1. This “warm start” greatly improves convergence speed relative to the standard ALS algorithm except in very low rank cases (see simulation result in Fig. 4). The warm start may also help to avoid poor local minima in this non-convex optimization problem. As a result, we are able to address the problems with robustness and scalability for large-scale datasets.

Figure 4:

Simulation results. Boxplots of the run time in seconds over 100 Monte Carlo trials are shown as a function of R. M denotes the number of random initializations when using original ALS algorithm. Top-left panel shows the zoomed-in results for a better comparison for lower rank data.

The full SRSCPD framework is shown in Algorithm II, for a third-order tensor example. The inputs of the algorithm are a tensor $\mathcal{X}\in {ℝ}^{I\times J\times K}$ and the desired maximum rank R. For each iteration r, a rank-r approximation is calculated using the original CP-ALS algorithm with initializations {A*,B*, C*,λ*}. The initializations are formed by concatenating the solutions {A^r−1,B^r−1,C^r−1,λ^r−1} from the previous iteration r − 1 with the rank-1 approximation {a′,b′,c′,λ′} of the residue tensor ${\mathcal{X}}_{res}$, where ${\mathcal{X}}_{res}$ is obtained by subtracting the reconstructed tensor using {A^r−1,B^r−1,C^r−1, λ^r−1} from the original data tensor $\mathcal{X}$.

SRSCPD is flexible in the sense that techniques that have been proposed to improve the ALS algorithm can be directly incorporated. For example, one can add a line search along the estimated gradient descend direction for each mode at the end of each major iteration of ALS [36], Moreover, constraints and regularization terms can be applied to each of the ALS sub-problems, e.g. non-negativity, sparsity, and smoothness.

B. Simulation

We simulated SEEG data [48] with 100 channels, 200 Hz sampling rate, 2 second duration for ranks from R = 1 to 10. In each component a total of N channels were co-activated where N was chosen randomly between 2 and 10. For each of the active channels for each component we generated a time series to represent a block activation pattern with the signal switching on and off, respectively, in active and inactive blocks. The number of active blocks over the 2-second period was selected randomly between 2 and 5 and both the minimum block length and the minimum interval between any adjacent activated blocks was set to 0.1 second. Within each active block, the signals in each component were unit amplitude sinusoids with frequencies chosen randomly between 10 and 80 Hz. Finally, we added white Gaussian noise to the simulated data with a range of SNRs.

The third-order tensor $\mathcal{X}$ was generated by calculating the magnitude squared of the complex Morlet wavelet transform (MWT) coefficients of the simulated data matrix with center frequency 1 Hz, time resolution full-width-half-maximum (FWHM) of 2 seconds [49] in a linearly spaced frequency range from 1 to 100 Hz with interval 1 Hz. Thus, the final tensor $\mathcal{X}$ has the dimensions of ${ℝ}^{I\times J\times K}$, where I = 100,j = 400, K = 100. An example of the model used to simulate the data is shown in Fig. 1. Note that overlaps between components may occur in any of the three modes.

Figure 1:

An example of the simulated data with 5 components. Each component is represented by a distinct color in all three modes. From left to right: The channel (spatial) mode shows the activated channels that participate in each network; The time (temporal) mode shows the block activation pattern for each network; The spectral mode shows the Morlet wavelet frequency spectrum for each network.

We first compared the robustness of the decomposition using the SRSCPD framework against ALS using 1, 2 and 5 random initializations (generated from a standard uniform distribution in the interval (0,1) using MATLAB (The MathWorks, Inc., Natick, MA, USA) function “rand”). The same convergence criterion was used for both algorithms and in all cases, we computed solutions from rank 1 to R. In both algorithms we used a non-negativity constraint on all loading matrices, because the squared magnitude of the wavelet coefficients are naturally non-negative and the constraint helps avoid degeneracy [50], Let $\mathit{A}\in {ℝ}^{I\times R}$, $\mathit{B}\in {ℝ}^{J\times R}$, $\mathit{C}\in {ℝ}^{K\times R}$ be the loading matrix in each of the three modes as described in Section II. Then in each sub-problem of the ALS, we used the following cost function for A (likewise for B and C)

$$\widehat{\mathit{A}}=\underset{\mathbf{\text{A}}}{ \text{argmin} }{\Vert {\mathit{X}}_{(1)}-\mathit{A}{(\mathit{C}\odot \mathit{B})}^T\Vert }_{F}s.t.\mathit{A}\succcurlyeq 0$$ (14)

where “≽” denotes the element-wise inequality.

Since we know the ground truth under the simulated settings, we assessed the quality of the solutions using the averaged congruence product (ACP) [51], ACP is a measure of correlations between components. Specifically, let A, B, C be the column-wise normalized ground truth loading matrices and $\widehat{\mathit{A}},\widehat{\mathit{B}},\widehat{\mathit{C}}$ their estimated counterparts. Then the ACP is defined per [51]as

$$\text{ACP}=\underset{p}{\text{max}} \text{tr} \left(\left({\mathit{A}}^T\widehat{\mathit{A}}\right)*\left({\mathit{B}}^T\widehat{\mathit{B}}\right)*\left({\mathit{C}}^T\widehat{\mathit{C}}\right)\mathit{p}\right)$$ (15)

where P is a permutation matrix accounting for the ambiguity of the ordering of the solutions [16] and tr(X) indicates the trace of X.

We evaluated the ACP of the solutions obtained from both ALS and SRSCPD as a function of R for SNR = 10 . For each R, we ran 100 Monte Carlo trials and boxplots of ACP were generated. For each simulated tensor, we repeated ALS M times, where M = l, 2, and 5, each time using a different random initialization. The final solution was selected as that which has the lowest cost. We also box-plotted the Frobenius norm error as shown in Eq. 10 for each trial. Additionally, we recorded the run time for each of the methods. We then repeated the above study, but instead of varying R we conducted the experiment as a function of SNR with R = 5.

C. Application to In-Vivo SEEG Dataset

We performed retrospective analysis of patient data collected under an Institutional Review Board approved protocol for SEEG evaluation and monitoring in the Epilepsy Center, Cleveland Clinic, OH, USA. The SEEG evaluation performs invasive pre-surgical electrophysiological mapping for patients who have pharmaco-resistant focal epilepsy. For each patient, the implantation was performed using multi-lead depth electrodes, with each electrode comprising typically 10 contacts spaced a few millimeters apart (AdTech, Racine, Wisconsin; Integra, Plainsboro, New Jersey; or PMT, Chanhassen, Minnesota). The electrode locations were determined after a multidisciplinary patient management conference where the hypotheses about the epileptogenic zone were drawn based on available noninvasive data: clinical history, video EEG, MRI, PET, ictal SPECT and MEG. The electrodes were implanted according to the Talairach stereotactic method using orthogonal or oblique trajectories [52], The implantation schemes are shown in Table I. The SEEG signals were recorded using a common reference on a Nihon Kohden EEG system with a sampling rate of 1000 Hz.

TABLE I:

Summary of the Patient Data

Subject ID	1		2
#of Channels	69		113
Epilepsy Type	Posterior cingulate epilepsy		Left fronto-parietal
Data Segment Time	09:23:00–09:33:00	Same Day: 13:22:02–13:32:02	21:50:30–22:00:30	Next Day: 08:35:20–08:45:20
Implantation Scheme

We chose two 10-minute data segments a minimum of 4 hours apart (see Table I) for each patient. The segments were selected using annotated video of the patients for periods of physical inactivity (e.g. reading, watching TV). Using co-registration of the post-implant X-ray CT to the patients’ MR image, we selected the subset of the SEEG contacts that were in gray matter.

For each data segment we applied the MWT with center frequency 1 Hz and time resolution FWHM of 2 seconds in a linearly spaced frequency range of 1 to 100 Hz with interval 1 Hz. We computed the squared magnitude of the wavelet coefficients and temporally down-sampled the resulting envelope data in each frequency band by a factor of 5, resulting in a new envelope sampling rate of 200 Hz. We performed a flattening of the power spectrum to compensate for its “1/f” characteristics to emphasize the higher frequency components. The resulting data were then represented as a third-order tensor $\mathcal{X}\in {ℝ}^{I\times J\times K}$, where I is the number of channels (see Table I), J = 120,000 is the number of time points and K = 100 isthe number of frequency bins.

We applied SRSCPD to each tensor with three additional constraints: non-negativity on all three modes due to the non-negative squared magnitude of the wavelet coefficients; a sparsity constraint on the spatial (channel) mode as we assumed that in each network only a small set of channels would be involved; and a smoothness constraint on the spectral mode reflecting the limited frequency resolution of the MWT. Let $\mathit{A}\in {ℝ}^{I\times R}$, $\mathit{B}\in {ℝ}^{J\times R}$, $\mathit{C}\in {ℝ}^{K\times R}$ be the loading matrices for each of the three modes. Then in the sub-problems of the ALS, we use the following cost functions:

$$\widehat{\mathit{A}}= \underset{\mathbf{\text{A}}}{ \text{argmin}}{\Vert {\mathit{X}}_{(1)}-\mathit{A}{(\mathit{C}\odot \mathit{B})}^T\Vert }_F^2+{\mu }_1\sum _i{\Vert {\mathit{A}}_{\cdot i}\Vert }_{l_1},s\text{.}t\text{. }\mathit{A}\succcurlyeq 0$$ (16)

$$\widehat{\mathit{B}}=\underset{B}{ \text{argmin}}{\Vert {\mathit{X}}_{(2)}-\mathit{B}{(\mathit{C}\odot \mathit{A})}^T\Vert }_F^2,s\text{.}t\text{.}\mathit{B}\succcurlyeq 0$$ (17)

$$\widehat{\mathit{C}}=\underset{C}{ \text{argmin}}{\Vert {\mathit{X}}_{(3)}-\mathit{C}{(\mathit{B}\odot \mathit{A})}^T\Vert }_F^2+{\mu }_3\sum _k{\Vert \nabla {\mathit{C}}_{\cdot k}\Vert }_{l_2}^2,s\text{.}t\text{. }\mathit{C}\succcurlyeq 0$$ (18)

where X._i denotes the i^th column of X, μ₁ and μ₃ are the regularization parameters which were set to 0.2 empirically $\Vert \cdot {\Vert }_{l_1}$ and $\Vert \cdot {\Vert }_{l_2}$ denotes the l₁ norm and l₂ norm respectively.▽ is the finite difference operator on the columns of C. We solved each of the convex sub-problems (16) – (18) using Auslender and Teboulle’s single-projection algorithm [53] in the TFOCS toolbox [54].

Finally, the rank of the tensor was estimated based on the decomposition results using CORCONDIA [45], This rank metric used the fact that the trilinearity of components starts decreasing in the case of over-factoring (the number of fitted components is greater than the actual rank).

IV. Results

A. Simulation

Fig. 2 shows performance of ALS vs SRSCPD as a function of rank R. For small R all results are similar. However, for larger ranks we see that the ALS results are strongly dependent on initialization and that performance for M = 5 is significantly better than for M = 2 and M = 1. SRSCPD benefits from using the results of the lower rank as an initialization, resulting in overall improved performance (higher median ACP) relative to all three versions of ALS.

Figure 2:

Simulation results. Boxplots of ACP over 100 Monte Carlo trials are shown as a function of R. M denotes the number of random initializations when using the original ALS algorithm.

Fig. 3 shows the corresponding Frobenius norm error. As with Fig. 2, ALS with M = 1 and 2 shows larger error than M = 5, with the difference increasing with rank. In contrast to the ACP metric in Fig. 2, the error for ALS with M = 5 in Fig. 3 is very similar to that for SRSCPD and sometimes smaller for higher ranks. Closer examination of these results revealed that in cases where this occurs, ALS fails to find one or more of the weaker components that SRSCPD does find. Instead, part of the noise in the data is fit to one of the tensor components. This in turn leads to a lower squared error in the fit even though the extracted components are a poorer fit to the ground truth as measured with ACP.

Figure 3:

Simulation results. Boxplots of the Frobenius norm error over 100 Monte Carlo trials are shown as a function of R. M denotes the number of random initializations when using the original ALS algorithm.

Fig. 4 shows the run time as a function of R (the run time was measured using MATLAB with Dell Precision T3610 computer, Intel Xeon E5-1650 v2 CPU). As expected, the ratios of the run time among the ALS methods are approximately proportional to M, the number of different initializations. The cost of SRSCPD is significantly lower than that for ALS with M = 2 and 5 restarts. As the rank increases (R > 4), the cost for SRSCPD is even lower than that for ALS without restart, M = 1. The reason for this is that the warm start in SRSCPD produces a better initialization that not only results in improved performance (Fig. 2) but also faster convergence of the ALS sub-problems as shown in Fig. SI in the supplemental material.

Fig. 5 shows that as the SNR increases, ACP also improves for all methods. SRSCPD shows generally similar performance to ALS with M = 5 restarts and is substantially better than results for M = l. However, for lower SNRs, the performance of ALS with M = 5 restarts is superior to SRSCPD.

Figure 5:

Simulation results. Boxplots of the ACP over 100 Monte Carlo trials are shown as a function of SNR. M denotes the number of random initializations when using the original ALS algorithm.

B. In-Vivo SEEG Dataset

1). Estimation of Rank

Fig. 6 shows plots of the CORCONDIA rank metric as a function of R for the two sessions for both subjects. Per recommendations in [45], rank should be chosen so that the CORCONDIA value is higher than 0.9. Based on the plots in Fig. 6, we selected R = 3 and 4 for the two sessions of Subject 1 and R = 5 and 3 for the two sessions of Subject 2.

Figure 6:

CORCONDIA rank metric are shown as a function of rank R for two sessions of both subjects.

2). Intra-subject Network Comparison

We found corresponding consistent components (CCs) across the two sessions as the pair of components that had the largest product of spatial congruence and spectral congruence, i.e. for each component ${\mathit{a}}_i^1,{\mathit{b}}_i^1,{\mathit{c}}_i^1$ in the first session, we found (${\mathit{a}}_j^2,{\mathit{b}}_j^2,{\mathit{c}}_j^2$) in the second session such that

$$j=\underset{j}{ \text{argmax}} \text{tr} (\left({\mathit{a}}_i^{1T}{\mathit{a}}_j^2\right)*\left(\left({\mathit{c}}_i^{1T}{\mathit{c}}_j^2\right)\right),i=1,\dots ,R$$ (19)

where ${\mathit{a}}_i^k,{\mathit{b}}_i^k,{\mathit{c}}_i^k$ represent the ith spatial, temporal and spectral component of the k^th (k = 1, 2) session, respectively.

Note that we do not expect temporal congruence across sessions. For Subject 1, we found three CCs with large congruence product (>0.6). Fig. 7 shows two of these components, the third is included in the supplemental material, Fig. S2. The congruence was 0.892 (spatial mode) and 0.997 (spectral mode) for component (a) and 0.984 (spatial mode) and 0.967 (spectral mode) for (b).

Figure 7:

Two consistent components for Subject 1 show: (a) locations consistent with activity in the default mode network within the alpha band and (b) Motor activity in the beta frequency range. For each pair of consistent components, we show modes for session 1 in red and session 2 in blue in the top row. From left to right: The channel (spatial) mode shows the activated channels that participate in each network; The time (temporal) mode shows the dynamic variations of each network (only the first 10 seconds is shown for better visualization); The spectral mode shows the frequency-dependent component of the tensor. In the bottom row the left and middle sub-figures show the spatial distribution of the activated channels mapped onto the subject’s smoothed cortical surface. For visualization purposes, a contact or channel is defined as activated if the value of the (normalized) channel mode at that contact exceeds a threshold of 0.05 in both sessions. The right sub-figure shows the Welch power spectrum of the temporal mode.

The first network for Subject 1 includes contacts in angular gyrus, mid-temporal gyrus, and precuneus all consistent with activation in the Default Mode Network (DMN) [55], Electrodes were not present in this subject for other regions that are typically included in the DMN, such as the superior frontal gyrus (SFG). The spectrum for this component is dominated by alpha rhythms, which is consistent with studies of the DMN in the EEG/MEG literature [56]–[58], The second network we found for this subject contains contacts mainly in the somatosensory and motor cortex with a peak in the beta band, which is consistent with sensorimotor rhythms as reported in the EEG/ECoG literature [59], [60]: beta activity appears most strongly in both motor and somatosensory cortex in movement preparation periods and steady contraction periods following a movement. These subjects were monitored during periods of natural inactivity when we might expect some amount of motor activity, for example page-turning while reading.

The temporal mode shows the power envelope of the SEEG signals, i.e. the dynamic variation in power across time of the spectral model. The bottom right sub-figure shows the power spectrum of this envelope estimated using the Welch method [61] after high-pass filtering with a cutoff frequency 0.02 Hz to remove DC drift. For both components, the power spectral density peaks at a frequency of approximately 0.1 Hz (0.05 Hz - 0.15 Hz), which is similar to the dominant frequency found in resting fMRI BOLD (Blood Oxygen Level Dependent) oscillations [62].

For Subject 2, we also found three CCs. Fig. 8 shows two of the three, the remaining one is shown in Fig. S3. Again, the similarity between sessions was high with congruence 0.652 (spatial mode) and 0.959 (spectral mode) for the DMN and 0.799 (spatial mode) and 0.957 (spectral mode) for the somatomotor network. In this subject there are electrodes in the SFG, unlike the first subject, and we now observe that the DMN does indeed include SFG. As with the first subject we see a second strong component in the somatosensory and motor cortex. However, unlike the first subject, the signal in this case is predominantly mu rather than beta. Again, this is consistent with periods of natural inactivity, where mu rhythms are typically observed in somatomotor cortex in parallel with alpha activity in the visual cortex during resting [63].

Figure 8:

Two consistent components for Subject 2: (a) default mode network in alpha frequency and (b) motor network centered on the mu range. Details as for Fig. 6.

3). Inter-subject network comparison

When comparing the results between Subject 1 and Subject 2, we found that in both cases the DMN activity is dominated by alpha activity while the motor network is predominantly beta or mu, despite the fact the two subjects had different electrode implantation schemes and different locations of their epileptogenic zones (see Table II).

4). Artifact Detection

SRSCPD not only can be used to identify brain networks, it can also detect artifacts. For example, in the supplemental material, Fig. S4 shows a component that was mismatched (i.e. low spatial and spectral congruence) between the two sessions for Subject 1. Similarly, Fig. S5 shows one of the components that was mismatched between sessions for Subject 2. These mismatched components are likely artifacts as they mostly contain a burst or bursts of activity on a very limited number of channels and the time courses do not look physiological in nature.

5). Comparison to Results Using ALS Algorithm

We also applied the traditional ALS algorithm to the same datasets and compared the components obtained from the two methods. ALS did not find as many functionally distinct networks as SRSCPD did from each individual session based on the same rank selection criterion. The rank was estimated to be R = 2 and 3 for the two sessions of Subject 1 and R = 3 and 3 for the two sessions of Subject 2 using the ALS algorithm as shown in Fig. S6. Moreover, we only found one CC (a DMN shown in Fig. S7) between the two sessions in one of the subjects. No other CCs were found as the maximum congruence product (Eq. 19) was less than 0.15 between all other pairs of components. These results show that ALS is not as robust as the SRSCPD algorithm especially when SNR is low. See supplemental materials (Fig. S6 – S16) for details.

V. Conclusion

We have described a novel framework for decomposition of electrophysiological data using a third-order tensor. Our SRSCPD approach is based on the original ALS algorithm, using a warm-start in a rank-recursive search to both improve the quality of results and reduce computation cost relative to conventional ALS with multi-start. The SRSCPD framework is scalable to large datasets due to its use of the warm start. We have shown its application to SEEG data in two subjects with epilepsy and found two consistent brain networks across different sessions of recordings several hours apart and in two different subjects. In contrast, ALS found only one consistent network (the default mode) in one subject. This consistency shows promise for the use of SRSCPD for robust identification of spontaneous brain network activity from invasively recorded EEG in individual subjects.