Multi-Subject Analysis for Brain Developmental Patterns Discovery via Tensor Decomposition of MEG Data

Abstract

Objective:

Identification of informative signatures from electrophysiological signals is important for understanding brain developmental patterns, where techniques such as magnetoencephalography (MEG) are particularly useful. However, less attention has been given to fully utilizing the multidimensional nature of MEG data for extracting components that describe these patterns.

Methods:

Tensor factorizations of MEG yield components that encapsulate the data’s multidimensional nature, providing parsimonious models identifying latent brain patterns for meaningful summarization of neural processes. To address the need for meaningful MEG signatures for studies of pediatric cohorts, we propose a tensor-based approach for extracting developmental signatures of multi-subject MEG data. We employ the canonical polyadic (CP) decomposition for estimating latent spatiotemporal components of the data, and use these components for group level statistical inference.

Results:

Using CP decomposition along with hierarchical clustering, we were able to extract typical early and late latency event-related field (ERF) components that were discriminative of high and low performance groups $(\mathit{p}<0.05)$ and significantly correlated with major cognitive domains such as attention, episodic memory, executive function, and language comprehension.

Conclusion:

We demonstrate that tensor-based group level statistical inference of MEG can produce signatures descriptive of the multidimensional MEG data. Furthermore, these features can be used to study group differences in brain patterns and cognitive function of healthy children.

Significance:

We provide an effective tool that may be useful for assessing child developmental status and brain function directly from electrophysiological measurements and facilitate the prospective assessment of cognitive processes.

1. Introduction

Characterization and identification of typical brain developmental patterns can provide important insights into brain organization and function. Brain function can be described by cognitive processes, which may include, but are not limited to, sensory, grammatical, semantic processing, memory retrieval, or motor events (Hernández, Puupponen, & Jantunen, 2022). One way of studying the timing properties of cognitive processes is by focusing on their underlying brain mechanisms. This can be achieved with the use of functional neuroimaging techniques such as magnetoencephalography (MEG) and electroen-cephalography (EEG) (Hernández et al., 2022) by extracting neural sources (latent components) describing these processes. MEG is a powerful neuroimaging technique that measures the magnetic fields generated by neuronal activity, which arises collectively within the brain from population neuronal responses to target stimuli (Hämäläinen, Hari, Ilmoniemi, Knuutila, & Lounasmaa, 1993). The direct measurement of neuronal currents by MEG, its high spatial resolution, and its excellent temporal resolution makes it an especially useful noninvasive technique for studying brain function. Thus, MEG is the preferred method for studying the fast spatiotemporal dynamics of brain activity (He & Liu, 2008). To this end, event-related fields (ERFs) and event-related potentials (ERPs) have been demonstrated as important research tools for understanding brain developmental patterns of pediatric cohorts. ERFs and ERPs represent a time-locked MEG/EEG activity that measures brain response elicited by stimuli.

Multi-subject MEG studies that focus on large pediatric cohorts have high potential to provide important insights into brain organization and brain development in children and adolescents. One approach to this is by using multi-subject latent component analysis, through which latent components within multiple MEG datasets are learned jointly by exploiting dependence across the datasets. However, as we discuss next, many existing multi-subject latent component analysis techniques in neuroimaging are based on matrix factorization methods. Matrix representations cannot account for the multiple dimensions of the data, such as participant, stimulus condition, variations in time and space, and the relationships across these dimensions.

Over the past decades, substantial efforts have been made in finding ways to model and extract common hemodynamic or electrophysiological components from multi-subject task and resting state neuroimaging (NI) data such as fMRI and MEG/EEG. The common components detected during the task may be indicative of a typical or atypical patient state and can be further used to discover the prognostic imaging biomarkers. Data fusion and joint analysis methods based on matrix decompositions such as joint independent component analysis (jICA) (Calhoun, Adali, Pearlson, & Kiehl, 2006), group ICA (GICA) (Calhoun & Adali, 2012; Calhoun, Adali, Pearlson, & Pekar, 2001; Labounek et al., 2018; Salman et al., 2019), dictionary learning (Akhavan, Baghestani, Kazemi, Karami, & Soltanian-Zadeh, 2022; Jin, Dontaraju, Kim, Akhonda, & Adali, 2020), independent vector analysis and its transposed variant (tIVA) (Adali, Levin-Schwartz, & Calhoun, 2015) have been used for analysis and fusion of NI data. A reason for the popularity of these methods is the convenience of presenting the time-varying NI data as a matrix of time × space. It was shown in many studies that these matrix-based approaches are powerful for extracting meaningful components (Calhoun & Adali, 2012). The group-level ICA methods exploit high-order statistics (Hyvärinen & Oja, 2000) of the data and enable assessment of complex spatiotemporal relationhips (Calhoun & Adali, 2012). However, a primary problem of two-way techniques is that components are defined only by two signatures, which are not determined uniquely without further constraints on the model. The uniqueness is achieved by imposing constraints such as independence or sparsity (Acar, Rasmussen, Savorani, Næs, & Bro, 2013; Adali, Anderson, & Fu, 2014; Akhavan et al., 2022; Jin et al., 2020; Lahat, Adali, & Jutten, 2015). To use matrix-based methods for the higher dimensional data, the unfolding and the dimension reduction into a matrix is required, which is done by concatenation or stacking of the data (Calhoun, Liu, & Adalı, 2009; Delorme & Makeig, 2004). Such unfolding inevitably discards the inherent multilinear structure of brain imaging data, and therefore, may ignore complex important interactions between/among the folded modes (Cong et al., 2015).

Given that most NI data can be conveniently expressed as a high order array, tensor decomposition techniques are preferred to represent the original data as a mixture of the latent components with corresponding signatures from each dimension. In addition, certain tensor decompositions provide uniqueness under mild constraints. The uniqueness property is critical for an unambiguous interpretation of the components, finding matches with neural processes and/or component signatures. Moreover, tensors provide a natural representation of the inherently multidimensional NI data and preserve the structural information among the tensor modes, thus effectively exploiting the multilinear correlation structure and enabling robust group-level statistical analyses for multiple datasets.

Among the tensor decomposition techniques, the canonical polyadic (CP) decomposition and the Tucker decomposition (Carroll & Chang, 1970; Kolda & Bader, 2009) are particularly useful in fMRI, MEG/EEG processing of real (Cong et al., 2015) and complex-valued data (Kuang et al., 2019). The main advantage of the CP model (Carroll & Chang, 1970) is that it is essentially unique up to scaling and permutations (Sidiropoulos & Bro, 2000). However, it is worth noting that the CP model cannot effectively take into account higher-order statistical information like ICA-based methods (Kroonenberg & De Leeuw, 1980). The disadvantage of the Tucker decomposition compared with the CP model is limited model interpretability without imposing the orthogonality constraint, which is unrealistic for the brain components. Thus, in this paper we chose the CP tensor format as our primary model of interest.

Tensor-based analysis of MEG/EEG has received increased attention during the last decade (Chatzichristos et al., 2022; Cong et al., 2012; Liu et al., 2021; D. Wang, Zhu, Ristaniemi, & Cong, 2018; X. Wang, Liu, Toiviainen, Ristaniemi, & Cong, 2020; Zhu et al., 2020). CP model was extensively used for high-order decompositions of multi-subject EEG data (organized as a channel × time × subject third-order tensor) or wavelet EEG (organized as a channel × time × frequency × subject) (Cong et al., 2012; Vanderperren et al., 2013; D. Wang et al., 2018; X. Wang et al., 2020). The specialized multiway algorithms have been proposed for ERP analysis of EEG to deal with noisy and nonstationary signals Bayesian CP model (Wu et al., 2014) and fifth-order ERP feature extraction (D. Wang et al., 2018). In (Kinney-Lang, Ebied, Auyeung, Chin, & Escudero, 2019; Kinney-Lang, Spyrou, Ebied, Chin, & Escudero, 2018) authors employed CP decomposition for developmental feature extraction from EEG pediatric datasets. In (Liu et al., 2021; Zhu et al., 2020) CP model was used to study functional connectivity patterns of MEG data (organized as time × frequency × connectivity third-order tensor).

Despite a substantial number of studies dedicated to high-order ERP analysis, the multidimensional nature of MEG has not been fully exploited for the data-driven extraction of sensor-level ERF components. MEG ERF components can better inform about the rapid spatiotemporal dynamics of brain information processing compared with EEG due to higher spatial resolution of MEG. Provided that ERFs are collected using the same stimuli, the assumption is that activity elicited by the same stimuli is highly correlated among subjects, which can be seen as a prerequisite for applying the CP decomposition. Thus, multisubject MEG studies generate ERFs that can be naturally represented using CP tensor format.

Several works (Ablin, Cardoso, & Gramfort, 2021; Boonyakitanont et al., 2022; Ikeda & Toyama, 2000; Jung et al., 2001; Stephen et al., 2013) focus on the characterization and identification of sensor-level MEG ERFs using matrix-based approaches such as ICA/jICA algorithms. The algorithms that have been used in (Ablin et al., 2021; Boonyakitanont et al., 2022; Ikeda & Toyama, 2000; Jung et al., 2001; Pinner, Coffman, & Stephen, 2020; Stephen et al., 2013) transform inherently three-dimensional (3D) multi-subject MEG data into two-dimensional (2D) matrix representation. For multi-subject MEG data, such a 2D transformation loses the multidimensional low-rank structure that may provide an intrinsic description of the spatiotemporal interactions. On the hand, the low-rank structure of MEG data can be fully captured by the CP tensor format as we propose. Hence, we model the multi-subject MEG data as a 3D tensor with dimensions of subject × time × channel. This high-order representation of the multi-subject MEG dataset maximizes the simultaneous use of spatiotemporal modes and multilinear interactions across modes within the data.

Our goal is the identification of typical brain developmental patterns that could be used as descriptive imaging signatures in a healthy population of children and adolescents. Using CP decomposition, we propose a group-level tensor analysis method to characterize and identify sensor-level ERF components in task-related multi-subject MEG data. The proposed model enables the analysis of a multi-subject MEG dataset as a third-order tensor and, thus, exploits the multidimensional nature of the group-level data. We use hierarchical clustering on principal components (HCPC) (Argüelles, Benavides, & Fernández, 2014; Husson, Josse, & Pagès, 2010) approach to find subject groups using a supplementary cognitive measures dataset.

We summarize our contributions as follows:

• The paper presents a CP analysis framework to robustly identify common brain developmental patterns for multi-subject sensor-level MEG data. The proposed formulation of the CP model shown in Fig. 2 was capable of extracting typical early (M150), and late latency (M300a and M400) ERF components representative of visual spatial attention, associative memory and semantic processing, similar to the results reported in existing ERF/ERP studies.

• We develop a group-level inference approach that allows performing robust statistical inferences directly using CP component matrices.

• We demonstrate the statistical significance of tensor group-level analyses by identifying the discriminative ERF components that can differentiate between high performant and low performant groups.

• We show that the discriminative ERF components were significantly correlated with major cognitive domains such as attention, episodic memory, executive function, and language comprehension.

A preliminary work using the same MEG data with similar clustering of subjects used ICA model and is presented as a conference contribution (Boonyakitanont et al., 2022). The current paper presents a novel formulation for group-level analysis using the CP model, a detailed description of the clustering approach (see Section 5.2), and novel experimental results.

Fig. 2.

Brain developmental patterns discovery via tensor decomposition. (a)-(b)Illustration of tensor analysis for multi-subject MEG data. (a) Tensor formation by arranging the subjects along the first dimension. (b) MEG tensor decomposition into $R$ rank-1 components. Each rank-1 component represents a distinct spatiotemporal brain activity pattern with subject weight $\left({\mathbf{a}}_r\right)$ and temporal $\left({\mathbf{b}}_{\mathbf{r}}\right)$ and spatial signatures $\left({\mathbf{c}}_{\mathbf{r}}\right)$. (c) Tensor group-level analysis, which includes subgroup identification and group-level statistical inference. (d) Neural association patterns with functional brain areas discovered via correlation analyses.

This paper is organized as follows. The description and notations for CP decomposition are introduced in Section 2.1. We describe multidimensional generative data model and CP tensor decomposition for multi-subject MEG data in Section 3. The experimental setup is described in Section 4.2. In Section 5, the typical ERF components extracted from the CP model and the group-level tensor-based statistical inference results are presented. The experimental results are discussed in Section 6. The conclusions and future work are presented in Section 8.

2. Materials and Methods

2.1. Notations and Definitions

In this paper, the mathematical notations and definitions are adopted from (Kolda & Bader, 2009) and (Cichocki et al., 2016). We denote scalar with the lower case letter $x$, vectors with boldface lowercase letters $(\mathbf{x},\mathbf{y},\mathbf{z},\cdots )$, matrices with boldface capital letters $(\mathbf{X},\mathbf{Y},\mathbf{Z},\cdots )$, and tensors with bold calligraphic uppercase letters $(𝒳,𝒴,𝒵,\cdots )$. The number of dimensions is called the order, and each dimension is referred to as a mode. $\Vert ·{\Vert }_F$ denotes the Frobenius norm, $\mathbf{A}\otimes \mathbf{B}$ denotes the Kronecker and $\mathbf{A}\odot \mathbf{B}$ denotes the Khatri-Rao product. A rank-1 tensor is expressed as the outer product of vectors, i.e., $X=\mathbf{a}\circ \mathbf{b}\circ \mathbf{c}$, where $\circ$ represents the vector outer product. The mode-$n$ matricitization of a given tensor along dimension $n$ is denoted by ${\mathbf{X}}_{\left(n\right)}\in {\mathbb{R}}^{I_n\times I_1{I}_2\cdots I_{n-1}{I}_{n+1}\cdots I_N}$ (Kolda & Bader, 2009). The $n$—mode product of a tensor $𝒳\in {\mathbb{R}}^{I_1\times \cdots \times I_n\times I_N}$ and a matrix $\mathbf{A}\in {\mathbb{R}}^{J_n{I}_n}$ along the $n$th mode, denoted as $𝒳{\times }_n\mathbf{A}$, is a tensor of size $I_1\times \cdots \times J_n\times \cdots \times I_N$.

2.2. Participants

The participants included 170 healthy children (89 male, 81 female) and adolescents between the ages of 9 and 15 ($M=11.92$ years, $SD=1.18$) with no reported clinical diagnoses from the Mind Research Network (MRN) in Albuquerque, New Mexico (90) and the University of Nebraska Medical Center (UNMC) in Omaha, Nebraska (80) as part of the Developmental Chronnecto-Genomics (Dev-CoG) study (Stephen et al., 2021). The participants and parents signed consent forms approved by each institutional review board (IRB) prior to joining the study. All procedures were approved by the MRN and UNMC IRBs prior to the start of the experiment.

2.3. Neuropsychological Testing

All the participants completed the Wechsler abbreviated scale of intelligence (Second Edition; WASI-II; (Wechsler, 2011)) to assess full-scale IQ (FSIQ) and NIH-Toolbox Cognitive Battery(Weintraub et al., 2013) (NIHTB-CB) tests assessing age-adjusted neuropsychological (T) scores in six cognitive domains: attention, episodic memory, executive function, language, processing speed, and working memory. The data collection also included the Conners 3 Inattention/Hyperactivity scores (Conners, 2008) for assessing attention-deficit hyperactivity disorder (ADHD), and children with diagnosed ADHD were excluded from the study. There were no significant differences $(p>0.05)$ in terms of age or gender with respect to the neurophysiological measures in participants from MRN and UNMC.

2.4. MEG Experimental Paradigm

Participants completed a multisensory task while MEG data were recorded (see Fig. 1). The visual stimulus was a full-screen, black and white vertical grating (0.25 cycles/degree). The auditory stimulus was a 40 Hz modulated 1000 Hz tone. For multisensory stimulus, the auditory and visual stimuli were presented simultaneously. The baseline fixation was a red box in the center of the screen. Subjects were instructed to press their index finger when they saw anything, heard anything, or both. Each MEG trial began with a fixation for an intertrial interval (ITI) that pseudo-randomly changed between 2400 and 2600 milliseconds (ms) in 10 ms increments. Following fixation, a sensory stimuli (auditory (AUD), visual (VIS), or audio-visual (AV)) was presented for 800 ms (Stephen et al., 2021). The total task duration was approximately 18 min.

Fig. 1.

Multisensory task paradigm. The presentation started with a baseline fixation screen, followed by the appearance of auditory (AUD), visual (VIS) or multisensory stimulus (AV).

2.5. MEG Data Acquisition and Image Preprocessing

The MEG data acquisition and preprocessing details were previously published in (Stephen et al., 2021). MEG recordings were acquired with the Elekta/MEGIN MEG system with 306 magnetic sensors (204 gradiometers and 102 magnetometers) in a magnetically shielded room. The MEG data were continuously sampled at 1000 kHz with a passband between 0.1 and 330 Hz. The MEG sensor-level artifacts were removed during prepossessing at both the MRN and UNMC sites. MEG epochs between −100 to 1000 ms (1100 time points) around the stimulus onset were averaged across 300 trials within respective stimuli and formed sensor-level ERFs time-locked to the stimulus condition (AUD, VIS, or AV).

3. Tensor Analysis of MEG Data for Brain Pattern Extraction

3.1. Multidimensional Model for Multi-Subject MEG Data

The MEG experimental paradigm shown in Fig. 1 results in simultaneously recorded neural measurements elicited in $C$ common sensors at $T$ timepoints across $K$ subjects. As a result, the observed MEG recordings are modeled as a mixture of the underlying neural sources of interest synchronized in time across subjects within a specific task. To identify the common brain developmental patterns elicited by sensory stimili across subjects, we applied the CP tensor decomposition to extract the latent brain activity patterns. The proposed approach has two important advantages. The CP representation of MEG data allows us to take into account the higher-order structure of the multi-subject data to extract common patterns across subgroups. By virtue of the CP decomposition, the MEG factorization provides a unique solution under mild constraints (Kruskal, 1977; Sidiropoulos & Bro, 2000). The importance of the uniqueness condition cannot be overstated since it allows finding meaningful components unambiguously and matching them to the true brain processes.

To preserve the intrinsic multidimensional nature of multichannel MEG data, the data are tensorized as third-order tensor $𝒳\in {\mathbb{R}}^{K\times T\times C}$ by stacking subject ERF matrices ${\mathbf{S}}_k\in {\mathbb{R}}^{C\times T}$ in subject mode. Fig. 2a shows the generative model for multidimensional representation of multi-subject MEG data.

3.2. Multi-Subject MEG Tensor Decomposition

By adopting the tensorization strategy shown in Fig. 2a, we present the CP model of the multisubject MEG data as a third-order tensor $𝒳\in {\mathbb{R}}^{K\times T\times C}$ $\left({\mathbb{R}}^{\text{subject}\times \text{time}\times \text{channel}}\right)$. The CP decomposition approximates tensor $𝒳\in {\mathbb{R}}^{K\times T\times C}$ as a sum of rank-1 tensors:

$$𝒳\approx \sum _{r=1}^R{\lambda }_r\circ {\mathbf{a}}_r\circ {\mathbf{b}}_r\circ {\mathbf{c}}_r=\mathbf{\Lambda }{\times }_1\mathbf{A}{\times }_2\mathbf{B}{\times }_3\mathbf{C}$$ (1)

where ${\mathbf{a}}_r\in {\mathbb{R}}^K,{\mathbf{b}}_r\in {\mathbb{R}}^T,{\mathbf{c}}_r\in {\mathbb{R}}^C$ are the factor vectors normalized to 2-unit norm; ${\lambda }_r$ represents the scale factor for each component, and the norms absorbed into diagonal matrix $\mathbf{\Lambda };\mathbf{A}\in {\mathbb{R}}^{K\times R},\mathbf{B}\in {\mathbb{R}}^{T\times R},\mathbf{C}\in {\mathbb{R}}^{C\times R}$ are the factor matrices and $R$ is the rank or number of components. Each rank-1 tensor obtained from the ${\lambda }_r\circ {\mathbf{a}}_r\circ {\mathbf{b}}_r\circ {\mathbf{c}}_r$ decomposition, can be interpreted as a distinct spatiotemporal brain pattern, where ${\mathbf{a}}_r,{\mathbf{b}}_r$ and ${\mathbf{c}}_r$ are the subject weights of the time-varying spatial patterns, timecourses, and spatial maps, respectively, as illustrated in Fig. 2b. The CP model optimizes a least-squares fit of the following cost function (Kolda & Bader, 2009):

$$\begin{gathered} f(\mathbf{\Lambda },\mathbf{A},\mathbf{B},\mathbf{C})= \\ \underset{\mathbf{\Lambda },\mathbf{A},\mathbf{B},\mathbf{C}}{\text{min}}\frac{1}{2}{‖𝒳-\mathbf{\Lambda }{\times }_1\mathbf{A}{\times }_2\mathbf{B}{\times }_3\mathbf{C}‖}_F^2, \\ \text{s.t}{‖{\mathbf{a}}_r‖}_2={‖{\mathbf{b}}_r‖}_2={‖{\mathbf{c}}_r‖}_2=1,\forall r=1,\cdots ,R. \end{gathered}$$ (2)

We apply alternating least squares (ALS) to estimate the factor matrices (Cichocki et al., 2016; Kolda & Bader, 2009). The minimization problem is solved by fixing two matrices and optimizing over the third-one. Each least squares subproblem is convex and has a closed-form solution (Kolda & Bader, 2009).

3.3. Component Number Estimation

As in many dimensionality reduction methods, a critical step is the selection of a number of components. We use two methods to make this choice for the CP decomposition: the core consistency diagnostic (CORCONDIA/CCD) (Bro & Kiers, 2003) and the average congruence product (ACP) (Tomasi & Bro, 2005) as a function of tensor rank $R$. The CCD measures the similarity between the estimated core and the superdiagonal ideal core, in the absence of noise (Bro & Kiers, 2003). According to (1), the CP core can be estimated as

$$𝒢=𝒳{\times }_1{\times }_2{\mathbf{A}}^{†}{\times }_3{\mathbf{B}}^{†}{\times }_3{\mathbf{C}}^{†}.$$ (3)

The CCD in (%) is defined as in (Bro & Kiers, 2003)

$$\text{CCD}\left(\%\right)=100\times \left(1-\frac{{‖{𝒢}_R-{\mathbf{ℐ}}_R‖}_F^2}{R}\right),$$ (4)

where $𝒢\times {\mathbb{R}}^{R\times R\cdots \times R}$ and $\mathbf{ℐ}\times {\mathbb{R}}^{R\times R\cdots \times R}$ are the estimated and ideal CP cores, respectively. We choose a model with the highest number of components such that

$${\hat{R}}_{\text{CCD}}=\text{arg}\underset{r}{\text{max}}(\text{CCD})\text{s.t}\text{CCD}(r)\ge \eta ,$$ (5)

where $0<\eta <100\%$ is the threshold coefficient, with $r=1,\cdots R$. Typically, $80\%<\eta <90\%$ is used.

Furthermore, to assess the number of components, we computed the ACP measure of all fitted models for a given tensor rank $R$. The ACP metric measures the correlation between components extracted from different models for a given tensor rank $R$:

$$\begin{gathered} \text{ACP}=\underset{\mathbf{P}}{\text{max}}\text{tr}\left(({\mathbf{A}}_r^{(1{)}^T}{\mathbf{A}}_r^{\left(2\right)})\right. \\ \left.({\mathbf{B}}_r^{(1{)}^T}{\mathbf{B}}_r^{\left(2\right)})({\mathbf{C}}_r^{(1{)}^T}{\mathbf{C}}_r^{\left(2\right)})\mathbf{P}\right), \end{gathered}$$ (6)

where ${\mathbf{A}}_r^{\left(k\right)},{\mathbf{B}}_r^{\left(k\right)},{\mathbf{C}}_r^{\left(k\right)}$ represents the $r$th component of the $i$th solution $i=1,\cdots I,r=1,\cdots R,\mathbf{P}$ is the permutation matrix that accounts for the ambiguity (Harshman et al., 1970) of ordering the solutions, and $\text{tr}(·)$ is the trace of the matrix. We select the model that produces the highest ACP value such that

$${\hat{R}}_{\text{ACP}}=\text{arg}\underset{r}{\text{max}}\left(\text{ACP}\right),r=1,\cdots R.$$ (7)

4. Numerical Experiments

4.1. Data Preprocessing

The MEG multi-subject dataset consists of 170 subjects taken from the Dev-CoG study (Stephen et al., 2021). Before the tensor analysis, we normalized the data by centering the third-order MEG tensor across the time mode, and scaling within the subject mode by its standard deviation (Bro & Smilde, 2003). We used 204 planar gradiometers of the 306 sensors to limit the spatial spread (Stephen et al., 2013) and reduce the noise in the original data. Thus, the data preprocessing resulted in $170C\times T$ ERF subject datasets with $C=204$, and $T=1100$. Furthermore, tensor analyses were performed separately for three stimulus conditions (AUD, VIS, and AV). We selected nine age-adjusted cognitive (T) scores from available neuropsychological measures (see Section 2.3) in the data analyses: the WASI-II FSIQ, Picture Sequence Memory (PSM) (T) score (Weintraub et al., 2013), Picture Vocabulary (PICVOCAB) (T) score (Weintraub et al., 2013), Oral Reading Recognition (ORRENG) (T) score (Weintraub et al., 2013), List Sorting Working Memory (LSWM) (T) score (Weintraub et al., 2013), Flanker Inhibitory Control and Attention (FICA) (T) score (Weintraub et al., 2013), Dimensional Card Sorting (DCCS) (T) score (Weintraub et al., 2013), and the Conners 3 Inattention/Hyperactivity scores. The neuropsychological (T) scores were aggregated to construct a cognitive score matrix $\mathbf{P}\in {\mathbb{R}}^{K\times L}$, where $K$ is the number of subjects and $L$ is the number of cognitive tests. Prior to clustering, the matrix was standardized by the $z$-score to account for scale differences. In addition to the neuropsychological (T) scores, we used parental socioeconomic status (SES), age and gender as model covariates (see the detailed protocol in Section 4.5.1).

4.2. Experimental Design

The goal of this study was to estimate common imaging patterns representing typical brain development in healthy children and adolescents. Three MEG data tensors were constructed (${𝒳}_{\text{VIS}},{𝒳}_{\text{AUD}}$, and ${𝒳}_{\text{AV}}$) for each stimulus condition according to the generative model shown in Fig. 2a. The tensor rank $R$ was estimated for each original data tensor as described in Section 3.3. Three separate CP decompositions were conducted for each stimulus condition with the chosen tensor rank. The fitted CP models resulted in three estimated tensors, consisting of $R$-component factor matrices $\mathbf{A}\in {\mathbb{R}}^{K\times R},\mathbf{B}\in {\mathbb{R}}^{T\times R},\mathbf{C}\in {\mathbb{R}}^{C\times R}$ that described the latent ERF spatiotemporal brain patterns.

To associate brain function with the cognitive performance observed in the neurophysiological tests, we partitioned the subject dataset into two distinct subgroups, high performance (HP) and low performance (LP), using HCPC method (Argüelles et al., 2014; Husson et al., 2010). Thus, we could perform group-level statistical analyses using the extracted ERF components to identify group-level discriminative brain developmental signatures. We identify associations between extracted latent ERF components and cognitive processes by correlating these latent components with children’s scores in the cognitive domains. We hypothesize that statistically significant latent ERF components can differentiate between children’s brain patterns in those with low vs. high performance and could indicate brain developmental trajectory or cognitive development status. Fig. 2 illustrates the application of tensor decomposition to identify brain developmental patterns using MEG data.

4.3. Execution Details

The CP model (2) was fit using CP-ALS (Kolda & Bader, 2009) from the TensorLy toolbox (Kossaifi, Panagakis, Anandkumar, & Pantic, 2019), MNE-Python (Gramfort et al., 2013) was used to generate topographic maps, and R software (Team et al., 2013) version 3.6.0 (R Foundation for Statistical Computing, Vienna, Austria) was used for statistical analyses. All experiments were performed on a Linux workstation with 4 Quad-Core Intel Xeon 3.1 processors and 16 GB memory.

4.4. Model Selection and Evaluation

The model performance was assessed with qualitative and quantitative metrics. The qualitative assessment used interpretations of the extracted components and comparisons with existing literature findings on adolescent cohorts. We computed the reconstruction error of the CP model as RMSE $=\Vert 𝒳-\widehat{𝒳}\Vert /\prod _{n=1}^N{I}_n$, and the model fit $\text{FIT}=\left(1-\Vert 𝒳-\widehat{𝒳}{\Vert }_F^2/\Vert 𝒳{\Vert }_F^2\right)$, where $𝒳$ is stands for the original data tensor, and $\widehat{𝒳}$ denotes the fitted model. The CP-ALS stopping criteria included reaching 1000 iterations or achieving a convergence tolerance of $ϵ\le {10}^{-8}$. We investigated the model order and stability by running the CP-ALS algorithm 100 times for each stimulus condition and $R$ values of one to ten, with each run randomly initialized. This procedure allowed us to determine whether some runs converged to local minima with high reconstruction error. The error plot in Fig. 3c reveals that all runs at fixed $R$ yielded the same RMSE with a standard error of the mean (SEM) < 0.0001. These results suggest that all CP-ALS local minima are similar and presumably also similar to the global minimum.

Fig. 3.

Estimation of the number of components for the CP model, showing the mean values $(N=100)$ of the CCD, ACP, and RMSE metrics as a function of tensor rank $R$ for 100 random initializations for each stimulus condition (VIS, AUD, and AV). (a)-(b) Boxplots summarize the distribution of the mean CCD and mean ACP as a function of tensor rank. Median values are represented by black lines inside the boxplot, with the top of the whisker lines indicating the 25th and 75th percentile values. Mean values are plotted in white circles, and red circles represent outliers. Error bars represent the standard error of the mean. The plot of mean values of RSME suggests that all runs at fixed $R$ yielded the same RMSE with a standard error of the mean < 0.0001. These results suggest that all CP-ALS local minima are similar and presumably also similar to the global minimum.

(a) CCD boxplot. (b) APC boxplot. (c) Mean and standard error of the RMSE and APC as a function of tensor rank $R$.

We assessed the number of components for the CP model (2) by generating average CCD (4) plots and average ACP (6) plots as a function of tensor rank $R$ for $R=1,\cdots ,10$. Figs. 3a–b show boxplots of the mean CCD (4) and mean APC (6) metrics for each stimulus, demonstrating the sensitivity of the solution to the selection of $R$ and initialization parameters of the CP-ALS algorithm.

According to (Bro & Kiers, 2003), the tensor rank of the CP model should be chosen such that the CCD value is greater than 90%. Fig. 3a reveals that $R=2$ should be chosen for the AUD $(M=97.9,SD=2.28)$ and VIS $(M=96.2,SD=2.25)$ conditions, while $R=3$ should be chosen for the AV $(M=93.1,SD=1.37)$ condition.

The ACP values for different $R$ was another method for assessing the number of components. Fig. 3b shows that adding more components resulted in lower mean and higher SEM values for the ACP metric. Similar to the CCD boxplot, the ACP boxplot confirms that $R=2$ is the correct number of components for the AUD $(M=0.988,SD=0.05)$ and VIS $(M=0.998,SD=0.02)$ conditions, while $R=3$ is the best for the AV $(M=0.988,SD=0.05)$ condition. The final solution was selected based on the chosen $R$, which produced the minimum RMSE value, and maximum CCD and ACP values.

4.5. Statistical Analysis

We quantified the CP model performance to produce latent factors for differentiating subject subgroups using mixed measures analysis of covariance (ANCOVA). We performed post hoc analyses with two-tailed parametric $t$-tests and corrections for multiple comparisons using the false discovery rate (FDR) (Benjamini, Drai, Elmer, Kafkafi, & Golani, 2001) with the significance level of $\alpha =0.05$ to determine statistical significance. The ANCOVA and post hoc analyses results were accompanied with $F$-statistics, $t$-statistics, $p$-values and effect size. The effect size was evaluated by generalized eta squared $\left({\eta }_G^2\right)$ value (Olejnik & Algina, 2003) and characterized as small (< 0.06), medium (0.06–0.14), or large (> 0.14), according to (Cohen, 2013). Additionally, we reported the mean $\left(M\right)$, and standard deviation (SD) of the measures of interest.

4.5.1. Group-Level Statistical Inference of CP Component Matrices

The columns of the factor matrices $\mathrm{A}\in {\mathbb{R}}^{K\times R}$ in subject mode contain the component loading factors (coefficients), with the column index corresponding to the loading factors of the given $R$th component. The loading factors of each component indicates how much of the component is required to reconstruct the subject’s source data (Acar, Schenker, Levin-Schwartz, Calhoun, & Adali, 2019). A higher subject loading factor signifies an increased contribution of that component (Stephen et al., 2013). Therefore, group-discriminative components can be obtained by statistically comparing the component loading factors of subgroups to determine significant between-group differences.

Group differences in the component loading factors were assessed with 2 × 2 mixed measure ANCOVAs with the stimuli condition (AUD, VIS, or AV) as a within-subject factor and subgroup (HP vs. LP) as a between-subject factor. The ANCOVAs were calculated for each component and condition while controlling for age, gender and parental SES. In addition to the ANCOVA tests, planned direct comparisons between HP and LP groups were made for each component and condition separately to determine if subgroups differed significantly in the component loading factors of any specific stimulus condition while controlling for the same covariates. We applied a two-tailed level of significance $(p<0.05)$ and an FDR correction for the number of tests performed for each condition.

4.5.2. Correlation Analysis between Component Loading Factors and Neurophysiological (T) Scores

To identify the specific neurophysiological scores associated with the ERF components, we separately correlated the ERF components with the neurophysiological (T) scores in the HP group, LP group and full sample. The relationships were evaluated with Pearson’s correlation tests. Partial correlation analyses (controlling for age, gender and parental SES) were performed between the component loading factors in subject mode (columns of matrix $\mathbf{A}$) and neurophysiological age-adjusted (T) scores. Specifically, we computed the two-tailed Pearson’s partial correlation coefficient $\left(r\right)$ between the component loading factors and the nine cognitive variables, namely, WASI-II FSIQ, PSM, PICVOCAB, ORRENG, LSWM, FICA, DCCS, and the Conners 3 Inattention/Hyperactivity scores. Partial correlations were considered significant below the FDR- corrected threshold $(p<0.05)$.

5. Results

The proposed model was used to extract MEG ERF components using CP decomposition, followed by statistical group-level analyses (see Sections 4.5.1 and 4.5.2). In this section, we describe the results of the multi-subject tensor analyses for extracting typical brain developmental patterns from the original MEG data.

5.1. Multi-Subject MEG Tensor Analysis Using the CP Model

We employed CP decomposition to determine the component structure of MEG ERF responses to multisensory task. The MEG tensor $𝒳\in {\mathbb{R}}^{K\times T\times C}$ was decomposed using the CP factorization model shown in Fig. 2b in the form of Subject × Time × Sensor tensor modes. The data tensor was constructed for each specific stimulus condition (VIS, AUD, and AV), and fitted with the number of components $R$ as determined in Section 4.4. The model reveals the $R$-component factor matrices $\mathbf{A}\in {\mathbb{R}}^{K\times R},\mathbf{B}\in {\mathbb{R}}^{T\times R},\mathbf{C}\in {\mathbb{R}}^{C\times R}$ corresponding to subject, time, and sensor modes. The average model FIT was determined on the basis of 100 runs of the CP model (see experimental setup in Section 4.4) for each stimulus condition and the chosen tensor rank $(\text{VIS}:R=2,M=0.93,SD=0.01;\text{AUD}:R=\mathrm{2,0.91},SD=0.01;\text{AV}:R=3,M=0.91,SD=0.01)$ indicates that the extracted factors account for a large part of the explained variance of the original datasets. Supplementary Fig. S. 4 and Supplementary Table S. 1 show the mean values of FIT metric of the fitted CP decomposition for each stimulus condition (VIS, AUD, and AV). To quantify the common associations between original MEG ERF subject’s datasets and the extracted ERF components, we performed repeated measures correlation analyses (Bakdash & Marusich, 2017) between these paired datasets (see Supplementary Methods Section 3.2).

The tensor analysis yielded seven ERF components (see Table 1), which describe patterns of temporal variance (temporal factors), spatial variance (spatial factors), and the subject factors. The spatial loadings provide the measure of activity in the MEG ERF as a function of time for each spatial factor. The subject loadings modulate the magnitude of these spatiotemporal patterns, representing the pattern’s activation strength for the specific subject. We categorized the components as functional MEG ERF components that correspond to prominent spatiotemporal peaks (Stephen et al., 2013), and the spatiotemporal variance explained $\left(R^2\right)$, which was determined by the repeated measures correlation analyses. Fig. 4 depicts the extracted temporal and spatial components time-locked to the target stimuli after CP tensor decomposition on the sensor-level MEG data. The temporal ERF components generated from individual sensor data averaged across subject ERF components are shown. The MEG topographic maps show the density of spatial patterns that correspond to prominent time peaks. The average ERF component (average across sensors, in cyan) and average ERF timecourse (in yellow) for each stimulus condition are plotted. The ERF components are well-matched to distinct peaks present on the average ERF timecourses. The temporal evolution of the MEG ERF topographic maps is shown in Supplementary Fig. S5.

Table 1.

Summary of MEG ERF components

	ERF Component
Stimuli condition	Occipital/M150	Right Temporal/M300a	Late Central/M400
VIS	✓	-	✓
AUD	-	✓	✓
AV	✓	✓	✓

Fig. 4.

Tensor decomposition results of sensor-level MEG data for target stimuli. Temporal and spatial patterns from the components of the CP model. Top: The topographic maps show the density of spatial patterns that correspond to prominent time peaks denoted with red and blue arrows. Bottom: ERF component with signal traces from all individual MEG sensors averaged across subject ERF components. The shaded areas around each line depict the standard error of the mean. The average stimulus-related ERF timecourse is shown in yellow, and the average ERF (average across sensors) component is plotted in cyan. (a)-(b) Occipital 130–150 ms component in the VIS and AV conditions. (c) Right temporal 280–300 ms component in the AUD and AUD conditions. (e)-(i) Late central 350–430 ms component in the VIS, AUD, and AV conditions.

The repeated measures correlation analyses (see Supplementary Methods Section 3.2) found significant correlations $(p<0.001)$ between ERF components and the original data (VIS, AUD, and AV conditions) for the overall common slope (Supplementary Table S.3). The ERF components (Supplementary Table S.3) accounted for 72%, 76%, and 74% of the spatiotemporal variance $\left(R^2\right)$ in the VIS, AUD and AV conditions, respectively.

5.1.1. Occipital Component/M150

The occipital component was found in the VIS and AV conditions, as shown in Figs. 4a–b. This component was associated with the first prominent visual peak at a latency of 130–150 ms. The spatial distribution map at 145–149 ms (see Figs. 4a–b) shows that the positive deflection reflects MEG activity in the bilateral and medial occipital scalp areas. The positive deflection resembles the visual P100 wave described in previous MEG/EEG studies, reflecting the allocation of attentional resources (Boehler, Schoenfeld, Heinze, & Hopf, 2008; Zhang & Luck, 2009).

5.1.2. Right Temporal Component/M300a

The right temporal component with the peak 280–300 ms was consistently found in the AUD and AV conditions in the right temporal and inferior left/right frontal regions (see Figs. 4c–d). The positive deflection at a latency of 280–300 ms corresponds to the early phase of the P300a component, which has been linked to different processes, such as detecting and evaluating novel and orienting responses (Pfefferbaum, Ford, Weller, & Kopell, 1985; Polich, 2007; Vogel, Luck, & Shapiro, 1998).

5.1.3. Late Central Component/M400

The late central component was extracted for all stimuli conditions (AUD, VIS, and AV), as shown in Figs. 4e–i. This component consists of a sequence of positive (VIS and AV) and negative peaks (AUD) at approximately 126–134 ms and a prominent peak at 350–430 ms. Figs. 4e–i show that this component is primarily distributed in the left temporal-parietal and right prefrontal regions. The prominent negative deflection resembles the late phase of the parietally distributed P400 component, which suggests association with semantic encoding (Halgren et al., 2002; Marinković, 2004).

5.2. Clustering Analysis for Subject Subgroup Identification

In this section, we present the clustering analysis results for identifying subgroups. A preliminary version of the clustering approach presented here using cognitive measures collected during the Dev-Cog study presented as a conference contribution (Boonyakitanont et al., 2022). The detailed clustering protocol is described in the Supplementary Methods Section 3.1. We partitioned the subject cohort $(N=170)$ into distinct subgroups using a neurophysiological dataset. We performed HCPC clustering using nine cognitive variables from six cognitive domains, including the Connors 3 inattention/hyperactivity scores: WASI-II FSIQ, ORRENG, PICVOCAB, PSM, LSWM, DCSS, FICA, INATTENTION and HYPERACTIVITY. The HCPC method (Argüelles et al., 2014; Husson et al., 2010) combines three standard techniques (principal component analysis (PCA), hierarchical clustering, and $K$-means algorithm) to obtain a higher quality clustering solution. A schematic view of the subgroup identification using the HCPC algorithm is presented in Fig. 5.

Fig. 5.

Hierarchical clustering on principal components of neuropsychological (T) scores for subject group identification using Ward’s D2 criterion (Murtagh & Legendre, 2014). PCA is used on the subject cognitive matrix $\mathbf{P}\in {\mathbb{R}}^{K\times L}$ to remove highly correlated continuous variables. Next, we apply hierarchical clustering using Ward’s D2 method on the distance matrix $\mathbf{D}$ to select the clusters based on the height of the hierarchical tree. The distance matrix $\mathbf{D}\in {\mathbb{R}}^{K\times K}$ is computed using the dissimilarity measure such as the distance correlation (Székely, Rizzo, & Bakirov, 2007) of the PCs. The initial number of clusters $N_{C_k}$ is assessed according to the compactness metrics (Halkidi, Batistakis, & Vazirgiannis, 2002a, 2002b), and the cluster stability is evaluated using the Jaccard similarity index (Jaccard, 1912) via a nonparametric bootstrap technique with a number of repetitions $n=1000$ (see detailed protocol in Supplementary Methods Section 3.1). We select significant clusters based on the approximately unbiased probability $p$-values (Efron, Halloran, & Holmes, 1996), as shown in Fig. 6a. We provide the final clustering solution by applying the $K$-means algorithm to the hierarchical clustering output.

First, we applied the PCA algorithm to the neurophysiological dataset, represented as a subject score matrix $\mathbf{P}\in {\mathbb{R}}^{K\times L},K=170,L=9$, to reduce the dataset into fewer dimensions called principal components (PCs), which are uncorrelated with each other. We computed a distance matrix $\mathbf{D}\in {\mathbb{R}}^{K\times K}$ of these PCs, which uses the dissimilarity measure such as distance correlation (Székely et al., 2007). The distance correlation measure allows to detect nonlinear correlations (Székely et al., 2007) that might not be identified by the Pearson correlation (Székely et al., 2007), which may result in suboptimal performance of the downstream tasks. Next, we applied hierarchical clustering using Ward’s D2 (Murtagh & Legendre, 2014) method on the distance matrix D to select the clusters based on the height of the hierarchical tree. The significant clusters are selected on the basis of the approximately unbiased (AU) probability (Efron et al., 1996) $p$-values with $p<0.05$. The quality of clustering is assessed according to the compactness metrics (Halkidi et al., 2002a, 2002b) (see Supplementary Methods Section 3.1 and Supplementary Fig. S.3). The cluster stability is evaluated as a function of the number of clusters using the Jaccard similarity index $\left(J\right)$ (Jaccard, 1912) via a nonparametric bootstrap technique with a number of repetitions $n=1000$ (Supplementary Methods Section 3.1). We obtained the final clustering solution by applying the $K$-means algorithm to the hierarchical clustering output.

The hierarchical clustering results using Ward’s D2 (Murtagh & Legendre, 2014) distance are shown in the dendrogram in Fig. 6a, with the height of the branches indicating the distance or dissimilarity between clusters. As depicted in Fig. 6a, two significant clusters were selected according to the approximately unbiased probability (AU) (Efron et al., 1996) $p$-values with $p<0.05$. The clustering solution projected on the PCs is shown in Fig. 6b. We used the HCPC clustering output to identify two subgroups with distinct distributions of mean PCA scores and categorized them as high $(n=89)$ or low $(n=81)$ performance. The distribution of mean PCA scores shown in Fig. 6c indicates that subjects in the HP group have higher PCA loading factors than the subjects in the LP group in all six cognitive domains except for the Conners 3 inattention and hyperactivity scores where a higher score implies greater inattention and hyperactivity.

Fig. 6.

Results of the hierarchical clustering on the principal components. (a) Dendrogram of hierarchical clustering based on Ward’s D2 criterion. The height of the branches indicates the dissimilarity between clusters. The number of retained clusters was chosen using an approximately unbiased probability measure (AU) (Efron et al., 1996). The significant clusters were selected based on unbiased probability $p$-values with $p<0.05$ corrected for multiple comparisons using FDR. The final clustering solution was obtained with the $K$-means algorithm. (b) Clustering solution projected on the principal components. (c)-(d) Subgroup associations with neuropsychological (T) scores. (c) Distribution of mean PCA scores averaged across subject subgroups. (d) Main effect of subject subgroup on neuropsychological (T) score. Independent samples twotailed $t$-test (FDR corrected, $p<0.05$) showed statistically significant differences in the WASI-II FSIQ, language, memory $(p<0.0001)$ and inattention scores $(p<0.01)$ between the LP and HP groups. Details can be found in Supplementary Table S.1. $****p<0.0001,**p<0.01$.

We evaluated the effect of the subject group (HP vs. LP) on the cognitive assessments using independent samples two-tailed $t$-tests with unequal variances corrected for multiple comparisons using FDR $(p<0.05)$. We present the summary statistics of the neurophysiological (T) score distribution according to subject subgroup in Supplementary Table S.1. The groups did not differ significantly in terms of gender $\left({\tilde{\chi }}^2=0.00,p=0.99)\right.$, age $(t_{167}=0.03,p=0.97)$ or parental SES $(t_{167}=0.517,p=0.61)$. However, the WASI-II FSIQs differed significantly $\left(t_{167}=9.16,p<0.0001)\right.$, with higher FSIQ scores in the HP group than in the LP group. Similarly, the language (PICVOCAB, ORRENG), memory (PSM, LSWM), and executive function (DCCS, FICA) (T) scores differed significantly by group (Supplementary Table S.1; Fig. 6d; $p<0.0001$), with cognitive (T) scores higher in the HP group than in the LP group. The Conners 3 hyperactivity score differed significantly by group $(t_{167}=-2.17,p=0.031)$, with lower scores in the HP group than in the LP group. The Conners 3 inattention score did not differ significantly between the HP and LP groups $(t_{167}=0.52,p=0.61)$. Fig. 6d shows the subject subgroup distribution of standard age-adjusted cognitive (T) scores.

5.3. Statistical Group-Level Analysis

In this section, we present the group-level analysis results described in Sections 4.5.1 and 4.5.2. This section has two subsections. The first subsection evaluates the statistical significance of the ERF components (see Section 5.1) to differentiate between the subgroups identified in Section 5.2. The second subsection assesses the covariant relationships between ERF components and neurophysiological measures to correlate brain responses with cognitive performance. The component loading factors in subject mode (columns of matrix A) and neurophysiological (T) scores were evaluated for normality. All analyses were corrected for multiple comparisons using FDR with a significance level of $\alpha =0.05$ unless stated otherwise.

5.3.1. Group-Level Discriminative Components

We applied mixed measures two-way ANCOVA (see Section 4.5.1) on the component loading factors in subject mode of each ERF component and stimulus condition to determine significant effects after controlling for the covariates. The mixed measures two-way ANCOVA comparison of the component loading factors showed a statistically significant stimulus condition × group interaction (see Table 2) for the Occipital/M150 $\left(F_{\mathrm{1,336}}=28.73,p<0.0001,{\eta }_G^2=0.101\right)$ and R.Temporal/M300a components $(F_{\mathrm{1,336}}=\left.6.82,p=0.03,{\eta }_G^2=0.098\right)$. There was no significant stimulus condition × group interaction for the L.Central/M400 component $(F_{\mathrm{2,494}}=\left.0.79,p=0.982,{\eta }_G^2=0.004\right)$. The main effect of the subject subgroup was statistically significant for each component (Table 2; Occipital/M150: $\left(F_{\mathrm{1,336}}=33.96,p<0.0001,{\eta }_G^2=0.113\right.$; R.Temporal/M300a: $\left(F_{\mathrm{1,336}}=101.35,p<0.0001,{\eta }_G^2=0.281;\right.$ L.Central/M400: $F_{\mathrm{1,494}}=176.73,p<0.0001,{\eta }_G^2=0.311$). Post hoc analyses with two-tailed $t$-tests corrected for multiple comparisons using FDR $(p<0.05)$ revealed six components with significant group differences (HP vs. LP) in the component loading factors. The details are shown in Table 3 and Fig. 7. Fig. 8 depicts the group ERF components as solid lines (blue for HP and red for LP). The group ERF components peaked at the same time as the average group ERF timecourses, drawn in dashed lines (blue for HP and red for LP).

Table 2.

Summary of ERF components loading factor ANCOVA results

Effect	Occipital/M150		R.Temporal/M300a		L.Central/M400
	$\mathit{F}\left(\mathrm{1,336}\right)$	$\mathit{p}$ value	$\mathit{F}\left(\mathrm{1,336}\right)$	$\mathit{p}$ value	$\mathit{F}\left(\mathrm{2,494}\right)$	$\mathit{p}$ value
Condition	0.05	0.996a	0.01	0.999b	0.22	0.969a
Group	33.96	< 0.0001a	101.35	< 0.0001a	176.73	< 0.0001a
Condition × Group	28.73	< 0.0001a	6.82	0.031 a	0.79	0.982a

^a$p$ values from ANCOVAs adjusted for age, gender and parental SES (FDR corrected, $p<0.05$).

Table 3.

Comparison of ERF Component loading factors by subject subgroup

Component	AUD				VIS				AV
	HP ($n=89$)	LP ($n=81$)	$t\left(165\right)$	$p$ value	HP ($n=89$)	LP ($n=81$)	$t\left(165\right)$	$p$ value	HP ($n=89$)	LP ($n=81$)	$t\left(165\right)$	$p$ value
	$M\left(SD\right)$				$M\left(SD\right)$				$M\left(SD\right)$
Occipital/M150	-	-	-	< 0.0001a	0.61(0.91)	−0.63(0.62)	7.86	< 0.0001a	0.015(1.07)	0.013(0.93)	0.166	0.864a
Right Temporal/M300a	−0.62(0.83)	0.56(0.79)	7.31	< 0.0001a	-	-	-	< 0.0001a	0.54(0.85)	−0.31(1.05)	5.65	< 0.001a
Late Central/M400	−0.57(0.81)	0.64(0.78)	−8.5	< 0.0001a	−0.51(0.75)	0.52(0.96)	−7.22	< 0.0001a	−0.56(0.83)	0.51(0.85)	−7.20	< 0.0001a

^a$p$ values (FDR corrected, $p<0.05$) from post hoc two-tailed $t$-tests adjusted for age, gender, and parental SES.

^bthe corresponding data are shown in Fig. 7.

Fig. 7.

Main effect of the subject group ($N=170$, HP vs. LP) on component loading factors in subject mode for each stimulus condition. Boxplots summarize the distribution of the mean values of the component loading factors in subject mode for the HP and LP groups. Median values are represented by black lines inside the boxplot, with the top of the whisker lines indicating the 25th and 75th percentile values. Mean values are plotted in white circles. Error bars represent the standard error of the mean. Six components were statistically significant in the post hoc two-tailed $t$-test results (FDR corrected, $p<0.05$), except for the occipital component/M150 in the AV condition, which did not differ between the HP and LP groups $(p=0.864)$. The post hoc $t$-tests results are shown in Table 3. $****p<0.0001,***p<0.001$ (post hoc and FDR corrected $p<0.05$), which means significantly different.

Fig. 8.

Discriminative group MEG ERF components from the CP decomposition for the VIS, AUD, and AV conditions are shown. The ERF components are indicated by solid lines (HP: high performance - blue, LP: low performance - red). The average group ERF timecourses are indicated by dashed lines. Main effect of the subject group (HP vs. LP, $N=170$) on the component loading factors in subject mode is summarized in the boxplots. The boxplots summarize the distribution of the mean values of component loading factors in the subject mode for the HP and LP groups. The median values are represented by black lines inside the boxplot, and the tops of the whisker lines indicate the 25th and 75th percentile values. The mean values are plotted in dark grey. The error bars represent the standard error of the mean. Post hoc analyses with two-tailed $t$-tests (FDR corrected, $p<0.05$) indicate that the mean value of the component loading factors of the HP group was significantly different than for LP group with $p<0.001$ for six ERF components except for the occipital/M150 component in the AV condition (AV/M150: $p=0.864$). (a)-(b) Occipital M150 component. (c)-(d) Right temporal M300a component. (e)-(f) Left central M400 component. $****p<0.0001,***p<0.001$ indicate significant differences (FDR corrected, $p<0.05$). The post hoc $t$-tests results are shown in Table 3.

Occipital Component/M150:

The Occipital/M150 (130–150 ms) group component is shown in Fig. 8a–b. The activity was concentrated in the left and right occipital and medial visual cortex. The HP group demonstrated a higher activation strength than LP group in the VIS condition (see Fig. 8a). The main effect of the group $(N=170,$ HP vs. LP) on the component loading factors was statistically significant for the VIS condition (Table 3, Fig. 7, Fig. 8a; VIS: $t_{165}=7.86,p<0.0001,\text{HP}>\text{LP}$, post hoc twotailed $t$-test). However, there were no significant differences in the component loading factors of the Occipital/M150 component between the HP and LP groups for the AV condition (Table 3, Fig. 7, Fig. 8b; AV: $t_{165}=0.166,p=0.864$, post hoc twotailed $t$-test).

Right Temporal Component/M300a:

The R.Temporal/M300a group component is shown in Figs. 8c–d. The component was associated with the peak at 280–300 ms and accounted for the activity in the right temporal and inferior left/right frontal regions. Post hoc two-tailed $t$-tests found a significant main effect of group $(N=170$, HP vs. LP) on the component loading factors for both the AUD and AV conditions (Table 3, Fig. 7, Figs. 8c–d; $\text{AUD}:t_{165}=7.31,p<0.0001,\left.\text{HP}>\text{LP};\text{AV}:t_{165}=5.65,p<0.001,\text{HP}>\text{LP}\right)$.

Late Central Component/M400:

The L.Central/M400 group component is shown in Figs. 8e–i. This component was associated with activity in the left temporal-parietal and right prefrontal cortex. Two-tailed $t$-tests identified a significant main effect of group ($N=170$, HP vs. LP) on the component loading factors for the VIS, AUD and AV conditions (Table 3, Fig. 7, Figs. 8e–i; $\text{AUD}:t_{165}=-8.5,p<0.0001,\text{HP}<\text{LP};\text{VIS}:t_{165}=-7.22,p<0.0001,\text{HP}<\text{LP};\text{AV}:\left.t_{165}=-7.20,p<0.0001,\text{HP}<\text{LP}\right)$.

5.4. Analysis of ERF Component Association with Cognitive Domains

To correlate neuropsychological scores with ERF components, we performed two-tailed partial Pearson’s correlation tests between component loading factors in subject mode (columns of matrix $\mathbf{A}$) for the nine cognitive age-adjusted neuropsychological (T) scores in each subject group $(\text{HP}:n=89;\text{LP}:n=81)$ and the full sample $(N=170)$ (see detailed protocol in Section 4.5.2). The partial correlation analyses were controlled for age, gender and parental SES. The correlation analyses indicated that three functional ERF components were significantly associated with the language, episodic memory and attention cognitive domains (see Table. 4). The results of two-tailed partial Pearson’s correlation analyses are summarized in Table 4, Figs. 9, 10 and 11.

Table 4.

Component loading factor and cognitive (T) score associations

	Cognitive (T) score
	PSM
Component	Condition	HP ($\mathit{n}=89$)			LP ($\mathit{n}=81$)
		$\mathit{r}$	$\mathit{t}\left(86\right)$	$\mathit{p}$ value	$\mathit{r}$	$\mathit{t}\left(78\right)$	$\mathit{p}$ value
Right Temporal	AUD	0.342	3.33	0.003 a	0.291	2.65	0.039 a
	AV	0.364	3.59	0.001 a	0.297	2.71	0.016 a
	PICVOCAB
Late Central	VIS	−0.242	−2.29	0.013 a	−0.12	−1.051	0.295a
	DCCS
Right Temporal	AUD	0.267	2.42	0.018 a	−0.017	−0.161	0.873a
	AV	0.277	2.51	0.014 a	−0.071	−0.564	0.561a
	HYPERACTIVITY
Occipital	VIS	−0.233	−2.21	0.030 a	−0.086	−0.685	0.992a

^a$p$ values (FDR corrected, $p<0.05$) from two-tailed partial Pearson’s correlation tests adjusted for age, gender and parental SES.

Fig. 9.

A schematic view of the ERF components associated with brain areas mapped to cognitive processes. (a) ERF component association patterns with cognitive domains. The circular form (Krzywinski et al., 2009) describes the functional associations between the ERF component and the cognitive domains. The arc widths denotes the association strength, expressed as the partial Pearson’s correlation coefficient $\left(r\right)$. (b) Brain areas associated with ERF components. (red: Occipital/M150 component is associated with the visual cortex, blue/yellow: R.Temporal/M300a component is associated with the inferior frontal and temporal cortex, green: L.Central/M400 component is associated with the left parietal lobe).

Fig. 10.

Significant (FDR corrected, $p<0.05$) twotailed partial Pearson’s correlations (correlation coefficient, $r$) between ERF components and neuropsychological (T) scores in the full sample. The linear fit and 95% confidence intervals (CIs) are shown. The blue dots denote HP group, and red dots denote LP group. (a) The occipital component was negatively correlated with the hyperactivity score in the VIS condition. (b) The late central component was negatively correlated with the PICVOCAB score in the VIS condition. (c)-(d) The right temporal component was positively correlated with the PSM (T) score in the AUD and AV conditions.

Fig. 11.

Significant (FDR corrected, $p<0.05$) two-tailed partial Pearson’s correlations (correlation coefficient, $r$) between ERF components and neuropsychological (T) scores in the HP and LP groups (HP: blue dots, LP: red dots). The group linear fit and 95% CIs are shown (HP: blue line, LP: red line). (a) The late central (VIS) component was negatively correlated with the PICVOCAB score and significant in the HP group. (b) The occipital (VIS) component was negatively correlated with the hyperactivity score and significant in the HP group. (c)-(d) The right temporal component (AUD/AV) was positively correlated with the PSM score and significant in both groups. (e)-(f) The right temporal component (AUD/AV) was positively correlated with the DCCS score, significant in the HP group and showed a negative trend in the LP group.

Among the nine cognitive variables, the PICVOCAB (T), PSM (T), DCCS (T), and Connors 3 hyperactivity scores were significantly correlated with ERF components. We present the the summary of functional association patterns between the ERF components and the cognitive domains in Fig. 9. In Fig. 9a, the arc widths reflect the association strength between ERF components and their cognitive domain-wise connections.

The Occipital/M150 component in the VIS condition (Figs. 4a and 8a) was negatively correlated with the hyperactivity score in the HP group $\left(r\right(86)=-0.233,p=0.03;$ Table 4; Fig. 11b). However, the correlation was not significant ($r_{\text{VIS-HYPERACTIVITY}}\left(167\right)=-0.194,p=0.051;$ Fig. 10a) in the full sample. The correlation was consistent with large group differences (HP > LP; $p<0.0001$) in the occipital component loading factors identified by two-tailed $t$-tests (see Table 3).

The R.Temporal/M300a component (Figs. 4c–d and 8c–d) had a statistically significant positive correlation with the PSM (T) score in both groups (Table 3; Figs. 11c–d; $\text{HP}:r_{\text{AUD}-\text{PSM}}\left(86\right)=0.342,p=0.003,r_{\text{AV}-\text{PSM}}\left(86\right)=0.364,p=0.001;\text{LP}:r_{\text{AUD}-\text{PSM}}\left(78\right)=0.291,p=0.039,r_{\text{AV}-\text{PSM}}\left(78\right)=0.297,p=0.016)$ and the full sample ${(r}_{\text{AUD-PSM}}\left(167\right)=0.239,p=0.004;r_{\text{AV-PSM}}\left(167\right)=0.303,p=0.001;$ Figs. 10c–d).

Additionally, the correlation tests revealed significant associations between the R.Temporal/M300a component and DCCS (T) score (see Table 3 and Figs. 11e–f) in the HP group (AUD: $r_{\text{AUD-DCCS}}\left(86\right)=0.267,p=0.018;\text{AV}:r_{\text{AV}-\text{DCCS}}\left(86\right)=0.277,p=0.014.$ The correlations between the R.Temporal/M300a component and PSM/DCCS scores were consistent with group differences (HP > LP; $p<0.001$) in the component loading factors for the AUD and AV conditions (see Table 3).

The L.Central/M400 component in the VIS condition (Figs. 4e and 8e) was negatively correlated with the PICVOCAB (T) score in the HP group ${(r}_{\text{VIS-PICVOCAB}}\left(86\right)=-0.242,p=0.013;$ Table 4; Fig. 11a) and the full sample $\left(r\right(167)=-0.208,p=0.017;$ Fig. 10b).

No significant associations were found in the HP group, LP group or full sample for the other cognitive scores in the component loading factors ($p>0.05$ all tests). There were no significant differences in partial correlations between the groups (HP vs. LP) or stimulus conditions (VIS vs. AUD vs. AV) within subject groups ($p>0.05$ all tests).

6. Discussion

This paper presents a tensor analysis-based model of MEG multi-subject data for identifying ERF components representative of typical brain developmental patterns in a healthy population of children and adolescents. The tensor analyses and tensor-based group-level statistical inferences outlined in this paper establish a foundational framework for extracting latent factors associated with children’s brain development from MEG datasets.

We contribute to the developmental neuroscience literature on the relationship between MEG activity and cognition by correlating ERF components from a healthy pediatric population with neuropsychological age-adjusted cognitive (T) scores and attentional indices.

6.1. ERF Components Extraction

To the best of our knowledge, this is the first study that modeled event-related field MEG data as a low-rank third-order tensor. We demonstrated that CP factorization could produce latent factors that result in functionally relevant ERF components and reveal meaningful spatiotemporal brain patterns. The CP model was shown to be highly effective in capturing informative data representation. For example, Fig. 4 and Supplementary Table. S. 2 illustrate that ERF components were well-matched with the average ERF waveforms and demonstrated significant correlations $(p<0.001)$ with the original datasets (see Supplementary Methods Section 3.2; Supplementary Table. S.3). The tensor analysis successfully identified latent brain developmental patterns across subjects for each stimulus condition. As expected, the Occipital/M150 component similar to the visual P100/M100 wave, was extracted from the VIS and AV conditions, as shown in Figs. 4a–b. We observed P300 component in the R.Temporal/M300a component (see Fig. 4c–d) extracted from the AUD and AV conditions. The absence of the visual Occipital/M150 component in the AUD condition and the absence of the R.Temporal/M300a component in the VIS condition confirms that the CP model can extract meaningful patterns corresponding to expected ERF responses. The L.Central/M400 component (Figs. 4e–i) was extracted from the AUD, VIS, and AV conditions, representing activity in the left temporal-parietal cortex and likely capturing the motor response required in all three conditions.

6.2. Functional Associations of Group-Discriminating Components

The CP decomposition of the multi-subject MEG data provides insight into how tensor analysis can be used to explore relationships between brain patterns and cognitive function in high- and low-performance subjects. The statistical tests confirmed the effect of the cognitive group on the relationship between the component loading factors and designated pediatric subgroups (see Section 5.3). We showed that children in the HP group were significantly different from those in the LP group (HP vs. LP; Table 3) in terms of the six components, with large effect sizes (${\eta }_G^2>0.113$, see Section 5.3.1). The group differences (HP > LP) were consistent with the neuropsychological (T) score $t$-tests, with the HP group scoring significantly higher than the LP group on all cognitive and behavioral tests, except for the Connors 3 inattention and hyperactivity scores. The subjects in the HP and LP groups did not show differences in the spatial distributions of all components; however, they demonstrated significant differences in the spatial activation strength and timecourse amplitude.

To identify ERF components as informative signatures of cognitive function, we correlated the ERF components with neuropsychological (T) scores in the full sample and each subject group. In the full sample, we found statistically significant correlations between neuropsychological (T) scores and specific ERF components (see details in Section 5.4), namely, the PSM (T), PICVOCAB (T) and Connors 3 hyperactivity scores.

The correlation analyses between the ERF components and neuropsychological scores revealed significant associations between ERF components and PSM (T) score in both the HP and LP pediatric groups, whereas the PICVOCAB (T), DCCS (T) and hyperactivity scores were significantly correlated with ERF components only in the HP group.

To associate ERF components with cognitive processes, we discuss how these components can be related to scores in the cognitive domains. The PSM test measures episodic memory (EM) (Dikmen et al., 2014), and the DCCS test measures cognitive flexibility and executive function (EF) (Weintraub et al., 2013). The PICVOCAB test measures verbal ability and language comprehension (LC) (Weintraub et al., 2013), and the Connors 3 hyperactivity score is an attentional (ATTN) index (Conners, 2008). The EM, EF, LC and ATTN metrics are foundational cognitive processes that change rapidly during development and vary across individuals.

Next, we characterize the relationships between EM, EF, ATTN, and LC cognitive functions and discriminative ERF components.

6.2.1. Early Latency Components

The hyperactivity score was negatively correlated with the Occipital/M150 component in the VIS condition using the full sample data. In addition, the analyses revealed a significant negative correlation between the hyperactivity score and the Occipital/M150 component in the HP group. Finally, as shown in Supplementary Table S.1, the hyperactivity score was significantly lower in the HP group than in the LP group. These findings suggest that the Occipital/M150 component patterns were consistent between the correlation results and ERF component group-level analyses (HP > LP; Table 3), as well as with the group-level neuropsychological (T) score analyses (Supplementary Table S.1). One interpretation of the Occipital/M150 pattern suggests that the cognitive enhancement in the HP group is closely related to the ability to concentrate on the task. In other words, the increased amplitude of the Occipital/M150 component in the HP group reflects the attentional processes designated to that task. The children in the HP group demonstrated effortless processing and sustained attention to the presented stimuli. In contrast, the children in the LP group experienced an increase in cognitive workload due to a lack of attention and therefore followed an effortful processing model. This result is consistent with the literature (Ghani, Signal, Niazi, & Taylor, 2020; Sokhadze et al., 2017), which states that early ERP components (N100, N200, and P200) are related to involuntary attention selection mechanisms. Our findings are consistent with the literature on healthy individuals (Allison & Polich, 2008; Ghani, Signal, Niazi, & Taylor, 2021; Kramer, Trejo, & Humphrey, 1995) and children with ADHD (Liotti, Pliszka, Higgins, Perez III, & Semrud-Clikeman, 2010; van Meel, Heslenfeld, Oosterlaan, & Sergeant, 2007), which reported lower early latency component amplitudes with increasing cognitive workload in patients relative to healthy controls.

6.2.2. Late Latency Components

The right temporal/M300a component in the AUD and AV conditions was positively associated with the PSM (T) score $(p<0.01)$ in the HP group, LP group, and the full sample. The partial correlation analyses for the PSM (T) score shown in Figs. 10c–d and 11c–d suggest that the increase in value of the component loading factors was associated with a better PSM (T) score, indicating an EM effect. Additionally, the right temporal/M300a component was significantly positively correlated with the DCCS (T) score in the HP group, which is consistent with the component loading factor group difference (HP > LP; Table 3). Similarly, the correlation analyses for the DCCS (T) score shown in Fig. 11e–f and Table 4 suggest that the increase in value of the component loading factors resulted in the improvement of the DCCS (T) score, indicating an EF effect. The HP group scored significantly higher (Supplementary Table S.1; $p<0.0001$) than the LP group in terms of both the PSM (T) and DCCS (T) scores. These results suggest that children in the HP group have higher implicit memory capacities and better EF function; thus, they could perform the task effortlessly, resulting in a lower cognitive workload and higher component loading factors. According to the literature, the P300a/M300a component amplitude represents an orienting response, reflecting involuntary orientations to attention-catching changes (Sur & Sinha, 2009). Additionally, the P300a/M300a component was categorized as an indicator of implicit memory and item familiarity (Friedman & Johnson Jr, 2000; Graf & Schacter, 1985; Rugg et al., 1998). Our findings aligned with the existing literature (Berti & Schröger, 2003; Dyke et al., 2015; Horat et al., 2016), with the P300a/M300a component indicating sustained attention and decreases in P300a amplitude with increasing cognitive workload.

The late central/M400 component showed a significant negative correlation with the PICVOCAB (T) score in the HP group and full sample. Similar to the N400 (Kutas & Federmeier, 2000) ERP component, the spatial scalp distribution was maximally concentrated in the left temporal-parietal regions. According to the literature, the N400/M400 component represents brain responses to words and other meaningful visual stimuli (Kutas & Federmeier, 2000, 2011). This component indicates semantic processing and has a lower amplitude for more frequent items (Kutas & Federmeier, 2000).

The association of the late central/M400 component with the PICVOCAB (T) score (see Figs. 10b and 11a) demonstrated that the reduced M400 component amplitude results in a higher PICVOCAB (T) score, indicating a language comprehension effect. Similar to computerized ERP language assessment studies (Byrne, Dywan, & Connolly, 1995; Pratarelli, 1994) in adults and children, our findings indicate that children in the HP group had lower M400 amplitude than children in the LP group, showing an incongruent processing pattern that caused larger negative outgoing amplitudes. This pattern suggests that N400 component ERPs reflect prediction error signals needed for learning; thus, larger ERP amplitudes are correlated with mismatches (Fitz & Chang, 2019).

7. Limitations and Future Work

The proposed generative model using CP decomposition implies that all subjects have the same number of latent components $R$, and all subjects share the common matrices $\mathbf{A}$ and $\mathbf{B}$. In other words, the CP model imposes strict assumptions such that the underlying brain patterns have identical timecourses and spatial maps across subjects. However, with real ERF MEG data, individual differences may exist in the timing and origin of the subject’s neural responses to the stimuli. For example, individual differences in the cognitive processing of the stimuli would result in differences in the timing and spatial distribution in MEG. To allow a variable number of components and spatiotemporal variability of brain patterns across subjects, a more flexible model can be used, such as constrained PARAFAC2 (Parallel Factor Analysis) (Afshar et al., 2018; Chatzichristos, Kofidis, Morante, & Theodoridis, 2019; Helwig & Snodgress, 2019), or higher-order block term decomposition (BTD2) (Chatzichristos et al., 2019). It has been shown in (Harshman et al., 1972; Helwig & Snodgress, 2019) that PARAFAC2 can handle the heterogeneity of subject’s responses and allows a variable number of latent components per subject (Afshar et al., 2018) via sparsity constraints. To address this limitation, our future work may include using the PARAFAC2 or BTD2 models to account for subject’s individual differences.

Another limitation of our generative model is fitting each stimulus condition as a separate CP decomposition. Alternatively, the multi-task multi-subject MEG data could be modeled as a coupled tensor-tensor decomposition (CTTD) (Chatzichristos et al., 2022; Jonmohamadi et al., 2020), where each stimulus (VIS, AV, and AV) represented as a third-order tensor and coupled in the subject mode. The multi-task joint learning enables the use of the complementary information (Acar et al., 2013; Lahat et al., 2015) from multiple stimuli and thus could result in the latent components with a higher discriminative power.

The extracted ERF components could be used as bioimaging markers for classification or prediction. Specifically, the subject loading factors found from MEG data using the CP model can be interpreted as feature extraction. The combination of machine learning techniques and multi-task tensor decomposition of MEG data could identify more reliable bioimaging markers that may enable the exploration of neurological differences associated with symptom onset, enabling early intervention. Thus, the application of multi-subject MEG tensor decomposition in context of machine learning is a promising direction for future research in cognitive neuroscience.

8. Conclusion

We demonstrated that CP decomposition can be used for effective identification and characterization of latent spatiotemporal components of multi-subject MEG data. We described the generative model for the multidimensional representation of multi-subject MEG data, latent component extraction and group-level statistical inference methodologies. We demonstrated that the grouplevel tensor decomposition recovers meaningful distinct brain patterns of varying spatiotemporal brain activity across subjects in healthy population of children/adolescents and in subgroups. The advantages of the proposed method include successful identification of the underlying latent brain patterns in the form of factor matrices via tensor factorization to allow for statistical assessment of the identified sources. The presented tensor-based group-level inference using CP component matrices eliminates the need to select specific regions of interest, such as time windows or specific sensor sites.

Using the proposed approach, we show that the tensor group-level analyses and tensor-based feature extraction allow us to investigate differences in brain activity between different subject groups. Given the importance of group-level inferences in neuroimaging studies, the extracted latent ERF components could be used to study differences in brain patterns across groups and aid in understanding how spatiotemporal brain activity can explain cognitive function and developmental changes directly from electrophysiological measurements. The application of MEG tensor decomposition used in this study is a promising direction for future research on other populations with different age ranges or developmental disorders.

Supplementary Material

Electronic Supplementary Material. This article contains supplementary material.

Data Availability.

The data supporting this study’s findings are available from the Dev-Cog MRN repository (Stephen et al., 2021)¹ upon user account request (contact email: vcalhoun@gsu.edu, jstephen@mrn.org).