Decoding the Spatiotemporal Dynamics of Neural Response Similarity in Auditory Processing: A Multivariate Analysis Based on OPM‐MEG

ABSTRACT

The brain represents information through the encoding of neural populations, where the activity patterns of these neural groups constitute the content of this information. Understanding these activity patterns and their dynamic changes is of significant importance to cognitive neuroscience and related research areas. Current studies focus more on brain regions that show differential responses to stimuli, but they lack the ability to capture information about the representational or process‐level dynamics within these regions. In this study, we recorded neural data from 10 healthy participants during auditory experiments using optically pumped magnetometer magnetoencephalography (OPM‐MEG) and electroencephalography (EEG). We constructed representational similarity matrices (RSMs) to investigate the similarity of neural response patterns during auditory decoding. The results indicate that RSA can reveal the dynamic changes in pattern similarity during different stages of auditory processing through the neural activity patterns reflected by OPM‐MEG. Comparisons with EEG results showed that both techniques captured the same processes during the early stages of auditory decoding. However, differences in sensitivity at later stages highlighted both common and distinct aspects of neural representation between the two modalities. Further analysis indicated that this process involved widespread neural network activation, including the Heschl's gyrus, superior temporal gyrus, middle temporal gyrus, inferior temporal gyrus, parahippocampal gyrus, and orbitofrontal gyrus. This study demonstrates that the combination of OPM‐MEG and RSA is sufficiently sensitive to detect changes in pattern similarity during neural representation processes and to identify their anatomical origins, offering new insights and references for the future application of RSA and other multivariate pattern analysis methods in the MEG field.

This study demonstrates the feasibility of using RSA combined with OPM‐MEG to decode neural response patterns across spatiotemporal dimensions, highlighting widespread neural network activations. It also reveals the similarities and differences in the sensitivity of OPM‐MEG and EEG when applying multivariate techniques to decode brain representations.

1. Introduction

A fundamental principle in contemporary neuroscience is that the brain encodes information through the population coding of numerous neurons (Georgopoulos et al. 1986). The activity patterns of neuron groups within a region are believed to collectively encode the content of these representations. Understanding brain activity patterns at different time points and their dynamic changes over time is crucial for many fields. Inspired by this concept, recent analyses of neural recordings and functional imaging data have increasingly focused on the activity patterns of large neuron populations within functional regions.

In recent years, various modalities for decoding brain activity have been developed. Among non‐invasive brain function measurement techniques, MEG directly measures neuronal electrical activity, capturing dynamic changes on a millisecond timescale (Hari and Puce 2023). MEG has become a crucial brain imaging technology with broad application prospects. Optically‐pumped magnetometer magnetoencephalography (OPM‐MEG) is a new magnetic field sensing technology that has recently emerged in the field of MEG (Allred et al. 2002). Its advantages, such as the ability to place sensors closer to the scalp, customizable sensor array layouts, and suitability for different populations, have made it a new research focus in brain neuroscience (Boto et al. 2021; Ma et al. 2024). The advent of OPM‐MEG offers new avenues for understanding neural representation content and processes. Numerous studies in domains such as vision (Wardle et al. 2016), language (Lyu et al. 2019), and audition (Guggenmos et al. 2018) demonstrate its high temporal resolution and rich spatial information, making it the preferred method for decoding the spatiotemporal dynamics of human brain activity.

Besides measurement modalities, significant progress has been made in methods for decoding neural activity, particularly univariate pattern analyses and multivariate pattern analyses (MVPA). Representational Similarity Analysis (RSA) is commonly used to examine local neural activity patterns. Compared to univariate methods, RSA is more sensitive to distributed neural representations, allowing for a more comprehensive understanding of brain activity patterns under specific tasks or conditions (Davis et al. 2014; Cichy and Oliva 2020), as it integrates the activity patterns of multiple voxels/neurons rather than simply averaging them, just as univariate analysis works. It has gained increasing popularity among researchers in recent years. RSA provides a powerful explanatory framework centered on the concept of representational space. In this representational space, each stimulus's neural response is represented as a high‐dimensional pattern vector. By calculating distances between patterns within this space, we can reveal the brain's representational processes, thus allowing for a comprehensive understanding of the brain's response patterns under specific tasks or conditions and providing deeper insights into the mechanisms of brain information processing. Numerous studies (Haxby et al. 2001; Cichy et al. 2014) have demonstrated that RSA, based on MVPA, is a powerful approach for decoding neural data. The study by Kriegeskorte et al. (2008) used this method to directly compare object representations in the inferior temporal cortex of monkeys and humans, revealing how different species perceive and categorize objects at the neural level. Traditionally, RSA has been applied in the field of functional magnetic resonance imaging (fMRI). In recent years, however, the method has gained increasing popularity in M/EEG research across various domains of neuroscience, including visual perception (Dobs et al. 2019; Cichy et al. 2019), auditory processing (Lowe et al. 2021), language (Hultén et al. 2021), and memory (Bae and Luck 2018; Staudigl et al. 2015). These applications highlight the significant potential of RSA in analyzing neural electrophysiological data.

OPM‐MEG can simultaneously measure brain activity at multiple locations with high spatiotemporal resolution, providing rich neuronal pattern information and precisely capturing the information and processes represented by large groups of neurons. This indicates that OPM‐MEG is ideally suited for decoding rapidly changing neural states in the human brain. The multivariate nature of RSA enables the simultaneous utilization of the spatial and temporal richness of multi‐channel data, facilitating the identification and interpretation of information contained within these complex activity patterns. Additionally, RSA does not rely on predefined stimulus categories but examines the response pattern for each stimulus independently, sampling the representational space more richly and covering more stimulus dimensions and features. It offers a means to study neural representations under rich experimental conditions, with significant advantages in experimental designs that focus on activity pattern information and utilize high‐resolution measurements (Kriegeskorte et al. 2008). Therefore, we believe that combining OPM‐MEG and RSA in neuroscience can capture the similarity of neural response patterns within the brain's representational processes, revealing the dynamic temporal evolution of these patterns and their underlying neural sources.

The structure of this paper is as follows: First, we designed an auditory experiment grounded in existing literature. This experiment incorporated a diverse array of sound stimuli, with the objective of comprehensively exploring the representational space and enhancing the stability and accuracy of neural decoding. The sound stimuli were presented to 10 participants, during which we recorded OPM‐MEG and EEG data. Subsequently, within the framework of representational space, we modeled the neural activity captured by OPM‐MEG and EEG as high‐dimensional neural response pattern vectors. By directly comparing the correlations between these patterns, we constructed a representational similarity matrix (RSM) at each time point, thus obtaining an abstract representation of the brain's encoded information and investigating the dynamic changes in pattern similarity during auditory processing. Third, we explored the anatomical origins of the changes in similarity along with the source localization results from M/EEG. Finally, we interpreted the aforementioned results and discussed potential future work related to RSA with OPM‐MEG.

2. Materials and Methods

2.1. Participants

The study was approved by the Ethics Committee of Beihang University (Nr. BM20200175), and written informed consent was obtained from all participants prior to the experiment. 10 participants (8 males, 2 females; mean age 27.7 ± 1.48 years) took part in the OPM‐MEG and EEG experiment. All participants were native Chinese speakers, right‐handed, with normal hearing, and no history of language or neurological disorders.

2.2. Experimental Design and Stimuli

Based on existing studies on auditory material classification and processing (Zhang et al. 2021; Moerel et al. 2012), we established six categories: human speeches, human non‐speeches, animal cries, scenes from life, scenes from nature, and musical instruments, resulting in 90 sound stimuli. Table 1 provides partial examples from each sound category. Each stimulus lasted approximately 0.8 s and was normalized to a consistent amplitude level using Praat software (https://www.fon.hum.uva.nl/praat/). The normalized stimuli were controlled via MATLAB (The MathWorks) and played back using Psychtoolbox‐3 (http://psychtoolbox.org/). Each sound was randomly repeated 9 times, resulting in a total of 810 trials. Sound stimuli were delivered binaurally to participants via silicone tubes. Before the experiment, sound levels were adjusted to a comfortable level, approximately 65 dB SPL.

TABLE 1.

A total of 6 categories of sounds, each with 15 stimulus samples.

Human speeches	Human non‐speeches	Animal cries	Scenes from life	Scenes from nature	Musical instruments
/ni/	Cry	Horse	Siren	Rain	Piano
/wo/	Laughter	Wolf	Bell chime	Wind	Flute
/good/	Coughing	Elephant	Train whistle	Thunder	Erhu
/hard/	Collapse	Cow	Doorbell	Flame	Drum
…	…	…	…	…	…

At the beginning of each block, a “Start sound” instructed the participants (Figure 1A). After a 4 s interval, the sound stimuli were presented. Following each sound stimulus, a 0.5 s response interval occurred, after which a button prompt was played. Participants were required to press a button on a feedback box upon hearing the cue, indicating whether the sound was emitted by an animate entity (left button) or an inanimate entity (right button), ensuring their sustained attention throughout the experiment. After a 0.5 s judgment period, the program proceeded to play the next sound stimulus. Each block lasted approximately 9 min. Each participant underwent 3 experimental sessions, with all stimuli presented in random order and repeated 3 times within each block session. There was a 5 min break between blocks, during which participants were verbally informed before the start of the next block (Figure 1B).

FIGURE 1.

Experimental paradigm and process. (A) Participants were instructed to make a judgment by pressing a key following the presentation of each auditory stimulus. (B) Each participant completed the experiment 3 times. All stimuli were presented randomly and repeated 3 times per experiment, totaling 810 trials, with a 5 min rest period between each block.

2.3. MEG and EEG Data Acquisition

The MEG acquisition system utilized 36 QuSpin (Quspin Inc., Louisville, CO, USA) second‐generation magnetometers (QZFM Gen2), 12.4 mm × 16.6 mm × 24.4 mm, with a sensitivity of $<15\text{fT}/\sqrt{\mathrm{Hz}}$. These sensors were uniformly placed in a 3D‐printed rigid helmet worn by participants during the experiment. The experiment was conducted in a magnetically‐shielded room (Figure 2A) with dimensions of 4 m × 3.5 m × 3 m and a central residual magnetic field of < 5 nT. The EEG system comprised a 128‐electrodes BrainCap, adhering to the 10–20 system. Data were recorded with an online filter range of 0.01–250 Hz. Throughout the experiment, the impedance across all channels was maintained below $30\mathrm{k\Omega }$, with the “FCz” channel serving as the online reference.

FIGURE 2.

OPM‐MEG and EEG data acquisition system. (A) One participant wore a rigid helmet for OPM‐MEG data acquisition. (B) Another participant underwent EEG data acquisition.

2.4. Co‐Registration System

A 3 T scanner (Siemens Medical Solutions, Erlangen, Germany) was used to acquire T1‐weighted MRI data for subsequent registration. The scanning parameters were: TR = 2200 ms, TE = 3.37 ms, TI = 1050 ms, FA = 7°, FOV = 256 × 256 mm, voxel size = 1.0 × 1.0 × 1.0 mm³. The MRI data were segmented using Freesurfer (https://surfer.nmr.mgh.harvard.edu/), resulting in a scalp surface model with 50,000 vertices. A structured light scanner (Occipital Inc., San Francisco, CA, USA) was employed to capture the 3D head shape of participants, producing a scanned point cloud. This point cloud was segmented into faces and helmets using region growing segmentation. Coarse and fine alignment of the 3D rigid helmet model and the helmet point cloud were achieved using the Sample Consensus Initial Alignment (SAC‐IA) and iterative closest point (ICP) algorithms, respectively. Nose tip points were localized based on the scan results and MRI images, and ICP was used to align the face scan point cloud with MRI images. Figure 3B illustrates the final registration results. Since the helmet design fixed the sensors' positions relative to the helmet, their positions relative to the MRI could be determined after two co‐registrations (Cao et al. 2021). Source localization on OPM‐MEG data was conducted using the MNE (https://mne.tools/stable/index.html) software package, identifying regions of significant activation. Initially, 3D head shape data from structural light scanning were co‐registered with MRI data to determine sensor positions relative to structural images. Using a head model and sensor positions, we computed the forward solution with MNE and estimated anatomical sources and source activation time series via dynamic statistical parametric mapping (dSPM).

FIGURE 3.

Schematic diagram of OPM‐MEG data processing. (A) Preprocessing. For OPM‐MEG data, preprocessing steps included filtering, removing bad channels and segments, and performing ICA. (B) Alignment. The arrows indicate the position and orientation of the sensors relative to the participant's MRI. (C) Source reconstruction. Based on this alignment, the physiological sources of pattern similarity changes are localized using established algorithms.

2.5. Preprocessing

For the OPM‐MEG data, trials with incorrect responses were first removed (Figure 3A). Offline bandpass filtering (1–30 Hz) and homogeneous field correction were then applied. Power spectral density (PSD) plots were inspected to identify and remove channels with poor signal quality (The final processed result showed that the average number of channels removed per participant was 0.70 ± 0.64). Trials were segmented based on synchronized trigger signals, extracting data from 100 ms pre‐stimulus to 1000 ms post‐stimulus to create epochs for subsequent analysis. Abnormal segments were excluded based on visual inspection of the PSD plots. Independent component analysis (ICA) was performed to remove artifacts from the segmented data.

For the EEG data, trials with incorrect responses were initially removed. Offline bandpass filtering (1–30 Hz) and whole‐brain reference averaging were applied. Channels were visually inspected and removed if indicated by PSD results (No EEG channels were removed from further analysis for each participant). The “FP1” and “FP2” channels were designated as EOG channels, and ICA was performed to remove artifacts from eye movements, heartbeats, and muscles. Data segmentation followed the trigger signals, with baseline correction applied, and epochs were extracted from 100 ms pre‐stimulus to 1000 ms post‐stimulus. Visual inspection was used to remove bad segments.

2.6. Representational Similarity Analysis

RSA was implemented using custom scripts combined with an existing Python toolkit (Lu and Ku 2020). We constructed RSMs using high‐dimensional neuroimaging data from each block of each participant. At each moment t (Figure 4A), all trials of the same stimulus were averaged to obtain the mean response, reducing the impact of potential outliers on the RSMs (Popal et al. 2019). The amplitudes of all channels at the same moment represent the brain's activity pattern, forming a vector of neural response patterns corresponding to the stimulus, with the vector's dimension M being the number of M/EEG channels. Pearson's correlation coefficient R (where R = 1 indicates perfectly correlated neural response patterns, and R = 0 indicates no correlation) was then calculated for all possible pairs of stimuli, resulting in a symmetric square matrix whose dimensions correspond to the number of conditions in the experiment. Each value in this matrix represents the neural response pattern similarity (NRPS) between pairs of stimuli, with each row and column representing the similarity of one stimulus to all others (including itself). Calculating R for all possible pairs over time yields a series of RSMs for a participant (Figure 4B). By averaging the off‐diagonal elements in the lower‐left corner of each RSM, we obtain the average R‐value at each time point, resulting in a temporal curve of average R‐values. This process is repeated for all participants and averaged across them to obtain the dynamics of pattern similarity over time.

FIGURE 4.

Flow of RSA. (A) Calculation of RSM at moment t. The amplitudes of all channels at a given moment represent the brain's neural response pattern vector to the corresponding stimuli. Extracting the pattern vectors of stimulus i and stimulus j at moment t, the response pattern similarity of the stimulus pair is calculated using Pearson's correlation coefficient, which gives the value of $R_{\mathit{ij}}$ in the RSM. This process is repeated for all possible stimulus pairs to obtain RSM at that moment. (B) Pattern similarity was calculated for all possible stimulus pairs point‐by‐point in time to obtain a temporal series of RSMs. The mean value of the off‐diagonal elements in the lower left corner of each matrix was calculated and then averaged across participants.

2.7. Model and M/EEG Similarity Analysis

To further verify and explain the neural RSM, we designed a binary classification model RSM (Figure 5). This model aims to distinguish the neural activity differences between animate sounds (human speeches, human nonspeeches, animal cries) and inanimate sounds (scenes from nature, musical instruments, scenes from life). We performed a correlation analysis between the model RSM and the neural RSM. The specific hypothesis of the model RSM is supported by a consistent pattern between the neural RSM and the model RSM.

FIGURE 5.

Correlation analysis between the model and neural RSMs. Neural RSMs were constructed at each time point from the M/EEG data, and the corresponding similarity to the model RSM was calculated, resulting in a curve of similarity changes.

Specifically, model RSM hypothesizes that, among the 90 sound stimuli, the similarity of neural activity patterns for all animate sounds is 1, the similarity for all inanimate sounds is also 1, and the similarity between animate and inanimate sounds is 0.

2.8. Statistical Testing

To identify time points where pattern similarity significantly differed from baseline, we performed the nonparametric Wilcoxon signed‐rank test (Wilcoxon 1992) at the epoch level. Specifically, we calculated the average similarity for each participant before the baseline period. We then merged the pre‐stimulus and post‐stimulus similarity results across time into a unified dataset, sorted them, and assigned ranks (averaging ranks for ties). We calculated the sum of ranks separately for pre‐stimulus and post‐stimulus data to obtain the Wilcoxon rank‐sum statistic. Distribution tables were consulted to determine the corresponding p‐values, with a significance level set at $\alpha =0.05$. If the p‐value was less than the significance level, the time point was considered significantly different from baseline and marked as such.

In addition, we performed the Bayesian population prevalence analysis (Ince et al. 2021, 2022), which aims to estimate the prevalence of the above significant effects in the population. This approach allows quantification of replication probabilities within participants. Specifically, we performed the above significance test for each participant and determined the number of participants that did and did not reach significance, obtaining the Bayesian posterior distribution function (alpha = 1 + n_significant; beta = 1 + n_nonsignificant). The beta distribution was then used to represent the population prevalence and determined the maximum a posteriori (MAP) estimation. The output of this Bayesian prevalence analysis also provided an interpretation of uncertainty, termed the highest posterior density interval (HPDI). The HPDI provided a range in which the true population prevalence could be found with a specified probability.

To elucidate the anatomical origins of similarity changes, we projected the time series of all source estimates onto a standardized brain template and averaged them across participants. T‐tests were employed to calculate t‐statistics for each voxel and time point, with corresponding p‐values derived and FDR correction applied to address multiple comparisons. Corrected p‐values replaced the original data, and thresholds of $\alpha$ = 0.05, 0.01, and 0.001 were set to determine statistically significant activations. The spatio‐temporal distribution of source activation was visualized using the MNE toolkit.

3. Results

3.1. Behavioral Results

We recorded responses from 10 participants under both modalities. The average accuracy for EEG was 0.965 ± 0.016 (mean ± SD), and for MEG, it was 0.967 ± 0.017 (mean ± SD). No significant behavioral differences were observed between the two modalities in terms of participant performance.

3.2. The Results of Representational Similarity Analysis for M/EEG

Representational similarity analysis across all participants revealed dynamic variations in similarity and significant time scales for both OPM‐MEG and EEG, with consistent trends in both modalities during the early stages of auditory processing (Figure 6A). According to the Bayesian prevalence analysis, the MAP estimations in the M/EEG representation similarity analysis were 0.89 (96% HPDI: [0.60, 0.99]; 50% HPDI: [0.82, 0.95]) and 0.79 (96% HPDI: [0.48, 0.96]; 50% HPDI: [0.70, 0.87]). This indicated that most participants exhibited significant effects in the M/EEG experiments. The HPDI results further showed the robustness of these findings (Figure 6C,D). Pattern similarity in OPM‐MEG significantly increased within the 80–300 ms post‐stimulus window, with statistical tests indicating significant differences compared to pre‐stimulus (Wilcoxon rank‐sum test, $\alpha$ = 0.05). Specifically, pattern similarity in OPM‐MEG showed a moderate similarity level from 100 ms pre‐stimulus to 80 ms post‐stimulus, averaging 0.52. This similarity sharply increased, reaching a first peak of 0.60 at 100 ms, then rapidly decreased to 0.56 at 120 ms. Another sharp increase occurred at 120 ms, reaching a second peak of 0.67 at 150 ms, followed by a rapid decrease to 0.54 at 210 ms. Another sharp increase occurred at 250 ms, reaching a third peak of 0.61, followed by a rapid decline. At 300 ms post‐stimulus, similarity gradually returned to the baseline level of 0.51 until the end of the stimulus.

FIGURE 6.

Neural response pattern similarity curve and Bayesian posterior distributions of population prevalence. The blue curve represents the mean similarity value, indicating pattern similarity over time. Light shading denotes the standard deviation, while red inverted triangle symbols represent significant time periods relative to the baseline. (A) The curve of OPM‐MEG pattern similarity. Compared to the pre‐stimulus period, significant differences began at 80 ms post‐stimulus and persisted until 300 ms (Wilcoxon rank‐sum test, $\alpha$ = 0.05), with a baseline similarity of 0.51. (B) EEG changes synchronized with OPM‐MEG from 80 to 300 ms, but also showed significant differences from 300 to 640 ms post‐stimulus (Wilcoxon rank‐sum test, $\alpha$ = 0.05), with a baseline similarity of 0.51. (C) MAP estimation of population prevalence in MEG representation similarity analysis. The estimated value of MAP was 0.89 (96% HPDI: [0.60, 0.99]; 50% HPDI: [0.82, 0.95]). The orange curve represents the posterior distribution of population prevalence, the solid circles represent MAP, and the thick and thin horizontal lines represent 50% and 96% HPDI. (D) MAP estimation of population prevalence in EEG representation similarity analysis. The estimated value of MAP was 0.79 (96% HPDI: [0.48, 0.96]; 50% HPDI: [0.70, 0.87]). The orange curve represents the posterior distribution of population prevalence, the solid circles represent MAP, and the thick and thin horizontal lines represent 50% and 96% HPDI.

EEG pattern similarity exhibited significance between 100 and 640 ms post‐stimulus (Figure 6B). EEG showed dynamic changes consistent with OPM‐MEG within the range from 100 ms pre‐stimulus to 300 ms post‐stimulus. Specifically, EEG exhibited pattern similarity at 100 ms pre‐stimulus, with a similarity of 0.51, persisting until 80 ms post‐stimulus. Subsequently, there was a sharp increase, reaching a first peak of 0.56 at 100 ms, followed by a continuous rise to a second peak of 0.66 at 150 ms. It then rapidly decreased to 0.62 at 210 ms, followed by another sharp increase, reaching a third peak of 0.71 at 250 ms, and then dropped quickly to 0.53 at 360 ms. EEG showed significant pattern similarity from 300 to 640 ms, with a broader peak at 500 ms, peaking at 0.57, whereas OPM‐MEG did not show significant differences during this time range. From 640 ms post‐stimulus, similarity gradually returned to the baseline level of 0.51. In the late phase, EEG was sensitive to significant changes in activity patterns compared to OPM‐MEG, suggesting distinct aspects of the two modalities with respect to MVPA.

3.3. The Results of Model and M/EEG Similarity Analysis

We compared the model RSM with the neural RSM obtained at each time point from M/EEG and calculated their temporal correlations. As shown in Figure 7, in certain time windows, the two RSMs exhibit significant similarities.

FIGURE 7.

The curve of similarity between the model and neural responses and Bayesian posterior distributions of population prevalence. The blue curve represents the average similarity at each time point, reflecting the pattern similarity over time. The light shading indicates the standard deviation, and the red inverted triangle symbols represent significant time periods relative to the baseline. (A) MEG‐RSM and model RSM pattern similarity curve. The continuous time window showed significant similarity compared to the pre‐stimulus period spans from 120 to 270 ms post‐stimulus (Wilcoxon signed‐rank test, p = 0.05). (B) EEG‐RSM and model RSM pattern similarity curve. The continuous time window showed significant similarity compared to the pre‐stimulus period spans from 100 to 350 ms post‐stimulus (Wilcoxon signed‐rank test, p = 0.05). (C) MAP estimation in the similarity analysis between MEG‐RSM and model RSM representations. The estimated value of MAP was 0.79 (96% HPDI: [0.48, 0.96]; 50% HPDI: [0.70, 0.87]). The orange curve represents the posterior distribution of population prevalence, the solid circles represent MAP, and the thick and thin horizontal lines represent 50% and 96% HPDI. (D) MAP estimation in the similarity analysis between EEG‐RSM and model RSM representations. The estimated value of MAP was 0.79 (96% HPDI: [0.48, 0.96]; 50% HPDI: [0.70, 0.87]). The orange curve represents the posterior distribution of population prevalence, the solid circles represent MAP, and the thick and thin horizontal lines represent 50% and 96% HPDI.

The similarity analysis between RSM based on MEG (hereinafter abbreviated as MEG‐RSM) and the model RSM revealed that, during the period from 100 ms prior to the stimulus onset to the onset of the stimulus, the similarity was approximately 0.50 and persisted until 30 ms after stimulus onset. The similarity then gradually increased, reaching 0.60 at 150 ms and stabilizing around 190 ms at 0.62, before slowly decreasing to 0.51 at 350 ms. Subsequently, the similarity showed a slight increase, but overall, it continued to decline, reaching 0.49 at 420 ms, after which it stabilized. During the period from 120 to 270 ms after stimulus onset, the MEG‐RSM and the model RSM showed significant correlation. For the RSM based on EEG (hereinafter abbreviated as EEG‐RSM) and model RSM similarity analysis, the similarity was about 0.49 from 100 ms before the stimulus onset to the stimulus onset and continued until 80 ms post‐stimulus. After this, the similarity began to rise, reaching 0.58 at 200 ms, and then slowly declined at the subsequent time points, reaching 0.48 at 370 ms, after which it stabilized. During the period from 100 to 350 ms post‐stimulus, the EEG‐RSM and the model RSM showed significant correlation. The MAP estimations in both the MEG‐RSM and EEG‐RSM similarity analyses with the Model RSM were 0.79 (96% HPDI: [0.48, 0.96]; 50% HPDI: [0.70, 0.87]), indicating a significant similarity between M/EEG and the binary classification model RSM in a large proportion (8/10) of participants (Figure 7C,D). This result showed that the significant effects exhibited high consistency and reliability at the individual level.

3.4. OPM‐MEG Source Location

To identify potential sources of increased pattern similarity relative to baseline, we conducted source‐level imaging across all participants (Figure 8). At 100 ms post‐stimulus, significant activations were observed in the left superior temporal gyrus (STG‐lh), especially in the anterior region, as well as in the right middle temporal gyrus (MTG‐lh) and the right inferior temporal gyrus (ITG‐lh). Similar activations were also detected in the left hemisphere, although with noticeably lower intensity and extent compared to the right hemisphere. Additionally, significant activations were identified in the right Heschl's gyrus (HG‐lh) and parahippocampal gyrus (PG‐lh). At 150 ms, significant bilateral activations were observed in the STG, HG, MTG, ITG, and orbitofrontal gyrus (OFG). Compared to 100 ms, both hemispheres exhibited anterior extension of activation, spanning from the posterior to anterior temporal regions, with increased intensity and extent, indicating continued auditory information processing. Additionally, significant and enhanced activations in the bilateral PG suggest increased auditory information processing in memory‐related brain areas. At 250 ms, significant activations were primarily observed in the PG, with reduced intensity and extent compared to 150 ms, yet still indicating ongoing auditory processing in memory‐related regions. Moderate activations were also noted in the middle part of the STG‐lh and HG, indicating a decline in activity within the primary auditory cortex. At 500 ms, no significant activations were observed in either hemisphere relative to baseline, suggesting a return or near return to baseline brain activity levels. This corresponds with the absence of significant differences in neural activity patterns compared to baseline during this timeframe, indicating the completion of higher‐level auditory information processing.

FIGURE 8.

Source localization of pattern similarity changes. The results indicate a significant increase in similarity in various brain regions between 100 and 250 ms post‐stimulus, particularly at the STG, MTG, ITG, HG, PG, and OFG of both hemispheres. These changes peaked at 150 ms, displaying broader and stronger activation. At 250 ms, significant activation was still observable, though the activation level diminished. At 500 ms, the brain's activation state had largely returned to baseline, with no significant differences. These findings reveal the dynamics of brain pattern similarity and identify the major brain regions involved in auditory processing.

3.5. EEG Source Location

After preprocessing to EEG data, we used MNE to obtain the source localization results of neural activity at different time points during auditory processing for each participant, which were then averaged across participants. As shown in Figure 9, significant activation was observed in the HG‐rh at 100 ms post‐stimulus. At 150 ms, significant activation was observed in the STG‐rh, MTG‐lh, left inferior ITG‐lh, and bilateral OFG. At 250 ms, bilateral PG showed significant activation. At 500 ms, no significant activation was observed in either hemisphere compared to baseline, except for the bilateral PG.

FIGURE 9.

EEG source localization of pattern similarity changes. At 100 ms post‐stimulus, significant activation was observed in the HG‐rh. At 150 ms, significant activation was observed in the bilateral temporal voice area (TVA), OFG, and PG, and it persisted until 250 ms. At 500 ms post‐stimulus, no significant activation was observed in other brain regions, except for the bilateral PG.

4. Discussion

In this study, we collected OPM‐MEG and EEG data under identical stimuli and task conditions. After preprocessing the data, we extracted response pattern vectors for each participant over time in response to auditory stimuli. We calculated pairwise similarities between these response patterns to construct time‐series RSMs, capturing the temporal dynamics of similarity during auditory processing. Our findings revealed that during the early stages of auditory processing, OPM‐MEG produced multivariate analysis results similar to EEG, indicating its capability to capture neural processing comparable to EEG. However, differences between OPM‐MEG and EEG emerged in the later stages of auditory processing, suggesting varying sensitivities in multivariate analysis. The source localization results from M/EEG consistently revealed the involvement of the bilateral TVA, PG, OFG, and HG in this process. In conclusion, integrating OPM‐MEG with RSA leverages their complementary strengths, uncovering dynamic changes in pattern similarity and anatomical origins during auditory decoding processes.

4.1. Pattern Similarity Changes in Early Perception

In adults, auditory information takes approximately 80–120 ms to reach the stage of conscious perception (Čeponienė et al. 2005). Our OPM‐MEG results indicate that pattern similarity begins to increase sharply at 80 ms post‐stimulus, peaking at 100 ms. The result is consistent with the study Ogg et al. (2020), which recorded SQUID‐MEG responses from participants following a range of natural sound stimuli. They used MVPA to study the process of auditory representations. The results showed that the brain began to discriminate between sound categories starting around 90 ms post‐stimulus. Concurrent changes in EEG results during this timeframe suggest that neural activity similarity is crucial in the early stages of auditory processing. This finding aligns with the views proposed by Correa et al. (2006) and Lange (2013), which suggest the presence of activity synchronization in the early stages of perception. Treisman suggested that the human cognitive system does not passively accept all external information but can modulate neural activity in specific brain regions (Treisman 1964).

In this experiment, we propose that the brain establishes neural representation synchronization among specific neuron groups during the early stages of sound processing. This synchronization increases pattern similarity, which subsequently affects perceptual processes (Fries 2005; Siegel et al. 2008). Synchronized neural firing has a stronger effect on neuronal populations than temporally disorganized firing (Doesburg et al. 2008). Moreover, our results are consistent with numerous studies (Maess et al. 2007; Bonte and Blomert 2004), which indicate that the results recorded within this timeframe consist of multiple subcomponents. These subcomponents are primarily distributed in the auditory regions of the temporal lobe, including HG, as well as the frontal regions (Gonzalez‐Heydrich et al. 2015). These studies suggest that the regions identified in this research form a neural network that enhances neural activity in response to stimuli during early perceptual stages.

The results of M/EEG show that the participants also presented a certain degree of similarity during the interval between sound stimulations, which is consistent with many similar studies. We believe that during the entire experiment, the participants listened to the sounds and made judgments, which required them to pay intense attention to the tasks. This sustained state of attention led to a high level of neural synchronization during the interval between adjacent stimuli. In other words, when participants adopted a consistent strategy for processing auditory stimuli (e.g., concentrating and making judgments), this strategy could enhance the consistency of brain activation patterns, leading to moderate neural pattern similarity even during the inter‐stimulus intervals. Hipp et al. (2011) found that during discrimination tasks, synchronization in relevant cortical areas had already increased before or just as the stimulus was presented, providing the brain with a “window” to process information rapidly after stimulus onset. Siegel's MEG study (Siegel et al. 2008) suggested that attention states not only influence neural representations after stimulus presentation but also assist task‐related selection by synchronizing specific neuronal populations before stimulus onset. Similar viewpoints have been expressed in other studies (Kastner and Ungerleider 2000; Breiter et al. 2001; Fox et al. 2005), which suggest that attention plays a role not only during the actual stimulus processing phase but also during anticipatory and preparatory states, allowing the brain to maintain sensitivity to upcoming stimuli.

4.2. Sensitivity to Sound in the Frontal and Temporal Regions

Our source localization results are consistent with the studies of Wingfield et al. (2017) and Su et al. (2014), who used SQUID‐MEG and EEG with RSA to find distributed patches in the bilateral temporal cortex of humans that are sensitive to sound stimuli. Growing evidence suggests that the human brain has specific neural pathways for speech processing within the auditory system. Using fMRI in adults, Belin identified sound‐sensitive TVA in the temporal lobe (Belin 2017). These areas form several clusters along the STG and superior temporal sulcus (STS) in both hemispheres. These brain regions are reported to be very close to the core auditory areas (approximately two to three synapses away) and contain numerous sound‐sensitive cells (Rogier et al. 2010). The result is well consistent with the study by Capilla et al. (2013). The study recorded the SQUID‐MEG responses of participants listening to animate sounds and inanimate sounds. The results showed that significant discrimination of the above two categories occurred 150 ms after stimulation and peaked near 200 ms. Further source localization analysis showed that it originated from the bilateral TVA. Activation of the TVA induces significant responses in the frontal and temporal regions of the scalp (Rogier et al. 2010). This response typically occurs between 100 and 250 ms after auditory stimulation, serving as a hallmark of rapid brain response to various stimuli and is well established (De Lucia et al. 2010; Perrodin et al. 2015). Charest et al. (2009) conducted a MEG study comparing neural responses to a wide range of auditory stimuli and observed this phenomenon at frontal and temporal electrodes. They further indicated that the brain's discrimination of sounds begins around 120–150 ms. Renvall et al. (2012) used SQUID‐MEG to find that, after presenting a large number of natural sound stimuli, participants showed strong responses at 100 and 250 ms post‐stimulus, which originated from the bilateral STS and HG.

We investigated pattern similarity in brain speech processing using OPM‐MEG and EEG. Our results are consistent with previous studies that describe activation in the frontal and temporal lobe regions in terms of timing and topographical distribution, suggesting their involvement in basic auditory discrimination processes at this stage. During this timeframe, we also observed significant activation in the bilateral STG, MTG, ITG, PG, and OFG, consistent with numerous anatomical and physiological studies (Romanski 2012; Remedios et al. 2009; Reber et al. 2003). Recently, Giordano et al. (2021) combined SQUID‐MEG and fMRI to show that in the early stages of category perception (157 ms post‐stimulus), bilateral frontal and temporal regions (such as STG and OFG) dominate brain representation. These studies have noted that neurons in these regions are sensitive to sound sources, playing crucial roles in sound understanding, retrieval, and discrimination, indicating that these cortical areas are part of the ventral auditory pathway. We constructed neural RSMs from the M/EEG signals of the participants to reveal the temporal dynamics of auditory information encoding and performed a correlation analysis with a binary classification model RSM. The results indicated that both MEG and EEG signals exhibited significant similarity with the model RSM within a similar time window (approximately 100–300 ms post‐stimulus). Our findings are consistent with current related studies. Combined with source localization results, the significant correlation during this time window suggests that the fronto‐temporal regions play a crucial role in the processing of sound discrimination.

Traditionally, faces have been considered special and processed differently from other types of visual stimuli (Tsao et al. 2008). However, increasing evidence suggests that despite the vastly different sensory inputs of faces and voices, the cognitive and neural processing mechanisms for these signals are remarkably similar, organized according to similar principles in their underlying functional architecture (Yovel and Belin 2013). Thus, they may generate similar neural representations. The changes in pattern similarity observed in this study exhibit temporal and functional organizational similarities with the aforementioned face perception processes, providing potential directions for future research. Given that in social interactions, similar types of information are often integrated through faces and voices (Yovel and Belin 2013), this is highly relevant for studying integration mechanisms.

4.3. The Establishment and Consolidation of Neural Representations

The M/EEG results reveal a significant similarity in neural responses within the 200–250 ms time period, which is consistent with findings from other studies (Karhu et al. 1997; Čeponienė et al. 2008), indicating that with repeated exposure to sounds, the activity similarity between neural ensembles responsible for processing these sounds increases. Our study utilized a large number of repeated stimuli, exposing the brain to these stimuli multiple times. This repeated exposure facilitated the gradual adaptation of the neural network to these stimuli, establishing and strengthening neural response patterns.

According to source localization results, bilateral activation of the PG is observed during sound processing, consistent with recent fMRI studies (Ceravolo et al. 2024; Leong et al. 2023), which suggest that, in everyday life scenarios, the processing of meaningful and non‐meaningful sounds involves the recruitment of the PG. Additionally, MEG connectivity analysis (Salvari et al. 2023) indicates a strong connection between PG and frontal regions, highlighting its crucial role in the brain network responsible for sound perception. Youssofzadeh et al. (2022) studied the SQUID‐MEG responses of healthy adults to auditory stimulation. The results showed that during the representation of the stimulus, the STG, MTG, and ITG were significantly activated. In addition, it was also found that the process involved the participation of the bilateral PG. Furthermore, an EEG study (Farahani et al. 2021) used source reconstruction technology and found that areas such as PG and OFG are one of the main sources of auditory steady‐state evoked responses. These studies provide direct evidence for the involvement of PG in auditory processing. The processing and rapid recognition of natural sounds require the decoding of complex dynamic features and rely on the collaboration of higher cognitive brain regions to complete auditory functions. The PG, located in the medial temporal lobe (MTL), is closely associated with a variety of higher cognitive functions. Although traditionally considered to be more responsive to visual stimuli, research has shown that the PG also responds to auditory stimuli (Mitterschiffthaler et al. 2007; Arnott et al. 2008; Engelien et al. 2006). Specifically, the PG is closely connected to the auditory cortex, playing a significant role in the encoding and retrieval of auditory memory (Baumann and Mattingley 2016; Aminoff et al. 2013; Pearson et al. 2011). The auditory cortex transmits auditory information to the MTL via the PG (Squire et al. 2001); on the other hand, the PG also receives and transmits auditory signals from the hippocampus, facilitating the propagation of auditory information to other cortical regions of the brain (Oh et al. 2014; Marschall et al. 2021). In Summary, our results indicate that the activation of these neural representations aligns with numerous current reports. MEG and fMRI studies suggest that activation predominantly originates from the temporal auditory cortex, particularly around HG (Papanicolaou et al. 2003; Price 2012). Some lesion and intracranial EEG studies also indicate activation in PG during this process (Knight 1996; Nie et al. 2019).

4.4. Commonalities and Individualities in Representational Similarity Analysis Between OPM‐MEG and EEG

In the OPM‐MEG experiment, participants were presented with the same stimuli and tasks. When comparing the results, we found that both representational analyses captured the millisecond‐scale spatiotemporal dynamics of auditory processing and produced similar outcomes. Between 80 and 300 ms, the representational analysis results were highly similar. The results measured by MEG and EEG stem from the same neural activity, as they are sister electrophysiological techniques that essentially reflect the same neural phenomena (Baillet 2017). Therefore, it is not surprising that EEG can detect the same multivariate analysis results as OPM‐MEG. This consistency aligns with previous studies (Cichy and Pantazis 2017), which reported similar results between EEG and MEG multivariate analysis effects.

In addition to producing similar results, we found a difference in the sensitivity of the two techniques in detecting pattern changes using RSA in the 300–640 ms time range. Specifically, EEG detected significant differences in pattern similarity within this time range, while OPM‐MEG did not show significant differences in response patterns compared to baseline. We believe this may be due to sensitivity differences between the two modalities. Although both techniques are sensitive to tangential dipole sources, EEG can detect radial dipole sources, allowing it to capture activities that MEG cannot detect (Boto et al. 2021). This indicates the presence of unique information encoded by each modality. Additionally, due to the fact that magnetic fields generated by deeper brain tissues are more susceptible to background noise, one of the main limitations of MEG, compared to EEG, is that the signal‐to‐noise ratio tends to significantly decrease when detecting activity from deeper brain regions (Zhu et al. 2020; Yang et al. 2024). Moreover, the electrical conductivities of the scalp, skull, and cerebrospinal fluid vary, whereas magnetic permeability remains relatively constant. Therefore, the spatiotemporal topology of MEG is less blurred and distorted compared to that of EEG produced by the same physiological brain sources. In summary, these differences make MEG and EEG sensitive to both shared and unique aspects of neural representations, meaning the two techniques do not always detect the same neurocognitive processes.

Recently, a study by Wang and Kuperberg (2023) showed that MEG and EEG multivariate analysis results are broadly similar in the early time window, but they differ in detecting neural activity in the late time window. The authors attribute this to the sensitivity differences between the two techniques to different neuroanatomical sources. Wu et al. (2024) visually compared and validated the complementary information of these two modalities, revealing their sensitivity differences in reflecting neuronal representations from the perspective of encoding sensitivity. We argue that despite differences, it is crucial to emphasize the complementarity and importance of both. This study combines OPM‐MEG and EEG, providing comprehensive information on both brain electrical and magnetic activities. This approach is advantageous for detailed investigations into complex brain networks by leveraging MEG's sensitivity to tangential sources and localization capabilities, alongside EEG's superior detection of radial components. In summary, our results demonstrate that RSA outcomes from OPM‐MEG and EEG provide commonalities and complementary perspectives on neural representational patterns of speech processing, thereby advancing their broader application in MEG and EEG research.

5. Limitations and Prospects

The EEG and OPM‐MEG data in this study were collected separately. However, simultaneous EEG‐MEG recording has been shown to have advantages (Mahmutoglu et al. 2022). In the future, we can address this issue by adopting simultaneous collection techniques. Secondly, compared to univariate pattern analysis, multivariate pattern analysis results lack direct physiological interpretation (Wang and Kuperberg 2023). It only indicates whether and when neural activity elicited by stimuli can be decoded. This study provides insights for future applications of RSA in OPM‐MEG and the broader field of neuroscience. In addition to continuing to use various neuroscience recording techniques (such as EEG and OPM‐MEG in this study), we will focus on developing new data analysis methods and models to extract more information from multivariate time series. Furthermore, as mentioned in Part 4.2, in social interactions, the same type of information (such as language, identity, emotion, gender, etc.) is often represented by both faces and voices, sharing comparable time frames and similar processing courses, making them suitable for studying integration mechanisms. This provides an intriguing avenue for future research. In summary, future research will build on this study and focus on addressing these issues.

6. Conclusion

In this study, we combined the high spatiotemporal resolution of OPM‐MEG measurement technology with the multivariate analysis method RSA to understand how neural representations unfold over time during task execution. We explored brain response patterns to rich auditory stimuli and compared the results with those from EEG. The findings demonstrate that OPM‐MEG combined with RSA is sufficiently sensitive to reveal fine‐grained changes in pattern similarity during brain representation processes. Comparison with EEG results indicates that both modalities are sensitive to similar processes in the early stages of auditory decoding but exhibit differences in sensitivity in later stages, suggesting both commonalities and unique aspects in RSA between these measurement techniques. The source localization results of M/EEG indicate widespread recruitment of regions such as the bilateral STG, MTG, ITG, PG, OFG, and HG, suggesting extensive activation of neural networks involved in this process. This provides an analytical foundation and framework for the future integration of OPM‐MEG with RSA and other MVPA methods.

Author Contributions

Conceptualization: C.L. Methodology: C.L. and H.W. Software: C.L. Validation: C.L., H.W., and X.L. Formal analysis: C.L. Investigation: C.L. and H.W. Resources: M.X. and X.N. Data curation: C.L., H.W., Y.M., and X.L. Writing – original draft preparation: C.L. Visualization: C.L. Supervision: H.W. and X.N. Project administration: X.N. Funding acquisition: M.X. All authors have read and agreed to the published version of the manuscript.

Ethics Statement

The study was approved by the Ethics Committee of Beihang University (Nr. BM20200175), and written informed consent was obtained from each participant before the experiment.

Consent

The authors have nothing to report.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.