Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jul 1.
Published in final edited form as: Neuroimage. 2015 Apr 2;114:49–56. doi: 10.1016/j.neuroimage.2015.03.066

Combined MEG and EEG show reliable patterns of electromagnetic brain activity during natural viewing

Wei-Tang Chang 1,*, Iiro P Jääskeläinen 2, John W Belliveau 1,3, Samantha Huang 1, An-Yi Hung 1,4, Stephanie Rossi 1, Jyrki Ahveninen 1
PMCID: PMC4446182  NIHMSID: NIHMS677390  PMID: 25842290

Abstract

Naturalistic stimuli such as movies are increasingly used to engage cognitive and emotional processes during fMRI of brain hemodynamic activity. Movies have been, however, little utilized during magnetoencephalography (MEG) and EEG that directly measure population-level neuronal activity at a millisecond resolution. Here, subjects watched a 17-min segment from the movie “Crash” (Lionsgate Films, 2004) twice during simultaneous MEG/EEG recordings. Physiological noise components, including ocular and cardiac artifacts, were removed using the DRIFTER algorithm. Dynamic estimates of cortical activity were calculated using MRI-informed minimum-norm estimation. To improve the signal-to-noise ratio (SNR), principal component analyses (PCA) were employed to extract the prevailing temporal characteristics within each anatomical parcel of the Freesurfer Desikan-Killiany cortical atlas. A variety of alternative inter-subject correlation (ISC) approaches were then utilized to investigate the reliability of inter-subject synchronization during natural viewing. In the first analysis, the ISCs of the time series of each anatomical region over the full time period across all subject pairs were calculated and averaged. In the second, dynamic ISC (dISC) analysis, the correlation was calculated over a sliding window of 200 ms with 3.3 ms steps. Finally, in a between-run ISC analysis, the between-run correlation was calculated over the dynamic ISCs of the two different runs after the Fisher z-transformation. Overall, the most reliable activations occurred in occipital/inferior temporal visual and superior temporal auditory cortices, as well as in the posterior cingulate, precuneus, pre- and post-central gyri, and right inferior and middle frontal gyri. Significant between-run ISCs were observed in superior temporal auditory cortices and inferior temporal visual cortices. Taken together, our results show that movies can be utilized as naturalistic stimuli in MEG/EEG similarly as in fMRI studies.

1. Introduction

Several important aspects of perceptual, cognitive, and emotional functions have been difficult to engage, and thus measure, using neuroimaging paradigms that are based on multiple repetitions of artificial stimuli in order to reduce the complexity of the measured brain activity. An increasing number of studies have utilized model-free naturalistic stimulation to mitigate this challenge. For example, cinema has been used as a novel naturalistic stimulus in the study of frontal lobe function, social cognition and the associated emotions (Spiers and Maguire, 2007, Jääskeläinen et al., 2008, Hasson et al., 2010, Nummenmaa et al., 2012). In studies of social cognition, feature films deliver more accurate and complete information on the history and context of events, which determine the behavior of interacting people, than what is often possible even in real-life observation. Well-directed movies can also elicit genuine and very strong emotional states in experimental subjects (Westermann et al., 1996), even in a relatively compromised environment such as the fMRI scanner (Jääskeläinen et al., 2008, Nummenmaa et al., 2012). As a further advantage, movies can be presented to subjects in a replicable manner, which makes it possible to specifically inspect reliability of brain hemodynamic activity in a model-free manner (Hasson et al., 2004, Hasson et al., 2010). In addition, social stimuli (e.g., presence of faces, emotional expressions, and social interactions) in the movie can be dynamically annotated (Lahnakoski et al., 2012). This helps provide multiple time courses that can be correlated with the subjects' brain activity patterns that are elicited during watching of the movies (Bartels and Zeki, 2004, Lahnakoski et al., 2012). Indeed, an accumulating body of research indicates that well directed movies, not only elicit very strong emotions in viewers, but the resulting fMRI activation patterns are highly similar across subjects (Jääskeläinen et al., 2008, Hasson et al., 2010).

While the idea of using fMRI with naturalistic stimulation paradigms is supported by an increasing number of studies (Hasson et al., 2004, Spiers and Maguire, 2007, Jääskeläinen et al., 2008, Hasson et al., 2010), a crucial problem is that the brain hemodynamic signals measured by fMRI are only indirect reflections of the underlying neuronal activity. The time resolution of fMRI is limited by the sluggishness of the hemodynamic response and the slow sampling rate (typically around 2 s) of whole head echo-planar imaging (EPI). Recently, naturalistic stimulation has also been utilized during magnetoencephalography (MEG) (Betti et al., 2013, Lankinen et al., 2014), which provides direct estimates of neuronal activity with high temporal resolution, with a better spatial precision than its electric counterpart EEG. In slightly more structured designs, MEG studies have also revealed content-specific phase modulations to 30-s audiovisual movie clips on auditory cortices (Luo et al., 2010) and investigated the role of cortical oscillations in segmentation and coding of continuous speech during a binaurally presented 7-min story sample (Gross et al., 2013). There is also evidence of consistent EEG/fMRI activations to luminance changes in repeated two minute video clips (Whittingstall et al., 2010). However, it is still uncertain whether reliable and replicable patterns of MEG/EEG activity can be observed during free viewing of movie stimuli comparable to naturalistic fMRI studies.

Achieving anatomically accurate non-invasive MEG/EEG estimates of brain activations during movie observation is limited by the ill-posed electromagnetic inverse problem. This problem can be mitigated by employing constraints that reduce the potential solution space. The source locations can be restricted to the cortical gray matter, where MEG and EEG signals are generated, based on anatomical MRI data (Dale and Sereno, 1993). Combining the complementary information provided by simultaneously measured MEG and EEG provides additional improvements over using MEG alone (Sharon et al., 2007). Here, we therefore studied the inter-subject synchronization of MEG/EEG activations during free observation of a movie stimulus using a cortically constrained source modeling approach. Physiological noise components such as respiratory and cardiac artifacts were modeled and then removed from the recorded data using a recently developed DRIFTER algorithm (Särkkä et al., 2012), a Bayesian method that allows the tracking of changes in both amplitude and shape in periodic noise and to separate the physiological noise from acquired data.

We hypothesized that the more unimodal sensory areas would demonstrate higher inter-subject synchronization than multisensory/modal association areas. We also expected that brain areas being more predominantly driven by the external stimuli (e.g. sensory areas, bottom-up attention networks (Corbetta and Shulman, 2002)) would show more replicable ISC patterns across separate runs, even though sensory cortices are also modulated by endogenous processes such as attention and anticipation. Therefore, we calculated the inter-subject correlation (ISC) both over the whole duration of the movie (1030 s) and using a sliding window (0.2 s) approach (i.e. dynamic ISC), and also calculated the correlation of dynamic ISC between two runs. The results not only support our hypotheses but also suggest that the ISC will increase when the subjects view the movie for the second time. In addition, the results suggest that it is possible to assess the functionality of a certain cortical regions such as fusiform gyrus and motor cortex based on MEG/EEG data collected during free viewing of a movie.

2. Materials and methods

2.1 Task and data acquisition

Nine subjects (age 18–27 years, 4 females) observed a 17-minute video clip from movie “Crash” (Lionsgate Films, 2004) twice, in two separate runs. Informed consents were obtained from each participant in accordance with the experimental protocol approved by the Massachusetts General Hospital Institutional Review Board. During the movie observation, 306-channel MEG (Elekta-Neuromag, Helsinki, Finland) and 74-channel EEG data were recorded simultaneously in a magnetically shielded room (sampling rate 600 Hz, passband 0.01–192 Hz). The position of the head relative to the MEG sensor array was monitored continuously using four Head-Position Indicator (HPI) coils attached to the scalp. Electro-oculogram (EOG) and electro-cardiogram (ECG) were recorded to monitor eye-blink and cardiac artifacts, respectively.

2.2 Signal preprocessing

External MEG noise was suppressed and subject movements, estimated continuously at 200-ms intervals, were compensated by using the signal-space separation method (Taulu and Simola, 2006) (Maxfilter, Elekta-Neuromag, Helsinki, Finland). The common-average reference was utilized for all analyses of EEG data. The MEG/EEG data were down-sampled (300 samples/s, passband 0.5–100 Hz). Next, we employed DRIFTER algorithm (Särkkä et al., 2012), which is a Bayesian method, to estimate and separate the physiological noise out from the MEG/EEG signal. Unlike the widely used approach RETROICOR (Glover et al., 2000), the DRIFTER is able to track the changes in both amplitude and shape in periodic noise and separate the physiological noise from acquired data. The temporal dynamics of frequency set in DRIFTER for cardiac and respiratory noise are estimated from ECG and EOG respectively. After the DRIFTER decomposed the MEG/EEG signal into activation related brain signal, physiological noise and white noise, the physiological noise was subtracted from the original time course.

To calculate l2 minimum-norm estimates (MNE) (Hämäläinen and Ilmoniemi, 1984), the information from structural segmentation of the individual MRIs and the MEG sensor and EEG electrode locations were used to compute the forward solutions for all source locations in the cortex using a three-compartment boundary element model (Hämäläinen et al., 1993). The shapes of the surfaces separating the scalp, skull, and brain compartments were determined from the anatomical MRI data using FreeSurfer 5.0 (http://surfer.nmr.mgh.harvard.edu/). The individual forward solutions for current dipoles placed at the vertices comprised the columns of the gain matrix. A noise covariance matrix C was estimated from the raw MEG/EEG data during a 4-minute session where the subjects kept their eyes open and rested without performing any task. To perform group analysis, individual raw inverse estimates were morphed onto the Freesurfer standard-brain surface. The points in source space were decimated to 1026 vertices per hemisphere.

This study has utilized an anatomically constrained MNE approach (Dale and Sereno, 1993), which restricts the possible solution to the cerebral gray matter, where a vast majority of recordable MEG/MEG activity is generated, to improve the spatial accuracy of source localization. The source localization accuracy is further improved by the explicit combination of MEG and EEG (Sharon et al., 2007), which provides complementary information of the same neuronal events. Nevertheless, it is good to bear in mind that source localization will likely be more precise in areas such the precentral, postcentral, and superior temporal cortices that are closer to the MEG sensors and EEG electrodes, and localization errors in areas further away from sensors, such as posterior cingulate cortex, are larger (Molins et al., 2008). Thus, one should interpret the source estimates at deep brain cautiously.

2.3 Cortical-label-based analysis

To perform group-level analysis, the individual cortical maps of time series were normalized from native anatomical space to a standard cortical surface space (Dale et al., 1999; Fischl et al., 1999). One of the challenges for analyzing the MEG raw data is the low signal-to-noise ratio (SNR). To increase the SNR, we therefore divided the cortical source space to 34 anatomical labels per hemisphere based on the Freesurfer Desikan-Killiany (Desikan et al., 2006) atlas (Figure 1), and employed PCA to capture the common feature within each regional label and to exclude the noise components. Assuming l source points are in a label, let Ml=[m1, m2, …ml] denote the collection of the time courses and mn be a t-by-1 column vector where n = 1… l and t denote the number of time points. First, the singular value decomposition (SVD) decomposed the t-by-l matrix Ml to be UlSlVlT, where Sl is a diagonal matrix with rank of l. We then selected the largest singular values and the corresponding singular vectors, which retained 20% of the total power, and discarded the rest of the components. After cleaning the time courses using SVD, we averaged them within each of the 34 cortical anatomical labels per hemisphere (Figure 1). All the following data analyses are performed on the resulting label time courses.

Figure 1.

Figure 1

Open in a new tab

Illustration of cortical labels of the Freesurfer Desikan-Killiany atlas (Desikan et al., 2006).

2.4 Inter-subject correlation (ISC) and dynamic inter-subject correlation (dISC)

We first calculated the correlation coefficients between subjects. With the normalized cortical maps of the time series in 9 subjects, we calculated the correlation coefficients between each pair of subjects over the 1030-s time and averaged the 36 pairs of cross-subject correlation coefficients. The dynamic ISC (dISC) was, in turn, utilized to calculate the ISC in a similar way but over a sliding time window of 200 ms at 3.3 ms-steps.

2.5 Between-run ISC

The between-run ISC tested the repetitiveness of inter-subject synchronization across runs. Firstly, we employed Fisher z-transformation (Fisher, 1915) to convert the dynamic ISCs in both of the runs to normally distributed values. Given the two z-transformed time series of dISC, between-run ISC calculated the correlation coefficient over the central 1029.8 s.

2.6 Time-frequency analyses on ISC

Time-frequency analyses were performed using a FFT taper approach with sliding time windows (Percival and Walden, 1993) at frequency bands centered at 6, 8, 10 and 12 Hz, ranging from the theta to the alpha band. An adaptive time-window of 3 cycles and a Hanning taper was used (Blackman and Tukey, 1959).

2.7 Statistical analysis by amplitude adjusted Fourier Transform (AAFT)

The statistical significance was tested using a surrogate data testing method due to the difficulty of modeling the statistical distribution after the multiple preprocessing steps. Surrogate data testing is a non-parametric approach, which does not depend on a particular model but on generating the surrogate data series from original dataset. Specifically, we employed amplitude adjusted Fourier transform (AAFT) method which preserves the autocorrelation function of data (Theiler and Prichard, 1996) to estimate the characteristic of null distribution. AAFT creates the surrogate data by scaling the data to a Gaussian distribution, performing the random phase method (Theiler and Prichard, 1996), and doing an inverse transformation to Gaussian scaling. The AAFT was repeated 100 times to estimate the mean and standard deviation of every type of ISCs (such as dISC, between-run ISC) for each label. Fisher z-transform then converted the correlation coefficients to be Gaussian distributed. With the Fisher z-transformed mean and standard deviation, we converted the correlation maps to z-score maps. All the AAFTs in this study were performed after cortical-label-based analysis. This study focuses only on the positive correlation coefficients.

3. Results

3.1 Within-run inter-subject synchronization analyses

Figures 2a and 2b show the statistical maps for the ISC analyses of the first and second viewing, respectively. Significant correlations were observed in both visual and auditory cortices, as well as in several frontal regions and medial surface. Specifically, the most significant inter-subject synchronizations related to movie watching were found bilaterally in pre-/post-central cortices, superior temporal cortices, and in the posterior cingulate.

Figure 2.

Figure 2

Open in a new tab

Statistical maps of ISCs for the first viewing (Run 1) and second viewing (Run 2). The data show strongest ISCs in sensory areas. Notably, the ISCs are clearly increased during Run 2, also in higher-order association areas such as ACC and prefrontal cortices. The data show the z-scores of ISCs.

We also found that the ISC generally increased during the second vs. first viewing of the movie. The increases of ISC were evident, not only in sensory regions such as somatosensory, motor, auditory and visual cortices, but also on association areas such as the middle frontal gyrus, Broca's area, precuneus, anterior cingulate cortex, and interior parietal cortex.

To assess the validity of our approach, we specifically examined the raw time courses of each subject, and the related dISC patterns, of two well-studied regions during natural viewing of Run 1. Figure 3 shows the results related to the fusiform gyrus, which is well-known to be related to the visual processing of facial stimuli. Three snapshots in the top row are associated with the highest three dISC peaks. Figure 3 also includes the respective label time courses of every subject and the dISC time series around these events. Notably, these data show significant peaks of dISC during movie frames containing faces, consistent with the presumed role of the fusiform gyrus in visual processing. To examine whether the correlations within this area were more selective to faces than other cortical regions, we also examined the 30 movie frames associated with the 30 highest dISC peaks on each of the 34 labeled regions, and counted the number of movie frames that show faces for each label. In general, the timings of the selected dISC peaks are uniformly distributed over the whole 1030-s period. The average number of the frames related with 30 highest dISC containing faces (across the 33 labeled regions, excluding fusiform gyrus) was 20.2 (standard deviation, SD, = 2.4). As for the fusiform gyrus, a considerably higher proportion, for 27 out of the 30 frames related with highest dISCs, included faces (p = 0.0013). This result suggests that the inter-subject synchronization of fusiform gyrus responded more often to face than other cortical regions.

Figure 3.

Figure 3

Open in a new tab

Selective functional preference of the fusiform gyrus. The three columns of figures from left to right correspond to the highest three dISCs. The time courses of each subject, normalized by the standard deviations for better visualization, in the fusiform gyrus are shown in the middle row. Different colors encode the time courses of different subjects. The dISC time courses of are shown at the bottom row, with the red traces indicating the dISCs that are larger than 0.3. The movie snapshots at the time points indicated by the asterisks, which all include faces, are shown at the top row.

The second region that was selected for validity testing was the pre-central cortex, which includes areas thought to belong into the mirror-neuron system that are strongly activated when the subject sees hands that manipulate objects or someone speaking (Rizzolatti and Craighero, 2004). Figure 4 shows the data during and around the movie frames associated with the highest three dISCs in this area in Run 1, all of which show hand-related movement or speaking. To further evaluate the functional selectivity of these dISCs, we also examined the 30 movie frames associated with the highest 30 dISCs in every other cortical label. In the pre-central cortex, 27 frames out of the 30 highest dISCs involved hand-related movement or speaking (p = 0.0147). The average number of the other 33 cortical regions that involve head-related movement or speaking was 20.6 (SD = 2.5).

Figure 4.

Figure 4

Open in a new tab

Selective functional preference of the pre-central sulcus. The three columns of figures from left to right correspond to the highest three correlation coefficients in this area. The time courses of each subject around the time points of these snapshots are shown in the middle row. The bottom row shows the time courses of dISC, with the timing of the respective movie frames shown at the top indicated by the asterisks. Notably, all these movie frames include speaking and/or hand movements.

3.2 Repetitiveness analysis across the two runs

The repetitiveness of inter-subject synchronization during free viewing was analyzed by between-run ISC analyses (Figure 5). Evidence for repeated inter-subject synchronization patterns across Run 1 and Run 2 were found in bilateral auditory cortices, visual cortices, precunei, the inferior parietal cortices, as well as in the posterior cingulate cortices. However, many of the association areas, which showed significant synchronization patterns across subjects within each run (Figure 2), did not show strong between-run correlations. This is consistent with an assumption that areas involved with bottom-up processing of stimulus properties or stimulus-driven orienting, would show better repetitiveness that are temporally more consistent across runs than those in areas associated with endogenous top-down control.

Figure 5.

Figure 5

Open in a new tab

The statistical maps of between-run ISC. The z scores are color-coded as shown in the color bar.

3.3 Spectral analyses

The statistical ISCs in different frequency bands are shown in Figure 6. The time-frequency analysis was performed on the data of Run 1. In general, the inter-subject synchronization was stronger around theta band (6–8 Hz) than around alpha band (8–12 Hz).

Figure 6.

Figure 6

Open in a new tab

Statistical maps of ISCs at frequency bands of 6, 8, 10 and 12 Hz during Run 1. The color scale shows the z scores.

3.4 Artifact removal using DRIFTER

Our results demonstrate that, by using the DRIFTER algorithm, physiological artifacts including muscular, ocular, and cardiac artifacts can be significantly reduced in MEG/EEG time series. This is especially important for MEG/EEG studies using naturalistic stimuli, because the individual data points that may contain essential information cannot, necessarily, be rejected together with physiological artifacts. An example of raw data segments in an individual subject before and after DRIFTER is shown in Figure 7.

Figure 7.

Figure 7

Open in a new tab

The demonstration of artifact removal by DRIFTER. The right panel shows the original MEG, EEG and EOG/ECG signals. The blue traces indicate the signal of magnetometer, red traces indicate gradiometer, green traces indicate EEG and gray traces indicate EOG/ECG. As demonstrated by this analysis, after the application of DRIFTER the artifacts induced by cardiac and eye blinks are largely suppressed.

Figure 8a shows quantitative analyses of the effect of DRIRTER, which compare the correlations between MEG/EEG time courses and horizontal EOG (hEOG), vertical EOG (vEOG) and ECG before and after the application of DRIFTER. The data show that the correlation coefficients between gradiometers/magnetometers/EEG and hEOG/vEOG are significantly reduced (p<0.001) after DRIFTER. The results also suggested that ECG produces significantly less interference on MEG/EEG than EOG does. Additionally, Figure 8b shows the cortical map of ISC before and after DRIFTER. The spatial pattern of ISC without DRIFTER is more diffuse and is quite different from that with DRIFTER. The results suggest that the artifacts induced by the EOG and ECG indeed affect the ISC.

Figure 8.

Figure 8

Open in a new tab

(a) The bar plots that demonstrate the quantified effect of DRIFTER. The correlation coefficients are calculated between gradiometers/magnetometers/EEG and hEOG/vEOG/ECG over the entire movie duration (1030 s) and then averaged across channels. Significant reductions of correlations are shown between both hEOG and vEOG derivations and MEG/EEG data. The error bars indicate the standard deviations across the gradiometer, magnetometer, and EEG channels, respectively (* p<0.05, ** p<0.001). (b) The cortical ISC maps demonstrating the effect of DRIFTER. ISC is reduced in frontal areas where synchronous eye-movements during movie-watching are likely to cause artifactual ISC due to synchronous electro-ocular artifacts being picked up by MEG/EEG sensors sensitive to currents in these brain areas.

4. Discussion

Our results suggest that reliable patterns of MEG/EEG activity can be observed during free movie viewing. The brain areas showing most consistent temporal patterns across subjects were consistent with those observed during previous fMRI studies (Jääskeläinen et al., 2008, Hasson et al., 2010), including those using a longer segment of exactly the same movie as stimulus (Jääskeläinen et al., 2008). The most of significant ISCs were found in visual (occipital, inferior temporal) and auditory cortices, as well as posterior cingulate and right frontal areas. The findings are in line with previous MEG and EEG/fMRI studies that have utilized more structured experimental designs, containing shorter and repetitive movie clips (Luo et al., 2010, Whittingstall et al., 2010). Evidence for inter-subject correlations in sensory areas during movie stimulation was also found in a very recent MEG study utilizing multi-set canonical correlation analyses (Lankinen et al., 2014).

The within-run ISCs generally increased during the second viewing of the movie, not only in predominantly unimodal sensory regions but also in association regions. Tentatively, it is possible that anticipation of the flow of events and stimuli increases during the second viewing of the movie clip, thus resulting in more similar timing of MEG/EEG activity. In the case of fMRI, there is some evidence suggesting that ISC is slightly lower during repeated viewing (Lahnakoski et al., 2014). While definite answers cannot be provided based on the current dataset, this discrepancy in findings might be explained by the ISC analysis suffering less from imprecisions in timing of brain responses. In the case of fMRI, the timing imprecision comes from much slower hemodynamics than in case of temporally highly accurate MEG/EEG. Thus, increased precision of response timings might be offset by possible fatigue effects in case of fMRI, resulting in tendency towards slightly lower ISC upon repeated viewing. There is also one previous EEG study that reported reduction in ISC during second viewing of a movie clip (Dmochowski et al., 2012). In the case of this previous EEG study, the correlation window used in the study was 5s and the sliding step was 1s. According to our study of dISC, however, the inter-subject synchronization may arise and subside within 400 ms (as shown in Figure 3). Hence, the ISC analysis in the EEG study may not contain all the available dynamic information.

However, the between-run ISC shown in Figure 5 indicate that, despite the increase of ISCs during second viewing, the subject might have synchronized their brain activities to different events during the second than the first viewing. For example, the between-run ISCs on pre- and post-central gyrus were not significant, although the ISCs were significant in the first viewing and the significance became even stronger in second time viewing. A similar phenomenon can be observed, for example, in the supramarginal, fusiform, and parahippocampal gyri. Our working hypothesis for future studies is that these effects are, analogously to the changes in ISC between Run 1 and 2, related anticipation effects that may have specifically affected time courses in association areas related to endogenous processing. These anticipation effects may have made the activation time courses different across runs, thus decreasing between-run ISCs, although the similarity between subjects within each run seemed to increase (thus increasing ISCs within each run after repetition). Such effects could have been further emphasized due to the high temporal resolution of our MEG/EEG estimates, which is several orders of magnitude more precise than that of fMRI.

Areas that showed similar inter-subject synchronization patterns across runs included regions such as the superior temporal gyrus (i.e., auditory cortex), pericalcarine cortex (i.e., primary visual cortex), and posterior cingulate cortex. Of these areas, particularly the two first ones are directly driven by external inputs. At the same time, previous fMRI studies suggest that the posterior cingulate cortex, which also showed evidence for significantly replicable activation patterns across runs, is particularly strongly activated by stimulus-driven visuospatial orienting (Hahn et al., 2007). We therefore speculate that these areas were might have been less significantly affected by the endogenous/anticipatory modulations related to movie repetition than areas related to endogenous processing. Further studies are needed to verify these interpretations.

The dISC demonstrated that the cortical-label-based MEG/EEG data were able to provide dynamic information of inter-subject synchronization with sliding time window of 200 ms. The reliability of dISC was examined on the well-studied areas – fusiform gyrus and pre-central gyrus. Both of the fusiform and pre-central gyrus showed significant functional specificity to face and human motion respectively.

The inter-subject synchronization was stronger in low (6–8 Hz) than high frequency bands (10–12Hz). It is possible that this effect is explainable by underlying event-related activities to individual movie events, such as slower "late" event-related components including the N2/P3 complex, which tend to be stronger, last longer, and have a better SNR at the theta than alpha band. At the same time, higher ISCs in low-frequency band may simply come from higher tolerance toward subtle timing difference across subjects. For higher frequency band oscillations, a slightly higher degree of synchrony might be needed in responses to stimulus or events to have the ISCs high. Although the statistical approach of AAFT counteracts slight temporal differences across the subjects to a certain degree, the frequency-dependent effect may not be suppressed completely.

The DRIFTER program was originally developed for the removal of physiological noise in fMRI. This study provides the evidence that DRIFTER is able to suppress the physiological noise in MEG/EEG significantly. In addition, the cortical maps of ISC become less diffuse after DRIFTER is applied. Specifically, ISC is reduced in frontal areas where synchronous eye-movements during movie-watching are likely to cause artifactual ISC due to synchronous electro-ocular artifacts being picked up by MEG/EEG sensors sensitive to currents in these brain areas. The pronounced difference between the cortical maps of ISC without and with DRIFTER suggests that the artifact removal by DRIFTER is beneficial.

In conclusion, to our knowledge, this is the first time that simultaneous MEG/EEG and cortical constrained inverse current sources were used in a movie study. Our results show that reliable patterns of MEG/EEG activity, consistent with previous fMRI results, can be measured during naturalistic observation.

Highlights.

PCA were employed to extract the temporal characteristics within cortical labels.

The ISCs generally increased during the second viewing of the movie

The dynamic ISC demonstrated significant functional specificity on well-studied areas

Similar temporal patterns of dISC across runs were mainly on primary sensory regions

Physiological MEG/EEG artefacts can be controlled with DRIFTER

ACKNOWLEGEMENTS

We thank Nao Suzuki and Drs. Seppo Ahlfors, Naoaki Tanaka, and Matti Hämäläinen for advice and support. This work was supported by National Institutes of Health (NIH) Grants R01MH083744, R21DC014134, R01HD040712, and R01NS037462. The research was carried out at the Athinoula A. Martinos Center for Biomedical Imaging at the Massachusetts General Hospital, using resources provided by the Center for Functional Neuroimaging Technologies, P41EB015896, a P41 Biotechnology Resource Grant supported by the National Institute of Biomedical Imaging and Bioengineering (NIBIB). The research environment was additionally supported by National Center for Research Resources Shared Instrumentation Grants S10RR014978, S10RR021110, S10RR019307, S10RR014798, and S10RR023401. Author I.P.J. was supported by the Academy Of Finland grant #138145. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

I have no conflicts of interest to disclose with regard to the subject matter of this manuscript.

REFERENCES

  1. Bartels A, Zeki S. Functional brain mapping during free viewing of natural scenes. Hum Brain Mapp. 2004;21:75–85. doi: 10.1002/hbm.10153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Betti V, Della Penna S, de Pasquale F, Mantini D, Marzetti L, Romani GL, Corbetta M. Natural scenes viewing alters the dynamics of functional connectivity in the human brain. Neuron. 2013;79:782–797. doi: 10.1016/j.neuron.2013.06.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Blackman RB, Tukey JW. The measurement of power spectra, from the point of view of communications engineering. New York: Dover Publications; 1959. [Google Scholar]
  4. Corbetta M, Shulman GL. Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci. 2002;3:201–215. doi: 10.1038/nrn755. [DOI] [PubMed] [Google Scholar]
  5. Dale A, Sereno M. Improved localization of cortical activity by combining EEG and MEG with MRI cortical surface reconstruction: A linear approach. J Cog Neurosci. 1993;5:162–176. doi: 10.1162/jocn.1993.5.2.162. [DOI] [PubMed] [Google Scholar]
  6. Desikan R, Segonne F, Fischl B, Quinn B, Dickerson B, Blacker D, Buckner R, Dale A, Maguire R, Hyman B, Albert M, Killiany R. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage. 2006;31:968–980. doi: 10.1016/j.neuroimage.2006.01.021. [DOI] [PubMed] [Google Scholar]
  7. Dmochowski JP, Sajda P, Dias J, Parra LC. Correlated components of ongoing EEG point to emotionally laden attention - a possible marker of engagement? Front Hum Neurosci. 2012;6:112. doi: 10.3389/fnhum.2012.00112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Fisher RA. Frequency distribution of the values of the correlation coefficient in samples of an indefinitely large population. Biometrika Trust. 1915;10:507–521. [Google Scholar]
  9. Glover GH, Li TQ, Ress D. Image-based method for retrospective correction of physiological motion effects in fMRI: RETROICOR. Magn Reson Med. 2000;44:162–167. doi: 10.1002/1522-2594(200007)44:1<162::aid-mrm23>3.0.co;2-e. [DOI] [PubMed] [Google Scholar]
  10. Gross J, Hoogenboom N, Thut G, Schyns P, Panzeri S, Belin P, Garrod S. Speech rhythms and multiplexed oscillatory sensory coding in the human brain. PLoS Biol. 2013;11:e1001752. doi: 10.1371/journal.pbio.1001752. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hahn B, Ross TJ, Stein EA. Cingulate activation increases dynamically with response speed under stimulus unpredictability. Cereb Cortex. 2007;17:1664–1671. doi: 10.1093/cercor/bhl075. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hämäläinen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounasmaa OV. Magnetoencephalography - Theory, Instrumentation, and Applications to Noninvasive Studies of the Working Human Brain. Reviews of Modern Physics. 1993;65:413–497. [Google Scholar]
  13. Hämäläinen M, Ilmoniemi R. Interpreting measured magnetic fields of the brain: estimates of current distributions. Helsinki, Finland: Helsinki University of Technology; 1984. [Google Scholar]
  14. Hasson U, Malach R, Heeger DJ. Reliability of cortical activity during natural stimulation. Trends Cogn Sci. 2010;14:40–48. doi: 10.1016/j.tics.2009.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R. Intersubject synchronization of cortical activity during natural vision. Science. 2004;303:1634–1640. doi: 10.1126/science.1089506. [DOI] [PubMed] [Google Scholar]
  16. Jääskeläinen IP, Koskentalo K, Balk MH, Autti T, Kauramaki J, Pomren C, Sams M. Inter-subject synchronization of prefrontal cortex hemodynamic activity during natural viewing. Open Neuroimag J. 2008;2:14–19. doi: 10.2174/1874440000802010014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Lahnakoski JM, Glerean E, Jaaskelainen IP, Hyona J, Hari R, Sams M, Nummenmaa L. Synchronous brain activity across individuals underlies shared psychological perspectives. Neuroimage. 2014;100:316–324. doi: 10.1016/j.neuroimage.2014.06.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Lahnakoski JM, Glerean E, Salmi J, Jaaskelainen IP, Sams M, Hari R, Nummenmaa L. Naturalistic FMRI mapping reveals superior temporal sulcus as the hub for the distributed brain network for social perception. Front Hum Neurosci. 2012;6:233. doi: 10.3389/fnhum.2012.00233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Lankinen K, Saari J, Hari R, Koskinen M. Intersubject consistency of cortical MEG signals during movie viewing. Neuroimage. 2014;92C:217–224. doi: 10.1016/j.neuroimage.2014.02.004. [DOI] [PubMed] [Google Scholar]
  20. Luo H, Liu Z, Poeppel D. Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation. PLoS Biol. 8:e1000445. doi: 10.1371/journal.pbio.1000445. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Molins A, Stufflebeam SM, Brown EN, Hamalainen MS. Quantification of the benefit from integrating MEG and EEG data in minimum l2-norm estimation. Neuroimage. 2008;42:1069–1077. doi: 10.1016/j.neuroimage.2008.05.064. [DOI] [PubMed] [Google Scholar]
  22. Nummenmaa L, Glerean E, Viinikainen M, Jaaskelainen IP, Hari R, Sams M. Emotions promote social interaction by synchronizing brain activity across individuals. Proc Natl Acad Sci U S A. 2012;109:9599–9604. doi: 10.1073/pnas.1206095109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Percival DB, Walden AT. Spectral analysis for physical applications : multitaper and conventional univariate techniques. Cambridge ; New York, NY, USA: Cambridge University Press; 1993. [Google Scholar]
  24. Rizzolatti G, Craighero L. The mirror-neuron system. Annu Rev Neurosci. 2004;27:169–192. doi: 10.1146/annurev.neuro.27.070203.144230. [DOI] [PubMed] [Google Scholar]
  25. Särkkä S, Solin A, Nummenmaa A, Vehtari A, Auranen T, Vanni S, Lin FH. Dynamic retrospective filtering of physiological noise in BOLD fMRI: DRIFTER. Neuroimage. 2012;60:1517–1527. doi: 10.1016/j.neuroimage.2012.01.067. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Sharon D, Hamalainen MS, Tootell RB, Halgren E, Belliveau JW. The advantage of combining MEG and EEG: comparison to fMRI in focally stimulated visual cortex. Neuroimage. 2007;36:1225–1235. doi: 10.1016/j.neuroimage.2007.03.066. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Spiers HJ, Maguire EA. Decoding human brain activity during real-world experiences. Trends Cogn Sci. 2007;11:356–365. doi: 10.1016/j.tics.2007.06.002. [DOI] [PubMed] [Google Scholar]
  28. Taulu S, Simola J. Spatiotemporal signal space separation method for rejecting nearby interference in MEG measurements. Physics in medicine and biology. 2006;51:1759–1768. doi: 10.1088/0031-9155/51/7/008. [DOI] [PubMed] [Google Scholar]
  29. Theiler J, Prichard D. Constrained-realization Monte-Carlo method for hypothesis testing. Physica D Nonlinear Phenomena. 1996;94:221–235. [Google Scholar]
  30. Westermann R, Spies K, Stahl G, Hesse FW. Relative effectiveness and validity of mood induction procedures: A meta-analysis. Eur J Soc Psychol. 1996;26:557–580. [Google Scholar]
  31. Whittingstall K, Bartels A, Singh V, Kwon S, Logothetis NK. Integration of EEG source imaging and fMRI during continuous viewing of natural movies. Magn Reson Imaging. 2010;28:1135–1142. doi: 10.1016/j.mri.2010.03.042. [DOI] [PubMed] [Google Scholar]

RESOURCES