The integration of information between functionally specialized and widely distributed brain regions (i.e., connectivity) is fundamentally important for cognition (Bressler and Menon, 2010), and can be estimated using electroencephalography (EEG) (Horwitz, 2003). However, the interpretation of connectivity measures from sensor-level EEG recordings is not straightforward. Instead, the neuroanatomical sources of recorded sensor-level data can be reconstructed by first finding the scalp potentials that result from hypothetical current distributions inside the head (i.e., the forward problem), then applying this to the actual EEG data to estimate back the sources that fit the measurements (i.e., the inverse problem). Several publicly available software packages have been developed for analysis of source-level EEG data. There exist many different models for solving the forward and inverse problems, and these can be implemented using several analysis packages. For source-level EEG to be a reliable tool for measuring connectivity, source estimates should be consistent across these different combinations of commonly used model parameters and software packages. If not, this would have important implications for reproducibility of EEG studies reconstructing brain activation and connectivity from neuronal sources.
Recently, Mahjoory et al. (2017) investigated the consistency of source localization and connectivity estimates across a host of widely used analysis pipelines. They tested 14 pipelines consisting of different combinations of (1) software packages including Brainstorm, FieldTrip and Berlin Toolbox; (2) inverse models including weighted minimum-norm estimate (WMNE), exact low resolution electromagnetic tomography (eLORETA) and linearly constrained minimum-variance (LCMV); and (3) forward models including boundary element, finite element and spherical harmonics expansions methods. Resting EEG data (eyes closed) was recorded from 65 healthy subjects as part of two different experiments; one on attentional processes (Fasor data) and the other as part of a brain-computer interface study (Würzburg data). In the latter, two sessions on separate days were conducted per subject. Source localization and connectivity estimates were computed for alpha-band (8–13 Hz) oscillations. Connectivity measures included imaginary coherence (Nolte et al., 2004), reflecting temporal correlations of neural activity between brain areas (i.e., functional connectivity), and phase slope index (Nolte et al., 2008), reflecting directed interactions (i.e. effective connectivity). To evaluate consistency across pipelines they computed grand-average correlations between localization or connectivity results for all pairs of pipelines. To evaluate between-study consistency they computed grand-average correlations between results of the two experiments for each pipeline. To evaluate within-subjects consistency they computed correlation between the two sessions of the Würzburg experiment for each subject and pipeline then averaged across subjects. To evaluate between-subjects consistency they computed correlations between datasets of distinct subjects separately for the Fasor and Würzburg experiments then averaged across pairs of subjects.
Mahjoory et al. (2017) showed that: connectivity patterns between EEG electrodes vary depending on the choice of electrical reference, supporting the use of source reconstruction for connectivity analyses. Source localization estimates, mapped onto cortical surface, have a smaller maximum and are more focally concentrated in the occipital region when LCMV is used than when eLORETA or WMNE is used. Patterns of interactions estimated from the reconstructed sources vary when different inverse methods are used. Across all pipelines source localization estimates are more consistent than functional connectivity estimates, followed by effective connectivity estimates. Average correlation across different combinations of forward models is higher than when varying inverse methods or software packages. For source localization and connectivity, correlation between WMNE and eLORETA based estimates exceeds correlation between LCMV and either eLORETA or WMNE based estimates. Source localization and connectivity results are most consistent between-studies, followed by within-subjects and then between-subjects. Between-study, within-subjects and between-subjects consistencies are highest for source localization estimates followed by functional and then effective connectivity estimates. Between-study, within-subjects and between-subjects consistencies, computed at the sensor-level, are in general similar to the average source-level results.
There are several important issues to bear in mind when considering the consistency of EEG measures in the source-space, not least of which is the physiological state of participants at the time of recording. The resting condition is a challenging state for source-level analysis, and likely represents a major source of inconsistency in the source localization and connectivity outcomes. Related to this is the arousal state of the subjects. This has been shown to affect physiological significance of source localization and alter connectivity estimates using various methods and modalities (Kaufmann et al., 2005; Massimini et al., 2005; Murphy et al., 2009; Ventouras et al., 2010). Therefore particularly important for within-subjects and between-subjects analysis, variability in the subjects' arousal state likely account for some of the variability in the estimates.
Based on the findings of Mahjoory et al. (2017), we highlight the importance of utilizing a priori assumptions based on physiological information determined using other modalities. Magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) are non-invasive recording techniques used to study human brain activity. It is suggested that combining EEG with these modalities may produce more accurate source localization estimates than either modality alone (Liu et al., 2002; Groening et al., 2009), presumably improving the consistency of source localization and connectivity estimates. Additionally, transcranial magnetic stimulation (TMS) is a technique that enables non-invasive manipulation of neural activity. TMS combined with EEG can be used to provide priors for EEG connectivity analysis and validate the connectivity results (Bortoletto et al., 2015).
There are several limitations of Mahjoory et al. (2017), many of which were addressed in the original work. However, one additional point to consider is the statistical approach for assessing consistency. In Mahjoory et al. (2017) consistency is quantified by reporting Pearson correlation coefficient. However, other measures may be suited to estimate consistency, such as intraclass correlation coefficient or the standard error of measurement (Bédard et al., 2000).
Mahjoory et al. (2017) presented the first comprehensive assessment of consistency of EEG source localization and connectivity estimates across widely used forward and inverse methods. Their study is an important contribution toward a consensus about source-level methodologies. Their findings highlight the need to take caution when interpreting source-level outcomes, particularly in clinical settings. However, this should not discourage researchers from studying EEG recordings in the source-space.
Author contributions
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
Conflict of interest statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Acknowledgments
The authors would like to thank Michael C Ridding for valuable feedback. BM would like to thank Luke Hallam for helpful discussions.
Footnotes
Funding. MG is supported by an NHMRC-ARC Dementia Research Development Fellowship (1102272).
References
- Bédard M., Martin N. J., Krueger P., Brazil K. (2000). Assessing reproducibility of data obtained with instruments based on continuous measurements. Exp. Aging Res. 26, 353–365. 10.1080/036107300750015741 [DOI] [PubMed] [Google Scholar]
- Bortoletto M., Veniero D., Thut G., Miniussi C. (2015). The contribution of TMS–EEG coregistration in the exploration of the human cortical connectome. Neurosci. Biobehav. Rev. 49, 114–124. 10.1016/j.neubiorev.2014.12.014 [DOI] [PubMed] [Google Scholar]
- Bressler S. L., Menon V. (2010). Large-scale brain networks in cognition: emerging methods and principles. Trends Cogn. Sci. 14, 277–290. 10.1016/j.tics.2010.04.004 [DOI] [PubMed] [Google Scholar]
- Ferri R., Rundo F., Bruni O., Terzano M. G., Stam C. J. (2007). Small-world network organization of functional connectivity of EEG slow-wave activity during sleep. Clin. Neurophysiol. 118, 449–456. 10.1016/j.clinph.2006.10.021 [DOI] [PubMed] [Google Scholar]
- Groening K., Brodbeck V., Moeller F., Wolff S., van Baalen A., Michel C. M., et al. (2009). Combination of EEG–fMRI and EEG source analysis improves interpretation of spike-associated activation networks in paediatric pharmacoresistant focal epilepsies. Neuroimage 46, 827–833. 10.1016/j.neuroimage.2009.02.026 [DOI] [PubMed] [Google Scholar]
- Horwitz B. (2003). The elusive concept of brain connectivity. Neuroimage 19, 466–470. 10.1016/S1053-8119(03)00112-5 [DOI] [PubMed] [Google Scholar]
- Kaufmann C., Wehrle R., Wetter T. C., Holsboer F., Auer D. P., Pollmächer T., et al. (2005). Brain activation and hypothalamic functional connectivity during human non-rapid eye movement sleep: an EEG/fMRI study. Brain 129, 655–667. 10.1093/brain/awh686 [DOI] [PubMed] [Google Scholar]
- Liu A. K., Dale A. M., Belliveau J. W. (2002). Monte Carlo simulation studies of EEG and MEG localization accuracy. Hum. Brain Mapp. 16, 47–62. 10.1002/hbm.10024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mahjoory K., Nikulin V. V., Botrel L., Linkenkaer-Hansen K., Fato M. M., Haufe S. (2017). Consistency of EEG source localization and connectivity estimates. Neuroimage 152, 590–601. 10.1016/j.neuroimage.2017.02.076 [DOI] [PubMed] [Google Scholar]
- Massimini M., Ferrarelli F., Huber R., Esser S. K., Singh H., Tononi G. (2005). Breakdown of cortical effective connectivity during sleep. Science 309, 2228–2232. 10.1126/science.1117256 [DOI] [PubMed] [Google Scholar]
- Murphy M., Riedner B. A., Huber R., Massimini M., Ferrarelli F., Tononi G. (2009). Source modeling sleep slow waves. Proc. Natl. Acad. Sci. U.S.A. 106, 1608–1613. 10.1073/pnas.0807933106 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Nolte G., Bai O., Wheaton L., Mari Z., Vorbach S., Hallett M. (2004). Identifying true brain interaction from EEG data using the imaginary part of coherency. Clin. Neurophysiol. 115, 2292–2307. 10.1016/j.clinph.2004.04.029 [DOI] [PubMed] [Google Scholar]
- Nolte G., Ziehe A., Nikulin V. V., Schlögl A., Krämer N., Brismar T., et al. (2008). Robustly estimating the flow direction of information in complex physical systems. Phys. Rev. Lett. 100:234101. 10.1103/PhysRevLett.100.234101 [DOI] [PubMed] [Google Scholar]
- Ventouras E. M., Ktonas P. Y., Tsekou H., Paparrigopoulos T., Kalatzis I., Soldatos C. R. (2010). Independent component analysis for source localization of EEG sleep spindle components. Comput. Intell. Neurosci. 2010:329436 10.1155/2010/329436 [DOI] [PMC free article] [PubMed] [Google Scholar]