Multimodal Approaches to Functional Connectivity in Autism Spectrum Disorders: An Integrative Perspective

Abstract

Atypical functional connectivity has been implicated in autism spectrum disorders (ASDs). However, the literature to date has been largely inconsistent, with mixed and conflicting reports of hypo- and hyper-connectivity. These discrepancies are partly due to differences between various neuroimaging modalities. Functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG) measure distinct indices of functional connectivity (e.g., blood-oxygenation level-dependent [BOLD] signal vs. electrical activity). Furthermore, each method has unique benefits and disadvantages with respect to spatial and temporal resolution, vulnerability to specific artifacts, and practical implementation. Thus far, functional connectivity research on ASDs has remained almost exclusively unimodal; therefore, interpreting findings across modalities remains a challenge. Multimodal integration of fMRI, EEG, and MEG data is critical in resolving discrepancies in the literature, and working toward a unifying framework for interpreting past and future findings. The current review aims to provide a theoretical foundation for future multimodal research on ASDs. First, we will discuss the merits and shortcomings of several popular theories in ASD functional connectivity research, using examples from the literature to date. Next, the neurophysiological relationships between imaging modalities, including their relationship with invasive neural recordings, will be reviewed. Finally, methodological approaches to multimodal data integration will be presented, and their future application to ASDs will be discussed. Analyses relating transient patterns of neural activity (“states”) are particularly promising. This strategy provides a comparable measure across modalities, captures complex spatiotemporal patterns, and is a natural extension of recent dynamic fMRI research in ASDs.

Introduction

Autism spectrum disorders (ASDs) are neurodevelopmental disorders characterized by deficits in social communication and restricted or repetitive behaviors (American Psychiatric Association, 2013). These symptoms typically emerge within the first few years of life, and persist throughout the lifespan. ASDs are a major research priority, as the Centers for Disease Control and Prevention estimate that 1 in 45 school-aged children in the United States are affected (Zablotsky et al., 2015). However, despite extensive neuroimaging research over the past decades, the brain bases of ASDs are still not well understood.

Neuroimaging studies of this population have taken various approaches, both structural (e.g., anatomical MRI, diffusion weighted imaging) and functional (e.g., functional MRI, electroencephalography, magnetoencephalography). The imaging literature broadly indicates that ASDs are characterized by altered connectivity within and between brain networks (Di Martino et al., 2014). However, reproducible biomarkers of ASDs have yet to be identified. This is likely, in part, due to the considerable heterogeneity across individuals on the autism spectrum. Phenotypic expression and outcome vary widely (Olsson et al., 2015) and etiologically distinct variants of ASDs are probable (Geschwind and State, 2015). However, another major source of variability is methodological. Despite obvious advantages of using multiple neuroimaging methods, different types of data have distinct neural bases. Failure to reconcile these differences has thwarted attempts to synthesize multimodal data into a coherent narrative describing atypical functional connectivity in ASDs.

The concept of ‘connectivity’, though widely used, is ill-defined and has been invoked in studies that test divergent neural phenomena (Horwitz, 2003). Unsurprisingly, different measures of connectivity can show poor correspondence (Reid et al., 2016). Therefore, it is essential to carefully define connectivity in the context of a given study, and to evaluate methodological choices accordingly. Broadly, ‘connectivity’ can be structural (i.e., physical properties of neuronal connections), functional (i.e., statistical relationship between activity in two or more regions), or effective (i.e., causal relationship between regions). With each of these approaches, connectivity can be further studied at different levels of granularity. For example, structural connectivity can refer to large axonal tracts, such as the inferior longitudinal fasciculus (detectable in diffusion weighted imaging), or to individual synapses connecting two neurons (detectable in microscopy). Structural properties influence functional relationships, but do not entirely account for them (Damoiseaux and Greicius, 2009). Both structural and functional connectivity have been studied extensively in ASDs; however, the current review will focus on functional connectivity, with an emphasis on multimodal integration.

The most commonly used measures of functional connectivity include functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and magnetoencephalography (MEG). Importantly, each modality measures (with specific limitations) a distinct physiological signal (see Figure 1), requiring uniquely suitable interpretation. EEG and MEG directly measure summated post-synaptic potentials (PSPs), arising mostly from pyramidal neurons. Sensors placed at the scalp measure extracellular electrical activity in EEG (Niedermeyer et al., 2011), and magnetic fields generated by intracellular currents in MEG (Supek and Aine, 2014). Both provide high temporal resolution, capturing neural oscillations across a broad frequency range (Buzsaki, 2006; see Table 1). Within given frequency bands, connectivity is typically measured in terms of power (a measure of voltage calculated from amplitude, often following time-frequency decomposition of waveforms), or coherence (phase-coupling across distant electrodes; for a review of coherence approaches see Bowyer, 2016). However, the spatial resolution of these methods is limited, particularly for deep brain structures. Additionally, the signal at the scalp represents the summation of signal from many sources, and skull and tissue conductivity may cause EEG signals to blur across sensors (van den Broek et al., 1998). Short-range coherence is particularly susceptible to increase from volume conductance, although this effect is considerably reduced in widely separated electrodes and can be minimized by computational methods such as a surface Laplacian spatial filter (Srinivasan et al., 2007).

Figure 1.

Illustration of key features of different imaging modalities, and relationships between multimodal signals. The top panel compares advantages and limitations of intracranial recordings, EEG/MEG, and fMRI. The bottom panel summarizes major findings linking the BOLD signal to the physiological signals measured by intracranial recordings and EEG/MEG, respectively. See also references to multiple reviews in the main text for more exhaustive discussion of specific topics beyond the scope of the current paper. (+) and (-) indicate positive and negative correlations.

Table 1.

EEG/MEG Rhythm Summary

Rhythms	Range (Hz)	Proposed Functional Relevance	Reviews
Delta	1 – 4	Deep sleep; signal matching; decision-making	Harmony (2013)
Theta	4 – 8	Drowsiness and sleep; selective attention; orienting; episodic memory	Basar et al. (1999) Nyhus and Curran (2010)
Alpha	8 – 13	Relaxed wakefulness; semantic memory; visual processing	Basar et al. (1997) Klimesch et al. (2011)
Beta	13 – 30	Focused attention; state maintenance; motor inhibition	Engel and Fries (2010)
Gamma	> 30	Memory; language; visual awareness; perceptual binding; sensory stimulation	Uhlhaas et al. (2011) Basar et al. (1999)

^†Summary of each standard EEG/MEG rhythms, including frequency range and functional relevance. Frequency ranges provided here may serve as a reasonable estimate for each rhythm, but the specific limits of each frequency band vary across studies. Functional relevance of each rhythm has been considerably simplified for the purpose of this paper; we present only the most commonly reported findings as a reference for readers unfamiliar with the EEG/MEG literature.

Localizing subcortical structures is similarly challenging in MEG, not due to conductivity, but because of the rapidly diminishing strength of magnetic fields as a function of distance (Hillebrand and Barnes, 2002). Another limitation of MEG is its failure to detect radial currents at the crests of gyri; this method is selectively sensitive to tangential currents, generated by pyramidal neurons in sulci. However, as only ~5% of cortical area is inaccessible to MEG for this reason, magnetic field strength is thought to be the primary limitation of MEG (Hillebrand and Barnes, 2002). Additionally, as in EEG volume conductance, magnetic field spread due to the distance separating coils from the scalp surface introduces artefact into coherence measures in MEG (Winter et al., 2007). This effect on MEG measures of coherence appears to be negligible in sensors separated by 20 cm or more.

In fMRI, the blood oxygenation level-dependent (BOLD) signal is a hemodynamic proxy for neural activity. The BOLD signal is an indirect measure of neural activity, but provides high spatial resolution. Unlike EEG and MEG, fMRI can detect signals equally from cerebral cortex and deep brain structures. Therefore, fMRI is useful for identifying both localized brain activity and correlations among functionally related areas at rest (Biswal et al., 1995; Fox and Raichle, 2007). Temporal resolution, however, is limited by low sampling rates (typically once every 2 seconds) and the sluggishness of the hemodynamic response, which peaks about 5 seconds after corresponding neuronal activity changes. BOLD correlations are therefore commonly detected in frequency bands below .1 Hz (Cordes et al., 2000), although functional networks may also be detected at slightly higher frequencies (Gohel and Biswal, 2015).

The relative strengths and limitations of each single technique underscore the necessity of multimodal approaches for a comprehensive characterization of neural network organization and dynamics. For example, fMRI is heavily impacted by physiological noise (e.g., heartbeat, respiration) and head motion (Liu, 2016), while EEG is more vulnerable to electric noise from the environment and muscle artifacts (e.g., eye blinks, jaw movement; Delorme et al., 2007). A spatial map derived from fMRI contains little information about dynamic aspects of brain activity; EEG and MEG reveal a wealth of complementary information in the time domain, capturing oscillatory activity inaccessible to fMRI (Buzsaki and Draguhn, 2004; Cohen, 2011). Advanced multimodal analyses take advantage of this, building integrated models representing interactions among network features observed through different modalities (Lei et al., 2011a).

The remainder of this paper will describe directions for an integrated, multimodal approach to studying functional connectivity in ASDs. First, fMRI and EEG/MEG findings from the ASD literature will be briefly discussed. A comprehensive review is beyond the scope of this paper, but seminal findings and inconsistencies that warrant novel, integrative methods will be highlighted. Next, the neurobiological bases of fMRI and EEG/MEG signals will be reviewed, including multimodal studies from the human and animal literature describing the relationship between them. Finally, different multimodal data fusion strategies will be introduced in the context of potential application to the functional connectivity literature on ASDs. We hope that these key points will serve as a framework for the integration of existing and future neuroimaging work, based in a nuanced understanding of methodological strengths and limitations.

Functional Connectivity in ASD

Despite the large and fast-growing number of functional connectivity studies on ASDs, little consensus on atypical functional network patterns in this disorder has emerged. This section will present several popular models that have been proposed to reconcile discrepancies in the literature, and briefly evaluate the evidence supporting each of these. Functional connectivity in ASDs has been comprehensively described in several reviews (Maximo et al., 2014; Mohammad-Rezazadeh et al., 2016; Vissers et al., 2012; Wass, 2011), in addition to modality-specific reviews focusing on fMRI (Hull et al., 2016; Müller et al., 2011; Rane et al., 2015) and EEG/MEG studies (O’Reilly et al., 2017). Rather than fully reiterate this literature, this paper aims solely to underscore the often perplexing heterogeneity of findings, and the inadequacy of current approaches to resolve apparent inconsistencies.

General Underconnectivity

Early functional connectivity findings in ASDs prompted the hypothesis of general underconnectivity within distributed networks. This theory was initially based on reduced BOLD signal correlations between several cortical regions during sentence comprehension in adults with ASDs (Just et al., 2004). Underconnectivity findings were subsequently replicated across a variety of other fMRI tasks involving different cortical regions and networks (e.g. Just et al., 2007; Kana et al., 2006; Kleinhans et al., 2008; Koshino et al., 2008; Mostofsky et al., 2009). Other studies have found more complex patterns of mixed over- and under-connectivity associated with memory tasks (Koshino et al., 2005; Noonan et al., 2009).

Task-free fMRI studies testing intrinsic functional connectivity (Van Dijk et al., 2010) reveal functional network architecture (i.e., BOLD correlations within networks) at rest. This approach has been widely applied to research on ASDs. Resting state fMRI studies have frequently found mixed patterns of under- and over-connectivity (e.g., Fishman et al., 2014; Supekar et al., 2013; for reviews, refer to Hull et al., 2016 and Rane et al., 2015). Methodological variables play an important role, as shown in a meta-analysis (Müller et al., 2011) and empirical comparative methods investigations (Jones et al., 2010; Nair et al., 2014), which highlighted the difference between fcMRI studies testing for task-driven BOLD correlations and intrinsic fcMRI studies examining spontaneous synchronization of BOLD signal changes.

EEG and MEG findings relevant to the underconnectivity hypothesis are equally inconsistent, with both increased and decreased coherence observed across various task designs (see Vissers et al., 2012). This is further complicated by differences between frequency bands and brain regions (Ye et al., 2014) (Kitzbichler et al., 2015). A recent review concluded that reduced coherence between distant electrodes, often interpreted as underconnectivity, is generally supported, but is found most consistently in low frequencies, with mixed findings in higher frequencies (for review, see O’Reilly et al., 2017). However, EEG findings in the gamma frequency range (>30 Hz) may be confounded by muscle artifact (Muthukumaraswamy, 2013), limiting the reliability of these results. Unlike coherence, power appears to be increased in ASDs in low and high frequencies, but decreased in middle frequencies, such as alpha and beta (Wang et al., 2013). Although power and coherence are each affected by synchronized neural activity, these are ultimately different measures of “connectivity”, and it is therefore difficult to interpret their seemingly divergent frequency-related patterns. Even more importantly, there is no clear translation between these constructs and those measured in fMRI (i.e., BOLD correlations). Therefore, EEG and MEG findings may not be readily suited to addressing the fcMRI underconnectivity hypothesis.

Long-range Underconnectivity, Local Overconnectivity

A related proposal suggests that ASDs are characterized by reduced connectivity between distant regions, but increased local connectivity (Belmonte et al., 2004). At first glance, this may appear to be an appealing explanation for the mixed findings described above, but the literature suggests that this theory is overly simplistic. As described in the previous section, long-range underconnectivity is not generally supported by intrinsic functional connectivity findings. In addition, not a single fcMRI study examining local connectivity, using graph theory (Itahashi et al., 2015; Keown et al., 2013) or regional homogeneity methods (Dajani and Uddin, 2016; Jiang et al., 2015; Maximo et al., 2013; Nair et al., 2017), has supported broadly consistent local overconnectivity in ASD. Instead, findings have been mixed and region-specific, even in studies analyzing fMRI data without regional homogeneity standardization (Maximo et al., 2013; Nair et al., 2017), which could theoretically obscure global group differences in local connectivity.

The EEG/MEG literature is highly mixed, with both local hypo- and hyper-connectivity revealed through coherence analyses (for review, see O’Reilly et al., 2017). However, EEG and MEG may not be suitable for examining local connectivity for several reasons. First, these methods lack the spatial resolution to distinguish activity from nearby generators. Second, signals from closely spaced EEG electrodes (within 10–15 cm) are confounded by volume conduction in EEG (Srinivasan et al., 2007) and field spread in MEG (Winter et al., 2007). Therefore, while EEG/MEG may be appropriate tools for examining long-range connectivity, they are less well-suited to local overconnectivity hypotheses.

Reduced Network Integration and Segregation

A promising theory of functional connectivity in ASDs relates to overall network organization. There is a growing literature suggesting that neural networks in ASDs may be inefficiently organized due to poor functional differentiation and integration. Specifically, several fMRI studies have revealed reduced connectivity within networks, but increased connectivity between networks (Fishman et al., 2014; Rudie et al., 2012b; Shih et al., 2011). This theory has found additional support in studies using graph theory metrics to characterize network properties of both fMRI (Chen et al., 2017; Keown et al., 2017; Rudie et al., 2012a) and EEG/MEG data (Peters et al., 2013; Pollonini et al., 2010; Tsiaras et al., 2011), suggesting possible consensus at the network level. However, opposite findings have also been reported in fMRI, suggesting increased modularity in ASDs (Barttfeld et al., 2011; Yerys et al., 2017). Additionally, group differences in network efficiency appear to vary across frequency bands (Kitzbichler et al., 2015). Although this is a promising research direction, the precise characterization of network organization in ASDs remains an area of active, ongoing study.

Developmental Trajectories

The characterization of brain networks in ASDs is further complicated by the neurodevelopmental nature of the disorder and by maturational changes in brain network organization in typical development (Fair et al., 2010; Supekar et al., 2009). Study samples usually include individuals within a limited age range, with relatively few neuroimaging studies including very young children or older adults, and no longitudinal studies capturing large segments of the lifespan. Therefore, there is little firm knowledge about age trajectories of connectivity in ASD. Uddin et al. (2013) proposed that overconnectivity is most prevalent in young children with ASD, while underconnectivity is primarily found in adolescents and adults. This hypothesis bears some analogy to structural (Courchesne et al., 2001) and diffusion-tensor imaging evidence (Wolff et al., 2012) suggesting a developmental trajectory characterized by very early brain overgrowth, which normalizes over time. However, structural findings show brain overgrowth only in the first few years of life, while the developmental hypothesis by Uddin et al. predicts overconnectivity in children until puberty. Furthermore, although some fMRI findings are consistent with the theory of early overconnectivity and later underconnectivity, many others are not (e.g., underconnectivity in toddlers reported by Dinstein et al., 2011; Shen et al., 2016; overconnectivity in adolescents reported by Shen et al., 2012; Redcay et al., 2013).

Similarly, EEG/MEG findings do not exclusively support overconnectivity early in life. Reduced long-range EEG coherence (Righi et al., 2014), and reduced power in all frequency bands (Tierney et al., 2012) have been observed in high-risk infants. Furthermore, the longitudinal trajectory of EEG power in the first few years of life appears to depend not only on eventual diagnostic status, but also on frequency band studied (Tierney et al., 2012). These findings do not necessarily refute overconnectivity findings in fMRI; as discussed above, connectivity abnormalities are likely affected by age, and differ by regions, networks, and frequency bands. Although a clear picture has yet to emerge, developmental stage is certainly one of several critical variables when modeling brain networks in a neurodevelopmental disorder.

Missing Links: Toward Multimodal Functional Connectivity Studies

Although several promising themes emerge across the fMRI and EEG literatures, no single hypothesis accommodates all reported findings. This highlights the importance of adopting multimodal strategies to better understand connectivity in ASDs. To date, few multimodal neuroimaging studies in this area have been published. A recent review (Yerys and Herrington, 2014) described a small body of literature combining structural methods (e.g., structural MRI, DTI) with functional methods (e.g., fMRI, EEG, MEG) to study ASDs. However, none of these studies employed multiple functional approaches. To our knowledge, only one simultaneous EEG-fMRI study has been published to date, reporting reduced EEG beta power and atypically reduced BOLD activation in auditory cortex during auditory stimulation in a small sample of adults with ASDs (Hames et al., 2016).

To jointly benefit from the temporal resolution of electrophysiological methods and the spatial resolution of MRI, improved methods of data integration are needed. Because EEG/MEG and fMRI are generated by different sources and suited to different analyses, the EEG/MEG and fMRI literatures have remained largely separate in ASD research. This has limited our ability to draw conclusions spanning findings across modalities. For an integrated interpretation of fcMRI, EEG, and MEG findings, the neurophysiological bases of different signals must first be reconciled.

Relationships Between Multimodal Signals

Intracortical Recordings and the BOLD signal

The relationships between EEG/MEG signals and neural activity are well-established. The BOLD signal, however, is a secondary hemodynamic measure that relies on neurovascular coupling, which is not perfectly understood (Hillman, 2014). Simultaneous fMRI and intracortical recordings have been used to investigate the neurovascular bases of BOLD changes (see summary in Figure 1). Of particular interest is whether BOLD signal changes are related to synaptic transmission, like EEG and MEG, as opposed to action potentials (i.e., “spiking”). Extracellular electrodes placed in the cortex can measure local field potentials (LFP), a measure of surrounding synaptic activity, as well as multiunit activity (MUA), which reflects spiking of multiple nearby neurons. Single-unit activity, or the spiking of a single neuron, can also be measured using single-unit recordings. Early work with anesthetized monkeys found that during visual stimulation BOLD changes in visual cortex were more closely associated with LFP than MUA (Logothetis et al., 2001). Spontaneous BOLD fluctuations are also related to LFP in awake and anesthetized animals, particularly within the gamma band (Hutchison et al., 2015; Magri et al., 2012; Shmuel and Leopold, 2008; cf. Bentley et al., 2016) Coupling between LFP gamma power and BOLD has been replicated in humans, both during tasks (Conner et al., 2011 ; Gaglianese et al., 2017; Nir et al., 2007) and at rest (He et al., 2008; Nir et al., 2007; Nir et al., 2008). Furthermore, it has been shown that the spiking of single neurons is poorly coupled to LFP gamma power and BOLD activity (Nir et al., 2007), suggesting that both BOLD and EEG/MEG signals reflect a summation of regional synaptic activity, rather than action potentials of specific neurons. However, some studies have found a linear relationship between BOLD and MUA as well as LFP in both humans (Mukamel et al., 2005) and animals (Shmuel et al., 2006; Shmuel and Leopold, 2008). This is expected, as spiking and synaptic activity tend to be highly correlated despite their partially distinct mechanisms (for further discussion see Logothetis, 2008; Shmuel and Maier, 2015).

Finally, intracortical recordings have provided insight into modulatory relationships between cortical oscillations at different frequencies (cross-frequency coupling), and BOLD activity both locally and across distributed networks, which is relevant to BOLD connectivity. One continuous electrocorticography (ECoG) study of epilepsy patients found that slow intracortical oscillations (< 4 Hz) and BOLD fluctuations shared similar correlation structure across specified ROIs, while gamma band power and BOLD correlations were related only when subjects were awake or in rapid eye-movement (REM) sleep, but not during slow-wave sleep (He et al., 2008). The authors suggested that slow cortical potentials serve as a foundational mechanism for modulating higher-frequency oscillations, which reflect “online” processing. A study in monkeys found that coherence in low-frequency (<20 Hz) LFP bands modulated both regional gamma amplitude and BOLD, but predicted BOLD connectivity across ROIs better than gamma coherence (Wang et al., 2012). These findings are paralleled by a recent study using simultaneous ECoG and fMRI in humans, reporting that BOLD activity is better predicted by a combination of band-limited power and phase-amplitude coupling (here: the degree to which beta coherence modulates gamma amplitude; Murta et al., 2017), compared to a model that only considered the effects of power. Together, these findings underscore a potential role of lower frequencies in coordinating high-frequency local activity across large-scale networks. Importantly, MEG studies have demonstrated disrupted alpha-gamma phase-amplitude coupling in ASDs both at rest (Berman et al., 2015) and during a face-viewing task (Khan et al., 2013). For an in-depth discussion of hierarchical effects of global rhythms on local networks, see Nunez and Srinivasan (2014).

EEG Power and BOLD Activity

Intracortical recordings have provided important insights about the neural basis of BOLD signal change that can only be inferred from invasive measures. Simultaneous EEG and fMRI acquisition is less invasive and therefore far more widely used in human studies (note that simultaneous MEG and fMRI recording remains infeasible). EEG-fMRI studies have sought to further clarify the relationship between electrical signals measured at the scalp, localized BOLD activity, and BOLD correlations across networks (for reviews, see Mulert, 2013; Murta et al., 2015). We will first focus on relationships between EEG power and local BOLD fluctuations (i.e., “BOLD activity”), and then turn to BOLD correlations across regions (i.e., “BOLD connectivity”) in the next section (for summary, see Figure 1).

Kilner et al. (2005) proposed a heuristic relating BOLD activity to EEG and LFP power based on mathematical assumptions about energy dissipation. This model posits that increased BOLD activity should be associated with increased power in high-frequency EEG and LFP, and that increases in lower frequency EEG and LFP should accompany BOLD decreases. Although this theory is partially supported (as described below), it somewhat oversimplifies the relationship between electrophysiological and BOLD activity.

Alpha rhythms (~8–12 Hz) are dominant at rest, and have been most widely studied in relation to spontaneous BOLD signal changes. Increases in alpha power are associated with reduced BOLD activity in frontal, occipital, and parietal areas (Bridwell et al., 2013; Goldman et al., 2002; Laufs et al., 2003; Olbrich et al., 2009); a similar relationship has been reported for beta power increases (Murta et al., 2017; Scheeringa et al., 2011). However, a consistently positive relationship has been found between alpha power and BOLD activity in the thalamus (Bridwell et al., 2013; de Munck et al., 2007; Goldman et al., 2002). Although the direction of these relationships is well-established, less is understood about their temporal lag structure. In particular, thalamic BOLD signal increases have been found to precede corresponding increases in EEG alpha power by several seconds (de Munck et al., 2007; Feige et al., 2005; Feige et al., 2017). This has been generally observed for positive EEG-BOLD relationships, but not negative ones (Feige et al., 2017). These findings warrant further study, and may be particularly relevant in ASD, considering evidence of atypical lag structure in this population (Mitra et al., 2017).

Although most early studies focused on the alpha frequency band, different EEG frequencies, each associated with distinct functional states (Buzsaki, 2006), may relate differently to the BOLD signal. Unique relationships between ICA-derived BOLD networks and specific EEG frequency bands have been demonstrated (Mantini et al., 2007; Neuner et al., 2014b). In line with the LFP and ECoG literature, EEG gamma power and BOLD activity have consistently been found to be positively associated (Bridwell et al., 2013; Murta et al., 2017; Scheeringa et al., 2011). Scheeringa and colleagues (2011) reported that fast (gamma) and slower (alpha and beta) oscillations predict unique variance in the BOLD signal.

EEG Power and BOLD Connectivity

The relationship between EEG power and interregional BOLD correlations is particularly relevant to the study of functional connectivity in ASDs. Despite the large literature relating EEG features to local BOLD increases and decreases, less is understood about links with BOLD correlations within and between networks. One relatively consistent finding is an inverse relationship between alpha power and coupling (positive or negative) between networks; in other words, increased alpha power is associated with less positive BOLD correlations in correlated networks, and less negative BOLD correlations in anticorrelated networks (Allen et al., 2017; Chang et al., 2013; Scheeringa et al., 2012). Chang and colleagues (2013) propose that reduced vigilance/attention may cause a shift away from alpha rhythms toward lower frequencies, as well as stronger differentiation between resting and task-driven networks (e.g., default-mode and dorsal attention networks). However, one study reported state-dependent relationships between BOLD connectivity and both alpha and beta power (Tagliazucchi et al., 2012). This study also identified many BOLD connections that were positively associated with gamma (which, however, may be confounded by muscle activity (Muthukumaraswamy, 2013).

BOLD connectivity within the default-mode network (DMN) has been reported to be positively and negatively related with beta and delta power, respectively (Hlinka et al., 2010). However, a recent study found that BOLD covariance matrices and source-localized EEG covariance matrices were most similar for low EEG frequency bands (Deligianni et al., 2014). These discrepancies are consistent with findings that the relationship between MEG and BOLD correlation structures varies between individuals, and that different MEG frequencies best predict connectivity between different ROI pairs (Hipp and Siegel, 2015). Overall, the exact relationship between BOLD correlations and electrophysiological power remains unclear, and appears to depend on ROIs selected, cognitive state, and frequency band studied.

Can Transient Connectivity States Unite EEG/MEG and fMRI?

Dynamic Connectivity Approaches

As pointed out earlier, electrophysiological and BOLD signals are conventionally analyzed in different ways. Direct comparisons of fMRI connectivity measures (e.g., BOLD correlations) with EEG connectivity measures (e.g., power, coherence) are therefore challenging at best, and possibly inappropriate. The study of transient brain states may provide a novel approach to this problem. This data-driven strategy assumes that neural networks dynamically fluctuate between a given number of transient, replicable connectivity patterns, or states. Each connectivity state is thought to correspond to a cognitive state. Crucially, it is expected that electrophysiological and hemodynamic measures of neural activity can be integrated more readily when assessed at the level of single states and their dynamic changes across time, compared to static connectivity approaches that have dominated the fcMRI literature on ASDs.

The dynamic approach commonly involves temporally clustering resting-state EEG data into a series of “microstates” (for review, see Khanna et al., 2015). Microstates can then be convolved with a hemodynamic response function (HRF) and used as regressors in a general linear model to predict BOLD activation. It has been shown that voxelwise BOLD activity maps generated purely from electrophysiological regressors spatially correspond to resting-state networks derived from BOLD ICA on the same sample (Britz et al., 2010; Musso et al., 2010). EEG microstates have also been shown to predict thalamic BOLD fluctuations (Schwab et al., 2015). Similarly, EEG data classified into “vigilance” states have also been shown to predict BOLD correlation patterns (Olbrich et al., 2009). These studies support the assumption that complex, transient EEG patterns directly relate to whole-brain BOLD correlations, and to cognitive states-of-mind (Milz et al., 2016). However, this does not necessarily suggest that each EEG microstate has a single corresponding BOLD resting-state network. Using temporal ICA on EEG data and spatial ICA on fMRI data, one study found that some BOLD networks were related to several EEG microstates (Yuan et al., 2012). These findings highlight the complex temporal dynamics of neural networks and their relationships with cognition and behavior, and underscore the benefits of integrating complementary methodologies.

Although EEG microstates have typically been used as predictors for BOLD data in simultaneous EEG-fMRI research, one recently published study took the opposite approach. Here, overlapping windows of BOLD time series were clustered into seven states, each of which corresponded to a unique EEG spectral signature (Allen et al., 2017). Additionally, they found that eye status (open or closed) affected the relative prevalence of different states. This study is the first to associate whole-brain BOLD connectivity states with EEG patterns, providing a unique perspective and novel insight into the relationships between EEG and fMRI networks.

Dynamic connectivity in ASD: First steps

Although none of these multimodal approaches have yet been applied to ASDs, interest in “dynamic” functional connectivity fMRI has recently grown. Dynamic functional connectivity studies aim to characterize the complex spatiotemporal patterns underlying neural networks, often focusing on properties of transient brain states (Chen et al., 2017; de Lacy et al., 2017; Watanabe and Rees, 2017). Dynamic methods have been recently used in a few ASD studies to explore dynamic connectivity variability (Falahpour et al., 2016) and to predict diagnosis (Zhu et al., 2016).

Sliding window analysis is the most popular dynamic connectivity approach (Allen et al., 2014; Hutchison et al., 2013). However, there is as yet little consensus on best practices concerning window length and overlap, which have been shown to affect findings (Hindriks et al., 2016; Shakil et al., 2016). While some have criticized the relevance and meaning of BOLD dynamics altogether (Laumann et al., 2016), others have found the approach to be generally reliable (Abrol et al., 2017). Furthermore, the temporal resolution of single-modality fMRI dynamic connectivity remains limited given the slowness of the BOLD response. However, the movement toward dynamic connectivity fMRI is a critical step in the right direction for ASD research. As described in the previous section, the study of transient connectivity states (rather than static connectivity across prolonged periods of scanning) may be the most promising avenue towards true integration between electrophysiological measures from EEG or MEG and hemodynamic measures from fMRI.

Analyzing Multimodal Signals

The preceding sections highlight the need for integrative and dynamic multimodal approaches to connectivity. We will now turn to strategies for approaching this goal. Common integration strategies for multimodal functional data (specifically EEG and fMRI) will be introduced. Key decision points will be emphasized, including advantages and disadvantages of each broad approach. This is intended to serve as a conceptual introduction rather than a comprehensive review. Readers interested in more exhaustive reviews are referred to (Daunizeau et al., 2009; Ritter and Villringer, 2006; Rosa et al., 2010; Uludag and Roebroeck, 2014). Detailed technical discussions of various fusion methods can be found in (Sui et al., 2012; Valdes-Sosa et al., 2009).

Data Integration Choices

Simultaneous vs. Separate Acquisition

Recent technological advances have improved the feasibility of simultaneous EEG and fMRI recording. For studies comparing spontaneous activity between two modalities, simultaneous recording is indispensable, as unpredictable fluctuations across each timecourse must be temporally matched. Moreover, simultaneous recordings collected in a single session may reduce participant attrition. However, there are also advantages to separate acquisition (Wibral et al., 2010), which is less expensive (given the high cost of MRI-safe EEG equipment) and avoids additional setup time and participant discomfort associated with simultaneous acquisition. This is especially relevant for studies including clinical populations such as ASDs, who may have difficulty tolerating long, uncomfortable protocols. Finally, each modality impacts data quality of the other, reducing the signal to noise ratio for simultaneously acquired data. Despite corrective methods (Eichele et al., 2010; Mullinger and Bowtell, 2010), specific artifacts (e.g., EEG ballistocardiographic effect) remain a challenge (Foged et al., 2017; Neuner et al., 2014a).

While separate data acquisition is often more practical, some research questions can only be addressed with simultaneous recording. Separately acquired data may be appropriate for highly controlled event-related paradigms, in which responses can be averaged across trials, whereas simultaneous acquisition is preferable when studying spontaneous EEG-fMRI activity or responses that vary across sessions (e.g., practice effects).

Model-driven vs. Data-driven Approaches

Model-based analysis requires a forward (i.e., generative) model that predicts the relationship between neural activity and measured signals from both modalities as a function of anatomy, volume conduction, and hemodynamic/metabolic interactions (Valdes-Sosa et al., 2009). However, generating such a model is challenging, and relies on assumptions about neurovascular coupling, which is not perfectly understood (Deneux and Faugeras, 2010; Rosa et al., 2010).

Data-driven approaches require no a priori models specifying the relationship between imaging data and underlying neural activity. This removes the constraints imposed by selected model parameters, and can reveal informative, complex relationships between measured EEG and fMRI data (Sui et al., 2012). However, interpreting results can be challenging without modeling underlying neural processes, as data-driven approaches can only describe relationships between observable measures (Valdes-Sosa et al., 2009). In such cases, it may be beneficial to relate findings from data-driven approaches to phenotypical variables such as symptom severity, cognitive function, and other neuropsychological measures in order to aid interpretation of results.

Symmetric vs. Asymmetric Analyses

A third consideration is whether each modality will be given equal priority (i.e., symmetric analysis), or if one modality will be treated as a predictor for the other (i.e., asymmetric analysis). For example, asymmetric analyses may be used to improve the temporal resolution of fMRI or the spatial resolution of EEG, by using data from one modality as a “constraint” for the other (Uludag and Roebroeck, 2014). While these asymmetric analyses take advantage of the complementary strengths of each modality, they assume superiority of one modality (considered the “predictor”). When one type of data is given greater importance, assumptions about the ground truth of that modality may introduce bias (Daunizeau et al., 2009; Rosa et al., 2010). Symmetric analyses, on the other hand, integrate EEG and fMRI data without prioritizing either modality (Valdes-Sosa et al., 2009). This approach is often driven by a generative model (described above), which specifies the relationship between neural activity and observed data from both modalities (Sotero and Trujillo-Barreto, 2008). However, symmetric analysis can also be data-driven or use hybrid methods (Lei et al., 2012b).

Common Data Integration Approaches

There are three major approaches to EEG-fMRI data integration: 1) EEG-informed fMRI, 2) fMRI-informed EEG, and 3) EEG-fMRI fusion (for discussion, see Rosa et al., 2010).

EEG-Informed fMRI

The goal of EEG-informed fMRI analyses is temporal prediction, with EEG time courses used as predictors of BOLD activity. Therefore, this is a data-driven, asymmetrical approach. Because this approach relies on temporal correlations, it can only be used with simultaneously collected data. Typically, some feature of EEG data is convolved with an HRF and entered as a regressor into a general linear model predicting voxelwise BOLD activation. Many of the studies discussed previously (see Relationships Between Multimodal Signals) have taken this approach in establishing the relationship between EEG and BOLD signals. EEG predictors can be discrete, event-related activations (Eichele et al., 2005), spontaneous power fluctuations in particular frequency bands (Goldman et al., 2002; Laufs et al., 2003), or “microstate” patterns identified by clustering (Britz et al., 2010; Musso et al., 2010; Schwab et al., 2015). With respect to connectivity, this approach can reveal voxelwise BOLD activity associated with EEG fluctuations, which can then be tested for links with known BOLD networks. Additionally, it can provide important information about moment-to-moment temporal dynamics in relation to BOLD networks, which would not be possible with fMRI alone. One limitation of this method is the risk of correlations between multimodal timecourses arising from shared noise.

fMRI-Informed EEG

Another common asymmetrical approach uses fMRI to spatially constrain EEG data. In other words, fMRI is used to improve source localization for EEG. As discussed earlier, identifying generators of EEG activity is complicated by volume conduction (Srinivasan et al., 2007). This creates an ill-posed inverse problem, with numerous loci of neural activity potentially accounting for any given spectral pattern. EEG source localization is typically done in the context of either an equivalent dipole model, which assumes that EEG activity arises from currents at specific points (i.e., dipoles), or a distributed source model, which estimates contributing activity across all voxels (Daunizeau et al., 2009). Using fMRI data as predictor can substantially improve the accuracy of EEG source localization (Lei et al., 2015). Recently, Bayesian techniques have been applied to fMRI-informed source reconstruction, as it operates on a more flexible set of assumptions about the EEG-fMRI relationship (Lei et al., 2012a; Phillips et al., 2005). One group developed a Bayesian procedure for using ICA-derived BOLD networks as priors for EEG localization (Lei et al., 2012a; Lei et al., 2011b). This method, known as Network-Based Source Imaging (NESOI), localizes EEG activity not to a single area of BOLD activity, but to a statistically probable functional BOLD network.

Applications of this method can also be used to characterize multimodal network connectivity. For example, NESOI has been applied as a means of matching ICA components derived separately from fMRI and EEG data. This method identified distinct and shared networks between modalities, and described interactions among these networks. Multimodal functional connectivity networks can thus be accurately modeled from both simulated and real data (Lei et al., 2011a). However, a major limitation to fMRI-informed source localization is the assumption that BOLD activity occurs at the location of EEG generators. This principle may be violated, as co-occurring activity does not always indicate a direct causal relationship.

EEG-fMRI Fusion

Fusion methods aim to integrate EEG and fMRI data in a symmetrical manner, with neither modality serving as a predictor for the other. This method makes no assumptions about the “correctness” of either data type relative to the other. Typically, fusion is accomplished by specifying a forward model accounting for findings in both modalities, as discussed earlier. Improved understanding of signal propagation and development of sophisticated modeling procedures has led to more frequent use of anatomically and physiologically realistic models (Deneux and Faugeras, 2010; Sotero and Trujillo-Barreto, 2008). Dynamic causal modeling is a popular approach for specifying these relationships (Bonstrup et al., 2016; Friston et al., 2003). However, creating truly realistic predictive models is challenging (Sotero and Trujillo-Barreto, 2008), and particularly relevant to this paper, generative models based on typical populations may not apply to ASDs, because atypical signal propagation may be part of the pathology of interest. For a review of model-based fusion, see Valdes-Sosa et al. (2009).

Data-driven approaches to symmetrical fusion have also been developed. For example, joint ICA (Moosmann et al., 2008) and parallel ICA (Liu and Calhoun, 2007) can simultaneously decompose multimodal data into meaningful components. Other methods, such as canonical partial least squares have been used to identify shared variance in multimodal datasets (Michalopoulos and Bourbakis, 2015). Interestingly, some fusion methods combine both model- and data-driven elements, incorporating data-driven findings into theoretical forward models (Daunizeau et al., 2007; Lei et al., 2010). ICA is particularly flexible, and has been integrated across numerous EEG-fMRI integration approaches (for review, see Lei et al., 2012b). However, an inherent limitation of ICA is that components may not be directly comparable between clinical and non-clinical populations.

Conclusions and Future Directions

Integrative multimodal studies constitute a critical next step in ASD functional connectivity research. As described above, no one proposal currently accommodates the numerous discrepant findings of over- and under-connectivity in ASDs for various imaging modalities, brain regions, and stages of life. Current studies are limited by a failure to appropriately merge electrophysiological and fMRI findings. Theoretical advances in ASD functional connectivity research will require more integrative approaches.

The different sources and characteristics of EEG, MEG, and fMRI signals present great challenges for cross-modal data interpretation. Comparison of these signals requires adequate understanding of the shared and distinct neural mechanisms underlying them. While there is strong support for a shared neural basis of electrophysiological signals and BOLD activity, the details of this association are complex (see Figure 1), and give rise to network features that may be visible to one modality, but invisible to another.

Advanced techniques for joint EEG-fMRI data analysis have been developed to bridge the gap between different imaging “languages”, allowing for more direct comparison and fusion between modalities. The application of these methods to ASD research remains limited, but is a particularly promising avenue of study for this field. The importance of connectivity has been well-established in ASDs, particularly at the network level. Joint analysis methods are ideally suited to these research questions, and could provide more exhaustive (and hopefully more accurate) models of how connectivity is disrupted in ASDs.

One particularly promising avenue of research includes defining EEG microstates to predict BOLD correlations across the brain and characterize widespread spatiotemporal patterns of inter- and intra-network connectivity. EEG microstates have provided useful insight into other psychiatric conditions, including panic disorder (Kikuchi et al., 2011), Alzheimer’s disease (Dierks et al., 1997; Nishida et al., 2013), and schizophrenia (Andreou et al., 2014; Nishida et al., 2013). Transient connectivity states provide a promising approach to translating different types of multimodal data into a common space for joint interpretation. Many of the seemingly inconsistent findings from the fcMRI, EEG, and MEG literature may be reconciled at the level of such transient states. First steps in this direction have been taken by recent dynamic connectivity fMRI studies of ASDs (Chen et al., 2017; de Lacy et al., 2017; Falahpour et al., 2016; Watanabe and Rees, 2017; Zhu et al., 2016). The strategy offers a novel approach to studying dynamic network properties, reaching beyond the familiar, often uninformative, and perhaps irresolvable questions of over- and under-connectivity.

Advances in multimodal data fusion have the potential to vastly improve our understanding of functional neural networks in ASDs and how they relate to everyday functioning and well-being. The dynamic properties of these systems, which likely play a role in a complex neurodevelopmental disorder such as ASD, cannot be accurately characterized by any single neuroimaging modality. Here, we have specifically focused on the physiological origins, advantages, and limitations of EEG, MEG, and fMRI, and the numerous possibilities offered by multimodal research. These methods have demonstrated their value in the neuroscience literature, but their implementation in ASDs has not yet fully come to fruition. There is a critical need for collaboration among scientists with broad, cross-disciplinary training, and with both clinical and technical expertise. If used appropriately, integrative multimodal analyses hold great promise for improved understanding of the brain bases of ASDs and the development of neurobiologically informed treatments.