Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2009 Aug 15.
Published in final edited form as: Neuroimage. 2008 Jul 2;42(2):799–806. doi: 10.1016/j.neuroimage.2008.05.036

Mapping the Connectivity with Structural Equation Modeling in an fMRI Study of Shape From Motion Task

Jiancheng Zhuang a, Scott Peltier b, Sheng He c, Stephen LaConte b, Xiaoping Hu b
PMCID: PMC2564811  NIHMSID: NIHMS67358  PMID: 18599316

Abstract

In this fMRI study, we explore the connectivity among brain regions in a shape-from-motion task using the causal mapping analysis of Structural Equation Modeling (SEM). An important distinction of our approach is that we have adapted SEM from its traditional role in confirmatory analysis to provide utility as an exploratory mapping technique. Our current approaches include (I) detecting brain regions that fit well in a hypothesized neural network model, and (II) identifying the best connectivity model at each brain region. We demonstrate that SEM effectively detects the dorsal and ventral visual pathways from the covariance structure in fMRI data, confirming previous neuroscience results. Further, our SEM mapping methodology found that the two pathways interact through specific cortical areas such as the superior lateral occipital cortex in the perception of shape from motion.

Keywords: functional magnetic resonance imaging, shape from motion, structural equation modeling, effective connectivity

Introduction

Functional Magnetic Resonance Imaging (fMRI) has become a successful investigative tool in basic and clinical neuroscience since the discovery of the Blood Oxygenation Level Dependent (BOLD) effect in 1992 (Ogawa et al.). Historically, the primary focus of fMRI researchers has been on localization of neural activity for particular tasks, even though their neuroscientific interest is often in the distribution of activity across different brain regions. Recently, more attention has been paid to the detection of causal interactions between cortical areas (McIntosh et al., 1994; Büchel and Friston, 1997; Goncalves and Hall, 2003; Penny et al., 2004; Zhuang et al., 2005; Smith et al., 2006).

However, ascertaining causal connectivity across cortical areas from fMRI data encounters three challenges. First, most current analysis methods, such as Independent Component Analysis and Principal Component Analysis, can reveal patterns of activity across multiple spatial regions, but are not able to provide directional information on the interaction between these regions. Second, the observed temporal latency of the BOLD response between different brain areas cannot be used to estimate the temporal order of neural activities. This is due to the fact that the local hemodynamic response, which depends on the local physiology, could be regionally specific, and thus blur or delay the temporal evolution of BOLD signals at different brain sites. The third challenge is that many traditional analysis methods fail to provide a complete account of interactions if more than two brain regions are involved. That is, one region can affect another directly or indirectly, i.e. via a third party in the model, and can be affected by several regions. As a result, correlation of two areas could include both direct and indirect effects, and most existing fMRI analysis methods based on correlations cannot tease these apart. For this type of “many-body problem” in neural interactions, a powerful statistical method has been utilized for functional neuroimaging research - structural equation modeling.

Structural equation modeling (SEM) is a statistical technique that is able to examine causal relationships between multiple variables. The parameters in the SEM are connection strengths or path coefficients between different variables, and reflect the effective connectivity in our neural network model. Each path in the model has direction, and a solvable model must have more than two variables. Parameters are estimated by minimizing the difference between the observed covariances and those implied by a structural or path model. Since SEM solves the whole path model at once, the solution will give the causal direction between multiple regions of interest (ROIs). Therefore the three challenges mentioned above can be overcome.

SEM was initially developed and applied in biology, psychology, economics, and other social sciences (Wright, 1920). In 1994, hypothesizing a connectivity model based on prior knowledge of anatomy and connectivity, McIntosh and Gonzalez-Lima (1994) applied SEM to PET data, demonstrating the dissociation between ventral and dorsal visual pathways in object and spatial vision. SEM was also used to characterize connectivity changes within the motor system of Parkinsonian patients (Grafton et al., 1994). Büchel and Friston (1997) used SEM analysis in an fMRI study to investigate the nonlinear interactions among V1, V5, posterior parietal cortex, and prefrontal cortex. Maguire et al. (2001) compared the structural equation models between human brains with and without bilateral hippocampal damage during a memory retrieval task. More recently, SEM was used as an exploratory approach to exhaustively compare all possible model combinations among a set of ROIs in an fMRI study of bimanual motor coordination (Zhuang et al. 2005). Another recent study investigated the effect of voxel selection on the application of SEM to an fMRI study of the visual areas V1, V2 and V5 (Goncalves and Hall, 2003).

While SEM is an effective method to ascertain path directions and coefficients from the covariance structure in fMRI data, in the current literature it is mostly applied using a few fixed models or fixed variables (ROIs in functional neuroimaging). Because these fixed models or ROIs need to be predefined based on existing knowledge, unknown brain areas cannot be explored using these methods. The present work uses SEM in an exploratory analysis to derive, on a voxel-by-voxel basis, the most significant path model and the corresponding path weights, such that we can generate maps corresponding to model statistical indices, especially in those areas in which the activation pattern is unknown.

To study neural interactions and test SEM methodology, we need an experimental paradigm that (1) is well-informed by neuroscience, providing knowledge of which brain regions are involved and allowing comparison of our statistical results with previous neuroscientific results; and that (2) explicitly modulates the interactions among brain regions, including different directions and strengths among these interactions, which can be tested by SEM. To satisfy these criteria, in this study we applied SEM with an fMRI paradigm to derive the effective connectivity in the human visual system.

There exist many neurological studies on the organization of the human visual cortex (Felleman and Van Eesen, 1991; Zeki et. al., 1991; Tootell et al., 1998; Sunaert et. al., 1999). It is now widely accepted that the human visual cortical areas are organized into two functional pathways, the occipitotemporal pathway, or “ventral stream”, and the occipitoparietal pathway, or “dorsal stream”. The ventral stream plays a specific role in object recognition and identification (‘what’), with the main cortex for identification of object shape located in the lateral occipital complex (LO) (Malach et al., 1995; Martin et al., 1996; Kanwisher et al., 1996). Meanwhile, the dorsal stream is involved in visual perception of the movement and location of objects relative to the observer (‘where’), with the major cortex for perception of the visual movement located at the midtemporal lobe (MT) or V5 area (Ungerleider and Mishkin, 1982). These well separated functional pathways provide a good test platform for SEM.

The integration of low-level image features into coherent visual shapes is still an important topic in visual perception. In a shape-from-motion (SFM) task, three-dimensional (3D) object shapes can be perceived using patterns of random-dot motion without explicit shape information based on contours (Wallach and O’Connell, 1953; Koenderink and Van Doorn, 1986; Siegel and Andersen, 1988; Treue et al., 1991). The phenomenon of SFM indicates that the motion information, which is processed in the dorsal pathway, can be transferred forward and be used in the object recognition, which is performed in the ventral stream (Malach et al., 1995; Kriegeskorte et. al., 2003). Ungerleider and Mishkin pointed out that the act of perceiving shape from motion suggested cross-talk between the dorsal and the ventral visual pathways (Ungerleider and Mishkin, 1982). Previous human fMRI studies (Jiang et al., 2002; Murray et al., 2003; Kourtzi et al., 2003; Kriegeskorte et al., 2003) have investigated this topic, but the focus has been on one or two regions, such as MT+ (Jiang et al., 2002), LO (Kourtzi et al., 2003; Kriegeskorte et al., 2003), or superior lateral occipital cortices (Murray et al., 2003). The analysis method in these papers does not provide a comprehensive picture at the macroscopic level, and the interpretation of their results was based on the assumption of connectivity between the individual detected regions.

In this paper, we used fMRI data from a shape-from-motion task to test the feasibility of SEM analysis in effective connectivity detection at a macroscopic level that includes multiple brain regions. SEM was applied as a mapping method rather than a confirmatory method to identify the best connectivity model at each brain region.

Materials and Methods

fMRI experiment

Seven subjects participated in this study according to the guidelines set forth by the institutional review board at Emory University. Informed consent was obtained from all subjects. Of these, five were male and two were female with age ranging from 18 to 31 years. All were right-handed with normal vision and no neurological deficits.

MR data were acquired on a 3T Siemens Trio whole body scanner. Anatomical scans were acquired using a T1-weighted MPRAGE sequence (TI/TR/TE/Flip angle: 800ms/25ms/6ms/15°; matrix: 256×256×140; FOV: 256×256×140 mm3). Functional data were then collected using a gradient echo EPI sequence (TR/TE/Flip angle: 1s/32ms/65°; slice thickness: 5.5 mm without gap; matrix: 64×64; FOV: 20×20 cm2). Fourteen oblique coronal slices were acquired to cover the occiptal, parietal and temporal lobes, with 300 images collected for a scan time of 5 minutes per run, matched to the visual task described below.

Subjects viewed visual stimuli in a block design paradigm that included the following conditions: 3D SFM, static shape contours, random moving dots, static random dots and fixation, each lasting 20s and repeated 3 times. The block-design fMRI scans were repeated twice for each subject. The order of the blocks was randomized. The pictures of random dots for visual presentation were exactly the same among the conditions of SFM, random moving dots and static random dots: white random dots uniformly spread on the black background with a small fixation cross at the center. The only difference between the SFM and random moving dots conditions was the display sequence. With a certain display sequence of dots, the subject can identify a cube or cylinder shape. In contrast, with other orders of presentation, or in the random moving dots condition, no shapes can be seen (Figure 1). The cube or cylinder contours were added on the random dots background during the static shape contour block.

Figure 1.

Figure 1

Open in a new tab

Examples of visual stimuli for the random dots (left) and shape from motion (right, without the explanatory dashed lines and rotation arrows) conditions.

The subject’s viewing angle of the rear screen when lying inside the scanner was approximately 16.5°. Stimuli were projected on the rear screen, with full display size of 34cm × 26cm. The visual stimuli consisted of 600 dots and a central fixation cross. The dots and the cross were displayed in white over a black background. The dots (six pixels) were round with a radius of 0.07° visual angle. All pictorial stimuli were displayed using Presentation (version 0.50, Neurobehavioral Systems).

Exploring with a fixed structural equation model

For each fMRI data set, head motion was corrected and activated areas were detected using BrainVoyager (Maastricht, Netherlands). Activation maps of V1/V2 areas were obtained using a t-test between all visual stimulus conditions (including SFM, random moving dots, static shapes and static random dots) and fixation baseline, with statistical significance p<0.001 (corrected for multiple comparisons) and a minimum cluster size of 5 voxels. Aided by the anatomical landmarks such as the lingual gyrus and the calcarine sulcus, we selected the activated areas that were closest to the coordinates cited in other studies to localize V1/V2 (Büchel and Friston, 1997; Goncalves et al., 2003).

MT+ was identified by a similar t-test between the motion conditions (SFM and random moving dots) and the static conditions (static shapes, static random dots and fixation), as well as by the anatomical landmarks published in the previous studies. Similarly, the LO area was detected by a t-test between shape conditions (SFM and static shapes) and all other conditions (random moving dots, static random dots and fixation). The time series of the voxels from each subject’s identified ROIs were first averaged, and then normalized into percentage scale by subtracting and dividing by the mean, and finally concatenated between both fMRI runs to create an intra-subject time series for each voxel.

To explain the whole analysis procedure, we will start with one model as an example. In this model, two visual pathways start from the unknown region to LO and MT+ area (Figure 2, Model 1) in each hemisphere. The normalized time courses from LO and MT+ areas were treated as two observed variables. The signal from each voxel within the hemisphere except the marked MT+ and LO areas was regarded as the third observation in the current structural equation model, and was evaluated with the SEM statistics.

Figure 2.

Figure 2

Open in a new tab

The sequence of SEM path models used to explore the causal connectivity of the visual system. For each model, the time courses from the voxels of ROIs were used to identify the unknown voxel that best fit the model.

For the assessment of the overall model fit, the goodness of fit index (GFI) and the adjusted goodness of fit index (AGFI) are the most commonly used fit indices in SEM analysis. For our purpose, AGFI is a more appropriate index of fit than GFI, in that AGFI accounts for the number of degrees of freedom in the model (see Equations 1 and 2; Byrne, 1994; Gerbing and Anderson, 1993; Hu and Bentler, 1999). These indices are derived by

GFI=1−tr[(∑−1S−I)2/tr[(∑−1S)2] (1)

where tr indicates the trace operation, S is the covariance matrix, and Σ is the estimated S;

AGFI=1−[p(p+1)/2df](1−GFI) (2)

where p the number of observations, and df the degree of freedom in the model.

The SEM software Lisrel 8 (Jöreskog and Sorbom, Scientific Software International Inc., Chicago, IL) was used to estimate the statistical significance of model fitting with the experimental data at each voxel. The AGFI value found in the SEM analysis was marked on the image voxel when two statistical significance thresholds in the model were reached at that point. First, a threshold of 0.90 was chosen for the overall model significance, represented by the AGFI. Second, the significance of each path, or the t value of each estimated path coefficient in the model was thresholded at 1.96, which corresponds to a probability level of 0.05 for the given degrees of freedom. The algorithm was implemented in Matlab (MathWorks, Natick, MA).

Exploring the best model at each voxel

The approach of AGFI mapping from one model can be extended to compare a range of possible models at each voxel, in order to identify voxels in the brain corresponding to a particular connectivity model or path. Among 13 cortical visual areas, 62 connectivity pathways were identified by previous neuroscience studies (Felleman and Van Essen, 1991). According to our topic and interests, six anatomically available connection models were assumed as in Figure 2. As an example, in the second model, we defined three variables and two paths: V1/V2, one unknown area, MT+ area, the path from V1/V2 to MT+, and the path from V1/V2 to the unknown area. The third model was similar to the second one, except that MT+ was replaced by LO. The purpose of selecting the first three models was to verify the feasibility of this method.

In the fourth and fifth models, two paths start from V1/V2 to LO and MT+ areas. One unknown variable is in the middle of the path either from LO to MT+ or from MT+ to LO (Figure 2, purple and blue models). In Model 6, the unknown area receives paths both from MT+ and LO, with MT+ and LO receiving inputs from V1/V2 area. The reason for including Models 4, 5 and 6 is to investigate the interaction between the two visual pathways and to identify unknown areas that may process and integrate very specific visual information into the shape perception.

Similar to the analysis of Model 1 described above, SEM was first used to estimate the statistical significance of each model’s fit with the observed activity in each voxel under consideration. Next, for each voxel the six models were sorted by their AGFI with a significance threshold of t-value (>±1.96) on each path. Finally, for each voxel, the best-fit model and its corresponding AGFI value were saved and displayed in the color-coded Model Index Map (MIM).

Using BrainVoyager (Maastricht, Netherlands), each subject’s brain was inflated into a smooth cortical surface. The statistical maps from the SEM, including the MIMs, were projected onto the inflated cortices of each subject for the easy visualization of the activation areas on one surface. For each model, the location of the voxel with peak activity in terms of highest AGFI value was identified, and the corresponding Talaraich coordinates were recorded. The time course from that voxel was retrieved, and fed back into the best fitting structural equation model to calculate the path coefficients for that model.

Results and Discussion

In this experiment, the activity of visual areas was observed using a t-test contrasting all visual conditions combined versus fixation for seven subjects individually. A subset of activated areas was located at V1/V2 areas. As illustrated in Figure 3, an ROI was selected based on the detected activity and anatomic landmarks of V1/V2 areas. The activity of the LO area was detected using a t-test between the 3D shape processing tasks (SFM and static shape) and all other conditions, and the activity of MT+ area was observed based on the test between motion perception (SFM and moving dots) condition and all other conditions. All t-test results were thresholded at p<0.001, corrected for multiple comparisons.

Figure 3.

Figure 3

Open in a new tab

Representative time courses and locations of Regions of Interests (ROIs) as derived from activation maps and anatomic landmarks, which were used in the voxel-by-voxel SEM analysis. The depicted areas are for one subject as an example.

The AGFI maps from Model 1 were determined and displayed in Figure 4 (red model). The results for all seven subjects demonstrated significant AGFI (>0.900) areas located in V1/V2, which is consistent with the well-known anatomical model of the two visual pathways. Table 1 lists the results of fitting Model 1 to the time courses of two representative voxels from a single subject. The first time course was retrieved from a voxel where the fitting was significant (in the detected red region of Figure 4), and the second was from a voxel where the fitting was not significant.

Figure 4.

Figure 4

Open in a new tab

The resulting Model Index Maps depicting the connectivity models of the human visual system, for the right hemisphere by subject. The color bars indicate the AGFI (Adjusted Goodness of Fit Index) value.

Table 1.

An example of the overall fit indices inferred by the SEM for Model 1 at two voxels (with Talaraich coordinates in units of mm) in one subject’s brain. Voxel A was selected from the red region of Figure 4, and Voxel B was selected from outside that region.

Model 1 Voxel A (-16, -91, 2) Voxel B (-30, -75, 12)
Goodness of Fit Index (GFI) 0.990 0.765
Adjusted Goodness of Fit Index (AGFI) 0.967 0.218
χ2 / degree of freedom 2.017 94.413
 P-value for χ2 (0.001) (0.109)
Root Mean Square Error of Approximation (RMSEA) 0.0351 0.336
  P-value for test of close fit (RMSEA < 0.05) (0.997) (0.6653)
Largest standardized residual 0.0107 10.605
Normed Fit Index (NFI) 0.997 0.883
Comparative Fit Index (CFI) 0.998 0.884
Open in a new tab

The Model Index Maps (Figure 4) show that Model 1 was the best fit for area V1/V2. Meanwhile, Model 2 fit best in LO for all seven subjects, and fit best in the inferior occipital-temporal area (IO/IT) for three of the seven subjects. Model 3 was the best model in MT+ (V5) area for all seven subjects, and was the best model in the mid-lateral parietal lobule for four subjects.

Model 4 was found to be the best fitting model in the left dorsal occipital regions (DO) for three subjects, and to be the best fitting model in the right dorsal occipital regions for four subjects. Almost found on all subjects, Model 5 had the best fit in the superior lateral occipital region (SLO), which was at the middle of the path from MT+ to LO. Model 6 did not fit any subject’s data well, implying that no cortical area was the converging point of two paths from MT+ and LO.

Table 2 presents the regions of peak AGFI value, on the left and right hemispheres respectively, for different models, their inter-subject averaged Talaraich coordinates, the number of subjects who exhibit the active region in the model, the inter-subject average of standardized path coefficients, and the average AGFIs. Table 2 shows that the AGFIs from the first five models are all high, above 0.948, which indicates a good fit for these models. Table 3 presents the Talaraich coordinates of the ROIs derived from localization of visual areas, and shows the results of SEM analysis specifically from these ROIs. Comparing the Talaraich coordinates listed in Tables 2 and 3, we can see that the detected regions from Models 1-3 and the corresponding pre-defined ROIs are extensively overlapped. However, the difference of SEM results between the model-identified regions and the corresponding ROIs indicates that they are not fully overlapped, for the reason that the statistics behind the ROI localization and SEM analysis are different in nature.

Table 2.

The region of peak activation value (determined by AGFI) detected by six models at two hemispheres. The Talaraich coordinates of regions, standardized path weights and AGFIs in the models, were inter-subject averaged. The numbers of subjects represent how many of the seven subjects’ data showed activity in this area. The values are presented as: Mean ± Standard Deviation. The unit of coordinates in Talaraich space is mm.

Model No. Peak Region Coordinate Subjects Num. Averaged Standardized Path Weight AGFI Ave.
x y z
Left Hemisphere
1 V1/V2 -15 -92 2 7 V1/V2→MT V1/V2→LO 0.966
±3 ±2 ±2 0.843 0.678 ±0.026
2 LO -37 -81 -3 7 V1/V2→MT V1/V2→LO 0.959
±6 ±5 ±2 0.799 0.656 ±0.031
3 MT -48 -72 3 7 V1/V2→MT V1/V2→LO 0.967
±4 ±9 ±3 0.877 0.708 ±0.029
4 DO -18 -89 22 3 V1/V2→MT V1/V2→LO LO→DO DO→MT 0.948
±5 ±11 ±4 0.681 0.833 0.090 0.096 ±0.037
5 SLO -34 -79 12 6 V1/V2→MT V1/V2→LO MT→SL0 SL0→LO 0.955
±4 ±7 ±5 0.678 0.848 0.101 0.099 ±0.024
Right Hemisphere
1 V1/V2 11 -95 1 7 V1/V2→MT V1/V2→LO 0.972
±4 ±7 ±3 0.815 0.682 ±0.021
2 LO 40 -79 -2 7 V1/V2→MT V1/V2→LO 0.968
±6 ±5 ±3 0.829 0.618 ±0.027
3 MT 47 -67 1 7 V1/V2→MT V1/V2→LO 0.970
±5 ±10 ±6 0.853 0.605 ±0.022
4 DO 23 -85 23 4 V1/V2→MT V1/V2→LO LO→DO DO→MT 0.956
±5 ±9 ±8 0.705 0.803 0.097 0.094 ±0.035
5 SLO 39 -78 10 7 V1/V2→MT V1/V2→LO MT→SL0 SL0→LO 0.951
±4 ±12 ±9 0.690 0.817 0.114 0.102 ±0.040
Open in a new tab

Table 3.

The ROIs used in SEM analysis at two hemispheres. The ROIs were defined by a series of t-tests and anatomic landmarks, as described in Methods and Materials. Talaraich coordinates of peak activity (determined by p-value) in the region were averaged among seven subjects. The time courses of each ROI were tested by a corresponding structural equation model (referred as “Tested Model” in table). The last column is the inter-subject averaged AGFI derived from the model. The values are presented as: Mean ± Standard Deviation. The unit of coordinates in Talaraich space is mm.

ROI Averaged Talaraich Coordinate
Tested Model No. AGFI Average
x y z
Left Hemisphere
V1/V2 -13 ±4 -91 ± 3 1 ± 3 1 0.942 ± 0.020
LO -36 ±5 -83 ± 5 -3 ± 2 2 0.943 ± 0.021
MT -48 ±3 -73 ± 6 4 ± 3 3 0.943 ± 0.020
Right Hemisphere
V1/V2 12 ±3 -95 ± 4 1 ± 3 1 0.946 ± 0.024
LO 37 ±6 -81 ± 4 -1 ± 3 2 0.949 ± 0.022
MT 48 ± 4 -66 ± 5 2 ± 4 3 0.948 ± 0.023
Open in a new tab

As an example, the single-voxel results in Table 1 indicate that Model 1 generated statistically significant result from one subject’s data at a sample voxel in V1/V2, but the fit did not reach statistical significance at a sample voxel in other areas. A previous study using a similar voxel-based SEM analysis on one subject’s data also demonstrated the validity of this approach (Goncalves and Hall, 2003). However, in that study, the SEM analysis was performed on just a few ROIs, rather than employed as a mapping method. In our study, the areas identified by SEM are consistent across seven subjects (Figure 4 and Table 2). It is important to note that with several models applied to more than 50,000 voxels, chance alone is bound to produce some significant results. The consistency of our findings across seven subjects lends support to the validity of our method.

The resulting maps from Models 1, 2 and 3 match with the known functional anatomy of the visual system. Area V1/V2 is the starting point of two visual pathways, which were mapped in Model 1. The detected areas from Model 2 are mostly in the ventral visual pathway, such as in the LO and IO/IT areas. The areas in Model 3 are located in the dorsal visual areas including MT+ and parietal cortex, which is also consistent with previous literature. The two primary visual pathways starting from V1/V2 to MT+ and LO were found consistently in the three independent model fittings. The results of the first three models verified that our mapping method of SEM can detect the paths in the visual system.

As described in the introduction, standard methods of analyzing BOLD fMRI are not suitable to make definitive conclusions regarding the causal connectivity based on the observed temporal differences in the fMRI activations as the hemodynamic response is too slow, while the cortical interconnection exhibited by neurophysiological oscillation on the millisecond scale can be studied by EEG or MEG (Darvas et al., 2004; David et al., 2006). The SEM approach is also applicable in analyzing EEG or MEG data in this regard (Babiloni et al., 2003; Astolfi et al., 2005). Some EEG and MEG studies (Schoenfeld et al., 2003; Cauquil et al., 2006) analyzed the neural processing sequence in the SFM task that is used in the present study, and found the interactions between ventral and dorsal visual pathways, such as between parietal, occipito-temporal and ventral–lateral occipital cortices. However, in general, neither EEG nor MEG has sufficiently high spatial resolution to reveal detailed cortical pathways. Therefore, we propose based on our current results that fMRI can serve as a complementary method for studying neural functions and neural interactions at a high spatial resolution using voxel-by-voxel SEM.

In our present study, the time courses from ROIs indicated that the specific paradigm design modulated the influence of the stimulus conditions on the activation of different brain areas. This enabled the use of time series from various groups of voxels to predict the interconnection of two visual pathways that correspond to the ventral and dorsal streams. Simple t-tests or cross correlation are insufficient to detect areas with a complex activation pattern. However, because activity in particular areas, which have connections to any known activated areas, must follow a certain modulation by characteristic tasks, we can circumnavigate this problem by estimating different structural equation models. As a demonstration, we removed the time points corresponding to the SFM condition at each voxel and then performed the same SEM analysis on Models 4-5. In this ad-hoc analysis, the previously detected region was no longer significant. This supports our suggestion that the results of the SEM approach yield a certain activation pattern which fits a causal connection model. In this way, the present exploratory SEM application represents a new mapping method for revealing the causal influence of “latent” or “hidden” areas, and further provides information about the connectivity at each voxel.

On the topic of shape perception from motion, all participants’ results indicated that there is intercommunication between the two visual “streams” (Models 4 and 5), but that there is no converging point between the two paths (Model 6), which is consistent with the known anatomy of the visual system (Felleman and Van Essen, 1991; Zeki et al., 1991; Tootell, 1998; Sunaert et al., 1999). According to Kourtzi (2000, 2001, 2002, 2003), shape processing with motion information follows ventral pathways, although the LO also responds to the low-level image features. This agrees with our Model 5, in which the information is passed from MT+ to LO. Furthermore, Model 5 revealed the indirect effect from MT+ to LO. In other words, increased activity in MT+ may somehow lead to correlated brain activity in the LO, despite the fact that the two cortical areas are thought to have no direct projections.

The detected SLO in Model 5 is in close proximity to the shape (LO) and motion (MT+) areas. Many researchers have pointed out that SLO is an important area in the interaction between MT+ and LO (Grill-Spector et al., 1998; Murray et al., 2003). The coding of 3-D shape from motion is assumed to be mediated by the projections of dorsal areas onto the ventral visual pathway through intermediate areas (Paradis et al., 2000; Kriegeskorte et al., 2003). SLO is a good candidate for such an area according to the results from the current study, which explain why SLO was also found activated in previous SFM studies (Murray et al., 2003). SLO appears to be involved in “complex” structure reconstruction from motion cues, as suggested by Murray’s and our results. The pathway from MT+ to SLO and on to LO may represent the procedure from analyzing motion flowfield, to processing the motion parallax, and on to reconstructing the surface structure of the motion-defined object.

The significant fit of Model 4 for a few subjects’ data implies that the information may be transferred in a reversed but weaker way from the ventral visual pathway to the dorsal visual pathway. The area MT/MT+ (V5), known to be primarily engaged in motion analysis, is also involved in shape processing independent of the objects’ motion and receives input from LO areas (Kourtzi et al., 2002; Jiang et al., 2002). Another possibility is that the activity of the parietal lobule (PL, which may include IPL and LIP) also corresponds to visual motion but to a lesser degree than MT+, as in the results from Model 3 (Figure 4). This could cause the observed covariance between the LO and PL. The detected area may be in the middle of a path between LO and PL areas. But this effect is probably not as robust as the causal interaction from MT+ to LO, because not all subjects demonstrated this interaction. According to previous studies on the human vision system (Gaska et al., 1988; Felleman and Van Essen et al., 1991; Tootell et al., 1997; Orban et al., 1999; Sunaert et al., 1999; Murray et al., 2003), the detected DO area in Model 4 is possibly located at V3A, which shares connections with areas in both parietal and temporal cortices, especially one of the principal connections to/from LO (Kim and Kim, 2005). It is also the most retinotopic motion area in humans (Tootell et al., 1997) and is considered to be a lower tier motion area after MT+/V5 (Van Essen and Zeki, 1978; Zeki, 1978; Gattass et al., 1988). Given this, our model suggests that V3A may aid in the integration of various cues for the perception of 3-D shape. Taken together, our findings from Models 4 and 5 suggest complicated distributed, rather than simple hierarchical, processing of global shapes in the visual system.

In interpreting our results, one should keep in mind the limitation of the selected models. The SFM task also involves other visual areas and other path network models. For example, many distinct visual areas are reciprocally connected, such as V1/V2 and MT. In this study, we focused only on the ventral and dorsal visual pathways and on the unidirectional intercommunications between them to reduce the model complexity. Even though our models did not include all involved regions, as with any SEM study that examines effective connectivity, the results are nonetheless relevant because any covariance between cortical regions mediated by other visual structures would be indirectly reflected in our structural equation models. Furthermore, for demonstrating the utility of exploratory SEM, the choice of six models is sufficient.

Conclusion

An exploratory SEM analysis of fMRI data was applied to derive effective connectivity maps in the visual system, using a shape-from-motion task. Possible connections between each brain voxel and selected ROIs were assessed by SEM using six possible network models. For each voxel in the analysis, the most significant model was identified; the results across voxels were then used to generate a Model Index Map (MIM). This map reflected the overall connectivity of the region and provided valuable information on functional networks in the brain. Interestingly, the SLO area was found to be connected to both dorsal and ventral visual pathways during the visual shape and motion analysis.

Acknowledgments

This work was supported by the National Institutes of Health (Grants R01EB002009 and RO1EB000331), the Georgia Research Alliance, and the Whitaker Foundation. We thank Dr. Mary Helen Immordino-Yang for her assistance with manuscript preparation.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Astolfi L, Cincotti F, Babiloni C, Carducci F, Basilisco A, Rossini PM, Salinari S, Mattia D, Cerutti S, Dayan DB, Ding L, Ni Y, He B, Babiloni F. Estimation of the cortical connectivity by high-resolution EEG and structural equation modeling: simulations and application to finger tapping data. IEEE Transactions on Biomedical Engineering. 2005;52:757–768. doi: 10.1109/TBME.2005.845371. [DOI] [PubMed] [Google Scholar]
  2. Babiloni F, Cincotti F, Basilisco A, Maso E, Bufano M, Babiloni C, Carducci F, Rossini P, Cerutti S, Rubin DBD. Frontoparietal cortical networks revealed by Structural Equation modeling and high resolution EEG during a short term memory task. Proceedings of First International IEEE EMBS Conference on Neural Engineering. 2003;20:79–82. [Google Scholar]
  3. Büchel C, Friston KJ. Modulation of connectivity in visual pathways by attention: cortical interactions evaluated with Structural Equation Modelling and fMRI. Cerebral Cortex. 1997;7:768–778. doi: 10.1093/cercor/7.8.768. [DOI] [PubMed] [Google Scholar]
  4. Buracas GT, Albright TD. Contribution of area MT to perception of three-dimensional shape: a computational study. Vision Res. 1996;36:869–887. doi: 10.1016/0042-6989(95)00192-1. [DOI] [PubMed] [Google Scholar]
  5. Byrne BM. Structural equation modeling and EQS and EQS windows: Basic concepts, applications, and programming. Sage Publications; Thousand Oaks, CA: 1994. [Google Scholar]
  6. Cauquil AS, Trotter Y, Taylor MJ. At what stage of neural processing do perspective depth cues make a difference? Exp Brain Res. 2006;170:457–463. doi: 10.1007/s00221-005-0229-1. [DOI] [PubMed] [Google Scholar]
  7. Darvas F, Pantazis D, Kucukaltun-Yildirim E, Leahy RM. Mapping human brain function with MEG and EEG: methods and validation. NeuroImage. 2004;23:S289–S299. doi: 10.1016/j.neuroimage.2004.07.014. [DOI] [PubMed] [Google Scholar]
  8. David O, Kiebel SJ, Harrison LM, Mattout J, Kilner JM, Friston KJ. Dynamic causal modeling of evoked responses in EEG and MEG. NeuroImage. 2006;30:1255–1272. doi: 10.1016/j.neuroimage.2005.10.045. [DOI] [PubMed] [Google Scholar]
  9. Felleman DJ, Van Essen DC. Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex. 1991;1:1–47. doi: 10.1093/cercor/1.1.1-a. [DOI] [PubMed] [Google Scholar]
  10. Gaska JP, Jacobson LD, Pollen D. Response suppression by extending sine wave gratings within the receptive fields of neurons in visual cortical area V3A of macaque monkey. Vision Research. 1987;27:1687–1692. doi: 10.1016/0042-6989(87)90098-8. [DOI] [PubMed] [Google Scholar]
  11. Gattass R, Sousa APB, Gross CG. Visuotopic organization and extent of V3 and V4 of the macaque. J Neurosci. 1988;8:1831–1845. doi: 10.1523/JNEUROSCI.08-06-01831.1988. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Gerbing D, Anderson J. Monte Carlo evaluations of Goodness-of-fit Indices. In: Bollen KA, Long JS, editors. Testing Structural Equation Models. Sage Publications; Newbury Park, CA: 1993. pp. 40–65. [Google Scholar]
  13. Goncalves MS, Hall DA. Connectivity analysis with structural equation modelling: an example of the effects of voxel selection. NeuroImage. 2003;20:1455–1467. doi: 10.1016/s1053-8119(03)00394-x. [DOI] [PubMed] [Google Scholar]
  14. Grafton ST, Sutton J, Couldwell W, Lew M, Waters C. Network analysis of motor system connectivity in Parkinson’s disease: modulation of thalamocortical interactions after pallidotomy. Hum Brain Mapping. 1994;2:45–55. [Google Scholar]
  15. Grill-Spector K, Kushnir T, Edelman S, Itzchak Y, Malach R. Cue-invariant activation in object-related areas of the human occipital lobe. Neuron. 1998;21:191–202. doi: 10.1016/s0896-6273(00)80526-7. [DOI] [PubMed] [Google Scholar]
  16. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Mod. 1999;6:1–55. [Google Scholar]
  17. Jiang Y, Haxby J. Human Neural System for Perceiving and Attending to Rotating 3D Objects. NeuroImage Human Brain Mapping. 2002;2002:20174. [Google Scholar]
  18. Kanwisher N, Chun MM, McDermott J, Ledden PJ. Functional imagining of human visual recognition. Brain Res Cogn Brain Res. 1996;5:55–67. doi: 10.1016/s0926-6410(96)00041-9. [DOI] [PubMed] [Google Scholar]
  19. Kim DS, Kim M. Combining functional and diffusion tensor MRI. Ann N Y Acad Sci. 2005;1064:1–15. doi: 10.1196/annals.1340.005. [DOI] [PubMed] [Google Scholar]
  20. Koenderink JJ, Van Doorn AJ. Depth and shape from differential perspective in the presence of bending deformations. J Opt Soc Am Ser A. 1986;3:242–249. doi: 10.1364/josaa.3.000242. [DOI] [PubMed] [Google Scholar]
  21. Kourtzi Z, Tolias AS, Altmann CF, Augath MA, Logothetis NK. Integration of local features into global Shapes: monkey and human fMRI Studies. Neuron. 2003;37(2):333–346. doi: 10.1016/s0896-6273(02)01174-1. [DOI] [PubMed] [Google Scholar]
  22. Kourtzi Z, Bülthoff HH, Erb M, Grodd W. Object-selective responses in the human motion area MT/MST. Nature Neuroscience. 2002;5:17–18. doi: 10.1038/nn780. [DOI] [PubMed] [Google Scholar]
  23. Kourtzi Z, Kanwisher N. Representation of perceived object shapes by the human lateral occipital complex. Science. 2001;293:1506–1509. doi: 10.1126/science.1061133. [DOI] [PubMed] [Google Scholar]
  24. Kourtzi Z, Kanwisher N. Cortical regions involved in perceiving object shape. Journal of Neuroscience. 2000;20(9):3310–3318. doi: 10.1523/JNEUROSCI.20-09-03310.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kriegeskorte N, Sorger B, Naumer M, Schwarzbach J, Boogert E, Hussy W, Goebel R. Human cortical object recognition from a visual motion flowfield. Journal of Neuroscience. 2003;23(4):1451–1463. doi: 10.1523/JNEUROSCI.23-04-01451.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Maguire E, Vargha-Khadem F, Mishkin M. The effects of bilateral hippocampal damage on fMRI regional activations and interactions during memory retrieval. Brain. 2001;124(6):1156–1170. doi: 10.1093/brain/124.6.1156. [DOI] [PubMed] [Google Scholar]
  27. Malach R, Reppas JB, Benson RR, Kwong KK, Jiang H, Kennedy WA, Ledden PJ, Brady TJ, Rosen BR, Tootell RBH. Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex. Proc Natl Acad Sci USA. 1995;92:8135–8139. doi: 10.1073/pnas.92.18.8135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Martin A, Wiggs CL, Ungerleider LG, Haxby JV. Neural correlates of category-specific knowledge. Nature. 1996;379:649–652. doi: 10.1038/379649a0. [DOI] [PubMed] [Google Scholar]
  29. McIntosh AR, Gonzalez-Lima F. Structural equation modeling and its application to network analysis in functional brain imaging. Human Brain Mapping. 1994;2:2–22. [Google Scholar]
  30. Murray SO, Olshausen BA, Woods DL. Processing shape, motion and three-dimensional shape-from-motion in the human cortex. Cerebral Cortex. 2003;13:508–516. doi: 10.1093/cercor/13.5.508. [DOI] [PubMed] [Google Scholar]
  31. Ogawa S, Tank DW, Menon R, Ellermann JM, Kim SG, Merkle H, Ugurbil K. Intrinsic signal changes accompanying sensory stimulation: Functional brain mapping with magnetic resonance imaging. Proc Natl Acad Sci USA. 1992;89:5951–5955. doi: 10.1073/pnas.89.13.5951. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Orban GA, Sunaert S, Todd JT, Hecke PV, Marchal G. Human cortical regions involved in extracting depth from motion. Neuron. 1999;24:929–940. doi: 10.1016/s0896-6273(00)81040-5. [DOI] [PubMed] [Google Scholar]
  33. Paradis AL, Cornilleau-Pérès V, Droulez J, Van de Moortele PF, Lobe lE, Berthoz A, Le Bihan D, Poline JB. Visual perception of motion and 3-D structure from motion: an fMRI Study. Cerebral Cortex. 2000;10(8):772–783. doi: 10.1093/cercor/10.8.772. [DOI] [PubMed] [Google Scholar]
  34. Penny WD, Stephan KE, Mechelli A, Friston KJ. Modelling functional integration: a comparison of structural equation and dynamic causal models. NeuroImage. 2004;23:S264–S274. doi: 10.1016/j.neuroimage.2004.07.041. [DOI] [PubMed] [Google Scholar]
  35. Schoenfeld MA, Woldorff M, Düzel E, Scheich H, Heinze HJ, Mangun GR. Form-from-motion: MEG evidence for time course and processing sequence. Journal of Cognitive Neuroscience. 2003;15:157–172. doi: 10.1162/089892903321208105. [DOI] [PubMed] [Google Scholar]
  36. Siegel RM, Andersen RA. Perception of three-dimensional structure from motion in monkey and man. Nature. 1988;331:259–261. doi: 10.1038/331259a0. [DOI] [PubMed] [Google Scholar]
  37. Smith JF, Chen K, Johnson S, Morrone-Strupinsky J, Reiman EM, Eric M, Nelson A, Moeller JR, Alexander GE. Network analysis of single-subject fMRI during a finger opposition task. NeuroImage. 2006;32:325–332. doi: 10.1016/j.neuroimage.2005.12.010. [DOI] [PubMed] [Google Scholar]
  38. Sunaert S, Hecke PV, Marchal G, Orban GA. Motion-responsive regions of the human brain. Exp Brain Res. 1999;127:355–370. doi: 10.1007/s002210050804. [DOI] [PubMed] [Google Scholar]
  39. Tootell RB, Reppas JB, Kwong KK, Malach R, Born RT, Brady TJ, Rosen BR, Belliveau JW. Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J Neurosci. 1995;15:3215–3230. doi: 10.1523/JNEUROSCI.15-04-03215.1995. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Tootell RB, Mendola JD, Hadjikhani NK, Ledden PJ, Liu AK, Reppas JB, Sereno MI, Dale AM. Functional analysis of V3A and related areas in human visual cortex. J Neurosci. 1997;17(18):7060–7078. doi: 10.1523/JNEUROSCI.17-18-07060.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Tootell RBH, Hadjikhani NK, Mendola JD, Marrett S, Dale AM. From retinotopy to recognition: fMRI in human visual cortex. Trends in Cognitive Science. 1998;2(5):174–183. doi: 10.1016/s1364-6613(98)01171-1. [DOI] [PubMed] [Google Scholar]
  42. Treue S, Husain M, Andersen RA. Human perception of structure from motion. Vision Research. 1991;31:59–75. doi: 10.1016/0042-6989(91)90074-f. [DOI] [PubMed] [Google Scholar]
  43. Ungerleider LG, Mishkin M. Two cortical visual systems. In: Ingle DJ, Mansfield RJW, Goodale MA, editors. The Analysis of Visual Behavior. MIT Press; Cambridge, MA: 1982. pp. 549–586. [Google Scholar]
  44. Van Essen DC, Zeki SM. The topographic organization of rhesus monkey prestriate cortex. J Physiol (Lond) 1978;277:193–226. doi: 10.1113/jphysiol.1978.sp012269. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Wallach H, O’Connell DN. The kinetic depth effect. J Exp Psychol. 1953;45:205–217. doi: 10.1037/h0056880. [DOI] [PubMed] [Google Scholar]
  46. Wright S. The relative importance of heredity and environment in determining piebald pattern of guinea pigs. Proc Natl Acad Sci USA. 1920;6:320–332. doi: 10.1073/pnas.6.6.320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Zeki SM. Functional specialisation in the visual cortex of the rhesus monkey. Nature. 1978;274:423–428. doi: 10.1038/274423a0. [DOI] [PubMed] [Google Scholar]
  48. Zeki S, Watson JDG, Lueck CJ, Friston KJ, Kennard C, Frackowiak RSJ. A direct demonstration of functional specialization in human visual cortex. J Neurosci. 1991;11:641–649. doi: 10.1523/JNEUROSCI.11-03-00641.1991. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Zhuang J, LaConte S, Peltier S, Zhang K, Hu X. Connectivity exploration with structural equation modeling: an fMRI study of bimanual motor coordination. NeuroImage. 2005;25:462–470. doi: 10.1016/j.neuroimage.2004.11.007. [DOI] [PubMed] [Google Scholar]

RESOURCES