Dissociation and Convergence of the Dorsal and Ventral Visual Streams in the Human Prefrontal Cortex

Abstract

Visual information is largely processed through two pathways in the primate brain: an object pathway from the primary visual cortex to the temporal cortex (ventral stream) and a spatial pathway to the parietal cortex (dorsal stream). Whether and to what extent dissociation exists in the human prefrontal cortex (PFC) has long been debated. We examined anatomical connections from functionally defined areas in the temporal and parietal cortices to the PFC, using noninvasive functional and diffusion-weighted magnetic resonance imaging. The right inferior frontal gyrus (IFG) received converging input from both streams, while the right superior frontal gyrus received input only from the dorsal stream. Interstream functional connectivity to the IFG was dynamically recruited only when both object and spatial information were processed. These results suggest that the human PFC receives dissociated and converging visual pathways, and that the right IFG region serves as an integrator of the two types of information.

Introduction

In the primate brain, visual information is conveyed from retina to primary visual cortex by three pathways: the magnocellular, parvocellular and koniocellular pathways (Livingstone and Hubel, 1988). Although recent studies have found significant intermixing of different pathways in the early primary visual cortex (Lechica et al., 1992; Yabuta and Callaway, 1998), and in V2 (Sincich and Horton, 2002), it has also been suggested that such intermixing follows a specific pattern, such that new parallel streams of information are conveyed to the extrastriate cortex (Nassi and Callaway, 2009). After leaving the primary visual cortex, visual information is largely processed through two pathways that involve different cortical regions: the object pathway extending from the primary visual cortex to the temporal cortex (ventral stream) and the spatial navigation pathway to the parietal cortex (dorsal stream). Despite extensive connections between the dorsal stream and ventral stream (Felleman and Van Essen, 1991), each represents different features.

The existence of these two separate visual processing pathways was first proposed by Schneider (1969) followed by Ungerleider and Mishkin (1982) who, based on their lesion studies, suggested that the dorsal stream is involved in the processing of visual spatial information, such as object localization (where), and the ventral stream is involved in the processing of visual object identification information (what) (Ungerleider and Mishkin, 1982). Since this initial proposal, it has been alternatively suggested that the dorsal pathway should be known as the ‘How’ pathway, as the visual spatial information processed here provides us with information about how to interact with objects (Goodale and Milner, 1992). For the purpose of object recognition, the neural focus is on the ventral stream.

This concept of separate representation of different features has been further supported by lesion studies (Goodale and Milner, 1992; Ungerleider and Haxby, 1994), electrophysiology studies (Gross et al., 1972; Sato et al., 1980; Bruce et al., 1981; Fuster and Jervey, 1981; Desimone et al., 1984; Miyashita and Chang, 1988; Miller et al., 1991; Batuev et al., 1985; Gnadt and Andersen, 1988; Koch and Fuster, 1989; Barash et al., 1991a, b; Chafee and Goldman-Rakic, 1998), and histological tracer studies (Andersen et al., 1985; Felleman and Van Essen, 1991; Kaas 2004; Lyon 2007) in non-human primates.

But how are object and spatial information represented in the prefrontal cortex (PFC), which is positioned at the end of the sensory processing stream (Levy and Goldman-Rakic 2000; Courtney 2004)? This question has long been debated in non-human primate studies of the PFC, and there continue to be varying viewpoints about whether these types of information are processed in the same or different regions of the PFC (Fuster and Alexander, 1971; Kubota and Niki, 1971; Funahashi et al., 1989; Wilson et al., 1993; Rao et al., 1997; O’Scalaidhe et al., 1997).

Functional magnetic resonance imaging (fMRI) studies in humans have suggested that object (what) information is supported by the inferior frontal gyrus (IFG) or ventrolateral PFC (VLPFC), and that spatial (where) information is supported by the superior frontal gyrus (SFG) (Wilson et al., 1993; Courtney et al., 1996; Courtney et al., 1998a; Belger et al., 1998; Courtney et al., 1998b; Mohr et al., 2006; Sala and Courtney 2007; Volle et al., 2008). Another model suggests a hierarchical organization of the PFC, with short-term maintenance functions ascribed to the IFG/VLPFC, and higher order non-mnemonic functions (e.g., manipulation of items in memory) ascribed to the dorsolateral PFC (DLPFC), which is located anterior to the SFG (Petit et al., 1996; Owen et al., 1996; D’Esposito et al., 1998; Mohr et al., 2006). Although these models do not necessarily exclude one another, questions remain about how the posterior cortices and the PFC interact, and how the information from the posterior cortices is represented within the PFC. The goal of the current study was to explore whether there is an area in the PFC in which the information from the two streams converges, and/or whether there is no such area but connections underlying the two streams dynamically combine the information from the two streams.

In the macaque or rhesus monkey, the terms “VLPFC” and “DLPFC” are often used for the upper and lower bank of the principal sulcus respectively, and the term “IFG” is used for an area located posterior to the arcuate sulcus in the inferior prefrontal region. However, it is more elusive in humans how to term multiple areas in the prefrontal cortex. In this paper, we used the term “VLPFC” in the human brain to mention a broad area lower to the inferior frontal sulcus where in part the inferior frontal gyrus is located (e.g. Gilbert and Burgess 2008), and used the term “IFG” as a part of the VLPFC. We intended to use the term “IFG” only in the context of the human study and in the description of the current human study (e.g. activity in the IFG).

The anatomical connections to the PFC in the non-human primate have provided essential information about functional localization. A number of anatomical studies have shown that the VLPFC and DLPFC regions receive different connections from posterior association cortices. The VLPFC is connected to the inferotemporal cortex (Kawamura and Naito, 1984; Barbas, 1988; Barbas and Pandya 1989), a region involved in the representation of visual objects. In contrast, the DLPFC receives dense projections from the parietal cortex (Mesulam, et al., 1977; Petrides and Pandya, 1984; Andersen et al., 1985; Barbas and Mesulam, 1985; Cavada and Goldman-Rakic, 1989), a region involved in visuo-spatial processing. Findings showing that the DLPFC and VLPFC in non-human primates are the recipients of different projections (object versus spatial) have suggested a parallel organization of functional dissociation within the PFC, and it would therefore be reasonable to expect that regions receiving these different visual inputs are functionally related to the ventral and dorsal visual-processing streams with which they are selectively connected (Goldman-Rakic, 1988). However, our understanding of human brain connections has not advanced at the same level as our understanding of connections in the non-human primate brain, because studies involving invasive tracer techniques cannot be conducted in living human subjects. Furthermore, in postmortem human brains, the current tracer techniques permit definition if only short-range connections.

Recent technical advances in diffusion tensor imaging (DTI) (Basser et al., 1994) have allowed us to look at long-range white matter connections in the human brain (Mori et al., 1999; Jones et al., 1999; Conturo et al., 1999). The process of reconstructing the 3-dimensional pathways is called diffusion tractography. An increasing number of studies have used DTI to show anatomical connections in the human brain (Conturo et al. 1999; Basser et al. 2000; Stieltjes et al. 2001; Xu et al. 2002; Behrens et al. 2003; Lehericy et al. 2004; Powell et al. 2004; for review, see Johansen-Berg and Rushworth, 2009; Catani and Thiebaut de Schotten, 2009), and recent studies have used DTI to assess connectivity between functionally defined regions of interest (ROIs) (Guye et al. 2003; Toosy et al. 2004; Dougherty et al. 2005; Kim et al. 2006; Takahashi et al. 2007, 2008). In this study, we use a boot-trac algorithm (Lazar and Alexander 2005; Takahashi et al. 2007, 2008) to create probabilistic maps of DTI tractography.

Given that a cortical area can be characterized by its pattern of connections to other areas (Schmahmann and Pandya, 2006), and that brain functions rely on anatomical connectivity, studying connectivity patterns between posterior cortices and the PFC in humans may provide an important clue to understanding the interactions between them. Toward this end, we used diffusion tractography to examine whether anatomical connections from the parietal and temporal cortices project to distinct areas in the PFC, or if they converge on the same areas in the PFC. Although DTI provides information about anatomical pathways in vivo, there is no functional interpretation in the reconstructed fibers themselves. We performed a whole-brain functional connectivity analysis, looking at the functional coupling between functionally identified ROIs.

Materials and Methods

Subjects

We tested fifteen healthy, normally sighted subjects (7 males and 8 females, aged 21-39, mean age 26), all of whom reported themselves to be native speakers of English, right handed, with no neurological or psychiatric history. Written informed consent, in accordance with the Declaration of Helsinki, was obtained from each subject after the nature and possible outcomes of the study were explained. Image acquisition was completed at Boston University School of Medicine, and the scanning procedures were approved by the BU Institutional Review Board.

Stimuli

All stimuli were presented on a tangential screen positioned 1.1 m from the subjects. Thirty-six face stimuli from the Max Planck Institute, Tübingen, Germany (http://www.kyb.mpg.de/) were used for the ‘face’ condition The faces were framed by ovals so that hair was not visible. Faces and control stimuli were grayscale on a black background, and occupied a visual angle of 3.20° × 3.60°; they were presented at the center of the screen during the face run, and at one of nine peripheral locations during the combined run, in the encoding and judgment periods (Fig. 1). Control stimuli for faces were generated by randomizing phases of original face images in Fourier domains, to preserve spatial frequency components.

Figure 1.

Experimental design for functional imaging.

White dot stimuli for the ‘spatial’ condition were generated using Matlab software (The Mathworks Inc., Natick, MA, USA). The dot stimuli were presented on a black background, at one of the 8 positions located 2.50° visual angle away from the center in the encoding and judgment periods. The same white dots served as control dot stimuli, and were presented at the center of the screen.

Between the encoding and judgment periods, white crosshairs appeared at the center of the screen, while red crosshairs appeared during inter-trial periods. All stimuli were presented using presentation software from Neurobehavioral Systems Inc., Albany, CA, USA.

Procedures

The functional experiment consisted of three independent runs: ‘face’, ‘spatial’, and ‘combined’ runs. fMRI data were acquired during all runs. There were 4 blocks in each run, and one block consisted of 4 trials. Each trial lasted 18 seconds, consisting of a 6 sec encoding period followed by 6 sec maintenance, 2 sec judgment, and 4 sec inter-trial periods (Fig. 1). In each encoding period, 3 stimuli were displayed for 2 sec each. Each face was presented only once throughout all of the encoding periods. Subjects were asked to keep their eyes on the center of the screen as much as possible, and to remember the stimuli they had seen (faces in the ‘face’ run, locations of the dots in the ‘spatial’ run, and faces and locations in the ‘combined’ run) for the maintenance period. In the judgment period that followed, only one stimulus was displayed, and subjects were asked to indicate by pressing one of the two buttons on a box held in the right hand whether or not the same stimulus was shown in the encoding period of the same trial.

In the control blocks in the ‘face’ and ‘combined’ condition, noise ovals were displayed to replace the face stimuli. In the control blocks in the ‘spatial’ condition, a white dot appeared at the center of the screen. For the control blocks, subjects pressed either one of the two buttons randomly after the target stimuli appeared. This task protocol has been used in many studies to localize face and spatial representation in the brain (Courtney et al., 1996; Haxby et al., 1995; Courtney et al., 1998a). We used these short-term memory tasks to enforce stimulus encoding, and to ensure that both the posterior cortices and PFC were recruited to complete the tasks. The ordering of the ‘face’, ‘spatial’, and ‘combined’ runs were counterbalanced across subjects. We used face stimuli because we wanted to study object recognition that has a direct link to the real life (not abstract objects etc.). Among the objects in the real life (e.g. real objects, animals etc.), activation in the ventral pathway was most reliably identified by using human face stimuli. The functional experiment lasted approximately 18 minutes. Eye movements were not measured. However, subjects were instructed to fixate their eye movements at the center of the screen during the task. Subjects reported their response by pressing one of two buttons.

Image acquisition

We used a 3 Tesla whole-body scanner (Intera, Philips) to acquire gradient-echo echo-planar (GE-EPI) T2*-weighted BOLD-sensitive images and diffusion-weighted images based on spin-echo echo-planar imaging (SE-EPI). Conventional T1-weighted structural images were also obtained to provide anatomical information for each subject. We used the following parameters for BOLD functional image acquisition: repetition time (TR) = 2.6 sec, echo time (TE) = 35 ms flip angle = 90°, in-plane matrix size 128 × 128, field of view (FOV) = 230 mm × 230 mm, voxel resolution = 1.8 mm × 1.8 mm, number of slices 36, slice thickness 4 mm. Slice orientation was axial, and the imaging volume was aligned to cover the whole brain. Each scanning run commenced with the acquisition of 5 dummy volumes, allowing tissue magnetization to achieve a steady state, after which functional volumes were acquired (115 volumes for each run).

We used the following parameters for DWI acquisition: TR = 17.1 sec, TE = 80 msec, in-plane matrix size 128 × 128, FOV 230 mm × 230 mm, voxel resolution 1.8 mm × 1.8 mm, number of slices = 96, slice thickness = 1.5 mm, with fat suppression, b = 1,000 sec/mm², 15 directions, gradient strength = 0.2 G/mm, p reduction = 2.0. For each subject we acquired 16 data sets (15 diffusion weighted + 1 non-diffusion weighted images). A total of 4 signal averages were collected to ensure sufficient signal-to-noise ratio (SNR) for high-quality tensor mapping. From these data we calculated diffusion tensors for all imaged pixels. To compensate for motion, scans were acquired separately and then co-registered with others before averaging. Use of the sensitivity encoding (SENSE) technique reduced susceptibility artifacts significantly. Diffusion-weighted data were acquired in the same FOV with BOLD images to simplify post hoc spatial registration. Subsequently, the foci of the BOLD activations were used as seeding points for diffusion tractography. The same procedures were previously applied to dissociate anatomical pathways related to specific cognitive function (Takahashi et al., 2007, 2008).

Functional MRI Analyses

All functional images were initially analyzed with the SPM99 software for statistical parametirc mapping (Wellcome Department of Neurology, UK), and later with custom-made Matlab programs that were previously used in our studies (Takahashi et al., 2007, 2008). For each subject, we realigned the acquired images to the first volume to correct for head movement. Differences in acquisition timing between each slice were corrected for by using sinc interpolation. We spatially normalized each volume to a standard EPI template of 2 mm cubic voxels in the Talairach and Tournoux space (Talairach et al., 1988), using nonlinear basis functions. Each image was smoothed spatially with a Gaussian kernel of 8 mm full-width half-maximum (FWHM), and the time series was smoothed temporally with a 4-sec FWHM Gaussian kernel. Slow signal drifts were removed by high-pass filtering using cut-off periods of 170 sec. For each voxel, data were best-fitted (least squares) using a linear combination of regressors. The regressors were constructed to correspond to each trial type for each subject, and then convolved with the standard hemodynamic response function (HRF). We analyzed all trials with three regressors that corresponded to encoding, maintenance, and judgment periods, respectively. We did not separate correct and incorrect trials within blocks. Contrasts were examined first at the single subject level; then the resulting images were analyzed at the group level using t-tests (random effect analysis).

We set an initial height threshold of p < 0.001, and then used a threshold of p < 0.01 corrected for whole-brain multiple comparisons at the cluster level, according to the SPM standard procedures (Friston et al., 1994). The reason why we did not perform the correction for non-sphericity of the data was based on published suggestions by Friston et al. (2005) in which they reported that the two-stage procedure (SPM standard procedure) with sphericity assumptions about the impact of first level variance components on second-level parameter estimates is a very reasonable approximation in the vast majority of experimental situations. In addition, all of the subjects who participate on our study completed tasks of the same duration; hence, first-level error variance can be considered to be the same across subjects.

The location of each cluster was indicated by peak voxels in the normalized structural images, and labeled using the Talairach and Tournoux nomenclature (Talairach et al., 1988). Table 1 lists all of the activated clusters that passed the threshold (p < 0.01, corrected for multiple comparisons at the cluster level). If the cluster extended into multiple Brodmann’s areas, we separately noted the peak voxels in different areas. Because activated regions in the FG and IPS were bilateral and continuous across the midline, we separated them at the midline to create regions of interest for diffusion tractography analyses.

Table 1.

Regions activated to face stimuli (a) and to spatial stimuli (b).

	Cluster size	T-value	Coordinates (x, y, z)	R/L	BA
a) Face stimuli vs. Controls
	9544	9.77	-32, -74, -6	L	37	Fusiform Gyrus
			34, -58, -14	R	37	Fusiform Gyrus
	1143	6.73	18, 16, 30	R	32	Anterior Cingulate Cortex
		6.44	28, 38, 2	R	45/46	ventral Inf. Frontal Gyrus
	317	6.28	34, -48, 52	R	40	Inferior Parietal Lobe
	688	6.20	-4, 20, 56	L	6	Medial Frontal Cortex
	294	5.73	42, 10, 28	R	44/9	dorsal Inf. Frontal Gyrus
b) Spatial stimuli vs. Controls
	2077	7.96	-18, -66, 56	L	7	Intraparietal Sulcus
		7.86	-34, -78, 22	L	19/39	Middle Temporal Gyrus
		7.82	20, -66, 60	R	7	Intraparietal Sulcus
	284	5.62	34, 6, 66	R	6	Superior Frontal Gyrus

Only clusters with statistically significant activity (p < 0.01) corrected for whole-brain multiple comparisons are listed. The coordinates and their T values are at the peak voxels in each cluster, and the coordinates and Brodmann’s areas (BA) are indicated in the Talairach and Tournoux atlas space. 1 voxel corresponds to 8 mm³. See also Supplementary Table S2.

DWI Analyses

We realigned the DWI images using the diffusion toolbox in SPM2. The first images of each run were realigned to the first image of the first run to remove eddy current-induced distortions. All of the images were then averaged across the 4 runs. For each voxel, diffusion tensor and fractional anisotropy (FA) were calculated using standard procedures (Mori et al., 1999). After fitting the 15 DWIs by an ellipsoid tensor we removed voxels that had very large residuals; i.e., those voxels with residuals exceeding 35% of the value of the apparent diffusion coefficients (ADC) averaged across the whole brain. In all, approximately 15% of voxels distributed along the edge of the brain were removed. We defined the starting points for diffusion tractography using the T maps generated by SPM99, which resulted from random effect analysis of 15 subjects, as described in the “Functional MRI analyses” section. We used p < 0.001 (uncorrected) as an initial height threshold and p < 0.01 (corrected at the cluster-level) as the starting point criteria (Takahashi et al., 2007, 2008).

The starting points for tractography were set at intervals of 1 mm in the foci of fMRI activation clusters. The coordinates of the activated clusters in the Talairach space were reverse-normalized into each subject’s native space, and were used as the basis for fiber tracking. We then transformed the coordinates of the end points in the native space to the Talirach space, averaged the data across all subjects, and superimposed the resulting maps onto the normalized T1-weighted images in Talairach space. We did sub-voxel seeding without grid shift, and after each 0.5mm step, an interpolated value was calculated for each DWI. For reverse normalization and normalization of the coordinates we used T1 and DWI scans in the same FOV. We have previously used this approach to define major white matter pathways (Takahashi et al., 2007, 2008); this method provides greater accuracy than direct spatial normalization of DWIs, in which resampling reduces image resolution.

Diffusion Tractography

We used a tractography algorithm based on the method described in Lazar et al. (Lazar et al., 2003), a tensor deflection approach, developed to overcome crossing fiber problem (Alexander et al. 2001), that uses the entire tensor information instead of a single eigenvector (Lazar et al. 2003). Tractography was not terminated in voxels where the minor eigenvalue was minimal and therefore the reconstructed 3-dimensional tensor model was in a “pancake” shape. When a tractography pathway encountered such a voxel, we let tractography continue to the adjacent voxel following the direction in the previous voxel. We have previously applied this method, showing many examples of known fiber bundles for validation (Takahashi et al. 2007, 2008). At every position along the fiber trajectory a diffusion tensor was interpolated (linear) and eigenvectors were computed. The eigenvector associated with the greatest eigenvalue indicates the principal direction of water diffusion. Fiber tracts were propagated along this direction over a small distance (0.5 mm) to the next point, where a new diffusion tensor was interpolated. Tractography was terminated when the angle between two consecutive eigenvectors was greater than a given threshold (60°), or when the FA value was smaller than a given threshold (0.14). FA < 0.14-0.15 has been reported to provide the best tradeoff between fewer erroneous tracts and penetration into the white matter (Thottakara et al. 2006); we used this same criterion in our previous studies (Takahashi et al., 2007, 2008). Although the tensor deflection algorithm is suggested to overcome the crossing fiber problem, and 15 directions that we used is more than double of the minimum number of the diffusion gradient directions, having more diffusion gradient directions would give more accurate tractography results.

Group analysis in diffusion tractography – Population map

For Figure 2A and 2C, we performed diffusion tractography for all voxels in each activated cluster. To establish dissociation and convergence of the anatomical connectivity of the dorsal and ventral pathways, we used a heuristic algorithm that propagated tensorlines from both regions. We determined the terminal points of the tensorlines by calculating across subjects the probability that an end point from a particular seed region would fall within a voxel (Takahashi et al., 2007, 2008). The resulting map was superimposed onto normalized T1-weighted anatomical images. The maps, which we defined as “population maps,” were reported as the percentage of subjects in whom connections were found.

Figure 2.

(Upper panel) Activation maps superimposed on a three-dimensional brain. Areas in green were active to spatial stimuli, while areas in red were active to face stimuli during the encoding phase. The image on the left side represents the right hemisphere, and includes all activated areas with a height threshold of p < 0.001 (uncorrected). Activated areas with a height threshold of p < 0.001, corrected for whole-brain multiple comparisons at the cluster level (p < 0.01), are listed in Table 1.

(Lower panel) Group analyses (n = 15) of tractography from the right FG (A, B) and right IPS (C, D). Population maps (A, C) and BOOT-TRAC probabilistic maps (B, D) are superimposed on a T1-weighted template brain. Talairach coordinates of each section are shown on top of each panel. Red areas in the population maps (right FG in panel A, right IPS in panel C) represent activated areas in the right FG (A) and right IPS (C). The dark blue arrowheads point to areas around the right SFG, while light blue arrowheads point to areas around the right ventral IFG. See also Supplementary Table S1 and Table S2.

Estimation of tractography error by the bootstrap method – BOOT-TRAC probabilistic map

To ensure the reliability of tractography and we estimated the dispersion errors in diffusion tractography using a statistical nonparametric bootstrap method (BOOT-TRAC) (Lazar and Alexander, 2005) to evaluate intra- and inter-subject variability (Figure 2B and 2D). For each gradient direction, we randomly selected and averaged two of four datasets (volumes), and obtained an averaged 15-direction dataset, diffusion tensors, and reconstructed fibers. To obtain a BOOT-TRAC probabilistic map from multiple seeds we first performed fiber tracking from all seed points, for each bootstrap sample. We repeated this procedure 100 times and created probabilistic maps based on how many times, out of 100, fibers passed each voxel in the brain. We assigned a value of “1” to voxels that were passed by at one or more fibers, and “0” to voxels not passed by any fiber. We repeated this procedures for 100 bootstrap samples and determined probabilities for each “1” voxel; we refer to the resulting maps as “BOOT-TRAC probabilistic maps” (Takahashi et al., 2007, 2008).

Estimation of anatomical convergence

For display purposes, we projected the population maps onto a 3-dimensional template brain (Fig. 3A). This procedure was done by modifying one of the programs in the SPM package that is used to project activated regions onto the 3-dimensional template brain. Instead of activation maps, we used the population maps after thresholding them at 5. The threshold of 5 was set for the population analyses only for a visualization purpose. We displayed the terminal points of tractography from the activated areas in the right IPS (terminal areas are shown in green) and in the right FG (terminal areas shown in red). Overlap areas are shown in yellow.

Figure 3.

(A) Population maps of the dorsal and ventral pathways projected onto a three-dimensional template brain. Terminal points of tractography (n > 5) from the activated area in the right IPS are shown in green, and terminal points of tractography (n > 5) from the activated area in the right FG are shown in red. Overlap areas from the two pathways were shown in yellow. The dark blue arrowhead indicates the right SFG, and light blue arrowheads indicate the right ventral IFG. See also Supplementary Fig. S2.

(B) Activation maps superimposed on a three-dimensional brain during the encoding (dark blue) and maintenance (light blue) phases of the combined condition. The image on the left side represents the right hemisphere, and includes all activated areas with a height threshold of p < 0.001 (uncorrected). The light blue arrowhead indicates the right ventral IFG.

To further estimate anatomical convergence in a quantitative way, we examined whether the anatomical pathways from the dorsal stream (right IPS) and ventral stream (right FG) converged on the same voxels. We noted areas that were continuous across more than 100 voxels, and that showed more than 50% BOOT-TRAC probabilities in both pathways from the right IPS and the right FG (Supplementary Table S1). Only peak voxels located more than 16 mm apart are listed.

Functional connectivity analyses

We performed functional connectivity analysis of the activated regions (McIntosh et al., 1994) using the same procedure we used in our previous study (Takahashi et al., 2008). The fMRI signals were preprocessed for realignment, slice timing correction, normalization and spatial smoothing, in the same way as described above (fMRI Analyses section). We then removed low-frequency drifts in the time course in each voxel. We did not include the realignment parameters in the design matrix for the functional connectivity analyses. Power et al. (2012) reported that many long-distance correlations are decreased by subject motion, whereas many short-distance correlations are increased. In the current study, we focused on long-distance anatomical connections and functional correlation, so the subject motion could decrease the correlations but we may not falsely detect the long-distance correlations.

Given the spatial smoothing of the fMRI data, the activity of a single voxel can also be considered representative of the activity of the region around the voxel. Peak voxels were used for functional connectivity analysis (Talairach coordinates 34, -58, -14 in the right FG and 20, -66, 60 in the right IPS). We obtained maps of the correlation coefficients between time courses in the peak voxel (either the right FG or right IPS) and calculated other voxels for individual subjects. To make inferences about the functional connectivity encoded by these correlation coefficients at the between-subject level, we transformed them into summary statistics using Fisher’s Z Transform. The resulting within-subject maps were then passed to a second-level or between-subject one-sample t-test to generate SPMs, using standard methods. T values were corrected for multiple comparisons for a whole brain at the cluster level (p < 0.01), using p < 0.001 (uncorrected) as an initial height threshold, to be comparable to the fMRI analyses.

Results

Subject performance

Reaction times were 2250±158 msec for the face task, 2120±183 msec for the spatial task, and 2428±384 msec for the combined task. The percentages of correct response in the total judgments were 100% in the face and spatial tasks, and 86.0±4.6% for the combined task.

Dissociation of activation to face and spatial stimuli

In response to face presentation, blood-oxygen level-dependent (BOLD) activity was observed in the bilateral fusiform gyri (FG; Brodmann’s Area [BA] 37), and in two regions in the right inferior frontal gyrus (IFG; BA 45/46 (dorsal IFG) and BA 44/9 (ventral IFG)) (Fig. 2, upper panel, Table 1a; n = 15; p < 0.01, corrected for whole-brain multiple comparisons at cluster level, according to the SPM standard procedures (Friston et al., 1994)). We note that although we use IFG in this results section, where we discuss only human data, when comparing across species in the introduction and discussion sections, we used IFG/VLPFC in reference to humans and VLPFC in reference to non-human primates. We also observed activation in the right anterior cingulate cortices (ACC; BA 32), a region in the left medial frontal cortex (MFC; BA 6), the right MFC (BA 9), and the right inferior parietal lobe (IPL; BA 40).

In contrast, activation in response to spatial stimuli was observed in the bilateral intraparietal sulcus (IPS; BA 7) extending to left visual areas (BA 19/39), and in the right SFG (BA 6) (Fig. 2, upper panel, Table 1b). The activated region in the right SFG (BA 6; Talairach coordinate 34, 6, 66) was located more dorsal from the frontal eye field (FEF; BA 8), which suggested that SFG activity was not due to subjects’ eye movements. We noted weak activation in the left SFG region in response to the spatial stimuli, below the threshold (p > 0.01 corrected for multiple comparisons). Significant functional activation in response to face and spatial stimuli was clearly dissociated in both hemispheres. The right IPS region activated by spatial stimuli was located dorsal to the right IPL region activated by the face stimuli, and there was no overlap (at threshold of p < 0.01 corrected for whole-brain multiple comparisons at the cluster level).

Anatomical connectivity from the posterior cortices to the PFC

We next examined how the dorsal (parietal cortex) and ventral (temporal cortex) streams anatomically project to the PFC. We performed diffusion tractography to explore anatomical connections from the activated regions. Supplementary Figure S1 shows all of the tractography fibers in one subject, from the right FG activated by face stimuli (red) and from the right IPS activated by spatial stimuli (green). The tractography pathways from the right FG were restricted primarily to the ventral areas, projecting to the right temporal cortex and the right IFG. In contrast, the right IPS activated by spatial stimuli projected to both dorsal and ventral areas: the right SFG, the right temporal cortex, and the right IFG.

The lower panels of Figure 2 (2A and 2C) show the percentage of subjects in whom we found connections from the right FG (Fig. 2 A) and the right IPS (Fig. 2 C) to each voxel (“population map”; see Materials and Methods section for more details). We found a high probability of connections from the right FG in the right temporal pole and in the right IFG; similarly, we found high probability of connections from the right IPS in the right SFG and IFG. Figure 2B and 2D show the result of the BOOT-TRAC averaged across all subjects. These BOOT-TRAC probabilistic maps evaluate both intra- and inter-subject variability on final reconstructed fibers. The results were very similar to what we observed in the “population map” (Fig. 2A and 2C), confirming that our results are reliable even when accounting for dispersion error.

The population maps of tractography from the right FG (Fig. 2A) and the right IPS (Fig. 2C) were projected on a 3-dimensional template brain (Fig. 3A). As described above, the ventral pathways (Fig. 3A red) remained mainly in the ventral areas, and the dorsal pathways (Fig. 3A green) projected to the right SFG (an arrowhead in dark blue) as well as to the right IFG (arrowheads in light blue). Overlap areas (Fig. 3A yellow) were located in areas from the right posterior parieto-occipital junction to the right middle temporal gyrus, the right parahippocampal gyrus and hippocampus, and the right IFG. The right IFG was the only PFC area where the two streams showed convergence. In the left hemisphere, pathways from the left FG and left IPS projected to similar areas in the right hemisphere, but there was no converging area in the left PFC (Fig. 3A, see also Supplementary Fig. S2).

Intra- and inter-stream anatomical connectivity

To address connectivity within streams (dorsal to dorsal or ventral to ventral) and across streams (dorsal to ventral or ventral to dorsal), we examined connections between activated areas, across all subjects. We first focused on five areas: right IPS and right SFG regions activated by spatial stimuli (dorsal stream), and right FG and two right IFG regions activated by face stimuli (ventral stream). We refer to the two right IFG regions activated by face stimuli as right dorsal IFG (BA 44/9) and right ventral IFG (BA 45/46). Intra-stream anatomical connections within the ventral stream were found between the right FG and right ventral IFG in 14 subjects (93.3 %), and between the right FG and right dorsal IFG in 10 subjects (66.7 %). Intra-stream connections within the dorsal stream were found between the right IPS and the right SFG in 12 subjects (80.0 %). Overall, we found very consistent intra-stream connections with high probability, which likely support the functional dissociation in the PFC during the timeframe of our study. Interestingly, we also found a high probability of some inter-stream connections. The right IPS was anatomically connected to the right dorsal IFG in 12 subjects (80.0 %), and to the right ventral IFG in 12 subjects (80.0 %). In contrast, the right FG was connected to the right SFG in only 1 subject (6.70 %). We also found inter-stream connections between posterior areas: the right FG and right IPS were connected in 15 subjects (100 %). Connectivity among all activated areas is reported in Supplementary Table S2.

Examination of voxel-level convergence of anatomical connectivity

We examined for projection overlaps from the dorsal stream (right IPS) and ventral stream (right FG). In the PFC, we found that projection overlap was highly probable (more than 50 %) only in the right ventral IFG (BA 47/10, Talairach coordinate 48, 48, -8), even though we found overlapping projections in other posterior areas (Supplementary Table S1a). In the left hemisphere, we saw no convergence area in the left PFC (Supplementary Table S1b), consistent with the results shown in the previous section (Fig. 3A and Supplementary Fig. S2).

As described above, the right dorsal IFG region activated by face stimuli received anatomical connections from both the FG and IPS (Supplementary Table S2), but was not found to have overlapping projections at the voxel level from the FG and IPS (Supplementary Table S1). This was because projections from the IPS and FG terminated at different places in these areas.

Intra- and inter-stream functional connectivity

We found that two regions in the right IFG were connected to both the dorsal stream (IPS) and ventral stream (FG). The right SFG activated by spatial stimuli was connected to the right IPS only, not to the right FG. The inter-stream anatomical connectivity between the right IPS (dorsal) and the right IFG (ventral) seemed to disagree with the functional dissociation in the PFC that we observed in this study, although it may be in line with previous observations of IFG activation in response to both spatial and object short-term memory tasks (Smith et al. 1995; Owen, 1996; Rao et al., 1997; D’Esposito, 1998; Postle et al., 2000; Nystrom et al., 2000).

To evaluate the functional strength of connectivity between these areas, we examined functional correlation among these areas during both the face task and the spatial task. In agreement with anatomical connectivity results, intra-stream functional connectivity from the right FG to the right ventral IFG, and from the right IPS to the right SFG were found during both face and spatial tasks (Table 2). We found that inter-stream functional connectivity between the right IPS and the right ventral IFG was not significant in either the object-only or the spatial-only conditions (Table 3).

Table 2.

Magnitude of functional connectivity along intra-stream connections observed from diffusion tractography.

Areas	Task Condition	Average Z Value (n = 15)	Significance (n = 15)
Intra-stream connectivity
right FG and right vIFG	Spatial	0.299	p < 0.001*
	Face	0.214	p < 0.001*
right IPS and right SFG	Spatial	0.508	p < 0.001*
	Face	0.289	p < 0.001*

The asterisks show significant connectivity.

FG: Fusiform Gyrus, vIFG: ventral Inferior Frontal Gyrus, IPS: Intraparietal Sulcus, SFG: Superior Frontal Gyrus.

Table 3.

Magnitude of functional connectivity along inter-stream connections observed from diffusion tractography.

Areas	Task Condition	Average Z Value (n = 15)	Significance (n = 15)
Inter-stream connectivity
right IPS and right vIFG	Spatial	0.146	p = 0.020
	Face	0.170	p = 0.021
	Combine	0.203	p < 0.001*
right FG and right SFG	Spatial	0.138	p = 0.015
	Face	0.115	p = 0.017
	Combine	0.084	p = 0.014

The asterisk shows significant connectivity.

FG: Fusiform Gyrus, vIFG: ventral Inferior Frontal Gyrus, IPS: Intraparietal Sulcus, SFG: Superior Frontal Gyrus.

Interstream functional connectivity during integration of two streams

Here, we hypothesized that the interstream connectivity between the right IPS and the right ventral IFG is important only when combined object and spatial information is processed in the brain. To test this hypothesis, we asked subjects to remember both the positions and identities of faces in combination (see Materials and Methods). We found that the right IFG region was specifically active during the maintenance phase, not during the perception or retrieval phases of the combined condition (Figure 3B). More interestingly, we found that functional connectivity along this interstream pathway became significant during the combined condition (Table 3). These results suggest that the interstream pathway is dynamically recruited only when combined object and spatial information is processed in the brain.

Discussion

In this study, two parallel pathways between the posterior cortices to the PFC were observed from the right FG to the right ventral IFG/VLPFC, and from the right IPS to the right SFG. The dorsal pathway was activated by spatial information, while the ventral pathway was activated by object (face) information. The right SFG was connected only to the right IPS, not to the right FG. This suggested that the right SFG receives mainly spatial information from the right IPS. We also identified pathways from the right IPS to the right ventral IFG/VLPFC across the dorsal and ventral pathways. The right ventral IFG/VLPFC region received anatomical connections from both the right FG and right IPS. Interestingly, we found that functional connectivity along this interstream pathway became significant during the combined condition. These results suggest that the right ventral IFG region serves as an integrator of the two types of information during short-term memory. In summary, our results suggest that the right ventral IFG is one prefrontal area in the human brain to receive converging long-range anatomical pathways from both the dorsal and ventral visual streams, and the right ventral IFG region may incorporate spatial information from the right IPS with object information from the right FG, so as to combine and maintain both types of information.

Intra-stream long-range anatomical connectivity - Comparisons with non-human primate studies

Although anatomical connections in non-human primates have been well characterized between the posterior cortices and the PFC, evidence for connectivity in humans is still limited, in part by technilogical limits. Our tractography results that showed pathways running from the right IPS to the right SFG likely correspond to the human homologue of the second branch of the superior longitudinal fasciculus (SLF II), as described in the human brain (Thiebaut de Schotten et al., 2005) and monkey brain (Schmahmann, and Pandya, 2006). In the monkey, the SLF II originates in the caudal inferior parietal lobe (corresponding to the human angular gyrus) and the occipito-parietal area, and projects to the dorsolateral prefrontal cortex (Schmahmann, and Pandya, 2006); these results mirror our tractography results showing connections between the right IPS and the right SFG, originating from broad areas around the parieto-occipital junction that includes the ventral end of the right IPS and dorsal edge of the right FG.

Our results on the pathway from the right FG to the right ventral IFG/VLPFC also showed strong agreement with previous observations in the ventral course of the inferior fronto-occipital fasciculus (IFOF) in the human brain, which connects the occipital/posterior temporal lobes and the inferior frontal lobe (Mori et al., 2007; Catani and Thiebaut de Schotten, 2009). In non-human primates, the uncinate fasciculus is the major connection from the temporal lobe to the frontal lobe, while in humans, diffusion tractography studies have repeatedly revealed that the ventral pathway of the IFOF originates from the occipital and posterior temporal lobe and courses directly to the ventral frontal lobe (Mori et al., 2007; Takahashi et al., 2007), and that the uncinate fasciculus originates from the anterior temporal lobe and courses to the ventral frontal lobe (Mori et al., 2007). The ventral pathways of the IFOF seem to be unique to the human brain; previous studies have led researchers to suspect that the IFOF is related to visual processing (Catani and Thiebaut de Schotten, 2009), and that its disconnection differentiates the ventral frontal cortex from more posterior sources of visual input related to object identification (Urbanski et al., 2008).

Inter-stream long-range anatomical connectivity - Comparisons with non-human primate studies

Interestingly, we identified inter-stream anatomical pathways from the right IPS to the right ventral IFG/VLPFC region. Based on the available literature, this pathway may correspond to a branch of the IFOF in humans that courses from the parietal lobe to the frontal lobe (Mori et al., 2007). In non-human primates, the DLPFC receives dense connections particularly from the IPS and the superior parietal cortex, less so from the inferior parietal cortex. The IFG that is located posterior to the VLPFC (posterior to the arcuate sulcus), is connected not only to infero-temporal cortex, but also to the inferior parietal cortex (Schmahmann and Pandya, 2006). The third branch of the SLF (SLF III) originates from the posterior parieto-occipital junction and surrounding areas, and courses to the VLPFC; tracer studies (Schmahmann and Pandya, 2006) and tractography studies (Schmahmann et al., 2007) demonstrating the monkey SLF III pathway may be related to our tractography results showing connections from the right IPS to the right ventral IFG. However, it is still not clear whether the SLF III in monkeys correlates to a branch of the IFOF in humans. The anatomical converging region that we observed in the ventral IFG/VLPFC (BA 47/10 in humans) seems to be located more anterior ventral than the terminal areas of the SLF III in monkeys (ventral BA 6, ventral BA 9/46, and BA 44 in monkeys), and more anterior ventral than suggested possible human frontal areas that receive pathways from the inferior parietal lobe (Petrides et al., 2012). Further comparative examinations between humans and monkeys will be necessary in order to conclude whether monkeys have an anatomical converging area in the PFC, and if they do, how that area correlates to the ventral IFG/VLPFC area in humans.

Role of the right ventral IFG/VLPFC

The right ventral IFG/VLPFC in humans may represent a convergence zone of the two streams of visual processing: (1) the occipito-temporo-frontal stream, which is dedicated to object processing and passes through the IFOF and the uncinate fasciculus, and (2) the parieto-frontal network, which is presumably connected by the human homologue of the third branch of the SLF. We speculate that this area may play an important role in integrating object and spatial information. Indeed, in monkeys, neurons in the lateral PFC respond to both the location and identity of previously presented visual objects, which allows the integration of “what” and “where” information (Rao et al., 1997). IFG/VLPFC is, however, strongly involved in processing of working memory, and we are not able to completely exclude the possibility that the connectivity between the IPS and IFG was recruited because our study involved working memory tasks. Given that the combination of multiple features is an important aspect of working memory processing, it is difficult to dissociate those features—this remains a challenge for future studies. Further correlation studies for functional and anatomical connectivity in humans and monkeys will be important to clarify the role of the IFG/VLPFC in integrating the information from the posterior cortices. A possible way to separate these two processes is to use a working memory task that needs the combined object and spatial information for completing the task, and to compare activated regions by another working memory task that can be completed without combining the two types of information but by other strategies.

We observed activation in response to face stimuli in two right IFG/VLPFC regions: the right dorsal IFG/VLPFC (BA 44/9) and the right ventral IFG/VLPFC (BA 45/47); thee results are consistent with previous reports (Courtney et al., 1996; Haxby et al., 1995; Courtney et al., 1997; Fiez et al., 1996; McCarthy et al., 1996; Cohen et al., 1997) that referred to these regions as the posterior mid-frontal cortex and inferior frontal cortex. These studies also found activation in the anterior mid-frontal cortex (BA 46)/DLPFC, which was more active during a memory delay period (Courtney et al., 1997). We did not find this activation, probably because our study included a relatively small number of items (3 locations) and short delay (6 sec). Courtney and colleagues reported that such a short delay period with low memory load results in right hemisphere-dominant activation (Courtney et al., 1996), which is consistent with our results. We also found right hemispere-dominant activation in response to spatial stimuli in a right SFG region, as has also been seen in previous reports (Courtney et al., 1996; Courtney et al., 1998a; Petit et al., 1996; Mellet et al., 1996; Mohr et al., 2006).

Hemispheric asymmetry of brain function and anatomical connectivity

In this study, we focused mainly on activation and connectivity in the right hemisphere, because activation to face or spatial stimuli occurs predominantly in the right hemisphere. In the left hemisphere, we found somewhat different results. While we found no anatomical convergence area in the left PFC, the right ventral IFG/VLPFC was a convergence area. Several factors may explain this hemispheric asymmetry. One possibility is that the observed hemispheric asymmetry was influenced by handedness. For example, recent imaging studies have reported the relationships between handedness and brain activity during motor, memory, and language tasks (e.g. Cuzzocreo et al., 2009; Propper et al., 2010; for review, Hatta 2007) and some of them suggested that handedness reflects the laterality of hemispheric brain activity in task-related brain regions. However, some other studies did not show such clear relationships (Hatta 2007), and further examination should be done to clarify the relationships between the degree of handedness and deviation of functional organization in each cognitive event.

The other possibilities could be related to the way of our data processing. First, the sizes of the activated areas in the IPS and FG were larger in the right hemisphere than in the left hemisphere. Since we explored pathways from regions of interests (activated areas), the observed laterality of the pathways could be largely due to the locations and sizes of the activated areas. Larger activated areas in the right hemisphere are likely to be related to larger numbers of connections from the activated regions, which could result in a more comprehensive network. Second, the sizes of the activated areas depend on the threshold used in this study. It is possible that outer regions in the activated area in the left IPS might be weakly activated at a sub-threshold. These outer regions might have anatomical connections with the left ventral IFG/VLPFC. Third, anatomical connection patterns themselves may have hemispheric asymmetry. Given that functional hemispheric asymmetry has been repeatedly found in the human brain, it is not surprising to observe such asymmetry in anatomical connections in humans. However, it may not be very likely that the major anatomical bundles would have such large hemispheric asymmetry to the extent we observed between the IPS and the ventral IFG/VLPFC in this study.

Activation in the right parietal lobe

In the right parietal lobe, we found activation to spatial stimuli in the right IPS, and activation to face stimuli in the right IPL although their activated regions did not overlap at the threshold of p < 0.01 (corrected). This does not necessarily go against the idea of representation of spatial information in the parietal cortex. The right IPL region may process some kind of spatial properties of face stimuli, or may process detailed features of the face stimuli themselves. Our experimental design does not explicitly distinguish these two possibilities, but if the right IPL processes spatial properties of the face stimuli, spatial information together with the information in the right IPS may be transferred via the IPS-IFG/VLPFC pathway. If the right IPL itself processes face information, the two modalities of information could converge to some extent in the local connections within the right parietal lobe before they are further processed in the PFC (Konen and Kastner, 2008).

The hierarchical model represents non-mnemonic higher order functions in the PFC, such that manipulation of items in memory is ascribed to anterior mid-frontal cortex (BA46), and short-term maintenance functions are ascribed to the IFG (BA 45/47) (Petit et al., 1996; Owen et al., 1996; D’Esposito et al., 1998; Mohr et al., 2006). In our previous study, we found anatomical connectivity between these areas, as well as two parallel pathways between these two PFC areas and the FG (Takahashi et al., 2007). Although we did not find activation in the anterior mid-frontal cortex/DLPFC in this study, these connections would be important for manipulating information maintained in the IFG/VLPFC (Petit et al., 1996).

Our findings on the existence of the inter-stream anatomical connectivity between the right IPS and right ventral IFG/VLPFC may be in line with previous observations that both spatial and object short-term memory tasks activated the IFG/VLPFC (e.g., Owen, 1997; D’Esposito et al., 1998). In this study, however, we did not find activation to spatial stimuli in the right IFG/VLPFC, probably because of our relatively small number of items (3 locations) and short delay (6 sec). Indeed, in previous studies, direct comparison of activity during face and spatial stimuli revealed functional dissociation between ventral and dorsal frontal cortices; with face information processed in the IFG/VLPFC and spatial information in the SFG (e.g., Courtney et al., 1998a). Thus, the functional dissociation of the dorsal and ventral frontal areas might not be absolute, but relative, particularly in the ventral regions.

Functional connectivity along anatomical connectivity

In agreement with the functional dissociation, we found that the inter-stream functional connectivity along this pathway was weak during the object-only and spatial-only conditions. This result suggests that anatomical inter-stream connectivity exists between the right IPS and right ventral IFG/VLPFC, but its functional strength is weak during tasks that require only one type of information, that is, either face or spatial information. This may explain why the ventral IFG/VLPFC regions were preferentially activated by face stimuli, but were also sometimes activated by spatial stimuli. Thus, the functional dissociation in the PFC may be mutually supported by anatomical connectivity and functional connectivity. Interestingly, we found that functional connectivity along this interstream pathway is dynamically recruited only when both object and spatial information are processed in the brain. Thus, the functional connections together with the anatomical connections form a flexible network for supporting both segregation and integration of the dorsal and ventral visual streams.

Limitation of the current study

In the current study design, it was possible that the spatial task could have induced micro saccades. Micro saccades have been reported to cause similar activation patterns as small visually guided saccades (e.g. Tse et al., 2010), and therefore we cannot exclude the possibility that the activated regions in the spatial task in the current study could have related to the induced micro saccades. However, as we showed in the result section, we did not find activation in the eye movement related areas. Therefore we suspect that the effect of eye-movement to the results of working memory was minimal in this study.

We presented visual stimuli in a block design, because it seemed to be confusing for subjects to challenge randomized types of trials that would require set-shifting when the trial types changed, and because set-shifting would in turn require preparation. Even though it would be possible to subtract post hoc the component of set-shifting related activation, the task itself would be confusing to the subjects, and the error rate would likely increase as a direct result of set-shifting, which is not a focus of this study. The correct trials were 100% for the simple face and spatial tasks, and we only used the activated areas in the tasks with 100% performance as seed regions to initiate tractography. Therefore, the error did not affect our main results (tractography). However, the activation in the combined task would contain not only working memory but also an error-monitoring component.

The spatial and object tasks differ not only in spatial versus object (face) processing but also in the eccentricity of the stimuli, and thus, in principle it is possible that this eccentricity difference is partially responsible for the differential activation. However, the spatial stimuli used in this study were located at a visual angle 2.5 degrees away from the center, which was within the foveal representation (2.6 degrees), and could drive ventral streams in the temporal cortex, as well as dorsal streams in the parietal cortex (Levy et al., 2001; Hasson et al., 2002). The nature of the stimuli was also different between tasks: a simple dot versus a face. However, we used the matched stimuli in control tasks, and therefore the effect of the stimulus difference should be mostly removed by control tasks that we used for each condition.