The Large-Scale Organization of Object-Responsive Cortex Is Reflected in Resting-State Network Architecture

Abstract

Neural responses to visually presented objects have a large-scale spatial organization across the cortex, related to the dimensions of animacy and object size. Most proposals about the origins of this organization point to the influence of differential connectivity with other cortical regions as the key organizing force that drives distinctions in object-responsive cortex. To explore this possibility, we used resting-state functional connectivity to examine the relationship between stimulus-evoked organization of objects, and distinctions in functional network architecture. Using a data-driven analysis, we found evidence for three distinct whole-brain resting-state networks that route through object-responsive cortex, and these naturally manifest the tripartite structure of the stimulus-evoked organization. However, object-responsive regions were also highly correlated with each other at rest. Together, these results point to a nested network architecture, with a local interconnected network across object-responsive cortex and distinctive subnetworks that specifically route these key object distinctions to distinct long-range regions. Broadly, these results point to the viability that long-range connections are a driving force of the large-scale organization of object-responsive cortex.

Introduction

Visually presented objects evoke distributed systematic patterns across occipitotemporal cortex that are similar in both humans and nonhuman primates, and are strongly linked to processing shape differences (Haxby et al. 2001; Tanaka 2003; Kriegeskorte et al. 2008; for review see Kourtzi and Connor 2011; Ungerleider and Bell 2011). Much work has focused on characterizing the structure of these neural response patterns, including characterizing their dimensionality (Haxby et al. 2011), identifying key dimensions (Kriegeskorte et al. 2008; Op de Beeck et al. 2008; Konkle and Oliva 2012; Konkle and Caramazza 2013), and mapping their spatial topography (Chao et al. 1999; Hasson et al. 2003; Bell et al. 2009; Konkle and Caramazza 2013; Grill-Spector and Weiner et al., 2014;Weiner et al. 2014). Studies examining the spatial organization of neural response preferences have found that the topography is not random, but shows a reproducible and consistent arrangement of clustered selectivities: Within this cortex there is a mosaic of meso-scale regions with strong responses for categories such as faces, bodies, and scenes (Kanwisher 2010). These regions are systematically located in a larger macro-scale map of object preferences that show a tripartite distinction between response preferences for animals, big objects, and small objects (Konkle and Caramazza 2013). Important and unanswered questions remain about this topography: Why do we see these particular divisions and why do they have such a consistent spatial organization across the cortical mantle?

Current theories regarding the origins of this systematic large-scale topography point to differential connectivity as one of the key organizing forces (Malach et al. 2002; Mahon and Caramazza 2011). One of the earliest proposals suggested that spatial arrangement of object regions in occipitotemporal cortex was linked to the retinotopic organization of early visual areas in adjacent occipital cortex (Levy et al. 2001; Hasson et al. 2002; Malach et al. 2002). They found that regions responsive to faces were more associated with foveal stimulation, while regions responsive to houses were more associated with peripheral stimulation, and suggested that differential connectivity with early visual cortex might account for the topography and kinds of selectivities observed in adjacent high-level visual cortex. More recently, Mahon and Caramazza (2011) proposed that ventral stream organization is constrained not only by its relationship with early visual cortex, but perhaps more strongly by relationship to other long-range (nonvisual) regions. Specifically, they argued that for broad evolutionarily relevant domains such as animals and tools, innately patterned connections might exist to create functional routes between different brain regions, for example, between animate-looking shape information and emotion and social processing regions, and between tool-like shape information and motor planning and manipulation regions. To what extent are these differential connectivity predictions borne out in measures of functional network architecture?

A number of different studies have begun to explore the links between stimulus-evoked response preferences and functional and structural network architecture of these object regions (Mahon et al. 2007; Turk-Browne et al. 2010; Zhu et al. 2011; Saygin et al. 2012; Simmons and Martin 2012; Baldassano et al. 2013; Garrido et al. 2013; O'Neil et al. 2014; Stevens et al. 2015; see Kravitz et al. 2013 for review). To date, however, these studies have mostly focused on a particular category, for example, either faces, scenes, or tools, targeting more focal regions within this larger topographic organization (but see Hutchison et al. 2014). In contrast, here we explore what functional routes exist between the entire stimulus-evoked organization of objects and the rest of the cortex. Specifically, do any long-range regions show differences in their connectivity structure across this object-responsive cortex, and if so, how do these differential connectivities relate to the mirrored tripartite organization of object responses? And, conversely, do any long-range regions show similarities in their connectivity structure that span the major distinctions of the stimulus-evoked object organization?

Our approach is to leverage the structure in intrinsic temporal correlations between brain regions in a resting state (Fox and Raichle 2007, Biswal et al. 2010; Smith et al. 2013). Such correlations between regions are known to reflect a combination of both direct and indirect pathways. Thus, resting-state structures allow us to explore “functional” network architecture, contrasting with “anatomical” network architecture (Honey et al. 2009). In the current study, we take this broader functional connectivity measure to be an advantage, as both direct and indirect connectivity can be driving influences on the organizational structure of the ventral stream. However, we note that all differential resting-state structures characterized below should be interpreted carefully, reflecting relative differences in the degree of influence between any set of regions, and should not be taken as evidence for differential white-matter tracts.

Materials and Methods

Participants

Nineteen human observers with normal or corrected-to-normal vision participated in a 1–2 h fMRI session (age 18–40 years, all right handed). Informed consent was obtained according the procedures approved by the Institutional Review Board at the University of Trento. The 12 observers who participated in Experiment 1 reflect a subset of the participants from Konkle and Caramazza (2013), for whom a resting-state scan was also acquired. The remaining 7 new participants were scanned for the replication Experiment 2.

MRI Acquisition

Functional magnetic resonance imaging was used to measure blood-oxygen level-dependent responses in all participants both during a resting state and while viewing images of objects. Imaging data were acquired on a Bruker BioSpin MedSpec 4 T scanner using an 8-channel head coil. Functional data were collected using an echo-planar 2D imaging sequence (TR: 2000 ms, TE: 33 ms, Flip angle: 73°, slice thickness = 3 mm, gap = 0.99 mm, with 3 × 3 in plane resolution). Volumes were acquired in the axial plane parallel to the anterior–posterior commissure in 34 slices, with ascending interleaved slice acquisition.

Resting-State and Stimulus-Driven Protocols

During resting-state scans, observers were instructed to keep their eyes closed, think of nothing in particular, and to avoid falling asleep. Spontaneous resting-state fluctuations were measured in one 10-min run (300 volumes) in Experiment 1 and in two 5-min runs (150 volumes) in Experiment 2.

During the stimulus scans, responses were measured to objects that varied in whether they were animate or inanimate (animacy) and whether they were big or small in their typical size in the world (real-world size). Observers were shown images of big animals, small animals, big objects, and small objects (e.g., bear, hamster, couch, lightbulb), presented at the same visual size on the screen (~8° × ~8° visual angle), using a standard blocked design. The stimulus set contained 240 unique images (60 per condition) selected to have broad coverage over the categories, and is available for download on T.K.'s website. Each block was 16 s long, in which 16 images were shown for 800 ms each followed by a 200 ms blank, presented in isolation on a white background. A 10 s fixation period intervened between each block. Each run had 4 blocks per condition (213 volumes). All 60 images for each condition were presented once per run (4 blocks of 15 unique images). Observers were instructed to pay attention to each item and to press a button when an exact image repeated back-to-back, which occurred once per block. In Experiment 1, there were 6 total runs yielding 24 blocks per condition (these data are reported in Konkle and Caramazza 2013); in Experiment 2, there were 4 total runs yielding 16 blocks per condition.

To map early visual organization in Experiment 2, observers viewed bands of flickering checkerboards in blocked design. The conditions included vertical meridian bands (~22° × 2.5°), horizontal meridian bands (~22° × 2.5°), upper and lower horizontal bands (~22° × 4.5°, offset ±4.5°), and iso-eccentricity bands covered by a central ring (radius ~1.2° to 2.4°), a peripheral ring (radius ~5.7° to 9.3°), and an extra wide peripheral ring (inner radius ~9.3°, filling the extent of the screen). Each block was 6 s, within which the checkerboard cycled at 8 Hz between states of black-and-white, randomly colored, white-and-black, and random colored. In each 5.7-min run (174 volumes), the 7 visual field band conditions and 1 fixation condition were repeated 7 times with their order randomly permuted within each repetition. Each run started and ended with a 6 s fixation period. Participants’ task was to maintain fixation, and press a button every time the fixation dot turned red, which happened once per block.

Preprocessing

Functional data were analyzed using Brain Voyager QX software and MATLAB. Resting-state runs were preprocessed with slice scan-time correction, 3D motion correction, spatial smoothing (6-mm FWHM kernel), and were transformed into Talairach coordinates. The time course for each voxel was temporally filtered to preserve low-frequency fluctuations (0.009–0.008 Hz), and whole-brain and ventricle time courses and motion correction predictors were subsequently regressed out of the time series of each voxel. For Experiment 2, preprocessing was separately performed on each run and then the two time series were concatenated across runs.

Preprocessing of the stimulus runs included slice scan-time correction, 3D motion correction, linear trend removal, temporal high-pass filtering (0.01-Hz cutoff), spatial smoothing (6-mm FWHM kernel), and transformation into Talairach coordinates. To estimate the strength of the response for each stimulus condition (beta weight), the time course of each voxel was modeled using a general linear model with square-wave regressors for each condition's presentation times convolved with a gamma-function to approximate the hemodynamic response.

Region of interest definitions

In all subjects, we calculated 3-way preference maps for big objects, all animals, and small objects within visually active voxels (all vs. rest, T > 2.0), following Konkle and Caramazza (2013). For each participant, we defined 5 regions of interest (ROIs) to capture the peaks of the animacy × object size organization: 1) parahippocampal cortex (PHC) reflecting big-object preferences, 2) fusiform (Fus) cortex reflecting animal preferences; 3) inferior temporal gyrus (ITG), reflecting small-object preferences; 4) lateral occipital (LO) cortex reflecting animal preferences; and 5) cortex near the transverse occipital sulcus (TOS) reflecting big-object preferences. These ROIs were defined separately for each participant in each hemisphere, using the following procedure. First, the coordinate near the center of each zone was manually defined based on both anatomy and functional response preferences. Next, within a 12-voxel radius of these coordinates, we selected the top 50 voxels with the strongest preference strengths for the corresponding stimulus condition (defined as the peak response beta weight relative to the mean of the nonpeak beta weights). These animacy-size ROIs defined from the stimulus-driven data were used in all subsequent resting-state analyses.

Foveal and peripheral ROIs were defined by hand for each participant on their inflated cortical surface following the contrast of central versus peripheral bands, and were constrained to fall within early visual fields areas V1–V3, which were defined based on the contrast of horizontal versus vertical meridians. No voxels overlapped between any of the ROIs.

Profile-Clustering Analysis

To discover and characterize regions that differentially correlate with these functional subdivisions within object-responsive cortex, we developed a profile-clustering analysis. First, we obtained an “Animacy-Size profile” for each voxel in each participant, defined as the correlation between that voxel's time course and the average time course of each of the 5 animacy-size ROIs. Next, we selected a subset of these voxels with significant variance in the degree of correlation with each zone. This was assessed for each voxel across Experiment 1 subjects using a one-way ANOVA, and was corrected for multiple comparisons (false discovery rate <0.05). To explore what kind of profiles were present in this subset of voxels without presupposing any specific pattern, we used K-means clustering to group voxels together based on the similarity of their animacy-size profiles. The K-means algorithm used a correlation distance metric and selected the solution with the least sum-squared error over 10 randomly seeded replicates. The number of clusters (k) was varied between 2 and 10.

To test the replicability of the clustering results, in Experiment 2 we calculated new animacy-size profiles for the selected voxels used in Experiment 1, and then conducted the same K-means clustering analysis. To quantify the convergence between clustering solutions, we used a signal-detection method. Specifically, for both Experiments 1 and 2, we created a matrix of voxels × voxels with values equal to 1 if the voxels were assigned to the same cluster and 0 if they were assigned to different clusters. Hit rate was calculated as the percent of voxel–voxel pairs that were assigned to the same cluster in Experiment 1 and that were also assigned to the same cluster in Experiment 2. False alarm rate was calculated as the percent of voxel–voxel pairs that were not assigned to the same cluster in E1 but were assigned to the same cluster in E2. Sensitivity (d’) was subsequently calculated as z(Hit)–z(FA). To ensure the convergence between Experiment 1 and Experiment 2 was not likely to happen by chance, we conducted a permutation analysis. For each voxel in Experiment 2, we shuffled the animacy-size ROI labels, and computed the convergence of the shuffled solution with Experiment 1 (holding the Experiment 1 clustering solution constant), over 100 iterations.

ROI Resting-State Analyses

To further analyze the resting-state correlations between this set of ROIs, we assessed the similarity in the whole-brain resting maps using a paired voxel-wise procedure. For every pair of voxels both within an ROI and between any two ROIs, we extracted the rest time courses, and correlated these with all other gray matter voxels, yielding two whole-brain correlation maps (rMaps). The two seed voxels and the other voxels in each ROI were excluded from both maps, and then the two rMaps were correlated with each other. These rMap correlations were computed for all pairs of voxels within and between zones, and the resting-state correlation was averaged across voxel pairs in order to estimate the resting-state relationship between any two ROIs, for each participant.

Next, we empirically computed the distribution of possible pair-wise rMap correlations, considering all pairs of voxels within a gray-matter mask, for each participant. To determine if the animacy-size rMaps were more similar than different, a paired t-test was conducted between the 50th percentile rMap correlation and the average observed rMap correlation among the animacy-size ROIs.

For statistical tests, correlation coefficients (r) were transformed using Fisher's z-transformation to represent normally distributed variable z. Paired t-tests were conducted to test for overall differences in the rMaps between regions with similar versus different response preferences, by comparing the average strength of TOS-PHC and LO-Fus within-preference links with the average strength of TOS-Fus, PHC-LO, TOS-LO, PHC-Fus across-preference links. Targeted t-tests directly comparing pairs of ROI–ROI resting stating correlation strengths are reported in Supplementary Table S1.

To compare the similarity of the resting-state networks for each of the category-specific regions, we used category-selective ROIs for faces (fusiform face area/occipital face area, FFA/OFA), bodies (extrastriate body area/fusiform body area, EBA/FBA), and scenes (parahippocampal place area/occipital place area, PPA/OPA), defined in each participant of both experiments. The similarities between all pairs of rMaps were computed and analyzed following the same procedure as used for the animacy-size ROIs. To examine whether regions with the same animacy preference (e.g., seeded by face and body regions) had more similar resting-state maps than regions with different animacy preferences (e.g., seeded by face and scene regions), we conducted a t-test over the average strength of the within-animacy links with the average strength of the between-animacy links (within-animacy links: FFA-FBA, EBA-OFA, FFA-OFA, FBA-EBA, FBA-OFA, PPA-OPA; between-animacy links: PPA-FFA, PPA-FBA, PPA-EBA, PPA-OFA, OPA-FFA, OPA-FBA, OPA-OFA, OPA-EBA).

Results

Distinct Functional Routes Through the Animacy-Size Organization

For each participant, the large-scale organization of animal and object size response preferences were mapped following the procedure reported in Konkle and Caramazza (2013). That is, given a set of visually responsive voxels, we compute a 3-way preference map, where each voxel is colored by the stimulus condition for which it has the strongest response among big objects, animals, and small objects. This analysis reveals the spatial distribution of voxel-wise response preferences for visually presented images of big objects (e.g., couch, car), animals (e.g., cow, chicken), and small objects (e.g., cup, keys), depicted for an example participant in Figure 1A. This stimulus-evoked organization has a large-scale mirrored topography (Konkle and Caramazza 2013): at both edges of the map there are zones of cortex with response preferences for big objects, near the TOS and along the PHC. Adjacent to these regions, there are zones with response preferences for animals, along both the Fus gyrus and lateral occipital cortex (LO). Finally, at the center of the map there is a region with response preferences for small objects, around the ITG.

Figure 1.

Stimulus-evoked large-scale organization of occipitotemporal cortex in a representative participant. (A) Three-way preference map showing the spatial organization of response preferences for big objects (blue), animals (purple), and small objects (orange). The strength of the preferential response is reflected in the color saturation. (B) Five ROIs were defined for each of the preference zones in each participant.

It is important to note that all the stimulus conditions drive all of these zones of cortex to different degrees, where each zone of cortex has graded response magnitudes for the big and small animal and object conditions (Konkle and Caramazza 2013). Here, we use the stimulus condition that drives the maximal response only as a label; these zones of cortex are not “category-selective regions” for animals, big objects, and small objects, but likely reflect the fact that some shape properties are particularly emphasized in the categories that elicit the strongest responses. Based on each individual's animacy-size map, 5 ROIs were defined corresponding to the most preferential voxels in each zone (see Materials and Methods). These ROIs are referred to subsequently as “animacy-size ROIs,” and are illustrated for an example participant in Figure 1B.

What functional routes exist between this animacy-size organization and the rest of the cortex? To explore this question, we analyzed the structure in resting-state correlations between these regions and other regions across the cortical surface. The goals of this analysis were to 1) isolate regions in the brain where there are reliable differential resting-state correlations with these object regions and 2) understand how these resting-state networks are related to the stimulus-evoked organization. To do so, we developed a profile-clustering analysis (Fig. 2), which has two critical features.

Figure 2.

Profile-clustering analysis procedure. (A) For all voxels in a gray matter mask (outside of the animacy-size ROIs), we computed an “animacy-size profile,” defined as the correlation between that voxel's time course and the average time course of each of the 5 animacy-size ROIs. (B) We selected the subset of voxels, which showed differential correlations with the 5 animacy-size ROIs, reliably across participants in Experiment 1. To explore what kind of animacy-size profiles were observed in these selected voxels, a K-means clustering algorithm was used. To test the replicability of the clustering solution, the same clustering analysis was carried out using the resting-state data from Experiment 2.

First, we targeted cortical regions that have distinctive functional connections with the animacy-size ROIs. For each voxel in the brain, we computed an “animacy-size profile,” which reflects the voxel's resting-state correlation with each of the 5 functionally defined ROIs (Fig. 2A). Given these profiles for all voxels and all participants, we selected only the voxels that reliably and differentially correlated with the animacy-size regions across participants in Experiment 1, based on a one-way ANOVA (see Materials and Methods). This voxel-selection method isolates the neural regions that have distinctive relationships with one or more of the animacy-size zones.

Second, we used a data-driven method to explore the network structure in this set of voxels. Specifically, we used a clustering analysis to group voxels with similar animacy-size profiles, both in the original experiment, and in a replication experiment (Fig. 2B, see Materials and Methods). The critical feature of this clustering analysis is that it does not presuppose any particular relationship between long-range regions and the animacy-size ROIs. For example, we are not specifically seeking long-range voxels that more strongly correlate at rest with the animal zones versus the other zones, or otherwise seeking connectivity structure that supports a mirrored tripartite organization. Instead, we take advantage of a data-driven clustering analysis to discover how long-range regions differentially correlate with these object regions. This analysis considers a large hypothesis space of possible profiles (e.g., some regions may correlate at rest most strongly with the lateral vs. medial surface, other regions may correlate at rest most strongly with both animal-preference zones over the other preference zones). Indeed, there are 2⁵ (32) different possible profiles if we simplify the space of possible profiles to only binary on–off correlations with each zone.

The best solution of the profile-clustering analysis, and its replication, are shown in Figure 3. Given the space of possible long-range connectivity profiles, the key result is that a tripartite division naturally emerges from the data-driven analysis. That is, voxels in cluster 1 are relatively more correlated with both big-object ROIs than animal or small-object ROIs; voxels in cluster 2 are relatively more correlated with both animal ROIs than the object ROIs; voxels in cluster 3 are relatively more correlated with the small-object ROI than the other animacy-size ROIs. The results are shown for the original experiment (Fig. 3A) and a replication experiment (Fig. 3B). Generally, these clusters indicate that big-object regions have stronger functional connectivity with early peripheral cortex and medial occipital cortex, animal regions have stronger functional connectivity with lateral temporal lobe regions, and small-object regions have stronger functional connectivity with parietal regions (Fig. 4). Additionally, we found that a 3-cluster solution is the best, considering a range of 2–10 possible clusters (see Supplementary Figs S1 and S2 for cluster solutions for k = 2–8, for both Experiment 1 and Experiment 2, respectively). In both experiments, the 3-cluster solution had the highest silhouette value, a measure that takes into account the average similarity of each voxel to its assigned cluster versus voxels in other clusters (Fig. 5A).

Figure 3.

Profile-clustering solution. (A) Voxels were clustered by similarity in their animacy-size profiles, and are color-coded by cluster. (B) Normalized 5-seed correlation profiles for each cluster. The relative r-values are shown on the y-axis, for each of the 5 ROIs along the y-axis. These profiles are referred to as the “cluster centers,” and reflect the relative correlation strength with each of the 5 seeds, averaged across voxels in the same cluster. Note the cluster center profiles are normalized—that is, they reflect relative differences in correlation strength, and the average across ROI-seeds must be zero. Left: Experiment 1. Right: Replication Experiment 2.

Figure 4.

Differential voxel locations. Histograms of the selected long-range voxels by their coarse anatomical locations, separated by the clusters solution of Experiment 1. Voxel counts are shown for the left and right hemisphere, binned by their anatomical location (based on the top 20 of annotated regions in freesurfer's 36-parcellation scheme).

Figure 5.

Profile-clustering analysis assessments. (A) For each clustering solution, the average silhouette value is plotted as a function of the number of clusters. Experiment 1 is plotted in the dashed line; replication Experiment 2 is plotted in the solid line. (B). Sensitivity of Experiment 2 to detect the clustering in Experiment 1 (d’) is plotted as a function of the number of clusters (solid line). Results of the permutation analysis of Experiment 2 reflect chance level (dashed line). The light gray bar highlights the peak silhouette values and peak sensitivity, and the corresponding number of clusters.

Qualitatively, there is a clear correspondence between the clustering solutions found in the original experiment and the replication experiment. To quantify the degree of convergence, we used a signal-detection analysis to compute the sensitivity of the Experiment 2 clustering solution to detect the Experiment 1 clustering (see Materials and Methods). Figure 5B shows that for cluster counts of 3 and higher, there was strong convergence between the two experiments (d’ > 1). The strongest convergence was at k = 3 (d’ = 1.5), which indicates that these long-range voxels are best grouped into 3 clusters based their animacy-size profiles. A permutation analysis revealed that the high convergence between experiments is very unlikely by chance (d’ = 0 for k = 2–10, see Materials and Methods). Taken together, these profile-clustering analyses demonstrate that there are long-range voxels that reliably and differentially correlate with object-responsive cortex in a way that matches the large-scale functional organization by animacy and object size. In other words, there is a naturally emergent long-range network architecture that manifests the stimulus-evoked network.

Relationship to Fovea and Periphery

While some proposals focus on downstream connections (beyond visual cortex) as key drivers of the organization of object cortex (Mahon and Caramazza 2011), other proposals highlight the relationship with early visual retinotopic organization as a key driving factor (Hasson et al., 2002). Indeed, in the profile-clustering analysis, some occipital regions were included in the voxel selection, consistent with the idea that these early visual regions may differentially correlate with object-responsive cortex. To test for this relationship directly, in the replication experiment we included visual field mapping protocol in order to define subject-specific ROIs for foveal and peripheral retinotopic cortex, and for upper and lower visual field cortex, spanning visual areas V1–V3. Then, we measured the resting-state correlations between the animacy-size regions with these large-scale divisions of retinotopic cortex.

First, we focused on the large-scale eccentricity organization, characterizing how each animacy-size ROI was correlated with foveal and peripheral cortex. Current proposals would predict that big-object regions are more correlated with peripheral cortex while animate and small-object regions are more correlated with foveal cortex (Levy et al. 2001; Hasson et al. 2002; Malach et al. 2002; Konkle and Oliva 2011). Overall, we observed a significant interaction in resting-state correlation strength between the eccentricity organization and the animacy-size organization (Fig. 6A; 2 × 5 repeated measures ANOVA: F(4,69) = 3.4, P = 0.026). Planned paired t-tests reveal that both big-object regions were more correlated at rest with the peripheral ROIs than foveal ROIs (TOS: t(6) = 3.01, P = 0.024, meanDiff = 0.16; PHC: t(6) = 5.42, P = 0.002, meanDiff=0.23). In contrast, the animal and small-object ROIs did not show a reliably stronger correlation with either the fovea or the periphery (all Ps > 0.3). These results suggest that there are indeed different functional routes between the eccentricity organization of early visual cortex and the animacy-size organization of object-responsive cortex, but these are largely driven by differential peripheral connections.

Next, we explored the other major retinotopic division between the upper and lower visual fields. While V1 has a complete visual field map, V2 and V3 have both a ventral and a dorsal component, corresponding to the upper and lower visual fields, respectively. Early visual cortex above the calcarine sulcus corresponds to the lower visual field; early visual cortex below the calcarine sulcus corresponds to the upper visual field (Wandell et al. 2007). Some of the earliest observations of mirrored object-responsive regions proposed this duplication may be driven by an extended retinotopic organization; it follows that the ventral surface regions (Fus/PHC) are extended from the adjacent upper visual field representations and the lateral surface regions (TOS/LOS) are extended from the adjacent lower visual field (Silson et al. 2015). To explore this possibility, we computed the resting-state correlations between the object regions and the upper and lower visual fields. No significant interaction was observed between the upper and lower visual field ROIs and the animacy-size regions (P > 0.2), but planned paired t-tests revealed both TOS and LO had stronger correlations with the lower visual field (Fig. 6B; TOS: t(6) = 2.60, P = 0.04, meanDiff = 0.10; LO: t(6) = −3.89, P = 0.008, meanDiff = 0.12). These results suggest that there is subtle evidence for upper/lower visual field differential connectivity, driven primarily in these data by lower visual field biases with the lateral surface of object-responsive cortex.

Figure 6.

Relationship to retinotopy. (A) Seed-time course correlations between the 5 animacy-size zones with the fovea and peripheral retinotopic cortex. Note that the band of peripheral cortex (dark blue) extends both to the lateral side (adjacent to TOS) and the ventral side (adjacent to PHC). (B). Seed-time course correlations between the 5 animacy-size zones with the upper and lower visual field. Note that early visual cortex corresponding to the lower visual field (dark green) is nearer to the lateral surface, while upper visual field (light green) cortex is nearer to the ventral surface. Adjacent to each plot, the ROIs are shown for one participant on an inflated cortical surface, with a view centered on the occipital pole.

Common Network Across the Animacy-Size Organization

Our analyses thus far explored the distinctive functional routes from early visual cortex, through these object-zones, to other long-range regions across the whole brain. In other words, we specifically isolated voxels that differentially correlate with the animacy-size organization. These analyses cannot highlight any commonalities that may exist in the whole-brain resting-state networks of these object regions. Thus, we next explored how each of the animacy-size ROIs correlate with the whole brain at rest, and then computed the similarity of these whole-brain resting-state maps.

The results of this analysis are shown in Figure 7A and for the replication experiment in Supplementary Figure S3. Overall, regions with similar stimulus response preferences also showed more similar whole-brain resting-state maps (t(11) = 2.40, P = 0.035). That is, the mirrored organization of object preferences is also reflected in a mirrored resting-state structure, consistent with the profile-clustering analysis. However, the most striking observation is that the whole-brain resting-state maps for each of these regions are highly similar. A visualization of these seeded resting-state maps is shown in Figure 7B.

Figure 7.

Whole-brain rest map correlations. (A) Similarity structure of the whole-brain resting-state maps. The average correlation for each ROI–ROI combination is plotted in a network diagram, with the color of the links indicating the average correlation between the two whole-brain resting-state networks. The nodes are organized with a pseudo-anatomical arrangement, to more clearly illustrate the mirrored resting-state structure. (B) Visualization of thresholded whole-brain resting-state maps, seeded for each animacy-size ROI. The colored cortex reflects all voxels with an average seed resting-state correlation >0.2.

 The results of this analysis are shown in Figure 7A and for the replication experiment in Supplementary Figure S3. Overall, regions with similar stimulus response preferences also showed more similar whole-brain resting-state maps (t(11) = 2.40, P = 0.035). That is, the mirrored organization of object preferences is also reflected in a mirrored resting-state structure, consistent with the profile-clustering analysis. However, the most striking observation is that the whole-brain resting-state maps for each of these regions are highly similar. A visualization of these seeded resting-state maps is shown in Figure 7B.

 Given that the whole-brain time course was regressed from each voxel in preprocessing, the range of possible whole-brain resting-state map correlations spans from fully anticorrelated (r = −1) to fully correlated (r = 1). Across the possible pairs of regions, we found that the resting-state map correlation is relatively high, with an average correlation of r = 0.40. To put this value into context, considering the empirical distribution of these correlations, a map similarity of r = 0.40 is greater than 80% of all possible pairs of voxels in the whole brain. Thus, in the space of possibilities, each of these object zones could have a highly distinctive whole-brain network; however, the results indicate that on average, their whole-brain resting networks are much more similar than different (t(11) = 6.55, P < 0.001).

 To understand which neural regions were commonly correlated at rest across all of the animacy-size ROIs, we visualized the overlap among the 5 resting-state maps (Fig. 8; see Supplementary Fig. S3). These 5 resting-state maps all strongly overlap primarily in a large contiguous extent spanning the entire extent of occipitotemporal cortex. There is little if any overlap in other longer-range regions across the cortical surface. These results point to the existence of a common, more localized, interconnected network across the extent of object-responsive cortex. This result is interesting in the context of different connectivity accounts of this organization, as it highlights that internal structure within the occipitotemporal cortex itself might also contribute to the large-scale clustering and segregation of response preferences across this cortex (Polk and Farah 1995; Op de Beeck et al. 2008; Behrmann and Plaut 2013).

Figure 8.

Overlap of seeded whole-brain resting-state maps. (A) Visualization of the overlapping locations of each of the 5 animacy-size seeded resting-state maps. (B) Histograms of the overlapping voxels by their coarse anatomical locations. Overlapping voxels were taken as those with a resting-state correlation >0.2 for all 5 of the animacy-size ROIs. Voxel counts are shown for the left and right hemisphere, binned by their anatomical location.

Comparing Category-Selective ROI Networks

One open question from these results is whether the three subnetworks we found routing through occipitotemporal cortex are relatively specific to our particular stimulus categories (animals, big objects, small objects), or whether they are more generic networks that reflect general large-scale routes shared by regions with similar spatial layouts in cortex. To gain some insight into this issue, we explored the resting-state maps seeded by meso-scale regions selective for faces, bodies, and scenes. Specifically, we can take advantage of the fact that both face and body regions fall within the animal-preferring zones (Konkle and Caramazza 2013). Do face and body ROI-seeds yield highly distinctive whole-brain networks, even though both ROIs have an animate preference? To the extent that the face and body networks are highly differentiable, it would suggest that there are likely (many) more subnetworks through OTC than we have characterized here. However, to the extent that these networks are highly similar, it would suggest that they may follow a more domain-general, animate-preferring network shared by regions with similar spatial layouts in cortex (cf. Weiner et al. 2014; Grill-Spector and Weiner 2014).

To approach this question, we quantified whether face and body resting-state networks were more similar to each other than to scene resting-state networks (see Materials and Methods). On average, the pairs of regions with similar animacy preferences (e.g., including FFA and EBA) had more similar resting-state networks than pairs of regions with different animacy preferences (e.g., FFA and PPA, EBA and OPA; t(18) = 5.18, P < 0.001). In addition, pairs of regions both of which preferred faces (FFA/OFA) or bodies (EBA/FBA) had whole-brain resting-state networks that were as similar as pairs of regions that spanned face/body preferences (FFA/FBA, FFA/EBA, OFA/FBA, OFA/EBA; t(17) = −0.73, P = 0.473). However, we also used a support-vector machine classification analysis to explore whether it was possible with a more sensitive method to tell apart face-seeded networks from the body-seeded networks, and found that most pairs spanning face and body-seeded resting-state maps could be classified above chance, with the exception of the FFA and FBA (see Supplementary Materials). Taken together, these results point to relatively domain-general, large-scale network distinctions shared by regions with similar layouts and response preferences, where face- and body-network differences might be thought of as refinements within a common animate-preferring network.

Discussion

Here, we traced the distinctive functional routes between early visual cortex, object zones, and the rest of the brain. Using a data-driven analysis, we show evidence for three distinctive whole-brain networks, which route through object-responsive cortex, consistent with the major tripartite division revealed by the stimulus-evoked responses. Specifically, regions with a response preference for animals were relatively more correlated with regions in the temporal lobe, regions with a response preference for small objects were relatively more correlated with parietal cortex along the intraparietal sulcus, and regions with a response preference for big objects were relatively more correlated with medial temporal and early peripheral visual cortex. Furthermore, we also observed that these object-responsive regions are part of a common network that spans the entire occipitotemporal cortex. Considering both the similarities and the differences of resting-state structure, the results point to a nested network architecture, characterized by a common local interconnected network across object-responsive cortex, with three distinct subnetworks of longer-range connections routing through the animal, small-object, and big-object responsive parts of this cortex. Broadly, these results provide empirical support for the proposals that argue differential network connectivity may drive the consistent large-scale organization of object responses across this cortex (Malach et al. 2002; Mahon and Caramazza 2011).

How should one interpret the finding that these regions are simultaneously part of a common network, but also part of distinctive subnetworks? One observation to consider is that this nested network architecture identified via resting-state measures bears a strong resemblance to the structure of the stimulus-evoked organization (Fig. 9). That is, all kinds of these visual stimuli (animals and objects alike) drive the entire occipitotemporal cortex relative to fixation, akin to the common network at rest (Fig. 9A). And, particular kinds of objects (e.g., big objects, animals, small objects) drive some parts of this cortex relatively more than other parts, and indeed those same stimulus differences are linked to the 3 distinctive subnetworks (Fig. 9B). Thus, there is an intriguing match between the hierarchical organization of “extrinsic” activation patterns and the similarity structure of the “intrinsic” network architecture. This match between evoked and intrinsic structures also dovetails with the broader resting-state literature, where the major task-evoked networks are quite similar to major resting-state networks (Cole et al. 2014; Laumann et al. 2015), with more subtle task-differences evident within targeted subnetwork components (Hasson et al. 2009, Buckner et al. 2013, Mennes et al. 2013, Krienen et al. 2014).

Figure 9.

Comparing extrinsic response profiles and intrinsic resting-state structure. (A) The contrast of all objects greater than a fixation baseline, for an example participant (left) is plotted next to a visualization of the common regions most strongly correlated at rest with each of the animacy-size ROIs (right, as in Fig. 8). (B) The 3-way response preference map for animals, big objects, and small objects, is shown for an example participant (left), plotted next to the profile-clustering results, which yielded 3 major distinctions in the long-range regions that route through this object-responsive cortex.

Relationship to Long-Range Regions

Mahon and Caramazza (2011) proposed that certain evolutionarily relevant domains will manifest in a large-scale spatial organization across ventral stream, linked to distinctive whole-brain network architecture. However, their account did not articulate how many domains there are likely to be. Here, we find evidence for 3 distinctive subnetworks that route through our animacy-size organization, following the tripartite division in the stimulus-evoked responses. Our data-driven analysis could have yielded a number of distinct long-range regions that correlated with different combinations of the 5 seeded animacy-size ROIs (e.g., a long-range region that correlated highly with TOS and LO and not the others); however, this more intricate structure did not fall out of the data-driven analysis. In addition, face and body-seeded resting-state maps were much more similar to each other than to scene resting-state maps, indicating that these functional regions are likely part of the same animacy subnetwork. Taken together, a strong interpretation of these data is that there are only three core networks that route through occipitotemporal cortex.

However, it is important to note that we probed for distinctive networks using small seeds regions spanning only a few stimulus-evoked divisions. This raises two key limitations. First, these results are based on only five functionally localized ROIs. If this cortex were divided into even more seed regions exhaustively covering the ventral stream, it is possible that more subnetworks might emerge. Our current data set is not well-suited to answer this question, as our targeted approach with a relatively small number of participants relies heavily on having single-subject functional localizers to define the seed regions (cf. Yeo et al., 2011 using a purely anatomical correspondence with 100 s of participants). Second, while we tested the stimulus distinctions between animacy, object size and the categories of faces, bodies, and scenes, these distinctions reflect only some of the extensive high-level visual information found in the multivoxel patterns across this cortex (Haxby et al. 2011, Huth et al. 2012). New methods that move beyond univariate functional connectivity to leverage multivariate patterns (Anzellotti et al. 2016) will be essential to the endeavor of linking stimulus-evoked and resting-state architecture.

Given these limitations, the strong proposal that there are exactly three core networks routing through the ventral stream may be too simple a view. In a more nuanced view, resting-state networks may be as textured at the stimulus-evoked organization. On this view, the entire ventral stream is largely part of a common resting-state network, within which there are major stimulus-evoked organizations with corresponding long-range networks, within which there are finer-grained response differences linked to more subtle differences in resting-state networks. The present results are consistent with this view, and provide empirical evidence that the major object distinctions of animacy and object size found in stimulus-evoked responses also have corresponding distinctions evident in the whole-brain resting-state architecture.

Relationship to Retinotopy

In a series of studies, Levy, Hasson, and Malach found that high-level object cortex was not only responsive to objects but contained subtle but reliable retinotopic biases, which were systematically related to the object-selectivities (Levy et al. 2001; Hasson et al. 2002; Malach et al. 2002). Subsequently, much work has explored the systematic relationships between category-selectivity and retinotopic biases (Levy et al. 2004; Sayres and Grill-Spector 2008; Troiani et al. 2012; Silson et al. 2015), mapping extended retinotopy all along the ventral streams (Larsson and Heeger 2006; Wandell et al. 2007; Arcaro et al. 2009). One likely account of these retinotopic response biases in high-level areas is that they are driven by differential connectivity with different parts of the retinotopic maps in early visual areas.

To explore this possibility, we probed the extent to which these biases and links were evident in the intrinsic resting-state structure between object cortex and early visual cortex. Our analyses yielded relatively subtle relationships between these two major cortical territories. Considering eccentricity, we only detected differences in peripheral functional connectivity, but not with foveal functional connectivity. Interestingly, this asymmetry is consistent with some recent stimulus-evoked results. For example, it has been found that big object-preferring regions have a stronger evoked response when stimuli are presented at larger retinal sizes than at smaller retinal sizes, showing a general peripheral preference (Konkle and Oliva 2012; see also Levy et al. 2004; Troiani et al. 2012), while the small-object-preferring regions do not have the opposite foveal bias (Konkle and Oliva 2012). Less is known about the relative strengths of the asymmetries between upper and lower visual field biases in these regions, but our resting-state results would predict stronger lower visual field biases along the lateral surface regions, generally consistent with reported lower visual field biases in LO (Sayres and Grill-Spector 2008; Schwarzlose et al. 2008; Kravitz et al. 2010; Silson et al. 2015, see also Kravitz et al. 2013).

Common Network and Nested Network Architecture

The strongest structure in these resting-state data shows that the animacy-size regions are largely part of a common interconnected network. What is the role of these local connections in the organization of object-responsive cortex? Some proposals suggest that this internal connectivity may actually be a prominent driver of the large-scale organization of object cortex (Polk and Farah 1995; Konkle and Oliva 2012; Behrmann and Plaut 2013; Srihasam et al., 2012). Specifically, the relative locations and sizes of different object-preferring regions may be driven by cooperative–competitive interactions within an integrated large-scale circuit. These accounts typically suggest that Hebbian self-organizing mechanisms might naturally segregate different kinds of images based primarily on the shape similarity and the extent and timing of experience. However, a key challenge for these proposals is that the relative positions of object-selectivities are remarkably consistent across people. Thus, these experience-driven accounts must also presuppose that the large-scale organization is either anchored by input and output connections, or it emerges based on an innate protomap that patterns this cortex (Srihasam et al. 2014).

Importantly, whether the organization is anchored by external connectivity or by an internal protomap, both of these factors likely only provide an initial blueprint of the ultimate object organization. These mechanisms cannot have evolved the specificity required to deal with the richness of the modern visual world (e.g., letter form selectivity in the visual word-form area, see Dehaene and Cohen 2007 for a clear discussion). Thus, there is little doubt that interconnections within occipitotemporal cortex are involved in sculpting the organization based on experience within the lifetime. The key question then is about the relative specificity of this innate blueprint and the scope of experience-driven plasticity: what is the scale and detail of these innate long-range connections and internal biases, and how much room do they leave for learning to change the large-scale organization?

Origins of Occipitotemporal Organization

Critically, we cannot directly probe the origins of the large-scale organization of object cortex directly, because we are characterizing the adult state. As such, we cannot determine whether the subnetworks reported here are initially in the blueprint of whole-brain network architecture or are pruned to have this tripartite-network structure via experience. On a pure innate account, these content divisions are prespecified by stable long-range connections that have been selected for over evolutionary time to facilitate particularly adaptive network architectures (Mahon and Caramazza 2011). On a pure experience-driven account, the content-distinctions between different kinds of visual objects emerge due to experience in the world, likely driven by differences in visual input statistics and interactions (Hasson et al., 2002; Konkle and Oliva 2012). In this scenario, long-range network architecture must be initially highly exuberant (Innocenti and Price 2005), and then undergo pruning to match the stimulus-evoked organization following local regional response tuning. A developmental approach will be required to understand the specificity of these long-range networks and to assess the degree of pruning triggered by development of object-responsive regions.

Exploration of these questions is beginning to bear fruit. On the one hand, congenitally blind individuals—who have had no visual experience in their lifetime—show similar neural response preferences to sighted individuals in some of these same “visual” object regions, for example, whether hearing the names of artifacts versus animals (Mahon et al. 2009), tools (Peelen et al. 2013), or large objects (He et al. 2013), by manually exploring objects (Pietrini et al. 2004; Amedi et al. 2010), or by hearing the shape of objects or bodies via auditory substitution (Striem-Amit et al. 2012; Striem-Amit and Amedi 2014). Thus, there is clear evidence that the organization of this object-responsive cortex is not solely driven by visual experience. On the other hand, recent evidence has shown that intensive early visual experience with symbols can systematically modify the large-scale organization of high-level visual cortex in monkeys, leading to spatially segregated regions that show symbol-selectivity (Srihasam et al. 2012, 2014). Thus, there is also evidence that “new” clustered functional regions can be formed through mechanisms of visual experience. However, more work is required to understand why the locations of these new regions are consistent across monkeys, and it is possible that these regions are in fact constrained by the available connectivity of those regions to downstream areas necessary for further processing of the stimulus set (Dehaene and Cohen 2011; Mahon and Caramazza 2011).

Ultimately, a region derives its selectivity and functional role through its inputs and outputs, and thus its representational role inherently depends on available connectivity. Different proposals about the organization and functional clustering in the ventral stream have emphasized different aspects of this connectivity—focusing on the role of the primary inputs from the visual cortex, the internal connections within the occipitotemporal cortex, or the downstream connections projecting to nonvisual parts of the brain. The present data cannot arbitrate the relative importance of these different connection motifs in driving the organization of object-cortex. However, these data do support the viability of innate whole-brain network-level connectivity as a potential driving force of object organization, as object-responsive cortex has differential functional connections both with early visual cortex and with longer-range regions, and these connections parallel the stimulus-evoked organization.

Supplementary Material

Supplementary material can be found at: http://www.cercor.oxfordjournals.org/.

Funding

The Fondazione Cassa di Risparmio di Trento e Rovereto (SMC) and the National Institutes of Health, National Eye Institute (Fellowship F32EY022863-01A1 to T.K.), and this work was conducted at the Laboratory for Functional Neuroimaging at the Center for Mind/Brain Sciences (CIMeC), University of Trento.

Notes

Conflict of Interest: None declared.