Representing object categories by connections: Evidence from a mutivariate connectivity pattern classification approach

Abstract

The representation of object categories is a classical question in cognitive neuroscience and compelling evidence has identified specific brain regions showing preferential activation to categories of evolutionary significance. However, the potential contributions to category processing by tuning the connectivity patterns are largely unknown. Adopting a continuous multicategory paradigm, we obtained whole‐brain functional connectivity (FC) patterns of each of four categories (faces, scenes, animals and tools) in healthy human adults and applied multivariate connectivity pattern classification analyses. We found that the whole‐brain FC patterns made high‐accuracy predictions of which category was being viewed. The decoding was successful even after the contributions of regions showing classical category‐selective activations were excluded. We further identified the discriminative network for each category, which span way beyond the classical category‐selective regions. Together, these results reveal novel mechanisms about how categorical information is represented in large‐scale FC patterns, with general implications for the interactive nature of distributed brain areas underlying high‐level cognition. Hum Brain Mapp 37:3685–3697, 2016. © 2016 Wiley Periodicals, Inc.

INTRODUCTION

The human brain is extremely good at categorization. The representation mechanisms for various object categories, such as faces, scenes, animals, and tools, have been intensively studied. In the ventral temporal cortex (VTC), objects of different categories have been shown to yield different responses in single neurons [Tanaka, 1996], regions [Kanwisher, 2010], distributed fMRI voxel patterns [Haxby et al., 2001; Kriegeskorte et al., 2008b] and temporal patterns of electrocorticographic signals [Majima et al., 2014]. The object's properties that are represented beyond VTC are also available and may affect how the object is visually processed [Fang and He, 2005; Mahon and Caramazza, 2011; Mahon et al., 2007]. Widely distributed regions outside VTC have been shown to be active in response to particular categories. For instance, in addition to occipital face area (OFA) and fusiform face area (FFA) in VTC [Kanwisher et al., 1997], face perception also more strongly (versus objects) activates a set of specific temporal and frontal regions, presumably driven by properties such as emotional valence or social needs that are more saliently associated with faces [Haxby et al., 2000; Zhen et al., 2013].

The distributed regions showing similar category selectivity in terms of activation preference have been commonly considered to be the functional network supporting the processing of a given category. Indeed, the patterns of functional connectivity (FC), measured as temporal correlations in the blood oxygenation level‐dependent (BOLD) activity between distinct brain regions [Friston et al., 1993; Smith, 2012], have been studied among various category‐selective regions during task and resting states, and it was commonly observed that these regions are intrinsically interconnected [He et al., 2013; Hutchison et al., 2014; Peelen et al., 2013; Stevens et al., 2015; Zhen et al., 2013] and retune their connection patterns according to task demands [Mechelli et al., 2004; Norman‐Haignere et al., 2012; Vitali et al., 2005]. However, the potential contributions to category processing made by regions through means other than stronger activation levels are largely unknown.

One such mechanism may involve adjusting the inter‐regional communication patterns according to a category of interest, which is not necessarily reflected by changes in the regional activity strengths. This notion has been recently supported by studies emphasizing widespread FC changes across various tasks in regions not showing elevated activation magnitudes during task performance [Gonzalez‐Castillo et al., 2015; Tomasi et al., 2014]. Given the tight relationships between activation and FC strengths during the task state, it is important to examine the effects of task FCs over and above those of activations and several approaches have been proposed to separate task FCs from activation magnitudes [Gonzalez‐Castillo et al., 2015; Norman‐Haignere et al., 2012; O'Reilly, et al., 2012]. In this study, we adopted the approach that was recently widely used–the continuous multitask paradigm [Gonzalez‐Castillo et al., 2015; Liang et al., 2013; Shirer et al., 2012], in which participants are continuously scanned for the same task block that is much longer than those in the traditional fMRI block design to allow for adequate estimation of within‐block correlations over time courses for the given task. The inter‐regional time series correlation computed this way has been argued to reflect functional connections that are not accounted for by first‐order activations [Gonzalez‐Castillo et al., 2015].

In the present study, we aim to explore the potential mechanisms of object category representation through whole‐brain FC patterns and the relationship between such connectivity‐based representation mechanism and classical activation‐based categorical regions. With participants continuously scanned for long blocks of viewing various object categories, we tested whether the categories being viewed can be decoded on the basis of whole‐brain FC patterns using multivariate pattern analyses and machine learning algorithms (fcMVPA). This paradigm is an extension (using FC as variates) of the classical multivariate pattern analysis [Haxby et al., 2014], where local activation values of multiple voxels in a given experimental condition constitute a brain pattern, with which techniques such as classification algorithms, or similarity analysis [Kriegeskorte et al., 2008a] are applied to reveal whether and how the cognitive information embedded in the experiment is represented in that pattern of activity. With successful decoding, we then depicted the characteristics of the category‐discriminative FC patterns in the context of resting‐state functional networks and more importantly, in the context of the category‐selective activation patterns. Our results identified widely distributed functional connections supporting category sensitivity beyond the classical categorical regions, revealing potential novel mechanisms for object recognition.

MATERIALS AND METHODS

Participants

Twenty healthy volunteers (10 males) aged 19–26 years (mean age = 22 years) participated in this study. All were right‐handed, native Chinese speakers, had normal or corrected‐to‐normal vision, reported no history of psychiatric or neurological diseases and gave written informed consent. The experiment was approved by the Institutional Review Board of the National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University.

Continuous Multicategory Paradigm

Task‐related imaging data were acquired when participants were continuously scanned over four block‐design runs (Fig. 1A). Each run began with 10 s of rest, contained four 80‐s stimulus blocks (one for each category), separated by 6‐s intervals of rest, and ended with 8 s of rest. Each block presented 40 pictures of different exemplars from a single category, and each picture was presented for 500 ms, with an interstimulus interval of 1,500 ms. The participants were instructed to watch the pictures carefully and to press a button when they detected that a picture appeared twice in a row. The presentation order of pictures within each block was randomized, with the same stimuli order across subjects. There were one to two catch trials per block, and the number of button presses was matched across categories (six times per category in total). The block order was counterbalanced across runs.

Figure 1.

The flowchart of the task and analysis procedures. (A) Participants were presented in various blocks (80 s in length) with pictures of four categories – faces (F), scenes (S), animals (A), and tools (T). (B) The 180 cortical and subcortical regions obtained from Craddock et al. [2012] were used as ROIs. Only ROIs in the left hemisphere are shown here. (C) Time courses were extracted from each ROI in each block. (D) ROI‐pairwise correlation matrices were computed from each block and averaged across four runs to obtain the FC patterns for each category, with brain regions ordered according to resting‐state functional networks. The FC values across four categories were averaged to obtain the task‐general FC pattern, which was then subtracted from the FC values of each category. (E) Category‐pairwise and linear ensemble classifiers were trained in all but one participant using linear SVM and tested in the remaining participant to assess decoding accuracy. The weights of each FC in the category‐pairwise classifiers were submitted to permutation tests and conjunction analyses, and the reliable discriminative FCs for one category were used to construct category‐discriminative networks. The results are mapped on cortical surfaces using the BrainNet Viewer [Xia et al., 2013].

The stimuli were 640 gray‐scale images (300 × 300 pixels, visual angle 7.92° × 7.92°) of four categories (160 images per category): neutral faces, scenes, animals, and tools. The face stimuli, selected from the Chinese affective picture system [Bai et al., 2005], consisted of 80 unique male and female neutral faces. The scene stimuli were all outdoor scenes (http://cvcl.mit.edu/database.htm), including mountains, countryside scenes, streets, and buildings, with 40 unique pictures of each type. The animals included mammals, birds, insects, and reptiles and were composed of 40 items, each with four exemplars. The tools included kitchen utensils, farm implements and other common indoor tools and also had 40 items, each with four exemplars. All images were matched for mean luminance and contrast using the SHINE toolbox [Willenbockel et al., 2010].

MRI Data Acquisition

All data were acquired using a Siemens TRIO 3‐Tesla scanner at the Imaging Center for Brain Research, Beijing Normal University. Resting‐state functional images were collected at the beginning, using an echo‐planar imaging (EPI) sequence with 33 interleaved axial slices (repetition time [TR] = 2,000 ms, echo time [TE] = 30 ms, flip angle [FA] = 90°; field of view [FOV] = 200 mm, voxel size = 3.125 × 3.125 × 4.2 mm³], during which the participants were instructed to close their eyes, and to remain still and awake. Four sessions of task‐state functional images were then collected using an EPI sequence (33 axial slices, TR = 2,000 ms, TE = 30 ms, FA = 90°, FOV = 192 mm, voxel size = 3 × 3 × 4.2 mm³). A three‐dimensional, T₁‐weighted magnetization‐prepared rapid gradient‐echo sequence was acquired after two task sessions (144 sagittal slices, TR = 2,530 ms, TE = 3.39 ms, FA = 7°, FOV = 200 mm, voxel size = 1.0 × 1.0 × 1.33 mm³).

fMRI Data Preprocessing

The preprocessing of functional images was performed using SPM8 (Wellcome Department of Cognitive Neurology, London, UK) and the DPARSF toolbox [Yan and Zang, 2010]. The preprocessing of both the resting‐state and task‐state BOLD time series included the deletion of the first five volumes for signal equilibrium, motion correction, normalization to the MNI space with voxel sizes resampled into 3 × 3 × 3 mm³, spatial smoothing with a 6 mm FWHM Gaussian kernel, linear trend removal and temporal band‐pass filtering (0.01–0.1 Hz). Several nuisance variables, including six motion parameters, the mean global signal and the averaged signals from white matter and ventricles, were then removed using multiple linear regression analysis. The residual time series were used for FC computation. Note that two subjects had one run with excessive head motion (> 3 mm or 3°) and thus these two runs were excluded from further analyses.

In a control analysis to remove the effects of univariate task activations in the FC‐based classification, four additional regressors corresponding to each category were included as the nuisance variables and these regressors were generated by convolving the hemodynamic response with boxcar functions with ones during the category blocks and zeros elsewhere. Finally, considering that the task signals may be present at higher frequencies than the slow time series fluctuations that the traditional resting‐state FC studies typically focus on, we also preprocessed the imaging data only with a high‐pass filter (f > 0.008 Hz) to evaluate the effects of frequency bands on the decoding results.

Estimation of FC Patterns

A collection of 180 cortical and subcortical regions of interest (ROIs), taken from the 200‐ROI parcellations in Craddock et al. [2012], was used as network nodes. This ROI set was adopted here because the ROIs were parcellated according to the homogeneity of the FC patterns of voxels while maintaining the spatial coherence of ROIs. We manually excluded 20 ROIs in the cerebellum and brain stem from the original 200‐ROI template and retained 180 cortical and subcortical regions.

Estimation of FC patterns were carried out on the preprocessed time series using the GRETNA toolbox [Wang et al., 2015]. The preprocessed resting‐state time series were directly submitted to GRETNA for FC computation. The task‐state time series of each task session were first segmented into separate blocks (faces, scenes, animals, and tools) using the following procedure. For each block with 40 volumes (80 s), the first four volumes (8 s) were discarded, and two volumes (4 s) of the subsequent block were included to account for the hemodynamic delay. Within each block, we first computed the mean time series of each of the 180 ROIs by averaging the time series of all the voxels in it and then calculated the ROI‐pairwise Pearson correlations of the time series ([180 × 179]/2 = 16,110 connections in total), which were finally Fisher z transformed and averaged across the four blocks of the same category to obtain the FC pattern of the category in each participant. We also computed the task‐general FC pattern by averaging the FC patterns across the four categories. Thus, there were six FC matrices for each participant, including one for the resting state, one for the task‐general state, and one per category.

Across‐Subject Classification of FC Patterns

A linear support vector machine (SVM; LIBSVM: http://www.csie.ntu.edu.tw/~cjlin/libsvm) with standard parameters was used in a leave‐one‐subject‐out cross‐validation scheme (LOOCV) for the FC pattern classification analysis. Note that we additionally carried out five‐fold, four‐fold, and two‐fold cross‐validations to re‐estimate the classification performance for validation purposes, and the performances were largely similar (see Supporting Information Table 1). With these various cross‐validation schemes, we performed two types of decoding: decoding between pairs of categories and decoding among the four categories (multicategory classification).

Because the interpretation of negative FCs remains controversial [Fox et al., 2009; Weissenbacher et al., 2009], we focused on positive FCs in this study. For each category, we performed a one‐sample t‐test across participants for each of the 16,110 connections and retained FCs that had values significantly greater than zero and multiple comparisons were corrected using false discovery rate (FDR) q = 0.01. This procedure identified 3,241 positive FCs for faces, 3,363 positive FCs for scenes, 3,446 positive FCs for animals, and 3,562 positive FCs for tools. Pooling these FCs together resulted in a total of 4,296 FCs that were significantly positive for at least one category state. The FC values of these connections in the task‐general FC pattern were then subtracted from the FC values of each category, and the resulting FCs were thought to reflect category‐selective information and were taken as features for pattern classification.

In the category‐pairwise decoding, there were two examples of each participant (one from each category, 20 participants × 2 examples = 40 examples in total), with each example containing 4,296 dimensions of features. In each iteration of the cross‐validation, each dimension of feature vectors of both training (38 examples of 19 participants) and test (2 examples of the remaining one participant) examples was z‐normalized by subtracting the mean and dividing by the standard deviation of the training data. The classifier was then trained on the training set and tested on the two examples of the remaining one participant by classifying each example into one of two categories.

To directly predict which category was presented, we constructed linear decoders for each category based on the category‐pairwise classifiers as previously described [Kamitani and Tong, 2005]. In each iteration of LOOCV, we first obtained six SVM classifiers for pairs of categories in all but one participant. These pairwise classifiers were then added to yield the linear ensemble decoder for each category, thus resulting in four linear ensemble decoders, with one per category. For each of the four test samples of the remaining participant, its feature vector was first normalized using the mean and standard deviation of the training data and then inputted to the four linear ensemble decoders. The decoder with the maximum value was chosen as the predicted category.

The statistical significance of decoding accuracy was assessed using permutation tests, in which category labels from each participant in the training set were randomly exchanged with each other, with a probability of 0.5 for category‐pairwise decoding and 0.25 for multicategory decoding, over 10,000 permutations. The P value for decoding accuracy was calculated as the fraction of accuracies from all permutations that were equal to or greater than the actual accuracy using correct labels.

Constructing Category‐Preferring Networks From Discriminative Functional Connections

In this section, we used the category‐pairwise classification weights for each FC to reflect its contribution to the classification [Cui et al., 2016; Ecker et al., 2010; Ekman et al., 2012; Mourao‐Miranda et al., 2005] and adopted conjunction analyses to identify those FCs showing reliable discriminative power to one category. From the 10,000 permutation tests as mentioned above, we first obtained the probability distribution of each FC weight. The weight vectors from the real dataset and permutations were normalized to length 1 and the P value for each FC weight was calculated as the fraction of weights from all permutations that were equal to or greater (for positive weights) or smaller (for negative weights) than actual weights in using correct labels. The network‐based statistic (NBS) approach was then adopted to localize connected components that show significant changes between two categories [Zalesky et al., 2010]. Briefly, after applying a primary FC‐wise threshold (P < 0.01), we defined a set of suprathreshold FCs among which the sizes of the connected components (number of links) were estimated based on the null distribution of the connected component size derived from the permutation tests. The FCs surviving the cluster‐defining threshold of P < 0.001 were considered to contribute significantly to the category‐pairwise classification. We then defined an FC as category‐preferring if it exhibited reliably significant discriminative power when classifying a particular category with each of other three categories. These category‐preferring discriminative FCs formed a network in which the edges showed either stronger or weaker connections when processing the target category compared with other categories. We also identified hub regions (defined as the region having the densest connections) in each network.

Distribution of Discriminative FCs in Resting‐State Functional Modules

Discriminative FCs can be characterized by illustrating their distribution among intrinsic brain networks [Dosenbach et al., 2010; Zeng et al., 2012]. The resting‐state functional modules were computed from a group‐weighted brain network by averaging the FC patterns at rest across all subjects. In this network, negative correlations were set to zero, and the top 20% of connections were extracted and submitted to a spectral optimization algorithm for module detection [Newman, 2006]. We found that the 180 ROIs were partitioned into four major functional modules, which corresponded to the visual (VIS), frontoparietal (FP), sensorimotor and subcortical (SM), and default mode (DM) networks. This configuration is largely consistent with those in the literature [Liang et al., 2013]. To assess the contribution of each resting‐state module in fcMVPA‐based decoding, we calculated the total number of discriminative FCs connected to each module, which was defined as the sum of the number of within‐module connections and 1/2 of the number of between‐module connections.

Participation of Activation‐Defined Category‐Preferring Regions in the Category‐Discriminative Networks

We investigated how the regions showing categorical preference in terms of activation were involved in the fcMVPA‐based classification. We first identified voxels showing categorical preference using a standard general linear model analysis for task activation. For the first‐level analysis, the spatially smoothed task fMRI images were modeled with four regressors, one for each category, convolved with the canonical SPM hemodynamic response function. The high‐pass filter was set at 128 s. Regions showing selective activations to faces, animals, tools, and scenes were defined on the group level (P < 0.0001, cluster size ≥ 10 voxels) using the contrast face > scene, animal > scene, tool > scene, and scene > (face + animal + tool), respectively. The obtained regions were overlapped with 180 ROIs, and the ROIs containing at least 10 category‐preferring voxels were considered to be category‐preferring and other ROIs category‐neutral. We then counted the number of category‐preferring regions and the number of FCs they connected in the category‐discriminative networks.

To test whether the high multicategory decoding accuracies could be simply due to these activation‐defined category‐preferring regions, we conducted multicategory decoding using randomly permuted FCs connected with category‐preferring regions and original FCs connected with category‐neutral regions as features. Specifically, the category labels of the FCs connected with category‐preferring regions in the training set were randomly shuffled over 30 permutations. This resulted in a distribution of accuracies, the mean of which was used to reflect how much category information is coded in category‐neutral regions. We then used one‐tailed one‐sample t‐test to examine whether the mean of the permuted accuracies was significantly different from the original decoding accuracy obtained from real data and the chance level of multicategory decoding (i.e., 25%).

RESULTS

We collected the BOLD time series during a picture viewing task when participants were continuously scanned for long blocks (80 s per block) of four evolutionarily salient categories (faces, scenes, animals and tools). The FC patterns for each category were obtained by computing the Pearson correlation coefficients of the mean time series between 180 cortical and subcortical brain regions [Craddock et al., 2012]. With the FCs patterns as features, the linear SVM classifier with standard parameters and a LOOCV were adopted to decode between pairs of categories and among the four categories. Based on the classification weights, we then constructed category‐preferring networks consisting of FCs showing reliable discriminative power for each category and examined each network's hub region(s) and their distribution among resting‐state functional modules. We finally evaluated the participation of activation‐defined category‐preferring regions in these networks. The flowchart of the task and analysis procedures is shown in Figure 1.

fcMVPA Successfully Differentiated Categories From Each Other

We subtracted the task‐general FC patterns from the FC patterns of each category and used the residual FC patterns in the classification of the object categories. In the main results below, we used the 4,296 FCs with significantly positive values in at least one category state at q(FDR) < 0.01 because the interpretation of negative FCs remains controversial [Fox et al., 2009; Weissenbacher et al., 2009]. As shown in Table 1 (first column), in the pairwise classification, all object pairs could be differentiated with high accuracy (Ps < 0.0001, 10,000 permutations). In decoding one out of the four categories (i.e., to predict which of the four candidate categories was being viewed), we constructed decoders for each category by combining pairwise classifiers and found that FC patterns could correctly identify the category being viewed with 85% accuracy, well above the 25% chance level (P < 0.0001, 10,000 permutations).

Table 1.

Accuracies of decoding visual categories using whole‐brain FC patterns

	0.01–0.1 Hz			0.008‐Inf
	Positive FCs (N = 4,296)	Positive FCs regressing out task effects (N = 4,284)	All FCs (N = 36,110)	Positive FCs (N = 4,194)	All FCs (N = 36,110)
Category pairwise classification (Chance level: 50%)
Face‐Scene	95%	95%	100%	90%	93%
Face‐Animal	85%	83%	85%	88%	88%
Face‐Tool	93%	90%	98%	93%	95%
Scene‐Animal	100%	90%	95%	100%	88%
Scene‐Tool	95%	93%	90%	90%	90%
Animal‐Tool	93%	95%	93%	98%	93%
Multicategory classification (Chance level: 25%)
	85%	81%	83%	86%	81%

We validated the reproducibility of decoding results across various preprocessing protocols, including extension to all FCs and frequency bands. The former issue has remained an unresolved issue for the FC computation, and the latter issue is particularly associated with task data, given the potential usefulness of high‐frequency signals during the cognitive tasks. We found, however, that the high classification accuracies using large‐scale FC patterns for category decoding were robust regardless of the preprocessing steps (Table 1). To rule out the possibility that the classifications results were directly driven by changes in regional activation levels across various categories, we regressed out the task effects as a nuisance variable and found that decoding accuracies were not influenced, indicating that univariate task activations were not necessary for fcMVPA‐based classification. Finally, using the fivefold, four‐fold, and two‐fold cross‐validation schemes, the classification accuracies were generally similar to those of the LOOCV and were all significantly higher than chance (although with overall slightly lower accuracies for twofold cross‐validation), indicating the robustness of our results (Supporting Information Table 1).

Identification and Characterization of Category‐Discriminative Networks

In this section, based on the classifier weights, we identified discriminative FCs in each category, defined as FCs that contributed significantly to the reliable discrimination between a particular category and the other three categories via conjunction analysis (FC‐wise P < 0.01 for each classification, NBS‐based cluster‐defining threshold, P < 0.001). Specifically, FCs that most strongly contributed to the identification of the target category from the other three categories (e.g., this is a face condition and not scene, tool, or animal) were considered discriminative FCs. These discriminative FCs, with either positive or negative weights, formed such a network in which the connection strengths of edges were either strengthened or weakened when processing the target category compared with other categories. Note that the positivity and negativity of weights corresponded to strengthening and weakening in raw FC values for a given category in our results, respectively. This analysis resulted in four category‐discriminative networks (Figure 2 and Supporting Information Table 2), all of which were widely distributed in both hemispheres.

Figure 2.

FcMVPA‐identified category‐discriminative networks. Orange and blue lines indicate FCs that are either strengthened or weakened when processing a given category compared to each of other three categories. The brain regions are scaled by the number of their connections.

Considering the most densely connected region(s) (i.e., hub regions) in each network, the left parahippocampal gyrus (lPHG, region center x,y,z: −29, −36, −12) was found to be the hub region for both faces and scenes, with different connection patterns. For faces, this region had weakening connections with bilateral occipitotemporal cortex and strengthening connections with right temporal pole and right planum temporale; for scenes, this region had weakening FCs with precuneous cortex and right temporal pole and strengthening FCs with various regions including bilateral dorsolateral occipital cortex, left lingual gyrus, right temporal occipital fusiform gyrus, right precentral gyrus, and left middle frontal gyrus. For animals, the hub region was left occipital pole, which had weakening connections with bilateral dorsolateral occipital cortex. Compared to other categories, connections in the tool network distributed more evenly, with left supramarginal gyrus, left inferior temporal gyrus and left lingual gyrus, each having two connections, serving as the hub regions.

The functional properties of discriminative FCs could be depicted referencing to the intrinsic functional modules during the resting state. Modularity analyses using a spectral optimization algorithm of the resting‐state FC patterns identified four intrinsic modules (Figure 3A) that are highly consistent with the literature [Liang et al., 2013]: the VIS, FP, SM, and DM networks. We quantified the distributions of discriminative FCs among these modules by counting the number of discriminative FCs according to the network membership of the regions connected by a given FC. As shown in Figure 3B, for the four categories, the VIS module had the largest proportion of discriminative FCs among the four resting‐state modules (faces: 58%; scenes: 61%; animals: 50%; tools: 44%).

Figure 3.

(A) Four resting‐state functional networks identified by module analysis, including VIS, FP, SM, and DM networks. (B) The distribution of category‐discriminative networks among resting‐state networks. Bar length represents the percentage of the total number of discriminative FCs connected to each network.

Characterization of category‐discriminative networks in relation to the activation‐based category selectivity

Category selectivity has been classically defined on the basis of focal activation strength differences. To examine the relationship between the fcMVPA‐defined and the activation‐defined category effects, we (1) evaluated to what extent the activation‐defined category‐preferring regions were included in category‐discriminative networks, and (2) conducted multicategory classification using FCs that were not connected with the activation‐defined category‐preferring regions, that is, excluding the effects of FCs of the activation‐defined category regions.

Identification of activation‐defined category‐preferring regions

We labeled a region in the template we used [Craddock et al., 2012] as category‐preferring if it contained clusters (minimum size: 10 voxels) showing preference for a given category (see Materials and Methods for the specific contrasts), as shown in Figure 4A, which nicely reproduced the well‐known distribution of category‐preferring regions in the literature. Some regions showed preference to more than one category, including five preferring both animals and tools (left inferior temporal gyrus, left occipital fusiform gyrus, and three regions in left lateral occipital cortex) and three preferring faces, animals, and tools (right lateral occipital cortex, right inferior, and posterior middle temporal gyrus). In total, activation analyses identified 27 category‐preferring regions, and the remaining 153 regions were labeled as category‐neutral.

Figure 4.

Identification of activation‐defined category‐preferring regions in the brain and their participation in category‐discriminative networks. (A) Category‐discriminative networks (gray dots and lines) are overlaid with activation maps (warm colors, uncorrected voxelwise P < 0.0001, cluster size ≥ 10 voxels, using the nearest voxel algorithm in the BrainNet Viewer [Xia et al., 2013]). Purple dots are ROIs we defined as category‐preferring in terms of activation magnitudes. (B) The percentage of activation‐defined category‐preferring regions in discriminative networks. (C) The distribution of discriminative FCs among activation‐defined category‐preferring and category‐neutral regions.

Inclusion of activation‐defined category‐preferring regions in the category‐discriminative networks

We observed that the discriminative networks included regions far beyond the regions with category selective activation and that the majority of nodes in the discriminative networks were category‐neutral according to activation analyses (faces: 89%; scenes: 71%; animals: 100%; tools: 69%; Figure 4B), suggesting the substantial contribution of the FCs by regions without significant category‐selective activation during the classification. We then counted the number of the discriminative FCs connecting with the activation‐defined category‐preferring and the category–neutral regions (Figure 4C). Only 11.1% of the scene connections and 37.5% of the tool connections were found within the category‐preferring regions, with other discriminative FCs being connected with at least one category‐neutral region. The percentage of FCs exclusively connected within category‐neutral regions were 69.2%, 27.8%, 100%, and 62.5% for faces, scenes, animals and tools, respectively. These results highlight the contribution of regions that did not show category sensitivity in terms of activation in category discrimination through FC changes.

Multicategory classification using FCs excluding the activation‐defined category‐preferring regions

To directly test the extent to which category differences were embedded in the patterns of these FCs, we performed multicategory decoding 30 times by randomly permuting FCs connected with the 27 regions showing category‐selective activations while maintaining the FCs connected within the category‐neutral regions. The permutated classifiers produced an average accuracy of 67.83%, which was lower than the original accuracy of 85% [t(29) = −39.00, P < 0.001], but significantly higher than the chance level of 25% [t(29) = 97.32, P < 0.001]. This result indicates that there is a significant amount of categorical information that is represented in the FC patterns among the activation‐defined category‐neutral regions. In fact, even when we excluded all the FCs connecting with the 27 regions with category‐selective activation and used the remaining 3,188 positive FCs within 153 category‐neutral regions as features of the multicategory classification, the category classifiers still produced an accuracy of 66%, despite the reduction of feature numbers.

Overlapping Regions Across Category‐Discriminative Networks

A set of regions have been observed in multiple category‐discriminative networks. Specifically, these regions, acting as “chameleons,” change their connection patterns based on various categorical conditions, and these changes significantly contribute to categorical discrimination. lPHG, a most classical scene‐selective region based on its activation sensitivity [Epstein and Kanwisher, 1998], is a good example. This region was identified to be a hub region by fcMVPA in not only the scene but also the face networks. That is, its FC patterns also contributed significantly to the classification of a target being a face or a non‐face. Among the 55 regions that the four category‐discriminative networks covered, 18 were such chameleon regions (Table 2), including two (bilateral inferior temporal gyrus) found in three networks (faces, scenes, tools) and 16 (including bilateral lateral occipital cortex, left lingual gyrus, left supramarginal gyrus) in two networks. We further examined the category preference of these regions in terms of activation and found that fcMVPA‐defined category discrimination in these regions is independent of their categorical differences in activation. First, seven chameleon regions did not show category preference in the activation analyses, including bilateral middle frontal gyrus, right temporal pole, left lingual gyrus, bilateral lateral occipital cortex. Seven other chameleon regions were found to show preference for only one category in terms of activation, including lPHG, bilateral lateral occipital cortex, left supramarginal gyrus. Finally, four chameleon regions were found to be involved in more than one category in both the activation and fcMVPA analyses, including left lateral occipital cortex, left occipital fusiform gyrus, and bilateral inferior temporal gyri, but the preferring categories substantially differed in these two analyses.

Table 2.

Overlapping regions across category‐discriminative networks

x	y	z	Label	fcMVPA‐defined preference	Activation‐defined preference
47	−51	−18	R Inferior Temporal Gyrus	Face, Scene, Tool	Face, Animal, Tool
−44	−51	−20	L Inferior Temporal Gyrus	Face, Scene, Tool	Animal, Tool
−37	−76	−16	L Occipital Fusiform Gyrus	Face, Scene	Animal, Tool
−50	−64	−6	L Lateral Occipital Cortex	Scene, Tool	Animal, Tool
−57	−27	36	L Supramarginal Gyrus	Face, Tool	Tool
27	−48	−14	R Temporal Occipital Fusiform Cortex	Scene, Animal	Scene
32	−76	30	R Lateral Occipital Cortex	Face, Scene	Scene
−27	−91	11	L Occipital Pole	Scene, Animal	Scene
−29	−36	−12	L Parahippocampal Gyrus	Face, Scene	Scene
35	−85	9	R Lateral Occipital Cortex	Scene, Animal	Scene
−35	−80	25	L Lateral Occipital Cortex	Face, Scene	Scene
−15	−73	−11	L Lingual Gyrus	Scene, Tool	None
−15	−64	55	L Lateral Occipital Cortex	Scene, Animal	None
29	−66	46	R Lateral Occipital Cortex	Scene, Animal	None
−27	−72	40	L Lateral Occipital Cortex	Scene, Animal	None
30	8	−20	R Temporal Pole	Scene, Animal	None
28	1	56	R Middle Frontal Gyrus	Scene, Tool	None
−46	10	31	L Middle Frontal Gyrus	Face, Scene	None

Notes: The coordinates of each node are the MNI coordinates of the center voxel within each brain region. The label for each node is given according to the Harvard‐Oxford cortical and subcortical structural atlas.

DISCUSSION

Using multivariate pattern classification analyses with FC strengths as variates, we showed that the visual categories (faces, scenes, animals, and tools) being viewed can be accurately decoded from the whole‐brain FC patterns. The decoding was successful even after the effects of regions showing category‐selective activations were excluded. The discriminative networks for each category were then constructed, composed of FCs that significantly contributed to all the classifications between a given category and other three categories. These networks included some category‐preferring regions defined by the traditional activation analyses, and importantly, the regions that were not conventionally defined as category‐specific based on activation analysis greatly contributed to fcMVPA‐based category discrimination. Regions participating in more than one category‐discriminative network were also identified. Together, these results demonstrate how large‐scale FC patterns reconfigure in processing various visual categories and extend the potential basis of categorical representation beyond the classical categorical‐selective regions, such as FFA for faces and PHG for scenes.

Our results that various visual categories can be successfully decoded from the whole‐brain FC patterns add to the recently growing body of evidence for the significant amount of information that whole‐brain FC patterns contain: fcMVPA‐based decoding can be applied to differentiate among various subject populations [Dosenbach et al., 2010; Zeng et al., 2012] and task or mental states [Cole et al., 2013; Gonzalez‐Castillo et al., 2015; Pantazatos et al., 2012; Shirer et al., 2012]. Our results further demonstrate its potential in differentially representing different object categories.

The literature on object processing is dominated by identifying regions showing category‐selective activations and revealing the mechanisms underlying regional selectivity [Kanwisher, 2010]. Although a few recent studies have begun to investigate the FC patterns associated with object categories during the resting state [He et al., 2013; Hutchison et al., 2014; Peelen et al., 2013; Stevens et al., 2015] or the task state [Mechelli et al., 2004; Norman‐Haignere et al., 2012; Vitali et al., 2005], they all seeded from or between activation‐defined category‐preferring regions. The potential effects of regions that are not classically considered to be category‐selective using the activation measure thus go undetected. Our results showed that the activity of many of these activation‐defined category‐neutral regions in fact contributed significantly to category decoding in that the FC patterns solely within these regions, that is, excluding classical activation‐based category‐selective regions, successfully differentiated categories. These results are consistent with the dissociation between mean response amplitude and connectivity recently presented in other contexts [Ekman et al., 2012; Gonzalez‐Castillo et al., 2015; Tomasi et al., 2014] and highlight the novel mechanisms of categorical information uncovered by fcMVPA.

Our approach, combining the whole‐brain large‐scale FC patterns and machine learning algorithms, allows us to unbiasedly uncover the contribution of functional connections and regions to categorical discrimination. Based on the classification weights, we constructed the discriminative network for each category, which could be considered as the skeleton connections supporting the processing of each category. That is, they may play a leading role in reorganizing the whole‐brain FC patterns for category differentiation. Closer examination of the discriminative FCs lends partial support to the notion that processing a given category involves coordinated activity between brain regions that process various aspects of that category (e.g., perceptual, motor, affective). For instance, the strengthening FC between right inferior temporal gyrus and left amygdala for faces may be related to emotional processing of faces [Fairhall and Ishai, 2007]. Regions implicated in scene perception and navigation, such as bilateral parahippocampal regions and dorsolateral occipital cortices, showed increased connections in processing scenes [Epstein, 2008]. For tools, the strengthening FCs between left supramarginal gyrus and ventral visual regions may be associated with the motor‐relevant information of tools [Lewis, 2006; Mahon et al., 2007].

Within each category‐discriminative network, we observed that compared to other categories, processing a given category is associated not only with boosting the intercommunication of particular brain regions (strengthening FCs) but also with suppressing the connections between some regions (weakening FCs). The interpretation of the weakening FCs is less straightforward. The decoupling between brain regions in visual category processing has been reported previously [Norman‐Haignere et al., 2012], showing that the connection between FFA and PPA was significantly decreased in the face condition compared to the scene condition. Given that Norman‐Haignere et al. included only two conditions (face and scenes) and examined only this connection, it was not clear whether the PPA‐FFA decoupling was associated with any types of non‐scene stimuli or specifically with faces. Our results, which considered the whole‐brain large‐scale FC patterns and four categories, showed the following: (1) the weakening FCs during the face condition connected PPA with not only FFA but also with OFA and their counterparts in the left hemisphere; (2) the weakening of these FCs is specifically related to face processing but not to scenes or other objects (animals or tools). That is, PPA indeed treats face processing in a special manner by decoupling with a range of face‐preferring regions. We speculate that the weakening connections among brain regions for a given category may be related to a partial breakdown of synchronizations that are irrelevant to the category being processed to promote better information flow among the regions involved [Cole et al., 2014]. This hypothesis naturally explains the observation of overlapping regions across category‐discriminative networks, such that they participated in the decoding of multiple categories by connecting or decoupling with various regions for specific categories. Note that it is also possible that some of chameleon regions may be involved in general cognitive functions, such as occipital regions in low‐level visual processing or frontal regions in executive control and thus selectively connect with other regions to accommodate to the special needs of particular categories.

Finally, there is an important general methodological issue in FC analyses to consider. The task FC patterns computed using correlations of fMRI time series data are a product of both intrinsic fluctuations and task‐evoked functional coupling. Although it is not straightforward how intrinsic fluctuations would contribute to classification, given the random ordering of blocks belonging to different categories in our study, it is nonetheless interesting to see whether explicitly taking out the intrinsic FCs would reduce noise, yield a set of task‐specific FCs that are more readily interpretable, and hence enhance the category decoding accuracy. A compelling approach was recently offered to correlate the same‐category blocks from different runs [Henriksson et al., 2015]. The FC profiles computed this way would potentially reflect the commonly task‐evoked processing of the given category. Intriguingly, however, using the cross‐block whole‐brain FC patterns to classify categories achieved an accuracy of 55%, which, although still significantly greater than the chance level (25%), was considerably lower than the accuracy using our original approach (85%). Subtracting the task‐general FC pattern (mean across categories) from each category did not improve the accuracy (55%), nor did including only the positive FCs (61.25%, using positive FCs for each category at q(FDR) < 0.05 which were mainly distributed in the posterior visual system; see Supporting Information for detailed methods and results of this new between‐block FC analysis). Future studies are desired to investigate the potentially different underlying mechanisms reflected by these different FC measures.

To conclude, using the fcMVPA‐based classification analyses, we showed that the large‐scale FC patterns contain rich information to accurately decode the category of objects being viewed. Not only do many regions that were not considered to be category‐selective on the basis of activation analysis in fact retune their functional connections in manners that can significantly differentiate categories, but that regions such as lPHG that were classically considered to be selective to a certain category were found to contribute to the decoding of multiple categories through reorganizing their connectivity patterns. Our findings extend the potential mechanisms of category representation, identify previously unnoticed regions whose functions are yet to be investigated, and highlight the importance of investigating FC patterns to achieve the full understanding of various cognitive processes.

Supporting information

Supporting Information