Representational Similarity of Body Parts in Human Occipitotemporal Cortex

Abstract

Regions in human lateral and ventral occipitotemporal cortices (OTC) respond selectively to pictures of the human body and its parts. What are the organizational principles underlying body part responses in these regions? Here we used representational similarity analysis (RSA) of fMRI data to test multiple possible organizational principles: shape similarity, physical proximity, cortical homunculus proximity, and semantic similarity. Participants viewed pictures of whole persons, chairs, and eight body parts (hands, arms, legs, feet, chests, waists, upper faces, and lower faces). The similarity of multivoxel activity patterns for all body part pairs was established in whole person-selective OTC regions. The resulting neural similarity matrices were then compared with similarity matrices capturing the hypothesized organizational principles. Results showed that the semantic similarity model best captured the neural similarity of body parts in lateral and ventral OTC, which followed an organization in three clusters: (1) body parts used as action effectors (hands, feet, arms, and legs), (2) noneffector body parts (chests and waists), and (3) face parts (upper and lower faces). Whole-brain RSA revealed, in addition to OTC, regions in parietal and frontal cortex in which neural similarity was related to semantic similarity. In contrast, neural similarity in occipital cortex was best predicted by shape similarity models. We suggest that the semantic organization of body parts in high-level visual cortex relates to the different functions associated with the three body part clusters, reflecting the unique processing and connectivity demands associated with the different types of information (e.g., action, social) different body parts (e.g., limbs, faces) convey.

SIGNIFICANCE STATEMENT While the organization of body part representations in motor and somatosensory cortices has been well characterized, the principles underlying body part representations in visual cortex have not yet been explored. In the present fMRI study we used multivoxel pattern analysis and representational similarity analysis to characterize the organization of body maps in human occipitotemporal cortex (OTC). Results indicate that visual and shape dimensions do not fully account for the organization of body part representations in OTC. Instead, the representational structure of body maps in OTC appears strongly related to functional-semantic properties of body parts. We suggest that this organization reflects the unique processing and connectivity demands associated with the different types of information different body parts convey.

Introduction

The human lateral and ventral occipitotemporal cortices (LOTC, VOTC) contain regions that respond preferentially to pictures of human bodies and faces. While early fMRI studies only made a distinction between body and face representations (Kanwisher et al., 1997; Peelen and Downing, 2007), recent studies additionally revealed dissociable responses to individual body parts such as hands and torsos (Bracci et al., 2010; Op de Beeck et al., 2010; Orlov et al., 2010). This raises the question of how representations of individual body parts are organized. Here we used representational similarity analysis (RSA; Kriegeskorte et al., 2008) of fMRI data to test multiple possible organizational principles: shape similarity, physical proximity, cortical homunculus proximity, and semantic similarity.

One plausible organizing principle is shape similarity; previous fMRI studies have shown that objects with similar shapes evoke relatively similar response patterns in OTC (Haushofer et al., 2008; Op de Beeck et al., 2008). This account predicts that body parts that are relatively similar in shape (e.g., legs and arms) evoke relatively similar response patterns in OTC. A second possible principle is physical proximity between body parts; in our daily life experience body parts are perceived in highly regular spatial configurations. Accordingly, body parts that lie nearby in the body space, and are therefore processed nearby in retinotopic visual cortex, might also be represented nearby in high-level visual cortex. Previous studies have indeed provided evidence that such visual regularities affect the representation of body parts in OTC (Chan et al., 2010). A third possibility is that body part representations in visual cortex follow the cortical homunculus organization observed in somatosensory and motor cortices (Penfield and Boldrey, 1937; Mountcastle, 1984). Evidence for functional links between motor/somatosensory cortices and body-selective OTC regions has been reported (Astafiev et al., 2004; Orlov et al., 2010). Finally, a fourth possibility relates to the semantic/functional similarity among body parts (Reed et al., 2004; Peelen and Caramazza, 2010). For example, representations of body parts that are primarily involved in performing actions (e.g., hands and feet) might cluster together and separate from representations of body parts that support social communication (e.g., eyes and mouth), reflecting different processing and connectivity demands associated with these different types of cues (Peelen and Caramazza, 2010).

To test these hypotheses, we used RSA to relate similarity matrices capturing the models to neural similarity matrices of whole person-selective regions in LOTC and VOTC, and across the whole brain. Thus our aim was to characterize the organization underlying the neural similarity of body parts within the wider whole person-selective OTC, rather than individuate and localize separate body part representations, as done previously (Schwarzlose et al., 2005; Bracci et al., 2010; Orlov et al., 2010; Weiner and Grill-Spector, 2010).

Materials and Methods

Participants.

The study included 15 right-handed adult volunteers (seven males and eight females, mean age = 28, range 21–42). All participants gave informed consent. The study was approved by the ethics committee of the University of Trento (Italy).

fMRI experimental design and stimuli.

Ten conditions were included in a block-design fMRI experiment: whole persons, hands, feet, arms, legs, chests, waists, upper faces, lower faces, and chairs (Fig. 1A). The study included eight runs lasting 6 min and 52 s each. Within each run a fully randomized sequence of 10 category blocks (each repeated four times) and fixation blocks (repeated eight times) was presented. Each run started and ended with 14 s of fixation. Within each stimulus block (8 s) 10 images from one category (e.g., whole person) were each presented for 400 ms, followed by a blank screen for 400 ms.

Figure 1.

Experimental conditions and ROIs. A, Example stimuli for the 10 conditions in the fMRI study. Each condition included 36 different images (4 shown here), which differed in viewpoint, posture, or gender. B, Whole person-selective ROIs in LOTC and VOTC (yellow outline) and chair-selective control ROIs in OC (blue outline) are shown in one representative participant at the uncorrected threshold of p < 0.001.

Each category consisted of two sets of 36 different grayscale images on a white background and had a size of 12 × 12° (400 × 400 pixels). One set of images was used for even runs and the other set of images was used for odd runs. Within each run, each presented image was unique (except for the repetitions related to the one-back task). Participants performed a one-back repetition detection task by pressing a button with their right hand any time the same picture was presented two times in succession. In each block, one or two repetitions were presented. Stimulus presentation was controlled by a PC running the Psychophysics Toolbox package (Brainard, 1997) in MATLAB. Pictures were projected onto a screen and were viewed through a mirror mounted on the head coil.

Imaging parameters.

Data collection was performed on a Bruker 4 T scanner (Bruker BioSpin) with standard head coil at the Center for Mind/Brain Sciences, University of Trento. MRI volumes were collected using EPI T2*-weighted scans. Acquisition parameters were as follows: TR, 2 s; TE, 33 ms; FA, 73°; FOV, 192 mm; matrix size of 64 × 64 and 3 × 3 mm in-plane voxel size. Each volume comprised 34 axial slices with 3 mm thickness and 1 mm gap. The T1-weighted anatomical images were acquired with an MP-RAGE sequence, with 1 × 1 × 1 mm resolution.

Imaging analysis.

Imaging data were preprocessed and analyzed using BrainVoyager QX (version 2.20; Brain Innovation) and MATLAB. Functional images underwent the following preprocessing steps: linear trend removal, three-dimensional head-motion correction, and high-pass temporal filtering (cutoff three cycles per functional run). Subsequently, functional images were aligned to the T1 anatomical images, which were then transformed into Talairach stereotaxic space and spatially smoothed by convolution of a Gaussian kernel of 4 mm full-width at half-maximum. The smoothing kernel was based on previous work showing an increase in signal-to-noise ratio at this level of smoothing for correlation-based multivoxel pattern analysis in OTC (Op de Beeck et al., 2008; Op de Beeck, 2010). The GLM included regressors for each experimental condition (persons, hands, feet, arms, legs, chests, waists, upper faces, lower faces, and chairs) and the six motion correction parameters (x, y, and z for translation and for rotation). Each predictor's time course was modeled by a boxcar function convolved with BrainVoyager QX default “two-gamma” function. Functional runs were z-normalized before GLM analysis.

Regions of interest.

Whole person-selective ROIs in LOTC and VOTC were defined in each individual participant by contrasting whole persons versus chairs (Fig. 1B, yellow outline). ROIs included all spatially contiguous voxels that exceeded the statistical threshold (p < 0.001, uncorrected) within the occipital and temporal lobe. Whole person-selective regions in LOTC and VOTC could be defined in both hemispheres in all 15 subjects. As shown in Figure 1B, ROIs defined in this study are likely to include previously described headless body-selective (Downing et al., 2001; Peelen and Downing, 2005) and face-selective (Puce et al., 1996; Kanwisher et al., 1997) areas. Additional control regions, in occipital cortex (OC), were defined by contrasting chairs versus whole persons (Fig. 1B, blue outline), following the same procedure as for the whole person-selective ROIs. ROIs did not overlap with each other. Table 1 reports average single-subject Talairach coordinates and cluster size for whole person-selective and control ROIs.

Table 1.

Single-subject mean Talairach coordinates and cluster size for each ROI

ROIs	x (SD)	y (SD)	z (SD)	mm3
Whole-person-selective ROIs
Left LOTC	−48 (5.8)	−71 (8.4)	5 (6.8)	8232
Right LOTC	44 (8.7)	−67 (8.3)	3 (8.4)	11566
Left VOTC	−40 (3.8)	−47 (8.4)	−19 (4.7)	2098
Right VOTC	37 (5.8)	−50 (9.1)	−19 (4.7)	2691
Chair-selective (control) ROIs
Left OC	−23 (7.0)	−89 (6.3)	−7 (7.6)	7112
Right OC	17 (7.3)	−86 (6.4)	−10 (6.2)	5486

Single-subject mean Talairach coordinates and cluster size (mm³) are reported for the whole person-selective regions (persons > chairs) in LOTC and VOTC and control regions (chairs > persons) in OC.

Multivoxel pattern analysis.

Correlation-based multivoxel pattern analysis (Haxby et al., 2001) was used to investigate voxelwise similarities in the distribution of responses to the different body parts in LOTC and VOTC. Parameter estimates for each condition (relative to baseline) were extracted for each participant and run for each voxel of each ROI. These patterns were correlated across even and odd runs. Correlations were Fisher transformed {0.5 * log[(1 + r)/(1 − r)]} and averaged across even and odd runs (e.g., hands odd—feet even and feet odd—hands even), resulting in a symmetric 8 × 8 correlation matrix for each participant and ROI. Chairs and whole persons were excluded from the multivoxel pattern analysis because these conditions were used to localize the ROIs.

Representational similarity analysis.

Similarity matrices were established for each of the five models. The physical shape similarity (shape 1) was computed using the shape context algorithm (Belongie et al., 2002). The perceived shape similarity (shape 2) and physical proximity were rated by an independent group of participants (n = 6) using the multiple object arrangement method (Kriegeskorte and Mur, 2012). Each participant rated a different subset of eight images (one image for each body part condition), taken from the set of images used in the functional neuroimaging study. For perceived shape similarity, participants were asked to arrange the body parts based on perceived shape similarity, disregarding similarity in low-level image features (e.g., size, brightness, orientation). For physical proximity, participants were asked to arrange the body parts based on physical proximity of body parts. Results were averaged across the participants. The cortical homunculus model reflected similarity rankings in the well established topography of body part representations in motor and somatosensory cortex (Penfield and Boldrey, 1937; Mountcastle, 1984). Finally, the semantic similarity of body parts was based on the frequency of body part word co-occurrence in large text corpora (Kolb, 2009), reflecting similarity of semantic/functional context in which body part words are used. Previous fMRI studies have shown that brain activity for semantic concepts can be reliably predicted using models based on word co-occurrence (Mitchell et al., 2008; Huth et al., 2012). Each body part condition was assigned with a label corresponding to the body part's name (hands, feet, arms, legs, chest, and waist). When this was not possible (i.e., upper face and lower face) words of the parts that are most salient for a given condition were used (eyes and forehead for the upper face condition; mouth and chin for the lower face condition). Values of semantic similarities were computed on pairs of labels. The five models are summarized in Figure 2 by dissimilarity matrices (Fig. 2A), two-dimensional arrangements derived from multidimensional scaling (Fig. 2B), and hierarchical plots (Fig. 2C).

Figure 2.

Models. For each hypothesis (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) the figure shows the dissimilarity matrix (A; light colors indicate larger dissimilarity), the 2D arrangement derived from multidimensional scaling (B), and the hierarchical plot derived from the hierarchical cluster analysis (C).

Subsequently, we related the five model-based similarity matrices to the neural similarity matrix of each ROI. To do so we used multiple regression analysis with dissimilarity matrices for the five models as the dependent variable and an ROI's neural dissimilarity matrix (1 − r) as the independent variable. The multiple regression analysis was run for each individual participant (N = 15). Differences between the resulting regression coefficients were tested using pairwise t tests. Additional correlation-based analyses are also reported.

Whole-brain RSA was performed using the spherical volume-based searchlight approach (Kriegeskorte et al., 2006). For each voxel in the brain, we defined a sphere of 9 mm radius and established the neural dissimilarity matrix for this sphere, following the procedures used for the ROI analysis. The neural dissimilarity matrix was then correlated with the dissimilarity matrices derived from the five models. Resulting correlation values were Fisher transformed and assigned to the center voxel of the sphere to generate whole-brain correlation maps for each of the models. Differences between these statistical maps were tested using random-effects whole-brain group analysis and displayed on the inflated surface of one individual subject. For the contrast of each model relative to the other four models in conjunction (Fig. 6A) results are shown at p < 0.01 (uncorrected). In addition, to correct for multiple comparisons for the contrast of each model relative to the average of the other four models, results are shown at p < 0.05 cluster corrected using BrainVoyager QX's cluster-level statistical threshold estimator (Fig. 6B).

Figure 6.

Whole-brain representational similarity analysis. A, Results of random-effects whole-brain RSA, contrasting each model with the other four models in conjunction (p < 0.01, uncorrected). B, Results of random-effects whole-brain RSA, contrasting each model with the average of the other three models (p < 0.05, cluster corrected for multiple comparisons).

Multidimensional scaling and hierarchical cluster analysis were used to visualize and compare neural similarity structures in LOTC and VOTC and similarity structures related to the five models. Metric multidimensional scaling was performed using MATLAB function “mdscale” normalized with the sum of squares of the dissimilarities. The hierarchical cluster analysis was performed using the MATLAB function “linkage” using the default nearest distance method.

Results

Whole person-selective ROIs in LOTC and VOTC were defined in each participant by contrasting persons versus chairs. As shown in Figure 1B, activity to individual body parts (relative to chairs) largely fell within these whole person-selective ROIs and activity to individual body parts showed a large degree of overlap (Fig. 1B). Nonetheless, analysis of multivoxel activity patterns to the eight body part conditions in whole person-selective LOTC and VOTC ROIs (Fig. 1B) revealed that, in all ROIs, within-category correlations for all conditions were significantly higher than the average of between-category correlations for these conditions (t₍₁₄₎ = 4.21, p < 0.001, for all tests but legs in left VOTC: t₍₁₄₎ = 2.06, p = 0.058), thus showing distinct activity patterns for individual body parts. Interestingly, as can be observed in the neural dissimilarity matrices (1 − r; Fig. 3A), the off-diagonal values differed substantially and consistently from each other, suggesting meaningful variability in between-part similarity. What are the principles underlying this variability?

Figure 3.

Representational structure in LOTC and VOTC. A, Neural dissimilarity (1 − r) and mean response profile (B) in whole person-selective LOTC and VOTC. C, MDS, performed on neural dissimilarity matrices (1 − r), shows pairwise distances in a 2D space. Pairwise distances reflect response-pattern similarity: body parts positioned next to each other elicited similar response patterns, whereas body parts positioned far from each other elicited dissimilar response patterns. D, For each ROI, the hierarchical plot derived from the hierarchical cluster analysis shows the activity-pattern similarity structure in these regions.

To address this question, we compared the neural similarity matrices with five hypothetical models of organization, illustrated in Figure 2. (1) The physical shape similarity (shape 1) model predicts relatively high similarity between upper faces, lower faces, chests, and waists. (2) The perceived shape similarity (shape 2) model predicts four separated clusters (legs and arms, hands and feet, chests and waists, and upper faces and lower faces) reflecting the rated shape similarities among body parts. (3) The physical proximity model predicts high similarity between physically connected body parts, such as hands and arms and feet and legs. (4) The cortical homunculus model predicts face parts to be distinct from lower limbs (legs and feet) but relatively similar to upper limbs (hands and arms). (5) Finally, the semantic similarity model predicts a high degree of similarity between response patterns evoked by hands, feet, arms, and legs.

Multiple regression analysis revealed that the perceived shape similarity model, the physical proximity model, and the semantic similarity model were all positively related to neural similarity in LOTC and VOTC ROIs (t₍₁₄₎ > 4.31, p < 0.001, for all tests; Fig. 4A), whereas the cortical homunculus model and the physical shape similarity model did not explain additional variance in neural similarity in any of the ROIs. Direct comparisons between the models showed that the semantic similarity model was significantly more related to neural similarity than were any of the other models (t₍₁₄₎ < 2.58, p < 0.02, for all tests; Fig. 4A).

Figure 4.

ROI representational similarity analysis. A, For each whole person-selective ROI, bar graphs show parameter estimates of regression analyses relating model-based dissimilarity matrices to neural dissimilarity matrices. The explained variance of the full regression models (adjusted R²) were as follows: left LOTC (R²_adj = 0.43), right LOTC (R²_adj = 0.37), left VOTC (R²_adj = 0.34), right VOTC (R²_adj = 0.30). B, Results of regression analyses in chair-selective OC control ROIs. The explained variance of the full regression models (adjusted R²) were as follows: left OC (R²_adj = 0.16), right OC (R²_adj = 0.14). Error bars indicate SEM.

To ensure that these results were not specific to the way we defined the LOTC and VOTC ROIs (persons vs chairs), we repeated the analysis in additional ROIs defined by the contrast between the average of the eight body part conditions versus chairs. Definition of these ROIs followed the same procedure as for the whole person-selective ROIs (see Materials and Methods). Bilateral ROIs could be defined in 13 of 15 participants. Results in these ROIs were very similar to results in whole person-selective ROIs: perceived shape similarity, physical proximity, and semantic similarity were all positively related to neural similarity in all ROIs (t₍₁₂₎ = 3.51, p < 0.004, for all tests), whereas cortical homunculus proximity and physical shape similarity were not. In all ROIs, neural similarity was significantly more related to the semantic similarity model than to the other four models (t₍₁₂₎ < 2.29, p < 0.04, for all tests). These results indicate that the body part representational structure observed in LOTC and VOTC is not specific to the type of contrast used to define body-selective ROIs.

Visual inspection of multidimensional scaling arrangements (MDS; Fig. 3C) and hierarchical plots (Fig. 3D) derived from the neural similarity matrices revealed an organization of three clearly separated clusters: one for body effectors (hands, feet, legs, and arms), one for noneffector body parts (chests and waists), and one for faces (upper faces and lower faces). These three body part clusters were similarly observed in LOTC and VOTC in both hemispheres, despite the different univariate activity profiles of these ROIs (Fig. 3B).

To test whether nearby regions showed a similar organization, we performed the above regression analysis in two additional ROIs (chairs vs persons; Fig. 1B) in left and right OC (see Materials and Methods). Results (Fig. 4B) revealed that, unlike the whole person-selective ROIs, only the two shape similarity models were positively related to neural similarity in OC regions (physical shape: t₍₁₄₎ = 3.50, p < 0.004, for both regions; perceived shape: t₍₁₄₎ = 4.38, p < 0.001, for both regions). Furthermore, the neural similarity in both OC regions was significantly more related to the physical shape model than to the other four models (t₍₁₄₎ = 2.60, p < 0.02, for all tests).

To directly test for differences between whole person-selective (LOTC/VOTC) and chair-selective (OC) ROIs, regression coefficients for each model in each ROI (averaged across whole person-selective regions and chair-selective regions) were tested in a 5 (Model) × 2 (ROI) ANOVA with Model (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) and ROI (LOTC/VOTC and OC) as within-subject factors. Results (Fig. 5A) revealed a significant Model × ROI interaction (F_(4,56) = 58.60; p < 0.001), indicating that the models differentially related to the neural similarity structures of person- and chair-selective ROIs. Confirming earlier analyses, in person-selective LOTC/VOTC the semantic similarity model was significantly more related to the neural similarity than were any of the other models (t₍₁₄₎ = 5.98, p < 0.001, for all tests), whereas in chair-selective OC the physical shape model was significantly more related to neural similarity than were the remaining models (t₍₁₄₎ = 5.19, p < 0.001, for all tests).

Figure 5.

Regression- and correlation-based representational similarity analysis. A, Regression results for the 5 (Model) × 2 (ROI) ANOVA with Model (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) and ROI (LOTC/VOTC and OC) as within-subject factors. B, Correlation results for the 5 (Model) × 2 (ROI) ANOVA with Model (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) and ROI (LOTC/VOTC and OC) as within-subject factors. The dotted line indicates the between-subject correlation of neural dissimilarity matrices (Op de Beeck et al., 2008) giving an estimate of the reliability of the neural data (Nili et al., 2014). This correlation was computed for each participant as the correlation between that participant's neural dissimilarity matrix and the average neural dissimilarity matrix of the remaining participants. These correlations were then averaged across participants to arrive at the value indicated by the dotted line. The correlations corresponding to each model (colored bars) were divided by the reliability estimate, giving the following results: physical shape similarity (person-selective LOTC/VOTC: 14%; chair-selective OC: 49%), perceived shape similarity (person-selective LOTC/VOTC: 32%; chair-selective OC: 46%), physical proximity (person-selective LOTC/VOTC: 43%; chair-selective OC: 12%), cortical homunculus (person-selective LOTC/VOTC: 27%; chair-selective OC: 2%), semantic similarity (person-selective LOTC/VOTC: 59%; chair-selective OC: −6%). Error bars indicate SEM.

To confirm these results, we repeated the analysis using correlation-based RSA, correlating the dissimilarity matrix of each model separately with neural dissimilarity matrices (i.e., not using multiple regression). Correlation values for each model and each ROI (whole person-selective LOTC/VOTC and chair-selective OC) were tested in a 5 (Model) × 2 (ROI) ANOVA with Model (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) and ROI (LOTC/VOTC and OC) as within-subject factors. Results (Fig. 5B) again revealed a significant Model × ROI interaction (F_(4,56) = 46.80; p < 0.001), indicating that the models differentially related to the neural similarity structures of person- and chair-selective ROIs. Post hoc pairwise t tests confirmed that in person-selective LOTC/VOTC the semantic similarity model was significantly more related to the neural similarity than were the other four models (t₍₁₄₎ = 3.64, p < 0.003, for all tests), whereas in chair-selective OC the two shape models were significantly more related to neural similarity than were the remaining models (t₍₁₄₎ = 3.56, p < 0.003, for all tests).

Finally, to test for additional brain regions where neural similarity was differentially related to one of the five models we performed RSA across the brain using a searchlight approach (see Materials and Methods). For each 9 mm sphere, the neural dissimilarity matrix was correlated with dissimilarity matrices derived from the four models (Fig. 2A) and these correlations were subsequently contrasted. Results from these contrasts are shown in Figure 6. Results confirmed and extended results from the ROI analyses: the contrast between the semantic similarity model with each of the other four models in conjunction (p < 0.01, uncorrected, for each of the four contrasts; Fig. 6A,Table 2) revealed clusters in left LOTC (−58, −55, −11) and left VOTC (−35, −65, −11). Additional clusters were observed in frontal and parietal cortex (see Table 2). Similar clusters were observed when the semantic similarity model was contrasted with the average of the other four models (p < 0.05, cluster corrected; Fig. 6B, Table 2). The contrast between the physical shape similarity model (shape 1) and each of the other four models in conjunction (p < 0.01, uncorrected) revealed clusters in posterior visual areas bilaterally (Fig. 6A). The contrast between the perceived shape similarity model (shape 2) and each of the other four models in conjunction (p < 0.01 uncorrected) revealed a small cluster in retrosplenial cortex (RSC). Contrasts testing for regions reflecting the physical proximity model and the cortical homunculus model did not yield any clusters.

Table 2.

Whole-brain RSA

Model contrasts	x	y	z
Semantic > all
Left VOTC	−35	−65	−11
Left LOTC	−58	−55	−11
Left SPL	−26	−67	39
Left IPS	−32	−39	50
Left IPS	−50	−38	43
Left PMv	−52	5	22
Left PMd	−36	−11	47
Right SPL	30	−74	31
Right SPL	23	−65	44
Semantic > all (conjunction)
Left VOTC	−31	−67	−8
Left LOTC	−57	−54	−10
Left SPL	−22	−68	43
Left SPL	−27	−74	24
Left IPS	−32	−39	50
Left PMv	−52	5	22
Right SPL	23	−70	39
Physical shape (shape 1) > all (conjunction)
Left OC	−36	−90	−9
Left OC	−17	−97	−1
Right OC	16	−95	0
Right OC	27	−88	1
Perceived shape (shape 2) > all (conjunction)
RSC	−2	−58	3

Results for the whole-brain RSA group average Talairach coordinates are reported for the contrasts shown in Figure 6. IPS, inferior parietal lobe; PMd, dorsal premotor cortex; PMv, ventral promotor cortex; SPL, superior parietal lobe.

Discussion

The representation of the body in motor and somatosensory cortices has been widely studied, and the organization of these body maps is relatively well characterized. In contrast, the principles underlying body maps in visual cortex are largely unknown. In the present study, we investigated the organization of body part representations in visual cortex by testing and contrasting multiple possible organizational principles (Fig. 2). Representational similarity analysis revealed that the neural similarity structures of person-selective VOTC and LOTC regions were most closely related to the semantic similarity model. Perceived shape similarity and physical proximity, but not physical shape similarity and homunculus proximity, explained additional variance in the neural similarity data. In contrast to VOTC and LOTC, the neural similarity structure of OC was most closely related to physical and perceived shape similarity models.

Of the models tested, the semantic similarity model best explained the neural similarity in lateral and ventral OTC (Figs. 4, Fig. 5). It should be noted, however, that neither this model alone nor all models combined fully accounted for the neural similarity structure. This suggests imperfections in the models and/or the existence of additional organizational principles that we did not consider. Visual inspection of the MDS and hierarchical clustering results reveals three separated clusters in VOTC and LOTC (Fig. 3): one for body parts that are used as effectors (hands, feet, legs, and arms), one for noneffector body parts (chests and waists), and one for face parts (upper faces and lower faces). This tripartite organization was highly similar in VOTC and LOTC, in both hemispheres.

The organization in three clusters (effectors, noneffectors, and faces) suggests an organization based on functional properties of body parts. This function-based organization was partly (though not fully) captured by the semantic similarity model (Fig. 2). The semantic similarity model was based on the frequency of body part word co-occurrence in written language; body parts that are used in similar functional (con)texts are more likely to co-occur, for example, in sentences describing body actions (involving effectors) versus sentences describing social interactions (involving faces). While there are other reasons for why body part words co-occur in texts, we interpret the commonality of the semantic similarity model and the neural similarity structure as reflecting the different functional roles of different body parts. For example, whereas limb postures and limb movements are important for action execution and understanding, facial features (and their configuration) are fundamental for social interactions. Behavioral evidence supports this interpretation, showing that the semantic representation of the human body was primarily organized by functional properties of body parts (Reed et al., 2004). The representational structure of body parts in human OTC may partly reflect the different computations that are relevant for the processing of these different types of information. For example, motion processing is relatively important for action recognition and action execution, while configural processing is relatively important for identity recognition.

Furthermore, connectivity demands differ as a function of information content (Peelen and Caramazza, 2010; Mahon and Caramazza, 2011): action-related body part information is relevant for downstream action networks, whereas social information (e.g., identity) extracted from faces needs to connect to brain areas involved in social cognition. Previous studies have indeed provided evidence for selective functional connectivity between OTC regions and domain-specific downstream networks (Mahon et al., 2007; Bracci et al., 2012; Simmons and Martin, 2012). Such downstream connectivity constraints have been proposed as a more general organizing principle of category selectivity in OTC (Mahon and Caramazza, 2011). Indeed, the organization observed here may reflect a broader organizational structure of OTC that applies to both objects and body parts. For example, objects that are used as effectors (e.g., tools) are represented nearby body effectors in OTC (Bracci et al., 2012; Bracci and Peelen, 2013). Future work that combines voxelwise selectivity measures and voxelwise functional or anatomical connectivity “fingerprints” could further test this hypothesis (e.g., Simmons and Martin, 2012; Saygin et al., 2012).

The aim of the present study was to reveal the organizational structure of body part representations within OTC. This approach differs from previous studies that focused on revealing the commonality of activation to different nonface body parts relative to object controls (Downing et al., 2001, 2007; Weiner and Grill-Spector, 2011). Consistent with these reports, our study revealed that different body parts activate largely overlapping regions in OTC when contrasted with chairs (Fig. 1). Importantly, however, our results revealed that within this region there is a consistent organization of body part-specific responses, one that is best explained by the semantic similarity of body parts.

Our study also differs from previous studies aimed at localizing circumscribed regions that respond preferentially to one body part relative to others (Schwarzlose et al., 2005; Bracci et al., 2010; Orlov et al., 2010). Unlike the current study, these studies did not attempt to reveal organizational principles that could explain the reported differential responses. For example, Op de Beeck et al. (2010) revealed distinct response patterns to hands and torsos but did not test what drove this distinction; all of the models tested here would have predicted this distinction (Fig. 2). Indeed, the current findings help to interpret previous findings, revealing the likely dimensions that drove previously reported distinctions between hands and torsos (Op de Beeck et al., 2010), hands and headless bodies (Bracci et al., 2010), and face and nonface body parts (Downing et al., 2001; Peelen and Downing, 2005; Schwarzlose et al., 2005). Because of the different analytical approach, our results do not contradict previous reports of focal regions selective for individual body parts when contrasted with each other (Orlov et al., 2010). For example, the finding that hand representations cluster with representations of other effectors within the broad OTC region considered here does not exclude the existence of a focal region selectively responsive to hands relative to other body parts (Bracci et al., 2010). What our results show is that such body part-selective responses cluster in meaningful ways, partly reflecting semantic/functional similarity.

The current approach highlights the larger scale organizational structure of body part representations in OTC, showing that this organization reflects the semantic/functional similarity of body parts. It is important to clarify that this conclusion pertains to the organization of body part representations in OTC, not to the representational content of these representations. Indeed, the finding of a semantic/functional organization does not exclude the possibility that OTC primarily represents the form of body parts. For example, the functional/semantic clustering of body form representations may reflect connectivity constraints to downstream networks involved in specific functions such as social cognition or action perception/execution (Bracci et al., 2012).

Interestingly, the whole-brain representational similarity analysis (Fig. 6) revealed regions beyond OTC, particularly in parietal and left frontal cortex, in which neural similarity was more strongly related to semantic similarity than to any of the other models. Similar to our interpretation of OTC results, the organization in these regions may reflect the functional similarity of body parts, differentiating action-related body parts (effectors) and noneffectors. This would be consistent with the previously reported sensorimotor function of these regions (Rizzolatti and Craighero, 2004; Gallivan and Culham, 2015). Indeed, it is increasingly appreciated that LOTC is part of an action network together with regions in frontal and parietal cortex (Lingnau and Downing, 2015). For example, a left LOTC region responsive to hands and tools was functionally connected to left premotor cortex and left intraparietal sulcus (Bracci et al., 2012), and these regions showed very much the same action-related functional profile to objects and body parts (Bracci and Peelen, 2013). Future studies could further investigate what drives the semantic organization of body parts in frontal and parietal cortex, and how these regions interact with body part representations in OTC during action perception and execution.

In addition to clusters organized by semantic similarity, the whole-brain analysis also revealed clusters, in OC, in which the physical shape similarity model outperformed the other models, including the perceived shape similarity model (Fig. 6A). Interestingly, a region in RSC selectively followed perceived (rather than physical) shape similarity. It should be noted, however, that these regions did not appear in the contrast maps that were corrected for multiple comparisons (Fig. 6B). Nevertheless, it will be of interest for future work to better investigate the putative dissociation between representations of perceived and physical object shape in visual cortex.

To conclude, the present study shows that visual dimensions, such as shape similarity and retinal proximity, do not fully account for the organization of body part representations in OTC. Instead, the representational structure of the body in OTC appears strongly related to functional-semantic properties of body parts. This organization is more closely related to that observed in frontal and parietal cortex than to that observed in OC. We suggest that this organization reflects the unique processing and connectivity demands associated with the different types of information different body parts convey.