Neural architecture underlying classification of face perception paradigms

Abstract

We present a novel strategy for deriving a classification system of functional neuroimaging paradigms that relies on hierarchical clustering of experiments archived in the BrainMap database. The goal of our proof-of-concept application was to examine the underlying neural architecture of the face perception literature from a meta-analytic perspective, as these studies include a wide range of tasks. Task-based results exhibiting similar activation patterns were grouped as similar, while tasks activating different brain networks were classified as functionally distinct. We identified four sub-classes of face tasks: (1) Visuospatial Attention and Visuomotor Coordination to Faces, (2) Perception and Recognition of Faces, (3) Social Processing and Episodic Recall of Faces, and (4) Face Naming and Lexical Retrieval. Interpretation of these sub-classes supports an extension of a well-known model of face perception to include a core system for visual analysis and extended systems for personal information, emotion, and salience processing. Overall, these results demonstrate that a large-scale data mining approach can inform the evolution of theoretical cognitive models by probing the range of behavioral manipulations across experimental tasks.

INTRODUCTION

As more resources are being developed and deployed for the management, sharing, and meta-analysis of “big data” in neuroimaging, the development of knowledge representation systems has likewise accelerated to enable objective and succinct descriptions of these data, including neurotechnological, neuroanatomical, and cognitive parameters. However, cognitive data descriptors are relatively under-developed compared to those from the neurotechnological and neuroanatomical domains. That is, as a community we are relatively more confident regarding data annotations differentiating sub-class or type of MRI scan (e.g., T2* or EPI images) or brain structure (e.g., hippocampus or amygdala) than in differentiating data that relates to memory (e.g., episodic or working). Nevertheless, semantic representation of cognitive and perceptual mental processes is a necessary component of large-scale, community-wide, and consensus-based mapping of structure-function correspondences in the human brain. Such a representation must include the full and robust definitions of mental processes; however, the identification and standardization of terms we use to describe the multitude and diversity of cognitive and perceptual functions is an inexact science. As a result, many alternative and often competitive terminologies exist. With the rise of high profile, high-impact projects such as the Human Connectome Project (Van Essen et al., 2013; Toga et al., 2012), the BRAIN Initiative (BRAIN Working Group, 2014), the Human Brain Project (Markram, 2012), and the RDoCs framework (Insel et al., 2010), the need for knowledge representations of cognitive aspects of neuroscience data has reached a critical point. Our community goal of mapping the human brain will surely require definition and standardization of the terms that are used to describe human thought and mental processes, as well as the behavioral tasks used to elicit them during neuroscience experiments.

Here, we propose and validate a strategy for deriving a classification system of functional neuroimaging paradigms using a proof-of-concept application. Our aim was to develop a meta-analytic data mining approach for paradigm classification based on neurobiological evidence provided by functional activation patterns, with the intent that such a strategy may mitigate the challenges associated with a lack of paradigm-related semantic consensus within a given domain. The overall premise of this work is that differences in activation patterns across studies should be captured and leveraged as they indicate meaningful segregations in brain function. Under this premise, tasks activating similar brain networks should be grouped as functionally similar in a cognitive schema, while tasks demonstrating differential activation patterns should be classified as functionally distinct.

As an exemplar domain, we demonstrate our approach in the context of face discrimination, as this category of neuroimaging tasks is highly heterogeneous and commonly employed across numerous perceptual, cognitive, and affective studies in both healthy and diseased populations. These studies broadly include visual stimulus presentation of human faces in which participants passively view faces or actively discriminate one or more aspects or features of face presentation (e.g., old/new, male/female, happy/sad/angry/fearful, etc.). Faces can be used as stimuli for classical conditioning, lip-reading, and naming tasks, or to cue autobiographical memory retrieval, emotion induction, or social processing. Given the wide scope of face tasks in the literature, our aim was to establish a neuroinformatics procedure capable of objectively decomposing the collective group into meaningful sub-categories. Using meta-analytic data reported across a diverse range of studies archived in the BrainMap database, we sought to determine if multiple functional networks distributed across the brain are differentially recruited for various task paradigms. Our ultimate goal was threefold: to develop a paradigm classification strategy for use by cognitive ontologies, to examine the underlying neural architecture of face perception from a meta-analytic perspective, and, more broadly, to assess whether an evidence-based data mining approach can inform the evolution of existing cognitive models.

METHODS

Meta-Analytic Data Extraction and Pre-Processing

The BrainMap database (Fox and Lancaster, 2002; Laird et al., 2005a; Laird et al., 2011a) currently archives brain activation locations from over 11,900 functional magnetic resonance imaging (fMRI) or positron emission tomography (PET) experiments (from over 2,400 journal articles). These experiments have been manually annotated with metadata that describe the experimental design of each archived study. Our study focused on a subset of tasks within BrainMap that were annotated with the paradigm class of “Face Monitor or Discrimination”; the relevant experiments were identified and downloaded for further analysis using the desktop search engine application, BrainMap Sleuth (http://www.brainmap.org/sleuth). Search results were filtered to include only face tasks performed by healthy adults to limit any potential bias due to effects of age, disease, or treatment differences. Information about the specific behavioral task performed by participants in each experiment, along with the experiment name, sample size, and stereotaxic coordinates of activation were exported as a tab-delimited text file. Exported coordinates reported in MNI space (Evans et al., 1993; Collins et al., 1994) were transformed to Talairach space (Talairach and Tournoux, 1988) using the Lancaster transform function icbm2tal (Lancaster et. al., 2007). icbm2tal was developed using global affine transforms to accommodate spatial disparity between Talairach and MNI coordinates as compared to the earlier mni2tal transform (Brett et al., 2001), and to minimize meta-analytic spatial dissonance due to template differences (Laird et al., 2010). Modeled activation (MA) maps were generated by modeling each coordinate of activation as a spherical Gaussian distribution of uncertainty to represent the probability of activation for each voxel, centered upon the experiment’s activation foci (Figure 1, Step 1). The algorithm includes an estimation of the inter-subject and inter-laboratory variability associated with each experiment, and is weighted by the number of subjects included in each experiment (Eickhoff et al., 2009). The per-experiment MA probability maps were converted into feature vectors of voxel values and concatenated horizontally to form an array of size n experiments by p voxels.

Figure 1. Data Analysis Pipeline.

Step 1: Per-experiment coordinate-based data were extracted from BrainMap and used to generate modeled activation (MA) maps. Step 2: A pairwise correlation analysis was performed in which correlation coefficients were calculated for each n experiment compared to every other n experiment. Step 3: Experiments were grouped into clusters using hierarchical clustering analysis. Step 4: Meta-analysis of the per-cluster set of experimental MA-maps was performed using the activation likelihood estimation (ALE) method.

Correlation Matrix Based Hierarchical Clustering Analysis

After generating the n X p matrix of MA probability maps, we employed a pairwise correlation analysis in which correlation coefficients were calculated for each n experiment compared to every other n experiment, to assess similarity of spatial topography across MA maps. Hence, the n X p array of MA maps was transformed into an n X n correlation matrix that captured the similarity of whole-brain modeled activation images across face discrimination experiments (Figure 1, Step 2). Experiments within the n X n correlation matrix were subsequently grouped into clusters using hierarchical clustering analysis, an agglomerative unsupervised classifier (Figure 1, Step 3). Previous implementation of correlation matrix based hierarchical clustering of resting state fMRI data (Liu et al., 2012; Keilholz et al., 2010) and hierarchical clustering of BrainMap-based meta-analytic images (Laird et al., 2011b) demonstrated optimal clustering using the average linkage algorithm and 1 – r as the distance between clusters, where r is the Pearson’s correlation coefficient. Following initial testing for optimal performance, these parameters were adopted in the present study. Notably, Pearson’s correlation distance maximizes the effects of direction, rather than magnitude, of the two observational vectors, thus identifying correlated MA maps as being topologically similar and anti-correlated MA maps as dissimilar.

The resultant dendrogram was examined to identify sets of experiments that clustered together. Selecting a clustering solution yielding an optimal parcellation of BrainMap experiments relied on two measures. The cophenetic distance between clusters at a specific model order (i.e., number of clusters) describes the dissimilarity between sub-clusters, and is intrinsically higher at low model orders (e.g., a two-cluster solution). Importantly, the relative difference in cophenetic distances when transitioning from model order x to the next highest model order x+1 can be informative when examining if cluster separation results in vastly different solutions. Therefore, we sought to determine the extent to which increasing model order resulted in substantially different activation patterns respective to each cluster by maximizing the relative difference, d_c, in cophenetic distances c_x and c_x+1, as model order, x, increased:

$$d_c=\frac{c_{x+1}-c_x}{c_{x+1}}$$ (Eqn. 1)

Related to the above measure of difference in cophenetic distances is the impact that increasing model order has on separating clusters into sub-clusters of proportionate number of variables (e.g., experiments). Increasing the cluster solution could potentially yield a segregation of experiments in which one sub-cluster dominates with a disproportionately large number of experiments. Therefore, we sought to minimize the effect of cluster segregation by calculating the maximum density of experiment separation. Essentially, we aimed to determine if increasing model order resulted in a disproportionate divergence of experiments. For example, if cluster i₀, consisted of n₀ experiments at model order x, and separated into clusters i₁ and i₂, with n₁ and n₂ experiments, respectively, then the density of experiment separation, d_s is calculated as:

$$d_s=\frac{n_1}{n_0},n_1\ge n_2$$ (Eqn. 2)

Thus, to achieve an optimal clustering solution of BrainMap experiments, the impact of parcellating clusters with increasing model order was assessed by both the relative difference in cophenetic distances and the experiment separation density.

ALE Meta-Analysis of Clustered Experiments

After selecting an optimal clustering solution, we investigated the underlying spatial features of the groups of experiments deemed similar in terms of their convergent activation patterns. Meta-analysis of the per-cluster set of experimental MA-maps was performed using the activation likelihood estimation (ALE) method (Turkeltaub et al., 2002; Laird et al., 2005b) (Figure 1, Step 4). Similar to the generation of the MA maps, ALE models the stereotaxic coordinates for each experiment as spherical Gaussian probability distributions. Using BrainMap GingerALE (version 2.3), an extended ALE algorithm was implemented that accounts for between-subject variability and between-template variance due to differences in spatial normalization methods across publications (Eickhoff et al., 2009), as well as eliminates within-experiment effects (Turkeltaub et al., 2012). Voxel-wise ALE scores were computed as the union of the probability distributions, which provide a quantitative representation of convergent brain activation patterns; statistical significance was assessed by analytically deriving the null distribution of random spatial association between experiments (Eickhoff et al., 2012). ALE maps for each cluster of the dendrogram were thresholded at P<0.05 (cluster-level family-wise error rate (FWER) corrected for multiple comparisons; cluster-forming threshold P<0.001 at the voxel-level; Woo et al., 2014).

Functional Decoding of Paradigm Clusters

Once the convergent spatial activation patterns were identified for each cluster, we examined the experimental design of the contributing tasks to determine which task features (e.g., stimulus, response, or instructions) led to the observation of similar activation patterns. That is, for a dendrogram yielding n clusters of activation patterns, we evaluated the behavioral tasks utilized in the experiments grouped together in cluster #1, the tasks in cluster #2, and so on for all n clusters. To this end, the associated metadata of the contributing experiments were analyzed to determine the combinations of stimulus, response, and instructions that were most likely to contribute to the observed clustering of activation patterns. Forward and reverse inference analyses (Poldrack, 2006; Yarkoni et al., 2011) were performed on the distributions of metadata fields for each cluster of experiments to determine over-representation compared to the entire BrainMap database (Cieslik et al., 2013; Nickl-Jockschat et al., In Press). Forward inference is the probability of observing activity in a brain region given knowledge of the psychological process, whereas reverse inference is the likelihood that a given metadata term resulted in an activation in the experiments within a cluster. In the forward inference approach, we tested whether the conditional probability of activation given a particular label [P(Activation|Task)] was higher than the baseline probability of activating the region in question per se [P(Activation)]. Significance was established using a binomial test (P < 0.05, corrected for multiple comparisons using Bonferroni’s method), which was appropriate since our goal was to compare the observed distribution to the expected distribution given only two categories (i.e., frequency of term within a cluster and across the entire database). In the reverse inference approach, we identified the most likely metadata term (e.g., behavioral domain) given activation in a particular cluster. This likelihood P(Task|Activation) can be derived from P(Activation|Task) as well as P(Task) and P(Activation) using Bayes rule. Significance (at P < 0.05, corrected for multiple comparisons using Bonferroni’s method) was then assessed by means of a chi-squared test. Tested metadata fields included the behavioral domain, stimulus type, response type, and instructions. In addition, the field of paradigm class was also analyzed; however, it was expected that every cluster would return a significant paradigm of “Face Monitor/Discrimination”, given that this was the field chosen for the initial BrainMap query. However, this field was still deemed potentially informative since studies can be annotated with multiple paradigm classes, when relevant.

RESULTS

Search of the BrainMap database for experiments meeting the criteria of paradigm class of “Face Monitor/Discrimination” in healthy individuals yielded 379 experiments, which included results reported across 2,917 brain locations in 4,945 participants. A complete listing of these experiments, along with the corresponding Sleuth workspace file, can be accessed in the Supplementary Materials. MA probability maps were generated for each of the 379 experiments, and a spatial correlation analysis was performed to compare the topography of each pair of experiments, thereby assessing the similarity of whole-brain modeled activation images across face discrimination experiments.

Correlation Matrix Based Hierarchical Clustering Analysis

Hierarchical clustering was performed on the 379 x 379 correlation matrix, yielding a complex dendrogram. The optimal clustering solution of “Face Monitor or Discrimination” experiments was identified by observing the impact of parcellating clusters with increasing model order through both the experiment separation density and relative change in cophenetic distances. We aimed to achieve a robust solution for characterizing the groupings of experiments in our data; here, we describe results for solutions between 3 and 10 clusters. While the metrics pertaining to higher cluster solutions were also examined, these fell well short of the optimal solution we aimed to achieve.

To achieve an clustering solution that optimally parcellated across different groups of experimental activation patterns, the relative difference in cophenetic distances and the experiment separation density were calculated as a function of increasing model order. Peak values for the relative difference in cophenetic distances were observed for solutions of 4, 6, and 8 clusters (Figure 2, blue), indicating model orders of activation patterns with substantially different topological characteristics. The second cluster criterion examined was the experiment separation density. The goal for this measure was to avoid solutions that segregated a single cluster into sub-clusters of disproportionate experiment density. Cluster solutions resulting in a decrease in experiment separation density from one model or to the next were observed for 4, 6, 7, and 10 clusters (Figure 2, red). Based on convergence of both metrics, clustering solutions of 4 and 6 were considered as candidate clustering solutions; these solutions agreed with visual inspection of the dendrogram. Ultimately, the 6-cluster solution was identified as the optimal solution due to the substantially larger relative difference in cophenetic distances compared to the 4-cluster solution.

Figure 2. Cluster Criteria.

The difference in cophenetic distances (blue) and density of experiment separation (red) are plotted as a function of candidate model orders. Peaks in the difference in cophenetic distances indicate that the corresponding clustering solution results in clusters with substantially different activation patterns than the model order below it. Decreases in density of experiment separation indicate a given solution yields clusters of non-trivial size, providing robustly different activation patterns. Convergence across both metrics was observed for a solution of 6 clusters.

Any clusters with less than 19 experiments (<5% of the dataset) were deemed as outlying data and excluded from further analysis. The small clusters on the far left and right of the dendrogram fell into this criterion (seen in white) with thirteen and two experiments, respectively, and were subsequently discarded. The cophenetic distances indicated that these clusters were quite different from the other clusters, further providing support for their omission. Four final clusters remained suitable for further analyzed: Cluster 1 (blue; 111 experiments), Cluster 2 (pink; 151 experiments), Cluster 3 (green; 68 experiments), and Cluster 4 (red; 34 experiments) (Figure 3).

Figure 3. Hierarchical Clustering of MA Maps.

Results of the hierarchical clustering analysis of face studies yielded a 4-cluster solution, as depicted via a dendrogram plot. Cluster 1 (blue) was comprised of 111 experiments, Cluster 2 (purple) = 151 experiments, Cluster 3 (green) = 68 experiments, Cluster 4 (red) = 34 experiments. Due to small representation (i.e., less than 5% of total number of experiments), clusters of size 13 and 2 (white) were omitted from further analysis.

ALE Meta-Analysis of Clustered Experiments

Coordinates of activation reported across each of the four clusters were separately meta-analyzed using the ALE approach (Eickhoff et al., 2012; Turkeltaub et al., 2002) to generate four maps illustrating the locations of convergent activations (P > 0.05; cluster-level FWE-corrected for multiple comparisons; cluster-forming threshold P < 0.001 at the voxel-level). As shown in Figure 4, the ALE meta-analysis map for Cluster 1 included significant areas of convergence in the anterior cingulate cortex, bilateral insula, precentral gyrus, premotor cortex and (pre-) supplementary motor area, lateral inferior frontal gyrus, dorsolateral prefrontal cortex (DLPFC), putamen, and left superior temporal sulcus. The Cluster 2 map indicated significant convergence in the bilateral fusiform gyrus, extrastriate cortex, superior and middle temporal cortex, lingual gyrus, amygdala, DLPFC, and cerebellum. Significant convergence was observed in Cluster 3 in the ventromedial prefrontal cortex, posterior cingulate cortex and precuneus, and right amygdala and corpus striatum. Lastly, Cluster 4 included significant convergence within the bilateral middle temporal gyrus and inferotemporal cortex, temporal pole, left superior temporal sulcus, parahippocampal gyrus, and inferior and superior frontal gyrus.

Figure 4. ALE Meta-Analysis of Experimental Clusters.

ALE meta-analysis results of experiments comprising each dendrogram cluster are shown for the left hemisphere (lateral and medial), axial slices, and right hemisphere (medial and lateral). ALE images were thresholded at P<0.05 (cluster-level FWE corrected for multiple comparisons; cluster-forming threshold P<0.001 at the voxel-level).

Functional Decoding of Paradigm Clusters

The clustering results indicated four robust groupings of experiments with similar within-cluster and different across-cluster spatial topographies. Forward and reverse inference analyses were performed on five different metadata fields of the BrainMap coding scheme: behavioral domain, paradigm class, stimulus type, response type, and instructions, to determine which metadata classes within each field exhibited an over-representation in each cluster’s experiment set compared to the distribution across experiments in the entire BrainMap database. The number of metadata classes within each field varied from 18 (response types) to 95 (paradigms), with a total of 258 classes investigated across all fields. Forward and reverse inference analyses indicated 14, 14, 10, and 8 significant classes within Clusters 1, 2, 3, and 4, respectively. Figure 5 illustrates the significant metadata BrainMap classes for each cluster, with pathways that connect one or more clusters indicating specific terms that were significantly associated with multiple clusters. For example, the stimulus type of “Faces” and the paradigm class of “Face Monitor/Discrimination” were associated with all clusters as predicted, whereas the instruction of “Discriminate” related only to Clusters 2 and 4. Notably, all clusters except Cluster 4 indicated significant association with several behavioral domain terms associated with emotional and visual processing, as shown in the center of Figure 5. Visualization of these metadata results was achieved using Cytoscape version 3.1.1 (Shannon et al., 2003).

Figure 5. Significant BrainMap Metadata Fields for Each Cluster.

Forward and reverse inference analyses were performed to identify significant BrainMap metadata for each cluster. The network of significant classes is visualized using Cytoscape (Shannon et al., 2003). Here, pathways that connect one or more clusters indicate specific terms that were significantly associated with multiple clusters.

The metadata in Figure 5 depict the standardized BrainMap terms that corresponded to each cluster; however, the results generated by the forward and reverse inference analyses lacked a sufficient level of detail needed to fully understand the subtle variations as to how experimental designs varied across clusters. Thus, we manually evaluated each experiment comprising a cluster by reviewing its experimental design, with emphasis on identifying key terms distinguishing task types. For example, a set of instructions across four different tasks may all be coded in BrainMap with the same instruction of “Discriminate”, while manual evaluation clarifies the differential nature of the instructed discriminations: (1) “Subjects performed a gender discrimination and indicated whether the presented face was male or female”, (2) “Subjects indicated the color presented behind the face”, (3) “Subjects determined if the two faces were the same or different”, or (4) “Subjects indicated which face they preferred”. For these example tasks, manually extracted terms included “gender”, “color”, “same/different”, and “preferred”, respectively. Using these salient terms identified during manual evaluation of clusters, we generated a secondary Cytoscape visualization of the complex relations between the combinations of task stimulus, response, and instructions and the newly classified brain activation patterns (Figure 6).

Figure 6. Enhanced Terms for Metadata Characterization.

Category assignments based on the rule set were generated for the original set of face experiments in BrainMap. A novel set of non-BrainMap, non-CogPO terms were derived to annotate each experiment with the specific rule that was applied in each instance. These terms were pooled within the clusters of experiments to visualize experimental trends, resulting in a clear summary picture of the distribution of face tasks across sub-classes.

The combined knowledge gained from both the BrainMap-based forward and reverse inference analyses and the manual inspection and identification of salient terms enabled a clear picture to emerge. Below is a summary of the four clusters across face tasks:

• Cluster 1 resulted in convergent activation patterns predominantly localized to the salience network (Seeley et al., 2007), including bilateral anterior insula and anterior cingulate cortices. BrainMap metadata convergence indicated significance correspondence in the forward and reverse analyses within the behavioral domains of “Emotion” and “Cognition.Soma” and the paradigm class of “Film Viewing”. Manual review of experimental design trends indicated that this cluster broadly focused on tasks that engage integrative processing across action, vision, and cognitive control, including spatial or location discrimination, working memory (e.g., Sternberg or n-back tasks involving face stimuli), or facial emotion imitation. Film clips or videos were presented while participants observed and/or performed dynamic movements (e.g., chewing, licking, etc.) or lip-reading (cross-represented in Cluster 4). Cluster 1 also encompassed experiments involving passive viewing of static or dynamic faces in which no discrimination was required. Lastly, Cluster 1 included experiments combining visual face presentation with auditory stimuli in which participants were required to perform pitch discrimination (e.g., discriminate gender of voice). Overall, the cluster consensus indicated that this diverse cluster was associated with Visuospatial Attention and Visuomotor Coordination to Faces.

• Cluster 2 resulted in convergent activation patterns in bilateral amygdala, fusiform face area, and other occipito-temporal areas. BrainMap metadata convergence indicated significance correspondence in the forward and reverse analyses within the behavioral domain of “Perception.Vision.Shape”, and the paradigm class of “Classical Conditioning”. Manual review of experimental design trends indicated that this cluster broadly focused on tasks involving active visual discrimination of faces, including emotional discrimination (e.g., intensity or valence discrimination, emotion matching, the Hariri task (Hariri et al., 2000), same/different discrimination), face/object discrimination, gender discrimination, memory tasks in which stimuli were encoded in a separate scan (unfamiliar vs. familiar, old/new), attending to one stimulus among multiple or divided attention, classical conditioning, and visual imagery. Overall, the cluster consensus indicated that this network was associated with Perception and Recognition of Faces.

• Cluster 3 was dominated by convergent activation in the default mode network (Raichle et al., 2001; Laird et al., 2009), including the medial prefrontal and posterior cingulate cortices. BrainMap metadata convergence indicated significance correspondence in the forward and reverse analyses within the behavioral domain of “Cognition.Attention” and the paradigm class of “Episodic Recall”. Manual review of experimental design trends indicated that this cluster broadly focused on tasks involving emotion induction, in which emotional faces were presented to participants with instructions to recall their own autobiographical memories associated with the presented emotion. Other tasks included discrimination of famous vs. unfamiliar faces, discrimination as to facial attractiveness, pleasant/unpleasant, or other subjective qualities, discriminations based on personal relationships (e.g., friends, mothers, children, peers, family, beloved), and tasks involving participants predicting an emotional response based on the presented facial stimuli. Overall, the cluster consensus indicated that this network was associated with Social Processing and Episodic Recall of Faces.

• Cluster 4 resulted in convergent activation patterns in the inferior middle temporal and parahippocampus gyri, although the convergence observed in this cluster was neither as robust nor as widespread compared to the other clusters. BrainMap metadata convergence indicated significant correspondence in the forward and reverse analyses within the behavioral domain of “Cognition.Language.Speech” and the paradigm class of “Overt Naming”. Manual review of experimental design trends indicated that this cluster was comparatively more narrow in focus, and primarily involved naming of faces, semantic memory, face recognition, and lip-reading (the latter being cross-represented with Cluster 1). Overall, the cluster consensus indicated that this network was associated with Face Naming and Lexical Retrieval.

DISCUSSION

To address the need for improved semantic representation of cognitive paradigms in neuroimaging studies, we developed an evidence-based cognitive neuroinformatics approach using face perception as a proof-of-concept application. Rather than subjectively generate a list of sub-classes to increase the existing classification granularity of face tasks, we systematically identified clusters of brain activation patterns and summary terms driving the segregation across these clusters. Our study was operationalized under the premise that tasks activating similar brain networks should be grouped as functionally similar in a given cognitive schema, while tasks demonstrating differential activation patterns should be classified as functionally distinct. We identified four sub-classes of face tasks: (1) Visuospatial Attention and Visuomotor Coordination to Faces, (2) Perception and Recognition of Faces, (3) Social Processing and Episodic Recall of Faces, and (4) Face Naming and Lexical Retrieval. Importantly, the methodological objectivity of our approach comes from the task groupings, which were derived from the modeled activation patterns and not from the metadata descriptors. Currently, there is no formal classification schema for mental tasks in the face perception literature; therefore, the present results demonstrating a robust four-part grouping are a new finding within this domain. More broadly, this work also demonstrates how data mining techniques can facilitate terminological refinement, thereby supporting continued development and expansion of neuroinformatics resources and tools.

Distributed Networks for Neural Representation of Faces

When perceiving faces, the standard prediction assumes activation of the fusiform face area (Sergent et al., 1992; Haxby et al., 1994; Kanwisher et al., 1997). However, our approach was to move beyond this single node to an expanded focus on multiple, distributed functional networks of regions that are differentially engaged by face task requirements (Ishai et al., 2005). Observed convergence in Cluster 1 across dorsal anterior cingulate and orbital frontoinsular cortices suggests a strong role in salience processing (Seeley et al., 2007) to detect and evaluate facial stimuli (Menon and Uddin, 2010; De Martino et al., 2009) in terms of external relevance to one’s self (Ramasubbu et al., 2011) and anticipated outcomes (Nahum et al., 2011), resulting in the mediation of cognitive-affective interactions (Luo et al., 2014). Facial processing of salient features includes not only analysis of a presented face, but also analysis of the emotional expression, gaze, and visuospatial motion, particularly speech-related lip movements (Bruce and Young, 1986). In addition to regions associated with the salience network, Cluster 1 included convergence in sensorimotor and premotor areas, driven by tasks associated with facial imitation and movement execution; lateral prefrontal cortex, explained by an association with cognitively demanding tasks (e.g., facial n-back task; Bzdok et al., 2012; Luo et al., 2014); and the superior temporal sulcus, which has been linked to assessment of gaze (Hoffman and Haxby, 2000; Grosbras et al., 2005), perceiving speech and non-speech mouth movements (Calvert et al., 1997; Puce et al., 1998; Rolls, 2007), and voice discrimination (Belin et al., 2000; Shah et al., 2001). Overall, Cluster 1 presented as the most heterogeneous grouping of tasks and appeared to represent a conflation across attention (Fox et al., 2005) and visuomotor responses to faces. We interpret the latter as extending somewhat beyond the neural substrates of face perception per se to involve task demands in which the visual face stimulus is a cue to execute a task requiring a visuomotor (e.g., lip-reading) or cognitive response (e.g., n-back).

In contrast to Cluster 1, Clusters 2, 3, and 4 all presented as moderately homogeneous experimental groupings. Cluster 2 corresponded to the canonical face perception network, most notably including activation of the bilateral fusiform face area in the lateral fusiform gyrus (Sergent et al., 1992; Haxby et al., 1994; 1999; Clark et al., 1996; Kanwisher et al., 1997; McCarthy et al., 1997; Halgren et al., 1999; Ishai et al., 1999; Hoffman and Haxby, 2000). Cluster 2 was associated with visual analysis of faces, and included known face-responsive regions in visual extrastriate cortex (beyond the fusiform face area) in the lateral inferior occipital gyri and the superior temporal sulcus (Halgren et al., 1999; Haxby et al., 1999; Hoffman and Haxby, 2000; Kanwisher et al., 1997; Puce et al., 1998). Cluster 3 strongly corresponded to tasks involving biographical episodic memory retrieval and social processing, giving rise to recruitment of the default mode network, including medial prefrontal, precuneus, posterior cingulate cortex, and amygdala (Spreng et al., 2009; Martinelli et al., 2013; Bzdok et al., 2012). Notably, it has previously been shown in a large-scale meta-analysis that regions of the default mode network exhibit increased activation in the context of certain cognitive tasks, such as those involved during social cognition or autobiographical memory (Laird et al., 2009). The distributed network revealed by this cluster has been associated with recognition of familiar faces, specifically famous faces (Leveroni et al., 2000; Gorno Tempini et al, 1998), faces personally known to the participants (Shah et al., 2001), and faces of one’s own children (Ramasubbu et al., 2007). Cluster 3 was also complemented by the striatum, which is known to play a role during social evaluation and decision-making (Kampe et al., 2001; Simon et al., 2014), particularly for valuation-based decisions (Ho et al., 2012). Cluster 4 included the least heterogeneous grouping and was primarily dominated by experiments involving face naming. Recollection of semantic knowledge of famous faces is known to recruit middle temporal cortex, especially the anterior middle temporal gyrus (Gorno Tempini et al., 1998; Brambati et al., 2010). Cluster 4 demonstrated correspondence to lexical knowledge and retrieval, relying on temporal pole, inferotemporal cortex, and middle temporal gyrus (Damasio et al., 1996; Price et al., 2005).

Although the clustering analysis was designed to identify dissociable co-activation patterns, we observed several nodes that were common across clusters. Clusters 2 and 3 revealed common recruitment of the temporoparietal junction, strengthening support for its role linking competing networks for external and internal information processing (Bzdok et al., 2013; Krall et al., In Press). Clusters 1 and 4 revealed convergence in Broca’s area, likely due to common recruitment across lip reading (Paulesu et al., 2003) and lexical retrieval (Damasio et al., 1996). Clusters 1 and 2 both recruited the DLPFC, which aligns with evidence indicating its involvement with perceptual decision-making (Druzgal and D’Esposito, 2001; Heekeren et al., 2006; Cieslik et al., 2013). Clusters 1 and 3 demonstrated overlap in the left posterior insula (Menon and Uddin, 2010). The anterior and posterior insula are known to have different functional and connectional profiles (Taylor et al., 2009; Kurth et al., 2010; Deen et al., 2011; Cauda et al., 2011; 2012); current evidence suggests that the anterior and posterior insula may be recruited during salience processing, whereas the posterior insula and posterior cingulate/precuneus during default mode processing. Thus, the present results suggest that the autonomic arousal role of the posterior insula (Craig, 2002; 2009; Taylor et al., 2009) may involve evaluation of environmental awareness during both salience processing and default mode processing. Lastly, the amygdala was common to Clusters 1, 2, and 3, and the high convergence of this region across face tasks suggests involvement in processing above and beyond emotional expressions (Breiter et al., 1996; Morris et al., 1996; Phillips et al., 1998; Hariri et al., 2000; Bzdok et al., 2011; Shi et al., 2013). Although the superior temporal sulcus was observed across multiple clusters, no topological overlap was observed. Notably, no region was observed across all four clusters.

An Integrated Model of Face Perception

A human’s ability to perceive, recognize, and respond appropriately to faces requires a highly developed range of complex perceptual, cognitive, and emotional processes (Bruce and Young, 1986; Burton et al., 1990), necessitating recruitment across a widely distributed neural network. A previous ALE meta-analysis of explicit and implicit emotional processing of faces identified convergence in visual, limbic, temporoparietal, prefrontal, and subcortical areas, independent of emotional valence (Fusar-Poli et al., 2009). Their face processing network demonstrated strong correspondence to the collective networks observed in the present study, but did not differentiate between varying processing components supporting face perception. In contrast, the hierarchical classification scheme proposed here bears strong similarities to the model of distributed neural systems first proposed by Haxby et al. (2002) and updated by Gobbini and Haxby (2007). This face perception model includes a core system for the visual analysis of faces in the inferior occipital gyrus, lateral fusiform gyrus, and posterior superior temporal sulcus. Also included is an extended system for personal knowledge of faces (i.e., representation of one’s relationship with the presented person, biographical information, and theory of mind processing) in the medial prefrontal, anterior temporal, and posterior cingulate cortices and temporoparietal junction, as well as for emotional response to faces in the amygdala, insula, and striatum. A recent study examined hierarchical clustering of functional connectivity patterns during execution of a face perception task. The identified nodes in their network-level analyses closely matched the brain regions derived from the present BrainMap data, as did the network structure, which differentiated between the temporo-occipital, medial fronto-parietal, and lateral fronto-parietal sub-networks (Zhen et al., 2013). This correspondence is notable, given that their proposed model of systems for facial identification, semantic retrieval, and expression analysis was limited to perception of novel faces with neutral expressions.

With respect to an integrated model of face perception, our results support the neural framework of a core plus extended systems. Here, Cluster 2 represents the core system for visual analysis of faces, while Cluster 3 corresponds to the extended system for personal knowledge, with Cluster 4 representing related lexical retrieval of face names. However, although affective processing is clearly a major component of the face perception model, a separate extended system for emotion was not observed in the present study. Rather, neural substrates underlying emotional responses were found to extend across multiple clusters. Clusters 1, 2, and 3 included varying convergence of activation in the amygdala, insula, and other limbic areas relevant to analysis of the emotional content of faces (Ishai et al., 2005; Gobbini and Haxby, 2007; Schulz et al., 2009; Bzdok et al., 2012). This result is not surprising given that many of the tasks involved visual presentation of emotional faces, and indeed, the Cytoscape representations of BrainMap fixed text terms (Figure 5) demonstrated correspondence between emotional terms and the first three clusters. Thus, we attribute the lack of an observed separate system for emotional responses to both the unique nature of our meta-analytic approach and the resultant coupling of emotional face presentation across numerous tasks. Moreover, while some research has focused on isolating regionally specific emotional states (e.g., sadness, happiness, anger; Fusar-Poli et al., 2009), we observed no major trends across clusters driven by different emotions, although we cannot rule out subtle emotion-dependent modulations of activation (Breiter et al., 1996) or deactivation (Sreenivas et al., 2012) since by nature coordinate-based meta-analysis results do not permit detailed analysis of the BOLD time series.

Beyond these original components of the Gobbini and Haxby model (Gobbini and Haxby, 2007), our results suggest an additional extended system related to attentional processing to determine the salience of presented faces, as indicated by Cluster 1. Overall, we propose an amendment of the Gobbini and Haxby (2007) model of face perception that includes a core visual system, and three extended systems for personal knowledge, emotion, and salience. Critically, we note that core and extended systems rely on the principle of functional integration in that no system operates independently but instead demonstrates varying degrees of relative recruitment strength based on changing task demands. What we have shown is that different systems in the collective network demonstrate functional specialization, with each performing a distinct aspect of processing and responding to face stimuli. We conclude that any given task implemented in the scanner, with its unique combination of stimulus, response, and instructions, may preferentially recruit across these networks, depending on whether processes associated with visual analysis, salience, personal knowledge, or emotion are actively engaged.

Methodological Issues and Limitations

There are several factors that potentially limit the results of the current study. With respect to the clustering method, a range of parameters exist that must be selected prior to analysis during any exploratory data mining approach (Pedrycz, 2005), which can somewhat limit any resultant conclusion. Hierarchical clustering involves selection of both a distance measure and a linkage algorithm; our results relied on clustering using the Pearson’s correlation as the distance measure and the average linkage algorithm. During our preliminary analysis, we evaluated several other distances (e.g., Spearman’s correlation, Hamming, Jaccard, Euclidean, Manhattan) and linkages (e.g., average, complete, single, and weighted) for comparative purposes. Across these, we determined that the Pearson’s and Spearman’s correlations provided reasonable distances and found precedence for their use (Laird et al., 2011b; Liu et al., 2012; Keilholz et al., 2010). In the case of the Spearman’s correlation, a ranked-value approach is used to calculate correlation coefficients and inherently provides more weight to those voxels shared between modeled activation images. Conversely, the Pearson’s approach maximizes the difference between spatial patterns in modeled activation images. While the results across these two distance metrics were largely similar, the Spearman’s correlation resulted in two additional clusters (subsets of Clusters 1 and 3). These additional sub-clusters did not yield dissimilar activation patterns from their parent networks, but rather included many of the same topological features. Furthermore, evaluation of the significant metadata indicated that many of the significant metadata terms were shared. Together, this suggests that no valuable information was gained by choosing the Spearman’s correlation over the Pearson’s correlation; thus, the Pearson’s approach was selected. Similarly, results utilizing different linkage algorithms were examined. The single and complete linkages yielded strongly cascading dendrograms, with little informational value. The average linkage was ultimately selected because of its high cophenetic correlation coefficient (high degree of similarity between cophenetic and pairwise distances) and corresponding clustering structure relative to the other three methods.

Selection of a clustering solution is another analytic stage requiring optimal parameterization and in some cases necessitates utilization of multiple information criteria (Kelly et al., 2010; Chang et al., 2012; Clos et al., 2013). Here, our ultimate goal was to identify a subset of classes for taxonomic and ontological purposes. Thus, we were predisposed towards simple solutions yielding 3–6 clusters, as opposed to solutions yielding more than 10 clusters, as this would have been a considerable number from a semantic representation perspective. Thus, our approach in using experimental separation density and cophenetic distances was more simplistic than what has been implemented in prior studies, but yielded robust results of the dimension we intended to target. However, we note that further decomposition into an ensuing level of classification is possible by additional investigation of the dendrogram structure to yield increased granularity and precision when defining cognitive paradigms.

Ontological Expansion

This study was motivated by a need for new strategies for developing a paradigm classification schema. Although ontological development is often characterized as a somewhat esoteric branch of biomedical research, the fundamental rationale behind this field is to develop improved methods for how scientific work is reported, discussed, and mined. In their seminal work, Bruce and Young (1986) stated their goal was to develop “a set of terms for understanding and discussing for how we recognize familiar faces, and the relationship between recognition and other aspects of face processing”. Here, our goal was to advance this ontological emphasis a step further and enhance the formal schema delineating variations of face processing tasks. The Cognitive Paradigm Ontology (CogPO) is a domain ontology of cognitive paradigms utilized in functional neuroimaging experiments (Turner and Laird, 2012). CogPO was originally developed using paradigm terms from the BrainMap taxonomy, but has since been expanded to include other paradigms, such as those also represented by the Cognitive Atlas (Poldrack et al., 2011). Although CogPO terms have been useful in providing functional interpretation of intrinsic connectivity networks (Laird et al., 2011b) and in automated multi-label annotation and classification of neuroimaging experiments (Turner et al., 2013), some of the included paradigm classes are fairly coarse and require decomposition into sub-classes. We observed that functional interpretation of the clusters in the present study was limited when utilizing existing BrainMap terms already represented in CogPO. To obtain a higher degree of clarity, manual review of experiments allowed the derivation of more salient terms and enhanced our overall results. Importantly, we note that the results presented here add depth and granularity to the CogPO ontology, as well as to the BrainMap taxonomy. However, this approach represents a semi-automated, not fully automated approach. While hierarchical clustering provided an objective and efficient means to aggregate experimental results into distinct groups, manual review required a moderate amount of labor. Future work will involve evaluation of text mining algorithms to develop a more automated approach for identifying and extracting key terms to enable functional decoding of experimental groupings.

Conclusions

We investigated the feasibility of an objective and evidence-based approach for identification of paradigm sub-classes used across the functional neuroimaging literature in the domain of face perception. The aim of our study was to highlight the variability in tasks and classify them into distinct groupings based on modeled activation patterns rather than task descriptions. We concluded four subclasses of face tasks: (1) Visuospatial Attention and Visuomotor Coordination to Faces, (2) Perception and Recognition of Faces, (3) Social Processing and Episodic Recall of Faces, and (4) Face Naming and Lexical Retrieval. Interpretation of these results supports an amendment of the distributed model of face perception proposed by Gobbini and Haxby (2007) to include a core system for visual analysis and extended systems for personal information, emotion, and salience processing. More broadly, our results demonstrate that large-scale meta-analysis of the BrainMap database allows theoretical mental models to be tested by probing the range of behavioral manipulations across tasks. In this way, neuroimaging meta-analysis and data mining are able to establish a link between theory and experimentalism to a degree that is more powerful than what can be accomplished with results achieved in a single empirical study. Our results demonstrate that the range of tasks involving presentation of human faces is complex, and the underlying neural architecture that collectively supports the execution of these tasks is similarly broad and diverse.