Decoding the Neural Representation of Self and Person Knowledge with Multivariate Pattern Analysis and Data Driven Approaches

Abstract

Multivariate pattern analysis and data driven approaches to understanding how the human brain encodes sensory information and higher level conceptual knowledge have become increasingly dominant in visual and cognitive neuroscience, however it is only in recent years that these methods have been applied to the domain of social information processing. This review examines recent research in the field of social cognitive neuroscience focusing on how multivariate pattern analysis (e.g., pattern classification, representational similarity analysis) and data-driven methods (.e.g, reverse correlation, intersubject correlation) have been used to decode and characterize high-level information about the self, other persons and social groups. We begin with a review of what is known about how self-referential processing and person perception are represented in the medial prefrontal cortex based on conventional activation-based neuroimaging approaches. This is followed by a non-technical overview of current multivariate pattern-based and data-driven neuroimaging methods designed to characterize and/or decode neural representations. The remainder of the review focuses on examining how these methods have been applied to the topic of self, person perception and the perception of social groups. Throughout, we highlight recent trends (e.g., analysis of social networks, decoding race and social groups, the use of naturalistic stimuli) and discuss several theoretical challenges that arise from the application of these new methods to the question of how the brain represents knowledge about the self and others.

Graphic/Visual Abstract and Caption:

Review of recent multivariate pattern analysis and data-driven approaches to decoding and characterizing the brain’s representation of self and person knowledge.

Introduction:

The medial prefrontal cortex (MPFC) is perhaps the most widely studied brain region in the field of social cognitive neuroscience. It has been implicated in a variety of social and emotional processes, from mentalizing and impression formation through to self-knowledge and empathy for others. One enduring theme in this research is that the MPFC is important for learning and retrieving information about the self and others (Amodio & Frith, 2006; Mitchell, 2008; Wagner, Haxby, & Heatherton, 2012). Although it took several years to gather steam, some of the earliest human neuroimaging work focused on how humans understand and infer the mental states of others. Using positron emission tomography (PET), these early studies demonstrated that the MPFC was activated when people infer the covert mental states of others (Fletcher et al., 1995) or when they judge whether familiar historical figures would have knowledge of modern everyday objects (Goel, Grafman, Sadato, & Hallett, 1995). Contemporaneous studies focusing on self-knowledge also demonstrated a role for the MPFC in simulating personal autobiographical memories (Fink et al., 1996) or in evaluating whether or not a trait described the self (Craik et al., 1999). As researchers moved to using functional magnetic resonance imaging (fMRI) to study human cognition, the greater flexibility of this technology allowed for a wider array of tasks and designs to become amenable to neuroimaging experiments. These developments allowed researchers to plumb more complex social cognitive phenomena, such as the self-referential memory effect, whereby individuals demonstrate improved recall for information when judged for its self-relevance over its semantic meaning (Rogers, Kuiper, & Kirker, 1977). Functional neuroimaging studies of this effect demonstrated that a region of the ventral MPFC (vMPFC) was recruited when evaluating whether psychological traits described the self vs. another person (Kelley et al., 2002) and, in an fMRI study optimized for memory recognition, activity in this region predicted both the self-relevance of traits as well as which traits were subsequently remembered in a surprise recognition test (Mitchell, Macrae, & Banaji, 2004). Based on these, and other findings, the notion that the vMPFC was engaged in self-related processes became virtual orthodoxy.

In contrast to the vMPFC, a more dorsal region of the MPFC (dMPFC) was often found to be involved in studies where people are tasked with inferring the mental states of other persons (Gallagher & Frith, 2003) or when they are required to form impressions of others from descriptions of their behavior (Amodio & Frith, 2006; Mitchell et al., 2004; Mitchell, Neil Macrae, & Banaji, 2005). Based on over a decade’s worth of such findings, meta-analytic studies suggested that there existed a ventral to dorsal gradient in the MPFC, such that ventral regions were more involved in self-related processes whereas dorsal regions were more typically recruited when processing information about other persons (Denny, Kober, Wager, & Ochsner, 2012; Wagner et al., 2012). Evidence from neuropsychological studies of patients with damage to the ventral portion of the MPFC appeared to lend support to this ventral-dorsal distinction. For example, studies of patients with vMPFC damage coinciding with the region of vMPFC in (Kelley et al., 2002) that was more active during self than other-referential judgements showed that the self-referential memory advantage in these patients was completely abolished relative to control patients and healthy participants (Philippi, Duff, Denburg, Tranel, & Rudrauf, 2012). In addition, patients with vMPFC damage typically maintain preserved social knowledge of other persons despite demonstrating a deficit in retrieving knowledge about themselves (Marquine et al., 2016). Studies of patients with damage to the dorsal portion of the MPFC are not as common as those with damage to ventral portions, however research using transcranial magnetic stimulation to induce temporary inactivation of the dMPFC suggest that this region is critical for forming impressions of others (Ferrari et al., 2016).

With few exceptions, most of the research on the role of the MPFC in social cognition is based on traditional activation-based neuroimaging approaches. These studies attempt to isolate psychological processes following the logic of cognitive subtraction whereby activation associated with a cognitive task or sensory feature of interest (e.g., memory, faces) has subtracted from it activation associated with a condition in which that process or feature is absent (Friston et al., 1996; Petersen, Fox, Posner, Mintun, & Raichle, 1988). Studies of this type rely on single voxel or regional differences in activation across conditions to infer the involvement of a brain region in a cognitive process and represent by far the most common type of experimental design and inference in fMRI research. As a result, many of the inferences that have been made regarding the role of the brain in self and person perception focus on whether a particular task or stimulus globally activates a brain region allowing researchers to make inferences about the social cognitive processes associated with that area (i.e., mentalizing, self-referential processing, emotion, etc.). However, with the advent of newer computational and machine learning based neuroimaging methods aimed at characterizing, predicting, or classifying neural representations based on their pattern of activity (e.g., Multivariate Pattern Analysis or MVPA) there are now readily available tools that permit social cognitive neuroscientists to make inferences regarding the informational content, representational structure, or computational transformations that are performed by different brain regions during social cognition. It is historically interesting, then, to note that a subset of these novel methods have long been in the toolbox of social psychologists who previously used multidimensional scaling and clustering of multivariate data in the 1970s and 1980s to investigate the cognitive representation of self and person knowledge (Forgas, 1976; Jones, 1983; Rosenberg, Nelson, & Vivekananthan, 1968).

Several years ago, in a review on the role of the MPFC in self and other-referential processes, Wagner, Haxby & Heatherton (Wagner et al., 2012) highlighted the emergence of MVPA methods to study neural representations in visual and cognitive neuroscience but noted that there was, at the time, a paucity of research in social cognitive neuroscience utilizing this approach. Since then, the number of researchers applying MVPA and related methods to the study of social cognition has steadily increased and we believe that there is now a sufficient mass of findings to warrant a review of their contribution to our understanding of how the brain represents the self and others. We begin this review with a non-technical overview of the various MVPA and data-driven methods that have been applied to the study of self and other neural representations. Following this, we review research in the last several years that utilizes these methods to investigate how midline structures, such as the MFPC, are involved in representing information related to the self as well as social knowledge of familiar and unfamiliar others (e.g., psychological traits, identity, mental states). In particular, we focus on studies demonstrating how activity in these regions can be used to decode features of our representation of other persons, such as their familiarity, personality, or social group. Finally, we discuss several current trends (e.g., social networks, naturalistic stimuli) and theoretical challenges that arise from the application of these new methods to decoding the brain’s representation of self and person knowledge.

A brief overview of multivariate and data driven approaches to studying neural representations

Since first being introduced to neuroimaging in the early 2000s (Haxby et al., 2001), multivariate pattern analysis (MVPA) has emerged as a critically important tool for understanding how information is represented in the brain (Cohen et al., 2017; Haxby, Connolly, & Guntupalli, 2014; Kriegeskorte, 2011; Naselaris, Kay, Nishimoto, & Gallant, 2011). Unlike conventional univariate approaches that relate psychological processes to the global level of activity within individual brain voxels or regions, MVPA methods focus on the fine-grained patterns of brain activity within a region or system that are represented across multiple voxels simultaneously. Many of the novel findings relating to how the brain encodes information about the self and others in social cognitive neuroscience have been driven in part by these advances in methodology and the myriad ways they allow researchers to characterize the function and computations of a brain region. These range from MVPA techniques designed to classify mental states or characterize the representational geometry of the information encoded in a brain region (e.g., representational similarity analysis) to data driven approaches such as inter-subject correlation and reverse correlation analysis. The increasing prevalence of MVPA methods has been aided by the availability of open, comprehensive, and flexible software package. Tools such as pyMVPA (Hanke et al., 2009) and nilearn (Abraham et al., 2014) implemented in the Python program language as well as CoSMoMVPA (Oosterhof, Connolly, & Haxby, 2016), and the Princeton MVPA Toolbox (Detre et al. 2006) implemented in Matlab, have greatly facilitated the uptake of these methods and have helped contribute to the standardization of practices. In this section we provide a brief overview of MVPA approaches as well as several data-driven but not necessarily multivariate methods (e.g., intersubject correlation, reverse correlation) in order to orient readers to the different types of inferences that they enable.

Multivariate Pattern Classification

Though MVPA can encompass many different types of multivariate pattern analyses, the most common application of MVPA methods remains multivariate pattern classification. In this analysis, a pattern classifier algorithm is ‘trained’ on a subset of the available neuroimaging data and learns to discriminate between different brain states (e.g., different conditions of a task, or different varieties of stimulus). Typically the classifier is linear and can range from standard regression based approaches such as linear discriminant analysis or logistic regression through to more complex algorithms from the field of machine learning such as support vector machines. Studies employing multivariate pattern classification typically aim to test whether patterns of brain activation can reliably classify (i.e. dissociate) two or more mental states or stimulus types, an approach that is often referred to as neural decoding. One issue with multivariate pattern classifiers is that their increased flexibility makes them vulnerable to noise, that is to say they can learn to exploit subtle differences between conditions that may be unrelated to the brain’s actual neural representation but instead reflects differences in the noise components of the signal. For this reason, it is common practice in machine learning and in MVP analysis to avoid overfitting by training the classifier on subsets of the data with subsequent evaluation of classifier performance being conducted on left-out independent portions of the data.

The first study to employ MVP analysis to neuroimaging data remains a paradigmatic example of this approach. In this study, Haxby and colleagues set out to investigate the representation of face and object categories in ventral temporal cortex (Haxby et al., 2001). Here, participants viewed images of eight different visual categories including faces, cats, and several varieties of man-made objects. The data in ventral temporal cortex associated with these categories were split into two independent datasets (based on odd and even runs of the experiment) and the correlation of voxel response patterns for each category were correlated to those of the corresponding and non-corresponding visual categories in the left out data fold. For each brain response pattern, it’s visual object category was classified according to the category of their highest correlation in the left-out data fold. This approach is referred to as a split-half correlation-based classifier (Haxby, 2012). Overall, the results of this study demonstrated the novel finding that the ventral-temporal cortex contained a distributed representation of faces and object categories and that these categories could be decoded from response patterns even when known category selective brain regions (i.e., the fusiform face area) were excluded from the analysis.

Correlation-based pattern classification analyses, like the one describe above, are still used in many studies. However, today it is much more common to use linear machine learning classifiers such as support vector machines, k-nearest neighbor and naïve Bayes. For each of these methods, researchers can choose to use brain responses from a given region of interest (e.g., anatomically defined regions, functional localizers) or to test local response patterns across the cortex using so-called searchlight information mapping (Kriegeskorte, Goebel, & Bandettini, 2006) whereby a multi-voxel ROI is iteratively moved throughout the brain and submitted to an MVP analysis.

Representational similarity analysis

As with other MVPA approaches, representational similarity analysis (RSA) interprets the neural response patterns associated with each experimental condition as the neural representation of a specific mental state (Kriegeskorte, Mur, & Bandettini, 2008). However, whereas neural decoding approaches based on multivariate pattern classification focus on discriminating between different experimental conditions based on neural response patterns, RSA focuses instead on the similarity of neural response patterns between pairs of conditions. One of the principle uses of RSA is to characterize the representational space of a set of neural response patterns, either by data-driven approaches that help characterize the geometry of this space (e.g., multidimensional scaling or clustering methods) or by directly comparing the neural representational space to spaces predicted from models based on stimulus features (Khaligh-Razavi & Kriegeskorte, 2014) or conceptual similarity (Connolly et al., 2012; Stolier & Freeman, 2016).

An example of how this method can be used to compare neural representational spaces with higher-order conceptual spaces can be found in a study by Parkinson and colleagues (Parkinson, Liu, & Wheatley, 2014) that investigated whether the brain uses a common metric for representing egocentric distance in both a literal sense (i.e., spatial distance from self) and a figurative one (i.e., temporal distance, social distance). In this study, participants made judgements as to the relative distance of two exemplars that varied along one of the three distance dimensions. More specifically, participants saw: images of objects that were spatially either closer or further according to the first-person perspective of the observer, photographs of personally familiar friends and acquaintances that varied in social distance from the participant, and verbal references to time that varied in terms of how distant in the future they were (e.g., “years from now” versus “in a few seconds”). Comparing the neural similarity structure to one generated based on participant’s subjective ratings of distance revealed a region of the right inferior parietal cortex that appeared to code for distance in all three dimensions (i.e., physical, temporal, social). In contrast to the pattern classification approach described above that seeks to find reliable patterns of activity that discriminate between mental states, the use of RSA here allows the experimenter to test whether the ensemble of mental states as a whole conform to a theoretical model of neural similarity (here defined based on the participant’s own subjective evaluations of distance) thus encapsulating multiple hypotheses concerning the similarity/dissimilarity of different mental states in a single test. This novel finding suggests that our understanding of spatial metaphors of distance (e.g., people who are “closer” to us) may be rooted in a common neural representation of literal distance from ourselves to objects in our environment. This study serves as an example of the utility of RSA as a means of abstracting away from low-level differences in both stimuli and modality (e.g., neural, sensory, cognitive) in order to test whether the higher-order conceptual and category structures, to which we have conscious access, are reflected in the neural activity patterns of different brain regions.

In addition to its ability to compare neural and cognitive representational spaces, RSA also allows cross-modal comparisons between imaging technologies and even across species (Kriegeskorte, Mur, & Bandettini, 2008). By abstracting away from idiosyncratic differences in the spatial (i.e., voxel) layout of neural representations, RSA allows for analysis and comparisons at the level of the representations themselves, thereby enabling comparisons between data collected using fMRI and electrophysiological recordings in different species. For example, using RSA, Kriegeskorte and colleagues (Kriegeskorte, Mur, Ruff, et al., 2008) showed that object representation in the inferior temporal cortex follow a common representational structure for monkeys and humans. Specifically, the representational spaces of both species demonstrated large category clusters corresponding to animate and inanimate objects, and, within animate objects, to faces and bodies, ultimately suggesting similarities in the neural code across primate species.

Inter-subject correlation of neural activity

In contrast to the multivariate pattern-based approaches outlined above, inter-subject correlation analysis (ISC) typically focuses on the univariate similarity of voxel timecourses between different participants. Whereas other methods typically, though not exclusively (Haxby et al., 2011), implement a different classifier or RSA model per participant, ISC analysis instead focuses on the similarity between participants. One advantage of this approach is that it is model-free, requiring no prior knowledge of the shape of the haemodynamic response function that gives rise to the signal measured by fMRI, nor does it require that the participants’ task be explicitly modelled. For this reason, it is especially well suited for studies of continuous viewing of naturalistic stimuli (e.g., movies, music). Experiments that employ ISC to investigate neural synchrony across different individuals are engaged in the fundamental question of whether neuronal populations across people process and encode sensory events in a similar fashion.

The first neuroimaging study to use this method examined neural synchrony as participants viewed the first thirty minutes of the movie The Good, the Bad and the Ugly (Hasson, 2004). For each subject, at each voxel, the time course of activity during movie viewing was correlated with that of another subject in the sample at the same location and this process was repeated for every possible pair of subjects. Taking the average ISC across all subject pairs revealed that approximately 30% of the cortical surface showed significant neural synchrony during movie viewing. Crucially, the implicated brain regions were not merely limited to primary auditory and visual cortices, but also included several higher-order association areas. This finding suggests that participants had similar subjective experiences of movie events (i.e., emotional content, meaning, attentional deployment) and that these operate at roughly the same timescale. In line with this last point, more recent work has begun to use this method to investigate whether there are meaningful differences in neural synchrony between groups of individuals that may reflect subjective experiences of a stimulus (e.g., (Ames, Honey, Chow, Todorov, & Hasson, 2015; Yeshurun et al., 2017). Finally, it is worth noting that although this method is essentially univariate and therefore blind to the potential information contained in patterns of activity over time (see (Adolphs, Nummenmaa, Todorov, & Haxby, 2016) there has, however, been a recent effort to extend this approach to measure the multivariate pattern similarity of voxel timecourses during movie viewing using canonical correlation analysis (Bilenko & Gallant, 2016; Kriegeskorte, 2015).

Reverse Correlation Analysis

As with intersubject correlation analysis, reverse correlation analysis is an approach aimed at characterizing neural responses during continuous viewing of naturalistic stimuli. In comparison to most conventional fMRI approaches, reverse correlation analysis inverts the typical analysis framework by using magnitude differences in the timecourse of a voxel or region of interest to identify those features of the stimulus that appear to drive neural activity. This method was first implemented in Hasson and colleagues (2004) investigating the timecourse of neural activity during movie viewing in three regions previously associated with strong category responses to faces, buildings and manual actions. Statistical peaks in the timecourse of activity in these regions were identified and used to extract those segments of the film that occured at the time of those peaks. Visualization of the movie frames associated with these peaks revealed that the putative category preference of these regions was maintained during natural viewing of a movie. For example, the activity peaks in the timecourse of the lateral fusiform gyrus, a region involved in face processing, was associated with movie frames in which there was a strong percept of a face (i.e., close-ups of actors’ faces). Since this initial study, reverse correlation analysis has been used to demonstrate the role of gestures in naturalistic language comprehension (Skipper, Goldin-Meadow, Nusbaum, & Small, 2009) and to highlight the role of the medial prefrontal cortex in processing social interactions during movie viewing (Wagner, Kelley, Haxby, & Heatherton, 2016). A potential limitation of this method is that the underlying signal used to derive the statistical peaks in the timecourse that are then correlated with movie features may itself be influenced by multiple unrelated components (e.g., noise, arousal, etc.) that might bias the method towards certain scenes over others. For this reason, removing potentially unrelated signal components, such as removing global signal changes estimated from task-relevant regions is critical (Hasson, Nir, Levy, Fuhrmann, & Malach, 2004; Wagner et al., 2016).

Repetition Suppression

Repetition suppression (RS), also known as fMRI-adaptation, is a method that capitalizes on the observation that neural responses are reduced or “suppressed” following the repetition of an identical stimulus (Grill-Spector, Henson, & Martin, 2006; Grill-Spector & Malach, 2001). This effect, first identified during single-cell electrophysiological recordings in animals, has been observed at multiple spatial scales from individual cortical neurons to the hemodynamic responses measured with fMRI. One of the advantages of this method is that it is believed to capture the functional properties of neuronal populations at subvoxel resolutions (Grill-Spector et al., 1999). With respect to higher-order cognition, this method is particularly useful in identifying neural populations involved in representing the conceptual, rather than lower-level, features of a stimulus (Barron, Garvert, & Behrens, 2016). For example, if RS is observed following presentation of two stimuli in succession that are equivalent in their conceptual meaning, but differ with respect to lower-level stimulus features, this implies that the two stimuli share some common representation of features in the observed voxel or brain region. As an example of the utility of this approach, studies employing RS provided some of the best evidence for a human mirror neuron system in the parietal lobe (Chong, Cunnington, Williams, Kanwisher, & Mattingley, 2008) and frontal gyrus (Kilner, Neal, Weiskopf, Friston, & Frith, 2009). Although prior work using univariate activation-based approaches had identified a potential homologue of the macaque mirror neurons (Gallese, Fadiga, Fogassi, & Rizzolatti, 1996), this finding rested primarily on the demonstration of overlapping activity clusters when humans observe or perform an action during PET (Rizzolatti et al. 1996; Decety et al. 1997) and fMRI (Buccino et al., 2001; Gazzola & Keysers, 2009). However, the observation of overlapping activity between two conditions does not allow for the inference that the same individual neurons are involved in both processes, an inference that is necessary to demonstrate that humans possess an analog of the monkey mirror neurons (Dinstein et al. 2008; Kilner and Lemon 2013). The advantage of RS analysis in studying this phenomenon is that the RS effect is thought to originate from the same neural populations in a voxel, thus the observation of repetition suppression following action observation and action performance implies that the same neurons are tuned to both conditions thereby demonstrating the necessary properties of mirror neurons and thus serving as strong evidence that humans possess mirror neurons.

Over the years, the phenomena of repetition suppression within the domain of fMRI has been challenged on the grounds that the RS effect is not necessarily indicative of the response tuning properties of local voxels but instead may be an artifact of attention or of participants’ stimulus expectations (Summerfield et al. 2008). However, it has been argued that these putative artifacts can be avoided with careful experimental design (Larsson & Smith, 2012). It is also worth noting that both MVPA methods and repetition suppression allow inferences regarding the response tuning function of a voxel or population of voxels, however in practice they may be sensitive to different aspects of neural representations. For instance, MVPA and RS have been used to show that regions of the inferior temporal cortex are sensitive to the identity and category of outdoor visual scenes, but MVPA appears to be more sensitive to categorical differences in the content of these scenes (Epstein & Morgan, 2012). On the basis of finding such as these, it has been suggested that RS might reflect the dynamic aspects of neural processes that modulate or exist in tandem with the underlying neural code that MVPA appears better suited to capture (Epstein & Morgan, 2012).

Decoding the neural representation of self knowledge

One of the most robust findings in social cognitive neuroscience is the identification of cortical midline structures, particularly the vMPFC, that appear to be highly sensitive to tasks involving some degree of self-evaluation or self-referential processing. Across dozens of studies, researchers have demonstrated that self-referential processing shows greater activation in the vMPFC when compared to evaluating the traits and characteristics of other persons, including thinking about famous others (Kelley et al., 2002) as well as friends (Heatherton et al., 2006). Other studies have also shown that, when thinking about other persons, vMPFC appears to be sensitive to the psychological closeness (Krienen, Tu, & Buckner, 2010) and similarity (Mitchell, Macrae, & Banaji, 2006) of these relationships. Building off of these early studies, later work aimed to further refine our understanding of how the brain supports the representation of self-relevant information. For example, Moran and colleagues (2006) found that vMPFC activity during self-referential thought appeared to track the self-relevance of information whereas other regions, more often implicated in emotion, were sensitive to the valence of that information (Moran, Macrae, Heatherton, Wyland, & Kelley, 2006). In addition, research showed that the vMPFC responds automatically to self-relevant information even in the absence of explicit instructions to engage in self-evaluation, such as when personal information (names, address) appears during an ostensibly unrelated task (Moran, Heatherton, & Kelley, 2009). More recently, Tamir & Mitchell (2012) examined the consequences of sharing self-relevant information (i.e., self-disclosure) and demonstrated that the vMPFC was more active when people generated information to disclose about themselves compared to information for others, but did not differentiate between sharing those disclosures vs. keeping them private. Instead, reward-related brain regions (such as the ventral striatum) showed increased activity when these self-disclosures were shared compared to when they were kept private, suggesting that self-disclosure is experienced as rewarding (Tamir & Mitchell, 2012). Another relatively recent area of research involves the investigation of neuropsychological patients with damage to the vMPFC. For instance, patients with damage to the vMPFC in the same location implicated in self-referential processing (e.g., the coordinates from Kelley et al. 2002) completely fail to demonstrate the normal self-referential memory advantage despite otherwise intact memory performance (Philippi et al., 2012). Similarly, in a recent case study, vMPFC damage was associated with reduced insight into the patient’s own traits despite intact knowledge of the traits of others (Marquine et al., 2016).

Taken together, this research suggests a critical role for the vMPFC in representing the psychological features of the self. However, the basic question still remains: What is it about the self that is being represented within the vMPFC? Based on the research discussed above, we still have limited understanding of what computations and what types of information are being operated on within the vMPFC that make it so fundamental to self-referential cognition. This issue is further compounded by the fact that a number of other domains (i.e., reward value, affect regulation) also appear to involve if not this same region then one that is at least partially overlapping (Acikalin, Gorgolewski, & Poldrack, 2017; Delgado et al., 2016). For instance, a number of studies and meta-analyses have attributed a role for the vMPFC in representing subjective reward value during decision making (Clithero & Rangel, 2014; Grabenhorst & Rolls, 2011; Rushworth, Noonan, Boorman, Walton, & Behrens, 2011). Recently, however, researchers have started to make progress in understanding the precise role of the vMPFC in self-referential thought using MVPA. Perhaps in no place has work in this area been more active than in understanding the intersection of self-related processes with positive affect and subjective value.

Over the last several years, researchers have noted the shared involvement of the vMPFC in both self-referential processing and in representing subjective value during decision making and have hypothesized that these two concepts may be inextricably linked (D’Argembeau, 2013; Northoff & Hayes, 2011) (Figure 1a). Although some work has been done to test this idea using conventional neuroimaging approaches (Enzi, de Greck, Prösch, Tempelmann, & Northoff, 2009), only within the last few years have researchers started to use MVPA approaches to study the relationship between subjective value and self in the vMPFC. In one study by Chavez, Heatherton, & Wagner (Chavez, Heatherton, & Wagner, 2017), participants were asked to complete two separate tasks: the first measures neural responses to positively and negatively valenced emotional scenes and the second task was a standard self-referential processing task e.g., (Kelley et al., 2002). To examine the relationship between self-referential processing and positive valence, the authors used a cross-domain multivariate pattern classification procedure in which a classifier trained in one domain (valence) was used to discriminate between conditions in another domain (self vs. other referential processing). Specifically, they trained a support vector machine classifier to distinguish positively and negatively valenced images and applied this classifier to data as participants evaluated themselves or a close friend. Their results showed that a region of the vMPFC trained on positive and negative valence also discriminated between thinking about the self and others (Figure 1b), thus providing initial evidence that positive affect and self-referential thought share a common underlying neural representation reflected in multivariate patterns of brain activity in the vMPFC (Chavez et al., 2017).

Figure 1.

(a) Overlap of reward and self-referential processing is largest in the ventral medial prefrontal cortex based on text-based meta-analysis in Neurosynth (Yarkoni, Poldrack, Nichols, Van Essen, & Wager, 2011). Red areas represent results from 671 studies using the term ‘reward’, blue areas represent results from 127 studies using the term ‘self referential’, and the the yellow areas represent the overlap between the two. Each map was generated using the forward inference procedure and thresholded at a z-value of 5 to highlight areas that showed a high probability of being activated across studies given the topic. (b) A cluster in the vMPFC where a cross-domain classifier trained on positive and negatively valenced material was able to decode self vs. other referential thought (Chavez, Heatherton & Wagner, 2017). (c) Regions of the vMPFC where a classifier trained on high and low reward-value could reliably decode self vs. other referential thought (Yankouskaya et al., 2017).

Other studies have found similar effects using different task paradigms. For instance, a recent study had participants complete a perceptual matching task to learn to associate different geometric shapes with either their own name, a friend’s name, a high-value reward, or a low-value reward (Yankouskaya et al., 2017). While in the scanner, participants were then presented with each shape and asked to respond with which category each shape was associated with to measure neural responses while thinking of each category. Using multivariate patterns of activity within multiple vMPFC ROIs arranged along an anterior to posterior axis, they trained a pattern classifier to dissociate high-reward from low-reward trials and tested the accuracy of this classifier on the self/other trials and vice versa. The largest classification accuracy was found within the anterior portion of the vMPFC (similar to that of Chavez(Chavez et al., 2017) with cross-modal accuracy decreasing to chance levels in more posterior ROIs (Figure 1c). More recently, another group of researchers used support vector regression to train a model to predict measures of self-esteem based on a multivariate brain activation while participants viewed pictures of their own face (Izuma, Kennedy, Fitzjohn, Sedikides, & Shibata, 2018). They found that activation patterns within areas associated with reward showed the greatest accuracy in predicting both implicit and explicit self-esteem, particularly within the vMPFC. Moreover, when excluding areas that have been implicated in reward related processing, the accuracy of the model showed a marked drop in performance. Taken together, these studies suggest that one role of the vMPFC in self-referential processing may be to compute the valuation and affective information that seem to be inextricable aspects of self-reflection.

In addition to the work on the intersection between self and affective processes, there is also a growing interest in more directly testing the relationship between the representations of self-knowledge and person-knowledge in the brain. Prior research using activation-based approaches had suggested that when individuals consider the traits of similar (Mitchell et al. 2006; Mitchell et al. 2005) they appear to use the same area of vMPFC as is involved in self-referential processing. A subsequent study using repetition suppression (see above) offered further evidence of this claim by demonstrating that the same neuronal populations involved in self-referential processing also show a repetition suppression effect when considering the traits of similar, but not dissimilar, others (Jenkins et al. 2008). Together, the results of these studies led to the suggestion that individuals use their own self-knowledge (e.g., traits, preferences and characteristics) as a model to enable inferences about similar others. More recently, a study by our group, examined this issue from a different perspective, specifically asking whether a individual’s neural representation of their own self is similar to the neural representation in the brains of friends, when these friends are reflecting on that individual’s traits. Or to put it differently, is the pattern of neural activity in the vMPFC when a person is thinking of themselves recapitulated in the brains of friends when they think of that same individual (Chavez & Wagner, under review). In this study, a group of close knit peers participated i in a round-robin trait evaluation task in which each subject in the study made trait judgments about their self, and for every other target in their social group. Thus, each participant was both perceiver and target for every one of their peers. . By calculating the neural similarity between each individual’s response patterns for self and those of their peers when thinking of that participant we computed a neural self-other similarity metric for each individual and examined if this relationship was not only greater than chance, but critically, if this relationship was stronger than when peers were thinking of different targets in the social group. Findings showed that the pattern of brain activity when individuals we’re thinking of themselves was similar to that of their peers thinking of that same individual and across the sample this neural similarity was greater than that computed for when peers were thinking of other persons in the social network, suggesting that these results are specific to each individual and not just the result of thinking about other friends in in general. These results provide the first piece of evidence that neural representation of our sense of self may be recapitulated in the brains of our friends when they are thinking of us, and that these processes are reflected in multivariate response patterns within the vMPFC.

The studies above highlight how MVPA methods have further elucidated our understanding of how self-knowledge is represented in the brain. However, there remains much more work to be done to provide a more comprehensive understanding of how cortical midline structures compute, encode and represent information about the self. For instance, it is clear that processes related to memory, interoception, motivation, and others are also related to self-referential cognition, but there are no MVPA studies explicitly testing the links between these processes. Moreover, although the vMPFC is the region found to be most consistently recruited during self-referential processing, the posterior cingulate cortex (PCC) also shows reliable activation when reflecting on the self. Though there are some theories to describe the role of the PCC in self-referential cognition using univariate methods (Brewer, Garrison, & Whitfield-Gabrieli, 2013), to our knowledge, no studies have been designed specifically to decode what form of information is being represented within the PCC during self-referential cognition. These areas are ripe for study and have the potential to further illuminate how the brain encodes the cognitive and affective information that forms the basis of its representation of self knowledge.

Decoding the neural representation of person knowledge

Previous research on person perception and person knowledge has revealed a network of brain areas implicated in forming impressions and thinking about the mental states of other persons. These typically include the temporoparietal junction (TPJ), posterior cingulate cortex (PCC), precuneus (PC), and, as will be our main focus, the medial prefrontal cortex (MPFC). With respect to the MPFC, findings from multiple lines of inquiry have suggested the presence of a functional gradient within this brain region such that more ventral areas are more involved in self-referential processes and more dorsal areas in forming impressions of others and making inferences as to the mental states and psychological traits of others (Denny et al., 2012; Mitchell, 2008; Van Overwalle, 2009; Wagner et al., 2012). Examples of this research have shown that the dMPFC is more active when people form impressions of others based on behavioral descriptions of their behavior (Mitchell et al., 2005) and character relative to impressions formed merely based on physical characteristics (Moran, Lee, & Gabrieli, 2011). Moreover, the dMPFC is also involved in updating impressions of persons, especially in situations where their behavior does not match expectations or conflicts with prior impressions (e.g., (Cloutier, Gabrieli, O’Young, & Ambady, 2011; Mende-Siedlecki, Cai, & Todorov, 2013).

Recently, researchers have turned to MVPA and related methods to better understand the nature of the information encoded in the MPFC during person perception. Central to much of the prior research on the role of the dMFPC in social cognition is the notion that this region is involved in trait and mental state inference. In one of the first studies to directly attempt to test whether trait information is encoded in the MPFC, Ma and colleagues (Ma et al., 2014) used repetition suppression (see above for an overview of the method) to examine the role of the MPFC in representing spontaneously inferred traits from verbal descriptions of behavior. The principal finding of this study was that the MPFC showed a repetition suppression effect when two different behavioral descriptions that implied the same or a similar trait followed each other. This finding extends the previous research based on more conventional activation based approaches that suggested a role for the MPFC in representing trait information by providing strong evidence that it is the inferred trait per se – not lower-level behavioral information, semantic similarity, or some epiphenomenal feature – that is encoded in the activity of the MPFC. Later repetition suppression studies by the same group extended this finding, demonstrating that the MPFC encodes not only trait information but also the identity of the person associated with those traits (Heleven & Van Overwalle, 2016), a finding which dovetails with previous work in experimental social psychology showing that the acquisition of trait knowledge is facilitated by associating it with a facial identity (Todorov & Uleman, 2002).

The findings of the repetition suppression studies reviewed above offer some initial evidence of a neural code in the MFPC for familiar others and raises the intriguing possibility that we might be able to use more sophisticated approaches, such as multivariate pattern classification, to decode familiar identities in the MPFC. Although prior research had suggested that viewing personally familiar faces might spontaneously activate representations of person knowledge in the MPFC (Cloutier, Kelley, & Heatherton, 2011; M. Ida Gobbini & Haxby, 2007; Todorov, Gobbini, Evans, & Haxby, 2007), it is only recently that researchers have turned to using MVPA in an attempt to decode familiar identities from activation in this area. In one such study, Viconti di Oleggio Castello and colleagues (2017) had participants perform a simple visual oddball-detection task in the scanner while viewing sets of familiar and unfamiliar faces. Using searchlight-based multivariate pattern classification, they found that regions implicated in mentalizing and impression formation (e.g., the MPFC, precuneus, and TPJ) could reliably distinguish between familiar and unfamiliar faces, even when tested on pairs of familiar and unfamiliar faces that were not part of the original training set (Visconti di Oleggio Castello, Halchenko, Guntupalli, Gors, & Gobbini, 2017). Moreover, these same regions also distinguished between the specific identities that participants had viewed (Figure 2b). Interestingly, the ability to decode identity in the MPFC was true of both familiar and unfamiliar identities. The Involvement of the MFPC in decoding unfamiliar identities is somewhat surprising given that this region typically shows a stronger univariate response for familiar as compared to unfamiliar identities (e.g., (Cloutier, Kelley, et al., 2011; M. Ida Gobbini, Leibenluft, Santiago, & Haxby, 2004; Leibenluft, Gobbini, Harrison, & Haxby, 2004; Todorov et al., 2007; Welborn & Lieberman, 2015). One possibility for why unfamiliar identities were able to be decoded from patterns of activity in the MPFC is that, over the course of the task, participants formed unique impressions of the unfamiliar faces despite having no biographical information to draw on for these faces. Indeed, there is large body of research demonstrating that individuals rapidly form impression of people’s traits from faces (Todorov et al. 2015; Willis and Todorov 2006). For instance, studies of first impressions show that there is broad consensus among individual’s when inferring the traits of trustworthiness and dominance for unfamiliar faces (Oosterhof and Todorov 2008). Moreover, theoretical accounts of these trait inferences, suggest that these judgements reflect unintended similarities between a target’s idiosyncratic facial structure and emotional expressions like anger (Zebrowitz and Montepare 2008; Todorov et al. 2008), similarity to typical male and female faces (Freeman and Ambady 2009) as well as their resemblance to baby faces (Zebrowitz 2017). Studies also show that neutral facial expressions themselves tend to be perceived not as emotionally neutral but rather being negatively emotionally valenced (Tottenham, Phuong, Flannery, Gabard-Durnam, & Goff, 2013).

Figure 2.

(a) Patterns of activity in the dMPFC during mental simulation of each of four different identities that varied in the personality dimensions of agreeableness and extraversion could be reliably discriminated using MVP classification (Hassabis et al., 2014). (b) Facial identity decoding in the the dMPFC and precuneus for familiar and unfamiliar identities (Visconti di Oleggio Castello et al., 2017). (c) Regions involved in social cognition demonstrate reliable identity-specific patterns of activity across split halves of a mentalizing task. These regions were then used in subsequent analyses demonstrating that identity-specific patterns of activity were associated with predictions based on social psychological theories of person perception (Thornton & Mitchell, 2017b). (d). Characteristics of an familiar individual’s social network position, such as centrality, can be decoded from patterns of neural activity associated with short video segments of each individual (Parkinson et al., 2017).

Decoding of identity and person knowledge based on patterns of neural activity has also been demonstrated in the context of novel identities constructed from trait implying biographical information. For instance, Hassabis et al. (Hassabis et al., 2014) had participants learn the personalities of four distinct identities that varied on the personality dimensions of agreeableness and extraversion. During scanning, participants mentally simulated each identity in a variety of everyday situations (e.g., in a bar). Results using MVPA demonstrated that patterns of activity within the dMPFC reliably discriminated between the four different identities (Figure 2a). Moreover, based on the results of a functional connectivity analysis, it was found that the MPFC, the precuneus and the left temporal cortex were functionally connected during mental simulation, leading the authors to speculate that the MPFC plays a central role in integrating across these different regions in order to assemble and update the representations of others’ personality (Freeman & Stolier, 2014; Hassabis et al., 2014).

Taken together, the repetition suppression and MVPA studies reviewed above provide evidence that person knowledge related to specific identities is encoded in the MPFC and that identity can be decoded from activity patterns in this region. Recently, a pair of studies have expanded on this work using representational similarity analysis to explore the representational structure of a broad array of identities. In the first of these studies, Thornton and Mitchell (2017) had participants engage in a mental simulation task similar to that of Hassabis and colleagues (2014) reviewed above. During this task, participants imagined a set of twenty personally familiar others in a variety of different contexts (e.g., eating at a restaurant). Using a searchlight-based representational similarity analysis, they tested whether the neural representational similarity structure was best fit by a model based on person identity or one based on the situational context. The results of this analysis demonstrated that the MPFC, precuneus and other areas commonly involved in impression formation, appear to encode person-specific patterns of neural activity. In contrast, the only regions that encoded the situational contexts was the precuneus and an additional area of occipital cortex (Thornton & Mitchell, 2017a). These regions were then used in a second large-scale RSA analysis examining how the neural similarity space of all these areas together might be related to several well-known social psychological models of person perception (e.g., the five-factor model of personality, stereotype content model, relational models theory). Results of this analysis revealed small but significant support only for the relational models theory (Fiske & Haslam, 1996), which characterizes the similarity between the twenty target identities in terms of the form of social relationships that participants have with each individual. Although the general findings of this study are in line with the previous MVP classification based approaches in that they demonstrate that the MPFC represents aspects of familiar identity, the fact remains that the specific type of person information that is being encoded in this region remains elusive. A second study by Thornton and Mitchell (2017b) again examined how different models of person perception might relate to the neural representations of others, this time extending the set of identities to 60 well-known public figures and, instead of a mental simulation task, participants were asked to make inferences as to the personal preferences of each identity (i.e., liked debating with others, see Mitchell 2006). The authors first identified regions that demonstrated reliable identity-specific patterns of activity by correlating voxelwise and pattern-wise similarities across split halves of the data (Figure 2c). The regions identified in this analysis, comprising the MPFC and several other regions implicated in social cognition, were then investigated using unique feature-space modeling approach whereby a mapping is learned between the dimensions underlying each model of person perception (e.g., trustworthiness and social dominance) and the neural data associated with evaluating the preferences of the different public figures minus a left-out identity for validation testing. More specifically, for each dimension of each model a representative pattern of neural response for that dimension is generated by taking the average of the pattern of neural activity associated with each identity weighted by that identities score on a given dimension (e.g., trustworthiness). Once these “canonical” neural patterns for each dimension of social evaluation were computed, the predicted pattern of neural activity for the left-out identity was generated by multiplying these patterns by the associated scores along each dimension and summing the results. The accuracy of this method was then tested by correlating the predicted patterns of brain activity with the actual measured l neural pattern associated with the left-out identity. In contrast to their prior study, this variant of task and analytic approach provided evidence that each of the standard models of person perception was associated with the accuracy of predicted neural data with no one model showing a clear advantage across variants of the analyses.. Together with the prior studies, these findings serve as strong initial evidence that the information represented in the MFPC during the perception of familiar identities is related to our social knowledge (e.g., traits, liking, relationships) of these different persons.

Another form of person knowledge not traditionally a part of standard social psychological theories of person perception is social network position. Recently, Parkinson and colleagues (2017) investigated whether the neural representation of familiar identity might incorporate information related to an individual’s social network characteristics. In this study, the authors characterized the social network of an entire academic cohort and invited a subset of these individuals to complete a neuroimaging study. During fMRI, participants viewed videos of members of their cohort who varied according to three different features of social network position (e.g., centrality, constraint and distance). Using representational similarity analysis, it was found that different sets of largely non-overlapping brain areas encoded these three metrics. Social distance was represented in a region spanning the inferior parietal lobule and the posterior aspect of the temporal gyrus, a finding that is consistent with prior work showing that conceptually similar but non-network based measures of social distance are also represented in this region (Parkinson et al., 2014). Another feature of social network position examined was eigenvector centrality (EC), or the extent to which an individual is well-connected to other well-connected others. EC, which is modestly correlated with extraversion, is thought to be a largely stable dispositional tendency, and therefore was expected to be represented in brain areas previously implicated in representing trait information. RSA analysis focusing on regions representing eigenvector centrality, showed that this metric was represented in the MPFC as well as the precuneus and PCC (Figure 2d) (Parkinson, Kleinbaum, & Wheatley, 2017).

The results of the multivariate pattern classification and RSA studies just described represent strong initial evidence that the MPFC encodes person knowledge in the form of a multidimensional representational space that is, at least in part, related to social psychological models of person perception and/or social network position and exists at a spatial scale accessible to current fMRI technology. Whether or not we are yet at the point where we can use neural data to arbitrate between competing psychological theories is less clear. In these studies, models defined by different psychological theories or by social network position tend to provide similar results although there appears to be some non-negligeable heterogeneity due to task or methodological approach (Hassabis et al., 2014; Parkinson et al., 2017; Thornton & Mitchell, 2017a, 2017b). Moreover, the specific applicability of different models of person perception may also vary as a function of task, social target or due to individual differences in the participants. Taken together, the findings reviewed in this section suggest that the representational space of social knowledge is related to psychological models of social perception at both the region level (Hassabis et al., 2014; Parkinson et al., 2017) as well as at the systems level incorporating a network of brain areas involved in social cognition (e.g., Thornton & Mitchell, 2017a,b). This suggests that at least some of the information contained in fine-grained patterns of localized activity is recapitulated at the scale of interregional differences throughout a network of brain areas (Thornton & Mitchell, 2017a).

Decoding race and social groups from faces

Research on the neural underpinnings of person perception using MVPA has also been conducted to study how the brain categorizes social groups and race. This work predominantly focuses on the Fusiform Face Area (FFA), a region of the inferior temporal cortex that is believed to represent the invariant features of faces that enable the recognition of facial identity (M. Ida Gobbini & Haxby, 2007; Kanwisher & Yovel, 2006; Rotshtein, Henson, Treves, Driver, & Dolan, 2005). Several initial studies on the representation of race in the FFA demonstrated that the racial category of a face could be decoded from multivariate patterns within the FFA even in the absence of univariate differences in the average response within this region (Brosch, Bar-David, & Phelps, 2013; Contreras, Banaji, & Mitchell, 2013; Kaul, Ratner, & Van Bavel, 2014; Ratner, Kaul, & Van Bavel, 2013; Stolier & Freeman, 2017). Interestingly, the ability to decode the race of a face based on patterns of activity within the FFA appears to vary as a function of individual characteristics of the perceiver as well as the context in which race categorization is made. For instance, one study examined the accuracy of race decoding in the FFA as a function of individual differences in implicit race bias as measured by the implicit association test (Greenwald, McGhee, & Schwartz, 1998). In this study, participants performed a simple 1-back task requiring them to press a key whenever a facial identity was repeated. Using multivariate pattern classification they demonstrated that classification accuracy in the FFA was significantly correlated with participants implicit racial bias scores such that those with a more pro-white score exhibited greater decoding accuracy (Brosch et al., 2013). Race decoding accuracy in the FFA was also found to vary as a function of participants’ task goals, for example when participants explicitly categorized faces by race, decoding accuracy in the the FFA decreased compared to when participants categorized faces along dimensions orthogonal to race (Kaul et al., 2014).

More recently, a pair of studies by Stolier and Freeman (2016, 2017) have demonstrated how social categories that share common stereotypic associations can become “entangled” leading to cross-category activation of social groups or emotional categories. In the first of these studies, participants viewed a series of faces that varied independently along the categories of race (Black, White, Asian), sex (Male, Female), and emotion (Angry, Happy). Building on prior behavioral work demonstrating that social groups with shared stereotypical associations bias social categorization (e.g., Black and Male sharing the stereotype of aggression; (Johnson, Freeman, & Pauker, 2012), this study employed a mouse-tracking paradigm to derive the subjective similarity between faces belonging to each of the different social categories. Using representational similarity analysis, the authors found that the participants’ biased perception of the different social categories was reflected in the representational structure of the fusiform gyrus and orbitofrontal cortex (Stolier & Freeman, 2016). Moreover, this effect obtained even after accounting for the visual similarities between facial stimuli (Figure 3). A second more recent study extended this work by employing a similar mouse-tracking task during fMRI scanning. In this task, participants categorized a series of computer-generated faces that varied in how the typical they were of their social group (e.g., how masculine/feminine a female face appeared). As in previous behavioral research, atypical faces drew participants’ hand movements toward the opponent category, suggesting these atypical faces activate the representation of both social categories. Using RSA, the authors demonstrated that the degree to which the neural pattern for a given atypical face resembled the average neural pattern for the typical opponent category (i.e. an atypical male face resembling the average typical female face) was correlated with the the degree to which this opponent category was activated during the mouse-tracking task (Stolier & Freeman, 2017). Together, the findings of these studies argue that neural representation associated with social targets that are multiply categorizable reflects both stereotypic associations and, in the case of atypical faces, opponent social categories.

Figure 3.

(a) Patterns of activity in the right fusiform and early visual cortex were associated with participant’s biased stereotypic associations between racial, gender and emotion. (b) After accounting for visual similarity between the faces, the right fusiform and orbitofrontal cortex (not shown) remain significantly associated with participants biased perception of racial and gender categories (Stolier & Freeman, 2016).

Thus far, much of this prior work has focused on detecting a social target’s race or group membership based on facial characteristics, however a pair of recent studies (Lau and Cikara 2017; Cikara et al. 2017) has examined how broader social groups based not on faces but on shared group affiliations (e.g., political preferences, assignment to teams) are represented in the brain. In both of these studies, a minimal groups approach (Tajfel et al. 1971) was used to assign a group of twenty-three self-identified democrat participants to one of two arbitrary teams (i.e, eagles or rattlers) in order to generate the feeling of group affiliation without the baggage of stereotypes associated with real-world social groups. In the first of these two studies, participants then underwent fMRI while viewing statements describing different individuals group membership (e.g., “Eve is a Democrat”, “Skyla is an Eagle”, “Niko is a Republican”). Using a repetition suppression approach, the authors identified regions that that were sensitive to in-group but generalized across both political groups and arbitrary teams (Lau and Cikara 2017). In contrast to the previous research focusing on categorization from faces, this study found that a set of fronto-parietal regions often associated with attentional control demonstrated a repetition suppression effect for group membership. This finding was largely replicated in a second study that used a multivariate pattern classification approach in which a pattern classifier was trained to discriminate group membership based on arbitrary teams and tested on political parties and vice versa. As in the repetition suppression study, the authors found a set of frontoparietal regions that discriminated between group membership but we’re most accurate at identifying in-group targets across social groups (i.e., political in-group members and arbitrary in-group members) (Cikara et al. 2017). Together these results suggest that detection of in-group members, even across different social group boundaries, may be prioritized by the brains system for attentional control.

Social Cognition During Viewing of Naturalistic Stimuli

The majority of the research covered in this review investigates the neural representation of different forms of social knowledge in experimental tasks that are necessarily limited in their number of conditions and stimuli. In recent years, there has been a growing trend towards using naturalistic stimuli (e.g., scenes, movies, virtual reality) with a variety of data-driven and MVPA approaches to study cognition in more real-world contexts (for reviews see (Hasson, Ghazanfar, Galantucci, Garrod, & Keysers, 2012; Hasson, Malach, & Heeger, 2010; Nummenmaa, Lahnakoski, & Glerean, 2018). Here we focus specifically on how these approaches have been applied to the domain of social cognition. One means of measuring participant responses in a reasonably naturalistic manner is to present them with visual scenes depicting various types of social and non-social interactions. Earlier work using stimuli depicting social interactions through animations of objects interacting or via photographs of people, demonstrated that individuals will spontaneously recruit the dMPFC even in the absence of any specific social cognitive task demands (Maria Ida Gobbini, Koralek, Bryan, Montgomery, & Haxby, 2007; Powers, Chavez, & Heatherton, 2016; Powers, Wagner, Norris, & Heatherton, 2013; Wagner, Kelley, & Heatherton, 2011; Wheatley, Milleville, & Martin, 2007). Moreover, the magnitude of this response has been shown to be moderated by individual differences in empathizing (Wagner et al., 2011) and was also found to predict real-world frequency of social interaction as measured via experience sampling (Powers et al., 2013). Although these prior activation-based studies suggest an important role for the dMPFC in spontaneous social cognition, they remain agnostic as to the precise features of a stimulus that appear to drive dMPFC activity. Recently, Wagner and colleagues employed a data-driven approach known as reverse correlation (see section above for an overview of the method) in an attempt to characterize the response profile of the dMPFC during natural viewing of a movie. In this study, participants viewed the first thirty minutes of a popular hollywood movie during fMRI scanning. Afterwards, the timeseries of brain activity from the dMPFC and several areas with known category preferences (e.g., the FFA) was used to identify those segments of the film that evoked the strongest and weakest response in each region. Replicating prior work using this method (Hasson et al., 2004), this study found that the response profile of the FFA was characterized by movie scenes involving a strong percept of a face. More interesting, however, were the results of the reverse correlation analysis in the dMPFC. Examination of the movie segments that elicited the strongest peaks in this region revealed that the dMPFC appeared to be preferentially tuned to movie scenes involving social interactions (Figure 4a). Subsequent content analysis of these movie segments confirmed this finding, as both verbal descriptions and explicit ratings (Figure 4b) suggested that this region was responding most strongly to segments of the movie involving complex social interactions (Wagner et al., 2016). The results of this data-driven analysis demonstrate that, even in the absence of any explicit task, the dMPFC is preferentially tuned to the social features of naturalistic stimuli, a finding in keeping with this region’s suggested role in impression formation and mental state inferences.

Figure 4.

(a) Result of a reverse correlation analysis of the neural response in the dMPFC during natural viewing of a thirty minute motion picture. The top right plot represents the volume-wise -statistic for a one-sample t-test across thirty-four participants. This timeseries of statistical peaks and troughs was used to extract movie frames corresponding to these timepoints. The bottom plot contains representative frames from these segments demonstrating that the dMPFC responds maximally to scenes involving social interaction between movie characters. (b) An independent set of one-hundred and thirty-two participants rated the movie segments generated form the reverse correlation procedure from several different brain regions (dMPFC, lateral and medial fusiform and the intraparietal sulcus) on a number of dimensions. Illustrated here are ratings of social complexity showing that video segments derived from the peak responses in the dMPFC were rated as more socially complex than those derived from three other regions with known category preferences. Figure adapted with permission from (Wagner et al. 2016).

More recently, a study used intersubject correlation analysis to examine how groups of people may interpret the same events in two very different ways. Specifically, ISC was used to examine whether the neural synchrony of brain activity timecourses differentiated people who share differing interpretations of a story (Yeshurun et al., 2017). In this study, participants were scanned while listening to a short story describing a husband searching for his wife after a party. Critically, half the participants were led to believe the wife was engaging in an extra-marital affair and the other half were led to believe the husband was simply being paranoid. Using ISC (see methods overview section for a description) the authors found greater similarity of neural timecourses between pairs of subjects who had the same interpretation of the characters compared to subjects in the group whose interpretation differed. These neural similarity differences were found primarily in regions implicated in social cognition (MPFC, precuneus, TPJ) and several regions involved in action and language comprehension (Yeshurun et al., 2017). Additional analyses explored whether the magnitude of between-group differences in neural response was correlated with segments of the story that were likely to have large between-group differences in interpretation of the characters’ beliefs, emotions, and intentions (as determined by expert raters). Findings suggested that differences in neural similarity between individuals with different interpretations of the same story were driven, at least in part, by segments of the story where the representation of a character’s beliefs and emotions was rated to be the most different between the two interpretations. This study, represents an exciting first step toward using brain data to explore spontaneous differences in the interpretations of complex social events, such as those that may arise between groups with different ideologies or political attitudes.

Conclusion

The increasing dominance of multivariate pattern analysis and data-driven neuroimaging methods to study neural representations has allowed researchers to move beyond brain mapping and inferences about process that are based on relative differences in activation within brain areas. By capitalizing on patterns of activity rather than voxel or region averaged responses, these methods make possible the exploration of the informational content and structure of neural representations associated with different perceptual and cognitive phenomena. In this article, we reviewed recent applications of MVPA and data-driven neuroimaging methods in social cognitive neuroscience that are aimed at decoding and/or characterizing the neural representation of knowledge about self and others. Findings reviewed in this paper have extended our understanding of how the self is represented in the MPFC, providing novel evidence of its association with subjective value and valence (Chavez et al., 2017; Yankouskaya et al., 2017) as well as suggesting that individuals and their friends share a common neural representation of a person’s self in the VMPFC (Chavez & Wagner, under review). With respect to research on the representation of others, we reviewed recent findings demonstrating that both familiarity and identity can be decoded from activity in the MPFC as well as other areas involved in social cognition (Visconti di Oleggio Castello et al., 2017) as well as studies using representational similarity analysis and repetition suppression that suggest that the representation of familiar identity in the MPFC is, at least in part, related to broad personality traits (Heleven & Van Overwalle, 2016; Thornton & Mitchell, 2017b) and an individual’s social network position (Parkinson et al., 2017).

In this review, we have focused primarily on the MPFC as it remains one of the most consistently recruited regions when it comes to self and person knowledge. Moreover, neuropsychological investigations of the deficits associated with damage to - or inactivation of - this area argue that it is a critical component in self-knowledge (Philippi et al., 2012) and may play a causal role in impression formation (Ferrari et al., 2016). That said, we have also discussed other areas that appear to be important for the representation of social knowledge (e.g., the precuneus, TPJ) as well as those involved in social categorization of faces (e.g., FFA). At present, there is still considerable ambiguity as to the precise computations being conducted in each of these regions. Further investigation into the differential contributions of these brain regions to our overall representations of social knowledge promises to be an important direction for future work. Indeed, developing a computational account of how representations in each of these core areas are transformed, communicated to other regions, and ultimately associated with behavior may eventually lead to a truly comprehensive model of the neural basis of social perception, one that is only now possible due to the methods and techniques reviewed here. However, enthusiasm for the possibilities afforded by these approaches may have to be tempered by the reality that, in general, the accuracy of MVPA approaches and computational models in the social domain lags far behind that found in vision neuroscience. Moreover, owing to both the idiosyncratic nature of social representations along with the difficulty inherent in modeling cognition that is both temporally, and often even cognitively uncoupled from the stimulus in a way that vision studies seldom are may put an upper limit on the ability to ever truly capture the complexity and variability of neural representations of social material. Nevertheless, the studies reviewed here offer some promising glimpses at what the next ten years of social cognitive neuroscience has in store and thus there is much to explore before worrying about exhausting the predictive ability of MVPA and related methods.

The manner by which social knowledge about the self and others is inferred from behavior, transformed by prior knowledge and expectations and ultimately distilled into a stable impression of a person has long been a fundamental topic of research in social psychology. Long ago, the psychologist Fritz Heider posited that people have a fundamental need to understand and explain their social world and theorized that individuals do so by searching for invariances in others’ behavior that can then be used to make attributions about their underlying motives, emotions and traits (Heider, 1958). In the last several years there have been numerous theories of medial prefrontal cortex function that suggest this region may play a central role in in this task by enabling individuals to explain and predict the thoughts and motives of others by mentally simulating their hidden mental states and in certain circumstances using the self as a template to understanding others (e.g. (Jenkins, Macrae, & Mitchell, 2008). For instance, researchers have theorized a role for the MPFC in simulating future events and reconstructing past ones (Buckner & Carroll, 2007), continuously generating predictions about social and non-social events (Bar, 2009), reasoning about ambiguous or counterfactual information (Mitchell, 2009) as well as generating models of others’ personalities and motives (Freeman & Stolier, 2014; Hassabis et al., 2014; Tamir & Thornton, 2018). Several years ago in a review on the role of the MPFC in representing knowledge about the self and others (Wagner et al. 2012) we concluded by suggesting that the adoption of MVPA methods for the study of social knowledge would shed new light on the neural and cognitive representations of social knowledge. The recent studies reviewed here have certainly done just that, even if they also raise many more questions. If we may be permitted to prognosticate once again, it would be to suggest that as social cognitive neuroscience enters its third decade, we will see great advances in the application of MVPA and related methods to our understanding of social phenomena. In addition, we also predict that the rise of large-scale sharing of neuroimaging data and the development of increasingly sophisticated tools for analysing large fMRI datasets and finding methods to abstract away from idiosyncratic neuroanatomy in order to improve analysis of fine-grained patterns across individuals (Haxby et al. 2011; Guntupalli et al. 2016)will drive novel discoveries, enable predictive models that work across-datasets and ultimately help the field move closer to providing a computational framework for how the brain infers and represents social knowledge. Finally, we anticipate that the use of naturalistic stimuli (e.g., movies, stories and virtual reality) to study the neural basis of social cognition (e.g., (Wagner et al., 2016; Yeshurun et al., 2017) will continue to be important as findings and models created using more conventional experimental approaches are tested in increasingly more naturalistic contexts.