Neural activity during social signal perception correlates with self-reported empathy

Abstract

Empathy is an important component of human relationships, yet the neural mechanisms that facilitate empathy are unclear. The broad construct of empathy incorporates both cognitive and affective components. Cognitive empathy includes mentalizing skills such as perspective-taking. Affective empathy consists of the affect produced in response to someone else's emotional state, a process which is facilitated by simulation or ‘mirroring’. Prior evidence shows that mentalizing tasks engage a neural network which includes the temporoparietal junction, superior temporal sulcus, and medial prefrontal cortex. On the other hand, simulation tasks engage the fronto-parietal mirror neuron system (MNS) which includes the inferior frontal gyrus (IFG) and the somotosensory related cortex (SRC). Here, we tested whether neural activity in these two neural networks was related to self reports of cognitive and affective empathy in daily life. Participants viewed social scenes in which the shift of direction of attention of a character did or did not change the character's mental and emotional state. As expected, the task robustly activated both mentalizing and MNS networks. We found that when detecting the character's change in mental and emotional state, neural activity in both networks is strongly related to cognitive empathy. Specifically, neural activity in the IFG, SRC, and STS were related to cognitive empathy. Activity in the precentral gyrus was related to affective empathy. The findings suggest that both simulation and mentalizing networks contribute to multiple components of empathy.

1. Introduction

The capacity to empathize with others is crucial for building and maintaining successful interpersonal relationships (Batson and Shaw, 1991; Davis, 1996). Empathy requires understanding someone else's mental and emotional state and responding to them appropriately - a process which incorporates both affective and cognitive components (Davis, 1996; Leiberg and Anders, 2006; Singer, 2006). The affective component of empathy consists, primarily, of the affective state that is produced in response to another person's emotional experience. This affective response often results in sharing the same emotion that is observed, such as feeling sad about someone else's loss, and it is related to the understanding of the other person's emotional state. The cognitive component of empathy consists of understanding a situation from another person's point of view and taking into account that the other person acts and reacts to a situation based on beliefs, goals, and intentions that may be different from one's own. This process is referred to as mentalizing or Theory of Mind.

Evidence suggests that these two components of empathy rely on different psychological and neurological mechanisms (Shamay-Tsoory et al., 2009). Affective empathy is hypothesized to arise via the process of simulation which relies on imitation (or “mirroring” activity) to facilitate emotion understanding and produce affective sharing (Decety and Jackson, 2004; Preston and de Waal, 2002). This simulation theory of empathy is conceptually linked to action-perception models (Preston and de Waal, 2002) and suggests that the observation of an emotional expression automatically activates the motor and somatosensory representation of that emotional expression in the motor and somatosensory regions of the fronto-parietal “mirror neuron system” (MNS) (Gallese and Goldman, 1998; Gallese et al., 2004; Gallese, 2007). The “mirroring” (i.e. the automatic and often subconscious imitation) of observed emotional expressions produces an embodied representation which can facilitate the decoding of the observed person's emotional state as well as induce that emotional state in the observer (Adolphs, 2002; Preston and de Waal, 2002). The ventrolateral premotor cortex and the inferior parietal cortex have been identified as key neural substrates involved in the “mirroring” of emotional expressions. This includes motor-related cortex, such as the precentral gyrus (BA 4, 6) and inferior frontal gyrus (IFG) (BA 44, 45) (Carr et al., 2003; Pfeifer et al., 2008) and somatosensory-related cortex (SRC), such as the postcentral gyrus (BA 3) and the supramarginal gyrus (BA 40) in the inferior parietal lobe (Adolphs et al., 2000; Gazzola et al., 2006). Neuroimaging studies show that the IFG is active during the imitation of facial expressions (Carr et al., 2003), and among children, the amount of activity in this region during imitation is related to self-reported empathy (Pfeifer et al., 2008). Furthermore, a lesion in the IFG is associated with poor emotion recognition skills and low affective empathy (Adolphs et al., 2000; Shamay-Tsoory et al., 2009).

On the other hand, mentalizing is a more cognitively effortful process that develops later in life and involves a different set of neural mechanisms (Saxe et al., 2004). Neuroimaging studies which require participants to represent the belief state or intentional stance of another person reliably activates a set of brain regions, including the temporoparietal junction (TPJ), the superior temporal sulcus (STS), the medial prefrontal cortex (MPFC), and the temporal poles (Frith and Frith, 2006; Gallagher and Frith, 2003). Lesion studies support the idea of a devoted neural network for processes related to cognitive empathy. For example, neurological patients with left superior temporal lesions have deficits on theory of mind tasks, such as the false belief task (Samson et al., 2004), and ventral MPFC lesion patients have low self-reported cognitive empathy whereas their affective empathy is in normal range (Shamay-Tsoory and Aharon-Peretz, 2007; Shamay-Tsoory et al., 2009).

Despite this initial evidence that imitation and mentalizing support affective and cognitive components of empathy, it is still unclear the extent to which they rely on dissociable neural regions. More importantly, it is also unknown how neural activity in regions associated with the two systems (MNS and mentalizing) is related to the use of these empathic processes in daily life. Neuroimaging studies that have sought to show differences in the MNS versus mentalizing networks have used different stimuli for each condition (e.g. (Hynes et al., 2006; Nummenmaa et al., 2008; Shamay-Tsoory et al., 2005). These studies show that certain regions are more sensitive to specific stimulus features. For example, Saxe and Powell (2006) investigated neural response to stories describing another person's thoughts as compared to another person's bodily states. They found that the TPJ was active to descriptions of thoughts and beliefs whereas the SRC was active to descriptions of bodily states such as hunger, thirst, and exhaustion (Saxe and Powell, 2006). While this suggests that designated regions are relatively more sensitive to specific features, it does not reveal how neural activity in response to these stimulus features support the complex process of empathizing with another person.

Furthermore, most tasks that involve social and emotional processing, particularly those that attempt to mimic social interactions, will engage neural response from both the MNS, and the mentalizing systems (Hynes et al., 2006; Schulte-Ruther et al., 2007). This underscores the fact that it is difficult to separate emotions and beliefs because emotional response is usually based on a person's belief about a situation. Additionally, the observer's understanding of another person's emotional state is dependent of the observer's understanding of context. For example, the facial display of surprise may use the same facial motor action regardless of whether that surprise occurs in the context of a positive or negative event (Ekman and Friesen, 1978); however, it is only when the context is integrated with the expression does the observer really understand what that person is feeling and is able to respond appropriately (Barrett, 2006; Barrett et al., 2007; Kim et al., 2004).

Here, we address these issues with a task aimed at engaging activity related to both mentalizing and MNS and then identifying whether mentalizing and imitation related regions are correlated with self-reports of cognitive and affective empathy, respectively. We created a series of complex social scenes in which each scene is a static snapshot of a different story scenario. In each scene, one character has full knowledge about what is happening in the scene (i.e. a “True Belief”) and one character has only partial knowledge or a misunderstanding about what is happening (i.e. a “False Belief”). Both characters display emotional expressions based on their beliefs about the situation. During the task, participants view the scene and have time to comprehend the social scenario. Then one of the characters in the scene changes their direction of attention by shifting their head and body orientation. In the primary condition of interest, The Social Change condition, the shift in direction of attention results in a visible change in mental state. The social change occurs because due to the direction of attention shift, the character with only partial knowledge sees something in the scene which changes their belief about the situation as well as their emotional response based on that belief. (See Supplemental Materials for a description of the scenarios.) The expectation is that the observed biological action associated with the change in emotional state, as understood from body gestures and facial expressions, will activate MNS regions, such as the ventrolateral premotor cortex and the somatosensory related cortex (SRC). At the same time, the change in belief state, i.e. changing from a false belief to a true belief will activate mentalizing regions, such as the TPJ, STS, and MPFC. The primary hypothesis is that activity in mentalizing regions will be significantly correlated with cognitive empathy and activity in MNS regions will be related to affective empathy.

In a second condition, the Physical Change condition the story character changes their direction attention by changing their body posture and/or head direction, but this change does not change their mental or emotional state. This condition, of biological movement only, is to identify whether activity in MNS and mentalizing regions in response to biological motion without mental state change will be related to empathy. We predict that while MNS and mentalizing regions will be active while processing these biological motion cues, activity in these regions will not be related to empathy. This prediction is supported by prior research and theory suggesting that MNS (and mentalizing) activity reflects the processing biological motion cues in order to build a higher level representation of that person's mental state or emotional state – i.e. the ultimate goal and intention associated with the action. The baseline comparison for both of these conditions is the same social scenes in which no change (no shift of attention) takes place (i.e. the No Change condition). See Figure 1 for an illustration of the task.

Figure 1.

Figure 1 shows an example of the stimuli. For both Physical and Social Change conditions, the first picture, which establishes the story, is presented for 4 seconds and then the Change picture is presented for 4 seconds. A). In the Social Change condition, a character changes their direction of attention which results in a change in mental state. For example, in the initial scene, the father does not know that his son has a failing grade. The father is happily reading the paper while his son approaches him looking afraid. In the “Social Change” scene, the father turns, sees the failed grade, and his emotion changes from happy to angry. B). In the Physical Change condition, a character changes their direction of attention but this does not result in a change in mental state. For example, in the initial scene, the father looks angrily at his son's messy room. The son is happily playing with his toys and does not see his father behind him. In the “Physical Change” scene, the son turns to the side, but still does not see his father, and he continues to be happy. Each Physical Change and Social Change trial has an accompanying No Change trial which contains the same story content but there is no change in direction of attention (i.e. the initial scene presented twice).

Cognitive and Affective Empathy is assessed with the Interpersonal Reactivity Index (IRI) (Davis, 1996). Cognitive Empathy score is composed of two subscales, Perspective Taking and Fantasy. Perspective-Taking (PT) is the tendency to “put yourself in someone else's shoe's” when trying to understand their point of view. The Fantasy scale (FS) assesses the ease with which a person can relate to fictional characters in novels and movies. Affective Empathy is composed of the subscales Empathic Concern (EC) and Personal Distress (PD). Empathic Concern is the tendency to have an emotional response to other people, such as sympathy when witnessing someone in distress. Personal Distress is the tendency to have negative emotions, such as worry and distress, in difficult situations (Davis, 1983; Davis, 1996).

2. Results

2.1. Behavioral Results

The analysis of behavioral performance on individual items showed that the majority of subjects made an error on one scene in the Social Change condition (mean accuracy 40%) and one scene in the Physical Change condition (mean accuracy 23%), suggesting that these two scenes were consistently misinterpreted. Therefore, we dropped these two trials and their accompanying No Change trial from further behavioral and imaging analysis; however, the inclusion or exclusion of these trials does not change the statistical significance of any of the results.

The mean accuracy for each condition was as follows: Social Change = 79% (s.d. +/- 14%); Physical Change = 85% (s.d. +/- 8%); No Change (Social stories) = 99% (s.d. +/-3%); No Change (Physical stories) = 99% (s.d. +/- 2%). Mean reaction time for each condition was as follows: Social Change = 1903ms (s.d. +/- 324ms); Physical Change = 2118ms (s.d. +/- 325ms); No Change (Social stories) = 1572ms (s.d. +/- 416ms); No Change (Physical stories) = 1560ms (s.d. +/- 382ms). Not surprisingly, paired t-tests show that subjects were faster and more accurate for the No Change condition as compared to the Change conditions [Accuracy: Social Change vs. No Change (social), t = 5.3, p<.01; Physical vs. No Change (physical) t = 5.9, p<.01. Reaction Time: Social Change vs. No Change (social), t = 3.7, p<.01; Physical vs. No Change (physical) t = 6.4, p<.01]. There was no difference in accuracy between the Social Change and Physical Change conditions (t= 1.6, ns). However, there was a difference in reaction time, such that subjects were faster to respond to the Social Change condition as compared to the Physical Change condition (t = 3.2, p<.01).

2.2. fMRI Results

2.2.1. Physical Change versus No Change

We first investigated neural activity in response to the Physical Change stories as compared to those same stories when no change occurred. As expected, we found enhanced neural activity in both MNS and mentalizing regions. Specifically, in the left hemisphere, there was activity in the inferior frontal gyrus, an area often associated with MNS processes. Mentalizing regions that were active in this contrast included the bilateral temporoparietal junction, the right superior temporal sulcus (STS), bilateral MPFC, and bilateral posterior cingulate. The posterior middle temporal gyrus was also active bilaterally. All significant brain activations in this contrast are listed in Table 1 and shown in Figure 2.

Table 1.

Brain regions that showed significantly greater activity for the Physical Change condition as compared to those same stories in the No Change condition. P<.001, cluster 10

Brain Region: Physical Change > No Change	R/L	BA	MNI Coordinates X, Y, Z	T value	Cluster Size
Frontal Cortex:
Lateral Orbital Frontal Gyrus (LOFC)	L	47	-36, 22, -22	7.0	2065#
Inferior Frontal Gyrus – Triangularis (IFG)	L	45	-54, 28, 12	5.8	(2065)#
Lateral Orbital Frontal Gyrus (LOFC)	R	47	46, 38, -18	5.3	125#
Middle Frontal Gyrus (DLPFC)	L	46	-38, 18, 42	5.04	286
Middle Frontal Gyrus (DLPFC)	R	46	46, 22, 42	7.1	546
Superior Frontal Gyrus (DMPFC)	R/L	9	-18, 58, 40	8.02	3993
Middle Frontal Gyrus – Anterior	R	10	38, 60, 6	4.15	23
Temporal Cortex
Posterior Superior Temporal Sulcus (pSTS) – Temporoparietal Junction (TPJ)	L	39	-52, -64, 30	9.5	1744 **
Middle Temporal Gyrus	L	21	-68, -36, -8	8.3	539 *
Posterior Superior Temporal Sulcus (pSTS)- Temporoparietal Junction (TPJ)	R	39	48, -62, 32	7.51	1712#
Middle Temporal Gyrus – extending to Superior Temporal Sulcus – middle portion	R	21	66, -30, -6	7.24	404*
Middle Temporal Gyrus	R	21	62, -14, -20	4.34	17 #
Parietal & Occipital Cortex
Paracentral Lobule	R/L	4	-4, -40, 78	7.5	676
Cuneus	R/L	18	8, -78, 28	6.8	578
Precuneus	R/L	7	-2, -70, 60	6.6	113
Posterior Cingulate Gyrus	L	23	-4, -42, 32	5.8	254
Calcarine Sulcus	L	17	-16, -68, 6	4.6	41
Subcortical & Cerebellar
Cerebellum	L		-42, -80, -48	8.2	1468
Cerebellum	R		22, -84, -42	7.1	1182

^**Peak activation is significant when corrected for multiple comparisons across the whole brain using Family Wise Error (FWE), p<.05

^*Peak activation is significant when corrected for multiple comparisons (FEW, p<.05) within the region of interest using the Small Volume Correction (SVC) tool in SPM

^#Peak activation is significant when corrected for multiple comparisons with False Discovery Rate, p<.05 within the region of interest using the Small Volume Correction (SVC) tool in SPM

Note: Cluster volume listed in parentheses indicates that the activation is incorporated into the bigger cluster directly above

Figure 2.

Figure 2 shows neural activation the two contrasts of interest at threshold t(14) = 3.8, p<.001. The right sagittal view (top panel) shows activation for both conditions in the superior temporal cortex-temporoparietal junction (TPJ) and the lateral orbital cortex (LOFC). The superior temporal sulcus (STS) activation is visible on the Social vs. No Change slide. The STS activity for the Physical vs. No Change condition was more lateral and is not shown at this slice. The axial view shows the medial prefrontal cortex activity as well as the IFG on the left.

2.2.2. Social Change vs. No Change

As expected, we found greater neural activity for Social Change vs. No Change (Social Stories) in both mentalizing and MNS regions. There was activity in the superior temporal cortex bilaterally. The strongest activation was in the TPJ, specifically the posterior ascending segment of the STS and extending to the angular gyrus. There was a separate cluster in the STS, anterior segment. In addition, the middle temporal gyrus was active bilaterally. Activity in the inferior frontal gyrus and ventrolateral premotor region extended from the lateral orbital frontal cortex (LOFC (BA 47) to the inferior frontal gyrus – triangularis. All significant brain activations are listed in Table 2 and shown in Figure 2.

Table 2.

Brain regions that showed significantly greater activity for the Social Change condition as compared to the same stories in the No Change condition. P<.001, cluster 10

Brain Region: Social Change > No Change	R/L	BA	MNI Coordinates X, Y, Z	T value	Volume in Voxels3
Frontal Cortex
Lateral Orbital Frontal Cortex (multiple peaks in IFG)	L	47	-42, 34, -12	8.4	1123 *
Anterior Insula/Lateral Orbital Frontal Gyrus (multiple peaks in IFG)	R	47	42, 22, -12	7.5	782*
Precentral Gyrus – Superior Portion	R	4/6	16, -28, 62	4.3	13
Superior Frontal Gyrus (DMPFC)	R/L	10	2, 58, 30	10.1	4917 **
Temporal Cortex
Posterior Superior Temporal Sulcus (pSTS) – Temporoparietal Junction (TPJ)	L	39	-46, -58, 22	8.4	1522 *
Posterior Superior Temporal Sulcus (pSTS) – Temporoparietal Junction (TPJ)	R	39	46, -66, 32	10.2	1806**
Middle Temporal Gyrus (extends to midSTS)	R		64, -32, -12	7.4	480**
Superior Temporal Sulcus – Middle	R	21/22	58, -32, 2	6.8	(480)**
Middle Temporal Gyrus	R	21	66, -12, -18	5.4	177 #
Middle Temporal Gyrus	L	21	-64, -38, -6	5.	100 #
Temporal Pole	L	38	-50, 20, -28	8.6	1611
Hippocampus	R	20	34, -22, -6	5.4	67
Hippocampus	L	20	-34, -24, -8	5.3	68
Parietal & Occipital Cortex
Paracentral Lobule	R/L	4	2, -38, 78	5.4	199
Lingual Gyrus	L	18	-16, -48, -2	7.73	833
Subcortical & Cerebellar
Thalamus	L		-14, -8, 16	5.03	46
Thalamus	R		10, -2, 10	4.6	66
Putamen	L		-18, 6, 14	4.4	18
Cerebellum	L		-24, -78, -46	8.52	1286
Cerebellum	R		26, -74, -36	6.7	1394

^**Peak activation is significant when corrected for multiple comparisons across the whole brain using Family Wise Error (FWE), p<.05

^*Peak activation is significant when corrected for multiple comparisons (FWE, p<.05) within the region of interest using the Small Volume Correction (SVC) tool in SPM

^#Peak activation is significant when corrected for multiple comparisons with False Discovery Rate, p<.05 within the region of interest using the Small Volume Correction (SVC) tool in SPM

Note: Cluster volume listed in parentheses indicates that the activation is incorporated into the bigger cluster directly above

Because the Social Change and Physical Change conditions have different story scenarios, these two conditions are not compared directly.

2.3. Correlation of Neural Activity and Self-Reported Empathy

To investigate the relationship between neural activity during the analysis of social signals and self-reported empathy in daily life, we correlated neural activity in each contrast with the sum scores for cognitive empathy and affective empathy as measured by the Interpersonal Reactivity Index. Cognitive Empathy consists of the subscales Perspective Taking (the tendency to cognitively imagine a situation from the other person's point of view) and Fantasy (the tendency to relate to characters in novels and movies). Affective empathy consists of the Empathic Concern (tendency to feel sympathy for someone else's misfortune) and Personal Distress (tendency to feel negative emotions, particularly in stressful situations).

2.3.1. Physical Change vs. No Change Correlated with Empathy

Neural activity during the Physical Change condition was not correlated with empathy in a priori identified MNS or mentalizing regions. However, neural activity was correlated with both Cognitive and Affective Empathy in regions associated with visual processing of faces and body parts. Physical Change vs. No Change (Physical Stories) was significantly correlated with Cognitive Empathy in the right middle temporal gyrus. The MTG activation is at the juncture of the inferior temporal gyrus and the middle occipital gyrus. This region is highly selective for human body parts and is referred to as the extrastriate body area (EBA) (Downing et al., 2001). Activity for Physical Change (vs. No Change) correlated with Affective Empathy scores in the fusiform gyrus bilaterally, the right thalamus, and the right putamen. The fusiform gyrus is highly responsive to faces (Kanwisher, 2001). The thalamus and putamen are commonly active during emotion processing. Table 3 lists all regions in which neural activity significantly correlated with Cognitive and Affective Empathy. Brain regions which correlated with each subscale are listed in Supplemental Table 1.

Table 3.

Brain regions in which greater activity for the Physical Change > No Change correlated with Cognitive and Affective Empathy.

Physical Change > No Change	R/L	BA	MNI Coordinates X, Y, Z	T value	Volume in Voxels
Physical Change > No Change Correlation with Cognitive Empathy
Superior Frontal Gyrus	R	6	20, -2, 58	5.18	34
Middle Temporal Gyrus	R	37	60, -64, -2	5.1	37
Physical Change > No Change Correlation with Affective Empathy
Fusiform Gyrus	R	19	30, -68, -16	4.8	74
Fusiform Gyrus	L	19	-22, -62, -16	5.7	72
Thalamus	R		20, -14, 6	5.6	29
Putamen	R		22, 12, -6	4.9	54

Additional analyses revealed correlations in target regions that were just below statistical threshold. The left IFG (Peak x, y, z, coordinates: -46, 12, 26, t = 3.38, p<.005) and the left primary somatosensory cortex (Peak x,y,z: -54, -10, t, = 3.5, p<.005) were both correlated with Cognitive Empathy. The right STS was correlated with Affective Empathy (Peak x, y, z, coordinates: 60, -32, -4, t = 3.5, p<.005). These findings suggest that activity in MNS and mentalizing regions when viewing social cues – even in the absence of a change of mental state – is weakly related to empathy.

2.3.2. Social Change vs. No Change correlated with Cognitive and Affective Empathy

Neural activity during the Social Change condition vs. No Change (Social Stories) was strongly related to Cognitive Empathy. Neural activity in only two regions was related to Affective Empathy.

Specifically, neural activity in both MNS and mentalizing regions in response to Social Change was related to Cognitive Empathy, including the left inferior frontal gyrus-opercularis (BA 44), left postcentral gyrus, left supramarginal gyrus (BA40), and right STS anterior segment. In addition, activity in the ventral portion of the MPFC as well as the bilateral middle temporal gyrus was correlated with Cognitive Empathy. Neural activity in the right precentral gyrus (BA6, 44) and superior parietal lobe (BA 7) during Social Change perception was correlated with Affective Empathy. Surprisingly, activity in the TPJ did not show a significant correlation with Cognitive Empathy. Further analysis revealed that neural activity in the TPJ was significantly correlated with the Perspective Taking subscale but it was not correlated with the Fantasy subscale. (See Supplemental Table 1). Table 4 lists brain regions with significant correlations with Cognitive and Affective Empathy. Data is shown in Figures 3 and 4.

Table 4.

Brain regions in which greater activity for the Social Change > No Change correlated with Cognitive and Affective Empathy.

Social Change – No Change Social	R/L	BA	MNI Coordinates X, Y, Z	T value	Cluster Size
Social Change vs. No Change Correlated with Cognitive Empathy
Frontal Cortex:
Inferior Frontal Gyrus – Opercularis	L	44	-50, 10, 12	4.2	16
Anterior Insula	L	48	-38, 2, 8	5.6	50
Medial Orbital Frontal Cortex (MOFC)	R	11	4, 24, -24	5.9	199
Medial Orbital Frontal Cortex (MOFC)	L	11	-6, -54, -14	5.2	77
Medial Orbital Frontal Cortex (MOFC)	L	11	-10, 32, -16	4.4	43
Middle Frontal Gyrus (DLPFC)	L	6	-18, 18, 42	4.8	95
Supplementary Motor Area	R	6	14, -54, 4	5.3	68
Temporal Cortex
Superior Temporal Sulcus	R	21,22	52, -28, 0	5.2	25 #
Middle Temporal Gyrus	R	37	50, -64, 0	4.2	13
Middle Temporal Gyrus	R	37	42, -54, 4	5.4	49
Middle Temporal Gyrus	L	37	-44, -56, 0	7.7	178*
Hippocampus	L		-28, -40, 4	5.8	457
Parietal & Occipital Cortices
Supramarginal Gyrus (Somatosensory Related Cortex (SRC))	L	40	-44, -28, 28	5.9	89
Postcentral Gyrus (SCI)	L	3	-20, -30, 48	5.5	87
Postcentral Gyrus (SCI)	L	3	-20, -40, 54	4.4	21
Middle Cingulate Gyrus	R	23	16, -30, 40	4.5	23
Subcortical & Cerebellar
Thalamus	R		20, -26, 8	5.7	75
Social Change vs. No Change Correlated with Affective Empathy
Precentral Gyrus	R	6,44	46, 2, 28	5.1	24
Middle Frontal Gyrus	L	9, 46	-24, 56, 34	4.7	28
Superior Parietal Lobe – Intraparietal Sulcus	L	7	-16, -70, 48	7.5	51

^*Peak activation is significant when corrected for multiple comparisons (FWE, p<.05) within the region of interest using the Small Volume Correction (SVC) tool in SPM

^#Peak activation is significant when corrected for multiple comparisons with False Discovery Rate, p<.05 within the region of interest using the Small Volume Correction (SVC) tool in SPM

Figure 3.

Regions in which greater neural activity in the Social Change condition versus the No Change condition was related to more self-reported Cognitive Empathy (sum of Perspective Taking and Fantasy subscales) as measured by the Interpersonal Reactivity Index. A). Right superior temporal sulcus (STS); B) Left somatosensory related cortices (SRC) in the supramarginal gyrus (BA 40) and left inferior frontal gyrus – opercularis (BA 44); C). Bilateral middle temporal lobe – a region consistent with the extrastriate body area (EBA).

Figure 4.

Greater neural activity in the precentral gyrus in response to Social vs. No Change condition positively correlated with self-reported Affective Empathy.

3. Discussion

We conducted an experiment to identify whether neural regions associated with MNS and mentalizing supported different components of empathy. The aim of the Social Change condition, in which a change in direction of attention resulted in a change in belief and emotional state, was to maximally activate both mentalizing and MNS systems and then identify whether neural activity was related to cognitive and affective empathy, respectively. We predicted that when processing social cues indicating a change in belief state and emotional state, neural activity in MNS regions, such as the IFG, precentral gyrus and somatosensory related cortices, would be sensitive to the change in emotional state and would therefore be related to affective empathy; whereas, neural activity in mentalizing regions, such as the STS, TPJ, and MPFC, would be sensitive to the change in belief state and would therefore be related to cognitive empathy. Because the social cues in the Physical Change condition (i.e. the shift in head and body posture) did not result in a change in belief state or emotional state, we predicted that MNS and mentalizing activity would not be related to empathy in this condition.

Results showed that across the group of participants, both the Social Change and Physical Change conditions produced robust activation in neural regions associated with mentalizing, including the STS, TPJ, and MPFC, as well as regions associated with MNS, such as the inferior frontal gyrus. Importantly, when detecting mental and emotional state change (i.e. the Social Change condition), neural activity in MNS (IFG and SRC) and mentalizing regions (STS) was strongly related to cognitive empathy. There was no significant relationship between neural activity in mentalizing regions and affective empathy; neural activity in one MNS region, the precentral gyrus, was related to affective empathy. As expected, when detecting a physical change (i.e. biological motion without change in mental state) there was no relationship between neural activity in MNS and mentalizing regions and cognitive or affective empathy. Interestingly, during both the Social and Physical Change conditions, neural activity in regions associated with face and body perception was related to both cognitive and affective empathy.

Overall, the results from the Social Change condition suggest three main conclusions: First, the results indicate that task-related neural activity associated with mentalizing, such as representing someone else's belief state, predicts the tendency to use cognitive empathy skills, such as perspective-taking, in daily life. The findings further suggest that task-related neural activity associated with mirroring, such as the internal modeling of motor and somatosensory representations of emotion, also predicts the tendency to engage in cognitive empathy. Secondly, the overall pattern of results suggests that, in healthy adults, neural activity in both MNS and mentalizing regions support both cognitive and affective empathy. While our data provide partial support for the hypothesis that MNS activity is related to affective empathy, we found that activity in both MNS and mentalizing regions were related to cognitive empathy. Thus a clear dissociation was not evident. Finally, the current results suggest that neural response to perceptual aspects of non-verbal social cues, such as the visual processing of bodies, is related to empathy.

The conclusions that can be drawn from the Physical Change condition are limited. The aim of this condition was to investigate whether the processing of social cues, such as shifts in head and body direction, without a change in mental state would be related to empathy. Although there was no significant relationship between activity in mentalizing and MNS regions in this condition and empathy, there was a relationship detected just below statistical threshold. Thus, it appears that both the Physical and Social conditions engaged mentalizing and MNS processing, but the Social Change condition drove the neural processing to a greater extent. Furthermore, subjects were slower to respond in the Physical Change condition, suggesting that it was more difficult to decipher whether a mental state change had taken place. Therefore, even if the mentalizing and MNS regions are more active during the representation of a new belief and emotional state, the Physical Change condition may have shown more than expected activity due to additional attention and processing of cues that could (but ultimately did not) indicate a change. Because of these concerns, we focus the majority of our discussion of the findings on the Social Change condition.

The current findings, particularly those from the Social Change condition, add to a growing literature that when using social cues to understand the mental and emotional state of another person, neural response in the IFG, SRC, and STS is related to empathic tendencies (Keysers and Gazzola, 2007; Pelphrey and Carter, 2008; Saxe, 2006; Shamay-Tsoory et al., 2006; Shamay-Tsoory et al., 2009; Singer, 2006; Tankersley et al., 2007). The extant neuroimaging literature, including the current study, suggests that, in healthy adults, the computational processes supported by these regions simultaneously contribute to multiple components of empathy. Here, we show that when processing social cues related to mental and emotional state change neural activity in the left inferior frontal gyrus-opercularis, and left supramarginal gyrus, and right superior temporal sulcus, significantly correlated with self-reported tendency to engage in cognitive empathy strategies. Cognitive empathy was defined here as the tendency to engage in perspective-taking (PT subscale) and relate to characters in stories, novels, and movies (FS subscale). Because our task incorporated multiple social cues, including body movement, emotional expression, and social context, to indicate a change in mental and emotional state, it is impossible to know exactly what aspect of the stimuli was being processed and what computation was being performed in each neural region. Nonetheless, it is clear that neural activity in response to these social cues is related to self-reported empathy.

Our data is consistent with the idea that activity in the IFG and SRC represents internal modeling (or ‘mirroring’) of emotional state and that the creation of a motor and somatosensory representation is related to both affective and cognitive empathy in daily life. This relationship between neural activity in MNS regions and empathy is apparent whether or not research participants are instructed to use simulation strategies such as imitation. For example, neural activity in the IFG when instructed to imitate facial expressions is related to the total score of the IRI as well as the Fantasy, Personal Distress, and Empathic Concern subscales (Pfeifer et al., 2008). However, tasks that involve the identifying emotional state without instructing simulation or imitation, also demonstrate a relationship between IFG and SRC activity and empathy. Neural activity in the IFG (as well as the STS) when identifying one's own as well as another person's emotional response to observed facial expressions was significantly correlated with affective empathy as measured by the Balanced Emotional Empathy Scale (BEES) and the Empathic Concern subscale of the IRI (Schulte-Ruther et al., 2007). Neural activity in the IFG (extending to the insula) during the observation of pleasant and unpleasant tastes was related to FS, PD and PT subscales (Jabbi et al., 2007). Neural activity in the IFG and SRC when inferring what a person's emotional response would be if their beliefs changed was related to a composite score of PT, FS and EC (Hooker et al., 2008). In addition, the amount of activity in the IFG and SRC when perceiving non-emotional actions, such as hand and mouth actions related to eating, varied with self-reported PT scores (Gazzola et al., 2006). Together these findings suggest a broad role for the MNS system in the internal modeling of actions, intentions, and emotions; it further suggests that the strength of those internal models (reflected in greater neural activity) and/or the tendency to spontaneously create those internal models predicts perspective-taking and empathic response in interpersonal relationships. This interpretation is consistent with the notion that cognitive empathy skills, such as perspective-taking, relies on the use of internal models of observed action in order to understand another person's situation (Gallese, 2007).

In addition, our findings support research showing that STS sensitivity to social cues is related to the ability to understand and use social cues to connect and communicate with others in daily life (Pelphrey and Carter, 2008). The STS is involved in multiple functions, including the analysis of biological motion cues (Allison et al., 2000), biological and non-biological attention cues (Mitchell, 2008), the perception of agency (Castelli et al., 2000), and the representation of another person's belief state or intentional stance (Frith and Frith, 2006; Gallagher and Frith, 2003). Though it is difficult to know what function the STS was performing during our task, the data is consistent with the idea that the STS is not only responsive to low-level perceptual cues, such as hand motion, but rather that this region uses multiple types of social cues to develop a representation of another person's mental state.

Interestingly, we also found evidence that in response to both Social and Physical Change conditions activity in neural regions associated with face and body perception were correlated with empathy. In particular, activity in a region of the posterior inferior temporal lobe during social and physical change was related to cognitive empathy. This region is consistently active when perceiving human body parts, such as body gestures and isolated hands, arms, feet and legs; it is more selective for bodies than for faces or objects and is referred to as the ‘extrastriate body area’ (EBA) (Downing et al., 2001; Peelen and Downing, 2007). Although it was initially thought that this region is responsive to perceptual aspects of body parts, research indicates that higher level processing might occur as well. For example, the EBA is more sensitive to allocentric views than for egocentric views (Saxe et al., 2006), suggesting a role for the EBA in self/other distinctions which is an important component of perspective-taking. The EBA is also active during the execution of actions, suggesting that this region is not specifically perceptual but that it may support the internal representations of actions (Astafiev et al., 2004). One study has investigated the involvement of the EBA in empathic process. This study focused on empathy for pain and concluded that the EBA is not involved in empathy for pain (Lamm and Decety, 2008).

However, in our study, body parts and changes in body position provided important information about mental and emotional state. Thus the association between EBA activity and cognitive empathy could reflect the fact that people with empathic tendencies have greater sensitivity in neural regions dedicated to social signal processing, such as the EBA, or that people with empathic tendencies are more likely to pay attention to body gestures that may provide communicative signals or clues to mental state.

We also found that activity in the fusiform gyrus during the physical condition was related to affective empathy. The fusiform gyrus is involved in face perception and is modulated by facial emotion (Vuilleumier and Driver, 2007). Facial affect was an important aspect of the current paradigm. Although head and body posture changed in Physical Change condition, facial expression did not. Therefore facial expression was a primary clue which indicated that the person's feelings and beliefs did not change. The fact that the relationship between fusiform activity and affective empathy occurred in the physical change condition but not the Social Change condition could reflect the greater scrutiny of facial expressions in the Physical Change condition.

In addition, during the physical change condition activity in the thalamus and putamen was also correlated with affective empathy. These regions are often active during emotion perception and emotion induction studies (Phan et al., 2002; Phan et al., 2004). This finding is consistent with the interpretation that participants were paying attention to the emotional content of the scene in order to identify that no change in emotion had taken place. It further suggests that activity in emotion related areas during emotional information processing is related to self-report of affective empathy.

Although the current data do not support the hypothesis that cognitive and affective empathy are dissociable systems, the regions which showed relative specificity in our study are consistent with those in lesion studies which showed a double dissociation between cognitive and affective empathy (Shamay-Tsoory and Aharon-Peretz, 2007; Shamay-Tsoory et al., 2009). We found that activity in the inferior precentral gyrus (BA 6) when detecting change in mental and emotional state was correlated with affective empathy. Activity in this region did not show a correlation with cognitive empathy. These findings are consistent with recent evidence showing that lesions in the ventral motor and premotor cortex, such as BA6 and BA 44, are associated with both emotion recognition deficits and lower than normal self-reports of affective empathy (Shamay-Tsoory et al., 2009). In addition, consistent with results showing that ventromedial prefrontal cortex lesions were associated with deficits in theory of mind skills and cognitive empathy (Shamay-Tsoory et al., 2009), we found that during the identification of mental state change, activity in multiple regions of the medial orbital frontal cortex were correlated with cognitive empathy.

Limitations and Caveats

There are several limitations of the current paradigm that could be addressed in future research. First, although the social and visual complexity is well controlled between the two stimuli in the Social Change vs. No Change (Social Stories) contrast, the Social Change condition contains apparent motion while the No Change condition does not. Thus, apparent motion, particularly biological motion, within a social context might be enough to drive the neural activity and correlation patterns observed in this study. While the results from the contrast Physical Change vs. No Change (Physical Stories) suggests that the relationship between activity in MNS and mentalizing regions and empathy is not driven by biological motion alone, this should be interpreted with caution, since the specific amount of biological motion was not equated in the two conditions. Furthermore, because the social scenarios in the Social and Physical Change conditions were different, a direct comparison between them is problematic. To more specifically identify whether the representation of affective state in MNS regions (and not just biological motion) is the key phenomenon related to empathy, future research should try to control all aspects of biological motion and isolate the change in affective state.

Nonetheless, results from the Social Change condition shows that during the analysis of multiple social cues indicating a change in mental and emotional state, neural activity in a broad network of regions is related to self-reported empathy in daily life. Still, it is important to note that the direction of this relationship is not clear. One possibility is that genetically determined neural sensitivity in specific regions, such as the IFG, SRC and STS, may cause a person to be more empathic. In other words, neural responsivity in mentalizing and MNS regions may facilitate the analysis of social cues and the internal representations of mental and emotional states. Because of this hardwired circuitry, such people are more likely to develop and use empathy related skills, like perspective-taking, and are more likely to have an affective response to the distress of others. On the other hand, the current results could reflect the fact that adults who have already developed empathy skills are more likely to pay attention to and analyze social cues in the stimuli and/or are more likely to use mentalizing and MNS strategies to complete the task. It will be important for future investigations to decipher whether individual differences in neural architecture determines empathic tendencies or whether other factors determine empathic tendencies and these established tendencies are reflected in the activity of specific neural circuits.

4. Experimental Procedure

4.1 Participants

15 healthy, English speaking adults (8 males; mean age is 21 years old, age range is 18-25 years old) volunteered and were paid for their participation. The subjects were screened for MR compatibility, neurological, and psychiatric illness. Prior to participating, subjects gave written, informed consent in accordance with the guidelines at the University of California, Berkeley.

4.2 Task and Stimuli

Before beginning the task, subjects completed the Interpersonal Reactivity Index, a 28 item self-report questionnaire assessing empathy and emotional reactivity (Davis, 1983; Davis, 1996). The IRI has 4 subscales: Empathic Concern (EC), the tendency to feel compassion toward others; Perspective Taking (PT), the tendency to take the point of view of another person; Fantasy Scale (FS), the tendency to relate to fictional characters; and Personal Distress (PD), the tendency to feel negative emotion in stressful situations. Each question is answered using a 0 – 4 Likert scale.

Subjects completed a short practice task consisting of 2 trials of each condition, in a fixed random sequence. Subjects were told, “Your job is to pay attention to what, if anything, changes from the first image to the second image, and then to classify the change as ‘Social’, ‘Physical’, or ‘No Change’. By a Social Change, we mean a change in the characters' beliefs, feelings, or understanding of the situation. By a Physical Change, we mean a physical change that does not result in a change the characters' feelings or beliefs. Lastly, No Change is self-explanatory – it means no change has taken place from the first to the second image.”

In the scanner, each subject completed 4 event-related runs. Each run began with a rest period (20s), followed by a cue indicating that the task was about to begin (2s), followed by 29 trials (8s each) with a jittered 4, 6, or 8s inter-trial interval (ITI). Each trial consisted of 4s of the social scene image, followed immediately by 4s the Change or No Change image and was accompanied by the answer choices which were: Physical, Social, or No Change.

The stimuli were 38 visual, static social scenes, in which the characters have different beliefs about what is happening in the scene. At least one of the characters has full knowledge about what is taking place (i.e. a “True Belief”) and one of the characters has only partial knowledge or a misunderstanding (i.e. a “False Belief”). There are between two and five characters in each scene. All characters display facial expressions of emotion that are based on their belief of the situation. The content of the social scenarios was inspired from mentalizing tasks that currently exist in the literature (Fletcher et al., 1995; Gallagher et al., 2000) and modified for the purpose of this experiment.

Pilot testing was done to verify that the scenes were understandable and that the emotional expressions were identifiable. (Pilot subjects were University of California at Berkeley undergraduates who received course credit). The scenes in the Physical and Social conditions were roughly equivalent in the types of emotions and number of people presented in each scene. Across all characters, the Physical Scenes (prior to any change) had the following emotions represented: Angry (6 scenes); Afraid (4 scenes), Happy (15 scenes), Sad (2 scenes), “Other” including worried, upset, bored (4 scenes). The Social scenes (prior to any change) had the following emotions represented: Angry (4 scenes); Afraid (4 scenes), Happy (17 scenes), Sad (2 scenes), “Other” including worried, upset, bored (5 scenes).

All story scenarios are described in the supplemental material. Many scenes involve a character unaware that a positively or negatively valenced event is about to happen (e.g. a terrified mother sees that a car is about to hit her son who is happily riding his bike across the street, or a sad girl is about to open the door to her house and is unaware that a surprise birthday party awaits her). Other scenes involve misinterpretation (e.g. a wife blames her husband for breaking a vase when the culprit really is a young boy who is hiding from her view) or deception in which one character tricks another (e.g. a man puts an ‘I'm stupid’ sign on his friend's back but the friend does not see it).

After presentation of the initial social scenario, half of the 38 scenes have a Physical Change and half of the scenes have a Social Change. In the Physical Change condition, there is a shift in the direction of attention (head and body motion) of one of the characters but it does not result in a change in their mental state. For example, in one scene a man believes he is being held-up at gun point when really a friend is playing a trick on him by sticking the end of a banana against his back. The man with the false belief is scared and the friend with the full understanding is amused. When the change occurs, the man with a false belief shifts from looking to the side to looking straight ahead, but he still does not see that the person behind him is a friend. Thus there is a change in direction of attention but no change in mental state. In the Social Change condition, there is a shift in the direction of attention (head and body motion) of one of the characters which results in a change in their belief about the situation and their emotional response to it. For example, in one scene a family photo is being taken (everyone is smiling) and the little brother is making “bunny ears” with his hand behind his sister's head as a joke. In the first scene she is unaware of the joke, in the “change” scene, she turns her head, sees her brother, and her expression changes from happy to annoyed. The control condition is the corresponding scene with no change: i.e. No Change (Physical Scenes) and No Change (Social Scenes).

All the pictures were made with the Poser 4 animation program (Curious Labs, Inc., Scotts Valley, CA) and had a cartoon-like quality. Facial expressions were created by using Facial Action Coding (FACS) algorithms developed for use with the Poser program.

4.3. FMRI Image Acquisition

All images were acquired at 4 Tesla using a Varian INOVA MR scanner (Palo Alto, CA) that was equipped with echo-planar imaging. For all experiments, a standard radiofrequency (RF) head coil was used, and a memory foam pillow comfortably restricted head motion. E-Prime software (PST, Pittsburgh, PA) controlled the stimulus display and recorded subject responses via a magnetic-compatible fiber-optic keypad. An LCD projector (Epson, Long Beach, CA) projected stimuli onto a backlit projection screen (Stewart, Torrance, CA) within the magnet bore, which the subject viewed via a mirror mounted in the head coil.

Functional images were acquired during four fMRI sessions which began with 5 dummy scans (with no data acquisition) and 4 “blank screen” scans which were subsequently dropped from analysis to insure steady state magnetization for all analyzed data. Images were acquired with parameters used to optimize signal in regions susceptible to drop-out due to magnetic field inhomogeneity. Each volume acquisition included 40, 3.5 mm thick coronal slices with a .5mm inter-slice gap, with a phase encode direction oriented in the superior-inferior direction. A one-shot T2* weighted echo-planar image (EPI) sequence (TR= 2000ms, TE = 28ms, FOV = 22.4cm2, matrix size=64×64) was used to acquire blood-oxygenated dependent (BOLD) signal. EPI voxel size at acquisition is 3.5 × 3.5 × 4mm. A high-resolution 3D T1-weighted structural scan (MPFLASH sequence) and an in-plane low resolution T2-weighted structural scan (GEMS) were acquired for anatomical localization.

4.4. Data Processing and Analysis

MRI data was processed and analyzed using SPM2 software. Each EPI volume was realigned in space to the first scan, using a six parameter, rigid body, least-squares transformation algorithm. Subjects who showed more than 3mm of movement across the session were dropped from analyses. After realignment, we re-sliced the coronal EPI data to the axial plane, and smoothed the data 8mm (FWHM). We then created and estimated a general linear model (GLM), and created contrast images of the difference between neural activity for each comparison of interest. (GLM and data analysis is detailed below). These contrast images were coregistered to the individual subject's co-planar (GEMS) and high resolution (MPFLASH) anatomical images, resliced to 2 × 2 × 2 isotropic voxels and then normalized to the Montreal Neurological Institute (MNI) atlas space.

In the creation of the GLM, the hemodynamic response for each event was modeled from the onset of the second picture (i.e. the change picture) with response duration of 4 seconds. We defined each trial type as a covariate of interest: 1) Physical Change; 2) Social Change; 3) No Change (Physical Stories); and 4) No Change (Social Stories).

We convolved the canonical hemodynamic response function (hrf) with brain activity at the onset of the trial type. Individual movement parameters were added to the GLM as covariates of no interest. Data was high-pass filtered at 200s, scaled by the global mean, and corrected for serial autocorrelation. We computed the difference in neural activity between two trial types of interest and then computed whether this difference was significant across subjects by entering the contrast value into a one sample t-test. This whole-brain random effects analysis was thresholded at p<.001 (uncorrected for multiple comparisons) with a cluster size of 10 voxels (k = 10). We have two planned comparisons of neural activity: Physical Change versus No Change (Physical Stories); Social Change versus No Change (Social Stories). All activations for these two contrasts are listed in Table 1 and Table 2. In addition, we identified whether clusters of activation in regions of a priori interest were significant after correcting for multiple comparisons within the anatomical region by using the Small Volume Correction (SVC) tool in SPM. We specifically investigated two regions:1) the superior temporal cortex region – which included the middle temporal gyrus, superior temporal sulcus, and superior temporal gyrus for the right and left hemispheres; 2) the ventral premotor region – which included the inferior frontal gyrus, lateral orbital gyrus and anterior insula for the right and left hemisphere. Each region in each hemisphere was investigated separately.

The correlation analysis was performed in SPM2 using a simple regression in a whole brain analysis. We entered each subject's contrast file of the comparison of interest [e.g. Social Change vs. No Change (Social Stories)] and then entered each subject's score on the IRI (e.g. the Cognitive Empathy score) as a regressor in the SPM model. Cognitive and Affective Empathy correlations were done in separate analyses. We then identified regions in the whole brain analysis in which relatively greater activity in the contrast is significantly correlated with higher scores on the IRI. There are 4 subscales of the IRI: Empathic Concern (EC), Perspective Taking (PT), Fantasy Scale (FS), and Personal Distress (PD). The Cognitive Empathy score is the sum of the subscales PT and FS. The Affective Empathy score is the sum of the subscales EC and PD (Davis, 1983; Davis, 1996; Eisenberg et al., 1994; Eisenberg et al., 1995; Eisenberg, 2000).

After performing the whole-brain correlation analysis, we identified (by viewing the group correlation maps overlaid on a single subject template brain) whether or not neural activity in our a priori regions of interest (e.g. IFG, STS) was significantly correlated with empathy at our defined threshold (p<.001, uncorrected). If there was a cluster of voxels which showed a significant correlation in an a priori region, we then extracted individual contrast values from that cluster and identified whether or not the correlation was driven by outliers. An outlier was defined as +/- 2.5 standard deviations from the mean on either the empathy score or neural activity.

Anatomical locations of the clusters was identified by consulting multiple neuroanatomy sources including MRI based software (e.g. MRIcro: http://www.sph.sc.edu/comd/rorden/mricron/), online anatomy tools (e.g. http://spot.colorado.edu/∼dubin/talks/brodmann/brodmann.html) and neuroanatomy atlas books (e.g. “The Human Brain”, by H.M. Duvernoy).