Skip to main content
NCBI home page
Brain and Behavior logoLink to Brain and Behavior
. 2021 May 5;11(6):e02140. doi: 10.1002/brb3.2140

Temporal lobe epilepsy alters neural responses to human and avatar facial expressions in the face perception network

Lorena Chantal Kegel 1,2,✉, Sascha Frühholz 2, Thomas Grunwald 1, Dieter Mersch 3, Anton Rey 4, Hennric Jokeit 1,2
PMCID: PMC8213650  PMID: 33951323

Abstract

Background and Objective

Although avatars are now widely used in advertisement, entertainment, and business today, no study has investigated whether brain lesions in neurological patients interfere with brain activation in response to dynamic avatar facial expressions. The aim of our event‐related fMRI study was to compare brain activation differences in people with epilepsy and controls during the processing of fearful and neutral dynamic expressions displayed by human or avatar faces.

Methods

Using functional magnetic resonance imaging (fMRI), we examined brain responses to dynamic facial expressions of trained actors and their avatar look‐alikes in 16 people with temporal lobe epilepsy (TLE) and 26 controls. The actors' fearful and neutral expressions were recorded on video and conveyed onto their avatar look‐alikes by face tracking.

Results

Our fMRI results show that people with TLE exhibited reduced response differences between fearful and neutral expressions displayed by humans in the right amygdala and the left superior temporal sulcus (STS). Further, TLE was associated with reduced response differences between human and avatar fearful expressions in the dorsal pathway of the face perception network (STS and inferior frontal gyrus) as well as in the medial prefrontal cortex.

Conclusions

Taken together, these findings suggest that brain responses to dynamic facial expressions are altered in people with TLE compared to neurologically healthy individuals—regardless of whether the face is human or computer‐generated. In TLE, areas sensitive to dynamic facial features and associated with processes relating to the self and others are particularly affected when processing dynamic human and avatar expressions. Our findings highlight that the impact of TLE on facial emotion processing must be extended to artificial faces and should be considered when applying dynamic avatars in the context of neurological conditions.

Keywords: avatar, emotion, facial expression, perception, temporal lobe epilepsy

The aim of our event‐related fMRI study was to compare brain activation differences in people with epilepsy and controls during the processing of fearful and neutral dynamic expressions displayed by human or avatar faces. Our fMRI results show that people with TLE exhibited reduced response differences between human and avatar fearful expressions in the dorsal pathway of the face perception network (STS and inferior frontal gyrus) as well as in the medial prefrontal cortex. Taken together, these findings suggest that brain responses to dynamic facial expressions are altered in people with TLE compared to neurologically healthy individuals—regardless of whether the face is human or computer‐generated.

graphic file with name BRB3-11-e02140-g003.jpg

1. INTRODUCTION

Advances in the development and animation of computer‐generated characters have led to the increased usage of anthropomorphic characters in digital applications and communication technologies (Miller, 2007). Accordingly, computer‐generated characters, or avatars, have also become popular for clinical and research settings as a complement to existing communication, assessment, and therapy options (Bohil et al., 2011; Bombari et al., 2015). As such, there have been initial studies examining the use of human‐like avatars in the assessment and training of patients with neurological conditions (Aljaroodi et al., 2017; Boucenna et al., 2014; Georgescu et al., 2014; Javor et al., 2016; Robitaille et al., 2017; Schilbach et al., 2007). When avatars are used in such settings, they—like humans—can accompany and influence interactions with facial expressions and thereby transmit social information. However, we do not know how flexibly we react to and integrate virtual non‐conspecifics into our social environment. What are the costs in terms of intensity and effort of emotional exchange in human‐avatar interactions compared to interactions between humans? This missing knowledge together with the increased exposure to avatars motivates the present investigation of the perception of humans and avatars. In particular, it is unclear how mesial temporal brain areas that play an eminent role in the processing of affective stimuli respond to these newly existing interaction partners. For this reason, it is essential to investigate whether lesions within the temporal lobe, such as those exhibited by individuals with temporal lobe epilepsy (TLE), impact the response. Determining how human and avatar faces are processed in the brain when TLE is present may provide significant insights about the importance of the affected brain regions. Hence, in the present study, we investigate whether brain responses to dynamic expressions displayed by human and avatar faces differ between people with TLE and neurologically healthy people.

In TLE, lesions in the amygdala, the hippocampus, or lateral temporal areas are associated with extensive structural and functional alterations in the temporal lobe and extratemporal regions such as frontal cortex (Bernhardt et al., 2013; Engel & Salamon, 2015; Jokeit et al., 1997). These changes encompass the network that is engaged during facial emotion perception and could thus be associated with impairments in the processing and recognition of emotions in people with TLE (Ives‐Deliperi & Jokeit, 2019; Jokeit et al., 2018; Milesi et al., 2014; Monti & Meletti, 2015; Schacher, Winkler, et al., 2006). In this neural face perception network, the superior temporal sulcus (STS) and the inferior frontal gyrus (IFG) belong to the dorsal pathway, which is sensitive to dynamic facial features such as facial motion and gaze. In addition, the inferior occipital gyrus (IOG), the fusiform gyrus (FG), and the anterior temporal lobe (ATL) form the ventral pathway sensitive to invariant facial features such as form and configuration. Moreover, the amygdala plays a central role in the processing of emotional facial expressions by contributing to the fast detection and evaluation of salient signals in our environment (Adolphs, 2001; LeDoux, 2000). This role is highlighted by amygdalar feedback connections to the dorsal and ventral pathway in the face perception network, which enable a modulatory effect on cortical face processing (Furl et al., 2013; Haxby et al., 2000; Vuilleumier, 2005). Together, this extended network forms the neural basis for the processing of facial expressions and facial identity (Duchaine & Yovel, 2015; Haxby et al., 2000).

In line with the above‐mentioned findings, previous research has reported that people with TLE show altered activity in face‐sensitive cortical and subcortical areas in response to human facial expressions. Accordingly, it has been shown that people with TLE displayed smaller responses in the amygdala, the occipital fusiform gyrus, the FG, and the posterior part of the STS than controls in response to dynamic fearful expressions (Åhs et al., 2014; Ives‐Deliperi et al., 2017; Labudda et al., 2014; Riley et al., 2015; Schacher, Haemmerle, et al., 2006; Toller et al., 2015; Vuilleumier et al., 2004). Furthermore, people with TLE showed extensive alterations of functional connectivity in distributed areas subserving facial emotion processing in contrast to controls (Broicher et al., 2012; Riley et al., 2015; Steiger & Jokeit, 2017). This highlights the importance of the affected regions in TLE and their influence on the whole‐brain network subserving facial emotion processing (Ives‐Deliperi & Jokeit, 2019).

Based on the evidence reported above, we may conclude that there are differences regarding the processing of dynamic human expressions between people with and without TLE. However, no previous study has tested whether response differences in the amygdala and the face perception network also translate to the processing of dynamic expressions of avatars. Recent evidence with neurologically healthy individuals suggests that areas in the dorsal pathway of the face perception network show stronger responses to human expressions than to avatar expressions. This has been shown for the STS and the IFG that are sensitive to dynamic features of faces and thus may show stronger responses to natural facial motion than to artificial facial motion (Duchaine & Yovel, 2015; Haxby et al., 2000; James et al., 2015; Kätsyri et al., 2020; Kegel et al., 2020; Sarkheil et al., 2013). Further, differences between dynamic human and avatar faces have so far only been reported for fearful expressions and not for neutral expressions (Kegel et al., 2020). Based on this, we may assume that emotional expressions exert a significant influence on human and avatar face processing in dorsal temporal areas, possibly via amygdalar‐cortical feedback connections (Furl et al., 2013). How TLE and associated structural and functional alterations in the temporal lobe and beyond may further affect this processing is unknown.

Hence, in the current study, we examined response differences to dynamic human and avatar expressions in people with TLE and controls with whole‐brain fMRI. Drawing on previous findings of the processing of human faces in TLE, we hypothesized that people with TLE would show overall attenuated brain responses to fearful human expressions versus neutral human expressions when compared to controls. We expected this response pattern to be present in dorsal and ventral areas of the face perception network as well as in the amygdala. Regarding response differences between human and avatar expressions, we expected that structural and functional alterations in people with TLE would affect the processing of both stimulus types. Therefore, we assumed a smaller response difference between human and avatar expressions for people with TLE compared to controls. Taking into account previous results with avatar faces, we assumed that such group differences would mainly occur in dorsal areas of the face perception network sensitive to dynamic features of faces.

2. MATERIALS AND METHODS

2.1. Sample

We examined 17 people with TLE and 30 controls that reported no diagnosed psychiatric or neurological disorders. People with TLE were recruited at the Swiss Epilepsy Center in Zurich. The main inclusion criterion was focal seizures originating in one or both temporal lobes. This criterion had been confirmed by ictal video‐EEG and the seizure type recorded during previous in‐patient stays at the center. In two people with TLE, this criterion was confirmed by interictal EEG and the seizure type reported by the affected person and/or an eyewitness, as both had not been examined as inpatients. Consequently, it was not possible to lateralize the seizure origin in these two people with TLE and both were only included for analyses of activation differences between the control group and the entire TLE group (regardless of seizure origin, see Section 2.4). The TLE diagnoses were made by epileptologists at the Swiss Epilepsy Center.

The control group was recruited via online advertising on a local community website and in‐house advertising targeted at the staff of the Swiss Epilepsy Center. All participants had to be able to follow and understand the information and study procedure (i.e., no language barrier, severe cognitive deficit, or psychiatric disease). Their vision was required to be normal or corrected to normal and all participants had to fulfill standard MRI safety criteria. All procedures as well as the study design were approved by the local ethics committee and participants were tested only following written informed consent in accordance with the Declaration of Helsinki.

During the preprocessing of the data (see Section 2.4), we had to exclude one participant from the TLE group due to severe atrophy of the left brain hemisphere which caused the preprocessing to fail. From the control group, two participants had to be excluded from final analyses due to excessive movement (>2 mm in either x‐, y‐, or z‐direction), one due to insufficient task engagement (verified by our control task described in Procedure and Stimuli), and one due to discomfort that led to the termination of the scanning session. This resulted in a sample of 16 people with TLE and 26 controls. Please see Table 1 for sociodemographic and clinical characteristics of the sample and Section 3.1 for analysis of group differences.

TABLE 1.

Sociodemographic and clinical characteristics of participants with and without temporal lobe epilepsy

Control group n = 26 TLE group n = 16 Left TLE group n = 7 Right TLE group n = 7
Sociodemographic characteristics
Gender (m/f) 13/13 4/12 2/5 2/5
Age in years, Mdn (range) 38.6 (18–62) 47.6 (21–64) 48.4 (21–58) 47.8 (28–64)
Years of full‐time education, Mdn (range) 13 (9–21) 12 (9–15) 12 (9–15) 13 (10–14)
Clinical characteristics
Age at epilepsy onset in years, Mdn (range) 22.5 (5–62) 17 (6–35) 22 (5–62)
Duration of epilepsy in years, Mdn (range) 20.5 (2–42) 27 (7–42) 19 (2–35)
Number of antiepileptic drugs per day, Mdn (range) 2 (0–3) 2 (0–3) 2 (1–2)
Hippocampal sclerosis (yes/no) 11/5 7/0 4/3
Open in a new tab

The analyses of group differences regarding the sociodemographic and clinical characteristics were not significant (all p > .05).

Abbreviations: f, female; m, male; Mdn, median; TLE, temporal lobe epilepsy.

2.2. Procedure and stimuli

We used an event‐related fMRI protocol presenting videos of actors' facial expressions and their avatar look‐alikes to measure blood‐oxygen‐level‐dependent (BOLD) responses associated with facial emotion processing. Participants completed 208 trials with videos of human and avatar faces showing fearful and neutral expressions, as well as scrambled versions of these videos (see Figure 1 and Videos S1 and S2 online). Furthermore, control videos with a red square centered on the displayed face or scrambled pattern were infrequently presented to which participants had to respond with a button press. The 208 trials were divided into two runs, so that each run consisted of 32 videos of human expressions (16 each fearful and neutral), 32 videos of avatar expressions (16 each fearful and neutral), 32 scrambled videos, and 8 control videos.

FIGURE 1.

FIGURE 1

Open in a new tab

Illustration of a female and a male actor (top panels) and their corresponding avatars (bottom panels) displaying neutral expressions

The videos lasted 3 s and were randomly presented with Cogent 2000 (version: 1.32; http://www.vislab.ucl.ac.uk/cogent.php, RRID: SCR_015672) under MATLAB (version 2015a; https://ch.mathworks.com/products/matlab.html; RRID: SCR_001622). In the scanner room, the videos were projected onto a projection screen situated at the front of the MRI bore (image width 103.5 cm, image height 85.5 cm). Participants were able to see the videos on the projection screen through a mirror attached to the head coil (visual angle of the faces: 7° (horizontal), 8.5° (vertical)). A Panasonic LCD Projector with wide angle optics, a screen resolution of 1,024 × 768 pixels, a brightness of 1,200 lm, and a frame rate of 50 Hz was used.

Participants were instructed to watch the videos attentively and to respond with a button press if a video with a red square was presented. The total number of button presses and the response times were recorded, so that participants' task engagement was verified. After scanning, participants were reimbursed with 30 Swiss Francs. Furthermore, participants were informed that they would be asked to rate the videos of human and avatar expressions according to their intensity in an online rating survey. Each survey contained 32 videos showing fearful human and avatar expressions (16 each) and four videos showing neutral human and avatar expressions (2 each) as a control condition. The intensity rating of the facial expressions could range from 1 (not very intense) to 6 (extremely intense) and had to be completed within 2 weeks.

The study protocol and data from controls were part of a previous analysis described in a published work by our group. For more details regarding the development of the videos displaying human and avatar expressions and the intensity rating, please refer to Kegel et al., (2020).

2.3. MRI data acquisition

All MRI data were collected using a 3 Tesla Philips Achieva scanner (Philips Medical Systems) with a 32‐channel head coil. Anatomical images were collected using a T1‐weighted MPRAGE sequence covering the whole brain and the following scanning parameters: TR/TE = 8.1 ms/3.7 ms, slices = 176 sagittal slices, voxel size = 1 × 1 × 1 mm, matrix size = 240 × 164 mm, FOV = 240 × 240 mm, flip angle = 8, no fat suppression, total acquisition time = 05:37. Functional images were acquired with an EPI sequence with 32 sequential ascending axial slices co‐planar to the AC‐PC line (TR/TE = 1,800 ms/30 ms, voxel size = 2.75 × 2.75 × 3.5 mm, interslice gap = 0.4 mm, matrix size = 80 × 82 mm, FOV = 222 × 222 mm, flip angle = 75, total acquisition time = 14:22). Per run, the first 10 volumes were discarded to allow the equilibration of T1 saturation effects so that in total 467 volumes were acquired.

2.4. Imaging preprocessing and analysis

Imaging preprocessing was carried out with SPM12 (version 6906; http://www.fil.ion.ucl.ac.uk/spm/; RRID: SCR_007037) on MATLAB (version 2017a; https://ch.mathworks.com/products/matlab.html; RRID: SCR_001622). Functional images were realigned to the first image in the series, followed by slice timing to the middle slice, and coregistration of the mean functional image to the individual anatomical image. Next, the anatomical scans were segmented into different tissue types and spatially normalized to the Montreal Neurological Institute template using DARTEL (Ashburner, 2007). Simultaneously, a mean anatomical template for the whole group was generated. Functional images were then resampled at a resolution of 2 × 2 × 2 mm and spatially smoothed (8 mm full‐width at half‐maximum Gaussian kernel) to reduce noise.

In the first‐level analysis, individual trials were modeled using a general linear model and the SPM12 default canonical hemodynamic response function defined by the onset and the duration of the videos. All images were high‐pass filtered (cut off 128 s) and the following conditions were modeled as regressors of interest: Condition face type (Human > Avatar), condition facial expression (Fear > Neutral), and condition scramble (nonscrambled > scrambled). Control trials were also modeled as regressors of interest but excluded for second‐level analyses, whereas realignment parameters were included as regressors of no interest.

In the second‐level analysis, we analyzed first‐level contrast images within independent regions of interest (ROI) that have been chosen a priori based on previous literature. Using a probabilistic atlas with particular reference to the temporal lobe (Hammers et al., 2003), we defined the following ROIs: the fusiform gyrus (FG), the posterior superior temporal sulcus (pSTS), the anterior superior temporal sulcus (aSTS), the inferior frontal gyrus (IFG), and the amygdala. Within these regions, average ROI signal was extracted and compared for the different conditions with two‐sample t‐tests (Control group > TLE group; TLE group > control group; right TLE > left TLE; left TLE > right TLE) using MarsBaR (version 0.44; http://marsbar.sourceforge.net/index.html; RRID: SCR_009605).

For comparisons between the control group and the TLE group, we pooled the data across participants with TLE to achieve greater statistical power to detect differences. Median rating differences between human and avatar faces were included as covariates of no interest in all analyses, as fearful human expressions were rated as more intense than fearful avatar expressions (see Section 3.1). The resulting two‐sample t‐test outcomes in the ROIs were considered significant if they were below p < .05. We report an uncorrected threshold (e.g., uncorrected for the number of regions in the ROI analysis) because Bonferroni's adjustment for multiple comparisons is often considered too conservative (Field, 2009). To detect potential group differences outside the a priori defined ROIs, we also analyzed first‐level images over the whole brain for the different contrasts. Regarding these results, we report BOLD activation clusters bigger than a cluster extent of k = 5 and remaining significant below a voxel‐wise FWE corrected p‐value of <.05.

2.5. Analysis of sample characteristics and behavioral data

Before analyzing sample characteristics and the intensity ratings, the respective data distributions were first visually inspected using boxplots. This visual inspection showed that most of the examined variables were not normally distributed. For this reason, between‐group comparisons were performed with Mann–Whitney U‐tests. Regarding intensity rating differences, we first compared group differences separately for ratings of fearful human and avatar expressions. Second, we investigated median differences between ratings of fearful human and avatar expressions pooled across the control group and the TLE group. All statistical analyses were performed using SPSS (Version 23; https://www.ibm.com/products/spss‐statistics; RRID: SCR_002865).

3. RESULTS

3.1. Sample characteristics

People with TLE did not differ significantly from controls in terms of age or years of full‐time education (all p > .05). People with right TLE did not differ from those with left TLE concerning age, years of full‐time education, age at epilepsy onset, duration of epilepsy, or number of antiepileptic drugs (all p > .05). Further, no significant differences were found between patients with hippocampal sclerosis and patients without hippocampal sclerosis in terms of their clinical characteristics (all p > .05). Please see Table 1 for details regarding sociodemographic and clinical characteristics of the sample.

3.2. Behavioral data

To verify participants' task engagement during the scanning session, they were required to respond with a button press to infrequently presented control videos with a red square centered on the displayed face or scrambled pattern. The average detection rate of control videos was near perfect in all groups (M = 98%–99%). The TLE group (Mdn = 772 ms) did not differ from the control group regarding their median response time (Mdn = 769 ms; Mann–Whitney U‐test: U = 191.000, p = .673).

No group differences were found between people with TLE and controls in their intensity rating of fearful human expressions (Mann–Whitney U‐test: U = 188.000, p = .593) or their intensity rating of fearful avatar expressions (Mann–Whitney U‐test: U = 195.500, p = .808). People with right TLE did not differ significantly from those with left TLE in their intensity rating of fearful human expressions (Mann–Whitney U‐test: U = 18.000, p = .462) or their intensity rating of fearful avatar expressions (Mann–Whitney U‐test: U = 14.500, p = .240). When comparing fearful human and avatar expressions over the control group and the TLE group, fearful human expressions were rated as more intense (Mdn = 5) than fearful avatar expressions (Mdn = 3; Wilcoxon singed‐rank test: z = −3.63, p = < .001). Please see Figure 2 for the distribution of intensity ratings per group.

FIGURE 2.

FIGURE 2

Open in a new tab

Box plots showing median intensity ratings for fearful human and avatar expressions per group. Whiskers indicate the 25th and 75th percentile. LTLE, left temporal lobe epilepsy; rTLE, right temporal lobe epilepsy

3.3. BOLD responses to human facial expressions in the extended face perception network

In the control group, fearful versus neutral human expressions evoked greater activation in almost all a priori defined ROIs (FG, pSTS, aSTS, IFG, AMY) except for the left pSTS and the left FG. In people with right TLE, a stronger response to fearful human expressions than to neutral human expressions was found in the right FG, the left aSTS, and bilateral amygdala. Further, people with left TLE did not exhibit a significantly stronger response to fearful human expressions than to neutral human expressions in any of the ROIs (see Table 2 for within‐group statistics). To examine whether this lack of activation difference indicates a lack of activation for people with left TLE in general, we also analyzed the response difference between fearful human expressions and their scrambled counterparts. In this case, people with left TLE showed a stronger response to fearful human expressions than to scrambled expressions in the right amygdala (t = 3.35, p = .003) and bilateral IFG (left: t = 3.05, p = .005; right: t = 2.65, p = .011).

TABLE 2.

Statistics for the contrast human fearful expression > human neutral expression within each group and for each a priori defined region of interest in the extended face perception network

FG pSTS aSTS IFG AMY
Left Right Left Right Left Right Left Right Left Right
Control group
t — 1.91 — 2.37 4.64 4.22 4.10 3.55 4.81 6.09
p ns 0.032 ns 0.012 <.001 <.001 <.001 <.001 <.001 <.001
RTLE group
t — 2.11 — — 2.09 — — — 3.22 1.95
p ns .029 ns ns .030 ns ns ns .004 .038
LTLE group
t — — — — — — — — — —
p ns ns ns ns ns ns ns ns ns ns
Open in a new tab

Results are thresholded at p < .05, uncorrected.

Abbreviations: AMY, amygdala; aSTS, anterior superior temporal sulcus; FG, fusiform gyrus; IFG, inferior frontal gyrus; ns, not significant; pSTS, posterior superior temporal sulcus,.

To test the hypothesis of lower activity in the extended face perception network (i.e., in the a priori defined ROIs) in people with TLE, we compared the response difference between fearful and neutral human expressions in the control group to that in the TLE group. We observed a larger response difference in the right amygdala (t = 2.09, p = .002) and the left aSTS (t = 1.71, p = .048) in controls compared to people with TLE (see Figure 3 for distribution of beta weights per condition and group). For the inverse contrast comparing the response difference in the TLE group to that in the control group, no significant difference between groups was apparent (all p > .05).

FIGURE 3.

FIGURE 3

Open in a new tab

Box plots showing the distribution of beta estimates in response to fearful and neutral human expressions per group and different a priori defined regions of interest. Significant group differences between the control group and the TLE group are displayed in the top panels, whereas significant group differences between people with right TLE and left TLE are displayed in the bottom panels. Whiskers indicate the 25th and the 75th percentile. *p < .05, uncorrected. aSTS, anterior superior temporal sulcus; FG, fusiform gyrus; FH, fearful human expression; LTLE, left temporal lobe epilepsy; NH, neutral human expression; RTLE, right temporal lobe epilepsy; TLE, temporal lobe epilepsy

We next compared people with right TLE to those with left TLE. For the right TLE group compared to the left TLE group, we found a larger response difference between fearful and neutral human expressions in the left amygdala (t = 1.94, p = .039) and the left FG (t = 1.82, p = .048; see Figure 3 for distribution of beta weights per condition and group). No difference was found between the two TLE groups, when we compared the activity in the left TLE group in response to fearful and neutral human expressions relative to the right TLE group (all p > .05).

Regarding analyses with avatar faces, we also compared the response difference between fearful and neutral avatar expressions in the control group to that in the TLE group. No significant response difference was found between the control group and the TLE group in any of the ROIs when comparing fearful and neutral avatar expressions (all p > .05). Similarly, no response difference was found between the two TLE groups when comparing fearful and neutral avatar expressions (all p > .05).

3.4. Do avatar facial expressions evoke different BOLD responses in the extended face perception network than human facial expressions?

When contrasting fearful human versus fearful avatar expressions between groups, we observed a larger response difference for controls in the right and left pSTS, the left aSTS, and the left IFG compared to people with TLE (see Table 3 for between‐group statistics regarding a priori defined ROIs). This indicates that in controls the difference in BOLD response between fearful human and avatar expressions was larger than in people with TLE in almost all the ROIs. This difference between groups was due to comparable responses (i.e., not significantly different) to fearful human and avatar expressions in people with TLE (see Figure 4 for distribution of beta weights per condition and group). The inverted contrast testing for larger response differences in the TLE group compared to the control group was not significant in any of the a priori defined ROIs.

TABLE 3.

Between‐group comparisons for the contrast human fearful expressions > avatar fearful expressions for each a priori defined region of interest in the extended face perception network

FG pSTS aSTS IFG AMY
Left Right Left Right Left Right Left Right Left Right
Control > TLE
t — — 1.88 2.07 2.53 — 1.85 — — —
p ns ns .034 .022 .008 ns .036 ns ns ns
RTLE > LTLE
t — — 2.26 — 3.56 — — — 2.51 —
p ns ns .023 ns .002 ns ns ns .014 ns
LTLE > RLTE
t — — — — — — — — — —
p ns ns ns ns ns ns ns ns ns ns
Open in a new tab

Results are thresholded at p < .05, uncorrected.

Abbreviations: AMY, amygdala; aSTS, anterior superior temporal sulcus; FG, fusiform gyrus; IFG, inferior frontal gyrus; ns, not significant.; pSTS, posterior superior temporal sulcus.

FIGURE 4.

FIGURE 4

Open in a new tab

Box plots showing the distribution of beta estimates in response to fearful human and avatar expressions per group and different a priori defined regions of interest. Whiskers indicate the 25th and the 75th percentile. *p < .05, uncorrected. aSTS, anterior superior temporal sulcus; FA, fearful avatar expression; FH, fearful human expression; IFG, inferior frontal gyrus; pSTS, posterior superior temporal sulcus; TLE, temporal lobe epilepsy

When comparing people with right TLE and left TLE, the right TLE group showed a larger response difference between fearful human and avatar expressions in the left amygdala, left pSTS, and left aSTS compared to the left TLE group. The left TLE group did not show a larger response difference in comparison to the right TLE group in any of the ROIs.

No significant response difference between any group was found when comparing neutral human and avatar expressions one with each other.

3.5. Whole‐brain group comparisons

To determine possible activation differences between people with TLE and controls that arise beyond the extended face perception network, group comparisons were analyzed across the whole brain. This analysis revealed one significant cluster: When comparing fearful human and avatar expressions between the control group and the TLE group, the control group showed a stronger response difference in the medial segment of the left prefrontal cortex (mPFC; MNI x, y, z = −2, 60, 12; t = 5.98; k = 32; p‐FWE = .008; see Figure 5). This group difference emerged because the activation cluster in the mPFC only occurred in the control group and was absent in the TLE group. Other group comparisons did not reach significance after correction for multiple comparisons (p‐FWE > .05).

FIGURE 5.

FIGURE 5

Open in a new tab

Group‐level statistical parametric map showing the larger response difference between the control group and the TLE group in response to fearful human compared to fearful avatar expressions (voxel‐wise p‐FWE < .05). The cluster in the medial prefrontal cortex is shown on the mean anatomical template of the study population (bottom and right image) and on the ‘mni152_2009bet’ template from MRIcroGL (top image)

4. DISCUSSION

4.1. Summary

We investigated whether brain responses to dynamic expressions displayed by human and avatar faces differ between people with TLE and controls. In line with previous research, we were able to demonstrate altered BOLD responses to dynamic human expressions within the face perception network in people with TLE relative to controls. More precisely, people with TLE showed a smaller activation difference between fearful and neutral human expressions in the right amygdala and the left aSTS than controls. When comparing the response difference between fearful and neutral human expressions among people with TLE, we found that the left amygdala and the left FG showed a stronger response difference in people with right TLE compared to those with left TLE. Remarkably, when we compared activity for fearful human and avatar expressions, we found a higher number of significantly different response clusters between groups. Controls showed stronger response differences in the right and left pSTS, the left aSTS, the left IFG, and the left mPFC compared to people with TLE. When investigating response differences between people with right TLE compared to those with left TLE, we observed that the right TLE group showed a stronger response difference between fearful human and avatar expressions contralaterally in the left amygdala, the left pSTS, and the left aSTS.

4.2. Altered responses to human facial expressions in temporal lobe epilepsy

In line with our first hypothesis, people with TLE showed an attenuated response difference in the right amygdala between fearful and neutral expressions portrayed by humans compared to controls. Thus, activity in the right amygdala was less increased in people with TLE when observing an emotional expression in another human. This finding is compatible with previous studies that investigated the processing of dynamic human expressions in TLE. This research consistently showed reduced amygdala activity in people with TLE compared to controls (Ives‐Deliperi et al., 2017; Ives‐Deliperi & Jokeit, 2019; Labudda et al., 2014; Schacher, Haemmerle, et al., 2006; Toller et al., 2015). Thus, our results extend earlier evidence to newly developed stimulus material and underline the importance of the right amygdala for the processing of emotional facial expressions.

We also expected people with TLE to show attenuated responses to expressions portrayed by humans in dorsal and ventral areas of the face perception network. However, this hypothesis was statistically confirmed only for the left aSTS showing a smaller response difference between fearful and neutral human expressions in people with TLE compared to controls. The aSTS, as part of the dorsal pathway in the face perception network, is sensitive to dynamic features of faces such as wrinkles and facial motion. This region thus plays a key role in decoding dynamic facial expressions (Duchaine & Yovel, 2015; Haxby et al., 2000; Pitcher et al., 2011). Summarized, this result supports previous studies showing that the epileptogenic network may interfere with (emotional) face processing in people with TLE (Åhs et al., 2014; Riley et al., 2015; Steiger et al., 2017).

Further, people with right TLE showed contralaterally larger response differences between human fearful and neutral expressions in the left amygdala and the left FG relative to people with left TLE. In contrast, there was no observation of larger response differences in left TLE compared to right TLE. The interpretation of this result must be addressed cautiously as previous findings are mixed. Whereas one study also reported stronger amygdala responses in people with right TLE than left TLE (Bonelli et al., 2009), several other studies reported less activity of the amygdala and face‐sensitive areas ipsilateral to seizure onset for right and left TLE (Ives‐Deliperi et al., 2017; Labudda et al., 2014; Schacher, Haemmerle, et al., 2006; Toller et al., 2015). Note, however, that the mentioned studies used different fMRI paradigms either comparing static fearful and neutral expressions (Bonelli et al., 2009) or comparing dynamic fearful expressions to complex landscape scenes (Ives‐Deliperi et al., 2017; Labudda et al., 2014; Schacher, Haemmerle, et al., 2006; Toller et al., 2015).

Compensatory brain activity in people with TLE in the nonaffected, contralateral hemisphere may be associated with larger response differences in right TLE than left TLE (Bettus et al., 2009; Doucet et al., 2013). Due to the preferential role of the right temporal lobe in emotion processing (De Winter et al., 2015; Gainotti, 1972), brain responses during facial emotion processing have been shown to be more affected in people with right TLE than in those with left TLE (Labudda et al., 2014; Steiger et al., 2017). Accordingly, people with right TLE may present stronger compensatory activity in the contralateral hemisphere than those with left TLE, which may account for the group differences found. The explanation would also be consistent with the larger activation difference in people with right TLE in the left amygdala, left pSTS, and left aSTS in response to fearful human and avatar expressions compared to people with left TLE.

4.3. Altered responses to avatar facial expressions in temporal lobe epilepsy

Corresponding to our second hypothesis, we found larger response differences for controls between fearful human and avatar expressions in the pSTS and aSTS compared to people with TLE. More precisely, no significantly different responses were found in people with TLE between fearful human and avatar expressions in dorsal temporal cortex. This result is comparable to a previous study examining brain responses in people with TLE after resection of the anterior temporal lobe (Åhs et al., 2014). In this study, individuals who underwent resection showed reduced responses in the pSTS to fearful human expressions compared to controls. Similar reduced responses of the pSTS to fearful human expressions were observed in people with TLE before resection (albeit not statistically significant; Riley et al., 2015). These results support the notion that structural and functional changes in the (mesial) temporal lobe affect brain functions in structurally intact face processing areas (Vuilleumier & Pourtois, 2007; Vuilleumier et al., 2004). Additionally, it gives support to the modulatory effect of mesial temporal areas, particularly the amygdala, on dorsal temporal cortex during (emotional) face processing (Furl et al., 2013). This highlights the influence of emotion on perceptual, cognitive, and motor responses to dynamic facial expressions (Sato et al., 2017; Vuilleumier & Pourtois, 2007).

The striking finding of our study was larger response differences in controls between fearful human and avatar expressions in frontal areas such as the IFG and the mPFC relative to people with TLE. We are the first to report activation differences in frontal areas during facial emotion processing between people with TLE and controls. This coincides with findings that altered functions in the mesial temporal lobe affect activity and connectivity throughout the entire brain (Ives‐Deliperi & Jokeit, 2019; Jokeit et al., 1997; Riley et al., 2015; Steiger et al., 2017). Moreover, this altered brain activity may not only be related to facial emotion processing, but to other socio‐cognitive processes such as self‐other distinction associated with the IFG (Sinigaglia & Rizzolatti, 2011), as well as mentalizing, perspective taking, or self‐referential processing related to the mPFC (Lieberman et al., 2019; Van Overwalle, 2009). Support for this hypothesis comes from a study investigating BOLD responses in people with TLE during processes related to the attribution of mental states. This study showed that people with TLE exhibited limited neural responses compared to controls when they observed animations of interactions that involved the attribution of mental states (Hennion et al., 2016).

4.4. Limitations and future directions

Our study is the first to apply dynamic avatar stimuli in the research on facial emotion processing in epilepsy. Understandably, we want to discuss certain limitations. First, the low sample size in the right and left TLE groups (n = 7 each) may have limited the statistical power to detect small differences between the two groups. Second, we report ROI results that are not corrected for multiple comparisons (i.e., not corrected for the total number of regions in the ROI analysis). As Bonferroni's adjustment for multiple comparisons is often too conservative (Field, 2009), we decided to report this exploratory, but initial evidence concerning the processing of human and avatar expressions in individuals with and without TLE.

Being the first study to apply dynamic avatar stimuli, we focused on fearful expressions given their evolutionary importance and their frequent use in research in TLE (Adolphs, 2008; Ives‐Deliperi & Jokeit, 2019). Moving on from this, future studies could incorporate expressions of additional emotions. Notably, this requires software solutions that enable us to render realistic emotional expressions with even subtle differences such as expressions of fear and surprise. Additionally, future studies may investigate whether processing differences between individuals with and without TLE also translate onto behavior toward avatars. This is highlighted by the fact that behavioral impairments in human emotion recognition in people with TLE are often subtle despite extensive structural and functional changes on a neural level (Monti & Meletti, 2015). Considering this, future studies may clarify whether tasks with avatars may be implemented for the clinical assessment of emotion perception in individuals with TLE.

5. CONCLUSIONS

Our results show that the neural processing of human and avatar facial expressions differs between individuals with and without TLE in (a) dorsal temporal and inferior frontal cortex sensitive to dynamic facial information and (b) medial prefrontal cortex associated with processes related to the self and others such as mentalizing, perspective taking, or self‐referential processing. Further, our findings support previous studies showing that BOLD activity in the amygdala and the face perception network is altered in individuals with TLE—in response to human as well as to avatar faces. Thus, in individuals with TLE, the influence of altered BOLD activity in the temporal lobe should also be extended to artificial facial expressions. Is this altered BOLD activity an expression of the underlying pathology or a response of a network that can overcome impairment due to temporal brain lesions? Since previous studies have shown that comparable changes in BOLD activity, including connectivity, are associated with impairments in human emotion recognition in people with TLE, but not necessarily with other forms of epilepsy, we can now convincingly argue that it is necessary to study the social domains of patients' behavior when using avatars (Broicher et al., 2012; Ives‐Deliperi & Jokeit, 2019; Labudda et al., 2014; Steiger et al., 2017; Toller et al., 2015). Considering the increased use of avatars in digital applications and remote communication technologies, this study highlights the importance of investigating neural and behavioral responses to computer‐generated characters in samples with neurological conditions as they may respond differentially to our new socio‐digital environment.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

PEER REVIEW

The peer review history for this article is available at https://publons.com/publon/10.1002/brb3.2140.

Supporting information

Video S1

Click here for additional data file. (562.4KB, avi)

Video S2

Click here for additional data file. (425.6KB, avi)

Video S1 and S2

Click here for additional data file. (13.3KB, docx)

ACKNOWLEDGMENTS

This project was funded by the Swiss National Science Foundation (SNSF; project number 166416). We wish to thank all participants who volunteered to participate in this study, and Rebecca Johannessen, Pascal Deschwanden, Alenka Schmid, Eric Diggelmann, and Daniela Casartelli, for their work with recruitment and data collection. Furthermore, we wish to acknowledge Victoria Reed, Bettina Steiger, Julia Bauer, and Teresa Sollfrank for their valuable support regarding the improvement of the manuscript.

Kegel LC, Frühholz S, Grunwald T, Mersch D, Rey A, Jokeit H. Temporal lobe epilepsy alters neural responses to human and avatar facial expressions in the face perception network. Brain Behav. 2021;11:e02140. 10.1002/brb3.2140

Used RRIDs: RRID: SCR_015672, RRID: SCR_001622, RRID: SCR_007037, RRID: SCR_009605, RRID: SCR_002865

Funding information

This project was funded by the Swiss National Science Foundation (SNSF; project number 166416).

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

REFERENCES

  1. Adolphs, R. (2001). The neurobiology of social cognition. Current Opinion in Neurobiology, 11(2), 231–239. 10.1016/S0959-4388(00)00202-6 [DOI] [PubMed] [Google Scholar]
  2. Adolphs, R. (2008). Fear, faces, and the human amygdala. Current Opinion in Neurobiology, 18(2), 166–172. 10.1016/j.conb.2008.06.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Åhs, F. , Engman, J. , Persson, J. , Larsson, E. M. , Wikström, J. , Kumlien, E. , & Fredrikson, M. (2014). Medial temporal lobe resection attenuates superior temporal sulcus response to faces. Neuropsychologia, 61(1), 291–298. 10.1016/j.neuropsychologia.2014.06.030 [DOI] [PubMed] [Google Scholar]
  4. Aljaroodi, H. M. , Adam, M. T. P. , Chiong, R. , Cornforth, D. J. , & Minichiello, M. (2017). Empathic avatars in stroke rehabilitation: A co‐designed mHealth artifact for stroke survivors. In Maedche A., vom Brocke J., & Hevner A. (Eds.), Designing the digital transformation (pp. 73–89). Springer International Publishing. 10.1007/978-3-319-59144-5_5 [DOI] [Google Scholar]
  5. Ashburner, J. (2007). A fast diffeomorphic image registration algorithm. NeuroImage, 38(1), 95–113. 10.1016/j.neuroimage.2007.07.007 [DOI] [PubMed] [Google Scholar]
  6. Bernhardt, B. C. , Hong, S. , Bernasconi, A. , & Bernasconi, N. (2013). Imaging structural and functional brain networks in temporal lobe epilepsy. Frontiers in Human Neuroscience, 7(624), 1–14. 10.3389/fnhum.2013.00624 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bettus, G. , Guedj, E. , Joyeux, F. , Confort‐Gouny, S. , Soulier, E. , Laguitton, V. , Cozzone, P. J. , Chauvel, P. , Ranjeva, J.‐P. , Bartolomei, F. , & Guye, M. (2009). Decreased basal fMRI functional connectivity in epileptogenic networks and contralateral compensatory mechanisms. Human Brain Mapping, 30, 1580–1591. 10.1002/hbm.20625 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bohil, C. J. , Bradly, A. , & Biocca, F. A. (2011). Virtual reality in neuroscience research and therapy. Nature Reviews Neuroscience, 12, 752–762. 10.1038/nrn3122 [DOI] [PubMed] [Google Scholar]
  9. Bombari, D. , Schmid Mast, M. , Canadas, E. , & Bachmann, M. (2015). Studying social interactions through immersive virtual environment technology: Virtues, pitfalls, and future challenges. Frontiers in Psychology, 6, 1–11. 10.3389/fpsyg.2015.00869 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bonelli, S. B. , Powell, R. , Yogarajah, M. , Thompson, P. J. , Symms, M. R. , Koepp, M. J. , & Duncan, J. S. (2009). Preoperative amygdala fMRI in temporal lobe epilepsy. Epilepsia, 50(2), 217–227. 10.1111/j.1528-1167.2008.01739.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Boucenna, S. , Narzisi, A. , Tilmont, E. , Muratori, F. , Pioggia, G. , Cohen, D. , & Chetouani, M. (2014). Interactive technologies for autistic children: A review. Cognitive Computation, 6(4), 722–740. 10.1007/s12559-014-9276-x [DOI] [Google Scholar]
  12. Broicher, S. D. , Frings, L. , Huppertz, H. J. , Grunwald, T. , Kurthen, M. , Krämer, G. , & Jokeit, H. (2012). Alterations in functional connectivity of the amygdala in unilateral mesial temporal lobe epilepsy. Journal of Neurology, 259(12), 2546–2554. 10.1007/s00415-012-6533-3 [DOI] [PubMed] [Google Scholar]
  13. De Winter, F.‐L. , Zhu, Q. I. , Van den Stock, J. , Nelissen, K. , Peeters, R. , de Gelder, B. , Vanduffel, W. , & Vandenbulcke, M. (2015). Lateralization for dynamic facial expressions in human superior temporal sulcus. NeuroImage, 106, 340–352. 10.1016/j.neuroimage.2014.11.020 [DOI] [PubMed] [Google Scholar]
  14. Doucet, G. E. , Skidmore, C. , Sharan, A. D. , Sperling, M. R. , & Tracy, J. I. (2013). Functional connectivity abnormalities vary by amygdala subdivision and are associated with psychiatric symptoms in unilateral temporal epilepsy. Brain and Cognition, 83(2), 171–182. 10.1016/j.bandc.2013.08.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Duchaine, B. , & Yovel, G. (2015). A revised neural framework for face processing. Annual Review of Vision Science, 1, 393–416. 10.1146/annurev-vision-082114-035518 [DOI] [PubMed] [Google Scholar]
  16. Engel, J. , & Salamon, N. (2015). Temporal lobe epilepsy. In Toga A. W. (Ed.), Brain mapping: An encyclopedic reference (pp. 853–860). Elsevier Inc. 10.1016/B978-0-12-397025-1.00101-9 [DOI] [Google Scholar]
  17. Field, A. (2009). Discovering statistics using SPSS (3rd ed.). Sage. [Google Scholar]
  18. Furl, N. , Henson, R. N. , Friston, K. J. , & Calder, A. J. (2013). Top‐down control of visual responses to fear by the amygdala. Journal of Neuroscience, 33(44), 17435–17443. 10.1523/JNEUROSCI.2992-13.2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gainotti, G. (1972). Emotional behavior and hemispheric side of the lesion. Cortex, 8(1), 41–55. 10.1016/S0010-9452(72)80026-1 [DOI] [PubMed] [Google Scholar]
  20. Georgescu, A. L. , Kuzmanovic, B. , Roth, D. , Bente, G. , & Vogeley, K. (2014). The use of virtual characters to assess and train non‐verbal communication in high‐functioning autism. Frontiers in Human Neuroscience, 8(807), 1–17. 10.3389/fnhum.2014.00807 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hammers, A. , Allom, R. , Koepp, M. J. , Free, S. L. , Myers, R. , Lemieux, L. , Mitchell, T. N. , Brooks, D. J. , & Duncan, J. S. (2003). Three‐dimensional maximum probability atlas of the human brain, with particular reference to the temporal lobe. Human Brain Mapping, 19(4), 224–247. 10.1002/hbm.10123 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Haxby, J. V. , Hoffman, E. A. , & Gobbini, M. I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4(6), 223–233. 10.1016/S1364-6613(00)01482-0 [DOI] [PubMed] [Google Scholar]
  23. Hennion, S. , Delbeuck, X. , Koelkebeck, K. , Brion, M. , Tyvaert, L. , Plomhause, L. , Derambure, P. , Lopes, R. , & Szurhaj, W. (2016). A functional magnetic resonance imaging investigation of theory of mind impairments in patients with temporal lobe epilepsy. Neuropsychologia, 93, 271–279. 10.1016/j.neuropsychologia.2016.11.007 [DOI] [PubMed] [Google Scholar]
  24. Ives‐Deliperi, V. L. , Butler, J. T. , & Jokeit, H. (2017). Left or right? Lateralizing temporal lobe epilepsy by dynamic amygdala fMRI. Epilepsy and Behavior, 70, 118–124. 10.1016/j.yebeh.2017.02.006 [DOI] [PubMed] [Google Scholar]
  25. Ives‐Deliperi, V. L. , & Jokeit, H. (2019). Impaired social cognition in epilepsy: A review of what we have learnt from neuroimaging studies. Frontiers in Neurology, 10, 1–10. 10.3389/fneur.2019.00940 [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. James, T. W. , Potter, R. F. , Lee, S. , Kim, S. , Stevenson, R. A. , & Lang, A. (2015). How realistic should avatars be? An initial fMRI investigation of activation of the face perception network by real and animated faces. Journal of Media Psychology, 27(3), 109–117. 10.1027/1864-1105/a000156 [DOI] [Google Scholar]
  27. Javor, A. , Ransmayr, G. , Struhal, W. , & Riedl, R. (2016). Parkinson patients' initial trust in avatars: Theory and evidence. PLoS One, 11(11), 1–21. 10.1371/journal.pone.0165998 [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Jokeit, H. , Eicher, M. , & Ives‐Deliperi, V. (2018). Toward social neuropsychology of epilepsy: A review on social cognition in epilepsy. Acta Epilepsy, 1, 8–17. 10.12107/ae.2018.1.3 [DOI] [Google Scholar]
  29. Jokeit, H. , Seitz, R. J. , Markowitsch, H. J. , Neumann, N. , Witte, O. W. , & Ebner, A. (1997). Prefrontal asymmetric interictal glucose hypometabolism and cognitive impairment in patients with temporal lobe epilepsy. Brain, 120(12), 2283–2294. 10.1093/brain/120.12.2283 [DOI] [PubMed] [Google Scholar]
  30. Kätsyri, J. , de Gelder, B. , & de Borst, A. (2020). Amygdala responds to direct gaze in real but not in computer‐generated faces. NeuroImage, 204, 1–12. 10.1016/j.neuroimage.2019.116216 [DOI] [PubMed] [Google Scholar]
  31. Kegel, L. C. , Brugger, P. , Frühholz, S. , Grunwald, T. , Hilfiker, P. , Kohnen, O. , Loertscher, M. L. , Mersch, D. , Rey, A. , Sollfrank, T. , Steiger, B. K. , Sternagel, J. , Weber, M. , & Jokeit, H. (2020). Dynamic human and avatar facial expressions elicit differential brain responses. Social Cognitive and Affective Neuroscience, 15(3), 303–317. 10.1093/scan/nsaa039 [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Labudda, K. , Mertens, M. , Steinkroeger, C. , Bien, C. G. , & Woermann, F. G. (2014). Lesion side matters – An fMRI study on the association between neural correlates of watching dynamic fearful faces and their evaluation in patients with temporal lobe epilepsy. Epilepsy and Behavior, 31, 321–328. 10.1016/j.yebeh.2013.10.014 [DOI] [PubMed] [Google Scholar]
  33. LeDoux, J. (2000). Emotion circuits in the brain. Demystifying the Brain, 23, 155–184. 10.1146/annurev.neuro.23.1.155 [DOI] [PubMed] [Google Scholar]
  34. Lieberman, M. D. , Straccia, M. A. , Meyer, M. L. , Du, M. , & Tan, K. M. (2019). Social, self, (situational), and affective processes in medial prefrontal cortex (MPFC): Causal, multivariate, and reverse inference evidence. Neuroscience and Biobehavioral Reviews, 99, 311–328. 10.1016/j.neubiorev.2018.12.021 [DOI] [PubMed] [Google Scholar]
  35. Milesi, V. , Cekic, S. , Péron, J. , Frühholz, S. , Cristinzio, C. , Seeck, M. , & Grandjean, D. (2014). Multimodal emotion perception after anterior temporal lobectomy (ATL). Frontiers in Human Neuroscience, 8(275), 1–13. 10.3389/fnhum.2014.00275 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Miller, G. (2007). The promise of parallel universes. Science, 317(5843), 1341–1343. 10.1126/science.317.5843.1341 [DOI] [PubMed] [Google Scholar]
  37. Monti, G. , & Meletti, S. (2015). Emotion recognition in temporal lobe epilepsy: A systematic review. Neuroscience and Biobehavioral Reviews, 55, 280–293. 10.1016/j.neubiorev.2015.05.009 [DOI] [PubMed] [Google Scholar]
  38. Pitcher, D. , Dilks, D. D. , Saxe, R. R. , Triantafyllou, C. , & Kanwisher, N. (2011). Differential selectivity for dynamic versus static information in face‐selective cortical regions. NeuroImage, 56(4), 2356–2363. 10.1016/j.neuroimage.2011.03.067 [DOI] [PubMed] [Google Scholar]
  39. Riley, J. D. , Fling, B. W. , Cramer, S. C. , & Lin, J. J. (2015). Altered organization of face‐processing networks in temporal lobe epilepsy. Epilepsia, 56(5), 762–771. 10.1111/epi.12976 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Robitaille, N. , Jackson, P. L. , Hébert, L. J. , Mercier, C. , Bouyer, L. J. , Fecteau, S. , Richards, C. L. , & McFadyen, B. J. (2017). A Virtual Reality avatar interaction (VRai) platform to assess residual executive dysfunction in active military personnel with previous mild traumatic brain injury: Proof of concept. Disability and Rehabilitation: Assistive Technology, 12(7), 758–764. 10.1080/17483107.2016.1229048 [DOI] [PubMed] [Google Scholar]
  41. Sarkheil, P. , Goebe, R. , Schneider, F. , & Mathiak, K. (2013). Emotion unfolded by motion: A role for parietal lobe in decoding dynamic facial expressions. Social Cognitive and Affective Neuroscience, 8(8), 950–957. 10.1093/scan/nss092 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Sato, W. , Kochiyama, T. , Uono, S. , Yoshikawa, S. , & Toichi, M. (2017). Direction of amygdala – Neocortex interaction during dynamic facial expression processing. Cerebral Cortex, 27(3), 1878–1890. 10.1093/cercor/bhw036 [DOI] [PubMed] [Google Scholar]
  43. Schacher, M. , Haemmerle, B. , Woermann, F. G. , Okujava, M. , Huber, D. , Grunwald, T. , Kramer, G. , & Jokeit, H. (2006). Amygdala fMRI lateralizes temporal lobe epilepsy. Neurology, 66(1), 81–87. 10.1212/01.wnl.0000191303.91188.00 [DOI] [PubMed] [Google Scholar]
  44. Schacher, M. , Winkler, R. , Grunwald, T. , Kraemer, G. , Kurthen, M. , Reed, V. , & Jokeit, H. (2006). Mesial temporal lobe epilepsy impairs advanced social cognition. Epilepsia, 47(12), 2141–2146. 10.1111/j.1528-1167.2006.00857.x [DOI] [PubMed] [Google Scholar]
  45. Schilbach, L. , Koubeissi, M. Z. , David, N. , Vogeley, K. , & Ritzl, E. K. (2007). Being with virtual others: Studying social cognition in temporal lobe epilepsy. Epilepsy and Behavior, 11(3), 316–323. 10.1016/j.yebeh.2007.06.006 [DOI] [PubMed] [Google Scholar]
  46. Sinigaglia, C. , & Rizzolatti, G. (2011). Through the looking glass: Self and others. Consciousness and Cognition, 20(1), 64–74. 10.1016/j.concog.2010.11.012 [DOI] [PubMed] [Google Scholar]
  47. Steiger, B. K. , & Jokeit, H. (2017). Why epilepsy challenges social life. Seizure, 44, 194–198. 10.1016/j.seizure.2016.09.008 [DOI] [PubMed] [Google Scholar]
  48. Steiger, B. K. , Muller, A. M. , Spirig, E. , Toller, G. , & Jokeit, H. (2017). Mesial temporal lobe epilepsy diminishes functional connectivity during emotion perception. Epilepsy Research, 134, 33–40. 10.1016/j.eplepsyres.2017.05.004 [DOI] [PubMed] [Google Scholar]
  49. Toller, G. , Adhimoolam, B. , Grunwald, T. , Huppertz, H. J. , Kurthen, M. , Rankin, K. P. , & Jokeit, H. (2015). Right mesial temporal lobe epilepsy impairs empathy‐related brain responses to dynamic fearful faces. Journal of Neurology, 262(3), 729–741. 10.1007/s00415-014-7622-2 [DOI] [PubMed] [Google Scholar]
  50. Van Overwalle, F. (2009). Social cognition and the brain: A meta‐analysis. Human Brain Mapping, 30(3), 829–858. 10.1002/hbm.20547 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Vuilleumier, P. (2005). How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9(12), 585–594. 10.1016/j.tics.2005.10.011 [DOI] [PubMed] [Google Scholar]
  52. Vuilleumier, P. , & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45(1), 174–194. 10.1016/j.neuropsychologia.2006.06.003 [DOI] [PubMed] [Google Scholar]
  53. Vuilleumier, P. , Richardson, M. P. , Armony, J. L. , Driver, J. , & Dolan, R. J. (2004). Distant influences of amygdala lesion on visual cortical activation during emotional face processing. Nature Neuroscience, 7(11), 1271–1278. 10.1038/nn1341 [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Video S1

Click here for additional data file. (562.4KB, avi)

Video S2

Click here for additional data file. (425.6KB, avi)

Video S1 and S2

Click here for additional data file. (13.3KB, docx)

Data Availability Statement

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Articles from Brain and Behavior are provided here courtesy of Wiley

RESOURCES