Right medial temporal lobe structures particularly impact early stages of affective picture processing

Abstract

Human vision prioritizes emotional stimuli. This is reflected in stronger electrocortical activation in response to emotional than neutral stimuli, measurable on the surface of the head. Feedback projections from brain structures deep within the medial temporal lobes (mTLs), in particular the amygdala, are thought to give rise to this phenomenon, although causal evidence is rare. Given the many pathways involved in visual processing, the influence of mTL structures could be restricted to specific time windows. Therefore, we delineate the temporal dynamics of the impact of right mTL structures on affective picture processing, investigating event‐related potentials (ERPs) in 19 patients (10 female) with right mTL resections and 19 individually matched healthy participants, while they viewed negative and neutral scenes. Groups differed significantly at early‐ and mid‐latency processing stages. Patients with right mTL resection, unlike controls, showed no (P1: 90–140 ms) or marginal (N1: 170–220 ms) emotion modulation. At mid‐latency (early posterior negativity: 220–370 ms), emotion modulation over the ipsi‐resectional right hemisphere was smaller in patients than in controls, but groups did not differ over the left hemisphere. During late parietal positivities (400–650 ms and 650–900 ms), both groups had similar emotion modulation. Our results demonstrate that right mTL structures attenuate particularly early processing of affectively negative scenes. This is theoretically consistent with an initial amygdala‐dependent feedforward sweep in visual emotion processing whose absence is successively compensated. Findings specify the impact of right mTL structures on emotional picture processing and highlight the value of time‐resolved measures in affective neuroscience.

We have compared event‐related potential correlates of affective picture processing in 19 patients with right temporal lobe resections including the amygdala and individually matched healthy controls. Our results show that particularly early processing stages (P1, ipsi‐resectional N1 and early posterior negativity) are impaired by the resections, whereas late positivities are largely intact. These findings constrain theories on the role of medial temporal lobe structures in visual affective stimulus processing.

1. INTRODUCTION

Emotionally arousing visual scenes attract human attention and facilitate perceptual processing (Pourtois, Schettino, & Vuilleumier, 2013), which is reflected in distinct electrocortical activity (Schupp, Flaisch, Stockburger, & Junghöfer, 2006).

A sequence of brain event‐related potentials (ERP) computed from the human electroencephalogram (EEG) is sensitive to emotional content. First markers of emotional modulation of vision can be observed in a positive peak around 100 ms after stimulus onset (P1) and the subsequent negativity (N1). Both components reflect preconscious, automatic attention allocation toward a salient stimulus (van Heijnsbergen, Meeren, Grèzes, & Gelder, 2007; Vogel & Luck, 2000). Middle‐ and long‐latency visual ERPs reflect attentional selection and stimulus evaluation (Olofsson, Nordin, Sequeira, & Polich, 2008). Here, the early posterior negativity (EPN), arising over occipitotemporal electrodes between 150 and 300 ms, is an indicator of selective processing of emotional stimuli (Junghöfer et al., 2006). The late positive potential (LPP), occurring over centroparietal electrode sites from about 400 ms after stimulus onset, reflects further elaboration and memory encoding (Schupp, Junghöfer, Weike, & Hamm, 2004).

Structures in the medial temporal lobe (mTL) are thought to be vital for this enhanced processing of emotional information (Méndez‐Bértolo et al., 2016; Phelps & LeDoux, 2005). It is widely assumed that emotional significance is evaluated in a first forward sweep in the amygdala and that subsequent feedback signals are sent to different ipsilateral cortical processing stages along the ventral visual pathway (Amaral, Behniea, & Kelly, 2003; Sabatinelli, Lang, Bradley, Costa, & Keil, 2009; Tamietto, Pullens, Gelder, Weiskrantz, & Goebel, 2012; Vuilleumier, 2005).

Accordingly, hemodynamic activity in distant brain areas has been reported to be reduced in patients with damaged mTL due to amygdala sclerosis or unilateral anterior lobectomy. This has been demonstrated for fearful faces (Vuilleumier, Richardson, Armony, Driver, & Dolan, 2004) and emotional body language (van de Vliet et al., 2018). Conversely, for emotional scenes, Edmiston et al. (2013) found preserved enhancement of hemodynamic activation in 13 patients withvarious types of temporal lobe resections that included the amygdala. In line with the multiple pathway model of emotion processing (Pessoa & Adolphs, 2010), these results challenge a strong causal impact of mTL structures.

However, highly time‐resolved electrophysiology measures may yield results that are different from or complementary to hemodynamic studies. So far, to our knowledge, only two electrophysiology studies investigated to what extent the mTL contributes to the generation of emotion‐sensitive surface ERPs (Framorando, Moses, Legrand, Seeck, & Pegna, 2021; Rotshtein et al., 2010). Both studies focused on face processing and found dampened responses to fearful faces on different ERPs. Rotshtein et al. (2010) report reduced P1 and late positive potentials, but intact N1 and mid‐latency responses, in seven patients with sclerosis in either the left or the right amygdala. Framorando et al. (2021) identified an N1 impairment in six right amygdalo‐hippocampectomy patients, but not in eight additional left amygdalo‐hippocampectomy patients. Theoretical considerations raise the possibility of distinct mTL contributions to emotion processing in specific time windows, which is supported by preliminary data, but any impact may depend on the stimuli employed, precluding generalization from emotional faces or body language to complex scenes. Moreover, the laterality of the damage may be decisive (Frühholz et al., 2015). Therefore, the causality of mTL contributions for ERP indices of visual emotion processing awaits further clarification.

Against the above background, our study aims to specify the role of right hemisphere mTL structures in time‐resolved visual processing of complex scenes. The right hemisphere is widely assumed to play a dominant role in emotion processing (Gainotti, 2012; Killgore & Yurgelun‐Todd, 2007). We focus on the processing of negative stimuli, for which processing enhancements are found reliably in the ERP (Carretié, Hinojosa, Martín‐Loeches, Mercado, & Tapia, 2004; Weinberg & Hajcak, 2010) and which are also supposed to be particularly effective in activating the amygdalae (LeDoux, 2007). We expect people with right mTL resections to have reduced emotional ERP modulation compared to healthy participants. Given the scarce and partly contradictory results of previous research, which focused on emotional face processing and suggested reductions on the P1 and LPP (Rotshtein et al., 2010) or N1 (Framorando et al., 2021), we make no strong predictions regarding the timing of any group differences and aim to test for them empirically. To this end, early, mid‐latency, and late visually evoked ERPs are analyzed in a homogenous sample of patients who have undergone right mTL resection including the amygdala (see Figure 1) and well‐matched healthy controls to delineate the temporal sequence of right mTL involvement in affective picture processing.

FIGURE 1.

Lesion extent and overlap of 19 right temporal lobe resection patients. The color bar represents number of patients with overlapping resections

2. MATERIALS AND METHODS

2.1. Participants

Data from 19 patients (10 female) who had undergone right anteromedial temporal lobe resection (rTLR) comprising amygdala, hippocampus, and surrounding structures at the Department of Epileptology (Krankenhaus Mara) of Bielefeld University and 19 matched controls are included in this report. Controls were individually matched to each patient regarding age (+/−1 year), gender, and educational background (highest educational qualification). Originally, 20 patients and 20 controls had participated in the study, but data of one patient were lost due to technical issues. Data of the corresponding healthy control were subsequently also omitted from the analyses to ensure the highest comparability between groups. All participants were right‐handed and had normal or corrected to normal vision. Sociodemographic data for both groups are detailed in Table 1. Clinical information for each patient is detailed in Table S1. Patients' surgical outcome was classified 24 months after surgery (Engel Jr, 1993). Eighteen patients were seizure‐free (Engel's I), and one patient reported two to three seizures per year (Engel's II). Eleven patients took anti‐epileptic drugs at the time of participation. Figure 1 shows the extent of the resections. Participants gave written informed consent according to the Declaration of Helsinki and received a monetary reward of 100€ for their participation in the entire study, which consisted of EEG, functional magnetic resonance imaging (fMRI), and extensive neuropsychology testing. fMRI and neuropsychology data will be reported elsewhere. The study was approved by the ethics committee of the German Psychological Association (DGPS).

TABLE 1.

Sociodemographic details for participants

	rTLR patients (n = 19)			Healthy controls (n = 19)
	Mean	SD	Range	Mean	SD	Range
Age	36.0	12.1	22–53	35.7	12	21–52
Education (years)	11.1	2.5	9–16	11.2	2.0	9–16
STAI‐S (state)	39.2	9.56	26–61	32.7	5.18	25–41
STAI‐T (trait)	40.1	11.2	24–63	34.1	9.02	24–58
BDI	8.16	6.77	0–24	2.89	3.03	0–10
Time since resection (months)	52.42	21.84	24–96	—	—	—
Age at onset (years)	15.87	8.78	2.00–39.00	—	—	—
Age at resection (years)	31.63	11.40	16.75–50.00	—	—	—
Duration until resection (years)	15.76	9.38	2.75–32.58	—	—	—

Note: BDI scores were higher for patients than controls (t[36] = 3.093, p = .004), but showed no severe depression for any subject. Further clinical information on the patients can be found in Table  S2 .

Abbreviations: BDI, Beck Depression Inventory, German Version (Hautzinger, Keller, & Kühner, 2009); STAI‐S, State–Trait Anxiety Inventory, State subscale; STAI‐T, State–Trait Anxiety Inventory, Trait subscale (Laux, Glanzmann, Schaffner, & Spielberger, 1981).

2.2. Stimuli and procedure

Emotionally negative and neutral pictures were taken from the International Affective Picture System (IAPS; Lang, Bradley, & Cuthbert, ) and an own, similarly constructed picture set. Physical features of the pictures, such as contrast, brightness, and complexity, were counterbalanced between content conditions, as was the number of pictures showing humans or animals. Table S2 lists the stimuli used in the experiment. Out of a set of 60 negative and 60 neutral pictures, for each participant, 40 pictures of each content were randomly chosen and presented in a passive viewing paradigm. The frequency of selection for each item did not differ between groups. For negative stimuli, the assumption of an equal distribution across both groups was confirmed by an independent sample Kolmogorov–Smirnov Test (p = .660) and chi‐square independence test (χ ²[59] = 45.53, p = 1). Likewise, the assumption of an equal distribution across both groups for neutral stimuli was confirmed by an independent sample Kolmogorov–Smirnov Test (p = 1) and chi‐square independence test (χ ²[59] = 38.78, p = 1). The selected stimulus set of 80 pictures (40 of negative and 40 neutral content) was divided into four blocks and presented in randomized order. Pictures subtending a size of 1,024 × 768 pixels were presented centrally on a 17‐in. TFT monitor with a refresh rate of 60 Hz that was positioned on a table in front of the participants. Participants were seated in a comfortable chair at a distance of about 70 cm from the center of the screen.

In line with previous research on affective picture processing (e.g., Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Foti, Hajcak, & Dien, 2009; Junghöfer et al., 2006; Keil et al., 2009), participants were instructed to attentively view eight blocks of randomly presented negative and neutral pictures. They were instructed to remain as quiet as possible and refrain from eye and body movements during each stimulation block. Self‐paced breaks were offered between stimulation blocks. After a run of four blocks, the entire presentation was repeated, so that every picture was presented twice. Stimuli were presented for 800 ms. A blank black screen was presented during the inter‐stimulus interval of 1,500 ms (±200 ms jitter). Participants also underwent an analogous fMRI experiment. The order of EEG and fMRI testing was counterbalanced.

Post‐experimentally, each participant rated valence and arousal of 10 neutral and 10 negative pictures randomly chosen from the previously presented stimulus set on a seven‐point Likert scale.

2.3. EEG recording and preprocessing

EEG signal was recorded from 128 channels using BioSemi active electrodes and Actiview Software (www.biosemi.com). Impendence of each sensor was kept below 25 Ω, in line with system recommendations. The recording sampling rate was 1,024 Hz. During EEG recording, electrodes were referenced to Cz. Offline, they were re‐referenced to the average reference. Data were bandpass filtered from 0.1 to 30 Hz, and eye movement was subtracted with the automatic artifact correction implemented in Brain Electrical Source Analysis (Ille, Berg, & Scherg, 2002). On average, 2.1% of electrodes were interpolated. The remaining artifacts were rejected based on an absolute threshold (<120 μV), signal gradient (<75 μV/∂T), and low signal (i.e., the SD of the gradient, >0.01 μV/∂T); 14.9% of all trials were rejected. Remaining number of trials (on average 68.05 per condition) did not differ between groups (F[1,36] = 0.200, p = .657, η _p ² = .006). In tendency, somewhat more neutral (15.8%) than negative (14.1%) trials were rejected (F[1,36] = 3.825, p = .058, η _p ² = .096). This pattern occurred in both groups with no interaction between stimulus content and group (F[1,36] = 0.277, p = .602, η _p ² = .008). Prior to statistical analysis, the filtered data were segmented from 100 before to 1,000 ms after stimulus onset and baseline corrected with a baseline interval from −100 ms until stimulus onset.

2.4. Statistical analyses

Electrode clusters for each component were selected in line with the literature (Codispoti, Ferrari, Junghöfer, & Schupp, 2006; Cuthbert et al., 2000; Foti et al., 2009) and following visual inspection of the current data. To investigate the P1, a time window ranging from 90 to 140 ms was segmented using an electrode cluster of 16 occipitoparietal electrodes. For the N1 and EPN time windows, the same symmetrical cluster of nine occipitotemporal electrodes each was used. Time windows ranged from 170 to 220 ms and from 220 to 370 ms, respectively. To investigate late effects (LPP) in two time windows, one from 400 to 650 and the other from 650 to 900 ms, a cluster of 20 centroparietal electrodes was used. Electrode clusters for statistical analyses are included in the results figures.

EEG data were statistically analyzed with emegs2.8 (Peyk, de Cesarei, & Junghöfer, 2011) and IBM SPSS Statistics 25 (IBM Corp., 2017). In each of five distinct time windows, a repeated measures ANOVA was conducted. For the P1 and LPP time windows, a 2 × 2 mixed ANOVA with the between factor group (Healthy Controls vs. rTLR Patients) and the within factor content (negative vs. neutral) was performed. For N1 and EPN time windows, the within factor laterality (left vs. right) was added. Effect sizes are reported using partial eta squared (η _p ²). According to Cohen (1992), effects are interpreted as small (η _p ² > 0.02), medium (>0.13), or large (>0.26). Significant interaction effects were followed‐up by independent or paired‐sampled t‐tests, as appropriate. Cohen's d (Cohen, 1992) was used as effect size for t tests with d = .20, .50, and .80 being small, medium, and large effects, respectively. The assumption of sphericity was met by the present design. Components for which homogeneity of variances was not confirmed (Levene's test) were analyzed with a Welch test. For all statistical tests, an alpha level of .05 two‐tailed was used.

3. RESULTS

3.1. Stimulus appraisals

In line with normative ratings, the negative pictures were rated as less pleasant (M = 1.961, SE = 0.105) than the neutral (M = 4.355, SE = 0.107) pictures (F[1,36] = 162.447, p < .001, η _p ² = .819) by the present participants, without difference between groups (F[1,36] = 0.099, p = .755, η _p ² = .003) or interaction of group and content (F[1,36] = 0.660, p = .442, η _p ² = .018).

Arousal ratings likewise differed between negative and neutral pictures (F[1,36] = 70.165, p < .001, η _p ² = .661). The negative pictures (M = 5.066, SE = 0.245) were rated as considerably more arousing than the neutral pictures (M = 2.837, SE = 0.202). Overall, arousal ratings did not differ between groups (F[1,36] = 0.45, p = .834, η _p ² = .001), but a trend‐level interaction of group and content (F[1,36] = 3.275, p = .079, η _p ² = .083) indicated that controls tended to assign more extreme arousal values than patients.

3.2. Event‐related potentials

3.2.1. P1

For the P1 (90–140 ms), negative pictures elicited higher amplitudes than neutral pictures (F[1,36] = 5.951, p = .020, η _p ² = .142), and right temporal lobe resection (rTLR) patients showed higher amplitudes than controls (F[1,36] = 5.421, p = .026, η _p ² = .131). Importantly, the groups differed in emotional modulation (F[1,36] = 4.808, p = .035, η _p ² = .118). In the healthy control group, negative pictures elicited higher amplitudes than neutral pictures (t[18] = 2.644, p = .016, d = 0.607), whereas this effect was absent for rTLR patients (t[18] = 0.256, p = .801, d = 0.021). Results for the P1 component are shown in Figure 2.

FIGURE 2.

Results for the P1 component (90–140 ms). (a) Difference topographies show electrocortical activity elicited by negative minus neutral pictures. (b) Bar charts show mean amplitudes in the electrode clusters used for statistical analyses. Overlaid dots represent individual observations. Error bars indicate +/−1 SE. Brackets indicate relevant comparisons within (lower level) or between (higher level) groups with symbols indicating significance of results. (c) Event‐related potential at a representative electrode (POz). The shaded area highlights the P1 time window. HC, healthy controls; neg, negative picture; ntr, neutral picture; rTLR, patients with right temporal lobe resection. *p < .05, ^n.s. p > .1

3.2.2. N1

For the N1 (170–220 ms), a significant main effect of picture content was found (F[1,36] = 24.808, p < .001, η _p ² = .408) with a more pronounced negative peak for negative than neutral pictures. Groups differed in emotion modulation (F[1,36] = 4.304, p = .045, η _p ² = .107). In healthy controls, a pronounced N1 amplitude was higher for negative than for neutral pictures (t[18] = −5.039, p = .000, d = −1.156), whereas in rTLR patients, amplitudes elicited by negative and neutral pictures differed only in tendency (t[18] = −2.035, p = .057, d = −0.467). No other main effects or interactions were found (all p values greater than p = .439). Results for the N1 are shown in the upper panel of Figure 3.

FIGURE 3.

Results for the N1 component (170–220 ms) and EPN (220–370 ms). (a) Difference topographies of scalp potentials elicited by negative minus neutral pictures. (b) Bar charts show mean amplitudes in the electrode clusters used for statistical analyses. Overlaid dots represent individual observations. Error bars indicate +/−1 SE. Brackets indicate relevant comparisons within (lower level) or between (higher level) groups with symbols indicating significance of results. (c) Event‐related potentials for two representative electrodes in the left (P7) and right (P8) electrode cluster. The shaded areas highlight the analyzed time windows. HC, healthy controls; neg, negative picture; ntr, neutral picture; rTLR, patients with right temporal lobe resection. ^o p < .1, *p < .05, **p < .01, ***p < .001, ^n.s. p > .1

3.2.3. EPN

In the EPN time window (220–370 ms), highly significant main effects for picture content (F[1,36] = 78.790, p < .001, η _p ² = .686) and laterality (F[1,36] = 13.758, p < .001, η _p ² = .276) were found. The EPN was stronger for negative compared to neutral pictures in that amplitudes were relatively more negative‐going for negative content (see also Schupp et al., 2006). Total amplitude was higher over the right electrode cluster, and groups did not differ in their overall amplitudes in this time window (F[1,36] = 0.111, p = .741, η _p ² = .003). However, groups did differ in emotional modulation of ERP amplitudes as reflected in an interaction of content by group (F[1,36] = 4.255, p = .046, η _p ² = .106) and the three‐way interaction of content by laterality by group (F[1,36] = 5.525, p = .024, η _p ² = .133).

Between‐group comparisons of the difference potentials (negative–neutral) for each hemisphere revealed no group difference in the left hemisphere cluster (t[36] = −1.009, p = .320, d = −0.327). By contrast, in the right, ipsi‐lesional, hemisphere cluster (t[36] = −2.682, p = .011, d = −0.870), emotion modulation was smaller in patients than in controls.

No other significant effects were found (all p values > .161). Figure 3 illustrates the results for N1 and EPN time windows.

3.2.4. Early LPP

As evident from Figure 4, in the early LPP (400–650 ms), a highly significant main effect of content (F[1,36] = 61.292, p < .001, η _p ² = .630) was found in that negative pictures elicited much higher amplitudes. The groups did not differ overall (F[1,36] = 1.378, p = .248, η _p ² = .037), and the interaction of content by group was marginally significant (F[1,36] = 4.027, p = .052, η _p ² = .114). Both groups showed a highly significant emotion modulation, although mean difference potentials (negative‐neutral) were numerically higher in controls (M = 1.451, SE = 0.191) than in patients (M = 0.859, SE = 0.225). A Welch's Test (t[36] = 2.007, p = .053, d = 0.651) revealed a borderline significant group difference.

FIGURE 4.

Results for the LPP component. The early LPP (400–650 ms) is shown in the left panels, and the late LPP (650–900 ms) is shown in the right panels. (a) Difference topographies show difference scalp potentials for negative minus neutral picture processing. (b) Averaged scalp potential for a representative electrode (Cz). The shaded areas indicate analyzed time windows. (c) Bar charts show mean amplitudes of the electrode cluster used for statistical analyses. Overlaid dots represent individual observations. Error bars indicate +/−1 SE. HC, healthy controls; neg, negative picture; ntr, neutral picture; rTLR, patients with right temporal lobe resection. ^o p < .1, *p < .05, **p < .01, ***p < .001, ^n.s. p > .1

3.2.5. Late LPP

For the late LPP (650–900 ms), again, negative pictures elicited much higher amplitudes than neutral pictures (F[1,36] = 49.937, p < .001, η _p ² = .581). Groups did not differ in their overall LPP amplitudes (F[1,36] = 0.162, p = .689, η _p ² = .004), and the interaction of group and content (F[1,36] = 2.644, p = .113, η _p ² = .068) was not significant, although descriptively emotion modulation was still higher in controls (M = 1.595, SE = 0.212) than in patients (M = 0.998, SE = 0.300). Figure 4 illustrates results for early and late LPPs.

3.3. Comparison across components

An additional repeated measure ANOVA with a within factor component (P1, N1, EPN, LPP1, and LPP2) and a between factor group was conducted to examine whether the reduction in emotional modulation in rTLR patients compared to healthy controls differed across components. For components comprising two clusters (N1 and EPN), difference potentials were averaged across both clusters as a rather conservative approach. A significant two‐way interaction (F[4,144] = 4.219, p = .003, η _p ² = .105) confirmed the presumption of time‐dependent group differences of emotional modulation.

Similarly, the onset of emotion modulation was assessed by testing in each group the difference potential (negative–neutral) against zero using one sample t tests. Results are shown in Table 2. They confirm an earlier onset of emotion effects in the control group.

TABLE 2.

One‐sample t tests of mean amplitude differences (negative–neutral) against zero

	rTLR patients (n = 19)				Healthy controls (n = 19)
	t value	df	p	Cohen's d	t value	df	p	Cohen's d
P1	0.256	18	.801	0.059	2.644	18.0	.016	0.607
N1	−2.035	18	.057	−0.467	−4.297	18.0	< .001	−0.986
EPN	−6.427	18	< .001	−1.474	−6.450	18.0	< .001	−1.480
LPP1	3.825	18	.001	0.878	7.581	18.0	< .001	1.739
LPP2	3.332	18	.004	0.764	7.528	18.0	< .001	1.727

Note: For N1 and EPN, the difference potentials were averaged across both electrode clusters.

Abbreviations: rTLR, right temporal lobe resection; df, degrees of freedom; p, p‐value (two‐tailed).

3.4. Timing of effects

Given previous literature, the theoretical focus of this report is on between group differences in emotion modulation of established ERP components as reflected by ERP amplitudes. However, visual inspection of ERPs may suggest subtle group differences in ERP latency. Given a component‐based approach, such timing differences could have affected the magnitude of effects, if ERP time windows matched the control group better than the patient group. To address this possibility, we additionally used a mass‐univariate approach, testing for emotion effects within each group at each individual sensor and time point. F tests were chosen over t tests to avoid polarity differences. The results of this control analysis are summarized in Figure 5, which depicts the temporal evolution of statistical emotion effects within the two groups. Results are further detailed topographically in Figure S1. These data confirm that emotion effects in the healthy comparison group have an earlier onset and larger extent than those in the rTLR patients and indicate that potential latency differences did not distort the component‐based analysis.

FIGURE 5.

Global emotion effects over time. The time course of statistical emotion effects for each group is shown collapsed across the entire scalp. For each time point and sensor, in a mass‐univariate approach, an F‐test contrasting event‐related activity evoked by negative and neutral pictures was calculated. For display purposes, the root‐mean‐square of these effects across all 128 sensors is shown for each time point and within each group of participants. The scalp topography of the effects is detailed in Figure S1. Solid lines indicate healthy controls, and dashed lines are patients. rTLR, patients with right temporal lobe resection

4. DISCUSSION

The main goal of our study was to clarify the impact of right mTL structures on the processing of complex emotional scenes as revealed by high temporal resolution ERPs, thereby providing a direct test of the re‐entry model of visual emotional processing (Vuilleumier et al., 2004). Overall, emotional modulation was reduced in patients with right mTL resections comprising amygdala, hippocampus, and surrounding structures. By contrast, self‐reported ratings of valence and arousal revealed no difference between groups, although patients tended to give less extreme arousal ratings than controls, which accords with previous studies reporting reduced ratings of emotional intensity following temporal lobe resection (Anderson, Spencer, Fulbright, & Phelps, 2000; Cristinzio, N'Diaye, Seeck, Vuilleumier, & Sander, 2010; Milesi et al., 2014).

For early cerebral processing, rTLR patients, unlike controls, showed no ERP enhancement for negative scenes (P1), although they had a generally larger P1 response than the controls. Emotion modulation on the N1 was marginal in patients, being substantially smaller than in controls. In the mid‐latency time window (EPN), rTLR patients' emotional modulation was significantly reduced ipsilateral to the resection but preserved on the contralateral side. While healthy controls showed over both hemispheres a symmetrically enhanced ERP in response to negative pictures, patients with rTLR had higher ERPs for negative content particularly over the contralesional left hemisphere. However, for late positivities, sustained emotion modulation was present in both groups, without statistical difference, although the effect was still numerically smaller in the patients. Overall, the mean difference potentials evoked by negative versus neutral content in patients align with the amplitude difference in controls as processing proceeds.

Our results demonstrate that the impact of mTL structures on emotional scene processing varies with time. The pattern indicates that early stages of emotion processing depend more in intact right mTL structures than do later ones.

Rapid emotion processing has often been attributed to a subcortical pathway to the amygdala that bypasses the cortex and is linked to unconscious perception (LeDoux, 2000; McFadyen, Mermillod, Mattingley, Halász, & Garrido, 2017; Morris, Öhman, & Dolan, 1999). Alternative accounts suggest that rapid responses to emotional stimuli or rapid amygdala activation need not require a subcortical visual “low route” and could be achieved via multiple connections (Pessoa & Adolphs, 2010), including fast feedforward connections from V1 to anterior inferotemporal regions and the amygdala (Kravitz, Saleem, Baker, Ungerleider, & Mishkin, 2013; Wyatte, Jilk, & O'Reilly, 2014). Current data could be consistent with both these accounts. While our results do not speak to the issue of how affective information reaches the mTLs, they do indicate that right mTL structures and perhaps primarily the right amygdala are particularly important for early electrophysiological indices of affective picture processing.

Considering the data within the framework of visual emotion processing as originally proposed by Vuilleumier (2005) and further developed by Pourtois et al. (2013), automatic detection and rapid appraisal of emotionally salient information appears strongly affected by the loss of right mTL structures. Present data show an emotion effect in healthy controls already on the P1, which is absent in rTLR patients. This is broadly consistent with findings by Rotshtein et al. (2010) for fearful faces, several other previous reports of P1 modulations by negative emotional stimuli (e.g., Carretié et al., 2004; for review see Olofsson et al., 2008) and the general notion of an initial, amygdala‐dependent feedforward sweep of affective stimulus processing. Rapid affective discrimination is also corroborated by various affective conditioning studies showing effects within the first 100 ms after stimulus onset (e.g., Steinberg et al., 2012; Steinberg, Bröckelmann, Rehbein, Dobel, & Junghöfer, 2013). However, interpretation of this effect is complicated by the generally larger P1 in the patient group. Moreover, previous controversies about P1 emotion effects raised the possibility that these effects are driven by low‐level perceptual stimulus characteristics rather than by affective significance per se. Low spatial frequencies, which are primarily processed by magnocellular pathways, might be perfectly correlated with some aspects of affective significance and drive early amygdala response for fearful faces (Méndez‐Bértolo et al., 2016), although this has not been confirmed by other studies (McFadyen et al., 2017). We have no evidence that perceptual characteristics drove our P1 effects, since picture assignment did not differ between groups, and previous data suggest no systematic interaction of affective significance and frequency content in the IAPS (Delplanque, N'diaye, Scherer, & Grandjean, 2007), but we did not use specifically frequency‐filtered stimuli. However, frequency content of the stimuli could hardly simultaneously account for the generally larger P1 and the lack of affective discrimination in the rTLR patients, as seen in the present data. A general inhibitory account of the P1 that assigns the P1 a role in early categorization, as proposed by Klimesch (2011), might be able to reconcile both findings: This account proposes that the “P1 amplitude reflects inhibition that enables the suppression of task‐irrelevant and potentially competing processes” (Klimesch, 2011, p. 52). If so, the rTLR patients' perceptual system might be generally more taxed by the consequences of the resection, therefore requiring more initial inhibition of competition, while not showing early affective discrimination, whereas in healthy controls, this competitive inhibition appears more pronounced for negative stimuli, reflecting early affective categorization.

Regarding the proposed subsequent feedback to ventral visual structures, our data suggest that signals from right mTL structures precede or outweigh those of other brain regions early on, since emotional modulation is still severely affected in rTLR patients in the second time window (N1, 170–220 ms). This finding is in line with the suggestion of Framorando et al. (2021) but deviates from the results of Rotshtein et al. (2010) who found intact N1 emotion modulation. The difference may be due higher experimental power in our considerably larger group, or other experimental differences. Rotshtein's study included seven patients with impaired (but not resected) amygdalae and was not controlled for side of sclerosis. Thus, residual amygdalae or intact right amygdalae in patients with left‐sided damage may have enhanced emotional modulation in that time window. This assumption is supported by the study of Framorando et al. (2021), who found preserved N1 enhancement in patients after left‐sided amygdalo‐hippocampectomy.

Both previous ERP studies used faces as stimuli. In general, the characteristics of evoked potentials differ between faces and scenes. Faces in isolation often evoke greater amplitudes in early time windows, including the N170 component (Schupp et al., 2004), while scenes evoke stronger responses in mid‐latency and late processing (Mavratzakis, Herbert, & Walla, 2016; Thom et al., 2014), thereby progressively recruiting more extensive cortical regions.

Presently, emotional modulation in mid‐latency processing, as reflected by the EPN, was reduced ipsilaterally to the resection, but preserved on the contralateral side. This indicates that, at least during processing of complex scenes, right mTL structures still critically contribute to enhanced mid‐latency visual processing of negative stimuli, but other mechanisms begin to compensate for the absence of those structures. In the EPN, such compensation is likely to arise from the intact left amygdala as the topography shift is in line with the assumption of ipsilateral re‐entry mechanisms (Vuilleumier et al., 2004). Compensation might be based on a mechanism of degeneracy enabling a system to perform a particular function via different components (Edelman & Gally, 2001). Thus, the loss of one structure involved in emotion processing may be compensated by other structures (Barrett, 2017; Bullmore & Sporns, 2009; Huang et al., 2020). Our findings suggest that this compensation comes into play at sustained, but not the earliest processing stages.

In later processing stages, the successive alignment between patients and controls regarding emotional modulation suggests a less unique contribution of mTL structures to sustained processing of complex scenes. In the early LPP, patients showed intact emotional enhancement, though still a tendency for reduced emotion modulation compared to healthy controls. In a comparable time window of 500–600 ms, Rotshtein et al. (2010) found reduced ERPs for fearful versus neutral faces in their sample with amygdala sclerosis. ERPs elicited by fearful faces did not differ significantly from those elicited by neutral faces, whereas in our sample, resection patients had significantly larger LPP responses to negative than to neutral scenes. Complex scenes typically contain many more perceptual details than isolated faces and thereby provide a much richer context for emotion perception (Reisch, Wegrzyn, Woermann, Bien, & Kissler, 2020; Sabatinelli et al., 2011). This may facilitate compensation by multiple brain sources for temporal lobe lesions.

Visual inspection of the data may suggest subtle ERP latency differences between rTLR patients and controls, particularly during early processing stages. Whereas latency was not the focus of our study as the hypothesis from previous research concerned amplitude differences, we tested whether potential latency shifts could have affected the above reported ERP effects, performing an additional data‐driven analysis on continuous ERPs. As shown in Figure 5 and Figure S1, the choice of temporal analysis windows or spatial clusters did not drive the pattern of results.

The present results are in line with the assumption that the amygdala is part of a distributed network of brain areas involved in emotion processing (Pessoa & Adolphs, 2010). This view was supported by a previous fMRI study (Edmiston et al., 2013) that found no significant difference in emotional modulation in patients with mTL resections and controls. The seemingly contradicting finding of no significant reduction of cortical blood oxygen level in patients with resection of the amygdala and surrounding tissue can be plausibly explained by compensations over the time course as specified by the present results. Indeed, fMRI data from the present project likewise suggest that during picture processing emotion modulation of BOLD responses in visual cortices is largely intact following right‐sided mTL resections (Reisch et al., in preparation; Stieghorst et al., 2018). Together, data indicate that mTL structures have a decisive contribution to the processing of complex scenes in early processing stages and a facilitating, but a less critical role on sustained emotion processing.

This study focused on right mTL resections, and its findings may not generalize to left temporal lobe structures, as left and right hemispheres appear to differ in their contributions to emotion processing (see Gainotti, 2019 for an overview), which was not explicitly considered by previous studies in the area. Extant theories suggest a right hemispheric predominance for emotion processing per se (Borod, Kent, Koff, Martin, & Alpert, 1988) or a hemispheric specialization for negative content (Adolphs, Jansari, & Tranel, 2001; Gainotti, 2012). Moreover, functional asymmetries of emotion processing may be based on timing differences between left and right amygdala (Gainotti, 2012; Ross & Pulusu, 2013).

The right amygdala has been found to be selectively activated by unconscious processing of emotional stimuli (Morris, Ohman, & Dolan, 1998; Nomura et al., 2004), and activation of the right amygdala has been observed to precede left amygdala activation in emotion processing (Hung et al., 2010; Kohno, Noriuchi, Iguchi, Kikuchi, & Hoshi, 2015). Right temporal lobe structures might be specialized in rapid detection and automatic processing of emotional stimuli, while the left amygdala might be particularly involved in sustained stimulus processing (Gläscher & Adolphs, 2003; Kohno et al., 2015). Indeed, our results support the assumption that the right amygdala is essential for early processing of emotional content and thus might be part of a rapid emotion detection system, since early emotional modulation is absent in patients with right mTL resection. By contrast, the left amygdala may kick in later, its impact being first reflected in left lateralized emotion enhancement in the EPN window. Preserved emotion enhancement in the LPP windows as well as largely intact explicit appraisals may then result from the combined impact of the intact amygdala and widely distributed extra‐temporal sources.

To our knowledge, this study is the first to evaluate ERPs during processing of complex negative and neutral scenes in a large homogenous group of right‐hemisphere temporal lobe resection patients. Our results are consistent with the general finding that neural processing of emotional stimuli is affected in patients with mTL impairment (Milesi et al., 2014; Rotshtein et al., 2010; van de Vliet et al., 2018; Vuilleumier et al., 2004; Wendling et al., 2015). Going beyond previous findings, our data delineate the time course of this impairment and assign it specifically to earliest processing stages.

In conclusion, our ERP data demonstrate that the absence of right mTL structures causes a time‐window specific reduction of emotion modulation for complex negative scenes. For patients with right mTL resection, emotion modulation was absent in early processing. In the mid‐latency time window, it was reduced ipsilaterally to the resection, but preserved contralaterally. Emotion modulation was increasingly normal in late processing. Thus, affective scenes are processed in multiple stages with varying impact of right mTL structures. While initial processing highly depends on intact right mTL structures, the absence of those structures is compensated by other brain sources in sustained processing, indicating a facilitating role of mTL structures for elaborated emotion processing.

CONFLICT OF INTEREST

The authors declare no competing financial interests.

Supporting information

Appendix S1: Supplementary Information

Mielke, M. , Reisch, L. M. , Mehlmann, A. , Schindler, S. , Bien, C. G. , & Kissler, J. (2022). Right medial temporal lobe structures particularly impact early stages of affective picture processing. Human Brain Mapping, 43(2), 787–798. 10.1002/hbm.25687

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available from the corresponding author upon reasonable request.