When affect overlaps with concept: emotion recognition in semantic variant of primary progressive aphasia

See Ibanez and Schulte (doi:10.1093/brain/awaa392) for a scientific commentary on this article.

Are emotions innate or do they depend on contextual – and thus cultural – knowledge? Using neuropsychological testing and neuroimaging in patients with the semantic variant of primary progressive aphasia, Bertoux et al. show that emotion recognition depends on semantic knowledge of emotion concepts.

Abstract

The most recent theories of emotions have postulated that their expression and recognition depend on acquired conceptual knowledge. In other words, the conceptual knowledge derived from prior experiences guide our ability to make sense of such emotions. However, clear evidence is still lacking to contradict more traditional theories, considering emotions as innate, distinct and universal physiological states. In addition, whether valence processing (i.e. recognition of the pleasant/unpleasant character of emotions) also relies on semantic knowledge is yet to be determined. To investigate the contribution of semantic knowledge to facial emotion recognition and valence processing, we conducted a behavioural and neuroimaging study in 20 controls and 16 patients with the semantic variant of primary progressive aphasia, a neurodegenerative disease that is prototypical of semantic memory impairment, and in which an emotion recognition deficit has already been described. We assessed participants’ knowledge of emotion concepts and recognition of 10 basic (e.g. anger) or self-conscious (e.g. embarrassment) facial emotional expressions presented both statically (images) and dynamically (videos). All participants also underwent a brain MRI. Group comparisons revealed deficits in both emotion concept knowledge and emotion recognition in patients, independently of type of emotion and presentation. These measures were significantly correlated with each other in patients and with semantic fluency in patients and controls. Neuroimaging analyses showed that both emotion recognition and emotion conceptual knowledge were correlated with reduced grey matter density in similar areas within frontal ventral, temporal, insular and striatal regions, together with white fibre degeneration in tracts connecting frontal regions with each other as well as with temporal regions. We then performed a qualitative analysis of responses made during the facial emotion recognition task, by delineating valence errors (when one emotion was mistaken for another of a different valence), from other errors made during the emotion recognition test. We found that patients made more valence errors. The number of valence errors correlated with emotion conceptual knowledge as well as with reduced grey matter volume in brain regions already retrieved to correlate with this score. Specificity analyses allowed us to conclude that this cognitive relationship and anatomical overlap were not mediated by a general effect of disease severity. Our findings suggest that semantic knowledge guides the recognition of emotions and is also involved in valence processing. Our study supports a constructionist view of emotion recognition and valence processing, and could help to refine current theories on the interweaving of semantic knowledge and emotion processing.

See Ibanez and Schulte (doi:10.1093/brain/awaa392) for a scientific commentary on this article.

Introduction

Emotion recognition is central to human life, as facial expressions are among the most salient types of social information that neurotypical humans process on a daily basis to understand their fellow beings and successfully navigate the social world. In the history of neurosciences, most of the early theories of emotion, such as the James-Lange theory (James, 1884; Lange, 1887), the Cannon-Bard theory (Cannon, 1927; Bard, 1928) or Papez’s proposal regarding the neural mechanisms of emotions (Papez, 1937) were formulated from a physiological perspective. Similarly, Ekman’s studies of facial emotion recognition (FER) have been deeply influential in suggesting that emotions are innate, biology-based processes (Ekman and Friesen, 1971), following Darwin’s hypothesis (Darwin, 1872), which he helped to popularize, together with the concept of basic emotions. According to this view, the experience and perception of these basic emotions (i.e. happiness, surprise, anger, disgust, sadness, and fear) are shaped by specific features and distinct neural responses that distinguish one emotion from another (Murphy et al., 2003; Vytal and Hamann, 2010; Ekman and Cordaro, 2011).

The parallel development of theories grounded in social psychology introduced the notion that a cognitive appraisal of the environment is necessary to guide the labelling of physiological arousal (Schachter and Singer, 1962). In appraisal theories, emotions are thought to result from individuals’ personal interpretation and explanation of an event, independently of their physiological arousal (Smith and Lazarus, 1993). These models led to a dedifferentiation of cognitive and affective representations, and provided an account of individual variability. A more integrative view on emotion has been developed in recent years, with the emergence of constructionist and embodied cognition theories (Barrett, 2006; Niedenthal, 2007; Barsalou, 2008). According to the constructionist view, emotions are constructed mental states, and the experience of emotion is an act of categorization guided by embodied knowledge about this emotion. Emotion recognition thus depends on acquired conceptual knowledge that is derived from prior experience and re-enacted during perception (Barrett et al., 2007). This conceptual act theory therefore contrasts with traditional emotion theories that postulate the existence of innate discrete emotion categories (Lindquist et al., 2013; Touroutoglou et al., 2015), arguing instead for acquired emotion conceptual knowledge, and plead against the specificity of these categories’ neural correlates (Lindquist and Barrett, 2012).

Independently of this debate, there is a consensus that valence (or hedonic tone) captures an essential aspect of emotions (Frijda, 1986; Russell, 2003; Barrett, 2006). Emotions such as happiness or pride are regarded as positive/pleasant, and anger, fear or shame as negative/unpleasant. It is generally agreed that valence processing is an early or less effortful step that precedes a more fine-grained interpretation of an emotional expression (Russell, 2003; Barrett, 2006; Calvo and Nummenmaa, 2016; Martinez, 2017; Qiu et al., 2017). Just as the concept of valence is central to theories of emotion, so it is essential in the fields of behaviour, motivation and mood (Joffily and Coricelli, 2013). However, valence has received multiple definitions over time, and its dichotomous character has been criticized in the past (Colombetti, 2005). The idea of mutually opposing positive and negative emotions has been judged by some to be simplistic and artificial (Solomon and Stone, 2002), resulting in a discussion about the universally innate and primitive nature of valence processing. As a concept that appears to be rooted in culture, morality and ethics (Solomon, 2001), we can hypothesize that valence is a construct, and as such cannot be independent of semantic memory. In line with this hypothesis, some authors have postulated that emotion concept knowledge can be divided into superordinate (valence) and subordinate (emotion labels) levels (Adolphs et al., 2003; Widen and Russell, 2003), but the few studies to have specifically tested this hypothesis have yielded inconsistent results (Lindquist et al., 2014; Macoir et al., 2019). It is therefore still not clear whether valence is independent of semantics.

A unique model for investigating the semantic contribution to FER and valence processing is provided by the semantic variant of primary progressive aphasia (svPPA), also known as semantic dementia, a neurodegenerative disease lying on the continuum of frontotemporal degeneration that is characterized by early and severe semantic memory impairments (Gorno-Tempini et al., 2011). Over the past 20 years, a major FER impairment for basic emotions, mostly those with a negative valence, in both static (photographs) and dynamic (videos) presentations have been revealed in svPPA (Perry et al., 2001; Rosen et al., 2002, 2004; Calabria et al., 2009; Kumfor et al., 2011; Hsieh 2012; Miller et al., 2012; Irish et al., 2013; Lindquist et al., 2014; Multani et al., 2017). So far, recognition of self-conscious emotions (e.g. embarrassment), which involves a deeper analysis of the social context, has rarely been investigated (but see Sturm et al., 2008). The neural correlates of these deficits have also been rarely investigated, but despite variable sample sizes and the choice to combine several disease groups to increase statistical power in some studies, findings indicate that temporal and orbitofrontal regions are central to FER impairment in svPPA (Rosen et al., 2002; Hsieh et al., 2012; Kumfor et al., 2018), especially on the right side (Kumfor et al., 2016). Regarding white fibres, the thalamic radiation, the uncinate and superior longitudinal fasciculi have been found to be implicated (Downey et al., 2015; Multani et al., 2017).

Regarding the possible link between semantic impairment and FER, inconsistent results have been reported in the rare studies to have addressed it but there is a general consensus to point out a primary FER deficit in svPPA despite a relative impact of language deficits (Kumfor et al., 2011, 2018; Hsieh et al., 2012; Fittipaldi et al., 2019). The two studies that specifically investigated the link between semantic and FER impairments yielded mixed results. Although both reported reduced recognition of discrete emotions, valence processing was judged to be preserved in one study (Lindquist et al., 2014), but not in the other (Macoir et al., 2019). In addition, as both studies only considered a single positive emotion (happiness) but diverse negative ones—like most FER investigations in neurodegeneration—this may have led to valence-incongruent errors in svPPA (e.g. mistaking a negative emotion for a positive one) being underestimated. To date, the study of semantic contribution to FER in svPPA has been limited to expressions sorting, decisions about forced associations between expressions and labels, and exploration of correlation between FER and naming performance. So far, no study has explored the specific semantics of emotions. We believe that taxonomic emotional knowledge could be easily measured through affect words matching or generation of emotion’s synonyms. Similarly, contextual knowledge of emotions could be assessed through the selection or production of examples of contexts typically associated with an emotion. If emotion recognition depends on acquired conceptual knowledge, one could expect a strong relationship between the evaluation of emotional concepts (EEC) and FER on the cognitive and neural levels. Beyond the need to better describe the interactions between semantic and emotion processing, the question of whether valence processing is preserved or not in svPPA remains a particularly interesting issue, regarding the debate between traditional and constructionist theories of emotion. Even if the two views converge to consider valence perception as a more or less automated process, no findings have formally disproved the hypothesis that valence can be represented as a superordinate semantic category. The goal of the present study was therefore to address this question by (i) describing FER abilities in svPPA using static and dynamic presentations; (ii) studying the relationship between valence and conceptual processing during FER; and (iii) exploring FER’s grey and white matter neural correlates using voxel-based morphometry (VBM) and diffusion tensor imaging (DTI). To overcome past limitations, we increased the number of emotions traditionally considered in the field, including self-conscious (pride, contempt, embarrassment) as well as basic emotions, and performed a qualitative analysis of the errors made by participants. We hypothesize to retrieve significant links between FER and emotion conceptual knowledge as well as a valence processing impairment in patients with svPPA.

Materials and methods

Participants

Participants included 20 control subjects and 16 patients with svPPA. Patients were recruited from memory clinics (university hospitals of Caen, Rennes, and Rouen, France) and seen through multidisciplinary consultations involving senior neurologists and neuropsychologists as well as speech therapists, all specialized in the assessment of neurodegenerative diseases. They all met Gorno-Tempini et al.’s (2011) diagnostic criteria. All patients presented a semantic memory deficit, reflected by anomia and word comprehension difficulties, as a predominant and inaugural symptom. At the time of this study, they were all well oriented in time and space and instrumental activities of daily living were preserved (except telephone use because of semantic difficulties for some). These patients could therefore continue to carry out everyday activities such as doing their own shopping, using public transport, remembering recent events and keeping GP appointments. Among the patients, nine had a lumbar puncture and a subsequent analysis of CSF biomarkers revealing the absence of an Alzheimer’s disease biomarker profile. No change of diagnosis was performed during the follow-up of patients, which therefore ensure that the clinical progression supported the clinical diagnosis (the worsening of semantic impairment was observed in all patients; stereotypies and impulsive behaviour were frequently observed later in the course of the disease). Additional clinical data are provided in the online Supplementary material. Regarding the lateralization of the atrophy, four patients were identified with predominantly right-sided atrophy. Basic demographic and clinical data for these patients are provided in Supplementary Table 1.

Cognitive complaint and a history of substance or alcohol use disorder, head trauma, and developmental, neurological or psychiatric conditions were exclusion criteria for control participants. The regional ethics committee approved the study, and all participants provided their written informed consent, in accordance with the Declaration of Helsinki.

Neuropsychological assessment

All participants underwent a comprehensive neuropsychological assessment described in the Supplementary material. Briefly, general efficiency, verbal fluency, verbal attention and working memory, executive functioning, visuo-praxic abilities, mentalizing as well as visual and semantic memory were assessed. Patients (n = 14) were also administered the Frontal Behavioural Inventory (Kertesz et al., 1997).

Evaluation of emotional concepts

To assess participants’ conceptual knowledge of emotions, we provided them with four emotion labels (anger, pride, surprise and embarrassment), and asked them four questions about each one. Participants had to provide a synonym of each emotion, then choose the word that was most closely related to it among four options (e.g. sad, tired, upset or satisfied for anger) (assessment of taxonomic knowledge). Next, participants had to provide an example of the context in which this emotion might be felt, and finally choose one context in which this emotion might be felt among four options (assessment of contextual knowledge). We computed four subscores: synonyms (scored/4), matching (/4), examples (/4), and context choice (/16 with 1 point for each correct ‘yes’ or ‘no’ answer) and a total score (maximum score/28).

Facial emotion recognition

This test involved the identification of facial emotions expressed by male and female actors, mostly white. Items were taken from the Amsterdam Dynamic Facial Expression Set (van der Schalk et al., 2011). The test had two modalities: static (photographs) and dynamic (short videos lasting 6–6.5 s). In each modality, participants saw 50 items depicting 10 different emotions, including seven basic ones (happiness, surprise, anger, fear, disgust, sadness, and neutral) and three self-conscious ones (pride, embarrassment, and contempt). Patients had to choose among the 10 labels that were presented. Regarding the valence, happiness and pride were considered as positive, and the other emotions as negative, except for neutral and surprise, which were regarded as neither positive nor negative. Different scores were calculated, reflecting the accurate recognition of positive, negative, basic, self-conscious and all emotions in each modality (static versus dynamic), as well as a total score for the test (maximum score/100).

Imaging acquisition

All imaging acquisitions were performed at the Cyceron centre (Caen). Participants underwent whole-brain imaging using a 3 T MRI Philips scanner with a standard quadrature head coil (eight channels). Structural high-resolution T₁-weighted images were acquired via the following sequence: 3D-T₁-FFE sagittal, 180 slices, slice thickness 1 mm, echo/repetition time = 4.6/20 ms, flip angle = 10°, matrix = 256 × 256 mm², 1-mm² in-plane resolution. Diffusion-weighted spin echo images (DWI) were acquired with the following parameters: axial orientation, 32 directions at b = 1000 s/mm², 70 slices, 2-mm slice thickness, echo/repetition time = 82/10 000 ms, flip angle = 90°, field of view = 224 × 224 mm², matrix = 112 × 112 and 2 mm² in-plane resolution. One no-diffusion weighted image at b = 0 s/mm² was also acquired.

Voxel-based morphometry

MRI data were analysed using FSL-VBM (Ashburner and Friston, 2000; Good et al., 2001), part of the FSL software package (Smith et al., 2004). Briefly, structural images were brain-extracted using the BET brain extraction tool, and tissue segmentation was conducted using the FAST automatic segmentation tool (Zhang et al., 2001). Grey matter partial volumes were aligned to the Montreal Neurological Institute standard space (MNI152) using the FNIRT non-linear registration approach (Andersson et al., 2007a, b), using a B-spline representation of the registration warp field (Rueckert et al., 1999). We created a study-specific template in which patients with svPPA and controls were equally represented, and the native grey matter images were registered non-linearly. Registered partial volumes maps were modulated by dividing them by the Jacobian modulation of the warp field to correct for local expansion or contraction. The Jacobian modulation step did not include the affine part of the registration, which meant that the data were normalized for head size as a scaling effect (Good et al., 2002). Modulated images were smoothed with an isotropic Gaussian kernel with a sigma of 3 mm.

Diffusion tensor imaging

The DWI images were preprocessed to create fractional anisotropy images using the FSL Diffusion Toolbox (http://fsl.fmrib.ox.ac.uk/fsl/fslwiki/FDT), and corrected for distortions due to eddy currents, then aligned to the b = 0 s/mm² image, using rigid-body registration for motion correction (Jenkinson et al., 2002). Fractional anisotropy images were created by fitting a tensor model to the diffusion images, and processed using tract-based spatial statistics for subsequent voxelwise statistical analysis (Smith et al., 2006). A 0.3-threshold was applied to each participant’s aligned fractional anisotropy image, to exclude low fractional anisotropy values that might be contaminated with partial volume effects from other non-white matter tissues and to minimize interparticipant variability (Segobin et al., 2015). The resulting image was then projected onto the mean skeleton by filling every voxel of the latter with the maximum fractional anisotropy value perpendicular to the skeleton structure. Voxel-based statistics were performed on skeletonized images. DTI data were missing for four patients.

Statistical analyses

Cognitive data

SPSS 20 (SPSS, Chicago, IL, USA) was used to perform the statistical analyses. Age was treated as a covariate for all analyses. Univariate analyses of variance (two-tailed) were performed to investigate intergroup differences. Partial eta-squared (η²) was computed as a measure of effect size. Frequency of FER labels were obtained with the Open Lexicon FR database (New et al., 2004).

We carried out a qualitative analysis to characterize the types of errors made during FER. In particular, this analysis serves to dissociate valence errors (i.e. when one emotion was mistaken for another of a different valence, e.g. happiness for fear) from non-valence errors (i.e. confusing two emotions of the same valence, e.g. sadness for fear). Spearman correlation analyses were computed to assess pairwise linear relations between cognitive scores in patients and results were corrected for multiple correlations using the Hochberg’s Step-Up procedure. As data for the dynamic presentation modality were missing for one patient, analyses of the total or dynamic FER scores only involved 15 patients.

Neuroimaging analyses

FSL was used to perform VBM and DTI analyses. First, we ran a two-sample t-test to contrast patients and controls, in order to identify specific grey matter regions that were either atrophied (VBM) or exhibited lower fractional anisotropy (DTI) in patients using permutation-based non-parametric testing, with 5000 permutations per contrast and correction for multiple comparisons (familywise error, FWE, P < 0.01) using threshold-free cluster enhancement (TFCE) for clusterwise correction (Smith and Nichols, 2009).

To examine the decreases in grey matter or fractional anisotropy associated with emotion recognition, valence recognition and conceptual knowledge, we entered the FER score, the number of valence errors, and the total EEC score into separate voxelwise general linear models through VBM and DTI analyses based on the same statistical parameters (i.e. 5000 permutations, FWE P < 0.01 using TFCE). Patients were combined to control for VBM analysis in a procedure that had previously been used in similar studies to achieve greater variance in scores, thereby increasing the statistical power to detect brain-behaviour relationships (Irish et al., 2014). As data for dynamic presentations were missing for one patient, we calculated an imputation on the mean and used it as a score. For all imaging analyses, the cluster threshold was set at 100 voxels, and age was treated as a nuisance variable.

We then conducted overlap mask analyses to identify grey matter regions that correlate with more than one cognitive measure. Here, pairs of statistical maps generated by the results (i.e. FER and EEC; valence errors and EEC) were scaled using a threshold of P = 0.01, after which the two maps were multiplied to create an overlap mask. This method was validated by a conjunction analysis performed a posteriori. Finally, to explore the relationship between FER and EEC regions or valence errors and EEC regions, we extracted mean grey matter intensity values for all participants from each map of results (thresholded at P = 0.01) and looked for correlations. To assess the specificity of the relations between FER or valence errors and EEC regions, we also identified the grey matter regions associated with the number of errors on the Brixton test (using the same parameters), and extracted the grey matter values. We decided to consider this executive score because significant between-group differences were observed, and it was neither an emotional nor a semantic score.

Data availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Results

Demographic, clinical and neuropsychological data

Patients differed from control subjects on age (P < 0.05), but not on education level (P > 0.1). Distribution of gender (males and females) did not differ across the groups (χ² = 2.031; P = 0.154). More details on other sociocultural variables matched across groups are given in the Supplementary material. In the neuropsychological examination, patients exhibited diminished global cognitive efficiency and a semantic knowledge impairment. Dysfunctions of visual episodic-like memory, working memory (only when high manipulation load was involved) and executive functioning were also observed. No significant effect of education or gender were retrieved to impact the following results (Table 1).

Table 1.

Demographic, clinical and neuropsychological participants' data

	Controls	Patients with svPPA	P-value (η2)
Demographic and clinical data
Age, years	63.26 (6.8)	67.89 (6.7)	<0.05 (0.109)
Education level, years	12.15 (3.1)	13.69 (4.3)	0.22 (0.044)
MoCA (/30)	27.60 (1.4)	19.87 (4.2)	<0.0005 (0.598)
Disease duration, months	NA	66.4 (47.8)	NA
Background neuropsychological data
Semantic knowledge score	143.80 (0.4)	122.60 (31.5)	<0.01 (0.176)
Lexical fluency (words)	24.35 (5.5)	14.33 (7.2)	<0.0005 (0.378)
Category fluency (words)	34.85 (6.9)	14.80 (9.5)	<0.0005 (0.558)
Forward verbal span	5.10 (0.8)	4.88 (1.2)	0.81 (0.002)
Backward verbal span	4.75 (0.9)	4.93 (0.9)	0.12 (0.074)
Letter and digit sequence span	10.50 (0.9)	8 (2.8)	<0.05 (0.129)
Stroop interference, s	52.20 (24.6)	104.4 (99.7)	0.12 (0.075)
TMT B − A, s	53.60 (30.0)	84.47 (43.9)	0.09 (0.085)
Brixton Spatial Anticipation Task (errors)	10.35 (3.9)	17.93 (4.1)	<0.0005 (0.415)
Rey complex figure (/36)	35.50 (0.8)	34.94 (1.5)	0.26 (0.038)
Rey complex figure, recall (/36)	21.50 (6.6)	13.38 (6.9)	<0.005 (0.286)
TOM-15 (/15)	14.05 (1.4)	12.87 (1.8)	0.06 (0.110)

Mean (standard deviation) demographic, clinical and neuropsychological scores, and differences (P-values and η² as effect size) between groups. MoCA = Montreal Cognitive Assessment; TMT = Trail Making Test; TOM-15 = Theory of Mind-15; NA = not available.

Evaluation of emotional concepts

Patients performed more poorly than controls on every subtest, although the differences on the synonyms and context choice subscores did not reach statistical significance. Patients made less accurate matching choices (P = 0.01, η² = 0.173) and were less able to provide an appropriate example of context (P = 0.04, η² = 0.128). The total score showed reduced conceptual knowledge of emotions in patients with svPPA (P = 0.01, η² = 0.166) (Table 2).

Table 2.

Evaluation of emotional concepts

Subscores	Controls	Patients with svPPA	P-value (η2)
Taxonomic knowledge
Synonyms	65.0% (30.8)	46.2% (39.3)	0.13 (0.071)
Matching	98.8% (5.6)	84.6% (24.0)	0.01 (0.173)
Contextual knowledge
Examples	97.5% (7.7)	80.8% (34.1)	0.04 (0.128)
Context choice	92.5% (6.3)	87.5% (10.5)	0.09 (0.087)
Total	88.4% (9.0)	74.6% (22.1)	0.01 (0.166)

Percentage performances (standard deviation) on the EEC subtests and differences (P-values and η² as effect size) between groups.

Facial emotion recognition

There was no effect of the frequency of labels used on performance in both groups (Rs < 0.10; P-values > 0.78). Taking the static and dynamic modalities together, patients significantly underperformed compared to controls on the recognition of emotions (P < 0.00001, η² = 0.572). Comparisons of performance between modalities showed that the FER performance was higher in the dynamic modality than in the static one for controls (P < 0.05, η² = 0.123), but not for patients (P = 0.85, η² = 0.001), with the interaction between modality and group not being significant (P = 0.08, η² = 0.023). Although we observed a trend towards greater recognition accuracy in controls for all dynamic emotions, this effect was only statistically significant for neutral (P < 0.01, η² = 0.146) and fear (P = 0.05, η² = 0.094). In patients, no effect of modality was observed for any emotion (all P-values > 0.35 and η²s < 0.026). When each emotion was considered separately, comparisons showed that with the exception of happiness, sadness, fear and contempt, patients performed significantly more poorly than controls on the recognition of emotions in the static modality, as shown in Fig. 1 and Supplementary Table 2. The same results were observed in the dynamic modality, except that fear was significantly better recognized by controls. Additional results regarding positive/negative and basic/self-conscious emotions are provided in the Supplementary material.

Figure 1.

Facial emotion recognition. Individual and group performance of controls (orange triangles and boxes) and patients with svPPA (blue dots and boxes) at the Amsterdam Dynamic Facial Expression Set for positive (happiness, pride), non-positive/negative (surprise, neutral), and negative (sadness, anger, fear, contempt, embarrassment, disgust) emotions in static (top) and dynamic (bottom) presentations. In the box and whisker plots, the rectangle represents the interquartile segment; group mean is indicated by a cross and median by a broken line within the box. Red asterisks represent the extent of the difference: *η² > 0.150; **η² > 0.250; ***η² > 0.400.

Valence errors

In both groups, valence errors were less frequent than non-valence errors, but there was an interaction between group and type of error (P < 0.001, η² = 0.145), as patients committed three times more valence errors than control subjects (Supplementary Table 5). There was no interaction with either modality or type of emotion (basic, self-conscious) in patients. In controls, however, non-valence errors were higher for self-conscious emotions, but only in the static modality (P < 0.005), implying that a dynamic presentation made it easier for this group to identify self-conscious emotions. When the total EEC score was considered as a covariate in the comparison between patients and controls, the difference in valence errors no longer reached significant difference.

Correlation with semantic processing

In patients with svPPA, there was a strong and significant correlation between the total FER and EEC scores (r = 0.80, P = 0.0001). FER and EEC were also correlated with category fluency score (r = 0.68, P < 0.005 and r = 0.65, P < 0.01, respectively). Finally, we observed a significant correlation between EEC and valence errors (r = –0.63, P = 0.01). In controls, a significant correlation between FER and category fluency score was observed (r = 0.56, P = 0.01) (Fig. 2). Other correlations (presented in the Supplementary material) were non-significant.

Figure 2.

Correlations plot. Correlations plot for svPPA (top, blue dots) and Control (bottom, orange triangles) groups between FER or valence errors committed during FER and semantic processing, including EEC and category fluency. Asterisks indicate significant correlations at **P < 0.0001 and *P < 0.01.

Neuroimaging findings

Group contrast

A direct contrast between patients and controls revealed an extended bilateral (predominantly left-sided) grey matter cluster (35 179 voxels, peak coordinates 27, 59, 11) encompassing the temporal poles, middle and inferior temporal gyri, fusiform gyrus, insula, subcallosal cortex and striatum (mainly in the putamen). A second bilateral and predominantly left-sided cluster (1013 voxels, peak coordinates 47, 32, 43) encompassed the calcarine sulcus, lingual gyrus and precuneus (Fig. 3).

Figure 3.

VBM (A) and DTI (B) contrasts between patients with svPPA and controls (P < 0.05 FWE-corrected) on sagittal and coronal views.

Nine clusters of decreased fractional anisotropy (ranging from 104 to 13 576 voxels) were identified in the right temporal superior longitudinal fasciculus, fornix, corticospinal tract and superior longitudinal fasciculus, as well as in the bilateral inferior fronto-occipital fasciculi, inferior longitudinal fasciculi in the temporal pole, corpus callosum (genu and anterior body), uncinate fasciculi, forceps minor, and anterior thalamic radiation.

Correlations between grey matter density, fibre degeneration and evaluation of emotional concepts

A left-sided grey matter cluster encompassed the lateral orbitofrontal cortex, anterior insula, striatum, and temporal lobe (from polar to temporo-occipital regions), as well as the amygdala, hippocampus, parahippocampal gyrus, and thalamus (Fig. 4 and Supplementary Table 6). No white matter correlates were identified.

Figure 4.

Grey matter correlates of EEC and FER (A), and white matter correlates of FER (B). Both at P < 0.05 FWE-corrected. POST = posterior.

Correlations between grey matter density, fibre degeneration and facial emotion recognition

As differences between controls and patients on scores, as well as correlations between emotion recognition and grey matter density, were similar for the static and dynamic presentations (Supplementary Fig. 1), the scores for these two modalities were summed in the analyses. As we were not interested in establishing the neural correlates of discrete emotions, but aimed instead to describe the neural mechanisms elicited during FER, we decided to group the scores for the different emotions under positive or negative emotion scores, and basic or self-conscious emotion scores (Fig. 4 and Supplementary Tables 7 and 8).

Total facial emotion recognition score

A grey matter cluster encompassed the bilateral subcallosal and orbitofrontal cortices, anterior insula, striatum, temporal pole, amygdala, hippocampi, anterior temporal lobe, and thalami. This cluster was more extensive on the left, involving the entire temporal lobe (including the superior, middle, inferior and fusiform gyri), posterior insula, and lateral occipital cortex.

Correlations with fractional anisotropy encompassed the bilateral forceps minor, uncinate fasciculi, inferior fronto-occipital fasciculi, anterior thalamic radiation, and left cingulum.

Recognition of positive emotions

Two grey matter clusters were identified, encompassing very similar regions to those identified in the previous analysis, but to a much smaller extent, and without the subcallosal and occipital cortices. No fractional anisotropy correlates were retrieved.

Recognition of negative emotions

Three grey matter clusters encompassing the same regions as for the FER total score were found. Correlations with fractional anisotropy involved left tracts only, including the uncinate, inferior fronto-occipital fasciculi, anterior thalamic radiations, forceps minor and cingulum.

Recognition of basic emotions

Grey matter correlates included the same regions as for the total FER score, in addition to the left supramarginal gyrus and posterior right superior temporal gyrus. The white matter fibres involved were identical to those involved in the recognition of negative emotions.

Recognition of self-conscious emotions

Grey matter correlates were identified in the left lateral orbitofrontal cortex, anterior insula, striatum, amygdala, temporal lobe, anterior parts of the hippocampus and parahippocampal gyrus, and lateral occipital cortex. A right-sided cluster also encompassed the amygdala.

Two clusters of fractional anisotropy correlates involved the tracts involved in the total FER score, in addition to the left corpus callosum.

Correlations between grey matter density, fibre degeneration and valence errors

Valence errors correlated with a grey matter cluster involving the left lateral orbitofrontal, insula, striatum, and temporal lobe (from polar to temporo-occipital regions), as well as the amygdala, hippocampus and parahippocampal gyrus, and lateral occipital cortex (Fig. 5 and Supplementary Table 9). No white matter correlates were found.

Figure 5.

Grey matter correlates of EEC (blue) and valence errors (green) on the FER. Both at P < 0.05 FWE-corrected represented on a 3D view (A) or on axial slices (B).

Grey matter density overlap analyses

To identify common grey matter regions implicated in FER or valence errors, and EEC, we ran overlap analyses. The overlap analysis between FER and EEC revealed common regions of atrophy in the left thalamus, orbitofrontal (lateral) cortex, anterior insula, anterior striatum (caudate, putamen, accumbens), amygdala, hippocampus, parahippocampal gyrus, and temporal pole—extending to the superior, middle, inferior and fusiform (all anterior and posterior parts) gyri. Overlap analysis between valence errors and EEC highlighted the same regions as in the previous analysis, except for the thalamus and posterior superior temporal gyrus. Conjunction analyses (corrected at P < 0.01) focusing on FER and EEC or EEC and valence error results yielded very similar results, as showed in Supplementary Figs 2 and 3 and Supplementary Tables 10 and 11.

Grey matter density correlation specificity analyses

To rule out the possibility that disease severity explained the relations between EEC and FER/valence errors, and to check the specificity of this relationship, mean grey matter values were extracted from the results maps obtained with EEC and the number of errors on the Brixton test. Grey matter values for EEC correlated significantly with the values for FER or valence errors, but not with the values for Brixton errors. No correlations were observed between grey matter values for Brixton errors and FER, valence errors or EEC grey matter values (Supplementary Fig. 4).

Discussion

Exploring the contribution of semantic knowledge to FER and valence processing during FER was the central aim of the present study. We therefore assessed emotion concept knowledge with the EEC and found lower scores in patients with svPPA. FER performance was also retrieved to be significantly decreased in patients. Notably, a strong correlation between total EEC and FER scores was observed in patients, thus indicating that semantic deficits play a critical role during FER in patients with svPPA. Through the analysis of participants’ responses in the FER task, we observed an impairment of valence processing in svPPA, which covaried with the deterioration of emotions conceptual knowledge. In addition, neuroimaging analyses retrieved that EEC, FER and valence error measures were related to a reduction of grey matter density in similar areas within frontal ventral, temporal, insular and striatal regions. Taken together, these findings indicate that the recognition of emotions is guided by conceptual knowledge of emotions, an ability that seems also at play in valence processing. This therefore supports a constructionist view of emotion recognition and valence processing.

In more detail, consistent with previous studies, we observed significant FER deficits in patients with svPPA. However, whereas previous results mostly emphasized impaired recognition of negative emotions (cf.Irish et al., 2013), we also observed impaired recognition of positive emotions—a result that may have been favoured by the higher number of positive emotions considered in our study. Extending previous findings, we observed an impairment for both basic and self-conscious emotions in patients with svPPA, although the differences from control subjects were generally greater for basic emotions. Whereas the dynamic presentation of emotions served as a facilitator for controls (especially for self-conscious emotions), thereby increasing their recognition accuracy compared with the static presentation, no such effect was observed in patients, who exhibited similar impairments in both presentation modalities. Coupled with a previous investigation of the impact of intensity on recognition performances (Kumfor et al., 2011), this finding suggests that attentional or perceptual deficits are not responsible for emotion recognition deficits in patients with svPPA.

Besides the general semantic knowledge impairment observed in patients with svPPA, we found lower EEC scores, showing that taxonomic and contextual knowledge on emotions were decreased in svPPA in comparison to controls. Critically, FER and EEC were strongly correlated. While the link between semantics and FER was already suggested by previous studies that investigated the correlation between object naming and emotion recognition in svPPA (Hsieh et al., 2012), formulating a conclusion about this specific relationship was not possible before given the inconsistent past results (Kumfor et al., 2018). In our study, assessing emotion conceptual knowledge specifically allowed us to show that emotion recognition depends on acquired conceptual knowledge about those emotions. Interestingly, FER performance was also correlated with category fluency in both patients and controls. As this task, often called ‘semantic fluency’, relies on exploiting existing links between related concepts (between the category label and the category members as well as among the category members) to generate responses, we believe that this result strengthens the observed link between semantics and FER. In addition, to study the possible impact of emotion concept knowledge on valence processing during FER, we performed a qualitative analysis of responses given by participants, and focused our investigations on valence errors. The number of valence errors on the FER test was significantly higher in the patient group, where it was also significantly correlated with EEC. This result indicates that valence processing during FER is impaired in patients with svPPA, a finding that contrasts with Lindquist et al. (2014), but is in line with Macoir et al. (2019). These two studies explored the recognition of the valence of emotional faces or scenes. Whereas the former concluded that valence processing is preserved in svPPA, the second concluded that valence processing is impaired. In our study, this disturbance of valence processing has to be considered in relation to the semantic deficit, as indicated by the correlation we observed between EEC and valence errors and the disappearance of group differences on the number of valence errors when we controlled for EEC performance. This strongly suggests that valence processing in patients with svPPA is not independent of the semantic processing of emotions.

A multimodal imaging approach allowed us to explore grey and white matter neural correlates of FER performance. Overall, the clusters we retrieved were left-lateralized. The entire temporal lobe, orbitofrontal cortex (including the subcallosal cortex), insula, striatum, thalamus and lateral occipital regions were correlated with FER performance, together with white tracts that mostly connected ventral frontal regions (e.g. forceps minor), as well as frontal and temporal regions (e.g. uncinate fasciculus), although long fibres such as the fronto-occipital fasciculus were also identified. In svPPA, the importance of the right uncinate fasciculus in FER has been suggested previously (Multani et al., 2017), alongside the role of the right anterior thalamic radiation and superior longitudinal fasciculus (Downey et al., 2015). By analysing correlations to measures reflecting different modalities, types of emotions and valence, we were able to identify more tracts involved in FER, notably the forceps minor, inferior fronto-occipital fasciculi and cingulum. Fewer regions and tracts were correlated with the recognition of positive emotions, compared with the recognition of negative emotions, for which the imaging results were stackable to those for FER—a result we had expected, given that recognition of negative emotions accounted for most of the FER score variance. The same effect was observed for basic versus self-conscious emotions, the latter being correlated with fewer regions and tracts, but also involving fewer emotions.

Overall, these findings are consistent with the literature on svPPA, which is mostly characterized by temporal, insular and orbitofrontal involvement (Kumfor and Piguet, 2012). Interestingly, the network retrieved to be involved in FER in our study seems very similar to the selective network involved in svPPA (Seeley et al., 2009). This would suggest that FER or any task that would involve social affective and conceptual processing together could thus represent efficient indicators of disease progression or measures for evaluating disease-modifying therapies. However, the bilateral but predominantly left-sided correlates retrieved here contrast with the preferentially right-sided correlations found previously. Left/right asymmetry has been a central finding in several studies conducted among patients with svPPA (Thompson et al., 2003; Josephs et al., 2009; Irish et al., 2013; Kamminga et al., 2015), and has led some to hypothesize that the right anterior temporal lobe is specialized for emotions and sometimes social cognition. However, an FER deficit has also been documented in left variant svPPA (Kumfor et al., 2013, 2016; Lindquist et al., 2014), as has the bilateral or left grey matter/white matter involvement in svPPA (Downey et al., 2015). Even though empirical findings suggest that right anterior temporal lobe atrophy is a key mechanism behind the FER deficit in patients with svPPA (Kumfor et al., 2016), we believe that these findings are not sufficiently substantial for us to dismiss the involvement of the left anterior temporal lobe in FER and, more generally, in social cognition. Like the right hemisphere hypothesis of emotions (Mills, 1912) that has not received convincing support from recent meta-analyses (Murphy et al., 2003), the hypothesis of lateralized anterior temporal lobe involvement in social versus general cognition has received little support from reviews (Gainotti, 2015) and meta-analyses (Rice et al., 2015). It seems that a graded hemispheric specialization for social and general semantic knowledge, rather than lateralization, is at play in the anterior temporal lobes (Rice et al., 2015; Pobric et al., 2016)—an approach that seems to reconcile clinical and functional MRI findings, as well as observations among patients with either left or right variant svPPA.

Interestingly, while no white matter correlates were identified for EEC, VBM analyses revealed a large overlap between regions correlated with this score and with FER performance. The thalamus, lateral orbitofrontal cortex, anterior insula and striatum, as well as medial, polar and ventral temporal regions, were found to covary with both EEC and FER scores. Furthermore, these same regions were correlated with valence errors. In sum, with the exception of the thalamus and the most posterior portion of the temporal lobe, the same regions were involved in EEC, FER and valence errors. In other words, regions involved in valence recognition, conceptual knowledge of emotions, and emotion labelling were similarly distributed across a fronto-insular-striatotemporal network. Some of these regions, such as the orbitofrontal cortex, insula and amygdala, are known to play a role in value/affect processing, salience and interoception, while others, such as the lateral and polar temporal regions, are involved in semantic processing (Peters et al., 2006; Olson et al., 2007, 2013 ; Patterson et al., 2007; Bertoux et al., 2012; Wang et al., 2014; García-Cordero et al., 2016; Adolfi et al., 2017; Rudebeck and Rich, 2018; Rolls, 2019). This overlapping of regions across these different dimensions of emotion processing reflects the correlations we observed between these scores (i.e. between EEC, FER and valence errors). In addition, although the impact of disease severity could not be directly ruled out in our analyses as a pure measure of severity was not available data in our study, it is unlikely that this anatomical and cognitive intertwining could be driven by a non-specific relationship between these dimensions, such as a general effect of disease severity, as revealed by dedicated analyses of specificity. These analyses showed that mean grey matter values correlated with EEC were also correlated with FER and valence errors, but not with a fourth measure (i.e. errors on the Brixton test, where patients had impaired performances). In addition, mean grey matter values correlated with Brixton test errors were not correlated with FER and EEC scores or valence errors.

Taken together, these results support the hypothesis that valence processing and the conceptual processing of emotions are intertwined during FER, and that these two components are not independent. Our findings do not support the idea that valence is a superordinate category within semantic memory, and instead suggest that semantics has an impact on valence processing. Because of the close relationship we observed between conceptual and valence processing during FER, this study does support a constructionist view of emotional valence. This view is further strengthened by the significant overlapping of brain regions involved in both positive and negative FER, thereby contradicting the notion of a 2D representation of valence in the brain, similar to what has already been shown (Lindquist and Barrett, 2012). Interestingly, the relationship between semantic knowledge and emotion recognition as well as the brain regions retrieved to be correlated to these cognitive performances seem also in line with the Social Context Network Model that emphasizes the role of ventral frontal, insular and polar temporal regions in the key mechanisms of social information integration and prediction related to context processing (Ibáñez and Manes, 2012). While this model is rather focused on context modulation on neurocognitive phenomenology (‘context in mind’), our findings also support the notion that the formation of neurocognitive representations during development also depends on context (‘mind in context’). In that perspective, the involvement of the temporal lobe retrieved in our study could reflect both the experiential learning and conceptual knowledge of situated social cognition process (Ibáñez and García, 2018), here applied to emotion categories and valence. Indeed, while the processing of contextual information through a fronto-temporo-insular network has been recently identified to guide social decision-making (O’Callaghan et al., 2016; Melloni et al., 2016; Ibáñez et al., 2017), our findings suggest that the disintegration of conceptual knowledge about the social world (in our study, emotional knowledge) could also have a prominent role in social cognition impairment. This fits nicely with the concept of intercognition (Ibáñez, 2019) that pleads against an isolationist view of cognition but rather tries to consider cross-domain synergies to explain behaviour. By conducting our investigations on svPPA, a disease characterized by semantic difficulties and a fronto-temporo-insular involvement, this study brought new clinical and lesion data to support the relevancy of the Social Context Network Model beyond data mainly originated from behavioural frontotemporal dementia or autism spectrum disorder populations (Baez et al., 2017).

Our study nevertheless had three main limitations that need to be highlighted. First, despite the fact that svPPA is a rare disease and our sample size was among the largest in the literature, the relatively small number of patients limited the statistical investigations that could be performed and required the inclusion of controls to boost statistical power during the imaging analysis. Second, our study did not reveal any significant group differences for sadness and contempt. Although there is a lack of data for the second emotion, previous studies have consistently reported a decrease in sadness recognition in svPPA (Kumfor et al., 2011, 2013), and the result observed in our study may thus be attributable to the specificity of the material we used, as performance of controls was lower than expected. This bring us to a third, more general, limitation, which was our reliance on the FER test, in which emotions are expressed by actors, and are thus more caricatural than real-life emotions, which mostly rely on far subtler and quicker changes linked to the dynamics of social interactions.

We believe that in order to specifically address the nature and extent of interactions between concept and valence in FER, future studies should seek to overcome the current limitations encountered by most studies in the field, especially those conducted in a neurodegenerative context. They should examine whether valence is hierarchically higher than emotion categories in the conceptual organization of emotional knowledge—a hypothesis that was not supported by our findings, and which has been recently challenged by others for both the facial and vocal modalities (Cowen et al., 2019; Cowen and Keltner, 2020). Similarly, determining whether the neural bases of valence recognition are restricted to a specific value- or reward-processing network or are distributed across semantic regions, as suggested by our study, would ultimately deepen our knowledge not only of valence and emotions, but also of moral judgements, beliefs, social norms, and cultural differences (Sharot and Garrett, 2016; Cowen et al., 2019). We believe that relying on neurodegenerative diseases as progressive lesion models remains an interesting and valuable approach (Hornberger and Bertoux, 2016), especially as neurodegeneration selectively targets neural networks (Seeley et al., 2009). Our study is a good illustration of this last point as the involvement of a network rather than a lesion in a specific site constituted an ideal setting to explore the interactions between several regions and functions. Nevertheless, for similar reasons, a cross-disorder context should perhaps be adopted, as considering different conditions with different performances and atrophy locations would not only help to increase the sample sizes to obtain more reliable statistics, but would also add more variability to the different measures, regardless of whether these measures addressed the same psychological constructs.

Regarding the material, the design of specific tasks should be envisaged. In this regard, we believe that more ecological emotion recognition tasks should be used, such as those relying on dynamic expressions but above all, featuring true and not caricatural expressions, in contrast to the majority of tasks used at present, including ours. As studies usually involve six or seven emotion categories, we also call for more emotion categories to be included in these tasks, in order to alleviate the generally high proportion of negative items in FER, and avoid happiness being the sole positive emotion. As language impairment is a common confounding factor in the field, the use of emoticons/prototypical emotions as buttons for possible answers during discrimination tasks might be considered, although this approach may be more appropriate for future generations. An overlooked dimension in the study of emotion that could also provide relevant findings to better understand the link between semantic deficits and FER is the ability to imitate facial emotions. So far, although this ability seems to rely partially on regions similarly involved in FER, it has been described to be relatively preserved in a small group of svPPA patients (Gola et al., 2017). Finally, as context is a crucial dimension when dealing with emotional material (Ibáñez and Manes, 2012), and because contextual modulations have been underlined as key factors for FER performance (e.g. in svPPA, see Kumfor et al., 2011, 2018), future tasks should consider the influence of context in the valence-label relationship, especially when contrasting basic and self-conscious emotions.

In conclusion, the present study, which was designed to explore the relationship between valence processing and conceptual processing during FER in patients with svPPA through a qualitative analysis of performances and error rates, as well as multimodal brain imaging methods, highlighted a close relationship between emotion recognition, valence processing, and emotion concept knowledge in svPPA. These three intercorrelated dimensions of emotion processing significantly overlapped, specifically in their neural correlates. Although neurosciences generally adopt an isolated, context-free, static and universalistic view of cognitive processes (Ibáñez and García, 2018), this study shows that cognitive functioning and social cognition in particular, strongly relies on semantic knowledge. In underlining the intertwining between affective and cognitive processes, emotion recognition and semantic memory functions and ultimately social cognition and language or memory domains, our findings not only plead against the general trends to compartmentalize cognitive processes, functions or domains, but also underline the importance of contextualized cognition, in light with constructionist theories and the Social Context Network Model. Both the behavioural and imaging findings of our study therefore support a constructionist view of emotions and emotional valence and contradict the idea that valence is a superordinate emotion category within semantic memory.

Funding

The French Ministry of Health supported this research (PHRC, ID-RCB: 2011-A00681-40). M.B. was supported by the University of Caen-Normandie.

Competing interests

The authors report no competing interests.

Supplementary material

Supplementary material is available at Brain online.

Glossary

DTI =

diffusion tensor imaging

EEC =

evaluation of emotional concepts

FER =

facial emotion recognition

svPPA =

semantic variant of primary progressive aphasia

VBM =

voxel-based morphometry