Emotion Recognition in Objects in Patients with Neurological Disease

Abstract

Objective.

Considerable research indicates that individuals with dementia have deficits in the ability to recognize emotion in other people. The present study examined ability to detect emotional qualities of objects.

Method.

52 patients with frontotemporal dementia (FTD), 20 patients with Alzheimer’s disease (AD), 18 patients awaiting surgery for intractable epilepsy, and 159 healthy controls completed a newly-developed test of ability to recognize emotional qualities of art (music and paintings), and pleasantness in simple sensory stimuli (tactile, olfactory, auditory), and to make aesthetic judgments (geometric shapes, room décor). A subset of participants also completed a test of ability to recognize emotions in other people.

Results.

Patients with FTD showed a marked deficit in ability to recognize the emotions conveyed in art, compared with both healthy individuals and patients with AD (relative to controls, deficits in patients with AD only approached significance). This deficit remained robust after controlling for FTD patients’ ability to recognize pleasantness in simple sensory stimuli, make aesthetic judgments, identify odors, and identify emotions in other people. Neither FTD nor AD patients showed deficits in recognizing pleasant sensory stimuli or making aesthetic judgments. Exploratory analysis of patients with epilepsy revealed no deficits in any of these domains.

Conclusion.

Patients with FTD (but not AD) showed a significant, specific deficit in ability to interpret emotional messages in art, echoing FTD-related deficits in recognizing emotions in other people. This finding adds to our understanding of the impact these diseases have on the lives of patients and their caregivers.

Introduction

Recognizing and responding to the emotional properties of environmental stimuli is important for successfully navigating the external world and can play a valuable role in supporting well-being. This broad category of functions includes several related but distinguishable abilities. At the simplest level, people can enjoy pleasant simple stimuli, such as the soft texture of a cashmere blanket or the scent of a rose. People also respond to more complex aesthetic features of the environment, such as the harmony of colors in a room. At a more sophisticated level, people can recognize emotions expressed by others through facial expressions and tone of voice and can even detect emotional messages communicated through the formal features of art (e.g., perceiving a piece of music as “happy” or “sad”). These abilities can provide much enjoyment in life, and even contribute to resilience and well-being during times of stress (e.g., Folkman, 1997; Shiota, 2006).

Much research has documented deficits among individuals with frontotemporal dementia (FTD) in aspects of empathy toward other people, including the ability to recognize their emotions (e.g., Goodkind, et al., 2015; Rosen, et al., 2004). Less is known about the implications of FTD and other neurological disorders for responding to emotionally relevant properties of objects in the environment such as art, aesthetic judgments, and simple sensory stimuli. The current study aims to help address this gap.

Emotional Responses to Aesthetics and Art

Human responses to the emotional properties of objects in the environment can take two broad forms: appreciating the object’s sensory or aesthetic pleasantness (vs. unpleasantness); and decoding its emotional message or tone (i.e., the emotion it communicates). While people can evaluate even very simple stimuli in terms of pleasantness, such as geometric shapes and scents, works of art (including music and painting) are often intended by the artist to convey a specific emotional quality, such as joy or anguish. Abilities to evaluate stimulus pleasantness and to decode emotional content in art overlap to some extent, but reflect distinct adaptive functions, and may be supported by distinct psychological and neural mechanisms.

The sensory stimuli we identify as pleasing are typically those that helped sustain the lives of our ancestors, activating approach behaviors that are critical for adaptive fitness (Shiota, et al., 2017). For example, the human sense of smell is closely linked to the sense of taste (Shepherd, 2006), and the most strongly preferred scents cross-culturally are those associated with palatable food (Gang, 2005). Similarly, people prefer soft, fuzzy textures rather than rough ones (Essick, et al., 2010), reflecting an innate preference for the feel of mammalian parents’ skin and fur (Harlow, 1958). In the domain of sound, people prefer individual tones and consonant (vs. dissonant) combinations of tones that are characteristic of human speech (Schwartz, Howe, & Purves, 2003). In the visual domain, people tend to prefer figures that are symmetrical (an indicator of health and developmental stability in living organisms; Moller, 1997); moderately complex, with sharper angles between sides and a combination of concave and convex angles (Friedenberg & Bertamini, 2015); and perceptually balanced around the geometric center of the image (Friedenberg & Bertamini, 2015).

In contrast, the ability to perceive emotional messages in art may be a byproduct of our ability to decode emotional expressions in other people and use metaphors to describe emotional feelings. For example, the acoustic properties of music that convey emotion have been found to correspond to those of the human voice that serve the same purpose (Juslin & Laukka, 2003). In the visual arts, simple properties of color and line are perceived as expressing affect in the same two-dimensional valence and arousal space people use to describe their own feelings (Collier, 1996). Even in verbal language, metaphors involving simple sensory experiences are more effective at evoking emotions than literal statements with the same meaning (Citron & Goldberg, 2014). The ability to associate simple stimulus properties with emotional meaning through metaphor, combined with the ability to perceive the emotion expressed by a being other than the self, may combine to produce the ability to decode the emotional meaning of art.

The available research on neural structures associated with these functions suggests some overlap, and some differentiation. A rich body of research identifies opioid receptor-rich regions of the nucleus accumbens and ventral palladium as highly responsive to sensory enjoyment or “liking” of simple stimuli such as pleasant tastes (Berridge & Kringelbach, 2013). Studies of neural activity during judgments of visual aesthetic preference, whether for simple geometric shapes or complex paintings, identify the orbitofrontal cortex (OFC) as an area that is more active while viewing preferred stimuli (e.g., Jacobsen, Schubotz, Hofel, & Cramon, 2006; Kawabata & Zeki, 2004). This is consistent with the evidence implicating OFC activation in stimulus valuation more generally, such as in the context of economic games, assessments of physical beauty, and aesthetic judgments (Chatterjee, 2011; FitzGerald, Seymour, & Dolan, 2009; Stalnaker, Cooch, & Schoenbaum, 2015). Multiple studies have also documented greater activity in anterior cingulate cortex (ACC) while viewing preferred relative to non-preferred paintings (Kawabata & Zeki, 2004; Vartanian & Goel, 2004).

Meta-analysis results suggest that processing of linguistic metaphor, relative to literal language, is associated with heightened activation in the inferior frontal gyrus (IFG), the amygdala, and the superior temporal gyrus (STG), among other regions (Bohrn, Altmann, & Jacobs, 2012; Citron & Goldberg, 2014). The latter of these are noteworthy as heightened amygdala and STG activation are also commonly seen in emotion recognition tasks with human face stimuli (Adolphs, 2002)(Adolphs, 2002). Empirical reviews and meta-analysis results further suggest that the amygdala, anterior insula, OFC, and regions of the ACC and medial prefrontal cortex show heightened activation during empathy tasks, although the exact profile of activation depends on specific task demands (e.g., infer versus feel the target’s emotion; Adolphs, 2002; Fan, Duncan, de Greck, & Northoff, 2011; Zaki & Ochsner, 2012).

Emotion Processing in Neurological Patients

Neurodegenerative diseases can affect brain circuitry that is critical for emotion processing (Rosen & Levenson, 2009). A number of studies have found that individuals with Alzheimer’s disease (AD) and frontotemporal dementia (FTD) have deficits in the ability to recognize emotion in other people (e.g., Fernandez-Duque & Black, 2005; Goodkind, et al., 2015; Hargrave, Maddock, & Stone, 2002; Hsieh, Hodges, & Piguet, 2013; Keane, Calder, Hodges, & Young, 2002; Lavenu, Pasquier, Lebert, Petit, & Van der Linden, 1999; Ostos, Schenk, Baenziger, & von Gunten, 2011; Rosen, et al., 2004; Rosen, Perry, et al., 2002; Weiss, et al., 2008; Werner, et al., 2007). When patients with AD and FTD have been compared, these deficits appear to be more pronounced in FTD (Goodkind, et al., 2015; Lavenu, et al., 1999). Among other neurological diseases, deficits in ability to recognize emotion in people have also been reported in individuals with epilepsy (Meletti, et al., 2003; Reynders, Broks, Dickson, Lee, & Turpin, 2005) and amygdala lesions (e.g., amygdala lesions; Adolphs, et al., 1999). In addition, similar deficits have been found in individuals with a number of psychiatric disorders (e.g., Ashwin, Chapman, Colle, & Baron-Cohen, 2006; Chan, Li, Cheung, & Gong, 2010; de Almeida Rocca, van den Heuvel, Caetano, & Lafer, 2009).

One of the intriguing findings in this literature has been that individuals with dementia have deficits in recognizing negative emotions in other people, but that the ability to recognize positive emotions may be preserved (Fernandez-Duque & Black, 2005; Kessels, et al., 2007; Lavenu, et al., 1999; Lough, et al., 2006). Although evolutionary accounts of emotion often emphasize the importance of negative emotions (which help us survive when confronted with threats), the ability to detect positive emotions is extremely important as well. Expressions of positive emotion signal potential relationship partners who may be committed to our welfare and able to provide important resources (e.g., Gonzaga, Keltner, Londahl, & Smith, 2001). Given the needs of individuals with dementia for care and assistance by others, preservation of the ability to recognize positive emotion would be important to document. However, as we have suggested elsewhere (Goodkind, et al., 2015), findings of preserved recognition of positive emotions may result from a structural asymmetry in most tests of emotion recognition. These tests typically utilize photographs of emotional facial expressions that have multiple negative emotions but only a single positive emotion (happiness portrayed by a smiling face), thus, making the detection of positive emotions much easier. Utilizing a film-based emotion identification test that assessed multiple positive (e.g., affection, amusement), negative (e.g., anger, fear), and self-conscious (e.g., embarrassment, pride) emotions, we (Goodkind, et al., 2015) found deficits in recognition for all three kinds of emotion in patients with FTD.

A much smaller group of studies has examined dementia patients’ ability to identify the emotional qualities of non-face targets, such as simple sensory stimuli and works of art (e.g., Boutoleau-Bretonniere, et al., 2016; Cohen, et al., 2016; Dellacherie, Ehrlé, & Samson, 2008; Omar, et al., 2011). For example, Boutoleau-Bretonniere and colleagues (Boutoleau-Bretonniere, et al., 2016) found that FTD patients generally rated abstract paintings as less pleasant and less emotionally moving than did non-patient controls. Cohen and colleagues (Cohen, et al., 2016) found that FTLD patients received lower scores than controls on a match-to-sample task relying on the affective valence of abstract art stimuli, with normative valence established through pilot research. Although these studies consistently identify deficits in FTD patients relative to controls, it is not clear from the tasks whether the deficit lies in evaluating the pleasantness/enjoyability of stimuli (i.e., which is more pleasant?), in decoding the emotional message of the stimuli (i.e., which communicates happiness?), or both. The present study uses a novel test that helps tease these two processes apart.

Comparing emotion recognition in different neurological disorders

As noted above, most studies of the ability of dementia patients to recognize emotion in people have been conducted with FTD or AD patients. FTD is a neurodegenerative disease that causes atrophy in the frontal and temporal lobes (Zhang, et al., 2009). In the initial stages of the disease, FTD patients show profound changes in emotion, personality, and behavior, including loss of empathy, loss of social awareness, disinhibition, apathy, loss of insight, and impaired emotion recognition (Rankin, Kramer, & Miller, 2005; Snowden, et al., 2001). In contrast, AD causes atrophy in the medial temporal and parietal lobes (Zhang, et al., 2009). In the initial stages, AD patients show marked deterioration of memory and cognitive abilities, with relative preservation of emotional functioning (e.g., Bucks & Radford, 2004; Goodkind, Gyurak, McCarthy, Miller, & Levenson, 2010; Goodkind, et al., 2015; Lavenu, et al., 1999). Although epilepsy is not considered to be a form of dementia, it is often accompanied by emotional changes (e.g., depression and anxiety; Hixson & Kirsch, 2009; Kanner, 2009). Moreover, seizure foci in epilepsy can occur in regions (e.g., temporal lobes) that are vulnerable in frontotemporal dementia (Zhang, et al., 2009). Despite the obvious virtue of comparing multiple neurological conditions, we are not aware of previous studies of emotion recognition that included FTD, AD, and epilepsy patients as well as healthy controls.

The present study

The present study assessed the ability of neurological patients with FTD and AD to recognize emotion in art (paintings and music) and pleasantness in simple sensory stimuli (olfactory, auditory, tactile), and to make aesthetic judgments (geometric shapes, room decor), in comparison to a group of healthy controls, and, in an exploratory analysis, a group of epilepsy patients awaiting surgery for intractable seizures. In a subgroup of the patients with FTD and AD, the ability to recognize emotion in facial expressions was also assessed.

Methods

Participants

Participants were 52 patients with FTD (25 males, 27 females), 20 patients with AD (11 males, 9 females), and 159 controls (82 males, 76 females, 1 who declined to indicate gender). Among the FTD patients, 25 were diagnosed with behavioral variant FTD, 21 with semantic dementia, and 6 with primary non-fluent aphasia. Patients with FTD and AD were recruited through the Memory and Aging Center of the University of California, San Francisco (UCSF), where they received comprehensive neurological and neuropsychological testing. Because significant numbers of participants were assessed prior to the appearance of newer diagnostic criteria (e.g., Gorno-Tempini, et al., 2011; Rascovsky, et al., 2007; Rascovsky, et al., 2011), the Neary clinical criteria (Neary, et al., 1998) were used to diagnose FTD. Similarly, the National Institute of Neurological and Communication Diseases criteria and those of the Stroke/Alzheimer’s Disease and Related Disorders Association (McKhann, et al., 1984) were used to diagnose AD. Controls were adult caregivers (mostly spouses) of patients with FTD and AD who were participating in a study of emotional functioning in dementia caregivers conducted at the Berkeley Psychophysiology Laboratory at the University of California, Berkeley.

An additional group of 18 patients with intractable epilepsy (6 males, 12 females) was recruited by one of the authors (HEK) at the UCSF Medical Center. These patients were all awaiting surgery to alleviate medically refractory seizures. Including patients with intractable focal epilepsy provides affords an important test of the specificity of findings by establishing whether deficits found in patients with dementia generalize to another neurological condition.

We included patients with AD, different FTD subtypes, and epilepsy to increase the anatomical and behavioral variability in our sample. AD is characterized by neurodegeneration in hippocampus, entorhinal cortex, and posterior cingulate and may leave emotional functioning relatively intact in the early stages of disease (Goodkind, et al., 2010; Lavenu & Pasquier, 2005). Compared to AD, FTD is characterized by degeneration in more anterior brain regions and produces more profound deficits in emotional functioning (Levenson & Miller, 2007). Among the three subtypes of FTD included in our sample: (a) behavioral variant FTD is characterized by bilateral frontal lobe degeneration and significant changes in emotion and personality; (b) semantic dementia is characterized by atrophy in anterior temporal areas and language problems; and (c) primary non-fluent aphasia is characterized by predominantly left-sided frontotemporal atrophy and problems with speaking (Neary, et al., 1998; Rosen, Gorno-Tempini, et al., 2002; Rosen, Kramer, et al., 2002). Patients with epilepsy who receive surgical treatment for intractable seizures often have seizure foci in the temporal region and may experience significant emotional changes (Meletti, et al., 2003; Reynders, et al., 2005). Network dysfunction in these patients with epilepsy is thought to involve frontotemporal regions, anatomic substrates that are similar to those affected in FTD. In our sample of 18 epilepsy patients, 15 had established temporal foci with nine having a primary left hemisphere focus and nine having a primary right hemisphere focus.

Patients with dementia and epilepsy and controls in this sample participate in various research projects conducted at UCSF and at UC Berkeley. AD and FTD patients and controls in this sample all participated in a comprehensive assessment of emotional functioning (Levenson, 2007) conducted at UC Berkeley. Some data from these assessments have been published previously (e.g., Goodkind, et al., 2015). However, no data from these participants related to detecting the emotional qualities of objects have been published previously.

Measures

Emotion in objects.

We utilized the Test of Emotional Aesthetics (TEA), a 16-item test developed at the Berkeley Psychophysiology Laboratory, to assess ability to recognize the emotional qualities of art (paintings, music), simple sensory stimuli (olfactory, auditory, tactile) and to make aesthetic judgments (room décor, geometric shapes). The instructions and visual and auditory stimuli for the TEA are presented on a laptop computer.

Stimuli for the art items included two pairs of paintings and two pairs of short musical excerpts; stimuli within each pair were always by the same artist. In the paintings (Picasso: “Weeping Woman” and “The Dream”; Van Gogh: “Wheat Field with Cypresses” and “Wheat Fields with Crows”), each painting in the pair depicts a similar subject through very different formal features, thereby conveying distinct emotional messages. Specifically, whereas “The Dream” and “Wheat Field with Cypresses” use lighter, primary, saturated colors and smoother, curved lines to convey calm positive affect, “Weeping Woman” and “Wheat Fields with Crows” both use darker, non-primary, less saturated colors and sharper, more jagged lines to convey negative affect (Collier, 1996). Similarly, pairs of musical excerpts (Living Out Loud soundtrack, “Ecstasy” and “She’s 34”; Bach Goldberg Variations: #21 and #22) present the same musical theme, one with a quick tempo and major key to convey happiness, and the other with slower tempo and minor key to convey sadness (Gabrielsson & Juslin, 2003). In each art item, the participant was asked to identify which of the pair was “more happy,” relying on the ability to decode the emotional messages of the two options.

Eleven items assessed preferences for simple sensory or aesthetic stimuli: (a) Olfactory 1 – scratch and sniff: apple versus natural gas; (b) Olfactory 2 – scratch and sniff: mint versus paint thinner; (c) Auditory 1 and 2 – consonant versus dissonant piano chords; (d) Tactile 1 – reach into bags: Brillo pad versus silk; (e) Tactile 2 – reach into bags: cotton versus sandpaper; (f) Aesthetic visual judgments 1 and 2 – two rooms decorated with different color schemes; and (j) Aesthetic visual judgments 3, 4, and 5 – pairs of geometric and abstract shapes, differing on complexity and balance (one of these was taken from the Visual Aesthetic Sensitivity Test; Götz, Borisy, Lynn, & Eysenck, 1979). In each pair of items, the correct response was based on the empirically supported principles of aesthetic preference discussed in the introduction section of this paper. Figure 1 presents example stimulus pairs from the aesthetic visual items. In each preference item, participants are asked which of the two options is “more pleasing.” The 16^th item assesses Odor Identification, presenting a single scratch and sniff rose smell with three label choices: smoke, rose, or lemon.

Figure 1.

Sample Visual Aesthetics Item Stimulus Pairs from Test of Emotional Aesthetics

Note. In each item, the participant is asked to identify which of the pair is “more pleasing.” Correct answers are (A) left; (B) right.

Each of the 16 items was scored correct (1) or incorrect (0), and normative responses were validated with an independent group of healthy controls. Percentage correct scores were computed for the following subscales: (a) Art (four items); Sensory stimuli (six items); Aesthetic visual judgments (five items); and Odor identification (one item). Copies of the TEA (along with instructions for creating the tactile and odor items) are available from the authors.

Emotion in people.

The Empathy Film Test (EFT; Goodkind, et al., 2015) assesses ability to recognize emotions in other people. It consists of 11 excerpts (approximately 40 sec in length) from commercial films each featuring a character expressing one of four positive (affection, amusement, calm, enthusiasm), four negative (anger, disgust, fear, sadness), or three self-conscious (embarrassment, shame, pride) emotions. After each clip, the participant is shown a photo of the main character with a neutral facial expression and a list of the 11 target emotions and asked: “What emotion did the character feel most strongly?” Reliabilities of the EFT subscales (Cronbach alpha: positive emotions = .63; negative emotions = .72, self-conscious emotions = .65) have been published previously (Goodkind, et al., 2015).

The EFT has several advantages over traditional tests of emotion recognition (e.g., Bowers, Blonder, & Heilman, 1991) in which respondents have to provide the correct emotion label (e.g., “happy”) for a static photograph of an isolated emotional facial expression (e.g., a single smiling person). The EFT has arguably greater ecological validity by virtue of assessing respondents’ ability to recognize emotion using dynamic, multi-modal (visual, auditory), and interpersonal stimuli. These conditions more closely mirror real-world emotional judgments compared to photographs, which are static, convey only visual information, and are usually not depicted in interpersonal situations (Goodkind, et al., 2015). Moreover, photograph-based tests typically assess multiple negative emotions (e.g., anger, fear, sadness, disgust) but only assess a single positive emotion (happiness as portrayed by a smiling face) and do not assess self-conscious emotions. In contrast, the EFT assesses multiple positive and multiple self-conscious emotions in addition to multiple negative emotions. Thus, in addition to its ecological validity, the EFT can provide a more comprehensive test of the ability to recognize emotion in people.

Each response on the EFT was scored correct (1) or incorrect (0) and a percentage correct score was determined for the following subscales: (a) Positive emotion (four items), (b) Negative emotion (four items), and (c) Self-conscious emotion (three items). Because the assessment of ability to recognize emotion in the present study focused primarily on positive emotion (i.e., happiness in art, pleasantness in sensory stimuli and aesthetic judgments), we used the Positive emotion score from the EFT in our primary analyses to control for deficits in emotion recognition in people when assessing deficits in emotion recognition in objects using the TEA. In secondary analyses, we also used the Negative and Self-conscious emotion scores from the EFT.

Procedures

Patients with FTD and AD participated in a comprehensive assessment of emotional functioning (Levenson, 2007) at the Berkeley Psychophysiology Laboratory. As part of this assessment, they completed the TEA and the EFT. Caregivers who accompanied the patients also completed the TEA and the EFT. Epilepsy patients completed the TEA at the UCSF Medical Center but did not complete the EFT. All participants gave informed consent prior to participation. The study was approved by the Committee for the Protection of Human Subjects at the University of California, Berkeley and the University of California, San Francisco.

Data Analysis

Analyses initially focused on the FTD and AD patients and controls because these groups had completed both the TEA and the EFT. An overall 3 x 4 (Diagnosis [FTD, AD, controls] x TEA subscale [Art, Sensory stimuli, Aesthetic judgments, Odor identification) repeated measures MANOVA was computed using the General Linear Model procedure in SPSS Version 24. In this MANOVA, TEA subscale was treated as a repeated measure and all main effects and interactions were analyzed using multivariate F tests (which obviate the need to control for violations of sphericity in the repeated measures). Significant effects were followed up by comparisons among means using least-significance difference tests and 95% confidence intervals (CI).

Results

Preliminary analyses

Psychometrics.

Because the TEA has not been used in prior published research, we conducted basic psychometric analyses using our sample of 159 healthy controls. These revealed that the internal consistency of the 16-item test was moderate (Cronbach alpha = .45). We expect this reflects both restriction in range in our neurologically healthy sample and the TEA’s assessment of different kinds of emotional judgments. Consistent with range restriction, for controls the percentage correct score averaged across all 16 items was 84% (SE=.89, range= 44%-100%) with the lowest percentage correct generally found for the aesthetic judgments of geometric shapes. An item-level reliability analysis revealed that the overall reliability of the scale was not markedly changed by removing any of the individual items.

Sample characteristics.

The average age for the sample of patients with FTD and AD and healthy controls was 62.4 years (SE = 0.6). This sample was well-educated (mean years of education = 16.5, SE = 0.2) and primarily Caucasian (88.0%). Analyses revealed no gender (χ²(2) = 0.35, p= .84), age (F(2,226) = .02, p = .99), or education (F (2,224)=1.43, p = .24) differences among the three groups. The sample of epilepsy patients used in the exploratory analysis was younger (mean age = 35.5, SE = 2.9) and less educated (mean years of education = 13.7, SE = 0.4).

Using data obtained from neuropsychological testing of patients with FTD and AD conducted at UCSF, mini-mental state (Folstein, Folstein, & McHugh, 1975) scores (which can range from 0-30, with lower scores indicating greater cognitive impairment) indicated moderate cognitive impairment (mean = 23.1, SE = 0.7) and clinical dementia rating (Morris, 1993) sum of boxes scores (which can range from 0 to 18, with higher scores indicating greater dementia severity) indicated moderate levels of dementia severity (mean sum of boxes = 5.0, SE= 0.4). Table 1 presents demographic and clinical data for the diagnostic groups.

Table 1.

Demographic and clinical information for diagnostic groups

	FTD	AD	Epilepsy	Controls	Total
N	52	20	18	159	249
Sex: % female	51.9	45.0	66.7	47.8	50.0
Age: Mean (SE)	62.6 (1.0)	62.2 (2.0)	35.5 (2.9)	62.4 (0.8)	60.5 (0.7)
Education: Years (SE)	16.0 (0.4)	17.0 (0.8)	13.7 (0.4)	16.5 (0.2)	16.3 (0.2)
Race/Ethnicity: %
Caucasian	86.2	88.9	n/a	88.5	88.0
Asian/Asian American	9.8	5.6	n/a	3.2	4.9
Hispanic	3.9	0	n/a	1.9	2.2
African American	0	0	n/a	1.3	.9
Native-Hawaiian/Pacific-Islander	0	0	n/a	1.3	.9
American-Indian/Native Alaskan	0	0	n/a	.6	.4
Refused to state/Unknown	0	5.6	n/a	0	.4
Clinical information: Mean (SE)
Mini-mental state examination	24.1 (0.8)	20.9 (1.0)	n/a	n/a	23.1 (0.7)
Clinical dementia rating sum of boxes	4.9 (0.5)	5.3 (0.5)	n/a	n/a	5.0 (0.4)

Note: FTD = frontotemporal dementia; AD = Alzheimer’s disease; n/a = not available; SE = standard error

Emotion in objects

FTD, AD, and Controls.

The MANOVA of the TEA revealed significant main effects for Diagnosis (F(2,228) = 21.96, p < .001) and TEA Subscale (F(3,226) = 16.88, p < .001) as well as a significant interaction of Diagnosis x TEA Subscale (F(6,454) = 7.80, p < .001). Examining the main effect for Diagnosis, both FTD patients and AD patients scored lower than controls (FTD vs controls: 95% CI = −17.15, −8.86, p < .001; AD vs controls: 95% CI = −16.73, −4.41, p = .001¹). Differences between FTD and AD patients were not significant (p = .483). Examining the main effect for TEA Subscale, performance on the Art subscale and Sensory stimuli subscale were higher than the Aesthetic judgments subscale (Art vs Aesthetic judgments: 95% CI = 8.38, 17.78, p < .001; Sensory stimuli vs Aesthetic judgments: 95% CI = 6.40, 15.27, p < .001) and Odor identification subscale (Art vs Odor identification: 95% CI = 10.63, 24.99, p < .001; Sensory stimuli vs Odor identification: 95% CI = 8.66, 22.47, p < .001).

The Diagnosis x Tea Subscale interaction was decomposed by computing univariate Diagnosis ANOVAs for each TEA subscale. These revealed that the main effect for Diagnosis was not significant for Sensory stimuli (F(2,228) = 0.57, p = .57) or for Aesthetic judgments (F(2,230) = 0.20, p = .82). However, the main effect for Diagnosis was significant for Art (F(2,228) = 23.08, p < .001) and Odor identification (F(2,228) = 19.01, p < .001). For Art, FTD patients scored lower than AD patients (95% CI = −18.86, −2.30, p = .013) and lower than controls (95% CI = −22.29, −12.23, p < .001). AD patients also scored lower than controls, but this difference only approached significance (95% CI = −14.15, .79, p = .079). For Odor identification, both dementia patient groups scored lower than controls (FTD vs controls: 95% CI = −42.57, −19.33, p < .001; AD vs controls: 95% CI = −52.82, −18.31, p < .001), but did not differ from each other (FTD vs AD: 95% CI = −14.52, 23.75, p = .635). Table 2 presents TEA data for the diagnostic groups.

Table 2.

Test of Emotional Aesthetics (TEA) and Empathy Film Task (EFT) scores for diagnostic groups

	FTD	AD	Epilepsy	Controls	Total
TEA: Mean (SE)
Overall	76.4 (1.8)	80.0 (1.8)	85.4 (2.8)	84.0 (0.9)	82.2 (0.8)
Art	76.9 (3.2)	87.5 (3.4)	93.1 (3.4)	94.2 (1.0)	90.0 (1.1)
Sensory stimuli	83.0 (2.7)	83.3 (3.0)	88.9 (3.6)	85.5 (1.2)	85.1 (1.0)
Aesthetic judgments	71.5 (2.8)	75.0 (4.1)	73.3 (5.1)	72.8 (1.8)	72.8 (1.4)
Odor Identification	59.6 (6.9)	55.0 (11.4)	94.4 (5.6)	90.6 (2.3)	81.5 (2.5)
EFT: Mean (SE)
Positive emotions	69.4 (5.3)	65.9 (9.7)	n/a	94.0 (1.7)	83.3 (2.5)
Negative emotions	76.2 (4.5)	93.2 (3.5)	n/a	97.2 (1.1)	90.2 (1.8)
Self-conscious emotions	44.1 (5.8)	59.1 (9.9)	n/a	80.2 (2.7)	66.7 (3.0)

Note: Values are mean percent correct on the indicated scale or subscale. FTD = frontotemporal dementia; AD = Alzheimer’s disease; n/a = not available; SE = standard error

Comparing FTD subtypes.

The sample sizes for FTD patients with behavioral variant FTD (N=25) and semantic dementia (N=21) were not large. Nonetheless, we thought it useful to conduct analyses to assess performance of these two FTD subtypes on the TEA and its subscales. Using univariate ANOVAs, the two FTD subtypes did not differ on overall TEA scores or on any of the subscales (Overall: F(1,44) = .25, p = .62; Art: F(1,44) = .57, p = .46; Sensory stimuli: F(1,44) = .40, p = .53; Aesthetic judgments: F(1,44) = .12, p = .73; Odor identification: F(1,44) = .01, p=.94). Moreover, there was no consistent pattern of one subtype performing better across the subscales. The smallish sample sizes require caution in interpreting this null finding, but the pattern of results does not suggest that deficits on the TEA in the larger FTD sample were driven by deficits in a particular subgroup. Table 3 presents TEA scores for the two FTD subtypes.

Table 3.

Test of Emotional Aesthetics (TEA) for FTD subtypes

	Behavioral variant FTD	Semantic dementia
N	25	21
TEA: Mean (SE)
Overall	77.0 (2.8)	75.0 (2.9)
Art	79.0 (4.0)	73.8 (5.8)
Sensory stimuli	84.0 (4.0)	80.2 (4.5)
Aesthetic judgments	71.2 (4.2)	73.3 (4.4)
Odor Identification	56.0 (10.1)	57.1 (11.1)

Note: FTD = frontotemporal dementia; SE = standard error

Summary.

Both FTD and AD patients showed lower performance overall on the TEA compared to controls. In terms of specific deficits, both FTD and AD patients showed deficits in Odor identification but no deficits in detecting pleasantness in Sensory stimuli or Aesthetic judgments. On the Art subscale, FTD patients showed significant deficits, and AD patients showed marginal deficits, compared to healthy controls. An exploratory analysis comparing TEA performance in two FTD subtypes found no evidence of differences between patients with behavioral variant FTD and semantic dementia.

Robustness of FTD deficits in recognizing emotion in art

Because not all of the participants who completed the TEA completed the EFT, this analysis was conducted using data from a smaller pool of participants (34 FTD, 11 AD, 62 controls) who completed both tests. In this smaller sample, an ANOVA of performance on the Positive emotion subscale of the EFT revealed a significant main effect for Diagnosis (F(2,104) = 16.61, p < .001) with both FTD patients and AD patients scoring lower than healthy controls (FTD vs controls: 95% CI = −34.15, −15.03 , p < .001; AD vs controls: 95% CI = −42.70, −13.38, p < .001). Table 2 presents EFT data for the diagnostic groups.

To evaluate the specificity of the deficits in recognizing emotion in art found for FTD patients, we determined whether this deficit was accounted for by deficits in ability to recognize emotion in other kinds of objects and in people. We conducted an analysis of covariance (ANCOVA) examining diagnostic group differences in performance on the Art subscale of the TEA, while controlling for the three other TEA subscales (Sensory stimuli, Aesthetic judgments, Odor identification) and for the Positive emotion subscale of the EFT. This ANCOVA revealed a main effect for diagnostic group that approached significance (F(2,106) = 2.97, p = .056). Follow-up comparisons revealed that FTD patients (estimated mean % = 81.85) still scored significantly lower than controls (estimated mean % = 91.51; 95% CI = −17.98, −1.34, p= .023), even when controlling for the four covariates.

In exploratory analyses, we determined whether FTD patients’ deficit in recognizing emotion in art would remain when controlling for their ability to recognize emotion in other objects and their ability to recognize either negative emotions or self-conscious emotions in people. We computed two ANCOVAs that controlled for the three other TEA subscales (Sensory stimuli, Aesthetic judgments, and Odor identification) as well as performance on either the Negative emotion subscale or the Self-conscious emotion subscale of the EFT. In both analyses, controlling for all four covariates, FTD patients still scored significantly lower than controls (Negative emotion on EFT: 95% CI = −19.73, −2.435, p=.013; Self-conscious emotion on EFT: 95% CI = −17.279, −.520, p=.038).

Are deficits in recognizing emotion in objects also found in patients with epilepsy?

To determine whether our TEA findings with FTD and AD patients generalized to another patient group, we recomputed our overall MANOVA adding data from the 18 patients with epilepsy. In this expanded ANOVA there were significant effects for Diagnosis (F(3,245) = 15.97, p < .001), TEA Subscale (F(3,243) = 16.39, p < .001), and Diagnosis x TEA Subscale (F(9,735) = 5.35, p < .001). Decomposing the Diagnosis X TEA interaction with univariate Diagnosis ANOVAs for each TEA subscale revealed that the epilepsy patients (see Table 2) did not differ from the controls in the Art (95% CI = −8.90, 6.65, p = .78), Sensory stimuli (95% CI = −4.47, 11.18, p = ..40), Aesthetic judgments (95% CI = −9.99, 11.00, p = .93), or Odor identification subscales (95% CI = −13.82, 21.58, p = .67). FTD patients scored lower than epilepsy patients on the Art subscale (95% CI = −24.68, −7.59, p < .001) and both FTD patients and AD patients scored lower than epilepsy patients on the Odor identification subscale (FTD vs epilepsy: 95% CI = −54.30, −15.36, p = .001; AD vs epilepsy: 95% CI = −62.57, −16.32, p = .001).

Discussion

Summary of findings

This study sought to determine whether patients with two kinds of dementia, FTD and AD, had deficits in recognizing emotional properties of objects as compared to healthy age-matched controls and, in an exploratory analysis, patients with epilepsy. Using the TEA, a newly developed test of ability to recognize emotional and aesthetic qualities in art and sensory stimuli, we found that patients with FTD showed significant and specific deficits in recognizing the emotions communicated by art (i.e., music and paintings); AD patients’ deficits on this subtest of the TEA only approached significance. Of note, FTD patients’ deficit in ability to decode emotional messages in art was quite robust, found even after controlling for the three other subscales of the TEA (Sensory stimuli, Aesthetic judgments, Odor identification) and the ability to recognize either positive emotion, negative emotion, or self-conscious emotion in other people (measured by the EFT). Both patient groups showed deficits in the ability to recognize positive emotion in other people (on the EFT), as well as deficits in identifying a specific odor. In contrast, neither FTD nor AD patients showed deficits in evaluating the pleasantness of simple sensory stimuli in normative ways, or in making aesthetic judgements for visual images.

We considered the possibility that the specific pattern of deficits shown by FTD patients in the TEA reflected item difficulty, with patients performing worst on the items that were the most difficult (i.e., aesthetic judgments of geometric shapes had the lowest percentage correct in the control sample). However, this was not the case; neither FTD nor AD patients differed significantly from controls on these items. We also considered whether the FTD patients’ deficits on the TEA might be driven by particular FTD subtypes (e.g., language problems in patients with semantic dementia). However, in an exploratory analysis we found no evidence for differential performance on the TEA between patients with behavioral variant FTD and patients with semantic dementia. Finally, to examine the possibility that deficits on the TEA found in patients with FTD and AD would also be found in other neurological disorders, we examined a group of epilepsy patients awaiting surgery to treat intractable seizures. Patients with epilepsy did not show deficits on any of the TEA subscales (they did not take the EFT). Thus, the deficits we found in FTD and AD patients did not generalize to another neurological disease that has also been found to affect emotional functioning (Hixson & Kirsch, 2009).

Emotion recognition in neurodegenerative disease

The present findings support and extend the existing literature on emotion recognition in neurological patients. Extensive prior evidence indicates deficits among dementia patients in the ability to recognize nonverbal expressions of emotion in other people – a crucial skill for successfully navigating the social world (e.g., Goodkind, et al., 2015; Lavenu, et al., 1999). Emotion recognition is subserved by complex neural circuitry (Adolphs, 2002; Preston & De Waal, 2002) and has proved to be vulnerable to a number of neurological disorders including AD and FTD (e.g., Cadieux & Greve, 1997; Goodkind, et al., 2015; Gray & Tickle-Degnen, 2010; Johnson, et al., 2007; Keane, et al., 2002) as well as a number of psychiatric disorders (e.g., Ashwin, et al., 2006; Chan, et al., 2010; de Almeida Rocca, et al., 2009). Neurodegenerative diseases like FTD and AD produce different patterns of deficits depending on the extent and location of neurodegeneration in large-scale brain networks (Seeley, Crawford, Zhou, Miller, & Greicius, 2009). For this reason, it is important to assess multiple aspects of emotion recognition (e.g., in objects and in people) in ways that are sensitive to patterns of lost and preserved functioning in different disorders and in different individuals.

Our findings reveal that patients with FTD and AD both show deficits in an understudied aspect of emotion recognition – the ability to decode emotional tone in works of art – but that these deficits are more profound and pervasive in FTD. These findings are consistent with prior studies that examined recognition of affective valence in art and abstract objects in FTD patients (Boutoleau-Bretonniere, et al., 2016; Cohen, et al., 2016; Dellacherie, et al., 2008; Omar, et al., 2011), and extend them by localizing the deficit to reading emotional tone in art, rather than appreciating the inherent pleasantness of stimuli such as scents and visual images. These deficits are particularly interesting given prior evidence that some neurological patients develop new artistic interests and skills during the course of the disease (Seeley, et al., 2008).

The ability to recognize emotional tone in art is analogous to the ability to read and respond to emotions in other people, and our findings suggest that patients with FTD show distinct deficits in both. As with empathy for other humans, describing one painting or piece of music as “happier” than another requires the ability to attribute a psychological state to something other than the self (i.e., mentalizing or theory of mind). With art, this judgment may be based on a visual or auditory metaphor and may involve having a visceral reaction to the target stimulus, using that reaction to estimate the target’s emotional “state.” These processes are often compromised in FTD. Prior research documents robust deficits in theory of mind, as well as deficits in comprehension of metaphor-laden, sarcastic, and nonliteral speech, in behavioral variant FTD in particular (e.g., Bora, Walterfang, & Velakoulis, 2015). Moreover, damage to structures such as the anterior insula that are critical for processing visceral feedback commonly accompanying emotion (Craig, 2009) can reduce the potency of experiential aspects of emotional responding (Verstaen, et al., 2016). We and others have shown that FTD patients have problems generating robust emotional responses (Eckart, Sturm, Miller, & Levenson, 2012; Sturm, Allison, Rosen, Miller, & Levenson, 2006; Sturm, Ascher, Miller, & Levenson, 2008); a deficit that may carry over into responding to emotional messages in works of art.

In contrast, we expect that judgments about stimulus pleasantness reflect a mix of hard-wired and deeply learned associations residing in brain areas that are not early targets of FTD and AD (e.g., opioid receptor-rich regions of the nucleus accumbens and ventral palladium; Berridge & Kringelbach, 2013). Although assessed only with a single item, our finding that FTD and AD patients had deficits in odor identification are consistent with previous findings of anosmia and other olfactory deficits in these and other neurodegenerative diseases (Luzzi, et al., 2007; McLaughlin & Westervelt, 2008; Mesholam, Moberg, Mahr, & Doty, 1998). Epilepsy has been associated with impairments in aspects of emotional functioning (Hixson & Kirsch, 2009; Kanner, 2009), but in our study these did not extend to problems recognizing emotion in objects. Although epilepsy and FTD may have some overlap in regions of neuroanatomical impairment, epilepsy is more chronic than degenerative; thus, preserved neural plasticity in individuals with epilepsy may allow other regions to carry out these emotional functions.

Strengths and Limitations

The present study had several strengths, including (a) assessing emotion recognition in a number of different kinds of objects, (b) assessing multiple patient groups, and (c) exploring the robustness of found deficits (by controlling for deficits in other aspects of emotion recognition in objects and people). Limitations included (a) small sample size (especially for patients with AD who completed both the TEA and the EFT, for patients with epilepsy, and for FTD subtypes), (b) lack of power for examining ethnic differences (the sample was predominately Caucasian), (c) focusing on positive emotional judgments (i.e., happiness and pleasantness) and not including negative emotional judgments (e.g., unhappiness and unpleasantness); and (d) differences in type of emotional judgments made for music and paintings (i.e., happiness) versus sensory stimuli and aesthetic judgments (i.e., pleasantness).

Concerning emotional judgments, we would expect similar findings if participants were asked to recognize negative emotional qualities rather than the positive emotional qualities we assessed. However, this clearly needs to be tested empirically. Similarly, additional research is needed to confirm that the patterns of deficits observed in this study are found specifically when assessing ability to detect the emotion conveyed by art (e.g., happiness in music) rather than its pleasantness. This may be particularly interesting in cases where an artistic stimulus conveys a negative emotion, yet is still pleasant (e.g., some very sad music is quite lovely), or vice versa.

Conclusion

The ability to recognize emotion in people and objects plays a vital role in our lives. The current study revealed that patients with FTD, and to a lesser extent those with AD, had impairments in recognizing emotions expressed in paintings and music, as compared to patients with epilepsy and controls. Deficits in the ability to recognize emotion in art may at first seem to be a fairly rarified loss. However, it is important to realize the important role that music and the visual arts play in the arsenal of emotion regulation. We often use music to alter our emotional state in desired ways, raising our spirits and deepening the experience of life’s losses. Art is often used strategically in our home and work environments to foster particular moods. Losing access to this aspect of emotion recognition can contribute to the impoverishment of emotion that is so often seen in patients with FTD, and can add to the burden and strain experienced by their caregivers (e.g., Brown, et al., 2018). These findings shed additional light on the ways that aspects of emotion recognition are preserved and diminished in different neurological diseases and add greater understanding to the profound impact that FTD can have on a wide range of emotional functions.

Public significance.

The ability to recognize emotions in others, and to enjoy pleasing sensations and aesthetic environments, is an important part of human life. Using a newly developed test of the ability to recognize emotional qualities in art, pleasantness in sensory stimuli, and to make aesthetic judgments, we found deficits in these abilities in patients with two kinds of dementia (most profoundly in frontotemporal dementia and less so in Alzheimer’s disease) but not in patients with epilepsy. With the aging population and increasing number of families caring for loved ones with dementia, it is important to understand how these diseases affect this important aspect of emotional functioning.