Emotion recognition from stimuli in different sensory modalities in post-encephalitic patients

Abstract

Emotion recognition from facial and non-facial stimuli was investigated in two post-encephalitic patients a few months after the onset of the disease. One patient who had a lesion relatively restricted to the amygdala and hippocampus experienced difficulty in recognizing fear from facial expressions. In contrast, the other patient who had a lesion that extended beyond the amygdala experienced difficulty in recognizing fear from non-facial (prosodic and written verbal) stimuli. We showed that impairment of emotion recognition was evident within a short duration after encephalitis and that recognizing emotion from different sensory modalities relies partly on integration of different neural systems.

Introduction

Limbic encephalitis has been shown to cause neuropsychological deficits, especially memory disturbances, because of damage to the medial temporal lobe. Although there have been well-documented case studies of patients with limbic encephalitis, there is little in the way of detailed documentation of emotion recognition in these patients. In addition, because previous studies have examined post-encephalitic patients a long time after the onset of symptoms, it is unknown when dysfunction begins.

In this study, we report our observations of 2 post-encephalitic patients in whom the medial temporal lobe, including the amygdala, was damaged after recovery from initial symptom. The role of the amygdala with respect to appropriate emotional behavior in humans was first reported by Adolphs et al.¹ This finding was supported by studies on individuals with impaired ability to recognize emotions from facial or prosodic expressions after amygdala damage.^2–6 However, the recognition of facial⁷ and prosodic^8,9 emotions is apparently normal in some patients with amygdala damage. Two factors related to this discrepancy have been suggested: 1) etiology (congenital vs late-onset amygdala damage) and 2) extent of the damage (selective amygdala damage vs broad damage to the medial temporal lobe).¹⁰ Neither explanation has been satisfactorily investigated. Moreover, impairments in recognizing emotion from facial and prosodic stimuli are inconsistent; certain patients are unable to recognize facial expressions, but can recognize prosodic emotion, whereas for some others, the converse is true.

Both the patients in this study had late-onset damage, but extent of the damage to the regions beyond the amygdala differed. We investigated the ability of these patients to recognize emotions from facial and non-facial (prosodic and written verbal) stimuli and analyzed their ability to recognize emotion from stimuli in different sensory modalities. In addition, we used both static (photographs) and dynamic (video-recorded) images as facial stimuli to determine the contribution of dynamic information in emotion recognition. We used a labeling task as the method of evaluation because it is considered more sensitive to impairments than a rating task; this is because recognition of facial or prosodic emotions by patients with amygdala damage is found to be consistently impaired during labeling tasks,^2,5,10,11 which is not always the case during a rating task.^4,7,9

Materials and methods

Participants

Patient MN was a 52-year-old ambidextrous male office worker. His illness began with symptoms of a common cold, and eventually, he experienced subacute disorientation (14 weeks before the present study). He was diagnosed with non-herpes limbic encephalitis. T2-weighted and fluid-attenuated inversion recovery (FLAIR) magnetic resonance imaging (MRI) scans revealed lesions that were relatively restricted to the left and right amygdala and hippocampus (Figure 1A). In addition to the MRI scan, single-photon emission computed tomography (SPECT) was also performed for functional imaging. After an iv bolus injection of 600 MBq technetium-^99mTc ethyl cysteinate dimer (ECD), projection data were acquired using a two-head gamma camera system (ECAM, Siemens, Hoffman Estates, IL). The results confirmed hypoperfusion of the bilateral medial temporal lobe, including the hippocampus and amygdala (Figure 2A), as was expected from the results of the MRI. Although he recovered from the encephalitis after appropriate medical treatment, his memory deficit persisted. Hence, the patient found it difficult to resume work.

Figure 1.

MRI scans of two patients (A, patient MN; B, patient YI) with encephalitis-induced damage to the medial temporal lobe. In MN, the images were obtained 6 weeks after the onset of the disease. In YI, the images were obtained 3 weeks after the onset of the disease.

Abbreviations: BDC, brain-damaged control group; NC, normal control group.

Figure 2.

Images of SPECT performed in two patients (A, patient MN; B, patient YI), obtained 14 and 4 weeks after the onset of the disease, respectively.

Patient YI was a 47-year-old right-handed woman, formerly working as a nurse. She contracted non-herpes simplex virus encephalitis 14 weeks before the start of this study. T2-weighted MRI, FLAIR MRI and SPECT revealed lesions in the bilateral medial temporal cortex, including the amygdala, hippocampus, parahippocampal gyrus, and the entorhinal and perirhinal cortices. The occipitoparietal cortex was not involved (Figures 1B and 2B). She initially had severe loss of consciousness and convulsive seizures for a short period, but her clinical condition gradually improved in 1 month. However, she continued to experience difficulty in short-term memory.

Table 1 presents results of neuropsychological tests for each of the aforementioned patients, including measures of orientation, memory, visuoperceptual ability, and depression. The score of mini-mental state examination (MMS) of YI (score, 24) was lower than that of MN (score, 30), which was largely attributable to the former patient’s inability to make detailed recollections. Both patients found it difficult to complete the memory tests. The memory of YI, excluding verbal memory, was worse than that of MN. However, both patients had similar scores for Raven’s colored progressive matrices (RCPM) test; memory and intellect were comparable in both cases and were sufficient for completing the following emotion recognition tasks. The visuoperceptual ability of both YI and MN was normal. When measuring the degree of depression, YI scored 34 (normal), whereas MN scored 52 (depressed) in the Zung self-rating depression scale (SDS). MN exhibited symptoms of depression and was occasionally uncooperative during testing. Neither patient exhibited signs of abnormal hearing or speech.

Table 1.

Baseline neuropsychological data (test scores) of patients MN and YI

		MN	YI
Orientation	Mini-mental state examination	30/30	24/30
	Raven’s colored progressive matrices	29/36	29/36
Memory	WMS-R (Semantic memory)
	Verbal memory	<50**	58*
	Visual memory	72*	53**
	General memory	<50*	<50*
	Attention and concentration	92	89
	Delayed recall	<50**	<50**
	Miyake paired-associate word learning test
	Related	4-6-6	2-3-3
	Unrelated	0-1-0	(incomplete)
Visuoperception	Facial identity discrimination	19/20	19/20
	Rey’s complex figure
	Copy	36/36	36/36
Mood	Zung self-rating depression scale	52	34

Notes: The Japanese version of each test was used. The asterisks against values under the Wechsler memory scale, revised edition (WMS-R) indicate that the scores are lower than those of standardized samples (*: 2SD, **: 3SD). The Miyake paired-associate word learning test is a memory test in which the subject is asked to recall 10 pairs of related and unrelated words (normal range of accuracy, 8–10 for related pairs of words and 5–9 for unrelated pairs of words). The Zung self-rating depressive scale is a measure of depression (normal, <40; borderline, 40–50; depressed, > 50).

Normal subjects with no history of neurological or psychiatric illnesses were included in the normal control group (NC group). The participants included 23 control subjects for facial stimuli (mean age, 55.4 ± standard deviation [SD], 16.1 years; 8 men and 15 women), 20 for prosodic stimuli (mean age, 56.3 ± 13.5 years; 10 men and 10 women), and 22 for written verbal stimuli (mean age, 59.4 ± 13.0 years; 14 men and 8 women). There were no significant differences in age between the patients and the NC group. In addition, there was a brain-damaged control group (BDC group, mean age, 67.2 ± 9.4 years; 3 men and 2 women), which included 5 subjects who did not have bilateral amygdala damage. These patients were tested for possible effects of brain damage. They had lesions in the lateral frontal lobe: two in the right, one in the left, and two in both sides. Their mean scores in the MMS and the SDS scale were 21 ± 5.6 and 43.5 ± 16.5, respectively. All of them sufficiently understood and completed the tasks. Informed consent was obtained from all participants for the study.

Stimuli

To assess the ability of each participant to recognize emotion from facial, prosodic, and written verbal stimuli, we used the same tasks as described by Kan et al.^12,13 For dynamic facial stimuli, we used video recordings of the six basic emotions: happiness, sadness, anger, fear, surprise, and disgust. We also used photographs depicting the six basic emotions as static facial stimuli. Prosodic stimuli were recordings of 4 semantically neutral sentences (eg, “good morning”) and 6 short nonsense sentences in which one of the 6 basic emotions was apparent. We used translations of written verbal stimuli from a previous report by Adolphs et al³ which comprised 30 sentences (5 clear depictions of each of the 6 basic emotions, such as “this person has just won a new car”).

Experimental design

Before testing, each subject was asked to explain the meaning of each of the 6 basic emotions in order to confirm that they understood them. The experiment was conducted in a comfortable, silent room. Each of the visual stimuli was individually presented in a randomized order by means of a television (for facial stimuli), tape recorder (for prosodic stimuli), or printed words (for written verbal stimuli). Each stimulus was presented once, except for facial stimuli, each of which was presented twice. The subjects were asked to label one of the 6 basic emotions that best described the emotional state represented by the stimulus. The subjects were instructed to consider each of the six alternatives carefully before responding.

Data analysis

We calculated the percentage of correct responses in MN and YI and the mean percentage of correct responses in the NC and BDC group. Thereafter, we converted the raw scores of MN and YI into z-scores using mean and SD of the control data and the mean scores of the two patients. The z-scores indicated the magnitude (in SD) of the difference between the performance of the patients and controls. We used one-tailed probabilities because our main focus was on impairments in emotion recognition. We compared performances of the NC and BDC groups by repeated measures analysis of variance (ANOVA) with group (NC and BDC) and emotion (the 6 basic emotions) as the factors.

Results

Figure 3 illustrates the performances of all the participants. The NC and BDC groups have been indicated by the bar graphs, and the performances of MN and YI have been superimposed as points on the graph.

Figure 3.

Performances of controls (normal control group [NC] and brain-damaged control group [BDC]) and patients (MN and YI). For the controls, data are expressed as the percentage mean ± SD of the correct responses. For the patients, data are expressed as the percentage of correct responses (see text for details).

Dynamic facial expressions

In emotion recognition from dynamic facial stimuli, there was a significant difference between the MN and NC groups with respect to fear recognition (z = −2.27, P < 0.05). The performance of YI did not significantly differ from that of the NC group.

The NC group performed better than the BDC group. The main effects of group [F (1, 26) = 34.1, P < 0.01] and emotion [F (5, 26) = 21.8, P < 0.01] were significant, and the interaction between these two factors was also significant [F (5, 130) = 9.87, P < 0.01]. The simple main effect revealed that recognition of all six emotions was impaired in the BDC group as compared to the NC group (P < 0.05 for happiness, sadness, and anger; P < 0.01 for fear, surprise, and disgust).

Static facial expressions

There was a significant difference between the MN and NC groups with respect to anger recognition (z = −1.76, P < 0.05) and fear (z = −2.29, P < 0.05) and between the YI and NC groups with respect to anger recognition (z = −1.76, P < 0.05).

The NC group performed better than the BDC group; however, statistical analysis revealed no significant difference in performance between these two groups [F (1, 12) = 2.53]. Only the main effect of emotion was significant [F (5, 12) = 12.4, P < 0.01]. A post hoc comparison using least significant difference (LSD) analysis revealed that subjects were able to recognize a happy expression better than any of the other five emotions. The subjects were unable to identify fear and disgust as accurately as the other four emotions, and fear was the least recognized of all the emotions (Mse = 0.0274, P < 0.05).

Prosodic expressions

A significant difference was observed between YI and the NC group with respect to the recognition of fear alone (z = −2.14, P < 0.05). The performance of MN did not significantly differ from that of the NC group.

Furthermore, compared to the BDC group, the NC group was better at recognizing all the six emotions. The main effects of group [F (1, 23) = 9.40, P < 0.01] and emotion [F (5, 23) = 8.33, P < 0.01] were significant, but the interaction between these two factors was not significant [F (5, 115) = 1.30]. A post hoc comparison using LSD analysis revealed that subjects were unable to recognize fear and disgust as accurately as the other four emotions, and that fear was the least recognized of all the emotions (Mse = 0.0274, P < 0.05).

Written verbal expressions

The performance of YI in recognizing fear was again significantly worse than that of the NC group (z = −2.01, P < 0.05). The performance of MN did not differ significantly from that of the NC group.

In the NC and BDC groups, there were significant main effects of group [F (1, 23) = 12.0, P < 0.01] and emotion [F (5, 23) = 3.62, P < 0.01], and the interaction between these two factors was also significant [F (5, 115) = 3.48, P < 0.01]. The main effect revealed that in the NC group, there were no significant differences with respect to recognition of the 6 emotions. BDC group performance was impaired when compared to the NC group with regard to recognition of anger [F (1, 23) = 12.0, P < 0.05], fear [F (1, 23) = 6.20, P < 0.05], and disgust [F (1, 23) = 22.63, P < 0.01].

Discussion

In the present study, we evaluated the ability of two post-encephalitic patients to recognize emotion from facial, prosodic, and written verbal stimuli. The following two findings will be discussed. First, in one patient, the ability to recognize fear from prosody and written verbal stimuli was impaired, but facial stimuli emotion recognition was not, in the other patient, the ability to recognize fear from facial stimuli was impaired, but recognizing fear from prosody and written verbal stimuli was not impaired. The BDC group exhibited relatively non-specific impairment in ability to recognize emotion. Second, in the patient with very extensive damage to the temporal lobe, including the amygdala, the ability to recognize fear from faces was not impaired, whereas in the patient with damage that was more restricted to the medial temporal lobe, the ability to recognize fearful facial expressions was impaired.

The first finding demonstrated that recognition of fear from different sensory modalities relies on neural systems that differ at least to some extent. Although the performance in terms of emotion recognition from prosodic and written verbal stimuli was worse than that from facial stimuli in normal control subjects, task difficulty alone cannot account for impairments in patients because one patient was able to recognize fear from modalities that even normal control subjects found difficult. Because the lesion in YI was more extensive than that in MN, the reduced ability of the former patient to recognize fear from prosodic stimuli may be attributable to damage caused to areas of the brain outside the amygdala. This would support the idea that in addition to the amygdala, a broader area of the medial temporal lobe is important to emotion recognition from prosodic stimuli.¹⁰ This is also observed with written verbal stimuli; hence, the same extra-amygdala structures may be involved in processing verbal emotions.

The second finding is more difficult to explain because it is counterintuitive and cannot be explained by lesion extent or etiology. Rather, it suggests that a fearful face can be recognized without the amygdala. Functional imaging has revealed that the amygdala is consistently activated when normal subjects are presented with fearful facial^14,15 or auditory^15,16 stimuli. Such activation of the amygdala occurs even when subjects are not actively paying attention to the stimulus; this finding indicates that the amygdala is activated automatically in response to fearful visual and auditory stimuli. However, although the automatic activation of the amygdala probably facilitates emotion recognition, it appears that the amygdala is not essential for this function. Indeed, several other brain structures, most notably in the inferior frontal cortex, have been implicated in emotion recognition.^17–20 Nakamura et al¹⁷ have suggested that the right inferior frontal cortex processes information on emotional communication, whereas the limbic system, particularly the amygdala, responds to stimuli that directly induce strong emotion.

Both patients found it easier to recognize anger from dynamic faces than from static faces. Several reports have stated that in patients with damage to the amygdala, the recognition of negative facial expressions such as anger, sadness, and fear is impaired. This study, by using both static and dynamic facial expressions as stimuli, suggests that dynamic information improves the ability of the post-encephalitic patients to recognize negative facial expressions. In addition, both patients tended to perform worse, even than the BDC group, in the static sad, angry, and disgusted faces trials (Figure 3). Although these differences did not reach statistical significance, there remains a possibility that the medial temporal lobe, including amygdala, play an important role in recognizing static negative facial expressions, as indicated in previous studies.

The results of this study indicate that impairments of emotion recognition are evident shortly after encephalitis when both patients had recovered from the disease and tried to resume work. We believe that it is clinically important to perform an in-depth examination of emotion recognition in post-encephalitic patients in order to promote social rehabilitation of these patients.