Emotion Processing in Early Blind and Sighted Individuals

Abstract

Objective

Emotion processing is known to be mediated by a complex network of cortical and subcortical regions with evidence of specialized hemispheric lateralization within the brain. In light of prior evidence indicating that lateralization of cognitive functions (such as language) may depend on normal visual development, we investigated whether the lack of prior visual experience would have an impact on the development of specialized hemispheric lateralization in emotional processing.

Method

We addressed this issue by comparing performance in early blind and sighted controls on a dichotic listening task requiring the detection of specific emotional vocalizations (i.e., suggestive of happiness or sadness) presented independently to either ear.

Results

Consistent with previous studies, we found that sighted individuals showed enhanced detection of positive vocalizations when presented in the right ear (i.e., processed within the left hemisphere) and negative vocalizations when presented in the left ear (i.e., right hemisphere). It is interesting to note that although blind individuals were as accurate as sighted controls in detecting the valance of the vocalization, performance was not consistent with any pattern of specialized hemispheric lateralization.

Conclusions

Overall, these results suggest that although the lack of prior visual experience may not lead to impaired emotion processing performance, the underlying neurophysiological substrate (i.e., degree of special hemispheric lateralization) may depend on normal visual development.

The ability to perceive the emotional states of others is crucial for adapted social interactions (Horstmann, 2003). A great deal of information regarding an individual’s emotional state is conveyed by visual social cues, such as facial and bodily expressions (for reviews, see de Gelder, de Borst, & Watson, 2015; de Gelder et al., 2010; Wood, Rychlowska, Korb, & Niedenthal, 2016). Nonetheless, emotional states can also be determined from information garnered from other sensory modalities (such as hearing) based on differences in prosody or emotional vocalizations (e.g., Altenmüller, Schmidt, & Zimmermann, 2013; Belyk & Brown, 2014). Accordingly, although only a few studies have specifically investigated emotion perception in blind individuals, available evidence suggests that visual experience is not needed to develop efficient emotional recognition skills. For example, blind and sighted individuals have been shown to possess similar emotion recognition accuracy in tasks requiring the haptic exploration of facial expressions (Kitada et al., 2013). Moreover, blind participants outperformed sighted control participants in recognizing negative emotions from body odours (Iversen, Ptito, Møller, & Kupers, 2015) and from prosody (Klinge, Röder, & Büchel, 2010). At the neurophysiological level, it has been reported that blind individuals show enhanced occipital cortex (specifically, the middle and superior occipital gyri) activity during the haptic recognition of facial expressions (Kitada et al., 2013), and enhanced amygdala responses to emotional auditory stimuli (Klinge et al., 2010) as compared with sighted controls. These results are in line with a contemporary view suggesting that early visual deprivation is associated with compensatory behaviors that are intimately related to underlying changes in the overall structural and functional organization of the brain (e.g., Hasson, Andric, Atilgan, & Collignon, 2016; Voss & Zatorre, 2015; for reviews, see Cattaneo et al., 2008; Ricciardi, Bonino, Pellegrini, & Pietrini, 2014).

When studying emotion processing in the blind, an interesting aspect to consider is hemispheric lateralization. In fact, if on one hand the brain exhibits a certain degree of specialized hemispheric lateralization with regards to the processing of emotions (for reviews Demaree, Everhart, Youngstrom, & Harrison, 2005; Wager, Phan, Liberzon, & Taylor, 2003), on the other hand a debated issue is whether the lack of normal visual development overall affects hemispheric specialization (beyond the emotional domain). Indeed, language functions may be less left-lateralized in early blind compared with sighted individuals (e.g., Lane et al., 2017), with reduced lateralization also observed for nonlinguistic functions (e.g., Bonino et al., 2015; Nava, Güntürkün, & Röder, 2013). With regards to emotions, converging behavioral and neurophysiological findings in the visual domain (e.g., Cunningham, Espinet, DeYoung, & Zelazo, 2005; Grimm et al., 2006; Lee et al., 2004; Lichtenstein-Vidne, Gabay, Cohen, & Henik, 2016) and in the auditory domain (Flores-Gutiérrez et al., 2007; Gagnon & Peretz, 2000; Schmidt & Trainor, 2001), suggest that in the (sighted) brain negative emotions are processed preferentially in the right hemisphere, whereas positive emotions are processed preferentially in the left hemisphere (i.e., the valence lateralization hypothesis; for reviews, see Demaree et al., 2005; Harmon-Jones, Gable, & Peterson, 2010; Wager et al., 2003). Whether the lack of prior visual experience also affects the development of specialized hemispheric lateralization in emotion processing is not known.

To address this issue, we compared performance in early blind and sighted participants on a dichotic listening task in which short emotional vocalizations of positive (i.e., happiness/laughing) or negative (i.e., sadness/crying) valences were presented independently in each ear (for a similar paradigm with a prosodic task, see Grimshaw, Séguin, & Godfrey, 2009). In light of previous evidence, we expected that blind individuals would recognize emotions as accurately as sighted individuals, if not better (see Iversen et al., 2015; Kitada et al., 2013; Klinge et al., 2010). In this scenario, preservation of hemispheric lateralization would presuppose that emotion cues garnered from other sensory modalities (e.g., auditory, haptic, and olfactory domains) may be sufficient for the specialization of related emotion processing brain networks. The absence of hemispheric lateralization (in the presence of a comparable, if not better, performance level) would infer that visual experience is not essential to develop good emotional recognition skills, but rather is crucial for the development of specialized (lateralized) brain networks.

Method

Participants

Sixteen early blind individuals (8 females), mean age 41.4 ± 3.0 years (age range from 21 to 58 years), mean education level of 13.6 ± 0.9 years, and 16 gender matched sighted control participants (8 females), mean age 40.9 ± 2.9 years (age range from 22 to 58 years), mean education level of 13.9 ± 0.9 years, took part in the experiment. The two groups did not significantly differ in terms of age, t(30) < 1, p = .91, or education level, t(30) < 1, p = .82. All the blind participants read Braille, and all participants were right-handed (based on self-report). All the blind participants were blind due to ocular related causes and were otherwise neurologically normal. The blind participants’ characteristics are listed in Table 1. Informed consent was obtained from all participants. The study was approved by the local ethical committee and the experiment was carried out in accordance to the tenants of the Declaration of Helsinki.

Table 1.

Characteristics of Blind Participants

Age	Gender	Educational level (years)	Cause of blindness	Residual visual ability
36	M	13	Retinopathy of prematurity	Lights
52	F	13	Retinopathy of prematurity	None
58	M	18	Congenital glaucoma	Lights, shadows, and colors
45	F	8	Micro ophthalmia	Lights
53	F	8	Optic nerve tumor	None
56	F	13	Retinopathy of prematurity	None
58	M	18	Retinitis pigmentosa	Lights
24	M	10	Optic nerve atrophy	None (could see lights and shadows until the age of 5)
45	F	8	Micro ophthalmia	Right eye lights and shadows, left eye none
39	F	13	Complications related to premature birth	Lights
28	F	13	Retinal degeneration/malformation	None
21	F	18	Ocular malformation	None
48	M	18	Leber’s congenital amaurosis	None
42	M	3	Optic nerve atrophy	Lights
28	M	13	Retinitis pigmentosa	Lights
30	M	13	Retinopathy of prematurity	None

Stimuli

Stimuli consisted of emotional vocalizations extracted from the Montreal Affective Voices database corresponding to a standardized and validated set of emotional nonlinguistic interjections expressing different emotions (Belin, Fillion-Bilodeau, & Gosselin, 2008). From this database, we selected happiness and sadness vocalizations (i.e., laughs and cries respectively) expressed by 10 different actors (5 males and 5 females). Stimuli were then cut into 1 second segments using Audacity audio editing software (http://www.audacityteam.org/). Four neutral vocalizations (expressed by 2 males and 2 females), consisting in an “Ah” vocalization, were also selected and characterized by a very small pitch variation (no prosody inclination) and cut in the same way.

Procedure

Participants wore headphones and were seated comfortably in front of a computer. Sighted participants were blindfolded during the entire experiment. We used a dichotic listening paradigm similar to that used in prior studies (e.g., Grimshaw et al., 2009), in which participants had to detect the presence of a target emotion. In each experimental trial, two auditory stimuli were presented dichotically. Specifically, one neutral and one emotional stimulus were presented simultaneously. The neutral and emotional stimuli were partially masked by surrounding white noise that was presented binaurally to increase task difficulty. To determine this experimental protocol, an earlier pilot study was carried out on a different group of sighted control participants confirming that detection performance was at ceiling levels when emotional and neutral stimuli were presented without the surrounding white noise. In the experiment, participants had to indicate as fast as possible whether the target emotion was “present” or “absent” (regardless of which ear it was presented) by pressing either a left or right key and using their two forefingers. Two experimental blocks were carried out; one in which the target emotion was happiness, and the second in which the target emotion was sadness. Each experimental block consisted of 80 trials, in which half of the target emotion vocalization was present. In target-absent trials of each block, a “distracter” emotional vocalization was presented (i.e., sadness for the target-happiness block, and happiness for the target-sadness block). Within each block, each emotional vocalization (target and distracter) was presented an equal number of times to both the left and right ear. The pairs of neutral and emotional stimuli presented in each trial were randomly selected with the only constraint that the two vocalizations belonged to the same gender voice. Before the main experiment, participants listened to all the auditory stimuli binaurally (without white noise), and performed six practice trials to familiarize themselves with the task. The order of target-emotion blocks and response key assignment were counterbalanced across all participants. The experiment lasted approximately 40 min (including debriefing). For the purposes of this experiment, responses given after 5000 ms (from the onset of the auditory stimulus) were excluded from further analysis (42 excluded trials in total, i.e., less than 1% of total number of trials).

Results

Mean accuracy scores from both groups are reported in Figure 1. The analysis of variance (ANOVA) performed on mean accuracy scores revealed no significant main effects of group, F(1, 30) < 1, p = .54, ${\eta }_{\mathrm{p}}^2=.01$; target emotion, F(1, 30) < 1, p = .70, ${\eta }_{\mathrm{p}}^2=.01$, and ear, F(1, 30) < 1, p = .96, ${\eta }_{\mathrm{p}}^2=.00$. The three-way interaction group by Target Emotion × Ear was significant, F(1, 30) = 5.05, p = .032, ${\eta }_{\mathrm{p}}^2=.14$. No other interactions reached significance (all ps > .09, maximum ${\eta }_{\mathrm{p}}^2=.09$). To further characterize this significant three-way interaction, we looked at the simple main effects of target emotion and ear within each group of participants.

Figure 1.

Mean accuracy scores in sighted and blind participants. Sighted participants tended to be better in detecting target-happy vocalizations when presented to the right ear, and target-sad vocalizations when presented to the left ear. No pattern of lateralization emerged in blind participants.

In sighted participants, the ANOVA did not reveal any significant main effect of target emotion, F(1, 15) < 1, p = .84, ${\eta }_{\mathrm{p}}^2=.003$, or ear, F(1, 15) < 1, p = .58, ${\eta }_{\mathrm{p}}^2=.02$. The interaction Target Emotion × Ear was significant, F(1, 15) = 7.15, p = .017, ${\eta }_{\mathrm{p}}^2=.32$, indicating that sighted participants tended to be better in detecting happiness-target vocalizations when presented to the right ear, t(15) = 1.38, p = .093, d = .45, and tended to be better in detecting sadness-target vocalizations when they were presented to the left ear, t(15) = 1.428, p = .087, d = .46 (see Figure 1). The ANOVA performed on blind participants’ responses revealed no significant main effect of target emotion, F(1, 15) < 1, p = .42, ${\eta }_{\mathrm{p}}^2=.04$, no significant main effect of ear, F(1, 15) < 1, p = .50, ${\eta }_{\mathrm{p}}^2=.03$, and no significant interaction Target Emotion × Ear, F(1, 15) < 1, p = .71, ${\eta }_{\mathrm{p}}^2=.01$.

We also performed a similar ANOVA on signal detection sensitivity (d′) and on the response Criterion (C) to look for possible confounding effects of response bias on hemispheric sensitivity. As shown in Figure 2a, the pattern of results for d′ was similar as for accuracy scores, but the interaction Group × Target Emotion × Ear did not reach significance, F(1, 30) = 2.60, p = .12, ${\eta }_{\mathrm{p}}^2=.08$. None of the main effects or of the other interactions reached statistical significance (all ps > .25, maximum ${\eta }_{\mathrm{p}}^2=.04$). The lack of significance for the three-way interaction is likely to reflect a lack of statistical power rather than an effect of response bias. In fact, the ANOVA on C scores did not show significant differences related to group or related to ear. The only significant effect when analyzing C was target emotion, F(1, 30) = 14.3, p = .001, ${\eta }_{\mathrm{p}}^2=.32$ (for all other main effects and interactions, ps > .14, maximum ${\eta }_{\mathrm{p}}^2=.07$). Negative C values arise when false-alarm rate exceeds the miss rate, and positive C values when it is lower. As shown in Figure 2b, our participants relied on a more conservative response criterion overall when target emotion was Sadness than Happiness.

Figure 2.

d′ (A) and response Criterion (B) scores for sighted and blind participants in the different experimental conditions. The pattern of d′ scores resembles that found when analyzing accuracy scores (although with d′ there is only a nonsignificant trend showing difference in emotional valence lateralization in the sighted). Overall, participants were more conservative in responding when the target emotion was sadness than happiness.

Analysis performed on mean correct RTs (see Figure 3) revealed a significant main effect of group, F(1, 30) = 8.20, p = .008, ${\eta }_{\mathrm{p}}^2=.22$, suggesting that overall blind participants were slower than sighted controls. The main effects of target emotion, F(1, 30) < 1, p = .33, ${\eta }_{\mathrm{p}}^2=.03$, and of ear, F(1, 30) < 1, p = .72, ${\eta }_{\mathrm{p}}^2=.00$, were not significant. No other interactions reached significance (all ps > .20, maximum ${\eta }_{\mathrm{p}}^2=.05$). Note that when analyzing RTs for incorrect responses the group difference was still there as a trend, independent samples t test: t(30) = 1.77, p = .087, d = .62.

Figure 3.

Mean correct RTs in sighted and blind participants. Error bars depict ±1 SEM. * p < .05.

Discussion

In this study, we used a dichotic listening task to investigate whether the lack of prior visual experience would have an impact on emotion processing and the development of specialized hemispheric lateralization. In line with prior evidence (e.g., Flores-Gutiérrez et al., 2007; Gagnon & Peretz, 2000; Schmidt & Trainor, 2001), a pattern of valence-dependent lateralization emerged in accuracy scores in our sighted participants in which positive vocalizations (i.e., laughing) tended to be better detected when presented to the right ear (i.e., processed within the left hemisphere) whereas negative vocalizations (i.e., crying) tended to be better detected when presented to the left ear (i.e., processed within the right hemisphere). In contrast, blind individuals did not show any advantage of one ear over the other, and for either the general condition (i.e., no evidence of any hemispheric dominance in emotional processing), or for the specific condition of hemispheric lateralization associated with the valence of the emotion presented. Moreover, overall blind individuals were significantly slower in responding compared with sighted controls.

Although in our study we did not have a control dichotic listening task sensitive to specialized hemispheric lateralization outside the emotional domain, our findings are in line with similar evidence in other domains suggesting that blindness may reduce, and in certain instances even reverse, the lateralization of perceptual and cognitive functions typically observed in sighted individuals (Hertrich, Dietrich, & Ackermann, 2013; Hugdahl et al., 2004; Kujala et al., 1995; Röder, Rösler, & Neville, 2000). Specifically, for the auditory domain, although some neuroimaging studies reported a left-lateralized activation in blind individuals during language tasks (Bedny et al., 2011; Amedi, Raz, Pianka, Malach, & Zohary, 2003; Burton et al., 2002), other evidence suggests that this may not be the case (differences possibly depending on the paradigms and material used). Indeed, blind individuals showed a reduced right ear advantage compared with sighted controls in dichotic listening tasks (Karavatos, Kaprinis, & Tzavaras, 1984; Larsen & Håkovsen, 1983). Röder and colleagues reported less left lateralized responses to language in blind individuals (Röder, Stock, Bien, Neville, & Rösler, 2002; Röder et al., 2000), with other groups even reporting a reversed lateralization pattern of language functions in the blind (Hertrich et al., 2013; Hertrich, Dietrich, Moos, Trouvain, & Ackermann, 2009; see also Hugdahl et al., 2004). A recent study directly comparing language laterality between blind and sighted individuals showed reduced laterality throughout frontotemporal language areas as well as in occipital cortices recruited for language processing in early blind individuals (Lane et al., 2017). Finally, in the spatial domain, the right hemisphere seems to be dominant in spatial metric tasks in the blind as well (Cattaneo, Fantino, Tinti, Silvanto, & Vecchi, 2010; Cattaneo, Fantino, Silvanto, Tinti, & Vecchi, 2011; Cattaneo Fantino, Tinti, et al., 2011; Sampaio, Gouarir, & Mvondo, 1995), although there is evidence that blindness affects the degree of lateralization in spatial imagery tasks (Bonino et al., 2015) and functional motor-spatial asymmetries (Nava et al., 2013).

How the lack of normal visual experience prevents the development of lateralized processing of emotions remains to be determined. One possibility relates to the intrinsic function of brain lateralization and specialization in certain processes (e.g., language). This is generally acknowledged to represent an enhancement in brain capacity and cognitive performance in both animals and humans (for review and discussion, see Rogers, 2010; Vallortigara & Rogers, 2005). In blind individuals, the lack of prior visual experience may alter how emotional information is accessed and ultimately processed given that facial and body expressions are typically perceived through the visual domain. In particular, the lack of visual experience related to the perception of face stimuli, known to be processed preferentially by a right-lateralized complex network of cortical and subcortical areas (e.g., Haxby, Hoffman, & Gobbini, 2000; Rossion, 2014), may ultimately have an impact on the development of specialized lateralization for emotion processing. In fact, recent evidence suggests that the often observed right-hemispheric dominance in processing facial emotional expressions may indeed depend on the right-lateralized network for face processing (Worley & Boles, 2016). In this regard, it is interesting that individuals who show developmental delays associated with autism spectrum disorder (and whose prevalence is particularly high in blind children, Jure, Pogonza, & Rapin, 2016) also present abnormal hemispheric lateralization in different tasks (Floris et al., 2016; Herringshaw, Ammons, DeRamus, & Kana, 2016; but see Baker, Montgomery, & Abramson, 2010), including atypical hemispheric specialization for faces (Keehn, Vogel-Farley, Tager-Flusberg, & Nelson, 2015).

Another, nonexclusive, possibility is that the lack of emotion lateralization observed in blind individuals may depend on a generic lack of lateralized specialization in auditory processing. Indeed, Coullon, Jiang, Fine, Watkins, and Bridge. (2015) have recently observed that the auditory cortex and the thalamus in blind individuals show equivalent responses to contralateral and ipsilateral verbal and musical stimulation (contrasting with the contralateral bias observed in sighted individuals). However, this interpretation would not account for prior results reporting a reverse lateralization pattern in processing linguistic material in blind individuals (Hertrich et al., 2013). Testing profoundly blind individuals who lost their vision later in life (and thus have prior visual experience and memoires) may help disentangle this issue. In fact, prior evidence suggests that acquired blindness may lead to different compensatory changes at the behavioral and neural level compared with early visual deprivation (Cattaneo, Vecchi, Monegato, Pece, & Cornoldi, 2007; Monegato, Cattaneo, Pece, & Vecchi, 2007; Shi et al., 2015; Voss, Pike, & Zatorre, 2014). In this respect, an interesting issue that deserves further consideration is whether a similar level of accuracy (despite lack of lateralization) is made possible by the recruitment of visual cortices in the blind during emotional processing. Although Lane et al. (2017) observed no correlation between the amount of visual cortical activation by language tasks and reduced lateralization in the frontotemporal network, whether occipital crossmodal plasticity and reduced lateralization occur via different mechanisms is an important question that deserves further investigation.

It should also be acknowledged that prior studies in sighted individuals also failed to consistently report a pattern of hemispheric lateralization related to the valence of auditory presented emotional stimuli. However, as mentioned earlier these inconsistent results may depend on the different paradigms and materials used (see Kotz, Meyer, & Paulmann, 2006; Rodway & Schepman, 2007, for examples and discussion). For instance, it has been suggested that valence-specific effects may be evident only for difficult and subtle discrimination of emotions (Jansari, Tranel, & Adolphs, 2000). Moreover, the use of a blocked paradigm, in which target emotions were presented in separate blocks rather than intermixed, may have also favored the appearance of lateralization effects (see also Cattaneo et al., 2014; Schepman, Rodway, & Geddes, 2012), by allowing for a (relative) long term focus on one specific emotion.

Analysis on detection sensitivity resembled the pattern observed with accuracy (but note that in this case the effects did not reach significance, likely because of a lack of statistical power). This analysis also revealed a general tendency (across sighted and blind subjects) to be more liberal in responding to happiness targets, in line with prior evidence showing attentional and perceptual biases toward positive stimuli (e.g., Pool, Brosch, Delplanque, & Sander, 2015). The similar level of accuracy achieved by blind and sighted individuals is in line with prior studies showing accurate emotion discrimination capacities in the early blind (Iversen et al., 2015; Klinge et al., 2010) and in a related domain, comparable performance in Theory of Mind tasks (Bedny, Pascual-Leone, & Saxe, 2009, 2013). Although blind participants in this study did not show any impairment or enhancement in emotion detection accuracy, they did show a comparatively slower reaction time (RT) in responding (across all trials). Prolonged response latencies in the blind in our study are unlikely to reflect selective difficulties in auditory (emotional) processing. Indeed, in prior studies blind individuals showed faster processing of auditory stimuli (e.g., Collignon & De Volder, 2009; Elbert et al., 2002; Jafari & Malayeri, 2014; Röder, Rösler, Hennighausen, & Näcker, 1996, 1999; Stevens & Weaver, 2005), even in tasks of auditory emotional discrimination (Klinge et al., 2010). It may be that the type of response required (i.e., multiple key pressing) was unfamiliar to our group of blind participants thus contributing to observed slower response times (but see Stevens & Weaver, 2005; Klinge et al., 2010). Future studies manipulating stimuli and task design may shed further light on this issue.

In interpreting our results, sample characteristics deserves consideration. Although the number of our early blind participants (N = 16) is in line with prior literature, our sample size was relatively small, this possibly reducing statistical power. Notwithstanding this limitation, whereas the significant difference between ears in emotion recognition reported in our sighted participants was expected in light of prior evidence (Flores-Gutiérrez et al., 2007; Gagnon & Peretz, 2000; Schmidt & Trainor, 2001), the lack of lateralization in our blind individuals is unlikely to depend on insufficient power since the interaction emotion by ear was in fact quite far from significance (p = .71). Moreover, one may question whether our sample was representative of blind individuals in general. In fact, our blind participants had on average a highschool degree (consider that the national policy tends to encourage people with disabilities to remain in the educational system beyond compulsory schooling), and they all attended the local blind union’s activities on a regular basis. This is typically the case of blind individuals enrolled in psychology research (at least in Western countries) that are usually recruited via local blind associations and thus tend to have good educational level. In this respect, they are unlikely to be representative of the whole population (note though that this may reflect a more general bias in psychology research sampling, e.g., Henrich, Heine, & Norenzayan, 2010). Nonetheless, it is worth noting that if level of education is known to affect cognitive performance (e.g., Ceci, 1991; Le Carret et al., 2003), it is not clear at this time how this factor would ultimately impact the recognition of basic emotional vocalizations (e.g., Kirouac & Dore, 1985). This is also suggested by the finding that children as young as 5 years are able to make fine distinctions between vocal-emotional signals (Sauter, Panattoni, & Happé, 2013).

In sum, our results suggest that specialized hemispheric lateralization implicated in the processing of different valenced emotions may depend on an intact visual experience. Future studies may further investigate the functional and neural mechanisms underlying emotion processing in blind individuals by considering a larger variety of emotional stimuli, as well as testing individuals with acquired blindness later in life. These studies are likely to shed further light on the compensatory mechanisms observed in blind individuals to perceive the emotional world around them in the absence of visual experience.

General Scientific Summary.

Many human capacities, including the capacity to recognize emotions, involve the left and right brain hemisphere to a different extent. Here we show that lateralization in emotional processing may depend on having a normal visual experience: blind individuals are good in recognizing emotions, but their brain uses different processes to sustain this capacity.