Abstract
The present study used event-related brain potentials to examine the hypothesis that emotional gestures draw attentional resources at the level of distinct processing stages. Twenty healthy volunteers viewed pictures of hand gestures with negative (insult) and positive (approval) emotional meaning as well as neutral control gestures (pointing) while dense sensor event-related potentials (ERPs) were recorded. Emotion effects were reflected in distinct ERP modulations in early and later time windows. Insult gestures elicited increased P1, early posterior negativity (EPN) and late positive potential (LPP) components as compared to neutral control gestures. Processing of approval gestures was associated with an increased P1 wave and enlarged EPN amplitudes during an early time window, while the LPP amplitude was not significantly modulated. Accordingly, negative insult gestures appear more potent than positive approval gestures in inducing a heightened state of attention during processing stages implicated in stimulus recognition and focused attention.
Keywords: gestures, emotion, affect, attention, EEG, ERP
INTRODUCTION
The perception and appropriate responding to non-verbal emotional signals are central to social interaction. In affective neuroscience, the study of neural mechanisms of emotion has been largely based on emotionally relevant signals such as facial expression and body posture (Vuilleumier, 2005; de Gelder, 2006). These displays are considered to reflect non-symbolic and non-propositional emotion communication (Buck and VanLear, 2002). Beyond such biologically shared emotion signals, humans also use learned symbolic communication displays to convey emotional meaning. In face-to-face interactions, symbolic gestures are frequently used to show approval, calm down a heated exchange or express insult, offense and threat (Morris, 1994). Being strongly related to language and unique to humans (Pika et al., 2005), gestures represent a distinct class of non-verbal emotion signals. They presumably build upon the general predisposition of shared intentionality, i.e. the motivation of humans ‘to share emotions, experience and activities with other persons’ (Tomasello et al., 2005). Accordingly, the relation between the type of display and the associated meaning of even the strongest emotional gestures is arbitrary, varies from culture to culture and depends on convention (Morris, 1994).
Two recent studies addressed the emotion–attention relationship of symbolic signal systems by studying emotionally pleasant, unpleasant and neutral hand gestures (Flaisch et al., 2009). Functional magnetic resonance imaging (fMRI) revealed increased brain activation during emotional compared to neutral gesture processing in selected extrastriate visual cortical regions devoted to hand gesture processing (i.e. posterior mid-occipital and lateral occipito-temporal). Furthermore, these effects were accentuated for negative insult as compared to positive approval gestures. These findings are conceptually similar to previous neuroimaging studies showing that pictures of erotica, mutilation and threat, emotional facial expressions and fearful body posture reliably enhanced activations in visual-associative cortical regions of the ventral processing stream (Vuilleumier et al., 2001; Pessoa et al., 2002; Bradley et al., 2003; Hadjikhani and de Gelder, 2003; de Gelder et al., 2004; Grosbras and Paus, 2005; Junghöfer et al., 2005; Sabatinelli et al., 2005; Junghöfer et al., 2006; Peelen and Downing, 2007).
Furthermore, event-related brain potentials delineated the temporal dynamics of emotion processing in visual-associative cortical regions. In a rapid serial presentation paradigm, it was observed that emotional gestures elicit an increased posterior negativity [early posterior negativity (EPN)], most pronounced in a time window from 200 to 300 ms after stimulus onset, as compared to neutral gestures (Flaisch et al., 2009). Similar to fMRI findings, these effects were accentuated for insult gestures. Accordingly, it has been suggested that emotional gestures are selected for preferential processing in perceptual representation regions (Schupp et al., 2006a). Enhanced EPN amplitudes to emotional stimuli have been similarly observed when viewing pictures of erotica, mutilation and threat as well as threatening and fearful faces (Junghöfer et al., 2001; Sato et al., 2001; Schupp et al., 2003, 2004; Leppänen et al., 2007). Taken together, previous findings support the notion that emotional gestures guide visual attention during initial stimulus perception.
The finding that the emotionality of gestures is able to regulate information processing during a relatively early processing stage provides the impetus for research investigating gesture processing in later processing stages. Previous research consistently demonstrates that the late positive potential (LPP) component is modulated by the intrinsic significance of emotional stimuli (Schupp et al., 2006a). For instance, pictures of erotica, threat and mutilation elicit an increased LPP component between 300 and 700 ms after stimulus onset compared to neutral stimulus contents (Cuthbert et al., 2000; Keil et al., 2002; Schupp et al., 2003). Similarly, larger LPP amplitudes are observed when viewing threatening or fearful as compared to neutral facial expressions (Schupp et al., 2004; Leppänen et al., 2007). Accordingly, the present study investigated the hypothesis of preferential processing of emotional hand gestures in a higher order processing stage that has been suggested to reflect a state of increased visual attention and stimulus representation in working memory (Nieuwenhuis et al., 2005; Sergent et al., 2005; Del Cul et al., 2007; Schupp et al., 2007).
The measurement of ‘earlier’ and ‘later’ processing periods furthermore allows comparing emotional modulation effects evinced by distinct event-related potential (ERP) components. Specifically, the differential sensitivity of the EPN and LPP component to the emotional meaning of the hand signs may change across processing time as they reflect functionally distinct processing stages (Luck et al., 2000; Schupp et al., 2006a). Previous research revealed that both components differ in their sensitivity to perceptual novelty and fluency and with regard to the emotion–attention relationship (Bradley et al., 2007; Schupp et al., 2007; Ferrari et al., 2009). Furthermore, distinct effects of positive and negative stimulus processing on early and late ERP components have been reported in studies presenting International Affective Picture System (IAPS) pictures, facial expressions and words (Esslen et al., 2004; Herbert et al., 2006; Codispoti et al., 2007; Flaisch et al., 2008b; Herbert et al., 2008; Pastor et al., 2008; Kissler et al., 2009). Examining earlier and later processing periods seems particularly relevant when considering differences in the processing of positive and negative hand gestures, i.e. increased responding to gestures of insult (Flaisch et al., 2009). Since negative as compared to positive gestures are usually associated with urgent action, they may be more efficient in capturing attentional resources at early processing stages, but comparable in drawing attention at later processing stages assessed by the LPP component. Alternatively, there may be a negativity bias in that preferential responding to gestures of insult is seen across the processing stream and obtained for EPN and LPP components.
The present study allowed furthermore assessing habituation effects and gender differences in emotional gesture processing. While previous studies with biologically shared emotion signals reported gender differences in that men were more responsive to pictures of erotica while females participants were more sensitive to pictures of threat and mutilation (e.g. Bradley et al., 2001; Sabatinelli et al., 2004), no gender differences were observed for emotional gestures (Flaisch et al., 2009). Furthermore, with regard to habituation effects, previous research with IAPS pictures and gestures consistently showed that preferential emotion processing is sustained across time (Schupp et al., 2006b; Codispoti et al., 2007; Flaisch et al., 2009). To further corroborate these findings, gender and habituation effects were examined across the processing stream.
The present study examined whether emotional hand gestures elicit a natural state of selective attention (Lang et al., 1997). Towards this end, participants viewed a rapid stream of positive (OK), neutral (Point) and negative (Insult) gestures in which each picture was shown for 118 ms [intertrial interval (ITI) = 894 ms]. Dense sensor event-related potentials were recorded to assess the temporal dynamics of emotional gesture processing. Specifically, it was examined (i) whether socially shared emotional gestures modulate later stages of processing similar to the effects of biologically shared emotion signals, (ii) whether LPP modulation is more accentuated for negative gestures, as was observed for the EPN component, (iii) whether effects of emotional gesture processing are obtained for both genders and (iv) whether the selective responding to emotional gestures habituates or is sustained across time. Previous findings regarding differential processing of emotional gestures during early processing as indexed by the EPN were expected to be replicated.
MATERIALS AND METHODS
Participants
Participants were 20 healthy adults (10 females) who received course credits or monetary compensation for participation. Two participants had to be excluded from data analysis because of excessive eye movements and artefact-contaminated electroencephalogram (EEG) data. Participants were between 20 and 25 years of age (M = 22.4 years). The ethical committee of the University of Konstanz approved the experimental procedure and all participants provided informed consent.
Stimulus materials
Three gestures bearing positive, negative and neutral emotional meaning were selected (cf. Flaisch et al., 2009). Among the strongest hand gestures of sexual insult is the middle-finger jerk produced by the upward thrusting of the stiff middle finger (Insult). The positive thumbs-up gesture, produced by the display of the erect thumb, signals approval and is also referred to as the Ok sign (OK). As emotionally neutral control gesture served the forefinger pointing in a specific direction (Point; Morris et al., 1979; Morris, 1994). All gestures are associated with a distinct, widely shared meaning in the German culture, which was confirmed by participants’ self-report collected after physiological data collection. Each of the three gestures (Insult, OK and Point) was posed by four women and four men. All gestures were displayed with the back of the hand rotated towards the viewer and with a neutral single-coloured grey–blue background. The exact location of each hand within a square-shaped image was kept constant by approximately positioning the back of the hand to the center of the picture. All pictures also appeared mirrored along the vertical axis to control for possible lateralization effects.
Apparatus and stimulus presentation
While the previous study used a rapid serial presentation paradigm in which pictures were shown for 330 ms with no interstimulus interval (Flaisch et al., 2009), the stimulus materials were displayed in the present study for 118 ms with an ITI of 894 ms. The brief picture presentation time was chosen to minimize eye movements as it allows for a single fixation of the pictures only (cf. Christianson et al., 1991).
To assure good signal-to-noise ratio, the entire picture set was repeated 45 times resulting in a total of 1296 picture presentations. The stimulus order was pseudo-randomized with several constraints to assure adequate control of sequence effects (cf. Flaisch et al., 2008a). Constraints included approximated transition frequencies for all categories, not more than three repetitions of the same category, as well as the presentation of the entire picture set before any single stimulus was repeated. Each participant viewed a different order of picture presentation.
Using presentation software (Neurobehavioral Systems, Inc., Albany, CA, USA), the pictures were shown on a 21-inch cathode ray tube (CRT)-monitor (75 Hz refresh rate) located ∼100 cm in front of the participant. Picture presentation lasted for 22 min with a short break in the middle of the session to allow for posture adjustments. Participants were instructed to keep their eyes comfortably focused on the centre of the screen and to simply view the pictures.
Self-report
Following ERP measurement, participants were asked to rate the gestures according to their pleasantness and arousal using the Self-Assessment Manikin rating scale (Bradley and Lang, 1994; valence: 1 = most pleasant, 9 = most unpleasant; arousal: 1 = least arousing, 9 = most arousing). For statistical analysis, both measures were entered into a one-factorial repeated measure ANOVA with the factor Gesture (Insult vs OK vs Point).
ERP data acquisition and analysis
Brain and ocular scalp potential fields were measured with a 256-lead geodesic sensor net (GSN 200 v2.0; Electrical Geodesics, Inc (EGI): Electrical Geodesics, Inc., Eugene, OR, USA), on-line bandpass filtered from 0.1 to 100 Hz, and sampled at 250 Hz using Netstation acquisition software and EGI amplifiers. Electrode impedance was kept below 50 kΩ, as recommended for this type of EEG amplifier by EGI guidelines. Data were recorded continuously with the vertex sensor as reference electrode. Continuous EEG data were low-pass filtered at 50 Hz using a zero-phase forward and reverse digital filter before stimulus synchronized epochs were extracted from 200 ms pre-stimulus onset to 1000 ms post-stimulus onset and baseline-corrected for pre-stimulus (100 ms) ERP activity.
Data editing and artefact rejection were based on a two-step method for statistical control of artefacts (Junghöfer et al., 2000). In a first step, based on the vertex reference, sensors contaminated across the session were identified and rejected. Furthermore, sensors containing trial epochs with artefact activity were rejected to avoid contamination when converting the data to an average reference. The rejection of artefact-contaminated trials and sensor epochs was based on the thresholds for a number of statistical parameters (e.g. absolute value over time, standard deviation over time; Junghöfer et al., 2000). In a second step, based on the average referenced data, sensors containing artefact-contaminated activity were replaced using spherical interpolation on the basis of all remaining sensors for the given trial. Average waveforms were calculated for the three experimental categories for each sensor and participant.
Waveform analyses
In a first stream of analyses, each time point and sensor was tested separately using a one-factorial (Insult vs OK vs Point) ANOVA. Significant effects were thresholded at P < 0.05 for at least eight continuous data points (32 ms) and two neighbouring sensors (Schupp et al., 2003) to provide a conservative guarding against chance findings (Sabbagh and Taylor, 2000). The resulting pattern of significant ERP modulation served to determine critical time periods as well as regions of interest for subsequent detailed statistical evaluation utilizing area score assessments.
Area score assessment
As shown in Figure 1, in a time interval between 80 and 140 ms post stimulus, the P1 component was scored over medial parieto-occipital clusters including EGI sensors 76, 77, 78, 79, 85, 86, 87, 88, 96, 97, 98, 99, 107, 108, 109, 110, 116, 117, 118, 119, 125, 126 (left) and 128, 129, 130, 131, 132, 139, 140, 141, 142, 143, 144, 150, 151, 152, 153, 154, 155, 160, 161, 162, 163, 164 (right). Over frontal sensor clusters, corresponding effects were apparent with reversed polarity. Statistical analyses revealed significant modulations mirroring posterior effects. For brevity, these analyses are not reported.
To analyse temporal changes in the EPN window, two time intervals, an early one from 160 to 220 ms and a later one from 220 to 280 ms were considered separately. Temporo-occipital clusters included EGI sensors 82, 83, 84, 90, 91, 92, 93, 94, 95, 101, 102, 103, 104, 105, 106, 112, 113, 114, 115, 116, 121, 122, 123, 124, 125, 134, 135, 136, 137, 146, 147, 256 (left) and 149, 150, 157, 158, 159, 160, 166, 167, 168, 169, 170, 171, 172, 175, 176, 177, 178, 179, 180, 188, 189, 190, 191, 192, 200, 201, 202, 209, 210, 217, 218, 232 (right). At fronto-central clusters, a corresponding polarity reversal was observed. Emotional modulation appeared with opposite polarity. Statistical analyses revealed significant modulations mirroring posterior effects, which are not reported for brevity.
The LPP was indexed as mean activity from 480 to 540 ms over fronto-central leads comprising EGI sensors 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 26, 27, 28, 29, 32, 33, 42, 43, 44, 52, 79, 80, 132, 133, 145, 186, 187, 198, 199, 208, 216, 257.
The P1 and EPN components were submitted to separate repeated-measures ANOVAs including the factors Gesture (Insult vs OK vs Point) and Laterality (left vs right), while analysis of the LPP included only the factor Gesture. When appropriate, the Greenhouse–Geisser procedure was used to correct for violations of sphericity.
RESULTS
Self-report ratings
Highly significant main effects for Gesture were observed for valence [F(2, 34) = 76.4, P < 0.001, ε = 0.78] and arousal ratings [F(2, 34) = 21.2, P < 0.001, ε = 0.73]. As expected, the Insult gesture was evaluated as being clearly negative (M = 7.6; s.d. = 1.3), the OK gesture as being positive (M = 2.8; s.d. = 1.2) and the Point gesture was rated as neutral (M = 5.6; s.d. = 0.8). Comparing the three different gestures shows that the insult gesture was rated significantly as being more negative than the OK gesture [t(17) = 10.2, P < 0.001] or the Point gesture [t(17) = 6.9, P < 0.001. The OK gesture was rated as being more positive than the Point gesture, t(17) = 7.2, P < 0.001. With regard to arousal, the Insult gesture (M = 5.8; s.d. = 1.8) was rated as more arousing than the OK gesture [M = 3.7; s.d. = 1.7; t(17) = 5.9, P < 0.001] or the Point gesture (M = 3.9; s.d. = 1.8; t(17) = 7.7, P < 0.001] which received similar arousal ratings.
ERPs
Waveform analyses
The single sensor waveform analyses revealed three highly significant modulations of the ERP as a function of the factor Gesture (see Figure 1). First, emotional as compared to neutral gestures were associated with an increased P1 peak (Figure 2). Second, the Insult gesture was associated with an increased early posterior negativity compared with both the OK and Point gestures. In addition, the OK gesture elicited an increased posterior negativity as compared with the Point gesture. However, this effect appeared attenuated in amplitude and brief in duration (Figure 3). Third, the Insult gesture elicited augmented LPP amplitudes as compared with both other gestures (Figure 4). These main effects of Gesture were further examined in separate repeated measures ANOVAs for the P1, EPN and LPP components based on the average ERP activity in selected sensor clusters and time windows showing most pronounced effects.
P1 component
Over parieto-occipital regions, a highly significant main effect of Gesture [F(2, 34) = 30.4, P < 0.001, ε = 0.97] indicated that the P1 component to emotional hand gestures was enlarged compared with the Point gesture [t(17) > 6.1, P < 0.001].
EPN component
In both time windows (160–220 and 220–280 ms), highly significant main effects of Gesture were obtained over temporo-occipital regions [F(2, 34) = 14.8 and 13.3, P < 0.001, ε = 0.95 and 0.98, respectively]. The Insult gesture elicited an increased negativity as compared to the Point and OK gestures in both time windows [Insult vs. Point t(17) = 5.3 and 4.7, P < 0.001, Insult versus OK t(17) = 2.8 and 4.5, P < 0.05 and 0.001, respectively]. The OK gesture elicited an increased negativity as compared with the Point gesture during the early-time window only, [t(17) = 2.7, P < 0.05].
LPP component
The LPP significantly varied as a function of gesture type [F(2, 34) = 16.0, P < 0.001, ε = 0.89]. Post hoc tests revealed increased LPP amplitudes for the Insult as compared to both, the OK gesture [t(17) = 5.8, P < 0.001] and the Point gesture [t(17) = 5.0, P < 0.001]. Furthermore, LPP amplitudes elicited by the OK and Point gestures were not significantly different [t(17) = 0.6, ns].
Additionally, L2-Minimum Norm solutions were calculated to provide an estimate of the generator sources of the differential processing of the Insult compared to Point and OK gestures. Calculation of the L2-Minimum-Norm was based on a four-shell spherical head model with evenly distributed 3 (radial, azimuthal and polar direction) × 350 dipoles as source model. A shell radius of 8 cm was chosen as best tradeoff between depth sensitivity and spatial resolution (Hauk, 2004). Consistent with the notion that the LPP reflects widespread cortical brain activity, L2-MNE analyses of the difference waves revealed increased dipole strength over fronto-central, inferior fronto-temporal and parietal locations for the Insult as compared to the OK and Point gestures (Figure 5).
Gender effects
Incorporating the factor Gender in these analyses allowed examining whether men and women differed in gesture processing. No gender differences were observed for the P1, EPN and LPP components (main effects and interactions incorporating Gender, F < 2.8, ns).
Habituation effects
To determine habituation, additional analyses were conducted including the factor Time (first half vs second half). Results reveal similar effects of Gesture type on P1, EPN and LPP component in the first and second half of the experiment and no higher order interaction involving Time reached significance (F < 1.7, ns).
DISCUSSION
The present findings reveal that social learning and experience is efficient in guiding attentional resources to non-verbal emotion signals. Increased EPN and LPP amplitudes to gestures of insult revealed that socially salient signals capture stimulus-driven attention. Interestingly, a notable difference emerged with regard to the processing of positive and negative gestures, i.e. accentuated EPN and LPP effects for the Insult compared to the OK gestures. This asymmetry may arise because negative cues usually require immediate behavioral responses (Taylor, 1991; Baumeister et al., 2001). Furthermore, the P1 was enlarged to emotional gestures, presumably because emotional gestures can be discriminated early in the processing stream by coarse stimulus analysis. Interestingly, across the processing stream, sensitivity to the emotionality of the gestures changed from the differentiation of emotional and neutral gestures during early processing to the specific responding towards Insult gestures in later processing periods. Thus, the findings provide corroborating evidence regarding the notion to examine emotional stimulus processing at the level of distinct processing stages (Hillyard et al., 1995; Luck et al., 2000).
The main finding of the present study is that the processing of Insult gestures elicited increased LPP amplitudes compared to the OK and Point gestures. One interpretation of these data is that Insult gestures draw more attentional resources at a processing stage related to stimulus representation in working memory (Nieuwenhuis et al., 2005; Schupp et al., 2006a). These results extend previous research examining the processing of stimulus contents, which relate to agendas set by evolution, such as fearful and threatening facial expressions and natural scenes displaying erotica, threat or violence (Schupp et al., 2006a). An evolutionary preparedness to specific emotional gestures, similar to emotional facial expression or body posture (Öhman and Mineka, 2001; de Gelder, 2006), seems unlikely considering that only humans use symbolic gestures. As the meaning of symbolic gestures is culturally transmitted, they provide unique insights into the power of social experience and learning to shape visual attention processes. Hand signs represent a unique emotional communication signal in that the information is carried by a unique configuration of the fingers, similar to facial expressions, but the display communicates symbolic information like words. Interestingly, recent research suggests that emotional word processing is associated with enlarged LPP amplitudes compared to neutral contents (Fischler and Bradley, 2006; Kissler et al., 2007; Kissler et al., 2009). Taken together, there is increasing evidence that emotional LPP modulation is elicited by biologically and socially shared emotion signals.
A noteworthy difference was observed regarding the processing of the Insult and OK gestures. Gestures of approval were less efficient in eliciting facilitated processing at the level of later ERP components, in particular the LPP. These differences were neither the consequence of gender differences in processing the OK gestures nor secondary to habituation effects. Instead, they may relate to differences in action disposition and response mobilization associated with the Insult and OK gestures (Taylor, 1991). The Insult gesture usually signals the need for immediate responses to prevent harm and injury while the approval gesture often occurs after action is completed. In this respect, the present result pattern shows resemblance to the study of other body signals. Several studies utilizing psychophysical measures demonstrated that anger was the emotion most reliably decoded from stimuli depicting dance or gesture (Dittrich et al., 1996; Boone and Cunningham, 1998; Pollick et al., 2001, 2002). Likewise, ERP research has shown stronger brain responses associated with threatening and fearful compared to happy facial expressions (Sato et al., 2001; Schupp et al., 2004; Leppänen et al., 2007). Thus, to meet potential threats and to minimize possible harm and trauma, the asymmetry in later processing stages between the Insult and OK gestures may arise because negative cues usually require immediate behavioural responses (Taylor, 1991; Brosch et al., 2008). Differences in the engagement of behavioural responses may also be reflected by participants’ evaluative self-report of the stimulus materials. Insult gestures are not only perceived as more unpleasant compared to the OK and Point gestures, but also rated higher in arousal. Previous research suggests that the emotional modulation of the LPP component is primarily related to differences in stimulus arousal (Schupp et al., 2003). According to this reasoning, Insult gestures appear more potent in engaging their corresponding motivational system compared to the OK gesture. Furthermore, valence differences observed in research studying emotional words are inconclusive. Specifically, emotional modulation of the LPP was observed for either pleasant or unpleasant words (Fischler and Bradley, 2006; Herbert et al., 2006, 2008; Kissler et al., 2007; Kissler et al., 2009). Accordingly, a larger array of gestures needs to be examined in future research to more conclusively investigate differences in positive and negative gesture processing.
The appearance of the emotional LPP modulation to hand gestures shows similarities and differences when compared to research examining IAPS pictures and facial expressions. With respect to polarity, the finding of increased LPP amplitudes to Insult gestures corresponds to previous studies investigating natural emotional scenes and facial expressions (Keil et al., 2002; Schupp et al., 2003, 2004, 2007; Leppänen et al., 2007). This reasoning is based on the consideration of difference potentials (Insult–OK and Insult–Point) to isolate ERP components reflecting emotion processing (Luck, 2005). Specifically, while inspection of Figure 4A reveals an overall negativity in the LPP time window, the difference between the processing of the Insult versus the Point and OK gestures is of positive polarity (Figure 4B and C). The advantage of considering the polarity difference in processing emotional and neutral stimulus materials becomes apparent when considering research with IAPS pictures. Depending on variations in experimental procedure (e.g. picture size, presentation time and rate), the ERP waveform to IAPS pictures may evince positive or negative polarity in the LPP time window (cf. Schupp et al., 2003; Flaisch et al., 2008b). However, despite pronounced differences in the overall ERP waveform across studies, the difference in processing emotional and neutral pictures is uniformly seen as a positive difference potential. Furthermore, with regard to timing and duration of the LPP modulation, the current findings appear similar to previous research investigating facial expressions and words (Schupp et al., 2004; Herbert et al., 2006, 2008; Leppänen et al., 2007). Previous research with IAPS picture materials indicates that emotional modulation effects appear both earlier in time and longer lasting. These differences are presumably secondary to the greater emotional engagement afforded by natural scenes of threat, mutilation and erotica (Bradley et al., 2003), which is also reflected by enlarged EPN and LPP effects (cf. Schupp et al., 2003, 2004; Flaisch et al., 2008b).
Clear differences to previous research emerged regarding the topography of the LPP component. Rather than appearing over centro-parietal leads, the LPP effect is observed over fronto-central sensors suggesting an at least partially different neural representation of gestures compared to biologically shared emotion signals (Figure 4C). One likely source for differences in the scalp topography of the LPP component is related to the stimulus materials. Compared to natural scenes and facial expressions, hand gesture processing is presumed to elicit a unique pattern of activation in higher-order visual-associative regions devoted to object perception (Downing et al., 2001; Malach et al., 2002). Thus, correlated brain activity in coupled networks might at least differ with regard to perceptual representation. Interestingly, source analysis revealed also commonalities in candidate generator structures of the emotional modulation of the LPP component observed in previous research. Specifically, L2-MNE analyses of the differential LPP activity elicited by Insult gestures was estimated to reflect activity in multiple cortical regions including generator structures in prefrontal, inferior temporal and parieto-occipital regions. Differential activation was also observed in these structures when studying pleasant and unpleasant natural scenes (e.g. erotica, mutilation, violence; Keil et al., 2002; Sabatinelli et al., 2007; Schupp et al., 2007). Overall, the data regarding the differential emotion processing indicated by the LPP are consistent with the notion that the LPP component reflects widespread neural generator sources in multiple distributed cortical association regions (Nieuwenhuis et al., 2005; Sergent et al., 2005; Del Cul et al., 2007). Studying a broad sample of emotional stimuli including natural scenes, facial expressions and gestures may enable to functionally decompose the contributions of different neural generator sources of the scalp-recorded LPP component in future studies.
A novel and somewhat unexpected finding was that the Insult and OK gestures elicited a larger P1 wave relative to the neutral Point gestures. The effect appeared sizeable over extended occipital and parietal regions in scalp topographical maps. According to explicit spatial attention research, enhanced P1 peaks for emotional gestures may reflect a gain mechanism enhancing processing in extrastriate visual cortex (Mangun et al., 1993). A modulation of the P1 amplitude has also been observed in research studying emotional faces (Sato et al., 2001; Eimer and Holmes, 2002; Pourtois et al., 2004). However, other studies found no reliable emotional P1 effects (Schupp et al., 2004; Leppänen et al., 2007) suggesting that P1 effects depend on type of emotion and task context (Vuilleumier and Pourtois, 2007). Studying more complex, natural scenes, a recent study systematically manipulated stimulus perceptibility by adding various amounts of visual noise. P1 amplitude linearly increased with picture perceptibility, similarly pronounced for emotional and neutral scenes (Schupp et al., 2008). Moreover, P1 effects seem to primarily rely on coarse processing carried by low spatial frequency inputs (Pourtois et al., 2005). Accordingly, one likely possibility is that emotional modulation of the P1 component is observed when low-level physical stimulus features and coarse stimulus processing facilitate the rapid extraction of emotional meaning.
The present results replicate the previously observed finding that emotional gestures elicit an increased early posterior negativity (Flaisch et al., 2009). A differential ERP activity was elicited by the Insult gesture closely corresponding in terms of polarity, topography and timing to the EPN component observed in a recent study of emotional gesture processing (Flaisch et al., 2009). Furthermore, the scalp topography of the EPN, i.e. a pronounced bilateral relative negativity over temporo-occipital sensors, is consistent with a recent fMRI study revealing that emotional gestures reliably engaged higher-order visual processing areas (Flaisch et al., 2009). Together, these findings implicate visual processing areas as underlying generator structure. With respect to the processing of OK gestures, a notable difference to previous findings emerged. Specifically, enlarged EPN amplitudes to the OK compared with the control gestures were obtained only during an early EPN time window (160–220 ms). Presentation rate of the pictures may account for these differences. Instead of a continuous 3-Hz stimulation as in previous research, pictures were shown for 118 ms in the current study with an interstimulus interval of 894 ms. A similar trend of less pronounced EPN modulation effects with slower presentation rates is seen in research with words and IAPS pictures (Kissler et al., 2007; Herbert et al., 2008; Peyk et al., 2009). A further difference concerns hemispheric asymmetries in the EPN effect. While emotional word effects appear to be more pronounced over left posterior regions (Kissler et al., 2007, 2009), research with IAPS pictures suggest more pronounced effects over right posterior regions (Junghöfer et al., 2001; Flaisch et al., 2008a). However, other studies observed bilateral emotional EPN modulations for both words and pictures (Schupp et al., 2003, 2004; Herbert et al., 2008). A similar variability is emerging with gestures in that the current findings reveal bilateral EPN modulation while the previous study observed more pronounced effects over right posterior regions (Flaisch et al., 2009). Contrasting hemispheric differences associated with these stimulus materials in a within-subject design may provide a test for the hypothesis that these differences reflect the emotional enhancement in cell assemblies with material-specific topographical neural representations (Peelen and Downing, 2007).
CONCLUSION
The present findings support the notion that emotional gestures efficiently recruit attentional resources during stimulus perception. Increased EPN and LPP amplitudes to emotional gestures of insult reveal the stimulus-driven attention capture of socially salient signals, particularly pronounced for the negative gestures. Furthermore, the P1 was enlarged to emotional gestures, presumably because emotional gestures can be discriminated early in the processing stream by coarse stimulus analysis. Overall, social learning and experience is efficient in guiding attentional resources to non-verbal emotion signals fostering the extraction of socially and affectively salient information.
Conflict of Interest
None declared.
Acknowledgments
This work was supported by grants from the Deutsche Forschungsgemeinschaft (DFG) to H. T. Schupp (Schu1074/11-2 and Schu1074/10-1).
REFERENCES
- Baumeister RF, Bratslavsky E, Finkenauer C, Vohs KD. Bad is stronger than good. Review of General Psychology. 2001;5:323–70. [Google Scholar]
- Boone RT, Cunningham JG. Children’s decoding of emotion in expressive body movement: the development of cue attunement. Developmental Psychology. 1998;34:1007–16. doi: 10.1037//0012-1649.34.5.1007. [DOI] [PubMed] [Google Scholar]
- Bradley MM, Lang PJ. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry. 1994;25:49–59. doi: 10.1016/0005-7916(94)90063-9. [DOI] [PubMed] [Google Scholar]
- Bradley MM, Codispoti M, Sabatinelli D, Lang PJ. Emotion and motivation II: sex differences in picture processing. Emotion. 2001;1:300–19. [PubMed] [Google Scholar]
- Bradley MM, Hamby S, Low A, Lang PJ. Brain potentials in perception: Picture complexity and emotional arousal. Psychophysiology. 2007;44:364–73. doi: 10.1111/j.1469-8986.2007.00520.x. [DOI] [PubMed] [Google Scholar]
- Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, King W, Desai P. Activation of the visual cortex in motivated attention. Behavioral Neuroscience. 2003;117:369–80. doi: 10.1037/0735-7044.117.2.369. [DOI] [PubMed] [Google Scholar]
- Brosch T, Sander D, Pourtois G, Scherer KR. Beyond fear: rapid spatial orienting toward positive emotional stimuli. Psychological Science. 2008;19:362–70. doi: 10.1111/j.1467-9280.2008.02094.x. [DOI] [PubMed] [Google Scholar]
- Buck R, VanLear CA. Verbal and nonverbal communication: distinguishing symbolic, spontaneous, and pseudo-spontaneous nonverbal behavior. Journal of Communication. 2002;52:522–41. [Google Scholar]
- Christianson SA, Loftus EF, Hoffman H, Loftus GR. Eye fixations and memory for emotional events. Journal of Experimental Psychology: Learning, Memory and Cognition. 1991;17:693–701. doi: 10.1037//0278-7393.17.4.693. [DOI] [PubMed] [Google Scholar]
- Codispoti M, Ferrari V, Bradley MM. Repetition and event-related potentials: distinguishing early and late processes in affective picture perception. Journal of Cognitive Neuroscience. 2007;19:577–86. doi: 10.1162/jocn.2007.19.4.577. [DOI] [PubMed] [Google Scholar]
- Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biological Psychology. 2000;52:95–111. doi: 10.1016/s0301-0511(99)00044-7. [DOI] [PubMed] [Google Scholar]
- de Gelder B. Towards the neurobiology of emotional body language. Nature Reviews Neuroscience. 2006;7:242–9. doi: 10.1038/nrn1872. [DOI] [PubMed] [Google Scholar]
- de Gelder B, Snyder J, Greve D, Gerard G, Hadjikhani N. Fear fosters flight: A mechanism for fear contagion when perceiving emotion expressed by a whole body. Proceedings of the National Academy of Sciences of the United States of America. 2004;101:16701–6. doi: 10.1073/pnas.0407042101. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Del Cul A, Baillet S, Dehaene S. Brain dynamics underlying the nonlinear threshold for access to consciousness. PLoS Biology. 2007;5:2408–23. doi: 10.1371/journal.pbio.0050260. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dittrich WH, Troscianko T, Lea SE, Morgan D. Perception of emotion from dynamic point-light displays represented in dance. Perception. 1996;25:727–38. doi: 10.1068/p250727. [DOI] [PubMed] [Google Scholar]
- Downing PE, Jiang Y, Shuman M, Kanwisher N. A cortical area selective for visual processing of the human body. Science. 2001;293:2470–3. doi: 10.1126/science.1063414. [DOI] [PubMed] [Google Scholar]
- Eimer M, Holmes A. An ERP study on the time course of emotional face processing. Neuroreport. 2002;13:427–31. doi: 10.1097/00001756-200203250-00013. [DOI] [PubMed] [Google Scholar]
- Esslen M, Pascual-Marqui RD, Hell D, Kochi K, Lehmann D. Brain areas and time course of emotional processing. NeuroImage. 2004;21:1189–203. doi: 10.1016/j.neuroimage.2003.10.001. [DOI] [PubMed] [Google Scholar]
- Ferrari V, Bradley MM, Codispoti M, Lang PJ. Detecting Novelty and Significance. Journal of Cognitive Neuroscience. 2010;22:404–11. doi: 10.1162/jocn.2009.21244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fischler I, Bradley M. Event-related potential studies of language and emotion: words, phrases, and task effects. Progress in Brain Research. 2006;156:185–203. doi: 10.1016/S0079-6123(06)56009-1. [DOI] [PubMed] [Google Scholar]
- Flaisch T, Junghöfer M, Bradley MM, Schupp HT, Lang PJ. Rapid picture processing: affective primes and targets. Psychophysiology. 2008a;45:1–10. doi: 10.1111/j.1469-8986.2007.00600.x. [DOI] [PubMed] [Google Scholar]
- Flaisch T, Schupp HT, Renner B, Junghöfer M. Neural systems of visual attention responding to emotional gestures. NeuroImage. 2009;45:1339–46. doi: 10.1016/j.neuroimage.2008.12.073. [DOI] [PubMed] [Google Scholar]
- Flaisch T, Stockburger J, Schupp HT. Affective prime and target picture processing: an ERP analysis of early and late interference effects. Brain Topography. 2008b;20:183–91. doi: 10.1007/s10548-008-0045-6. [DOI] [PubMed] [Google Scholar]
- Grosbras MH, Paus T. Brain Networks Involved in Viewing Angry Hands or Faces. Cerebral Cortex. 2005;16:1087–96. doi: 10.1093/cercor/bhj050. [DOI] [PubMed] [Google Scholar]
- Hadjikhani N, de Gelder B. Seeing fearful body expressions activates the fusiform cortex and amygdala. Current Biology. 2003;13:2201–5. doi: 10.1016/j.cub.2003.11.049. [DOI] [PubMed] [Google Scholar]
- Hauk O. Keep it simple: A case for using classical minimum norm estimation in the analysis of EEG and MEG data. NeuroImage. 2004;21:1612–21. doi: 10.1016/j.neuroimage.2003.12.018. [DOI] [PubMed] [Google Scholar]
- Herbert C, Kissler J, Junghöfer M, Peyk P, Rockstroh B. Processing of emotional adjectives: evidence from startle EMG and ERPs. Psychophysiology. 2006;43:197–206. doi: 10.1111/j.1469-8986.2006.00385.x. [DOI] [PubMed] [Google Scholar]
- Herbert C, Junghöfer M, Kissler J. Event related potentials to emotional adjectives during reading. Psychophysiology. 2008;45:487–98. doi: 10.1111/j.1469-8986.2007.00638.x. [DOI] [PubMed] [Google Scholar]
- Hillyard SA, Mangun GR, Woldorff MG, Luck SJ. Neural systems mediating selective attention. In: Gazzaniga MS, editor. The Cognitive Neurosciences. Cambridge, MA: MIT Press; 1995. pp. 665–81. [Google Scholar]
- Junghöfer M, Elbert T, Tucker DM, Rockstroh B. Statistical control of artifacts in dense array EEG/MEG studies. Psychophysiology. 2000;37:523–32. [PubMed] [Google Scholar]
- Junghöfer M, Bradley MM, Elbert TR, Lang PJ. Fleeting images: A new look at early emotion discrimination. Psychophysiology. 2001;38:175–8. [PubMed] [Google Scholar]
- Junghöfer M, Schupp HT, Stark R, Vaitl D. Neuroimaging of emotion: Empirical effects of proportional global signal scaling in fMRI data analysis. NeuroImage. 2005;25:520–6. doi: 10.1016/j.neuroimage.2004.12.011. [DOI] [PubMed] [Google Scholar]
- Junghöfer M, Sabatinelli D, Bradley MM, Schupp HT, Elbert TR, Lang PJ. Fleeting images: Rapid affect discrimination in the visual cortex. Neuroreport. 2006;17:225–9. doi: 10.1097/01.wnr.0000198437.59883.bb. [DOI] [PubMed] [Google Scholar]
- Keil A, Bradley MM, Hauk O, Rockstroh B, Elbert T, Lang PJ. Large-scale neural correlates of affective picture processing. Psychophysiology. 2002;39:641–9. doi: 10.1017.S0048577202394162. [DOI] [PubMed] [Google Scholar]
- Kissler J, Herbert C, Peyk P, Junghöfer M. Buzzwords: Early cortical responses to emotional words during reading. Psychological Science. 2007;18:475–80. doi: 10.1111/j.1467-9280.2007.01924.x. [DOI] [PubMed] [Google Scholar]
- Kissler J, Herbert C, Winkler I, Junghöfer M. Emotion and attention in visual word processing: an ERP study. Biological Psychology. 2009;80:75–83. doi: 10.1016/j.biopsycho.2008.03.004. [DOI] [PubMed] [Google Scholar]
- Lang PJ, Bradley MM, Cuthbert BN. Motivated attention: affect, activation, and action. In: Lang PJ, Simons RF, Balaban M, editors. Attention and Emotion: Sensory and Motivational Processes. Mahwah, NJ: Erlbaum; 1997. pp. 97–135. [Google Scholar]
- Leppänen JM, Kauppinen PK, Peltola MJ, Hietanen JK. Differential electrocortical responses to increasing intensities of fearful and happy emotional expressions. Brain Research. 2007;1166:103–9. doi: 10.1016/j.brainres.2007.06.060. [DOI] [PubMed] [Google Scholar]
- Luck SJ. Ten simple rules for designing ERP experiments. In: Handy TC, editor. Event-related Potentials: A Methods Handbook. Cambridge, MA: MIT Press; 2005. pp. 17–32. [Google Scholar]
- Luck SJ, Woodman GF, Vogel EK. Event-related potential studies of attention. Trends in Cognitive Sciences. 2000;4:432–40. doi: 10.1016/s1364-6613(00)01545-x. [DOI] [PubMed] [Google Scholar]
- Malach R, Levy I, Hasson U. The topography of high-order human object areas. Trends in Cognitive Sciences. 2002;6:176–84. doi: 10.1016/s1364-6613(02)01870-3. [DOI] [PubMed] [Google Scholar]
- Mangun GR, Hillyard SA, Luck SJ. Electrocortical substrates of visual selective attention. In: Meyer DE, Kornblum S, editors. Attention and Performance 14: Synergies in Experimental Psychology, Artificial Intelligence, and Cognitive Neuroscience. Cambridge, MA, US: The MIT Press; 1993. pp. 219–43. [Google Scholar]
- Morris D. Bodytalk: The Meaning of Human Gestures. New York: Crown Publishers; 1994. [Google Scholar]
- Morris D, Collett P, Marsh P, O’Shaughnessy M. Gestures: Their Origins and Distribution. London: Jonathan Cape; 1979. [Google Scholar]
- Nieuwenhuis S, Aston-Jones G, Cohen JD. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychological Bulletin. 2005;131:510–32. doi: 10.1037/0033-2909.131.4.510. [DOI] [PubMed] [Google Scholar]
- Öhman A, Mineka S. Fears, phobias, and prepardness: toward an evolved module of fear and fear learning. Psychological Review. 2001;108:483–522. doi: 10.1037/0033-295x.108.3.483. [DOI] [PubMed] [Google Scholar]
- Pastor MC, Bradley MM, Low A, Versace F, Molto J, Lang PJ. Affective picture perception: emotion, context, and the late positive potential. Brain Research. 2008;1189:145–51. doi: 10.1016/j.brainres.2007.10.072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Peelen MV, Downing PE. The neural basis of visual body perception. Nature Reviews Neuroscience. 2007;8:636–48. doi: 10.1038/nrn2195. [DOI] [PubMed] [Google Scholar]
- Pessoa L, McKenna M, Gutierrez E, Ungerleider LG. Neural processing of emotional faces requires attention. Proceedings of the National Academy of Sciences of the United States of America. 2002;99:11458–63. doi: 10.1073/pnas.172403899. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Peyk P, Schupp HT, Keil A, Elbert T, Junghöfer M. Parallel processing of affective visual stimuli. Psychophysiology. 2009;46:200–8. doi: 10.1111/j.1469-8986.2008.00755.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pika S, Liebal K, Call J, Tomasello M. The gestural communication of apes. Gesture. 2005;5:41–56. [Google Scholar]
- Pollick FE, Lestou V, Ryu J, Cho SB. Estimating the efficiency of recognizing gender and affect from biological motion. Vision Research. 2002;42:2345–55. doi: 10.1016/s0042-6989(02)00196-7. [DOI] [PubMed] [Google Scholar]
- Pollick FE, Paterson HM, Bruderlin A, Sanford AJ. Perceiving affect from arm movement. Cognition. 2001;82:B51–B61. doi: 10.1016/s0010-0277(01)00147-0. [DOI] [PubMed] [Google Scholar]
- Pourtois G, Dan ES, Grandjean D, Sander D, Vuilleumier P. Enhanced extrastriate visual response to bandpass spatial frequency filtered fearful faces: time course and topographic evoked-potentials mapping. Human Brain Mapping. 2005;26:65–79. doi: 10.1002/hbm.20130. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pourtois G, Grandjean D, Sander D, Vuilleumier P. Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex. 2004;14:619–33. doi: 10.1093/cercor/bhh023. [DOI] [PubMed] [Google Scholar]
- Sabatinelli D, Bradley MM, Fitzsimmons JR, Lang PJ. Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance. NeuroImage. 2005;24:1265–70. doi: 10.1016/j.neuroimage.2004.12.015. [DOI] [PubMed] [Google Scholar]
- Sabatinelli D, Flaisch T, Bradley MM, Fitzsimmons JR, Lang PJ. Affective picture perception: gender differences in visual cortex? Neuroreport. 2004;15:1109–12. doi: 10.1097/00001756-200405190-00005. [DOI] [PubMed] [Google Scholar]
- Sabatinelli D, Lang PJ, Keil A, Bradley MM. Emotional Perception: Correlation of Functional MRI and Event-Related Potentials. Cerebral Cortex. 2007;17:1085–91. doi: 10.1093/cercor/bhl017. [DOI] [PubMed] [Google Scholar]
- Sabbagh MA, Taylor M. Neural correlates of theory-of-mind reasoning: An event-related potential study. Psychological Science. 2000;11:46–50. doi: 10.1111/1467-9280.00213. [DOI] [PubMed] [Google Scholar]
- Sato W, Kochiyama T, Yoshikawa S, Matsumura M. Emotional expression boosts early visual processing of the face: ERP recording and its decomposition by independent component analysis. Neuroreport. 2001;12:709–14. doi: 10.1097/00001756-200103260-00019. [DOI] [PubMed] [Google Scholar]
- Schupp HT, Flaisch T, Stockburger J, Junghöfer M. Emotion and attention: Event-related brain potential studies. Progress in Brain Research. 2006a;156:31–51. doi: 10.1016/S0079-6123(06)56002-9. [DOI] [PubMed] [Google Scholar]
- Schupp HT, Junghöfer M, Weike AI, Hamm AO. Emotional facilitation of sensory processing in the visual cortex. Psychological Science. 2003;14:7–13. doi: 10.1111/1467-9280.01411. [DOI] [PubMed] [Google Scholar]
- Schupp HT, Öhman A, Junghöfer M, Weike AI, Stockburger J, Hamm AO. The facilitated processing of threatening faces: an ERP analysis. Emotion. 2004;4:189–200. doi: 10.1037/1528-3542.4.2.189. [DOI] [PubMed] [Google Scholar]
- Schupp HT, Stockburger J, Codispoti M, Junghöfer M, Weike AI, Hamm AO. Stimulus novelty and emotion perception: the near absence of habituation in the visual cortex. Neuroreport. 2006b;17:365–9. doi: 10.1097/01.wnr.0000203355.88061.c6. [DOI] [PubMed] [Google Scholar]
- Schupp HT, Stockburger J, Codispoti M, Junghöfer M, Weike AI, Hamm AO. Selective visual attention to emotion. Journal of Neuroscience. 2007;27:1082–9. doi: 10.1523/JNEUROSCI.3223-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schupp HT, Stockburger J, Schmälzle R, Bublatzky F, Weike AI, Hamm AO. Visual noise effects on emotion perception: Brain potentials and stimulus identification. Neuroreport. 2008;19:167–71. doi: 10.1097/WNR.0b013e3282f4aa42. [DOI] [PubMed] [Google Scholar]
- Sergent C, Baillet S, Dehaene S. Timing of the brain events underlying access to consciousness during the attentional blink. Nature Neuroscience. 2005;8:1391–400. doi: 10.1038/nn1549. [DOI] [PubMed] [Google Scholar]
- Taylor SE. Asymmetrical effects of positive and negative events: The mobilization-minimization hypothesis. Psychological Bulletin. 1991;110:67–85. doi: 10.1037/0033-2909.110.1.67. [DOI] [PubMed] [Google Scholar]
- Tomasello M, Carpenter M, Call J, Behne T, Moll H. Understanding and sharing intentions: the origins of cultural cognition. Behavioral and Brain Sciences. 2005;28:675–91; discussion 91–735. doi: 10.1017/S0140525X05000129. [DOI] [PubMed] [Google Scholar]
- Vuilleumier P. How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences. 2005;9:585–94. doi: 10.1016/j.tics.2005.10.011. [DOI] [PubMed] [Google Scholar]
- Vuilleumier P, Pourtois G. Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia. 2007;45:174–94. doi: 10.1016/j.neuropsychologia.2006.06.003. [DOI] [PubMed] [Google Scholar]
- Vuilleumier P, Armony JL, Driver J, Dolan RJ. Effects of attention and emotion on face processing in the human brain: An event-related fMRI study. Neuron. 2001;30:829–41. doi: 10.1016/s0896-6273(01)00328-2. [DOI] [PubMed] [Google Scholar]