The Bodily Expressive Action Stimulus Test (BEAST). Construction and Validation of a Stimulus Basis for Measuring Perception of Whole Body Expression of Emotions

Abstract

Whole body expressions are among the main visual stimulus categories that are naturally associated with faces and the neuroscientific investigation of how body expressions are processed has entered the research agenda this last decade. Here we describe the stimulus set of whole body expressions termed bodily expressive action stimulus test (BEAST), and we provide validation data for use of these materials by the community of emotion researchers. The database was composed of 254 whole body expressions from 46 actors expressing 4 emotions (anger, fear, happiness, and sadness). In all pictures the face of the actor was blurred and participants were asked to categorize the emotions expressed in the stimuli in a four alternative-forced-choice task. The results show that all emotions are well recognized, with sadness being the easiest, followed by fear, whereas happiness was the most difficult. The BEAST appears a valuable addition to currently available tools for assessing recognition of affective signals. It can be used in explicit recognition tasks as well as in matching tasks and in implicit tasks, combined either with facial expressions, with affective prosody, or presented with affective pictures as context in healthy subjects as well as in clinical populations.

Introduction

Humans are considered to be among the most social species and there are only limited moments during which we are not interacting with conspecifics, either face to face or more indirectly for instance on the telephone. As we routinely take part in a wide range of heterogeneous social interactions on a daily basis, interpreting the intentions and emotions of others has significant adaptive value. In the most influential emotion models, emotions are considered more than emotional feelings and additionally include action components (Frijda, 1986; Damasio, 2000). One of the leading emotion theories was proposed by Frijda (1986) and it capitalizes on the function of emotions as action related mechanisms. Frijda suggests that emotions follow “appraisal” that is not necessarily consciously experienced (Frijda, 2007) and can lead to both intentional and unintentional (impulsive) actions (Frijda, 2010a,b), reactive to ongoing events (Frijda, 1986) based on a match or mismatch with our aims and goals and thereby tuning our interaction with the either social or non-social environment (Frijda, 1953, 1969, 1982). Although the focus of emotion perception research in affective neuroscience has so far been primarily on how we perceive, process, and recognize facial expressions, the theories and concepts proposed by Frijda push the envelope towards a more ambitious research instrumentarium and necessitates to move beyond investigating perception of facial expressions. Indeed, in our natural environment, faces are not perceived in isolation and usually co-occur with a wide variety of visual, auditory, olfactory, somatosensory, and gustatory stimuli. Whole body expressions are among the main visual stimulus categories that are naturally associated with faces and the neuroscientific investigation of how body expressions are processed has entered the research agenda this last decade (e.g., Hadjikhani and de Gelder, 2003; de Gelder et al., 2004), in line with the emotion theories proposed by Frijda (1986, 2007). The study of whole body expressions has significant additive value over that of facial expressions (de Gelder, 2006; Peelen and Downing, 2007; de Gelder et al., 2010). We have developed a database consisting of whole body expressions that served as stimulus materials in several experiments investigating saccades to emotional bodies (Bannerman et al., 2009, 2010), the neural basis of emotional whole body perception (Hadjikhani and de Gelder, 2003; de Gelder et al., 2004; van de Riet et al., 2009), contextual effects of whole body expressions (Van den Stock et al., 2007; Kret and de Gelder, 2010), and emotional body perception in patient populations including autism (Hadjikhani et al., 2009), prosopagnosia (Righart and de Gelder, 2007; Van den Stock et al., 2008b), neglect (Tamietto et al., 2007), and blindsight (Tamietto et al., 2009). Despite a number of previous reports on whole body expressions, dating back to the early work of Darwin (Darwin, 1872; James, 1932; Carmichael et al., 1937; Bull and Gidro-Frank, 1950; Gidro-Frank and Bull, 1950; McClenney and Neiss, 1989; Dittrich et al., 1996; Wallbott, 1998; Sprengelmeyer et al., 1999; Pollick et al., 2001; Heberlein et al., 2004), only one other validation study of a whole body expression stimulus set has been described in detail (Atkinson et al., 2004) and used in several other experiments (Atkinson et al., 2007; Peelen et al., 2007, 2009). That stimulus set consists of 10 identities (5 women) displaying whole body expressions (anger, disgust, fear, happiness, and sadness) in full light and point light displays. The bodily postures were generated by actors who had been given the instruction to express each emotion with their whole body. In developing the present stimulus set, our goal was to emphasize the action dimension of whole body expressions rather than the pure expression of inner feelings and to relate it to specific context in which the emotion expressing actions are appropriate.

Here we describe the stimulus set of whole body expressions termed the bodily expressive action stimulus test (BEAST), and we provide validation data for use of these materials by the community of emotion researchers.

Materials and Methods

Stimulus construction

Forty-six non-professional actors (31 female) were recruited through announcements at Tilburg University. Actors participated in exchange for course credits or were paid a small amount. Additionally, to enhance motivation, actors were competing for providing the best recognized expressions and the winner received an iPod. Pictures were taken against a white background and under controlled lightening conditions in a professional photo studio with a digital camera mounted on a static tripod. Actors were individually instructed in a standardized procedure to display four expressions (anger, fear, happiness, and sadness) with the whole body, and the instructions provided a few specific and representative daily event scenarios typically associated with each emotion. For example, anger was associated with being in a quarrel and threatening to fight back; fear with a pursuing attacker; happiness with encountering an old friend not seen in years and being very pleased to see; and sadness with learning from a friend that a very dear friend has passed away. We did not include other emotions like disgust at this stage, because a pilot study indicated these were hard to recognize without information conveyed by the face. Based on the photographer's impression of the expressiveness of the posture, in a few instances more than one picture of the same actor and expression was taken in order to enrich the total set and allow for small variations in expressiveness that may prove useful for one or another design. The total number of pictures was 254 (64 anger, 67 fear, 61 happiness, and 62 sadness). The order of the scenarios was randomized across subjects. After the photo shoot, the pictures were desaturated and the facial area was blurred. See Figure 1 for examples.

Figure 1.

Examples of edited stimuli showing a female (top row) and male (bottom row) actor. The expressions display (from left to right): anger, fear, happiness, sadness.

Validation procedure

Nineteen participants [11 female, mean (std) age: 22.5 (2.4)] took part in the validation experiment in exchange for course credits. All 254 stimuli were randomly presented one by one for 4000 ms with a 4000-ms inter-stimulus interval during which a blank screen was presented. The participants were instructed to categorize the emotion expressed in the whole body stimulus on an answering sheet in a four alternative-forced-choice task (anger, fear, happiness, sadness). The validation procedure consisted of three blocks. Duration of the first and second block was around 30 min per block and 100 pictures were presented in both blocks. The remaining 54 were presented in the third block, which lasted around 20 min. Fifteen-minute breaks were inserted between blocks, adding up to a total duration of about 120 min for the whole validation experiment.

Results

The stimuli and validation data can be downloaded at www.beatricedegelder.com/beast.html. The total number of data points added up to 4826 (19 subjects each rating 254 stimuli), of which 10 were missing values. The maximum number of missing values was 2 across participants and 3 across stimuli. We calculated the frequency of every response alternative for every stimulus condition and constructed a confusion matrix on the basis of this analysis (see Table 1). The overall rater agreement was calculated using Fleiss’ Generalized kappa and measured 0.839. Mean categorization accuracy of all stimuli was 92.5%. Bonferroni corrected paired-sample t-tests showed that sad expressions were better recognized than angry [t(18) = 2.960, p < 0.0083], fearful [t(18) = 4.332, p < 0.0004], and happy expressions [t(18) = 6.054, p < 0.0001] and that anger [t(18) = 3.670, p < 0.0017] and fear [t(18) = 3.601, p < 0.0020] expressions were recognized better than happy expressions (see Figure 2).

Table 1.

Number and percentage of responses according to emotion expressed in the stimulus category.

		Response
		Anger		Fear		Happiness		Sadness		Total
		No.	%	No.	%	No.	%	No.	%	No.	%
Target	Anger	1136	93.6	27	2.2	40	3.3	10	0.8	1213	25.2
	Fear	19	1.5	1194	93.9	49	3.9	10	0.8	1272	26.4
	Happiness	65	5.6	66	5.7	986	85.4	37	3.2	1154	24.0
	Sadness	7	0.6	16	1.4	3	0.3	1151	97.8	1177	24.4
	Total	1227	25.5	1303	27.1	1078	22.4	1208	25.1

Figure 2.

Proportion correct categorizations according to bodily expression. *p < 0. 01, **p < 0.001.

Bonferroni corrected paired-sample t-tests were performed on the number of incorrect responses to investigate whether the intended target emotion was systematically confused with one specific non-target response alternative. For example, we investigated for all (intended target) angry expressions whether the number of (incorrect) “fear” responses differed from the number of (incorrect) “happy” or “sad” responses. This revealed that target angry expressions were categorized significantly more often as “happy” [t(18) = 3.174, p < 0.0053] than as “sad”; and that target fear expressions were categorized significantly more often as “angry” [t(18) = 3.426, p < 0.003] and “happy” [t(18) = 4.025, p < 0.001] than as “sad.”

To investigate whether the variability of judgments for the different emotions represent variations, we transformed the stimulus categorizations to accuracies and calculated for every emotion the SD. The results are displayed in Table 2 and show that the variability is highest for happiness, followed by fear, anger, and sadness. Bonferroni corrected paired-sample t-tests were performed to compare the SD and showed that the variability for happiness is significantly higher than all other emotions [t(18) > 3.358, p < 0.002] and that the variability of sadness is lower than all other emotions [t(18) > 3.580, p < 0.002].

Table 2.

Cronbach's alpha and mean SD of the accuracies of the emotion categorization.

	SD	I.C.
Anger	0.22	0.74
Fear	0.23	0.50
Happiness	0.33	0.82
Sadness	0.10	0.72

To investigate the extent to which the different stimuli expressing the same emotion inter-correlated, we computed for every emotion the Cronbach's alpha as a measure of internal consistency. The results are displayed in Table 2 and show acceptable values for all emotions, except for fear.

To investigate inter-individual differences in emotion recognition, we correlated recognition performances for the four emotion categories over subjects. This revealed no significant correlation between any of the combinations (the highest absolute value of all correlations was below 0.40 and the lowest p value above 0.088), based on bivariate correlation tests.

Discussion

We constructed a database of 254 face-blurred whole body expressions, consisting of 46 actors expressing 4 basic emotions that can be recognized without information conveyed by the facial expression (anger, fear, happiness, and sadness) and asked participants to categorize the emotions expressed in the stimuli in a four alternative-forced-choice task. The results show that all emotions are well recognized, with sadness being the easiest, followed by fear, whereas happiness was the most difficult. The same pattern is reported for (non-exaggerated) static body expressions in a study by Atkinson et al. (2004) who used a similar validation instruction, although the procedure for constructing the stimuli differed significantly from the one followed in the present study. The convergence between the results of both studies may point to a priori differences in recognisability of bodily emotions, although the findings are influenced by task variables. In a recent study we avoided the use of verbal labels by presenting stimuli from the same database in a simultaneous match-to-sample paradigm and found that fearful expressions were the most difficult to match (Van den Stock et al., 2007). The discrepancy in the results of these studies indicates that different processes underlie verbal labeling and matching of whole body expressions. A possible explanation may be that, next to the emotional processing involved in both tasks, verbal labeling relies more on language processes and idiosyncratic semantic knowledge about features in whole body expressions (for instance that anger is associated with clenched fists), whereas matching relies more on correspondence of features between stimuli. The finding that fearful body expressions are hardest to match then indicates that this expression is more ambiguous (Van den Stock et al., 2007). This is also reflected in the internal consistency measure.

Analysis of the incorrect categorizations reveals that anger is more often confused with happiness than with sadness and that fear is more confused with anger and happiness than with sadness. It is remarkable that bodily expressions with opposite valence are more confused than expressions with similar valence. This contrasts with findings from facial expression studies, where recognition of happiness in faces is usually superior to recognition of negative emotions (e.g., Milders et al., 2003; Goeleven et al., 2008). It was also our experience during the construction of the body expression stimuli that happiness is most difficult to properly instruct as well as to perform by the actors, whereas facial expressions of happiness are very easy to instruct and perform.

An important issue concerns the fact that all the emotions were expressed voluntarily. Face emotion studies have shown differences in spontaneous and voluntary emotional expressions (Zuckerman et al., 1976). Furthermore, the actors in were instructed and motivated to express highly recognizable emotions. This does not necessarily correspond to how we express emotions in daily life and might have induced exaggerated emotional expressions. However, by presenting the actors with real life scenarios, we made an attempt to increase the ecological validity of the expressions.

Finally, we found no significant across subject correlations in emotion categorization accuracy, indicating there are no significant inter-individual differences in the recognition of whole body expressions. This means that the performance of individual subjects compared to the other subjects on for example categorization of angry whole body expressions is not systematically associated with performance on categorization of fearful, happy, or sad whole body expressions. It shows that subjects are heterogeneous in the emotional category they recognize better than others.

Explicit recognition has been tested in normal populations, in clinical populations with autism, with schizophrenia, and in individuals with prosopagnosia. The stimulus set has also proven useful for investigating high anxiety subjects (Kret and de Gelder, under review) and violent offenders (Kret and de Gelder, under review). Besides behavioral measures, EEG, MEG, and fMRI have been used to investigate the neurofunctional basis in normal and in abnormal populations. Useful information was also collected from EMG measurements and from eye movement recordings.

Another area we have begun to explore concerns cultural determinants of perception. So far one study compared Dutch and Chinese observers (Sinke et al., under review).

The role of attention and of awareness was also investigated. Implicit recognition of these bodily emotion expressions was tested in a few cases of V1 lesion that leads to clinical blindness and of hemispatial neglect associated with inattention to contralesional stimuli. Implicit recognition was also investigated in normal subjects with the use of a masking paradigm (Stienen and de Gelder,  2011).

Furthermore, the stimuli have been used in experiments using composite stimuli or combinations of a bodily expression with another source of affective information, either visual or auditory. Visual contexts that have been explored include facial expressions and environmental or social scenes (Van den Stock et al., under review). As auditory contexts we have used music fragments, spoken sentence fragments, and environmental sounds in normal (Van den Stock et al., 2008a, 2009) and clinical (Van den Stock et al., 2011) populations. Finally, a computational model for body expression recognition was developed (Schindler et al., 2008).

Conclusion

The BEAST is a valuable addition to currently available tools for assessing recognition of affective information. As has already been illustrated it can be used in explicit recognition tasks as well as in implicit ones, when combined with facial expressions or affective prosody and this in healthy subjects as well as in neurological and psychiatric populations.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.