Context Facilitates Performance on a Classic Cross-Cultural Emotion Perception Task

Katie Hoemann; Alyssa N Crittenden; Shani Msafiri; Qiang Liu; Chaojie Li; Debi Roberson; Gregory A Ruark; Maria Gendron; Lisa Feldman Barrett

doi:10.1037/emo0000501

. Author manuscript; available in PMC: 2020 Oct 1.

Published in final edited form as: Emotion. 2018 Nov 26;19(7):1292–1313. doi: 10.1037/emo0000501

Context Facilitates Performance on a Classic Cross-Cultural Emotion Perception Task

Katie Hoemann ^a, Alyssa N Crittenden ^b, Shani Msafiri ^c, Qiang Liu ^d, Chaojie Li ^d, Debi Roberson ^e, Gregory A Ruark ^f, Maria Gendron ^a,^*, Lisa Feldman Barrett ^a,^g,^*

PMCID: PMC6535382 NIHMSID: NIHMS1014210 PMID: 30475026

Abstract

The majority of studies designed to assess cross-cultural emotion perception use a choice-from-array task in which participants are presented with brief emotion stories and asked to choose between target and foil cues. This task has been widely criticized, evoking a lively and prolonged debate about whether it inadvertently helps participants to perform better than they otherwise would, resulting in the appearance of universality. In three studies, we provide a strong test of the hypothesis that the classic choice-from-array task constitutes a potent source of context that shapes performance. Participants from a remote small-scale (the Hadza hunter-gatherers of Tanzania) and two urban industrialized (China and the United States) cultural samples selected target vocalizations that were contrived for six non-English, non-universal emotion categories at levels significantly above chance. In studies of anger, sadness, fear, disgust, happiness, and surprise, above chance performance is interpreted as evidence of universality. These studies support the hypothesis that choice-from-array tasks encourage evidence for cross-cultural emotion perception. We discuss these findings with reference to the history of cross-cultural emotion perception studies, and suggest several processes that may, together, give rise to the appearance of universal emotions.

Keywords: emotion, perception, vocalizations, universality, culture

There has been a lively and prolonged debate about whether or not certain emotion categories are universally expressed and recognized. According to the strongest version of the universality hypothesis, all humans (barring illness) innately produce and perceive anger, disgust, fear, happiness, sadness and surprise in nonverbal behaviors, independent of cultural background and learning.¹ The original tests of this hypothesis were conducted in a handful of studies between 1969 and 1975 in two remote, small-scale societies, the Fore and Dani of Papua New Guinea (Ekman, 1972; Ekman & Friesen, 1971; Ekman, Sorenson, & Friesen, 1969; Sorenson, 1975) (for a review, see Gendron et al., in press). The studies employed several versions of a choice-from-array task pioneered by Dashiell (1927): participants heard a brief story about an emotional episode with an emotion word embedded (e.g., “Her child has died and she feels very sad.”) and are then asked to select the target stimulus (e.g., a photograph of a posed facial configuration of a frown) presented alongside one or two foils.² Participants chose the target stimulus more frequently than chance, leading to the conclusion that anger, sadness, fear, happiness, surprise, and disgust were universal emotions (on the assumption that people infer emotion in certain facial configurations only if those configurations express emotion with a certain degree of fidelity; for discussion, see Jack, Sun, Delis, Garrod, & Schyns, 2016). Since then, hundreds of studies have been published using a choice-from-array task with participants from various urban cultural contexts with some degree of exposure to western cultural practices and norms, such as Brazil, China, Estonia, Ethiopia, France, Greece, India, Indonesia, Israel, Japan, Kenya, Malaysia, Mexico, Singapore, Turkey, and Zambia (Elfenbein & Ambady, 2002; Laukka et al., 2013), providing further support for the universality hypothesis (for discussion, see Ekman, 2017). The strongest evidence continues to come from the original studies of emotion perception in the Fore and Dani of Papua New Guinea (Ekman, 1972; Ekman & Friesen, 1971; Ekman, Heider, Friesen, & Heider, 1972), as well as from choice-from-array tasks used with the Himba of Namibia (Sauter, Eisner, Ekman, & Scott, 2010) and Bhutanese villagers (Cordaro, Keltner, Tshering, Wangchuk, & Flynn, 2016), because participants in these studies had limited exposure to western cultural practices and norms, including media, minimizing alternative explanations for any cross-cultural consistencies that were observed (Norenzayan & Heine, 2005). In the present paper, however, we present three studies to support the hypothesis that the choice-from-array task itself creates evidence for universal emotions. This evidence, we suggest, is not simply the result of demand characteristics or confirmatory bias. The task does not prime innate emotion knowledge. Instead, we suggest that choice-from-array tasks contain psychologically potent features that guide participants as they make emotional meaning of novel stimuli.

Choice-From-Array Tasks

There is now evidence from a variety of experimental studies that choice-from-array tasks provide a potent context that helps participants choose the stimuli that experimenters expect, creating stronger evidence for universal emotions than might occur otherwise (L. F. Barrett, Adolphs, Marsella, Martinez, & Pollak, forthcoming; Russell, 1994; Russell, Bachorowski, & Fernandez-Dols, 2003). Numerous lab-based studies with both children and adults (L. F. Barrett et al., forthcoming), as well as almost a dozen cross-cultural studies published since 2008, which sample a broader range of remote, small-scale populations (see Table S1; also Gendron et al., in press) and ask people to infer emotional meaning in both facial poses and vocalizations, provide evidence that choice-from-array tasks are not psychologically inert. Instead, as reviewed next, these tasks can contain elements that are known to shape participants’ responses (for a broader discussion of contextual factors that influence emotion perception, see Aviezer & Hassin, 2017; L. F. Barrett, Mesquita, & Gendron, 2011; de Gelder, 2016; Gendron, Mesquita, & Barrett, 2013; Hess & Hareli, 2017; Wieser & Brosch, 2012).³

Emotion words.

The emotion words used in choice-from-array tasks may encourage participants to assign emotional meaning to facial actions and vocalizations in ways they would otherwise not. When emotion words are familiar (i.e., associated with known emotion concepts), they directly shape participants’ perceptions, such that facial configurations like scowls and pouts are seen differently when words like “angry” and “sad” are present during the experiment vs. when they are absent (Gendron, Lindquist, Barsalou, & Barrett, 2012). In such contexts, emotion words influence which facial movements are predicted, encoded, and remembered (Chanes, Wormwood, Betz, & Barrett, 2018; Doyle & Lindquist, 2018; Fugate, Gendron, Nakashima, & Barrett, 2017; Fugate, Gouzoules, & Barrett, 2010). These findings suggest the interpretation that emotion words are aids that help perceivers recognize universal emotions more easily. Emotion words may assist perceivers in choosing the target stimulus from an array by strengthening their predictions or enhancing their sensitivity for the target or its perceptual features (Chanes et al., 2018; Mohanty & Sussman, 2013; Vogt, De Houwer, Moors, Van Damme, & Crombez, 2010). Other evidence suggests that the effect of emotion words is more potent than that, potentially creating emotion perceptions that would otherwise not occur. For example, participants label scowling faces as “determined” or “puzzled,” wide-eyed faces as “hopeful” and gasping faces as “pained” when they are provided with stories about those emotions rather than with stories of anger, surprise, and fear (Carroll & Russell, 1996, Study 2). Scowling faces are also more likely to be perceived as fearful when paired with the description of danger (Carroll & Russell, 1996, Study 1).

Words are also particularly potent when people are presented with cues, such as facial movements, that are unfamiliar and have no prior emotional meaning (Fugate et al., 2010), consistent with the broader finding that words support perception for unfamiliar objects (Lupyan, Rakison, & McClelland, 2007). Furthermore, experimental tasks that place fewer constraints on how participants respond, such as asking them to freely label facial poses or vocalizations, rarely provide evidence for the universality hypothesis (see Table S1; L. F. Barrett et al., forthcoming; Gendron, 2017; Gendron et al., in press). Finally, children who do not explicitly possess conceptual knowledge about emotions such as anger, sadness, fear, and who do not understand the meaning of emotion words beyond their affective content, as well as adults who have lost that semantic knowledge from neurodegeneration, are experientially blind to the emotional meaning of facial poses and vocalizations: they only perceive affect (Lindquist, Gendron, Barrett, & Dickerson, 2014; Widen, 2016).

Elaborate manipulation checks.

Choice-from-array tasks have been known to use an elaborate manipulation check procedure that has the potential to teach novel emotion concepts to naïve participants. For example, Sauter et al. (2010) used what they referred to as a “rigorous” manipulation check that did more than ask for verbal confirmation of whether individuals from the Himba culture of northwestern Namibia understood anger, sadness, fear, and the other concepts represented in brief stories used in the experiment. As stated in Sauter et al. (2010), after hearing each emotion story, participants were “asked how the person was feeling” (p. 2411, italics added) to confirm that “they had understood the intended emotion of the story” (p. 2408). In their later commentary, Sauter et al. (2015) elaborated that:

Participants [were allowed] to listen several times to a given recorded story (if needed), until they could explain the intended emotion in their own words, before they proceeded to the experimental trials for that story. The inclusion of a rigorous manipulation check with experimenter verification, rather than reliance on participants’ reports, was thus crucial. (p. 355).

Individuals in the Himba cultural group display opacity of mind, however; they do not make mental state inferences as frequently as more westernized people (H. C. Barrett et al., 2016), focusing on emotion as situated action rather than internal feeling.⁴ As a consequence, when Sauter et al. asked participants to verbally describe the English emotion concept that was portrayed in the story, they perhaps (unwittingly) encouraged concept learning in their experiment (as hypothesized by Gendron, Roberson, & Barrett, 2015). They did not allow Himba participants to proceed to the experimental trials for a given story until they could conceptualize the emotion stories in a manner consistent with western cultural expectations.

By contrast, a subsequent study using a choice-from-array task with the Himba, but without the elaborate manipulation check, was unable to replicate the Sauter et al. (2010) findings (Gendron, Roberson, van der Vyver, & Barrett, 2014a). Our study used a manipulation check only to verify that participants understood how to perform the task. Participants were given the opportunity to indicate that they understood the emotion stories from their own cultural perspective. We took great pains to ensure participants were attending to the experimental task, understood the action described in the brief stories, and understood what was required of them: we conducted an attention check before every trial, verbally confirmed understanding (yes or no response), and allowed participants to replay the story for any reason. The concern that participants may have missed some of the stories, due to an attentional lapse or a failure of memory, is assuaged by that fact that the stories were repeated many times over the course of the experiment. Analyses of the different foil conditions revealed that Himba participants performed better than chance only when the target and foil differed in valence, such that pleasantness or unpleasantness could be used to distinguish between the two vocalizations. The general pattern of these findings was replicated in a free-labeling task (Gendron et al., 2014a, Study 1).

Repeated trials.

Choice-from-array tasks present the same stimuli over and over. The fact that participants are repeatedly exposed to the same facial configurations and emotion words creates a context for them to learn the intended pairings, even if they do not know them to begin with (Nelson & Russell, 2016a). For example, children learn to label an artificially constructed facial expression (e.g., a blowfish expression) with the word “pax” in a choice-from-array task (Nelson & Russell, 2016b). And participants have been shown to use a process of elimination strategy to complete choice-from-array tasks, boosting agreement levels (DiGirolamo & Russell, 2017). In fact, if the target stimulus is presented against only one foil, all that is required in this task arrangement is to figure out which of two vocalizations or two faces is least expected in the given context. For example, after hearing a story about anger, a participant hears a shout and a laugh, and can choose the shout merely by realizing the laugh is not correct (on the basis of valence). This is similar to selecting the correct answer on a multiple-choice test by eliminating the incorrect alternatives.

Choice-from-array tasks have also been known to block trials by emotion category, which may further encourage perceptual learning and a process of elimination strategy. For example, Sauter et al. (2010) blocked their trials: a Himba participant heard a story, verbalized an English emotion concept, and then heard a series of trials with portrayed vocal bursts for the target emotion category and a foil. For example, after hearing a story about sadness several times and describing the English concept for sadness, a participant would hear a cry (target) and a laugh (foil), then a shout (foil) and a cry (target), and so on. When Himba participants completed a similar task with randomized trials, they did not choose vocalizations for anger, sadness, fear, and the other categories tested in a way that was supportive of the universality hypothesis (Gendron et al., 2014a).

Meaning Making

It is tempting to assume that these design considerations are mere methodological footnotes, but we suggest an alternative hypothesis. Taken together, these findings suggest that the design elements in a choice-from-array task are psychologically potent: they provide a context for perceivers to make meaning of ambiguous physical cues like facial movements and vocalizations, particularly when those cues have no inherent emotional meaning. As demonstrated by a recent review, hundreds of studies of healthy adults across cultures, newborns and young children, and people who are congenitally blind or deaf consistently find that specific facial movements, such as smiling, scowling, and frowning are context-specific in their emotional meaning: instances of the same emotion category, such as instances of anger, are expressed with more variable facial movements than generally acknowledged, and similar facial movements, such as a scowl, can communicate a variety of different emotions, or even carry non-emotional information (L. F. Barrett et al., forthcoming). As a result, perceivers implicitly use situational context to make meaning and infer emotional information in facial movements. The same is likely true of vocalizations (Russell et al., 2003). As a result, choice-from-array tasks may provide a context that allows participants to infer emotional meaning differently than they otherwise would, shaping their emotion perception performance. If this hypothesis is correct, then it provides an alternative explanation for the hundreds of studies that support the hypothesis that people universally recognize emotion in nonverbal behaviors in an automatic and obligatory way.

The hypothesis that choice-from-array tasks facilitate emotion perception fits with the observation that humans are active meaning-makers. Human brains are wired to transduce changes in light, air pressure, and chemicals, and go beyond the information given (Bruner, 1957) to create the sights, sounds, and smells of our surrounds. The same changes in air pressure can be experienced as a person’s laugh in one setting and a sob in another (Belin, Fillion-Bilodeau, & Gosselin, 2008). The psychological literature is full of experiments that manipulate context to examine its effect on meaning. For example, visual context facilitates the recognition of scene-consistent objects (for a review, see Bar, 2004). And it is well known that the acoustic and linguistic context influences which phonemes are heard (Massaro & Cohen, 1983). There are notable examples of how certain experimental design features evoke powerful psychological effects that are hidden contexts for other psychological processes (such as controlled processing; for a review, see Pashler, Johnston, & Ruthruff, 2001). Context effects are well-established in how people perceive emotion in faces, voices and bodies (e.g., Aviezer et al., 2008; Aviezer, Trope, & Todorov, 2012; Calbi, Angelini, Gallese, & Umiltà, 2017; Fernández-Dols & Ruiz-Belda, 1997; Fridlund, 1991; Mobbs et al., 2006; Ruiz-Belda, Fernández-Dols, Carrera, & Barchard, 2003; Van den Stock, Righart, & de Gelder, 2007; Wallbott, 1988).

Our hypothesis here is that choice-from-array tasks contain features that encourage participants to infer particular emotional meaning in vocalizations such as shouts, sighs, and laughs, creating the appearance of universal emotions. Participant performance on tests of cross-cultural emotion perception may be the result of multiple processes, including identifying perceptual similarities or employing process-of-elimination strategies (e.g., Nelson & Russell, 2016a), perceiving affect (e.g., Gendron et al., 2014a), and learning categories online (e.g., Ferry, Hespos, & Waxman, 2010), calling into question the validity of interpreting such performance as direct evidence for the innate universality of certain emotion categories. In addition, it is possible that participants may use conceptual combination to complete cross-cultural emotion perception tasks when faced with unfamiliar emotion categories and exemplars. Conceptual combination is a fundamental cognitive capacity that allows individuals to construct instances of novel categories (such as emotion categories not present in their culture) by flexibly combining previously acquired conceptual knowledge (Barsalou, 1987). Conceptual combination does not imply that all properties of the original concepts will be invoked in a novel instance (Wu & Barsalou, 2009), but only those that are relevant for a particular situation (e.g., a given emotion story). This is the way that people can perceive instances of novel categories, including emotion categories, for which they have no single vocabulary word or even prior experience (for discussion, see L. F. Barrett, 2017a). As such, conceptual combination is a plausible process by which participants leverage features of the experimental context to infer emotional meaning in novel stimuli.

The Present Studies

In the studies that follow, we tested the potency of the classic choice-from-array task to create a context that allows participants to infer emotional meaning for novel vocalizations that have no inherent emotional meaning for them. We present data from one sample of U.S. participants (Study 3) and from samples in two other cultural contexts. In Study 1, we test individuals from a remote, small-scale society with relatively little exposure to western cultural norms, practices, and values (Henrich, Heine, & Norenzayan, 2010; Norenzayan & Heine, 2005) – hunter-gatherers from northern central Tanzania – who may not, on their own, make emotional meaning of vocalizations such as laughs, sighs, and shouts in the same way as U.S. participants. Our recent study of emotion perception in the Hadza indicates that they do not freely label scowls, smiles, frowns and other facial poses in the presumed universal way (Gendron et al., under review). Therefore, Study 1 provides a strong test of our hypothesis. Study 2 tests Chinese participants living in China who have access to western cultural norms, practices, and values, but who also have their own as well, enhancing the variable emotional meanings that are available for physical cues such as facial movements and vocalizations. Recent research with dynamic facial movements comparing U.S. and Chinese participants shows that instances of the same emotion are expressed with multiple sets of facial movements and similar facial movements express different emotion categories (Jack et al., 2016). As a consequence, Study 2 provides a test of the choice-from-array task’s potency for creating a context for emotional meaning making under conditions of enhanced ambiguity. Across three studies, we demonstrate that the choice-from-array task typically used in studies of cross-cultural emotion perception actively (although unintentionally) facilitates task performance, resulting in findings that make non-universal emotion categories appear universal.

In all three studies, the goal was not to separately test the potency of individual elements in the choice-from-array task, such as emotion words, repeated and blocked trials, or elaborate manipulation checks. Rather, we sought to demonstrate that, together, these elements create a highly structured version of the task that encourages emotional meaning making, and therefore task performance, providing evidence for universal emotions that may not otherwise emerge. A more mechanistic approach is not optimal at this stage of our research for two reasons. First, while it may be possible to conduct systematic control conditions in laboratory experiments, it is often not feasible in field studies with remote populations limited in number (such as the Hadza), where researchers must operate under site constraints. Second, it is common practice to establish the replicability and robustness of an effect before investing the resources to undertake the long series of studies required to isolate each feature and testing its unique contribution, or manipulating features to model their synergistic impact.

There are several notable aspects to the studies we report in this paper. First, to provide a stringent test of our hypothesis, we traveled to northern central Tanzania to test emotion perception in members of a remote small-scale hunter-gatherer culture, the Hadza (Study 1). This paper reports the first study of emotion perception ever conducted with the Hadza (the second being our study of emotion perception in faces, reported in Gendron et al., under review). These studies are particularly important because, according to ideas from evolutionary psychology, universal and innate emotional expressions evolved to solve to the recurring fitness challenges of hunting and gathering in small groups on the African savanna (Pinker, 1997; Shariff & Tracy, 2011; Tooby & Cosmides, 2008). Therefore, the Hadza provide the strongest test of whether certain emotion categories are universal. Their cultural isolation is rapidly under assault, becoming contaminated by tourists and assimilation, and so the opportunity to study their emotional lives is rapidly disappearing. We examined the replicability of our findings by conducting the same study in an industrialized culture from the East (China, Study 2) and the West (the United States, Study 3).

Second, we chose to study the impact of the choice-from-array context on six emotion categories that would be novel to our participants: gigil, greng jai, gluckschmerz, itoshii, lajja, and liget (Table 1). These categories are not translatable by a single word in English, meaning that participants likely do not invoke these specific category boundaries in their daily lives. (It does not mean that participants would be unable to understand or relate in some way to the experienced described.) Nor do these emotion categories meet the usual criteria for universality: they do not appear in Darwin’s The Expression of the Emotions in Man and Animals (1872/2005), they do not appear to be evidenced in the behaviors of non-human animals (see Tracy & Randles, 2011), and are they not thought to solve a recurring evolutionary challenge for our Pleistocene ancestors (see Shariff & Tracy, 2011). We provided participants with short descriptive scenarios and stipulated (i.e., made-up) vocalizations for each emotion category (Table 1, see Supplemental Material for details). We verified that the six novel emotion categories were unfamiliar in the cultural samples being tested, in that participants did not consistently and specifically associate the scenarios or vocalizations with pre-existing, emotion categories in their native language. This verification process is reported in the method section of each study.

Table 1.

Novel Emotion Categories

Category	Pronunciation	Origin	Description	Scenario
Gigil	GHEE-ghil	Philippines	The overwhelming urge to squeeze or pinch something that is very cute	Someone sees a small, chubby, lovely baby and wants to squeeze it tightly. They feel gigil.
Glückschmerz	GLOOK-shmairts	Germany	Displeasure derived from another’s pleasure	Someone hears that a bad person had some good fortune, and feels upset about it. They feel glückschmerz.
Greng jai	kreng-JAI	Thailand	The feeling you get when you don’t want someone to do something for you because it would be a pain for them	Someone is offered help from others, but does not want it, because it is too much trouble for the others. The person feels greng jai.
Itoshii	ee-toe-SHEE	Japan	Bittersweet longing for an absent loved one	Someone thinks pleasant things about their loved one who has moved away (to another camp).^* They feel itoshii.
Lajja	lah-ZHAH	India	Respectful restraint or playful shame; pleasant adherence to social norms	Someone makes a small mistake that others will notice and feels bad, but also acts playful. They feel lajja.
Liget	LI-gut	Ilongot (Philippines)	Intense focus, passion, and energy associated with actively pursuing a challenge	Someone works very hard toward a goal, and feels a rush of energy and intense focus. They feel liget.

Open in a new tab

Note: Additional testing verified that these categories were unknown to the three cultural samples. Participants were asked to freely label vocalizations and scenarios developed for each category. Examination of the labels confirmed that participants produced neither consistent nor specific labels. See Supplemental Material (p 7–17) for details.

The content in parentheses was included in Study 1 for clarification, but not Studies 2 and 3.

On each trial in our studies, participants heard scenarios describing an emotional experience for which they did not have a pre-existing category or word, and then heard two contrived vocalizations, one that was invented as the target for the novel emotion category and the other invented for different novel emotion category (the foil). We hypothesized that the choice-from-array task would aid participants in selecting stipulated target vocalizations for six novel emotion categories from around the world with a level of agreement normally interpreted as evidence for universality. That is, the choice-from-array task would make six non-universal categories look universal in three different samples from around the world.

If certain emotion categories are indeed universal, and the choice-from-array task does not encourage participants to choose the expected answers, we would expect to find support for the null hypothesis: participants would not choose the stipulated vocalizations for these novel emotion categories at levels greater than chance. In contrast, support for our hypothesis would be found if participants perform at above chance levels, choosing the vocalizations we invented for the emotion categories that are not traditionally part of U.S., Chinese, or Hadza culture. Specifically, these results would be consistent with (even if they do not directly test) an account of novel emotion meaning-making based on conceptual combination, in addition to other general processes like affect perception and category learning.

Use of a Bayesian Analytic Approach

We used Bayesian hypothesis testing to quantify the evidence for universality hypothesis (corresponding to the null hypothesis) that participants would select target vocalizations at chance levels for the novel emotion categories, as well as our alternative (task-as-context, henceforth ‘context’) hypothesis that they would select the vocalizations at levels greater than chance. The ability to quantify evidence in favor of both null and alternative hypotheses one of the main advantages of Bayesian hypothesis testing over frequentist approaches such as t-tests and ANOVAs (e.g., Edwards, Lindman, & Savage, 1963; Rouder, Morey, Speckman, & Province, 2012; Wagenmakers et al., 2015) because p-values cannot provide support for the null (Wetzels et al., 2011). Bayesian hypothesis testing allowed us to assess the ratio between the probability of our context hypothesis given the data, as well as the probability of the null (universal) hypothesis given the data.

Another key advantage to Bayesian hypothesis testing is that it yields a de facto power analysis and replaced the need for a separate power analysis (Berger & Mortera, 1999; Berger & Wolpert, 1988; Rouder, 2014; Wagenmakers et al., 2015). The main statistic of interest is the Bayes factor, which expresses the relative evidence for null and alternative hypotheses and consistently trends toward truth as data accumulate, unlike a p-value (Rouder, Speckman, Sun, Morey, & Iverson, 2009). Using this approach, we were able to determine that we had adequate sample sizes to test both hypotheses. We supplemented our Bayesian analyses with hierarchical generalized linear modeling analyses (HGLM; Raudenbush, Bryk, Cheong, Congdon, & du Toit, 2004), as discussed in the results sections of each study.

Study 1: Remote Small-Scale Sample – the Hadza of Tanzania

The Hadza are a semi-nomadic, hunter-gatherer population who live in small bands in Tanzania and remain relatively isolated from other cultural practices and norms. They are egalitarian foragers who bring food back to camp for distribution (Berbesque, Wood, Crittenden, Mabulla, & Marlowe, 2016). They consume a diet that is primarily based on the collection of wild foods (Apicella & Crittenden, 2015); the women gather and the men hunt and collect honey. The Hadza are not a Paleolithic population, but have been continuously hunting and gathering for thousands of years in East Africa, and they are one of the last groups to live in a social and physical context closer to that of our Pleistocene ancestors than our own (Marlowe, 2010). An estimated 150–200 Hadza individuals currently live in remote camps (Jones, 2016). In Study 1, we recruited and tested 55 of these individuals. Prior to our visit to the Hadza in 2016, they had not participated in any studies of emotion perception, although some had previously participated in social cognition research (H. C. Barrett et al., 2016; Bryant et al., 2016). During this visit, we conducted two studies (Study 1 reported here, and a second study with posed emotional faces reported in Gendron et al., under review).

Method

Participants.

Participants were 55 native Hadzane speakers (21 female) from the Hadza ethnic group, recruited from the area surrounding three camps located southeast of Lake Eyasi in northern central Tanzania: camp 1 (n = 20), camp 2 (n = 27), and camp 3 (n = 8) (Figure S7). All three camps were far from the regional towns, and there was no evidence of recent tourism to these areas. During the time of data collection, the majority of the Hadza individuals we tested did not travel except to other Hadza camps, although regional towns and small cities were visited occasionally for access to supplies and medical care. Only three individuals we tested reported venturing further afield to the larger cities of Arusha, Dar es Salaam, or parts of Kenya. Participants ranged in age from 18 to mid-70s, with a median decade of 30–40; Hadza individuals do not keep track of their biological age, so estimates were based on maternal/paternal history and personal knowledge of our translator (the third author, S.M.), who is ethnically Hadza as well as an experienced research assistant.

Participants were tested in their native language, Hadzane, which is a click-based language considered to be a linguistic isolate (Sands, 1998). Almost all Hadza speak some Swahili, however, as they use this language to communicate with neighboring ethnic groups. Twenty adults in our sample reported fair to excellent knowledge of Swahili, whereas the other 35 most reported little knowledge of Swahili. In addition, five individuals reported modest knowledge of another neighboring language, such as Datoga, Iraqw, or Isanzu. Only one participant claimed rudimentary knowledge of English, but did not use it with the experimenters. Sixteen participants in the sample had attended local primary school for between two and seven years. One participant attended the regional secondary school for two years.

Data collection was approved by the Office of Human Subject Research Protection Institutional Review Board at Northeastern University as well as the Tanzanian Commission for Science and Technology (COSTECH). Hadza participants were verbally consented prior to participation and were remunerated with gifts (clothing, cookware, etc.).

Stimuli.

Stimuli were six short scenarios (one per emotion category) and 60 non-word vocalizations (10 per emotion category). For comparison, Sauter et al. (2010) used short scenarios from nine emotion categories (achievement, amusement, anger, disgust, fear, sensual pleasure, relief, sadness, and surprise), each with 10 non-word vocalizations.

Emotion concept selection.

We selected six emotion concepts from the Interdisciplinary Affective Science Laboratory’s database of emotion concepts that have been deemed ‘untranslatable’ into English. We compiled this database (available upon request) using published psychological, anthropological, and linguistic literatures (e.g., Lomas, 2016; Rosaldo, 1980; Russell, 1991), as well as websites and publications in the popular media (e.g., BetterThanEnglish, 2012; De Boinod, 2007; Lin, 2013). We selected six emotion concepts that were easy to portray with static facial poses and vocalizations. None of these concepts are labeled with single words in English. None are frequently described in the English media. And none have been studied in a scientific context or stipulated as universal. We refer to these as ‘novel’ emotion concepts because they are unfamiliar to English and Hadzane speakers alike (i.e., they are not represented as existing emotion categories in either English or Hadzane). Fluent English speakers from the United States on Amazon’s Mechanical Turk (MTurk) rated the six emotion concepts on valence and arousal (n = approx. 15 per concept; 43.5% female; mean age = 32.2, SD = 8.79; see Figure S1).

Emotion scenarios.

We generated six short scenarios, one for each novel emotion concept. The scenarios (Table 1) were developed based on concept descriptions by native speakers of the source languages, and were adapted for cultural fit by the second author (A.N.C.), a nutritional and ecological anthropologist with extensive field experience working among the Hadza. Our translator, the third author (S.M.), a native speaker of Hadzane and a fluent speaker of English, also reviewed the scenarios, and recorded them in Hadzane. He is one of the few Hadza to have attended university, and has previously been involved in anthropological research (e.g., Raichlen et al., 2017). Each scenario was one to two sentences long, and did not differ substantially in structure from those used in Sauter et al. (2010), which were themselves adapted from the scenarios used by Ekman, Sorenson, and Friesen (1969). Each scenario briefly described an emotional situation in concrete terms and concluded with a description of how the protagonist felt using the target emotion word in its original language. For example, “Someone sees a small, chubby, lovely baby and wants to squeeze it tightly. They feel gigil.”

Vocalizations.

We (the authors) invented the short vocal bursts that served as posed vocal expressions of each emotion category (i.e., we stipulated their pitch, tone, duration) based on our folk understanding of the novel emotion concepts in Table 1. We then directed five posers from the United States (three women, two men) to generate these short vocal bursts. Imitating emotion poses is a well-established method for stimulus generation in studies of emotion perception (e.g., Ekman, Levenson, & Friesen, 1983; Schröder, 2003). Posers were first provided with descriptions of the desired vocalizations for each emotion category (Table S3), which were also modeled by the first author. Posers then generated the vocalizations while imagining the emotional situations described in the scenarios. All vocalizations were cleaned for ambient noise and adjusted for mean peak amplitude using Audacity (Audacity Team, 2015).

Next, we examined whether the vocalizations portraying the same emotion category clustered together in multidimensional space. We opted for a multidimensional scaling (MDS) approach (Hout, Papesh, & Goldinger, 2013; Jaworska & Chupetlovska-Anastasova, 2009) because it allowed us to discover, rather than confirm, whether the vocalization associated with the same emotion category are similarly positioned in perceptual space. Prior MDS analyses confirm that vocal stimuli sharing similar acoustic features do cluster together (Gygi, Kidd, & Watson, 2007). Pilot participants (N = 7) heard pairs of vocalizations, one member of the pair played after the other, and then rated their similarity from 1 (very similar) to 9 (very dissimilar). All possible pairwise comparisons were presented in randomized order, with vocalizations randomly assigned to play first or second. A single matrix of mean similarity ratings was computed for all vocalization pairs and subjected to an ALSCAL procedure (Young & Lewyckyj, 1979). Results indicated that vocalizations posed for the same novel emotion category clustered together in multidimensional space (Figures S4 and S5). A stress-by-dimensionality plot (Figure S3) indicated that the three-dimensional solution provided the best fit for the data. The dimension loadings for the vocalizations suggested that the first two dimensions are valence and arousal, replicating prior findings from MDS analyses of affective stimuli (e.g., Russell, 1980; Sauter, Eisner, Calder, & Scott, 2010 also report similar findings using principal component analysis [PCA]). Cluster locations were similar to explicit valence and arousal ratings for the emotion scenarios (Figure S1). A separate sample of MTurk raters (N = 25; 8 female; mean age = 31.56, SD = 7.04) also verified that the posed vocalizations for the same novel emotion category clustered together in multidimensional space (Figure S6). We elicited a second set of vocalizations from five Namibian posers (three women, two men) as part of a separate study, based on these normed vocalizations, for a total of 10 per emotion category.

Procedure

Verifying vocalization novelty.

Before completing the choice-from-array task, nineteen participants freely labeled six of the made-up vocalizations, one for each emotion category under investigation. Participants were asked to describe how the target person making each sound felt using a word or short phrase. Participants who provided a description of a situation or behavior were prompted to describe how the target person was feeling, to replicate the method described by Sauter et al. (2010, 2015). Participant-provided labels were coded according to whether they represented a known emotion category in English, and whether or not this category was subordinate to or synonymous with a purportedly universal emotion category (i.e., anger, disgust, fear, happiness, sadness, surprise; see Supplemental Material for details on the coding procedure). We expected, and found, that the consistency and specificity of labeling each vocalization was low. Participants did not label the vocalizations with the words for the novel emotion concepts, nor did they provide labels such as angry, fearful, sad, etc., and their synonyms. In fact, most participants provided labels for the vocalizations that did not represent a known emotion category in English (for gigil, 79% of responses did not refer to a discrete emotion category; gluckschmerz, 37%; greng jai, 81%; itoshii,78%; lajja, 33%; liget,53%), (panel A of Figure S8). Furthermore, participants labeled each vocalization with words that ranged in their affective features, using both positively or negatively valenced labels, as well as labels that were high or low in arousal, indicating that there was little consistency and specificity in the affective features associated with the vocalizations (panel A of Figures S9 and S10).

Choice-from-array task.

Each participant was tested individually and completed the classic choice-from-array task, following the methods used in Sauter et al. (2010; for additional details, see Sauter et al., 2015). All participants were tested individually using a Dell ATG laptop and headphones. Participants listened to a pre-recorded scenario in their native language (e.g., in English, “Someone sees a small, chubby, lovely baby and wants to squeeze it tightly. They feel gigil.”). Participants then completed an extensive manipulation check (panel A of Figure 1) as described in Sauter et al. (2015). They were asked to describe how the scenario protagonist feels, and a correct answer would contain features other than simply repeating the target emotion label used in the scenario. For example, a response (in Hadzane or Swahili) of “he feels gigil” was not accepted, whereas a response of “he feels good”, “he feels loving”, or “he feels angry” was acceptable. Participants were allowed to listen to the recorded scenario multiple times, and received verbal feedback in their native language on their responses. Because the Hadza are a pre-literate society, we communicated verbal instructions and feedback through our Hadzane translator (S.M.).

Once participants had passed the manipulation check for a given emotion category, they then completed a block of trials for that category (panel B of Figure 1). For example, on every trial within a gigil block, participants listened to the audio recording of the scenario followed by target (an invented gigil sound) and a foil vocalization (e.g., an invented sound for the lajja category). Each trial drew from a list of 10 possible vocalizations for each novel emotion category. Foils were drawn randomly from one of the other (non-target) categories and were matched for sex of poser. As the first vocalization played, an icon appeared on the left side of the computer screen, and then disappeared. While the second vocalization played, the mirror version of the icon appeared on the right side of the screen, and then disappeared. Both icons then appeared simultaneously, and participants pressed the left or right icon to indicate which vocalization matched the emotion portrayed in the scenario. Target position (left or right) was randomized for target and foil. On a given trial, scenarios and vocalizations were repeated for participants who wished to hear them again. Upon completing the first trial, participants heard the gigil scenario again, followed by an invented gigil sound and another foil (e.g., an invented sound for the liget category). Once the block of trials for a given emotion category was complete, participants proceeded to another block of trials for a different emotion category. Emotion category blocks were presented in randomized order. Participants completed six emotion category blocks of five trials each, for a total of 30 trials.

Fig. 1. — The classic, highly structured choice-from-array task with novel emotion categories. Pre-recorded material presented over headphones is depicted in light gray boxes; verbal interactions with the experimenter are presented in dark gray boxes. A. Manipulation check: Participants listened to a scenario in their native language and were then asked to describe how the protagonist in the story feels. B. Perception trials blocked by emotion category. Following the manipulation check for a given emotion category, participants completed a block of trials for that category. On every trial, participants listened to the scenario again followed by target and foil vocalizations. Vocalizations were played, one at a time, with an icon presented concurrently on either the left or right side of the screen (no words were present on screen). Participants touched an icon to select a vocalization. Once a trial was complete, participants completed another trial from the same emotion category. Targets were presented randomly on the left or right within a block.

Field site constraints dictated that each participant spend a similar amount of time in testing. As a result, all participants completed all experimental trials, but a participant’s data for a given block of trials were removed prior to analysis if he or she failed the manipulation check for that emotion category three times. The sample size used in the analyses can be found in Figure 3.

Verifying emotion scenario novelty.

Following completion of the choice-from-array task, participants who freely labeled the vocalizations also freely labeled long-form versions of the scenarios that did not have any emotion word embedded. Only six participants were available to do this norming due to time constraints. As expected, participant-provided labels showed little agreement across participants and did not converge with known English concepts anger, disgust, fear, happiness, sadness, and surprise (see panel A of Figure S11). A similar finding was observed for valence and arousal properties (see panel A of Figures S12 and S13).

Results

Bayesian hypothesis testing.

We conducted a Bayesian one sample t-test to directly evaluate the degree of support for the null (universal) hypothesis (i.e., chance-level performance at .5), as well as to evaluate the sequentially accumulating support for the alternative (context) hypothesis (i.e., mean performance greater than chance) (Rouder et al., 2009; Wetzels, Raaijmakers, Jakab, & Wagenmakers, 2009). Analyses were computed in JASP (JASP Team, 2017). Given that the effects sizes that support the context hypothesis might vary, a Bayesian approach evaluates a distribution of expected effect sizes rather than a single estimate. In JASP, we estimated the probability that the context hypothesis was true given a one-sided prior probability distribution where the median effect size was equivalent to Cohen’s δ = .7. This is consistent with strong evidence for the universality of emotion categories in choice-from-array tasks (Haidt & Keltner, 1999).⁵

A Bayesian one sample t-test conducted on aggregate response data demonstrated robust support for the hypothesis that participants would perform above chance relative to the null hypothesis. That is, Hadza participants performed at a level typically interpreted as evidence of universality (i.e., perceived the intended novel emotions) when tested using the highly structured version of the classic choice-from-array task used by Sauter et al. (2010). As can be seen in panel A of Figure 2, the Bayes factor for our Hadza participants was BF₁₀ = 187.905, indicating that the observed data are 187.905 times more likely under the alternative (context) hypothesis than under the null (universal) hypothesis. This is considered “very strong” (Bayes factor greater than 150; Kass & Raftery, 1995) or “decisive” (Bayes factor greater than 10^2; Jeffreys, 1961) evidence for the context hypothesis.

These findings were robust, even when we varied the parameters of our analysis. For example, we varied the width of the prior probabilities for the effect size under the alternative (context) hypothesis. Narrower priors allow for a smaller range of expected effect sizes, and therefore represent a more stringent test of the context hypothesis when compared to the null (universal) hypothesis (i.e., a prior width very close to zero most favors finding evidence for the null hypothesis). Under these conditions, the Bayes factor still indicates substantial support for the context hypothesis (i.e., BF₁₀ > 3).

We also generated sequential analysis plots to examine the development of the Bayes factors as data accumulated. We observed that cumulative data increasingly provided evidence for the alternative (context) hypothesis when compared to the null (universal) hypothesis, lending confidence that our sample size was large enough and we were sufficiently powered to test our hypotheses (panel D in Figure 2).

Fig. 2. — Inferential plots for Study 1, based on Bayesian one-sample t-tests on overall performance of Hadza participants. a. Pizza plot providing a proportional representation of the ratio of evidence for the alternative (context) hypothesis to evidence for null (universal) hypothesis. b. Posterior distribution based on a one-sided prior distribution. Posterior mass to the right of zero indicates that participant performance is consistently above chance-level responding of .5. c. Bayes factor robustness check. Narrower priors indicate a smaller range of expected effect sizes, favoring the null (universal) hypothesis. d. Sequential analysis of evidence accumulated over the course of data collection. All figures adjusted from JASP (JASP Team, 2017).

Hierarchical Generalized Linear Modeling (HGLM).

In addition, we used hierarchical generalized linear modeling (HGLM; Raudenbush et al., 2004) to examine whether participants’ performance was above chance-level responding when analyzed by novel emotion category. We chose HGLM for two reasons. First, the main dependent measure in the choice-from-array task is dichotomous: Did a participant pick the target vocalization on a given trial? (1 = yes, correct; 0 = no, incorrect). Participants’ performance across multiple trials can be modeled as a binomial distribution, bounded at both ends (i.e., has values between 0 and 1). Traditional parametric approaches such as one-sample t-tests and ANOVAs with binomial or categorical data cannot be used to analyze data from the choice-from-array task because binomial response data must be treated as proportions or percentages, resulting in confidence intervals that can extend beyond the interpretable values between 0 and 1, leading to spurious results (e.g., Agresti, 2002; Jaeger, 2008). HGLM accounts for data that are not normally distributed by using a nonlinear link function (Nelder & Wedderburn, 1972).

The second reason we chose HGLM is that non-parametric approaches such as chi-square (e.g., Sauter, Eisner, Ekman, et al., 2010) and exact binomial t-tests (e.g., Cordaro et al., 2016) do not account for the nested, non-independent nature of the data in a repeated measures choice-from-array task. Trials are non-independent because the probability of a correct response on one trial could influence the probability of a correct response on subsequent trials (e.g., due to perceptual similarity of the target vocalizations within a category, which are blocked together in the highly structured version of the task). HGLM is well suited for these dependencies because trials can be grouped into clusters, allowing for the error term to be partitioned and increasing the power of the model to detect the effect (Guo & Zhao, 2000; Kenny, Korchmaros, & Bolger, 2003).

We analyzed the data using a Bernoulli multilevel model, which estimated a log-odds (i.e., the probability of performing above chance of .5) using a log linear link function. Data were structured in a two-level model. Trials on which participants selected either the target or foil vocalization for six emotion categories (level-1) were nested within individuals (level-2). We used an intercept-as-outcome approach with dummy codes for each emotion category (Raudenbush et al., 2004). We have used the intercepts-as-outcomes approach to analyze other repeated measures data (e.g., Anderson, Siegel, & Barrett, 2011; L. F. Barrett & Niedenthal, 2004). We used a random effects model to compute the population-average estimates with robust standard errors, allowing us to generalize the average probability of success beyond those individuals included in the sample. All HGLM analyses were conducted in HLM7 (SSI Inc., Lincolnwood, IL). See Supplemental Material for model specifications.

The results are reported in Figure 3, panel A (see also Table S4 for detailed results). The analysis indicated that participants selected target vocalizations for gigil, itoshii, and liget at levels significantly above chance (p’s range from .005 to .030). Participants selected targets for lajja and glückschmerz at levels approaching conventional levels of statistical significance (m = .58, p = .089 and m = .57, p = .093, respectively); notably, performance means were identical to liget (m = .58). The glückschmerz category obtained a bimodal distribution of responses: while 12 participants performed below chance, 17 participants performed above chance, including 10 who performed at or near ceiling. The lajja category was underpowered because many participants failed the manipulation check after describing the feeling as unpleasant, whereas it is experienced as pleasant in the original Oriya Hindu culture (Menon & Shweder, 1994). Of the 12 participants who passed the manipulation check for lajja, however, eight selected target vocalizations at levels above chance. (See Figure S14 for distribution of performance above and below chance per novel emotion category.)

Fig. 3. — Results from Study 1 (panel A) compared against Sauter et al. (2010; panel B). *Abbreviations*: (panel A) gig, gigil; gre, greng jai; glü, glückschmerz; ito, itoshii; laj, lajja; lig, liget; (panel B) ach, achievement; amu, amusement; ang, anger; dis, disgust; fea, fear; ple, sensual pleasure; rel, relief; sad, sadness; sur, surprise. Results data from Sauter et al. (2010) were originally presented in terms of mean number of correct responses (out of four trials per emotion category). These data have been re-plotted in descending order of proportion correct in order to facilitate direct comparison with the present study. Dashed line indicates chance-level performance (.5). Sample size per emotion category reported in white font at the bottom of each column. Standard error bars (± 1 SE) are provided as distributional information only: the location of error bars above the chance line is not indicative of significant above-chance performance because these data were binomially rather than normally distributed. Effect sizes for Study 1 are reported above each column using the odds ratio (OR), which expresses group difference in probabilities when the outcome is dichotomous and the data are analyzed by logistic regression (Fleiss & Berlin, 2009). The OR is obtained by transforming the b weights using e^b (Feingold, 2013). In line with our strong a priori hypotheses, all tests were conducted using one-tailed probability thresholds to avoid Type II errors.

Comparison to Sauter et al. (2010).

We used the highly structured choice-from-array task (i.e., including emotion words, repeated and blocked trials, and elaborate manipulation checks) from Sauter et al. (2010) to assess the ability of Hadza participants to match stipulated vocalizations to descriptions of novel emotion categories. As can be seen in panel B of Figure 4, the overall pattern of results resembled the pattern of findings in Sauter et al. (2010) for anger, sadness, fear, and other emotion categories that have been claimed as universal. In Sauter et al. (2010), individuals from the Himba culture in northwestern Namibia were asked to complete the same choice-from-array task, and chose the expected vocalizations at rates above chance, providing apparent support for the universality hypothesis. These results stand in contrast to those obtained when Himba participants freely labeled the vocalizations, which revealed little evidence that posed vocalizations were perceived universally as anger, fear, etc. (Gendron et al., 2014a).

Studies 2 and 3: Urban Industrialized Samples – China and U.S.A.

In Study 1, we predicted and found support for the hypothesis that the classic choice-from-array task provides an experimental context that helps participants choose the target stimuli for emotion categories. A remote sample of Hadza hunter-gatherers selected stipulated (i.e., made-up) target vocalizations for novel emotion categories at levels significantly above chance, appearing to provide evidence of cross-cultural emotion perception despite verified lack of exposure. In Studies 2 and 3, we replicated the study in samples of Chinese and U.S. participants from urban industrialized cultural contexts.