Context reconsidered: Complex signal ensembles, relational meaning and population thinking in psychological science

Abstract

Every experiment contains a multitude of factors that might influence the phenomenon of interest. This paper considers the status and study of “context” in psychological science through the lens of research on emotional expressions. We begin by updating three well-trod methodological debates on the role of context in emotional expressions to reconsider several fundamental assumptions lurking within the dominant methodological tradition: namely, that certain expressive movements have biologically prepared, inherent emotional meanings because they issue from singular, universal processes which are independent of but can interact with contextual influences. The second part of this paper considers the scientific opportunities that await if we set aside this traditional understanding of “context” as a moderator of signals with inherent psychological meaning and instead consider the possibility that psychological events emerge in ecosystems of signal ensembles, such that the psychological meaning of any individual signal is entirely relational. Such a fundamental shift has radical implications not only for the science of emotion, but for psychological science more generally. It offers opportunities to improve the validity and trustworthiness of psychological science beyond what can be achieved with improvements to methodological rigor alone.

Take a look at the woman in Figure 1, screaming in terror. Her eyebrows are furrowed, her eyes are pinched tight, and her mouth is agape. She could be in danger or witnessing a horrific scene. She’s obviously terrified out of her wits.

Figure 1:

A woman’s face (Photo credit: Barton Silverman/The New York Times/Redux.)

Except… she isn’t. This is actually a triumphant Serena Williams after she beat her sister, Venus, in the 2008 U.S. Open tennis finals (see Appendix for the full photograph). When viewed in context, Ms. Williams’s configuration of facial muscles instantly takes on a different emotional meaning.

I first published this example in 2011 to demonstrate the power of context to subtly transform the emotion you experience in another person’s face (Barrett et al., 2011; for similar examples, see Barrett, Lindquist, et al., 2007; Aviezer et al., 2012). Beneath this simple phenomenon lies a microcosm of century-long debates about the nature of emotions — the events we refer to with words like “anger,” “sadness,” “compassion,” and “awe.” The first half of this paper carefully reviews and updates three well-trod methodological debates on the role of context in emotional expressions to unearth several problematic assumptions, or ontological commitments, about emotions as psychological events caused by isolated, universal processes that are independent of but moderated by context. The second half of the paper weaves the existing evidence, plus ideas and evidence from other scientific disciplines, into an alternative approach with different ontological commitments: that psychological events emerge in ecosystems of signal ensembles; and that psychological meaning of any individual signal is determined by the other signals in the ensemble (i.e., the meaning is entirely relational). To be clear, I’m not suggesting that emotions and other psychological phenomena are illusions that don’t exist in physical reality. They are indeed physically real, but real in a different sense than is traditionally understood (e.g., Barrett, 2012).

These entwined efforts —reconsidering hidden ontological commitments in our methods, and then crafting a different set of commitments within a multidisciplinary milieu — have been the focus of my work with mentees and collaborators for the past thirty years. Similar critiques of psychology’s treatment of “context,” particularly within laboratory experiments, have been raised now and then throughout the history of our field, giving rise to important but siloed lines of research. Yet somehow the dominant scientific paradigm in psychology — one that considers context a mere moderator of universal processes that can be observed in a lawful way by manipulating one or two isolated variables — prevails. This traditional view of “context” remains resilient, in part, because prior critiques have rallied efforts to improve the methodological rigor of those experiments, rather than to question their suitability for studying psychological phenomena in the first place. Considerable scientific opportunities still await, it turns out, particularly for improving the validity and trustworthiness of psychological science, if we take more seriously the idea of a fully relational science of psychology and reconsider our shared empirical practices accordingly. The discussion in the second half of the paper is more aspirational than prescriptive, a sort of conceptual “call to arms” by design. It sketches an emerging scientific landscape that offers exciting opportunities for scientific discovery, particularly for the next generation of psychological scientists with intrepid spirits and open minds.¹

The Importance of Context

A single configuration of facial muscles in motion, such as a smile or a scowl, can take on different emotional meanings depending on the context. This observation is not controversial. The power of context to shape one person’s experience of another person’s face has been consistently noted for thousands of years (see Supplementary Online Materials, Box 1). In the early 20^th century, formal psychological demonstrations of context effects began appearing in scientific journals (e.g., Sherman, 1927) and have continued ever since (for reviews, see these papers and references therein: (Barrett et al., 2011; Barrett et al., 2019, Supplementary Online Materials Box 3; Gendron et al., 2013; Gendron & Barrett, 2017). A common interpretation of such findings is that context situationally tunes biologically prepared responses. Certain configurations of facial muscle movements are assumed to have inherent emotional meaning — they are presumed to display a person’s internal emotional state with specificity and consistency across situations, people, and cultural contexts. Scowls are thought to display anger, frowns are thought to display sadness, raised eyebrows with a slight smile are thought to display interest, and so on. I’ll refer to this view as a typological view of emotion because it assumes that emotions such as fear, awe, anger, and disgust are a taxonomy of biologically-prepared states, each with its own pattern of diagnostic signals (e.g., Cowen & Keltner, 2021) that can be adjusted by independent contextual influences. In this view, all things being equal, a person’s facial movements are assumed to be a pretty good guide to the emotional state they are in. A typological view of emotion, therefore, does not deny the influence of contextual factors. But the surrounding context is assumed to be independent of the processes that cause emotions (e.g., such cultural dialects, display rules, emotion regulation strategies, differences in induction methods, measurement error, etc.; Elfenbein, 2013; Levenson, 2011; Matsumoto, 1990; Roseman, 2001; Tracy & Randles, 2011). Therefore, contextual factors are thought to merely tweak, modify or moderate inherent emotional meanings.

To better understand the role of context in a typological view, consider that each type of emotion is, in actuality, a category of many individual instances. Each instance (e.g., each instance of anger) can be described as a collection of features. Some features are physical, such as expressive facial movements and autonomic nervous system changes; and some are mental, such as affective feelings (e.g., pleasure/displeasure, level of arousal, effort), goals (e.g., to protect against threat, to affiliate), and appraisals which describe how a person experiences their situation (e.g., novelty, goal congruence, etc.). In principle, any reference to “an emotion” is inherently ambiguous, because it is unclear whether the referent is an individual instance or the entire category. In practice, a typological view reduces this ambiguity by assuming that each type of emotion, as a category, has a prototype, i.e., an instance with a pattern of features that best describes all the category’s instances. A category’s prototype might be its most frequently observed instance (i.e., a typical instance) or its most representative instance. Individual instances of the category might vary in their features across situations, people, and cultures, creating a distribution (or family) of physical signals, but the prototype, as a conceptual representation of the entire distribution, must share a family resemblance with them. The prototype’s features must be similar enough to the other category instances in the distribution, and different enough from the prototypes of other categories, to diagnose a new instance reliably and specifically as belonging to its specific emotion category. Indeed, in a typological view of emotion, an emotion prototype is considered to be a reliable suite of coordinated features (e.g., in peripheral physiology, motivation, and behavior) that serves as an evolved adaptation to a specific fitness-relevant challenge (Shariff & Tracy, 2011). Prototype categories have fuzzy boundaries, meaning their instances occasionally share some features with instances of other categories, and this is where context comes in. A wrinkled nose and scrunched up eyes, for example, are assumed by themselves to be an evolutionarily-preserved, prototypical expression of disgust (Shariff & Tracy, 2011), but this configuration might express anger when it occurs attached to a body with balled fists (see Aviezer et al., 2008).

Evidence for a typological view of emotion can be found in hundreds of published studies that tried to remove as many contextual influences as possible, in the best tradition of rigorous experimental design within psychological science. The assessment of these designs as “rigorous” rests on the assumption that isolating the face from other contextual influences allows a precise causal inference between one person’s facial movements and any change in another person’s experience of emotional meanings in those movements (i.e., the behavior of interest). These studies, referred to as “emotion recognition” studies, almost exclusively employed photographs of posed, disembodied faces that are considered devoid of context. Participants were asked to choose from an array of options which emotion they perceived in the posed facial configurations. At greater levels than chance, participants around the world saw anger in scowling configurations, sadness in frowning configurations, fear in wide-eyed gasping configurations, and so on. Controlling for context supposedly allows the inference that any emotional meanings in the facial movements experienced by participants must, logically, emanate from the faces alone. And if emotional expressions are universally recognized, the argument goes, they must also be universally produced; and if they are universally produced, the argument continues, then they must be innate (Ekman, Heider, et al., 1972; Shariff & Tracy, 2011).

But were these faces, disembodied though they were, actually presented to participants without any context? Classic critiques of similar experimental designs have pointed out that they are composed of complex arrays of influences beyond those variables that are the intended focus of the experiment (e.g., Cronbach, 1975; Gergen, 1978; McGuire, 1973). In line with these criticisms, a number of scientists (including my collaborators, mentees, and me) suggested that these recognition experiments actually include potent contextual features, lurking unnoticed by the experimenters and participants, with sufficient power to mold the emotional meaning of the faces. The implication is not that these studies are flawed. They discovered something important — just not what their designers think they discovered: that context may not function just as a moderator of biologically prepared meanings but serve as full-fledged cause of those meanings. If so, then these contextual factors travel along with the experimental methods from culture to culture, like stowaways, encouraging participants around the world to “recognize” certain emotional meanings, and allowing their responses to be misinterpreted as evidence for universal emotional expressions, all the while obscuring the complex nature of causality.

Several forms of context have been the focus of persistent debate over the past century, creating a hornet’s nest of methodological controversies. We’ll carefully re-examine three sources of context that have been widely debated in the literature, extending the discussion with new findings about hidden contextual elements: within choice-from-array methods, in repeating and block trials (allowing for cross-trial learning), and in the stimulus arrays selected for study. The goal in these discussions is to not suggest the need for more experimental rigor in these designs (in the traditional sense of psychology’s so-called replication crisis or similar crises of the past), but to question psychology’s use of a reductionistic experimental tradition in the first place. What hopefully becomes clear is that mere disagreements about methods (i.e., epistemological controversies) reveal deeper ontological disputes about the nature of emotion, the nature of a human mind, and how psychological phenomena are caused. That is, our goal is nothing less than to reconsider what causal influences actually exist in the world and how to best study them with a scientific method.

Methods Issue #1: Context Embedded Within Choice-From-Array

A choice-from-array method presents participants with a stimulus (e.g., a face, a set of eyes, a vocalization, a scenario, a word) and asks them to select the corresponding target from a small array of choices (e.g., a set of words, faces, vocalizations, etc.). In studies of emotion recognition, this method comes in several flavors, all of which strongly constrain the emotional meanings that participants could experience in the facial movements of another person.² In one common design, a participant views a photo of a scowling face and then must choose a word to describe its emotional meaning from a small set of options, such as “anger,” “disgust,” “fear,” “happiness,” “sadness” and “surprise.” This means the participant cannot opt for “confusion,” “groaning at a bad pun,” or “gastric discomfort.” Other versions of choice-from-array have similar constraints, such as when a participant is presented with a scenario or an emotion word and must choose the matching facial configuration from two or three options. Research has shown that these contextual constraints harmonize participants’ responses and increase the consistency of answers across participants, particularly when testing people from non-western cultures (for reviews, see Barrett et al., 2019; Gendron et al., 2018; Russell, 1994). This harmonization is, in fact, one reason that choice-from-array was chosen as the preferred method for studies of emotion recognition (Ekman & Friesen, 1971, p. 125).

Of the hundreds of studies that seemingly provide evidence for universal emotion recognition (and are interpreted as evidence for universal emotional expressions), the vast majority employ choice-from-array methods. For example, in the most recent major meta-analysis of emotion recognition research, 95% of the included studies used choice-from-array and observed that participants from Western parts of the world (e.g., Germany, France, Italy, etc.) chose the expected word or face about 85% of the time on average; the results were slightly lower (72%) in cultures that are less similar to the US (e.g., Japan, Malaysia, Ethiopia, etc.; Elfenbein & Ambady, 2002). These observed consistencies are directly caused by the faces in context, rather than by the facial configurations alone, because choice-from-array shapes how participants experience the faces.

Research suggests that the emotion words shown in these experimental trials are potent enough to actively shape the emotional meaning that the participants experience in another person’s facial movements (e.g., Doyle & Lindquist, 2018; Gendron et al., 2012; Lindquist & Gendron, 2013; Satpute & Lindquist, 2021). When the contextual features embedded in choice-from-array are removed, giving participants more freedom in their responses, an explosion of variation is observed, in participants from the US and other large, urban cultural settings (Barrett et al., 2019; Kollareth et al., 2021; Russell, 1994; Russell et al., 2003) and references therein) and in people from small-scale cultures from around the world. Since 2008, eleven published papers studying participants small-scale cultures, and using a variety of less-constraining methods, have documented considerable variation in the psychological meanings experienced in the facial poses that have been hypothesized to be prototypic expressions of emotion (Gendron et al., 2018; Gendron, Hoemann, et al., 2020) and references therein).

Some amount of variation is consistent with a typological view, as noted earlier, because types of emotion are thought to be expressed by a family of related physical signals, each type having its own prototype. Some natural variation is therefore expected in the configurations that will be recognized as expressions of that type (e.g., Ekman, 1992). In other words, an expression of anger can look somewhat different on different occasions and still remain an expression of the underlying type for the proposed families of related signals (e.g., see Table 1 in Barrett et al., 2019). So, what’s the big deal? The amount of variation is vast. For example, on viewing a scowling configuration, participants might infer that the poser is expressing sadness, concentration, hunger, or a desire to avoid a social interaction. When participants are offered the opportunity to apply non-emotional labels to facial configurations, they routinely choose to do so, sometimes at higher rates than the emotional options (e.g., Crivelli, Jarillo, et al., 2016; Crivelli, Russell, et al., 2016; Crivelli & Fridlund, 2018; Rychlowska et al., 2015). So, the issue boils down to magnitude. How many psychological meanings can a single facial configuration support and still be interpreted as support for a typological hypothesis? Likewise, how many different configurations can be experienced as expressing a single type of emotion and still be considered evidence for an expressive prototype with inherent biological meaning? How far can a typological view be stretched before it breaks?

Methods Issue #2: Context Hidden Within Repeated Trials

The context effect in choice-from-array studies, even if we accept it in general terms, cannot explain another source of evidence that supports the typological view: some participants from small-scale, remote cultures match emotion words (with or without scenarios) and posed facial configurations (or vocalizations) at above chance levels when choosing from a small array of options (Ekman et al., 1969; Ekman & Friesen, 1971; Sauter et al., 2010; Tracy & Robins, 2008). Restricting participants’ choices would harmonize their responses only if participants were already familiar with the facial configurations and their emotional meanings in the first place; without such familiarity, participants would have responded randomly. Such familiarity could only be learned, the logic goes, by persistent contact with US cultural practices and norms, which these participants presumably had little of. But recent research suggests that certain contextual features in classic choice-from-array designs have the power to subtly teach participants the answers that the experimenters expected, creating observations that have been interpreted as evidence for universal emotion recognition (and by implication, biologically prepared prototypes of emotional expression).

Humans are powerful statistical learners who can absorb complex, dynamically changing patterns of information in a short time, particularly when words are available to help. Words are invitations to learn categories (Waxman & Gelman, 2010), even for very young infants (Vouloumanos & Waxman, 2014). Such “supervised category learning” may, in fact, be an important source of emotional development (Hoemann, Wu, et al., 2020; Hoemann, Xu, et al., 2019) and support the transmission of cultural knowledge across generations, called cultural inheritance, more generally (Gelman & Roberts, 2017; Gendron, Mesquita, et al., 2020). For example, young children quickly learn the emotional meaning of facial movements that they have probably never encountered before; children learned new categorizations for facial movements, based on associations with contextual features (including words), after only 12 minutes of experience (Woodard, Plate, & Pollak, 2021); also see Plate et al., 2019, 2022; Woodard, Plate, Morningstar, et al., 2021). Relatedly, US children performing a choice-from-array task learned to label an artificially constructed facial expression (e.g., a blowfish configuration) with the word “pax” at levels comparable to those for the expressive configurations that are thought to be universal (Nelson & Russell, 2016). Statistical learning may contribute to a process-of-elimination strategy: since the same words and photos are recycled across trials, words that are not chosen on prior trials are selected more frequently, inflating agreement levels (DiGirolamo & Russell, 2017).

A closer look at the aforementioned studies of small-scale cultures (e.g., Sauter et al., 2010, 2015) reveals other contextual elements that enhanced to participants’ ability to learn novel emotion categories across trials (as discussed in Gendron et al., 2015; Hoemann, Crittenden, et al., 2019). Experimental trials were blocked by emotion category, not randomized, which encouraged pattern learning and other strategies for completing the experiment. At the beginning of each block, participants completed an elaborate manipulation check that had the potential to teach novel emotion concepts (e.g., they listened to a brief emotional scenario, sometimes multiple times, and were required to explain their emotional understanding of the scenario; they were allowed to proceed to the experimental trials containing that scenario only when they explained the scenario “correctly,” i.e., as expected by the Western experimenters). During each trial of a block, participants were presented with a scenario, a foil that varied from trial to trial, and a target stimulus that did not vary much, providing an opportunity to learn the intended emotional meaning of those stimuli, even if they did not know those meanings at the outset (Gendron et al. 2015).

This “cross-trial learning effect” was confirmed in a study with US, Chinese and Hadza (hunter-gatherer) participants, who learned to match completely invented expressive signals (in this case, vocalizations) to non-universal emotion categories from around the globe (Hoemann et al., 2019). Participants in all three samples, who did not have a pre-existing category or word for any of the emotion categories being studied, nonetheless selected the contrived, target vocalizations at levels significantly above chance — at levels similar to those of prior studies in small-scale, remote cultures (as reported in Sauter et al., 2010) — strongly suggesting that the task constraints guided participants to respond in ways that made it appear as if the (contrived) expressions for non-universal emotion categories were universally recognized. Further evidence for the importance of cross-trial learning can be found in people who cannot learn new emotion concepts nor reassemble prior emotion knowledge, as needed, to complete the experimental tasks (Calabria et al., 2009; Lindquist et al., 2014; Roberson, Davidoff, & Braisby,1999), in people who are experimentally prevented from reassembling that knowledge (e.g., Gendron et al., 2012; Lindquist et al., 2006; Roberson & Davidoff, 2000), and in young infants who do not have that knowledge to begin with (e.g., Caron et al., 1985) (see SOM Box 2). Findings like these suggests an alternative hypothesis to the typological view of emotion: people might not be detecting emotional meanings that are biologically embedded in facial movements; they might be constructing those meanings in relation to contextual factors.

Methods Issue #3: Context Hidden in the Sampling of Facial Configurations

The third contextual influence that we’ll consider is the role of stipulation, rather than discovery, in emotion recognition experiments. Most studies of emotion recognition show participants a single facial configuration to represent the hypothesized prototypic expression for each emotion category, posed by several actors. The exact configuration tested varies slightly from study to study (see Table 1 in Barrett et al., 2019; see Supplementary Table 1 in Le Mau et al., 2021), and by design, each pose is usually exaggerated (Ekman, Friesen & Ellsworth, 1972), i.e., they are caricatures of everyday movements (Goldstone et al., 2003). But these hypothesized prototypic expressions were not discovered by observing people as they express emotions in situ — they were chosen by scientists who were inspired by Charles Darwin (Darwin, 1965), who, by proclamation rather than observation, stipulated various configurations of facial muscle movements as expressions of emotions. Darwin’s stipulations were based on drawings by Charles Bell (Bell, 1806) and photographs by Guillaume-Benjamin-Amand Duchenne (Duchenne, 1990).³

An obvious avenue for improvement in evaluating a typological view of emotion would be to use a variety of facial configurations culled from everyday life, rather than a single, posed configuration for each emotion type, and examine the emotional meanings that participants experience in those faces. A recently published series of studies has done just that (summarized in Cowen & Keltner, 2021). Each study sampled thousands (and in certain cases, millions) of stimuli that were more naturalistic than the posed, disembodied caricatures of prior studies. Participants rated the emotional meanings they experienced in thousands of physical signals: in faces within the context of video clips (Cowen et al., 2021), in photographed faces that included body postures and surrounding context (Cowen & Keltner, 2020), in posed non-verbal vocalizations (e.g., laughs, screams, etc.; Cowen, Elfenbein, et al., 2019), and in thousands of speech samples that varied in prosody, spoken by actors from five countries (e.g., rhythm, timbre, etc.; Cowen, Laukka, et al., 2019). The studies also included a few other design elements in an attempt to avoid limitations of previous studies. They used sophisticated machine-learning (ML) methods. They used both choice-from-array involving 31 emotion words and a relaxed choice-from-array so large that it functioned like a less-constrained free-response format, compared the two, and found no statistical difference in participants’ responses.⁴ They also found that situational context had a minimal influence on the emotional meanings that people experienced in face-plus-body stimuli (Cowen, Elfenbein et al., 2019). All studies in the series appear to support the hypothesis of emotion prototypes (although the number of types varies from study to study). In the authors’ words, “Upwards of 25 distinct varieties of emotional experience have distinct profiles of associated antecedents and expressions” (Cowen & Keltner, 2021, p. 124), and “pure expressions of fear, surprise, and awe are bridged by gradients of composite facial–bodily and vocal displays that reliably transmit intermediate meanings. Although there may be modal emotion-related responses, much of human emotional life is more complex” (Cowen & Keltner, 2021, p. 5). Similar observations were reported in a recent study of college students in China, India, Japan, Korea, and the United States who listened to a brief scenario describing an event that might cause each of 22 emotions (e.g., “You have been insulted, and you are very angry about it”) and were instructed to pose the facial expression they believed they would make if the events in the scenario were happening to them (Cordaro et al., 2017). Such findings, when viewed through a typological lens, appear to provide convincing evidence that people express distinct types of emotion with variable physical signals, and recognize the biologically prepared expressive meaning of those signals, in a way that is independent of, but can occasionally be moderated by, situational context.

How do we square these newer findings with the larger landscape of studies (some of which we discussed earlier) in which participants experienced many psychological meanings in smiles, frowns, scowls, and other facial configurations — much more variation, in fact, than can comfortably be accounted for by a prototype structure? One possibility is that these new ML studies simply disconfirm all those other studies and put this debate to rest, finally.

Another possibility is that these ML studies, by virtue of their design and modeling choices, continue the tradition of introducing experimenters’ beliefs and stipulations as contextual constraints that are not present in real life, thereby restricting possible observations in subtle yet pernicious ways. We all know that ML algorithms do not prevent biases from creeping into stimuli, training data, and test data, and the same is true for these newer ML studies (see Barrett, 2021). For our purpose in this paper, however, let’s focus on just one vector for embedding experimenter beliefs: the stimuli sampled for these studies, i.e., the range of physical signals that participants were exposed to.

In these newer ML studies, the face stimuli, though numerous and variable, were curated according to experimenters’ beliefs about the nature of (English) emotion categories they chose to study, rather than being randomly sampled.⁵ There are thousands of anatomically viable facial configurations.⁶ Given this plausible variation, what is the likelihood that a search for facial configurations will capture the breadth and variation of expression that exists in the real world (e.g., laughing in anger, sobbing in happiness, or falling asleep in fear) when that search is supervised by typological beliefs about emotion?⁷

To be fair, the ML papers in question explicitly state that stimuli were not chosen to maximize their “resemblance to category prototypes” (Cowen & Keltner, 2020, p. 353). Yet there is evidence of bias in ML studies of emotional expression (see Domnich & Anbarjafari, 2021; Rhue, 2019). The stimuli used to induce instances of emotion are selected in line with typological beliefs (e.g., Kragel, Reddan, et al., 2019; Wager et al., 2015). And experimental evidence shows that if an experimenter has a prototype in mind, and curates stimuli guided by that belief, participants will be able to infer that prototype, even though they never see the prototype (Posner & Keele, 1968). It is therefore plausible that stimuli used in the aforementioned ML studies failed to sample the full range of variation in the real world simply because the researchers who “selected expressions for apparent authenticity” (Cowen & Keltner, 2020, p. 353) believe strongly in the existence of biologically prepared emotion types.⁸

Luckily, we don’t have to argue hypotheticals. Other published studies of emotional expression demonstrate that small changes in how stimuli are curated have a big impact on the results, no matter how vast the data set and no matter how sophisticated the ML, in part because those studies included stimuli that allowed for the possibility of disconfirming a typological hypothesis. Consider a recent study, for example, in which researchers identified English nouns, verbs, adjectives and adverbs that refer to emotion categories, along with their semantic and lexical relations (Srinivasan & Martinez, 2021). These words were translated into six languages (Spanish, Mandarin Chinese, Farsi, Arabic, and Russian) and used in a variety of internet search engines to identify and download over seven million images of human faces. Each image was electronically coded for its facial muscle configuration using the Facial Action Coding System and code accuracy was manually verified. Only 35 facial configurations, which appeared in only 1.87% of the 7M culled images, were found in images derived from the search in all six languages, and none matched the configurations proposed by Darwin or presumed to be prototypes in prior research. (Another eight were identified as common to images mined in one or more, but not all, of these languages).⁹ When participants freely labeled the 35 configurations for their emotional meaning, substantial variation was observed (more than mere accents on a universal core expression, and more than blends of “prototypic” expressive forms). Once again, when participants were granted more freedom to experience a variety of meanings in facial configurations, the results had more variation than a static view of biologically prepared emotion prototypes can easily handle.

A different sampling strategy — one that was not supervised by specific emotion words or specific emotional situations — yielded even more situated variation in expressive configurations, while also providing yet more evidence of the causal power of context (Le Mau et al., 2020). We used photographs posed by one hundred and eighty well-known, highly experienced actors to portray characters in over 600 realistic scenarios rich with psychological meaning (e.g., “[A woman confronts] her lover, who has rejected her, and his wife as they come out of a restaurant”); each scenario was posed only once (Schatz, H. & Ornstein, B. J., 2006; Schatz, H., Edwards, E. & Ornstein, B. J., 2013). These photos are a goldmine for emotion research; even though the facial configurations were not spontaneous, they were posed by award-winning, world-class actors whose professional reputations and livelihoods depend on conveying emotional meaning in realistic ways. Each evocative scenario was rated for emotional meaning using a choice-from-array method (by design, to tilt the odds in favor of universality), as was the corresponding facial pose, presented either alone or with its scenario for context, and the facial configurations were also coded using FACS.¹⁰ Considerable variation was observed in the facial poses associated with scenarios of similar emotional meaning, replicating the Srinivasan & Martinez (2021) study.¹¹ A ML analysis of the facial configurations, supervised by emotional ratings of their corresponding scenarios, indicated that experienced actors posed a wide variety of context-sensitive facial configurations for each emotion category. The usual prototypes proposed in the literature were neither typical nor representative, although some were posed more often than expected by chance. These findings, which call the existence of stable, static prototypes into question, were statistically robust (as assessed with a multiverse analysis (Steegen et al., 2016) that examined the findings across a range of analytic choices, including exhausting all potential combinations between them).

A recently published meta-analysis of spontaneous facial expressions of emotion (Durán & Fernández-Dols, 2021) conceptually replicates the other studies that call a traditional prototype view of emotions into doubt. The proposed prototypic expressions, in their full configuration, were not observed at greater than chance levels within this meta-analysis (87% of almost 4,000 participants across 69 experiments). Portions of the configurations (i.e., some, but not all the facial movements in each proposed prototype) were observed as expected at greater than chance levels, but even so, they were neither typical nor representative expressions. Instead, most of the time, participants expressed instances of an emotion category with a wide variety of facial movements. For example, experienced actors scowl and study participants partially scowl about 35% of the time in instances of anger, which suggests that scowling is one expression of anger in certain situations but not a prototype, because a majority of the time (65% of the time) people were observed to express anger with other patterns of facial movements that share no family resemblance to scowling. Similar observations held for all the emotion categories included in both studies (and for similar findings in the spontaneous expressions in children, see Castro et al., 2017).¹² Specificity of expression was not assessed in the Duran and Fernandez-Dols (2021) meta-analysis because it was infrequently reported in the original papers, but is low when assessed, such as in the Le Mau et al. 2020 study of experienced actors (the highest specificity coefficient was, in fact, for scowling in anger at 0.52, corresponding to a false positive rate of 0.48!). Together, these findings do more than cast any view of static, stable emotion prototypes into doubt — they suggest that any category of emotion may be expressed with a population of variable, situated facial movements, a hypothesis that was clearly reinforced by our unsupervised ML analyses (Le Mau et al., 2021).¹³

From Epistemology to Ontology

Let’s pause to consider what we’ve learned so far. Numerous studies suggest that people around the world reliably and robustly experience certain facial movements (physical signals) as prototypic expressions of certain emotion categories, provided various contextual factors are in place, either alone or in combination. We discussed three of these factors: the sample of signals observed (methods issue #3), the manner in which they were observed (methods issue #2), and the response options that were permitted (methods issue #1). When these contextual factors are recognized and their constraints are relaxed or removed, scientists instead routinely, reliably and robustly observe evidence of variation of a magnitude that is more akin to populations than prototypes, and emotional meaning that is functional because its inherently situated and relational rather than biologically prepared: people around the world experience the same emotional meaning in a variety of facial configurations (i.e., as expressing instances of the same emotion category) and the same configuration of facial movements as having a variety of psychological meanings, not all of which are emotional in nature. This situated variation in what people perceive is also mirrored by situated variation in the movements that people make to express emotion. When taken together, this set of empirical observations cannot be easily squeezed into typological notions that enumerate a few dozen types of emotion, each supposedly composed of a prototype of physical signals and triggered by a unique collection of neurons, all of which allegedly have an inherent emotional meaning across situations, people and cultures that can be merely tweaked by contextual influences.

In principle, no psychological scientist, not even someone who subscribes to strict typological thinking, would reject the existence of contextual influences. Typological approaches, which treat contextual factors as moderators, and fully relational views in which the emotional meaning of any physical signal, such as facial muscle movements, is fully causally dependent on those factors, define a continuum of theoretical proposals rather than a strict dichotomy (see SOM Figure 1). Nonetheless, it’s time for a more concerted effort to appreciate the deep significance of these observations we’ve just discussed, because these findings call into question certain background assumptions about what emotional phenomena are and how they are caused, which ultimately leads to discussions of how best to study them. I’d like us to seriously entertain the possibility that the informational value of any signal, such as the ability of one person’s facial movements to convey emotional meaning to another person, may be causally dependent and therefore conditional on an entire ensemble of factors, the milieu, in which those movements are made and observed. Not surprisingly, our entry point is a reconsideration of what psychologists call “context.”

“Context” Reconsidered

As scientists, we understand that every experiment contains a multitude of factors that might influence our phenomenon of interest. Participants are tested at particular times of day in places with particular smells, temperatures, sights, and sounds. They arrive at an experiment with their own contextual factors, i.e., a particular set of experiences and beliefs, having had particular amounts of sleep, having ingested particular amounts of food and caffeine, and having breathed air with particular concentrations of carbon dioxide, all of which could influence their metabolic dynamics. Participants interact with experimenters who themselves have particular beliefs, memories, sleep habits, and momentary physical states, tone of voice, facial and body movements, word choice, etc. And participants register their responses using particular actions as prescribed by the experimental setup. It is tempting to assume the contextual factors that do not interest us scientifically are epiphenomenal (at best) or sources of outright error (at worst). In the study of emotion recognition, as we’ve just discussed, such factors are usually considered epiphenomenal to the inherent emotional meanings that are assumed to be broadcast by facial movements, and therefore constitute moderators or even sources of noise in recognizing (i.e., detecting) said emotional meanings. As a consequence, attempts are routinely made to experimentally control these contextual factors by holding them constant across participants, by measuring their influence and statistically removing their variance, by increasing sample sizes in an attempt to drown them out with the signal of interest, or by ignoring them and hoping the (error) variance they contribute will be randomly distributed across observations.

This approach — to control or reduce the impact of contextual factors rather than to model and analyze their causal influence — reflects a belief that a mind is a system of independent, separable mechanisms with precise laws of cause and effect, and each mechanism can be studied individually without affecting the others. This assumption goes by many names in philosophy of science (the nuances in their similarities and differences are a discussion for another day). But they all conform to a “machine metaphor” (Lewontin, 2000) as a set of ontological commitments or meta-narrative about the nature of the world and the manner of scientific inquiry that is required to study such a world. The machine metaphor has its origins in the scientific revolution of Descartes and Newton, is associated with a reductionist approach to scientific inquiry, and remains a dominant philosophy of science in the writings and practice of psychological science. Types of perceptions, cognitions, emotions, decision-making, and so on are thought to be distinct psychological states, caused by distinct processes that are implemented in distinct populations of neurons that function in a law-like manner. It is assumed that, because they are independent of one another, cognitions, emotions, and the like can interact, thereby causing behavior. Any other influences are moderators, or error. Typological views of emotion are examples of this kind of thinking.

Some scientists consider the machine metaphor and its associated epistemology to be one of the greatest accomplishments of psychological science, in the best tradition of the physical sciences. Others consider it an outdated view of physics and other physical sciences (e.g., Lewontin, 2000; Mayr, 2004), even going to far as to refer to it as an ideology, grounded in Western individualism, that biases scientific thinking (e.g., Lewontin, 1991). Throughout my career, with the help of mentees and generous colleagues (some of whom have educated me scholarly fields far from my own), I have increasingly questioned the ontological commitments of typologies in the science of emotion and more generally in psychology and neuroscience (e.g., Barrett, 2009, 2012, 2017; Barrett & Satpute, 2013, 2019; Lindquist & Barrett, 2012; Mesquita et al., 2010; Barrett & Russell, 2015). Other like-minded scientists have raised similar questions in studies of development, motor movement, perception, social relations, psychopathology, concepts, consciousness and many other domains, including basic brain functions that arise from a dynamic relations among multiple, simultaneous, weak causal influences (for discussions and additional references, see for example, Buzsáki, 2019; Cisek, 2019; Eidelson, 1997; Fausto-Sterling, 2020; Gergen, 1978; Heft, 2001; Kirchhoff, 2018; Maturana & Varela, 2012; Mesquita, 2022; Mesquita et al., 2010; Russell, 2003; Smith et al., 2018; Smith & Thelen, 2003; Wright & Woods, 2020; Zelazo, 2013).¹⁴ Again and again, experimental evidence calls the notion of types into doubt. Studies that do not usually cannot, in large part because they have been designed a priori to confirm their existence. All of us follow in the footsteps of philosophers and psychological scientists who repeatedly criticized the notion of typologies, and the methodological designs they engender, pretty much since the dawn of psychological science, stemming back to William James’s notion of the “stimulus situation” (again, see Gergen, 1978; Heft, 2001; Zelazo, 2013 and references therein; historical references related to the science of emotion specifically can be found in Barrett, 2017 and Barrett & Lida, in press).

These critiques have sometimes given rise to new scientific paradigms, such as ecological psychology, cybernetics-inspired dynamical systems approaches, and a variety of constructionist approaches (one of which we will return to later). Each effort is admirable in his own right, but none has sparked a scientific revolution on par with the downfall of typologies in other sciences, despite a century of discussion. This is no criticism of those efforts, but a testament to the difficulty of the task at hand. A conventional typological approach to psychological science, organized into cognitions, emotions, perceptions, motivations, and other Western folk-psychological categories, reflects a particular, culturally-specific theory of mind (Danziger, 1997). Since the professional and publishing institutions of psychological science are filled with people who routinely employ typological views, we cannot, as a field, reconsider these most basic assumptions unless we question the causal understanding of our own lives, akin to questioning our experience of gravity and the solidity of the objects we interact with. This picture suggests that for a very long time, psychological scientists may have misunderstood the nature of the very phenomena that we are attempting to understand. With these ideas in mind, let’s strap on our seatbelts and take the plunge. The goal here is not to offer answers to age-old mysteries, but to stimulate better questions that have, in principle, a better chance of actually being answered scientifically. As in the first section of the paper, we’ll consider the implications through the lens of affective science.

Ensembles of interwoven signals.

Let’s start with the possibility that so-called contextual factors actually play a fully causal role in creating an instance of emotion or any mental event —even factors that are typically assigned to the so-called background. Here, contextual factors do not merely tweak the hypothetical, biologically prepared, psychological meanings of physical signals. They form a larger, complex web of causality, a dynamic ensemble of interactions that give rise to mental events and actions, and from which psychological meaning emerges, but whose behavior is not predictable from its components in isolation. That is, a complex system consists of many elements that interact, often non-linearly, and produce outputs probabilistically.

A full exposition of complexity theory and complex adaptive systems is beyond the scope of this paper, but discussions of complexity theory can be found in virtually every field within the natural sciences. In biology and related fields, for example, study after study has shown that a living organism is not an assemblage of separable mechanisms that can be studied bit by bit. Rather, contextual factors that may be weak on their own interact and coordinate in non-linear ways to powerfully create phenomena that cannot be reduced to any weak factor in isolation. And importantly, it’s not possible to manipulate one factor separately and leave the others unaffected. Therefore, modeling and analysis is more important than isolation and manipulation because reductionism is impossible in reality. Also, signals in nature are much more uncertain than is assumed in a typological approach, so we must acknowledge the uncertainty and try to discover and model its structure.

How might our understanding of emotion, or more broadly, psychological events of any kind, change if our experimental inquiries assumed that every behavior, every experience — every mental event — arises from such a web of complex causation? For a start, complexity theory reinforces our earlier observation that surrounding signals (i.e., contextual factors) might be considered part of the web of causal forces that are at play in any laboratory experiment. A growing number of studies give evidence of the profound and surprising ways in which so-called contextual factors influence psychological events, even when those factors are studied in isolation (e.g., time of day, e.g., Hahn et al., 2012; odor, e.g., Leleu et al., 2015; Sorge et al., 2014; CO2 concentration, e.g., Scully et al., 2019 ; sleep quality, e.g., Prather et al., 2013; heart rate, breathing and other conditions of the body, e.g., Al et al., 2020; Galvez-Pol et al., 2020; Grund et al., 2022; Kluger et al., 2021). It is well established that contextual factors influence even the most basic aspects of movement perception and object perception (e.g., Bar, 2004; Brandman & Peelen, 2017; Castelhano & Pereira, 2017). Even something as basic as whether a participant saccades to a stimulus vs. passively views a stimulus influences brain responses in primary visual cortex (e.g., MacEvoy et al., 2008; Zirnsak & Moore, 2014). Such findings imply that, in psychology experiments, these and other potentially impactful influences go unmeasured most of the time, producing variation that masquerades as error. As a consequence, the modest effect sizes in psychological research may, in part, be a consequence of mistakenly trying to model complex phenomena as a simple, linear mechanistic systems (Barrett, 2020a, 2020b). Notions of complex causation suggest we adopt experimental designs that seek to discover and model the extent of variation, with the potential to forge a more robust and replicable psychological science.

Also consider that many of the “contextual” signals that are most relevant to any mental event are found in a person’s brain. A brain does not detect features in the world and body; a brain assembles features as ensembles of interwoven physical signals to create meaning. Some features are closer in detail to the raw sense data coming from the sensory surfaces of the body and that guide organ function, metabolism, immune function, and muscle fiber contractions (a.k.a. higher-dimensional features). Some are more abstract (lower-dimensional), referred to as mental features, such as “goal,” “value,” “threat,” “reward,” “valence,” “arousal,” “novelty,” and so on. Mental features are compressed, multi-modal summaries of the higher-dimensional sensory and motor signals (as discussed in (Barrett, 2017b; Katsumi, Kamona, et al., 2021; Katsumi, Theriault, et al., 2021). A word for a psychological category, such as anger, is an efficient way to communicate a pattern of features without listing them. An important hypothesis here is that sensory signals from the world are not the only, nor even the primary, source of these signals; they are often generated intrinsically within the brain as it continually converses with the body, reassembling past events that are similar to the present in some way (i.e., the brain constructs features of equivalence; for discussion, see Barrett, 2017). These features of equivalence are the means by which a brain generalizes from the past to regulate the internal systems of the body, guide action, and create experience, thereby giving psychological meaning to higher-dimensional sensory and motor signals. The result is what the neuroscientist Gerald Edelman referred to as “the remembered present” (Edelman, 1989).

This hypothesis implies that measuring only facial muscle movements, changes in autonomic nervous system activity, or even the brainstem circuitry that supports running, freezing, and other skeletomotor actions will never reveal the nature of emotion. Nor will measuring the prefrontal and parietal areas in the cerebral cortex reveal the nature of attention (e.g., Gonzales-Castello et al., 2012). This is because these signals are only a small subset of a larger ensemble that together give rise to any psychological meaning. The nature of any mental event must be understood as part of an ensemble of interwoven signals that includes those involved with meaning-making (Barrett, 2009, 2012). (This hypothesis is consistent with appraisal views of emotion, in which appraisals are mental features that describe a person’s experience of themselves in a particular situation; Barrett, Mesquita, et al., 2007; Clore & Ortony, 2013). Accordingly, a rigorous epistemological strategy will require that scientists cast their nets much more widely than is currently the case if we hope to ever hope to understand the physical basis of any psychological phenomenon.

If psychological meaning arises within a complex system of many weak, nonlinear, interacting causes, some of which are found in the brain of a perceiver, then perhaps when you experienced terror or elation in Serena Williams’s furrowed eyebrows, pinched eyes, and gaping mouth, you were not recognizing an emotional display. Instead, your experience emerged as a system-level event, the product of signals that derive their psychological meaning within a complex, dynamic system of other signals, including the signals in your brain and the context you carry around with you (e.g., the state of your own body). There is also a broader spatio-temporal context (e.g., where are you located? what time of day is it? what led up to the present moment? what might happen in a moment from now?), all embedded in a broader cultural context. In the real world, one rarely encounters a disembodied face like the floating Wizard-of-Oz in the Emerald City. Real faces appear in rich, dynamic, multi-sensory, temporally extended contexts. They usually appear on heads that utter words with vocal prosody, are attached to bodies that move in certain ways and carry certain smells and are associated with other surrounding details. Such dynamic spatio-temporal arrays are the natural ecology in which we experience everything, including the emotions that we see (and hear and feel) in others. Facial movements may be made psychologically meaningful in those signal arrays within your brain, your body, and the external context. They may interact with one another in probabilistic ways to create a mental event, such as whether you experience the rise of an eyebrow or the curl of a lip as an expression of someone’s emotional state or as noise to be ignored. Together, these causal factors may create a constantly changing web of influence that adapts to the present situation using what is learned from vast arrays of past experience.

It remains to be seen whether complexity theory is a worthy ontological guide for the science of emotion and psychological science more broadly. In the practice of science, the devil is in the details, and there are a lot of details to consider. Adopting these commitments suggests that we should design our experiments to consider a range of weak, interacting influences, even those factors that are not of central interest, in an effort to gain explanatory power. If your head is shaking as you read this and consider the pragmatic nightmare that I am suggesting, then it is mirroring my own as I write this. I don’t have to tell you that designing and implementing such studies will be expensive, time consuming and frankly frustrating (i.e., it’ll be a bitch). To study ensembles of interwoven signals, including those that are within the brain of a perceiver, requires a scientific approach that investigates any mental event as something that emerges from a system of signals as a whole, in interaction with its environment. We’ll need to specify which elements are part of the system and which are inputs and outputs (e.g., are physical signals from the body part of the complex system that gives rise to mental events, or are they inputs to the system? Are the physical signals coming from an interaction partner better modeled as part of the complex system, or should they be modeled as inputs to a system that is bounded by the skin of a single person?). We’ll have to upend certain policies and practices that guide the professional aspects of our science, train our students differently, and secure resources that might not yet be available. But complexity theory is still worth our curiosity as a viable basis for reconceptualizing what a mind is and how it works. It lets us explore whether the phenomena that culturally constitute a particular version of the human mind — with cognitions, emotions, and the like — with various cognitions, emotions (both their generation and their perception), and the like — emerge from more basic, domain-general, and universal causal processes that are shaped by evolution and development. If so, then these processes could also account for individual differences in the granularity of categories that constitute a human mind (e.g., Barrett, 2017a; Hoemann et al., 2021, 2022; Kashdan et al., 2015; Wilson-Mendenhall & Dunne, 2021) and for the different mental categories that have been observed in other cultures. On the other hand, if we continue to believe, as a field, that isolating and manipulating a couple of variables and their interactions produces reliable and generalizable scientific results, then we are fooling ourselves.

Relational meaning.

Thus far, we’ve considered the possibility that psychological phenomena emerge as complex ensembles of interwoven signals, some that reside in the surrounding situation and in your body, and some that are constructed in your brain (to model the world and your body, including abstract features that we refer to as mental features). An unsettling yet far-reaching implication of this hypothesis is that the psychological meaning of any physical signal exists only in relation to the rest of the ensemble, a web of variation in other physical signals. In other words, physical signals may have no inherent psychological meaning. The mental features that give a physical signal its psychological meaning do not reside in the signal itself. They are constructed, as relative information, in another set of signals — the signals in a human brain that can remember (i.e., reassemble) features from the past. In a relational view of meaning, any physical signal of interest in the science of psychology — heart rate variability, skin conductance, glucose metabolism, cytokine concentrations, dopamine, serotonin, and norepinephrine release and uptake — has no biologically-prepared, perceiver-independent psychological meaning. Even the electrical activity in a population of neurons has no inherent psychological function. The meaning of any firing neuron is always in relation to other physical signals (e.g., McIntosh, 2004), most especially the other neurons receiving that neuron’s action potential (e.g., when a single pattern of action potentials from a single neuron is received by a motor neuron, it is considered a motor signal; when the same action potentials are received by sensory neurons, they are considered sensory signals).

By implication, facial movements like smiles, frowns, scowls, and other physical signals may not have evolved, inherent meaning as emotions. A scowl becomes meaningful as an expression of anger only in particular contexts that include the brain of a person with particular past experiences. A scowl can also meaningfully signify heartburn, or the enjoyment of a pun, depending on how the movement is categorized (i.e., which memories are reassembled to generalize and construct the features to give the signal meaning). Serena Williams’s furrowed eyebrows, pinched eyes, and gaping mouth, as physical signals, took on meaning only in relation to an ensemble of other physical signals, some of which emerged only in your brain as part of this meaning-making. Goals, value, affect, and other mental features are not properties that exist in the world or the body. They exist only in the brain that creates these relational ensembles.

Correspondingly, Williams’s brain, upon completing her victory over her sister Venus in the US Open (or even milliseconds before), assembled an action plan (from a population of possible plans) as it constructed an instance of elation, and in so doing made certain physical movements meaningful as an instance of this emotion category in relation to an ensemble of other signals, including those that construct mental features to create meaning, guide action, and create experience. In our lab, we hypothesize that the psychological meaning of her movements is part of this relational meaning-making process, not determined by it. Through this lens, emotional communication, or communication of any sort, is not detection of biologically-prepared, inherent meanings, but is a synchrony of relational meanings across brains (e.g., Gendron & Barrett, 2018; Nguyen et al., 2021; Nozawa et al., 2019).

Relational meaning —the hypothesis that physical signals only have meaning in relation to other physical signals — is unintuitive and perhaps discomforting in its implications, but it is not “extreme relativism.” It is a hypothesis that psychological meaning arises from a complex web of interdependent signals that involves other people. It is a realism that is consistent with our evolved roles as social animals and our ability to collectively create social reality (Barrett, 2012, 2017a, b, 2020). This realism differs from the usual dichotomy drawn between idealism (reality exists in your head) and materialism (reality exists in the world). It suggests that to properly understand a psychological phenomenon, such as an expression of emotion, scientists must measure more than just facial muscle movements or vocalizations. We must also measure the signals that give those physical signals their psychological meaning. That is, the “relative information” shared between signals (Shannon & Weaver, 1949/1964) must be observed and modeled (e.g., for examples of studies of emotion recognition that use information theory, see Jack & Schyns, 2017).

Population thinking.

If instances of emotion and other mental events emerge in a complex web of causal factors full of relational meaning, and we design our experiments accordingly, then we have a combinatorial explosion of possible patterns. In the words of the evolutionary biologist Richard Lewontin, “Organisms are … extremely internally heterogeneous. Their states and motions are consequences of many intersecting causal pathways, and it is unusual that normal variation in any one of these pathways has a strong effect on the outcome.… Indeed, we may define ‘normality’ as the condition in which no single causal pathway controls the organism.… All attempts to understand causes must necessarily involve the observation of variations” (Lewontin, 2000, pp. 93-94). Simply put, variation is the norm. The same physical signals that implement abstract mental features can be associated with a variety of physical signals that implement conditions of the body, muscle movements, and the outside world. Conversely, the same signals that relate to the conditions of the body and world can be associated with different mental features in different situations. And so on. The possible variety is not limitless, but it is considerably greater than what is hypothesized by typological views of the mind. Attempts to understand the causes of emotion, or any psychological phenomenon, will be hampered if non-rigorous sampling of participants, stimuli, and measurements limits this variation, as we observed earlier in ML studies of emotional expression.

The variation that is typically assigned the role of error or moderator in any given study may be structured in reliable, predictable ways, as demonstrated by the Le Mau et al. (2021) study of actors’ portrayals. In another recent study that was designed a priori to capture structured variation (Hoemann, Khan, et al., 2020), we predicted and observed (via unsupervised ML) that patterns of physiological activity (a.k.a. physiological motifs) for an emotion category varied in context-dependent ways within an individual participant. Instances of a given emotion category, such as anger or happiness, showed a considerable amount of variation in a given participant’s physical and mental features. Furthermore, the study suggests that reliable patterns of physiological signals have no inherent emotional meaning; a given motif was made meaningful as a variety of emotions, again, within an individual participant.

Historically, multiple physiological motifs have been observed for a given emotion category across studies, and there is substantial overlap of motifs across different emotion categories (for recent a meta-analysis of this literature, see Siegel et al., 2018). That is, scientists have been unable to identify a physiological prototype for any emotion category that is sufficiently specific and reliable across studies, despite decades of search for emotion-specific physiological motifs. The Le Mau et al. and Hoemann et al. studies and others like them suggest an interpretive frame for this tangle of observations: a given emotion category may have there may be multiple, situated patterns; in fact, a given category could have multiple motifs in the same situation (a.k.a. degeneracy; e.g., Edelman & Gally, 2001).

A similar situation exists with brain imaging studies of emotion. In the last decade, numerous studies employing supervised machine learning approaches have searched for the prototypic brain state corresponding to a specific emotion category, such as fear, which is subsequently interpreted as the brain biomarker for that category (e.g., Zhou et al., 2021). The identified patterns, however, differ across published studies (e.g., Horikawa et al., 2020; Kassam et al., 2013; Kragel & Labar, 2015; Saarimaki et al., 2016; Wager et al., 2015; Wilson-Mendenhall et al., 2015) and might instead indicate the presence of meaningful, structured variation. This is supported by brain imaging studies that have documented variation in the neural correlates of different instances of emotion within the same emotion category (e.g., Lebois et al., 2020; Wilson-Mendenhall et al., 2011, 2015). We have also applied various unsupervised models to discover this structured variation in brain imaging data during emotional experience when no a priori labels are applied to the data (Azari et al., 2020). Even participants watching the same video clip have substantial variation in their brain activity and state dynamics (Singh et al., 2021). Our findings again suggest that emotion category labels refer to populations of variable, context-specific instances that would be better sampled and modeled using methods suitable for estimating structured variation (also see Boiger et al., 2018). Such an approach would allow scientists to discover, rather than presume, conditions when such variation might generalize across situations, people, and cultures.

Fortunately, Charles Darwin (1858/2001) gave us a conceptual tool for thinking about this magnitude of structured variation. It’s called population thinking, named by the evolutionary biologist Ernst Mayr (Mayr, 2004). Population thinking, as articulated in On the Origin of Species, refers to the idea that a biological category, such as a species, is a conceptual category of individuals with variable physical features, and whose fitness is inherently relative to the conditions of the immediate environment. William James adapted Darwin’s observation to the nature of emotion: “The varieties of emotion are innumerable… The trouble with the emotions in psychology is that they are regarded too much as … eternal and sacred psychic entities, like the old immutable [pre-Darwinian] species in natural history … all that can be done is with them is reverently to catalogue their separate characters, points, and effects.”. And James continued, “But if we regard them … as ‘species’ are now regarded as products of heredity and variation, the mere distinguishing and cataloguing becomes of subsidiary importance.” (James, 1998) p. 449). Since then, population thinking has been periodically revisited in psychological science a number of times (e.g., Estes, 1956; Gallistel, 2012, 2013). My lab’s hypothesis — that any biological category, and correspondingly, any psychological category, including emotion categories, is a population of situation-dependent instances with variable features — is similarly inspired by this Darwinian idea (as discussed in Barrett, 2013; Barrett, 2017a, b; Clark-Polner et al., 2016; Siegel etal., 2018).

In population thinking, variation among a category’s instances is assumed to be real in nature, structured, and meaningfully related to the situations in which those instances emerge. Any abstract summary of a category, such as its mean or a prototype, is a fiction. (By analogy, the average US household size in 2020 was 2.53 people, but no real family contains 2.53 individuals.) In ML analyses that search for a single pattern to summarize multiple participants, any such pattern is an abstraction that need not exist in any participant’s data; i.e., in any given brain imaging study, the so-called biomarkers for emotions are abstractions (per population thinking), not actual brain states (per typological thinking; for a mathematical simulation, see Clark-Polner et al., 2017). When viewed through the lens of population thinking, the proposed expressive prototypes for each emotion category — smiling in happiness, scowling in anger, frowning in sadness, and so on — are stereotypes (Barrett et al., 2019). They are oversimplified beliefs about emotional expressions that are taken to be more applicable and diagnostic than they actually are.

As a scientific tool for psychological science, population thinking is enriched by the discovery of ad hoc conceptual categories (Barsalou, 1983; Barsalou et al., 2003; Casasanto & Lupyan, 2015). Categorization is the grouping of objects or events according to their similarities (Murphy, 2002). Similarities are features of equivalence. The similarities shared by a category of instances, therefore, need not be physical; they can be mental, defined in relation to the categorizer’s goals in a specific situation, i.e., the particular function the instances serve for the categorizer in a specific situation. For example, the categories “flower” and “weed” are defined functionally, not physically. A bright yellow dandelion with green leaves might be categorized a nasty weed to pluck from a garden, a beautiful flower to place in a vase of wildflowers, or even a nutritious food to eat in a salad, depending on a person’s goals in a particular situation (Barrett, 2012). Likewise, the movement of “scowling” is a situated conceptual category because its instances are created by physical changes that vary across instances and people (i.e., individual differences in facial anatomy and the brain’s control of facial muscles cause varied execution at the muscular level, even when facial movements look the same to the naked eye; for discussion, see Barrett et al., 2019, SOM Box 5 and references therein). Even sounds (Barsalou, 1992) and smells (Cleland & Borthakur, 2020) are processed as conceptual categories. From this perspective, a brain’s assembly of physical signals, some of which are abstract mental features, can be thought of as construction of a situated conceptual category.

Our lab studies emotion categories as situated, ad hoc conceptual categories (Barrett, 2006, 2017b, 2017a; Lebois et al., 2020; Wilson-Mendenhall et al., 2011, 2013, 2015), predicting and observing the variation reviewed earlier in this paper. We hypothesize that as a brain continually assembles features of equivalence (i.e., similarities), which are part of larger ensembles of interwoven physical signals, it is actually constructing a situated conceptual category, such as an emotion category tailored to goals and functional requirements of the present situation. In a human brain, instances of the same category need not have similar sensory and motor features (i.e., instances of anger need not involve the same changes in blood pressure, respiration, physical actions); the features of equivalence that make the instances similar in a particular situation are abstract, mental features (i.e., the lower-dimensional, compressed, multi-modal summaries). A single abstract mental feature, or a single pattern of abstract features, can be associated with variable sensory and motor features for controlling the body and creating experience (i.e., with an entire distribution of possible neural assemblies that we call a situated category). These variable patterns of sensory and motor features each have some probability of fitting the present situation (i.e., a prior probability) based on similarity to past experiences that also contained the abstract features of equivalence; that is, by virtue of their shared abstract feature(s), different sensory and motor signals share the same (relational) psychological meaning in a specific situation (where a situation is defined as anything going on outside the brain, i.e., in the body and the world). The pattern that best matches the high-dimensional details in the situation gives meaning to those sensory and motor signals (i.e., categorizes them). That is, the features of equivalence are the means by which a brain generalizes from past experiences to categorize incoming sensory signals and outgoing motor signals, giving them psychological meaning in a specific situation as a brain regulates the body, guides action, and creates experience. In our lab, we’ve hypothesized that the brain’s continual, situated category construction creates all mental events (Barrett, 2017b, 2017a; Shaffer et al., 2022). Cognitions, perceptions, motivations, and the usual psychological phenomena named in the tables of contents of introductory textbooks (and the pages of journals like this one) might be considered ad hoc events that are assembled across the whole brain, not states that exist in distinct territories of neurons.

The implication is that every category is a situated event with no static, perceiver-independent prototype (for more on the view of conceptualization as a process, see (Barsalou, 1987; Barsalou et al., 2010; Casasanto & Lupyan, 2015; Spivey, 2007). Its features of equivalence are always constructed by a particular perceiver for a particular function in a particular situation. The summary of any situated category is analogous to a prototype that best suits the functional goal of the categorizer in that specific situation (Barsalou & Hale, 1993; Voorspoels et al., 2011). For a given perceiver, a given emotion category therefore has as many prototypes as there are different functional contexts or situations for that perceiver. Fear of starving in the woods, as a situated conceptual category, may have a different prototype for a given perceiver than fear in a haunted house, fear of being stung by a bee, fear of being rejected by a lover, or fear of accidentally harming a friend. Accordingly, that perceiver might cry in fear, laugh in fear, startle in fear, hug someone in fear, or even fall asleep in fear, whatever action their brain has learned to construct to mitigate threat in a given situation; and the corresponding physiological motif that supports each action vary accordingly (Obrist, 1981; Obrist et al., 1970). The same physical signals therefore have relational emotional meanings that can vary by situation and person. And two people who live in the same culture will learn to construct similar situated prototypes (a.k.a. similar ensembles of entwined, related physical signals), allowing them to communicate.

At this point, you might ask, “Amidst all this variation, what makes instances of fear what they are — fear — and not some other kind of emotion?” If so, you are asking a typological question that is not meaningful from a population-thinking perspective. Across the entire population of fear instances for all creatures whose brains are equipped to make instances of fear, the features of equivalence (i.e., the similarities that are used to categorize, creating an instance of fear) can be person- and situation-dependent, resulting in patterns of features that are highly variable.

Ultimately, population thinking changes the questions we ask about psychological phenomena. We won’t ask questions about the nature of fear, or any psychological category, as if it is a third-person phenomenon that happens independent of person and spatio-temporal context. Instead, we ask questions about how features of equivalence are chosen and constructed, how category construction works, how categorization proceeds, how bodily regulation, actions, and experiences emerge from this construction process, and what conditions produce similar prototypes across situations and people to allow for communication and category learning.

Constructionism

The ideas that we’ve been discussing — ensembles of interwoven signals, relational meaning, and population thinking — is consistent with a naturalistic philosophy of science (Gleiser, 2015; Godfrey-Smith, 2003) and bears a family resemblance to other systems of ideas, including William James’s radical empiricism (1996), ecological psychology (e.g., Heff, 2002 and references therein), grounded cognition (e.g., Barsalou, 2008), complex dynamical systems approaches to development (e.g., Zelazo, 2013 and references therein), the social cognition hypotheses of situationism, construal and dynamic tension systems (Ross & Nisbett, 1991) and what is now being called “radical embodied cognitive neuroscience” (Raja & Anderson, 2019). It’s also consistent with a variety of constructionist approaches to emotion.

The psychologist George Mandler first named constructionism as an approach to the science of emotion in his 1984 book, Mind and Body: Psychology of Emotion and Stress (in a section titled “The Construction of Emotion”; Mandler, 1984; see also Mandler, 1990), but nascent constructionist ideas can be easily traced back from early to mid-20^th century to the 19^th century, with historical tendrils reaching back even further (Barrett, 2017a; Gendron & Barrett, 2009). In the modern era, a constructionist perspective to emotion was synonymous with social constructionist views for many years until psychological construction was introduced in 2003 by the psychologist James Russell (Russell, 2003). In a social construction view, instances of emotion are hypothesized to derive from social and cultural ingredients (i.e., cultural artifacts), including social roles, beliefs, values, other people’s actions towards you, and various sociocultural structures (e.g., Averill, 1980; Boiger & Mesquita, 2012; Harre, 1986; for a historical and anthropological overview of social constructionism, see Reddy, 1997). Psychological construction views propose that the ingredients of emotion are psychological processes: emotional instances are hypothesized to arise from affective feelings when they are categorized, conceptualized, or otherwise made meaningful as emotions with a mental mechanism (e.g., Cunningham et al., 2013; Lindquist, 2013; Mandler, 1984; Russell, 2003; also see Barrett & Russell, 2015).

I have been developing a constructionist view with mentees and collaborators for three decades, first as the conceptual act theory (Barrett, 2006, 2007, 2012), which developed into the theory of constructed emotion (Barrett, 2017a,b) and has now been expanded to the constructed mind approach (Shaffer et al., 2022) as a multidisciplinary approach to understand how mental events and associated behavior arise within a brain that is in continual, dynamic conversation with its body and the surrounding world, including the social world. The specific hypotheses and evidence for this approach have been fleshed out in additional published papers (e.g., Barrett & Finlay, 2018; Barrett & Lida, in press; Barrett & Satpute, 2019, and references below). The details of our approach are beyond the scope of this article, but three key hypotheses are worth noting.

Prediction.

Our approach hypothesizes that situated category construction occurs via predictive processing (see these papers and references therein (Barrett, 2017b; Barrett & Simmons, 2015; Chanes & Barrett, 2016; Hutchinson & Barrett, 2019; Katsumi, Kamona, et al., 2021; Katsumi, Theriault, et al., 2021). A brain constructs prediction signals by computing features based on the past, checking those predictions against ongoing signals from the body’s sensory surfaces, and correcting those predictions if needed (a.k.a. learning). The physical signals hitting the sensory surfaces of your body have no inherent psychological meaning. Instead, they’re made meaningful — categorized — in relation to the signals in a brain, and a brain is wired to model its body and the world it inhabits. We hypothesize that each event begins as a situated category, constructed as an ensemble of interrelated, temporally evolving features that are assembled across the entire brain. When the computed features are in sync with the signals from the sensory surfaces of the body, those signals are said to be categorized, and mental features are experienced as explaining the sensations and their associated actions. Predictive processing, when understood as continuous category construction, offers a coherent, neurobiological research framework to unify many proposed constructs for how a brain creates relational meaning, such as appraisal, construal, generalization, memory, perceptual inference, conceptualization, and simulation.

Coordination and regulation of bodily systems.

Our approach hypothesizes that psychological meaning is rooted, fundamentally, in the brain’s predictive regulation of the body, called allostasis (Sterling, 2012; for a discussion of some of the neuroscience details, see e.g., Kleckner; Sterling & Laughlin, 2015; for modeling details, see Sennesh et al., 2021). This is consistent with evolutionary approaches to understanding nervous system function, in which the fundamental function of a brain is not to build knowledge about the world, but to control an animal’s energetic state as it navigates its niche (e.g., Cisek, 2019). From this perspective, every situated, conceptual category, such as an ad hoc category for fear, begins as abstract, mental features that include an abstract action concept or intention — a descending cascade of potential motor patterns to control the systems within the body (e.g., the autonomic nervous system, immune system, endocrine system, etc.) that support movements of the body (i.e., the skeletomotor system).

Action creates experience.

The dynamics of predictive processing suggest that action preparation gives rise to experience, not the other way around. During conceptual category construction, prediction signals that prepare motor action simultaneously cascade to simulate the expected sensory consequences of the expected motor movements (called an efference copy or corollary discharge). This hypothesis runs counter to typological views, which hypothesize that your brain works the other way around: it detects events in the world and constructs a perception, then evaluates the perception to create a cognition or emotion or some interaction of the two, which then results in an action plan. We hypothesize instead that perception and experience arise from predicted actions, rather than causing those actions, and experiences and actions are always constructed with respect to predicted future energy (allostatic) needs.

Cultural inheritance.

Culture has an evolutionary role in transmitting information from one generation to the next. The hypothesis is not that humans evolved particular signals, such as facial movements, physiological changes, or even patterns of neural firing with particular genetically encoded emotional meanings, which is a standard hypothesis in evolutionary psychology.¹⁵ Instead, there is growing evidence that a human is born with their brain under construction (e.g., Gao et al., 2017; Gilmore et al., 2018; Grayson & Fair, 2017; Zuo et al., 2017). Signals from the physical and social world are necessary inputs for the brain to develop the capacity to model its body in the world and to compute abstract mental features. This creates an opportunity for cultural inheritance (e.g., Boyd et al., 2011; Richerson & Boyd, 2008), in addition to genes, to transfer information across generations. During development and the processes that scientists call “socialization,” via the words (Gelman & Roberts, 2017) and actions of others (e.g., Atzil et al., 2018; Gendron, Mesquita, et al., 2020; Mesquita, 2022), it is hypothesized that culture creates recurrent situations that allow a brain to learn specific, situated meanings of particular signals in the natural and cultural ecology of a person’s environment. As human brains develop, they grow the micro-wiring to construct mental features in culturally relevant ways, including their attentional capacities for deciding which signals are relevant and which are noise to be safely ignored. (An obvious example is the ways in which a young brain tunes and prunes with experience to hear certain speech sounds while losing the capacity to hear others.)

In this way, a human brain develops the wiring to model its body and the world it inhabits. It becomes encultured with the knowledge to create meanings that are relevant to a particular set of cultural practices and values. As children develop into adults and interact with their world, they create some of the signals in the environment (by their words and actions) that will wire the brains of the next generation. What evolution produced is a human brain architecture with the capacity for flexible, situated meaning-making that can be synchronized across minds, and across generations. This capacity comes at a cost. Without the physical signals from the world, a brain does not receive the necessary wiring instructions to develop and function in a neurotypical fashion (e.g., McLaughlin et al., 2019).

Scientific Validity

These entwined ideas of complex signal ensembles, relational meaning, and population thinking have many important implications for how we understand our work as practicing scientists. I’ll mention just a few here.

Humans vs. Non-Human Animals

When viewed through the lens of relational meaning, all brains form situated concepts to categorize anticipated sensory inputs and guide action. What differs among species from this perspective is the degree of abstraction that a brain can support — the degree of compression in the features that are constructed — not the computational principles that govern their construction. These differences result from general brain-scaling functions (Workman et al., 2013) and the information available in an animal’s niche. For example, the human brain has expanded association cortices in the frontal lobes, parietal cortex and inferotemporal cortex compared to other primates, including other great apes (Sherwood et al., 2012, 2017), along with metabolic and neuropil changes in the upper layers of cortex (see (Theriault et al., 2021) and references therein). This expansion potentially allows for increased information compression and dimensionality reduction, suggesting that human brains are capable of multimodal summaries (i.e., features) characterized by greater abstraction (see (Finlay & Uchiyama, 2015; Katsumi, Kamona, et al., 2021). This hypothesis has important implications for how to generalize from non-human animals to humans.

Who is Constructing What?

It is common for human scientists to observe a fly freezing, a rat running, and a human gasping with a wide-eyed stare, and conclude (categorize) that all three animals are in a state of fear, functionally defined by the goal to protect from threat. A human brain can construct such a category, despite the vast physical differences in the three events, because it can compute an abstract feature of equivalence that creates the similarity (in this example, the goal). Now consider fly brains and rat brains — are they architecturally equipped to compute such abstract features? If not, then in whose brain does this state reside? (Hint: it’s not the fly’s or the rat’s.) This is my point when I describe instances of emotion and emotion perception as first person, perceiver-dependent events, not third-person, perceiver-independent phenomena (Barrett, 2012). A fly’s fearful state is real for human scientists, but perhaps not for the fly whose brain may not be capable of computing abstract features like “a goal to protect against threat” when making sensory signals meaningful as actions in its niche. Even perceiving an animal as “running” is, in fact, an abstraction from briefer, more basic (and perhaps innate) muscle motifs that can be flexibly assembled in a specific situation (in relation to the signals therein; e.g., (Datta, 2019)). Such notions call into question the “perceiver-independence” of functional views of emotion and the mind that confuse scientific consensus with objectivity.

The Importance of “Context,” Again

An animal’s body and its ecological niche are as important to the nature of its mind as the circuitry in its brain (and they help determine the sorts of meanings that its brain can compute). Such observations reinforce the importance of studying mental phenomena in the wild, rather than in traditional laboratory settings, or to create laboratory settings that are similar in complexity to the real world. Anything you learn about the mind from an experiment is constrained by the experimental setting. Watching a film clip while lying flat on your back, completely still in the bore of a large magnet, is not at all like strolling across campus with a friend in the early morning while sipping tea, confronting your boss in a board room with glaring lights and too much air conditioning, or lying in bed with your lover in the middle of the afternoon. As these examples imply, there is a crucial need to measure the state of a participant’s body (and their brain’s modeling of that body), even when studying psychological phenomena that you believe are purely cognitive and having nothing to do with the body, such as cognitive control (e.g., Kragel, Bianciardi, et al., 2019).

The Credibility of Psychological Research

When a study’s findings don’t replicate, the usual assumption is that the first study was flawed, or the proposed mechanistic cause was not sufficiently robust. (Or perhaps that professional pressures of organized science have led to a “creative” use of statistics.) Any behavioral effect, it seems, should be easy to replicate in any lab at any time of the day with any sample of participants as long as the strong causal influence is present. This is the underlying assumption of experimental designs that are common in experimental psychological science, designed with the machine metaphor in mind. A typical experiment isolates one or two causal influences and manipulates them with the hopes of observing a strong effect on the behavioral outcome of interest.

But there’s another possibility. If the psychological meaning of physical signals is relational, dependent on a complex web of other physical signals entangled in many weak, nonlinear interactions, then it is very likely that hidden causal factors are lurking in the context of the original experiment and differ in the attempted replication. This idea has been criticized as unscientific — so-called weasel words to avoid taking scientific responsibility for failed experiments (e.g., Yong, 2018). But from a view of relational meaning, grounded in complexity of causation, this possibility seems obvious and is likely very common. Psychological scientists rarely attempt to measure or manipulate the fuller web of influences. As a consequence, the impact of those influences — the variance they cause —mistakenly appears as error variance. This realization makes effect sizes of .30 look like an accomplishment rather than an embarrassment.

At minimum, problems with replication require us to reconsider the whole endeavor of multi-trial, stimulus-response-style experiments that are the bread and butter of experimental psychology, but that catastrophically fail to account for the full causal web of influences. In principle, many scientists would not defend that the mind works in independent and discrete chunks in time. In practice, however, many experiments fail to recognize that a participant’s response on any given trial is some combination of the signals that create the participant’s internal model, the signals of a given stimulus, and the “background” signals of the participant’s body and context. Ideally, one should model as many relevant signals as possible to maximize the robustness of scientific findings. Research has highlighted the utility of modeling brain and behavior in terms of continuous, temporally dependent processes (e.g., Huk et al., 2018; Spivey, 2007).

An apt example comes from the science of molecular genetics (described in Lewontin, 2000). A standard method for demonstrating that a gene is the source of a phenotypic characteristic, such as the development of neurotypical wings in a species of drosophila, is to identify mutations that disrupt normal development, such as producing curly wings rather than the usual straight ones. This mutation, however, produces curly wings only in a lab where contextual conditions like temperature and humidity are hidden in the “background context,” carefully controlled, and not in the real world where temperature and humidity vary across a broad range of environments.

From this perspective, the science of emotion recognition is a cautionary tale about the risks of valuing reliability over validity. A finding that robustly replicates again and again (think universal emotion recognition via choice-from-array) is not necessarily evidence that the tested hypothesis is valid (think flies with curly wings). And yet, choice-from-array remains in broad use within psychology, in both scientific and clinical practice (e.g., see Betz et al., 2019 for the influence of choice-from-array in the “Reading the Mind in the Eyes” test), despite an entire century of evidence that this method nudges (or shoehorns) participants to provide certain responses, limiting what scientists can learn.

Conclusion

This paper opened with a simple, everyday occurrence: looking at someone’s face and seeing evidence of their psychological state. The experience is so automatic and effortless that it feels natural, as if we were detecting a biologically prepared, universal meaning. As perceivers, we’re largely unaware of the multitude of factors that guide our actions and give rise to our experiences (including the signals from our own body). This ignorance is reflected in the typological approaches that, despite more than a century of scrutiny and critique, still dominate large swaths of psychological science to this day. When these other factors are considered, they are usually called “context” for the one or two causal influences that are the focus of experimental interest. We grabbed hold of that single string — context — tugged it and unraveled the dominant paradigm guiding psychological science. Then we gathered the various threads — complex signal ensembles, relational meaning and population thinking — and began to weave a new approach, an alternative that, if taken seriously, could radically change our conceptions of what a mind is and how to best study it, in full awareness that our new story is not yet complete and must be compared and integrated where possible with approaches that have been proposed as responses to the critiques of typologies.

Complexity, relational meaning, and population thinking have each been linked to paradigm shifts in other scientific fields. When Darwin (1858/2001) proposed that a species (as a biological category) was a population of variable instances rather than a type, he prompted a paradigm shift in biology, a scientific revolution whose tremors are still felt today. Likewise for the early 20^th century physicists who introduced quantum mechanics and observed that the world of solid objects, gravity, and so-called physical reality is actually a vast web of interacting quantities of energy, whose properties exist only in interaction with other quantities (Di Biagio & Rovelli, 2021; Rovelli, 2020; van Fraassen, 2010). And, who knows, a scientific revolution may still be in the offing for the mid-20^th century scientists, engineers, and mathematicians who conceived of cybernetics, systems theory, and eventually complexity theory and complex adaptive systems (for a brief history, see Tilak et al., 2021) and references therein).

Maybe now it’s psychology’s turn. Many psychological scientists continue to formulate their hypotheses and understand their scientific practices in terms of a mechanistic model of causation that arose in a scientific revolution of the 16^th century and remained unchallenged until the 19^th and 20^th centuries. This has serious implications. Science is more than a conceptual system for understanding how phenomena are caused. It is also a conceptual system for what phenomena are — an ontology of what exists. The science of emotion is a useful example in this regard. A recent survey indicates that a substantial number of psychological scientists who are experts in the science of emotion (80% of respondents) accept some form of a typological view (Ekman, 2016). Contextual factors, even powerful ones that transformed Serena Williams’ face from terror to elation, are generally treated as mere moderators of prototypic signals with inherent emotional meanings — i.e., as the exception rather than the rule.¹⁶ As a result, facial movements are routinely called “facial expressions,” as if they always display inherent psychological meaning (e.g., most recently, Cowen et al., 2021), which is a hypothesis to be tested, rather than a fact to be assumed; and certain configurations of facial movements are equated with “emotional expressions” (e.g., referring to scowling faces as “anger expressions”), rather than treating this correspondence as a hypothesis to be tested (e.g., for a recent example see Schneider et al., 2022).

Breaking free of our typological roots may require a radical conceptual shift. This paper has sketched one option: guiding our hypotheses and scientific practices by the idea that a mind emerges from a network of relations among signals, not as a collection of psychological modules with inherent, biologically prepared, independent psychological meanings. In this view, there is no single, universal human nature with a single set of universal psychological categories. The categories that a human brain is wired to construct, and the experiences of the world and the psychological meanings of actions that result, are not necessarily universal (as evidenced from numerous ethnographies in cultural and psychological anthropology). Instead, complexity, diversity, and the construction of relational meaning (i.e., the neural processes that create categories, and cultural shaping of category learning) could be the hypothesized universals.

How can such a conceptual shift best be accomplished? Single investigators, their intrepid band of courageous mentees, and their trusting collaborators can craft and test novel hypotheses in the privacy of their own labs (assuming they can convince anyone to fund them). Siloed communities of other scientists can craft and test similar hypotheses as they have for more than a century. But the resulting work becomes knowledge only by the consent of a broad swath of scientists. Science is a human activity that operates in a social context. What counts as knowledge in any science depends on shared goals and agreements about which questions are admissible and which methods count as acceptable tools of inquiry. Ultimately, the value of the ideas and interpretations in this paper depend on you, the reader. The questions you ask next, the studies you design from here on out, and the lessons you teach your students will help determine, in a complex ensemble of other scientists, whether psychology is ready to reconsider its typological preoccupations. The future of psychology as a science could depend on it.

Public Significance Statement:

Science is a way of understanding how thoughts, feelings, and other psychological events are caused. But it is more than that. It is also a set of assumptions about what a mind is and how it works. These assumptions reverberate much further, influencing medicine, education, industry, and other aspects of public life. This paper uses three well-trod methodological debates about emotional expressions as a lens to challenge a particular set of assumptions, and consider an alternative.

Appendix

Figure A:

An ecstatic Serena Williams after she beat her sister, Venus, in the 2008 U.S. Open tennis finals. (Photo credit: Barton Silverman/The New York Times/Redux.)