Concepts Dissolve Artificial Boundaries in the Study of Emotion and Cognition, Uniting Body, Brain, and Mind

Abstract

Theories of emotion have often maintained artificial boundaries: for instance, that cognition and emotion are separable, and that an emotion concept is separable from the emotional events that comprise its category (e.g., ‘fear’ is distinct from instances of fear). Over the past several years, research has dissolved these artificial boundaries, suggesting instead that conceptual construction is a domain-general process—a process by which the brain makes meaning of the world. The brain constructs emotion concepts, but also cognitions and perceptions, all in the service of guiding action. In this view, concepts are multimodal constructions, dynamically prepared from a set of highly variable instances. This approach obviates old questions (e.g., how does cognition regulate emotion?) but generates new ones (e.g., how does a brain learn emotion concepts?). In this paper, we review this constructionist, predictive coding account of emotion, considering its implications for health and well-being, culture and development.

Theories of emotion have long been guided by folk intuitions about the mind. One intuition is that cognition and emotion are distinct, biologically-based categories of phenomena that cause one another and compete for the control of behaviour. Another intuition is that an emotion concept (i.e., a mental representation of emotion) is distinct from the physiological changes, actions, and experience of an emotional event itself. These distinctions are called into question by recent accounts that offer a common computational framework for understanding how a brain predictively constructs thoughts, feelings, and other experiences in the service of action (Clark, 2013; Friston, 2010; Hohwy, 2013), and place concepts at the centre of the construction process (Barrett, 2017a, 2017b). In the first section of this paper, we introduce a constructionist, predictive coding account of brain function and consider its consequences for the relationship between cognition and emotion. In the second section, we discuss the hypothesis that emotion concepts are embodied, highly variable, and dynamic prediction signals. In the third and final section, we consider the role of language in the development and construction of emotion concepts.

Cognitions and Emotions are Constructed by the Brain as a Dynamic, Predictive Biological System

Cognition and emotion are often viewed as separate mental forces: at times opposing, at times interdependent (e.g., Clore & Huntsinger, 2007; Damasio & Carvalho, 2013). In many modern accounts, cognitions are hypothesised to cause or be caused by emotions (e.g., Lazarus, 1991; Oatley & Johnson-Laird, 1987; Ortony, Clore, & Collins, 1990; Schwarz & Clore, 1996). According to a causal appraisal theory (e.g., Scherer, 1999), for example, hearing a sudden noise while walking home in the dark would evoke cognitive evaluations (e.g., of threat), which then trigger the experience of fear: the racing heart and urge to run that motivate decisions and direct attention. Cognitions are also hypothesised to regulate emotions after the fact (e.g., Ochsner & Gross, 2005). Fear could be attenuated by considering the situation from a different perspective (e.g., it could be interesting wildlife), reinterpreting sensations (e.g., as excitement), remembering previous situations in which no harm occurred, etc. Phenomenologically, there seems to be a clear distinction between the aspects of such a scenario that correspond to emotion (e.g., physiological changes, actions, feelings of (un)pleasantness), and those that correspond to cognition (e.g., conscious decisions, memories, experiences of effort and volition). Consequently, scientific theories have traditionally assumed that emotions and cognitions are ontologically distinct categories of experience, generated by architecturally separate systems in the brain as it reacts to its environment.

Accumulating evidence does not support these assumptions, however (e.g., Duncan & Barrett, 2007). Instead, there is growing consensus that all experiences are constructed via the interaction of domain-general systems, in a brain that predictively, rather than reactively, guides behaviour. These recent accounts offer a common computational framework for how the brain guides action and makes meaning of sensation – to create cognitions, emotions, and perceptions – through the process of predictive coding (e.g., Barrett, 2017a, 2017b; Clark, 2013; Friston, 2010; Hohwy, 2013; Huang & Rao, 2011; Spratling, 2016).

A predictive coding account seeks to explain how the brain optimises energy efficiency while keeping the body’s physiological systems in balance. To minimise metabolic costs, the brain needs to infer the causes of the sensations it receives from both exteroceptive (world) and interoceptive (body) sensory channels. By accurately inferring causes, the brain can anticipate the needs of the body, and prepare to meet those needs before they arise (Sterling, 2012; Sterling & Laughlin, 2015). However, sensory input is noisy, incomplete, and can – like the sudden noise in the dark – have many different causes. According to a predictive coding account, the brain identifies which cause is most likely by comparing the current sensory array to prior experiences and determining what is most similar. As an internal model of the world (Buckner, 2012; Hassabis & Maguire, 2009), including the body and its internal milieu (Barrett & Simmons, 2015; Garfinkel, Seth, Barrett, Suzuki, & Critchley, 2015), the brain uses the statistical regularities of the past to predict which sensations are most probable in the future, and which actions are most beneficial to deal with those sensations (e.g., Barrett & Simmons, 2015; Chanes & Barrett, 2016).

Prediction is neither a deliberate nor a consciously accessible act. Rather, it is the process by which neurons communicate to generate behaviour and construct experience. Predictions prepare the brain by flexibly changing the firing of sensory and motor neurons in anticipation of the next moment (Denève & Jardri, 2016; Denève & Machens, 2016). These changes emerge as updated brain states, or patterns of distributed neural activity. Predictions also guide which sensory inputs are attended to and which are ignored. Anticipated inputs confirm predictions, categorising sensations and making them psychologically meaningful (Lochmann & Deneve, 2011). Unanticipated sensory inputs are prediction errors (the discrepancy between what was predicted and what actually occurred), creating an opportunity to modify the internal model, so the brain can predict more accurately in the future.

When past experiences of an emotion (e.g., fear) are the best fit for the current sensory array, the brain uses this emotion as its best guess at what will cause sensory inputs and what should be done about them. Once this prediction is sufficiently corrected by any prediction error, sensations are categorised and explained as emotion. That is, the emotion is understood as the cause of actions and physical changes in the body, giving rise to the folk intuition that emotions are central drivers of behaviour and experience. Cognitions, as well as perceptions, are constructed in a similar way (Huang & Rao, 2011; Spratling, 2016). What distinguishes between apparent categories of experience is the brain’s attentional focus, or which inputs are foregrounded (Barrett, Wilson-Mendenhall, & Barsalou, 2015). The experience of cognition occurs when the brain foregrounds mental contents and processes. The experience of emotion occurs when, in relation to the current situation, the brain foregrounds bodily changes. When walking home late at night, the brain may use past experience (of a sudden noise, an elevated pulse, the dark) to predict fear.

Every categorisation of sensation (e.g., as fear) updates the neural context in which the brain is making predictions for the body. The brain will subsequently prioritise perceptions, actions, emotions, and cognitions that have previously been reinforced in similar situations. This iterative process of constructing and confirming predictions gives rise to the folk intuition that cognition and emotion cause one another or compete for control. For instance, when a cognition (e.g., mental speech) precedes a change in emotion (e.g., attenuated fear), this is understood as emotion regulation. However, a predictive coding account argues that regulation does not exist separately from construction. Rather than having separate causes, cognitions and emotions are constructed based on the temporal dynamics of the brain (Spivey, 2007). As the brain transitions through all possible patterns of neural activation (i.e., state space), the current brain state, in combination with inputs from the body and world, influences the probability of future brain states (Barrett, 2009). A predictive coding account therefore revises hypotheses about the relationship between cognition and emotion. Cognition does not control emotion in a top-down fashion, nor do emotions provoke cognitions; the transition from one to the other occurs in an uninterrupted, domain-general meaning-making process (Figure 1).

Figure 1.

Schematic depiction of the dynamics of a mental event (e.g., an instance of emotional experience) from prediction to categorisation. Blue lines indicate top-down signal; red lines indicate bottom-up signal. Based on the current brain state, previous experience is used to generate a cascade of predictions focused on meeting the body’s expected needs for action (i.e., allostasis). As depicted, changes in sensory input (i.e., prediction error) may result in further tuning of the predictions. When predictions are confirmed, the current sensory array has been categorised and a new brain state instantiated. In turn, visceromotor changes and actions impact sensory inputs from the body and world, respectively. Current experience also updates the internal model, becoming part of the previous experience that will be brought to bear in future predictions.

A predictive coding account has many implications for the study of experience. For one, it suggests that traditional laboratory paradigms may limit the generalisability of experimental findings to real-world predictive processing. These paradigms typically present randomised sequences of stimulus and response, with trials treated as independent so they can be analysed in aggregate. As such, they put a continuously predicting brain into an unnatural environment, disrupting rather than modelling the temporal dependencies inherent in brain function. A predictive coding account suggests that experience is better assessed using a holistic approach, in which continuous measures of activity in the brain and body are used to capture cognitions and emotions unfolding over time (e.g., Ariff, Donchin, Nanayakkara, & Shadmehr, 2002; Müller et al., 2008), and at different levels of analysis (e.g., Mack, Preston, & Love, 2013; Purcell et al., 2010). Using computational models that account for complex, nonlinear dynamics (e.g., Friston, Harrison, & Penny, 2003; McClelland et al., 2010; Pezzulo et al., 2013), scientists can examine behaviour and experience as the brain continues on its probabilistic trajectory through state space. These recommendations lend themselves to empirically testable hypotheses and questions about the nature of cognition and emotion (Table 1, numbers 1–3).

Table 1.

Empirically Testable Hypotheses and Questions Generated by a Predictive Coding Account of Cognition and Emotion

Cognitions and Emotions are Constructed by the Brain as a Dynamic, Predictive Biological System

1Hypotheses: Continuous measures of neural activity (e.g., EEG, fMRI) will reveal that the spatiotemporal patterns for instances of the same category of mental event (e.g., fear) vary from one another as much as from instances of different categories. Similarly, detailed self-report measures will reveal variation in the associated mental features. 2Questions: How do phenomenological boundaries in the experience of cognitive and emotional events (e.g., Zacks & Swallow, 2007) map to continuous measures of neural activity? Are the same boundaries observed in cultures where there is no linguistic distinction made between ‘thinking’ and ‘feeling’ (e.g., Ifaluk ‘nunuwan’; Lutz, 1985)? 3Hypothesis: Brain states, and their associated mental events, evidence properties of complex, non-linear, dynamical systems (e.g, 1/f scaling, fractality; Richardson & Chemero, 2014).

Concepts, as Predictions, are Intrinsically Embodied and Highly Variable

4Hypotheses: When measured at an idiographic level, the mental and physical features of emotion categories might be more consistent and specific than at a nomothetic level. There will be individual differences in the number of emotion categories and variability of their instances. 5Hypothesis: The physical features and internal bodily sensations associated with categories of mental events (e.g., Nummenmaa, Hari, Hietanen, & Glerean, 2018) will vary across cultures. 6Question: Which leads to more efficient physiological regulation: increasing variability in category instances (i.e., within-category diversity for a given emotion category such as fear), or increasing the precision and number of emotion categories (i.e., more fine grained categories with less variation from one another)?

Language Plays a Central Role in the Development and Construction of Concepts

7Questions: How is the conceptual system (i.e., the brain’s internal model) updated when new emotion words are acquired, either by observation or instruction, and how does this impact embodied experience? 8Hypotheses: Increased similarity in individuals’ momentary emotion concepts (and therefore in their emotional experiences and perceptions) will result in synchrony, as well as decreased interpersonal tension and associated metabolic costs. 9Question: What is the most effective way to teach emotion concepts (e.g., Maurer & Brackett, 2004) to improve cross-cultural communication and acculturation?

Concepts, as Predictions, are Intrinsically Embodied and Highly Variable

Scientific theories have traditionally assumed that a firm boundary exists between categories and concepts. Members of a category are instances, events, or objects that exist in the natural world, while a concept is a mental representation of that category inside the head (for reviews, see Goldstone & Kersten, 2003; Smith & Medin, 1981).¹ For example, the concept of ‘fear’ is dissociable from the actions and sensations of actual fear events. In many of these views, concepts are considered relatively stable objects of cognition that have a set of automatically-activated, context-independent properties (for reviews, see Lebois, Wilson‐Mendenhall, & Barsalou, 2015; Mahon & Hickok, 2016). The central features of ‘fear’ (e.g., typical physical sensations, behaviours, affect) are maintained regardless of whether it is instantiated when walking home in the dark or giving a public speech. Accordingly, concepts are understood as amodal symbols that operate independently of the brain’s systems for perception and action (e.g., Mahon, 2015).

In contrast, a predictive coding account is consistent with proposals that concepts and categories are constructed ad hoc, according to situation-specific functions (Barsalou, 1991, 2003; Barsalou, Simmons, Barbey, & Wilson, 2003; Casasanto & Lupyan, 2015). In these views, concepts are multimodal, grounded simulations represented by the activation of the same neurons that underlie sensation and movement (e.g., Barsalou, 2008; Kan, Barsalou, Olseth Solomon, Minor, & Thompson-Schill, 2003; Pulvermüller & Fadiga, 2010). A concept’s features are fully context dependent: when walking home in the dark, ‘fear’ may involve a racing heart and the propensity to shriek; when giving a public speech, ‘fear’ may involve a tense stomach and a stammering voice. In other words, concepts are the predictions that the brain uses to categorise sensory inputs and motor actions (Barrett, 2017a, 2017b). When the brain constructs an emotion concept, the result is emotional meaning. In turn, these category members become part of the internal model used as a basis for future predictions (Hoemann, Gendron, & Barrett, 2017). The emotion categories that emerge from this process are conceptual categories (Barrett, 2012), in that within-category similarities and between-category differences are not based on perceptual features, but imposed by the brain according to the function that category serves.

The core task of the brain is to keep the body’s physiological systems in balance (Sterling, 2012). Because of this, all concepts – whether they deal with emotion or not – involve information about the body in the world (Barrett, 2017a, 2017b). Accordingly, emotion concepts are partial re-enactments of visceromotor, motor, and other sensory changes that were engaged in past emotional experiences (e.g., Niedenthal, Barsalou, Winkielman, Krauth-Gruber, & Ric, 2005). Brain areas responsible for movement and physiological regulation are activated by emotion words (Moseley et al., 2015), and observing and producing a smile activate the same facial muscles (Foroni & Semin, 2009). Embodiment also plays a constitutive role in the construction of emotion concepts (Niedenthal, 2007). Deficits in emotion perception are observed after spontaneous activity of associated facial muscles is inhibited (e.g., Niedenthal, Brauer, Halberstadt, & Innes-Ker, 2001), and after neural processes associated with embodiment are disrupted via transcranial magnetic stimulation (Pitcher, Garrido, Walsh, & Duchaine, 2008). Further, emotional experiences are constructed by brain networks involved in implementing emotion concepts (Lindquist, Wager, Kober, Bliss-Moreau, & Barrett, 2012; Wilson-Mendenhall, Barrett, & Barsalou, 2015; Wilson-Mendenhall, Barrett, Simmons, & Barsalou, 2011) – the same networks that contain the visceromotor circuitry that regulates the systems of the body (Kleckner et al., 2017).

When the brain constructs embodied, context-sensitive emotion concepts, it produces variation in the associated physiological and motor responses – a heterogeneity that is apparent in meta-analyses of faces (Barrett, Adolphs, Marsella, Martinez, & Pollak, forthcoming), brains (e.g., Clark-Polner, Wager, Satpute, & Barrett, 2016), and bodies (Siegel et al., 2018). Notably, significant variation within emotion categories has been observed across 202 studies measuring autonomic nervous system activity during lab-based inductions (Siegel et al, 2018). Patterns of activity did not consistently or specifically distinguish between emotion categories (e.g., both anger and fear inductions resulted in increased heart rate when compared to a neutral baseline, but with significant statistical heterogeneity; see also Cacioppo, Berntson, Larsen, Poehlmann, & Ito, 2000; Stemmler, 2004). This variability could not be fully explained by induction method or other experimental moderators (Siegel et al, 2018).² Likewise, brain activations for the same emotion have been shown to differ as a function of situation-specific features (Wilson-Mendenhall et al., 2015; Wilson-Mendenhall et al., 2011). These descriptive features (mental, physical, internal, external) do not overlap completely with other instances in the same emotion category, but can (and often do) occur in other emotion categories (Hoemann et al., 2017; Wilson-Mendenhall, Barrett, & Barsalou, 2013). A particular instance of ‘fear’ may be more similar to an instance of ‘anger’ (e.g., both involve social threat and intense stomach sensations) than to another instance of ‘fear’ (e.g., that involves pleasant thrill-seeking, such as a haunted house).

A predictive coding account considers this variation meaningful rather than random, which carries implications for theory and measurement. Foremost, it suggests that studies must account for individual and context-based variation. Although one might argue that variation discredits physiological perspectives on emotion and cognition, we disagree. Emotion concepts have a biological basis, even if emotion categories do not cut nature at its joints with distinct, diagnostic sets of features. Physiological variation in emotional experience is functional: it occurs because concepts are created to meet body’s present and predicted metabolic needs. By acknowledging variation, theories can generalise beyond the lab and account for the vicissitudes of everyday life. A predictive coding account suggests that experience is better assessed using an idiographic approach, in which experience sampling methods (e.g., Conner & Mehl, 2015; Nezlek, Vansteelandt, Van Mechelen, & Kuppens, 2008) are used to test whether consistent and specific emotion categories exist within individuals (Table 1, number 4). Variation within a given emotion category can be modelled by manipulating fine-grained contextual features (e.g., situational demands; Wilson-Mendenhall et al., 2011), allowing scientists to better map cross-cultural variation (Table 1, number 5). By modelling individual differences in momentary experience and physiology, scientists can assess person-specific impacts for mental and physical health (e.g., Barrett, 2017a; Kashdan, Barrett, & McKnight, 2015; Lumley, Beyer, & Radcliffe, 2008), as well as design targeted interventions (Table 1, number 6).

Language Plays a Central Role in the Development and Construction of Concepts

Variation poses a challenge: the brain needs a way to learn the statistical regularities necessary to make accurate predictions. Language may serve this purpose, playing a key role in concept acquisition by directing attention and communicating intentionality (Chen & Waxman, 2013; Ferry, Hespos, & Waxman, 2010; Gelman, 2009). Words serve as invitations to make meaning from sensory input, creating similarity between exemplars that do not share perceptual features (e.g., Graham, Booth, & Waxman, 2012). Emotion categories may be especially reliant on the cohesion provided by words to achieve conceptual consistency. Contrary to accounts that discrete emotion concepts such as ‘fear’ and ‘anger’ are a form of inborn or early-to-develop knowledge (e.g., Izard, 1994; Kobiella, Grossmann, Reid, & Striano, 2008), data suggest they develop gradually across childhood as the brain learns from experience. Emotion categories and their corresponding words are initially applied broadly and then their use narrows over time, suggesting concepts are being refined (e.g., Widen & Russell, 2003, 2008). While young children anchor on valence-based information (pleasure, displeasure), adults have a more elaborated, multidimensional organisation that includes arousal (i.e., level of activation). This conceptual development has been shown to be uniquely mediated by increasing verbal knowledge (Nook, Sasse, Lambert, McLaughlin, & Somerville, 2017), further underscoring the role of language in emotional learning.

Language may also play an active role in shaping experience (for reviews, see Boroditsky, 2010; Lupyan, 2012). Rather than a means of simply activating stored knowledge, words are a special type of sensory input in the predictive process (Elman, 2009; Lupyan & Clark, 2015). Words highlight functional similarity between past and present experiences, forming networks of semantic associations, such that hearing the word “fear” may cue prior experiences of ‘anxiety’, ‘tense stomach’, or ‘public speaking’. As such, words create a flexible context for the online construction of concepts (Barrett, 2017a; Casasanto & Lupyan, 2015). Hearing the word “fear” while preparing a public speech may make the construction of ‘nervous’ more likely; while riding a roller coaster, it may encourage ‘thrilled’. The brain uses words to tune prediction, as shown by studies of object recognition (e.g., Boutonnet & Lupyan, 2015; Lupyan & Thompson-Schill, 2012), category learning (e.g., Lupyan & Casasanto, 2014; Lupyan, Rakison, & McClelland, 2007), and visual awareness (e.g., Lupyan & Ward, 2013; Ostarek & Huettig, 2017). These effects have recently been demonstrated for the prediction, perception, and memory of emotional expressions (Chanes, Wormwood, Betz, & Barrett, 2018; Doyle & Lindquist, 2018; Fugate, Gendron, Nakashima, & Barrett, 2017). Further, labelling or writing about emotional experiences can help reduce their intensity, with important therapeutic implications (e.g., Kircanski, Lieberman, & Craske, 2012; Lieberman et al., 2007; Pennebaker, 1997).

Language structures both individual and shared experience. Emotions can be shared through language, allowing predictions to be collectively constructed (e.g., Rimé, 2007, 2009). Concepts are inherited through language: through devices such as labels (e.g., “fear”) and generic statements (e.g., “people scream in fear”), language aligns concepts and cultural practices across generations (Gelman & Roberts, 2017). For example, as children hear their parents use emotion labels in a variety of perceptually dissimilar situations (e.g., “fear” as applied to both public speaking and the dark), they come to associate these instances as functionally similar. This implies that both the ontological and evolutionary development of emotion concepts are shaped by the language practices in a given culture (e.g., Richerson & Boyd, 2005). Future work can investigate how emotion concepts systematically influence the brain (Kitayama & Salvador, 2017) as well as the body (Niedenthal, Winkielman, Mondillon, & Vermeulen, 2009; Seth, 2013) by examining the social and linguistic context of their use (for discussion, see Barrett, 2017a; Gendron, Mesquita, & Barrett, in press) (Table 1, number 7). Moreover, emotion concepts might function as a tool for cultural coordination (e.g., De Leersnyder, Boiger, & Mesquita, 2013; Mesquita, Boiger, & De Leersnyder, 2016), helping individuals physiologically regulate one another (Barrett, 2017a) (Table 1, number 8). Interventions designed to teach emotion language and concepts (e.g., Hagelskamp, Brackett, Rivers, & Salovey, 2013) may therefore lead to shifts in emotional meaning-making, facilitating communication and acculturation (Table 1, number 9).