A sensorimotor control framework for understanding emotional communication and regulation

Abstract

Our research team was asked to consider the relationship of the neuroscience of sensorimotor control to the language of emotions and feelings. Actions are the principal means for the communication of emotions and feelings in both humans and other animals, and the allostatic mechanisms controlling action also apply to the regulation of emotional states by the self and others. We consider how motor control of hierarchically organised, feedback-based, goal-directed action has evolved in humans, within a context of consciousness, appraisal and cultural learning, to serve emotions and feelings. In our linguistic analysis, we found that many emotion and feelings words could be assigned to stages in the sensorimotor learning process, but the assignment was often arbitrary. The embodied nature of emotional communication means that action words are frequently used, but that the meanings or senses of the word depend on its contextual use, just as the relationship of an action to an emotion is also contextually dependent.

1. Introduction

This review on the neuroscience of action and affect is being undertaken as part of the ‘The Human Affectome Project’, a 2016 initiative organised by the non-profit organisation Neuroqualia. The project aims to produce a series of overarching reviews that can summarise much of what is currently known about affective neuroscience while simultaneously exploring the language that we use to convey feelings and emotions. The project is comprised of twelve teams organised into a taskforce focused on the development of a comprehensive and integrated model of affect that can serve as a common focal point for affective research in the future.

To that end, our team was specifically tasked to review the neuroscience research related to actions, the way that people communicate feelings that relate to actions, and whether or not the feelings terms that people convey in communication might inform the way we approach action-related neuroscience research.

The evolutionary origins of feelings and emotions lie in their critical role in regulating behaviour, and so consequently, emotions are tied closely to actions. One might consider these regulatory behaviours to be of two types. Firstly, there are those behaviours that serve to regulate an individual’s behaviour so that he or she can maintain a physiologically healthy state. Secondly, many animals live in social groups of varying size and complexity, and so additional behaviours are designed to influence the behaviour of others. Humans stand out as being different to other animal species in the way that actions relate to emotions, and this paper will review these differences. We consider that there may not be fundamental differences but that these are largely a matter of degree. Most obviously, whereas in other animals, actions are highly limited in their variability, in humans they are highly flexible and adaptive to variation in context. The exception occurs in states of mental disorder where action patterns may become stereotyped and repetitive.

Given that humans are a highly social species, living in large, socially complex societies, the action repertoire for social communication has become particularly extensive. In addition to language, we communicate our emotions to one another through actions such as body posture, speech (tone, volume or intonation), facial expression and hand gestures (Vaessen et al., 2018), and complex control systems have evolved to serve this function. In humans, emotional states are constructed through a self-awareness of actions, associated with contextual and cultural learning. Flexibility depends upon complex interaction between a subcortical network serving associative learning and cortical mechanisms allowing for contextual influence and sensorimotor learning. Potential conflicts between outcomes of intentions at varying levels within a hierarchical system of goals are managed, and systems constantly learn and become updated through evaluation and appraisal mechanisms.

In this review we aim to consider how the relationship between action and emotion has evolved in humans, examining how an enhanced capacity for flexible motor control and motor learning has resulted in a complex system for the communication and regulation of emotional expression and communication, involving cultural learning and consciousness. We consider what happens to this capacity in states of disorder and disease, and how, through embodied cognition, action words are often employed to describe emotions and feeling states.

Firstly though, we will consider two principles that underpin our review, the first of which is expressed through predictive models of perception, action, and cognition, which argue for an active inference account of the mind (Friston, 2010; Clark, 2013).

Within active inference accounts, the primary goal of the brain is to maintain allostasis. Allostasis is a term used to describe how the body maintains stability through change. It differs slightly from homeostasis in allowing learning and anticipatory responding to vary set-levels of parameters in order for the organism to adapt to its environment, rather than keeping predetermined levels constant (Ramsay et al., 2014). In the brain, allostasis is achieved primarily by comparing bottom-up sensory inputs from the world and body to top-down ‘predictions’ about the world and body (Clark, 2013). Mismatch between feedback and feedforward processes gives rise to ‘prediction errors’, presenting potential risks to stability. These prediction errors trigger action in order to address the cause of the error and restore equilibrium. Rather than perception being a blank canvas onto which the state of the world is painted, perception is the result of comparing predictions about the world to actual sensory inputs. When an organism detects a discrepancy between predicted and experience inputs, this brings key threats to homeostasis to attention, triggering action to address these issues.

These actions are usually intentional which means that they are enacted with a plan to achieve a specific goal. Therefore, all actions have a motivation or an emotional value, which result in the action being planned to achieve a desirable goal with an associated sensory feedback. This process has been learned from birth ever since the infant starts to act on his or her environment in an effort to achieve desirable ends. This learning process utilises feedback-feedforward processes. Internal sensorimotor models are encoded in the brain which associate internal models for action patterns with those that encode their sensory consequences and goal-achievement. Consequently, ideation of the action’s goal triggers an associated motor plan, which determines the selection, sequences, and power of muscular contractions that form actions, needed to achieve the goal. The action itself triggers a range of sensory consequences, occurring across all modalities, whether visual, tactile, vestibular or kinaesthetic, creating further feedback. This feedback is then compared to the predicted consequences of any planned action, actively inferring causes of any error and modifying the internal sensorimotor model, so that prediction error is decreased to an acceptable range, actions are coordinated to achieve goals in an optimal fashion, and allostasis is maintained.

The second principle is that internal models of goal-sensorimotor relationships are organised in hierarchies. Any action can be reduced to a set of specific muscular contractions which combine to form simple actions and then complex actions, which are enacted to serve short term plans, and finally longer-term plans. The longer-term plans may be so distal to the actions taken to achieve them that they are largely independent of these individual actions. Consider for example, the combination of muscle contractions required to grasp a door handle and open the door, which may be serving an immediate goal of leaving a room. This may be serving a higher goal of leaving a meeting, which could be an act which communicates an expression of a desire to leave a group. At each level wider contextual factors characterise the intention.

In this review we consider the relationship of sensorimotor control mechanisms to feelings and emotion, before considering whether feelings and emotion words can inform the way in which we approach research related to sensorimotor control. The principles are summarised in Fig. 1. One notable feature of this model is the omission of something specifically called “executive function” since the function of control is an integral aspect of the relationship between higher levels of the hierarchy of the sensorimotor-goal model and limbic associations. Similarly, we do not, at this stage, make a distinction between social and non-social actions, but assume that the strong motivational value of interpersonal function will mean that limbic associations will be particularly important in the development of those sensorimotor goal relationships. Finally, we do not discuss anatomical correlates at this point though these are considered in more detail in Section 4.

Fig. 1.

Schema of components of internal model of sensorimotor relationships involved in the encoding and enactment of actions with emotional values.

2. Comparative psychology of emotion expressed in action

The expression of emotion through action reflects the biological continuity of emotional communication with other animals. Behaviours can convey emotion explicitly through displays, such as when an animal shows aggression or courtship behaviour, or passively, such as when an animal reveals its distress. In many species, the emotional expressions of conspecifics affect observers’ actions.

Humans lie at the end of a continuum in their ability to vary their expressions of emotion, through a combination of intentional and automatic control of actions controlling body posture, manual gesture, facial movements or vocalisation. Apart from the vertebrates, most other animal species are relatively inflexible regarding their behavioural repertoire including motor patterns or vocalisations and rely on fixed, innate patterns of action. Most animals have fixed patterns of vocal expression that are largely innate (the chicken is a good example). However, many primates do adjust their actions or vocalisations to their observers, and some species may even flexibly mimic the vocalisations of other species including those of predators or for instance of human speech. The parrot is the most obvious example of that, but the phenomenon has been observed in other species too (e.g., seals producing speech-like sounds; Ralls et al., 1985). Vocal learning is observed in several distantly related mammalian species including bats, cetaceans, elephants and seals (Chakraborty and Jarvis, 2015). Some birds are comparable to humans in having a capacity for an extensive and constantly variable vocal repertoire (e.g., songbirds). Whether this extends to other aspects of action is a question hardly explored but research in raven and parrot species, shows that they adjust their social behaviours and related actions when knowing that they are being observed by conspecifics.

To facilitate emotion transmission, humans have evolved exceptionally communicative faces where all the expressive parts are enlarged and accentuated (Kobayashi and Kohshima, 1997; Kret, 2015; Kret and Tomonaga, 2016; Tomasello et al., 2007). As well as evolving a high level of intentional motor control over expression, human facial features are adapted to maximising expressiveness. For example, our visible eye-white, which facilitates emotion expression and gaze following, is larger than in other species, likewise lip colour and eyebrow size are emphasised. Using our unique collection of facial features, humans express their emotions explicitly through emotional signals that can be subject to intentional control (such as smiling). These emotional signals may also combine with other forms of expression that may be under a lesser degree of intentional control or awareness such as pupil dilation (Harrison et al., 2009; Kret et al., 2013a, b; Kret et al., 2014, 2015; Kret and De Dreu, 2017), blushing (Dij et al., 2009; Leary et al., 1992), and other subtle autonomic cues (Kret, 2015; Levenson and Gottmann, 1983; Reed et al., 2013).

Given the degree to which humans have evolved a capacity to express emotion so clearly, one might expect that the emotional signals would be unambiguous. However, the opposite is more often the case, and the emotion being conveyed by any particular form of expression depends on a variety of factors. For example, the smile is an expression seen across cultures, which participants in lab experiments are supposed to link with the label ‘happy’. However, oftentimes it is not happiness that is expressed in a smile, but something completely different (e.g., nervousness or contempt). For instance, when participants were shown the facial expression of an athlete winning a medal at the Olympics (thus a truly happy person), they were not able to say whether the athlete had won or lost the competition (Aviezer et al., 2012).

In this case, examination of the way that expressions are used by other species may throw a light on their adaptive function. To continue with the example of the smile, this expression stems from the so-called ‘bared teeth display’, an expression shared amongst primates (Van Hooff, 1976; Waller et al., 2006). It is shown when a primate is afraid, but also to signal subordination to a more dominant individual. The smile still has this function in humans – during interactions between a person high, and a person low in power, it is the latter smiling most (Hecht and LaFrance, 1998). In humans, the smile has become ritualised, and next to expressing nervousness or subordination, communicates affiliation, love and affection. Over our lifetime, we learn when to use or reciprocate this expression and when to inhibit it (Hess and Fischer, 2013).

Another example of a facial expression long-thought to be a universal expression of emotion is that of fear. However, there is no universal agreement on what constitutes a fearful expression. The expression associated with fear in Western stimuli (e.g., Ekman, 1993) is instead interpreted as threatening and aggressive in other societies such as in a case study of the people of the Trobriand Islands of Papua New Guinea (Crivelli et al., 2016). This expression is also a common image in apotropaic art featuring threat displays that are meant to ward off harm and deter evil (Kret and Straffon, 2018). Representations of this expression in apotropaic art generally show staring or bulging eyes, flared nostrils, open mouth, flaunted tongue, face distortions, and very often bared fangs or teeth (Emigh, 2011).

The lack of universal agreement on what constitutes a fearful expression is also signified in popular validated facial expression stimulus sets used in psychology, including the one used in the study by Crivelli et al. (2016), which intermix different facial expressions in the category ‘fear’. The faces of some of the actors in the set show widened eyes in combination with the display of upper and lower teeth, similar to the primate bared teeth display (Andersson, 1980; Waller et al., 2006), whilst others within the same face set show the typical expression epitomised by Edvard Munch’s painting ‘The Scream’ or the ‘Home Alone’ film poster; the eyes are enlarged and the mouth is wide open but the teeth do not show. Still other stimuli show a mixture of the two (Kret and Straffon, 2018). This example demonstrates why it is important to make a clear distinction between different negative expressions – they likely have different evolutionary origins and their meanings are context-dependent. The bared teeth face, like other threat displays, probably evolved from the ritualisation of attack or pre-fight movements or intentions, such as biting. In contrast, the gasping face most likely evolved as a fear display from screaming or calling behaviour (Andersson, 1980).

The study of facial expressions in our closest living relatives, the chimpanzees and bonobos, along with studies in more distantly related species such as macaques, can help resolve such ambiguities. Behavioural observations have demonstrated that non-human primate emotional expressions and human emotional expressions can play similar functional roles. For example, human infants use a pout face to solicit their mother’s attention, and a similar facial expression can be found in infant chimpanzees for the same bonding functions (Blurton Jones, 1971; van Lawick-Goodall, 1968). Therefore, cultural appropriation of expressions to serve conventional understandings of associated emotional states would seem to be one factor that has led to a difference in the emotion associated with an expression in some primate groups to that of humans. Notably, the exception of pouting occurs in infants who are the least exposed to cultural influences.

Another factor that may be important in modifying the meaning of a facial expression is bodily expression. In fact, bodily expression of emotion may be just as important as facial expression. However the existing literature has largely focused on posed facial expressions (Adolphs, 2002; Dimberg, 1982; Ekman, 1993; Ekman and Rosenberg, 1997; Frijda, 2016; Hess and Bougeois, 2010), which may affect the ecological-validity of the findings (Kret, 2015). In fact, two of the most illustrious theoreticians of emotion, Darwin and James, discussed whole-body expressions at great length. Darwin famously included postural descriptions in ‘The expression of the emotions in man and animals’ (Darwin, 1872), and James (1890) investigated recognition of emotion with photographs of whole-body posture.

Faces and bodies are equally salient and familiar in daily life and often convey some of the same information; when they do not, it is oftentimes the body that reveals expressers’ genuine feelings (i.e., Aviezer et al., 2012; for a review, see de Gelder et al., 2010). In recent decades researchers have taken up the issue of bodily expression recognition, and results from several behavioural experiments using independent stimulus sets now allow us to conclude that recognition performance for bodily expressions is similar for face and body stimuli (Kret et al., 2013a, b; Kret et al., 2011; de Valk et al., 2015).

In conclusion, most other animals show relatively limited range of emotional expressions, and tight correlations between emotional state, body posture and facial expressions. However, the capacity for social learning and intentional control over expression results in a departure from these relationships. Such a departure can result in more flexible behaviours, such as the diverse repertoires of vocalisation in songbirds. In humans who have evolved an especially strong capacity for intentional control and social learning, facial expressions of emotion may be modified to convey varied and subtle meanings in the context of bodily expression, autonomic reactivity, and cultural convention.

3. Interoception and action in emotion

Modern theory increasingly recognises emotion as providing important influences upon action to make it serve an adaptive function, in much the same way as cognition and perception (Moors and Fischer, 2019). This stands in great contrast to the previous thinking of emotions as maladaptive disruptors of decision-making and action. In fact, it is increasingly recognised that the common principles of motor learning and control are likewise applicable to the awareness and regulation of emotions. More importantly, the ‘active inference’ principles of sensorimotor control discussed in the introduction can also be applied to emotion. Within active inference accounts of emotion, emotion represents the inferred causes of a ‘prediction error’ (Barrett et al., 2016), wherein emotions represent inference about why there is a discrepancy between the expected and experienced sensory input. This prediction error then arises as a readiness for action (Ridderinkhof, 2017). Emotions act as motivators for action, through facilitating ideation of goals (Moors et al., 2017), and thus are not separate from cognition and perception, but rely upon the same processes.

In daily life, emotional experiences are almost always accompanied by physical action, whether through unintentional facial expressions or intentional actions to either communicate the emotional state or address its causes. Evolutionary theory suggests that emotions are, at their core, adaptations that motivate action to benefit the organism (Damasio, 2018). This can be as simple as the fear that motivates an individual to run from danger, or as complex as the sympathy and guilt that motivates an individual to donate to charity. In this way, emotion is closely tied to action and perception.

Yet a major factor that makes human actions so complex and variable is the existence of subjective awareness – the ability to be consciously aware of one’s own emotional experiences. Subjective awareness allows humans to consciously control how emotions are expressed through action, resulting in greater variability of emotional expressions, as well as variability in how emotions influence actions and decision-making (Moors and Fischer, 2019). More nuanced awareness of one’s own emotions facilitates the more effective selection of appropriate emotional responses, but also more accurate perception of others' emotional actions. Yet this relationship is unlikely to be unidirectional – the imitation and production of emotional actions during development also seems to be related to developing more sophisticated and nuanced emotion concepts, leading to greater awareness of the emotions of the self and other.

It is increasingly argued that awareness of one’s own emotions is variable across individuals, and this variability has socioemotional and clinical implications (Smidt and Suvak, 2015). This variability is best understood through the constructed theory of emotion (Barrett, 2017), which posits that emotions are not categorical ‘natural kind’ experiences with limited associated action plans, but dimensional experiences constructed through consolidating bodily sensations with contextual information and prior learning. Subsequently, individual variation in emotional self-awareness can be attributed to sensitivity to bodily sensations, as well as individual and cultural learning. In this way, more fine-tuned interoception – the sense of the emotional state of the body – is associated with more nuanced awareness of one’s own emotions (Craig, 2003).

The association between interoception and emotional self-awareness has been demonstrated by empirical studies examining accuracy in perceiving one’s own heartbeats. More accurate heartbeat perception is associated with greater ability to identify and describe one’s own emotions in adults (Herbert et al., 2011) and children (Koch and Pollatos, 2014), as well as in people with autism spectrum disorder (Shah et al., 2016). This association has also been replicated with other measures of interoceptive sensitivity, such as the ability to discriminate between similar levels of muscular strain (Murphy et al., 2018). Intervention work targeting interoceptive abilities has also been found to improve emotional self-awareness in healthy participants (Bornemann and Singer, 2017), suggesting a causal nature to this relationship. Such findings illustrate both that emotional experiences are inherently embodied, and that degree of conscious awareness of these experiences is dependent upon sensitivity to physical sensation.

Given the importance of physical sensation to the conscious experience of emotion, it would follow that motor action and its kinaesthetic feedback are likewise important. Most of the work on action and emotion has focused on how motor actions relate to the perception of emotion in others. For instance, imitating viewed facial expressions facilitates faster and more accurate emotional recognition (Wood et al., 2016). Social learning models suggest that mimicking others’ actions allows us to share in their subjective experience, and mimicry early in life facilitates development of an understanding of how actions relate to subjective experiences (Decety and Meyer, 2008). In support of this argument, Niedenthal et al. (2012) report that pacifier use in male infants is associated with lower emotional intelligence later in life, through inhibiting infants’ abilities to mimic the facial expressions of others, thus limiting opportunities for social and emotional learning.

Cross-sectional research has likewise found associations between emotional self-awareness and emotional actions. Poorer emotional self-awareness is associated with diminished ability to imitate and spontaneously produce emotional facial expressions (Trinkler et al., 2017), as well as lower expressivity in social and non-social situations (Wagner and Lee, 2008). Such findings indicate how conscious awareness of one’s own emotions facilitates more diverse and effective emotional communication. This is likely as subjectivity facilitates more refined conscious control of emotional actions.

The association between subjective awareness of emotions and conscious control of emotions can be seen in emotional regulation research. Identifying emotions is considered a fundamental step in effective emotional regulation (Gross, 2015). Furthermore, more differentiated awareness of emotions is associated with more frequent emotional regulation (Barrett et al., 2001). Greater emotional awareness and regulation has also been found to be associated with greater social success (Kimhy et al., 2016), as this allows for individuals to select more appropriate emotional actions. Likewise, difficulties interpreting and imitating motor actions is associated with greater emotional self-awareness difficulties (Brezis et al., 2017).

The subjectivity of emotion facilitates the intentional cognitive control of emotional actions, which in turn regulates emotional experiences. Moreover, this subjectivity leads to a wide diversity in emotional actions and expressions – the same emotion can result in many different actions, and the same action may be associated with many different emotions. This variability is reflected in diversity of emotional language, particularly in the frequency and diversity of emotion terms relating to action.

The relationship between emotion and action can also be seen on the neural level. The next section further details how these relations can be seen in the brain, before further discussing its relation to emotional regulation and psychopathology.

4. Brain bases for emotional communication

Ample neuroscientific evidence in monkeys and in humans has shown that the cortical sensorimotor regions, specifically the premotor and parietal cortices, are involved in emotional communication (Sato et al., 2015; Trautmann-Lengsfeld et al., 2013). These studies were inspired by the discovery of mirror neurons in monkeys. Single-unit recording studies in monkeys revealed that specific neurons of the ventral premotor cortex discharge both when the monkey executes specific hand actions and when it observes experimenters performing similar actions (di Pellegrino et al., 1992). These neurons have been named mirror neurons (Gallese et al., 1996; Rizzolatti et al., 1996). Later, mirror neurons were also found in the parietal cortices (Fogassi et al., 2005). As the superior temporal sulcus (and its adjacent temporal regions) contains neurons that respond during the observation of actions (Perrett et al., 1985), this region is thought to provide input to the mirror neurons in the premotor and parietal regions. Some researchers have proposed that these regions constitute a functional network, as the mirror neuron system, and are involved in important social cognitive functions, such as imitation and intention understanding (e.g., Williams et al., 2001). Hamilton (2008) proposed that the superior temporal region, parietal region, and inferior frontal gyrus represent the visual, goal, and motor features, respectively.

Direct evidence from a single-unit recording study in monkeys revealed that the neurons in the ventral premotor cortex discharge during observation of emotional facial communication, such as lip smacking (Ferrari et al., 2003). Several neuroimaging studies using functional magnetic resonance imaging (fMRI) in humans have confirmed the involvement of the premotor or parietal cortex in the processing of dynamic emotional facial expressions (Arsalidou et al., 2011; LaBar et al., 2003; Sato et al., 2004; Schultz and Pilz, 2009; Trautmann et al., 2009).

For example, in one fMRI study (Sato et al., 2004), brain activity was measured during observation of dynamic facial expressions, static expressions, and dynamic mosaic images. The results revealed that certain regions in the mirror neuron system, including the inferior frontal gyrus (the human homologue of the ventral premotor cortex in the monkey; Rizzolatti and Arbib, 1998), inferior parietal lobule, and superior temporal sulcus, were more active in response to dynamic facial expressions than to static expressions and dynamic mosaics.

Some other studies showed that observation of dynamic bodily gestures also activated the premotor or parietal cortex (Grèzes et al., 2007; Kret et al., 2011). Electroencephalography and magnetoencephalography studies have supported the rapid activation of premotor and parietal regions in response to dynamic emotion expressions. For example, an electroencephalography study reported activation of the inferior frontal gyrus within 200–300 ms in response to dynamic emotional (happy and disgusted) facial expressions, compared with dynamic neutral expressions (Trautmann-Lengsfeld et al., 2013).

Neuroimaging studies have suggested that activity in the mirror neuron system regions during observation of dynamic emotional expressions is related to the matching of observation and execution of actions (Carr et al., 2003; Hennenlotter et al., 2005; Kircher et al., 2013; Leslie et al., 2004; Likowski et al., 2012; van der Gaag et al., 2007). For example, Likowski et al. (2012) measured facial electromyography and fMRI simultaneously during observation of dynamic emotional facial expressions and found a positive association between facial muscle activity and activity in certain brain regions, including the inferior frontal gyrus. Hennenlotter et al. (2005) evaluated common patterns among brain regions in the observation and execution of smiling facial expressions and found shared activation in brain regions, such as the inferior frontal gyrus

Theoretical and empirical studies have explored the functional networking patterns of brain regions in the mirror neuron system during emotional communication. Hamilton (2008) proposed that the superior temporal region, parietal region, and inferior frontal gyrus represent the visual, goal, and motor features, respectively. In this model, mimicry can be implemented by direct connectivity from the superior temporal gyrus region to the inferior frontal gyrus, and goal-directed imitation can be accomplished by connectivity among the superior temporal sulcus region, parietal region, and inferior frontal gyrus. Sato et al.’s (2012, 2015) fMRI and magnetoencephalography studies applied dynamic causal modelling analysis to brain activity data obtained during observation of dynamic facial expressions versus dynamic mosaic images and found that the optimal model accounting for the data involved bidirectional (feedforward and feedback) modulatory connectivity between the superior temporal sulcus region and inferior frontal gyrus, which was accomplished as early as 200 ms after stimulus onset. Engelen et al.’s (2018) combined stimulation and fMRI study revealed that that the inferior parietal lobule communicates with the premotor cortex, as well as a number of other regions, including the amygdala, when processing the emotional content of actions. In short, these data suggest that the premotor and parietal cortices are involved in emotional communication and are possibly responsible for the matching observations with execution of emotional actions.

Beyond these sensorimotor regions, substantial neuroscientific evidence indicates an extended cortical and subcortical network, including the amygdala, insula, anterior cingulate gyrus, and orbitofrontal cortex (OFC). The evidence further suggests that, as in the case of the cortical mirror neuron system, these regions can be activated by mirroring actions (see meta-analysis by Molenberghs et al., 2012a).

The amygdala has been consistently implicated in the recognition, and experience, of emotion from faces, voices and bodies (Schirmer and Adolphs, 2017) and forms part of a neural network enabling context-appropriate social behaviours (Adolphs, 2010). Mimicking smiles has been linked to activity in the amygdala, as well as the striatum (Lee et al., 2006; Schilbach et al., 2008) and the amygdala is a key component of the Simulation of Smiles model, in which embodied simulation can be used to understand different types of smiles (Niedenthal et al., 2010). In humans, the amygdala and motor-related areas are coactivated when perceiving emotions (e.g., Van den Stock et al., 2011). In addition, there are direct pathways between the amygdala and cortical motor areas, linked to emotion-related brain structures (such as the STS and OFC) involved in emotional communication (Grèzes et al., 2014).

The insula is activated both in the experience of disgust (as evoked by unpleasant odours) and the observation of disgusted facial expressions (Wicker et al., 2003). Weakened spontaneous expressions of disgust in response to odours (Hayes et al., 2009a), and reduced ability to voluntarily pose disgusted expressions (Hayes et al., 2009b) and imitate at least some basic facial expressions (Trinkler et al., 2011) has also been reported in patients with Huntington’s disease, which is associated with a loss of volume in key social network structures, including the amygdala and insular cortex (Kordsachia et al., 2017). More recently, Braadbaart et al. (2014) have argued that the insula plays an important role in learning facial expressions, which would make this structure sensitive to mismatches between observed and imitated facial expressions.

Empathy has been argued to be based on an action-perception mechanism, with ‘affective’ empathy, such as mimicry and emotional contagion (which is likely to be shared across species) likely to be associated with neural systems involved in sensation, movement and emotion (i.e., premotor-parietal, temporal and subcortical regions; Ferrari and Coudé, 2018). Meta-analysis has shown that the anterior insular cortex, along with medial and anterior cingulate cortex are involved in empathy for pain and the direct experience of pain (Lamm et al., 2011). Along similar lines, a recent study used fMRI to compare representations of self and others' expressions of pain and found selectively greater activity for ‘self’ pain-related stimuli in the anterior mid-cingulate cortex, a region critical for pain perception and recognition (Benuzzi et al., 2018). Interestingly, areas of the insula and amygdala were more active during emotional expressions from a mother’s own child than another child, and brain responses were correlated with an indirect measure of empathy (Lenzi et al., 2009).

One way to recognise emotions in others might be to internally simulate the emotional state (e.g., Decety and Chaminade, 2003; Gallese, 2003; Goldman and Sripada, 2005; Keysers and Gazzola, 2007; Niedenthal et al., 2010; Winkielman, 2010). Simulation could occur relatively automatically and involve neural substrates that were activated for both recognition and experience (Heberlein and Atkinson, 2009). Simulation might be more important for understanding dynamic, ambiguous expressions than prototypical ones (Niedenthal and Ric, 2017; Rychlowska et al., 2014; Sato and Yoshikawa, 2007) and might also be more effective for some people than others (Hess and Fischer, 2014; Niedenthal and Ric, 2017). Heberlein and Atkinson (2009) suggest that evidence is consistent with shared substrates for emotion recognition and experience in the amygdala and OFC, but less clearly consistent with a simulation model (c.f. somatosensory cortices). However, by prioritising and enhancing processing emotional information (at least visual, but perhaps also auditory) the amygdala and OFC could be influencing simulation processes in other parts of the network (Heberlein and Atkinson, 2009). Finally, Williams (2013) suggests that systems for goal-directed action are connected with amygdala-orbitofrontal circuits central to emotional learning.

5. The hierarchical organization of motor control and emotional regulation

For animals with a limited range of stereotyped behavioural responses to either rewarding or aversive conditions, the relationship between conditioning and sensorimotor systems can be made relatively easily. However, in humans, the expression and communication of emotional responses is highly flexible and ever-evolving in relation to cultural demands. This requires another level of control over action execution which is intentional and self-aware. In this section we consider two actions. First, we consider a monkey in a motor learning experiment, moving a robotic arm to guide a cursor onto a target on the computer screen to acquire a juice reward. Secondly, we consider a parent witnessing her child stepping dangerously out into the road. Her initial reaction is to generate and exhibit fear, but she realises this reaction might scare the child and make things worse. Therefore, some cognitive control involving self-awareness is likely to moderate the initial sensorimotor response.

As discussed in the introduction, for any action, we plan (goal setting), execute the plan (action), and adjust the action according to feedback (error detection and learning) until the desired outcome is achieved. In the motor learning experiment, where macaque monkeys move a robotic arm to acquire a juice reward, the monkeys manage this easily, achieving optimal performance within just a few tens of trials of training. A clockwise force field is then turned on, and the reach trajectory is perturbed in the force direction such that the monkeys fail to reach the target in time. Sensing the change, the monkeys learn to adapt to the force field and again in a few tens of trials, can accomplish the task – the movement trajectory becomes straight and dynamics reflects an optimal profile. The monkeys appear to have learned to reset the movement strategy to accomplish the goal. If at this point the force field is removed, movement trajectories are once again deviated, and the monkeys start another adaptation cycle to accommodate the perturbation.

Neuronal activities have been recorded from motor cortical structures in these experiments (Li et al., 2001; Padoa-Schioppa et al., 2002, 2004). In the motor cortex, ensemble neuronal activities encode the target direction and movement synergy and these activities are aligned when no force field is present. When a force field is turned on, the ensemble activities initially align with the movement kinematics (target location) but gradually change to reflect the dynamics rather than the desired kinematics of the upcoming movement. Thus, the neuronal activities reflect a sensorimotor or kinematics-dynamics transformation to meet the desired goal. This simple example highlights the core component processes of motor control: goal (to reach the target; specified in kinematics), movement (muscle synergy required to execute the movement or dynamics), error detection and post-error learning behavioural adjustment.

There are a few issues worth considering from these motor learning studies. First, movement control is hierarchical only in the sense that the sequence of events unfolds in time but not that the processes of control are unamenable to change. Second, when the environment is stable, hierarchical interaction is established in favour of goal to action translation rather than outcome monitoring and goal resetting. As goal action translation becomes most expedient, a habit is formed and the behavioural contingency may transpire without awareness. Third, much of neuroscience research have focused on understanding the neural processes subserving the “linear” chain of command and less is known about the mechanisms serving how the outcome resets the goal.

In this example of motor control the goal is clear – monkeys must reach the target in order to obtain juice reward. Whereas this applies to many of the actions one routinely performs, social and emotional communication is a different matter. Then there are conflicting goals and behaviour needs to be optimised to meet these goals. If we now consider the example in which a parent whose child is about to step dangerously out into the road, she faces a conflict between exhibiting a prepotent response and an anticipation of the effects of her behaviour on that of another person, which conflicts with the prepotent response. Therefore, and additional level of cognitive control is employed to moderate the initial reaction.

5.1. Conflict control

Cognitive control facilitates decision making in a changing environment. One of the cardinal features of cognitive control is the ability to learn from the outcome of our actions and revise our knowledge of world and action plans accordingly. This ability to learn and change is supported by a brain system that integrates moment-to-moment information into our behavioural repertoire. By exploring the changing environment, individuals strengthen behavioural routines that lead to positive outcomes and revise those in association with negative consequences. Cognitive control is particularly critical in the face of conflicting goals.

In the laboratory, investigators have combined brain imaging or electrophysiological recording and a variety of behavioural tests (e,g., go/no-go; Simon, Stroop flankers; stop signal task) to examine the neural processes of cognitive control. Here we use recent studies of the stop signal task (SST) as an example to highlight the neural circuits of cognitive control and the interactive nature of regional processes to support optimal performance in the face of conflicting goals.

In the SST, a frequent “go” signal instructs participants to quickly respond (by pressing a button) and an occasional “stop” signal (1/3 or 1/4 in frequency) instructs participants to withhold the response. The stop signal follows the go signal and the time interval – stop signal delay (SSD) – determines how difficult it is for participants to withhold the response. With a long SSD, the motor command to respond has likely been relayed to the muscle and reached a “point of no return” (Logan, 2015), and a stop error ensues. In a typical SST experiment, the SSD is adjusted stop-trial by stop-trial either pseudo-randomly or following a staircase procedure, so that participants achieve success in only half of the stop-trials. There are two main reasons in manipulating this variable in the SST. First, it would allow the computation of the stop signal reaction time – the time needed to stop a motor response – using the race model (Logan et al., 1984). Second, there will be a sufficient number of error trials, so one can examine the neural processes underlying error detection and post-error behavioural adjustment (Chang et al., 2014; Ide and Li, 2011; Li et al., 2008a, b). Participants are confronted with two conflicting goals in the SST – a speeded response to the go-signal to meet a time window and a cautious act so the response can be withheld when the stop-signal appears. As a result of these conflicting goals, participants typically fluctuate in go-trial reaction time (RT) and slow down after committing a stop-error – an observation termed post-error slowing. That is, compared to a go-trial that follows another go-trial, the go-trial that follows a stop-error (or stop success) trial is prolonged in RT. There are different accounts of why participants slow down following a conflict (stop-trial), including diversion of attention (Van der Borght et al., 2016) and conflict-elicited control. Without going into the details of the debate, here we elaborate on the behavioural and neural evidence in support of conflict-elicited cognitive control.

In a series of studies, investigators posited that, because stop-trial occurs randomly but influences go-trial RT, it is possible that individuals track the occurrences of stop-trial and slow down in response when they anticipate a stop signal. Using a Bayesian model, Yu and colleagues estimated the trial by trial likelihood of stop signal or P (Stop) and showed that a higher P(Stop) is associated with prolonged go-trial RT – an observation termed “sequential effect.” (Yu and Cohen, 2008). That is, participants proactively prolong the response if they anticipate that a stop signal will occur. This provides a strategy to negotiate the conflicting goals between speedy and cautious go-responses.

Combining fMRI and the Bayesian model of SST performance, studies have delineated the neural correlates of conflict anticipation, RT slowing, and unsigned prediction error or the absolute discrepancy between anticipated and actual outcome – a surprise signal. Regional activities in response to P(Stop) are located in the anterior pre-supplementary motor area (pre-SMA) and bilateral inferior parietal cortices (Hu et al., 2015). RT slowing engages the posterior pre-SMA and bilateral anterior insula, the latter of which has an acknowledged role in conflict awareness (Ullsperger et al., 2010). Importantly, using a Granger causality analysis, investigators are able to demonstrate directional influence of P(Stop) on RT activities. An event-related potential study in combination with source reconstruction confirmed these findings (Chang et al., 2017). Thus, these studies together support proactive control of motor response in the SST. Further, a distinct area of the medial prefrontal cortex (mPFC) – in the dorsal anterior cingulate cortex – responds to unsigned prediction error, highlighting the functional heterogeneity of the mPFC (Hu et al., 2015; Ide et al., 2013). An important question pertaining to the control hierarchy concerns the roles of prediction error signal in driving SST performance and remains unanswered.

The aforementioned studies describe how we proactively control motor response in anticipation of conflicting goals. A complementary process of cognitive control is the reaction evoked by an infrequent, behaviourally relevant stimulus. In the SST, the stop signal appears infrequently and is highly relevant, as it instructs an interruption of the motor command. In imaging studies of the SST this issue is commonly addressed by computing the stop signal reaction time (SSRT), as estimated from the race model, and identifying its neural correlates on the basis of between-subject analyses. A circuit involving right inferior frontal cortex, anterior pre-SMA and subcortical structures including the caudate nucleus has been identified in supporting reactive response inhibition (Cai et al., 2017; Duann et al., 2009; Li et al., 2006). Brain regions within this circuit interact to respond to the stop signal and interrupt the motor action. RT slowing following an error can therefore also be conceived in terms of a reactive process. That is, error signals may engage cerebral processes of control and prolongs RT in the next trial.

Earlier imaging studies demonstrated that a cortical-thalamic-cerebellar-cortical circuit, in congruence with known anatomical connectivity, supports the reactive response of post-error slowing (Hendrick et al., 2010; Ide and Li, 2011; Li et al., 2008b). Thus, there are both reactive and proactive processes that subserve the hierarchy of cognitive control. In contrast with motor control, which we illustrate with a force field learning experiment in macaque monkeys, cognitive control requires constant outcome monitoring and resolving conflicting goals, and engages multiple loops of reactive and proactive control circuits. The hierarchical nature of cognitive control can only be meaningfully considered with this complexity in mind.

A few issues can be considered in contrasting cognitive and motor control (though we might equally refer to high level and low level motor control). First, motor control often comes with a clearly set goal whereas cognitive control is demanded in situations where multiple goals are in place and often in conflict. Second, with a set goal, motor control may become a routine with repeated practice or a habit that is “closed-loop” and expressed without concomitant awareness. Cognitive control, in contrast, is often engaged to override a habit and requires active monitoring (and awareness) of performance to be effective. Finally, these control mechanisms involve very distinct circuits even when the same sensory (input) and motor (output) modalities are engaged. These mechanisms exist across primate species and similar mechanism located in the dorsal aspect of the anterior cingulate and prefrontal cortex, show many similarities in serving cognitive control (Mansouri et al., 2017).

5.2. Hierarchical control of complex actions

One of the other differences between the monkey and the parent in our examples is the involvement of other psychological processes such as the recall of several behavioural rules and conventions the mother may have learnt. An important aspect of motor control is the way that a set of actions and rules can be integrated to determine a coherent response, by incorporating abstract concepts into the organisation of response. Importantly, studies have converged to suggest a hierarchical organization or rostrocaudal gradient in the frontal cortex with the rostral and caudal regions respectively supporting more abstract and concrete representations (Azuar et al., 2014; Badre and D’Esposito, 2009; Badre et al., 2008; Koechlin et al., 2003). It is suggested that more rostral regions may be critical for progressively later stages of perception and action (Fuster and Bressler, 2012).

This complexity of cognitive functional organisation has also been construed in terms of the time scale of activities. It is posited that the frontal cortex embodies a rostro-caudal hierarchy that is sensitive to different time scales of environmental dynamics, with caudal and rostral regions each engaging faster (shorter time scale) and slower dynamics (Badre, 2008; Botvinick, 2008; Fuster, 2004; Koechlin and Hyafil, 2007; Koechlin et al., 2003; see, however, Zhang and Rowe, 2015). The time scale of activities has also been explored for neural circuits beyond the prefrontal cortex. For instance, by creating distinct narratives with word changes while preserving the grammatical structure across stories, investigators reported different neural responses between the stories that gradually increased along the hierarchy of processing timescales (Yeshurun et al., 2017). In early perceptual auditory cortex the differences in neural responses between stories were relatively small. In contrast, in areas with the longest integration windows, such as the precuneus, temporal parietal junction, and medial frontal cortices, there were large differences in neural responses between stories. Further, this gradual increase in neural differences between the stories was correlated with an area's ability to integrate information over time. These findings suggest that hierarchical control of complex mental act may unfold according to the temporal scales at which component processes take place.

5.3. Valence

Whether actions are favoured or discouraged depends upon learning systems which attribute a valence to their outcomes. Encountered environmental stimuli are encoded with positive or negative valence and mediate behavioural changes accordingly, with changes being encoded in hypothalamic nuclei, which also mediate neuroendocrine stress responses (Kim et al., 2019). According to traditional learning theory, when valence is attributed to environmental stimuli, associated behaviours are either reinforced or punished, meaning that they either increase or decrease. A range of psychological models propose systems that explain how these systems might work in humans. (e.g., Davidson, 1992, 2000; Davidson et al., 2002; Gray, 1982, 1987; Lang and Bradley, 2010). Neurologically based approach and avoidance systems are thought to mediate emotional sensitivity, personality, positive and negative affective experiences, and goal- directed behaviour (e.g., Davidson, 1998; Fowles, 1988; Lang and Bradley, 2010; Laricchiuta, 2015; McNaughton and Corr, 2014).

Gray’s (1982) early two-system model of motivation proposed a behaviour inhibition system (BIS) and a behaviour activation system (BAS). BIS activation is sensitive to anticipation of threatening stimuli and inhibiting aversive outcomes, and responsible for regulating negative feelings such as anxiety and fear. The BAS is sensitive to anticipation of reward and approaching appetitive experiences, and responsible for regulating positive feelings such as hope, elation, happiness. Within Gray’s model, positive affect (PA) and negative affect (NA) are viewed as state manifestations of underlying regulatory reward-driven and punishment-driven motivational systems. The BIS is thought to primarily involve serotonergic and noradrenergic pathways (Gray, 1994), whereas, the BAS is thought to be mediated by dopaminergic pathways (Depue and Iacono, 1989).

Self-report motivational scales designed to assess motivational systems such as the BIS/BAS are not direct measures of motivation or underlying neurophysiological activation. Ongoing integrative research investigating neurological activation, explicit (effortful, awareness) and implicit (autonomous, spontaneous) motivational, cognitive and affective processes is required to better understand motivated action. A more recent theoretical development by McNaughton and Corr (2014) distinguishes underlying independent motivational systems from more surface level behaviours based on approach and avoidance interactions that may lead to the activation of approach-avoidance conflicts. Their model purports that more surface level behaviour may be determined interactively, even when the underlying approach and avoidance motivational systems are independent (Corr, 2013).

5.4. Self-regulation and being regulated by others

Further models place the valence systems within a context of emotion regulation. Higgin’s Regulatory Focus Theory (RFT; Higgins, 2000) and Carver and Scheier’s (1998, 2001) self-regulatory models provide a theoretical framework for investigating the interface between motivational, cognitive and affective systems involved in goal-directed action and emotion (Higgins, 2000; Carver and Scheier, 1998, 2001).

Rooted in self-regulatory motivational sensitivities, approach and avoidance goal striving actions represent sustained activity towards desirable outcomes and away from undesirable outcomes, respectively. Goal-directed action is guided by the process of ongoing self-regulation that modulates an individual’s thoughts, affect and attention (e.g., Dickson et al., 2017; Winch et al., 2015). In sum, a two-system view of motivation has persisted over time, even though different labels have been put forward to define approach and avoidance systems. Approach-and avoidance-oriented actions and emotional sensitivities in response to rewarding or threatening stimuli are seen as rooted in specific neurological brain systems (Gentry et al., 2016; Steinman et al., 2018). Laricchiuta (2015) posits that brain networks are implicated in instigating approach and avoidance behaviours in reaction to salient stimuli. Such networks include cerebral nodes interconnected as prefrontal cortex, amygdala, hypothalamus, striatum and cerebellum.

There is also evidence that the dopaminergic system and interconnected brain regions process positive and negative stimuli to reinforce approach and avoidance behaviours (Gentry et al., 2018). Although sensorimotor reactions to appetitive or aversive stimuli are typically spontaneous and automatic, goal-directed conflict, lack of goal progress or unpleasant emotions may stimulate reflective awareness, goal planning and more effortful cognitive control. McNaughton and Corr (2014) draw an important distinction between underlying orthogonal motivational systems and possible approach and avoidance interactive surface level behavioural conflicts.

A key aspect to emotional regulation is the capacity to be regulated by others, whether during childhood by adults or by peers. This requires bridges to be built between codings for one’s own emotion-action states and those of others. We are able to do this by generating sensory changes in our own body state to identify how someone else is feeling (Craig, 2003; Seth, 2013), a process controlled by the somatosensory and prefrontal cortices (Adolphs et al., 2000; de Gelder, 2006; Hornak et al., 2003; Radice-Neumann et al., 2007). Although these internally generated emotional responses generally lack intentional control and awareness, they significantly impact our recognition of nonverbal emotion cues (Naranjo et al., 2011; Neumann et al., 2014). They also reflect our desired outcome for the social interaction we are engaged in and thus modulate our emotional experiences in response to these cues (Naranjo et al., 2011; Soussignan, 2002). Our interoceptive response, desired outcome and ultimate interpretation of the emotional experience are influenced by gender, social roles and culture (Chaplin et al., 2005; Fischer et al., 2004). As outlined in the embodied-contextual model of emotion, our interpretation of others’ feelings is further mediated by previous experience and the environmental context in which the interaction took place (Barrett, 2017; Eder, 2017).

The influence of context becomes more apparent as we develop and gain more sophisticated cognitive skills. With increased cognition, we learn that an emotion expression may have multiple (and often conflicting) meanings depending on the context in which it is produced. In response, we learn to rely upon our prior experience and memories to accurately interpret and respond to the emotion expressions of others (Boone and Cunningham, 1998; Buck, 1991; de Gelder, 2006). Thus, recognising and appropriately responding to emotion operates as part of a feedback system, one in which our analysis of the actions and movements of others as well as our own internally generated sensory changes, leads to learning. The responses we receive during social interactions provide feedback and guide our future behaviour – if the response is a rewarding one, we are more likely to behave similarly in future social interactions, but if the response is a punishing one, we will learn to adjust our behaviour to pursue a more positive emotional outcome (Baumeister et al., 2007; Gendolla, 2017). We are constantly appraising the meaning of the interaction and modifying our emotional actions in response (Ridderinkhof, 2017). We then enact cognitive and motor control to guide future responses in similar contexts through goal striving actions that result in an (usually desirable) outcome (Griffiths et al., 2014; Higgins, 2000). This learning should ultimately contribute to conscious adaptation that leads us to choose actions that are appropriate within a social context (Baumeister et al., 2007).

It is therefore evidence that the 'regulation' of emotion is directly concerned with learning patterns of behavioural responses to environmental stimuli such that they minimise the experience of negative valence and maximise the positive. This requires ongoing and iterative motor learning and conditioning. Emotion regulation for humans, involves constant appraisal and reappraisal through explicit or implicit regulatory processes (Braunstein et al., 2017). People draw from a large number of different strategies in the service of regulating their emotions (Heiy and Cheavens, 2014), but the neural correlates of emotion regulation have been studied primarily through fMRI studies of reappraisal (the cognitive reinterpretation of emotionally evocative events), and sometimes distraction or expressive suppression (Etkin et al., 2015; Frank et al., 2014).

Broadly speaking, explicit emotion regulation through reappraisal recruits frontal cognitive control regions of the brain, including regions involved in sensorimotor control, with concomitant changes in subcortical regions, including the amygdala and ventral striatum (Ochsner et al., 2012). A consistent finding across meta-analyses is that down-regulation of emotions (particularly through reappraisal) recruits the dorsolateral prefrontal cortex, ventrolateral prefrontal cortex, and anterior cingulate cortex (Buhle et al., 2014; Etkin et al., 2015; Frank et al., 2014; Kohn et al., 2014).

These findings mesh with psychological models of the process of emotion regulation whereby reappraisal involves working memory and selective attention to generate and maintain the reappraisal representation, inhibition to prevent prepotent responses, and monitoring to assess the effectiveness of the reappraisal response (e.g., Ochsner et al., 2012). For example, Kohn et al. (2014) note that activation of the ventrolateral prefrontal cortex also occurs during emotion generation and appraisal, and, as such may reflect emotional salience as well as the operation of regulatory processes like inhibition. In addition, the anterior middle cingulate cortex has been described as a limbic motor control region, involved in controlling motor responses in situations of reward and punishment (Kohn et al., 2014). In tandem with these regions of activation, explicit down-regulation of negative emotions involves reduced activity in the amygdala, known for signalling the presence of emotionally-arousing stimuli, and the ventral striatum, known for representing the reward value of stimuli. Other regions may also be implicated in explicit emotion regulation. For example, the supplementary motor area, which also is active during emotional mimicry tasks and mental imagery studies, and plays a role in preparatory motor movement, is noted to be active in up-regulation and down-regulation of emotions (Etkin et al., 2015; Frank et al., 2014; Kohn et al., 2014).

6. Development, psychopathology and disordered states

Due to its deep evolutionary roots, our ability to perceive emotion begins early in life and is thought to be an automatic and spontaneous component of social interaction (Boone and Cunningham, 1998) connected to early mimicry (Decety and Meyer, 2008). A recent study showed that newborns appear to be sensitive to dynamic faces expressing emotions at birth (Addabbo et al., 2018). Infants between five and seven months of age start to preferentially attend to fearful faces rather than happy faces, and disengage attention less readily from fearful faces, than from happy or neutral faces (Hoehl, 2014). The various neurological structures (e.g., prefrontal cortex, amygdala) that control our capacity to recognise and express emotions continue to develop throughout childhood and adolescence, allowing us to perceive more nuanced and subtle differences in the emotions expressed by others (Herba and Phillips, 2004; Thomas et al., 2007). Hence, the accuracy in which we can differentiate between emotions is mediated by both age and gender, with female children and adolescents showing more accurate perception than males (Herba et al., 2006; Lawrence et al., 2015; McClure, 2000). Cognition is also strongly associated with recognising how someone else is feeling (Lawrence et al., 2015; Thomas et al., 2007), likely because accurate perception requires simultaneous processing and integration of many different cues.

Our choice of actions within the emotional regulation system can be significantly affected by experience. For instance, studies with infants and children who have been abused have shown that this aberrant social experience alters their perception of facial and bodily movements indicative of anger (Pollak et al., 2000; Pollak and Kistler, 2002; Pollak and Sinha, 2002). Specifically, Pollak and his colleagues found children who were abused to be more in-tune with nonverbal expressions of anger in their environment. Although this may be an adaptive response reflecting a desire to avoid punishing responses in future interactions, results of these studies additionally indicate that children who are abused may attribute anger to expressions intended to elicit a more sympathetic or positive response. These results indicate that children who are abused may view even rewarding responses as punishing ones, and thus respond negatively or withdraw from the interaction, thereby minimising the frequency and range of social interactions they have with others.

Breakdowns in the feedback system are seen in many neurological populations where the neuroanatomical circuitry necessary for recognising, analysing, and responding to the facial and bodily movements of others has been damaged. For instance, people with traumatic brain injury (TBI) who commonly experience damage to the prefrontal cortex, limbic system, and parietal cortex, have been shown to have poor social outcomes due to the difficulty they have understanding and identifying their own emotions (i.e., alexithymia; Henry et al., 2006; Neumann et al., 2014; Williams and Wood, 2010) as well as difficulties in recognising, interpreting, and accurately responding to the nonverbal emotional expressions of others (Babbage et al., 2011; McDonald, 2005; Milders et al., 2003; Neumann et al., 2012; Zupan et al., 2014, 2016). Patients with Parkinson’s Disease, resulting in damage to the basal ganglia, show impaired facial expression recognition, which is linked with voluntary control of facial muscles (Gray and Tickle-Degnen, 2010; Marneweck et al., 2014). Disruption of amygdala-cortical pathways, such as in autism spectrum disorder (Gotts et al., 2012) or amyotrophic lateral sclerosis (ALS, Passamonti et al., 2013), may also affect emotional perception and social interaction.

Another way that brain disorder impacts upon the action-emotion relationship is to diminish flexibility. The diversity, flexibility and range of action seems to be diminished in psychopathological conditions like schizophrenia, autism spectrum disorder and obsessive compulsive disorder, where behaviours are often quite inflexible and stereotyped, and the outward expression of emotion quite fixed.

Schizophrenia for example, is well characterised by negative symptoms including flat or blunted affect, emotional withdrawal and apathy. Reduced emotional expressivity in the context of intact subjective emotional experience (Kring and Moran, 2008) has led some to conceptualise the symptom of blunted affect in schizophrenia as reflecting or mirroring abnormality, given the previously described role of the motor system in the physical action of emotion expression and the simulation of others’ emotive states (Gaebel and Wolwer, 1992). Several different studies show mirror neuron disturbances in schizophrenia (Enticott et al., 2008a; Mehta et al., 2014a), which directly correlate with negative symptoms such as affective blunting, anhedonia, avolition and alogia; as well performance on facial emotion processing tasks (Kohler et al., 2003, 2010; Lee et al., 2014; Turetsky et al., 2007).

Deficits in facial affect processing are also core to the social cognitive profile of schizophrenia (Kring and Elis, 2013), and are consistently associated with reduced recruitment of a neural network encompassing limbic and prefrontal areas including the mirror neuron enriched inferior frontal gyrus, as well as regions in the occipital and temporal cortex (Gur et al., 2007, 2002; Kilner et al., 2009; Leitman et al., 2011; Taylor et al., 2012). These widespread neural abnormalities and associated behavioural deficits appear to reflect disruption in the activity and integration of several systems involved in general face perception, motor behaviour and emotional states (Eimer et al., 2011; McCleery et al., 2015; Rossell et al., 2014; Taylor et al., 2012; Van Rheenen et al., 2017). Relevantly, mirror neuron-related motor system abnormalities in schizophrenia may result in an inability to adequately mimic and recognise the emotional expressions of others, and thus the extent to which their emotional state can be internally simulated (Enticott et al., 2008b; Haker and Rössler, 2009).

Mirror neuron disturbances have been directly linked to poor theory of mind in schizophrenia (Mehta et al., 2014b). However, it is possible that this relationship is somehow mediated by other top-down abnormalities as the mirror neuron system is likely a predictive system that is activated not only by the visual representation of an action but by its goal or intention (Kilner et al., 2007a; Umiltá et al., 2001). Indeed, mirror neuron activity appears to be moderated by the context in which an action occurs (Liepelt et al., 2009), as well as the biases of the observer (Liepelt and Brass, 2010; Molenberghs et al., 2012b). Thus, intentions inferred by observing actions are biased by prior knowledge, reflecting the outcome of the brain’s attempt to minimise differences between what is observed and what is expected (i.e., a prediction error) (Kilner et al., 2007b; Maranesi et al., 2014; Miall, 2003).

In schizophrenia, aberrant predictive coding may contribute to symptoms by reducing precision for prior expectations, leading to abnormal attentional control over sensory information and altered integration of top-down and bottom-up input (Adams et al., 2013; Stephan et al., 2009; Tschacher et al., 2017). Several studies show abnormal cognitive control of bottom-up emotional experience in schizophrenia, indicating deficient emotion regulation by lateral prefrontal control regions that are consistently hypoactive in the presence of emotionally evocative stimuli, with this hypoactivation perpetuated in patients with affectively relevant negative symptoms, such as alogia, avolition and blunted affect (Anticevic et al., 2012; Dichter et al., 2008; Potkin et al., 2002; Vai et al., 2015). Further, ventrolateral-orbitofrontal cortex activation during emotion processing does not appear to be modulated by context in schizophrenia as it is in healthy individuals, suggesting that patients with schizophrenia do not adequately integrate prefrontal representations of existing knowledge into their evaluations of social stimuli (Leitman et al., 2011).

Indeed, it has been shown that negative symptoms in schizophrenia are associated with an increased tendency to over-weight sensory information relative to prior expectations when making inferences about the social actions of others, which results in an inability to accurately predict others’ intentions (Chambon et al., 2011). On the contrary, when inferring intent during interactions between others and meaningless objects, patients with schizophrenia over-weigh prior expectations over visual (sensory) information, and this top-down bias correlates with the severity of positive symptoms (Chambon et al., 2011). Thus, it appears that non-social situations in schizophrenia invoke heightened conviction in prior beliefs even in the face of contradictory external evidence – a mismatch of which would normally give rise to a prediction error that allows for the readjustment of one’s worldview. This over-reliance on (potentially aberrant) prior beliefs fits with arguments that schizophrenia reflects an impaired separation of one’s own intentions from that of others, resulting in a disconnect between action and free will that gives rise to positive symptoms involving passivity experiences (the misattribution of intentions to non-agents) or paranoia (attributing intent in the absence of any) (Chambon et al., 2011; Frith, 1987).

7. Linguistics

With this understanding of the current state of action research as a backdrop, our team was specifically tasked to review the language that people use to express feelings related to action. To better understand, the range of verbally articulated feelings that are expressed in the English language, a small task team within the Human Affectome Project led a computational linguistics research effort to identify feeling words (Siddharthan et al., 2018).

Results were extracted from the Google n-gram corpus (Younes and Reips, 2019), which includes roughly 8 million books and then manually annotated by more than one hundred researchers from this project. This resulted in 9 proposed categories of feelings and a new affective dataset that identifies 3664 word senses as feelings. Of relevance to this review is a category related to Actions and “Prospects”, which was defined as follows:

“Feelings related to goals, tasks and actions (e.g. purpose, inspired), including feelings related to planning of actions or goals (e.g., ambitious), feelings related to readiness and capacity of planned actions (e.g. ready, daunted), feelings related to levels of arousal, typically involving changes to heart rate, blood pressure, alertness, etc., physical and mental states of calmness and excitement (e.g. relaxed, excited, etc.), feelings related to a person’s approach, progress or unfolding circumstances as it relates to tasks/ goals within the context of the surrounding environment (e.g. organised, overwhelmed, surprised, cautious, etc.), feelings related to prospects (e.g. afraid, anxious, hopeful, tense, etc.).“

This subset of the results included about 1137 feeling words, including 130 words that were judged by individual raters to express AF within the context of planning (251 words). About 48 anticipatory feeling words were exclusively related to feelings of fear and anxiety, whereas about 54 words expressed feelings of optimism, while three smaller clusters of about 10–15 words expressed feelings of hope, suspicion or suspense (see word corpus of the Human Affectome Project). Although it was not within the scope of this effort to undertake a formal analysis of this dataset, we reviewed these feelings words and we attempted to roughly organize the words into discernable categories. The individual word senses and this sorting attempt can be found in the supplemental data accompanying this review. However, a degree of caution should be exercised in the interpretation of this sorting effort, as it was created only to give us an initial sense of how feeling words related to the various stages of sensorimotor function, as shown in Fig. 2.

Fig. 2.

Schema of how feeling words may relate to stages of sensorimotor function. Emotion word categories are considered sub-categories of action stages (see Fig. 1) which may have a social or non-social dimension. In the case of inclination and anticipation, further sub-categories are listed.

From our perspective, these feeling words are interesting, relevant and warrant further study. A significant number of words simply described general levels of arousal (e.g., calm, aroused), but many feeling words were very specific and reflected different aspects of action-related thought. For example, feelings related to the Hierarchy of Goals included having a sense of purpose, immediate physiological needs, social/moral obligations, external influences (e.g., social prodding), the acquisition of resources, competitiveness, sentimentality, and even fate/superstition. In this area, recent research in monkeys has provided new insights about the role of the frontopolar cortex in monitoring the significance of current and alternative goals (Mansouri et al., 2017). Current goal-management models involve arbitration processes between exploitation and exploration behaviours (Donoso et al., 2014) and additional research is needed to determine whether humans may have additional cognitive capacities for the directed exploration of concurrent alternative strategies (Mansouri et al., 2017). So, this roster of articulated feelings which appears to help us better understand the range of goal priorities in humans will be useful when formulating future research.

Other feelings that we reviewed related to Planning and Coordination. This included competing priorities, the degree of creativity needed/employed, decision speed, risk involved, readiness, optimism/pessimism about prospective outcomes, the degree of rationality (e.g. irrational, rational), inclination towards action (e.g., reluctant, undecided, inclined), the degree of caution to be exercised, the level of aggressiveness employed, and assessments of persistence (e.g., resistless, persistent). In neuroscience research, the prefrontal cortex is the primary focus when it comes to planning, executive attention, decision-making, and inhibitory controls (Fuster, 2019), and many of aspects of planning and coordination have been already been subjected to a considerable degree of research. However, the full scope of PFC function is still not well known (see Burgess and Stuss, 2017, for a historical review), so just having an initial inventory of articulated feelings in this area is helpful.

Feelings related to the outcomes of actions, assessed the degree of success, external assistance, luck involved, the predictability of the result and overall acceptability of the outcome. While feelings of suspense reflected unresolved circumstances. Finally, feelings related to outcomes from a personal perspective additionally related freedom, composure, understanding, skill level and power.

A principal observation made during this categorisation exercise was that there are many categorical overlaps. Many words will fit into more than one category because the stages are interdependent to a large degree. A single word sense may describe the goal of an action as well as its motivation, whilst motivation is also dependent upon ability (e.g., knowing that you can do something successfully is a prerequisite to having the drive to do it). Given the hierarchical, feedback-dependent nature of motor control, one may always argue that it is the outcome that is really the goal of on action. For example, feelings of courage in the execution of an action may suggest careful judgment of risk in its planning. A person’s judgement of risk would impact upon motivation. Alternatively, feelings of courage may be considered to reflect that person’s ability as a character trait, or an attribution that is dependent upon judgement by others. Indeed, it may be the goal of the person undertaking these actions to be to be considered courageous by others, and this may subsequently be the outcome of the action.

Similarly, affordance learning theory argues that the properties of an object will shape the action that is enacted towards it. Social affordance theory similarly (Marsh et al., 2009) argues that sensorimotor states are determined by expectations of the consequences of an action. Therefore, a word used to describe the goal of an action may also describe the action’s expected consequences on others, as well as the way that action is performed.

Also, because of the very nature of embodied cognition, action words often reflect emotions or feelings being expressed metaphorically. This is particularly the case for words used to describe how one person might affect another person. Therefore, words that can be used to describe a simple action can also be used to describe the nature of one person’s behaviour towards another. In this respect, words such as ‘snare’ or ‘stifle’, which are used to describe the goal of an action as to have a restricting effect, can also describe social oppositionality.

Another way of explaining this is to consider that some words have evolved to serve actions at a high level of the action organisational hierarchy. As such they have more metaphorical properties and will serve a range of actions, whether they have literal, selfish or social properties. A word such as emancipate would seem to be free from any specific action form but describes the relationship between an action and its consequences.

Nonetheless, we do think that this inventory of English feeling words may have some utility. Fig. 2 is not intended as a final model, and it has not been tied to corresponding neuroanatomy. Rather, it is intended only as an illustration of the ways in which articulated feelings might be related sensorimotor control. Additional research in each of these areas will be needed to determine whether our feelings can be closely tied to these areas of neural function.

8. Conclusions

In the first part of our paper we reviewed the relationship between emotion and the sensorimotor system. We showed that the sensorimotor system is intrinsic to the communication of emotional states between individuals, and how the nature of this communication develops and evolves from expression of simple emotional states to much more complex communication at a more abstract level and higher hierarchical level of organisation. Sensorimotor control mechanisms regulating emotional states have evolved from serving simple allostatic functions through selection of stereotyped actions and optimised motor performance in most non-human animals, to having a capacity for highly flexible, novel and creative responses, drawing upon a wide range of cultural and physical influences, operating across differing temporal scales, to satisfy conflicting social and appetitive goals. The sensorimotor system of representation is key as part of appraisal and reappraisal systems that modulate action and emotional awareness. In the second part of this paper, we reviewed the process creating a linguistic framework that categorises feeling words within the context of these appraisal and reappraisal systems.

Considering the more granular array of feeling words that were generated in this project, a large proportion of the words in the original list were considered to describe feelings related to actions. This is unsurprising given that embodied cognition theory argues that feelings and emotions are intrinsic to the sensorimotor states that are used to communicate them. We found that further categorisation of words according to action stage was an exercise with limited value with respect to obtaining a neat and reliable classification but was of more value for learning the reasons why this was the case. It was often found that words applied to multiple categories, which is unsurprising given that the meaning of a word is dependent upon its context. This was especially so where words described action properties at a relatively high level within the action progress hierarchy. At these high levels, categorisation would depend upon the tense which is applied, whether it is considered as being done by the person or to the person, or whether there is a social context. Multiple classifications of the same word also seemed dependent on the integrated nature of sensorimotor control, by which we mean that even allocating aspects of sensorimotor control to action stages is arbitrary to some degree. For example, the desire for a successful outcome is inherent to motivation, which is at least partially determined by ability and skill. Nevertheless, despite these reservations, we can conclude that a sensorimotor action framework can throw a useful light on the classification of emotions and feelings.