Emotional expressions beyond facial muscle actions. A call for studying autonomic signals and their impact on social perception

Abstract

Humans are well adapted to quickly recognize and adequately respond to another’s emotions. Different theories propose that mimicry of emotional expressions (facial or otherwise) mechanistically underlies, or at least facilitates, these swift adaptive reactions. When people unconsciously mimic their interaction partner’s expressions of emotion, they come to feel reflections of those companions’ emotions, which in turn influence the observer’s own emotional and empathic behavior. The majority of research has focused on facial actions as expressions of emotion. However, the fact that emotions are not just expressed by facial muscles alone is often still ignored in emotion perception research. In this article, I therefore argue for a broader exploration of emotion signals from sources beyond the face muscles that are more automatic and difficult to control. Specifically, I will focus on the perception of implicit sources such as gaze and tears and autonomic responses such as pupil-dilation, eyeblinks and blushing that are subtle yet visible to observers and because they can hardly be controlled or regulated by the sender, provide important “veridical” information. Recently, more research is emerging about the mimicry of these subtle affective signals including pupil-mimicry. I will here review this literature and suggest avenues for future research that will eventually lead to a better comprehension of how these signals help in making social judgments and understand each other’s emotions.

Introduction

Imagine how different life would be if we were unable to recognize another’s expressions and from that, infer or even feel how the other must be feeling? During interactions with others, we automatically make use of another’s facial expressions and bodily signals and use that information to contextualize what is being said. Often, we are not aware of being influenced by these signals, except when they are absent from a “conversation,” for example during email or phone (mis)communication. Modern communication media aim for making conversations as natural as possible. However, even a Skype-conversation which from all communication media best simulates a natural interaction, does not even approach the richness and quality of real face-to-face interactions. One key limitation of Skype is that it is impossible to make eye-contact. Making eye-contact over Skype would require both interaction partners to look into the camera (and miss out upon each other’s eyes). Table 1 gives an overview of the various cues available in different communication mediums. In this review I will focus on the visual domain and on the face alone (columns E and F in Table 1) and explain how humans use a variety of sources from others’ facial signals during natural interactions.

Table 1.

Cues available in different communication mediums.

Medium	(A) Literal Message	(B) Shared environment	(C) Auditory	(D) Other sensory cues	(E) Facial expression	(F) Eye contact	(G) Body language
Real life	✓	✓	✓	✓	✓	✓	✓
Skype	✓	–	✓	✓	✓	–	✓/–
Phone	✓	–	✓	–	–	–	–
Email	✓	–	–	–	–	–	–

Depending on the type of communication medium, more is transmitted than just the literal, “cold” message (A). Apart from interfering visual or auditory delays, one large difference between real life interactions and interactions via other mediums is the being versus not being in the same environment, both in terms of time and place (B). In addition, during email contact, one misses out upon important auditory cues such as the tone of voice of the interaction partner, and his intonation or use of sarcasm which might change the meaning of the literal message (C). Apart from auditory cues, other sensory information such as smell and touch might impact on how the literal message is transferred and interpreted (D). The face, its expression and autonomic signals that are visible in it provide important cues about the emotional state of the interaction partner such as intensity or genuineness (E). Eye-contact is very important during interactions and is impossible via modern communication mediums (F). Body language and autonomic signals visible from the body reveal important emotional information (G).

Emotion processing is a broad and general term that refers to a complex of affective, behavioral and cognitive mechanisms that underlie our emotions. Given the impact of our emotions on a wide range of mental processes (e.g., perception, impression formation, decision making, memory) and manifest behaviors (e.g., helping or aggressive and abusive behavior), being able to recognize and regulate our emotions is of crucial importance and an essential feature of mental health (Kret and Ploeger, 2015).

During social interaction, interaction partners continuously express and regulate emotional states and simultaneously process affective cues expressed by the other. They orient to their partner’s tractable characteristics, such as facial or bodily features and emotion expressions. By attending to the stream of subtle dynamic facial reactions during an interaction, they “feel themselves into” the emotional landscapes inhabited by their partners; they rely on, and are influenced by implicit signals from their partner’s face that are autonomic and not under someone’s control, yet reflective of his or her emotions and intentions (Hatfield et al., 1994). This interchange of emotion processing influences impressions that are formed during a social interaction.

Figure 1 shows how emotions that are expressed during a social interaction by Person A, through emotional contagion, influence the emotions and expressions of Person B. Person A and B not only mimic each other’s facial expression, they also link on the physiological level and without being aware of it, synchronize on the level of arousal. Whereas they from time to time may “force” social smiles when considered appropriate (this is where ‘cognition’ comes into play), they have no or very little control over their autonomic responses such as blushing, sweating and pupil dilation which may nonetheless spread to the other person. Emotions and feelings, the extent to which they are expressed and converged with, together with cognitive processes influence how the other person is perceived. I call this ‘social perception,’ which includes impressions, beliefs about how the other person is feeling, trust, liking etc.

FIGURE 1.

Schematic representation of emotion processing during a social interaction.

The fact that emotions are not just expressed by the face and its muscle actions alone is often ignored in emotion perception research. In this article, I therefore argue for a broader exploration of emotion signals from sources beyond the face muscles that are more difficult to control. More specifically, I will focus on the perception of implicit sources that may be equally salient and familiar in daily life such as eye-gaze, pupil-dilation, eyeblinks, blushing and sweating. Although these cues may be subtle, they are visible to observers and may add to the perceived intensity of an emotional expression, or to its perceived genuineness. But prior to discussing these autonomic responses, I will review the literature on facial emotion perception.

Emotions Shown by the Whole Face

Expressions of emotion have evolved not only to move blood and tears in the service of emotions, but also to provide us with a rich set of tools that help us communicate and signal the nature of our internal emotional experiences so that we can call on others empathy and receive help.

Emotions Expressed by Face Muscles

Humans are experts at processing faces. They can recognize the identity of 1000s of individual faces and in addition quickly decode a variety of emotional expressions. Like the face itself, facial expressions of emotion may be processed configurally, a processing style that presumably enables speed and efficiency (Tanaka and Farah, 1993). The hallmark of this processing routine is the inversion effect, i.e., the impaired recognition performance when a face is shown upside down (Yin, 1969; for a review, see Maurer et al., 2002). Inversion appears to weaken or remove emotion category boundaries along continua of morphed stimuli and it has therefore been argued that the categorization of facial expressions draws upon configural information (de Gelder et al., 1997). It has also been proposed that categorical processing can be based on features, for example for happy expressions with the salient mouth curvature feature or for fearful faces with the display of eye-white, and, thus, precede affective attribution at the stage of configural processing (Calvo et al., 2012). This might explain why inversion has no effect when facial expressions are employed in a visual search task (Lipp et al., 2009; Savage and Lipp, 2014).

The many distinct facial expressions of emotions can be accurately encoded with aid of the facial action coding system (FACS; Ekman and Friesen, 1978). This anatomically based system segments the visible effects of facial muscle activation into “action units” (AUs). Each AU relates to one or more facial muscles. FACS describes facial activity on the basis of 33 unique AUs. This system not only helped in developing standardized stimuli sets of facial expressions of emotion [for example JACFEE (Biehl et al., 1997) and Ekmans pictures (Ekman and Friesen, 1978)], but also resulted in the development of automated facial expression recognition software.

Emotional expressions, facial expressions included, are highly contagious. It has been suggested that facial mimicry, i.e., the imitation of others’ facial displays by an observer, plays an important role in the communication of affective states. Evidence exists that mimicry accompanies the perception of a facial expression (Dimberg, 1982; Bush et al., 1989). The mere observation of a facial expression can also evoke the corresponding emotion in perceivers (Berger, 1962; Bandura and Rosenthal, 1966; Hygge, 1976). Other research shows that mimicry can be involved in the detection of change in facial expressions of emotion. Specifically, evidence suggests that individuals detect changes in the facial expression of another person through the feedback, and perhaps change in subjective state, caused by facial mimicry (Zajonc et al., 1987; Wallbott, 1991; Niedenthal, 1992; see also Niedenthal and Showers, 1991). Facial mimicry can occur unconsciously and has for example been observed in response to subliminally presented pictures and in cortically blind patients (Dimberg et al., 2000; Tamietto et al., 2009).

The majority of emotion studies make use of prototypical static facial expressions in their experimental paradigms which in fact are fairly impoverished representations of facial expressions in real life. In reality, facial expressions are sometimes partly occluded and not fully visible and almost always paired with other expressive signals including prosody and body language that provide a context (for example, see Regenbogen et al., 2012). The perception of facial expressions of emotion is influenced by such context cues. Context, even when it needs to be ignored within an experimental task, can completely shift the emotional category recognized in a facial expression (Meeren et al., 2005; Righart and de Gelder, 2006; Van den Stock et al., 2007; Aviezer et al., 2008; Kret and de Gelder, 2013; Kret et al., 2013a,b). In addition, facial expressions are dynamic by nature, with varying intensity and ambiguous. For brevity I will not further discuss these aspects, although I think it is important to mention that dynamics in facial stimuli are especially important not only to enhance recognition or increase naturalness, but also to create the impression of an interaction in observers [a face that slowly starts to smile (video) might be perceived more as a reaction to the participant than an already smiling face (picture)].

In addition to facial actions, there are other ways for the face to reveal emotions than via its muscle movements. When highly emotional, our forehead may show sweat drops, our cheeks may blush, our eyes may tear and our pupils may dilate. All these automatic and autonomic reactions are not specific for one particular emotion and may also occur when in pain or during sport. Importantly, they may also happen during an emotional experience, and because they are much harder to control than our facial muscles, and are visible to others, they might add to the perceived intensity of a facial expression or even overrule the emotion signal the facial muscles try to reveal. Not much is known about how these autonomic reactions or signals impact on emotion perception and whether they can modulate or even change the perception of facial expressions, and provide a context. For example, it is possible that a person with an angry, reddened face and sweat pearls on his forehead is perceived as more intensely angry than a person without these signs of arousal. See Figure 2.

FIGURE 2.

Expressing anger. The face on the right is reddened (except for the eyes and teeth) and shows drops of sweat. How these expressions of autonomic arousal impact on the perception of the emotional facial expression is not known.

Facial Reddening

Facial redness is associated with the perfusion of the skin with oxygenated blood. Research shows that facial redness is sexually dimorphic (men are slightly redder than women; Edwards and Duntley, 1939; Frost, 1994), and is strongly androgen-dependent in men (Jeghers, 1944). Moreover, increased skin blood perfusion is related to health and is enhanced by physical training (Johnson, 1998) and reduced in different patient groups (Richardson, 1987; Panza et al., 1990; Charkoudian, 2003). Skin redness enhances the healthy appearance of faces, also in dark-skinned people (Stephen et al., 2009). A recent study examined whether one’s own positive and negative emotions affect how healthy we perceive our own face to appear as compared to how others faces appear. Participants reported on their own affective state and then judged their own face, either healthy (red-tinged) or unhealthy looking (green-tinged) against stranger’s faces. Results show that participants high in negative affectivity judged themselves as equivalent to less healthy looking versions of their own face and a stranger’s face (Mirams et al., 2014).

In addition to a healthy appearance, facial redness may also signal emotionality. Anger, for example, also increases the blood flow to the face (Drummond and Quah, 2001; Montoya et al., 2005). Stephen et al. (2012) asked a group of women to manipulate pictures of men’s faces to make them as attractive as possible. Women in this study made the skin tone redder and even added more red when asked to make the men look more dominant. However, it was also observed that the women associated very red faces with aggression. Clearly, these studies suggest that redness is associated with health, physical dominance and anger or aggression, and someone with reddened cheeks will likely be perceived as dominant, angry or aggressive. However, facial redness might also reflect shyness in the form of blushing.

Many socially anxious individuals are anxious about blushing and regard it as the cause of their social difficulties rather than as a symptom of them. Although there is a lot of research on why, when and who blushes (Leary et al., 1992), there is no research on how blushing is actually perceived by others. Dijk et al. (2011) examined whether blushing after a sociomoral transgression remediates trustworthiness in an interdependent context. In their study, participants played a computerized prisoner’s dilemma game with a virtual partner who defected in the second round of the game. After the defection, a picture of the opponent was shown, displaying a blushing (reddened) or a non-blushing face. In a subsequent trust-game, participants invested more money in the blushing opponent than in the non-blushing opponent. In addition, participants indicated that they trusted the blushing opponent more, that they expected a lower probability of future defeat, and judged the blushing opponent in general more positively.

Above research indicates that the redness of a face impacts on observers’ social judgments. However, exactly how a red face is interpreted (“shy blushing,” “angry,” more ambiguously “aroused” or simply “hot”) is unsure and deserves further investigation and experimental control. Another avenue for future research is to use dynamic morphs from normally colored to slightly reddened faces and to manipulate the redness of the face so that the observers get the impression that the reddening of the face is a reaction to them, as in social interaction. In addition, it would be interesting to test whether humans would mimic or synchronize with such changes in color in their own face, which could be predicted based on previous research showing that people synchronize their level of arousal (Levenson and Gottman, 1983; Feldman et al., 2011; Cooper et al., 2014).

Emotions Shown by the Eyes

Among the many implicit cues that may inform assessments of someone’s internal state, the human eye region stands out as salient and powerful. Especially during short distance intimate interactions, both infants and adults focus on their interaction partner’s eyes, grasp emotion signals and follow gaze (Baron-Cohen, 1997; Driver et al., 1999; Farroni et al., 2002; Macrae et al., 2002).

Eyes and Gaze-Direction

The eyes are richly informative and important for understanding emotion and communicative intention of other individuals (Emery, 2000). Emotion-driven complex musculature contractions such as the raising of the eyebrows in fear enables observers to decode emotions from just the eye region (Baron-Cohen, 1997). The eye region captures more attention than other areas of the face in adults (Janik et al., 1978; Adolphs et al., 2005) as well as in infants (Haith et al., 1977) and this bias may reflect an innate predisposition (Argyle and Dean, 1965). In view of the importance of the eye region, one may predict that information from the eyes is robust such as to resist influence from the surrounding context. We often only see the eyes because items such as caps, hats, helmets, medical masks, beards or headdresses hide the rest of the face. Whether the perception of the expression of the eyes is sensitive to such visual context cues is a question that has hardly ever been asked.

There is some evidence that emotion categorization from the eye region is a process that is triggered automatically and unconsciously in a bottom–up fashion on the basis of the information available from the position of the eyebrows (Sadrô et al., 2003; Leppänen et al., 2008) and the visibility or display of eye white (Whalen et al., 2004). However, other research suggests that it is too early to rule out that context does play a role in the perception of expressions from the eyes. For example, Kret and de Gelder (2012) investigated how briefly presented angry, fearful, happy, and sad expressions were recognized when presented in different contexts including Islamic headdresses, a cap or a scarf. Results show that participants (all with a non-Islamic background) were better at recognizing fear from women wearing a niqāb as compared to from women wearing a cap and a shawl. An opposite effect was found for expressions of sadness and happiness. Furthermore, response patterns showed that ‘anger’ and ‘fear’ were more often chosen when the observed woman wore a niqāb as compared to a cap and a shawl. An opposite pattern was again found for the label ‘happy.’ Islamic cues triggered negative associations with the Islam, which influenced how emotions from the eyes were perceived. In line with the face literature this study shows that the recognition of emotional expressions from the eyes is also influenced by context.

Direct eye gaze captures and holds visual attention more efficiently than averted gaze and signals an expressor’s approach-avoidance behavioral tendency (Stern, 1977; von Grünau and Anston, 1995; Conty et al., 2010; Palanica and Itier, 2012; Böckler et al., 2014). Baron-Cohen (1995) states that this innate capacity to process gaze direction plays an important role in the development of a ‘Theory Of Mind.’

Gaze direction of an expressor also impacts on how observers perceive an expressed emotion. Adams and Kleck (2003) showed that expressions of anger and happiness were easier identified when presented with direct versus averted gaze, whereas fear and sadness expressions were more quickly labeled when presented with averted than with direct gaze. In a fear-conditioning experiment, Wieser et al. (2014) did not find an interaction between gaze direction and emotional expression either in terms of visually evoked steady-state potentials amplitude or affective ratings. This research underscores the importance of incorporating gaze direction in future work on facial expression perception.

Recent research has shown that social cognition including emotion perception is fundamentally different when we are engaged with others in real-time social interaction with eye-contact (‘online’ social cognition), rather than merely observing them (‘offline’ social cognition; e.g., Pfeiffer et al., 2013; Schilbach et al., 2013). The time has come to study emotion perception during real interactions between two participants as the mechanisms underlying actual social interactions in real life are insufficiently understood.

Eyeblinks

Apart from social interest or attention that may be inferred from gaze cues, research suggests that humans are also sensitive to another’s eyeblinks. Humans spontaneously blink every few seconds. Eyeblinks are necessary to moisturize the eye, but occur more frequently than necessary. The spontaneous eye blink is considered to be a suitable ocular indicator for fatigue diagnostics (Stern et al., 1984) and reflects the influence of central nervous activation without voluntary manipulation (Blount, 1928; Ponder and Kennedy, 1928). Nakano et al. (2009) showed that spontaneous blinks synchronized between and within subjects when they viewed short video stories. Synchronized blinks occurred during scenes that required less attention such as at the conclusion of an action, during the absence of the main character, during a long shot and during repeated presentations of the same scene. In contrast, blink synchronization was not observed when subjects viewed a background video or when they listened to a story that was read aloud. The results suggest that humans share a mechanism for controlling the timing of blinks that optimizes the processing of critical information, in order to not miss out on important information while viewing a stream of visual events. In a next study, Nakano et al. (2013) show that eyeblinks are involved in attentional disengagement. Authors demonstrated that during cognitive load, eyeblinks momentarily activate the brain’s default-mode network, while deactivating the dorsal attention network. It is thus far unknown whether the observation of eyeblinks in another person influences how that person is perceived.

Tears

Crying is a typically and, as far as known, uniquely human form of emotional expression. This phenomenon is controlled by the sympathetic and the parasympathetic nervous system, the latter being responsible for the tears (Botelho, 1964). Recent research has offered several accounts of how tears may have become adaptive over the course of evolution. Two main functions of crying have been distinguished, namely (i) tension relief and promoting the recovery of psychological and physiological homeostasis, and (ii) communication. Thus, on the one hand crying may impact on psychobiological processes in the individual and facilitate the physiological and psychological recovery after distress but on the other hand it may elicit positive or negative reactions from the social environment (Hendriks and Vingerhoets, 2002). For example, Hill and Martin (1997) observed that crying female confederates were more sympathized with than non-crying confederates. The results of this study suggest that crying may communicate the need for emotional support and help and calls on empathy. Crying is very contagious. Even newborn children may spontaneously start crying when they hear other children cry (Simner, 1971). It has been argued that empathy is related to the capacity to react to and mimic emotional others. In line with this argument, Wiesenfeld et al. (1984) observed that highly empathic women were more likely to react with a corresponding facial reaction when observing crying infants.

Research has shown that tears are an important visual cue that add meaning to facial expressions. Provine et al. (2009) and Zeifman and Brown (2011) have demonstrated that tears are helpful to identify sadness and to perceive a need for help and comfort. Similarly, Balsters et al. (2013) show that tears increase the accurate recognition of sadness and also the extent to which the displayed crying persons were in need of social support. Interestingly, in this last study, images were shown for just 50 ms, demonstrating the strength of tears even at an early pre-attentive level.

One consequence of crying is that the eyes redden. Although research on the perception of sclera color exists, this is mostly focusing on perceived health, age and attractiveness. For example, faces with artificially reddened sclera are rated less attractive (Provine et al., 2011). Whether red eyes also impact on the perception of a facial expression is an open question.

The Perception of Another’s’ Pupil-Dilation

Pupil size is an interesting social signal because it cannot be controlled or faked, in contrast to features such as eye gaze and facial expression and reflects much more than changes in light, namely, our inner cognitive and affective state (Bradshaw, 1967). Precisely because pupil changes are unconscious, they provide a veridical reflection of the person’s inner state.

Hess (1975) was the first to recognize the social potential of pupil dilation. In one study, he presented a group of men a series of pictures of which two showed an attractive young woman. One of them had been retouched to make the woman’s pupils larger and the other smaller than the original version. Interestingly, participant’s pupil response to the picture with the large pupils was larger than to the one with small pupils. Despite being unaware of the manipulation, participants liked the woman with the large pupils better, describing her as “more feminine,” “prettier,” and “softer” than the woman with small pupils (Hess, 1975; Figure 3A). In order to control for possible effects of luminance, in a later study, Hess created schematic eyes that consisted of a circle with a small, medium or large black dot in the middle. The circles were presented in isolation, in pairs with equally sized black dots, or with three of those in a row. Participants observed these stimuli whilst their pupil size was measured. Hess observed that both male and female participants showed the greatest pupil response to the ‘eye-like’ pair with the large black dot in the middle, an effect that could not be explained by luminance (Hess, 1975).

FIGURE 3.

Observed pupil size impacts on social evaluations. (A) The woman on the left is judged positive and attractive, and the one on the right with small pupils, as cold and distant. Picture is taken from Hess (1975). (B) A partner with dilating pupils is better trusted than a partner with static or constricting pupils, especially when the participant (on the left) synchronizes his own pupil size with the observed partner. Visual representation of the experimental setup of the study by Kret et al. (2015).

Later research replicated this early work and showed that pupillary changes are indeed picked up by observers and influence assessments: partners with large pupils are judged positive and attractive, and those with small pupils cold and distant (Demos et al., 2008; Amemiya and Ohtomo, 2012).

Kret et al. (2014) suggest that this positive association with large pupils is formed through pupil-mimicry. Within a close social interaction, mutual pupil dilation might indicate and reinforce social interest and liking. In their study, human and chimpanzee subjects viewed the eye region of both species. In this eye region, the stimulus pupils were manipulated to dilate or constrict dynamically, suggesting a response in view of the participant, as in an interaction. Results show that pupil-mimicry is not uniquely human, but has also been observed in chimpanzees. Even more interestingly, the phenomenon was restricted to within-species interactions (i.e. humans with humans, and chimpanzees with chimpanzees). It should be noted that the human participants in this study were mostly Japanese with pupils hardly distinguishable from their dark irises. Even in them, pupil-mimicry occurred, which is in line with the great importance of pupil size in the Japanese culture. As in Disney figures, good and bad cartoon characters in Japanese Manga are characterized by large and small pupils respectively.

The study by Kret et al. (2014) suggests that apart from passively reflecting inner states, pupils have a social signaling function and provide a basic form of communication between members of one’s own species. A more recent study further supports that idea. Kret et al. (2015; Figure 3B) observed that participants not only synchronized their pupil size with the pupil size of their virtual partner, but that this also helped them to trust that partner. More precisely, in their study, they used eye-images of Dutch and Japanese students and presented these to Dutch students. The pupils within these images dilated, constricted, or remained static. Whereas pupil dilation mimicry was amplified in interactions with the Dutch partners (who were considered closer to the in-group than the Japanese partners), pupil-constriction mimicry was stronger with the Japanese partners. Interestingly, a link was observed between pupil dilation mimicry and trust in Dutch, but not Japanese partners. Whether pupil-mimicry impacts on real interactions is unsure; pupils are prone to changes in ambient light and as mentioned earlier, are not equally visible in all individuals or circumstances. However, the fact that pupils have such profound effects in lab studies with virtual partners demonstrates their large potential in future live interaction studies with two participants.

Conclusion

During social interactions, humans willingly and unwillingly express their emotional state through facial muscle movements but also via other channels including pupil-dilation, eyeblinks and blushing. Over the course of human evolution, it became adaptive to not only perceive and be aware of emotions in one self, but also to process emotions including these implicit cues of others. Such perceptual abilities insure smooth interpersonal cooperation, for example by permitting the monitoring of expressions of fear and facilitating quick actions in response to environmental threat. In the past century, research on this topic has accumulated steadily and shown that humans, from early age on, respond to the emotions of others, empathize with them, and are very good at distinguishing between different expressions of emotion (i.e., Izard, 1971; Ekman and Friesen, 1982). The majority of this research has focused on the perception of pictures of faces showing prototypical, intense and unambiguous emotional expressions. Research has for example shown that humans mimic facial expressions (Dimberg, 1982) and according to the theory of emotional contagion, this might even improve emotion recognition (Hatfield et al., 1994).

At the beginning of this article, I noted the importance of context in the perception of emotions from the face. Emotions as displayed in the face are perceived in the context of the person’s body posture and the social emotional scene context (Kret and de Gelder, 2010). Clearly, the studies that did take these factors into account show that the percept of a face can change as a function of these contextual factors (Meeren et al., 2005; Righart and de Gelder, 2006; Van den Stock et al., 2007; Aviezer et al., 2008; Kret et al., 2013a,b). Also, emotions seen from just the eye region appear different, depending on what is visible around the eyes such as various types of clothing including caps, scarves and Islamic headdresses (Kret and de Gelder, 2012). My argument that autonomic expressions of arousal such as pupil dilation are less prone to cognitive control than facial expressions does not imply that they are not sensitive to contextual factors. On the contrary, pupil-mimicry for example, a phenomenon individuals are absolutely unaware of when doing it, is enhanced in interactions with members of an in-group (other humans and own culture) as compared to members of an out-group (chimpanzees and other cultural and ethnical background; Kret et al., 2014, 2015). I would like to argue that facial expressions of emotion beyond facial muscle actions can provide an additional context for the interpretation of emotional expressions.

When experiencing an emotion, it is not just the facial muscles that communicate that to other individuals. The current article is not the first call for studying ‘emotion-signaling sources’ beyond the well-known facial muscle movements. Other research has for example stressed the importance of studying the perception of bodily expressions (Dittrich et al., 1996; Pollick et al., 2001; Kret et al., 2011a,b,c, 2013a,b; Sokolov et al., 2011; Krüger et al., 2013). Although facial muscle actions can be unconscious, it is possible to control or regulate them via ‘top–down’ cognitive processes, yet our verical emotion often speaks through our body (for a review, see de Gelder et al., 2010). While fully acknowledging the importance of that work, I here would like to stress that also within a face there are more verical emotion signals than the signals sent by the facial muscles. Instead, autonomic expressions of arousal are much harder to control and more driven in a bottom-up fashion. For that reason, autonomic expressions might provide implicit insight into another’s experienced emotions, and might impact on the perceived intensity or genuineness of the expressed emotion.

To conclude, psychophysiological research has shown that the perception of emotional expressions in another individual triggers autonomic reactions that are for example measurable via electrodes measuring skin conductance (sweat), eye-tracking equipment (fixated attention and pupil dilation), (i.e. Bradley et al., 2008) or thermal imaging (to measure facial heating; for a review, see Ioannou et al., 2014). All these measures, these dependent variables, are in principle visible to others, and could serve as independent variables, possibly impacting on the total percept of a person and his emotional expression, its intensity and/or genuineness. The shedding of tears in sadness or joy, the reddening of the face and the pearls of sweat on ones forehead in anger or love, the blushing of the face and averting of gaze in embarrassment or shame, the dilating pupils indicating social interest and trust, happiness or stress are just examples of emotion signals that have received very little attention in the literature. Studying emotion perception beyond the face muscles including the perception of autonomic signals and the synchronization therewith is a promising avenue for future research.

Conflict of Interest Statement

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.