Skip to main content
BMC Psychology logoLink to BMC Psychology
. 2025 Mar 10;13:223. doi: 10.1186/s40359-025-02590-7

Overconfident, but angry at least. AI-Based investigation of facial emotional expressions and self-assessment bias in human adults

Roland Kasek 1,, Enikő Sepsi 2, Imre Lázár 1,2
PMCID: PMC11895137  PMID: 40065465

Abstract

Metacognition and facial emotional expressions both play a major role in human social interactions [1, 2] as inner narrative and primary communicational display, and both are limited by self-monitoring, control and their interaction with personal and social reference frames. The study aims to investigate how metacognitive abilities relate to facial emotional expressions, as the inner narrative of a subject might project subconsciously and primes facial emotional expressions in a non-social setting. Subjects were presented online to a set of digitalised short-term memory tasks and attended a screening of artistic and artificial stimuli, where their facial emotional expressions were recorded and analyzed by artificial intelligence. Results show self-assessment bias in association with emotional expressivity – neutrality, saturation, transparency – and the display of anger and hostility as an individually specific trait expressed at modality-dependent degrees. Our results indicate that self-assessment bias interplays in subconscious communication – the expression, control and recognition of facial emotions, especially – with empathetic skills and manipulation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40359-025-02590-7.

Keywords: Facial emotional expression, Self-assessment bias, Nonverbal communication, Metacognition, Artificial intelligence

Introduction

Nonverbal communication holds significant importance in shaping interpersonal relationships, serving as a channel for expressing emotions and conveying information above content, it complements and alters primary meanings. Human emotion perception and recognition are automatic, enhanced by both organic (mirror neurons) and functional abilities (empathy).

Observation of facial muscles movements has been shown in many studies to be sufficient in itself to assess the emotional state of a subject [36] enabled by distinct neuronal pathways [7], and in specific cases to show universal patterns across species and cultures [8]. For the basic emotions of joy, surprise, anger, disgust, sadness, fear, it is generally accepted that they can be reliably described by identical faciomuscular movement configurations and that they have individual, social and other meanings beyond the expression of emotion [9].

Although the recent academic debate argues whether or not emotions are directly linked to facial expressions and challenges the status quo that they are universally recognized as summarized by Heaven [10], Cowen et al. [11] suggests a high-dimensional framework to map the phenomena of human emotional expression and experience.

Artificial intelligence and emotion recognition

Recent studies conclude that while facial expressions are seemingly universal with a culturally dependent saturation of meaning, the underlying affective states rarely correlate with emotions that, in this sense, consist of spontaneous neurobiological changes and their cognitive evaluation, leading to cognition-dependent activation or inhibition of specific conscious and instinctive responses [12, 13].

With the development of imaging and image analysis technologies, it has also become possible to code facial expressions and recognize emotions based on categorical [5] or continuous [14] models using artificial neural networks and machine learning in real time from recordings to use the resulting data for analysis, primarily for marketing purposes. [eg. 15,16] However, Automatic Facial Coding (AFC) systems developed to understand and influence consumers’ emotions in line with business purposes, are also ideally suited for psychological assessments, taking their limitations and potential into account and benefit both the research and the development [17].

The discrepancies of systematically manipulated content - e.g. the role of modalities, influence of artistic effects - in individual and social situations can be directly investigated when using high-quality videos recorded under laboratory conditions [18]. At the same time, when complemented with psychophysiological and psychometric measurement tools, AFC systems allow for sensitive data acquisition and – compared to traditional methods based on typically self-reported phenomenological interpretations – enables highly accurate statistical analysis, especially in the context of implicit process diagnostics [19].

A further advantage of video-based emotion recognition is that it can be analyzed with a machine learning and artificial intelligence-enabled system, based on the judgement of emotion recognition experts – developed in time-consuming and expensive recruitment procedures – cross-referenced with relevant investigations, so that the human error can be eliminated in the analysis, resulting in AFCs to outperform humans [20].

Evolutionary background

Humans, compared to apes [21], exhibit reduced short-term visual memory in urban settings but excel in distinct cortical functioning e.g. long-term memory, imagination, and intentionality in deceit detection. The increased neocortex supports a reduced energy consumption, enhancing cognitive capacity for logical thinking and intention attribution in long-term mating strategy [22].

Despite inherent inaccuracies [23], imagination plays a vital role in creating adaptive strategies for varying situations and stakes, hence assists risk assessment, meaning constant awareness and ability to mentally manipulate any and all information about the potential partner in order to reinforce or inhibit affiliation [24].

Facial emotional expression in social interactions

According to Ekman and Friesen [25], lying is nothing more than a dissonance of cognition and emotion that leads to nonverbal leakage along channels. The modality constraint and the separation of channels greatly affect the observer’s picture of the observed person’s affective-cognitive coherence - even without the content [26].

Analyzing facial expression, as a pivotal channel in human interactions [27], in terms of cognitive functions it is clear that it engages visual short-term and semantic visual memories associated with vision and its processing [28], just as much as the decoding of body language [29].

Human cognition extends beyond processing ‘what is’ to also consider ‘what could be’ in nonverbal communication as well, particularly deception detection, which relies on brief visual cues, with the ‘presentation’ time of micro-expressions being as short as 1/5 − 1/25 milliseconds [25, 30]. The challenge lies in the detection of the emotional leakage that manifests in micro-expressions and contradicts the primary message. Detecting nonverbal leakage is complex, compounded by its atypical appearance. In human social interactions, distorted truths foster justifications, confirming suspicions and breeding stronger illusions to resolve cognitive dissonance.

While cognitive biases and stereotypes can be useful in everyday life, peer influence also puts evolutionary pressure on self-deception as an adaptive strategy [31], they are activated uncontrollably when the right risk (time pressure, stakes, danger, emotional context) is present, and it is almost impossible to ignore them.

Just like in Flavell’s [32] subjects, self-awareness [33] is key to performance – from fast and accurate perception to being able to be simultaneously aware of a situation and its possible outcomes – and especially in communication, probabilistic prediction of intentions and successful manipulation of others – and a capacity that allows these simultaneous processes to perform efficiently – allows one to time the use of resources optimally, and being aware of both limitations and potentials. Awareness relieves cognitive capacity of unnecessary burdens, and volitional inhibition can be applied to types of nonverbal leakage that are more difficult to detect, while those that are easier to assess lead to more accurate – or, defined by the inner speech or narrative, self-assuring – findings.

Aim

In social interactions, intentional masking and display of emotional states lead to the appearance of micro-expressions and empathy-driven mimicry, therefore, we have hypothesised that the defining inner narrative enabled by self-awareness – based on self-confidence and personality – of a subject might project subconsciously and primes facial emotional expressions. To observe such phenomena and let faces serve as a purely inner-state dependent display, subjects were prevented from maintaining interaction during data collection, hence allowing honest – or at least less deceitful – reactions to stimuli.

While laboratory settings offer a sterile context to capture what might really be happening, they also miss out on revealing the real-life characteristics of social interactions.

Our goal was to develop a setting that allows more space for the observation of the life-like approach to facial emotional expression using artistic and artificial experience as tools to understand the differences in emotional expressions and experiences between man-made and machine generated stimuli.

As activation and inhibition of expressing emotions are motivated and enabled by individually different interplay of cognitive and affective factors, we have also hypothesized that the removal of direct social contact will promote the temporal and qualitative extension of facio-muscular activation and therefore reveal otherwise voluntarily or subconsciously hidden characteristics of the associations between self-confidence, personality traits and facial emotional expressions.

Method

Subjects were presented online on their own devices to a set of digitalised short-term memory tasks [34] that measure cognitive capacity and metacognitive abilities simultaneously. In the second phase of the experiment a subgroup of previously tested adults based on availability and willingness attended a screening of artificial audio, human audio and video stimuli, where their actions were recorded for further facial emotional expression analysis with Noldus FaceReader 8.1 as described in the graphical abstract (Appendix GA).

Participants

The study investigates a sample of 35 mentally healthy native Hungarian adult subjects – 14 men (age = 28–51 years; M = 35.1 years) and 21 women (age = 19–48; M = 30.8 years) – who were randomly selected from a previously psychometrically tested larger population of volunteers recruited at the university and by social media based on availability and willingness. Before participating, subjects anonymously stated that they are mentally healthy adults and consented to the collection and use of their data.

Laboratory equipment and setting

Close-up studio cameras were set up in front of the subjects while they listened to the three literary excerpts in Hungarian with a total duration of 16 min interpreted by a professional actor in order: first with actor-performed audio (sound only), then with actor-performed experience (sound and image), and finally with artificially generated audio (sound only). After the first screening the order of stimuli was shuffled to avoid artefacts that may arise from order effect confound, despite the fact that we were only going to investigate modality dependent changes and individually specific emotion expressions.

Apparatus

Performance tests

The test software includes four memory tasks (span of 3 to 9) of increasing complexity. The basic task (Task 1) is a digitalised version of the Corsi Block Tapping Test [35], requiring the correct sequential localization of sequentially presented homogeneous stimulus material (dots), while the second task (Task 2) requires the spatial recall of simultaneously presented numbers in ascending order [36]. The third task (Task 3) is a repetition of the second with a self-paced presentation time [32]. The fourth task (Task 4) requires spatial-numerical interference inhibition and correct spatial recall [34]. At the end of the tasks, participants intuitively rate their own performance relative to their peers on a Likert-scale of 6 [37].

Facial action coding with artificial intelligence

The FaceReader by Noldus uses an artificial neural network to classify emotional expressions, which yields data such as basic expressions, individual expressions, head orientation, gaze direction, personality characteristics, valence and arousal, as well as heart rate and heart rate variability with frame-by-frame resolution limited by the analyzed video material’s frame rate and outputs data with milliseconds accuracy. By default, the diagnostic software measures the expression of the six basic emotions (joy, fear, anger, sadness, surprise, disgust) as a percentage and the temporal expression of the most prominent emotion along reaction time. A validation study conducted by Stöckli et al. [38] found that FaceReader 6 performed the best of the major emotion classification software available at the time, with an average accuracy of 88%. Another study done by Lewinski, den Uyl and Butler [19] states that FaceReader correctly recognized 88% of expressions on average in the WSEFEP and ADFES pictures, whereas human participants only recognized 85%, and outperforms humans 90 to 59% when it comes to neutral faces [20]. FaceReader 8.0 reached a higher test-retest reliability than human coders [39], and the since improved version of FaceReader - version 8.1 was used in this study - achieved an even higher score of 96% in emotional recognition according to the developer, so we can at least assume it is as good as or even better than human coders and valid in minimum 88% of the cases.

Statistical analysis

Descriptive statistics, Kruskal-Wallis tests with Dwass-Steel-Critchlow-Fligner pairwise comparisons and Brunner-Munzel tests – the most reliable nonparametric procedure for relatively small sample comparisons [40] – were conducted using jamovi 2.3 [41] for the software’s unique ability to perform the latter. The threshold for statistical significance was set at p <.05.

Variables

Performance and self-assessment

Memory performance (MP) was measured simply by sum of hits across all tasks excluding effortlessly acquirable – first and last – hits out of the accumulation. Based on hits accumulated across the four tasks, results were recorded test by test as baseline for the post-task self-assessment Likert-scale of 1–6, that was compared to actual, 1–6 scaled performance and subtracted and averaged across T1-T4 into self-assessment bias index (SAB), allowing the grouping of subjects by bias polarity–lower or higher than 0 – (PSAB) and accurate or biased self-confidence (ASAB).

Facial action coding with artificial intelligence

Facial emotional expressions were recorded and analysed throughout the entire screening time with a chosen analysis frame rate of 30 frames per second resolution (Borges et al., 2019) – a unique ability of the artificial intelligence – into cumulated percentage values by basic (happy, surprised, sad, scared, disgusted, angry), other and neutral emotion category and each modality (‘HV Angry 28.36’ for the total percentage of anger expressed during each Human Video setting), then averaged into a modality-independent facial expression index (‘AE Angry 12.31’).

Based on our preliminary exploration of the raw recordings and the collected data, we have identified the need to develop emotional expression ratios as new variables to be able to compare the characteristics across different modalities. Emotional Saturation (SE) shows the ratio of ‘dominant’ to ‘all other’ emotions, to explore the subject’s emotional span. Emotional Transparency (TE) represents the ratio of ‘basic six’ to ‘other’ emotions and reveals how difficult might be to decode the social target’s emotional state just by registering their facial expressions.

Emotion experience and personality

Since emotions and personality traits are interrelated [eg. 42,43], based on therapeutic considerations [44] we have grouped basic emotions into three personality categories – Extroversion (happy and surprised), Neuroticism (sad and scared) and Hostility (disgusted and angry) by Izard’s theoretical framework and results [45] – to be able to analyse the associations of self-confidence and personality dimensions as defining factors of how subjects empathize, experience and express emotions.

Results

Gender differences

Brunner-Munzel testing confirmed no evidence of gender differences (Table 1) in our sample regarding age (Age), memory performance (MP), self-assessment bias (SAB), accuracy (ASAB) and polarity (PSAB).

Table 1.

Brunner-Munzel tests confirms no evidence of differences between basic variables (Age, memory performance, Self-Assessment bias, accuracy and Polarity) and biological gender (Sex)

Statistic df p Relative effect
AGE -1.559 31.8 0.129 0.350
MP -0.182 29.9 0.857 0.481
SAB -0.441 28.9 0.662 0.459
ASAB 0.406 24.3 0.688 0.524
PSAB -0.538 27.9 0.595 0.452

Note. HaP ^(1 < 2) + ½P ^(1 = 2) ≠ ½

Modalities

Compared to audio presentation (sound only), the full artistic experience (sound and video) seemingly rather dimmed than enhanced emotional expressions. Contrary to our expectations, Kruskal-Wallis testing of emotional expressions by modalities – Human Audio, Human Video, Artificial Audio and their modality-independent Average – confirmed no evidence of such phenomena (Table 2), even in DSCF pairwise comparisons (Appendix Table 1 A).

Table 2.

Kruskal-Wallis tests confirms no evidence of differences between each modality (HA, HV, AA) and their average (AE)

Emotion χ² df p ε²
Neutral 0.740 3 0.864 0.00532
Happy 0.741 3 0.863 0.00533
Surprised 2.249 3 0.522 0.01618
Sad 0.608 3 0.895 0.00437
Scared 2.015 3 0.569 0.01449
Disgusted 0.540 3 0.910 0.00388
Angry 3.074 3 0.380 0.02212
Other 5.107 3 0.164 0.03674
SE 1.801 3 0.615 0.01296
TE 0.343 3 0.952 0.00247
Extroversion 1.473 3 0.689 0.01060
Neuroticism 0.530 3 0.912 0.00382
Hostility 2.511 3 0.473 0.01807

Note. * p <.05 ** p <.01, *** p <.001

Self-assessment bias and emotions

General analysis of the sample revealed that self-assessment bias is moderately associated with the expression of anger (χ²= 7.3736, df = 2, p =.25, ε²=0.21687) and hostile tendency (χ²= 7.4567, df = 2, p =.24, ε²=0.21931). Surprisingly and quite contrary to our expectations based on a previous phenomenological approach [46], the difference between the expressions of other basic emotions were not significant (Table 3).

Table 3.

Kruskal-Wallis tests confirms evidence of differences between self-assessment bias (SAB) and the average of facial emotional expressions (AE angry, AE Hostility)

Emotion χ² df p ε²
Neutral 3.7451 2 0.154 0.11015
Happy 2.3276 2 0.312 0.06846
Surprised 0.0597 2 0.971 0.00176
Sad 2.0917 2 0.351 0.06152
Scared 1.5000 2 0.472 0.04412
Disgusted 2.6545 2 0.265 0.07807
Angry 7.3736 2 0.025* 0.21687
Other 5.3169 2 0.070 0.15638
SE 3.5787 2 0.167 0.10526
TE 2.7109 2 0.258 0.07973
Extroversion 1.3546 2 0.508 0.03984
Neuroticism 2.0917 2 0.351 0.06152
Hostility 7.4567 2 0.024* 0.21931

Note. * p <.05, ** p <.01, *** p <.001

Self-assessment accuracy

Categorical grouping and Brunner-Munzel testing (Table 4) of our sample by accurate or biased self-assessment (ASAB) confirmed evidences that Neutrality (BM = 2.97, df = 5.58, p =.014, RE = 0.798) as lack the of detectable expressions and Transparency (BM = 2.33, df = 26.01, p =.014, RE = 0.702) as the incidence of basic emotional expressions was strongly higher, while Saturation (BM=-2.62, df = 9.08, p =.005, RE = 0.210) as the variety of emotional experiences was moderately lower in the biased group.

Table 4.

Brunner-Munzel tests confirms evidence of differences by self-assessment bias accuracy (ASAB) and emotional neutrality, emotional saturation (SE) and emotional transparency (TE) across modalities and their average (AE)

Emotion Modality Statistic df p Relative effect
Neutral HA 1.88 8.96 0.047* 0.694
HV 2.87 4.20 0.021* 0.815
AA 4.03 5.50 0.004** 0.847
AE 2.97 5.58 0.014* 0.798
S E HA -2.72 14.51 0.008** 0.258
HV -1.87 4.27 0.065 0.258
AA -2.62 7.56 0.016* 0.242
A E -3.26 9.08 0.005** 0.210
TE HA 1.33 3.96 0.128 0.694
HV 1.14 4.18 0.158 0.665
AA 4.11 12.28 < 0.001*** 0.819
AE 2.33 26.01 0.014* 0.702

Note. HaP ^(1 < 2) + ½P ^(1 = 2) > ½; * p <.05, ** p <.01, *** p <.001

Exploring the differences between modalities, the artificial audio (AA) setting emerged both in emotional Neutrality (BM = 4.03, df = 5.50, p =.004, RE = 0.847) and Transparency (BM = 4.11, df = 12.28, p <.001, RE = 0.819) in the biased group with a strong relative effect, while the human audio (HA) setting elicited moderately Saturated expressions (BM=-2.72, df = 14.51, p =.008, RE = 0.258) in the accurate group.

Self-assessment bias Polarity

Brunner-Munzel tests revealed (Table 5) relatively strong associations of positive bias polarity (PSAB) and Hostility across modalities and their Average (BM = 2.91, df = 33.0, p =.003, RE = 0.732), as a result of strong synergetic tendencies of Disgust (BM = 1.74, df = 27.0, p =.047, RE = 0.608) and Anger (BM = 2.35, df = 27.0, p =.013, RE = 0.670).

Table 5.

Brunner-Munzel tests confirms evidence of differences by self-assessment bias Polarity (PSAB) and disgust, anger and hostility across modalities and their average (AE)

Emotion Modality Statistic df p Relative effect
Disgusted HA 1.74 27.0 0.047* 0.608
HV 1.67 29.0 0.053 0.605
AA 0.97 30.1 0.171 0.552
AE 1.67 29.0 0.053 0.605
Angry HA 2.35 27.0 0.013* 0.670
HV 1.67 29.0 0.053 0.605
AA 1.46 25.0 0.078 0.585
A E 2.40 31.9 0.011* 0.681
Hostility HA 3.38 31.3 < 0.001*** 0.755
HV 1.93 32.9 0.031* 0.647
AA 1.82 30.7 0.039* 0.632
A E 2.91 33.0 0.003** 0.732

Note. HaP ^(1 < 2) + ½P ^(1 = 2) > ½; * p <.05, ** p <.01, *** p <.001

Summary

Altogether, the subjects of the observed sample reacted typically with neutral expressions to the presented stimuli, which can be explained by the reduced attentional capacity and the resulting lower receptivity and empathy due to the fear of the epidemic and the stress arising from the restrictions, as well as by the laboratory situation itself.

It is also a surprising result that despite the lack of direct evidence of emotional experiences between modalities, we have found strong but modality-dependent associations with self-assessment bias and emotional expressions, which leads to several conclusions:

  1. That the actor’s predominantly negative-neutral facial expressions influence the recipient’s facial expressions in terms of affective transfer; and.

  2. That the emotional intensity of the audio-only experience is higher because this form presumably allows more room for imagination, hence reserves cognitive processing power to the detriment of the control of facial expressions.

The artificial sound, on the other hand, has often resulted both in temporary serenity in case of negative emotions and in enhanced positive emotions, most probably because of its comic intonation and inconsistent emphasis which were violating natural patterns, resulting in relatively strong associations between self-assessment accuracy, emotional transparency and neutrality. Accurate self-assessment is associated with lower facial emotional control (lower Neutrality), higher emotional Saturation and lower Transparency, compared to any sign of bias, while Anger and Hostility are typical in the overconfident group.

Discussion

Self-reflection and facial emotional expression

The aim of the study was to investigate how cognitive capacity and metacognitive abilities relate to facial emotional expressions. The short-term memory tasks were essential to measure an objective output in a non-familiar situation, on which subjects were able to intuitively reflect on and estimate their performance on a scale of 1–6 compared to peers. Processing these objective and subjective factors gained us access to the individually attributive degree of self-assessment bias, which allowed the exploration of the subjects’ inner narrative without the typical influencing factors of self-reported results. To let faces serve as a purely inner-state dependent display, subjects were prevented to maintain interaction during the screening, hence allowing unfiltered – or at least less voluntary – facial emotional reactions to stimuli.

As we have hypothesised, the defining inner narrative of a subject projects subconsciously towards peers and influences facial emotional expressions. The intensity of facial emotional expression depends on whether and to what extent the subject is organically and functionally able and willing to communicate emotional state to others in a given situation. Cognitive capacity directly determines the ability of self-control – associated with the approach or inhibition of certain behaviours – and metacognitive ability which together influence self-awareness, social interactions, and are particularly relevant in the case of intentional expression or repression of facial emotions. However, in the applied laboratory setting none of this mattered: subjects were practically prevented interaction, and their purely receiving role allowed us to observe facio-muscular reactions without their normally active facial expression filters.

Happiness, sadness and surprise appeared on the faces regardless of the array and magnitude of self-assessment bias, leading us to a conclusion that their inner narrative aligned with past experiences and primes expressions by eliciting sympathy and support in others for the results – whether they lag behind promises or meet expectations – might be independent of self-confidence. Hostile – disgust and mainly angeremotional expressions, on the contrary, stem in overconfidence and mitigates the expected harm that might be done by others for the results are thought to fall short of requirements, providing a higher ground in a predicted conflict on a ‘best defense is – attack’ basis and as a constructive, reparative response [47]. Our finding indicates that further research may explore how overconfidence exactly relates to inner speech, the expression of anger and hostility, and whether and how these results might serve therapeutic aspects.

Neutrality and happiness were detected in cases of congruent self-reflection, independent of cognitive capacity but related to gender disparities. Male subjects of minimally biased or precise self-assessment seemingly enjoyed the screening more than females, which can either be attributed to.

  1. The male actor’s appearance or the play itself,

  2. The mood and topics of the literary excerpts, or.

  3. Simply the fact they were able to relax and focus on the experience of having a normally peaceful inner narrative due to their accurate self-reflection and social reference frame.

In conclusion, self-assessment bias seemingly related to how subjects manipulate their environment, and depending on their projected expectations based on past experiences, self-confidence and personality, their expression reflected on how they want others to see them: hostile, friendly or lost.

Nonverbal communication in solitary

The costs and benefits of expressing or repressing emotional states are purely situational and depend on capacity and motivation. Whilst in social interactions sympathy, empathy – a special form of metacognition, being aware and in control of feelings – and intention define verbal-nonverbal congruency, inhibition of emotional leakage becomes futile for subjects who are experiencing temporary social deprivation or isolation.

The reception of artistic performance as a one-way communication triggered more intense emotional reactions when imagination reserved cognitive capacity, while the appearance of the actor saturated the experience resulting in similar, but decreased amplitude of facial expressions. Observing such phenomena allows us to conclude that artistic audio-visual experience might develop empathy and empathetic skills by solitary exploration of affective changes which improve self-awareness by practicing adaptation to another person without the consequences of the attempt.

In conclusion, the removal of direct social contact promotes the temporal and qualitative extension of facial emotional expressions and therefore linked to cognitive capacity, metacognitive abilities and personality traits.

Constraints and limitations

Although the relatively small sample size, the self-reported mental health status and known effects of the pandemic – and its due restrictions – on cognitive abilities [48, 49] prevent us from directly generalize these findings to the majority of the population, our results indicate that further investigation of the yet revealed pattern would be beneficiary in the development of diagnostic tools and therapeutic interventions.

The experiments were carried out between May and June 2020, after months of obligatory use of surgical masks, social deprivation, existential threat and uncertainty which altogether might have altered emotional experience and expression due to known increase in mental illnesses and disorders [50, 51]. It is important to underline that recent findings suggest that isolation and physical distancing itself - which has been misinterpreted by mainstream media as social distancing - may also be responsible for functional uncertainty, as it is associated with neuropsychological and neurobiological changes in early childhood and with age [52], and therefore may alter nonverbal communication patterns.

Further and better funded investigation involving artificial intelligence that enables temporal data processing – e.g.: emotional Saturation and Transparency – in a larger sample may also be required to determine whether and how metacognitive abilities interplay in the expression of facial emotions and recognition, especially in relation with psychophysiological correlates – e.g. heart rate variability, galvanic skin resistance – of emotion production, empathetic skills and manipulation.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 2 (3.9MB, docx)

Acknowledgements

The authors would like to thank all the reviewers whose insights were invaluable in reaching the final form of the study; the participants; their colleagues, Anikó Kónya, Mónika Albu, Réka Bogdányi, Emese Fejes, dr. Marianna Török, Szabolcs Takács and Bence Gergely; and the students who contributed to the research. The authors were provided with the screening materials, recording equipment, and postproduction services by the Art Sciences and Media Education Division of Benda Kálmán College for Advanced Studies at Károli University.

Author contributions

Conceptualization, R.K., E.S., and I.L.; methodology, R.K., E.S., and I.L.; software, R.K. and I.L.; validation, I.L.; formal analysis, R.K.; investigation, R.K.; resources, I.L.; data curation, I.L. and R.K.; writing—original draft preparation, R.K.; writing—review and editing, R.K. and I.L.; visualization, R.K.; supervision, I.L.; project administration, I.L.; funding acquisition, R.K., E.S. and I.L. All authors have read and agreed to the published version of the manuscript.

Funding

Open access funding provided by Semmelweis University.

The research leading to these results has received auxiliary funding for FAC analysis from the Doctoral Research Grant and non-financial support from the Doctoral School’s Workgroup for Science Management at Semmelweis University.

Data availability

The data presented in this study are available upon request from the corresponding author for legal agreements and ethical reasons.

Declarations

Ethics approval and consent to participate

All procedures complied with the Helsinki Declaration and institutional guidelines and were approved by Semmelweis University Regional and Institutional Committee of Science and Research Ethics at SE RKEB 149/2019 on 31st July 2019.

Informed consent

All participants provided written informed consent.

Consent for publication

Not Applicable.

Competing interests

The authors declare no competing interests.

Footnotes

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Frith CD. The role of metacognition in human social interactions. Philosophical Trans Royal Soc B: Biol Sci. 2012;367(1599):2213–23. 10.1098/rstb.2012.0123. PMid:22734064, PMCid:PMC3385688. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Barrett LF, Mesquita B, Gendron M. Context in emotion perception. Curr Dir Psychol Sci. 2011;20(5):286–90. 10.1177/0963721411422522 [Google Scholar]
  • 3.Russell JA, Bachorowski JA, Fernández-Dols JM. Facial and vocal expressions of emotion. Ann Rev Psychol. 2003;54(1):329–49. 10.1146/annurev.psych.54.101601.145102. PMid:12415074. [DOI] [PubMed] [Google Scholar]
  • 4.Phillips ML, Drevets WC, Rauch SL, Lane R. Neurobiology of emotion perception I: the neural basis of normal emotion perception. Biol Psychiatry. 2003;54(5):504–14. 10.1016/S0006-3223(03)00168-9. PMid:12946879. [DOI] [PubMed] [Google Scholar]
  • 5.Ekman P, Rosenberg EL. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA. 2nd Edition. 2005 10.1093/acprof:oso/9780195179644.001.0001
  • 6.Aviezer H, Ensenberg N, Hassin RR. The inherently contextualized nature of facial emotion perception. Cur-rent Opin Psychol. 2017;17:47–54. 10.1016/j.copsyc.2017.06.006. PMid:28950972. [DOI] [PubMed] [Google Scholar]
  • 7.Calder AJ, Lawrence AD, Young AW. Neuropsychology of fear and loathing. Nat Rev Neurosci. 2001;2(5):352–63. 10.1038/35072584. PMid:11331919. [DOI] [PubMed] [Google Scholar]
  • 8.Ekman P, Friesen WV. Constants across cultures in the face and emotion. J Personal Soc Psychol. 1971;17(2):124. 10.1037/h0030377. PMid:5542557. [DOI] [PubMed] [Google Scholar]
  • 9.Ekman P. Universals and cultural differences in facial expressions of emotion. In Nebraska symposium on motivation. University of Nebraska Press. 1971
  • 10.Heaven D. Why faces don’t always tell the truth about feelings. Nature. 2020;578(7796):502–5. 10.1038/d41586-020-00507-5. PMid:32103200. [DOI] [PubMed] [Google Scholar]
  • 11.Cowen A, Sauter D, Tracy JL, Keltner D. Mapping the passions: toward a high-dimensional taxonomy of emotional experience and expression. Psychol Sci Public Interest. 2019;20(1):69–90. 10.1177/1529100619850176. PMid:31313637, PMCid:PMC6675572. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Barrett LF, Gross J, Christensen TC, Benvenuto M. Knowing what you’re feeling and knowing what to do about it: mapping the relation between emotion differentiation and emotion regulation. Cognition Emot. 2001;15(6):713–24. 10.1080/02699930143000239 [Google Scholar]
  • 13.Lindquist KA, Wager TD, Kober H, Bliss-Moreau E, Barrett LF. The brain basis of emotion: a me-ta-analytic review. Behav Brain Sci. 2012;35(3):121. 10.1017/S0140525X11000446. PMid:22617651, PMCid:PMC4329228. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Russell JA. Core affect and the psychological construction of emotion. Psychol Rev. 2003;110(1):145. 10.1037/0033-295X.110.1.145. PMid:12529060. [DOI] [PubMed] [Google Scholar]
  • 15.Hadinejad A, Moyle BD, Scott N, Kralj A. Emotional responses to tourism advertisements: the application of facereader™. Tourism Recreation Res. 2019;44(1):131–5. 10.1080/02508281.2018.1505228 [Google Scholar]
  • 16.da Peixoto CL, Gonçalves MM, C. A., da Silva EC. Proposition of the ISIE framework and the applicability of facereader to measure emotions in marketing research on sustainable consumption. REMark: Revista Brasileira De Mark, 2024;23(1).
  • 17.Martinez A, Du S. A model of the perception of facial expressions of emotion by humans: research overview and perspectives. J Mach Learn Res, 2012;13(5). [PMC free article] [PubMed]
  • 18.Landmann E. I can see how you feel—Methodological considerations and handling of Noldus’s facereader software for emotion measurement. Technol Forecast Soc Chang. 2023;197:122889. 10.1016/j.techfore.2023.122889 [Google Scholar]
  • 19.Lewinski P, den Uyl TM, Butler C. Automated facial coding: validation of basic emotions and FACS AUs in facereader. J Neurosci Psychol Econ. 2014;7(4):227. 10.1037/npe0000028 [Google Scholar]
  • 20.Lewinski P. Automated facial coding software outperforms people in recognizing neutral faces as neutral from standardized datasets. Front Psychol. 2015;6:1386. 10.3389/fpsyg.2015.01386 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Fagot J, De Lillo C. A Comparative Study of Working Memory: Immediate Serial Spatial Recall in Baboons (Papio Papio) And Humans, Neurosychologia. 2011;49;3870–3880. 10.1016/j.neuropsychologia.2011.10.003, PMid:22015260. [DOI] [PubMed]
  • 22.Dunbar, R. I. (1998). The social brain hypothesis. Evolutionary Anthropology: Issues, News, and Reviews: Issues, News, and Reviews, 6(5), 178–190. 10.1002/(SICI)1520-6505(1998)6:5%3C178::AID-EVAN5%3E3.0.CO;2-8
  • 23.O’Connor KP, Aardema F. The imagination: cognitive, pre-cognitive, and meta-cognitive aspects. Conscious Cogn. 2005;14(2):233–56. 10.1016/j.concog.2004.07.005. PMid:15950880. [DOI] [PubMed] [Google Scholar]
  • 24.Murray SL, Holmes JG, Griffin DW, Bellavia G, Rose P. The mismeasure of love: how self-doubt contaminates relationship beliefs. Pers Soc Psychol Bull. 2001;27(4):423–36. 10.1177/0146167201274004 [Google Scholar]
  • 25.Ekman P, Friesen WV. Nonverbal leakage and clues to deception. Psychiatry. 1969;32(1):88–106. 10.1080/00332747.1969.11023575. PMid:5779090. [DOI] [PubMed] [Google Scholar]
  • 26.O’Sullivan M, Ekman P, Friesen W, Scherer K. Of Others, Interpersonal Relations And Group Processes. J Personal Soc Psychol. 1985;48(1):54–62. 10.1037//0022-3514.48.1.54. PMid:3981393. What You Say And How You Say It: The Contribution Of Speech Quality To Judgements. [DOI] [PubMed]
  • 27.Schmidt KL, Cohn JF. Human facial expressions as adaptations: evolutionary questions in facial expression research. Am J Phys Anthropology: Official Publication Am Association Phys Anthropologists. 2001;116(S33):3–24. 10.1002/ajpa.20001. PMid:11786989, PMCid:PMC2238342. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Brosch T, Scherer K, Grandjean D, Sander D. The impact of emotion on perception, attention, memory, and decision-making. Swiss medical weekly, 2013;143(1920):w13786-w13786. 10.4414/smw.2013.13786, PMid:23740562. [DOI] [PubMed]
  • 29.Abramson L, Marom I, Petranker R, Aviezer H. Is fear in your head? A comparison of instructed and real-life expressions of emotion in the face and body. Emotion. 2017;17(3):557. 10.1037/emo0000252 [DOI] [PubMed] [Google Scholar]
  • 30.Ekman P. Lie Catching and Microexpressions. In: Martin C, editor. The Philosophy of Deception. New York, USA: Oxford University Press; 2009;118–133. 10.1093/acprof:oso/9780195327939.003.0008
  • 31.Hippel W, Trivers R. The evolution and psychology of Self-Deception. Behav Brain Sci. 2011;34:1–56. 10.1017/S0140525X10001354. PMid:21288379. [DOI] [PubMed] [Google Scholar]
  • 32.Flavell JH. Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. Am Psychol. 1979;34(10):906. 10.1037/0003-066X.34.10.906 [Google Scholar]
  • 33.Morin A. Possible links between self-awareness and inner speech theoretical background, underlying mechanisms, and empirical evidence. J Conscious Stud. 2005;12(4–5):115–34. [Google Scholar]
  • 34.Kasek R, Albu M. Are visual types basically visual? DuCog VI Language and conceptual development. Croatia, May 23 (in program and abstracts. Dubrovnik; 2014;41.
  • 35.M Corsi P. Human memory and the medial Temporal region of the brain. Dissertation Abstracts Int. 1973;34(2–B):891. [Google Scholar]
  • 36.Inoue S, Matsuzawa T. Working memory of numerals in chimpanzees. Curr Biol. 2007;17(23). 10.1016/j.cub.2007.10.027. PMid:18054758. R1004-R1005. [DOI] [PubMed]
  • 37.Kasek R, Takacs S, Konya A. Visual expertise in immediate Spatial localisation tasks. Cogn Process. 2015;16:S91. [Google Scholar]
  • 38.Stöckli S, Schulte-Mecklenbeck M, Borer S, Samson AC. Facial expression analysis with AFFDEX and FACET: A validation study. Behav Res Methods. 2018;50(4):1446–60. 10.3758/s13428-017-0996-1. PMid:29218587. [DOI] [PubMed] [Google Scholar]
  • 39.Borsos Z, Jakab Z, Stefanik K, Bogdán B, Gyori M. Test–retest reliability in automated emotional facial expression analysis: exploring facereader 8.0 on data from typically developing children and children with autism. Appl Sci. 2022;12(15):7759. 10.3390/app12157759 [Google Scholar]
  • 40.Karch JD. Psychologists should use Brunner-Munzel’s instead of Mann-Whitney’s U test as the default nonpara-metric procedure. Adv Methods Practices Psychol Sci. 2021;4(2):2515245921999602. 10.1177/2515245921999602 [Google Scholar]
  • 41.The jamovi project. (2022). jamovi. (Version 2.3) [Computer Software]. Retrieved from https://www.jamovi.org
  • 42.Reisenzein R, Weber H. Personality and emotion. Camb Handb Personality Psychol. 2009;1:54–71. 10.1017/CBO9780511596544.007 [Google Scholar]
  • 43.Martínez-Tejada LA, Maruyama Y, Yoshimura N, Koike Y. Analysis of personality and EEG features in emotion recognition using machine learning techniques to classify arousal and Valence labels. Mach Learn Knowl Extr. 2020;2(2):7. 10.3390/make2020007 [Google Scholar]
  • 44.Harkness AR, Reynolds SM, Lilienfeld SO. A review of systems for psychology and psychiatry: Adaptive systems, personality psychopathology five (PSY–5), and the DSM–5. Journal of personality assessment, 2014;96(2):121–139. 10.1080/00223891.2013.823438, PMid:23941204. [DOI] [PubMed]
  • 45.Izard CE, Libero DZ, Putnam P, Haynes OM. Stability of emotion experiences and their relations to traits of personality. J Personal Soc Psychol. 1993;64(5):847. 10.1037/0022-3514.64.5.847. PMid:8505713. [DOI] [PubMed] [Google Scholar]
  • 46.Kasek R, Lázár I. Anger inside out. Metacognitive bias primes facial emotional expression in human adults? Semmelweis Symposium, Budapest, 2021;16–18.
  • 47.Karppinen H, King O, Russell PS. Hostile emotions and close relationships: anger can be related to construc-tive responses. Pers Indiv Differ. 2023;212:112258. 10.1016/j.paid.2023.112258 [Google Scholar]
  • 48.Kasek R. Emlékezet És metakogníció Bizonytalanság idején. Érzelmek élettana Járvány idején. Budapest: Károli Gáspár Református Egyetem L’Harmattan Kiadó; 2021;276–87. (Memory and metacognition in times of uncertainty). [Google Scholar]
  • 49.Kasek R, Takacs S, Konya A, Lazar I. Impacts of COVID-19 control measures on metacognition and Spatial cognition. Cogn Process. 2021;22:S53. [Google Scholar]
  • 50.Jung SJ, Jun JY. Mental health and psychological intervention amid COVID-19 outbreak: perspectives from South Korea. Yonsei Med J. 2020;61(4):271–2. 10.3349/ymj.2020.61.4.271. PMid:32233168, PMCid:PMC7105405. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Park SC, Park YC. Mental health care measures in response to the 2019 novel coronavirus outbreak in Korea. Psychiatry Invest. 2020;17(2):85. 10.30773/pi.2020.0058. PMid:32093458, PMCid:PMC7047003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Bzdok D, Dunbar RI. The neurobiology of social distance. Trends Cogn Sci. 2020;24(9):717–33. 10.1016/j.tics.2020.05.016. PMid:32561254, PMCid:PMC7266757. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material 2 (3.9MB, docx)

Data Availability Statement

The data presented in this study are available upon request from the corresponding author for legal agreements and ethical reasons.


Articles from BMC Psychology are provided here courtesy of BMC

RESOURCES