Impact of face masks and sunglasses on emotion recognition in South Koreans

Garam Kim; So Hyun Seong; Seok-Sung Hong; Eunsoo Choi

doi:10.1371/journal.pone.0263466

. 2022 Feb 3;17(2):e0263466. doi: 10.1371/journal.pone.0263466

Impact of face masks and sunglasses on emotion recognition in South Koreans

Garam Kim ¹, So Hyun Seong ¹, Seok-Sung Hong ², Eunsoo Choi ^1,^*

Editor: Marina A Pavlova³

PMCID: PMC8812856 PMID: 35113970

Abstract

Due to the prolonged COVID-19 pandemic, wearing masks has become essential for social interaction, disturbing emotion recognition in daily life. In the present study, a total of 39 Korean participants (female = 20, mean age = 24.2 years) inferred seven emotions (happiness, surprise, fear, sadness, disgust, anger, surprise, and neutral) from uncovered, mask-covered, sunglasses-covered faces. The recognition rates were the lowest under mask conditions, followed by the sunglasses and uncovered conditions. In identifying emotions, different emotion types were associated with different areas of the face. Specifically, the mouth was the most critical area for happiness, surprise, sadness, disgust, and anger recognition, but fear was most recognized from the eyes. By simultaneously comparing faces with different parts covered, we were able to more accurately examine the impact of different facial areas on emotion recognition. We discuss the potential cultural differences and the ways in which individuals can cope with communication in which facial expressions are paramount.

Introduction

Without doubt, we all have had trouble identifying emotional expressions from others in recent years when mask wearing became the norm. In this COVID-19-era, we now live in an environment where wearing masks is a necessity in our daily lives. As a result, we often encounter faces that are only partially exposed, impairing our daily social interactions due to our diminished ability to recognize facial expressions and their associated emotions. Thus, it is particularly important to understand the specific ways in which the ability to correctly infer emotions is restricted when parts of the face are occluded in this unusual time.

Overall, researchers agree that facial expression recognition is hindered when parts of the face are covered. The two key areas of the face that are important for reading facial expressions are the mouth and eyes [1–4]. There have been debates about which is more important, either the eyes or the mouth, in recognizing facial expressions. Notably, emotion type is considered to play a critical role in determining the key facial areas for reading emotions.

Past research focusing on six basic emotions (i.e., happiness, sadness, fear, anger, disgust, and surprise) has found that some emotions render consistent results, while others do not. First, studies on happiness or fear reported consistent results such that the mouth plays a key role in recognizing happiness [5–8] and the eyes in fear recognition [9–13]. Second, as for disgust, a growing number of studies have shown that the mouth has a greater effect than the eyes [11, 12]. Other basic emotions such as surprise, sadness, and anger showed mixed results [6–8, 11–14]. For instance, in detecting sadness, some studies showed that covering the mouth affects facial expression recognition [8], while others have shown more areas including the mouth, eyebrows, eyes, and rigid head posture also contribute to this [5]. Given the current state of the literature, further research is needed.

Some recent studies that empirically confirmed the impact of facial masks, thereby increasing the realism during the COVID-19 outbreak, were noticeable [9, 15–17]. As reviewed by Pavlova & Sokolov [18], recent studies that examined the effects of mask-wearing showed that mask hurts emotion recognition and that the impairment in recognition varied by specific emotions. Carbon [9] conducted the first study to test the effects of mask on recognition for six basic emotions (anger, disgust, fear, happiness, neutral, and sadness) with German adults. He found that when people recognize angry, disgusted, happy, and sad facial expressions, their performance was more impaired under the mask condition than under the no-mask condition, suggesting that the mouth area provides an important source of information for inferring these emotions from the face. However, there was no significant difference between the conditions in recognition for fear, indicating that the eyes (with and without a visible mouth) were sufficient for detecting fear. This study sheds light on the effects of wearing a mask, that is, the effects of occlusion of the mouth area on the recognition of different emotions in everyday contexts. These findings were also replicated with children age 9–10 years [15].

One limitation of these studies, however, is that the effects of eyes, which are critical in emotional expression and recognition of faces, were not examined. These studies make it difficult to identify whether the impairment is entirely due to the lack of information from the mouth or additional interference from the eyes. Alternatively, disturbance in emotion recognition for faces covered with masks may be buffered by the information obtained from the eyes. Thus, it would be important to make direct and simultaneous comparisons between the mouth-covered, eye-covered, and uncovered conditions, which would allow us to determine which facial area is important, and whether it depends on types of emotions.

Recently, there have been few studies that addressed this limitation and examined the effects of occlusion of both the mouth and the eyes on emotion recognition [16, 17]. Most notably, Noyes et al. [17] conducted an experiment that included the mask, sunglasses, and control conditions with UK adults. They found that the overall emotion recognition accuracy was the lowest when the mouth was occluded. However, the effects of the conditions differed by specific emotions. For instance, the effects of mask condition and sunglasses condition on the recognition of angry faces were not significantly different, and the mask condition rendered a greater accuracy rate than the sunglasses condition for sad faces. However, these findings seem inconsistent. For example, Carbon’s study [9] showed that there was an impairment in recognition for sadness under masked conditions. Thus, more research is required to test whether the findings are robust.

Finally, one glaring limitation of prior research is that the participants were primarily from Western cultural contexts. It has generally been found that there is a cultural difference in emotional recognition between Asians and Westerners [19–21]. Specifically, previous studies document that East Asians tend to focus on the eyes rather than the mouth, whereas Westerners tend to focus on the entire face [22]. Other studies have shown that East Asians concentrate more on the eye area (upper face) compared to Westerners [10, 21, 23]. If this were the case, then Asians’ emotion recognition for mask wearing faces would be less impaired, but somewhat more impaired for faces covered with sunglasses. Moreover, whereas Westerners are more accustomed to faces with sunglasses than to masked faces [17], East Asians have been wearing masks since pre-COVID-19 and are relatively more familiar with masked faces [24]. Also, the effects of masks or sunglasses on reading other person’s facial expressions may differ depending on the cultural context (for a review see [18]). As an example, consider the findings that a face covered with Islamic headdresses such as niqāb impacts the recognition of emotions differently by cultural groups [13, 25], suggesting that a culturally attached meaning of headdress may play a role. As for East Asians, they are not only less accustomed to sunglasses than Westerners, but sunglasses are often considered rude in interpersonal relationships [26, 27]. Given such cultural background, thus, it is necessary to test whether the effects of masks and sunglasses on facial expression recognition that are documented with Western participants would also apply to East Asians.

Present study

Building on past studies, the present study aims to fill the gaps in previous research. To do this, we first conducted an integrative investigation that examined the effects of both the mouth and eye regions with realistic facial occlusions (i.e., mask and sunglasses) on six basic emotions. Second, going beyond the past studies that have been primarily conducted in the West, we aimed to test whether similar findings are observed in non-Western samples. This is important because many previous studies have demonstrated that there are cultural differences in the areas of the face to which people pay attention.

In the present study, we used a within-subject design with Korean undergraduate students as participants. To enhance the realism of the stimuli, we took steps to edit real-world masks and sunglasses images to face stimuli rather than simply cutting or covering images with a black box, as in previous studies [8, 13, 14, 28]. In addition to the main research question, we tested the effect of sex on facial expression recognition. According to previous studies, women appear to perform better than men when distinguishing facial expressions in faces covered with masks [29, 30]. In addition, it was found that individuals better identified (and labeled) emotions from facial expressions when the target was of different sex [31]. Thus, we examined whether emotion recognition differs according to the sex of the participant and the sex of the face stimuli.

Methods

Participants

Based on a power analysis (G-power software version 3.1), a total of 28 participants were required for the current experiment, which was designed with a repeated-measures ANOVA. Given an expected effect size of .25, and α = 0.05, this led to an acceptable power of .8 [32]. However, we sampled more participants than the required number for potential participants who would get excluded if computers malfunction or those who did not meet the pre-set exclusion criteria (less than a 50% correct facial expression recognition answer rate when presented with a fully visible face without a face mask or sunglasses were excluded). This pre-determined criterion was based on that of Carbon’s method [9]. Through an advertisement post on a Korean university community website, 40 students voluntarily registered for the study. All participants had normal or corrected-to-normal vision with no abnormal color discrimination ability. Participants received a 10,000 won (approximately 9 dollars) beverage gift card for participation in the study. This study was approved by the Institutional Review Board of Korea University (KUIRB-2020-0317-01). During the analysis, participants with less than a 50% correct facial expression recognition answer rate when presented with a fully visible face without a face mask or sunglasses were excluded. With 39 participants exceeding a 50% correct answer rate (average = 0.67), the final sample consisted of 39 participants (male = 19, female = 20, M_age = 24.2 years, SD_age = 4.7 years). The data can be accessed at https://osf.io/fcg4d/.

Materials

The Korean facial expression data used in the study was obtained from the Korea University Facial Expression Collection (KUFEC) [33]. The KUFEC was developed to minimize the cross-race effect by using the faces of Korean models [34]. Through an agreement between two researchers, six male and female models with relatively more accurate facial expressions were selected from a total of 49 models. Each identity posed a facial expression depicting happiness, surprise, fear, sadness, disgust, anger, and a neutral emotion. These were then photoshopped using Adobe Photoshop CC 2020 by adding surgical face masks (mask condition) and sunglasses (sunglasses condition). In total, there were 252 facial stimuli: 2 (sex) × 6 (individuals) × 7 (emotions) × 3 (uncovered vs. mask vs. sunglasses). All factors were within-subject factors.

Procedure

The experiments were conducted in individually assigned cubicles, where the participants were briefly informed about the study and signed an informed consent form. During the experiment, participants were instructed to recognize facial expressions in a picture in which the facial stimuli were presented randomly across all factors (sex, emotion type, and condition). The stimuli were presented using E-Prime 3.0. The facial expression stimuli were presented on the left side of the monitor and a 3 × 3 table on the right side, with each emotional example entered as corresponding positions up to seven on a numeric keyboard. Participants were instructed to respond as fast and accurately as possible by pressing the numeric key that was aligned to the facial expression represented by the presented picture. An example of this task is shown in Fig 1.

Fig 1 — Participants chose the emotional example that aligned to the facial expressions presented after 0.5-second fixation. All facial stimuli were presented randomly. Due to copyright, the presented face is an example photo, not KUFEC.

The participants completed six practice trials and then performed 252 test trials. After completion of the test trials, participants responded to a questionnaire regarding demographic information, were involved in a debriefing of the study, and received their participation rewards.

Results

We tested the accuracy of the participants’ baseline recognition rates by presenting the uncovered faces. Taking into consideration the chance rate of 14.2%, we can safely assume that the average correct answer rate of 68% for a fully visible face was well above the chance level (χ² = 21312.458, p < .001, Cramer’s V = 0.601, p < .001). We tested whether there were any sex or age effects on facial recognition accuracy. Sex differences in the overall correct answer rate were not statistically significant (t = -1.303, p = .203), nor was there a significant relationship between age recognition accuracy (F = .032, p = .86).

Before analyzing the main results, all data were checked for normality distribution. Normality was violated for some variables but we decided to use the repeated measures ANOVA because ANOVAs are generally considered robust to nonnormality with sample sizes being equal (within-subject design) [35]. A 3 (face condition) × 7 (emotion type) within-subjects ANOVA was conducted to test the main and interaction effects of the two variables: face regions that were covered and type of emotions. We reported the results of Greenhouse–Geisser correction whenever Mauchly’s test of sphericity was significant. First, we predicted that the accuracy would be lower for recognizing faces wearing masks than for uncovered faces. The results showed that there was a significant main effect of face condition, F(2, 76) = 147.34, p < .001, η_p² = .80, with the highest recognition rate observed for the uncovered condition (M = .68, SD = .01), followed by the sunglasses (M = .64, SD = .01) and mask conditions (M = .51, SD = .01). Specifically, pairwise comparisons demonstrated that the accuracy under the mask condition was lower than both the uncovered condition, t(38) = -15.98, p < .001, and the sunglasses condition, t(38) = -11.4, p < .001. Not surprisingly, recognition accuracy under the sunglasses condition was lower than that under the uncovered condition, t(38) = -4.74, p < .001. Covering parts of the face, whether it is the eyes or mouth, hinders the performance of facial expression recognition. In particular, covering the mouth rather than the eyes made it more difficult to identify emotions.

There was also a significant main effect of emotion, using the Greenhouse-Geisser correction, F(4.04, 153.44) = 211.96, p < .001, η_p² = .85. Specifically, happiness and neutral emotion had higher recognition rates compared to all other emotions (ts ≥ 3.45, ps ≤ .012; | ts | ≥ 3.78, ps ≤ .004, respectively), whereas fear showed the lowest recognition rates (| ts | ≥ 12.28, ps < .001). In summary, facial expression recognition accuracy was high in the order of happiness, neutral—surprise—sadness, disgust, anger—fear.

Do different face regions (eye vs. mouth) make a difference in recognizing specific emotions?

In the previous analysis, participants’ emotion recognition was significantly impaired when the mouth (and also a part of the nose) was covered (mask condition) than when the eyes were covered (sunglasses condition). Next, we tested whether the impaired facial recognition by mouth occlusion (vs. eye occlusion) would depend on specific emotional expressions in the face. Based on prior research, we expected that emotions such as happiness and disgust, the recognition of which were more influenced by the mouth (vs. the eyes) would be more impaired by wearing a mask than sunglasses; in contrast, we expected that for emotions such as fear, for which the eye has a higher impact on emotion recognition, would be less recognized by wearing sunglasses compared to a mask.

As shown in Fig 2, the interaction between face conditions and emotions was significant, F(7.29, 277.04) = 15.22, p < .001, η_p² = .29. To interpret the interaction between the face conditions and the emotion type, we conducted pairwise comparisons by face condition in each emotion and examined whether the differences in accuracy rates between the mask, sunglasses and uncovered conditions would depend on the specific emotions. For instance, emotions such as happiness and disgust, which require cues primarily from the mouth, will show particularly lower accuracy under mask conditions than under sunglasses conditions. The results are as follows:

Happiness

The recognition accuracy of happiness under the mask condition (M = .84, SE = .02) was lower than that of both the sunglasses (M = .97, SE = .01), t(38) = -8.8, p < .001, and the uncovered conditions (M = .92, SE = .02), t(38) = -3.58, p = .001. The recognition accuracy of the faces with sunglasses was higher than that of the uncovered faces, t(38) = 2.76, p = .009. This result was consistent with prior research that documented that the mouth is especially informative in the identification of happiness. However, it is noteworthy that participant accuracy in the sunglasses condition, which covered the eyes and left the mouth visible, was higher than that in the uncovered face condition. A possible reason for why people recognized happy faces with sunglasses better, and not worse, than uncovered happy faces may be because this allowed participants to concentrate only on the mouth without being distracted by the eyes. Previous studies have demonstrated that facial recognition performance improves by covering less important parts of the face [25]. When including both the mask and the sunglasses conditions, we were able to show that covering relatively less important areas in a face can actually increase recognition performance of certain emotions. Specifically, the present findings showed that people can perceive happiness by focusing only on the mouth, and information from the eyes interferes, rather than facilitates, emotion recognition.

Disgust

For disgust, participants were less accurate when faces were covered by a mask (M = .34, SE = .03) than when they were uncovered (M = .56, SE = .03), t(38) = -6.31, p < .001, and when sunglasses were worn (M = .52, SE = .03), t(38) = -5.97, p < .001. Recognition accuracy for uncovered faces and sunglasses did not differ, t(38) = -1.42, p = .164. Considering that there was no significant difference between the sunglasses and uncovered conditions in disgust, disgust seems to be recognized from the mouth only, and the eyes do not play a major role.

Sadness

Sad faces with masks (M = .32, SE = .04) showed lower detection accuracy than uncovered faces (M = .64, SE = .03), t(38) = -9.13, p < .001, and faces with sunglasses (M = .48, SE = .03), t(38) = -4.15, p < .001. Recognition accuracy under the sunglasses condition was also lower than that under the uncovered condition, t(38) = -6.34, p < .001. That is, it was particularly difficult to recognize sadness when the mouth was covered compared to when the eyes were covered.

Anger

As for anger, faces covered by masks showed lower accuracy (M = .42, SE = .03) than uncovered faces (M = .62, SE = .03), t(38) = -5.58, p < .001, and those covered by sunglasses (M = .53, SE = .04), t(38) = -2.68, p = .011. Participants recognized angry faces with sunglasses less accurately than uncovered faces, t(38) = -2.75, p = .009. Thus, as with sadness, facial expression recognition of anger showed a significant decrease in accuracy in both the sunglasses and mask conditions, but especially in the mask condition. In other words, when the eyes, but not the mouth, were covered, people could still make emotional inferences of sad or angry faces from the mouth to some degree. Furthermore, the lower accuracy in the sunglasses condition compared to the uncovered condition further corroborates that people detect sadness and anger from the eyes as well.

Surprise

As for the emotion of surprise, participants recognized the emotions in the masked condition less accurately (M = .61, SE = .03) than in the uncovered condition (M = .93, SE = .02), t(38) = -12.05, p < .001, and the sunglasses condition (M = .9, SE = .02), t(38) = -10.35, p < .001. The recognition accuracy of uncovered faces and faces with sunglasses did not differ, t(38) = 1.36, p = .181. That is, people were able to recognize surprise by focusing on the mouth and did not gain much from focusing on the eyes.

Fear

Interestingly, recognition of fear did not seem to be affected by the covering of the mouth. The recognition accuracy of fear under the mask condition (M = .14, SE = .03) was similar to the uncovered condition (M = .15, SE = .03), t(38) = -.53, p = .602. However, accuracy under the sunglasses condition (M = .1, SE = .02) was lower than that under the other two conditions, | ts | ≥ 2.14, ps ≤ .039. This suggests that covering the eyes, but not the mouth, decreased recognition. Consistent with previous results, people tend to recognize fearful faces from the eyes (upper facial area) [9, 12].

Neutral

Finally, neutral faces with sunglasses showed higher recognition rates (M = .95, SE = 0.01) compared to uncovered faces (M = .91, SE = .02), t(38) = 2.4, p = .021, and faces covered by masks (M = .90, SE = .03), t(38) = 2.61, p = .013. The recognition accuracy of neutral expressions under the mask and uncovered conditions did not differ, t(38) = .937, p = .355.

How were the emotions misinterpreted?

We further analyzed how participants incorrectly recognized the emotions of a face covered by a mask. This will allow us to better understand how the lack of cues from masked faces is dealt with and provides real-world implications for individuals living during a pandemic. The percentages of correct and incorrect answers to all emotions are presented in the confusion matrix of emotions (Fig 3).

Fig 3 — Rows correspond to the recognized emotion (participants’ responses) and columns correspond to the presented (correct) emotion.

The correct answer rate of sadness in an uncovered face was 64.1%, but the correct answer rate of the sad face wearing a mask fell dramatically to 32.3%, of which 20.7% of participants incorrectly recognized as disgust. As for the emotion of disgust, the correct answer rate for the uncovered faces was 56.2%, but it dropped substantially to 34% when the face was covered with a mask. In this case, 24.8% of participants mistakenly recognized it as sadness and 23.7% as anger. This suggests that without the facial configuration of the mouth, sadness can easily be confused with disgust. Consistent with the present study, Carbon [9] also demonstrated that sadness, disgust, and anger were confused with each other in masked faces. Additionally, sadness was detected with 64.1% accuracy under uncovered conditions; however, this decreased to 47.9% in the sunglasses condition, where 23.9% misrecognized sadness as disgust. Furthermore, in the case of anger, the 61.5% accuracy rate in the uncovered condition decreased to 52.6% under the sunglasses condition, and 25.2% misrecognized it as disgust and 14.3% as neutral. Therefore, sadness, disgust, and anger were likely to be confused with each other under sunglasses conditions as well.

As for surprise, the recognition rate was 61.3% in the mask condition, and 28.8% of participants misrecognized it as a neutral emotion. However, for the neutral faces with masks, the correct answer rate was 89.5%, with very few people mistaking these for other emotions. That is, wearing a mask increased the possibility of misrecognizing surprise as a neutral expression, but the reverse was not the case.

The other emotions discussed earlier (i.e., sadness, disgust, and anger) were likely to be confused with each other, but interestingly, the confusion between surprise and neutral emotions was asymmetrical. Interestingly, surprise was not confused with a neutral emotion in the sunglasses condition, as indicated by its high recognition rate of 90%; the misrecognition of surprise as a neutral emotion was only observed when the mouth was covered.

Another point to note in our findings is the misinterpretation of fear as surprise. Under the sunglasses condition, 66% of the participants incorrectly recognized fear as surprise. Moreover, 65% of the participants recognized fear as surprise when a face was uncovered, and these were often confused with one another [12, 36, 37]. In a study that used the KUFEC, participants were not able to make a clear distinction between surprise and fear [34].

Does sex matter?

In this experiment, picture stimuli consisted of six male and six female models. We tested whether there was a difference in the recognition accuracy of facial expressions depending on the sex of the target face. We conducted a repeated-measures ANOVA with the sex of the face as a within-subject factor and the sex of the participant as a between-subject factor. The results showed that the main effect of the sex of the stimuli was statistically significant, F(1, 37) = 5.87, p = .02, η_p² = .14, indicating that the accuracy was higher when female faces (vs. male faces) were presented, t(37) = 2.42, p = .02. However, the main effect of the sex of the participants and the interaction between the sex of the participants and the sex of the face stimuli were not statistically significant, F(1,37) = 1.74, p = .20; F(1, 37) = 1.42, p = .24; respectively. Thus, participants were more accurate in recognizing the emotions of female faces, regardless of their sex.

Discussion

In this experiment, we set out to test whether facial occlusion in real-world settings such as occlusion with facial masks and sunglasses impairs recognition of emotions in a face, with a particular focus on the effects of mask wearing, which is still mandatory in many parts of the world due to the COVID-19 pandemic. We did this by comparing the recognition of six basic emotions under masked conditions compared to sunglasses or uncovered conditions. We found that the recognition rates for faces with masks and sunglasses were lower than for the uncovered faces. Wearing a mask particularly harmed the recognition of emotional expressions in the face, as indicated by a greater decrease in recognition under the mask condition compared to under the sunglasses condition. Specifically, happiness, surprise, sadness, disgust, and anger showed the lowest recognition accuracy when the mouth was covered (i.e., mask condition), suggesting the important role played by the mouth in these emotions. Fear, on the other hand, showed the lowest recognition accuracy when the eyes were covered (i.e., the sunglasses condition). As for the neutral face, there was no significant difference between the uncovered and masked faces. Taken together, our findings suggest that facial masks cause the most striking decline in recognition of the majority of basic emotions among Koreans, which is consistent with prior research on Western participants. In addition to replicating prior studies, the present study has several noteworthy findings.

By simultaneously comparing faces with different parts covered, the present study found that covering certain parts of the facial areas increases, not decreases, recognition of some emotions. For instance, when the eyes were covered, happiness was recognized better than when faces were uncovered, suggesting that covering the upper facial area, which is not critical for perceiving happiness, could actually facilitate emotion recognition. Moreover, with the addition of the sunglasses condition in the present study, the critical role of the mouth in emotion recognition was confirmed in emotions such as surprise and disgust, as similar accuracy rates were observed for faces with sunglasses and uncovered faces. On the other hand, emotions such as sadness and anger in faces with sunglasses were not as recognizable as in uncovered faces, indicating that people attain facial expression information for these emotions from the eyes in addition to the mouth.

In addition to examining the overall recognition rate of different emotions when different parts of the face are covered, it is worthwhile to have a closer look at how participants misinterpreted the emotional expressions of these faces. Specifically, negative emotions, including sadness, disgust, and anger, were often confused with one another. In particular, when the mouth was covered (i.e., mask condition), sadness and disgust were often misidentified as one another. That is, without any cues from the mouth, people have trouble distinguishing these emotions. These findings offer practical advice for people who communicate while wearing a face mask. For instance, individuals who are sad may benefit from knowing that people may misrecognize their sadness as disgust. Likewise, it would be helpful for people to know that what seems like a disgusted face may actually be a sad face.

Finally, to our knowledge, this is the first study to test the effect of mask wearing on emotion recognition with a direct comparison to the effects of sunglasses among East Asians. It is widely known that East Asians focus on the eyes while Westerners focus on the mouth when recognizing facial expressions. Thus, we expected a decrease in recognition accuracy in the sunglasses condition, as the eyes are especially important in recognizing facial expressions for East Asians. The results showed that there was indeed a decrease in the recognition of faces with sunglasses. However, the largest decrease in recognition was observed for the masked faces, more than the faces with sunglasses, which is consistent with Noyes et al. [17], suggesting that the mouth is the most important source of information for most basic emotions for Koreans as well. The overall facial expression recognition accuracy was similar to that of Westerners [9, 17], but some emotions showed different patterns. For instance, in our findings, the mask had destructive effects on sad faces, but not on fearful faces, which was consistent with Germans [9] but not with British people [17]. Specifically, comparing faces covered with masks and faces covered with sunglasses, our participants recognized fearful faces more accurately in mask conditions and sad faces in sunglasses conditions, while British participants showed the opposite patterns. In addition, our participants had difficulty classifying surprise and neutral states from the faces who wore shades compared to uncovered faces, but there was no difference in British participants. This suggests that East Asians retrieve information about emotions from eyes more than Westerners when reading facial expressions of surprise. However, there was also a finding that Westerners, rather than East Asians, read emotions from eyes. For instance, our participants showed higher accuracy for angry faces in sunglasses than mask conditions, but UK participants showed no significant difference between these conditions, suggesting that the mouth area was more informative for Koreans, but both mouth and eye areas were informative for British people. However, since these participants were not collected at the same time with the same procedures, the current study cannot directly compare with British people, making it difficult to determine the influence of culture. Therefore, future research that directly compares the differences between East Asian and Western participants is needed.

Finally, participant gender did not have any effect on recognition rate nor did they interact with the target’s gender. However, the target stimuli’s gender mattered. Overall, participants recognized female faces better than male faces. More research is needed to replicate this effect.

Limitations and future research

Some limitations of our study should be considered in future research. First, because the participants in our study were limited to undergraduate students, possible age effects were not tested. Some studies suggest that people have more difficulty recognizing older faces than middle-aged or younger faces [9, 38], and that middle-aged participants identify facial expressions better than children or older adults [39]. Future research should therefore examine the effects of the age of both the raters and the target faces with realistic facial occlusions. Second, the present study used still photos of facial expressions, which is far from the real-world communication of facial expressions. Recently, researchers measured the accuracy of emotional recognition using video stimulation, which adds a static background to the dynamic facial expression set [40]. However, since this study included only two emotions, happiness and sadness, it is difficult to grasp the effect of on recognition of various emotions in facial expression. Moreover, the face is not the only channel through which emotions are recognized. Other information such as context [41], body language [42, 43], and voice [44] are also used to express or read emotions. Thus, future studies using video stimuli that include various emotions and involve other sources of information would render a more ecologically valid effect of facial occlusion on emotion recognition.

Wearing a facial mask will not go out of fashion anytime soon, as COVID-19 is unlikely to cease and continues to stay with us as an endemic influenza [45]. Given this situation, the present line of research could be developed so that we can find ways in which people can adapt to restricted communication due to facial masks. An intervention that targets specific groups that are particularly vulnerable to the current situation is a good example. For instance, service employees who are required to recognize customers’ facial expressions while masks are worn are especially at a disadvantage. Given that tourism and hospitality staff showed an increase in both accuracy and speed of facial expression recognition [46], a similar training would enhance emotion recognition for masked faces as well. Another group that requires special attention is children. In recent studies, researchers have found that children’s emotion recognition is also affected by facial masks [15, 16, 47]. Given that childhood is an important developmental stage for the socialization of emotion [48, 49] and a potential influence of mask wearing on the development of necessary social interaction skills [18], it would be important not only to understand the effect of facial occlusion among children but also to develop strategies to help young children learn to read emotions from others.

In conclusion, our results suggest that the mouth is more important than the eyes in facial expression recognition. As mask wearing is becoming increasingly important, future studies should investigate effective ways, such as interventions, to minimize miscommunication from incorrectly reading emotions from a face.

Acknowledgments

We would like to thank Dr. June-Seek Choi for letting us use the KUFEC stimuli set for this study.

Data Availability

The raw data is available from the Open Science Framework (https://osf.io/fcg4d/).

Funding Statement

E.C received the grant from School of Psychology, Korea University (K2110491). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

References

1.Eisenbarth H, Alpers GW. Happy mouth and sad eyes: scanning emotional facial expressions. Emotion. 2011;11(4):860–5. doi: 10.1037/a0022758 [DOI] [PubMed] [Google Scholar]
2.Gosselin F, Schyns PG. Bubbles: a technique to reveal the use of information in recognition tasks. Vision Research. 2001;41(17):2261–71. doi: 10.1016/s0042-6989(01)00097-9 [DOI] [PubMed] [Google Scholar]
3.Kontsevich LL, Tyler CW. What makes Mona Lisa smile? Vision Research. 2004;44(13):1493–8. doi: 10.1016/j.visres.2003.11.027 [DOI] [PubMed] [Google Scholar]
4.Schmidtmann G, Logan AJ, Carbon C-C, Loong JT, Gold I. In the Blink of an Eye: Reading Mental States From Briefly Presented Eye Regions. i-Perception. 2020;11(5). doi: 10.1177/2041669520961116 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Bombari D, Schmid PC, Schmid Mast M, Birri S, Mast FW, Lobmaier JS. Emotion Recognition: The Role of Featural and Configural Face Information. Quarterly Journal of Experimental Psychology. 2013;66(12):2426–42. doi: 10.1080/17470218.2013.789065 [DOI] [PubMed] [Google Scholar]
6.Cunningham DW, Kleiner M, Bülthoff HH, Wallraven C. The components of conversational facial expressions. Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization; Los Angeles, California, USA: Association for Computing Machinery; 2004; 143–50. doi: 10.1145/1012551.1012578 [DOI] [Google Scholar]
7.Fraser IH, Craig GL, Parker DM. Reaction Time Measures of Feature Saliency in Schematic Faces. Perception. 1990;19(5):661–73. doi: 10.1068/p190661 [DOI] [PubMed] [Google Scholar]
8.Kotsia I, Buciu I, Pitas I. An analysis of facial expression recognition under partial facial image occlusion. Image and Vision Computing. 2008;26(7):1052–67. doi: 10.1016/j.imavis.2007.11.004 [DOI] [Google Scholar]
9.Carbon C-C. Wearing Face Masks Strongly Confuses Counterparts in Reading Emotions. Frontiers in Psychology. 2020;11. doi: 10.3389/fpsyg.2020.566886 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Jack RE, Garrod OGB, Yu H, Caldara R, Schyns PG. Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences. 2012;109(19):7241–4. doi: 10.1037/a0023463 [DOI] [PMC free article] [PubMed] [Google Scholar]
11.Nestor MS, Fischer D, Arnold D. “Masking” our emotions: Botulinum toxin, facial expression, and well-being in the age of COVID-19. Journal of Cosmetic Dermatology. 2020;19(9):2154–60. doi: 10.1111/jocd.13569 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Wegrzyn M, Vogt M, Kireclioglu B, Schneider J, Kissler J. Mapping the emotional face. How individual face parts contribute to successful emotion recognition. PLOS ONE. 2017;12(5). doi: 10.1371/journal.pone.0177239 [DOI] [PMC free article] [PubMed] [Google Scholar]
13.Fischer AH, Gillebaart M, Rotteveel M, Becker D, Vliek M. Veiled Emotions: The Effect of Covered Faces on Emotion Perception and Attitudes. Social Psychological and Personality Science. 2012;3(3):266–73. doi: 10.1177/1948550611418534 [DOI] [Google Scholar]
14.Song I-H, Cho K-J, Chim H-R. The effect of emotional dimension and facial expression’s presenting areas on facial expression’s recognition: A comparison of gender differences. Korean Journal of Woman Psychology. 2008;13(2):197–214. [Google Scholar]
15.Carbon C-C, Serrano M. The Impact of Face Masks on the Emotional Reading Abilities of Children—A Lesson From a Joint School–University Project. i-Perception. 2021;12(4). doi: 10.1177/20416695211038265 [DOI] [PMC free article] [PubMed] [Google Scholar]
16.Ruba AL, Pollak SD. Children’s emotion inferences from masked faces: Implications for social interactions during COVID-19. PLOS ONE. 2020;15(12). doi: 10.1371/journal.pone.0243708 [DOI] [PMC free article] [PubMed] [Google Scholar]
17.Noyes E, Davis JP, Petrov N, Gray KLH, Ritchie KL. The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers. Royal Society Open Science. 2021;8(3). doi: 10.1098/rsos.201169 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Pavlova MA, Sokolov AA. Reading Covered Faces. Cerebral Cortex. 2021;32(2):249–65. doi: 10.1093/cercor/bhab311 [DOI] [PubMed] [Google Scholar]
19.Ekman P, Friesen WV, O’Sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, et al. Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology. 1987;53(4):712–7. doi: 10.1037//0022-3514.53.4.712 [DOI] [PubMed] [Google Scholar]
20.Matsumoto D, Ekman P. American-Japanese cultural differences in intensity ratings of facial expressions of emotion. Motivation and Emotion. 1989;13(2):143–57. doi: 10.1007/BF00992959 [DOI] [Google Scholar]
21.Yuki M, Maddux WW, Masuda T. Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology. 2007;43(2):303–11. doi: 10.1016/j.jesp.2006.02.004 [DOI] [Google Scholar]
22.Jack RE, Blais C, Scheepers C, Schyns PG, Caldara R. Cultural Confusions Show that Facial Expressions Are Not Universal. Current Biology. 2009;19(18):1543–8. doi: 10.1016/j.cub.2009.07.051 [DOI] [PubMed] [Google Scholar]
23.Haensel JX, Danvers M, Ishikawa M, Itakura S, Tucciarelli R, Smith TJ, et al. Culture modulates face scanning during dyadic social interactions. Scientific Reports. 2020;10(1):1–11. doi: 10.1038/s41598-019-56847-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
24.Baehr P. City under siege: authoritarian toleration, mask culture, and the SARS crisis in Hong Kong. Networked Disease: Emerging Infections in the Global City. 2008;138–51. doi: 10.1002/9781444305012.ch8 [DOI] [Google Scholar]
25.Kret M, De Gelder B. Islamic Headdress Influences How Emotion is Recognized from the Eyes. Frontiers in Psychology. 2012;3. doi: 10.3389/fpsyg.2012.00110 [DOI] [PMC free article] [PubMed] [Google Scholar]
26.Gesteland RR. Cross-cultural business behavior: A guide for global management. Copenhagen Business School Press DK; 2012. [Google Scholar]
27.Miyazawa D, Kaneko G. Japan’s key “X-factor” for low COVID-19 mortality may be its culture of wearing face masks. Authorea Preprints. 2020. doi: 0.22541/au.159373225.52467275 [Google Scholar]
28.Roberson D, Kikutani M, Döge P, Whitaker L, Majid A. Shades of emotion: What the addition of sunglasses or masks to faces reveals about the development of facial expression processing. Cognition. 2012;125(2):195–206. doi: 10.1016/j.cognition.2012.06.018 [DOI] [PubMed] [Google Scholar]
29.Freud E, Stajduhar A, Rosenbaum RS, Avidan G, Ganel T. The COVID-19 pandemic masks the way people perceive faces. Scientific Reports. 2020;10(1). doi: 10.1038/s41598-020-78986-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
30.Grundmann F, Epstude K, Scheibe S. Face masks reduce emotion-recognition accuracy and perceived closeness. PLOS ONE. 2021;16(4). doi: 10.1371/journal.pone.0249792 [DOI] [PMC free article] [PubMed] [Google Scholar]
31.Hofmann SG, Suvak M, Litz BT. Sex differences in face recognition and influence of facial affect. Personality and Individual Differences. 2006;40(8):1683–90. doi: 10.1016/j.paid.2005.12.014 [DOI] [Google Scholar]
32.Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Abingdon. England: Routledge. 1988. [Google Scholar]
33.Lee TH, Lee KY, Lee K, Choi J-S, Kim HT. The Korea University facial expression collection: KUFEC. Lab of Behavioral Neuroscience, Dept of Psychology, Korea University, Seoul, South Korea. 2006. [Google Scholar]
34.Kim MW, Choi J-S, Cho YS. The Korea University Facial Expression Collection (KUFEC) and semantic differential ratings of emotion. Korean Journal of Psychology: General. 2011;30(4):1189–2111. [Google Scholar]
35.Blanca Mena MJ, Alarcón Postigo R, Arnau Gras J, Bono Cabré R, Bendayan R. Non-normal data: Is ANOVA still a valid option? Psicothema. 2017;29(4):552–7. doi: 10.7334/psicothema2016.383 [DOI] [PubMed] [Google Scholar]
36.Ekman P. Emotions revealed: recognizing faces and feelings to improve communication and emotional life. New York. NY: Times books. 2003. [Google Scholar]
37.Roy-Charland A, Perron M, Beaudry O, Eady K. Confusion of fear and surprise: A test of the perceptual-attentional limitation hypothesis with eye movement monitoring. Cognition and Emotion. 2014;28(7):1214–22. doi: 10.1080/02699931.2013.878687 [DOI] [PubMed] [Google Scholar]
38.Fölster M, Hess U, Werheid K. Facial age affects emotional expression decoding. Frontiers in Psychology. 2014;5. doi: 10.3389/fpsyg.2014.00030 [DOI] [PMC free article] [PubMed] [Google Scholar]
39.Horning SM, Cornwell RE, Davis HP. The recognition of facial expressions: An investigation of the influence of age and cognition. Aging, Neuropsychology, and Cognition. 2012;19(6):657–76. doi: 10.1080/13825585.2011.645011 [DOI] [PubMed] [Google Scholar]
40.Kastendieck T, Zillmer S, Hess U. (Un)mask yourself! Effects of face masks on facial mimicry and emotion perception during the COVID-19 pandemic. Cognition and Emotion. 2021:1–11. doi: 10.1080/02699931.2021.1950639 [DOI] [PubMed] [Google Scholar]
41.Chóliz M, Fernández-Abascal EG. Recognition of Emotional Facial Expressions: The Role of Facial and Contextual Information in the Accuracy of Recognition. Psychological Reports. 2012;110(1):338–50. doi: 10.2466/07.09.17.PR0.110.1.338-350 [DOI] [PubMed] [Google Scholar]
42.Van den Stock J, Righart R, de Gelder B. Body expressions influence recognition of emotions in the face and voice. Emotion. 2007;7(3):487–94. doi: 10.1037/1528-3542.7.3.487 [DOI] [PubMed] [Google Scholar]
43.Clarke TJ, Bradshaw MF, Field DT, Hampson SE, Rose D. The Perception of Emotion from Body Movement in Point-Light Displays of Interpersonal Dialogue. Perception. 2005;34(10):1171–80. doi: 10.1068/p5203 [DOI] [PubMed] [Google Scholar]
44.de Gelder B, Vroomen J. The perception of emotions by ear and by eye. Cognition and Emotion. 2000;14(3):289–311. doi: 10.1080/026999300378824 [DOI] [Google Scholar]
45.Beams AB, Bateman R, Adler FR. Will SARS-CoV-2 Become Just Another Seasonal Coronavirus? Viruses. 2021;13(5):854. doi: 10.3390/v13050854 [DOI] [PMC free article] [PubMed] [Google Scholar]
46.Koc E, Boz H. Development of hospitality and tourism employees’ emotional intelligence through developing their emotion recognition abilities. Journal of Hospitality Marketing & Management. 2020;29(2):121–38. doi: 10.1080/19368623.2019.1608885 [DOI] [Google Scholar]
47.Gori M, Schiatti L, Amadeo MB. Masking Emotions: Face Masks Impair How We Read Emotions. Frontiers in Psychology. 2021;12. doi: 10.3389/fpsyg.2021.669432 [DOI] [PMC free article] [PubMed] [Google Scholar]
48.Denham SA, Bassett HH, Zinsser K, Wyatt TM. How Preschoolers’ Social–Emotional Learning Predicts Their Early School Success: Developing Theory-Promoting, Competency-Based Assessments. Infant and Child Development. 2014;23(4):426–54. doi: 10.1002/icd.1840 [DOI] [Google Scholar]
49.Izard C, Fine S, Schultz D, Mostow A, Ackerman B, Youngstrom E. Emotion Knowledge as a Predictor of Social Behavior and Academic Competence in Children at Risk. Psychological Science. 2001;12(1):18–23. doi: 10.1111/1467-9280.00304 [DOI] [PubMed] [Google Scholar]

PLoS One. doi: 10.1371/journal.pone.0263466.r001

Decision Letter 0

Marina A Pavlova

22 Sep 2021

PONE-D-21-25701Is wearing a mask masking one's emotion?PLOS ONE

Dear Dr. Choi

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. We have now feedback from Reviewer ! who is an expert in the field, and (since we have some difficulties in securing reviewers due to summer time) I had a close look at your submission. Both of us find your work interesting. Yet we both require a major revision of your manuscript. Reviewer 1 provided you with a thorough review and a number of major and minor concerns and suggestions (please find below). My main concerns are:1) You have update your Introduction and Discussion taking into account latest studies in the field. Our analysis Pavlova and Sokolov READING COVERED FACES. Cerebral Cortex 2021 may be of substantial help; 2) I agree with Reviewer 1 that the title should be modified to better reflect the specificity and novelty of your study. Please carefully consider what is new in your work as compare with previous research; 3) IMPORTANT; did you check the data sets for normality of distribution and how? Please make a cklear statement on it in the Method Section. Parametric statistics can be used only if the data sets are normally distributed, otherwise, please use non-parametric statistics; 4) I appreciate that you analysed the effect of a poser gender. But did you analyse the effect of observers' gender on performance? PLease address all issues on the point-by-point basis in your rebuttal letter to me.

Please submit your revised manuscript by Nov 06 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Marina A. Pavlova, PhD

Academic Editor

PLOS ONE

Journal requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following financial disclosure:

“E.C received the grant from School of Psychology, Korea University (K2110491).”

Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

If this statement is not correct you must amend it as needed.

Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in your Competing Interests section:

“NO authors have competing interests”

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state "The authors have declared that no competing interests exist.", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

This information should be included in your cover letter; we will change the online submission form on your behalf

4. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The general topic of this paper is still timely as the pandemic is seemingly never ending, so research about the impact of wearing masks is relevant, and emotional reading in particular is one cognitive function which is obviously strongly impoverished when a main part of the face is covered.

In particularly liked that the authors employed Korean images and added a further, ecological relevant variable by using sunglasses as a kind of control condition to face masks to test effects of mouth vs. eyes-covering. Additionally I appreciate that they offered high quality analyses which are nicely depicted. The paper is clear, only the introduction seems to miss some points.

I will comment on this in detail in my report to the authors below.

Major points

1. The authors claim that the Ruba & Pollack study was “particularly informative”; I would like to disagree, and the authors sum this up competently later on: That study was a (very) limited one; additional to the critical points the authors already raised, I would like to add that the material and experimental design was highly problematic as we cannot assign the variance to the specific face or the specific intervention (type of covering) as different faces were used across conditions—in the end, the authors devote a very long section on that paper but the validity of it is currently debated. Please consider new studies which addressed these critical points, e.g. Carbon & Serrano (2021).

2. G_Power-analyses: this makes sense to conduct such an analysis before conducting the study, but why then were 39 instead of 28 persons tested. Was the test power analysis in the end just done for the sake of having done it? Please verify this.

3. Title: this is typically NOT a major issue, but here the title is so underspecified that I would not recommend it (and I do this to protect the authors, therefore raising it to “major” as I believe that more informative papers will help to be read by others—and I truly believe that this paper deserves many readings by interesting readers!).

Minor points

1. Please reduce the “precision” with values which are imprecisely measured, e.g. age.

2. Please add units to values (e.g., again, age)

3. Greek letters should not be italicized

4. It is great that you have offered the data on OSF! Please consider (later on) to provide the analyses, too

5. Figure 1 is very illustrative, thank you! I would extend the arrow and put a “…” at the end, juts to make clear it goes on and on and that a trial is NOT a couple of two pictures but that this diagram just shows a sequence of two typical trials (the quality of the scale could be increased in the final version—it is hard to read currently).

6. Please insert spaces between values and units

7. Please add effect sizes, also to the Chi2-tests

8. Fig.2 is very nice, but please add the method how you analysed the pairwise comparisons; was it alpha-corrected? How were the error bars calculated? What do they represent? Just make everything clear, please!

9. “subject”: I would (just a recommendation, no rule or obligation—personal taste) not speak of “subjects” but persons / participants / etc.-

10. Discussion: About video material: please check the literature as there are already papers on video-related mask research

11. Baron-Cohen’s study [see ref.1] was additionally tested by Schmidtmann and colleagues with non-clinical participants meanwhile. Please check.

12. Discussion: children: see the latest paper on children-related samples

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2022 Feb 3;17(2):e0263466. doi: 10.1371/journal.pone.0263466.r002

Author response to Decision Letter 0

9 Nov 2021

Dear Managing Editor,

PLOS ONE Reviews

Thank you for inviting us to submit a revised draft of our manuscript entitled, “Does masking one's facial area masks emotions? The impact of face masks and sunglasses on emotion recognition in South Koreans (revision)” to PLOS ONE. We express our cordial gratitude for the valuable and constructive review of our paper. The reviewers’ comments have helped us to further strengthen the overall quality of the paper. We have incorporated changes that reflect the detailed suggestions you have graciously provided. We also hope that our edits and the responses we provide below satisfactorily address all the issues and concerns you and the reviewer have noted. The specific responses to the reviewer’s comments are listed below.

The Editor’s comments:

Comment 1: You have update your Introduction and Discussion taking into account latest studies in the field.

Response:

Thank you for the suggestion and recommending the recent papers on this topic. We have updated the introduction in the following manner:

We cited the paper by Pavlova and Sokolov (2021) and introduced the most recent relevant studies that were reviewed in the paper. Specifically, we incorporated the findings from Carbon & Serrano (2021) and Noyes et al., (2021) which were the most up-to-date studies that tested the effects of masked faces on emotion recognition. In addition, following reviewer 1’s suggestion, we have significantly reduced the part on the findings of Ruba & Pollack’s (2020) study. Furthermore, we updated the findings that gender differences exist in facial expression recognition of covered faces with the latest study synthesizing face masks.

In the discussion section, we provided a more comprehensive discussion by comparing our findings with Noyes et al. (2021) as well as Carbon (2020). In addition, we updated the latest studies in regard to reading emotions among children.

Comment 2: the title should be modified to better reflect the specificity and novelty of your study.

Response: As per your suggestion, we have changed the title to “Does masking one's facial area masks emotions? The impact of face masks and sunglasses on emotion recognition in South Koreans”. The manuscript title has been updated and now it is more meaningful.

Comment 3: did you check the data sets for normality of distribution and how? Please make a clear statement on it in the Method Section.

Response: Thank you for your suggestion. We tested normality of distribution and found that some variables were not normally distributed. However, we decided to continue using ANOVA as of the robustness of the statistic against violations of normality with equal sample sizes (Glass et al., 1972). The ANOVA is reasonably robust to violations of homogeneity of variance when group sizes are equal, and minor violations of the normal distribution can also still produce results that are similar to those when the data are normally distributed (Schmider et al. 2010). We have confirmed that many studies use the repeated measures ANOVA based on F test's robust to non-normality, even if the data are nonnormal (Ackermann et al., 2019; Adriaens et al., 2018; Everman et al., 2018; Ghazali et al., 2018; Goheen et al., 2013; Savage et al., 2020; Zhang et al., 2018). We also described that the violation of normality assumption and the reason we still use within-subject ANOVA in the Results section (p. 9, lines 184-187). Nevertheless, if you still think to need a nonnormality test, we will proceed with a new analysis.

“Before analyzing the main results, all data were checked for normality distribution. Normality was violated for some variables but we decided to use the repeated measures ANOVA because ANOVAs are generally considered robust to nonnormality with sample sizes being equal (within-subject design) [34].”

Additionally, we added a sentence related to how to present results according to the tests of sphericity (p. 9, lines 189-190).

“We reported the results of Greenhouse–Geisser correction whenever Mauchly’s test of sphericity was significant.”

References

Glass G V, Peckham PD, & Sanders JR. Consequences of failure to meet assumptions underlying the fixed effects analyses of variance and covariance. Review of educational research. 1972;42(3): 237-288.

Schmider E, Ziegler M, Danay E, Beyer L, Bühner M. Is it really robust?. Methodology. 2010 Sep 8.

Ackermann C, Beggiato M, Bluhm LF, Löw A, Krems JF. Deceleration parameters and their applicability as informal communication signal between pedestrians and automated vehicles. Transportation research part F: traffic psychology and behaviour. 2019;62: 757-768.

Adriaens K, Van Gucht D, Baeyens F. IQOSTM vs. e-cigarette vs. tobacco cigarette: a direct comparison of short-term effects after overnight-abstinence. International journal of environmental research and public health. 2018;15(12): 2902.

Everman ER, Delzeit JL, Hunter FK, Gleason JM,Morgan TJ. Costs of cold acclimation on survival and reproductive behavior in Drosophila melanogaster. PLOS ONE. 2018;13(5): e0197822.

Ghazali AS, Ham J, Barakova E, Markopoulos P. The influence of social cues in persuasive social robots on psychological reactance and compliance. Computers in Human Behavior. 2018;87: 58-65.

Goheen JR, Palmer TM, Charles GK, Helgen KM, Kinyua SN, Maclean JE, ... Pringle RM. Piecewise disassembly of a large-herbivore community across a rainfall gradient: the UHURU experiment. PLOS ONE. 2013; 8(2): e55192.

Savage MJ, James R, Magistro D, Donaldson J, Healy LC, Nevill M, Hennis PJ. Mental health and movement behaviour during the COVID-19 pandemic in UK university students: Prospective cohort study. Mental Health and Physical Activity. 2020;19: 100357.

Zhang Z, Zhang B, Cao C, Chen W. The effects of using an active workstation on executive function in Chinese college students. PLOS ONE. 2018;13(6): e0197740.

Comment 4: Did you analyze the effect of observers' gender on performance?

Response: Yes. We described this at the beginning of the Results section. However to express it more effectively, we have revised the text to “However, the main effect of the sex of the participants and the interaction between the sex of the participants and the sex of the face stimuli were not statistically significant, F(1,37) = 1.74, p = .20; F(1, 37) = 1.42, p = .24; respectively.” (p. 17, lines 338-341).

The Reviewer's comments:

Comment 1: The authors claim that the Ruba & Pollack study was “particularly informative”; I would like to disagree, and the authors sum this up competently later on: That study was a (very) limited one; additional to the critical points the authors already raised, I would like to add that the material and experimental design was highly problematic as we cannot assign the variance to the specific face or the specific intervention (type of covering) as different faces were used across conditions—in the end, the authors devote a very long section on that paper but the validity of it is currently debated. Please consider new studies which addressed these critical points, e.g. Carbon & Serrano (2021).

Response: Thank you for pointing this out. The editor also expressed the same concern. Based on your and the editor’s recommendation, we have updated the introduction in the following manner:

Comment 2: G_Power -analyses: this makes sense to conduct such an analysis before conducting the study, but why then were 39 instead of 28 persons tested. Was the test power analysis in the end just done for the sake of having done it? Please verify this.

Response: First, we calculated the number of participants required in order to meet the assumptions of independence and normality in our experiment using G-power. After that, we recruited more participants than we needed in anticipation of dropouts during the experiment or malfunctioning experimental computers or programs. In addition, more participants were recruited for more stable results because there may be participants who did not meet the pre-set exclusion criteria (less than a 50% correct facial expression recognition answer rate when presented with a fully visible face without a face mask or sunglasses were excluded). A total of 40 participants were recruited, but all 39 were used in the analysis because there was no reason to exclude them from the data except one. (p. 6, lines 130-134).

“However, we sampled more participants than the required number for potential participants who would get excluded if computers malfunction or those who did not meet the pre-set exclusion criteria (less than a 50% correct facial expression recognition answer rate when presented with a fully visible face without a face mask or sunglasses were excluded). This pre-determined criterion was based on that of Carbon’s method [9].”

Comment 3: Title : this is typically NOT a major issue, but here the title is so underspecified that I would not recommend.

Response: As per your suggestion, we have changed the title to “Does masking one's facial area masks emotions? The impact of face masks and sunglasses on emotion recognition in South Koreans”.

Comment 4: Please reduce the “precision” with values which are imprecisely measured, e.g. age.

Response: We did not understand this comment. Could you please explain what you meant by “reduce the precision”? Thank you.

Comment 5: Please add units to values (e.g., again, age).

Response: We have reflected this comment by adding units to age (p. 2, lines 23; p. 7, line 143).

“mean age = 24.23 years”

“Mage = 24.23 years, SDage = 4.68 years”

Comment 6: Greek letters should not be italicized.

Response: Thank you for your comments. We have revised the reporting of all partial eta-squared (ηp2) statistics throughout the paper. You can check them without italics.

Comment 7: It is great that you have offered the data on OSF! Please consider (later on) to provide the analyses, too

Response: Thank you for your suggestion. We will update with the syntax for the data analyses before the paper is published.

Comment 8: Figure 1 is very illustrative, thank you! I would extend the arrow and put a “…” at the end, juts to make clear it goes on and on and that a trial is NOT a couple of two pictures but that this diagram just shows a sequence of two typical trials (the quality of the scale could be increased in the final version—it is hard to read currently).

Response: We have now figure 1 with arrows and “…”. We think these changes now better to describe our trials. We hope that you agree.

Comment 9: Please insert spaces between values and units

Response: We checked it throughout the paper.

Comment 10: Please add effect sizes, also to the Chi2-tests.

Response: We calculated effect size of the χ2 using Cramer's V which is for categorical comparisons. Cramer's V coefficient and p-value were reported (p. 9, lines 180).

“(χ^2 = 21312.458, p < .001, Cramer’s V = 0.601, p < .001)”

Comment 11: Fig.2 is very nice, but please add the method how you analysed the pairwise comparisons; was it alpha-corrected? How were the error bars calculated? What do they represent? Just make everything clear, please!

Response: We computed a repeated-measures ANOVA and pairwise comparisons using SPSS 25 software, and constructed Fig 2 using the alpha and standard errors provided by the package. We agree with you and have clarified how we analyzed (under the Fig.2). We think this change now is better for readers to understand our analysis. We hope that you agree.

Comment 12: “subject”: I would (just a recommendation, no rule or obligation—personal taste) not speak of “subjects” but persons / participants / etc.

Response: We have reflected this comment by replacing the term “subject” throughout the paper with “participants”.

Comment 13: Discussion: About video material : please check the literature as there are already papers on video-related mask research.

Response: Thank you for the reminder. As you have mentioned about existing literature on video materials, we managed to find that a paper with regards to video materials with faces wearing masks and thus, included it in our Discussion section. However, since the paper has limitations in that, it contains only two emotions, happy and sad, we decided to maintain the argument that video research is necessary for the effect of masks on facial expressions recognition (p. 20, lines 418-422; 424-426).

“Recently, researchers measured the accuracy of emotional recognition using video stimulation, which adds a static background to the dynamic facial expression set [40]. However, since this study included only two emotions, happiness and sadness, and sad, it is difficult to grasp the effect of on recognition of various emotions in facial expression.”

“Thus, future studies using video stimuli that include various emotions and involve other sources of information would render a more ecologically valid effect of facial occlusion on emotion recognition.”

Comment 14: Baron-Cohen’s study [see ref.1] was additionally tested by Schmidtmann and colleagues with non-clinical participants meanwhile. Please check.

Response: We checked the study of Schmidtmann et al. (2020) and changed the references. Thank you for your suggestion.

Comment 15: Discussion: children : see the latest paper on children-related samples

Response: We checked the paper (Carbon & Serrano, 2021) that examined the effect of masks on facial expression recognition in children and added it in the corresponding part. (p. 21, lines 436-437)

“In recent studies, researchers have found that children’s emotion recognition is also affected by facial masks [15, 16, 47].”

Again, thank you for giving us the opportunity to strengthen our manuscript with your valuable comments and queries. We have worked hard to incorporate your feedback and hope that these revisions persuade you to accept our submission.

Attachment

Submitted filename: Rebuttal Letter.docx

Click here for additional data file.^{(30.5KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0263466.r003

Decision Letter 1

Marina A Pavlova

21 Dec 2021

PONE-D-21-25701R1Does masking one's facial area masks emotions? The impact of face masks and sunglasses on emotion recognition in South KoreansPLOS ONE

Dear Dr. Choi:

Please submit your revised manuscript by Feb 04 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Marina A. Pavlova, PhD

Academic Editor

PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Partly

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

**********

6. Review Comments to the Author

Reviewer #1: Dear Authors, thanks for providing such a precise and clear rebuttal letter which greatly supports my job as a reviewer. I found your replies convincing and well founded.

I would only recommend to address some (very!) minor points.

(very) minor points

1. The effect of gender was not significant as you explained in your letter; please state 1-2 sentences whether this was expected or not and on which basis you argue pro or contra such effects.

2. About Comment 4 (from my review before): please use only the precision the base of measurement provides, e.g. age is typically asked regarding years, but not more precise. So any outcome of mean age should not be more precise than about 1/10 of the original precision. So for instance 24.2 instead of 24.23 years.

3. [Just a “Thank you!”: Thanks for providing your data and analyses on OSF—the scientific community will appreciate this very much!]

4. Please leave out [Internet] for references

5. IMPORTANT POINT: Fig.3: PLEASE DOUBLE CHECK THIS (just because it is highly unrealistic): was fear really mostly misinterpreted as SURPRISE? I have never seen such a massive systematic confusion of emotional states across different presentation conditions.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

PLoS One. 2022 Feb 3;17(2):e0263466. doi: 10.1371/journal.pone.0263466.r004

Author response to Decision Letter 1

5 Jan 2022

1. The effect of gender was not significant as you explained in your letter; please state 1-2 sentences whether this was expected or not and on which basis you argue pro or contra such effects.

Response: Although gender was not a major concern of our study, we expected two main effects of gender based on previous studies. First, we expected that women would show higher recognition rates than men. Second, we expected the participants to perform better when the target face is of opposite sex. However, the results showed that the main effect of participant sex and the interaction between participant gender and the gender of target were not significant. Unexpectedly, there was a main effect of stimuli gender was significant. Specifically, we found that participants perceived the emotions of female (vs. male) faces more accurately. As we know of no theoretical reason for this result, more research is needed to replicate such findings first.

The following paragraph is added in the revised manuscript.

“Finally, participant gender did not have any effect on recognition rate nor did they interact with the target’s gender. However, target stimuli’s gender mattered. Overall, participants recognized female faces better than male faces. More research is needed to replicate this effect.” (page 20, lines 410-413)

Response: Thank you for your kind explanation. As per your suggestion, we have revised the precision of age by rounding up to the first digit after the decimal point instead of the second digit.

3. [Just a “Thank you!”: Thanks for providing your data and analyses on OSF—the scientific community will appreciate this very much!]

Response: Thank you for the reminder. We updated the syntax for the data analyses as well as our figures.

4. Please leave out [Internet] for references

Response: We removed [Internet] from the references.

Response: In this experiment, participants tended to recognize fear as surprise rather than as fear itself. (We were also surprised by these patterns, and we checked the results again several times.) Not only did 41% of participants incorrectly recognize fear as surprise in a mask condition, 65% of participants misread fear as surprise when in uncovered face.

It is true that surprise and fear are often confused (Ekman, 2003) but in our study, the misinterpretation rate was very high. This may suggest that there the facial expression of "fear” in the present facial stimuli set is problematic. In line with this argument, in a study conducted by Kim et al. (2011) where the same KUFEC database was used, participants were not able to make a significant distinction between surprise and fear.

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(22.6KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0263466.r005

Decision Letter 2

Marina A Pavlova

10 Jan 2022

PONE-D-21-25701R2Does masking one's facial area masks emotions? The impact of face masks and sunglasses on emotion recognition in South KoreansPLOS ONE

Dear Dr.Choi,

Please submit your revised manuscript by Feb 24 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

We look forward to receiving your revised manuscript.

Kind regards,

Marina A. Pavlova, PhD

Academic Editor

PLOS ONE

Journal Requirements:

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

Reviewer #1: Yes

**********

6. Review Comments to the Author

Reviewer #1: Thanks for your quick makeover; all points are now addressed (thanks also for clarifying the issue with SURPRISE vs. FEAR) and I wish the authors alle the best for their paper!

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

PLoS One. 2022 Feb 3;17(2):e0263466. doi: 10.1371/journal.pone.0263466.r006

Author response to Decision Letter 2

19 Jan 2022

The Editor's comments:

Comment 1: I suggest to shorten the title of your paper to 'Impact of face masks and sunglasses on emotion recognition in South Koreans.

Response: As per your suggestion, we have shortened the title to “Impact of face masks and sunglasses on emotion recognition in South Koreans.” We hope that the change will make it easier for readers to access our paper.

Comment 2: You also completely ignored my request for more attentive reading of the recent review READING COVERED FACES, and for adding more information on cultural differences on recognition of covered by masks emotions. I hope you can do this promptly.

Response: We apologize for carelessly omitting the response to your comment regarding cross-cultural implications of the present research. The current manuscript has added more information on cultural differences on recognition of covered faces by citing studies from your review paper. We greatly appreciate your helpful suggestions.

See p. 5, lines 101-109

" Also, the effects of masks or sunglasses on reading other person's facial expressions may differ depending on the cultural context (for a review see [18]). As an example, consider the findings that a face covered with Islamic headdress such as niqāb impacts the recognition of emotions differently by cultural groups [13, 25], suggesting that a culturally attached meaning of headdress may play a role. As for East Asians, they are not only less accustomed to sunglasses than Westerners, but sunglasses are often considered rude in interpersonal relationships [26, 27]. Given such cultural background, thus, it is necessary to test whether the effects of masks and sunglasses on facial expression recognition that are documented with Western participants would also apply to East Asians.”

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(21KB, docx)}

PLoS One. doi: 10.1371/journal.pone.0263466.r007

Decision Letter 3

Marina A Pavlova

20 Jan 2022

Impact of face masks and sunglasses on emotion recognition in South Koreans

PONE-D-21-25701R3

Dear Dr. Choi:

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Marina A. Pavlova, PhD

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

PLoS One. doi: 10.1371/journal.pone.0263466.r008

Acceptance letter

Marina A Pavlova

24 Jan 2022

PONE-D-21-25701R3

Impact of face masks and sunglasses on emotion recognition in South Koreans

Dear Dr. Choi:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Prof. Marina A. Pavlova

Academic Editor

PLOS ONE

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Attachment

Submitted filename: Rebuttal Letter.docx

Click here for additional data file.^{(30.5KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(22.6KB, docx)}

Attachment

Submitted filename: Response to Reviewers.docx

Click here for additional data file.^{(21KB, docx)}

Data Availability Statement

The raw data is available from the Open Science Framework (https://osf.io/fcg4d/).

[pone.0263466.ref001] 1.Eisenbarth H, Alpers GW. Happy mouth and sad eyes: scanning emotional facial expressions. Emotion. 2011;11(4):860–5. doi: 10.1037/a0022758 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref002] 2.Gosselin F, Schyns PG. Bubbles: a technique to reveal the use of information in recognition tasks. Vision Research. 2001;41(17):2261–71. doi: 10.1016/s0042-6989(01)00097-9 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref003] 3.Kontsevich LL, Tyler CW. What makes Mona Lisa smile? Vision Research. 2004;44(13):1493–8. doi: 10.1016/j.visres.2003.11.027 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref004] 4.Schmidtmann G, Logan AJ, Carbon C-C, Loong JT, Gold I. In the Blink of an Eye: Reading Mental States From Briefly Presented Eye Regions. i-Perception. 2020;11(5). doi: 10.1177/2041669520961116 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref005] 5.Bombari D, Schmid PC, Schmid Mast M, Birri S, Mast FW, Lobmaier JS. Emotion Recognition: The Role of Featural and Configural Face Information. Quarterly Journal of Experimental Psychology. 2013;66(12):2426–42. doi: 10.1080/17470218.2013.789065 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref006] 6.Cunningham DW, Kleiner M, Bülthoff HH, Wallraven C. The components of conversational facial expressions. Proceedings of the 1st Symposium on Applied Perception in Graphics and Visualization; Los Angeles, California, USA: Association for Computing Machinery; 2004; 143–50. doi: 10.1145/1012551.1012578 [DOI] [Google Scholar]

[pone.0263466.ref007] 7.Fraser IH, Craig GL, Parker DM. Reaction Time Measures of Feature Saliency in Schematic Faces. Perception. 1990;19(5):661–73. doi: 10.1068/p190661 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref008] 8.Kotsia I, Buciu I, Pitas I. An analysis of facial expression recognition under partial facial image occlusion. Image and Vision Computing. 2008;26(7):1052–67. doi: 10.1016/j.imavis.2007.11.004 [DOI] [Google Scholar]

[pone.0263466.ref009] 9.Carbon C-C. Wearing Face Masks Strongly Confuses Counterparts in Reading Emotions. Frontiers in Psychology. 2020;11. doi: 10.3389/fpsyg.2020.566886 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref010] 10.Jack RE, Garrod OGB, Yu H, Caldara R, Schyns PG. Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences. 2012;109(19):7241–4. doi: 10.1037/a0023463 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref011] 11.Nestor MS, Fischer D, Arnold D. “Masking” our emotions: Botulinum toxin, facial expression, and well-being in the age of COVID-19. Journal of Cosmetic Dermatology. 2020;19(9):2154–60. doi: 10.1111/jocd.13569 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref012] 12.Wegrzyn M, Vogt M, Kireclioglu B, Schneider J, Kissler J. Mapping the emotional face. How individual face parts contribute to successful emotion recognition. PLOS ONE. 2017;12(5). doi: 10.1371/journal.pone.0177239 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref013] 13.Fischer AH, Gillebaart M, Rotteveel M, Becker D, Vliek M. Veiled Emotions: The Effect of Covered Faces on Emotion Perception and Attitudes. Social Psychological and Personality Science. 2012;3(3):266–73. doi: 10.1177/1948550611418534 [DOI] [Google Scholar]

[pone.0263466.ref014] 14.Song I-H, Cho K-J, Chim H-R. The effect of emotional dimension and facial expression’s presenting areas on facial expression’s recognition: A comparison of gender differences. Korean Journal of Woman Psychology. 2008;13(2):197–214. [Google Scholar]

[pone.0263466.ref015] 15.Carbon C-C, Serrano M. The Impact of Face Masks on the Emotional Reading Abilities of Children—A Lesson From a Joint School–University Project. i-Perception. 2021;12(4). doi: 10.1177/20416695211038265 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref016] 16.Ruba AL, Pollak SD. Children’s emotion inferences from masked faces: Implications for social interactions during COVID-19. PLOS ONE. 2020;15(12). doi: 10.1371/journal.pone.0243708 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref017] 17.Noyes E, Davis JP, Petrov N, Gray KLH, Ritchie KL. The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers. Royal Society Open Science. 2021;8(3). doi: 10.1098/rsos.201169 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref018] 18.Pavlova MA, Sokolov AA. Reading Covered Faces. Cerebral Cortex. 2021;32(2):249–65. doi: 10.1093/cercor/bhab311 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref019] 19.Ekman P, Friesen WV, O’Sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, et al. Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology. 1987;53(4):712–7. doi: 10.1037//0022-3514.53.4.712 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref020] 20.Matsumoto D, Ekman P. American-Japanese cultural differences in intensity ratings of facial expressions of emotion. Motivation and Emotion. 1989;13(2):143–57. doi: 10.1007/BF00992959 [DOI] [Google Scholar]

[pone.0263466.ref021] 21.Yuki M, Maddux WW, Masuda T. Are the windows to the soul the same in the East and West? Cultural differences in using the eyes and mouth as cues to recognize emotions in Japan and the United States. Journal of Experimental Social Psychology. 2007;43(2):303–11. doi: 10.1016/j.jesp.2006.02.004 [DOI] [Google Scholar]

[pone.0263466.ref022] 22.Jack RE, Blais C, Scheepers C, Schyns PG, Caldara R. Cultural Confusions Show that Facial Expressions Are Not Universal. Current Biology. 2009;19(18):1543–8. doi: 10.1016/j.cub.2009.07.051 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref023] 23.Haensel JX, Danvers M, Ishikawa M, Itakura S, Tucciarelli R, Smith TJ, et al. Culture modulates face scanning during dyadic social interactions. Scientific Reports. 2020;10(1):1–11. doi: 10.1038/s41598-019-56847-4 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref024] 24.Baehr P. City under siege: authoritarian toleration, mask culture, and the SARS crisis in Hong Kong. Networked Disease: Emerging Infections in the Global City. 2008;138–51. doi: 10.1002/9781444305012.ch8 [DOI] [Google Scholar]

[pone.0263466.ref025] 25.Kret M, De Gelder B. Islamic Headdress Influences How Emotion is Recognized from the Eyes. Frontiers in Psychology. 2012;3. doi: 10.3389/fpsyg.2012.00110 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref026] 26.Gesteland RR. Cross-cultural business behavior: A guide for global management. Copenhagen Business School Press DK; 2012. [Google Scholar]

[pone.0263466.ref027] 27.Miyazawa D, Kaneko G. Japan’s key “X-factor” for low COVID-19 mortality may be its culture of wearing face masks. Authorea Preprints. 2020. doi: 0.22541/au.159373225.52467275 [Google Scholar]

[pone.0263466.ref028] 28.Roberson D, Kikutani M, Döge P, Whitaker L, Majid A. Shades of emotion: What the addition of sunglasses or masks to faces reveals about the development of facial expression processing. Cognition. 2012;125(2):195–206. doi: 10.1016/j.cognition.2012.06.018 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref029] 29.Freud E, Stajduhar A, Rosenbaum RS, Avidan G, Ganel T. The COVID-19 pandemic masks the way people perceive faces. Scientific Reports. 2020;10(1). doi: 10.1038/s41598-020-78986-9 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref030] 30.Grundmann F, Epstude K, Scheibe S. Face masks reduce emotion-recognition accuracy and perceived closeness. PLOS ONE. 2021;16(4). doi: 10.1371/journal.pone.0249792 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref031] 31.Hofmann SG, Suvak M, Litz BT. Sex differences in face recognition and influence of facial affect. Personality and Individual Differences. 2006;40(8):1683–90. doi: 10.1016/j.paid.2005.12.014 [DOI] [Google Scholar]

[pone.0263466.ref032] 32.Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Abingdon. England: Routledge. 1988. [Google Scholar]

[pone.0263466.ref033] 33.Lee TH, Lee KY, Lee K, Choi J-S, Kim HT. The Korea University facial expression collection: KUFEC. Lab of Behavioral Neuroscience, Dept of Psychology, Korea University, Seoul, South Korea. 2006. [Google Scholar]

[pone.0263466.ref034] 34.Kim MW, Choi J-S, Cho YS. The Korea University Facial Expression Collection (KUFEC) and semantic differential ratings of emotion. Korean Journal of Psychology: General. 2011;30(4):1189–2111. [Google Scholar]

[pone.0263466.ref035] 35.Blanca Mena MJ, Alarcón Postigo R, Arnau Gras J, Bono Cabré R, Bendayan R. Non-normal data: Is ANOVA still a valid option? Psicothema. 2017;29(4):552–7. doi: 10.7334/psicothema2016.383 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref036] 36.Ekman P. Emotions revealed: recognizing faces and feelings to improve communication and emotional life. New York. NY: Times books. 2003. [Google Scholar]

[pone.0263466.ref037] 37.Roy-Charland A, Perron M, Beaudry O, Eady K. Confusion of fear and surprise: A test of the perceptual-attentional limitation hypothesis with eye movement monitoring. Cognition and Emotion. 2014;28(7):1214–22. doi: 10.1080/02699931.2013.878687 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref038] 38.Fölster M, Hess U, Werheid K. Facial age affects emotional expression decoding. Frontiers in Psychology. 2014;5. doi: 10.3389/fpsyg.2014.00030 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref039] 39.Horning SM, Cornwell RE, Davis HP. The recognition of facial expressions: An investigation of the influence of age and cognition. Aging, Neuropsychology, and Cognition. 2012;19(6):657–76. doi: 10.1080/13825585.2011.645011 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref040] 40.Kastendieck T, Zillmer S, Hess U. (Un)mask yourself! Effects of face masks on facial mimicry and emotion perception during the COVID-19 pandemic. Cognition and Emotion. 2021:1–11. doi: 10.1080/02699931.2021.1950639 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref041] 41.Chóliz M, Fernández-Abascal EG. Recognition of Emotional Facial Expressions: The Role of Facial and Contextual Information in the Accuracy of Recognition. Psychological Reports. 2012;110(1):338–50. doi: 10.2466/07.09.17.PR0.110.1.338-350 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref042] 42.Van den Stock J, Righart R, de Gelder B. Body expressions influence recognition of emotions in the face and voice. Emotion. 2007;7(3):487–94. doi: 10.1037/1528-3542.7.3.487 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref043] 43.Clarke TJ, Bradshaw MF, Field DT, Hampson SE, Rose D. The Perception of Emotion from Body Movement in Point-Light Displays of Interpersonal Dialogue. Perception. 2005;34(10):1171–80. doi: 10.1068/p5203 [DOI] [PubMed] [Google Scholar]

[pone.0263466.ref044] 44.de Gelder B, Vroomen J. The perception of emotions by ear and by eye. Cognition and Emotion. 2000;14(3):289–311. doi: 10.1080/026999300378824 [DOI] [Google Scholar]

[pone.0263466.ref045] 45.Beams AB, Bateman R, Adler FR. Will SARS-CoV-2 Become Just Another Seasonal Coronavirus? Viruses. 2021;13(5):854. doi: 10.3390/v13050854 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref046] 46.Koc E, Boz H. Development of hospitality and tourism employees’ emotional intelligence through developing their emotion recognition abilities. Journal of Hospitality Marketing & Management. 2020;29(2):121–38. doi: 10.1080/19368623.2019.1608885 [DOI] [Google Scholar]

[pone.0263466.ref047] 47.Gori M, Schiatti L, Amadeo MB. Masking Emotions: Face Masks Impair How We Read Emotions. Frontiers in Psychology. 2021;12. doi: 10.3389/fpsyg.2021.669432 [DOI] [PMC free article] [PubMed] [Google Scholar]

[pone.0263466.ref048] 48.Denham SA, Bassett HH, Zinsser K, Wyatt TM. How Preschoolers’ Social–Emotional Learning Predicts Their Early School Success: Developing Theory-Promoting, Competency-Based Assessments. Infant and Child Development. 2014;23(4):426–54. doi: 10.1002/icd.1840 [DOI] [Google Scholar]

[pone.0263466.ref049] 49.Izard C, Fine S, Schultz D, Mostow A, Ackerman B, Youngstrom E. Emotion Knowledge as a Predictor of Social Behavior and Academic Competence in Children at Risk. Psychological Science. 2001;12(1):18–23. doi: 10.1111/1467-9280.00304 [DOI] [PubMed] [Google Scholar]

PERMALINK

Impact of face masks and sunglasses on emotion recognition in South Koreans

Garam Kim

So Hyun Seong

Seok-Sung Hong

Eunsoo Choi

Roles

Abstract

Introduction

Present study

Methods

Participants

Materials

Procedure

Fig 1. An example of the facial expression recognition task.

Results

Do different face regions (eye vs. mouth) make a difference in recognizing specific emotions?

Fig 2. Facial recognition accuracy across conditions and emotions.

Happiness

Disgust

Sadness

Anger

Surprise

Fear

Neutral

How were the emotions misinterpreted?

Fig 3. Percentage of participants responding to each emotion.

Does sex matter?

Discussion

Limitations and future research

Acknowledgments

Data Availability

Funding Statement

References

Decision Letter 0

Marina A Pavlova

Roles

Author response to Decision Letter 0

Decision Letter 1

Marina A Pavlova

Roles

Author response to Decision Letter 1

Decision Letter 2

Marina A Pavlova

Roles

Author response to Decision Letter 2

Decision Letter 3

Marina A Pavlova

Roles

Acceptance letter

Marina A Pavlova

Roles

Associated Data

Supplementary Materials

Data Availability Statement

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases