Abstract
Face masks are an effective and important tool to prevent the spread of COVID‐19, including among children. However, occluding parts of the face can impact emotion recognition, which is fundamental to effective social interactions. Social distancing, stress, and changes to routines because of the pandemic have also altered the social landscape of children, with implications for social development. To better understand how social input and context impact emotion recognition, the current study investigated emotion recognition in children (7–12 years old, N = 131) using images of both masked and unmasked emotional faces. We also assessed a subsample of participants (“pre‐pandemic subsample,” n = 35) who had completed the same emotion recognition task with unmasked faces before and during the pandemic. Masking of faces was related to worse emotion recognition, with more pronounced effects for happy, sad, and fearful faces than angry and neutral faces. Masking was more strongly related to emotion recognition among children whose families reported greater social disruption in response to the pandemic. Finally, in the pre‐pandemic subsample, emotion recognition of sad faces was lower during versus before the pandemic relative to other emotions. Together, findings show that occluding face parts and the broader social context (i.e., global pandemic) both impact emotion‐relevant judgments in school‐aged children.
Keywords: COVID‐19 pandemic, development, emotion recognition, family context, parenting
1. INTRODUCTION
Emotion recognition is fundamental to human social interactions (Niedenthal & Brauer, 2012). Emotion recognition improves across childhood from infancy to 11 years of age as social interactions are thought to increase in complexity (Chronaki et al., 2015). Accordingly, children with better emotion recognition skills evidence higher peer status, friendship quality, and teacher‐rated social competence (Denham et al., 2003). In contrast, impaired emotion recognition is linked to risk for psychopathology, including conduct disorder (Kohls et al., 2020) and depression (Demenescu et al., 2010). Late childhood is a key developmental period for understanding risk for psychopathology (Costello et al., 2011), for which emotion recognition plays a prominent role (Southam‐Gerow & Kendall, 2002). Thus, a better understanding of the factors that influence children's emotion recognition, including contextual factors or social cues, can provide insight into both adaptive and maladaptive socioemotional development.
Research from children and adults shows that emotions are generally inferred holistically from facial configurations (Pellicano & Rhodes, 2003), such that when some regions of the face are occluded, other features become compensatory diagnostic cues (Gagnon et al., 2014). Nevertheless, research using eye‐tracking shows emotion‐specific variation. For example, we fixate more on the upper face region for sad and fearful cues, but the lower region for happy signals (Bombari et al., 2013). Likewise, research on facial action units provides evidence for the different contributions of facial regions across emotion displays. For example, occluding the eyes undermines emotion recognition for fearful and sad expressions (Gagnon et al., 2014; Wegrzyn et al., 2017), whereas occluding the mouth reduces recognition of happy faces (Fischer et al., 2012; Wegrzyn et al., 2017).
Despite this literature, few studies have investigated how facial occlusion is related to emotion recognition in children, which is surprising given that facial emotion recognition improves rapidly during childhood (Batty & Taylor, 2006; Chronaki et al., 2015). In particular, the neural systems that support perception of emotional faces show significant change across childhood (Batty & Taylor, 2006) and older age is related to enhanced performance and speed on emotion processing tasks (De Sonneville et al., 2002). Importantly, children increasingly spend time with peers in the transition from middle‐to‐late childhood into adolescence (Lam et al., 2014). Accurate perception of peers’ emotion signals is vital for social development and academic competence (Izard et al., 2001), with known variability in how facial expressions are produced by children of different ages (Grossard et al., 2018). However, no prior studies have explored how facial occlusion is related to emotion recognition by children when emotions are expressed by child faces.
Added to this literature, the COVID‐19 pandemic has created new incentives for understanding emotion recognition, particularly in children. To reduce airborne transmission of the virus between 2020–2022, mask‐wearing in the United States became largely normalized, and in most states, government mandated. For example, state‐issued mask mandates were in effect in 73.6% of counties between March 1 and December 31, 2020 (Guy et al., 2021) and applied to children as young as 2‐ or 3‐years‐old across daycare and school settings (Mickells et al., 2021). Thus, a majority of children had to learn to infer facial cues of both children and adults with noses and mouths occluded. Prior to the pandemic, mask‐wearing outside health settings was rare, meaning that few children would have encountered faces occluded in this way. In addition, the COVID‐19 pandemic necessitated major changes to daily routines that altered typical social inputs relevant to emotion recognition, including school closures, suspension of physical activities, and “stay‐at‐home orders” (Griffith, 2020). No studies have explored how changes to social interactions or the degree of social distancing adopted by families during the pandemic is related to emotion recognition in children, which could give insight into the impacts of the pandemic on socioemotional development, as well as the role of social context in shaping emotion recognition more broadly.
Consistent with research on partially occluded faces, mask‐wearing during the pandemic has been linked to worse emotion recognition among adults, including lower face matching performance (Carragher & Hancock, 2020), impaired emotion processing abilities (Freud et al., 2020), and reduced emotion recognition accuracy (Carbon, 2020; Marini et al., 2021). Research on the impact of masking on emotion recognition in children is still nascent, despite childhood being a critical period for the development of emotion recognition (Chronaki et al., 2015; Widen, 2013). In four exceptions, emotion recognition accuracy for masked versus unmasked adult faces was significantly lower among 3–5 year olds (N = 276) (Schneider et al., 2022), 3–5‐year‐olds (N = 31) and 6–8‐year‐olds (N = 49) (Gori et al., 2021), 7–13‐year‐olds (N = 81) (Ruba & Pollak, 2020), and 9–11‐year‐olds (N = 57) (Carbon & Serrano, 2021). Across all studies, however, emotion recognition accuracy remained above chance, perhaps due to the salience of the eye region in providing cues about emotional expressions (Gagnon et al., 2014; Roberson et al., 2012; Wegrzyn et al., 2017).
All four prior studies of children have in common the use of a behavioral approach where participants assessed the emotional state of static images of adult faces (but not child faces). Studies differed with respect to the stimuli used (MPI FACES database, Carbon & Serrano, 2021; Ruba & Pollak, 2020; ER‐40 color emotional stimuli database, Gori et al., 2021; self‐created, Schneider et al., 2022), which enhances generalizability of the findings. The studies also differed in rating procedures (e.g., verbal, computerized vs. card selection responses) and the emotions tested (e.g., negative emotions [sad, angry, fearful] vs. “basic emotions” [angry, disgust, fearful, happy, neutral, sad]). However, research is needed that utilizes stimuli with masked peer‐aged faces, especially given that emotion recognition may be more accurate for same‐aged expressers (Rhodes & Anastasi, 2012). Indeed, many children have returned to school or camp settings with mask mandates still in place, meaning that masks may still be present during both peer and non‐peer interactions. Being able to make predictions about the feeling states of peers is essential for successful peer social interactions and relationships (Barth & Bastiani, 1997; Denham et al., 2003; Edwards et al., 1984) and we need to understand how mask‐wearing is related to this critical skill.
The current study aimed to advance knowledge of different stimulus and contextual factors related to emotion recognition in children. Although, face masks are known to be a highly effective tool against the spread of the COVID‐19 virus (Li et al., 2021), our first aim was to replicate findings from four prior studies showing lower emotion recognition for masked versus unmasked faces, including, for the first time, child‐aged expressers. We hypothesized that emotion recognition would be lower for masked faces but left as exploratory the question of whether this effect would differ in magnitude for child‐ versus adult‐aged expressers (though we expected that children would show better emotion recognition for child vs. adult actors, regardless of masking, consistent with prior research that emotion recognition is thought to be more accurate for same‐aged expressers; Rhodes & Anastasi, 2012). Consistent with the emotion‐specific relevance of distinct facial regions (e.g., Bombari et al., 2013; Fischer et al., 2012; Wegrzyn et al., 2017), we hypothesized that masking would be related to worse emotion recognition for emotions expressed more saliently by the lower region of the face (e.g., happy) versus upper region of the face (e.g., sad, fearful). As an exploratory analysis, we tested whether masking would be differentially related to emotion recognition based on participant age (De Sonneville et al., 2002). Our second aim was to investigate how pandemic‐related disruptions were associated with emotion recognition. Consistent with prior research suggesting that sensory input can affect emotion recognition (Plate et al., 2019), we hypothesized that children whose parents reported more pandemic‐related restrictions and concerns (e.g., worry, social distancing, limiting of activities) would show worse emotion recognition. Finally, we investigated emotion recognition in a smaller subsample for whom data for unmasked faces were collected in the immediate months before COVID‐19 was declared a global pandemic (referred to as “pre‐pandemic subsample”). By including these participants, we could, for the first time, examine longitudinal changes in emotion recognition within the same children across the time period in which the pandemic was occurring. While we could not directly establish the facial input that children had received, prior research suggests that children adjust emotion recognition based on the input of emotions observed in their environments (Plate et al., 2019; Woodard et al., 2021). Thus, we tested the hypothesis that emotion recognition would have worsened or stagnated during the pandemic as children received less exposure to unmasked faces.
2. METHOD
De‐identified datasets, analysis scripts, and stimuli are available online at Open Science Framework (https://osf.io/69pnu/?view_only=e80f144b1fce4cabac3003fa26bef79a). The experimental task is available upon request. R version 3.6.3 was used for all analyses (R Core Team, 2019). We used the tidyverse package (Wickham et al., 2019) for data organization, lme4 package for linear mixed effect models (Bates et al., 2015), and the sjPlot and ggplot2 packages for visualization (Lüdecke et al., 2021).
2.1. Participants
Participants were recruited as part of the online Family And Child Emotion Socialization (FACES) study, a multi‐site study investigating social competence in late‐childhood (data collected between 12/02/2020 and 05/18/2021). The sample was children aged 7–12 years (N = 131; M age = 9.13 SD age = 1.16, 66 male, 64 female, one participant did not report gender; 58.02% White, 14.50% Black/African American, 10.69% Asian, 16.03% mixed race/other; 9.92% Hispanic or Latinx) recruited from two cities in the northeastern United States (Boston, n = 69; Philadelphia, n = 62). We focused on 7–12 year olds because during this development period, children rely less solely on the eye region for emotion recognition (Roberson et al., 2012; Ruba & Pollak, 2020), with evidence that emotion recognition continues to develop during this age period (Chronaki et al., 2015; Widen, 2013), particularly in terms of speed of recognition (De Sonneville et al., 2002). Parents were highly educated (.76% less than High School Diploma, 6.11% High School Diploma/GED 4.58% Some College, 2.29% Associate's Degree, 26.72% Bachelor's Degree, 56.49% Graduate Degree). Two additional participants were excluded for not completing the experimental task. A pre‐pandemic subsample (N = 35; M age = 9.15 SD age = 1.12, 21 female, 14 male; 11.11% Asian, 41.67% Black/African American, 30.56% White, 16.67% mixed race/other; 25.00% Hispanic or Latinx) had also completed the emotion recognition task (with unmasked faces only) in the months immediately before pandemic‐related restrictions on data collection were enacted (data collected between 11/06/2019 and 03/05/2020; the World Health Organization declared COVID‐19 to be a global pandemic on 03/11/2020).
2.2. Measures
2.2.1. Emotion recognition
We assessed emotion recognition in children and young adults using the Dynamic Affect Recognition Task (DART). The task includes static images of actors (child or adult) displaying one of five emotions: happy, sad, angry, fearful, or neutral (Ekman, 1992; Izard, 2007). While use of “basic” emotions has received criticism (Barrett et al., 2019; Barrett & Westlin, 2021), we focused on these five emotions to ensure that our results were comparable with recent research on masking in children. Stimuli from child actors were obtained from the National Institute of Mental Health Child Emotional Faces Picture Set (NIMH‐ChEFS) (Egger et al., 2011), a validated set of high quality, color images of children (50% female, 50% male; 50% Black/African American, 50% White). Adult stimuli were obtained from an independently validated stimulus set of amateur actors (50% female, 50% male; 36.67% Black/African American, 33% Non‐Hispanic White, 13% Asian, 16% Hispanic/Latinx) with standardized luminosity and distance (see, Figure 1 and Figure S1). There was natural variation in the facial action units conveyed by each actor. However, validation analyses established that the emotional signals were distinguishable. For the current study, the DART was adapted by adding surgical masks to stimuli using Photoshop. All participants saw the same images but were randomly assigned to which 50% of the stimuli were masked or unmasked (Set A or Set B; Figure 1A).
During the DART, the experimenter explained, “You will match different faces with an emotion. First, you will see a photo of a person expressing an emotion. Try to think of what this emotion might be. On the next screen, you will see five pictures of five emotions to choose from. The goal is to match the emotion you saw on the first screen with one of the five options”.
Participants completed two practice trials (same stimuli for everyone; one unmasked adult sad; one unmasked adult happy) and received feedback. During the task, each image was presented for 1 s to ensure brevity without sacrificing number of trials. Following stimulus presentation, participants clicked on the image they believed to best represent the emotion shown. If the participant did not respond within 6 s, the trial counted as invalid (<1% of trials were invalid). Participants completed a total of 40 trials (i.e., eight trials per emotion). For each emotion (happy, sad, angry, scared, or neutral), participants saw two masked child faces, two unmasked child faces, two masked adult faces, and two unmasked adult faces, with one exception: due an experiment building error, participants randomized to set B saw an unbalanced number of sad adults (one masked, three unmasked). We used a validated pictorial response scale that depicts prototypical facial configurations (happy, sad, neutral, scared, and angry) with the corresponding emotion word displayed below each image (see Figure 1B and Supplemental Materials for validation details from an independent sample, as well as the description of the facial action units captured by the pictorial response scale, Figure S1). Prior research supports the idea that children can assess emotion information using pictorial images (Brechet, 2017), which we intended to reduce reliance on verbal or written labels.
2.2.2. Pandemic‐related disruptions
We measured parental worries about the COVID‐19 pandemic using a total score of six items that assessed level of concern about the pandemic (e.g., dying from the virus, family contracting the virus, having financial burden) with items rated on a 5‐point scale (1 = not at all, 5 = great deal; see, Table S1 for descriptive data and frequencies for individual item responses) (α = .86, 95%CI [.83, .90]) (Barzilay et al., 2020; Waller et al., 2021). We also assessed the degree to which families adjusted to the pandemic using an average score of three items assessing social disruption (e.g., degree of social distancing, restrictions, and virtual interactions), with each item rated on a 4‐point scale (see Table S1 for descriptive data and frequencies for item responses) (α = .67, 95%CI [.57, .78]). We focus presentation of results on total scores for both the worries and social disruption scales but also report findings at the individual item‐level in the Supplemental Materials.
2.3. Procedure
Data were collected via Zoom during a single 30‐min session, which included the DART and two other tasks completed online by children. Following the Zoom session, parents completed an online Qualtrics survey within 2 days. The survey included questionnaires assessing worries and concerns about the pandemic, social disruptions due to the pandemic, and demographic information, as well as additional questionnaires that were collected as part of a broader study on emotion development. Children in the pre‐pandemic subsample completed a version of the DART featuring only unmasked faces during in‐person lab visits conducted prior to the pandemic. The data were collected as a part of the multi‐site (masked for review), which included the DART among a battery of tasks and questionnaire measures during a 3 hr lab visit. Parents provided written consent, children provided verbal assent, and the study was approved by Institutional Review Boards at both sites.
2.4. Analytic plan
To address our first aim, whether masking influences emotion recognition, we used a logistic mixed effects model and regressed accuracy on each trial (correct = 1, incorrect = 0) on stimulus type (unmasked = ‐.5, masked = .5), age of the expresser (i.e., adult = ‐.5, child = .5) and the following interactions: stimulus type by age of the expresser and stimulus type by age of the participant. To assess whether emotion recognition accuracy varied based on emotion type, we ran a second model in which we regressed accuracy on stimulus type, emotion, and their interaction. To address our second aim, and explore whether experiences during the pandemic influenced emotion recognition, we tested the effects of the pandemic worries and social disruption scales in separate logistic mixed effects models using the full sample (i.e., dependent variables were the composite scores for pandemic worries and social disruption; see Table S2 for results with individual items). Finally, to investigate change in emotion recognition across the pandemic, we used data from the pre‐pandemic subsample and regressed emotion recognition accuracy on time (before the pandemic = ‐.5, during the pandemic = .5), followed by regressing accuracy on time, emotion, and their interaction. For all models, we included a by‐participant random intercept. We also included participant age (mean‐centered), site (Boston = ‐.5, Philadelphia = .5), participant gender (female = ‐.5, male = .5), and stimulus set (i.e., which stimuli were masked, Set A = ‐.5, Set B = .5) as covariates.
3. RESULTS
Descriptive statistics and bivariate correlations between main study variables, including emotion recognition accuracy, are presented in Table 1.
TABLE 1.
N | Min, Max | M | SD | 1. | 2. | 3. | 4. | 5. | |
---|---|---|---|---|---|---|---|---|---|
1. Age (years) | 130 | 7.08, 11.87 | 9.13 | 1.16 | |||||
2. COVID‐19 worries (parent‐reported) | 128 | 6.00, 30.00 | 14.7 | 5.44 | .20* | ||||
3. COVID‐19 social disruptions (parent‐reported) | 128 | 1.33, 4.00 | 3.42 | .54 | ‐.01 | .04 | |||
4. Emotion recognition accuracy for unmasked faces (% correct) | 128 | 52%, 100% | 90.0 | 9.0 | .33*** | .11 | .09 | ||
5. Emotion recognition accuracy for masked faces (% correct) | 128 | 25%, 100% | 85.0 | 12.0 | .19* | .09 | ‐.09 | .43*** | |
6. Emotion recognition accuracy total (% correct) | 128 | 42%, 98% | 87.0 | 9.0 | .29*** | .11 | ‐.02 | .79*** | .89*** |
p < .05.
p < .01.
p < .001.
3.1. Aim 1: Emotion recognition for masked and unmasked adult and child faces
First, masking was related to lower emotion recognition accuracy (b = ‐.58, SE = .09, z = −6.57, p < .001). That is, children were less accurate at inferring the emotional cues from masked versus unmasked faces. Emotion recognition accuracy improved with age (b = .20, SE = .05, z = 3.76, p < .001). However, there was no effect of expresser age (adult vs. child) on emotion recognition (b = .12, SE = .09, z = 1.43, p = .15). Moreover, neither the interactions between expresser age and masking (b = ‐.21, SE = .17, z = −1.21, p = .23) nor between masking and participant age (b = ‐.14, SE = .07, z = −1.86, p = .06) were significant (see Table S3 for full reporting of results including covariate effects). That is, we found evidence for lower emotion recognition of masked faces, regardless of whether the emotion was produced by a child or adult.
In terms of specific emotions, emotion recognition was worse overall for sad faces compared to happy, fearful, angry, and neutral faces (omnibus: χ 2(1) = 37.34, p < .001; see Table S4 for pairwise comparisons), but no other differences in accuracy emerged between emotions. The main effect of masking was qualified by an interaction with emotion (χ 2(1) = 44.15, p < .001), such that masking was related to emotion recognition accuracy differently depending on the emotion (Table 2). Specifically, participants showed higher accuracy for unmasked versus masked faces specifically for happy (b = 1.74, SE = .29, z = 6.11, p < .001), sad (b = .60, SE = .14, z = 4.21, p < .001) and fearful (b = 1.31, SE = .24, z = 5.47, p < .001) faces. However, masking was unrelated to recognition of angry (b = .33, SE = .24, z = 1.34, p = .18) and neutral (b = ‐.45, SE = .24, z = −1.82, p = .07) faces (see, Table 2 for means and standard deviations). There was also an effect of stimulus set (b = .55, SE = .12, z = 4.47, p < .001) with higher emotion recognition accuracy among participants who were randomly assigned to stimulus set B. However, due to an experiment building error, set B contained three unmasked and only one masked sad face instead of two of each. Thus, the significantly higher accuracy for set B was likely a methodological artifact arising from participants having to identify one fewer masked sad face.
TABLE 2.
Unmasked | Masked | |||
---|---|---|---|---|
Mean | St. Dev | Mean | St. Dev | |
Happy | .97 | .17 | .85 | .35 |
Sad | .75 | .43 | .62 | .49 |
Fearful | .95 | .22 | .85 | .36 |
Angry | .94 | .24 | .92 | .28 |
Neutral | .91 | .28 | .94 | .24 |
Note: Masking was related to emotion recognition differently depending on emotion (χ2(1) = 44.15, p < .001). Stimuli included masked child faces, unmasked child faces, masked adult faces, and unmasked adult faces for each of five emotions: happy, sad, fearful, angry, and neutral. Emotion recognition was lower for masked versus unmasked faces, and specifically for happy, sad, and fearful faces, but not angry or neutral faces.
3.2. Aim 2: Associations between pandemic‐related social restrictions and concerns and emotion recognition
Frequency data and descriptive statistics for pandemic‐related social restrictions and concerns, as well as bivariate correlations for study variables are presented in Tables 1 and S1. There was no main effect of social disruptions on emotion recognition (b = .05, χ2(1) = .16, p = .69; Table S2). However, the interaction between masking and social disruptions was significant (b = ‐.34, χ2(1) = 4.31, p = .04), such that the effect of masking was related to lower emotion recognition accuracy in families reporting more social disruption (Figure 2). Parental worries about the pandemic were not related to emotion recognition nor did they moderate the effect of masking on emotion recognition (Table S2).
In the pre‐pandemic subsample, there was no overall change in emotion recognition accuracy across the pandemic (b = ‐.25, χ2(1) = 1.43, p = .23). However, emotion recognition differed over time for sad relative to all other emotions (omnibus: χ2(1) = 22.18, p < .001). Specifically, emotion recognition for sad faces decreased over time (b = −1.31, SE = .33, z = −4.04, p < .001), while emotion recognition for happy (b = .74, SE = .81, z = .91, p = .36), angry (b = .63, SE = .54, z = 1.18, p = .24), and neutral faces did not change (b = .04, SE = .44, z = .09, p = .93; means and standard deviations are in Table 3). Finally, children's emotion recognition of fearful faces improved across the period of the pandemic (b = 1.56, SE = .77, z = 2.04, p = .04).
TABLE 3.
Pre | During | |||
---|---|---|---|---|
Mean | St. Dev | Mean | St. Dev | |
Happy | .97 | .17 | .99 | .12 |
Sad | .92 | .28 | .77 | .42 |
Fearful | .93 | .25 | .99 | .12 |
Angry | .93 | .25 | .96 | .19 |
Neutral | .93 | .26 | .94 | .25 |
Note: Time (i.e., measurement of emotion recognition pre vs. during the pandemic) was related to emotion recognition differently depending on emotion (χ2(1) = 22.18, p < .001). Stimuli included masked child faces, unmasked child faces, masked adult faces, and unmasked adult faces for each of five emotions: happy, sad, fearful, angry, and neutral. Children showed specific impairments in identifying sad faces during the pandemic compared to pre‐pandemic, while they showed improvement in identifying fearful faces during the pandemic compared to pre‐pandemic.
4. DISCUSSION
We investigated children's emotion recognition in the context of the COVID‐19 pandemic. Consistent with hypotheses and prior research in both children (Carbon & Serrano, 2021; Gori et al., 2021; Ruba & Pollak, 2020; Schneider et al., 2022) and adults (Carbon, 2020; Carragher & Hancock, 2020; Freud et al., 2020; Gori et al., 2021), children showed worse emotion recognition for masked happy, sad, and fearful faces versus unmasked faces exhibiting these emotions. We add to the existing literature by showing, for the first time, the impact of masking for recognizing emotions displayed by child expressers. As children spend time in school or camp settings, in many cases with mask mandates remaining in place, it is critical to understand how children are making inferences about the emotions expressed by other children, including on the basis of reduced visual information. That is, teachers and other childcare providers may need to consider additional strategies to promote emotion understanding during peer or classroom interactions (Cameron & Tenenbaum, 2021; Sprung et al., 2015).
Notably, masking had a more pronounced negative impact on emotion recognition for happy, sad, and fearful faces compared to angry or neutral faces. The finding for happy is consistent with our hypothesis and evidence from prior research that employed eye‐tracking and facial action units to establish that the lower region the face (i.e., covered by a mask) is especially important for recognizing happy (Bombari et al., 2013; Fischer et al., 2012; Wegrzyn et al., 2017). However, our findings are inconsistent with prior research in adults suggesting that the eyes (i.e., still visible with a mask) are more important features for inferring sad and fearful. Thus, unlike adults, children may suffer more in the absence of holistic cues from several different regions of the face when inferring sad and fearful. At the same time, we still observed relatively high emotion recognition accuracy even for masked faces (i.e., >75%), suggesting that children can still infer the emotions of others even with faces masked (also see, Ruba & Pollak, 2020; Gori et al., 2021).
In relation to the pandemic, the level of parental worry about COVID‐19 (i.e., total worries score) was not related to children's emotion recognition (though see Supplemental Materials for an effect with a specific item focused on contracting COVID‐19 and reduced emotion recognition for masked faces). However, children whose parents reported more social disruptions showed significantly worse emotion recognition for masked versus unmasked faces. One explanation is that children who experienced a greater degree of social distancing and virtual interactions stayed at home more where they were surrounded by unmasked faces of family members and thus became less proficient at inferring emotions from masked faces. That is, we may have observed an absence of a practice effect for masked faces or lack of relevant social input, which would have led children to become more skilled in recognizing emotions behind a mask (Leppänen & Nelson, 2006; Plate et al., 2019; Woodard et al., 2021).
We found no change in emotion recognition accuracy among our small subsample of children who completed the task with unmasked faces prior to the pandemic and again during the pandemic. Despite the small sample size, however, our findings highlight the need for additional research to better characterize the apparent lack of change over time in recognition accuracy. For example, no change in emotion recognition over time could indicate minimal effect of masking on emotion‐relevant skills. Alternatively, no change in emotion recognition over time could suggest a stagnation in emotion recognition skills, given that emotion recognition normally improves during this age range (Chronaki et al., 2015; Schneider et al., 2022; Widen, 2013). Indeed, across the full sample, we saw age‐related effects on emotion recognition overall, with older children showing higher degrees of accuracy. At the same time, the follow‐up period between our two assessments (≈12–18 months) may not have been long enough to capture developmental change in emotion recognition accuracy. Other metrics, including speed of response (e.g., De Sonneville et al., 2002) may be more sensitive to increases in emotion processing abilities during middle‐to‐late childhood and should be the target of future longitudinal efforts.
Notably, while there was no difference in total scores for emotion recognition in the pre‐pandemic versus pandemic assessment, we did find emotion‐specific differences. Namely, emotion recognition for sad faces worsened across the pandemic, while emotion recognition for fearful faces improved across the pandemic. One explanation for these divergent findings is that masking allows for compensatory mechanisms to exaggerate the expression of fearful (i.e., through the eyes and eyebrows) but stymies such compensation cues for sad (i.e., more exaggerated down‐turning of the mouth would be hidden under a mask) (Carbon, 2020). This explanation fits with evidence that we actively tune our facial expressions when communicating with others (Greenaway et al., 2021) and is consistent with some of the data here in which some participants show high emotion recognition of masked faces. An alternative explanation has to do with established developmental trajectories of emotion recognition, which already suggest more delayed normative proficiency and less accuracy for recognizing sad relative to other emotions, a trend which may have been exacerbated in the context of the pandemic (Chronaki et al., 2015; Herba et al., 2006).
There were a number of strengths to the current study, including a new task featuring both child and adult expressers, masked and unmasked stimuli, and a subsample leveraging a prospective, longitudinal design. However, there were several limitations. First, child stimuli came from a database and adult stimuli were generated by actors, such that emotions may not generalize to spontaneous “real” expressions (Barrett et al., 2019). In addition, while both stimulus sets were validated without masks, this study is the first use face masks. Task stimuli are freely available (https://osf.io/69pnu/?view_only=e80f144b1fce4cabac3003fa26bef79a), thus encouraging their use in future studies. Second, our pictorial scale was new and prior studies have not established whether the pictorial depictions map consistently onto the emotions expressed by the masked and unmasked faces. Third, the sample was drawn from two northeastern cities with a high prevalence of mask‐wearing and strict social restrictions. Thus, our results may not generalize to children who experienced varying social inputs during the pandemic based on well‐documented differences between states in mask mandates, physical distancing regulations, and school closures (Yang et al., 2022). Fourth, the parents in our sample tended to be highly educated (i.e., more than half had a graduate‐level degree), such that our results may not generalize to children of parents with lower levels of education, among whom the pandemic may have affected in unmeasured ways that were relevant to our study variables (e.g., ability to socially distance) (Patel et al., 2020). Finally, although the scale assessing worries about the pandemic had been validated in prior studies (Barzilay et al., 2020; Waller et al., 2021), items assessing social disruption due to the pandemic not been used or validated in tasks prior to the current study.
In sum, we used a new paradigm to evaluate emotion recognition during a global pandemic that has disrupted the richness and quality of social inputs for children. We add to prior research indicating that facial occlusion (in this case, due to mask‐wearing) negatively impacts emotion recognition accuracy. We also showed that greater social disruption exacerbated the negative impact of mask‐wearing on emotion recognition. Finally, we show that changes in emotion recognition varied across the pandemic, suggesting that compensatory diagnostic cues may be more effective for some emotions than others (Gagnon et al., 2014). Findings underscore the importance of future work to investigate how children use their social context (pandemic or otherwise) to make sense of emotion cues. Through these efforts, we can design more effective strategies to target the emotion recognition difficulties that are known to put children at risk for psychpathology through novel methods that alter children's social input (e.g., Vajawat et al., 2021).
CONFLICTS OF INTEREST
The authors report no conflicts of interest.
5.
Supporting information
ACKNOWLEDGMENTS
This work was funded by institutional support from the University of Pennsylvania (R.W.) and Boston University (N.J.W.), a University of Pennsylvania MindCORE Postdoctoral Fellowship (R.C.P.), funding from the John and Polly Sparks Foundation (American Psychological Foundation) (R.W.), and funding from the National Institute of Mental Health (R.W., N.J.W.; R01MH125904).
Chester, M. , Plate, R. C. , Powell, T. , Rodriguez, Y. , Wagner, N. J. , & Waller, R. (2022). The COVID‐19 pandemic, mask‐wearing, and emotion recognition during late‐childhood. Social Development, 00, 1–14. 10.1111/sode.12631
Contributor Information
Nicholas J. Wagner, Email: njwagner@bu.edu.
Rebecca Waller, Email: rwaller@sas.upenn.edu.
DATA AVAILABILITY STATEMENT
De‐identified datasets, analysis scripts, and stimuli are available online at Open Science Framework (https://osf.io/69pnu/?view_only=e80f144b1fce4cabac3003fa26bef79a).
REFERENCES
- Barrett, L. F. , Adolphs, R. , Marsella, S. , Martinez, A. M. , & Pollak, S. D. (2019). Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest, 20(1), 1–68. 10.1177/1529100619832930 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barrett, L. F. , & Westlin, C. (2021). Navigating the science of emotion. In Emotion measurement (pp. 39–84). Elsevier. [Google Scholar]
- Barth, J. M. , & Bastiani, A. (1997). A longitudinal study of emotion recognition and preschool children's social behavior. Merrill‐Palmer Quarterly (1982‐), 43, 107–128. [Google Scholar]
- Barzilay, R. , Moore, T. M. , Greenberg, D. M. , DiDomenico, G. E. , Brown, L. A. , White, L. K. , Gur, R. C. , & Gur, R. E. (2020). Resilience, COVID‐19‐related stress, anxiety and depression during the pandemic in a large population enriched for healthcare providers. Translational Psychiatry, 10(1), 1–8. 10.1038/s41398-020-00982-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bates, D. , Kliegl, R. , Vasishth, S. , & Baayen, H. (2015). Parsimonious mixed models. https://arxiv.org/abs/1506.04967
- Batty, M. , & Taylor, M. J. (2006). The development of emotional face processing during childhood. Developmental Science, 9(2), 207–220. 10.1111/j.1467-7687.2006.00480.x [DOI] [PubMed] [Google Scholar]
- Bombari, D. , Schmid, P. C. , Schmid Mast, M. , Birri, S. , Mast, F. W. , & Lobmaier, J. S. (2013). Emotion recognition: The role of featural and configural face information. Quarterly Journal of Experimental Psychology, 66(12), 2426–2442. 10.1080/17470218.2013.789065 [DOI] [PubMed] [Google Scholar]
- Brechet, C. (2017). Children's recognition of emotional facial expressions through photographs and drawings. The Journal of Genetic Psychology, 178(2), 139–146. 10.1080/00221325.2017.1286630 [DOI] [PubMed] [Google Scholar]
- Cameron, L. , & Tenenbaum, H. R. (2021). Lessons from developmental science to mitigate the effects of the COVID‐19 restrictions on social development. Group Processes & Intergroup Relations, 24(2), 231–236. [Google Scholar]
- Carbon, C.‐C. (2020). Wearing face masks strongly confuses counterparts in reading emotions. Frontiers in Psychology, 11, 566886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carbon, C.‐C. , & Serrano, M. (2021). The impact of face masks on the emotional reading abilities of children—A lesson from a joint school–university project. i‐Perception, 12(4), 204166952110382. 10.1177/20416695211038265 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carragher, D. J. , & Hancock, P. J. (2020). Surgical face masks impair human face matching performance for familiar and unfamiliar faces. Cognitive Research: Principles and Implications, 5(1), 1–15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chronaki, G. , Hadwin, J. A. , Garner, M. , Maurage, P. , & Sonuga‐Barke, E. J. (2015). The development of emotion recognition from facial expressions and non‐linguistic vocalizations during childhood. British Journal of Developmental Psychology, 33(2), 218–236. 10.1111/bjdp.12075 [DOI] [PubMed] [Google Scholar]
- Costello, E. J. , Copeland, W. , & Angold, A. (2011). Trends in psychopathology across the adolescent years: What changes when children become adolescents, and when adolescents become adults? Journal of Child Psychology and Psychiatry, 52(10), 1015–1025. 10.1111/j.1469-7610.2011.02446.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- De Sonneville, L. , Verschoor, C. , Njiokiktjien, C. , Op het Veld, V. , Toorenaar, N. , & Vranken, M. (2002). Facial identity and facial emotions: Speed, accuracy, and processing strategies in children and adults. Journal of Clinical and Experimental Neuropsychology, 24(2), 200–213. [DOI] [PubMed] [Google Scholar]
- Demenescu, L. R. , Kortekaas, R. , den Boer, J. A. , & Aleman, A. (2010). Impaired attribution of emotion to facial expressions in anxiety and major depression. Plos One, 5(12), e15058. 10.1371/journal.pone.0015058 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Denham, S. A. , Blair, K. A. , DeMulder, E. , Levitas, J. , Sawyer, K. , Auerbach–Major, S. , & Queenan, P. (2003). Preschool emotional competence: Pathway to social competence? Child Development, 74(1), 238–256. 10.1111/1467-8624.00533 [DOI] [PubMed] [Google Scholar]
- Edwards, R. , Manstead, A. S. R. , & Macdonald, C. J. (1984). The relationship between children's sociometric status and ability to recognize facial expressions of emotion. European Journal of Social Psychology, 14(2), 235–238. 10.1002/ejsp.2420140212 [DOI] [Google Scholar]
- Egger, H. L. , Pine, D. S. , Nelson, E. , Leibenluft, E. , Ernst, M. , Towbin, K. E. , & Angold, A. (2011). The NIMH Child Emotional Faces Picture Set (NIMH‐ChEFS): A new set of children's facial emotion stimuli. International Journal of Methods in Psychiatric Research, 20(3), 145–156. 10.1002/mpr.343 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6(3‐4), 169–200. [Google Scholar]
- Fischer, M. , Ekenel, H. K. , & Stiefelhagen, R. (2012). Analysis of partial least squares for pose‐invariant face recognition . Paper presented at the 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).
- Freud, E. , Stajduhar, A. , Rosenbaum, R. S. , Avidan, G. , & Ganel, T. (2020). The COVID‐19 pandemic masks the way people perceive faces. Scientific Reports, 10(1), 1–8. 10.1038/s41598-020-78986-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gagnon, M. , Gosselin, P. , & Maassarani, R. (2014). Children's ability to recognize emotions from partial and complete facial expressions. The Journal of Genetic Psychology, 175(5), 416–430. 10.1080/00221325.2014.941322 [DOI] [PubMed] [Google Scholar]
- Gori, M. , Schiatti, L. , & Amadeo, M. B. (2021). Masking emotions: Face masks impair how we read emotions. Frontiers in Psychology, 12, 1541. 10.3389/fpsyg.2021.669432 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greenaway, K. H. , Kalokerinos, E. K. , Hinton, S. , & Hawkins, G. E. (2021). Emotion experience and expression goals shape emotion regulation strategy choice. Emotion (Washington, D.C.), 21, 1452–1469. 10.1037/emo0001012 [DOI] [PubMed] [Google Scholar]
- Griffith, A. K. (2020). Parental burnout and child maltreatment during the COVID‐19 pandemic. Journal of Family Violence, 37, 725–731. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grossard, C. , Chaby, L. , Hun, S. , Pellerin, H. , Bourgeois, J. , Dapogny, A. , Ding, H. , Serret, S. , Foulon, P. , Chetouani, M. , Chen, L. , Bailly, K. , Grynszpan, O. , & Cohen, D. (2018). Children facial expression production: Influence of age, gender, emotion subtype, elicitation condition and culture. Frontiers in Psychology, 9, 446. 10.3389/fpsyg.2018.00446 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guy, G. P. , Massetti, G. M. , & Sauber‐Schatz, E. (2021). Mask mandates, on‐premises dining, and COVID‐19. JAMA, 325(21), 2199–2200. 10.1001/jama.2021.5455 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Herba, C. M. , Landau, S. , Russell, T. , Ecker, C. , & Phillips, M. L. (2006). The development of emotion‐processing in children: Effects of age, emotion, and intensity. Journal of Child Psychology and Psychiatry, 47(11), 1098–1106. 10.1111/j.1469-7610.2006.01652.x [DOI] [PubMed] [Google Scholar]
- Izard, C. , Fine, S. , Schultz, D. , Mostow, A. , Ackerman, B. , & Youngstrom, E. (2001). Emotion knowledge as a predictor of social behavior and academic competence in children at risk. Psychological Science, 12(1), 18–23. 10.1111/1467-9280.00304 [DOI] [PubMed] [Google Scholar]
- Izard, C. E. (2007). Basic emotions, natural kinds, emotion schemas, and a new paradigm. Perspectives on Psychological Science, 2(3), 260–280. 10.1111/j.1745-6916.2007.00044.x [DOI] [PubMed] [Google Scholar]
- Kohls, G. , Baumann, S. , Gundlach, M. , Scharke, W. , Bernhard, A. , Martinelli, A. , Ackermann, K. , Kersten, L. , Prätzlich, M. , Oldenhof, H. , Jansen, L. , van den Boogaard, L. , Smaragdi, A. , Gonzalez‐Madruga, K. , Cornwell, H. , Rogers, J. C. , Pauli, R. , Clanton, R. , Baker, R. , … Konrad, K. (2020). Investigating sex differences in emotion recognition, learning, and regulation among youths with conduct disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 59(2), 263–273. [DOI] [PubMed] [Google Scholar]
- Lam, C. B. , McHale, S. M. , & Crouter, A. C. (2014). Time with peers from middle childhood to late adolescence: Developmental course and adjustment correlates. Child Development, 85(4), 1677–1693. 10.1111/cdev.12235 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leppänen, J. M. , & Nelson, C. A. (2006). The development and neural bases of facial emotion recognition. Advances in Child Development and Behavior, 34, 207–246. 10.1016/S0065-2407(06)80008-X [DOI] [PubMed] [Google Scholar]
- Li, Y. , Liang, M. , Gao, L. , Ahmed, M. A. , Uy, J. P. , Cheng, C. , Zhou, Q. , & Sun, C. (2021). Face masks to prevent transmission of COVID‐19: A systematic review and meta‐analysis. American Journal of Infection Control, 49(7), 900–906. 10.1016/j.ajic.2020.12.007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lüdecke, D. , Ben‐Shachar, M. S. , Patil, I. , Waggoner, P. , & Makowski, D. (2021). performance: An R package for assessment, comparison and testing of statistical models. Journal of Open Source Software, 6(60), 3139. 10.21105/joss.03139 [DOI] [Google Scholar]
- Marini, M. , Ansani, A. , Paglieri, F. , Caruana, F. , & Viola, M. (2021). The impact of facemasks on emotion recognition, trust attribution and re‐identification. Scientific Reports, 11(1), 1–14. 10.1038/s41598-021-84806-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mickells, G. E. , Figueroa, J. , West, K. W. , Wood, A. , & McElhanon, B. O. (2021). Adherence to masking requirement during the COVID‐19 pandemic by early elementary school children. Journal of School Health, 91(7), 555–561. 10.1111/josh.13033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Niedenthal, P. M. , & Brauer, M. (2012). Social functionality of human emotion. Annual Review of Psychology, 63, 259–285. 10.1146/annurev.psych.121208.131605 [DOI] [PubMed] [Google Scholar]
- Patel, J. , Nielsen, F. , Badiani, A. , Assi, S. , Unadkat, V. , Patel, B. , Ravindrane, R. , & Wardle, H. (2020). Poverty, inequality and COVID‐19: The forgotten vulnerable. Public Health, 183, 110. 10.1016/j.puhe.2020.05.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pellicano, E. , & Rhodes, G. (2003). Holistic processing of faces in preschool children and adults. Psychological Science, 14(6), 618–622. 10.1046/j.0956-7976.2003.psci_1474.x [DOI] [PubMed] [Google Scholar]
- Plate, R. C. , Wood, A. , Woodard, K. , & Pollak, S. D. (2019). Probabilistic learning of emotion categories. Journal of Experimental Psychology: General, 148(10), 1814. 10.1037/xge0000529 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rhodes, M. G. , & Anastasi, J. S. (2012). The own‐age bias in face recognition: A meta‐analytic and theoretical review. Psychological Bulletin, 138(1), 146. 10.1037/a0025750 [DOI] [PubMed] [Google Scholar]
- Roberson, D. , Kikutani, M. , Döge, P. , Whitaker, L. , & Majid, A. (2012). Shades of emotion: What the addition of sunglasses or masks to faces reveals about the development of facial expression processing. Cognition, 125(2), 195–206. 10.1016/j.cognition.2012.06.018 [DOI] [PubMed] [Google Scholar]
- Ruba, A. L. , & Pollak, S. D. (2020). Children's emotion inferences from masked faces: Implications for social interactions during COVID‐19. Plos One, 15(12), e0243708. 10.1371/journal.pone.0243708 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schneider, J. , Sandoz, V. , Equey, L. , Williams‐Smith, J. , Horsch, A. , & Graz, M. B. (2022). The role of face masks in the recognition of emotions by preschool children. JAMA Pediatrics, 176(1), 96–98. 10.1001/jamapediatrics.2021.4556 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Southam‐Gerow, M. A. , & Kendall, P. C. (2002). Emotion regulation and understanding: Implications for child psychopathology and therapy. Clinical Psychology Review, 22(2), 189–222. 10.1016/S0272-7358(01)00087-3 [DOI] [PubMed] [Google Scholar]
- Sprung, M. , Münch, H. M. , Harris, P. L. , Ebesutani, C. , & Hofmann, S. G. (2015). Children's emotion understanding: A meta‐analysis of training studies. Developmental Review, 37, 41–65. 10.1016/j.dr.2015.05.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Team, R. C. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing. [Google Scholar]
- Vajawat, B. , Varshney, P. , & Banerjee, D. (2021). Digital gaming interventions in psychiatry: Evidence, applications and challenges. Psychiatry Research, 295, 113585. 10.1016/j.psychres.2020.113585 [DOI] [PubMed] [Google Scholar]
- Waller, R. , Powell, T. , Rodriguez, Y. , Corbett, N. , Perlstein, S. , White, L. K. , Barzilay, R. , & Wagner, N. J. (2021). The impact of the COVID‐19 pandemic on children's conduct problems and callous‐unemotional traits. Child Psychiatry & Human Development, 52(6), 1012–1023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wegrzyn, M. , Vogt, M. , Kireclioglu, B. , Schneider, J. , & Kissler, J. (2017). Mapping the emotional face. How individual face parts contribute to successful emotion recognition. Plos One, 12(5), e0177239. 10.1371/journal.pone.0177239 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wickham, H. , Averick, M. , Bryan, J. , Chang, W. , McGowan, L. D. A. , François, R. , Grolemund, G. , Hayes, A. , Henry, L. , Hester, J. , Kuhn, M. , Pedersen, T. L. , Miller, E. , Bache, S. M. , Müller, K. , Ooms, J. , Robinson, D. , Seidel, D. P. , Spinu, V. , … Yutani, H. (2019). Welcome to the Tidyverse. Journal of Open Source Software, 4(43), 1686. 10.21105/joss.01686 [DOI] [Google Scholar]
- Widen, S. C. (2013). Children's interpretation of facial expressions: The long path from valence‐based to specific discrete categories. Emotion Review, 5(1), 72–77. 10.1177/1754073912451492 [DOI] [Google Scholar]
- Woodard, K. , Plate, R. C. , & Pollak, S. D. (2022). Children track probabilistic distributions of facial cues across individuals. Journal of Experimental Psychology: General, 151(2), 506–511. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yang, Y. T. , DeRoo, S. S. , & Morain, S. R. (2022). State mask mandate bans for schools: Law, science, and public health. Pediatrics, 149(1), e2021054724. 10.1542/peds.2021-054724 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
De‐identified datasets, analysis scripts, and stimuli are available online at Open Science Framework (https://osf.io/69pnu/?view_only=e80f144b1fce4cabac3003fa26bef79a).