Abstract
We investigated the influence of contextual expressions on emotion recognition accuracy and gaze patterns among American and Chinese participants. We expected Chinese participants would be more influenced by, and attend more to, contextual information than Americans. Consistent with our hypothesis, Americans were more accurate than Chinese participants at recognizing emotions embedded in the context of other emotional expressions. Eye tracking data suggest that, for some emotions, Americans attended more to the target faces and made more gaze transitions to the target face than Chinese. For all emotions except anger and disgust, Americans appeared to use more of a contrasting strategy where each face was individually contrasted with the target face, compared with Chinese who used less of a contrasting strategy. Both cultures were influenced by contextual information, although the benefit of contextual information depended upon the perceptual dissimilarity of the contextual emotions to the target emotion and the gaze pattern employed during the recognition task.
Eye tracking work suggests that there are cultural differences in how Easterners and Westerners scan an emotional expression (Jack, Blais, Scheepers, Schyns, & Caldara, 2009). Recent work by Masuda and colleagues (Masuda et al., 2008) suggests that cultural differences in attentional patterns to contextual elements also influence emotion recognition: Japanese participants’ intensity ratings varied according to the emotional expression of surrounding faces, while Westerners’ intensity ratings did not. Attentional patterns consistent with these cultural differences were confirmed with eye tracking and recognition memory.
These findings may be a reflection of cultural differences in worldview, termed the social orientation hypothesis (see Markus & Kitayama, 1991; Nisbett, Peng, Choi, & Norenzayan, 2001 for reviews; Varnum, Grossmann, Kitayama, & Nisbett, 2010). Differences in worldview may relate to attention: European Americans, coming from a more individualist culture, tend to exhibit an analytic pattern of attention and interpret objects in a scene by their defining attributes, while East Asians, coming from a more collectivist culture, tend toward a holistic pattern of attention and perceive objects in terms of their relationships to other objects (e.g., Masuda & Nisbett, 2001). In emotion recognition research this may be best reflected in differences in the purposes for which members of each culture attend to contextual information. Several studies suggest that Americans do use contextual information when making emotion judgments (Aviezer et al., 2008; Barrett & Kensinger, 2010). Instead of an “attend to context vs. ignore context” dichotomy, an investigation of cultural differences in how context is used in emotion perception may be more informative. In the present study we aimed to do just that by asking participants from different cultures to identify the emotion expressed by a target face surrounded by contextual faces of a different emotion. We sought to identify how context, defined as these surrounding facial expressions, was used to make emotion judgments by recording eye movements to the target and contextual faces during the emotion recognition task. Applying the social orientation hypothesis to emotion recognition in context adds texture to the simple idea that one group considers context while the other ignores it: Emotion judgments in both cultures may be influenced by context, but cultural differences in worldview may lead to differences in how the contextual information is used.
Cultural Differences in the Use of Context in Emotion Recognition
Given that context has been shown to influence emotion attributions in American samples, and East Asian cultures exhibit context-sensitivity in their visual attention (Masuda & Nisbett, 2001; Nisbett et al., 2001), it is clear that both individualist and collectivist cultures attend to contextual information. How such contextual information is used when making an emotion judgment is less clear, although the outcome of this process does appear to differ across cultures (Masuda et al., 2008). Thus, in the present study, we used eye tracking to specify cultural differences in attention patterns to contextual information in an emotion recognition task. In particular, we wondered whether members of an individualist culture would tend toward a more analytic (contrasting context) attentional style than members of a collectivist culture when judging the emotional expression of a target face surrounded by other facial expressions. We chose to use stimuli of real faces, rather than the cartoons used in prior cross-cultural emotion recognition research (Masuda et al., 2008), to enhance the ecological validity of the task.
Hypotheses
Hypothesis 1
We expected Chinese participants’ recognition accuracy1 to be more affected by the surrounding faces than Americans’ recognition. Thus, for our first hypothesis we expected Americans to be more accurate at identifying a target face surrounded by other facial expressions than Chinese.
Hypothesis 2
We also expected cultural differences in accuracy as a function of the emotion of the contextual faces. We expected American participants’ accuracy (but not Chinese participants’ accuracy) to benefit more on trials where a perceptually similar emotion is the context, compared to other contextual emotions. Past work suggests that when a misidentification is made, both American and Chinese participants commonly confuse surprise with fear (or vice versa) and anger with disgust (or vice versa; e.g., Isaacowitz et al., 2007; Wang, K. et al., 2006), perhaps because of their perceptual similarity (Aviezer et al., 2008; Susskind, Littlewort, Bartlett, Movellan, & Anderson, 2007) . If Chinese participants are more likely than Americans to integrate the expressions of the contextual faces with the target face, the combination of surprise and fear faces (or anger and disgust faces) in the display may lead to a perception of the contextual face rather than the target. On the other hand, if Americans are more likely to use a contrasting strategy, the presence of often-confused contextual faces may help differentiate the target face from the often confused expression.
Hypothesis 3
For the eye tracking data, we expected American participants would fixate more on the target face than Chinese participants because we thought Americans would focus on the target’s emotion as an individual emotion but Chinese would also attend to the contextual faces. Attending more to the contextual faces would indicate a more holistic attentional style because the contextual faces may be receiving more weight or consideration in the emotion judgment; whereas less relative fixation duration to contextual faces would be evidence for more of an analytic attentional style (Masuda et al., 2008). For our first eye tracking variable of interest, relative fixation duration, we expected American participants to spend greater relative fixation duration on the target faces than Chinese participants.
Hypothesis 4
The second way eye tracking data could suggest differences in patterns of attention is in the gaze pattern of the fixations from one area of interest (AOI) to another. An analytic pattern of attention may be executed by looking at the target face and then looking at one of the contextual faces and then looking back at the target face…and so on for each contextual face, suggesting that a contrasting strategy is being utilized to judge the emotion of the target face. This gaze pattern can be captured in eye tracking data by counting the transitions from one AOI to another that include the target face. If an individual has more transitions that include the target face, or target transitions, this means they are shifting their gaze back to the target face more often than an individual with fewer target transitions. Thus, a greater number of target transitions would reflect an analytic contrasting strategy. Thus, for our second eye tracking variable of interest, target transitions, we expected Americans to use a contrasting viewing strategy more than Chinese, as exhibited by Americans having a greater percentage of target transitions than Chinese.
Hypothesis 5
We also expected differences in eye tracking patterns to be related to emotion recognition accuracy, with greater target transitions being related to greater accuracy, perhaps differentially for emotions that exhibit the largest cultural differences.
Method
Participants
We recruited 184 individuals for this study: 90 Americans and 94 Hong Kong Chinese. We only recruited individuals who were born and raised in America [Hong Kong] or had lived outside the U.S. [Hong Kong] for three years or less. Three American and eight Chinese participants were excluded because their emotion responses were lost due to technical failures. This left 87 American (62 female, 25 male; age ranges 18–29 and 62–86 years; Mage of young adults = 20.45 years, SD = 2.32; Mage of older adults = 72.87 years, SD = 6.48) and 86 Chinese (57 female, 29 male; age ranges 18–30 and 60–82 years; Mage of young adults = 21.00 years, SD = 2.68; Mage of older adults = 68.87 years, SD = 6.03) participants for analysis.
Materials and Procedure
The equipment, software, stimuli, and set-up were identical in America and Hong Kong (e.g., the same brand and model of eye tracker, with identical settings, was used in the two locations). Verbal communication during the experimental sessions was in English in America and in Cantonese in Hong Kong. First, the experimenter calibrated the eye tracker camera (Model 504; Applied Science Laboratories, Bedford, MA; sampling rate of 60 Hz) to the participant’s left eye using a 17-point calibration. Next, participants judged the target emotion displayed in 216 slides which consisted of 36 target slides for each of six emotions (i.e., 36 slides where the intended emotion of the target face was anger, 36 slides where the intended emotion of the target face was disgust…fear…happy…sad…and surprise). The slides were presented in one of four different pseudo-random orders with GazeTracker 7.2 software (EyeTellect, LLC, Charlottesville, VA) on a 17 inch monitor with a screen resolution of 1280 × 1024 pixels and a screen refresh rate of 60 Hz. Each slide displayed the faces of five different people (balanced across gender) in a cross shape with the target face indicated by a red box surrounding it (see Figure 1 for sample slides). Half of the slides depicted five Caucasian faces taken from the Pictures of Facial Affect (Ekman & Friesen, 1976) and Japanese and Caucasian Facial Expressions of Emotion (Matsumoto & Ekman, 1988) face sets; the other half of the slides depicted five Chinese faces from the face set created by Wang and Markham (Wang, L. & Markham, 1999). We cropped the images as close to the faces (including hair) as possible, eliminating background, in order to standardize the size of the face across stimuli. We also standardized the remaining background of the images to gray. For each slide, the five images were displayed on a black background (998 × 796 pixels) with 7.56 pixels of space between images. The size of each face image was 159 × 239 pixels. Participants were seated at a distance of 80 cm from the screen, and images were subtended 3.01 × 4.52 of visual angle. A card listing the six emotion choices (written in English in America; and in Chinese characters in Hong Kong) was available throughout the task: anger, disgust, fear, happy, sad, or surprise.
Figure 1.
Sample slides. A: Ethnicity of Faces = American. Congruency = Congruent. Location of Target = Center. B: Ethnicity of Faces = Chinese. Congruency = Incongruent. Contextual Face Type = Fear. Location of Target = Left. Target face was indicated to participants by a red line surrounding the photo (indicated here by a dotted white line).
Of the 36 target slides for each emotion, 24 slides depicted the target face in the center of the cross (Figure 1A) and 12 depicted the target in one of the other positions in the cross (Figure 1B), to reduce the likelihood that participants would always simply focus on the center face in each trial. We also included six slides for each emotion in which all five faces displayed the same emotion (congruent; Figure 1A). The remaining 30 trials for each emotion displayed four faces with the same emotion, but the target face displayed a different emotion (incongruent; Figure 1B). On these 30 incongruent trials for each emotion, each of the five non-target emotions appeared as the contextual faces on six slides. We focused our analysis on the incongruent slides only.2
The following instructions were given to participants:
On each slide, you will see five people who all belong to a group together. One of the pictures will have a red box around it; this is the target picture. Your task is to verbally say out loud what emotion the target picture is expressing. Please use one of the following multiple choice labels: anger, disgust, fear, happy, sad, or surprise.
Participants were also instructed to keep their head as still as possible during the task. Throughout the task, the experimenter controlled the eye tracking camera to continuously track the participants’ eye. Participants received 10 practice trials. Vocal responses were timed and recorded and the task was self-paced. On average, Chinese participants (M = 3,575.50 ms, SD = 1,396.17) spent more time on each slide than Americans (M = 2,754.96 ms, SD = 763.65), t(124.83)3 = 4.68, p < .001, d = .73.
Scoring
The uncorrected percentage accuracy hit rates for each emotion, separately by culture, are presented in Table 1. To account for response biases, we computed corrected accuracy for each emotion according to the method outlined by Wagner (1993), and then the proportions were normalized using an arcsine transformation. We present the analyses using the unbiased hit rates, but we also conducted the analyses on the raw accuracy hit rates for comparison with other studies.
Table 1.
Uncorrected (Raw) Emotion Recognition Accuracy and Confusion Percentages for American (AM) and Chinese (CH) Participants
| Intended Target Expression |
Intended Contextual Expression |
Response |
|||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Anger | Disgust | Fear | Happy | Sad | Surprise | ||||||||
| AM | CH | AM | CH | AM | CH | AM | CH | AM | CH | AM | CH | ||
| Anger | Disgust | 79.12% | 72.29% | 5.56% | 17.64% | 2.49% | 2.71% | 0.38% | 0.19% | 11.30% | 5.04% | 1.15% | 1.74% |
| Fear | 78.16% | 65.70% | 5.17% | 17.64% | 4.21% | 3.88% | 0.19% | 0.39% | 11.30% | 6.78% | 0.96% | 5.23% | |
| Happy | 74.33% | 65.70% | 9.00% | 21.12% | 1.92% | 2.91% | 0.38% | 0.00% | 13.22% | 7.56% | 1.15% | 2.71% | |
| Sad | 75.86% | 58.33% | 8.05% | 15.89% | 5.94% | 5.62% | 0.00% | 0.19% | 5.36% | 5.62% | 4.79% | 13.95% | |
| Surprise | 77.78% | 69.19% | 5.17% | 18.41% | 1.53% | 2.52% | 0.19% | 0.78% | 14.94% | 6.20% | 0.38% | 2.71% | |
| Disgust | Anger | 16.86% | 33.14% | 80.84% | 50.19% | 0.57% | 5.62% | 0.19% | 1.36% | 0.00% | 6.40% | 1.53% | 3.10% |
| Fear | 5.94% | 21.90% | 89.66% | 55.04% | 2.49% | 7.95% | 0.19% | 0.97% | 1.15% | 7.17% | 0.57% | 6.20% | |
| Happy | 19.92% | 32.56% | 71.26% | 46.90% | 0.96% | 3.49% | 3.07% | 3.88% | 3.45% | 8.14% | 1.34% | 4.26% | |
| Sad | 20.11% | 36.05% | 74.90% | 50.00% | 0.96% | 2.91% | 0.19% | 0.58% | 3.45% | 8.33% | 0.38% | 1.74% | |
| Surprise | 12.07% | 24.03% | 81.23% | 55.04% | 1.53% | 5.62% | 0.00% | 0.39% | 3.07% | 11.63% | 2.11% | 2.91% | |
| Fear | Anger | 13.60% | 14.73% | 14.56% | 17.64% | 46.17% | 19.96% | 0.00% | 0.78% | 5.75% | 9.50% | 19.92% | 37.21% |
| Disgust | 5.75% | 9.11% | 6.32% | 8.72% | 54.21% | 29.07% | 0.19% | 0.97% | 6.32% | 6.20% | 27.20% | 45.54% | |
| Happy | 10.15% | 8.91% | 18.97% | 18.22% | 36.78% | 22.29% | 0.96% | 2.91% | 6.51% | 7.17% | 26.63% | 40.12% | |
| Sad | 2.87% | 6.98% | 2.11% | 7.95% | 51.34% | 25.39% | 0.57% | 0.78% | 8.24% | 8.72% | 34.87% | 50.00% | |
| Surprise | 4.60% | 6.40% | 11.49% | 13.37% | 50.00% | 24.61% | 0.57% | 0.58% | 4.79% | 6.20% | 28.54% | 48.26% | |
| Happy | Anger | 0.00% | 0.39% | 0.00% | 0.00% | 0.00% | 0.58% | 100% | 98.64% | 0.00% | 0.39% | 0.00% | 0.00% |
| Disgust | 0.38% | 0.19% | 0.38% | 1.55% | 0.00% | 0.39% | 99.04% | 96.90% | 0.19% | 0.78% | 0.00% | 0.19% | |
| Fear | 0.00% | 0.19% | 0.00% | 1.16% | 0.00% | 0.00% | 99.62% | 97.67% | 0.38% | 0.00% | 0.00% | 0.97% | |
| Sad | 0.00% | 0.00% | 0.00% | 0.97% | 0.00% | 0.39% | 99.62% | 98.45% | 0.38% | 0.19% | 0.00% | 0.00% | |
| Surprise | 0.19% | 0.00% | 0.19% | 0.19% | 0.00% | 0.00% | 98.66% | 98.45% | 0.38% | 0.58% | 0.57% | 0.78% | |
| Sad | Anger | 7.28% | 11.24% | 7.09% | 18.60% | 4.60% | 7.36% | 0.00% | 0.39% | 79.89% | 59.11% | 1.15% | 2.52% |
| Disgust | 4.60% | 7.36% | 10.15% | 17.44% | 8.43% | 10.85% | 4.60% | 4.07% | 66.86% | 54.84% | 5.36% | 4.07% | |
| Fear | 2.87% | 9.88% | 4.21% | 18.80% | 4.02% | 6.98% | 0.19% | 1.16% | 86.21% | 57.95% | 2.49% | 3.49% | |
| Happy | 3.83% | 10.27% | 9.00% | 17.83% | 6.13% | 6.01% | 0.57% | 3.29% | 78.35% | 59.30% | 2.11% | 2.13% | |
| Surprise | 2.30% | 6.59% | 5.17% | 16.09% | 5.36% | 6.59% | 0.00% | 0.19% | 83.33% | 66.09% | 3.83% | 3.68% | |
| Surprise | Anger | 0.38% | 0.78% | 0.00% | 2.91% | 3.64% | 9.11% | 1.15% | 9.30% | 0.19% | 1.74% | 94.64% | 75.97% |
| Disgust | 0.00% | 1.36% | 0.77% | 6.20% | 7.28% | 11.82% | 1.34% | 3.10% | 0.00% | 1.36% | 90.61% | 75.39% | |
| Fear | 0.38% | 1.74% | 0.19% | 6.40% | 5.56% | 10.08% | 2.30% | 10.08% | 0.96% | 1.16% | 90.61% | 70.16% | |
| Happy | 0.77% | 1.55% | 0.00% | 4.26% | 8.62% | 11.82% | 0.77% | 3.49% | 0.00% | 0.39% | 89.85% | 78.10% | |
| Sad | 0.96% | 2.71% | 0.00% | 4.65% | 4.41% | 11.43% | 0.00% | 4.07% | 0.38% | 1.36% | 94.25% | 75.78% | |
Note. Hit-rate accuracy coefficients are in boldface. Values may not add to 100% because of rounding and missing data.
Results
Accuracy
To test the first hypothesis that American participants would be more accurate than Chinese participants, we examined culture effects when unbiased emotion recognition accuracy was submitted to a 2 (Culture) x 2 (Sex) x 2 (Age Group) x 6 (Target Emotion) mixed-model ANOVA. In support of Hypothesis 1, there was a main effect of Culture with Americans (M = .75, SE = .02) exhibiting greater emotion recognition accuracy than Chinese (M = .52, SE = .02), F(1, 165) = 108.57, p < .001, ηp2 = .40. Consistent with the aging literature (Ruffman, Henry, Livingstone, & Phillips, 2008), there was also a main effect of Age Group, F(1, 165) = 43.57, p < .001, ηp2 = .21, with young adults (M = .71, SE = .02) exhibiting greater accuracy than older adults (M = .56, SE = .02). The main effect for Sex was not significant, F(1, 165) = 1.99, p = .16, ηp2 = .01. The Culture x Age Group interaction was significant, F(1, 165) = 35.12, p < .001, ηp2 = .18. Follow-up ANOVAs separately for each age group revealed that the main effect of Culture was greater for older adults, F(1, 86) = 160.46, p < .001, ηp2 = .65, (MOlderAmericans = .74, SE = .02; MOlderChinese = .39, SE = .02) than young adults, F(1, 79) = 8.47, p < .01, ηp2 = .10 (MYoungAmericans = .76, SE = .03; MYoungChinese = .66, SE = .02).
When the analysis above was repeated using the raw accuracy hit-rates (Table 1), a similar pattern emerged. Consistent with the results from the unbiased hit rates, and in support of Hypothesis 1, Americans (M = .78, SE = .01) exhibited greater emotion recognition accuracy than Chinese (M = .63, SE = .01), F(1, 165) = 128.42, p < .001, ηp2 = .44.
Accuracy by Contextual Face Type and Error Analyses
Next, we investigated whether certain contextual face trial types would differentially benefit accuracy, and whether this pattern differed by culture. Specifically, we were interested in four target-contextual face combinations: surprise target/fear contextual faces, fear target/surprise contextual faces, anger target/disgust contextual faces, and disgust target/anger contextual faces. For each of the four target-contextual face combinations, we examined unbiased emotion recognition accuracy in a 2 (Culture) x 5 (Contextual Face Type) mixed-model ANOVA and followed up significant results with planned paired sample t-tests comparing accuracy on trials with the often-confused contextual emotion to the other four contextual emotions, separately by culture (e.g., for the surprise target/fear contextual face combination we compared surprise recognition accuracy on trials with fear contextual faces to trials with anger, disgust, happy, and sad contextual faces separately for American and Chinese participants). The results of these analyses are presented in Table 2 (as well as the raw accuracy results for comparison with other studies) and the confusion matrix in Table 1 presents the descriptives (using raw accuracy hit rates) for each type of error.
Table 2.
Culture x Contextual Face Type ANOVA Results on Unbiased Emotion Recognition Accuracy
| Target-Contextual Face Combination |
df (effect, error) |
F | p < | ηp2 | Direction of Effect | |
|---|---|---|---|---|---|---|
| Surprise Target/Fear Context | ||||||
| Culture | (1, 171) | 96.88 | .001 | .36 | AM > CH | |
| Contextual Face Type | (3.75, 641.60) | 4.10 | .01 | .02 | Fear < Happy, Sad, Anger | |
| Culture x Context | (3.75, 641.60) | 3.37 | .05 | .02 | AM: Fear < Anger, Sad CH: Fear < Anger, Sad, Disgust, Happy |
|
| Fear Target/Surprise Context | ||||||
| Culture | (1, 166) | 79.95 | .001 | .33 | AM > CH | |
| Contextual Face Type | (3.39, 563.08) | 13.76 | .001 | .08 | Surprise < Disgust; Surprise > Angry, Happy | |
| Culture x Context | (3.39, 563.08) | 3.15 | .05 | .02 | AM: Surprise > Happy CH: Disgust > Surprise |
|
| Anger Target/Disgust Context | ||||||
| Culture | (1, 170) | 38.40 | .001 | .18 | AM > CH | |
| Contextual Face Type | (3.79, 643.83) | 6.97 | .001 | .04 | Disgust > Fear, Happy, Sad | |
| Culture x Context | (3.79, 648.83) | 2.15 | .073 | .01 | AM: Disgust > Happy CH: Disgust > Fear, Happy, Sad |
|
| Disgust Target/Anger Context | ||||||
| Culture | (1, 171) | 141.41 | .001 | .45 | AM > CH | |
| Contextual Face Type | (3.77, 644.14) | 28.19 | .001 | .14 | Anger > Happy; Anger < Fear | |
| Culture x Context | (3.77, 644.14) | 6.64 | .001 | .04 | AM: Anger > Happy, Sad; Anger < Fear CH: Anger > Happy; Anger < Fear, Surprise |
|
| Surprise Target/Fear Context | ||||||
| Culture | (1, 171) | 29.47 | .001 | .15 | AM > CH | |
| Contextual Face Type | (3.69, 631.52) | 3.89 | .01 | .02 | Fear < Happy, Sad, Anger | |
| Culture x Context | (3.69, 631.52) | 2.95 | .05 | .02 | AM: Fear < Anger, Sad CH: Fear < Anger, Sad, Disgust, Happy |
|
| Fear Target/Surprise Context | ||||||
| Culture | (1, 171) | 62.87 | .001 | .27 | AM > CH | |
| Contextual Face Type | (4, 684) | 13.52 | .001 | .07 | Surprise < Disgust; Surprise > Angry, Happy | |
| Culture x Context | (4, 684) | 3.80 | .01 | .02 | AM: Surprise > Happy CH: Surprise = Anger, Disgust, Happy, Sad |
|
| Anger Target/Disgust Context | ||||||
| Culture | (1, 171) | 14.10 | .001 | .08 | AM > CH | |
| Contextual Face Type | (3.75, 641.10) | 7.71 | .001 | .04 | Disgust > Fear, Happy, Sad | |
| Culture x Context | (3.75, 641.10) | 3.27 | .05 | .02 | AM: Disgust > Happy CH: Disgust > Fear, Happy, Sad |
|
| Disgust Target/Anger Context | ||||||
| Culture | (1, 171) | 73.75 | .001 | .30 | AM > CH | |
| Contextual Face Type | (4, 684) | 19.32 | .001 | .10 | Anger > Happy; Anger < Fear | |
| Culture x Context | (4, 684) | 3.56 | .01 | .02 | AM: Anger > Happy, Sad; Anger < Fear CH: Anger < Fear |
|
Notes. AM = American, CH = Chinese. The Greenhouse-Geisser correction was used to correct for violations of sphericity. For the unbiased emotion recognition results, one Chinese participant was excluded from the Anger Target/Disgust Context analysis and 6 were excluded from the Fear Target/Surprise Context analysis due to numerous missing responses in the category.
Surprise target/fear contextual faces
For surprise target/fear contextual faces, Americans were less accurate at recognizing a surprise target face on slides with fear contextual faces compared with angry or sad contextual faces. Chinese participants, however, were less accurate on trials with fear contextual faces than all other contextual face types, suggesting that both cultures were hindered by the presence of fear contextual faces.
Fear target/surprise contextual faces
Americans were less accurate at recognizing a fear target face surrounded by happy contextual faces than a fear target face surrounded by surprise contextual faces and Chinese participants were less accurate at identifying fear target faces surrounded by surprise contextual faces than a fear target surrounded by disgusted contextual faces. This suggests that, consistent with our hypothesis, fear targets surrounded by surprise contextual faces boosted accuracy for Americans, compared to other contextual face type fear trials, but reduced the accuracy of Chinese participants (compared to disgust contextual faces).
Anger target/disgust contextual faces
Americans were less accurate at recognizing an anger target surrounded by happy contextual faces than by disgust faces. Chinese participants were more accurate on trials with disgust contextual faces than fear, happy, or sad contextual faces. Contrary to our hypothesis, participants in both cultures benefited from disgust contextual faces when identifying an anger target face.
Disgust target/anger contextual faces
Americans were less accurate at recognizing a disgust target face surrounded by happy or sad contextual faces than by anger faces, but were more accurate at identifying a disgusted face surrounded by fear faces than anger faces. Chinese participants were more accurate on trials with anger contextual faces than happy contextual faces and more accurate on trials with surprise and fear contextual faces than anger contextual faces. Thus, neither culture seemed to benefit from the presence of the easily confused emotion (anger) as the contextual face.
Eye-Tracking Analyses
For the eye-tracking analyses, we only included those slides where participants’ eyes were successfully tracked 40 percent or more of the time the slide was shown, any less was considered too much loss of data, and these slides were excluded from analysis, on a trial-by-trial basis.4 We also excluded three American and four Chinese participants listwise from the eye-tracking analyses because they were missing more than 70 percent of the slides due to poor tracking, leaving N = 166 (84 American, 82 Chinese). For the remaining participants, there was not a significant difference between the two cultures in the number of slides excluded out of 180, t(164) = 1.33, p = .18, Americans: M = 17.00, SE = 2.80; Chinese: M = 12.06, SE = 2.41. On average when examining all slides for the remaining participants, most slides were tracked for a majority of time the slide was shown (MSlideTracked = 82.76 percent of time shown, SD = 17.21%; see Table 3 for statistics broken down by emotion, culture, and age group).
Table 3.
Percent Time Slide Tracked: Descriptives and ANOVA Results
| American (n = 84) |
Chinese (n = 82) |
ANOVA Results, F(1, 162) | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| YA | OA | YA | OA | Culture Main Effect | Age Group Main Effect | Culture x Age Group | |||||||
| F | p | ηp2 | F | p | ηp2 | F | p | ηp2 | |||||
| Anger | 80.87 (2.57) |
83.71 (2.57) |
87.61 (2.60) |
80.62 (2.60) |
.50 | .48 | .003 | .65 | .42 | .004 | 3.62 | .06 | .022 |
| Disgust | 81.59 (2.58) |
82.50 (2.58) |
88.28 (2.61) |
80.46 (2.61) |
.81 | .37 | .005 | 1.78 | .18 | .011 | 2.83 | .09 | .017 |
| Fear | 81.14 (2.82) |
82.53 (2.82) |
86.58 (2.86) |
80.13 (2.86) |
.29 | .59 | .002 | .79 | .38 | .005 | 1.90 | .17 | .012 |
| Happy | 80.36 (2.70) |
80.91 (2.70) |
87.11 (2.74) |
81.78 (2.74) |
1.97 | .16 | .012 | .77 | .38 | .005 | 1.17 | .28 | .007 |
| Sad | 79.77 (2.74) |
83.13 (2.74) |
87.75 (2.77) |
79.97 (2.77) |
.77 | .38 | .005 | .65 | .42 | .004 | 4.10 | .05 | .025 |
| Surprise | 79.87 (2.73) |
81.89 (2.73) |
86.90 (2.77) |
81.05 (2.77) |
1.26 | .26 | .008 | .49 | .49 | .003 | 2.05 | .15 | .012 |
Note. YA = young adults (18–30 years), OA = older adults (60–86 years). Descriptives are means of percent time slide tracked with standard errors of the mean in parentheses.
A fixation was defined as an interval when gaze was focused for 100 ms or more within 1° visual angle (Manor & Gordon, 2003). To examine relative fixation duration to the target and contextual face AOIs (see Figure 2), we computed percent fixation duration to the target AOI out of percent duration fixation to all AOIs (percent duration fixation to target plus percent fixation duration to contextual faces) and percent fixation duration to the contextual faces AOIs out of percent duration fixation to all AOIs. The relative fixation duration to targets was greater for Americans (M = 57.20, SE = 1.65) than Chinese (M = 54.89, SE = 1.58). Of course, this also means that in terms of relative fixation duration to the contextual faces, Chinese participants (M = 42.67, SE = 1.46) spent greater relative fixation duration than Americans (M = 41.61, SE = 1.58). To address the dependency between target and contextual face AOIs, we then computed difference scores (Target Preference Score = relative % fixation to target – relative % fixation to contextual faces) reflecting the degree of fixation preference for target over contextual faces. Seven participants (2 Chinese and 5 Americans) were excluded because they had zero fixations in any of the AOIs for one or more emotion categories, leaving N = 159 (79 Americans and 80 Chinese).
Figure 2.
Anger target (in center), disgust contextual face slide. AOIs are the areas inside the boxes marked with white dashes (shown here for presentation purposes; dashed white lines were not present when participants viewed the stimuli). Each of the five photos of faces was an AOI, with one target AOI and four contextual face AOIs for each slide.
To check for age and sex differences, we first submitted the target preference scores to a mixed-model ANOVA with Culture, Age Group, and Sex as between-subject factors and Target Emotion as the within-subject factor. Because there were no significant main effects or interactions with Age Group or Sex, we excluded these variables from further analyses. To determine whether American participants fixated on the target faces for a greater relative duration than Chinese (Hypothesis 3), the target preference scores were submitted to a mixed-model ANOVA with Culture as a between-subject factor and Target Emotion as a within-subject factor. A main effect of Culture emerged, F(1, 157) = 4.23, p = .04, ηp2 = .03, with Americans (M = 19.72, SE = 2.49) exhibiting a greater target preference score than Chinese (M = 12.52, SE = 2.47). This main effect was qualified by a Culture x Target Emotion interaction, F(4.40, 690.04) = 2.48, p = .03, ηp2 = .02. Follow-up simple effects analysis revealed significant Culture differences in target fixation preferences for anger, t(159) = 2.33, p = .02, d = .37, happiness, t(157)5 = 2.14, p = .03, d = .34, and surprise, t(158) = 2.20, p = .03, d = .35, with Americans (Manger = 20.62, SE = 2.95; Mhappy = 19.12, SE = 2.91, Msurprise = 19.30, SE = 2.91) exhibiting a greater target preference score than Chinese participants (Manger = 10.28, SE = 3.31; Mhappy = 11.22, SE = 2.30; Msurprise = 9.98, SE = 3.08; see Figure 3). For the other emotions, although Americans had a greater target preference score than Chinese, the difference was not significant.
Figure 3.
Culture differences in target preference scores (average percentage relative fixation duration to target out of all AOIs – average percentage relative fixation duration to contextual faces out of all AOIs). Bars represent standard errors of the mean. *Difference between cultures is significant, p < .05.
To examine whether these cultural differences were driven by the slides where the target face did not appear in the center, we re-ran the analyses above without the 60 slides where the target was not in the center. Both the main effect of Culture, F(1, 156) = 5.55, p = .02, ηp2 = .03, and the Culture x Target Emotion interaction remained significant, F(4.37, 681.42) = 3.81, p = .003, ηp2 = .02. Thus, Hypothesis 3, which predicted that American participants would have greater relative fixation duration to the target faces than Chinese participants, was partially supported (for anger, happiness, and surprise).
We were also interested in whether American and Chinese participants differed in their gaze patterns. If Americans were more likely than Chinese to treat the target face as independent from the contextual faces in the display, they might be more likely to use a contrasting viewing strategy by alternating between the target face and each contextual face (see Figure 4A for an example). We expected Chinese participants might be less likely to use a contrasting viewing strategy by progressively viewing each face without referring back to the target face as often as Americans (see Figure 4B for an example). To examine gaze patterns, we computed the number of transitions between fixations 1) to target and contextual faces (target transitions), and 2) to different contextual faces (contextual transitions). A transition was defined as a fixation in a new AOI. That is, we examined transitions from one of the five AOIs (target and four contextual face AOIs) to a different AOI only. Then we classified any transition that included the target face, a target transition. All other transitions were classified as contextual transitions. On average, participants made 3.48 transitions per slide between the five AOIs. Americans made more target transitions (M = 3.28, SE = .18) and contextual transitions (M = 3.51, SE = .28) per slide than Chinese participants (target transitions: M = 2.22, SE = .18; contextual transitions: M = 2.56, SE = .28). Because Americans (M= 6.68, SE = .59) made more total transitions (target transitions + contextual transitions) than Chinese (M = 4.71, SE = .21), t(102.04) = 3.13, p = .002, d = .49, we computed percent target transitions of total transitions (target transitions / total transitions) so we could compare the two cultures. A Culture x Sex x Age Group x Target Emotion mixed-model ANOVA examining percent target transitions found a main effect of Culture, F(1, 151)6 = 8.14, p = .005, ηp2 = .05. Consistent with the hypothesis that Americans use more of a contrasting strategy than Chinese, Americans (M = .51, SE = .01) had a greater percentage of target transitions than Chinese (M = .49, SE = .01). There was also a main effect of Age Group, F(1, 151) = 21.52, p < .001, ηp2 = .13, with young adults (M = .52, SE = .01) making more target transitions than older adults (M = .48, SE = .01). These main effects were qualified by two significant interactions: a Culture x Age Group interaction, F(1, 151) = 15.88, p < .001, ηp2 = .10, and a Culture x Target Emotion interaction, F(4.24, 640.16) = 17.63, p < .001, ηp2 = .11. Follow-up simple effects analyses revealed that the age differences in percent target transitions were only significant for the Chinese, F(1, 75) = 38.61, p < .001, ηp2 = .34. Follow-up analyses for the Target Emotion x Culture interaction revealed that the cultural differences were only significant for fear, happy, sad, and surprise (see Figure 5).
Figure 4.
Examples of hypothesized (A) American and (B) Chinese gaze patterns. For both slide examples, the target picture is on the left and was indicated to participants by a red outline surrounding the target face (shown with a dotted white line here). The patterns of circles with connecting lines represent our hypothesized gaze patterns among the target and contextual faces.
Figure 5.
Culture differences in % target transitions (target transitions / total transitions). Bars represent standard errors of the mean. *Difference between cultures is significant, p < .05.
We re-ran the above Culture x Sex x Age Group x Target Emotion mixed-model ANOVA examining percent target transitions without the 60 non-center target slides. There remained a significant interaction of Target Emotion with Culture (it also interacted with age group), Culture x Target Emotion x Age Group interaction, F(4.19, 552.52) = 3.65, p = .005, ηp2 = .03. These results partially support Hypothesis 4 predicting differential gaze patterns for the Americans and the Chinese: for four of the six emotions, Americans made more target transitions relative to total transitions than Chinese.
We next examined the Pearson correlation coefficients between accuracy and our eye tracking variables (Table 4). As expected (Hypothesis 5), accuracy is positively related to percent target transitions for Happy (r = .26, p = .001), Sad (r = .33, p < .001), and Surprise (r = .33, p < .001), suggesting that the contrasting strategy is related to greater accuracy of the target emotion for some emotions.
Table 4.
Pearson’s Correlation Coefficients for Recognition Accuracy and % Target Transitions for the Six Emotions (N = 166)
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | ||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | Anger Unbiased Hit Rate | 1.00 | |||||||||||
| 2 | Disgust Unbiased Hit Rate | 0.73 | 1.00 | ||||||||||
| 3 | Fear Unbiased Hit Rate | 0.51 | 0.59 | 1.00 | |||||||||
| 4 | Happy Unbiased Hit Rate | 0.45 | 0.44 | 0.30 | 1.00 | ||||||||
| 5 | Sad Unbiased Hit Rate | 0.71 | 0.64 | 0.43 | 0.48 | 1.00 | |||||||
| 6 | Surprise Unbiased Hit Rate | 0.58 | 0.68 | 0.71 | 0.58 | 0.58 | 1.00 | ||||||
| 7 | Anger Percent Target Transitions | 0.11 | 0.03 | −0.11 | 0.16 | 0.16 | −0.02 | 1.00 | |||||
| 8 | Disgust Percent Target Transitions | 0.16 | 0.15 | 0.03 | 0.19 | 0.17 | 0.08 | 0.42 | 1.00 | ||||
| 9 | Fear Percent Target Transitions | 0.31 | 0.34 | 0.08 | 0.28 | 0.34 | 0.20 | 0.52 | 0.58 | 1.00 | |||
| 10 | Happy Percent Target Transitions | 0.23 | 0.26 | 0.06 | 0.26 | 0.23 | 0.20 | 0.37 | 0.40 | 0.50 | 1.00 | ||
| 11 | Sad Percent Target Transitions | 0.34 | 0.38 | 0.19 | 0.33 | 0.33 | 0.31 | 0.38 | 0.56 | 0.61 | 0.58 | 1.00 | |
| 12 | Surprise Percent Target Transitions | 0.30 | 0.36 | 0.19 | 0.38 | 0.25 | 0.33 | 0.31 | 0.46 | 0.59 | 0.69 | 0.77 | 1.00 |
Note. Correlations in bold are significant at p < .05. Shaded boxes represent accuracy correlated with percent target transitions for each emotion.
Discussion
This study found that Americans and Chinese differ in 1) how much they attend to contextual emotional expressions when judging emotions, and 2) how they attend to the contextual information. Both cultures’ emotion judgments were influenced by the contextual facial expressions, but the way the contextual information was processed during the task differed by culture as well as by emotion. Overall, in this task, the contrasting strategy that the Americans were more likely to exhibit was related to greater accuracy. The task was set-up to favor an emphasis on differentiating the target face from the contextual faces (i.e., contrasting), rather than integrating the contextual faces with the target face. The degree to which a contrasting vs. integrating strategy would be successful in social situations remains an open question.
Emotion Recognition Accuracy
Consistent with the idea that members of collectivist cultures attend to contextual elements when making emotion judgments (e.g., Masuda et al., 2008), Chinese participants were less accurate than Americans at identifying emotions surrounded by different contextual expressions. However, we also found that American participants’ accuracy was influenced by the contextual expressions. The presence of the often-confused surprise expression as a contextual face improved Americans’ fear recognition accuracy compared to other contextual faces combined with fear targets. Thus, in contrast with the Masuda findings (2008), American participants’ emotion judgments were also influenced by context. For American participants, coming from a more individualist culture, dissimilar expressions may have served to disambiguate some expressions (e.g., fear) from others (e.g., surprise). Although this result is surprising given the Masuda and colleagues’ (2008) study, it does fit with work which suggests that Americans do use context in emotion perception (Aviezer et al., 2008; Barrett & Kensinger, 2010). These findings suggest that everyone’s emotion perception can be influenced by contextual factors under certain circumstances, but how context influences emotion judgments varies by culture.
The findings of our study may differ from the Masuda (2008) study for several reasons: 1) we looked at emotion recognition for six emotions rather than three (angry, happy, and sad recognition), 2) we used accuracy of emotion identification rather than intensity ratings, 3) we used pictures of real faces rather than cartoons, 4) we compared Chinese to Americans rather than Japanese, and 5) we used participants from across the lifespan rather than college students. We designed the study so that the most frequently confused facial expression for the target expression was one of the response options and served as a contextual face. This situation may have exploited the benefits of using a contrasting viewing strategy (as Americans do) for recognizing the target face because an exemplar of the most often mistaken face was available and thus able to be eliminated as a possibility. It may also be that our participants were more motivated than a typical college-only sample to perform the task well.
Eye Tracking Analyses
The eye-tracking data are consistent with the idea that Americans may have been more likely than Chinese to contrast contextual faces with the target face in order to disambiguate between easily confused emotions, which may be a relatively quick analytic process; whereas Chinese participants spent less time contrasting the target face with the contextual faces, perhaps to integrate all the faces in the display together to interpret the expression of the target face, a process that may take longer than only contrasting. On average, Chinese participants spent more time on each slide than Americans, which could be because these two differently-paced approaches were being differentially favored. By examining the gaze patterns of American and Chinese participants, we determined that American participants made more transitions to the target face, exhibiting more of a contrasting strategy, moving back and forth between the target face and a contextual face, for some emotions. The gaze patterns of Chinese participants had fewer target transitions, indicating a pattern where each contextual face was fixated without always fixating on the target face in between. It should be noted, however, that for both cultures, nearly half of the transitions they made to AOIs were target transitions. Both cultures were clearly contrasting the target face with the contextual faces, but the Americans exhibited this pattern more than the Chinese.
Differential Emotion Effects
The extent to which there were cultural differences in attention to context varied by emotion in both the accuracy and eye tracking data. We expected culture differences in the influence of context on emotion judgments would be greatest for targets presented with frequent “false-alarm” contextual faces (e.g., fear/surprise and anger/disgust) because we thought the presence of the common mistake would confer a benefit for Americans who might be more likely to contrast the contextual face with the target face. Only the fear target/surprise contextual faces combination showed culture differences with Americans, but not Chinese, exhibiting a benefit in fear recognition when presented with surprise contextual faces compared to other contextual faces. The converse of this, the surprise target/fear combination, showed worse accuracy than other surprise/contextual face combinations for both cultures. Perhaps the differences in the arousal level of fear and surprise changed the influence of four contextual faces on one target face in these two combinations. Having more of the higher arousal level fear faces (Mandal, Bryden, & Bulman-Fleming, 1996) may have overwhelmed the interpretation of the less-arousing surprise target face but the reverse may not have been so overwhelming, so that the American group was able to benefit from contrasting the contextual surprise faces with the more-arousing fear face. In other words, perhaps the more-arousing fear target face was strong enough to withstand comparison without becoming washed out.
Like fear and surprise, anger and disgust are perceptually similar and often-confused with each other (Isaacowitz et al., 2007; Susskind et al., 2007; Wang, K. et al., 2006), however the anger/disgust combinations were sometimes beneficial to both Americans and Chinese, suggesting that on these trials the two cultures may have both benefited from contrasting the contextual faces with the target face. Consistent with these findings in accuracy, anger and disgust were the only two emotions that did not exhibit culture differences in how many target transitions were made. For fear, happy, sad, and surprise, Americans made more percent target transitions than Chinese. Furthermore, this strategy seemed effective, at least for happy, sad, and surprise, as suggested by positive correlations between percent target transitions and accuracy for these emotions. Why might anger and disgust be different from fear, happy, sad, and surprise? Although speculative, multidimensional scaling results suggest that participants do rate fear, happy, sad, and surprise as more similar than anger and disgust, which are set somewhat further apart from the other four (e.g., see figure 7a of Susskind et al., 2007). Taken together with findings that show how an “anchor” can influence the interpretation of a target in the opposite direction (seeing a neutral face after a sad face makes the neutral face appear more happy; seeing the same neutral face after a happy face makes the face seem more sad; Russell & Fehr, 1987), it may be that anger and disgust have more of these “opposites” in this task (four: fear, happy, sad, and surprise) to propel them across the multidimensional perceptual similarity space to the intended label for the target. Thus, because anger and disgust were more likely to have a contextual face that was opposite, accuracy on anger and disgust trials was less related to a contrasting strategy than the other emotions because it was narrowed down by having fewer options with perceptual similarity to the target (or dissimilarity to the contextual faces). Compared to the other four emotions, which only have two opposites (anger and disgust) which would propel them toward the four possible targets when presented as contextual faces, the contrasting strategy may be relevant to both cultures for anger and disgust trials because of the dissimilarity of the contextual faces to the target. This can be seen in the data in that both cultures exhibited a similar level of percent target transitions on anger and disgust trials as Americans did on other trials. It would be interesting to replicate this study with a different number of possible response labels to see how it influences cultural differences in attention to context.
Limitations and Future Directions
The results of this study are tempered by several limitations. First, because we wanted to use actual face stimuli, the displays of faces were less cohesive than a group picture. Nevertheless, we still found cultural differences in the influence of contextual faces on accuracy and attentional patterns. It is also possible that the participants of each culture were interpreting the instructions of the task differently. While we cannot rule out this possibility, the cultural differences in fixation duration and gaze patterns are consistent with the notion that the two cultures were using context in different ways to make their judgments. Another possible limitation is that we did not include intensity ratings or confidence judgments (Beaupré & Hess, 2006), which may have been even more sensitive to cultural differences in interpretation. In addition, the faces in the Chinese stimuli were photographed in Beijing, but the Chinese participants in the study were from Hong Kong. There are cultural differences between these groups, including that they speak different languages. Future work should try to more closely match the origin of the stimuli with the culture of the participants.
Conclusions
Although there seem to be many similarities across cultures in typical emotion recognition tasks devoid of context (Ekman, 1992, 1994; Ekman & Friesen, 1971; Ekman et al., 1987), situational factors may increase cross-cultural differences in emotion judgments. With the increasing globalization of our society and workplace, it is important to understand how cross-cultural differences in worldview might contribute to differences in understanding others’ emotions (Elfenbein & Ambady, 2002).
Although numerous individual studies have shown that 1) emotion judgments of Americans are influenced by contextual factors (Aviezer et al., 2008; Carrera-Levillain & Fernandez-Dols, 1994; Carroll & Russell, 1996; Fernandez-Dols, Sierra, & Ruiz-Belda, 1993; Goodenough & Tinker, 1931), and 2) the visual attention patterns of East Asians are more context-sensitive than North Americans (Masuda & Nisbett, 2001; Nisbett et al., 2001), this is the first study we know of to find pronounced but differential context effects on emotion recognition accuracy for Americans and East Asians within the same study. In the present study, we found that both Chinese and Americans were influenced by the contextual faces, but in different ways. This study is therefore the first to highlight the different ways in which contextual information is used by Chinese and American participants when making emotion attributions. Such findings are a warning not to think too narrowly about cultural differences in attention to context. Context always matters; but how it is used seems to differ cross-culturally and based on the perceptual similarity of the emotional context to the target.
Acknowledgments
Funding
This research was supported by the National Institute on Aging [grant numbers T32 AG-00204, R01AG-026323].
Footnotes
Portions of this paper were presented at the annual meetings of the Gerontological Society of America, Atlanta, GA, 2009 and the Society for Personality and Social Psychology, Las Vegas, NV, 2010. We are grateful to Hanna Negami and Amy Stricoff for their assistance with data collection.
Derek M. Isaacowitz is now at Department of Psychology, Northeastern University.
For the purposes of the present study, we defined accuracy to “refer to ‘correct’ information of some nature being obtained, by some means, from facial behavior” (Ekman, Friesen, & Ellsworth, 1972). The instructions in our task were to identify the emotion expressed by the target face. Ekman and O’Sullivan (Ekman & O'Sullivan, 1988) suggest that “judgment accuracy can also be evaluated with posed expressions if one wants to know whether observers can tell what emotion the poser intended to show” p. 87. It is in this spirit that we use “accuracy”.
We focused on the incongruent trials because there were not enough congruent trials for each emotion to reliably compare with the incongruent trials (6 congruent trials vs. 30 incongruent trials for each emotion). The pattern of results remained the same when we included the congruent trials in the analysis. For unbiased emotion recognition accuracy, the main effect of Culture remained significant when congruent trials were included in the analysis, F(1, 165) = 127.23, p < .001, ηp2 = .44. When the analysis for target preference scores was re-run with the congruent slides included, a significant Culture x Target Emotion x Congruency interaction emerged, F(4.75, 745.41) = 2.73, p = .02, ηp2 = .02. Follow-up simple effects analysis revealed significant Culture differences in target fixation preferences for incongruent anger, congruent sad, and marginally for congruent fear and incongruent surprise, all with Americans exhibiting greater target preference scores than Chinese.
A Culture x Sex x Age Group x Target Emotion x Congruency mixed-model ANOVA examining percent target transitions found a main effect of Culture, F(1, 106) = 8.57, p = .004, ηp2 = .08. Consistent with the hypothesis that Americans use more of a contrasting strategy than Chinese, Americans (M = .52, SE = .01) had a greater percentage of target transitions than Chinese (M = .48, SE = .01). This main effect was qualified by a significant Congruency x Culture x Target Emotion interaction, F(4.50, 477.37) = 2.60, = .03, ηp2 = .02. Follow-up simple analyses revealed that the cultural differences were only significant for congruent trials of anger (Chinese > Americans), sad (Americans > Chinese), surprise (Americans > Chinese), and incongruent trials of fear, happy, sad, and surprise (Americans > Chinese). In addition to positive correlations for happy, sad, and surprise incongruent trials with their respective percent target transitions, there was also a positive correlation between congruent sad unbiased emotion recognition accuracy and congruent sad percent target transitions, r = .21, p < .05.
Whenever appropriate, statistics for equal variances not assumed are reported and the Greenhouse-Geisser correction was used for violations of sphericity.
We chose the rather lenient criterion of including slides in which eye-tracking data was collected for 40 percent or more of the time the slide was shown for several reasons. First, participants did have to move their gaze away from the monitor in order to look at the card of emotion label choices attached to the right side of the monitor. Thus, any time the participant was glancing at the emotion label choices, their eyes were not being tracked. Participants also sometimes looked away from the monitor to ask the experimenter a question, although they were instructed to keep their heads as still as possible. Second, given that all of our eye-tracking hypotheses relied on relative indices, we chose to maximize our data despite differential missing data across slides.
Degrees of freedom for t-tests do not match because additional participants were excluded listwise from analysis for emotion categories that showed no fixation in any AOI for the entire emotion category.
Two Americans and 3 Chinese participants did not make any transitions on one of the emotions and were excluded from this analysis.
References
- Aviezer H, Hassin RR, Ryan J, Grady CL, Susskind J, Anderson AK, Bentin S. Angry, disgusted, or afraid? Studies on the malleability of emotion perception. Psychological Science. 2008;19:724–432. doi: 10.1111/j.1467-9280.2008.02148.x. [DOI] [PubMed] [Google Scholar]
- Barrett LF, Kensinger EA. Context is routinely encoded during emotion perception. Psychological Science. 2010;21:595–599. doi: 10.1177/0956797610363547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beaupré MG, Hess U. An ingroup advantage for confidence in emotion recognition judgments: The moderating effect of familiarity with the expressions of outgroup members. Personality and Social Psychology Bulletin. 2006;32:16–26. doi: 10.1177/0146167205277097. [DOI] [PubMed] [Google Scholar]
- Carrera-Levillain P, Fernandez-Dols J-M. Neutral faces in context: Their emotional meaning and their function. Journal of Nonverbal Behavior. 1994;18:281–299. [Google Scholar]
- Carroll JM, Russell JA. Do facial expressions signal specific emotions? Judging emotion from the face in context. Journal of Personality and Social Psychology. 1996;70:205–218. doi: 10.1037//0022-3514.70.2.205. [DOI] [PubMed] [Google Scholar]
- Ekman P. An argument for basic emotions. Cognition & Emotion. 1992;6:169–200. [Google Scholar]
- Ekman P. Strong evidence for universals in facial expressions: A reply to Russell’s mistaken critique. Psychological Bulletin. 1994;115:268–287. doi: 10.1037/0033-2909.115.2.268. [DOI] [PubMed] [Google Scholar]
- Ekman P, Friesen WV. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology. 1971;17:124–129. doi: 10.1037/h0030377. [DOI] [PubMed] [Google Scholar]
- Ekman P, Friesen WV. Pictures of Facial Affect [Photos] Palo Alto, CA: Consulting Psychologists Press; 1976. [Google Scholar]
- Ekman P, Friesen WV, Ellsworth PC. Emotion in the Human Face: Guidelines for Research and an Integration of Findings. Oxford, England: Pergamon Press; 1972. [Google Scholar]
- Ekman P, Friesen WV, O’Sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, Tzavaras A. Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology. 1987;53:712–717. doi: 10.1037//0022-3514.53.4.712. [DOI] [PubMed] [Google Scholar]
- Ekman P, O’Sullivan M. The role of context in interpreting facial expression: Comment on Russell and Fehr (1987) Journal of Experimental Psychology: General. 1988;117:86–88. doi: 10.1037//0096-3445.117.1.86. [DOI] [PubMed] [Google Scholar]
- Elfenbein HA, Ambady N. Predicting workplace outcomes from the ability to eavesdrop on feelings. Journal of Applied Psychology. 2002;87:963–971. doi: 10.1037/0021-9010.87.5.963. [DOI] [PubMed] [Google Scholar]
- Fernandez-Dols J-M, Sierra B, Ruiz-Belda MA. On the clarity of expressive and contextual information in the recognition of emotions: A methodological critique. European Journal of Social Psychology. 1993;23:195–202. [Google Scholar]
- Goodenough FL, Tinker MA. The relative potency of facial expression and verbal description of stimulus in the judgment of emotion. Journal of Comparative Psychology. 1931;12:365–370. [Google Scholar]
- Isaacowitz DM, Loeckenhoff C, Lane R, Wright R, Sechrest L, Riedel R, Costa PT. Age differences in recognition of emotion in lexical stimuli and facial expressions. Psychology and Aging. 2007;22:147–259. doi: 10.1037/0882-7974.22.1.147. [DOI] [PubMed] [Google Scholar]
- Jack RE, Blais C, Scheepers C, Schyns PG, Caldara R. Cultural confusions show that facial expressions are not universal. Current Biology. 2009;19:1543–1548. doi: 10.1016/j.cub.2009.07.051. [DOI] [PubMed] [Google Scholar]
- Mandal MK, Bryden MP, Bulman-Fleming MB. Similarities and variations in facial expressions of emotions: Cross-cultural evidence. International Journal of Psychology. 1996;31:49–59. [Google Scholar]
- Manor BR, Gordon E. Defining the temporal threshold for ocular fixation in free-viewing visuocognitive tasks. Journal of Neuroscience Methods. 2003;128:85–93. doi: 10.1016/s0165-0270(03)00151-1. [DOI] [PubMed] [Google Scholar]
- Markus HR, Kitayama S. Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review. 1991;98:224–253. [Google Scholar]
- Masuda T, Ellsworth PC, Mesquita B, Leu J, Tanida S, Van de Veerdonk E. Placing the face in context: Cultural differences in the perception of facial emotion. Journal of Personality and Social Psychology. 2008;94:365–381. doi: 10.1037/0022-3514.94.3.365. [DOI] [PubMed] [Google Scholar]
- Masuda T, Nisbett RE. Attending holistically versus analytically: Comparing the context sensitivity of Japanese and Americans. Journal of Personality and Social Psychology. 2001;81:922–934. doi: 10.1037//0022-3514.81.5.922. [DOI] [PubMed] [Google Scholar]
- Matsumoto D, Ekman P. Japanese and Caucasian Facial Expressions of Emotion (JACFEE) [Photos] San Francisco, CA: Intercultural and Emotion Research Laboratory, Department of Psychology, San Francisco State University; 1988. [Google Scholar]
- Nisbett RE, Peng K, Choi I, Norenzayan A. Culture and systems of thought: Holistic versus analytic cognition. Psychological Review. 2001;108:291–310. doi: 10.1037/0033-295x.108.2.291. [DOI] [PubMed] [Google Scholar]
- Ruffman T, Henry JD, Livingstone V, Phillips LH. A meta-analytic review of emotion recognition and aging: Implications for neuropsychological models of aging. Neuroscience & Biobehavioral Reviews. 2008;32:863–881. doi: 10.1016/j.neubiorev.2008.01.001. [DOI] [PubMed] [Google Scholar]
- Russell JA, Fehr B. Relativity in the perception of emotion in facial expressions. Journal of Experimental Psychology: General. 1987;166:223–237. [Google Scholar]
- Susskind JM, Littlewort G, Bartlett MS, Movellan J, Anderson AK. Human and computer recognition of facial expressions of emotion. Neuropsychologia. 2007;45:152–162. doi: 10.1016/j.neuropsychologia.2006.05.001. [DOI] [PubMed] [Google Scholar]
- Varnum MEW, Grossmann I, Kitayama S, Nisbett RE. The origin of cultural differences in cognition: The Social Orientation Hypothesis. Current Directions in Psychological Science. 2010;19:9–13. doi: 10.1177/0963721409359301. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wagner HL. On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior. 1993;17:3–28. [Google Scholar]
- Wang K, Hoosain R, Lee TMC, Meng Y, Fu J, Yang R. Perception of six basic emotional facial expressions by the Chinese. Journal of Cross-Cultural Psychology. 2006;37:623–629. [Google Scholar]
- Wang L, Markham R. The development of a series of photographs of Chinese facial expressions of emotion. Journal of Cross-Cultural Psychology. 1999;30:397–410. [Google Scholar]