Eye Movements When Looking at Unusual/Weird Scenes: Are There Cultural Differences?

Abstract

Recent studies have suggested that eye movement patterns while viewing scenes differ for people from different cultural backgrounds and that these differences in how scenes are viewed are due to differences in the prioritization of information (background or foreground). The current study examined whether there are cultural differences in how quickly eye movements are drawn to highly unusual aspects of a scene. American and Chinese viewers examined photographic scenes while performing a preference rating task. For each scene, participants were presented with either a normal or an unusual/weird version. Even though there were differences between the normal and weird versions of the scenes, there was no evidence of any cultural differences while viewing either scene type. The present study, along with other recent reports, raises doubts about the notion that cultural differences can influence oculomotor control in scene perception.

Some recent research has suggested that Asian participants look at scenes differently from the way American participants do. Specifically, Chua, Boland, and Nisbett (2005) reported that Chinese viewers spent less time looking at the focal objects in a scene and more time looking at the background of the scene than did their American counterparts. These results were discussed in the wider context of a general theory of cultural differences in cognition (Nisbett, 2003) whereby Asian cultures lead people to not place as much value on the individual as on the group, while American culture places more emphasis on the individual. According to Chua et al.’s reasoning, this underlying cultural difference in thinking led the Chinese viewers to look more at the background and spend relatively less time (in comparison to the Americans) looking at the focal objects.

However, two recent reports have at least raised some questions about the validity of Chua et al.’s (2005) findings. Rayner, Li, Williams, Cave, and Well (2007) reported no differences in the viewing patterns for Chinese and American participants, with both groups looking more at focal objects than at the background information. Boland, Chua, and Nisbett (2008) noted that the Rayner et al. study was not a direct replication of Chua et al., given that the same materials were not used (i.e., the focal objects were more apparent in the Chua et al. study) and the task varied between the two studies (i.e., the expectation of a memory test in Rayner et al. vs. a rating task for scene likeability in Chua et al.). However, Evans, Rotello, Li, and Rayner (in press) used the original scenes used by Chua et al. (as well as additional scenes for increased power) and the same task and also found no differences between the two groups (both with the entire set of stimuli and with the subset that had been previously used by Chua et al.).

In the present study, rather than dividing scenes into focal and background regions and examining eye movements, we used a different type of manipulation. Specifically, we asked Chinese and American participants to look at scenes that had a rather unusual or weird component to them. We reasoned that if cultural differences can influence where viewers look in a scene (and how quickly they look there), then there could be differences in how quickly Chinese viewers look at the unusual/weird parts of the scene. That is, if they truly are intent on looking more at the background information, it is likely that the unusual/weird object would not be as apparent to them as quickly as it would be for the American viewers. On the other hand, Masuda and Nisbett (2006) found that in a change blindness flicker paradigm, configural changes (e.g. the spacing between two objects) were noticed sooner by Japanese viewers than by Americans. Thus, it could also be argued that because many of the weird regions in the present study were configurally odd, the Chinese viewers might look to the unusual/weird part of the scene sooner due to a better ability to detect these configural differences. Either of these two positions would lead to differences between the Chinese and American viewers in terms of how quickly they looked at the weird part of the scene. Alternatively, and consistent with Rayner et al. (2007) and Evans et al. (in press), there might not be differences between the two cultural groups (suggesting that culture doesn’t strongly influence oculomotor control in scene perception).

Although we used a manipulation in which we examined how quickly viewers from two different cultures looked at highly unusual aspects of a scene, it is also the case that how quickly viewers (apart from any cultural differences) do this is actually somewhat contentious. That is, some experiments indicate that the eyes are quickly drawn to unusual or emotional aspects of a scene, while others suggest that they are not. Loftus and Mackworth (1978) embedded a tractor in an underwater scene and an octopus in a farm scene. They found that viewers moved their eyes much more quickly to the octopus in the farm scene than to the same octopus in an underwater scene. However, Rayner and Pollatsek (1992) pointed out that the eyes may be drawn to objects that are out of place because they are physically distinctive in the scene. Indeed, DeGraef (1998), DeGraef, Christiaens, and d’Ydewalle (1990), and Henderson, Weeks, and Hollingworth (1999) controlled for visual distinctiveness, and none of them replicated the finding that semantically inconsistent objects were fixated earlier than were consistent objects (see also Brockmole & Henderson, 2008).

Becker, Pashler, and Lubin (2007) and Harris, Kaplan, and Pashler (2008) also recently examined the extent to which odd (or semantically out-of-place) objects in scenes draw attention and eye movements. Unlike many earlier experiments, which used line drawings of scenes, both studies used color photographs as stimuli. In Becker et al.’s study, scenes were presented either normally or with a change that yielded anomalous objects. For example, a person’s hand was changed from flesh color to green, and a stop sign was green instead of its normal red color. The primary finding was that viewers fixated the anomalous objects earlier (both in time and in order of eye fixations) than the normal objects. Becker et al. argued that the results indicate that violations of canonical form can be detected from extrafoveal vision and can affect the likelihood of fixating them. In Harris et al.’s study, viewers saw either normal scenes or scenes in which a normal object in the scene was replaced by an odd/emotional object. Like Becker et al., Harris et al. found that viewers looked much earlier at the emotional aspect of the scene; that is, in a beach scene they looked earlier at a baby flying through the air than at a beach ball flying through the air. Other recent studies (Calvo & Lang, 2005; Calvo & Nummenmaa, 2007; Nummenmaa, Hyönä, & Calvo, 2006) have likewise reported that the eyes move more quickly to emotional objects/scenes (though in these studies, the object/scene was usually presented in parafoveal vision and the latency of a saccade from a central fixation point was measured for an emotional object/scene vs. a neutral object/scene).

In the research reported by both Becker et al. (2007) and Harris et al. (2008), a research strategy was used wherein a small number of scenes were presented to a large number of viewers. The rationale was that if viewers continually viewed odd scenes then they might adopt an unusual viewing strategy. While this concern obviously has some validity, it is important (given prior results) to replicate these findings with a larger stimulus set. We obtained the scenes used by Harris et al.¹ and collected a number of other unusual scenes and presented them to viewers, who were asked to rate how much they liked each scene. Thus, in addition to the main goal of examining cultural differences in the extent to which the eyes are drawn to unusual or weird aspects of a scene, our secondary goal was to replicate Harris et al.’s results (as well as in principle Becker et al.’s results) with a wider range of stimuli.

Method

Participants

Twenty-four members of the University of Massachusetts community participated in the experiment. They were compensated with either credit toward an introductory psychology course or $7. Twelve participants were native English speakers born in the United States (we refer to these participants as the American group below), and 12 were native Chinese speakers who grew up in China (the Chinese group).²

Stimuli

Forty full-color photographs—20 sets of scenes, each set with two versions: a normal scene and an anomalous scene in which one object (e.g., a baby³) replaced another (e.g., a beach ball)—were used as stimuli. Of these scenes, three were from the Harris et al. (2008) study. In addition, we collected 17 other weird pictures from the Internet and then, using Adobe Photoshop, we prepared a control version of each scene in which the unusual object was replaced with the background elements or with another suitable object. The photographs were viewed by three independent observers to confirm that the modifications were not easily detectable. The scenes typically had a focal object or a central group of objects. For the weird condition, the weird aspect of the scene was a part of a single object or group that was incompatible with the rest of the scene. Figure 1 shows three examples of scenes from our collection that were edited from weird to normal. In the first example, the weird condition arises from the empty boot by the soldier’s leg, which makes it look as though, upon first glance, he is crouching with an extra foot extended. In the second example, the extended arm of two players is at first interpreted as the unnaturally long arm of one player. In the third example, two dogs seem to be pondering a move on the game board. There are two things to note from these examples: The weird objects were not weird in the sense that they were contrary to the scene context, but rather the configuration of the objects allowed for a strange interpretation. Of the 20 weird scenes used, 9 yielded strange misinterpretations (like Examples 1 and 2 in Figure 1), and 11 were funny configurations that are somewhat unexpected (like Example 3 in Figure 1 and the scenes taken from Harris et al., 2008). The focus of this study was not on focal versus background elements of the scenes but rather on the fixation of its components, depending on whether they were out of place or strangely configured.

Figure 1.

Three sample stimuli are depicted, and for each, the normal version is presented on the left and the weird version is presented on the right.

Apparatus

The eye movements of each participant were tracked with an Eyelink 1000 eye tracker (SR Research, Osgoode Ontario, Canada). The scenes (which subtended a visual angle of 35° horizontally and 28° vertically) were displayed at a resolution of 800 × 600 pixels on a 19-in. (48.26 cm) monitor with a refresh rate of 100 Hz.

Procedure

Participants were seated 60 cm from the monitor, and a chinrest restricted head movements. They were instructed to view each scene and then rate how much they liked it (as per the Chua et al., 2005, study) on a scale from 1 (don’t like at all) to 7 (like very much). After the instructions were given to the participant, the eye tracker was calibrated. Calibration was assessed before each trial, and the participant was recalibrated if the tracking error was greater than 0.4°. At the beginning of each trial, a fixation marker appeared on either the far left or the far right of the screen. When a fixation on the marker for at least 250 ms was detected, the marker disappeared and the scene then appeared. The location of the fixation marker was counterbalanced across conditions and participants. The scene was presented for 5 s and was then followed by a gray screen on which the preference scale was displayed (digits 1–7). Participants gave their preference rating by fixating on their preference rating for the scene viewed immediately prior to pressing a response button. There were 20 trials composed of either the normal or weird picture conditions, counterbalanced across participants. The entire experiment lasted about 25 min.

Results

Eye movement measures as a function of cultural differences will be discussed first, followed by analyses of the entire scene (see Table 1) and analyses of eye movement measures related to the normal and weird interest areas (see Table 2). For each unusual/weird scene, a rectangular region encompassing the weird interest area—the modified area, when comparing the two versions of the scene—was defined. These same areas were also used to define a region of interest (ROI) in the normal scenes for comparison purposes.

Table 1.

Eye Movement Measures for the Entire Scene for Each Experimental Condition and Participant Group

	Chinese		American
Time (ms)	Normal	Weird	Normal	Weird
Average duration of fixation on scene	282	281	280	284
Elapsed time to fixating the scene	821	797	818	833
Total viewing time	4,108	4,152	4,182	4,198

Table 2.

Eye Movement Measures for the Region of Interest (ROI) by Experimental Condition and Participant Group

	Chinese		American
Measure	Normal	Weird	Normal	Weird
Time until ROI is fixated
Elapsed time to fixation (ms)	1,415	1,270	1,352	1,257
No. of fixations	3.48	2.92	3.60	3.17
Saccade length into ROI (degrees)	5.88	5.86	6.33	6.03
Initial examination of ROI
First fixation duration (ms)	241	250	227	241
First gaze duration (ms)	615	851	551	947
No. of gaze fixations	2.12	2.95	2.15	3.38
Subsequent examination of ROI
Average no. of times examined	2.08	2.26	1.98	2.26
Second gaze duration (ms)	516	774	546	783
Average fixation duration (ms)	284	301	261	294
Total time (ms)	1,331	1,845	1,324	2,030
Total no. of fixations	3.92	5.71	3.99	6.13

Cultural Differences

The most important finding with respect to the central question of whether there are cultural differences in eye movements during scene perception—a finding that holds true across analyses of the whole scene and for ROI analyses—is that there were no systematic differences between the participant groups and no significant interactions involving participant group (see Tables 1 and 2). Importantly, all F values were less than 1 (so that estimates of standard measures of effect size such as omega squared and Cohen’s f are zero). Table 3 shows the difference score and the 95% confidence interval (CI) between the two participant groups; in most cases the difference in means was nowhere near the edge of the 95% CI. These results are consistent with those reported by Rayner et al. (2007) and Evans et al. (in press) in indicating no cultural differences concerning where Chinese and Americans look in scenes. Further discussion of this lack of an effect occurs in the Discussion section. For the remainder of the Results section, we concentrate on the differences between the normal and unusual/weird conditions.⁴ We first discuss analyses concerning the entire scene.

Table 3.

Difference Scores (Between Chinese and American Participants) and 95% Confidence Intervals (CIs) for Eye Movement Measures

	Normal		Weird
Measure	Difference	95% CI	Difference	95% CI
Time until ROI is fixated
Elapsed time to fixation (ms)	63	192	13	151
No. of fixations	−0.12	0.72	−0.25	0.54
Saccade length into ROI (degrees)	−0.45	0.65	−0.17	0.52
Initial examination of ROI
First fixation duration (ms)	14	29	9	44
First gaze duration (ms)	64	169	−96	199
No. of gaze fixations	−0.03	0.33	−0.43	0.64
Subsequent examination of ROI
Average no. of times examined	0.10	0.27	0.00	0.24
Second gaze duration (ms)	−30	174	−9	170
Average fixation duration (ms)	23	31	7	31
Total time (ms)	7	213	−185	203
Total no. of fixations	−0.07	0.49	−0.42	0.72

Note. ROI = region of interest.

Whole Scene Analysis

Although it is true that parts of the scene other than the predefined ROI could be fixated differently due to the addition or subtraction of weird regions, it is difficult to objectively decide a priori which regions were likely to be most affected by these configural changes. Therefore, we report eye movement measures on the scene overall to try to capture this possibility (see Table 1).

Average fixation duration

On average, participants’ fixation durations on the scene were 282 ms, which is in the range expected for fixations on natural scenes (Henderson, 2003; Rayner, 1998). Average fixation durations were not significantly longer for the weird scenes when compared with those for the normal scenes, F < 1.

Total viewing time

The average total viewing time on the scene (the time spent fixating) was 4,160 ms. Most of the remainder of the 5 s viewing time was in the elapsed time to the first fixation on the scene (from the fixation marker), which averaged 817 ms.⁵ Total viewing times for the weird scenes were 30 ms numerically longer than those for the normal scenes, but they were not significantly different, F < 1.

Region of Interest Analysis

The analyses of the ROI include three types of eye movement measures: (1) measures reflecting the time until the ROI was fixated; (2) measures reflecting initial examination of the region; and (3) aggregate measures reflecting the time spent examining the region. For each measure, we conducted a two-way analysis of variance with cultural group as the between-subjects variable and weirdness conditions as a within-subject variable.⁶ We address each in turn below. A summary of all the means for these measures is presented in Table 2.

Time until the ROI was fixated

The first set of measures reflects how quickly and how far away fixations were when the ROI was selected as the next saccade target for each experimental condition and each group. Elapsed time to region was defined as the time from the onset of the scene to the first fixation on the weird region. The weird regions were fixated 119 ms sooner than were the normal regions, which was marginally significant, F(1, 22) = 3.27, p = .08. The number of fixations made from the onset of the scene to the first fixation on the region was also measured. As with the elapsed time measure, the number of fixations to the weird regions were fewer than to the normal region and also marginally significant, F(1, 22) = 3.8, p = .06. As a more direct measure of how well the weird regions attracted fixations compared with the normal regions, the average length of the saccade to the region was analyzed. Overall, viewers’ saccades to the region tended to be between 5° and 6° of visual angle. However, there was no significant difference between conditions (F < 1). Thus, weird regions were fixated slightly sooner but did not attract fixations from farther away.

Initial examination of the ROI

For the initial examination of the region, we examined the duration of the first fixation made on the region as well as the duration and number of fixations for the first gaze (defined as the time spent examining a region before the eyes moved to a location outside the region). Although the first fixation duration on the ROI was numerically longer for the weird region than for the normal region, the difference was not significant, F < 1. However, there was a significant difference in the first gaze duration, F(1, 22) = 16.6, p < .01. The gaze duration was 315 ms longer on the weird region than on the normal region. As would be expected, there was also a significant difference in the number of fixations made during the first gaze, F(1, 22) = 21.8, p < .01, with weird regions receiving on average one more fixation than did normal regions.

Subsequent examination of the ROI

We also examined, beyond the initial examination, how often the region was returned to for continued examination, as well as the total amount of time and total number of fixations on the region. First, after the eyes moved to another area of the scene, the weird region was more likely to be fixated again than was the normal region, F(1, 22) = 4.93, p < .05. For those instances in which the region was returned to for further examination, the second gaze duration for the weird region was also 248 ms longer than that for the normal region, F(1, 22) = 18.8, p < .01. Second, the average fixation duration (for all fixations on the region) was significantly longer by 25 ms for the weird region than for the normal region, F(1, 22) = 7.25, p < .05. Third, the total amount of time spent examining the region (sum of all fixations on the region) was 610 ms longer for the weird region than for the normal region, F(1, 22) = 46.48, p < .01. And similarly, the number of total fixations made on the region was significantly more for the weird region (by about two fixations over the course of viewing the scene) than for the normal region, F(1, 22) = 46.63, p < .01.

Discussion

Our primary goal in the experiment was to examine possible cultural differences in scene viewing. Specifically, Chua et al. (2005) reported that Chinese viewers spend less time than American viewers looking at focal objects in a scene and more time looking at the background of the scene. As we noted earlier, some recent studies examining this issue have not found support for this difference (Evans et al., in press; Rayner et al., 2007). In the present study, rather than examining eye movement patterns for focal and background objects, we examined how quickly Chinese and American viewers looked at a weird/unusual object in a scene. For the scenes in the weird condition, a portion of a central object (or group of objects) was configured differently, resulting in an odd interpretation of the picture. The question addressed in the present study was whether Chinese viewers would be slower or faster to fixate the odd objects than would the American viewers. To address either of these possibilities, we examined the viewing of regions of interest. Across all eye movement measures, there were no differences between the Chinese and American viewers. Thus, the present study found no evidence that the unusual/weird regions were fixated later (or earlier) or for differing amounts of time as a function of cultural difference (see also Evans et al., in press; Rayner et al., 2007).

Rayner et al. (2007) did report some differences in the overall patterns of eye movements, with Chinese viewers making more fixations for shorter durations than did American viewers (see also Chua et al., 2005). To investigate the possibility of an overall difference in eye movement patterns (unrelated to the unusual/weird condition), we examined the number of fixations and the average fixation duration made across the entire scene. However, examination of these measures found no significant effects of viewer group and no significant interactions between the viewer group and weird condition. If anything, the fixation time measures on the ROI show just the opposite pattern, in that the Chinese viewers’ average first fixation durations during the initial examination of the ROI and their average fixation duration on subsequent examination of the region (see Table 2) were longer than the Americans’ (though not significantly so) and they made slightly fewer fixations. It is not obvious why we did not replicate the earlier finding reported by Rayner et al. that Chinese viewers trade off fixation duration for number of fixations, but it is instructive that Evans et al. (in press) also did not find evidence for this tradeoff.

It is important to note that we do not view the present results as compromising Nisbett’s (2003) theory of cultural differences in thinking. We have no doubts that there are cultural differences between Chinese and Americans that manifest themselves in different ways of thinking about various issues (Nisbett, 2003). However, the present results, along with those of Rayner et al. (2007) and Evans et al. (in press), raise doubts about the more extreme notion that cultural differences can influence processing at the basic level of oculomotor control.

A secondary goal of the present study was to examine the finding that the eyes are drawn quickly to unusual aspects of a scene. We found that the weird regions attracted fixation slightly sooner than did normal regions, although the regions were not fixated from farther away. The regions were also examined for a longer period of time and returned to for further examination more often over the course of viewing the scene. This finding is consistent with past studies showing that anomalies in scenes are fixated in total for a longer period of time (Friedman, 1979; Henderson et al., 1999). The important difference with the current study is that the weird regions were not odd in the same sense as those used in past studies. Rather than using objects that did not fit with the scene’s gist or schema, in the present study we used objects that suggested strange or impossible configurations in the weird regions. These differences rely not on an understanding of the whole scene compared with one of its parts but on an understanding (or lack thereof) of the parts’ relationships with each other.

In summary, our results are consistent with those of Becker et al. (2007) and Harris et al. (2008) in showing that (a) weird regions attract fixations slightly sooner than do normal regions and (b) viewers look longer at weird regions than at normal regions. More importantly, our results are consistent with those reported by Rayner et al. (2007) and Evans et al. (in press) in showing that cultural differences have little influence on oculomotor control during scene perception.