Skip to main content
NCBI home page
Human Brain Mapping logoLink to Human Brain Mapping
. 2014 Oct 18;36(2):619–632. doi: 10.1002/hbm.22651

The Mona Lisa effect: Neural correlates of centered and off‐centered gaze

Evgenia Boyarskaya 1,2,✉, Alexandra Sebastian 2,3, Thomas Bauermann 2,4, Heiko Hecht 1,2, Oliver Tüscher 2,3,5,6
PMCID: PMC6869588  PMID: 25327821

Abstract

The Mona Lisa effect describes the phenomenon when the eyes of a portrait appear to look at the observer regardless of the observer's position. Recently, the metaphor of a cone of gaze has been proposed to describe the range of gaze directions within which a person feels looked at. The width of the gaze cone is about five degrees of visual angle to either side of a given gaze direction. We used functional magnetic resonance imaging to investigate how the brain regions involved in gaze direction discrimination would differ between centered and decentered presentation positions of a portrait exhibiting eye contact. Subjects observed a given portrait's eyes. By presenting portraits with varying gaze directions—eye contact (0°), gaze at the edge of the gaze cone (5°), and clearly averted gaze (10°), we revealed that brain response to gaze at the edge of the gaze cone was similar to that produced by eye contact and different from that produced by averted gaze. Right fusiform gyrus and right superior temporal sulcus showed stronger activation when the gaze was averted as compared to eye contact. Gaze sensitive areas, however, were not affected by the portrait's presentation location. In sum, although the brain clearly distinguishes averted from centered gaze, a substantial change of vantage point does not alter neural activity, thus providing a possible explanation why the feeling of eye contact is upheld even in decentered stimulus positions. Hum Brain Mapp 36:619–632, 2015. © 2014 Wiley Periodicals, Inc.

Keywords: Mona Lisa effect, cone of gaze, gaze perception, eye contact, superior temporal sulcus, fusiform gyrus

INTRODUCTION

The eyes are special—they do not only transfer sensory input to the brain, but also convey a rich source of information, which can be used by other individuals. The eyes have been determined to provide important information about attractiveness and to drive attention when observing another's face [Bruce and Young, 1998; Itier et al., 2007a, 2007b; Janik et al., 1978; Yarbus, 1967]. Also, the eyes play a central role in face recognition [Itier et al., 2006; Vinette et al., 2004].

Eye gaze not only attracts attention, it also reveals information about the looker's current state, be it is her/his object of interest or his/her emotional and intentional states [e.g., Emery, 2000; Gläscher et al., 2004]. Gaze direction is a powerful cue for implicit (nonverbal) communication. The direction of gaze expresses communication disposition or attendance, synchronizes turn talking, regulates level of emotionality, intimacy or affiliation and dominance, signals liking, attraction, credibility, and even mental health [Argyle and Cook 1976; Argyle and Dean, 1965; Argyle et al., 1972; von Cranach and Ellgring, 1973; Kendon, 1967; Kleinke, 1986]. Gaze understanding plays an essential role for successful social cognition and “theory of mind” [e.g., Baron‐Cohen, 1995]. Furthermore, gaze‐following behavior is thought to be a key element in establishing joint attention [Dunham and Moore, 1995], which is essential for language acquisition [Bruner, 1983; Tomasello and Farrar, 1986]. Thus, the ability to detect gaze and to correctly determine where exactly our interaction partner aims her or his gaze is of crucial significance. Many studies about gaze perception exist; however, gaze is typically studied with pictorial stimuli, which have some peculiar properties that may question if results obtained with pictorial face stimuli generalize to three‐dimensional (3D) heads. One such peculiarity is the Mona Lisa effect. Here, we investigate if the neural correlates of the Mona Lisa effect correspond to the gaze cone width of about 5 degrees to either side that is typically obtained with physical lookers as well as with portraits viewed from the proper straight‐ahead vantage point [Gamer and Hecht, 2007].

The Mona Lisa Effect

When looking at a portrait (whether it is a photograph, painting, drawing, or computer animation), the portrait's eyes appear to “follow” us as we move around and change our viewing angle—the Mona Lisa effect [see e.g., Brewster, 1832]. For example, if we look at a portrait with straight‐ahead gaze, it appears to look us in the eyes even if we step to the side. As we arbitrarily change our position relative to the picture, the portrait seems to maintain eye‐contact, following us with its eyes. This “rotation effect” applies as well to portraits with averted gaze: the portrait will always appear to be looking to the side, regardless of our position.

A number of studies have found strong evidence for the Mona Lisa effect using various picture and observer displacements [see e.g., Goldstein, 1987; Maruyama et al., 1985; Rogers et al., 2003]. The effect holds for portraits with direct gaze and observer displacements in the horizontal, vertical, and diagonal planes [Boyarskaya and Hecht, 2012]. This effect persists, albeit in a less perfect manner, if the portrait has an averted gaze. The Mona Lisa effect is very robust also in extremely oblique viewing conditions (Hecht et al., in press). However, hardly anything is known about the neural basis of the Mona Lisa effect. Would the brain respond equally to the gaze of a portrait if it is centered versus displaced to the side, even if the subject fixates the gazing portrait in both cases?

Gaze Perception and Cone of Gaze

Human eyes have a unique morphology and are extraordinarily salient and prominent, simplifying the detection of gaze direction [Kobayashi and Koshima, 1997; 2001a, 2001b]. In the few animals that have a high‐contrast difference between sclera and iris, such as dogs, the sclera is usually occluded by the eye‐lid. This extraordinary salience may have evolved as an adaptation to enhance the gaze signal, thereby enabling communication using eye signals as a new social function in human evolution [Kobayashi and Koshima, 2001b; Tomasello et al., 2007]. Human infants show sensitivity to eye appearance from birth [Batki et al., 2000]. Moreover, newborns are able to distinguish between mutual gaze and averted gaze by preferring faces with direct gaze to those with averted gaze [Farroni et al., 2002, 2004, 2006]. In this context it is not surprising that humans can estimate the gaze direction of others relatively accurately, especially if the gaze is directed toward themselves [Anstis et al., 1969; Cline, 1967; Gibson and Pick, 1963; Symons et al., 2004].

More recently, Gamer and Hecht [2007] demonstrated that despite the ability to discriminate gaze directions well within a degree of visual angle, the subjective criterion for being engaged in eye contact is surprisingly liberal. It comprises about nine or 10 degrees. Thus, gaze direction is best conceived of as a cone. This result was obtained with portraits viewed straight‐on and validated using a virtual 3D head and a live gazing subject. Gamer and Hecht measured the gaze cone with an interactive task in which observers were required to move the eyes of the head to the edge of the gaze cone. The gaze cone of approximately 10 degrees was also found to be only slightly wider in pictures [two‐dimensional (2D) head] compared to 3D images and the physical head. The cone width was roughly maintained even when looking at a mere line‐drawing of a head.

Perception of Direct and Averted Gaze

Perception of eye contact and perception of averted gaze have very different behavioral and cognitive implications [see also George and Conty, 2008; Senju and Johnson, 2009]. For instance, the subjective eye contact enhances the memorability of faces [Adams et al., 2010; Hood et al., 2003; Macrae et al., 2002; Mason et al., 2004; Smith et al., 2006; Vuilleumier et al., 2005]. Faces with direct gaze, at least in European cultures, are perceived to be more attractive than those with averted gaze [Ewing et al., 2010; Kampe et al., 2001]. Even the desirability of an object grows when it is associated with an attractive face displaying direct gaze [Strick et al., 2008]. Further, eye contact elevates the likability of the face [Kuzmanovic et al., 2009; Mason et al., 2005] and makes the face appear more trustworthy [Wyland and Forgas, 2010]. Eye contact may also have a distracting power [Conty et al., 2010] and capture visuospatial attention [Senju and Hasegawa, 2005]. The perception of eye contact increases physiological arousal as measured by galvanic skin responses [Hietanen et al., 2008; Helminen et al., 2011; Nichols and Champness, 1971] and thumb blood volume [Mazur et al., 1980]. Also, EEG arousal increases in the case of mutual gaze [Gale et al., 1972, 1975].

In contrast to direct gaze, perceived averted gaze is consistently associated with the general disposition of humans to orient their own visual spatial attention toward the direction indicated by eye gaze direction of others—a phenomenon described in terms of gaze cueing of attention [see Frischen et al., 2007 for review]. Whereas some authors claim that the attention shift in response to another's gaze direction is reflexive / automatic / bottom‐up / exogenous [Driver et al., 1999; Farroni et al., 2004; Friesen and Kingstone, 1998; Hood et al., 1998; Mathews et al., 2003], others suggest that this effect is top‐down / endogenous in nature [Itier et al., 2007a, 2007b; Ristic and Kingstone, 2005; Vecera and Rizzo, 2004, 2006]. Furthermore, objects are liked more when they are looked at by another person [Bayliss et al., 2006].

Computing the gaze direction is an obligatory component of the processing of facial expression [Ganel et al., 2005]. For example, angry faces exhibiting direct gaze are perceived to be even angrier, and fearful faces displaying averted gaze are perceived to be even more fearful [Adams and Kleck, 2003, 2005; Sander et al., 2007].

Neural Correlates of Gaze Perception

Given that eye contact and averted gaze effectively trigger different psychological mechanisms, it may be expected that they activate distinctive brain networks [George and Conty, 2008]. There has been considerable research on the neural bases underlying gaze processing [for review, see Emery, 2000; Grosbras et al., 2005; Itier and Batty, 2009; Nummenmaa and Calder, 2009; Senju and Johnson, 2009]. Neurophysiological primate studies and neuroimaging methods in humans highlight the role of superior temporal sulcus (STS) for gaze perception [Akiyama et al., 2006a, 2006b; Allison et al., 2000; Bristow et al., 2007; Calder et al., 2002, 2007; Campbell et al., 1990; Carlin et al., 2011; Engell and Haxby, 2007; Ethofer et al. 2011; Hoffman and Haxby, 2000; Hooker et al., 2003; Kuzmanovic et al., 2009; Laube et al., 2011; Nummenmaa and Calder, 2009; Nummenmaa et al., 2010; Pageler et al., 2003; Pelphrey et al., 2003, 2004; Perrett et al., 1985, 1992; Puce et al., 1998; Schilbach et al., 2006; Sato et al., 2008; Wicker et al., 1998, 2003]. Especially Carlin et al. [2011] emphasized that STS contains a higher‐order, fine grained, and head‐view‐invariant gaze direction code. Furthermore, the involvement of fusiform gyrus (FG) in gaze perception is underlined by many studies [Calder et al., 2002; George et al., 2001; Hoffman and Haxby, 2000; Pageler et al., 2003]. Further structures that have been reported to be involved in gaze perception are: intraparietal sulcus [IPS; Bristow et al., 2007; Ethofer et al., 2011; Hadjikhani et al., 2008; Hoffman and Haxby, 2000; Pageler et al., 2003; Pelphrey et al., 2003; Puce et al., 1998], amygdala [Adams et al., 2003; George et al., 2001; Hadjikhani et al., 2008; Hooker et al., 2003; Kawashima et al., 1999; Sato et al., 2004, 2010; Straube et al., 2010; Wicker et al., 2003], MT/V5 [Ethofer et al., 2011; Hadjikhani et al., 2008; Kuzmanovic et al., 2009; Watanabe et al., 2001; Wicker et al., 2003], and medial prefrontal cortex [MPFC; Calder et al., 2002; Conty et al., 2007; Kampe et al., 2003; Schilbach et al., 2006]. Senju and Johnson [2009] explicitly focused in their review on the neural correlates of eye contact in a number of studies and outlined six brain areas that respond differentially to eye contact versus averted gaze: the FG, right anterior and posterior STS region, MPFC, orbitofrontal cortex and amygdala. The inconsistency among studies, however, does not conclusively support a universal model to entirely describe neural correlates of gaze perception. We compared the findings about cerebral structures involved in eye contact perception and perception of averted gaze which are summarized in Table 1. However, the findings remain controversial and most specific questions are still to be answered.

Table 1.

Brain areas, which differentiated between direct and averted gaze

Direct > averted (more active during direct vs. averted gaze) Averted > direct (more active during averted vs. direct gaze)
FG (George et al., 2001; Calder et al., 2002; Pageler et al., 2003) STS (Hoffman and Haxby, 2000; Engell and Haxby, 2007; Sato et al., 2008)
STS (Pelphrey et al., 2004; Kuzmanovic et al., 2009) IPS (Hoffman and Haxby, 2000)
amygdala (Wicker et al., 2003; Sato et al., 2004; Kawashima et al., 1999—only right) amygdala (Straube et al., 2010)
MPFC (Calder et al., 2002; Schilbach et al., 2006) parahippocampal gyrus (Sato et al., 2008)
PCC (Kampe et al., 2003) angular gyrus (Sato et al., 2008)
temporal poles (Kampe et al., 2003) MFG (Calder et al., 2002; Sato et al., 2008)
MT/V5 (Kuzmanovic et al., 2009) cerebellum (Sato et al., 2008)
STG (Calder et al., 2002)
Open in a new tab

FG, fusiform gyrus; STS, superior temporal sulcus; MPFC, medial prefrontal cortex; PCC, paracingulate cortex; STG, superior temporal gyrus; IPS, intraparietal sulcus; MFG, middle frontal gyrus.

Given that a surprisingly wide range of metrically averted gaze directions are judged as being directed at the observer [Gamer and Hecht, 2007], some brain regions should respond preferentially to clearly averted but not to more subtly averted gaze (at the edge of the cone of gaze). Further, the putative gaze discrimination regions should make similar gaze direction distinctions regardless of the portrait's location with respect to the observer. We test these predictions and compare the brain response to eye contact observed for different gaze directions. More importantly, we investigate how the activation in gaze sensitive regions is modulated by the portrait's presentation position, while subjects maintain eye contact with the portrait. To address these questions we used photographs of face stimuli with different degrees of direct or averted gaze (±5 or 10 degrees) in an functional magnetic resonance imaging (fMRI) study.

METHODS

Participants

Twenty‐two healthy right‐handed volunteers (12 female, 10 male; age range 19–39 years; mean age 24.8 years) participated in the fMRI experiment and in a short behavioral test outside the scanner. Two additional participants were excluded from the analysis due to excessive head movement during the scanning session. All participants had normal visual acuity as assessed with a traditional Snellen eye chart. They were naïve as to the purpose of the current experiment and reported no history of neurological or psychiatric illnesses. The study was approved by the ethical review committee of the Johannes Gutenberg University Mainz; participants provided written informed consent and were paid for participating.

Stimuli

Stimuli were digitized frontal full face portrait photographs of a blue‐gray‐eyed female model with a neutral facial expression, who was gazing in one of three directions whereby only the eyes differed (see Fig. 1). The model directed her gaze either directly at the camera (gaze angle of 0°, straight ahead), or at two targets placed in the horizontal plane of the camera resulting in gaze angles of 5° and 10° to the cameras' right. Gaze angles refer to deviations from the straight‐ahead of the model's gaze. The camera lens was positioned 1 m in front of the model. Each photograph was taken from the 0° position of the viewer. Thus, the photographed face had three gaze directions: 0°—eye contact, 5° to the right—the edge of the cone of gaze, and 10° to the right—clearly averted gaze. Directions were given from the observer's point of view. The three gaze directions were fully crossed with three picture presentation positions: central, shifted to the right, and shifted to the left by 28° of visual angle, resulting in nine conditions. Further, we altered the color of the eyes to green using Photoshop. The difference between original blue‐gray eyes and altered green eyes was detectable (as proven in a short test with a sample of 10 participants), but not saliently evident. The stimuli were presented randomly in both color variants; frequency of portraits with blue‐gray and green eyes was counterbalanced. All stimuli, fixation cross and task questions were presented against uniformly light gray background.

Figure 1.

Figure 1

Open in a new tab

Examples of stimuli. (a) face with direct gaze (0°); (b) face with gaze directed 5° to the right; (c) face with gaze averted 10° to the right. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

fMRI Experiment

Design

Each face (three gaze directions × three picture positions) was presented 12 times (six times with blue‐gray and six times with green eyes) in each of three runs, resulting in 324 randomized trials per subject. The test duration was approximately 15 min per run, with 2‐min breaks between runs.

Task

Participants were instructed to fixate each face. The task assuring fixation was to decide whether or not the eyes of the presented portrait were green by giving the answer “yes” or “no.” Subjects indicated their task responses using a button box held in their right hand. This irrelevant task was designed to ascertain that subjects paid attention to the portrait's eyes and to implicitly assess gaze direction processing.

Trial sequence

Participants attended to a fixation cross that was presented centrally and had a variable duration of 1.5–4.05 s. The variable duration was used to prevent expectancy effects. The stimulus face was presented for a fixed duration of 1.5 s and then disappeared from view, followed by the fixation cross with duration of 1 s. Then, the task question was displayed for constant time interval of 3 s during which participants gave their responses. Thus, the interstimulus interval varied from 7 to 9.55 s.

Stimulus presentation

Stimuli were randomly presented in an event‐related design, programmed using Matlab‐based Cogent software. Images were projected via a projector (resolution 1024×768) onto a large custom‐designed rear projection screen (138 × 82 cm2), which was placed at a distance of approximately 5 m from the subject. Participants viewed the screen via a two‐mirror system, situated above their eyes approximately 12 cm away from the bridge of the nose and mounted on the head‐coil gradient set. The mirror position was individually adjusted for each participant to ascertain that the images were entirely visible and vertically centered at the respective eye level. The mirror was 12 cm high and 16 cm wide, thus, the field of view was approximately 67°. The lateral shift of picture position (the midpoint between the portrait's eyes referred to as center of projection) resulted in approximately 28° of visual angle to the left or right.

fMRI data acquisition

MR imaging was performed with a 3Tesla Magnetom Trio magnetic resonance imaging scanner (Siemens, Germany) with an eight‐channel phased‐array head coil gradient set at the University Medical Center of the Johannes Gutenberg—University Mainz. Functional brain data were acquired with T2*‐weighted echo‐planar image (EPI) volumes, sensitive to blood‐oxygen level dependent (BOLD) signal contrast (TE = 30 ms, TR = 2,000 ms, flip angle = 90°, FOV = 192 mm) covering the whole brain (33 transversal slices in ascending series, matrix = 64×64, voxel size = 3×3×3 mm + 0.6 mm gap). For each subject, a total of 1,365 functional volumes were acquired (455 per run). Before functional imaging, a T1*‐weighted structural image was obtained (TE = 2.52 ms, TR = 1,900 ms, flip angle = 90°, FOV = 250 mm, TI = 900 ms, 176 sagittal slices, voxel size = 1×1×1 mm) for registration purposes. Head movement was minimized using bilateral foam padding.

Data preprocessing and analysis

Data preprocessing and analysis was performed using SPM8 software (http://www.fil.ion.ucl.ac.uk/spm). Images were screened for motion artifacts prior to data analysis. Excessive head motion (>2 mm) was observed in two subjects who were excluded from data analysis, which was performed on the remaining 22 subjects. Prior to analysis, the functional EPI scans were spatially realigned to correct for slight head motion, coregistered to individual subject anatomical images, stereotacticly normalized to the Montreal Neurologic Institute (MNI) version of the standardized coordinate space (MNI—International Consortium for Brain mapping) and spatially smoothed with an isotropic Gaussian kernel [full width at half maximum (FWHM) = 8 mm] to increase the signal‐to‐noise ratio. Using the onset times for each event (face appearance), the canonical SPM‐specific event‐related hemodynamic response function was modeled applying the general linear model. Regressors representing estimated translational and rotational movements (altogether six degrees of freedom) acquired in the realignment procedure were included in the model as covariates of no interest. For each subject, a contrast for each of the gaze directions (0°, 5°, and 10°) and presentation positions (central, left, and right) was calculated. To assess the neural correlates of the cone of gaze, single subject contrast images comparing gaze angles beyond and within the cone of gaze were subjected to second level random effects models using t‐tests (i.e., 10° > 0° and 10° > 5°). The neural correlates of gaze direction were assessed by contrasting direct gaze displayed centrally to direct gaze shifted laterally either on the left or on the right (i.e., central > left, central > right, and vice versa). Finally, to test whether the neural correlates underlying eye contact remain unchained when the portrait is presented at decentered view points, we performed a 3 (gaze direction: 0°, 5°, 10°) × 3 (presentation position: central, left, right) factorial analysis of variance (ANOVA) to test for an interaction of gaze direction and presentation position. Statistical maps were thresholded at P < 0.05 familywise error‐corrected on the cluster level with a minimum cluster size of 10 voxels. The primary cluster defining threshold was set to P < 0.001 uncorrected. For visualization purposes, the activations were overlaid on an SPM8 MNI MRI single‐subject template brain (threshold at P < 0.001 uncorrected with a minimum cluster size of 10 voxels for display only).

Behavioral Test

Design

To assess explicit gaze direction judgments, the behavioral test containing the same stimuli was conducted outside the scanner after the fMRI procedure. Stimuli were presented on a computer monitor (33.3 × 20.8 cm, resolution 1280 × 800) from a distance of 40 cm in a dark room to keep the light conditions similar to those during fMRI session. The stimulus size in terms of visual angle was comparable to that of the original fMRI experiment. Each face (three gaze directions × three picture positions) was presented 4 times (2 times with blue‐gray and 2 times with green eyes) fully randomized in two blocks with a short break between them.

Task

Participants had to perform gaze discrimination judgments. After face disappearance, they were required to judge the gaze direction of each face. Participants choose between four answer possibilities presented on the screen in constant order using a mouse cursor: was looking directly at myself / was looking slightly to my right / was looking to my right / was looking clearly to my right.

Trial sequence

Trial sequence and presentation duration were identical to those in the original fMRI experiment. However, the next trial appeared always as soon as an answer had been given, without a standardized waiting time interval.

RESULTS

Behavioral Data During fMRI Scanning

All participants performed the eye‐color discrimination task very accurately. Participants indicated blue‐gray eye color in 97.3% correctly and in 0.7% incorrectly, with 2% missing values. In the case of green eyes, 95.6% correct and 2.8% incorrect responses were made, with 1.6% missing values. Thus, the attendance to the eyes was very high during the whole experiment across all participants. The task to discriminate eye color required foveation due to the rather subtle difference between blue‐gray and green eye color of the iris. The experimenters were unable to tell iris color when looking outside the eye‐region of the stimulus. This indicates that the subjects did fixate the eyes although gaze direction may or may not have been noticed. Thus, the position of the stimulus on the retina was largely independent of the portrait's lateral position on the screen.

fMRI Data

Cone of gaze

As we were interested in specific differences between brain responses to gaze angles beyond and within the cone of gaze respectively, we compared increased activation to faces with clearly averted gaze (10° to the right) with activation to faces making eye contact (0°) and to those looking 5° to the right (at the edge of the cone of gaze). According to the concept of the cone of gaze, we expected increased BOLD signal during clearly averted gaze as compared to direct gaze (10°>0°) as well as to gaze averted by 5° on the border of the gaze cone (10°> 5°).

Contrasting gaze direction of 10° to gaze direction of 0° (i.e., 10°>0°) revealed significant clusters of activation in the right FG, right middle temporal gyrus including portions of STS, and left calcarine gyrus. Increased activity was also observed in right superior parietal lobule, right precuneus, and left postcentral gyrus. See also Figure 2a, Table 2.

Figure 2.

Figure 2

Open in a new tab

Brain activation for the comparisons of gaze (a) clearly averted to the right minus direct gaze (10°>0°); (b) clearly averted to the right minus gaze at the right boundary of the gaze cone (10°>5°); (c) conjunction of clearly averted minus direct gaze (10°>0° and 10°>5°). P < 0.05 FWE‐corrected on the cluster level, minimum cluster size = 10 voxels. Activations are projected onto the MNI standard brain template (for display purposes: P < 0.001 uncorr., minimum cluster size = 10 voxels; color bar indicates T‐score). Lateral views, axial and coronal slices are presented. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Table 2.

Effects of observed clearly averted versus direct gaze and gaze at the edge of the gaze cone

Brain areas Side 10°>0° 10°>5° Conjunction of 10°>0° and 10°>5°
BA MNI coordinates t voxels BA MNI coordinates t voxels BA MNI coordinates t voxels
x y z x y z x y z
middle occipital gyrus L 18 −30 −91 −2 4.43 218a
inferior occipital gyrus L 19 −39 −79 −11 4.58 218a
calcarine gyrus L 17 −18 −64 7 5.76 1165a
FG L 37 −36 −55 −14 4.41 218a
FG R 37 33 −52 −11 5.87 1165a 37 36 −43 −17 7.33 657a 37 30 −40 −17 3.72 151
supramarginal gyrus R 40/48 66 −34 25 5.34 357a 49 45 −37 25 3.43 206a
MTG/STS R 37/21 60 −58 4 6.68 1165a 21 48 −28 −5 5.82 65 37/22 63 −49 16 4.35 206a
superior parietal lobule R 7 27 −55 64 4.87 96a
precuneus R 5 15 −52 55 4.82 96a
postcentral gyrus L 3 −51 −16 19 4.99 134 2 −51 −22 19 4.95 179a
rolandic operculum L −45 −16 22 4.61 179a
MFG R 44/46 36 17 40 5.56 104a
SFG R 8 18 23 46 4.41 104a
IFG/ pars triangularis R 44/45 48 29 7 4.95 100a
Open in a new tab

P < 0.05 FWE−corrected on the cluster level, minimum cluster size = 10 voxels.

a

This subcluster is confluent with another subcluster.

The voxel number represents the total cluster size.

FG, fusiform gyrus; MTG, medial temporal gyrus; STS, superior temporal sulcus; MFG, middle frontal gyrus; SFG, superior frontal gyrus; IFG, inferior frontal gyrus.

Contrasting gaze direction of 10° to gaze direction of 5° (i.e., 10°>5°) revealed increased activation in FG bilaterally, although activation in this region was more pronounced in the right hemisphere. This activation extended to left middle and inferior occipital gyri. Activity was also observed in the right STS. Additional activation was found in right supramarginal gyrus, left postcentral gyrus, and left rolandic operculum. Frontal regions included right middle frontal gyrus, right superior frontal gyrus, and right inferior frontal gyrus / pars triangularis. See also Figure 2b, Table 2.

Both contrasts, 10°>0° and 10°>5°, resulted in activation in the right FG, right STS, and left postcentral gyrus. We performed a minimum statistic conjunction analysis [“conjunction null”; Nichols et al., 2005] to test whether activation in these regions was mutually present for averted versus direct gaze (within the cone of gaze as well as at the edge of the cone of gaze). The conjunction analysis revealed mutual activation for averted versus direct gaze in the STS as well as in the FG (Table 2 and Figure 2c). This suggests that both, the STS and the FG mutually subserve the discrimination of clearly averted gaze (10°) as compared to direct gaze within (0°) as well as at the edge (5°) of the cone of gaze. Further, the overlapping networks resulting from this conjunction indicate that the gaze direction of 5° might have been perceived as mutual gaze.

Furthermore, conditions with a straight‐ahead look were compared with those where the face gazed 5° to the right (0°>5° and 5°>0°). We also compared neural responses to direct gaze versus 10° gaze aversion (0°>10°) and to gaze directed 5° to the right versus 10° gaze aversion (5°>10°). No significant activation differences were found in either of these comparisons.

The Mona Lisa effect

To determine whether faces making eye contact with the observer yielded different neural responses depending on picture presentation position, we first compared direct gaze (gaze direction 0°) shown centrally, laterally on the left, and on the right using t‐tests. The contrasts “central>right” and “central>left” were performed to identify regions with greater response to central picture presentation position than to lateral picture presentation position. The contrasts “right>central” and “left>central” were computed to reveal activation patterns associated with lateral portrait positions.

Prominent activations for faces displaying eye contact presented centrally as compared to those presented shifted to the right were concentrated in right superior occipital gyrus, right calacarine gyrus, and in inferior occipital gyrus / lingual gyrus. When contrasting faces with direct gaze presented centrally to those shifted to the left, the left inferior occipital gyrus / lingual gyrus was more active. On a more liberal threshold, contrasting “central>left” revealed increased activation in inferior occipital gyrus / lingual gyrus also in the right hemisphere as well (30, −94, −2; P uncorrected < 0.001; k = 10 on peak‐level). A minimum statistic conjunction analysis [“conjunction null”; Nichols et al., 2005] confirmed mutual activation for central as compared to lateral (i.e., left or right) presentation position in bilateral inferior occipital gyri. For more details see Figure 3 and Table 3.

Figure 3.

Figure 3

Open in a new tab

Brain activation for the comparisons of eye contact in pictures presented (a) central minus shifted to the right; (b) central minus shifted to the left. P < 0.05 FWE‐corrected on the cluster level, minimum cluster size = 10 voxels. Activations are projected onto the MNI standard brain template(for display purposes: P < 0.001 uncorr., minimum cluster size = 10 voxels; color bar indicates T‐score). Lateral views and axial slices are presented. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Table 3.

Effects of presentation position: central versus lateral

Brain areas Side Central > right Central > left Conjuction of central > right and central > left
BA MNI coordinates t voxels BA MNI coordinates t voxels BA MNI coordinates t voxels
x y z x y z x y z
superior occipital gyrus R 17 24 −100 7 5.67 279a
calcarine gyrus R 17 15 −100 −5 5.11 279a
inferior occipital gyrus/ lingual gyrus L 18 −24 −94 −5 7.36 186 18 −24 −91 −5 7.41 234 18 −24 −94 −5 5.22 153
inferior occipital gyrus/lingual gyrus R 18 33 −88 −11 7.28 279a 18 30 −94 −2 5.17 23b 18 30 −94 −2 4.73 67
Open in a new tab

P < 0.05 FWE‐corrected on the cluster level, minimum cluster size = 10 voxels

a

This subcluster is confluent with another subcluster.

The voxel number represents the total cluster size

b

P uncorrected < 0.001; k = 10 on peak level.

The contrast “right>central” revealed significant signal increase in a large cluster of voxels covering right cuneus and bilateral calcarine gyrus. The contrast “left>central” revealed significant activation in left calcarine gyrus extending to left lingual gyrus. For overview see Figure 4, and Table 4. A minimum statistic conjunction analysis [“conjunction null”; Nichols et al., 2005] confirmed mutual activation for lateral (i.e., left or right) as compared to central presentation position in left calcarine gyrus and bilateral lingual gyri. Interestingly, the same contrasts (central versus right or left and vice versa) including also the averted gaze stimuli, yielded similar results (data not shown).

Figure 4.

Figure 4

Open in a new tab

Brain activation for the comparisons of eye contact in pictures presented (a) shifted to the right minuscentral; (b) shifted to the left minus central. P < 0.05 FWE‐corrected on the cluster level, minimum cluster size = 10 voxels. Activations are projected onto the MNI standard brain template (for display purposes: P < 0.001 uncorr., minimum cluster size = 10 voxels; color bar indicates T‐score). Lateral views and axial slices are presented. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Table 4.

Effects of presentation position: lateral versus central

Brain areas Side Right > central Left > central Conjunction of right > central and left > central
BA MNI coordinates t voxels BA MNI coordinates t voxels BA MNI coordinates t voxels
x y z x y z x y z
cuneus R 19 6 −82 37 8.81 2848a
calcarine gyrus R 17 9 −73 10 12.71 2848a 17/18 3 −73 1 5.92 2496a
calcarine gyrus L 17 −9 −67 7 9.60 2848a 17/18 −12 −79 7 12.13 2843a 17/18 −15 −61 2 7.12 2496a
lingual gyrus L 18 −9 −70 1 16.03 2843a
superior parietal lobule R 5 21 −55 55 3.13 19
Open in a new tab

P < 0.05 FWE‐corrected on the cluster level, minimum cluster size = 10 voxels

a

This subcluster is confluent with another subcluster.

The voxel number represents the total cluster size.

To test for an interaction of gaze direction and presentation position, we then performed a 3 (gaze direction: 0°, 5°, 10°) × 3 (presentation position: central, left, right) factorial ANOVA. No interaction effects could, however, be observed (data not shown). This implicates that presentation position had no impact on the Mona Lisa Effect.

Behavioral test (outside scanner)

Participants were able to discriminate gaze directions of the presented faces and could estimate their gaze direction from all viewing points accurately. We conducted a correspondence analysis of the obtained gaze direction judgments to describe and visualize the data, using SPSS 19.0 (Statistical Package for the Social Sciences).

Correspondence analysis is a statistical method that seeks to represent the relationships between the rows and columns of a table as points in a spatial 2D map [Greenacre, 2007]. Figure 5 shows row points for the actual gaze direction of a portrait, and column points for the estimated gaze direction. The chart suggests that subjectively perceived direct gaze was strongly related to the actual direct gaze of a portrait. Further, the eyes looking 5° to the right were judged as looking slightly rightward. Finally, the portraits with the gaze averted 10° to the right were closely associated with apparent gaze direction to the right or clearly to the right. Thus, the gaze direction estimations had a strong correspondence to the actually exhibited gaze of the face, and this across all presentation positions.

Figure 5.

Figure 5

Open in a new tab

A correspondence map: Biplot of symmetrically normalized row (actual gaze direction of a face) and column profiles (judged gaze direction of a face).

Further analysis demonstrated that the face with gaze directed 5° to the participant's right was most prone to errors. On average, the 5° off‐centered gaze was misperceived as being directed into the observer's eyes in 28.8% of the cases. In comparison, the 10° averted gaze was misperceived as being direct gaze in only 2.5%.

DISCUSSION

In this study we have identified brain areas that respond selectively to averted gaze versus. direct gaze, and gaze at the edge of the gaze cone, thereby supporting the concept of the cone of gaze. We have also shown that these neural correlates of eye contact remain unchanged when the portrait is presented at decentered view points. That is, the Mona Lisa effect was manifest in the pattern of neural activation.

Cone of Gaze

Cone of gaze refers to all gaze directions within which a person feels looked at. Therefore, we expected that averted gaze would yield distinctly different neural responses as compared to direct gaze within (10°>0°) as well as at the edge (10°>5°) of the cone of gaze. Accordingly, we identified the right STS and FG to be mutually stronger activated during clearly averted gaze as compared to direct gaze within and at the edge of the cone of gaze. This finding was further supported by that fact that brain activation patters associated with direct gaze versus gaze at the edge of the gaze cone (0°>5°; 5°>0°) did not differ significantly. Further support for the hypothesis that gaze at the edge of the cone of gaze is perceived similar to direct gaze stems from the behavioral task outside the scanner. During the explicit gaze discrimination task, the same subjects could discriminate between 0° and 5° gaze eccentricity. However, most errors were made in the 5° condition with a 5° averted gaze being misperceived as direct gaze. That is, the neural responses to eye contact were somewhat more liberal than what would be expected if gaze direction had been judged explicitly. We had opted to use the indirect task of judging eye color as opposed to a direct judgment task because the latter is known to be vulnerable to sizable adaptation effects [e.g., Jenkins et al., 2006].

Averted Gaze Versus Eye Contact

We focused on areas that showed greater activation to clearly averted gaze than to eye contact. We found a considerable overlap in increased BOLD signals elicited by faces with clearly averted gaze in the right FG and the right STS.

Our results regarding stronger right STS response to averted gaze in comparison to eye contact are in line with previous findings [Engell and Haxby, 2007; Sato et al., 2008], although Sato et al. [2008] found increased bilateral STS activity. Also, the enhanced STS activity in response to averted gaze as compared to direct gaze was demonstrated by Hoffman and Haxby [2000], however, it was located in the left hemisphere in their study. Two other studies report opposite findings. Pelphrey et al. [2004] and Kuzmanovic et al. [2009] revealed more pronounced STS activation during eye contact relative to observing averted gaze. However, both studies used animated dynamical stimuli with gaze shifts and, thus, eye motion. In contrast, stimuli in our study as in the studies by Engell and Haxby [2007] and Sato et al. [2008] were static face photographs. Furthermore, the initial eye orientation (before shifting the gaze to make eye contact) of the animated characters in the studies of Pelphrey et al. [2004] and Kuzmanovic et al. [2009] was averted to the side. We may speculate that participants had habituated to the averted gaze direction, thus, the stronger STS response when centering the gaze may ultimately be compatible with our results. This suggestion needs, however, further exploration.

Enhanced activity in FG when observing averted gaze in comparison to direct gaze found in our study appears to contradict findings by George et al. [2001] and Pageler et al. [2003]. However, these authors might likely have captured face processing in addition to gaze direction processing. Participants in the study of George et al. [2001] viewed 34 different faces. Participants in the study of Pageler et al. [2003] saw sixty different faces. Behavioral studies have demonstrated that direct gaze enhances processing and memory encoding of faces [Hood et al., 2003; Mason et al., 2004; Vuilleumier et al., 2005]. Hence, in the case of direct gaze, a deeper face processing and encoding of a novel face might produce the larger response in FG, since this brain structure is crucial for face processing [Kanwisher et al., 1997; McCarthy et al., 1997; Sergent al., 1992]. In this study, we used one single face portrait, so that the eye position was the only marked difference across conditions. This enabled us to make a direct comparison of gaze directions. Effects of memory and other aspects of face processing could thus be minimized.

It is important to note that other methodological differences make it difficult to relate our findings to existing studies. We asked our participants to perform an irrelevant eye‐color discrimination task to implicitly assess gaze perception. Contrary to this approach, participants in the study of Pageler et al. [2003] were required to accomplish an explicit gaze discrimination task. George et al. [2001] also used an irrelevant task—gender discrimination. However, Vuilleumier et al. [2005] demonstrated, also using a gender discrimination task, that the judgments were slower for faces with direct gaze than for those with averted gaze. Thus, it is likely that gender discrimination of faces with direct gaze needs more cognitive resources for face categorization and may cause increased activity in FG. We have ruled out such processing using a simple task and a constant face.

Further methodological differences consist in the viewing angle of the faces: George et al. [2001] as well as Pageler et al. [2003] both used frontal and oblique face orientations. Whereas the former study revealed stronger bilateral FG activation for direct gaze relative to averted gaze across all head angles, the latter reported an increase in FG activity for faces with direct gaze as compared to those with averted for frontal face orientation only.

The most striking difference, however, consists in the angle of gaze deflection of the presented stimuli: gaze averted by 30° and 45° in studies of George et al. [2001] and Pageler et al. [2003] respectively, versus 10° gaze deflection in our study. We chose our stimuli explicitly to be comparatively close to the edge of the gaze cone. Although the gaze directed 10° to the observer's right was easily identified as "clearly averted" by our participants, the eye deviations of 30° or even 45° might engage a different sort of processing. The decision that mutual gaze is not given is certainly much easier to make. The exploration of neural activity modulation by use of parametric stimuli with small increments between gaze directions across a wide range might be valuable for better understanding of activity in the FG (and other structures) during gaze perception. Knowing the size of the gaze cone (±5°), we may suppose three rather than two categories of gaze: mutual gaze, averted gaze, looking in an altogether different direction.

The Mona Lisa Effect

Behavioral studies have shown that a portrait's direct gaze is perceived as eye contact even when the observer is displaced [e.g., Boyarskaya and Hecht, 2012]. In this study, we presented face photographs coupled with direct gaze from three different viewpoints (central, i.e., straight ahead, shifted 28° to the observer's right, shifted 28° to the left).

We have identified brain regions that differentiated between presentation positions. Regions, responding selectively to the portrait's position were situated mainly in BA17 and BA18, extending to BA 19: superior occipital gyrus, calcarine gyrus, inferior occipital /lingual gyrus, and cuneus, thus, in primary and secondary visual cortex and in early portions of ventral and dorsal visual stream areas [see Mishkin et al., 1983]. The activation pattern found as a function of the eye position of the observer is similar to that reported by Deutschländer et al. [2005] for mere eye position. In their study, participants fixated a red LED that appeared at one of three positions: central, shifted 10° to the right, and shifted 10° to the left, for the duration of 22.5 s. The modulation of neural activity in response to observation of eye contact from different viewing angles demonstrated in our study is comparable with the eye position dependent activity modulation produced alone by fixation on a red LED straight ahead or laterally right and left. Thus, our picture produced the same activation pattern, once its position—and the associated eye position of the observing subject—is accounted for.

We found that the increased cerebral activity during fixations at laterally presented stimuli was mainly ipsilateral to the corresponding visual field (see Table 4, Fig. 4). Similar results were obtained by comparison of lateral (right and left) fixations relative to central fixations in the study carried out by Deutschländer et al. [2005]. The BOLD signal increase in the hemisphere ipsilateral to the fixation target was interpreted as less deactivation of correspondent brain areas during lateral than during central fixation. Indeed, the authors revealed deactivations in the higher‐order visual areas (lingual and fusiform gyri, posterosuperior cuneus) during 10° lateral fixation predominantly in the hemisphere contralateral to the fixation target.

Importantly, our results show that brain regions broadly agreed to be involved in gaze processing, specifically in the processing of eye contact [see e.g., Itier and Batty, 2009; Senju and Johnson, 2009], produce the same patterns for portraits viewed centrally and portraits viewed from oblique vantage points (the portraits making eye contact in all cases). Moreover, no effect of presentation position on gaze direction was present. Therefore, gaze perception was not affected by the portrait's position providing positive evidence for the Mona Lisa effect.

CONCLUSION

Our study of gaze‐contingent activation is unique in focusing on gaze per se and on picture displacement with respect to the observer, who fixated the target at all times. Taken together, we assessed gaze perception during or close to eye contact, inside, outside, and at the edge of the cone of gaze. We used an identical face all conditions (as opposed to changing facial expression or even changing the person on the picture), and we chose a very subtle manipulation (the color of blue‐gray eyes was only slightly different from the green eyes). This allowed us to probe into gaze direction perception in the range where mutual gaze is detected while minimizing the processing of other facial features. Averted gaze versus mutual gaze produced distinctly different brain activations in FG and STS. These gaze sensitive brain areas were unaffected by the portrait's presentation location, providing positive evidence for the Mona Lisa effect. Finally, activation patterns of averted gaze versus eye contact were comparable to those associated with averted gaze versus gaze at the edge of the gaze cone (directed 5° to the side). Thus, the neural response to gaze at the edge of the gaze cone (5°) is closer to that elicited by eye contact (0°) than to clearly averted gaze averted (10°). In sum, although the brain clearly distinguishes averted from direct gaze, a substantial change of vantage point does not alter neural activity, thus providing a possible explanation why the feeling of eye contact is upheld even in decentered stimulus positions.

ACKNOWLEDGMENTS

The authors would like to express gratitude and very special thanks to Agnes Münch for her programming assistance and invaluable technical support. The authors also thank Hannah Gamerdinger, Yevheniy Sukharevskyy, and Bernhard Both for their help with stimuli development, data acquisition, and analysis of behavioral data.

REFERENCES

  1. Adams RB Jr, Kleck RE (2003): Perceived gaze direction and the processing of facial displays of emotion. Psychol Sci 14:644–647. [DOI] [PubMed] [Google Scholar]
  2. Adams RB, Kleck RE (2005): Effects of direct and averted gaze on the perception of facially communicated emotion. Emotion 5:3–11. [DOI] [PubMed] [Google Scholar]
  3. Adams RB Jr, Gordon HL, Baird AA, Ambady N, Kleck RE (2003): Effects of gaze on amygdala sensitivity to anger and fear faces. Science 300:1536. [DOI] [PubMed] [Google Scholar]
  4. Adams RB, Pauker K, Weisbuch M. (2010): Looking the other way: The role of gaze direction in the cross‐race memory effect. J Exp Soc Psychol 46:478–481. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Akiyama T, Kato M, Muramatsu T, Saito F, Nakachi R, Kashima H (2006a): A deficit in discriminating gaze direction in a case with right superior temporal gyrus lesion. Neuropsychologia 44:161–170. [DOI] [PubMed] [Google Scholar]
  6. Akiyama T, Kato M, Muramatsu T, Saito F, Umeda S, Kashima H (2006b): Gaze but not arrows: A dissociative impairment after right superior temporal gyrus damage. Neuropsychologia 44:1804–1810. [DOI] [PubMed] [Google Scholar]
  7. Allison T, Puce A, McCarthy, G (2000): Social perception from visual cues: Role of the STS region. Trends Cogn Sci 4:267–278. [DOI] [PubMed] [Google Scholar]
  8. Anstis SM, Mayhew JW, Morley T (1969): The perception of where a face or television 'portrait' is looking. Am J Psychol 82:474–489. [PubMed] [Google Scholar]
  9. Argyle M, Dean, J (1965): Eye‐contact, distance and affiliation. Sociometry 28:289–304. [PubMed] [Google Scholar]
  10. Argyle M, Cook, M (1976): Gaze and Mutual Gaze. Oxford, England: Cambridge University Press. [Google Scholar]
  11. Argyle M, Ingham R, Alkema F, McCallin M (1972): The different functions of gaze. Semiotica 7:19–32. [Google Scholar]
  12. Baron‐Cohen S (1995): Mindblindness: An Essay on Autism and Theory of Mind. Cambridge, MA: MIT Press/Bradford Books. [Google Scholar]
  13. Batki A, Baron‐Cohen S, Wheelwright S, Connellan J, Ahluwalia J (2000): Is there an innate gaze module? Evidence from human neonates. Infant Behav Dev 23:223–229. [Google Scholar]
  14. Bayliss AP, Paul MA, Cannon PR, Tipper SP (2006): Gaze cuing and affective judgments of objects: I like what you look at. Psychon Bull Rev 13:1061–1066. [DOI] [PubMed] [Google Scholar]
  15. Boyarskaya E, Hecht H (2012): Differentiating the differential rotation effect. Acta Psychol 140:252–265. [DOI] [PubMed] [Google Scholar]
  16. Brewster D (1832): Letters on Natural Magic. Adressed to Sir Walter Scott, Bart. London: John Murray. [Google Scholar]
  17. Bristow D, Rees G, Frith CD (2007): Social interaction modifies neural response to gaze shifts. Soc Cogn Affect Neurosci 2:52–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Bruce V, Young AW (1998): In the Eye of the Beholder: The Science of Face Perception. Oxford, England; New York: Oxford University Press. [Google Scholar]
  19. Bruner J (1983): Child's Talk: Learning to Use Language. New York: Norton. [Google Scholar]
  20. Calder AJ, Lawrence AD, Keane J, Scott SK, Owen AM, Christoffels I, Young AW (2002): Reading the mind from eye gaze. Neuropsychologia 40:1129–1138. [DOI] [PubMed] [Google Scholar]
  21. Calder AJ, Beaver JD, Winston JS, Dolan RJ, Jenkins R, Eger E, Henson RN (2007): Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule. Curr Biol 17:20–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Campbell R, Heywood CA, Cowey A, Regard M, Landis T (1990): Sensitivity to eye gaze in prosopagnosic patients and monkeys with superior temporal sulcus ablation. Neuropsychologia 28:1123–1142. [DOI] [PubMed] [Google Scholar]
  23. Carlin JD, Calder AJ, Kriegeskorte N, Nili H, Rowe JB (2011): A head view‐invariant representation of gaze direction in anterior superior temporal sulcus. Curr Biol 21:1817–1821. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Cline MG (1967): The perception of where a person is looking. Am J Psychol 80:45–50. [PubMed] [Google Scholar]
  25. Conty L, N'Diaye K, Tijus C, George N (2007): When eye creates the contact! ERP evidence for early dissociation between direct and averted gaze motion processing. Neuropsychologia 453:3024–3037. [DOI] [PubMed] [Google Scholar]
  26. Conty L, Gimmig D, Belletier C, George N, Huguet P (2010): The cost of being watched: Stroop interference increases under concomitant eye contact. Cognition 115:133–139. [DOI] [PubMed] [Google Scholar]
  27. Deutschländer A, Marx E, Stephan T, Riedel E, Wiesmann M, Dieterich M, Brandt T (2005): Asymmetric modulation of human visual cortex activity during 10° lateral gaze (fMRI study). NeuroImage 28:4–13. [DOI] [PubMed] [Google Scholar]
  28. Driver J, Davis G, Ricciardelli P, Kidd P, Maxwell E, Baron‐Cohen S (1999): Gaze perception triggers reflexive visuospatial orienting. Vis Cogn 6:509–540. [Google Scholar]
  29. Dunham PJ, Moore C (1995): Current themes in research of joint attention In: Moore C, Dunham P. editors. Joint Attention: Its Origin and Role in Development. Hillsadle, NJ: Lawrence Erlbaum; pp 15–28. [Google Scholar]
  30. Emery N (2000): The eyes have it: The neuroethology, function and evolution of social gaze. Neurosci Biobehav Rev 24:581–604. [DOI] [PubMed] [Google Scholar]
  31. Engell AD, Haxby JV (2007). Facial expression and gaze‐direction in human superior temporal sulcus. Neuropsychologia 45:3234–3241. [DOI] [PubMed] [Google Scholar]
  32. Ethofer T, Gschwind M, Vuilleumier P (2011): Processing social aspects of human gaze: A combined fMRI‐DTI study. NeuroImage 55:411–419. [DOI] [PubMed] [Google Scholar]
  33. Ewing L, Rhodes G, Pellicano E (2010): Have you got the look? Gaze direction affects judgements of facial attractiveness. Vis Cogn 18:321–330. [Google Scholar]
  34. Farroni T, Csibra G, Simion F, Johnson M (2002): Eye contact detection in humans from birth. Proc Natl Acad Sci USA 99:9602–9605. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Farroni T, Pividori D, Simion F, Massaccesi S, Johnson MH (2004): Eye gaze cueing of attention in newborns. Infancy 5:39–60. [Google Scholar]
  36. Farroni T, Menon E, Johnson MH (2006): Factors influencing newborns' preference for faces with eye contact. J Exp Child Psychol 95:298–308. [DOI] [PubMed] [Google Scholar]
  37. Friesen CK, Kingstone A (1998): The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychon Bull Rev 5:490–495. [Google Scholar]
  38. Frischen A, Bayliss AP, Tipper SP (2007): Gaze cueing of attention: Visual attention, social cognition, and individual differences. Psychol Bull 133:694–724. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Gale A, Lucas B, Nissim R, Harpham B (1972): Some EEG correlates of face‐to‐face contact. Br J Soc Clin Psychol 11:326–332. [DOI] [PubMed] [Google Scholar]
  40. Gale A, Spratt G, Chapman AJ, Smallbone A (1975): EEG correlates of eye contact and interpersonal distance. Biol Psychol 3:237–245. [DOI] [PubMed] [Google Scholar]
  41. Gamer M, Hecht H (2007): Are you looking at me? Measuring the cone of gaze. J Exp Psychol: Hum Percept Perform 33:705–715. [DOI] [PubMed] [Google Scholar]
  42. Ganel T, Goshen‐Gottstein Y, Goodale MA (2005): Interactions between the processing of gaze direction and facial expression. Vision Res 45:1191–1200. [DOI] [PubMed] [Google Scholar]
  43. George N, Conty L (2008): Facing the gaze of others. Neurophysiol Clin 38:197–207. [DOI] [PubMed] [Google Scholar]
  44. George N, Driver J, Dolan RJ (2001): Seen gaze‐direction modulates fusiform activity and its coupling with other brain areas during face processing. NeuroImage 13:1102–1112. [DOI] [PubMed] [Google Scholar]
  45. Gibson JJ, Pick AD (1963): Perception of another person's looking behaviour. Am J Psychol 76:386–394. [PubMed] [Google Scholar]
  46. Gläscher J, Tüscher O, Weiller C, Büchel C (2004): Elevated responses to constant facial emotions in different faces in the human amygdala: An fMRI study of facial identity and expression. BMC Neurosci 5:45. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Goldstein EB (1987): Spatial layout, orientation relative to the observer, and perceived projection in pictures viewed at an angle. J Exp Psychol: Hum Percept Perform 13:256–266. [DOI] [PubMed] [Google Scholar]
  48. Greenacre M (2007): Correspondence analysis in practice, 2nd ed. London: Chapman & Hall/CRC. [Google Scholar]
  49. Grosbras M‐H, Laird AR, Paus T (2005): Cortical regions involved in eye movements, shifts of attention, and gaze perception. Hum Brain Mapp 25:140–154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Hadjikhani N, Hoge R, Snyder J, de Gelder B (2008): Pointing with the eyes: The role of gaze in communicating danger. Brain Cogn 68:1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Hecht H, Boyarskaya E, Kitaoka A: The Mona Lisa effect: Testing the limits of perceptual robustness vis‐à‐vis slanted images. (in press).
  52. Helminen TM, Kaasinen SM, Hietanen JK (2011): Eye contact and arousal: The effects of stimulus duration. Biol Psychol 88:124–130. [DOI] [PubMed] [Google Scholar]
  53. Hietanen JK, Leppänen JM, Peltola MJ, Linna‐aho, K , Ruuhiala HJ (2008): Seeing direct and averted gaze activates the approach–avoidance motivational brain systems. Neuropsychologia 46:2423–2430. [DOI] [PubMed] [Google Scholar]
  54. Hoffman EA, Haxby JV (2000): Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nat Neurosci 3:80–84. [DOI] [PubMed] [Google Scholar]
  55. Hood BM, Willen JD, Driver J (1998): Adult's eyes trigger shifts of visual attention in human infants. Psychol Sci 9:131–134. [Google Scholar]
  56. Hood BM, Macrae CN, Cole‐Davies V, Dias M (2003): Eye remember you: the effects of gaze direction on face recognition in children and adults. Dev Sci 6:67–71. [Google Scholar]
  57. Hooker CI, Paller KA, Gitelman DR, Parrish TB, Mesulam MM, Reber PJ (2003) Brain networks for analyzing eye gaze. Cogn Brain Res 17:406–418. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Itier RJ, Batty M (2009): Neural bases of eye and gaze processing: The core of social cognition. Neurosci Biobeh Rev 33:843–863. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Itier RJ, Latinus M, Taylor MJ (2006): Face, eye and object early processing: What is the face specificity? NeuroImage 29:667–676. [DOI] [PubMed] [Google Scholar]
  60. Itier RJ, Alain C, Sedore K, McIntosh AR (2007a): Early face processing specificity: It's in the eyes! J Cog Neurosci 19:1815–1826. [DOI] [PubMed] [Google Scholar]
  61. Itier RJ, Villate C, Ryan JD (2007b): Eyes always attract attention but gaze orienting is task‐dependent: Evidence from eye movement monitoring. Neuropsychologia 45:1019–1028. [DOI] [PubMed] [Google Scholar]
  62. Janik SW, Wellens AR, Goldberg ML, Dell'Osso LF (1978): Eyes as the center of focus in the visual examination of human faces. Percept Mot Skills 47(Pt 1):857–858. [DOI] [PubMed] [Google Scholar]
  63. Jenkins R, Beaver JD, Calder AJ (2006): I thought you were looking at me: direction‐specific aftereffects in gaze perception. Psychol Sci 17:506–513. [DOI] [PubMed] [Google Scholar]
  64. Kampe KK, Frith CD, Dolan RJ, Frith U (2001): Reward value of attractiveness and gaze. Nature 413:589. [DOI] [PubMed] [Google Scholar]
  65. Kampe KK, Frith CD, Frith U (2003): "Hey John": signals conveying communicative intention toward the self activate brain regions associated with "mentalizing," regardless of modality. J Neurosci 23:5258–5263. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Kanwisher N, McDermott J, Chun MM (1997): The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neurosci 17:4302–4311. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Kawashima R, Sugiura M, Kato T, Nakamura A, Hatano K, Ito K, Fukuda H, Kojima S, Nakamura K (1999): The human amygdala plays an important role in gaze monitoring: A PET study. Brain 122:779–783. [DOI] [PubMed] [Google Scholar]
  68. Kendon A (1967): Some functions of gaze‐direction in social interaction. Acta Psychol 26:22–63. [DOI] [PubMed] [Google Scholar]
  69. Kleinke CL (1986): Gaze and eye contact: a research review. Psychol Bull, 100: 78–100. [PubMed] [Google Scholar]
  70. Kobayashi H, Kohshima S (2001a): Unique morphology of the human eye. Nature 387:767–768. [DOI] [PubMed] [Google Scholar]
  71. Kobayashi H, Kohshima S (2001b): Unique morphology of the human eye and its adaptive meaning: Comparative studies on external morphology of the primate eye. J Hum Evol 40:419–435. [DOI] [PubMed] [Google Scholar]
  72. Kuzmanovic B, Georgesc, AL , Eickhoff SB, Shah NJ, Bente G, Fink GR, Vogeley K (2009): Duration matters: Dissociating neural correlates of detection and evaluation of social gaze. NeuroImage 46:1154–1163. [DOI] [PubMed] [Google Scholar]
  73. Laube I, Kamphuis S, Dicke PW, Thier P (2011): Cortical processing of head‐ and eye‐gaze cues guiding joint social attention. NeuroImage 54:1643–1653. [DOI] [PubMed] [Google Scholar]
  74. Macrae CN, Hood B, Milne AB, Rowe AC, Mason MF (2002): Are you looking at me? Eye gaze and person perception. Psychol Sci 13:460–464. [DOI] [PubMed] [Google Scholar]
  75. Maruyama K, Endo M, Sakurai K (1985): An experimental consideration on “Mona Lisa gaze effect”. Tohoku Psychologica Folia 44:109–121. [Google Scholar]
  76. Mason MF, Hood BM, Macrae CN (2004): Look into my eyes: Gaze direction and person memory. Memory 12:637–643. [DOI] [PubMed] [Google Scholar]
  77. Mason MF, Tatkow EP, Macrae CN (2005): The look of love. Psychol Sci 16:236–239. [DOI] [PubMed] [Google Scholar]
  78. Mathews A, Fox E, Yiend J, Calder AJ (2003): The face of fear: Effects of eye gaze and emotion on visual attention. Vis Cogn 10:823–835. [DOI] [PMC free article] [PubMed] [Google Scholar]
  79. Mazur A, Rosa E, Faupel M, Heller J, Leen R, Thurman B (1980): Physiological aspects of communication via mutual gaze. AJS 86:50–74. [DOI] [PubMed] [Google Scholar]
  80. McCarthy G, Puce A, Gore JC, Allison T (1997): Face‐specific processing in the human fusiform gyrus. J Cog Neurosci 9:605–610. [DOI] [PubMed] [Google Scholar]
  81. Mishkin M, Ungerleider LG, Macko KA (1983) Object vision and spatial vision: two cortical pathways. Trends Neurosci 6:414–417. [Google Scholar]
  82. Nichols K, Champness B (1971): Eye gaze and the GSR. J Exp Soc Psychol 7:623–626. [Google Scholar]
  83. Nichols T, Brett M, Andersson J, Wager T, Poline JB (2005): Valid conjunction inference with the minimum statistic. NeuroImage 25:653–660. [DOI] [PubMed] [Google Scholar]
  84. Nummenmaa L, Calder AJ (2009): Neural mechanisms of social attention. Trends Cogn Sci 13:135–143. [DOI] [PubMed] [Google Scholar]
  85. Nummenmaa L, Passamonti L, Rowe J, Engell AD, Calder AJ (2010): Connectivity analysis reveals a cortical Network for eye gaze perception. Cereb Cortex 20:1780–1787. [DOI] [PMC free article] [PubMed] [Google Scholar]
  86. Pageler NM, Menon V, Merin NM, Eliez S, Brown WE, Reiss AL (2003): Effect of head orientation on gaze processing in fusiform gyrus and superior temporal sulcus. NeuroImage 20:318–329. [DOI] [PubMed] [Google Scholar]
  87. Pelphrey KA, Singerman JD, Allison T, McCarthy G (2003): Brain activation evoked by perception of gaze shifts: The influence of context. Neuropsychologia 41:156–170. [DOI] [PubMed] [Google Scholar]
  88. Pelphrey KA, Viola RJ, McCarthy G (2004): When strangers pass. Psychol Sci 15:598–603. [DOI] [PubMed] [Google Scholar]
  89. Perrett DI, Smith PA, Potter DD, Mistlin AJ, Head AS, Milner AD, Jeeves MA (1985): Visual cells in the temporal cortex sensitive to face view and gaze direction. Proc Soc Lond B Biol Sci 223:293–317. [DOI] [PubMed] [Google Scholar]
  90. Perrett DI, Hietanen JK, Oram MW, Benson PJ (1992): Organization and functions of cells responsive to faces in the temporal cortex. Philos Trans R Soc Lon B Biol Sci 335: 23–30. [DOI] [PubMed] [Google Scholar]
  91. Puce A, Allison T, Bentin S, Gore JC, McCarthy G (1998): Temporal cortex activation in humans viewing eye and mouth movements. J Neurosci 18:2188–2199. [DOI] [PMC free article] [PubMed] [Google Scholar]
  92. Ristic J, Kingstone A (2005): Taking control of reflexive social attention. Cognition 94:B55–B65. [DOI] [PubMed] [Google Scholar]
  93. Rogers S, Lunsford M, Strother L, Kubovy M (2003): The Mona Lisa effect: Perception of gaze direction in real and pictured faces In: Rogers S, Effken J, editors. Studies in Perception and Action VII. Mahwah: Lawrence Erlbaum Associates, Inc; pp 19–24. [Google Scholar]
  94. Sander D, Grandjean D, Kaiser S, Wehrle T, Scherer KR (2007): Interaction effects of perceived gaze direction and dynamic facial expression: Evidence for appraisal theories of emotion. Eur J Cogn Psychol 19:470–480. [Google Scholar]
  95. Sato W, Yoshikawa S, Kochiyama T, Matsumura M (2004): The amygdala processes the emotional significance of facial expressions: An fMRI investigation using the interaction between expression and face direction. NeuroImage 22:1006–1013. [DOI] [PubMed] [Google Scholar]
  96. Sato W, Kochiyama T, Uono S, Yoshikawa S (2008): Time course of superior temporal sulcus activity in response to eye gaze: A combined fMRI and MEG study. Soc Cogn Affect Neurosci 3:224–232. [DOI] [PMC free article] [PubMed] [Google Scholar]
  97. Sato W, Kochiyama T, Uono S, Yoshikawa S (2010): Amygdala integrates emotional expression and gaze direction in response to dynamic facial expressions. NeuroImage 50:1658–1665. [DOI] [PubMed] [Google Scholar]
  98. Schilbach L, Wohlschlaeger AM, Kraemer NC, Newen A, Shah NJ, Fink GR, Vogeley K (2006): Being with virtual others: Neural correlates of social interaction. Neuropsychologia 44:718–730. [DOI] [PubMed] [Google Scholar]
  99. Senju A, Hasegawa T (2005): Direct gaze captures visuospatial attention. Vis Cogn 12:127–144. [Google Scholar]
  100. Senju A, Johnson MH (2009): The eye contact effect: mechanisms and development. Trends Cogn Sci 13:127–134. [DOI] [PubMed] [Google Scholar]
  101. Sergent J, Ohta S, MacDonald B (1992): Functional neuroanatomy of face and object processing. A positron emission tomography study. Brain 115 (Pt 1):15–36. [DOI] [PubMed] [Google Scholar]
  102. Smith A, Hood B, Hector K (2006): Eye remember you two: Gaze direction modulates face recognition in a developmental study. Dev Sci 9:465–472. [DOI] [PubMed] [Google Scholar]
  103. Straube T, Langohr B, Schmidt S, Mentzel H‐J, Miltner WH (2010): Increased amygdala activation to averted versus direct gaze in humans is independent of valence of facial expression. NeuroImage 49:2680–2686. [DOI] [PubMed] [Google Scholar]
  104. Strick M, Holland RW, van Knippenberg A (2008): Seductive eyes: Attractiveness and direct gaze increase desire for associated objects. Cognition 106:1487–1496. [DOI] [PubMed] [Google Scholar]
  105. Symons LA, Lee K, Cedrone CC, Nishimura M (2004): What are you looking at? Acuity for triadic eye gaze. J Gen Psychol 131:451–469. [PMC free article] [PubMed] [Google Scholar]
  106. Tomasello M, Farrar MJ (1986): Joint attention and early language. Child Dev 57:1454–1463. [PubMed] [Google Scholar]
  107. Tomasello M, Hare B, Lehmann H, Call J (2007): Reliance on head versus eyes in the gaze following of great apes and human infants: The cooperative eye hypothesis. J Hum Evol 52:314–320. [DOI] [PubMed] [Google Scholar]
  108. Vecera SP, Rizzo M (2004): What are you looking at? Impaired ‘social attention’ following frontal‐lobe damage. Neuropsychologia 42:1657–1665. [DOI] [PubMed] [Google Scholar]
  109. Vecera SP, Rizzo M (2006): Eye gaze does not produce reflexive shifts of attention: Evidence from frontal‐lobe damage. Neuropsychologia 44:150–159. [DOI] [PubMed] [Google Scholar]
  110. Vinette C, Gosselin F, Schyns P (2004): Spatio‐temporal dynamics of face recognition in a flash: It's in the eyes. Cogn Sci 28, 289–301. [Google Scholar]
  111. von Cranach M, Ellgring JH (1973): Problems in the recognition of gaze direction In: von Cranach M, Vine I, editors. Social Communication and Movement: Studies of Interaction and Expression in Man and Chimpanzee. London: Academic Press; pp 419–443. [Google Scholar]
  112. Vuilleumier P, George N, Lister V, Armony J, Driver J (2005): Effects of perceived mutual gaze and gender on face processing and recognition memory. Vis Cogn 12:85–101. [Google Scholar]
  113. Watanabe S, Kakigi R, Puce A (2001): Occipitotemporal activity elicited by viewing eye movements: A magnetoencephalographic study. NeuroImage 13:351–363. [DOI] [PubMed] [Google Scholar]
  114. Wicker B, Michel F, Henaff MA, Decety J (1998): Brain regions involved in the perception of gaze: A PET study. NeuroImage 8:221–227. [DOI] [PubMed] [Google Scholar]
  115. Wicker B, Perrett DI, Baron‐Cohen S, Decety J (2003): Being the target of another's emotion: A PET study. Neuropsychologia 41:139–146. [DOI] [PubMed] [Google Scholar]
  116. Wyland CL, Forgas JP (2010): Here's looking at you kid: Mood effects on processing eye gaze as a heuristic cue. Soc Cogn 28:133–144. [Google Scholar]
  117. Yarbus, AL (1967): Eye Movements and Vision. New York: Plenum Press. [Google Scholar]

Articles from Human Brain Mapping are provided here courtesy of Wiley

RESOURCES