Videos of conspecifics elicit interactive looking patterns and facial expressions in monkeys

Abstract

A broader understanding of the neural basis of social behavior in primates requires the use of species-specific stimuli that elicit spontaneous, but reproducible and tractable behaviors. In this context of natural behaviors, individual variation can further inform about the factors that influence social interactions. To approximate natural social interactions similar to those documented by field studies, we used unedited video footage to induce in viewer monkeys spontaneous facial expressions and looking patterns in the laboratory setting. Three adult male monkeys, previously behaviorally and genetically (5-HTTLPR) characterized (Gibboni et al., 2009), were monitored while they watched 10 s video segments depicting unfamiliar monkeys (movie monkeys) displaying affiliative, neutral, and aggressive behaviors. The gaze and head orientation of the movie monkeys alternated between ‘averted’ and ‘directed’ at the viewer. The viewers were not reinforced for watching the movies, thus their looking patterns indicated their interest and social engagement with the stimuli. The behavior of the movie monkey accounted for differences in the looking patterns and facial expressions displayed by the viewers. We also found multiple significant differences in the behavior of the viewers that correlated with their interest in these stimuli. These socially relevant dynamic stimuli elicited spontaneous social behaviors, such as eye-contact induced reciprocation of facial expression, gaze aversion, and gaze following, that were previously not observed in response to static images. This approach opens a unique opportunity to understanding the mechanisms that trigger spontaneous social behaviors in humans and non-human primates.

Introduction

Behavioral neuroscience is most powerful when discoveries extend past the artificial confines of the laboratory to explain how an organism's brain functions in its natural setting (Høgh-Olsen, 2006; Kingstone, 2009; Shepherd, Steckenfinger, Hasson, & Ghazanfar, 2010). The major challenge with ethologically valid experiments is identifying those behaviors that are natural to the species and can be reproduced in the laboratory. A good compromise between a reductionist's well-controlled experiments and the spontaneity of natural behavior exploits those mechanisms that were built into the nervous system during the evolution of the species. Because an individual's neural mechanisms depend in part on its unique ontogeny, an ethological approach should consider those behaviors that are both consistent among all individuals of a species as well as unique to a subset of individuals, enlarging the spectrum of factors that determine naturally-occurring behaviors.

In primate societies the survival of the individual depends on adequate decoding and production of social signals, so much that they are considered the primary selective pressure in the evolution of the primate brain (Shultz & Dunbar, 2007). Primates have probably acquired neural specializations for the accurate decoding of social signals that inform about the emotions and intentions of others (Adolphs, 2009; Reader & Laland, 2002). The sophisticated perceptual processing of social inputs is matched by a response system that elaborates nuanced social signals by combining elements from a repertoire of displays that include facial expressions, vocalizations, postures, and gestures. The meaning of these signals depends both on the ‘emitter’ and the ‘receiver’ and also on the context in which the exchange of social signals takes place (Chevalier-Skolnikoff, 1973; Darwin, 1872; Hinde & Rowell, 1962; Redican, 1975; van Hoff, 1967). The eye movements or looking patterns of monkeys during social interactions are informative for both the perceptual and response components of social cognition.

The analysis of scanpaths (the sequence of saccades and fixations) on static images of faces has provided rich information about how monkeys process social signals. When viewing facial expressions monkeys selectively attend to different facial features; for example, the mouth of the fear-grimace is looked at more than the mouth of a neutral expression (Chance, 1967; Dittrich & Lea, 1994; Gibboni, Zimmerman, & Gothard, 2009; Gothard, Battaglia, Erickson, Spitler, & Amaral, 2007; Gothard, Erickson, & Amaral, 2004; Kaufmann, 1967; Maestripieri, 1997). The value of these observations notwithstanding, static images are poor predictors of where a monkey might look during a real-life encounter. It is highly unlikely, for example, that in real life a fear-grimacing monkey would maintain eye contact (direct gaze) with the intended recipient of this display for several seconds, especially if the recipient is higher ranking. In this particular case, changes of head and gaze direction toward and away from the eyes of the recipient not only enhances the intended signal of submission, but also creates windows of opportunity for the receiver to reciprocate (or not) direct gaze. Accordingly, eye movements during natural social interactions are expected to reveal how monkeys process emotional stimuli (e.g., which facial features they attend to) and also how they use eye gaze shifts to communicate their emotions and intentions (e.g., to signal submission).

In the current study we sought to expand on the results of our previous static image-viewing studies and explored, in addition to scanpaths, the behavioral responses of monkeys to video stimuli of conspecifics. These experiments rest on the premise that monkeys are engaged by videos and are even willing to forgo reward to view social stimuli (Andrews & Rosenblum, 1993; Berg, Boehnke, Marino, Munoz, & Itti, 2009; Nahm, Perret, Amaral, & Albright, 1997; Shepherd, et al., 2010). We expanded on earlier observation of looking patterns of monkeys on videos by adding a strong social component – displays of appeasing, neutral, and aggressive behaviors. We hypothesized that videos with species-specific content would thus elicit looking patterns and behaviors comparable to those reported by field studies (Chance, 1967; Chevalier-Skolnikoff, 1973; Maestripieri, 1997; Redican, 1975; Shepherd & Platt, 2008; Tomasello, Call, & Hare, 1998). To enhance our ability to capture facial expressions that might be used by the viewer monkeys to ‘interact’ with the monkey depicted in the videos, we also monitored the viewers' facial expressions and the electromyographic activity of several muscles. We hypothesized that eye contact between the movie monkey and the viewer monkey would results in partial or fully fledged facial displays as the viewer “interacted” with the movie monkey.

Scanpaths inform about the content of the stimulus and also about the viewer. Gender, age, social status, and emotional state have been show to contribute to the variability of scanpaths in monkeys (Gibboni, et al., 2009; Klein, Shepherd, & Platt, 2009; Tsuchida & Izumi, 2009) and in humans (e.g., Perlman et al., 2009) In addition to exploring the social determinants of scanpaths, we investigated how socially-triggered looking behaviors varied with the sociability of the viewer monkeys. Monkeys exhibit perceptual bias towards images of their own species (Gothard, Brooks, & Peterson, 2009; Leopold & Rhodes, 2010) and prefer looking at, or interacting with, certain individuals (Deaner, Khera, & Platt, 2005). Based on our previous studies (Gibboni, et al., 2009), that correlated the viewer's 5-HTTLPR genotype, early life experience and other behavioral tendencies with looking patterns or static images we predicted large differences in the subjects' movie viewing preferences.

The use of videos with social content to elicit looking patterns and facial expressions in monkeys was motivated by the need to take natural behaviors observed in the field and make them tractable in a laboratory setting. Such an approach allows us to exert experimental control “while keeping one foot firmly placed in real experience” (Kingstone, 2009). At the same time these experiments prepare the groundwork for future studies in exploring the neural bases of social and emotional perception.

Methods

Subjects

Scanpaths (the sequence of eye movements and fixations) were recorded from 3 adult male monkeys (Macaca mulatta): Q, V, and T. The life history, dominance status, motivation, and serotonin transporter linked polymorphic region (5-HTTLPR) of each monkey has been previously characterized (Gibboni, et al., 2009) and is summarized in Table 1. At the time of the study, all monkeys were between 8-13 years of age and weighed 10-14 kg. Initially the monkeys were pair-housed with partners others than the subjects of these experiments. Eventually each subject monkey got into fights with his cage mate (T with H, V with M, and Q with H) and from this point on each monkey was housed alone, in a double cage, with visual access to all other monkeys in the colony.

Table 1. Biographical, genetic, and temperamental characteristics of the viewer monkeys.

	Age	5HT-gen.	Rearing	Dominance	Motivation	Risk-aversion
Q	10 years	l/l	Mother-reared in large outdoor troupe (up to the age of 1 year)	Dominant over T and V. Aggressive towards certain caretakers.	Highly motivated to improve performance. Responds to verbal praise as well as food reward. Fast learner.	Initially avoids novel potentially harmful objects (rubber snake). Overcomes this aversion to achieve food reward and even manipulates object.
V	8 years	l/s	Mother-reared in large outdoor troupe (up to the age of 1 year)	V aggresses Q and T. Very submissive to caretakers.	Least-experienced worker. Somewhat motivated by food reward. Unmotivated by verbal praise. Intermediate learner.	Exhibits greatest avoidance of novel potentially harmful objects, but eventually overcomes this aversion for food reward.
T	13 years	s/s	Nursery-reared, and then transferred to indoor housing	Appears to be unconcerned with hierarchy. Submissive towards caretakers.	Highly motivated by food reward. During training is impatient and rushes to action. Perpetually paces in cage. Slow learner.	Completely disregards novel potentially harmful objects to obtain food-reward.

Surgery

All experimental procedures were performed in compliance with the guidelines of the National Institutes of Health for the use of primates in research and were approved by the Institutional Animal Care and Use Committee at the University of Arizona. In order to record eye movements, each monkey was fitted with a 3-point head fixation device attached to the skull under isoflurane anesthesia (Gothard, et al., 2007). During the same surgery, 3 electrodes were placed intramuscularly in different compartments of the left auricularis muscle (Waller, Parr, Gothard, Burrows, & Fuglevand, 2008) and attached to a connector affixed to the skull, allowing intramuscular electromyographic recordings (EMG).

Stimuli

Subjects viewed movies of unfamiliar monkeys (4 female, 6 male) ranging in age from 3-20 years. The monkeys depicted in the movies were filmed in a cage with a Plexiglass front as they gestured toward and away from a video camera (for more details on the stimuli, see (Gothard, et al., 2007; Gothard, et al., 2004). Each stimulus monkey was shown in 3 movies, displaying affiliative (lipsmack), neutral, and agonistic (open-mouth threat) expressions, yielding a total of 30 stimulus movies (10 monkey identities × 3 expressions = 30 movies). Movies spanned 10 s of continuous unedited footage in which there was minimal movement by the camera. In the course of the 10 s the movie monkeys look either away or directly at the camera generating segments when the viewer perceives either averted or direct gaze. Examples of movies with superimposed eye movements are shown in Supplementary Movie 1.

Behavioral task and recording procedures

During each data collection session, 6 movies (2 monkeys × 3 expressions) were chosen from the total of 30 movies and presented in pseudorandom sequence. When all 6 movies had been presented once, the block of movies was repeated in a new pseudorandom sequence until a total of 7 repetitions of each movie had been obtained.

Subject monkeys were seated in a custom-built primate chair with their eyes at 57 cm from an LCD monitor spanning 37×28 degrees of visual angle (dva) with a refresh rate of 60 Hz. A webcam (Logitech Quickcam, 30 fps, 8 megapixel resolution) was positioned in front and to the left of the monkey to capture the viewer's facial expressions. Prior to presentation of a movie, a visual cue of 10 × 9 dva was presented for a random duration of 750-1250 ms. Following the cue, a blank screen was shown for 1 s, and then the movie was presented. Each movie spanned 26 × 18 dva and consisted of 299 frames displayed at a resolution of 30 fps (10 sec). Each movie was followed by a 9-15 s inter-movie-interval; during this interval the screen remained blank. NBS Presentation software (Albany, CA) was used for cue and movie presentation. During the task, monkeys were free to look anywhere on or off the monitor. No reward or punishment was delivered; the monkeys were motivated only by their interest in the movie content.

Prior to the experiment, monkeys were trained on a 9 point calibration with a precision of ±1 dva. Scanpaths were recorded using an infrared camera with a sampling rate of 240 Hz (ISCAN Inc., Woburn, MA) and collected as an analog signal through a CED Power 1401 data acquisition system and Spike 2 software (Cambridge Electronic Devices, UK). The display time of each movie frame was digitally sent by Presentation and encoded by the Power 1401. EMG was sampled at 2 kHz and the webcam video was time-locked to the Spike 2 data file.

Data analysis

Ethograms were used to quantify the content of the stimulus videos as well as the scanpaths and facial expressions of the viewer monkey (Fig. 1).

Figure 1. Example ethograms.

(A) Ethogram of a 10 s movie depicting a lipsmacking 3 year-old female monkey. The four traces quantify: movement (purple), facial expression (grey), direction of eye gaze (pink), and head direction (aqua) of the movie monkey in each frame of this movie. (B) Ethogram quantifying the target of the viewer's fixation on each frame of the movie. Each viewer monkey (Q: top, V: middle, and T: bottom panel) looked at the same movie seven times. Note the similarity of looking patterns between the trials from the same viewer and also between different viewers. For example, the yellow band at 3.5s indicates that monkey Q consistently looked at the chest of the movie monkey on all seven trials. When monkeys V and T viewed the same movie they also tended to look at the chest of the movie monkey around the same time. (C) Ethogram showing the time periods during which the viewer monkeys (Q: top, V: middle, and T: bottom) lipsmacked (orange) during the same movie over the course of seven trials.

Ethogram of the movie monkey's behavior

The behavior of the stimulus monkey was quantified in a custom ethogram with 33 ms resolution (1 frame = 33 ms). Each frame was scored for eye direction (direct, averted) and facial expression (neutral, lipsmack, or threat). Head direction was quantified as the angle formed by placing points centered on each eye orbit and between the nostrils (Fig 1 A). If this eye-nose-eye angle was between 50-70 degrees, the head was quantified as being directed at the viewer, which agreed with qualitative observation. An angle greater than this indicated that the monkey was looking up or down whereas an angle near 0 indicated complete profile in either the left or right direction. The distance that the eye position moved from one frame to the next gave a frame-by-frame measure of the amount of motion in the video.

Ethogram of the viewer monkey's scanpath

Fixations and saccades subtending more than 2 dva were distinguished in the eye position data using custom MATLAB scripts (The MathWorks Inc., MA, USA). Each fixation point was user-verified and blinks were excluded from the analysis. For each movie frame presented, a researcher manually identified the region upon which the viewer monkey fixated: (eyes, brows, mouth, nose/midface, ears, cheeks/jaw, top of head, chest/nipples, stomach, hands, feet, arms, legs, perineum, genitals, tail, back, cage, outside of movie frame). Scoring of the regions fixated in each frame was performed by three independent raters who agreed in greater than 90% of cases (Fig 1 B).

Ethogram of the viewer monkey's facial expressions

For each frame of the time-locked webcam recording of the viewer monkey, a researcher identified whether or not the viewer was making an overt lipsmack display (puckering of the lips). The occurrence of a lipsmack was additionally verified as an increase in the amplitude of the auricularis EMG that exceeded more than 2 standard deviations (∼0.2 mV) relative to the resting amplitude. This deflection in the EMG correlates with the viewer monkey pulling his ears back against his head, a characteristic component of the lipsmack expression (see Supplementary Movie 2 and Fig. 1 C). The subject's jaw movements were limited by the collar that was held in the yoke of the primate chair, therefore yawns and open-mouth threats were not possible while in head immobilization.

Statistics

Unless otherwise stated in the text, the following guidelines were used in the statistical analyses of data: for group comparisons of more than two categories (viewer monkey, movie monkey expression, etc.) standard analysis of variance (ANOVA) tests were used and post-hoc Tukey-Kramer tests were applied. Paired two-tail t-tests were used to compare data of two test conditions (e.g., left vs. right side face looking). Chi-squared test statistics (χ²) were used to compare the probability of occurrences (e.g., percent viewer lipsmacking to movies of different expressions). In all statistical tests, significance levels were set to α= 0.05.

Results

General scanpath properties

Although the monkeys were neither rewarded nor punished, all three monkeys were visually attentive to the movies. For each monkey the looking pattern varied from the first to the last trial (Fig. 2A) and there were large differences in looking patterns between viewers Fig. 2B, Supplementary Movie 1).

Figure 2. Changes in looking time and scanpath dispersion.

(A) Example scanpaths of monkey Q as he viewed a movie of a threatening 20 year-old male. For illustration purposes, scanpaths are superimposed on an image constructed from different frames of the movie. During the first trial (left panel), monkey Q kept his eyes primarily on the movie-monkey. As the trials progressed (middle and right panels), the viewer's scanpath became gradually more dispersed. The viewer spent less time exploring the movie monkey's face and frequently looked away from the movie screen. The color bar on the left indicates the time of each saccade relative to ten, 1 s periods. (B) Mean looking time (expressed as % of total looking) calculated for each viewer monkey averaged across all 30 movies, during the course of seven trials. From the first trial, monkey Q spent significantly more time watching the movies than monkeys T and V (p <0.05). Starting with the second trial, all viewers showed a decrease in looking time (p<0.05). However, the extent of this decrease was viewer-dependent. By the last trial, monkey Q was still watching the movies 50% of the time, whereas monkeys T and V looked less than 15 % of the time. (C) Scanpath dispersion calculated for each monkey averaged across all 30 movies over the course of seven trials. Dispersion was computed as the standard deviation of all fixation positions during a single movie presentation. If the viewer spent less than 15 % of the time viewing the movie, dispersion was not computed. Scanpaths of monkeys Q and T were significantly more dispersed on the second compared to the first trial (p<0.05). Dispersion increased monotonically until the final trial. This last trial, however, was still significantly more dispersed than the first (p<0.05).

During the first trial, monkey Q looked significantly longer at the movies than the other two monkeys (one-way ANOVA on total looking time with factor = viewer monkey; F_2,87=10.896, p=0.000; Fig. 2B). As the movies were repeated, all three monkeys looked less, though monkey Q exhibited the least attenuation in viewing time (Fig. 2A, B). Not only was looking time decreased during the second trial, each monkey also tended to look at different movie areas upon trial repetition (% of frames where the scanpath is within 2 dva during first and second trial; Q: 52±17%, V: 22±20%, T: 12±8%). During the initial presentation of a given movie, the three monkeys tended to examine the same area (within 2 dva) of a given frame 20±15% of the time (range = 0-51%). While the viewers' scanpaths tended to concentrate around the movie monkey's face during the first trial, they dispersed as the trials progressed and the viewers attended to other regions (e.g., the body of the movie monkey or the cage) (Fig. 2A, B). During the first trial, the scanpaths of monkeys T and V were more dispersed than that of monkey Q (one-way ANOVA on dispersion with factor = subjects, F_2,87=5.04, p =0.009, Fig. 2C), suggesting that these two monkeys looked less overall at the movie monkey's face and instead explored other regions of the movies. The observed increase in dispersion and decrease in looking time with repeated viewing of the same movie might reflect habituation/boredom for a given movie manifested in a shift of the viewer's attention to unexplored regions of the movie, but it is also possible that with trial repetition, the viewers are better able to predict the content of the movies and choose to actively avoid the movie monkey. To distinguish between these possibilities, the relationship between looking time and the expression of the movie monkey was examined.

Visual exploration of facial features

When the results of 7 repetitions of all 30 movies were combined, we found a significant variability in the total looking time among viewers (mean percent looking for all trials: Q=72.4%, V=16.0%, T=27.6%), and also a large variability in the looking time elicited by different movies (standard deviation of total movie looking: Q=±11.9%, V=±5.6%, T=±14.7%).

Although the facial expressions of the movie monkeys did not influence total looking-time (one-way ANOVAs on total looking time with factor = expression, Q: F_2,27=1.61, p=0.219; V: F_2,27=0.44, p=0.646; T: F_2,27=0.37, p=0.694), an expression-dependence existed at a finer scale. Further analysis of scanpath-dispersion indicated that monkey Q looked at the face of the movie monkey for a significantly greater portion of time than monkey T, who looked less at the face and instead explored other regions of the movie. (Fig. 3A, two-way ANOVA on proportion of face looking during first trial with factors = viewer × movie monkey expression, main effect of viewer F(1)_2,87=12.24, p=0.000). In an attempt to determine what area of the face was explored by each monkey, the amount of time spent looking at the left vs. right side of the face was quantified. This measure was motivated by the observation that healthy human subjects tend to spend a greater portion of time looking at the left side of the image (i.e., in anatomical position the right side of the stimulus subject's face), an effect that is exacerbated when viewing emotional facial expressions (Indersmitten & Gur, 2003). Moreover, subjects with anxiety, depression, and other psychiatric disorders show a reduced or absent left-side bias (Gooding, Luh, & Tallent, 2001; Heller, Etienne, & Miller, 1995; Redican, 1975). Although the magnitude of the left-side bias did not depend on facial expression (one-way ANOVAs on left-side looking time during first trial with factor = expression, Q: F_2,27=2.88, p=0.073; V: F_2,27=3.22, p=0.057; T: F_2,27=0.63, p=0539), there were considerable differences in left-side bias among the three viewers (see Fig. 3E). Monkeys Q and V spent twice the amount of time looking at the left side of the face, whereas monkey T exhibited no significant side bias (Fig. 3F, two-tail paired t-tests on left and right-side looking time during first trial Q: t₅₈=7.41, p=0.000; V: t₅₈=13.90, p=0.000; T: t₅₈=1.76, p=0.084).

Figure 3. Facial expression determines the allocation of looking time to facial features.

(A) Percent looking time on the face calculated for each facial expression (ne=neutral, ls=lipsmack, th=threat) for all three viewers during the first trial of all 30 movies. Monkey Q spent a significantly greater proportion of time (p<0.05) looking at the movie monkey's face, compared to monkey V and monkey T. (B) Eye looking (expressed as a percent of face looking) calculated for each facial expression across the first trials of all 30 movies. All three monkeys looked significantly longer at the eyes of neutral movies (p<0.05). (C) Mouth looking (expressed as a percent of face looking). All three monkeys spent proportionally more time exploring the mouth of threats, followed by lipsmacks, and neutrals (p<0.05). (D) The nose was explored the most during movies of lipsmacking monkeys (p<0.05). Monkey T spent a greater proportion of time looking at the nose than monkey Q (p<0.5). (E) Example scanpaths from all three viewers during a 1.5 s segment of a video when there was no movement. (F) Percent time looking at the left vs. right side of the face. Monkeys Q and V spent more time looking at the left side of the movie monkey's face (p<0.05). Monkey T showed no significant side bias.

We next analyzed the relationship of looking time on each feature and facial expressions. Even though the overall face looking (Fig. 3A) and side bias were not influenced by facial expression, the time spent exploring different facial features varied considerably for different expressions. All three viewer monkeys tended to look most at the mouth of threatening displays, less at the mouth of lipsmacks, and least of all at the mouth of neutrals (Fig. 3C, two-way ANOVA on portion of mouth looking time with factors = viewer × expression of movie monkey, main effect of expression F(2)_2,87=13.86, p=0.000). The nose region (including the midface) was explored significantly more while viewing lipsmacking displays, with monkey T spending more time looking at the nose than monkey Q (Fig. 3D, two-way ANOVA on portion of nose looking time with factors = viewer × expression of movie monkey, main effect of viewer F(1)_2,87=4.22, p=0.018, main effect of expression F(2)_2,87=4.14, p=0.019). The ears were examined more during neutral than lipsmacking or threatening displays (two-way ANOVA on portion of ear looking time with factors = viewer × expression of movie monkey, main effect of expression F(2)_2,87=5.28, p=0.007), whereas no facial expression biased the time spent looking at the cheeks or jaw-line. More than any other feature, all three viewers spent the greatest proportion of time examining the eyes of the movie monkeys, attending most to the eyes of movie monkeys with neutral expressions (Fig. 3B, two-way ANOVA on portion of eye looking time with factors = viewer × movie monkey expression, main effect of expression F(2)_2,87=3.36, p=0.040).

Although the overall proportion of eye-looking time did not vary among viewers (Fig. 3B), each viewer showed a different pattern of fixations on the eyes. The length of time that monkey Q looked contiguously at the eyes before saccading to another movie region was nearly twice that of monkey T. Monkey V had contiguous eye-looking periods shorter than Q but longer than T (Q: 0.56±0.65 s; V: 0.43±0.45 s; T: 0.30±0.31 s; one-way ANOVA on duration of eye-looking periods during first trial with factor = viewer, F_2,2178=48.34, p=0.000). Despite looking at the eyes for a shorter contiguous duration, both monkeys T and V looked back at the eyes more frequently than monkey Q (Q: 0.90±0.78 s^-1; V: 1.19±1.12 s^-1; T: 1.15±0.77 s^-1; one-way ANOVA on frequency of eye-looking during first trial with factor = viewer, F_2,1745=8.33, p=0.000). Neutral expressions elicited from all three viewers longer contiguous eye-looking periods than lipsmacks (two-way ANOVAs on duration of eye-looking periods with factors=movie monkey expression × age-group, main effect of expression Q: F(1)_2,1193=21.88, p=0.000; V: F(1)_2,347=3.2, p=0.042; T: F(1)_2,632=4.28, p=0.014). Monkey Q, but not the other two viewers, also made saccades to the eyes of threatening monkeys more frequently (two-way ANOVA on frequency of eye-looking with factors=movie monkey expression × age-group, main effect of expression Q: F(1)_2,1193=4.82, p=0.008). Additionally, monkey Q looked at the eyes of old monkeys for shorter contiguous durations than the eyes of young monkeys (two-way ANOVA on frequency of eye-looking with factors=movie monkey expression × age-group, main effect of age Q: F(2)_2,1193=21.40, p=0.000). Given the salient role that the eyes play in social communication, the remainder of this article focuses on three distinct aspects of eye-looking: (1) direct eye contact between the viewer and movie monkey, (2) active avoidance of eye contact with the movie monkey, and (3) gaze following.

Direct eye contact and the production of facial expressions

As quantified by the ethograms (see Fig. 1A for an example), all the stimulus movies contained segments when the movie monkey looked directly at the camera. During these segments, the viewer monkey experienced direct gaze from the movie monkey, which was often reciprocated (eye contact). These periods of eye contact were significantly longer than periods of looking at the averted eyes of the movie monkey (two-tail paired t-tests on duration of eye-looking at averted and direct gaze; Q: averted 0.54±0.67 s, direct 0.68±0.63 s, t₁₁₆₂=2.84, p=0.005; V: averted 0.38±0.43 s, direct 0.64±0.50 s, t₃₄₈=4.24, p=0.000; T: averted 0.29±0.32 s, direct 0.36±0.26 s, t₆₂₈=2.27, p=0.023). As in humans, direct eye contact in monkeys signals an invitation for social interaction. Accordingly, eye contact is often accompanied by an overt act of aggression (e.g., yawn, open-mouth threat) or a gesture of appeasement (e.g., gaze aversion, fear-grimace, lipsmack) (Redican, 1975). To test whether the viewer monkey participated in such forthright behaviors, facial expressions of the viewer monkey were monitored using a webcam and EMG from the auricularis muscle.

Despite being under head immobilization, all three viewer monkeys made overt facial expressions while watching the movies. All three viewer monkeys routinely moved their ears and mouths in a manner that is characteristic for lipsmack displays (see Fig. 1C). With few exceptions (2/53 instances in Q), if the viewer monkey didn't lipsmack on the first trial of a video, he was unlikely to lipsmack on subsequent presentations of that same movie. As trials progressed, the viewers tended to lipsmack at later times during the movie display (see Fig. 1C). Most of the time, the viewers did not begin lipsmacking until well into the movie presentation (Q: n=53, min=0.5 s, max=9.9 s, median=4.9 s; V: n=10, min=2.5 s, max=8.5 s, median=5.3 s; T: n=16, min=0.3 s, max=9.0 s, median=4.2 s). Although monkey Q lipsmacked more than the other two monkeys over all trials (% of movie presentations that monkeys lipsmacked: Q=25%, V=8%, T=8%, Chi-squared test on total proportion of time spent lipsmacking across all trials, χ²₂=33.40, p=0.000), during the first trial all three monkeys lipsmacked for comparable durations (Chi-squared test on total proportion of time spent lipsmacking during first trial,χ²₂=0.289, p=0.866);

Based on the purported role of eye contact in social communication, it was hypothesized that viewers would be more likely to lipsmack during and following sustained periods of direct eye contact. Indeed, both monkeys Q and V were more likely to lipsmack when maintaining eye contact for long durations than for short durations (Fig. 4A, left and middle panels, Movie 1). Monkey T demonstrated no significant relationship between his inclination to lipsmack and the duration that he maintained eye contact (Fig. 4A, right panel). A complementary analysis showed that monkeys Q and V (but not monkey T) maintained direct eye contact for longer durations when they were lipsmacking than when they were not lipsmacking (two-tail paired t-test on consecutive eye looking duration when the viewer did/didn't lipsmack Q: 0.62±0.54 s / 0.36±0.41 s, t₂₄₀=18.38, p =0.000; V: 0.64±0.46 s / 0.29+0.21 s, t₆₂=13.25, p =0.001; T: 0.34±0.21 s / 0.32±0.18 s, t₇₄=0.26, p=0.615).

Figure 4. Movie content determines the amount of lipsmacking of the viewer monkeys.

(A) The dependence of lipsmacking probability on the durations of eye contact between the viewer and the movie monkeys. The probability of lipsmacking was calculated separately for each viewer (Q, V, and T) for three different durations of eye contact (short: pale grey, medium: gray, and long: black). Probabilities are centered around the initiation of eye contact (time zero on the x axis) by the viewer monkey. For monkey Q (left panel), the probability of lipsmacking was low when eye contact was not maintained but increased for periods of long eye contact. Even more pronounced in monkey V (middle panel), prior to the initiation of eye contact there was virtually no distinction in the probability of the viewer lipsmacking but after long periods of eye contact the probability of lipsmacking increased significantly. Monkey T (right panel) showed a slight increase in lipsmacking probability for long periods of eye contact but to a lesser extent than in monkeys Q and V. All periods of eye contact (short, medium, and long) were scaled to the minimum and maximum eye contact durations of each viewer, so that for a given viewer, each duration category includes the same number of data points (Q n=242; V n=64, T n=76). (B) Proportion of time that each viewer spent lipsmacking movies of other monkeys displaying lipsmacking (purple), and neutral (blue), threatening (orange) expressions. Responses are further subdivided by the age group of the movie monkey such that younger monkeys are represented by lighter hues of the same color. All three viewers lipsmacked movies of lipsmacking monkeys the most and threatening monkeys the least. Additionally, all three viewers tended to lipsmack young monkeys more than monkeys in their peer group or older.

In Rhesus societies, a direct stare is considered by some researchers to be a mild form of aggression, and can be appropriately responded to with a gesture of appeasement (e.g., fear-grimace or lipsmack)(Redican, 1975). In this context, the higher probability of a viewer lipsmacking for longer periods of direct eye contact, suggests that the lipsmack is primarily a means of appeasing a threatening situation. Contrary to this expectation, however, all three viewer monkeys produced more lipsmacks in response to movies of lipsmacking monkeys rather than in response to threatening monkeys; (Fig. 4B). Not only did the threatening movies elicit the least amount of lipsmacks, but all three monkeys tended to spend the greatest time lipsmacking young monkeys, possibly the least intimidating individuals (Fig. 4B, Chi-squared test on proportion of time spent lipsmacking, groups=age-group × expression, Q: χ²₄=203.4, p =0.000; V: χ²₄=112.4, p =0.000; T: χ²₄=239.8, p=0.000). The two movies that elicited the most numerous lipsmacks from all the three viewers depicted a 3 year-old female displaying a lipsmacking or neutral expression.

Check-looks as a measure of active avoidance

The progressively decreasing time spent looking at movies during later trials might be explained by less interest or habituation to the content of the movie. It is also plausible that viewers sometimes looked away from the monitor to purposefully avoid social interaction with the movie monkeys. In these cases, the viewer could maintain a covert level of attention and, in doing so, might occasionally look at the monitor for very brief periods to “check-look” on what the movie monkey is doing. We have defined check-looks as instances when the viewer saccaded from outside the movie frame to briefly [<400 ms] fixate on a single movie region, before saccading back to an areas outside the frame. Such check-looks were performed by all three viewer monkeys (number of movies with check-looks on first trial: Q=0, V=9, T=7; total number of check-looks across all trials: Q=66, V=82, T=118). Whereas monkey Q often looked back at the eyes of the movie monkey, monkey T tended to saccade more frequently to the cage in the vicinity of the movie monkey, avoiding the eyes (Fig 5A). Monkey V, on the other hand, was equally likely to look at the eyes or the cage. Interestingly, both monkeys T and V (but not monkey Q) tended to wait for the movie monkey to avert his gaze before taking the chance to check-look on the movie (Fig. 5B), suggesting that these viewers were aware of the direction of the gaze of the movie monkey and they were actively avoiding eye contact. The number of check-looks made by a viewer did not depend on the expression of the movie monkey.

Figure 5. Check-looks indicate that viewer monkeys actively avoid looking at movie monkeys.

(A) Each plot shows the probability of check-looks targeting the eyes of the movie monkey or a region of the cage in the vicinity of the movie monkey's face. Probabilities are centered around the start of the check-look fixation (time zero on the x axis represent the beginning of a check-look fixation). In accordance with the definition of a check-look, prior to and following the fixation on the movie the probability that the viewer is looking at any movie region is zero. Monkey Q (left panel) looked primarily at the movie monkey's eyes (gray histogram) during check-looks, monkey T (right panel) looked primarily at the cage (black trace). Monkey V (middle panel) showed an equal probability of looking at either the eyes or the cage (n Q=66; n V=82; n T=118). (B)Probability of check-look occurring as a function of the movie monkey's gaze. Time zero on the x axis represents the start of the check-look fixations. In viewer monkey Q, the probability of a check-look did not depend on the movie monkey's gaze. Monkeys T and V were less likely to perform a check-look when the movie monkey's gaze was directed at them.

Gaze following

Because the eyes reveal what an individual is overtly paying attention to, gaze following is purported to play a significant role in normal social behavior. In all stimulus movies, the movie monkey's gaze was averted during some segment of the movie, potentially allowing for “opportunities” during which the viewer monkey could gaze follow. Even though there were no other monkeys or objects in the movie frame that were directly in the line of sight of the movie monkey, all three viewers consistently followed the movie monkeys' gaze (Q: n=67, V: n=39, T: n=45) (Fig. 6). To qualify as a gaze-following saccade, the eye movements of the viewer monkey had to meet stringent criteria: viewers had to have first fixated on the eyes of the movie monkey and then saccade within ±30 degrees of the movie monkey's line of sight (e.g., Fig. 6A, B, C, D). If the target of such a saccade was a body part it was not considered a gaze following event. In some instances, the viewers looked in the same direction as the movie monkey, and when they were unable to discern what the movie monkey was looking at, they made a saccade back to the movie monkey's eyes before performing another gaze following saccade.

Figure 6. Viewer monkeys spontaneously follow the gaze of movie monkeys.

Example of frames during which all three viewers fixated on the eyes of the movie monkey and then saccaded in the direction of the monkey's gaze. Frames that contained gaze-following saccades were extracted from movies depicting (A) a lipsmacking 11 year-old female, (B) a threatening 20 year-old male, (C) a neutral 13 year-old male, (D) a lipsmacking 4 year-old male. (E) Cross correlation of gaze-following saccade times showing an increase in the probability of all three viewers gaze followed around the same frame (depicting the same event). The probability that viewers gaze followed during the same time period (within ±0.5 s of each other) is greater than would be expected by chance (compare with probability of gaze follows occurring > ± 0.5 s from each other). (F) Average level of motion in the movie before and after a gaze-following saccade (n=151). Time zero on the x axis indicates the start of the gaze-following saccade. Gaze following was preceded by an increase in movement in the video.

Gaze-following saccades did not occur randomly. If a viewer gaze-followed during one trial of a movie, he was twice as likely to gaze-follow at the same time (± 15 frames) on a different trial than at some other random time during the movie. Moreover, all three monkeys often gaze-followed during the same frames of a given movie, an overlap that is not expected by random chance (Fig. 6E, also see examples in Fig. 6 A, B, C, and D). Gaze-following events were unequally distributed across movies and did not depend on the expression of the movie monkey. The two movies that elicited the most gaze-following events from all three viewers depicted two different 20 year-old males: one displaying a lipsmack and the other a threat. The propensity to gaze follow likely depended on factors that were inherent in the movies and could not be equalized across all movies, e.g. gaze following was more likely to occur when the movie monkey in the film made a rapid movement of the head (Fig. 6F).

Discussion

The findings reported here show that spontaneous natural behaviors can be reliably elicited, quantified, and controlled in the laboratory. We show that those self-same social interactions observed in the field can also be observed in the laboratory as looking patterns (e.g., gaze-aversion and gaze-following) triggered by naturalistic social stimuli (Chevalier-Skolnikoff, 1973; Redican, 1975; Shepherd & Platt, 2008). We also demonstrate that when monkeys use their gaze to “interact” with a video, they often display overt facial expressions. Because the monkeys in this study were neither required to watch the videos nor given external reward/punishment for doing so, all behaviors described here are spontaneous and might reflect meaningful species-specific social signaling.

Monkeys look at the salient features of facial expressions

As in humans, the meaning of a facial expression in monkeys depends on context (e.g., relative rank, body posture, history of interactions, distance between animals) (Chevalier-Skolnikoff, 1973; Hinde & Rowell, 1962; Maestripieri, 1997; Redican, 1975), the intensity of the display (e.g., open mouth threat with/without head bobbing), and the relative place of an expression along a continuum between expressions (e.g., gradations from yawn-threat-feargrimace-teethchattering-lipsmacking-pucker) (Chevalier-Skolnikoff, 1973; Redican, 1975). Intense expressions can be easily categorized based on the salient contributions of different facial features, but less explicit expression gain significance from subtle changes in posture, the duration of the display, or the context (e.g., preceded by a more intense display of the same type).

When monkeys watched videos of facial expressions, they tended to explore those facial features that best define each expression. The mouth was explored most during open-mouth threats and least during neutral expressions. The midface was attended to more during lipsmacks (which includes the wrinkling of the nose) than neutral or threatening expressions. The ears were examined most during neutral expressions, likely because their movement can signal even subtle changes in attention and emotion. All three viewer monkeys looked longer at the eyes than at any other feature. The eyes indicate most accurately the affective, attentional, and intentional states of an individual (Baron-Cohen, Wheelwright, Hill, Raste, & Plumb, 2001; Klein, et al., 2009). Indeed, humans are able to accurately identify the emotional state of an individual by looking at images of the eyes alone (Adolphs, 2009).

Significant differences were observed between the feature preference of the viewers reported here and earlier reports from experiments that used static images of facial expressions (Gibboni, et al., 2009; Gothard, et al., 2004; Keating & Keating, 1982; Nahm, et al., 1997). These differences can be attributed to several factors: (1) only the face is usually presented in static images and not the whole body as in our video segments, (2) in many previous studies monkeys are often required to maintain their gaze on the image for a predefined duration in order to get reward, and/or (3) videos are inherently more salient and the movement alone can “hold” the viewer's attention. In a similar video-presentation study, Ghazanfar and colleagues (2006) have shown that monkeys look mostly at the eyes during videos of vocalizing monkeys but attend to the mouth when it begins to move at the onset of a vocalization.

Although the three monkeys in the current study spent proportionally equivalent amounts of time looking at different facial features, monkeys Q and V were biased towards looking at features on the left-side of the face whereas monkey T showed no such side bias. Guo and colleagues (2009) have demonstrated that monkeys show a left-side bias towards static images of upright monkey and human faces but not towards inverted faces, suggesting that the left-side bias is related to face processing expertise. A left-side face-looking bias has been well-documented in humans where it is closely linked with a perceptual bias to process information in the left visual field (Indersmitten & Gur, 2003). When shown chimeric face stimuli (e.g., a composite face which is smiling on the left half but frowning on the right half), healthy human subjects are biased in perceiving the emotion or identity on the left-side of the image unlike subject with Asperger's syndrome that show reduced side bias (Ashwin, Wheelwright, & Baron-Cohen, 2005; Indersmitten & Gur, 2003). Individuals with schizophrenia also exhibit an ablated or reduced side-bias towards faces (Gooding, et al., 2001).

Lipsmacking is an affiliative behavior that is facilitated by direct eye contact

When presented with video stimuli under conditions of head fixation, our monkeys reliably and consistently displayed lipsmacking expressions. Viewers were more likely to lipsmack during periods of sustained eye-contact. Moreover, movies of young lipsmacking monkeys elicited the greatest number of responses whereas movies of threatening monkeys elicited the fewest. Rhesus facial expressions are often interpreted in terms of negative emotional states and behavioral intentions (e.g., the goal of a lipsmack is to appease threatening and fearful situations, a yawn is used to express anxiety or agitation). These results, however, argue strongly in favor of interpreting the lipsmack as a prosocial behavior, and not an attempt to diffuse perceived aggression. Indeed, several field biologists have placed lipsmacking in a prosocial category of facial gestures. Although lipsmacking occurs in a variety of circumstances, it almost always results in a positive social advance (Hinde & Rowell, 1962). It is the most prominent expression during grooming and it occurs frequently during sexual intercourse and same-sex mounting (Maestripieri, 1997; Redican, 1975). Lipsmacking initiates playful behaviors in juvenile monkeys, is used by mothers to beckon their infants, and is performed by males to attract females in estrus (Maestripieri, 1997; Redican, 1975). Although also used when a subordinant makes direct eye contact with a dominant animal, in these cases the lipsmack communicates an intention to engage in affiliative behaviors, and not simply inhibit aggression and end social contact (Maestripieri, 1997). Though lipsmacks occur in response to threats, they are rarely the first reaction and are typically intermixed with fear-grimacing and gaze aversion (Maestripieri, 1997; Redican, 1975). The finding that monkeys lipsmacked the least while viewing movies of threats and the most when viewing movies of young-lipsmacking corroborates the idea that lipsmacking is primarily prosocial.

Likewise, direct eye-contact can not only be considered a mild form of threat but can also be used to signal the intention for a positive social interaction which is reciprocated with a lipsmack (Redican, 1975). It may be possible that the heightened arousal associated with direct eye-contact compels a monkey to externalize his arousal through a gestural display (Linnankoski, Gronroos, & Pertovaara, 1993; Perrett & Mistlin, 1991; Wada, 1961). Lipsmacks, therefore, may not only promote social bonding, but “may also reflect excitement” (Maestripieri, 1997). Indeed, when a female monkey displays her hindquarters, a male will lipsmack, masturbate, and ejaculate but only when eye-contact is made (Linnankoski, et al., 1993). On a more subtle level, both humans and monkeys exhibit increased autonomic arousal when making facial expressions and when observing the facial expressions of others (Ekman, Levenson, & Friesen, 1983; Hoffman, Gothard, Schmid, & Logothetis, 2007; Kuraoka & Nakamura, 2011; Nakayama, Goto, Kuraoka, & Nakamura, 2005). Finally, in much the same way that happiness and smiling are “contagious” in humans, the propensity of our monkeys to lipsmack other lipsmacking monkeys could reflect a form of emotional contagion accompanied by facial mimicry (Hatfield, Rapson, & Le, 2009).

Even though all three viewer monkeys lipsmacked the movies, monkey T, who is peer-reared, showed the least propensity to lipsmack during direct eye contact. In agreement with field observations, mother-reared monkeys make more eye contact and lipsmack more often than peer-reared animals (Emery, 2000; Redican, 1975). When caged monkeys are allowed to watch movies of other monkeys, animals with high sociability display aggressive and affiliative expressions more often than low sociable monkeys (Capitanio, 2002). Moreover, low sociable animals have a higher propensity to lipsmack the movie monkey when it's engaged in a nonsocial behavior (e.g., eating food or manipulating cage) (Capitanio, 2002). Monkeys that carry at least one short allele of the 5-HTTLPR gene exhibit more aggressive behaviors than monkeys homozygous for two long alleles (Izquierdo, Newman, Higley, & Murray, 2007). Although our monkeys did not reliably display open-mouth threats (likely because head fixation makes the expression physically difficult and instills the subject with a perceived sense of vulnerability) it is possible that threatening microexpressions could be identified using video footage and facial muscle EMG (Waller, et al., 2008). Indeed, when monkeys are free to move around in a cage they display overtly aggressive gestures towards movies (Capitanio, 2002).

Being able to reliably elicit lipsmacks in monkeys in the laboratory opens multiple opportunities for further research. As one of the first facial expressions to appear during development (Redican, 1975), how does a maturing monkey learn to adapt lipsmacking to such a wide variety of circumstances? Given that lipsmacking is more prevalent during estrus (Carpenter, 1942) and threatening gestures are linked to testosterone (Chevalier-Skolnikoff, 1973; Clarke & Boinski, 1995), how do hormones influence facial expressions in the laboratory? Perhaps most compelling of all in a field dominated by fear-conditioning and aggression in Rhesus monkeys, lipsmacks offers a means of understanding the neural bases of positive social and emotional interaction.

Covert attention is used to avoid direct eye contact

Eye contact between monkeys can signal an intention for social interaction. This interaction can be affiliative (e.g., lipsmack with eye contact preceding grooming or sex) or agonistic (e.g., threat stare). (Maestripieri, 1997; Redican, 1975). In several species of primates active avoidance of eye contact (gaze aversion) is a submissive response and is the most common response (more so even than the fear grimace) to threatening or dominant individuals (Masataka & Fujii, 1980; Redican, 1975). Monkeys often signal submission by redirecting their attention toward another individual or taking sudden and intense interest in an object (cage grooming) or empty point in space (Redican, 1975). Although the monkey's eyes remain averted, his attention is covertly drawn toward the potentially threatening individual. Covert attention conceals intentions but enables monkeys to make quick behavioral responses in case the situation escalates (Klein, et al., 2009). Although gaze aversion can be used effectively to appease aggressive individuals, relinquishing eye contact could also be a product of intense arousal. Averting one's gaze allows a monkey to literally cut off the further perceptual processing of social stimuli, effectively reducing his level of arousal without requiring him to physically withdraw from social contact (Redican, 1975).

Monkey Q maintained eye contact with movie monkeys for longer durations than the other two viewers. Perhaps maintenance of eye-contact with the movie monkey reflects a social confidence stemming from monkey Q's high dominance-rank within his home-colony. Both monkeys V and T maintained shorter periods of eye contact, but looked more frequently at the eyes. In field studies, submissive monkeys tend to look frequently at threatening individuals but maintain eye contact for shorter periods than dominant monkeys (Redican, 1975). Although a viewer's rank in his home-colony might be related to his level of sociability, speculations about the viewer's rank relative to the movie-monkey's are tenuous (the movie-monkeys are unfamiliar and their relationship to one-another is unknown by the viewer). Future studies that use video stimuli depicting multiple monkeys interacting and/or individuals familiar to the monkey might be able to provide the information necessary to determine a relationship between social rank and looking patterns.

The looking patterns described here replicate only in part earlier findings by Watson and colleagues (2009) and Gibboni and colleagues (2009), who reported that monkeys with at least one copy of the short allele (s/l, s/s) of the 5-HTTLPR gene explore the eyes less than monkeys with two long copies (l/l). Both these studies used static face images as stimuli. We report here that the looking behavior of the same monkeys changes when the static images are replaced with dynamic images. Monkey T for example, whose genotype is s/s, looked intensely and obsessively at the eyes of static monkey faces (Gibboni, et al., 2009) but avoided looking at the eyes of monkeys shown in videos. These findings underscore the importance of using the most naturalistic designs possible for behavioral experiments in primates.

Detailed scrutiny of the pattern of eye contact, or lack thereof, identified many instances when the viewer was not looking at the movie screen but would occasionally make a saccade and a single fixation on the movie to “check-look” on what was happening. Check-looks were relatively more frequent in monkeys V and T as they spent less time looking at the movies than monkey Q. The check-looks of monkey Q targeted predominantly the eyes of the movie monkey; unlike Q, monkey V and especially monkey T tended to look at the cage in the vicinity of the movie monkey, further suggesting a form of avoidance. Moreover, both monkeys V and T waited until the movie monkey had averted his gaze before electing to make a check-look. Together, these findings suggest an active gaze avoidance by monkeys T and V. We propose that check-looks could be a way of identifying naturally induced and socially-relevant covert attention in the laboratory (during a check-look, both viewers appeared to know that the movie monkey had averted his gaze even though they weren't overtly attending to the movie).

Monkeys follow the gaze of other individuals

It is clear that direct eye-contact is autonomically arousing in monkeys, but averted gaze can also be arousing in certain circumstances (Hoffman, et al., 2007). The direction of eye gaze contains essential information about the target of another individual's attention (Emery, 2000; Shepherd, 2010). The ability to derive from another individual's eye gaze the target of his interest and to follow his gaze, is hypothesized to be a key component in the development of “theory of mind,” the ability of an animal to infer the mental state of another individual (Emery, 2000; Høgh-Olsen, 2006; Langton, Watt, & Bruce, 2000). Gaze following and shared attention (the latter is the ability to use the gaze of another individual to attend to the same object) are basic building blocks of primate social behavior (Chance, 1967; Emery, 2000). Quantifying gaze following and shared attention in the laboratory can be met by significant challenges. If the “social” task is extensively controlled, the paradigm can become artificial and cease to be socially relevant under real world circumstances (e.g., autistics may present with no gaze following deficits during an experiment even though they clearly exhibit deficits in shared attention outside the confines of the laboratory (Kingstone, 2009). In object-choice experiments (e.g., two objects are displayed to either side of an image of an individual whose gaze is oriented towards one of the objects), viewers saccade more often to the object that is congruent with the stimulus animal's gaze (Deaner & Platt, 2003; Emery, Lorincz, Perrett, Oram, & Baker, 1997). While informative in understanding mechanisms of shared attention, highly constrained experiments lack ecological validity and can lead to overestimates in gaze responses (Høgh-Olsen, 2006). A majority of studies in monkeys have explored the possibility that gaze following occurs in response to humans, a stimulus with little ecological relevance and which fails to take advantage of the monkeys natural ability to recognize the subtle social cues of conspecifics (Ferrari, Kohler, Fogassi, & Gallese, 2000; Goossens, Dekleva, Reader, Sterck, & Bolhuis, 2008; Langton, et al., 2000; Tomasello, Hare, & Fogleman, 2001). A more informative approach used in the field has been to attract the attention of one monkey using food and subsequently observe whether other conspecifics redirect their attention along the gaze of the attentive animal (Tomasello, et al., 1998). Shepherd and Platt have elegantly demonstrated spontaneous gaze following of lemurs in the field by using head-mounted video cameras (Shepherd & Platt, 2008). Field biologists have described instances where one monkey uses his gaze to trick another monkey into looking in a certain direction, giving the deceiver a chance to grab a piece of fruit that he saw in the opposite direction (Emery, et al., 1997).

In the present study, the viewer monkeys were allowed to freely view movies of conspecifics, and they all showed spontaneous gaze-following saccades. By using stringent gaze following criteria and showing that (1) individual viewers gaze followed during the same movie frames across multiple trials and (2) all three viewers gaze followed during the same movie frames, we argue that these events were neither random nor selected with bias. While viewers sometimes gaze-followed to an object that they subsequently explored (shared attention), there were many instances where there was no object of interest and the viewer gaze followed to a position lying outside the movie screen. Oftentimes when viewers gaze followed and could not localize an object of interest, they fixated again on the eyes and then made another gaze-following saccade in the same direction. These “double-checks” argue against the idea that gaze following is purely a reflexive mechanism (Shepherd, 2010). Although monkeys have been shown to gaze follow eyes alone, it is thought that they rely more on head direction (or at least coordination of head and eye direction facilitates gaze following) (Emery, et al., 1997; Ferrari, et al., 2000; Lorincz, Baker, & Perrett, 1999). We show that head movement is one of the best predictors that a gaze following event will occur, corroborating the claim that head orientation facilitates gaze following.

It has been hypothesized that monkeys are more apt to gaze follow certain individuals or particular facial expressions (Goossens, et al., 2008; Shepherd, 2010). While we observed no bias of the monkeys to gaze follow certain expressions more than others, all three viewers gaze followed two movies of old male monkeys the most. The lack of a strong identity or expression effect may have resulted from our selection of the movie segments; the movies were not explicitly controlled for neither the amount of head movement nor the congruency/incongruency of eye and head angles.

Knowing that gaze following can occur in response to video stimuli suggests that future research could dissociate the various effectors of gaze-following by carefully controlling for movement and head-eye orientation. Such video stimuli will probably be most effective if they are several seconds in duration (allowing for the monkey to be fully engaged by the stimulus) and are unedited segments of continuous footage. (Shepherd, et al., 2010). Shepherd and colleagues (2010) report that “humans, but not monkeys, strongly attend the foci of the other individual's attention and activity [in a video stimulus].” However, they and others concede that this effect could be due to “cuts” or other aspects of cinematography, which might appear unnatural to a viewer monkey (Berg, et al., 2009; Shepherd, et al., 2010). The use of carefully selected video stimuli has the potential to elucidate the developmental trajectory of gaze following (e.g., why infant monkeys tend to gaze follow reflexively but adults will habituate to a “boy-who-cried-wolf” gaze follow) (Langton, et al., 2000), the influence of sexual hormones on the propensity to gaze follow (Klein, et al., 2009), and the association between gaze following and “theory of mind” (Emery, 2000; Høgh-Olsen, 2006; Langton, et al., 2000).

In summary, using videos of conspecifics elicited reproducible and spontaneous social behaviors of Rhesus monkeys in a laboratory setting. These observations represent a range of species-specific natural behaviors, some of which are consistent across all three viewers and others which are unique to one or two individuals (Table 2). Whereas consistent behaviors might reflect neural mechanism that are to ingrained in the entire species, those behaviors that are subject to large individual variation might reflect neural mechanisms that depend on the ontogeny of the individual.

Table 2. Summary of the species-specific behaviors elicited by movies of conspecifics.

	Similarities between viewers	Individual differences
Overall looking time and habituation	scanpaths of viewers converged 20% of the time during the first trial total looking time was significantly less during the second trial and further decreased with trial repetition scanpaths became more disperse with trial repetition	scanpaths of viewers diverged 80% of the time during the first trial Q spent more time looking at the movies than V and T scanpaths of T and V were more disperse than Q
Expression-dependent looking patterns	facial features were explored in an expression dependent manner (e.g. mouth of threat looked at more than mouth of neutral) the eyes were explored more than any other facial feature direct eyes were looked at for longer continuous periods than averted eyes (maintenance of eye contact) total looking time did not depend on the appeasing, neutral, or aggressive behavior of the movie monkeys	Q spent a greater portion of time looking at the face of the movie monkey than T Q and V exhibited a left-side looking preference on faces. T did not. Q fixated on the eyes for consecutively longer periods than V; V looked longer than T. T and V saccaded back to the eyes more frequently than Q. Q looked at the eyes of threatening monkeys more frequently
Facial expressions of the viewer monkey	all viewers lipsmacked movies; if they didn't lipsmack on the first trial it was unlikely for them to do so on later trials during the first trial, all viewers lipsmacked for similar durations viewers lipsmacked young monkeys the most	pooling over all trial repetitions, Q lipsmacked the most to the movies Q and V were more likely to lipsmack when making direct eye-contact with the movie monkey. T did not show a tendency to lipsmack during direct eye contact.
Gaze avoidance & check-looks	all monkeys performed check-looks	T performed the most check-looks followed by V and then Q. Q saccaded to the eyes during check-looks but T saccades to the cage. V performed the same amount of check-looks to the eyes and cage T and V were more likely to perform a check-look when the movie monkey had averted his gaze
Gaze following	viewers spontaneously followed the gaze of the movie monkeys. gaze following events occurred in response to the same movie segment (within viewer and across viewers)

Viewer monkeys spent 20% of the time looking at the same areas of the videos (comparable with the 27% reported by Shepherd, et al., 2010), but the majority of the time the viewers looked at different areas of the movies, adding to the rich catalogue of observations that scanpaths reveal information about the identity (gender, age, race), personality, temperament, and pathology of a viewer. Although the viewers varied on numerous background factors (e.g. 5-HTTLPR genotype, rearing-history, autonomic-characteristics), we argue that when these factors are considered together they converge on a relative measure of sociability. The specifics of each viewer's scanpaths, together with the viewer's genetic, behavioral, and autonomic characteristics suggest that a large portion of the variability in scanpath is due to different levels of sociability. Monkey Q who was the most sociable of the three viewers based on these qualifications, exhibited the greatest social flexibility in responding to the signals of movie monkey (e.g. more eye contact, left-side face looking bias, least habituation, most prosocial lipsmacking toward the movie-monkey). Monkey T consistently exhibited some degree of social inaptitude (e.g., lack of left-side bias, no increase in lipsmacking probability during sustained eye-contact). Monkey V was situated somewhere in between. If monkeys Q and T (the two social extremes) displayed a consistent behavior, monkey V always displayed that behavior as well.

Some of the aspects of individual variability highlighted here, might aid in the development of diagnostic tools for social cognition in both monkeys and humans. We expect that further studies with this task will elucidate how natural social behaviors are regulated by different hormones, how they appear during development, how they vary with genotype, and how they are generated at the level of individual neurons and neural circuits. While our current analyses are by no means exhaustive, the behaviors we have described are reproducible, quantifiable, and are a naturally-occurring subset of the complex repertoire of social interactions performed by Rhesus monkeys.

Supplementary Material

S1

Movie 1: Example stimulus videos with superimposed eye movements of all three viewer monkeys. The eye movements of monkey Q (red), V (blue) and T (yellow) during the first trial of a movie presentation are superimposed on four stimulus videos. For illustration purposes, each video is displayed first at normal speed (30 fps) and a second time at half speed (15 fps).

S2

Movie 2: Video of the viewer's face, eye movements, and EMG during video presentation. The movie at the left side of the screen shows the eye movements of monkey Q (green dot) as he viewed a 3 year-old female with neutral expression. The movie on the right shows the facial expressions of monkey Q as he watched this movie. The top trace shows the simultaneous EMG from the viewer's lateral auricularis muscle. When the viewer displayed a lipsmack, there are significant increases in the EMG. Note that lipsmacking is enhanced by direct eye contact.