A Cross-species Comparison of Facial Morphology and Movement in Humans and Chimpanzees Using the Facial Action Coding System (FACS)

Abstract

A comparative perspective has remained central to the study of human facial expressions since Darwin’s [(1872/1998). The expression of the emotions in man and animals (3rd ed.). New York: Oxford University Press] insightful observations on the presence and significance of cross-species continuities and species-unique phenomena. However, cross-species comparisons are often difficult to draw due to methodological limitations. We report the application of a common methodology, the Facial Action Coding System (FACS) to examine facial movement across two species of hominoids, namely humans and chimpanzees. FACS [Ekman & Friesen (1978). Facial action coding system. CA: Consulting Psychology Press] has been employed to identify the repertoire of human facial movements. We demonstrate that FACS can be applied to other species, but highlight that any modifications must be based on both underlying anatomy and detailed observational analysis of movements. Here we describe the ChimpFACS and use it to compare the repertoire of facial movement in chimpanzees and humans. While the underlying mimetic musculature shows minimal differences, important differences in facial morphology impact upon the identification and detection of related surface appearance changes across these two species.

Introduction

Elaborate facial communication is a primate adaptation and a comparative approach has long been integral to our understanding of facial expressions (Andrew, 1963; Chevalier-Skolnikoff, 1973; Darwin, 1872/1998; Redican, 1975; van Hooff, 1972, 1976). This evolutionary perspective on the form and function of facial movements allows us to gain further insight into the pressures shaping communicative capacities in primates (Parr, Waller, & Fugate, 2005; Schmidt & Cohn, 2001; Sherwood et al., 2005). Despite the literature describing facial displays in primates, including the study of both captive and wild chimpanzees (Ladygina-Kohts, 1935/2002; Marler, 1976; Parr, Cohn, & de Waal, 2005; van Hooff, 1973; van Lawick-Goodall, 1968), our understanding of nonhuman primate facial communication has been limited by the lack of an appropriate measurement tool for recording facial behaviors and allowing meaningful cross-species comparisons (Chevalier-Skolnikoff, 1973; Fridlund, 1994). To fully understand species-unique characteristics of the human face, ideally the same measurement tools should be used. Here we describe the development and application of ChimpFACS to examine the phylogeny of facial displays in terms of both underlying functional anatomy and related appearance changes.

In contrast to nonhuman primate research, the investigation of many of the complexities of human facial communication has been greatly facilitated by the development of the Facial Action Coding System (FACS; Ekman & Friesen, 1978; Ekman, Friesen, & Hager, 2002); this method allows for the objective and standardized measurement of facial movement based on the underlying mimetic musculature. While many studies measure the inferences made by observers when viewing faces, some questions are better addressed by measuring what the face itself does (Ekman & Rosenberg, 2005). FACS is anatomically based and describes minimal units of facial movement called Action Units (AUs) with detailed descriptions of the resulting changes in facial appearance¹. In addition, the use of multiple redundant cues addresses the often overlooked interaction between underlying muscle movements and the considerable individual variation in facial morphology, which impacts the precise form of observable appearance changes (Ekman & Friesen, 1978; Marsh, Adams, & Kleck, 2005; Schmidt & Cohn, 2001). The development of such a comprehensive coding system with the common language of AUs, with numerical codes and neutral labelling, has enabled researchers across a wide variety of sub-disciplines, often with diverging theoretical positions, to communicate and evaluate findings using a common language (see Ekman & Rosenberg, 2005).

FACS identifies, describes, and illustrates 32 AUs, in addition to codes for miscellaneous facial movements (such as tongue showing), head and eye movements, and the visibility of each facial area. Most AUs are the result of a single underlying muscle action (e.g., the zygomatic major muscle underlies AU12: Lip Corner Puller), or of different portions of the same muscle (e.g., the frontalis muscle underlies both AU1 and AU2, Inner Brow Raise and Outer Brow Raise, respectively). Some AUs are caused by the synergistic action of more than one muscle, which are rarely observed to move independently (e.g., AU4: Brow Lowerer is the combined action of the corrugator supercilli, depressor supercilli, and procerus muscles). FACS is based upon the understanding of anatomy combined with the analysis of visual appearance changes during facial movement, which enables actions to be identified and described in detail. It is this relationship between musculature and observable movement that makes the FACS methodology such a valuable research tool. FACS has already been modified for human infants, with AUs being translated to accommodate the differences in facial morphology (BabyFACS: Oster, 2003, unpublished manual; Oster & Ekman, 1978). Thus, the FACS is a methodology which is amenable to modification across populations, and the anatomical basis suggested it would be robust enough to accommodate the greater modifications required for cross-species application.

The FACS approach enables us to address questions about the mechanisms underlying the production of primate facial displays. Indeed, its value has been recognized in previous studies of facial behavior in nonhuman primates; Preuschoft and van Hooff (1995) addressed the issue of homology of nonhuman primate bared teeth facial displays with human smiling, while Steiner, Glaser, Hawilo, and Berridge (2001) and Ueno, Ueno, and Tomanaga (2004) described facial responses to taste in primates. These initial studies highlight the usefulness and desirability of adapting FACS for cross-species application. However, no studies to date have systematically modified FACS for use with other species and in order to make comparisons an equivalently comprehensive tool is desirable. For instance, in these studies, wrinkling beneath eyes is always identified as resulting from AU6 Cheek Raise (Preuschoft & van Hooff, 1995; Steiner et al., 2001; Ueno et al., 2004). In humans, wrinkling beneath the eyes can be caused by AU6 as well as several other actions, including some lower face actions (Ekman et al., 2002). When similarity is assessed using appearance rather than muscle function (Preuschoft & van Hooff, 1995), the AUs are no longer strictly comparable as the integrity of the FACS approach requires that the movement be related to underlying structure.

Moreover, the utility of FACS resides in the clarity of detailed descriptions. For example, Ueno et al. (2004) replicated Steiner et al.’s (2001) taste study with infant chimpanzees and found that their observations differed to those for adult chimpanzees found in the original study. It may be that the facial repertoire expands as the infants develop, but the authors also concede that this discrepancy could be due to differences in experimental protocols. Any adaptations of FACS should follow the process of its developers (Ekman & Friesen, 1978; Oster, 2003, unpublished manual); underlying structure and musculature must be integral to the development and use of the system, which should then be carefully documented to allow replication. The value of the FACS approach is that it provides a common language for describing changes in facial appearance; ad hoc modifications and the use of FACS to support a particular theoretical position is likely to reduce its utility as a research tool.

We developed a modification of the FACS for describing chimpanzee facial movement. ChimpFACS is based upon the comparative anatomy and facial morphology of chimpanzees. Although the literature revealed some disagreement regarding the muscle plan of the chimpanzee (Gibbs, Collard, & Wood, 2002; Huber, 1931; Pellatt, 1979; Seiler, 1971), many of these inconsistencies have been resolved using a more refined facial mask dissection method (see Burrows, Waller, Parr, & Bonar, 2006). Contrary to previous reports suggesting unique complexity and refinement of human facial musculature (Huber, 1931), there is in fact a striking similarity between humans and chimpanzees. Moreover, the study of functional anatomy also reveals comparable muscle movements in these species, with individual facial movements recognizable during intramuscular stimulation (see Waller, Vick, Parr, Smith Pasquialini, Bard, Gothard, et al. 2006). From this starting point, we then undertook extensive observations of chimpanzee facial movements and this paper provides an overview of how these actions appear on the chimpanzee face. Using this approach, we can examine the underlying mechanism of facial expression production in chimpanzees and subsequently, use ChimpFACS to explore both perception and function using a truly comparative research tool.

The FACS method is particularly well suited to comparative studies as it is provides clear descriptions for the identification of each AU, listing various appearance changes (movement of facial landmarks, changes to shape of facial features) that can be directly compared. Thus, we can identify how and why a given facial movement may appear dissimilar across species in terms of underlying facial structure, musculature and surface coloration, and textures. As noted above, facial expressions interact with the underlying facial morphology to create salient signals; in primates, coloration or shape of facial features can emphasize or camouflage facial movements (Andrew, 1963; Kaplan & Rogers, 2002; Kobayashi & Kohshima, 2001; Napier & Napier, 1985; Preuschoft, 2000). For example, pigmentation of the upper eyelids in some species, or retention of hairy brows on hairless forehead in humans, may highlight actions of the brows (Ekman, 1979; Campbell, Benson, Wallace, Doesbergh, & Coleman, 1999; Preuschoft, 2000). In addition, there is much individual variation in both chimpanzee and human faces, which needs to be accommodated in these descriptions of appearance changes (Ekman & Friesen, 1978).

As with BabyFACS (Oster, 2003, unpublished manual), it is apparent that translating FACS for different facial morphologies identifies the key commonalities and differences in the basic repertoire of facial movements and their associated changes in appearance. The development of the ChimpFACS allows us to address two central questions in a more systematic manner than previously. First, what are the underlying mechanisms of chimpanzee facial expression production? Second, how do these facial movements compare with the human facial repertoire, in terms of not only the number of identifiable independent actions, but also in terms of their functional movement and resulting appearance changes? Here we describe the basic facial repertoire of the chimpanzee in terms of minimal independent movements using ChimpFACS. For each AU identified, basic appearance changes will be outlined and compared to the equivalent AU as described by the human in FACS.

Method

Development of Chimp FACS

Adapting the FACS for other primate species is challenging for a number of reasons. Due to the anatomical basis of FACS, the first step must be a consideration of facial musculature for the study species. However, initial attempts to produce a basic muscle plan of the chimpanzee face were hindered by the inconsistencies in reported dissections. Therefore, new dissections were conducted in order to clarify our understanding of cross-species differences in facial musculature, and these are described elsewhere (see Burrows et al., 2006). As there were no data available on facial muscle function in the chimpanzee, a comparative study of muscle action was also undertaken in order to ascertain whether independence of facial movements was a common feature, and also to document these actions in isolation (Waller et al., 2006). See Table 1 for summary of muscle presence and action in chimpanzees in relation to human AUs and their underlying musculature.

Table 1.

Facial musculature underlying AUs in humans (Ekman et al., 2002), identifying their presence (Burrows et al., 2006) and comparable muscle movement when stimulated in chimpanzees (Waller et al., 2006)

	FACS		Chimpanzee
AU	Name	Underlying musculature	Presence	Movement
AU1	Inner Brow Raise	Frontalis (medial)	✓	✓
AU2	Outer Brow Raise	Frontalis (lateral)	✓	✓
AU4	Brow Lowerer	Procerus	✓	✓
		Depressor supercilli	✓	✘
		Corrugator	✓	✓
AU5	Upper Lid Raise	Orbicularis oculi	✓	–
AU6	Cheek Raise	Orbicularis oculi, pars orbitalis	✓	✓
AU7	Lid Tightener	Orbicularis oculi, pars palebralis	✓	–
AU9	Nose Wrinkle	Levator labii superioris alaeque nasi	✓	✓
AU10	Upper Lip raiser	Levator labii superioris	✓	✓
AU11	Nasiolabial Furrow Deepener	Zygomatic minor	✓	–
AU12	Lip Corner Puller	Zygomatic major	✓	✓
AU13	Cheek Puffer	Caninus	✓	✘
AU14	Dimpler	Buccinator	✓	✘
AU15	Lip Corner Depressor	Triangularis	✓	✓
AU16	Lower Lip Depressor	Depressor labii	✓	✓
AU17	Chin Raise	Mentalis	✓	✓
AU18	Lip Pucker	Incisivii labii (superioris and inferioris)	✘	✘
AU20	Lip Stretch	Risorius	✓	✘
AU22	Lip Funnel	Orbicularis oris	✓	✓
AU23	Lip Tighten	Orbicularis oris	✓	✘
AU24	Lip Press	Orbicularis oris	✓	–

✓ = muscle present/comparable muscle action, ✘= not identified/contracted, – = not tested

In terms of observation of facial movements, our initial database of existing expression footage and images (facial expressions) were insufficient to develop the ChimpFACS. It was necessary to collect close-up footage of the chimpanzee face in order to describe appearance changes in detail; more subtle and isolated facial movements (occurring without full facial expression combinations) were also studied and described using frame by frame analysis. Thus, our aim was to detail the movements of the chimpanzee face rather than to analyze facial expressions per se: this bottom up approach in is central to the FACS methodology. It should be noted that FACS was developed using primarily highly controlled observations of voluntarily produced facial movements (Ekman et al., 2002). Interestingly, many of the AUs are difficult to produce on demand, at least without some training. So even for humans, obtaining clear facial actions is difficult and time consuming. However, this approach is not feasible with other species; all our observations are spontaneously produced facial movements and these also give us insight into the repertoire of facial movements in the chimpanzee.

Our observations are based upon static and video images of around 150 chimpanzees; this included an existing database of over 200 short-video clips (behavioral episodes containing a main expression configuration) and over 100 static facial expression images,² in addition to around 40 h of focal footage focusing closely upon the face. Video images and static images were examined by trained FACS coders (SJV and BMW) for potential signs that a given AU had acted. These were then reviewed until a catalog of video and related still images were available for the description of the appearance changes relating to the action. This collection was studied using frame-by-frame analysis and common features were detailed for each movement. As other AUs were developed, earlier AUs were modified in light of any relevant new information. Some AUs were rarely seen in isolation, but could also be inferred by comparing combinations of movement and extrapolating the minimal units. Using the above method, we have been able to describe many of the core AUs in the chimpanzee.

The original FACS, presents each AU in terms of underlying musculature (location and direction of action), appearance changes (multiple cues for identifying AUs), reference for AUs (subtle differences between AU combinations), how to do the AU (voluntary production of AU in isolation), and intensity scoring for the AU (criteria for coding decisions). We only address the first three of these as; voluntary production was not applicable and intensity coding will be addressed in later studies. Thus, we describe each AU in terms of the underlying muscle action, the multiple appearance changes that may be used to identify the action, in addition to any other information for consideration when making coding decisions, such as the impact of co-occurring actions upon facial appearance. When translating a coding system across different face types, some morphological features can be misleading. These false indicators resemble the cues used to discriminate an AU in another population (e.g., in BabyFACS; Oster, 2003, unpublished manual). Amongst the several cues described to identify a given AU, some are usually more discriminating for the identification of an AU and these best cues are also described for each AU.

Chimpanzee and human faces differ in terms of underlying anatomy; chimpanzees have more prognathic faces with an elongated mouth, a lower forehead with more substantial brows, a flatter nasal area, much less check fat, and no bony chin boss. The contrast visible on the face is also often limited due to skin pigmentation, reduced outer lip vermillion, and substantially different eye morphology, which most notably lacks the white sclera which make human eye movements (and changes in the surrounding area) more readily perceptible (Kobayashi & Kohshima, 1997, 2001; Ekman et al., 2002). While these features can make identifying facial movement more challenging, the texture of the chimpanzee face (with many visible lines and wrinkles) can facilitate some AU identifications. Importantly, chimpanzees show considerable individual variation in many of these features, making the FACS approach of using multiple and redundant cues to detect actions extremely valuable for the usability of ChimpFACS. Figure 1 identifies commonalities and differences in facial landmarks across species.

Fig. 1.

Facial morphology in humans and chimpanzees

It is important to note that there are still some unresolved questions and practical restraints upon the perception or reliable coding of these often small and subtle movements (in highly mobile subjects). Several core facial movements are omitted; some actions are either so uncommon, or usually masked by co-occurring actions, that we currently lack good clear exemplars on which to base our descriptions. Others are simply extremely difficult to detect upon the chimpanzee face, or their precise anatomical basis has not been confirmed. Omissions are identified along with an explanation for their absence because, given the similarity in underlying musculature, these also serve to inform our understanding of how facial morphology and movements interact across species.

Results

We will follow the basic structure of the FACS manual and present core facial AUs with clearly identified anatomical basis (not miscellaneous movements) in the following groupings: (a) movements of the upper face (AU1 AU2 AU4 AU5 AU6 AU7) and movements of the lower face: (b) up/down actions (AU9 AU10 AU16 AU15 AU17), (c) horizontal actions (AU14 AU20), (d) oblique actions (AU11 AU12 AU13), and (e) orbital actions (AU18 AU22; supplemented with a very brief description of some additional movements of the mouth). The functional appearance changes caused by the action presence of an AU will be described and compared for humans and chimpanzees. A briefer summary of omitted AUs is also given for each facial movement grouping. Table 2 summarizes the comparisons of appearance changes for AUs in humans and chimpanzees.

Table 2.

Summary of AUs and comparison with appearance changes in chimpanzees

AU	Name	Comparison with original FACS
AU1	Inner Brow Raise	These are not seen to act independently in the chimpanzee
AU2	Outer Brow Raise	but entire brow raises are common and readily identifiable
AU4	Brow Lowerer	This action has not been identified in the chimpanzee despite presence of musculature, perhaps due to difference in brow morphology
AU5	Upper Lid Raise	Muscle present, difficult to identify given eye morphology
AU6	Cheek Raise	This action is readily identifiable in the chimpanzee, but seems to be less localized and more orbital in action
AU7	Lid Tightener	Muscle present, difficult to identify given eye morphology
AU9	Nose Wrinkle	Present and recognizable due to distinctive wrinkling at nasal root in both species
AU10	Upper Lip raiser	Present but distinctive squarish shape to upper lip not seen as lip has smooth elliptical shape when raised in chimpanzees
AU11	Nasiolabial Furrow Deepener	Muscle present but the absence of cheek fat may mean that this action is undetectable in chimpanzees
AU12	Lip Corner Puller	Present and readily recognized across species
AU13	Cheek Puffer	Action not seen in chimpanzee, extremely uncommon in humans
AU14	Dimpler	Buccinator muscle present but absence of cheek fat may make this action undetectable in chimpanzee
AU15	Lip Corner Depressor	Not seen to act independently, may co-occur with AU17 chin raise
AU16	Lower Lip Depressor	Present in chimpanzee but AU160 lower lip relax also added as new code to describe related movement in chimpanzees
AU17	Chin Raise	Present in chimpanzee, though absence of bony chin boss means that cues differ
AU18	Lip Pucker	Unclear whether proposed musculature present: AU not described in the chimpanzee
AU20	Lip Stretch	Musculature confirmed but not yet seen to act independently
AU22	Lip Funnel	Present in chimpanzees, though lip peaks in center and does not have characteristic squareness seen in humans
AU23	Lip Tighten	Not clearly identifiable due to lack of everted lips in chimpanzees
AU24	Lip Press	Present and readily recognized in chimpanzees

Movement and Morphology in the Upper Face

Brow raising (AU1 + 2) and cheek raising (AU6) are both evident in the chimpanzee and described in comparison to original FACS descriptions. Omissions include two subtle actions around the eyes (AU5 Upper Lid Raise and AU7 Lids Tighten), which are not reliably distinguished due to differences in facial morphology, and AU4 Brow Lowerer, although the underlying reason for this absence is less clear.

AU 1 + 2 inner and Outer Brow Raiser

Humans are able to raise the inner and outer portions of the brow (AU1 Inner Brow Raise and AU2 Outer Brow Raise) independently, and also in combination to lift the entire brow. The principle appearance change for AU1 (Inner Brow Raise) is the pulling up of the inner (or medial) portion of the brows. It causes wrinkling, usually only in the center of the forehead, and gives the brows an oblique shape. AU2 (Outer Brow Raise) similarly raises the outer (or lateral) section, which produces an arched shape to the brow and can raise the lateral eye-cover fold and leads to wrinkles above the lateral brow. When AU1 and AU2 act together, the entire brow is raised upwards, there are wrinkles across the forehead, and changes to both medial and lateral eye-cover fold as the skin beneath the brow is pulled upwards.

Unlike in humans, the medial (central) frontalis does not appear to act independently from the lateral portion of the muscle in chimpanzees, so that AU1 and AU2 are always expected to act in unison. Therefore, AU1 and AU2 are combined into AU1 + 2: both components of the label are maintained so that descriptions are contiguous with the equivalent action in humans. The horizontal forehead wrinkles which appear with the brow raising (described by AU1 + 2 in humans) are less indicative of this action in the chimpanzee. There are permanent wrinkles on the chimpanzee forehead that will deepen with this action, but these may also serve as a potential false indicator of brow raising; in order to code AU1 + 2, it is important that some deepening or movement of these facial landmarks is observed. Moreover, while the frontalis muscle is longer in chimpanzees compared to humans, in some chimpanzees the forehead is almost entirely covered by hair, so that any wrinkling will in most cases only be visible close to the brows. In chimpanzees, this AU is more readily detected by noting movement of or upon the brow itself. In this case, the entire brow is raised and this movement is often observable, particularly in individuals with large heavy brows or when viewing the chimpanzee in profile. While chimpanzees do not have the contrast of eyebrows on a hairless face, the texture of the brow often changes as the skin is pulled upwards over the brow ridge. Given the prominence of chimpanzee brows, perception of brow position is easily influenced by both chimpanzee head movements and changes to the observer’s viewing angle.

AU6 Cheek Raiser and Lid Compressor

AU6 Cheek Raiser is described as an orbital action, which pulls skin around the eyes inwards toward the eye from the temple and the cheeks. The action is the result of the contraction of the Orbicularis oculi, pars orbitalis, which circles the eyes and extends into the brow and below the lower eyelid furrow. In humans, AU6 Cheek Raiser draws in the skin around the eyes and narrows the eye aperture. This action raises the infraorbital triangle as it pushes the cheek upwards and leads to bagging and wrinkling around the eyes; the lower eyelid furrow is deepened and crow’s feet, which extend radially from the outer eye corner, may appear. This action may also lower the lateral brow and stronger actions may deepen the nasiolabial and infra-orbital furrows and even cause the lateral portions of the upper lip to raise slightly.

In chimpanzees, AU6 is clearly recognizable as the skin around the eyes is drawn inwards with similar increases in bagging and wrinkling in the eye area and narrowing of eye aperture apparent. However, the distinctive crow’s feet at the eye corner are absent. Rather than oblique lines radiating from lateral eye corner, the chimpanzee wrinkles seem to be parallel to each other and more horizontal. In addition, the lack of contrasting sclera means that the reduced aperture and its direction are not as perceptible as in humans. In addition, AU6 in chimpanzees seems to cause more change in the entire eye region, for example, lowering both medial and lateral brow, than is typically seen in humans. That is, the action seems to be more orbital and less localized in the chimpanzee. The nasiolabial furrow, which deepens in humans, is not present to be used as a cue to this action in the chimpanzee. In both humans and chimpanzees, movement causing increased wrinkling in eye area must be seen to pull medially inwards from the temple, not only to be pushed upwards as this can be result of other AUs (e.g., AU9 Nose Wrinkle which also pushes upwards on the medial eye region). Chimpanzees generally have more bagging and wrinkling around the eye area in the neutral face and these false indicators should not be used as a basis for coding AU6; there has to be movement toward the eye and an increase in bagging or pouching and a deepening of wrinkles and furrows.

Omission of AU4 Brow Lowerer, AU5 Upper Lid Raiser, and AU7 Lid Tightener

There are obvious morphological differences between the human and chimpanzee upper face, which have led to the omission of three upper face AUs. Most notably, the heavier brows and reduced contrast in terms of eye morphology (Kobayashi & Kohshima, 1997, 2001) make small movements around the eyes difficult to detect. In humans, AU5 Upper Lid Raiser and AU7 Lids Tightener both depend upon detecting changes to extent of visible sclera. These two AUs are therefore not described for the chimpanzee. These are subtle actions which may be masked by co-occurring movements, such as head and brow movements, and eye movements visible beneath the eye-lid itself. The AU4 Brow Lowerer is also omitted despite recent confirmation regarding the presence of the requisite musculature (Burrows et al., 2006). This AU is the result of combined corrugator, procerus, and depressor supercilli action in humans, which acts to lower and knit the brows together. This distinct drawing together of the brows has not been identified in the chimpanzee. Possible explanations are that the morphology of the chimpanzee brow may hinder the actions of the muscles in the manner documented for humans, or the action may be either entirely absent from repertoire (Darwin, 1872/1998) or just uncommon (Parr, Preuschoft, & de Waal, 2002). Despite this, frowns have been reported in infant chimpanzee (K. A. Bard, personal observation) and bonobos (Gaspard, personal communication) where there is an absence of the developed brow ridge.

Up/down Actions of the Lower Face

These AUs describe actions, which pull the face vertically either toward the brows or chin. AU9 Nose Wrinkle, AU10 Upper Lip Raise, AU16 Lower Lip Depress, and AU17 Chin Raiser are described, with AU15 Lip Corner Depressor being the only omission in this grouping.

AU9 Nose Wrinkle

AU9 Nose Wrinkle describes the appearance changes caused by action of the Levator labii aleque nasi muscle; this attaches close to the nostril wings and when contracted pulls upwards toward the root of the nose. As the skin adjacent to the nose is pulled, it causes distinct wrinkling along the sides and at the root of the nose. The infraorbital triangle is also raised, deepening the infraorbital furrow, and causing changes around the eyes, namely reducing eye aperture and causing bagging or pouching of the skin beneath the eyes. In addition, the inner corners of the brow are also drawn downwards toward the nasal root, as a result of recruitment of the procerus, and the medial section of the lips are also pulled upwards slightly, while stronger actions may part the lips (AU25 Lips Part). Due to presence of the procerus muscle in both actions, AU4 Brow Lowerer cannot be scored with AU9 Nose Wrinkle as brow lowering cannot be reliably identified (unless these actions have clearly asynchronous onsets).

AU9 Nose Wrinkle is readily identifiable in chimpanzees; it pulls the nostril wing area medially upwards toward the root of the nose and draws the inner brows downwards. As in humans, this action causes distinctive wrinkling at the root of the nose, bagging and wrinkling beneath the eyes, lowering of the inner brow, and stronger actions may similarly raise the upper lip. The deepening of the nasiolabial furrow is not available as a cue to this action, but there are other cues, which indicate AU9 Nose Wrinkle in chimpanzees that are not noted for humans. The nose can be seen to move upwards a little, which results in the vertical groove down the center of the chimpanzee nose visibly deepening, and also deepening of the subnasal furrow, particularly the lateral portions. There are permanent lines at the root of the nose in chimpanzees, which are potentially a false indicator of this action, so movement should be seen before this action is coded.

AU10 Upper Lip Raise

The levator labi superior muscle, which underlies the AU10 Upper Lip Raise is a little lateral to that underlying AU9 Nose Wrinkle. When this muscle contracts, it pulls the upper lip upwards, with the center of the lip being raised slightly more than lateral sections, so that the lip has a squarish, angular shape. The infraorbital triangle is pushed upwards so that the infraorbital furrow becomes more evident, the nasiolabial furrow pouches distinctively and the nostril wings are raised and widened. Stronger actions part the lips, unless another AU is acting to close the lips, and AU25 Lips Part should be noted.

AU10 is readily recognizable in the chimpanzee; the top lip is pulled upwards, shortening the distance between nose and upper lip and usually parting the lips, revealing the upper teeth and gums to a varying extent. However, retraction does not cause the upper lip to appear square (as in humans), but rather raises the lip in a smoother, elliptical curve. The upper lip thickens along its edge as it is pulled upwards and more of the inner lip usually becomes visible. Stronger actions of AU10 deepen the subnasal furrow and in profile views, the lip may appear to be bulging slightly. These appearance changes are dependent upon the presence of other co-occuring AUs, for example, AU12 Lip Corner Puller may act to pull and stretch the upper lip so that the thickening of the lip due to AU10 is less evident. The distinctive shape of the nasiolabial furrow used to discriminate this action from similar movements in humans, is not available as a cue in the chimpanzee.

AU16 Lower Lip Depressor

This action pulls the lower lip downwards toward the chin, changing the appearance of the lower lip as it is stretched and pulled laterally so that it may either protrude or flatten. This action usually parts the lips (AU25 Lips Part) and exposes the lower teeth, while lower gums may become visible with stronger actions. Flattening and wrinkling of the chin can also be seen, and a wrinkle may appear directly below the lip.

In chimpanzees, retraction of the lower lip is identifiable; the lower lip is pulled down and the lips are parted, so as to reveal lower teeth and gums, and the lower lip may be seen to protrude. While the skin may be seen to move downwards over the chin, the absence of a defined chin boss means that the flattening and wrinkling of the chin are much less evident in chimpanzees. Other actions, such as AU12 Lip Corner Puller, may serve to flatten the lip so that protrusion of the lip is no longer available as a cue for presence of AU16 Lower Lip Depressor. Caution needs to be taken to ensure that appearance changes are due to lip retraction and not to other factors which may impact upon the visibility of lower teeth and gums, such as viewing angle or, jaw dropping (AU26 Jaw Drop) or jaw stretching (AU27 Jaw stretch). Moreover, in chimpanzees, relaxation of the lower lip causes it to droop away from the face and ChimpFACS draws a distinction between lower lip depression AU16 and relaxation of the lip, which is denoted as AD160 Lower Lip Relax. It may be that this could be best described by coding AU16 an extreme intensity AU25 Lips Part, but the resulting appearance changes in chimpanzees are not comparable those listed in FACS for AU16 (the presence of a bony chin boss in humans may limit how much the lip droops). AU16 and AD160 can be distinguished by tension in the lower lip. While AU16 Lower Lip Depressor pulls the lower lips downwards toward the chin, AD160 Lower Lip Relax results in the lip drooping away from the face.

AU17 Chin Raiser

AU17 describes the appearance changes caused by the contraction of the mentalis muscle, which pushes the chin and lower lip upwards, often causing it to protrude. As the skin on the chin boss is stretched, it may wrinkle and sometimes a depression is evident below the center of lip. As the center of the lip is pushed upwards, the mouth corners appear to be pulled downwards.

Chimpanzees lack a defined boss, so that the textural changes on the chin used to identify AU17 Chin Raiser are less evident on the chimpanzee face. As in humans, the chin is pushed upwards, with the action causing the lower lip to rise and protrude so that the mouth corners may appear to be lowered. However, viewing angle influences the perception of the distance between jaw-line and lower lip, which is the best indicator of this action in chimpanzees, so that a clear upwards movement needs to be observed before AU17 Chin Raiser should be coded.

Omissions of AU15 Lip Corner Depressor

AU15 Lip Corner Depressor results from contraction of the triangularus muscle, which, pulls the mouth corners obliquely downwards and usually appears to stretch the lower lip horizontally. Although chimpanzees have the muscle underlying this movement (Burrows et al., 2006), and we know that the muscle functions in a comparable manner in both species (Waller et al., 2006), we have not yet seen any clear examples of AU15 Lip Corner Depressor which would allow us to extract cues for identifying lip corner depression. There are movements which may appear to pull the mouth corners down, namely AU17 Chin Raiser which pushes the medial portion of the lower lip upwards, and it is likely that some of our examples of stronger AU17 Chin Raiser include some degree of lip corner depression. Appearance changes used to identify this action in humans include deepening of the lower portion of the nasiolabial furrow and flattening or bulging of the chin boss, both features are absent as cues on the chimpanzee face and perhaps hinder detection of the this action.

Horizontal Actions of the Lower Face

There are only two horizontal actions of the lower face and neither of these, AU14 Dimpler nor AU20 Lip Stretcher, is described for the chimpanzee. Important differences in lower face morphology means that some features which are used to detect some actions on human faces are not available to aid identification of these movements in chimpanzees, namely the absence of cheek fat and distinctive nasio-labial furrow. It may also be the case that due to the prognathic lower face, horizontal actions are simply less perceptible in chimpanzees, however, analysis of profile footage has also failed to identify these actions.

Ommision of AU14 Dimpler and AU20 Lip Stretcher

AU14 Dimpler (buccinator action) causes tightening at the lip corners and leads to distinctive dimpling. The lip corners may be raised or lowered slightly by this action, and it may also deepen the nasiolabial furrow and cause changes to the chin boss. We have no evidence for independent action of AU14 Dimpler in chimpanzees. Although they have the buccinator muscle, which may be used during mastication, they lack the substantial cheek fat found in humans so it is not clear how this action would change the appearance of the face. While AU14 also acts to tighten the mouth corners, other actions can also do so, such as AU22 Lip Funneler, so this alone cannot be used to detect this action.

In humans, AU20 Lip Stretcher pulls the lip corners laterally backwards toward the ears as the risorious muscle contracts. This horizontal action elongates the mouth and the lips can become flattened and stretched as the corners are pulled. AU20 also results in a lateral pull on the nasiolabial furrow, wrinkling and flattening on the lower cheek, flattening or wrinkling on the chin, and a lateral pull on the nostril wings. Although there has been some debate as to whether chimpanzees have the risorious muscle that underlies this action (Waller et al., 2006), we now have confirmation that it is indeed present in the chimpanzee (Burrows et al., 2006). We have not been able to isolate and or describe any action that resembles risorious action in the chimpanzee, although it is highly probable that such action occurs with extreme vocalizations, such as during screaming, but is perhaps masked by the jaw lowering which also occurs.

Oblique Actions of the Lower Face

There are three oblique lower face AUs: AU12 Lip Corner Puller, AU13 Sharp Lip Corner Puller, and AU11 Nasiolabial Furrow Deepener. Only AU12 Lip Corner Puller is described in the chimpanzee. While chimpanzees have the muscles underlying both AU11 Nasiolabial Furrow Deepener (putatively the zygomatic minor, though the underlying musculature remains uncertain, Ekman et al., 2002, p. 176) and AU13 Sharp Lip Corner Puller (caninus), we have no examples of these movements in the chimpanzee. It may be that the cues to identifying these actions are absent, making detection problematic or that these are rare actions in chimpanzees, as indeed they are in humans (Ekman et al., 2002).

AU12 Lip Corner Puller

This action is the result of the contraction of the zygomatic major muscle, which pulls the lip corners obliquely upwards toward the ears. The nasiolabial furrow is deepened by the action and the infraorbital triangle may raise and deepen the infraorbital furrow. Stronger actions can cause appearance changes beneath the eyes, namely, bagging in the lower eyelid, narrowing of the eye aperture, and the appearance of crow’s feet. Given the theoretical significance of distinguishing whether AU12 Lip Corner Puller is co-occuring with AU6 Cheek Raise (to produce a Duchenne or felt smile, as opposed to AU12 alone which is termed nonDuchenne or social smile) it is important to distinguish between changes around the eye caused by stronger actions of AU12 which may also mask those indicative of AU6 (see Ekman & Rosenberg, 2005). The best cues that AU6 is also acting are the slight lowering of the lateral brow and the slight raise of the medial cheek; these changes are not seen when AU12 occurs independently.

The chimpanzee mouth often shows some degree of curvature, which may be mistaken for subtle AU12 action, especially when seen in static images. Nonetheless, the oblique action of AU12 Lip Corner Puller is readily recognizable on the chimpanzee face. It causes the overall shape of the lower face to alter; the entire lower face has an oval shape when seen en face but this is pulled into a more diamond like shape as the mouth is elongated into a crescent. The lips are tightened by this action, but due to the differing lip morphology this is identified by the lessening of any existing visible lip lines and a slightly fuller appearance of the lips in chimpanzees. Furrows at the mouth corners deepen into wrinkles (these are better observed in profile views, as are more subtle actions of AU12) and in stronger actions the lips will retract to reveal the teeth or gums. Similarly to humans, changes to the upper face are also apparent in AU12 actions in the chimpanzee with pouching and bagging of the infra-orbital triangle evident. Other actions may co-occur to retract the lips more fully, namely AU10 Upper Lip Raiser and AU16 Lower Lip Depressor, these can be identified by the greater amount of retraction, or by asynchrony in onsets or offsets of these actions. The distinctive deepening of the nasiolabial furrow is not evident when AU12 acts in chimpanzees, but several cheek furrows become more evident with stronger actions.

Omission of AU11 Nasiolabial Furrow Deepener and AU13 Cheek Puffer

AU11 Nasiolabial Furrow Deepener is an oblique lower face movement, which causes the upper lip to be pulled slightly upwards and laterally deepens the upper and middle portion on the nasiolabial furrow as it does so. The upper medial portion of the infraorbital triangle is subtly raised and may seem to puff a little. This is an uncommon action in humans and indeed there is considerable individual variation in the presence of zygomatic minor, which underlies this action (Pessa, Zadoo, Adrian, Yuan, & Garza, 1998, but see also Sato, 1968).

The difficulty translating this AU for chimpanzees may be due to the absence of check fat and distinctive nasiolabial furrow, as observable changes in this feature indicates AU11 has acted. AU13 Cheek Puffer pulls the mouth corners sharply upwards (at a sharper angle than AU12 Lip Corner Puller) and puffs the cheeks up as the infraorbital triangle is raised upwards. It should be noted that AU13 Cheek Puffer is an extremely rare action in humans, and to date we have no evidence of this movement as an independent observable action in the chimpanzees.

Orbital Actions of the Lower Face

Several actions are the result of the action of the Orbicularis Oris, a large band of muscle, which encircles the mouth. These orbital actions describe those actions that change the appearance of the mouth in terms of aperture and appearance of the lips. However, lip morphology differs across the two species; humans have considerable lip eversion so that the inner vermilion of the lip is highly salient and although chimpanzees have slightly everted lips (Duckworth, 1910), both the contrast and definition of the outer lip is far less evident so that these cues are much less readily detected in chimpanzees. We only focus here on two distinct actions of the orbicularis oris in humans, a puckering action (AU18 Lip Pucker which also uses the incisivor muscles) and a funneling action (AU22 Lip Funneler). In the chimpanzee, we have identified and described only the latter funneling action. Other orbital actions are noted briefly in addition to some miscellaneous orbital actions identified in the chimpanzee.

AU22 Lip Funneler

In humans, AU22 is caused by the contraction of the outer bands of the orbicularis oris muscle so that skin surrounding the lips is tightened, the lip corners are drawn medially inwards and the lips are funneled outwards. This action causes the lips to part so that AU25 Lips Part is also coded unless some other action such as AU17 Chin Raise causes the lips to remain closed. The teeth and gums are visible and the vermillion part of the lip is more exposed. These cues are usually more evident in the lower lip and should not be confused with AU16 Lower Lip Depressor. The chin boss may be flattened or wrinkled by this action. It is unusual to see this action in a single lip.

In chimpanzees, AU22 is readily recognizable as the action pushes the lips forwards so that they flare outwards into a funnel shape; the lip corners are pulled forwards and the medial section of the lips is pushed forwards into a diamond shape. The lips are usually parted to some extent (AU25) but the drawing in of the mouth corners means that the aperture is small and round, unless another AU acts to retract the lip corners, such as AU12 Lip Corner Puller. The vertical wrinkles of the upper lip can be seen to deepen as they are pulled obliquely inwards toward the center of the lip. The inner vermillion of the lips is more visible (usually more evident in the bottom lip). The chin is pulled slightly upwards as the lower lip is pushed outwards. This action can be distinguished from other AUs by the distinctive shape of the lip which appears to be pulled outwards into a peak from the center, contrasting with the more squarish appearance of this action in humans (see also AU10 Upper Lip Raise). Most notably, this action is seen relatively frequently in only one lip in the chimpanzee, primarily the upper lip.

Omission of AU18 Lip Pucker

In humans, AU18 Lip Pucker is thought to describe the contraction of the small insisivor muscles, which draw the lips medially inwards and de-elongates the mouth, making the mouth opening smaller and rounder (as if puckering up for a kiss). Short wrinkles may appear above or below the lips or on the lips themselves, and the chin boss may also appear wrinkled or flattened. The insisivor muscles are not discussed in any of the chimpanzee anatomy literature. Moreover, the elongated mouth and lack of eversion in chimpanzee lips means that this action would appear quite dissimilar on the chimpanzee. In any case, for chimpanzees we have not observed an action, which is comparable to AU18 Lip Pucker as seen in humans (actions which draw the lips inwards all contain the flaring of the lips described by AU22 Lip Funneler).

Other Movements of the Mouth

Other mouth movements have been identified in the chimpanzee in addition to the core facial movements; for example, AU25 Lips Part, AU26 Jaw Drop, and AU27 Jaw Stretch describe how the lips and jaws are parted. Although not described in detail here, we have also identified AU24 Lip Press and AU28 Lip Suck. We have not described AU23 Lips Tighten due to the difficulty in detecting such a movement given reduction in visible outer lip in the chimpanzee. Other miscellaneous mouth movements are readily identifiable in the chimpanzee, including AU32 Lip Bite, AU36 Tongue Bulge, AU19 Tongue Show, and AU30 Jaw Sideways. It is noteworthy that chimpanzees do seem to be capable of much more independence of movement in the upper and lower lips compared to humans.

Discussion

We have demonstrated that the FACS approach is amenable to modification for use with other species of primate and that this common method allows us to compare facial repertoires across species. The process of developing a FACS for chimpanzees allows us to understand how and why chimpanzee and human facial movements differ in terms of underlying musculature, facial morphology, and the interaction between these. In terms of the overall repertoire, we have successfully isolated and described nine of the most commonly seen core facial actions, and several other miscellaneous movements. In many cases, facial actions are quite readily identifiable once differences in underlying morphology have been taken into account, but for other AUs there are clear differences in appearance across species. While there are several omissions, with the notable exceptions of AU4 Brow Lowerer and AU5 Upper Lid Raiser, the omitted AUs are also less crucial to expression production in humans. For example, some of these are described as being rarely seen in humans (AU13 Cheek Puffer and AU11 Nasiolabial Furrow Deepener: Ekman et al., 2002), so that it may be that we have simply yet to see clearly identifiable actions to form the basis of their description and illustration. Other actions we think are likely to be present but are not yet clearly identified due to lack of independent action (such as AU20 Mouth Stretch and AU15 Lip Corner Depressor). Unlike the original FACS, we cannot differentiate such rare actions by training the face to perform these movements in isolation (Ekman & Friesen, 1978), or even by having more stringent control over observations in terms of lightening, viewing angles, and so on, so that these masked actions can be more readily extracted from any co-occurring movements. In some cases it is unclear whether omissions are due to these being extremely uncommon movements in the chimpanzee facial repertoire or due to the absence of salient cues for detecting these actions. However, due to the close similarities in muscle plans between these species (Burrows et al., 2006) it is clear that this is not the cause of divergence, for example, as had been suggested as an explanation for the absence of frowning in nonhuman primates (Fridlund, 1994).

The chimpanzee face lacks the various sources of contrast seen in humans; for examples, white sclera and everted lips. The most striking aspect of the translation process is the importance of contrast for the detection of facial movement; even when those facial movements are carefully studied, the lack of contrast makes the movement less salient in the chimpanzee. It has already been suggested that human eyes have more signal value than those of other primates (Kobayashi & Kohshima, 2001), but only gaze direction detection was considered and it is apparent that lack of scleral contrast also impedes the detection of facial actions in the eye region. While chimpanzees have larger more prominent brows than humans, movements of the brow are also enhanced in humans with the contrast against the hairless forehead (Campbell et al., 1999). Likewise, our everted lips with the visible vermillion make subtler movements, which change the shape of the lips far more readily detectable. It would be interesting to explore the psychophysics of facial movement across species in order to assess the contribution of contrast to expression recognition. However, not all differences are simply a matter of contrast in facial features. Notably, the lack of cheek fat means that the nasiolabial furrow, which is used to help identify several AUs in humans and is also important for expression recognition (Rubin, 1999), is absent in the chimpanzee and thus discriminating these movements is challenging.

It is obviously difficult to speculate why human facial morphology takes the specific form that it does. Some features may be related to function, such as nasal structure, while other may be primarily sexually selected, such as the retention of neotonous features into adulthood (Morris, 1967). It has been noted that primates often have distinctive facial morphology and coloration in infancy (Preuschoft, 2000; Kuze, Malim, & Kohshima, 2005), and it may be that this signal of immaturity aids their interactions with other group members. However, as we have described, facial morphology may also impact upon expression recognition in important ways and thus infant facial morphology may also serve some specific communicative function. In any case, human facial morphology is particularly well suited to producing highly salient changes in facial appearance with even the subtlest movement. In addition, several facial expressions have been shown to be visible at some distance, with the smile being most salient and identifiable at up to 100 m (Hager & Ekman, 1979). Chimpanzees have both a similar facial muscle plan and a wide repertoire of independent movements, but their facial morphology generally does not seem to enhance the visual impact of facial movements in the same manner as in humans. However, chimpanzee facial expressions can also result from extremely salient appearance changes, such as retraction of the lips to reveal inner lips and gums. It would thus be of interest to measure expression recognition distances in chimpanzees to assess how readily these signals are perceived by others, perhaps in relation to their social function (e.g., Waller & Dunbar, 2005). While the uniqueness of human facial morphology is by no means a novel observation, our approach has allowed us to identify exactly which factors impact upon the detection of facial movements across species, in terms of the specific cues used to identify each action.

Interestingly, people show high agreement when attributing personality and other attributes based solely on facial morphology (e.g., Zebrowitz & Collins, 1997). Conversely, expressions can also change the facial features to give the impression of a different facial morphology. For example, fear expressions may serve to make the face appear more neotonous (wide eyed), while anger expressions may enhance the perceived maturity of facial features (by lowering the brow and narrowing the lips, Marsh et al., 2005). It remains unclear whether other facial expressions will yield so easily to this form of analysis, but it is clearly important to consider the interactions between morphology and movement, and their subsequent interpretation by others. Such factors may be particularly influential when considering perception of facial displays in other species; configural face processing may bias the perception of expressions due to differences in the precise spatial relationships between features.

Finally, the success of the FACS itself means that there are many paradigms developed for studies with humans, some of which may be suitable for modification for studies with chimpanzees (see Ekman & Rosenberg, 2005). That is, the development of ChimpFACS allows for directly comparable methods to be employed in cross-species research and contribute to the questions surrounding both chimpanzee communication and the evolution of human facial expressions. As Oster and Ekman (1978) state in relation to the FACS method, ‘‘a similar system for different non-human primate species would be valuable in attempts to establish homologies between human and nonhuman primate expressions’’ (p.245). To date, questions of homology have primarily been answered on the basis of similarity in appearance of expressions (Preuschoft & van Hooff, 1995) and a more detailed and anatomically based means of comparison is long overdue (Fridlund, 1994). It is only by making detailed cross-species comparisons with the same measurement tool that we will gain a true understanding of what is and what is not uniquely human.