Development of Face Processing

Recognition of individuals at first sight is extremely important for social species such as human and non-human primates. An individual’s decision to fight, flee, mate, cooperate, or seek protection often depends on the outcome of such recognition. If some species recognize individuals using olfaction or audition (1), primates will primarily do it via their visual system, using faces.. Although it is still controversial as to whether humans’ superior face processing abilities reflect an innate, specialized face processing mechanism or a more general phenomenon of visual learning due to massive exposure to faces (2, 3,4,5), most researchers agree that: (a) adults are experts in face processing, and (b) adults’ ability to process faces is tuned by their experience.

Faces are multi-dimensional visual stimuli, providing a broad range of information to an observer (6). This information can be organized into two major categories: face traits and face states. Face traits refer to the visual information in the face that is relatively permanent and stable. These traits include: facedness (face or non-face); species (e.g., humans or dogs); gender (male or female); race (e.g., Chinese or Caucasian); aesthetics (attractive or unattractive); age (old or young); and identity (e.g., John or Mary). Face states refer to dynamic and transient facial cues. Face state information is used to process speech (e.g., the McGurk effect), emotional expressions, attention, and intentions (7). Exactly which components of the face processing system are present at birth, which develop first, and at what stage the system becomes adult-like are still hotly debated topics. However, we will not talk about face states here.

Viewing a face triggers at least two automatic and fast processes: categorization of the stimulus as a face belonging or not belonging to our own group or species, and recognition of the face at an individual level. Bruce and Young posit a stepwise model whereby categorization of faces according to their race, gender, and other social attributes happens at an early stage of “structural encoding”, before the ‘face recognition units’ or the ‘person identity nodes’ are involved. In this framework, categorization refers to the process by which information about a stimulus is used to relate the stimulus to classes exhibiting common traits, whereas individuation or recognition is the process by which unique traits are linked to an individual (8). According to this model, when presented with a face, we first determine automatically to which species it belongs, its gender, race and then proceed to individuating information. However, opposing this sequential model, the recent work by Ge et al. (2009) shows that individuation and categorization can be achieved at different rates depending on the kind of face processed (9). We will comment further on the distinction between categorization and recognition in the course of our review.

The aim of the present article is to review our current knowledge and understanding of the development of the face processing system from birth, during infancy, and through childhood, until it becomes the sophisticated system observed in adults. We will first provide a brief review of our current knowledge and understanding of the adult face processing system. Secondly, we will present an overview of the development of the face processing system. Finally, we will review the various theoretical accounts/models that have been proposed to explain the development of the system.

Face Processing as Expertise

Over the last three decades, a great deal has been learned about human adults’ ability to process faces (10). The evidence that adults are experts at face processing includes their ability to discriminate between highly similar unfamiliar faces and their extraordinary memory for hundreds of faces that is relatively unaffected by the passage of time (e.g., for 35 years: 11). Many visual effects are particularly pronounced with faces, compared to non-face stimuli. One example is the inversion effect which is one of the most commonly studied effects in face recognition. It refers to the fact that faces are recognised more accurately and faster when presented in their canonical orientation than when presented upside-down (12). Two types of information have been identified as crucial for face processing (10). Featural information refers to aspects of a face that can be thought about in relative isolation. Among the different types of face featural information, a distinction has been made between the internal face features (eyes, nose, and mouth) and the external features (hairstyle and jaw-line). While both types of featural information are important for face recognition, the internal features are considered to be more critical in adult face processing expertise (13, 14, 15). The other important source of information is the configural that refers to the spatial relationships between the features of a face (e.g., distance between the eyes, nose, and mouth). Holistic information refers to the facial “gestalt”, which represents a fusion of featural and configural information into an integral and unbroken whole (e.g., 10) (see figure 1). It is admitted that the inversion effect is due to our inability to use configural processing when presented with upsidedown faces. Another face-specific effect is the Tanaka-Farah effect: face parts are better recognized in an intact face than in isolation relative to non-face objects (houses) or inverted faces (10).

Figure 1.

Example of a featural change (i.e., eyes) made by switching the eyes of the two original faces on the left to make the two altered faces on the right (A). Example of configural changes (i.e., male face: spacing between the eyes; female face: spacing between the nose and mouth) with original faces on the left and altered faces on the right (B). Example of stimuli that can be used to examine holistic face processing (C). If asked to decide whether the top parts of two faces are identical or different, the composite faces on the left (which have different bottom parts) make it harder to simply process the top part of the face in isolation from the bottom part of the face – a task that is easier when the top and bottom parts of the face are segregated as shown on the right.

Categorization-Individual Recognition

It has been suggested that whereas novices categorize visual objects at the basic level of abstraction, perceptual experts identify objects in their domain of expertise at a more specific, subordinate level (17, 18). For example, a novice will identify a brown object with feathers and wings at the basic level of “bird,” but an expert bird watcher will identify this same object at the subordinate level of “white crowned sparrow.” Thus, a hallmark of perceptual expertise is a downward shift in recognition to a more specific, subordinate level of abstraction. Face expertise constitutes the most specific, downward shift in recognition in that a familiar face can be identified at the level of the unique individual (e.g., Bob, Mary) rather than categorized at the more generic level of race or gender (19).

It is hypothesized that as face expertise increases, the accuracy and speed for processing the individuating information improves at the expense of processing the categorical information. This categorization-individuation trade-off associated with expertise has strong empirical support. For example, Levin (20) found that while Caucasians recognized Caucasian faces better than African faces (own-race recognition advantage; Adults are typically more proficient at recognizing faces from their own racial group, as opposed to faces from other racial groups . This is commonly known as the other-race effect and will be discussed later.), paradoxically, they were better at racial classification of African faces than Caucasian faces (i.e., other-race categorization advantage). It should be noted that earlier experiments were inconsistent regarding the other-race categorization advantage, likely due to issues of stimulus selection, participant types, and procedure (21). Subsequent studies have shown the effect to be robust once these issues have been addressed. Furthermore, Ge et al. recently significantly extended Levin’s findings (21). They found that the size of Chinese and Caucasian participants’ other-race categorization advantage was reliably correlated with the size of their own-race recognition advantage. This evidence suggests that when a person becomes expert at recognizing the identity of faces in one category (e.g., Caucasian faces), their ability to categorize those faces becomes “compromised” (i.e., is slower and less accurate) relative to the categorization of faces in the non-expert category.

Featural-Configural Processing

Another, more narrow and perhaps more controversial criterion of face expertise is the ability to process configural as opposed to featural information during face identity recognition (10). Although feature information may be useful for adult face recognition, researchers have shown that adults rely heavily on, and are highly proficient at processing facial configural information. Thus, by this configural-featural definition, becoming a face recognition expert means that one needs to acquire the ability to process the configural, as well as, if not better than, featural information. However, more recent work has suggested that both featural and configural information play crucial roles in face perception (22).

Role of Experience in Face Processing

Researchers of adult face processing agree that experience plays a critical role in the acquisition of face expertise. However, exactly how experience exerts its effect is not clear. Results from the Greeble (an artificial visual stimulus) training studies (3) suggest that processing at the subordinate level may improve gradually in a quantitative way. However, experimental (23) and modeling (24) studies suggest that becoming a face expert may involve qualitative changes at certain points (25). Valentine (26) suggested that faces are encoded in a face-space framework in terms of vectors in a multidimensional perceptual space. The origin of the space represents the average of all faces experienced by an individual. More typical faces are close to the origin and more distinctive faces are further out in the space. New faces are encoded in terms of their deviation from this norm face. Recent work suggests that by age five, the face space of young children approximates the face space demonstrated by young adults (27).

The face space model also provides an explanation of the other-race effect.. The origin of scientific interest in this area stemmed from an observation that is still made by people today, which is: ‘they (people of other-races) all look the same to me’. Face spaces are tuned toward the faces present in the environment and are less effective in processing faces from a different ethnic group. Frequent exposure to a certain face category (e.g., one’s own race or species) eventually causes a warp in the face space that maximizes the differences between faces of the familiar category such that they can be optimally discriminated (24). The cost of the warping process is that one becomes less proficient at processing less frequently experienced face categories (e.g., faces of other races).

Nelson (25) draws a parallel between language development and the development of face recognition. In language development, it has been shown that experience results in infants’ ability to discriminate sounds from their native language better than sounds unique to non-native languages (28). According to Kuhl’s (29) theory, early contact with a native language restructures one’s innate perceptual space to maximize differences between sounds within that language, which may result in difficulty discriminating sounds of non-native languages. Similarly, Nelson proposes that the face processing system develops during the first years of life from a broad non-specific system to a human-tuned face processor. Furthermore, he suggests that the faces observed within the infants’ visual environment shape and influence the developing face system.

Limited evidence exists regarding the specifics of the processes that generate face expertise. Training adults to become experts in processing artificially created visual stimuli (e.g., Greebles) is a viable approach, given excellent stimulus control and flexible experimental manipulation. Also, it allows researchers to draw causal conclusions. However, despite these strengths, this approach using artificial stimuli has several shortcomings. First, expertise with Greebles is acquired after participants are already expert processors of another class of stimuli (faces). Adults may recruit their face processing expertise to learn about Greebles. In other words, the adult training approach may only reveal how one acquires a new expertise based on a pre-existing expertise (akin to second language acquisition (30)). In contrast, face expertise develops naturally from “scratch”, along with many other types of expertise (e.g., language) from an initial state of immaturity to adult-level sophistication. Second, while one can create artificial categories for the Greebles (e.g., gender), it is difficult to mimic natural face categories and the amount of exposure that individuals experience during the development of face expertise. Training participants to individuate other-race faces also improves subsequent performance in recognition tasks (31). Thus, while the training approach with adults is useful to address some research questions, a developmental approach seems better suited to examine how face expertise develops naturally in children.

Development of Face Processing

The development of face processing during infancy and childhood is one of the most extensively researched topics in perceptual development (6, 32). There is general agreement that face processing begins at birth because newborns: prefer face-like stimuli (see figure 2) to non-face stimuli (4); imitate another’s facial movements (33); prefer familiar over unfamiliar faces (34, 35); and attend to attractive over unattractive faces (36). However, some of these preferences at birth may reflect newborns’ general tendency to look at certain stimulus-general configurations that happen to be similar to faces (37). Despite this early start, researchers also agree that face expertise development is protracted with some investigators arguing that children reach adult levels only after puberty (38). More recently, however, Crookes and McKone (39) suggested that the face processing system was already very much mature by 5-6 years of age. The extensive literature notwithstanding, surprisingly, the role of experience in the development of face processing ability has until recently received little attention.

Figure 2.

Example of a face-like stimulus (left) that is preferred over a non-face stimulus (right) by infants.

Categorization-Individual Recognition

In the last two decades, studies have indicated that infants process faces as members of various categories in a manner similar to adults. For example, around 3 months of age, they categorize nonhuman animal stimuli such as cats versus dogs based on face information (28). During the initial months of life, they categorize stimuli into human vs. nonhuman categories, cats vs. dogs, and even different breeds of cats and dogs (30). In the next several months, they categorize human faces into male vs. female and prefer female faces (40), attractive vs. unattractive faces and prefer attractive faces (41), and Caucasian vs. Chinese faces (42). They also categorize faces in different age categories, preferring child over adult faces (43). In addition, Rennels and Davis (44) reported that infants experience far more female faces than male faces in their daily interactions with others. Quinn, et al. (40) found that infants who were raised by mothers preferred female over male faces while those raised by fathers preferred male over female faces. Kelly et al. (45, 46) showed that both Chinese and Caucasian 3-month-old infants prefer looking at their own-race faces than at other-race ones. These findings suggest a role for experience in early face category processing. Overall, the existing evidence indicates that children are not simply processing individual faces; they also categorize faces according to race, species, gender, and age, and show preferences for one face category over another that are due to selective experience.

Featural-Configural Processing

With regard to configural face processing, there has been considerable controversy (6). The influential encoding switch hypothesis (47) posited that before 10 years of age children relied mainly on isolated facial features to recognize faces and after 10 years, children begin to rely on configural information. The subsequent three decades of research has led to significant challenges to this theory. Counter evidence comes from developmental studies that directly and indirectly manipulated configural information. In all cases, infants as young as 3 months of age and children under 10 years have been found to be sensitive to configural information and use such information in face identity processing. For example, sensitivity to configural changes among facial features emerges between 3 to 5 months of age (48; 49). In addition, Slater et al. (36) found that newborns’ preference for attractive faces is disrupted when the faces are inverted. Quinn et al. (40) found that 3-month-old infants’ preference for female faces over male faces is not found if the faces are presented with inversion. Turati, Sangrigoli, Ruel, and de Schonen (50) investigated 4-month-old infants’ ability to recognize inverted faces when stimuli are learned in various poses. They found a deficit which contrasted with infants’ abilities to recognize faces both in the upright condition and the inverted condition when familiarization was done with the same picture. In children, many studies have demonstrated the existence of inversion effects even at a young age (6). Collectively, these findings refute the strong interpretation of the encoding switch hypothesis. However, there has been consistent evidence to show that young children may have more difficulty processing configural than featural information (6). For example, Mondloch et al. showed that children do not reach the level of adult performance in a configural task until 15 years of age (51). One possible conclusion that can be drawn from this existing and still controversial literature is that whereas younger children show greater sensitivity to featural information relative to configural information their configural processing ability still undergoes significant development from childhood to adolescence.

With regard to featural face processing, the developmental story is that featural processing of faces begins at birth, with newborns and young infants attending more to the outer (e.g., hairline) than internal face features (eyes, nose, and mouth). The latter begins about 2-3 months of age (52). Newborn recognition of the maternal face can be disrupted by removing the hairline (35), suggesting that primitive facial identity is based on both the inner and outer facial features, with the outer contour dominating initially. Turati et al. (53) investigated the importance of outer and inner information for unfamiliar faces. Newborns were familiarized with the full face, just the inner part, or just the outer part of the face. The recognition test was then conducted with the identical picture paired with a novel picture. The results showed that children could recognize the familiar picture in all the conditions. They then investigated recognition of inner parts when a full face had been learned or recognition of full face when only the inner part has been learned. Their results show that newborns do not present signs of recognition if important changes occurred between the habituation and the test phase. These results are very important as they demonstrate that newborns are able to process the inner or outer facial features, but recognition of a stimulus that has been learned with all face information available can hardly be done when some of the features are removed.

Infants rapidly acquire the ability to process the internal features of a face (54). Between later infancy and childhood (55, 14), it has been shown that featural processing becomes adult-like (although with better processing of the eyes relative to the mouth and nose) except that children are generally less accurate at remembering face features than adults. Also, their face feature memory is more vulnerable to extraneous interferences than that of adults (e.g., confusions due to similarity and paraphernalia: 55). However, Ge et al. (21) found that experience still affects children’s (as well as adults’) face feature processing: for example, children tend to rely on outer features more when processing unfamiliar faces but inner features more when processing familiar faces. With regard to the development of holistic face processing, evidence to date suggests that infants as young as 6 months of age are able to process holistic information (54). By the preschool to early elementary school years, children show similar holistic information related effects (e.g., the Tanaka-Farah effect) as adults, but the magnitude of these effects does not reach the level of adults until adolescence (56, 57).

Role of Experience in Face Processing

In the last decade, there has been significant progress in research on the role of experience on children’s processing of face category and individual face identity. At this point, it appears that experience with certain face categories does affect children’s ability to categorize and recognize faces. For example, Quinn et al. (40) reported that mother-reared 3-month-olds looked longer at a novel female over a novel male face after habituation to a number of different male faces, but they showed no preference for novel male faces at testing after habituation to a number of different female faces. This asymmetry may reflect the possibility that female faces receive more visual attention than male faces. The female faces may also be more deeply encoded than the male faces. For example, mother-reared 3-month-olds looked longer at a novel female over a familiar female face after habituation to a number of different female faces, but they showed no preference for a novel male face over a familiar one at testing, after habituation to a number of different male faces.

The gender findings are in line with several theories on the representation of expertise in the adult perceptual learning and categorization literature that have posited the downward shift in recognition with increased experience with exemplars within a domain (3). They are also consistent with a hypothesized representational shift from a summary structure (i.e., a category) to exemplar-based individual memory (i.e., identity) during the time-course of category learning (57). Consistent with this idea, Pascalis et al. (23) have found that 6-month-olds show recognition memory for different human faces as well as for different monkey faces. However, such recognition memory for individual monkey faces in the same species disappears by 9 months of age unless experience with such faces is provided (59). Anzures et al. (42) also found that 9-month-old Caucasian infants could both categorize and individuate Caucasian faces (referred to as categorization in the literature); however, they engaged in categorical perception of Chinese faces, where the Chinese faces were responded to equivalently and not individuated. Clearly, own-race experience already plays an important role in infants’ categorization of face race.

With regard to the effect of race category on individual face processing, our recent studies showed a similar perceptual narrowing phenomenon: while 3-month-old Caucasians prefer own-race faces, they are able to discriminate among faces of African, Middle Eastern, Chinese, and Caucasian faces, followed by 6-month-olds’ ability to discriminate only Chinese and Caucasian faces, and then at 9 months of age only Caucasian faces (60). Experience with other races and lack of further experience with own-race faces in early childhood has also been shown to reverse this other-race effect in recognition. For example, Korean children adopted into Caucasian families in a predominantly Caucasian environment exhibited an advantage in their recognition of Caucasian faces relative to their recognition of Korean faces when adults (61). However, deHeering et al. (62) tested children who had been adopted only a few years before and did not find such a difference. They observed that the children were equally good at discriminating own-race and foster parent-race faces. Pascalis et al. (59) investigated whether or not it was possible to maintain the more general processing properties of the face system by exposing infants to other species’ faces between the ages of 6-9 months. Six-month-olds were exposed regularly to monkey faces during a three-month period and their ability to discriminate monkey faces was then assessed at 9 months. Their discrimination performance was compared with a control group of 9-month-olds who received no training. Nine-month-old infants in the training group demonstrated recognition of monkey faces whereas the control infants did not. In addition, maintaining accurate recognition memory for monkey faces in the same species appears to require not just mere exposure to monkey faces, but rather, exposure to such faces accompanied by consistent individuation (i.e., repeatedly referring to each monkey face by a specific name) between such faces (63)

These and other recent studies suggest that experience (as defined by general and specific contact with individuals of other races) affects children’s processing of individual faces as early as 3 months. In addition, our recent study (15) showed that Chinese children become increasingly better at recognizing their own gender faces, likely due to increased gender segregation during the school years. Another recent study (64) showed 3-year-olds to have differential ability at recognizing different aged child and adult faces and that such ability is influenced by their experience (e.g., whether they had a sibling).

So far we have demonstrated that the face processing system is developing very fast during infancy and that experience is sculpting this cognitive ability. The system remains, however, extremely flexible and the child’s face processing system is not adult-like until late adolescence. Can we explain the paradox of a very efficient system present at 1 year of age and a seemingly less efficient system in children that continues to differ from the adult one until much later? From our review of the literature on the development of face categorization and face recognition, it is clear that more data are available during infancy than during childhood. There are also more consistent results during infancy than during childhood. This might be due to the fact that researchers have higher expectations from children than from infants and are perhaps not taking into account other cognitive factors such as memory and attention (16; 39).

Neural Correlates of the Development of Face Processing

It is important to note that imaging and electrophysiological analyses are providing a different view of the development of face processing. In adults, faces elicit a negative deflection around 170 ms after stimulus onset (N170) which is of larger amplitude and shorter latency than the one elicited by objects (65). The N170 is found to be of larger amplitude and longer latency for inverted human faces compared to upright human faces, but not for animal faces or objects (66, 67). During development, when 6-month-olds are presented with a face, an ‘infant N170’ in 6-month-olds is recorded at 290 ms, and a similar latency range for both upright and inverted faces is observed. This is followed by a positivity at 400 ms that is different when the face is inverted (66). When presented with monkey faces, 6-month-olds’ ERPs are similarly affected by the inversion. At around 12 months of age, the adult ERP patterns are observed (68). Until recently we thought that the N170 does not become adult like until late in childhood as illustrated by a series of studies conducted by Taylor and her collaborators (69). If the N170 was clearly found for faces across age groups its latency, however, was much longer in young children and had not reached adult values by mid-adolescence. However, recently Kuefner et al. (70) have demonstrated that the changes observed are linked to the N100. This wave is large and may be contaminating the following N170. When the N100 is taken into account, the N170 seems rather adult-like early on. This result supports the Mckone and Crookes (39) view that the face processing system is well developed during childhood and that the differences observed are more likely reflective of general improvement in other cognitive functions.

fMRI studies have identified 3 cortical areas that appear to be at the core of face processing in adults: the Inferior Occipital Gyrus (‘OFA’), Middle Fusiform Gyrus (‘FFA’), and Superior Temporal Sulcus (STS) . These areas are more activated in humans while viewing human faces than while viewing other objects (71). There is a general agreement that the neural circuits involved in face processing will differ somehow during early infancy, but it is rather controversial to determine how much and when it becomes adult-like and whether additional neural substrates are involved in face processing during infancy and childhood. There are two alternative views of the development of the neural system of face processing. One favours a specialized system from the beginning which will of course develop with the cortical maturation over the years. The second shows a scenario where specialization does not appear before childhood and its development into the adult system is complete only around 15 years of age.

Only two studies have looked at face processing during infancy with imaging techniques. Using a Positron Emission Tomography (PET) technique, Tzourio-Mazoyer et al., (72) found brain activation in 2-month-old infants presented with faces in brain regions that are activated by faces in an adult population in both fMRI and PET studies. It is important to note that the PET technique has a poor spatial resolution and prevents any conclusions regarding the similarity of the structures involved. However, faces also activated areas that are typically devoted to language in adults, suggesting an early link between the visual and auditory systems or a more distributed network. Otsuka et al. (73) recorded the hemodynamic response of the brain, using near infrared spectroscopy, while 5- to 8-month-olds were watching upright or inverted faces and objects. The sensors were placed over the temporal and occipito-temporal region which correspond to the location of the STS in adults. They found that upright faces create a different signal relative to objects, but not inverted faces.

Several fMRI studies have been investigating the development of face processing at different ages. Some studies have reported an FFA activation in 10- to 11-year-olds but not at earlier ages(73; (75). In a more recent study, Golarai et al. (76) compared recognition of objects and faces in children (7 to 11), teenagers, and adults. They found similar activation in the same areas for the three age groups for objects. For faces, they demonstrated that the FFA is three times larger in adults than in children which may explain why some studies could not find it in young children. These new techniques are helping us to understand what changes are happening between infancy and childhood, and if they are from a general domain or specific to face processing.

Concluding Remarks

This review of the developmental literature reveals early competence in face processing abilities, with infants presenting a preference for face stimuli and facial discrimination using featural, configural, and holistic cues. This early competence is then later refined as evidenced by age-related changes throughout childhood. Some of the refinements are likely due to the development of general cognitive abilities, whereas some others (e.g., configural processing) may be face-specific.

Although biological factors may initially play some role in biasing the newborn’s visual system towards faces in their environment, the existing evidence overwhelmingly suggests the important role of experience in the development of face processing expertise. The role of experience has been implicated in all aspects of the development of face processing expertise and from infancy through childhood. For example, it is evident that experience plays a crucial role in infants’ discrimination and children’s identity judgments for different categories of faces (i.e., species, race, gender, age), with better recognition abilities for the more familiar face categories (i.e., own-species, own-race, female, and own-age). However, considering the quasi-experimental nature of the existing studies, well-controlled experiments that directly manipulate experience (e.g., training studies) are needed to establish fully the causal linkage between differential experience and the development of face processing abilities.

The present review of the existing literature on the development of face processing has also revealed significant gaps in our research endeavors. While most of the recent exciting discoveries have been made with infants in all aspects of face processing, relatively limited knowledge has been gained about childhood face processing abilities, except for the development of facial configural processing. Future studies need to examine how children’s classification of faces at the basic and various subordinate categorical levels develops with age and to what extent such classification is related to their increasing abilities to process faces at the individual level, which in turn will provide a more comprehensive developmental account of the formation of face processing expertise.