More Than Meets the Eye: Split-Second Social Perception

Abstract

Recent research suggests that visual perception of social categories is shaped not only by facial features but also by higher-order social cognitive processes (e.g., stereotypes, attitudes, goals). Building on neural computational models of social perception, we outline a perspective of how multiple bottom-up visual cues are flexibly integrated with a range of top-down processes to form perceptions, and we identify a set of key brain regions involved. During this integration, ‘hidden’ social category activations are often triggered which temporarily impact perception without manifesting in explicit perceptual judgments. Importantly, these hidden impacts and other aspects of the perceptual process predict downstream social consequences – from politicians’ electoral success to several evaluative biases – independently of the outcomes of that process.

Visual Perception of Social Categories

Based upon the mere glimpse of another individual, knowledge about that person’s gender, race, and other social categories seems to spring to mind spontaneously. The visual construal of social categories feels instantaneous and immediate, as if it were a direct product of ‘reading’ visible facial cues. This experience of social perception aligns with early research on the topic, which emphasized its automatic, immediate, and unavoidable nature [1–3].

A considerable body of research has shown that multiple social categories are perceived spontaneously when we encounter another individual [2]. Consequently, early work in the field of social psychology focused on either the inevitability of categorization or on its varied downstream consequences. Meanwhile, research in the cognitive, neural, and vision sciences sought to uncover the determinant cues and basic mechanisms driving perceptions of facial stimuli. Recently, a unified social vision (see Glossary) approach has emerged [4–6] in which the visual aspects of social perception are theoretically and empirically integrated with the products that follow. This approach moves beyond the historic disciplinary divide in which each level of analysis was studied in relative isolation. As such, the social vision approach has afforded novel insights into how we form our initial perceptions of others, revealing important complexity that occurs before a stabilized judgment is formed. Moreover, such dynamics during the process of forming categorical percepts of other people have recently been shown to carry downstream social consequences – predicting politicians’ electoral success, producing gender bias, and even eliciting prejudice against racial or sexual minorities.

Because downstream interpersonal consequences typically took center-stage in social perception research, a feed-forward approach was generally assumed (Figure 1A, Key Figure). As such, perceptual cues were presumed to activate a single, dominant social category representation, which in turn elicited related stereotypes, attitudes, and goals, thereafter impacting evaluative judgments and interpersonal behavior [1–3]. In this article we offer a more comprehensive perspective in which social perception is characterized as the dynamic integration of both visual processing of facial features and higher-order social cognitive processes (e.g., stereotypes, attitudes, goals) that were historically probed only as a ‘downstream’ product of social perception.

Figure 1 Key Figure. Dynamic Interactive (DI) Model of Social Perception.

(A) A feed-forward approach to social perception (top) assumes that the face of a target is visually encoded by some feature space or set of feature detectors, which thereafter activates related social categories. Those categories then automatically activate related stereotypes, attitudes, and goals, which affect and may be affected by high-level behavior. The dynamic interactive (DI) model [4] (bottom), posits additional feedback influences inherent to the system, wherein stereotypes, attitudes, and goals constrain and inform category activation and featural representation. In contrast to the feed-forward approach, the DI model predicts that category activation (e.g., male) and featural representation (e.g., larger jaw) are impacted not only by lower-level visual processing but also by activated stereotypes (e.g., aggressive) in a top-down fashion. Featural representation is simplified for illustration; feature spaces using pixel-based models or computational models of ventral-visual processing may also be implemented. Permission to use the sample face image was obtained [86]. (B)Aneural network for flexible split-second social perception, including the fusiform gyrus (FG), orbitofrontal cortex(OFC), and anteriortemporal lobe (ATL). The FG is involved in visual processing of faces, and the ATL retrieves social-conceptual associations related to perceived characteristics such as social categories (e.g., stereotypes, person-knowledge). Such social-conceptual information may then be used by the OFC to implement top-down visual predictions that modulate FG representations of faces in line with those predictions. This network supports rapid and flexible integration of bottom-up facial cues and higher-order social cognitive processes.

One formalization of this approach is the dynamic interactive (DI) model, a neural computational model of social perception [4] (Figure 1A). The DI Model treats an initial percept as an automatic but gradual negotiation between the multiple visual features inherent in a target of perception and the prior knowledge or high-level cognitive states that a perceiver brings to the perceptual process (e.g., stereotypes, attitudes, goals). Thus, initial perceptions are not discrete ‘read outs’ of visual features; they evolve over hundreds of milliseconds and may be dynamically shaped by context and higher-order social cognition. Some models of social reasoning and high-level impression formation have long considered similar integrative processes and feedback influences [7], but these treated initial social perceptions as theoretical givens and as input into the model that thereafter impacted the higher-order processes of interest. By contrast, the DI model aims to explain and predict the initial split-second perception itself [4].

Malleability and Bias of Initial Perceptions

Perceivers are highly sensitive not only to discrete social categories of faces but also to within-category variation in their cues (e.g., gender or racial typicality). Such variation impacts initial categorizations [8,9], stereotype activation [10,11] and application [12], evaluative judgments [13], and other interpersonal behaviors. For instance, more prototypically Black faces more strongly activate Black-related stereotypes, and this portends harsher criminal sentences [14] including capital punishment [15]. Similarly, higher levels of gender typicality impact gender categorizations [8], activate stronger gender-based stereotypes [11], and elicit more favorable social evaluations [16]. For example, American female politicians with more feminine facial features enjoy higher levels of electoral success, particularly for elections held in conservative American states [17,18]. In general, such typicality biases persist even when controlling for evaluative factors such as perceived attractiveness and competence (e.g., [17]). Moreover, effects of implicit stereotypes and attitudes often exist independently of the gender, race, or other category memberships of the perceiver because they tend to be learned, although not necessarily endorsed, by all members within a society [19].

Although downstream evaluative biases commonly stem from negative attitudes related to a perceived category (or deviations from that category), recent work reveals that some biases are rooted in the initial perception of the category itself. Prejudice against mixed-race individuals is predicted by a more unstable perceptual experience during the processing of mixed-race faces – one that involves abruptly shifting race-category representations – an effect that is stronger in White observers who have little exposure to Black individuals [20]. Similarly, prejudice against sexual minorities occurs in part because gender-atypical faces tend to be categorized as gay owing to stereotypical associations [21] and such faces are therefore more difficult to process. Specifically, the effect of gay categorization on negative evaluations is statistically mediated by greater difficulty in perceiving the gender of the face [22]. Moreover, systematically exposing observers to either masculine or feminine features in visual-adaptation paradigms produces visual after-effects that shift perceived norms for what is regarded as ‘typical’ and ‘atypical’, thus also altering social evaluations. For instance, exposing observers to more extreme levels of masculine features adapts the visual system such that a subsequent face requires more masculinity to be perceived as male. Faces that deviate from this new male norm (which now appear more feminine to the perceiver) are more difficult to categorize, in turn leading them to be evaluated more negatively [23]. Thus, some forms of prejudice may originate in aspects of the initial perceptual process itself, such as processing ease or stability, independently of the categorical outcomes of that process [24].

Social perceptions are not only sensitive to bottom-up cues originating in the target of perception but also to top-down factors harbored within perceivers, such as stereotypes [25,26], goals [27–29], and prior person-knowledge [30,31]. Which race-category is initially perceived from a face is influenced by prior racial labels [32], stereotypes [25,33], social dominance [34], visual context [35,36], and political and economic factors [29,37]. Perceptions of the race, gender, and emotion categories of a face are biased by stereotypical expectations, and these biases manifest at relatively early levels of visual representation in the brain [91]. Higher-order goals, such as self-protection motives, can also bias perceptions, leading ambiguous targets to more readily be perceived as male or as Black due to associations with the potential to harm [16].

Thus, extant work suggests that initial perceptions are not solely determined by visible cues, thereafter driving evaluation and behavior (Figure 1). Instead, perceivers are sensitive to subtle variation in facial features that impact the perceptual process, and perceptual experiences themselves can affect subsequent interpersonal processes independently of explicit categorizations. In addition, social perceptions are malleable and are influenced by feedback from higher-order social cognitive processes traditionally considered as being downstream from initial perceptions. Together, these findings suggest that initial perceptions involve a rapid and flexible integration of multiple bottom-up visual cues and top-down processes. Next, we delve into the underlying mechanisms that permit such integration in social perception.

A Landscape of Hidden Social Categories

Social targets inhabit multiple categories simultaneously (e.g., young Asian female). Classic work suggested this ‘multiple category problem’ is solved by only one dominant category coming to the fore, while others are actively suppressed [38,39]. Recent research, however, reveals that multiple categories both across and within dimensions coexist throughout processing [4,40]. Although perceiver judgments stabilize on a single categorization rapidly, numerous ‘hidden’ categories are concurrently activated – even though these parallel activations may not be apparent in explicit behavioral responses. These concurrent activations are elicited by both bottom-up cues that originate in the target of perception (e.g., subtle feminine cues on a male face partially activate the female category [41]) and top-down factors harbored within the perceiver (e.g., stereotypical expectations that Black individuals are hostile partially activate the angry category even when not displaying any anger [42,91]. While bottom-up and top-down processes dynamically impinge on social perceptions, competition between category alternatives via lateral inhibition typically clears such ‘incorrect’ hidden activations from processing [4] (Figure 1).

Evidence for hidden social category activations has typically utilized computer mouse-tracking paradigms (Box 1). Participants rapidly categorize faces by moving the computer cursor from the bottom-center of the screen into one of two category responses in either top corners. The premise is that the hand in motion reveals ‘mind in motion’ and the real-time unfolding of cognitive processes. Such mouse-tracking paradigms have now been leveraged in many domains across the cognitive sciences [43–45]. Early studies explored natural within-category variation in facial cues and the hidden activations they trigger. No face is a perfect prototype of a given category – instead inhabiting middle-points along the pertinent social-perceptual continua (e.g., gender, race) – and this is subtly revealed in mouse trajectories. For instance, when categorizing a face bearing subtle cues of the opposite gender, participants’ mouse trajectories exhibited a partial attraction toward the opposite gender-category response before ultimately selecting the correct category [8]. This provided evidence that an alternative social category (e. g., female) became partially activated, even though it did not manifest in the explicit perceptual judgment (e.g., male). Once activated, these partial-parallel activations of social categories (e.g., female) cascade into the partial-parallel triggering of associated stereotypes (e.g., caring) [11] (Figure 1). Importantly, these temporary activations also predict meaningful downstream consequences and real-world outcomes, independently of category membership itself (Figure 2A).

Box 1. Computer Mouse-Tracking Paradigm.

Mouse-tracking tasks have been used across a variety of research areas in the cognitive sciences, including recent social perception research [43,45,87]. Although a two-choice paradigm is most popular, many other arrangements exist. In such tasks, the trajectory of the hand is recorded during the real-time evolution of a behavioral response, providing highfidelity information about a participant’s tentative commitments to multiple alternatives over time. This allows a discrete reaction time or button-press to be opened up into a continuous stream of cognitive output, which is warranted because motor dynamics are coextensive with cognitive dynamics [43]. For instance, in perceptual decision-making tasks, monkey neurophysiological work and human electroencephalography (EEG) studies demonstrate that, when required to make a hand-guided response, information continuously flows into the premotor cortex driving the motor response while the decision process evolves over time [88,89].

Participants begin a trial by clicking a button at the bottom-center of the screen, after which they are presented with the stimulus (e.g., a face). They then quickly move the cursor from the bottom-center of the screen to response options in either top corners (e.g., ‘male’ vs ‘female’). Despite their explicit selection, the hand movements of the participant can simultaneously exhibit a ‘hidden’ partial attraction toward the unselected alternative (on the opposite side of the screen), indicating a competing, partial activation of that alternative. Numerous properties of these competitive motor dynamics are informative for understanding cognitive dynamics, including the magnitude of that activation and its real-time stability (Figure I). The millisecond-resolution timing of these dynamics is also highly valuable for dissociating the temporal profile of when different cues are processed, when different cognitive processes come into play, or how individual or crosscultural differences shape processing [36,47,90]. In the same way as responses involving motor demands in other speeded tasks (e.g., longer reaction times in semantic or evaluative priming, or the IAT) are often interpreted as reflecting implicit processes, there is good reason to suspect that mouse-trajectory dynamics in speeded tasks may also reflect implicit processes. That said, future research would benefit from exploring the more implicit or explicit nature of these dynamics, their boundary conditions, and the roles of automaticity and control.

Figure I. Mouse-Tracking Measures for Understanding Cognitive Dynamics.

A ‘hidden’ partial activation of the male category is illustrated, indicated by the attraction of the hand to the male category response. (A) The magnitude or strength of the activation can be measured by maximum perpendicular deviation (broken line) or area under the curve (blue highlight) relative to an idealized straight-line response trajectory (examples in Figure 2A,B in main text). (B) The stability or instability of category activation dynamics can be measured by the number of abrupt directional changes along the horizontal x axis (axis of decision) (example in Figure 2C in main text).

Figure 2. Examples of ‘Hidden’ Social Category Activations.

In the mouse-tracking tasks, one face was presented at a time at the center of the screen, and participants moved the cursor from the bottom-center into response options at either top corners of the screen. (A) The extent to which participants were partially attracted to the ‘male’ response when categorizing the face of a female politician (i.e., the strength of partial male category activation) predicted a decreased likelihood of winning her electoral contest, particularly in US conservative states [17]. The trajectories depicted are a schematic illustration of the main result. (B) When categorizing faces by race, low-status attire (e.g., janitor uniform) led trajectories to be partially drawn to the ‘Black’ response owing to stereotypes linking Blacks to low status. Although not depicted, high-status attire (e.g., business suit) conversely compelled trajectories toward the ‘White’ response [25]. Adapted from [25]. (C) When categorizing racially-ambiguous faces, White participants with higher exposure to Black individuals showed more stable coactivation of the White and Black categories. Those with lower exposure to Black individuals showed more unstable co-activation, indicated by abrupt White–Black category shifting [20]. Additional work suggested these unstable category-activation dynamics were due to stereotypes that White and Black individuals are highly dissimilar, which affects the process of perception (Figure 3). The trajectories depicted are a schematic illustration of the main result. Adapted from [20].

The hidden triggering of social categories has now been observed during perceptions of gender, race, age, emotion, and sexual-orientation categories when specific facial cues resemble an alternative category [9,8,42,46,47]. These concurrent activations are also elicited by the top-down influence of stereotypes [25,42], prior knowledge [46], and expectations due to context [36]. For instance, owing to stereotypical associations between racial categories and social status, the hand trajectories of participants were initially attracted to the Black response when categorizing White faces surrounded by low-status attire (e.g., janitor uniform), and attracted to the White response when categorizing Black faces surrounded by high-status attire (e.g., business suit); these stereotypic biases were exacerbated as race became more ambiguous because bottom-up ambiguity opens the door wider to top-down influences [25] (Figure 2B).

These concurrent activations differ not only in their strength but also in their qualitative dynamics. When the environmental context surrounding a target is more consistent with an alternative category (e.g., a White face in an Asian-related scene), the alternative category became partially activated, as indicated by a temporary partial attraction toward the alternative (e.g., Asian). Interestingly, this temporary activation occurred significantly earlier among perceivers from more collectivist, ‘high-context’ cultures than among perceivers from more individualist, ‘low-context’ cultures. This finding implies that a cultural preparedness to process context shapes the timing of its impact on perception [36]. Additional perceiver characteristics, such as exposure to different racial groups, influence the stability with which racial categories activate during initial perception. For Whites with lower interracial exposure, racially-ambiguous faces elicited coactivations of the White and Black categories that competed in a more unstable fashion, as indicated by abrupt, back-and-forth shifting between category responses and higher velocity in hand motion (Figures 2C and 3). In turn, this more unstable perceptual experience of racial ambiguity predicted more negative evaluations of mixed-race individuals. Beyond interracial exposure, hidden impacts on perception and temporary category activations are sensitive to several individual differences, including the strength of stereotypic associations [48,91], attitudes [42], cultural differences [36], and neurodevelopmental disorders [49] – often without any impact on explicit judgments or ultimate perceptual outcomes.

Figure 3. Race Perception and Interracial Exposure.

The dynamic interactive (DI)model treats social categories as attractors. In this illustration, two energy minima indicate the White and Black attractor states in a system in which two forces jointly drive race perception: visual processing (blue arrows) and conceptual knowledge (red arrows). Conceptual knowledge (red arrows) reflects the natural force in the system to minimize energy and descend down the landscape, resembling the compulsion of a ball to roll down a hill. The trajectory of the system (represented by the motion of the ball) reflects the process of race perception, where gradual descent into an attractor reflects an increasing activation of that social category, and an increasingly confident interpretation that the category is represented in the visual input. When confronted by a racially-ambiguous face, visual processing initially seeks to drive the system toward the ridge in between the attractors, ‘pushing’ the White and Black activations together to be in line with the ambiguous facial input. Conceptual knowledge, by contrast, seeks to rapidly ‘pull’ the two categories apart, seeking to drive the system into one alternative or the other. When White and Black categories are more conceptually dissimilar (as in White individuals with lower interracial exposure), the attractors will be more differentiated [20], creating a steeper descent and a stronger ‘pull’ to descend into the categories. When low-exposure perceivers are presented with racially-ambiguous faces, visual processing tries to ‘push’ together two categories that conceptual knowledge is trying to rapidly ‘pull’ apart, creating more unstable dynamics. Computational simulations and mouse-tracking findings (Figure 2C) support these predictions [20]. Adapted from [20].

Inherent Intersection of Social Categories

Social category dimensions were historically considered to be processed in isolation, but recent studies have demonstrated their inherently intersectional nature. For example, the DI model (Figure 1) predicts that multiple social categories will be perceived interdependently when stereotypes related to two ostensibly unrelated categories overlap. This is because the processing of one category dimension (e.g., gender) will activate stereotypes that in turn bias the perception of other category dimensions (e.g., race) while perception is still unfolding. To the extent that two social categories (e.g., male and Black) share stereotypes (e.g., ‘aggressive’), those social categories themselves will become inextricably linked, down to a perceptual level.

Accordingly, recent studies have discovered that race categories bias gender perception and vice versa, such that Black female faces partially activate the male category and Asian male faces partially activate the female category during face perception. These ‘race is gendered’ effects are exacerbated for perceivers holding stronger overlapping stereotypes between Black and male categories, and Asian and female categories [48,50]. These perceptual biases also have lasting impacts, predicting leadership selection or interracial dating [51]. Other categories also become perceptually entangled with race and gender, such as emotion. Stereotypically, men and Black individuals are both associated with expressing anger, while women are associated with expressing joy; consequently, emotion perception is biased in a consistent fashion [33,52].

In a set of fMRI studies [91], participants viewed faces varying by gender, race, and emotion, and additionally completed post-scan mouse-tracking tasks (Figure 4). The extent of a participant’s conceptual (i.e., stereotype) overlap between pairs of categories was also assessed. Overall, mouse trajectories showed intersectional bias toward a related but orthogonal category: male and Black toward anger, female toward happy, Black toward male, and Asian toward female. Despite these overall tendencies, participants’ unique stereotype overlaps between categories predicted the extent of those stereotypes’ hidden impact on perceptions. At the neural level, these stereotypically biased effects on perception were reflected in the similarity of the multivoxel representations of the categories in the fusiform gyrus (FG) and orbitofrontal cortex (OFC) (Figure 1B). For instance, a stronger attraction to select the ‘angry’ response for Black faces predicted greater neural-pattern similarity between angry and Black category patterns in these regions. This held true even when stimuli were matched on low-level visual properties and after controlling for several bottom-up visual models (e.g., pixel-based similarity or models of ventral-visual processing), ensuring that such perceptual biases were not due to mere physical confounds (Figure 4). That these stereotype-driven biases on perception manifest in the representational structure of visual face-processing regions (FG) supports the notion that top-down feedback from social cognitive processes can reach relatively low levels of perceptual representation. Finally, additional studies have found that brain regions associated with conflict monitoring and inhibition may also be involved in helping to correct these temporary biases during perception (e.g., attraction to ‘angry’ for a non-angry Black face) once they become activated [42].

Figure 4. The Inherent Intersection of Social Categories.

Illustration of the MVPA representational similarity analysis findings of [91]. At the top left, an example dissimilarity matrix (DM) depicts the similarity/dissimilarity in conceptual knowledge of all pairwise gender, race, and emotion categories, based on trait ratings regarding social categories (no faces involved). Blue colors indicate similarity and red colors indicate dissimilarity. For instance, the Black and Angry categories tended to be similar, indicating conceptual overlap (bottom left). At the top middle, another example DM depicts the similarity/dissimilarity in perceptions based on mouse-tracking data, where similarity indicates bias. For instance, Black and Angry faces tended to elicit similar mouse-tracking patterns, such as a greater bias to perceive Black faces as ‘Angry’ even when not displaying angry cues (bottom middle). The structure of the conceptual DM significantly predicted the structure of the perceptual DM, which in turn predicted the structure of multi-voxel patterns in the fusiform gyrus (FG) (and orbitofrontal cortex, OFC). Thus,in this example, a greater conceptual overlap between Black and Angry categories predicted a greater bias to perceive faces belonging to those categories similarly (e.g., Black faces as angry), and this in turn predicted a greater similarity in multi-voxel patterns for the Black and Angry categories when viewing such faces in the scanner.

A Neural Network for Flexible Social Perception

As outlined above, recent behavioral studies point to a split-second social perception process that involves flexible integration of multiple bottom-up visual cues and top-down social cognitive factors – a process that often involves the hidden triggering of multiple category activations that impact perception. At the neural level, extant research suggests that a network of three regions is important for this process, the FG, OFC, and the anterior temporal lobe (ATL), which are all densely interconnected [53,54] (Figure 1B). We focus on these regions for their specific role in integrating bottom-up and top-down information during split-second perception of the face of another person, but clearly numerous other regions participate in additional components of social perception more broadly.

The FG is centrally involved in face perception, and it is highly responsive to face and face-like stimuli [55]. Whereas extrastriate areas are sensitive to physical variations in facial features, the FG is more sensitive to abstract, categorical distinctions [56]. Indeed, the FG has been shown to be specifically involved in representing social-categorical information from faces. For instance, activity in the FG is sensitive to within-category variation in social categories [57], and multi-voxel patterns within the region reliably distinguish gender, race, and emotion categories [58–61]. Although bottom-up visual cues are clearly the driving input to the region, FG activity can also be shaped by higher-order social cognitive processes, including stereotypes [91], goals [62], and implicit biases [61–63]. Indeed, recent studies combining multi-voxel pattern analyses (MVPA) with mouse-tracking suggest that hidden social category activations driven by such higher-order processes impact initial perceptions via biased representations in the FG [91] (Figure 4).

Although models of face perception recognize that an extensive network of downstream regions are involved in subsequent processing once a face is initially analyzed [64], these regions are generally not regarded as participating in the initial face-perceptual process. However, recent evidence suggests that the OFC and ATL may be involved in this process, helping to situate the face within a larger social context and align perceptual representations with higher-order processes. Indeed, the notion that top-down feedback from higher-order neural systems may modulate lower-level visual processing is hardly new, and top-down effects have been observed as early as primary visual cortex (V1) for simple visual stimuli [65–67]. In cases of face stimuli, a primary target of top-down feedback is the FG [68,69].

Although in theory any number of social cognitive processes may constrain initial perceptions (e. g., stereotypes, attitudes, goals), the top-down impact of stereotypes has the strongest support at a mechanistic level. Stereotypes are merely semantic associations activated by social categories, and these are theorized to automatically trigger expectations and predictions [4]; more generally, semantic contexts have long been known to constrain perception of objects or words [70,71]. Indeed, models such as the DI model [4,20,25] (Figure 1) find that the top-down effects of stereotypes on face perception occur through computational principles that are similar to classic context effects on word perception [70]. As stereotypes are implicit expectations generated during perception [10,11], they may influence face perception analogously to how context influences object or word perception (Figure 5), supporting the possibility of domaingeneral mechanisms.

Figure 5. The Many Faces of Context.

The top-down impact of stereotypes on face perception shares a fundamental similarity with more general top-down impacts of context on object or word perception. Cues in the visual periphery of a face or even within a face (e.g., its other category memberships) provide a context that may be semantically (i.e., stereotypically) associated with the focal percept. (A) An identical ambiguous object is readily perceived to be a drill when in a garage but to be a hairdryer when in a bathroom. Adapted from [71]. (B) An identical ambiguous A/H letter is readily disambiguated as ‘A’ in the context of ‘CAT’ but ‘H’ in the context of ‘THE’ as a result of prior knowledge. (C) Contextual attire cues bias perception of a racially-ambiguous face to be White when surrounded by high-status attire, but to be Black when surrounded by low-status attire, owing to stereotypic associations between race and social status [25]. Adapted from [25]. (D) Contextual cues may be embedded within the face itself, such as when an emotionally-ambiguous face is perceived to be angry when male, but happy when female, as a result of stereotypic associations linking men to anger and women to joy [52,91].

Considering this similarity with other context effects, numerous studies have implicated the OFC in modulating visual-object recognition via top-down expectations activated by contextual or associative cues [72,73]. Top-down perceptual ‘priors’ from the OFC may have a modulatory role on ventral-visual representation [69,71], and expectations about face stimuli in particular enhance top-down effective connectivity from the OFC to the FG [68,69]. Magnetoencephalography (MEG) studies show that activity related to successful object recognition is present in the OFC 50–85 ms earlier than in ventral-visual object-processing regions, consistent with the ability of the OFC to exert top-down modulation on perceptual regions [74]. The OFC has additionally been linked to integration of faces’ social category cues with context-based expectations about those cues [72], the retrieval of person knowledge [75], and the accessing of stereotypes [76,77].

The OFC also responds to within-category variation in gender and race [35,57], and recent work has found that neural patterns in the OFC and FG exhibit a representational structure of faces’ multiple social categories (gender, race, and emotion) that are biased by stereotypical associations [91]. For example, when categorizing Black faces that exhibited no actual angry cues, the degree of a participant’s temporary attraction to select the ‘angry’ response predicted greater neural-pattern similarity between Black and angry patterns in these regions [91] (Figure 4). One possibility is that, in response to a face, social-conceptual associations are automatically retrieved (e.g., Black-angry), in turn leading the OFC to implement a top-down prediction of ‘angry’ that is then imposed on FG representation. Thus, recent studies suggest that the OFC plays an important role in initial social perceptions, possibly by implementing visual predictions based on stereotypes, person-knowledge, or other socially-relevant expectations, to then sensitize perceptual regions (such as the FG) to fall in line with those predictions.

Although the OFC may implement visual predictions, it likely coordinates with the ATL to access social-conceptual associations that form those predictions. The ATL is consistently activated during the storage and retrieval of semantic associations, and one popular theory argues for its role as an amodal ‘semantic hub’ [78]. However, the ATL exhibits selective responses to social as opposed to non-social concepts and associations, such as person-knowledge and stereotypes [58,79,80], leading some to argue for its selective role in the processing of social-semantic knowledge [79]. It also commonly activates in response to faces, especially when associated with prior knowledge [79,81], and monkey studies show that ATL neurons rapidly generate associations between face stimuli and related abstract knowledge [82]. Demonstrating a causal role in activation of social-conceptual associations, temporarily disrupting ATL activity decreases the associative strength of stereotypes (e.g., Arab = terrorist), as measured by the implicit association test (IAT); however, no such decrease is observed for non-social associations [83]. Such studies support ATL involvement in accessing a diverse range of social-conceptual information, which may then be used by the OFC for visual predictions that in turn modulate FG face processing.

Taken together, emerging research suggests that a key network involving the FG, OFC, and ATL is involved in integrating bottom-up facial cues with top-down socially-relevant expectations or social-conceptual information to form initial perceptions of others.

Concluding Remarks

Traditional approaches have emphasized that visual construal of the social categories of other people reflects an automatic ‘read-out’ of facial features, which thereafter unleashes a linear chain of events from initial perceptions up to higher-order processes. In light of current evidence, we argue here that initial social perceptions are, in fact, hardly ‘initial’. They reflect a dynamic cascade of interactive influences, wherein factors that were long presumed to be downstream products of social perception, such as stereotypes, attitudes, and goals, constrain initial perceptions. In this process, bottom-up visual cues inherent to targets, and a range of top-down social cognitive processes harbored within perceivers, are rapidly integrated to form split-second perceptions. As this integration unfolds, multiple ‘hidden’ category activations may impact perception without being observed in explicit perceptual judgments. Although quickly cleared from processing, these hidden perceptual impacts and other aspects of the initial perceptual process can linger, driving important downstream social consequences, independently of the outcomes of that process. This overall perspective is consistent with neural computational models of social perception (Figure 1) and points to a key network comprising the FG, OFC, and ATL in helping to form these perceptions. In such a network, perceptual representations of another’s face available in the FG may be routinely modulated by top-down visual predictions of the OFC, which are guided by social-conceptual associations retrieved in the ATL.

Together, the work presented here suggests that the nature of social perception may be far more bidirectional than is typically appreciated. Not only are visual perceptions of other people malleable to higher-order social cognition, but the dynamics of these perceptions, in turn, may uniquely impact higher-order processes and bear downstream effects on social evaluation and behavior. However, such research is only emerging and many questions remain (see Outstanding Questions). Moreover, this recent work has predominantly focused on perceiving social categories, but in the future this perspective can be leveraged for other instances of social perception, such as face-based perceptions of traits or mental states. In addition, although our focus has been on perceptions via facial cues, perceptions via bodily or vocal cues show similar integrative effects, and how these function together warrants further investigation [26,84,85]. We are hopeful that advances in understanding split-second social perception will be furthered from burgeoning social vision approaches and increasing crosstalk between social psychology and the cognitive, neural, and vision sciences.

Outstanding Questions.

Extensive research has explored face-based trait inferences (e.g., perceiving trustworthiness and dominance) and social evaluation (e.g., positive/negative) from faces. Do such perceptions share similar mechanisms as perceiving social categories? How can computational and neural models of perceptions involved in social categorization be integrated with those involved in trait inference and social evaluation?

Much research outside social neuroscience has examined the role of the OFC in top-down expectations that constrain visual-object processing, which provides a precedent for understanding how top-down stereotypical expectations may constrain face processing. Other higher-order factors such as attitudes and goals can affect perceptions, but do such top-down effects depend on OFC modulation or other mechanisms?

The functional relationships between the FG, OFC, and ATL, as well as their precise roles in the context of initial social perceptions, remain unclear. How can network modeling, connectivity analyses, and other techniques such as MEG clarify the interplay and timing of these regions’ participation in flexible social perception?

At exactly which levels of representation do top-down impacts of social cognitive processes manifest? MVPA combined with mouse-tracking and other sensitive behavioral techniques will likely be important in addressing this question. How can such work inform recent resurgences of the argument for cognitive impenetrability of perception?

The flexibility and bias in initial social perceptions can be considered as being both adaptive (facilitation by context and expectations that streamlines processing; motivational biases to selfprotect) but also maladaptive (perceptual errors, suboptimal performance, as well as stereotyping). What are the evolutionary origins of such top-down effects on perception: do they represent an adaptation or a byproduct of more general perceptual-cognitive interactions (or some other process)?

Trends.

Recent research shows that visual perceptions of the social categories of others are not only highly sensitive to bottom-up facial features but are also affected by higher-order social cognitive factors (e.g., stereotypes, attitudes, and goals).

Emerging work suggests a rapid and flexible integration among multiple bottom-up visual cues and top-down social cognitive processes – a process that often triggers ‘hidden’ social category activations that are not observed in explicit perceptual judgments.

Aspects of the initial perception process itself appear to drive important downstream social consequences (e.g., evaluative biases or politicians’ electoral success) independently of the outcomes of that process.

Recent studies point to a key network comprised of the fusiform gyrus, orbitofrontal cortex, and anterior temporal lobe in helping to form flexible social perceptions through an integration of facial cues and top-down social-conceptual information.

Glossary

Attitudes

a positive or negative evaluation related to a social category (or person, object, thing, or event). Attractors: a state toward which a dynamical system tends to evolve. Computational model: a mathematical model used to study the behavior of complex systems. Neural computational models provide an algorithmic and process-level description of cognition using core principles of information processing in neural systems

Effective connectivity

analyses that use statistical models of functional neuroimaging data to infer that the activity of one neural region is exerting a directed and causal influence on the activity of another neural region

Electroencephalography (EEG)

a noninvasive technique that measures electrical potentials in the brain, and which provides high-resolution temporal information but low spatial resolution

Extrastriate areas

cortex next to V1, involved in early visual processing of stimuli

Feed-forward

a system or form of processing in which information moves in only one direction in a directed fashion without any feedback influences

Implicit association test (IAT)

a reaction-time measure used to assess the strength of the implicit associations in memory of an individual, independently of any conscious expression

Lateral inhibition

the process of an excited neuron, node, or representation diminishing the activity of its neighbors, creating competition between them

Magnetoencephalography (MEG)

a noninvasive technique that measures magnetic fields generated by electrical currents in the brain, and which provides high-resolution temporal information and typically better spatial resolution than EEG (although worse than fMRI). Multi-voxel pattern analyses (MVPA) or multi-voxel representations: a multivariate approach to fMRI data where individual stimuli, conditions, or tasks are distinguished on the basis of voxel response patterns

Social vision

an emerging field combining social psychology and vision science

Stereotypes

beliefs, expectations, or conceptual associations related to a social category, which may be positive, negative, or neutral, and which perceivers may implicitly hold without necessarily endorsing personally

Ventral-visual processing

ventral-visual processing or representation refers to the ventral pathway in the brain, extending from V1 through extrastriate areas and ventral regions of the temporal cortex, where visual stimuli are processed in detail and represented

Visual after-effects

effects where prolonged exposure to stimuli on a visual dimension (e.g., orientation, color, facial gender) results in adaptation that biases perception of other stimuli on that dimension in a systematic fashion