Out of Sight but Not Out of Mind: Unseen Affective Faces Influence Evaluations and Social Impressions

Abstract

Using Continuous Flash Suppression (CFS), we demonstrated in four experiments that affective information extracted from unseen faces influences both affective and personality judgments of neutral faces. In four experiments, participants judged neutral faces as more pleasant or unpleasant (Studies 1 and 2) or as more or less trustworthy, likable, and attractive (Study 3) or as more or less competent or interpersonally warm (Study 4) when paired with unseen smiling or scowling faces compared to when paired with unseen neutral faces. These findings suggest that affective influences are a normal part of everyday experience and provide evidence for the affective foundations consciousness. Affective misattribution arises even when affective changes occur after a neutral stimulus is presented, demonstrating that these affective influences cannot be explained as a simple semantic priming effect. These findings have implications for understanding the constructive nature of experience, as well as the role of affect in social impressions.

Food is delicious or distasteful. Paintings are beautiful or ugly. People are nice or mean. Perceptions of the world around us are always infused with affective meaning. Indeed, objects in the world are said to be “positive” or “negative” by virtue of their capacity to influence a person’s affective state. If the perception of a snake (or a person) involves unpleasant, high arousal affect, then the snake (or person) is said to be negative and arousing. In this article, we introduce the use of continuous flash suppression (CFS) to examine how affective information can be unhinged from its source and misattributed to a concurrently perceived stimulus. In CFS, perceivers are presented with dynamic visual images to one eye, while the other eye is presented with a still image. Participants experience seeing only the flashing dynamic images while the still image remains unseen, suppressed from conscious experience (Tsuchiya & Koch, 2005). In four studies, we found that the affective consequences of unseen faces are misattributed to consciously seen structurally neutral faces, so that the visible neutral faces are judged as pleasant or unpleasant (Studies 1 and 2) or as more or less trustworthy, likable, and attractive (Study 3) or as more or less competent or interpersonally warm (Study 4). CFS provides a laboratory model for understanding the constructive role of affect in consciousness, and in so doing suggests a perceiver’s affective state plays a role in normal experience.

The current research builds on the affect-as-information perspective, where “affect assigns value to whatever seems to be causing it” (p. 393, Clore & Huntsinger, 2007; also see Clore et al., 2001; Schwarz & Clore, 1983). “Seems” is the crucial word here, because it is very easy for affective changes to be caused by one thing but for perceivers to misattribute this change to another. Thus, people are more satisfied with their lives on sunny (as opposed to rainy) days (Schwarz & Clore, 1983). Buildings appear higher after perceivers view high arousal photographs (Stefanucci & Storbeck, 2009). Chinese ideographs are more pleasant when people are briefly exposed to smiling (vs. scowling) faces immediately before making judgments (Murphy & Zajonc, 1993; Winkielman, Zajonc, & Schwarz, 1997). Thirsty people pour and drink more of a fruit-flavored beverage and are willing to pay more for it when they have been previously exposed to brief presentations of smiling compared to frowning faces (Winkielman, Berridge, & Wilbarger, 2005; also see Berridge & Winkielman, 2003; Payne, Cheng, Govorun, & Stewart, 2005). As these examples show, affect plays an important role in how we experience and report on the world, even when the reaction originates from an unknown or irrelevant source and even when people are largely unaware of the affective changes.

In this article, we build on this prior research to explore how affective misattribution applies to perceptions of other people. When an invisible image contains affective information, will that affective information be incorporated in to their experience of another person, such that an otherwise neutral person is experienced as having real affective importance or value for well-being?

To explore affective misattribution, we used the phenomenon of continuous flash suppression (CFS, Tsuchiya & Koch, 2005). In CFS, perceivers view dynamic visual images in one eye, while the other eye is presented with a still image. Participants experience seeing only the flashing dynamic images while the still image remains unseen, suppressed from conscious experience.

For a number of reasons, CFS offers a promising paradigm to explore affective misattribution. First, CFS renders images invisible for long periods of time (up to 3 minutes, Tsuchiya & Koch, 2005) so that affect need not be manipulated with a second task such as a mood induction. As a consequence, it is possible that perceivers remain truly unaware that affective stimuli have been presented.

Second, the mechanisms underlying CFS allow much clearer inferences about the processes underlying affective misattribution during visual processing. Although the use of CFS cannot prove any particular neural hypothesis in a behavioral experiment, this work was actively inspired by existing neuroscience research (using it as a “context for discovery”), and so we present a brief review of these findings here. A highly schematic sketch of the neuroscience of visual processing and the links between the visual system and the brain’s affective workspace can be found in Figure 1. There is accumulating evidence that at the initial stages of perception, the brain, via fast, low spatial frequency (magnocellular) pathways in the dorsal visual stream (the “where is it” and “what do I need to do about it” stream), quickly makes an initial prediction about the “gist” of the scene or object to which visual sensations refer (Bar, 2007; Bar et al., 2006). These brain regions project to the medial part of orbitofrontal cortex (also called the mediodorsal system; Barbas, 1988; Carmichael & Price, 1995; Öngür, Ferry, & Price, 2003) within 80 ms in a feedforward sweep (Bullier, 2001; Lamme & Roelfsema, 2000; also see Bar et al., 2006; Rempel-Clower & Barbas, 2000; Rudrauf et al., 2008; Shipp, 2003; Thorpe, Rolls, & Maddison, 1983; for a recent review, see Pessoa & Adolphs, 2010). This portion of OFC regulates autonomic reactivity both by projecting directly to spinal cord autonomic nuclei, and by multiple cascading projections to striatal, hypothalamic, and brainstem areas that themselves project to spinal cord autonomic nervous system regulation centers (Öngür, Ferry, & Price, 2003). In this way, visual information is relayed to the brain’s affective workspace in less than 100 ms after stimulus onset. These signals not only create the substrate that is available for subjective affective experience (via the insula), but via the lateral OFC (also called the basoventral system) they are projected back to modulate the ventral visual stream (e.g., Barbas, 1988; for a review, see Kveraga, Boshyan, & Bar, 2007) whose activation is correlated with visual consciousness (Tong, Nakayama, Vaughan, & Kanwisher, 1998). Because ventral stream activity is initiated 15–30 ms after the dorsal stream (because the ventral stream carries slower, higher spatial frequency parvocellular information; Klistorner, Crewther, & Crewther, 1997; Laycock, Crewther, & Crewther, 2008), there is ample time for affect in the perceiver (which can also be modulated by other sources) to become part of the conscious rendering of a percept, producing affective misattribution. In this way, affect might literally shape the way people experience the world around them.

Figure 1. Neuroanatomical Depiction of Affective Vision.

During perception, the brain, via magnocellular pathways in the dorsal visual stream (where is it and how do I act on it), makes an initial “gist” level prediction about the scene or object to which visual sensations refer. These areas project to the medial part of OFC, which regulates visceromotor control and has multiple cascading projections to striatum, hypothalamus, brainstem and spinal cord autonomic regulation centers, changing the perceiver’s body state. About 15–30 ms after the onset of this process the visual system begins to process more detailed visual input via parvocellular pathway in the ventral stream that is associated with visual consciousness. This information reaches lateral OFC quickly. With a lot of back and forth between visual cortex and the more lateral areas of prefrontal cortex (via the direct projections that connect them), a finer level of categorization is achieved until finally the object is precisely recognized. This 15–30 ms magnocelluar advantage is the window of affective projection where the body state is changed and helps shape visual perception. For ease of presentation, many other connections are not shown here (e.g., reciprocal connections between the ventral and dorsal visual streams, as well as reciprocal connections between the medial and lateral sections of the OFC). Medial OFC dorsal stream connections are similar to Barbas and Pandya’s (1989) “mediodorsal” system within the orbital sector of the prefrontal cortex, whereas the lateral OFC ventral stream connections are similar to their “basoventral” system.

CFS takes advantage of these neuroanatomical connections for the purpose of studying affective misattribution. Some unseen images (particularly tools) suppressed under CFS are predominantly represented in the dorsal visual stream (Almeida, Mahon, & Caramazza, 2010; Almeida, Mahon, Nakayama, & Caramazza, 2008; Fang & He, 2005). As a result, it might be possible to present affective information with a suppressed image, and then this affective information would influence how concurrently viewed stimuli are evaluated.

The neuroanatomical connections between visual streams and the OFC are important, because they indicate something about the mechanism by which affective misattribution during CFS happens. Because CFS seems to prioritize dorsal stream processing, any effect observed in a CFS paradigm would likely not occur via traditional semantic priming. Therefore, affective misattribution created with CFS might be due to a different mechanism than the “spreading activation” mechanism thought to account for affective priming in sequential priming paradigms (e.g., Ferguson, Bargh, & Nayak, 2005). As such, CFS studies are not simply a conceptual replication of prior studies, but instead might reveal a novel mechanism by which affect can serve as information.

Furthermore, the affective misattribution created by CFS builds in important ways on previous experiments on “unconscious emotion.” These studies relied on brief presentations of affective stimuli that were then backwardly masked to prevent the perceivers from becoming aware of the triggering stimuli. Recent evidence has suggested that briefly presented primes used to manipulate the affective state of the perceiver were probably not completely invisible (Pessoa, Japee, Sturman, & Ungerleider, 2006). Briefly presented, masked objects are subjectively invisible but some perceivers have objective awareness of the objects (Pessoa et al., 2006) and the objects produce some activity in the ventral visual stream (Dehaene et al., 2001) that is typically associated with visual awareness. Furthermore, masks themselves can blend or merge with the briefly presented target stimulus to construct an amalgam of a percept (rather than the mask simply replacing the target in visual cortex; Kim et al., 2010). In contrast, images suppressed under CFS seem to be predominantly processed by the dorsal visual stream without ventral stream activity, particularly at short presentation durations (Fang & He, 2005), although ventral stream activity has recently been demonstrated at longer durations (Jiang et al., 2009; Sterzer, Haynes, & Rees, 2008; Yang, Hong, & Blake, 2010). As a consequence, CFS is an excellent candidate for presenting affective content in a way that is outside of awareness and examining the consequences.

In the present studies, we explored whether affective faces made invisible by CFS can produce instances of affective misattribution when presented just after a visible, structurally neutral target face is presented (Studies 1 and 2), and whether such invisible images influence first impressions of these neutral target faces (Studies 3 and 4). In Study 1, we examined the extent to which affective misattribution occurred during the affective evaluation of neutral faces. On each trial, we first presented participants with supraliminal neutral face, followed immediately by alternating CFS images to one eye paired with a scowling, smiling, or neutral face suppressed in the second eye. We reasoned that if affective misattribution occurred, then any affective information from the unseen face would be attributed to the consciously seen neutral face, resulting in that neutral face being perceived as imbued with affective value. To further distinguish our affective misattribution hypothesis from the more typical semantic priming explanation, unseen affectively potent faces were presented after the neutral target faces. We predicted that visible neutral faces would be judged as more unpleasant when followed by an invisible scowling face and more pleasant when followed by an invisible smiling face when compared with visible neutral faces that were followed by suppressed neutral faces. Study 2 replicated Study 1 in a community sample at the Museum of Science, Boston with a small number of trials, speaking to the validity of the affective misattribution findings. In Study 3, we extended the effect to show that it can occur during first impressions of personality judgments of other people. In Study 4, we implemented an objective measure of awareness to strengthen our claim that the suppressed images were rendered invisible by CFS. Furthermore, we presented the unseen affective faces for a shorter duration (200 ms), because recent evidence shows that suppressed images presented for longer than 600 ms produce some ventral stream activation whereas this activation has thus far been absent at shorter durations (Jiang et al., 2009; Sterzer, Haynes, & Rees, 2008; Yang, Hong, & Blake, 2010). Throughout these studies we demonstrate that suppressed affective information influence the first impression of concurrently viewed faces.

Study 1

Materials and Methods

Participants

Participants were 67 (47 female) Boston College students (age 17–22 years; M = 19.64). All participants reported normal or corrected-to-normal acuity and were naïve to the purpose of the experiment. Individuals wearing glasses were excluded from this experiment because glasses interfere with the proper function of the stereoscope. Participants received one departmental research credit or $10 for participating.

Materials and Procedure

Instructions and stimuli were presented using E-Prime Version 2 running on a Dell Optiplex 725 and a 17-inch Dell LCD flat-screen monitor (1280 × 1024). Participants sat with their head positioned on a chin rest while they viewed stimuli through a mirror stereoscope at a distance of approximately 55 cm. Stimuli subtended approximately 3.5 × 5.0 degrees of visual angle and were presented in grayscale surrounded by a frame to facilitate fusion. We determined eye dominance for each participant using the Dolman method.

On a given trial, perceivers were presented with a fixation point to both eyes for 500 ms, followed by a face displaying a neutral facial expression (the target face) for 200 ms. The dominant eye was then presented with a series of eight “Mondrian” type images (see Figure 2, Tsuchiya & Koch, 2005). Each Mondrian image was shown for 100 ms to achieve continuous flash suppression. Concurrently, the nondominant eye was presented with a low-contrast low-luminance face matching the target identity for the 800 ms that the Mondrian images were displayed; faces were either smiling, frowning, or displayed a neutral expression. Following this sequence, a backward mask was presented to both eyes for 500 ms. Perceivers were instructed to hold still and focus on the central fixation point throughout each trial, and typically experienced seeing only the stimuli presented to the dominant eye (see Figure 2). Following stimulus presentation, perceivers were asked to rate the neutral target face on a 5-point scale from pleasant to unpleasant using a standard keyboard. Perceiver responses in the CFS task were coded −2 (unpleasant) to 2 ( pleasant).

Figure 2. Study 1: Example of Trial Structure.

Following a fixation dot, both eyes were presented with a face depicting a neutral expression for 200 ms. Immediately following, eight Mondrian images were presented, each for 100 ms. Concurrently, a low luminance face (depicting scowling, smiling, or neutral expressions) was presented to the suppressed eye for 800 ms.

To probe whether participants consciously perceived the suppressed face, they were asked to indicate whether they saw another image following the offset of the initial neutral face (in addition to the flashing Mondrian images). This subjective measure allowed us to determine (and remove) trials where suppressed faces broke through to awareness. We excluded from analysis trials when perceivers reported seeing the suppressed stimuli (19.3% of all trials). Additionally, participants completed a perceptual memory awareness check at the end of the study (see online supplementary materials).

Thirty unique identities were presented; 10 were paired with each type of suppressed face type (scowling, smiling, neutral) for a total of 30 trials. These 30 trials were repeated in four blocks, so each identity was shown four times (always with the same suppressed facial expression) to yield 120 trials. Due to a programming error, one identity was shown with two different suppressed expressions; these trials were removed from analysis creating 112 trials per participant.

Results and Discussion

As predicted, suppressed affectively potent faces changed how perceivers judged visible neutral faces (Figure 3; Table 1). A repeated measures ANOVA (with suppressed face type as the repeated measure) was statistically significant, F(1, 66) = 5.90, p < .019 (df adjusted to correct for sphericity assumptions), such that neutral target faces presented just prior to suppressed scowling faces were judged as more unpleasant than were neutral target faces presented before suppressed neutral faces, t(66) = 2.10, p < .041. Neutral target faces presented just prior to suppressed smiling faces were judged as more pleasant than were neutral faces presented prior to suppressed neutral faces, t(66) = 1.93, p < .06, or before suppressed scowling faces, t(66) = 2.76, p < .009.

Figure 3. Study 1: Mean Ratings of Faces by Suppressed Face Type.

Error bars represent standard errors. Ratings in graph were centered on individuals’ grand mean. All means were statistically different at p < .05, except the comparison between neutral and smiling face types, which only reached trend level significance ( p < .06).

Table 1.

Mean Ratings of Target Neutral Faces by Study and Condition. Standard Error Given in Parentheses

Judgment	Scowling	Neutral	Smiling	Scale (range)
Study 1
Pleasantness	−.229 (.050)	−.130 (.044)	−.050 (.043)	5 point (−2 to 2)
Study 2
Pleasantness	.0038 (.051)	—	.0736 (.045)	2 point (−1 & 1)
Study 3
Trustworthiness	2.522 (.089)	2.748 (.085)	3.101 (.087)	5 point (1 to 5)
Likability	2.620 (.079)	2.679 (.080)	2.914 (.086)
Attractiveness	2.501 (.093)	2.663 (.090)	2.697 (.087)
Study 4
Trustworthiness	2.438 (.052)	2.493 (.046)	2.622 (.042)	4 point (1 to 4)
Competence	2.649 (.047)	2.671 (.043)	2.757 (.037)
Warmth	2.241 (.038)	2.271 (.034)	2.477 (.047)

We also briefly explored whether repeated exposure increased the magnitude of participants ratings. For instance, did participants’ ratings to target faces (which were paired with suppressed scowling faces) become more negative with each pairing over the four blocks? We found no evidence of this type of “learning effect” suggesting that the effects demonstrated here are short-lived and do not carry over to future blocks (more information is presented in the online supplementary materials).

These findings demonstrate that people use affective information to go beyond the information given in social perception, even when that affect is incidental to the task at hand, outside of awareness, and occurred after a target face was presented. On the CFS trials reported here, perceivers were unaware of the value of the unseen image (or even whether an image was there in the first place). And although static faces broke through on some trials for some participants, potentially alerting those participants to the fact that unseen images were presented on some trials, this cannot account for the within-subject patterns we report here. Furthermore, demonstrating affective misattribution with CFS suggests that affective influences can occur via processes that do not involve semantic priming.

Study 2

Recent work calls into question research that is based exclusively on college-aged students (Henrich, Heine, & Norenzayan, 2010). As an important step in exploring whether affective misattribution extends beyond the college campus, we replicated Study 1 in a diverse community sample collected in collaboration with the Living Laboratory initiative at the Museum of Science, Boston. Participants were 87 (40 Female) museum visitors ages 8–78 (mean age = 27.8; SD = 15.5) To conform with the Museum’s requirements, Study 2 used only 20 trials (10 with smiling and 10 with scowling suppressed faces), and participants rated the target neutral face using two response options ( pleasant or unpleasant) on a numeric keypad. All other CFS parameters were identical to Study 1. Museum visitors were included in the analysis only if they had 10 or more trials where static faces were completely suppressed and, as in Study 1, trials during which breakthrough occurred were not included in this analysis (6.9% of trials for the 87 participants analyzed). Pleasant responses were coded as 1, and unpleasant responses were coded as −1. As in Study 1, we found that perceivers judged neutral target faces as more unpleasant when they were paired with suppressed scowling faces (M = 0.0038, SD = 0.47) compared to neutral target faces followed by suppressed smiling faces (M = 0.0736, SD = 0.42), t(86) = 1.7, p < .048, one-tailed test. Although the effect size is small, Study 2 makes clear that it is possible to observe meaningful affective influences in a community sample with only 20 trials in a less controlled context where there are many sounds and smells that could potentially impact perceivers’ judgments. Because there were only 20 trials, no face identities were repeated, so there could be no “learning effect” with multiple exposures.

Study 3

In Study 3, we explored whether affective misattribution could be observed in first impression trait judgments, and whether this influence would occur even when the neutral target being evaluated differed in identity and gender from the unseen affectively potent face. To date, research on first impressions has focused on how stimulus-based properties of a person’s face influence judgments about that person (Ballew & Todorov, 2007; Blair, Judd, & Chapleau, 2004; Zebrowitz & Montepare, 2005). We explored whether presentation of an unseen affective face would influence perceivers’ affective disposition toward a concurrently presented neutral target face, influencing judgments of trustworthiness, attractiveness, and likability about that face. Additionally, in this experiment we used target and suppressed faces that differed in identity and gender. This was done to reduce the possibility that features from the two faces would blend together. As in Studies 1 and 2, trials were removed from analysis if participants reported seeing the suppressed gender (breakthrough) or reported ambiguous or blended gender (indicating a blend and potential feature fusion), so that we analyzed only trials during which participants reported seeing only the target face. We predicted that visible neutral faces would be judged as having more negative traits when paired with an invisible scowling face and more positive traits when paired with an invisible smiling face when compared with visible neutral faces that were followed by suppressed neutral faces.

Materials and Methods

Participants

Participants were 38 (25 female) Boston College students ranging in age from 18–24 (M = 19.63). Participant recruitment, remuneration, and exclusion criteria were the same as Study 1. Three additional participants were dropped from analysis because they saw the suppressed face on the majority of trials (75% of trials).

Materials and Procedure

Each trial started with a 500 ms fixation dot, then the perceiver’s dominant eye was presented with a series of flashing images, including structurally neutral faces and “Mondrian” type images (see Figure 4). Concurrently, the suppressed eye was presented with a low-contrast low-luminance smiling, scowling, or neutral face. Unlike Studies 1 and 2, the identity and gender of the two faces differed. To avoid abrupt transitions, this suppressed image was “faded-in” by first presenting a black field for 66 ms, followed by a very-low-contrast face for 66 ms. The suppressed face was then presented for the remainder of the trial. The perceived effect was a face that faded in over 132 ms. Following this sequence, a backward mask was presented to both eyes for 500 ms. Perceivers typically experienced seeing only the stimuli sequence presented to the dominant eye. Following the stimuli presentation, perceivers were asked to rate the neutral target face on a 5-point scale from unattractive to very attractive using the 1–5 keys on a standard keyboard (when rating the attractiveness of the neutral face, participants were instructed to rate “how attractive the face is objectively, as opposed to how attractive the face is to you”). Perceivers were then asked to rate the neutral target face on the same 5-point scale from unlikeable to very likable and then from untrustworthy to very trustworthy.

Figure 4. Study 3: Trial Structure.

Following a fixation dot, the dominant eye was presented with a series of intermingled faces, white fields, and Mondrian images. Twenty images were presented, each for 33 ms for a total time of 660 ms. Concurrently, the suppressed eye was presented with a black field for the first 66 ms, followed by a low luminance face (either smiling, scowling, or neutral) presented for the remaining 594 ms of the trial.

To probe whether participants consciously perceived the suppressed face, they were asked to indicate the gender of the face on each trial. They were asked to chose “male,” “female,” or “don’t know” and were instructed to chose “don’t know” if they had trouble determining the gender, saw more than one gender, or saw a blend of two genders. This subjective measure allowed us to identify trials when suppressed faces broke through to awareness. As in the previous studies, these trials (18.6%) were removed from analysis.

Thirty-six identities were presented (18 as each dominant and suppressed images); six were paired with each type of suppressed face (scowling, smiling, neutral). These 18 trials were repeated in six blocks, so each identity was shown six times (always with the same suppressed face) to yield 108 trials. The identity and suppressed face type were counterbalanced across participants.

Results and Discussion

As predicted, suppressed affective faces influenced first impressions of consciously seen neutral faces (Figure 5; Table 1). A repeated measures ANOVA (with suppressed face type as the repeated measure) was statistically significant for trustworthiness judgments F(2, 80) = 24.48, p < .001, such that visible neutral target faces presented with suppressed smiling faces were judged as more trustworthy than were neutral target faces presented with suppressed neutral faces, t(37) = 2.87, p < .01, and with suppressed scowling faces t(37) = 4.18, p < .001. Visible neutral target faces presented with suppressed scowling faces were judged as less trustworthy than were neutral target faces presented with suppressed neutral faces, t(37) = 2.01, p < .05. We found similar effects on judgments of likability F(2, 80) = 22.74, p < .001, such that visible neutral target faces presented with suppressed smiling faces were judged as more likable than were neutral target faces presented with suppressed neutral faces, t(37) = 2.94, p < .01, and with suppressed scowling faces t(37) = 4.18, p < .001. The repeated measures ANOVA approached statistically significance for attractiveness F(2, 80) = 2.15, p < .09, such that visible neutral target faces presented with suppressed smiling faces were judged as significantly more attractive than were neutral target faces presented with suppressed scowling faces, t(37) = 2.25, p < .05.

Figure 5. Study 3: Mean social ratings by face type.

Error bars represent standard errors. A) Trustworthiness; B) Likeability; C) Attractiveness.

These findings indicate that trait judgments of other people can be influenced by incidental affective stimuli—showing that social impressions formed quickly and efficiently are, at least in part, contingent upon affective information that co-occurs in the environment.

Study 4

In the first three studies reported here, we determined whether the suppressed image was in fact invisible by using a measure of subjective awareness by asking participants: “Did you see X?” The advantage of this measure was that it allowed us to exclude trials on which participants reported seeing the suppressed image. However, some participants might report not seeing the suppressed image because they didn’t see it clearly, even if they saw part of it— enough to have some sense of the valence of the image. Even though our questions were worded in such a way that participants probably over reported seeing the suppressed image (because they were asked to report seeing anything in addition to the dominant image), we conducted another experiment using a forced choice measure of objective awareness to assess degree of visual awareness of the suppressed image. After the test phase where they made first impression judgments of the seen neutral faces, participants completed another set of trials where they reported the orientation of suppressed faces (right-side-up or up-side-down). Participants who performed better than chance at the orientation task where excluded from analysis.

In addition, while published studies thus far indicate that briefly presented (200 ms) suppressed images are only processed by the dorsal stream during CFS (Almeida et al., 2008; Fang & He, 2005), more recent reports using longer presentation times (600 ms) have found that suppressed houses and faces have different pattern of neural activation in ventral stream areas (using multi-voxel pattern analysis; Sterzer et al., 2008). This finding suggests that at longer presentation duration, a suppressed image might be represented in the ventral stream. In Study 4, therefore, we shortened the duration that the suppressed image was presented to 200 ms to further rule out ventral stream activity, thereby presenting a stronger test of the idea that the observed affective misattribution effects are nonconscious. As in the previous studies, we predicted that visible neutral faces would be experienced as more negative when followed by an invisible scowling face and more positive when followed by an invisible smiling face.

Materials and Methods

Participants

Participants were 88 (52 female) Boston College students ranging in age from 18–22 (M = 19.26; SD = 1.13). Participant recruitment and remuneration were the same as Study 1 and 3. Sixteen participants were excluded from analysis because they performed better than chance on the awareness task (see below).

Materials and Procedure

Study 4 was identical to Study 3 except as outlined below. The experimental session included three tasks: a contrast adjustment task, the experimental CFS task, and an objective awareness task. Each task used the same basic trial structure.

Participants first completed a contrast adjustment task during which the contrast of the suppressed image was adjusted to improve suppression. The logic of this section was to determine what contrast level rendered the suppressed image invisible for each participant. Trials followed the same structure as the CFS task described below, with the following changes. Houses (upside-down or right-side-up) were used instead of faces as the suppressed image, and a series of three “Mondrian” images were presented to the dominant eye (instead of a neutral face). Four contrast levels of stimuli were created by reducing contrast and luminance to 75%, 50%, 25%, and 12.5% of the image’s original contrast and luminance. The contrast adjustment task started using the highest contrast stimuli (75%). Participants viewed 20 trials and were asked to guess the orientation of the suppressed house on each trial. Additionally, we asked participants how clear their perception of the house was using the PAS scale (Ramsøy & Overgaard, 2004). On this scale participants report their subjective awareness of the suppressed house using one of four responses: no experience, vague experience, almost clear experience, or absolutely clear experience. If participants correctly guessed the orientation of the suppressed house on 14 or more trials, or they reported “no experience” of the house on fewer than 15 trials, the contrast level was reduced to the 50% level. This procedure was repeated until participants correctly guessed the orientation on 13 or few trials and reported “no experience” on at least 15 trials or until the 12.5% contrast level was reached. This contrast level was then used for the rest of the experiment.

During the experimental task, each CFS trial started with a 500 ms fixation dot, then the perceiver’s dominant eye was presented with a “Mondrian” type image for 100 ms, followed by a structurally neutral face for 100 ms, followed by another “Mondrian” image for 100 ms. Concurrent with the onset of the structurally neutral face, the suppressed eye was presented with a low-contrast low-luminance smiling, scowling, or neutral face for 200 ms that terminated with the offset of the final “Mondrian” image presented to the dominant eye. As in Studies 1 and 2, the identity of the suppressed and dominant face matched. Following this sequence, a backward mask was presented to both eyes for 500 ms. Following the stimuli presentation, perceivers were asked to make three trait judgments about the neutral target using 4-point scales. The first judgment was “how trustworthy is this person?” (from untrustworthy to trustworthy); the second was “how competent is this person?” (from incompetent to competent); and the third was “how (interpersonally) warm is this person?” (from cold to warm). Thirty unique identities were presented; 10 were paired with each type of suppressed face type (scowling, smiling, neutral) for a total 30 trials. These 30 trials were repeated in four blocks, so each identity was shown four times (always with the same suppressed facial expression, counterbalanced across participants) to yield 120 trials.

The final task was an objective awareness check. During this phase participants were asked to guess the orientation of a suppressed face (upside-down or right-side-up). These trials were identical to the experimental trials except that a scrambled face was presented to the dominant eye instead of a neutral face. Participants completed 60 trials using the same 30 suppressed faces used in the experimental task and the same 30 faces flipped upside-down (rotated 180 degrees). Fifteen participants were removed from analysis because they correctly guessed the orientation on more than 61.66% percent of the trials and one was removed because he or she guessed less than 38.33% correct (better or worse than chance, p < .046 for each tail). The participant that performed at worse than chance was removed because he or she might have mistakenly reversed their key responses (and was able to identify some of the suppressed images) which would lead to this pattern of responses.

Results and Discussion

As in the previous studies, the affective value of the suppressed face influenced first impression trait judgments of the visible neutral target face (see Figure 6). Repeated measures ANOVAs (with suppressed face type as the repeated measure) were statistically significant for trustworthiness judgments F(2,142) = 11.726, p < .002, competence judgments F(2, 142) = 5.803, p < .02, and warmth judgments F(2, 142) = 14.838, p < .001. Follow-up paired t tests revealed that seen neutral faces paired with suppressed smiling faces were judged as more trustworthy t(71) = 3.334, p < .002, competent t(71) = 2.515, p < .015, and warm t(71) = 4.46, p < .001 than were seen neutral faces paired with suppressed neutral faces. Similarly, neutral target faces paired with suppressed smiling faces were judged as more trustworthy t(71) = 4.164, p < .001, competent t(71) = 2.73, p < .009, and warm t(71) = 4.079, p < .001, than were neutral faces paired with suppressed scowling faces. Suppressed scowling faces did not result in significantly more negative judgments when compared to suppressed neutral faces for judgments of competence t(71) = .88, p < .381, and warmth t(71) = .873, p < .385. Seen neutral target faces were rated as marginally less trustworthy when paired with suppressed scowling faces when compared to suppressed neutral faces t(71) = 1.65, p < .101, replicating Study 3.

Figure 6. Study 4: Mean social ratings by face type.

Error bars represent standard errors. A) Trustworthiness; B) Competence; C) Warmth.

Study 4 replicated our basic finding that suppressed affective images influence first impression judgments of concurrently viewed neutral faces. These effects were observed even when using a more stringent method of determining awareness of the suppressed faces. Trustworthiness judgments were most robustly influenced by incidental affect, although the phenomenon of affective misattribution was still evident in judgments of warmth and competence.

General Discussion

Across four studies, we demonstrated that incidental affect plays a role in how humans evaluate each other— unseen images with affective impact influence the judgments of visible neutral faces—imbuing otherwise neutral faces with affective value. In Studies 1 and 2, fully visible neutral faces were judged as more unpleasant when immediately followed by invisible scowling faces and more pleasant when immediately followed by invisible smiling faces. In Study 3, visible neural faces were judged as more trustworthy, likable, and attractive when simultaneously presented with unseen smiling faces, and as more untrustworthy and unlikeable when presented with unseen scowling faces. Interestingly, in Study 3 it appears that affect had its most potent influence on judgments that were, perhaps, the least explicitly affective of the three traits (trustworthiness). Study 4 replicated the finding that judgments of trustworthiness were most influence by unseen affective faces and partially extended the effect to judgments of warmth and competence. In addition, Study 4 also demonstrated that the effects were not due to participants consciously seeing the suppressed affective faces. It is also interesting that unseen smiling faces had the most potent influence on target faces in Study 4 since affective influences in vision are often demonstrated most strongly with negative stimuli (Anderson, Siegel, Bliss-Moreau, & Barrett, 2011).

Our results are consistent with the idea that affect is part of the unified conscious field, so that affective changes are routinely integrated with sights, sounds, and other exteroceptive sensations to produce perceptions of the world (for discussion see Barrett & Bar, 2009; Duncan & Barrett, 2007). Additionally, this work builds both theoretically and methodologically on studies of “unconscious affect” (e.g., Winkielman et al., 2005). However, in this work we showed for the first time that incidental affective information is projected onto faces. The existing neuroanatomical evidence allows us to implicate the dorsal stream as the probable mechanism for the effects we demonstrated. Because backward masking leaves ventral processing intact (Dehaene et al., 2001), while CFS blocks ventral stream activity (at least for some stimuli), particularly at short durations (Almeida et al., 2010; Almeida et al., 2008; Fang & He, 2005), our current findings are consistent with the neuroanatomical hypothesis that the dorsal stream is responsible for processing nonconscious information (Lyon, Nassi, & Callaway, 2010; Shipp, 2003), particularly when it has affective value (Barrett & Bar, 2009; Pessoa & Adolphs, 2010; Tamietto et al., 2009). Since we did not directly measure neural activation in the present studies, this hypothesis is still somewhat speculative, however, directly testing these neuroanatomical hypotheses is an interesting direction for future research.

Limitations

Of course, no study is without limitations. First, we suspect, based on the neuroanatomy, that invisible faces were influencing the perceivers’ core affective state. Because this change in state occurred very close in time to the presentation of the neutral target face, it was bound to the consciously seen neutral image, resulting in the neutral face being judged as more or less valenced, trustworthy, likable, or attractive. Of course, one possibility is that no peripheral nervous system changes actually took place, and that the affective infusion observed was achieved completely via cortical representations of affect. A second possibility is that physiological changes did take place (as in Tamietto et al., 2009), but that no affective feelings were accessible, so that the physiological changes contributed to visual perception in the way that one sensory modality can influence another, similar to the McGurk effect in which visual information is integral to the normal processing of speech (see, McGurk & MacDonald, 1976). A third possibility is that peripheral autonomic changes were actually felt as affective experience, which was then incorporated during the perception process. Additionally, these possible influences could have taken place at different stages in the perception-to-judgment process. Future studies will focus on directly testing these specific hypotheses.

Additionally, it would be prudent in future studies to have our neutral target faces rated by an independent set of participants on trustworthiness, likability, attractiveness, competence, warmth, dominance, and several other trait dimensions. Neutral faces differ in their physiognomy (the physical structure of the face) which influences social judgments (Todorov, Said, Engell, & Oosterhoft, 2008) such as trustworthiness, and this could have influenced participants’ judgments of the seen neutral faces. Nonetheless, we found that manipulating the perceiver’s affective state is sufficient to influence social judgments over and above trial-to-trial variations in neutral physiognomy.

Finally, future studies might examine the boundary conditions of these effects. For example, what is the relative contribution of top-down (perceiver-based) and bottom-up (stimulus based) affective information in social perception? Can a terrific résumé still get you the job when your interviewer has just come from a fight with his or her superior? Will a flawless argument be enough to overcome a judge’s bias because he or she was stuck in traffic earlier? Will a reviewer recommend your paper for acceptance when he or she has just suffered a frustrating faculty meeting? Future studies are needed to directly address questions about the strength and durability of affective misattribution in the formation of social judgments.

Implications

In the broader context, our findings are consistent with the emerging view that perceptual experience is constructed and strongly influenced by perceiver-based (i.e., top-down) contributions (cf. Bar, 2007). Our findings add to this perspective by suggesting that affect is an important top-down influence in perception. A perceiver is not a blank slate. Affect appears to be a normal and important ingredient that allows perceivers to make sense of their world (see Duncan & Barrett, 2007). Affect helps to select the information that reaches conscious awareness (Alpers, Ruhleder, Walz, Mühlberger, & Pauli, 2005; Anderson, Siegel, & Barrett, 2011) perhaps by shaping the formation of neuronal assemblies associated with conscious experience (Kuhbandner et al., 2009). Additionally, via heteromodal brain areas like orbitofrontal cortex, external sensory information becomes integrated with somatovisceral sensations from the body (Kringelbach & Rolls, 2004; Öngür, Ferry, & Price, 2003) so that conscious percepts of the external world are intrinsically infused with affective content. As such, the human brain has the capacity to infuse images and objects with affective meaning that does not derive from the images and objects themselves (affective misattribution).

We can speculate that affective misattribution causes a kind of “affective realism” that is a normal consequence of how the brain processes visual sensations in the context of sensations from the body. At times, somatic and visceral sensations from the body are in the foreground of consciousness so that they are experienced as our own internal reactions to the world: we like or dislike a food, a painting, or a person. When such sensations are in the background because they are not in the focus of attention, they are experienced as a property of the object itself, leading perceivers to experience those objects as affectively significant (in what is called world focused affect; for discussions, see Barrett & Bliss-Moreau, 2009; Lambie & Marcel, 2002). Affective realism can be thought of as a type of naïve realism, which is a commonsense theory held by most people that perception puts them in direct contact with the external world—as naïve realists, most people believe their eyes provide an accurate representation of the way the world really is (for a review of this and other issues faced by philosophers of perception, see Gendler & Hawthorne, 2006; Crane, 2001). As affective realists, people can believe that objects and people in the world are literally negative or positive.

Affective misattributions are not just a laboratory novelty—they very likely have potent, real world consequences. The current work suggests that, at least at the outset, individuals bring powerful internal affective influences when making first impressions. And as previous studies have demonstrated, first impressions can have enduring impact, such as when trait evaluations predict electoral success (Ballew & Todorov, 2007) and sentencing decisions (Blair et al., 2004). Additionally first impressions can shape our behavior toward others, which in turn influences how they respond— creating a self-fulfilling prophecy in social interactions (Fazio, Effrein, & Falender, 1981).

While affect plays an important top-down role in normal experience, it also has the capacity to contribute to disordered experience. Abnormal affective reactivity has been implicated in a variety of mental illnesses, and has been observed in patients who experience delusions associated with psychopathology. For instance, some schizophrenic patients suffering from delusions exhibit increased levels of baseline skin conductance (used as a measure of affective arousal) in response to neutral stimuli (compared to controls); this arousal in turn correlated with the severity of their delusional experiences (Holt, Lebron-Milad, et al., 2009). Similarly, patients with delusions were more likely to experience neutral stimuli as negative (Holt et al., 2006). Neuroimaging results suggest that these findings may be due to impaired functioning of paralimbic midline structures (Holt, Lakshmanan, et al., 2009) that are involved in regulating somatovisceral aspects of the body and representing those changes as affect. Very possibly, enhanced affective reactivity can lead to stronger affective realism, leading some people to feel that neutral objects or events are very negative, or even sinister, despite evidence to the contrary. It could be that individuals suffering from delusions are simply more sensitive to affective information changes that come from a particular target object or person (for discussion see Bell, Halligan, & Ellis, 2006), or that they are more likely to misattribute strong, incidental affective feelings. If this is the case, CFS might provide a useful paradigm for studying “affective delusions” where incidental affect is in appropriately used as information about the world. An important line of future research would be to explore how deep the similarity is between the affective realism we observe in the lab and the affective delusions associated with psychopathology.

Perceivers need not suffer from psychopathology to be lead astray by their own affective responses, however. Affective realism can contribute to naïve realism, in that perceivers assume that their perceptions reflect true and authentic information about the world, including their impressions of other people. This realism has real implications, such as how conflicts are perceived by liberals and conservatives (Robinson, Keltner, Ward, & Ross, 1995) and even influences how disputes are resolved (Ross & Ward, 1995). An exciting line of future research will be to leverage the current CFS paradigm to explore how affective realism contributes to real world social interactions.