Affect of the unconscious: Visually suppressed angry faces modulate our decisions

Abstract

Emotional and affective processing imposes itself over cognitive processes and modulates our perception of the surrounding environment. In two experiments, we addressed the issue of whether nonconscious processing of affect can take place even under deep states of unawareness, such as those induced by interocular suppression techniques, and can elicit an affective response that can influence our understanding of the surrounding environment. In Experiment 1, participants judged the likeability of an unfamiliar item—a Chinese character—that was preceded by a face expressing a particular emotion (either happy or angry). The face was rendered invisible through an interocular suppression technique (continuous flash suppression; CFS). In Experiment 2, backward masking (BM), a less robust masking technique, was used to render the facial expressions invisible. We found that despite equivalent phenomenological suppression of the visual primes under CFS and BM, different patterns of affective processing were obtained with the two masking techniques. Under BM, nonconscious affective priming was obtained for both happy and angry invisible facial expressions. However, under CFS, nonconscious affective priming was obtained only for angry facial expressions. We discuss an interpretation of this dissociation between affective processing and visual masking techniques in terms of distinct routes from the retina to the amygdala.

Our ability to process incoming stimuli in terms of their emotional and affective content is automatic and extremely robust. It has been shown that the processing of emotionally charged stimuli (e.g., facial expressions) can override ongoing cognitive processes (e.g., Vuilleumier, Armony, Driver, & Dolan, 2001). Research has also uncovered one of the most important functions of the affect system—that of quickly modulating our impressions of a particular situation. Coarse affective responses can be elicited by stimuli and transferred to our perception of the environment, influencing our immediate judgments or our approach/avoidance responses to those stimuli or the context (e.g., Dannlowski & Suslow, 2006; Murphy & Zajonc, 1993; Rotteveel, de Groot, Geutskens, & Phaf, 2001; Winkielman, Berridge, & Wilbarger, 2005). Interestingly, this fast and efficient emotional and affective processing can take place even under states of unawareness and under attentional load, perhaps due to dedicated neural routes that we may share with many of our evolutionary ancestors (e.g., LeDoux, 2003; Luo et al., 2010; Vuilleumier, Armony, Driver, & Dolan, 2003; Whalen et al., 2004; but see Pessoa, McKenna, Gutierrez, & Ungerleider, 2002).

Recently, there has been a resurgence of interest in the mechanisms that may underlie affective processing under unawareness. This renewed interest has been spurred, in part, by results obtained in research on the processing of facial expressions under interocular suppression techniques such as continuous flash suppression (CFS; Adams, Gray, Garner, & Graf, 2010; Jiang & He, 2006; Pasley, Mayes, & Schultz, 2004; see also Williams, Morris, McGlone, Abbott, & Mattingley, 2004). CFS induces very strong and prolonged states of unawareness (Tsuchiya & Koch, 2005). Researchers have shown that despite complete and prolonged unawareness, emotional stimuli rendered invisible through CFS, when compared to nonemotional (facial and nonfacial) stimuli, strongly activate limbic structures, including the amygdala, known to process emotional content (Jiang & He, 2006; Pasley et al., 2004; Troiani, Hunyadi, Riley, Herrington, & Schultz, 2010; Williams et al., 2004). Such activation in response to facial expressions under CFS occurs despite the near absence of stimulus-specific activation within inferior and ventro-temporal face-preferring regions (e.g., Jiang & He, 2006; Pasley et al., 2004; see also Kreiman, Fried, & Koch, 2002; Sheinberg & Logothetis, 1997; Tong, Nakayama, Vaughan, & Kanwisher, 1998; Williams et al., 2004; but see Sterzer, Haynes, & Rees, 2008) that are known to be the principal afferents to the amygdala (e.g., Amaral & Price, 1984). The results on the processing of facial expressions under CFS have shown that affective information is rapidly and automatically transmitted to the amygdala independently of the participant’s conscious awareness and can only be minimally dependent on input from ventral temporal face-preferring cortex.

It is not known, however, whether under these conditions the affective response to emotionally charged facial expressions can be transferred to subsequently presented unrelated stimuli. Nonconscious affective priming has been studied with the use of masking techniques that have different neural and behavioral signatures from those presented by CFS (e.g., backward masking [BM]; see, e.g., Murphy, Monahan, & Zajonc, 1995; Murphy & Zajonc, 1993; Niedenthal & Showers, 1991; Winkielman et al., 2005). For instance, we have shown that BM and CFS lead to different behavioral priming results with nonaffective stimuli (Almeida, Mahon, & Caramazza, 2010; Almeida, Mahon, Nakayama, & Caramazza, 2008). These differences in behavioral priming were rather qualitative, suggesting that distinct types of processing are afforded by the two masking procedures. Moreover, the visual processing of the suppressed stimulus at the neural level also seems to be dissimilar in the two techniques. Under BM, inferior and ventro-temporal regions present activation levels in response to visual stimulation (albeit lower than those obtained from visible presentations; e.g., Dehaene et al., 2001; Rolls & Tovee, 1994). In fact, under BM there seems to be a sort of generalized decrement of activation across many cortical regions. As such, these regions can serve as afferents to the amygdala (e.g., Amaral & Price, 1984). As described above, however, under CFS a discrete set of regions seem to maintain their response levels to certain types of stimuli (e.g., the amygdala; certain regions in the parietal lobe, the pulvinar; and the superior colliculi; e.g., Fang & He, 2005; Pasley et al., 2004), whereas others suffer a marked decrease in neural response (e.g., ventral temporal cortex). This qualitative difference between CFS and BM may be related to the locus of the perceptual resolution of binocular rivalry. Rivalry resulting from the presentation of two different images to the same retinal location in the different eyes may be resolved very early on within visual cortex (e.g., Blake, 1989; Polonsky, Blake, Braun, & Heeger, 2000; Tong & Engel, 2001), with the dominant image obliterating the processing of the suppressed image further up the visual system. This would drastically curtail regular geniculate striatal processing for the suppressed images, leaving subcortical processing relatively unaffected. Incidentally, then, CFS could be considered to be functionally similar to the blindsight phenomenon, in that putative processing of information is maintained mainly through subcortical structures (e.g., Stoerig & Cowey, 1997). Interestingly, it has been robustly reported that certain patients with blindsight can nevertheless process affective information (e.g., de Gelder, Vroomen, Pourtois, & Weiskrantz, 1999).

Because of the characteristics of BM and CFS, there may be differences in how nonconscious affect interacts with decision making under these two techniques. In particular, the differences regarding the strength of suppression between the two methods—that is, the differences in the loci and degree of cortical activation elicited by each method (most notably, the differences in ventral temporal activation)—raise interesting questions regarding nonconscious affect. Can we elicit an affective response to valenced invisible stimuli that can then be transferred onto unrelated neutral targets in situations in which visual processing is very limited in face-preferring regions in the ventral temporal cortex? Can this affective reaction have behavioral consequences, while at the same time not being “noticed” by participants? And finally, given the bias within the amygdala (and the putative pathways that bypass ventral temporal cortex) for the processing of threat-related emotions (e.g., LeDoux, 2003), would nonconscious affective priming under CFS be more prominent for negative-valenced facial expressions?

The experiments

In Experiment 1, we combined a CFS nonconscious affective priming paradigm with a likeability judgment task. Following Murphy and Zajonc (1993), we asked participants to rate, on the basis of their first impressions, how much they liked a series of novel items (Chinese characters). Unbeknownst to the participants, a prime picture, rendered invisible through CFS, was presented prior to each target. The prime picture could be one of four types: an angry or happy face, a polygon, or a noise image. The first two sets of primes were our experimental primes and were used to test for a positive or a negative affective response, respectively. Emotional facial expressions are critical social stimuli that may convey and elicit basic affective reactions (e.g., Niedenthal & Showers, 1991), and as such have been widely used to study nonconscious affect (e.g., Murphy & Zajonc, 1993). The other two prime conditions were used as control primes. The polygons were used as irrelevant primes, and the noise images were used as a “no-prime” condition that nevertheless maintained the temporal properties of each trial. In Experiment 2, we used BM—as originally used by Murphy and Zajonc—to render the primes invisible.

Method

Participants

A group of 54 right-handed Harvard undergraduates participated in two experiments. Stringent criteria for including participant data in the main analyses were employed with regard to the prime awareness data (the objective and subjective criteria will be described below). As a consequence of these criteria, 26 participants were excluded from the main analysis (15 in Exp. 1, 11 in Exp. 2), whereas the remaining 28 participants were included in the main analysis (i.e., the analysis of nonconscious affective priming; 14 participants each in Exps. 1 and 2). All of the participants had normal or corrected-to-normal vision, were naive as to the experimental hypotheses, gave written informed consent, and reported no familiarity with Chinese characters. The project was approved by the appropriate institutional review board.

Materials

Eight angry and eight happy faces (four male and four female faces shown under both emotional expressions; Ekman & Friesen, 1976), eight polygon figures (Shepard & Metzler, 1971), and eight noise images were used as the prime stimuli (see Fig. 1c). Each prime was presented five times throughout the experiment.

Fig. 1.

Experimental design. (a) In Experiment 1, we used continuous flash suppression (CFS) to suppress prime pictures. CFS is an interocular suppression technique whereby two different images are presented at the same time, one to each eye (Tsuchiya & Koch, 2005). Eye dominance was assessed with the Miles test. We presented a low-luminance, low-contrast version of the prime stimulus to the participant’s nondominant eye and a dynamic, high-contrast random-noise pattern—changing every 100 ms—to the dominant eye. This ensured that participants perceived only the dynamic random-dot pattern, while the prime did not reach consciousness. Participants wore red/green anaglyph glasses that allowed for dioptic presentation of the images. Prime stimuli were restricted to the green RGB channel, whereas the high-contrast random patterns were restricted to the red RGB channel. (b) In Experiment 2, we used backward masking (BM) to render the pictures invisible. A black-and-white backward mask was generated by using the same algorithm that was used to generate the high-contrast random-noise patterns for CFS. (c) Examples of the primes used

A set of 120 Chinese characters were used as target stimuli. Two thirds of the target Chinese characters were presented only once during the experiment: 40 paired with the random-dot displays and 40 with the polygons. The remaining 40 targets were presented twice: once with an angry face and once with a happy face. The Chinese characters were randomly assigned to each condition for each participant independently. All experiments were run on a Dell PC, with a ViewSonic Ultrabrite A90_f 100-Hz CRT monitor. The stimuli were presented centrally and subtended 7° of visual angle. We used MATLAB and the Psychophysics Toolbox extensions (e.g., Brainard, 1997) to present the stimuli.

Procedure

Each experiment consisted of two independent tasks: the experimental task, which required participants to perform likeability judgments over the novel items (Chinese characters), and a prime discrimination task, which was performed immediately after the experimental task.

In the experimental task, participants were asked to rate, on the basis of their first impressions, how much they liked a series of Chinese characters, using a scale from 1 to 5 (1 = do not like at all, 5 = like a lot).

In Experiment 1, we used CFS to suppress the prime pictures (see Fig. 1a). The trial structure for this experiment was as follows: A fixation cross appeared on the screen (500 ms), followed immediately by the prime picture accompanied by the first random-noise pattern (100 ms), followed immediately by the same prime picture accompanied by the second random-noise pattern (100 ms), followed by the target Chinese character (2,000 ms). Participants were asked to respond while the target picture was on the screen.

In Experiment 2, we used BM to render the pictures invisible (see Fig. 1b). The trial sequence was the following: A fixation cross appeared on the screen (500 ms), followed immediately by the prime picture (10 ms; presumably the same prime duration as in Murphy & Zajonc’s 1993 experiments; Rotteveel et al., 2001), followed by a black-and-white random-noise mask (100 ms), followed by the target Chinese character (2,000 ms). Everything else was the same as in Experiment 1.

The prime discrimination task measured the extent to which participants were aware of the prime pictures. The trial sequence was exactly the same as in the main experiment, with the exception that participants were asked to categorize the primes as to whether or not they were faces. Participants were asked to press the “f” key if they thought that the prime was a face, and the “j” key if it was a nonface object. Nonface objects were defined as geometrical shapes or noise images.

The two experiments used nonoverlapping groups of participants. The trials from all experimental conditions were distributed randomly throughout the experiment. The participants were seated comfortably and at a distance of ~76 cm from the screen.

Analysis

Nonconscious affective priming

For both experiments, we analyzed the mean standardized ratings (z scores), calculated on a within-participants basis, in order to account for individual fluctuations in Likert ratings of baseline valence. A 4 (prime condition: happy face, angry face, polygon, or noise images) × 2 (experiment: 1 vs. 2) ANOVA was performed over the standardized ratings, with the last factor analyzed between subjects. One-sample t tests were performed over the interaction between the two factors (prime conditions and experiments) on the four conditions (i.e., one-sample t tests against zero for the z-score mean ratings across the entire experiment, determined on a participant-by-participant basis).

Prime awareness

Percent correct performance on the prime awareness task was one criterion used for selecting participants for the present experiments (the other being unfamiliarity with Chinese characters). The individual overall percent correct performance and d′ scores obtained for the face/non-face discrimination task were analyzed. Participants were then included in the main analysis if (1) they were not subjectively aware of the primes in the experiment and prime discrimination task (i.e., if no breakthroughs from suppression occurred throughout the experiment); (2) overall performance in the discrimination task was at chance level (tested with a z test for one proportion); (3) performance on face and nonface trials in the discrimination task and on the trials for each experimental condition (i.e., responses to angry and happy faces, noise images, and polygons) was no different from chance (tested with a z test for one proportion); (4) performance levels on the faces and nonfaces did not differ from each other significantly (tested by using a z test for two proportions); and finally, (5) performance levels on the angry and happy primes did not differ from each other significantly (tested by using a z test for two proportions).

These strict criteria were used to ensure, as best as possible, that the affective reaction was truly elicited by the invisible primes and was not consciously processed (i.e., no consciously mediated signal could be used to categorize the prime images).

Results

Nonconscious affective priming

The analysis of the standardized ratings revealed that the presence of the prime pictures influenced the likeability ratings obtained for the neutral Chinese characters [F(3,78) = 5.521, MSE = 0.014, p = .002, η² = .175; the main effect of the between-subjects factor Experiment did not reach significance, F < 1]. The main effect of the prime conditions expressed itself differently in each experiment, as evidenced by the interaction between the two factors [F(3, 78) = 2.719, MSE = 0.014, p = .05, η² = .095]. In Experiment 1 (n = 14), in which the primes were presented under CFS, the likeability judgments were significantly more negative when the characters were preceded by angry-face primes [one-sample t tests against zero using standardized ratings: t(13) = 2.84, p = .014, mean standard rating = −0.076, SEM = 0.027; see Fig. 2). The other conditions did not differ from the mean [all ts < 1, except for the polygon condition: t(13) = 1.97, p = .07].

Fig. 2.

Mean standardized likeability ratings. The average standardized likeability ratings are plotted as a function of the experimental conditions for the two experiments. Error bars represent SEMs for the mean standardized likeability ratings across participants for each condition. *p < .05. **p < .001

In Experiment 2 (n = 14), in which we used BM to render the primes invisible, happy-face primes led to higher-than-average likeability ratings [t(13) = 4.46, p = .0006, mean standard rating = 0.092, SEM = 0.021], whereas angry-face primes led to lower-than-average likeability ratings [t(13) = 2.49, p = .027, mean standard rating = −0.075, SEM = 0.03]. The other conditions did not differ from the mean (all ts < 1). The findings in Experiment 2 are in line with the BM results reported by Murphy and Zajonc (1993).

Finally, for both experiments, the mean unstandardized likeability rating for all Chinese characters was around the value 3, suggesting that these Chinese characters were considered, in general, neutral (Exp. 1, mean = 3.11; Exp. 2, mean = 3.05; see Table 1 for the mean unstandardized likeability ratings). Moreover, we replicated the results of Experiment 1 in an experiment in which we used a different set of participants and a different way of assigning Chinese characters to the prime conditions (see Supplementary Fig. S1).

Table 1.

Mean unstandardized ratings for Experiments 1 and 2 for each prime condition

	Primes
	Noise images	Polygons	Happy	Angry
Experiment 1	3.14	3.17	3.10	3.01
Experiment 2	3.05	3.02	3.16	2.96

Prime awareness

A total of 26 participants were excluded from the main analysis on the basis of the criteria described above. The results for d′ and percent correct performance for the participants entered in the main analyses did not differ from 0 and 50 %, respectively (all ts < 1; see Fig. 3 for the distribution of the percent correct and d′ scores for each experiment). As such, the participants who were included in the main analysis did not experience any (conscious) breakthrough of the visual prime stimuli (either for BM or CFS).

Fig. 3.

Individual overall proportions correct and d′ scores on the prime awareness measures. Percent correct scores are presented in the decimal system (i.e., proportions correct) to simplify their presentation alongside d′ scores. Filled circles correspond to the individual d′ scores, whereas filled diamonds correspond to the individual percent correct scores for (a) Experiment 1 and (b) Experiment 2

Discussion

In two experiments, we showed that despite the strong visual suppression evoked by CFS, primes depicting facial expressions do indeed elicit an affective response that influences the way in which we react to our environment. Prime pictures of angry faces influenced likeability ratings over neutral novel items, turning them subjectively more negative. This nonconscious affective response—or at minimum, its modulatory effect on how participants evaluated seemingly neutral novel objects—was restricted to prime faces that expressed threat-related emotions (i.e., anger). However, when the same facial expressions were rendered invisible with a technique that imposed less selective cortical processing restrictions (i.e., BM), both happy and angry prime faces influenced participants’ likeability ratings, suggesting a more general affective response in this case (cf. Murphy & Zajonc, 1993).

Under CFS, there is very limited processing of prime-related information, particularly within ventral temporal regions (e.g., Jiang & He, 2006; Pasley et al., 2004; for data on the strength of suppression under CFS, see also Tsuchiya, Koch, Gilroy, & Blake, 2006). Prime-related information may undergo processing only over processing pathways that are more resilient, and perhaps hardwired (e.g., Pasley et al., 2004; Troiani et al., 2010). On the other hand, under BM, information flows toward different neural regions (e.g., Dehaene et al., 2001). This less selective suppression may be responsible for the generalized affective effect obtained. In fact, the difference as to the availability of information between the two techniques can also be assessed via the timing differences between them: Despite the fact that primes under BM are presented for durations ten times shorter than the primes presented under CFS, more information passes through, due to more lenient suppression under BM.

But why should the affective modulation under CFS be threat-specific? As we described in the introduction, under CFS, processing within face-preferring ventral temporal regions—some of the most important afferents to the amygdala—seems to be highly disrupted. Thus, the amygdala will process information that originates from alternative pathways. One possibility is that, under CFS, information processing is biased toward putative subcortical connections that link the retina to the amygdala via the superior colliculus and the pulvinar nucleus (e.g., LeDoux, 2003; see Fig. 4). In line with this finding, Pasley and colleagues (2004; Troiani et al., 2010) showed that the pulvinar and superior colliculus mediate the activation of the amygdala. In a related CFS study, Jiang and He (2006) found that BOLD signal changes in the amygdala in response to emotional faces did not correlate with activity in either V1 or face-preferring regions within ventral temporal cortex. The potential involvement of the retinotectal subcortical route in the processing of emotions under CFS would account for the threat-related affective effect that we obtained, as the role of the retinotectal route in emotional processing has been closely tied to the fast and efficient detection of threats in the environment (e.g., LeDoux, 2003). This retinotectal route is thought to be a “quick and dirty” mechanism for finding biologically critical stimuli, and it probably involves coarse (low spatial frequency) processing of stimuli.

Fig. 4.

Schematic depiction of information processing routes from the retina to the amygdala. The route depicted in green corresponds to the putative subcortical route from the retina to the amygdala, through the superior colliculus and the pulvinar nucleus, whereas the route depicted in red corresponds to the regular geniculate–striatal route, whereby information travels from the retina to the lateral geniculate nucleus, to early visual areas, and reaches the amygdala via high-level visual areas

Despite the appeal of this proposal, the existence of such subcortical connections in humans has been questioned (e.g., Pessoa et al., 2002). An alternative possibility has been recently proposed (e.g., Barrett & Bar, 2009), whereby the same “quick and dirty” processing of affective information is performed by dorsal-visual-stream regions and the orbitofrontal cortex, also bypassing ventral visual stream regions. Particularly interesting, in terms of our results, is the observation of neural activity within dorsal-stream parietal regions for the processing of negative facial expressions suppressed under CFS (Troiani et al., 2010). These results are in line with the possibility that the processing within the orbitofrontal cortex/dorsal-stream pathway may be underlying our “threat-specific” effects under CFS.

Irrespective of the neural pathways involved, and in line with the threat-specific effect that we obtained under CFS, Yang, Zald, and Blake (2007) showed that negative emotions (e.g., fear) break from CFS suppression faster than other emotions do, suggesting an advantage for the processing of threat-related emotions under CFS.

Our results show that contrary to other aspects of face processing (e.g., face identity; Moradi, Koch, & Shimojo, 2005), information on facial expressions can survive strong states of visual suppression, such as those induced by CFS. Other groups have presented data that corroborates this conclusion. Adams et al. (2010), for instance, presented emotional faces (displaying different positive and negative emotions) under CFS for a prolonged period of time. These faces were used as adapting stimuli in an emotional face adaptation study and influenced the perception of a subsequently presented face.

Our results go beyond those obtained by Adams et al. (2010), as we have shown that, besides altering the phenomenological perception of emotional faces, CFS-suppressed facial expressions can elicit affective responses that modulate our reactions to the surrounding world. Note, however, that in contrast to our CFS result in Experiment 1, Adams et al. obtained adaptation effects for both positive- and negative-valenced faces. This may be due to the fact that the two studies may be tapping into different mechanisms—Adams and his colleagues may have tapped into face-specific perceptual processing, whereas we are dealing with nonconscious affect and its behavioral consequences. It may well be that these two types of processing make use of different sets of information and types of processing. For instance, the adaptation experiment of Adams et al. may have hinged on low-level visual similarity between the adaptor and the test face. On the other hand, nonconscious affect-priming experiments may be more dependent on approach/avoidance, affect-related information that can be extracted from facial expressions. Moreover, our primes were presented for a much shorter period of time than were the adaptor faces of Adams et al., and thus did not allow for possible effects of information accumulation through time.

In conclusion, facial expressions under strong states of unawareness induced by CFS can nevertheless elicit coarse affective reactions that modulate our impressions and judgments of the surrounding environment. Interestingly, this nonconscious affective reaction is restricted to threat-related emotions (e.g., anger); CFS-suppressed angry faces lead to a bias toward more negative evaluations of neutral novel objects. This threat-specific effect could be driven by the fact that under CFS there may be a processing bias toward the putative retinotectal route from the retina to the amygdala (or toward regions of the dorsal stream and orbitofrontal cortex) that bypasses the slow but detailed geniculate–striate pathway. This alternative processing would result in crude, low-level markers (e.g., how much sclera is exposed as a proxy to the presence of danger; Whalen et al., 2004) that would signal potential threat in the environment and would boost the participant’s arousal levels, enhance his or her readiness to act (Schutter, Hofman, & van Honk, 2008), and signal regions of interests for further detailed geniculate–striate processing in which there may be potential danger.