ERPs reveal subliminal processing of fearful faces

Abstract

To investigate whether facial expression is processed in the absence of conscious awareness, ERPs were recorded in a task where participants had to identify the expression of masked fearful and neutral target faces. On supraliminal trials (200 ms target duration), where identification performance was high, a sustained positivity to fearful versus neutral target faces started 140 ms after target face onset. On subliminal trials (8 ms target duration), identification performance was at chance level, but ERPs still showed systematic fear-specific effects. An early positivity to fearful target faces was present but smaller than on supraliminal trials. A subsequent enhanced N2 to fearful faces was only present for subliminal trials. In contrast, a P3 enhancement to fearful faces was observed on supraliminal but not subliminal trials. Results demonstrate rapid emotional expression processing in the absence of awareness.

Investigations of the neural basis of emotional processes have become one of the most active research areas in cognitive neuroscience (see Adolphs, ^{​2002, ​2003;} Dolan, 2002, for reviews). FMRI studies have revealed a complex interconnected network of brain areas that are involved in the detection and processing of emotional faces and other types of affective stimuli. This network includes the amygdala, orbitofrontal cortex, and ventral striatum, where perceived visual events are classified in terms of their emotional significance, as well as paralimbic and higher cortical areas such as somatosensory cortex, anterior cingulate, and medial prefrontal cortex, where conscious representations of emotional states are generated that are used in the strategic control of behaviour (see Adolphs, 2003, for more details). The rapid evaluation of the emotional content of facial expression appears to be mediated by the amygdala and orbitofrontal cortex, while structures such as the anterior cingulate and prefrontal cortex are linked to the conscious representation of perceived facial expression.

Given the adaptive significance of emotional information, it is often assumed that the processing of affectively salient stimuli such as facial expressions can occur even when these stimuli are inaccessible to conscious awareness. This type of subliminal processing of emotional faces might be mediated by a hypothetical subcortical pathway that sends retinal input directly to the amygdala via the superior colliculus and the pulvinar (see Pessoa, 2005, for a more detailed discussion). Evidence that emotionally salient events are processed in the absence of awareness comes from fMRI studies demonstrating fear-specific amygdala activation during binocular suppression (Pasley, Mayes, & Schultz, 2004; Williams, Morris, McGlone, Abbott, & Mattingley, 2004b), and in extinction patients for face stimuli that remain undetected as a result of right parietal damage (Vuilleumier, Clarke, Husain, Driver, & Dolan, 2002). Further support for the subliminal processing of emotional faces comes from fMRI studies that used backward masking procedures. Whalen et al. (1998) found stronger amygdala activations for fearful relative to happy faces under conditions where these faces were presented for 33 ms and then immediately replaced by a neutral face mask, and participants did not report any awareness of emotional faces after the experiment (see also Morris, Öhman, & Dolan, 1998, for similar findings). However, another more recent fMRI study (Phillips et al., 2004) failed to obtain analogous results. When fearful faces were shown for 170 ms (supraliminal condition), fear-specific amygdala responses were found, but these responses were absent in an unaware condition where these faces were only presented for 30 ms.

One potential problem for the interpretation of these fMRI results is that the absence of conscious awareness of fearful faces under stimulation conditions described as ‘subliminal’ was not established in a rigorous fashion. In psychophysical experiments, objective thresholds for conscious stimulus detection or discrimination are usually determined on the basis of chance-level performance in forced-choice tasks, as indicated by measures of perceptual sensitivity such as d’ (see Macmillan & Creelman, 1991). Because no formal threshold measurements were included in previous fMRI studies of subliminal emotion processing (Whalen et al., 1998; Morris et al., 1998; Phillips et al., 2004), it is unclear whether subliminal conditions realized in these studies did in fact correspond to objective thresholds. Using objective stimulus detection measures, Pessoa, Japee, and Ungerleider (2005) have recently demonstrated that many observers can reliably detect masked fearful faces presented for 33 ms (i.e., equivalent to the face duration used by Whalen et al., 1998, and Morris et al., 1998). Some observers still showed above-chance performance when target duration was reduced to 17 ms. Furthermore, participants varied greatly in their perceptual sensitivity to fearful faces, suggesting that it may be very difficult to determine interindividually constant objective thresholds of awareness in the perception of emotional faces. Thus, some of the emotion-specific effects observed in previous fMRI studies in response to masked emotional faces may have been due to the fact that at least some observers were aware of these faces on a subset of trials.

This assumption was further supported by an fMRI experiment (Pessoa, Japee, Sturman, & Ungerleider, 2006) that demonstrated strong links between fear-specific amygdala activation by masked faces and their detectability. Observers who were able to detect masked fearful faces presented for 67 ms, but performed at chance level when their duration was reduced to 33 ms, showed differential amygdala activity only in the former case. In contrast, observers who reliably detected fearful faces also when they were shown for only 33 ms showed fear-specific amygdala effects throughout. Importantly, when results were analysed as a function of observers’ perceptual reports on single trials, stronger amygdala responses were found on trials where observers reported a fearful face than on trials where they missed the presence of a fearful face. These findings suggest that fear-specific amygdala activations may be closely linked to variations in the visibility of masked fearful faces.

In addition to the question whether the detection of masked emotional faces was measured in a sufficiently rigorous way in previous fMRI experiments, these studies cannot provide direct insights into the temporal dynamics of subliminal emotional processing. In order to gain such insights, fMRI measures need to be complemented with brain activity measures that afford superior temporal resolution, such as EEG/ERP or MEG. In the only ERP study to date investigating the subliminal processing of emotional faces, Liddell, Williams, Rathjen, Shevrin, and Gordon (2004; see also Williams et al., 2004a) compared ERPs triggered by masked fearful faces and neutral faces in blocks where fearful or neutral faces were presented for either 10 ms (subliminal condition) or 170 ms (supraliminal condition), and were followed by a neutral face mask, while observers passively watched these stimuli. When compared to blocks with neutral faces, fearful faces triggered an enhanced negativity in the N2 time range between 200 and 300 ms after stimulus onset in the subliminal condition, suggesting that fear-specific brain responses can be elicited in the absence of conscious awareness. This N2 modulation was absent in the supraliminal condition. A later enhancement of the P3 component was found at Pz for fearful as compared to neutral faces in the supraliminal, but not in the subliminal condition. In a forced choice experiment conducted with a different set of participants, Liddell et al. (2004) found that when masked fearful faces were presented for 10 ms, emotion identification performance did not differ significantly from chance. These authors interpreted the N2 enhancement for subliminally presented fearful faces as evidence for an automatic orienting response triggered independently of awareness. In contrast, the subsequent P3 modulation observed for supraliminal trials only was seen as reflecting the conscious integration of an emotionally salient event into the current stimulus context.

While the results reported by Liddell et al. (2004) provide initial evidence that ERPs might be sensitive to the subliminal processing of emotional faces, this conclusion must remain tentative due to several potential methodological problems inherent in this study. First, the measures needed to demonstrate that presenting masked faces for only 10 ms makes above-chance identification objectively impossible were obtained in a separate study with a different set of observers, but not during the EEG recording session. Taking into account the marked differences in observers’ sensitivities to masked emotional faces demonstrated by Pessoa et al. (2005), it is therefore possible that at least some participants may have been able to detect masked fearful faces on at least some subliminal trials during EEG recording, and that this residual awareness was responsible for the differential ERP effects observed for fearful versus neutral subliminal faces. Also, the fact that Liddell et al. (2004) obtained ERP measures while participants were merely passively watching the stimulus arrays leaves open the possibility that participants were not always actively attending to the masked face stimuli, which could have resulted in the absence of emotion-specific ERP effects that might have been observed with fully focused attention. Finally, the choice of neutral faces as masks is potentially problematic in a situation where these are preceded by fearful or neutral faces. Presenting neutral face masks on every trial inevitably leads to fearful face items being overall less likely than neutral faces, which could be reflected by modulations of ERP components sensitive to a priori stimulus probabilities such as N2 and P3. For example, the observation by Liddell et al. (2004) that supraliminal fearful faces elicited an enhanced P3 relative to supraliminal neutral faces might be related to such probability differences, rather than to emotional expression as such.

The aim of the present experiment was to further investigate whether and how the subliminal processing of emotional faces is reflected in ERPs under conditions where the methodological issues discussed above are resolved. As in the study by Liddell et al. (2004), we recorded ERPs to masked fearful and neutral faces under supraliminal and subliminal presentation conditions. On subliminal trials, a fearful or neutral target face was presented for 8 ms and was then immediately replaced by a mask. On supraliminal trials, which were randomly intermingled with subliminal trials, the duration of a target face was 200 ms. In order to obtain a continuous measure of face discrimination performance throughout the EEG recording session, and to ensure that focal attention was always fully allocated to the masked faces, participants had to discriminate fearful and neutral faces on every trial. To prevent any differences in the a priori probability of fearful and neutral faces that might result when neutral face masks are employed, masking stimuli were generated by scrambling face segments. In the resulting scrambled face masks (see Figure 1), local features such as the nose, eye, and mouth remained largely intact, while global face configuration was disrupted. Performance measures (see below) demonstrated that these masks were highly effective in preventing above-chance emotion discrimination on subliminal trials, while allowing accurate performance on supraliminal trials.

Figure 1.

Example of the stimulus sequence in a trial with fearful target face. The target face was presented for 200 ms on supraliminal trials, or for 8 ms on subliminal trials, and was followed immediately by a scrambled face mask (150 ms duration).

ERPs obtained on trials with fearful or neutral target faces were computed separately for supraliminal trials (200 ms presentation duration) and subliminal trials (8 ms presentation duration). For supraliminal trials, where fearful and neutral faces were clearly visible, we expected to find emotion-specific ERP effects similar to those observed in previous studies in response to unmasked faces. In these studies, fearful faces were found to trigger an early enhanced positivity at frontocentral electrodes relative to neutral faces (Ashley, Vuilleumier, & Swick, 2004; Eimer & Holmes, 2002; Eimer, Holmes, & McGlone, 2003; Holmes, Vuilleumier, & Eimer, 2003). This emotional expression effect is usually triggered within 150 ms after stimulus onset, has been found to start as early as 120 ms (Eimer & Holmes, 2002), and has been interpreted as reflecting the rapid detection of facial expression in prefrontal areas involved in the processing of emotional stimuli (see Eimer & Holmes, 2007, for more details). Following this early response at frontocentral electrodes, emotional faces also trigger a more broadly distributed sustained positivity beyond 200 ms post-stimulus (Ashley et al., 2004; Eimer & Holmes, 2002; Schupp, Öhman, Junghöfer, Weike, Stockburger, & Hamm, 2004) that is often reflected by enhanced P3 amplitudes for emotional relative to neutral faces (e.g., Schupp et al., 2004^; Liddell et al., 2004). These longer-latency emotional expression effects are usually interpreted as indicating the activation of higher-level stages of emotional face processing, where emotional content is consciously evaluated and integrated.

The critical question was whether similar effects would also be found on subliminal trials under conditions where fearful and neutral faces could not be consciously discriminated. The presence of early anterior emotional expression effects for subliminal trials would indicate that they reflect the activity of brain mechanisms involved in the rapid analysis of emotional stimuli that operate independently of awareness. The absence on subliminal trials of any longer-latency ERP modulations, such as an enhanced P3 for fearful as compared to neutral faces, would suggest that these ERP effects indicate the conscious processing of emotional events. A further question was whether the observation of Liddell et al. (2004) that a fear-specific N2 enhancement is triggered with subliminal, but not supraliminal stimuli can be confirmed under conditions where methodological problems related to discrimination thresholds, focal attention, and a priori stimulus probabilities are eliminated. In order to firmly establish that any systematic fear-specific ERP effects with subliminal faces were not due to some form of residual awareness, we also investigated whether such effects were systematically linked to variations in face discrimination performance across participants.

Finally, we also investigated whether or not the face-specific N170 component is sensitive to facial expression by comparing this component for trials with fearful versus neutral target faces. While several previous ERP studies have found that the N170 is unaffected by emotional facial expression (Eimer & Holmes, 2002; Eimer et al., 2003), other studies (e.g., Batty & Taylor, 2003) have found evidence for a modulatory effect of facial expression on the N170.

Method

Participants

Fourteen volunteers (6 males) participated in this experiment. Mean age was 29.2 years. All participants were right-handed, and had normal or corrected-to-normal vision.

Stimuli and procedure

Stimuli were presented on a CRT monitor (Sony FD Trinitron; 120 Hz refresh rate) at a viewing distance of 75 cm. E-Prime software (Psychology Software Tools, Pittsburgh, PA) was used for stimulus presentation and behavioural response collection. On every trial, a target face and a mask were presented in rapid succession (see Figure 1). Target faces were randomly drawn from a set of 40 photographs of 20 different individuals with either neutral or fearful expression. These were taken from a standard set of emotional faces (Ekman & Friesen, 1976). The twenty neutral faces employed as targets were used to generate scrambled face masks. This was done by using a purpose-written Matlab script that divided the inner region of each neutral face into a 5 × 5 matrix of tiles and then randomly rearranged these tiles. In this way, most local facial features (nose, eye, mouth) remained intact, but the global structure of the face was disrupted, and no attribution of facial expression was possible (see Figure 1). Only neutral faces were used to generate scrambled face masks in order to avoid that any of these masks would contain segments with facial features recognisably associated with fear (such as wide open eyes). Each face image subtended 7.6° × 11.2° of visual angle, and the size of the scrambled area of the mask was 5.1° × 7.4°. All stimuli were displayed in the centre of the screen.

Participants performed 12 blocks of 60 trials where fearful or neutral face targets were presented with equal probability and in random order for 8 ms (subliminal trials) or for 200 ms, resulting in 15 trials per block for each combination of target duration and facial expression. On each trial, a mask was randomly selected from the set of 20 scrambled face masks. The mask was presented immediately after target face offset for 150 ms, and was then replaced by a blank screen with a central fixation cross (0.5° × 0.5° visual angle). The next trial started 1950 ms after mask offset. Participants were instructed to decide on each trial whether the target face was fearful or neutral, and to guess when they were unable to discriminate its facial expression. Response mappings (left button press indicating a fearful target face, and right button press indicating a neutral target face, or vice versa) were balanced across participants.

EEG data acquisition and analysis

EEG data were recorded and digitized at a sampling rate of 200 Hz using a NeuroScan SynAmps amplifier (0.1-40 Hz bandpass). Signals were recorded from 23 electrodes mounted in an elastic cap at scalp sites Fpz, F7, F3, Fz, F4, F8, FC5, FC6, T7, C3, Cz, C4, T8, CP5, CP6, P7, P3, Pz, P4, P8, PO7, PO8, and Oz. Horizontal eye movements were measured from two electrodes placed at the outer canthi of the eyes. All impedances were kept below 5 kΩ. Scalp electrodes were referenced to linked earlobes. No additional filters were applied after EEG recording.

The continuous EEG was epoched offline from 100 ms before to 700 ms after the onset of a masked face target. Epochs with activity exceeding ±30 μV in the HEOG channel (reflecting horizontal eye movements) or ±60 μV at Fpz (indicating eye blinks or vertical eye movements) were excluded from further analysis, as were epochs with voltages exceeding ±80 μV at any other electrode. On average, 3.6% of all trials were removed from analysis due to the presence of artefacts (with trial rejection rate ranging from 0.1% to 13.1% across participants). Waveforms were then averaged separately for each combination of target face expression (fearful vs. neutral) and target face duration (subliminal: 8 ms; supraliminal: 200 ms). For supraliminal trials, ERPs were computed only for trials where participants correctly reported a fearful or neutral face target. For subliminal trials, where discrimination performance was at chance level (see below), ERPs were collapsed across trials where participants responded correctly or incorrectly.

To investigate early frontocentral effects of emotional facial expression, ERP mean amplitudes were computed within three successive time windows (140-180 ms, 180-250 ms, and 250-350 relative to target face onset) for frontopolar (F7, Fpz, F8), frontal (F3, Fz, F4), and central (C3, Cz, C4) electrodes. Initially, exploratory omnibus repeated-measures analyses of variance (ANOVAs) were conducted for each time window for the factors area (frontopolar versus frontal versus central), laterality (left versus midline versus right electrode), emotional expression (fearful versus neutral target face), and target duration (200 ms versus 8 ms). However, such analyses across different face target durations are potentially problematic, because visual ERPs elicited by face target offset and mask onset are inevitably triggered at different latencies relative to the onset of a target face. Therefore, other analyses were performed separately for both face target durations. To investigate the impact of facial expression on the face-specific N170 component, additional analyses were conducted for mean amplitudes obtained in the 140-180 ms time window at occipito-temporal electrodes (P7/8) where this component is maximal. Additional analyses were conducted for P3 amplitudes that were quantified on the basis of mean amplitudes between 400 and 600 ms post-stimulus at Pz (where Liddell et al., 2004, found a P3 enhancement for supraliminally presented fearful faces).

Results

Behavioural results

In supraliminal trials (200 ms target face duration), response accuracy was 94% and clearly above chance (50%; t(13)=36.1, p<.001). In contrast, the accuracy of facial expression discrimination in subliminal trials (8 ms target duration) was 50.5% and did not differ from chance (t<1). To obtain an additional estimate of participants’ ability to detect fearful target faces in supraliminal and subliminal trials, d’ values (Macmillan & Creelman, 1991) were calculated on the basis of correct responses on trials with fearful targets (defined as Hits) and incorrect responses on trials with neutral face targets (defined as False Alarms). When target faces were presented for 200 ms (supraliminal trials), mean d’ was 3.39, which was significantly greater than zero, t(13)=16.4, p<.001. For subliminal trials (8 ms target duration), mean d’ was 0.01, which was smaller than the d’ observed in the supraliminal condition, t(13)=16.4, p<.001, and did not significantly differ from zero (t<1). Response bias c (Macmillan & Creelman, 1991) was computed separately for supraliminal and subliminal trials. Although participants were slightly more likely to report the presence of a neutral face on subliminal trials (c=0.34), this tendency did not reach significance, t(13)=1.9, p=.075. For supraliminal trials, c was 0.08, and did not differ from zero, indicating the absence of any response bias.

Reaction times (RTs) were analysed separately for both target durations. Participants were faster to correctly report a fearful face than a neutral face when these were presented for 200 ms (588 vs. 626 ms; t(13)=4.1, p<.001). In the subliminal condition (8 ms target duration), RTs for trials containing fearful and neutral target faces did not differ (674 vs. 671 ms; t<1).

ERP results

Figure 2 shows ERP waveforms elicited in the 350 ms interval after target onset on supraliminal trials where fearful and neutral masked target faces were presented for 200 ms, and their facial expression was correctly identified. As expected, a sustained and broadly distributed enhanced positivity was elicited for fearful as compared to neutral faces that started about 140 ms after the onset of masked target faces.

Figure 2.

Grand-averaged ERPs elicited in the 350 ms interval after face target onset by correctly reported masked neutral faces (solid lines) and correctly reported masked fearful faces (dashed lines) on supraliminal trials (200 ms target duration).

Figure 3 shows the corresponding ERPs for fearful and neutral target faces on subliminal trials where these faces were presented for only 8 ms. Because discrimination performance in this condition was at chance level, ERPs to fearful and neutral target faces were computed irrespective of whether responses were correct or incorrect. An enhanced early positivity in response to fearful as compared to neutral target faces was elicited at anterior electrodes in the subliminal condition. This effect emerged at the same time as the emotional positivity observed in the supraliminal condition, but was considerably smaller and disappeared at about 180 ms post-stimulus. An enhanced N2 component that was not observed with supraliminal faces was triggered by fearful relative to neutral face targets at frontal and central electrodes. In addition, the face-specific N170 component observed at lateral posterior electrodes P7/P8) appears unaffected by emotional facial expression in supraliminal trials (Figure 2) as well as in subliminal trials (Figure 3).

Figure 3.

Grand-averaged ERPs elicited in the 350 ms interval after face target onset by masked neutral faces (solid lines) and masked fearful faces (dashed lines) on subliminal trials (8 ms target duration).

Finally, Figure 4 shows ERPs obtained at Pz in the 600 ms interval after face target onset to fearful and neutral target faces on supraliminal (top panel) and subliminal (bottom panel) trials. An enhanced P3 to fearful as compared to neutral target faces is present for supraliminal trials, but appears entirely absent on subliminal trials. These informal observations were then substantiated by statistical analyses.

Figure 4.

Grand-averaged ERPs elicited in the 600 ms interval after face target onset at electrode Pz on supraliminal trials (top panel) and subliminal trials (bottom panel) by masked neutral faces (solid lines) and masked fearful faces (dashed lines).

140-180 ms post-stimulus

In the initial omnibus ANOVA, a highly significant main effect of emotional expression, F(1,13)=40.4, p<.001, reflected the presence of the early enhanced positivity for fearful as compared to neutral target faces shown in Figures 2 and 3 for supraliminal and subliminal trials, respectively. Importantly, a significant target duration × emotional expression interaction was also obtained, F(1,13)=6.3, p<.03, indicating that the size of this early emotional expression effect was larger for supraliminal relative to subliminal trials. However, while differing in size, there was no evidence that this early effect also differed in terms of its topography between supraliminal and subliminal trials, as none of the relevant interactions (target duration × emotional expression × area; target duration × emotional expression × laterality; target duration × emotional expression × area × laterality) approached statistical significance (all p>.12).

The other analyses were conducted separately for supraliminal and subliminal trials. For supraliminal trials, main effects of emotional expression were present at frontopolar, frontal, and central electrodes, F(1,13)=10.1, 10.2, and 16.4, respectively, all p<.007. In subliminal trials, a significant main effect of emotional expression was present at frontopolar electrodes, F(1,13)=4.9, p<.05, demonstrating a reliable early emotional positivity in response to masked fearful relative to neutral faces. Although Figure 3 suggests that this early modulation may also be present at frontal electrodes, the emotional expression effect was neither significant for frontal sites, F(1,13)=2.9, p=.113, nor for central sites, F(1,13)=2.0, p=.181.

To further investigate whether the reliable emotion-specific early positivity at frontopolar electrodes for subliminal trials might be due to some residual awareness of fearful faces for some participants, two additional analyses were conducted. First, we ran a median-split analysis based on individual d’ values obtained in the subliminal condition, with a group of seven participants with d’ below the group median (mean d’ -0.16) and another group of seven participants with a d’ above the group median (mean d’ 0.18). A main effect of emotional expression, F(1,12)=4.8, p<.05, was again obtained for mean amplitudes in the 140-180 ms time window at frontopolar electrodes, but there was no indication of any emotional expression × group interaction (F<1), thus indicating that variations in face discrimination performance across participants were not systematically related to the early emotional positivity. To provide additional evidence for this conclusion, we then computed Pearson correlation coefficients (r) between participants’ d’ values and the size of the early emotional expression effect at Fpz (quantified as mean difference amplitudes between ERPs to fearful versus neutral target faces in the 140-180 ms time window). An r of -.36 indicated a small and non-significant tendency (p=.20) for participants with higher d’ values (i.e., better detection performance) to have a smaller frontopolar emotional expression effect, thereby again confirming that any residual awareness of fearful target faces in some participants was not in any way linked to the size of this effect.

In addition, the face-specific N170 component was not affected by emotional expression on supraliminal or subliminal trials (see Figures 2 and 3). This component was quantified on the basis of ERP mean amplitudes obtained between 140 and 180 ms post-stimulus at lateral posterior electrodes P7/8. There was no indication of any emotional expression effect, F<1, or any target duration × emotional expression interaction, F<1, thus strongly suggesting that N170 amplitudes did not differ between trials with fearful and neutral target faces, both F<1.

180-250 ms post-stimulus

In the omnibus ANOVA, a main effect of emotional expression, F(1,13)=5.6, p<.035, was accompanied by a highly significant target duration × emotional expression interaction, F(1,13)=20.8, reflecting the fact that emotional expression effects of opposite polarity were triggered during this time window for supraliminal and subliminal trials. On supraliminal trials, main effects of emotional expression remained present at frontopolar, frontal, and central sites, F(1,13)=18.8, 13.9 and 7.3, all p<.02, reflecting the continued presence of an enhanced positivity for fearful target faces (see Figure 2). On subliminal trials, a main effect of emotional expression was present at frontal and central electrodes, F(1,13)=8.7 and 5.6, p<.012 and .034, respectively. This was due to a small but systematic enhancement of the N2 component in response to masked fearful as compared to neutral faces (see Figure 3), similar to previous observations by Liddell et al. (2004). No significant emotional expression effect was present for subliminal trials at frontopolar electrodes.

Analogous to the procedures used for the early frontopolar positivity observed for subliminal trials in the 140-180 ms time window, two follow-up analyses were conducted to test for the possibility that the enhanced N2 to masked fearful faces on subliminal trials might result from awareness of fearful faces in some participants. The median-split analysis based on participants’ d’ scores (see above) produced a main effect of emotional expression at frontal and central electrodes in the 180-250 ms time window, F(1,12)=8.1 and 5.5, p<.015 and .037, respectively, but no indication of any emotional expression × group interactions (both F<1), indicating that the N2 modulation did not depend on above-average discrimination performance. Next, Pearson’s correlation coefficients (r) were computed between participants’ d’ values and the size of the expression-induced N2 modulation at Fz (quantified as mean difference amplitudes between ERPs to fearful versus neutral target faces in the 180-250 ms time window). An r of -.012 (p=.966) demonstrated that d’ scores and the N2 enhancement in response to masked fearful faces on subliminal trials were essentially unrelated.

250-350 ms post-stimulus

In the omnibus ANOVA, a main effect of emotional expression, F(1,13)=11.9, p<.004, was again accompanied by a significant target duration × emotional expression interaction, F(1,13)=17.1, p<.001. On supraliminal trials, the sustained presence of an enhanced positivity in response to fearful target faces was reflected in main effects of emotional expression at frontopolar, frontal, and central electrodes, F(1,13)=13.1, 13.5, and 16.4, respectively, all p<.003. In contrast, no indication of any emotional expression effects were observed on subliminal trials, all F<1.1, all p>.335.

400-600 ms post-stimulus (P3)

The analysis of P3 mean amplitudes obtained at Pz revealed a main effect of emotional expression, F(1,13)=13.1, p<.003, together with a target duration × emotional expression interaction, F(1,13)=41.4, p<.001. As can be seen in Figure 4, an enhanced P3 to fearful as compared to neutral target faces was clearly present on supraliminal trials, and this was reflected in a significant emotional expression effect when ERPs for these trials were analysed separately, F(1,13)=26.1, p<.001. In contrast, no such P3 modulation for fearful versus neutral target faces was present on subliminal trials, F<1.

Discussion

The aim of the present study was to identify ERP correlates of subliminal fear processing that are triggered by masked fearful faces under conditions where these faces cannot be consciously discriminated from neutral faces. We compared ERPs in response to fearful and neutral target faces followed by scrambled face masks on supraliminal trials, where these faces were presented for 200 ms and were thus clearly visible, and on subliminal trials, where they were shown for only 8 ms. Participants were instructed to discriminate fearful and neutral target faces throughout the experiment. On supraliminal trials, where discrimination accuracy was 94%, ERP results confirmed the findings previously observed in studies with unmasked fearful faces (Ashley et al., 2004; Eimer & Holmes, 2002; Eimer et al., 2003; Holmes et al., 2003). Relative to masked neutral faces, fearful faces elicited an early enhanced positivity at frontopolar, frontal, and central electrodes that started at about 140 ms post-stimulus (see Figure 2). In addition, enhanced P3 amplitudes were observed at Pz for supraliminal trials containing fearful as compared to neutral faces (see Figure 4), again in line with previous observations (Schupp et al., 2004; Liddell et al., 2004).

The critical question was whether any emotion-specific ERP modulations obtained on supraliminal trials would also be present on subliminal trials. On these trials, participants’ face discrimination performance was at chance level. This demonstrated that the scrambled face masks employed in the present study were highly effective in preventing any conscious discrimination of emotional expression on subliminal trials. In other words, target faces were at or below the objective identification threshold. In spite of the fact that participants were unable to discriminate fearful and neutral target faces, ERPs obtained on subliminal trials showed systematic emotional expression effects (see Figure 3). A small but reliable enhanced positivity for fearful as compared to neutral faces was elicited at frontopolar electrodes between 140 and 180 ms post-stimulus. Importantly, the observation that this early emotion-specific ERP modulation on subliminal trials was entirely unrelated to variations in participants’ discrimination performance strongly suggests that it is not a reflection of any residual awareness of fearful faces that may have occurred for some participants on some subliminal trials.

This early emotional expression effect on subliminal trials was similar in terms of its onset, polarity, and topography to the effect observed for supraliminal trials, although the limited number of electrodes used in the present study would have precluded the detection of subtle topographic differences between subliminal and supraliminal trials. Given its presence on subliminal as well as supraliminal trials, this early fear-induced ERP modulation might reflect the activity of prefrontal brain processes involved in the rapid detection of emotionally significant sensory signals that is triggered even when such signals are insufficient to result in perceptual awareness. Emotion-specific single cell responses have indeed been recorded in prefrontal cortex at latencies comparable to the latencies of the ERP effects observed in the present study (Kawasaki et al., 2001).

Importantly, the early emotional positivity was significantly larger on supraliminal relative to subliminal trial. This difference might point towards a threshold mechanism in the rapid cortical processing of emotional facial expression, with conscious awareness of emotional faces emerging only when the sensory evidence available to prefrontal areas is sufficiently strong. Additional studies where the duration and detectability of masked emotional faces is systematically varied are required to further investigate the relationship between the size of early emotion-specific ERP effects and awareness. It should also be noted that in contrast to the present results, Liddell et al. (2004) did not find an early emotional positivity under supraliminal or subliminal viewing conditions. There are several possible reasons for this discrepancy that need to be explored in future experiments. For example, it is conceivable that the different types of masks used in these experiments (intact neutral faces versus scrambled face masks) may have yielded a different pattern of early ERP modulations. In addition, participants in the Liddell et al. (2004) study were only required to passively view masked faces, whereas the face target discrimination instructions used in the present experiment ensured that participants attended to the location of target faces on every trial. Such differences in attentional task requirements may have been partially responsible for the presence versus absence of early emotional expression effects in subliminal trials. As discussed in more detail below, previous studies (Holmes et al., 2003; Eimer et al., 2003) have indeed demonstrated a strong modulatory influence of spatial attention on early emotion-specific ERP effects.

While an early emotional positivity was present on supraliminal as well as subliminal trials between 140 and 180 ms post-stimulus, qualitative differences between these two trial conditions emerged beyond this time window. In marked contrast to supraliminal trials, where the emotional positivity remained present, an enhanced N2 to fearful versus neutral target faces was instead found at frontal and central electrodes for subliminal trials (see Figure 3). This remarkable dissociation between emotion-specific ERP responses on supraliminal and subliminal trials is similar to the observations by Liddell et al. (2004), who also found an N2 enhancement to subliminally, but not supraliminally, presented fearful faces. The fact that this effect was reproduced in the present study, where different masking procedures were used, focal attention was required throughout, and discrimination performance was assessed continuously, strongly suggests that it reflects a genuine and distinct electrophysiological correlate of subliminal emotion processing.

Liddell et al. (2004) suggested that the N2 enhancement to subliminally presented faces might reflect an orienting response to an emotionally salient event that is triggered prior to, and sometimes independently of conscious awareness, and is mediated by a subcortical pathway from the thalamus to the amygdala (see also Morris et al., 1999^; Pessoa, 2005). The present results support this view in so far as they demonstrate that this N2 modulation is elicited under conditions where face identification performance is continuously measured, and any impact of a residual conscious detection of fearful faces can be ruled out. However, given its nuclear structure of clustered neurones, the amygdala is an electrically closed system and thus largely inaccessible to ERP measures. It is thus unlikely that the N2 effect observed here and by Liddell et al. (2004), or the early frontopolar emotional positivity observed in the present study, directly reflect fear-specific amygdala activation. These effects are most likely generated at early neocortical stages of emotional processing that may however be directly contingent upon amygdala input.

A notable finding of the present experiment was that unlike the early emotional positivity, which was present in subliminal as well as supraliminal trials, the fear-specific N2 modulation was only elicited on subliminal trials, and was absent when faces were clearly visible. If this effect reflected an orienting response to emotionally significant events, such a response should presumably also be triggered, and perhaps even more strongly, to supraliminal fearful faces. One possibility is that a fear-induced N2 enhancement was in fact also elicited on supraliminal trials, but was superimposed by the much more pronounced sustained emotional positivity triggered within the same time range (see Figure 2), and was thus not visible in the ERP waveforms for supraliminal trials. Alternatively, it is conceivable that this N2 effect is only triggered under conditions where the presence of a fearful face is initially registered pre-consciously, but is then not confirmed by subsequent perceptual checks based on re-entrant top-down control signals, because the mask has by then removed all relevant sensory evidence. Again, further research is needed to find out whether the N2 enhancement triggered by subliminal fearful faces that was observed in the present study as well as by Liddell et al. (2004) is restricted to situations where a backward mask prevents the conscious registration of such stimuli.

Another dissociation between emotional face processing on subliminal and supraliminal trials was found for the P3 component. While a P3 enhancement for fearful relative to neutral target faces was clearly present on supraliminal trials, no such difference was observed on subliminal trials (see Figure 4). This observation, which is again in line with previous findings by Liddell et al. (2004), further confirms the interpretation of P3 amplitude modulations in response to emotional facial expression as reflecting the controlled activation of higher-level emotional face processing stages involved in the conscious evaluation and integration of emotional information.

Another finding of the present study is the absence of any systematic modulation of the face-specific N170 component by emotional facial expression on subliminal as well as on supraliminal trials. This confirms results from several previous ERP experiments, which have found the N170 to be insensitive to manipulations of facial expression (Eimer & Holmes, 2002; Eimer et al., 2003), although other studies (e.g., Batty & Taylor, 2003) have reported different findings (see Vuilleumier & Pourtois, 2007, for further discussion).

On a more general level, the fact that reliable early emotional expression effects were observed in the present study for subliminal trials may initially seem inconsistent with previous observations that such ERP effects are completely eliminated when attention is actively directed away from faces to the location of other task-relevant stimuli (Holmes et al., 2003; Eimer et al., 2003). If these effects depend critically on focal spatial attention, as suggested by these earlier studies, one could argue that they should also show similar sensitivity to variations in perceptual awareness induced by backward masking. However, the fact that early emotional expression effects were observed in the present study for subliminal trials, but were absent in previous studies in the absence of focal attention, might actually reflect an interesting dissociation between attention and awareness. Given that participants had to discriminate facial expression on every trial, and the fact that trials with clearly visible supraliminal face targets were randomly intermixed with subliminal trials, it can be assumed that spatial attention was consistently focused on the location of target faces in the present study. This fact alone may have been sufficient for early ERP effects of emotional expression to emerge even on subliminal trials, in spite of the fact that facial expression could not be consciously discriminated. In contrast, when spatial attention is directed elsewhere, such effects appear to be absent even when fearful faces are presented for much longer durations, and without subsequent mask. In other words, focal spatial attention might be necessary for the processing of supraliminal as well as subliminal emotional information (see also Pessoa & Ungerleider, 2004, for an analogous argument in favour of the attention-dependence of subliminal emotion perception).

In summary, the present study has provided new electrophysiological evidence that masked fearful faces elicit brain responses indicative of subliminal emotional processing when these stimuli are presented at or below an objective identification threshold. While an early emotional positivity that is triggered by subliminal fearful faces is also elicited to faces that are clearly visible, a subsequent enhanced N2 component may be specific to subliminal presentation conditions. Future ERP experiments will need to investigate the subliminal processing of emotional expression with facial expressions other than fear. For example, it will be important to find out whether a similar ERP signature of subliminal emotional processing can also be observed when negative expressions such as threat and anger are presented below the threshold of conscious awareness, or even when positive facial expressions such as happiness are used instead.