Interaction Between Conscious and Unconscious Information-Processing of Faces and Words

Abstract

It is widely acknowledged that holistic processing is a key characteristic of face perception. Although holistic processing implies the automatic integration of face parts, it is unclear whether such processing requires the awareness of face parts. Here, we investigated the interactions between visible face parts and face parts rendered invisible using continuous flash suppression (CFS). In the first experiment with the upper half-face visible and the lower half-face invisible, the results showed that perceived face identity was influenced by the invisible lower half-face, suggesting that integration occurs between the visible and invisible face parts, a variant of the “composite face effect”. In the second experiment, we investigated the influence of visible face parts on the processing of invisible face parts, as measured by the time it took for the invisible parts to break out from CFS. The results showed a visible-to-invisible facilitation effect, that the aligned invisible face parts broke through CFS faster than when the visible and invisible face parts were misaligned. Visible eyes had a stronger influence on the invisible nose/mouth than the other way around. Such facilitation of processing from visible to invisible parts was also found when Chinese characters were used as stimuli. These results show that information integration occurs across the consciousness boundary.

Introduction

Exploring the role of conscious awareness is an important endeavor of cognitive neuroscience [1]. One way to understand the function of consciousness is to investigate the capacity of unconscious processing [2–5]. Unconscious processes can affect a variety of cognitive, affective, and behavioral processes. Previous studies have shown that consciousness is not necessary to process information about orientation, luminance, or even facial expression [6–11]. The present research aimed to explore the holistic interaction between information processed consciously and unconsciously.

Since conscious and unconscious information-processing often occur at the same time, a natural question is whether they are based on fundamentally different mechanisms. More specifically, how much interaction occurs between conscious and unconscious information-processing? Related to this issue, previous studies have shown that contextual modulation effects can occur when the context is rendered invisible, and such contextual effects include orientation-processing, the size-contrast illusion, and symbolic object processing [12–15]. In the current study, we investigated the interaction between the information-processing of conscious and unconscious components when the components belong to the same object; specifically, when half of a stimulus is invisible and the other half remains visible. Is the observer able to integrate information across the visibility gap? Since faces were used as stimuli, the question becomes whether integration occurs across the conscious boundary for visual objects that are normally processed holistically.

The holistic processing effect refers to the processing of an object as a whole rather than a set of independent features, with different objects having different degrees of holistic processing [16]. Face recognition is widely considered to be the best example of holistic processing [17, 18]. Much evidence has demonstrated the holistic processing of faces, such as the part and whole effect [19], the inversion effect [20], and evidence from EEG studies [21]. However, the composite face effect is probably the most direct evidence [22]. In this effect, when two identical upper half-faces are paired with different lower half-faces, the judgment of the identical upper part is automatically affected by the lower half and perceived as different.

In the current study we used a variant version of the composite face effect. We rendered half of the face invisible and tested whether the two parts of the face still interacted. Regarding whether the composite face effect is carried out unconsciously, previous experiments reported negative results [23]. Here, we improved the sensitivity by optimizing the paradigm, for example, by adding transitional stimuli and blurring the identity of the upper face. Moreover, we tested another category of visual objects with which observers have an expert level of experience, namely written words. There is strong evidence that Chinese characters, similar to faces in that they have parts-whole relationships, also show a high degree of the holistic processing effect [24] and may induce similar processing effects in expert readers as found in face recognition [11, 25].

We conducted two complementary experiments in this study: In the first experiment on the contribution from unconscious to conscious information-processing, we specifically explored whether the invisible part of a face can still affect the identity judgment of the visible part of the face. In the second experiment on the influence from conscious to unconscious information-processing, we investigated whether the visible part of a face (or word) facilitated the processing of the invisible part of the face (or word). These two experiments together tested the interaction between the visible and invisible parts of a stimulus.

Materials and Methods

Participants

15 volunteers participated in Experiment 1a, 18 in Experiment 1b, and 15 in Experiment 2. All participants were healthy adults with normal or corrected-to-normal vision. We determined that a sample size of 15 was needed for each experiment, based on sample-size calculation using the software G*power [26], with α = 0.95, power = 0.8, and the estimated effect size for t-test = 0.8, that for ANOVA = 0.4, based on a pilot experiment. The experimental protocols were approved by the Institutional Review Board of the Institute of Biophysics, Chinese Academy of Sciences.

Invisible Modulation

Continuous flash suppression (CFS) [4, 27] was used to render stimuli in the unconscious state. In this paradigm, a train of dense color-rich rectangular Mondrian-like masks were flashed into one eye (Figs. 1A, 2A), effectively covering part of the. stimulus presented to the other eye. As we were interested in holistic processing across the conscious and unconscious states, so half of the objects were masked: the edge of the suppression coincided with the centerline of the stimulus image, and the other edges were 0.5° from the border (Figs. 1A, 2A). A mirror stereoscope was used to fuse the images that were displayed side-by-side to project them to each eye. A chin rest, background frame (10° × 12°), and fixation spot were set to achieve stable image merging in the eyes (Figs 1A, 2A).

Fig. 1.

Paradigm and stimuli of Experiment 1. A Paradigm of Experiment 1. After 500 ms of fixation, the stimulus appears for 200 ms. The stimulus location is semi-randomized for each presentation. The participants indicated a two-alternative forced-choice (2AFC) response of the identity of the face. B Example of face stimuli used in Experiment 1. The test stimuli gradually change from a happy female to a sad male. The change of the face is much more salient in the lower half of the face, with the upper half of the face being similar except in the original faces HF0 and SM0. With the same procedure, we generated three sets of face stimuli. C Example of the control condition used in Experiment 1a. The upper half-faces are the same for the test and control conditions. In the control condition, the lower facial components are spatially shuffled. D Examples of stimuli used in Experiment 1b. The stimuli modulations in the main experiment are aligned as shown in B for the test condition, and top-only and misaligned for the control conditions. E Example of face stimuli used in Experiment 1–exploratory control experiment. The stimuli are two groups of faces with the same set of five upper parts, with one group combined with the original lower face part HF0 and the other group combined with the original SM0 from B.

Fig. 2.

Paradigm and stimuli of Experiment 2. A The break-suppression paradigm of Experiment 2. After 500 ms of fixation, the stimulus appears and stays until the observer responds, seeing the stimulus break from the mask. The stimulus location is semi-randomized for each presentation. B The stimuli of Experiment 2. Ten faces and ten words are used in this part of experiment. C Manipulations of stimuli used in Experiment 2. The position of the CFS mask depends on the manipulation.

Stimuli

Stimuli were presented on a 15-inch NESOJXC FS210A monitor (1024 × 768 at 60 Hz, Nanchang, China) using the MatLab Psychophysics Toolbox [28]. The viewing distance was 70 cm. The whole experiment was carried out in a dark room.

In Experiments 1a and 1b, the stimuli were a series of faces (see, for example, Fig. 1B), with identity gradually changing from happy female (HF) to sad male (SM). Adobe Photoshop CS6 was used to manipulate the images, and FantaMorph 2.5 generated the faces at intermediate levels of identity. As we wanted to generate faces with similar upper parts and quite different lower parts, we first morphed the original faces (HF0 and SM0) to get a series of faces that gradually changed from HF0 to SM0. Second, we picked two images, 30% and 70% SM0, so that they looked like HF0 or SM0, respectively, but without very strong characteristics. Third, we only used the upper part (above midline) of these two selected faces, composited with the original HF0 and SM0’s lower part (below midline), to create faces HF1 and SM1. Fourth, we morphed HF1 and SM1 and picked three middle images, 25%, 50%, and 75% SM1, to create HF2, MF (middle face), and SM2. Finally, we obtained seven images with similar upper parts (0%, 30%, 40%, 50%, 60%, 70%, and 100% SM0 upper parts) and quite different lower parts (0%, 0%, 25%, 50%, 75%, 100%, and 100% SM0 lower parts). In addition, if there was only one set of faces, the subject might easily find features with great difference between the two faces, so that the identity of the face could be judged based only on local details, ignoring the information of the lower half-face hidden in the CFS; to prevent this, we used three sets of faces.

In Experiment 2, we used 10 neutral faces (5 male) and 10 Chinese characters (Fig. 2B). To avoid collinear interference as much as possible, each Chinese character had no strokes passing through the centerline, and to make each character easier to integrate, the upper and lower parts were not independent words.

General Design and Procedures

In Experiment 1, we combined a variant version of the composite face effect paradigm with CFS; instead of comparing two consecutive pictures [22, 29], we measured the psychometric function of forced-choice identity judgment for the morphed faces, to explore whether the lower part of the face was still able to change the perception of the upper part even when the lower part was processed without awareness. After the manipulated face stimulus was flashed for 200 ms, participants were required to judge if the stimulus looked more like a HF or a SM (Fig. 1A). As the visible parts of the stimuli were modulated to be very similar and the invisible parts held most of the identity information, if a holistic interaction occurred between conscious and unconscious perception, the invisible part would contribute to the discrimination of the visible part of the face, and the slope of the psychometric function would be steeper.

In Experiment 2, we used the breaking CFS paradigm to check whether the visible part of the face or Chinese character influenced the invisible part (Fig. 2A). If so, the suppression time would differ between the alignment and misalignment modulations. We also investigated whether inversion and masking of different components (component-suppressed) affected holistic processing in the absence of awareness. To explore these two factors, this experiment used a 2 (upright/inverted) × 2 (lower-suppressed/upper-suppressed) design, comparing the differences between the aligned and misaligned modulations under different manipulations (Fig. 2C).

Experiment 1: Contribution to the Composite Face Effect from Invisible Face-Parts

Aiming to address the question more comprehensively and rigorously, we performed two variations of the experiment. In Experiment 1a, we kept the components of the lower half of the face, only disrupting their position (Fig. 1C). In Experiment 1b, we compared the effects among three conditions: upper and lower half-faces aligned, misaligned, and top-only (Fig. 1D). In the exploratory control experiment, we tested a possible sensitivity-based explanation for why we obtained positive effects but previous studies did not. Similar to previous studies, we used intact faces with different invisible face-parts as the comparison conditions. The difference was that we added transitional faces to stimuli and participants were only required to judge the identity of one stimulus instead of comparing two stimuli.

Experiment 1a: Comparison with the Component-Shuffled Condition

In this part of the experiment, we used the shuffled face components as the control condition, so that the control condition contained spatially unorganized facial component information. We used the online visibility test to evaluate the visibility of the suppressed components as well as Bayesian statistics [30] to assess the significance level of detecting the suppressed components.

In Experiment 1a, we shuffled the components of the lower part as the control condition (Fig. 1C), and the three groups of stimuli were shuffled in different ways. The contrast of the test figure (5° × 4.2°) was ramped up gradually (from 0% to 100% over 200 ms), presented to the randomly-selected left or right eye (balanced between the eyes) at a random location centered within an area 1.3°–2.0° above the fixation point; 0.23°–0.42° laterally from fixation either to the left or right side of fixation with equal probability (Fig. 1A).

In Experiments 1a and 1b, every participant completed two pre-tests before the main test to ensure that the experiment was carried out unconsciously and to exclude participants with poor face integration ability even in the conscious state. In the first pre-test, we adjusted the CFS contrast by staircase. The stimulus was presented in the same way as in the main experiment, and the task was to see whether the stimulus broke from the mask or not. After at least 30 trials (until the staircase reached a stable contrast level), if the steady CFS contrast was <50%, the participant was accepted, and this contrast was used in the main test. The contrast was in the range 40%–70% for face and 30%–50% for CFS. The second pre-test was an identity two-alternative forced-choice task with the whole face visible and top-only conditions. If the response for face recognition at any morph level in the psychometric function of the whole-face condition differed by >3 SDs compared with other participants, or the difference between the psychometric function of the whole face and that of the top-only condition was too small, the participant was excluded.

In the main test, participants were required to answer three questions in each trial. First, in the identity two-alternative forced-choice task, they had to identify whether the face looked more like a HF or a SM. Second, in the subjective visibility report, they reported how much of the image they had seen under CFS: saw something (left arrow key), not sure (down arrow key), or nothing (right arrow key). Third, in the objective forced-choice test, they chose whether the lower part had intact component positions or shuffled component positions. Response keys were presented (“happy female or sad male,” “left clear right none,” “intact face or shuffled nonface”) to reduce memory load and improve accuracy. Each participant completed at least 3 rounds, each round had 84 trials, and intact faces and component-shuffled faces were balanced within each round.

Experiment 1b: Using the Top-Only and Top with Invisible Misaligned Lower Part as Control Conditions

We learned from Experiment 1a that the information of the lower part of the intact face processed without consciousness still affected the recognition of the upper half of the face. In this part of the experiment, we investigated whether the effect of the lower part was based on “integration with the upper part” or simply from an influence of the lower part on its own, by changing the spatial relationship between the upper and lower parts of the faces.

The stimuli in Experiment 1b were almost the same as those in Experiment 1a, except that different control stimuli were used. The stimuli were top-only and top with aligned or misaligned invisible lower part (Fig. 1D). The image was separated from the midline for all the control conditions. The top-only condition had only the upper half of the face, and the misaligned condition moved the lower half of the face to the left or right by 1/3 of the width of the image in the main test. The visibility test stimuli shifted the lower half of the face to the left or right by 1/25 of the width of the image, as this level of spatial shift was well above the detection threshold when the images were visible.

In the main test, after presenting the lower-suppressed image, participants were required to report whether the low half-face broke from the mask or not, and pressed the space key if it did to end the trial; the trial was added back to this round at the end. If it did not break, participants judged if the face looked more like a HF (left-arrow key) or a SM (right-arrow key). All stimulus conditions (aligned face, misaligned, and top-only) were mixed and balanced within the round. Each participant performed at least 4 rounds with 252 original trials in each round. If the suppression was broken 4 times consecutively in the supplemental trials, the round ended.

The task in the objective visibility test was to determine whether the lower part under CFS was shifted to the left or right relative to the visible top part of the face when CFS was not broken; if it was broken, the trial passed. If the lower parts of the faces were processed without consciousness, then the position judgment should be at chance level. The amplitude of the distance shift was well above the threshold in the conscious state. Each participant performed at least 2 rounds with 90 original trials in each round. There were at least 170 successfully masked trials in the visibility test.

Exploratory Control Experiment: Positive Holistic Effect Due to More Sensitive Testing Stimuli

A previous study [23] reported a lack of holistic processing of faces without awareness. However, this null result could be due to a combination of weak effect and insensitive testing methods. An important difference in study design between our study and theirs is that more transitional faces were included in our study and the visible upper faces were smoothly blended. As a result, the differences between upper half-faces were more refined, making it more sensitive to the influence of the information from the invisible lower half-face. In this experiment, we tested this assumption in order to reconcile the previous null results and the current positive findings.

The traditional composite face illusion paradigm typically compares two faces with very different lower parts and the exact same upper part, to see whether the different lower part modulates the perception of the identical upper part. Here, we modified this approach while adding transitional faces introducing refined differences in the identity of the upper faces so that the upper part could be more sensitive to the influence from the invisible lower part.

Similar to the stimulus generation in Experiments 1a and 1b, the original HF and SM faces were subjected to morph treatments to generate three intermediate stimuli, resulting in five different upper faces transitioning from HF to SM. These five upper half-faces were then combined with the lower half of the two original faces to generate the two sets of stimuli. These two sets of stimuli, with the same five upper half faces, were compared against each other, to see whether the lower parts with different identity influenced the upper half-faces so they would be perceived differently. We hypothesized that the middle range transitional upper faces could make the test more sensitive to the influence of the lower half-faces (Fig. 1E).

In this experiment, we used a relatively simple design. After the stimulus was presented (200 ms), participants first judged whether the lower part of the face was visible (i.e., ineffective CFS suppression). If it was not fully suppressed the trial was skipped. Otherwise, they were required to judge the identity of the face. The same contrast of the face stimuli (50%) and CFS (30%) was used for all participants.

Experiment 2: Visible Parts of a Face (or Word) Facilitate the Processing of Invisible Parts of the Face (or Word)

In the first experiment, we found that face recognition based on the upper part of faces was influenced by the invisible lower part of faces through integration of the facial parts. In Experiment 2, we investigated the reverse interaction from the visible half-image to the invisible half-image. In addition to faces, we included Chinese characters as stimuli in this experiment. The inclusion of Chinese characters provided an opportunity to see if the potential influence from visible to invisible components of a stimulus is unique to faces or more generally applies to stimuli that we have expertise in processing.

The contrast of the test figure (4.5° × 3.7° for an aligned face) was ramped up gradually (from 0% to 100% over 200 ms) to one of the observer’s eyes (balanced between the two eyes with a random sequence) at a random location: 1°–1.5° below the fixation point; 0.467°–0.767° randomly on left or right side of the fixation point with equal probability (Fig. 2A). For the misaligned condition, we kept the top part still and moved the lower part to the opposite side of the fixation point. The task was to react as soon as possible when the mask failed by choosing which side of the fixation point the suppressed part popped out of (left side left-arrow key, right side right-arrow key). In order to allow the participants to assign spatial attention more easily and stably, in one round, CFS only appeared above or below the fixation point. Faces and words were also assigned in different rounds. Conditions were balanced and mixed within a round, each round had 80 trials, each participant completed at least 4 rounds, and the round order was balanced across participants.

Results

Experiment 1a: Comparison with the Component-shuffled Condition

We divided all the trials in the main test into three groups according to the response to the second question, the subjective visibility level. Then, within subjective levels, we measured the accuracy of the response to the third question, the objective visibility two-alternative forced-choice task. Using the binomial distribution z-test, we tested whether the accuracy was at chance level, except in the “saw something” group. If the forced-choice accuracy in the visibility level of “saw nothing” was not at chance level, this participant was excluded. Otherwise, all the trials in this visibility level were classified as effective invisible trials. If the forced-choice accuracy in the visibility level of “not sure” was also at chance level, these trials were included as effective invisible trials as well. The effective invisible trials were used to calculate percentage responses for face recognition at each morph level faces in different conditions (Fig. 3A). The slope for each participant was calculated using a logistic psychometric function [31] (Fig. 3B). Participants with data points in the psychometric function 3 SDs outside of the mean were excluded from further analysis. Eighteen subjects completed this test, with data from three participants excluded from further analysis based on the exclusion criteria.

Fig. 3.

Composite face effect across visible and invisible facial components in Experiment 1a. A The psychometric functions are for face identity judgments for the morphed faces in two conditions. The x-axis represents the morph levels of the upper parts, and the y-axis represents the percentage of response of sad male (SM). B The slope for the condition with invisible intact lower faces is sharper than that for the shuffled control condition, and the mean matching slope differs significantly across stimulus modulations (t₍₁₄₎ = 3.066, P = 0.008, Cohen’s d = 0.792, BF₁₀ = 6.447). Error bars denote the SEM. **P <0.01.

The slope of the psychometric function was considered to represent the ability to judge the identity of the face images. Comparing the slopes between the test and control conditions with the Bayesian paired samples t-test, the slope difference was t₍₁₄₎ = 3.066, P = 0.008, Cohen’s d = 0.792, BF₁₀ = 6.447 (6.447 times against the null hypothesis using JASP software [32]), indicating that discrimination performance was significantly higher in the intact face condition than in the component-shuffled face condition (Fig. 3B).

Experiment 1b: Using the Top-Only and Top with Invisible Misaligned Lower Part as Control Conditions

First, we tested whether participants’ performance in the forced-choice visibility test was above chance level using the binomial distribution z-test. Two participants performed above chance and their data were not analyzed further.

Then, the remaining participants’ data were used to generate the psychometric function corresponding to the stimulus conditions (Fig. 4A). Next, logistic fitting was used to derive the slope of each curve. The within‐subjects ANOVA of the slope revealed a significant main effect of modulation, F_{(2, 34)} = 18.074, P <0.001, partial η² = 0.515, BF₁₀ = 3103.652. Comparing the slope between conditions using the paired samples t-test, the aligned condition was significantly higher than the top-only condition (t₍₁₇₎ = 4.558, P <0.001, Cohen’s d = 1.074, BF₁₀ = 113.871, after Bonferroni correction) and the misaligned condition (t₍₁₇₎ = 4.712, P <0.001, Cohen’s d = 1.111, BF₁₀ = 151.977), and there was no significant difference between the two control conditions (t₍₁₇₎ = 0.872, P = 0.396, Cohen’s d = 0.205, BF₁₀ = 0.340; Fig. 4B).

Fig. 4.

Composite face effect from integrating visible and invisible facial components in Experiment 1b. A The psychometric functions of face identity judgments for the morphed faces under three conditions. The x-axis represents the morph levels of upper parts, and the y-axis represents the percentage of response of sad male (SM), showing that the slope for the condition with invisible aligned lower faces is sharper than that for the misaligned and top-only control conditions. B The mean matching slopes across stimulus modulations are significantly different for the test condition compared with the control conditions (Top-only: t₍₁₇₎ = 4.558, P <0.001, Cohen’s d = 1.074, BF₁₀ = 113.871; Misaligned: t₍₁₇₎ = − 4.712, P <0.001, Cohen’s d = 1.111, BF₁₀ = 151.977). C The relevance (r = − 0.137, P = 0.587) of the scatterplot indicates that there is no association between the slope for the ability to recognize identity (x-axis) and the forced-choice accuracy of the visibility test (y-axis). The results are corrected for multiple comparisons. Error bars denote the SEM. **P <0.01.

To further verify that the effect found in the main experiment was not due to the potential residual visibility of the lower part, we calculated the correlation between the effect size and the forced-choice accuracy of the visibility test (Fig. 4C). The difference between the slopes of the aligned and misaligned conditions was taken as the effect size. The results showed that there was no correlation (r = −0.137, P = 0.587). This result showed that the unconscious part of the face contributed to the identity judgment of the conscious part of the face only when the two parts of the face were aligned, supporting the hypothesis that the effect came from the integration between the two parts.

Exploratory Experiment: Positive Holistic Effect Due to More Sensitive Testing Stimuli

In this experiment, we were interested in whether the identity discrimination of each pair of faces (SM vs HF) with the upper half-faces visible would be influenced by the invisible lower half of the face. Figure 5 shows plots of the percentage of reporting “Sad Male” for the two groups of five images (Fig. 1E). The same group of five upper faces was perceived differently depending on the invisible lower half-faces. Faces with invisible lower faces from SM/HF were more likely to be reported as SM/HF. Data were collected from 9 participants, with results clearly showing the increased sensitivity due to stimuli selection.

Fig. 5.

Composite face effect in the exploratory control experiment. The points represent face identity judgments for morphed faces in the upper part with different original lower parts. The x-axis represents the morph levels of upper parts, and the y-axis represents the percentage of response of sad male (SM). Between the two groups, only the responses of the transitional faces are significantly different (F2: t₍₈₎ = 3.774, P = 0.005 Cohen’s d = 1.258, BF₁₀ = 10.355; F3: t₍₈₎ = 2.701, P = 0.027, Cohen’s d = 0.900, BF₁₀ = 2.917; F4: t₍₈₎ = 2.454, P = 0.04, Cohen’s d = 0.818, BF₁₀ = 2.174). Error bars denote the SEM. *P <0.05,**P <0.01.

For the purpose of this experiment, it is important to note that the influence from the invisible lower half-face was significant for the three middle face conditions (F2: t₍₈₎ = 3.774, P = 0.005 Cohen’s d = 1.258, BF₁₀ = 10.355; F3: t₍₈₎ = 2.701, P = 0.027, Cohen’s d = 0.900, BF₁₀ = 2.917; F4: t₍₈₎ = 2.454, P = 0.04, Cohen’s d = 0.818, BF₁₀ = 2.174), but not significant for the two end conditions with the original HF and SM faces (F1: t₍₈₎ = 1.795, P = 0.11, Cohen’s d = 0.598, BF₁₀ = 1.021; F5: t₍₈₎ = 1.053, P = 0.323, Cohen’s d = 0.351, BF₁₀ = 0.502, Fig. 5).These results support our hypothesis that the transitional upper half-faces are more sensitive to the influence from the invisible lower half-faces, and thus able to reveal the effect of holistic face processing without consciousness.

Experiment 2: Visible Parts of a Face (or Word) Facilitate the Processing of Invisible Parts of the Face (or Word)

The critical measure in this experiment was the response time, i.e., how long it took for the initially suppressed parts of a face (or word) to break suppression and be detected. We calculated the correct rate (all >93%) and every condition’s mean reaction time (Fig. 6). The suppressed half of the image would be more effective in breaking suppression if the visible half of the image exerted a facilitatory influence on the conscious access of the invisible half. The time it took the invisible half of the image to break suppression should reflect this potential facilitation effect. More specifically, since the stimuli included both the upper-lower aligned and misaligned images, the reaction time difference between the misaligned vs aligned conditions could be used to index the facilitation of conscious access from the visible to the invisible parts.

Fig. 6.

The results of Experiment 2. A Facilitation from visible to invisible face parts. The mean break-suppression times for different pairs of conditions. For both the upright and inverted faces, lower face part suppressed conditions and inverted faces with lower suppressed condition show significant differences (upright lower-suppressed: t₍₁₄₎ = 5.706, P <0.001, Cohen’s d = 1.473, BF₁₀ = 490.696; upright upper-suppressed: t₍₁₄₎ = 3.748, P = 0.008, Cohen’s d = 0.968, BF₁₀ = 20.056; inverted lower-suppressed: t₍₁₄₎ = 3.527, P = 0.012, Cohen’s d = 0.911, BF₁₀ = 13.836). The results are corrected for multiple comparisons. Error bars denote the SEM. B Facilitation from visible to invisible Chinese character parts. The mean break-suppression times for different pairs of conditions. Only the upright lower word part suppressed condition showed a significant difference (upright lower-suppressed: t₍₁₄₎ = 5.422, P <0.001, Cohen’s d = 1.400, BF₁₀ = 314.628). The results are corrected for multiple comparisons. Error bars denote the SEM. *P < 0.05, **P < 0. 01, ***P < 0.001.

For the face simulation, reaction time differences among conditions were analyzed using a three-way repeated measures ANOVA involving alignment (aligned/misaligned), inversion (upright/inverted), and component-suppression (lower-suppressed/upper-suppressed). The reaction times for each condition are shown in Fig. 6A. The ANOVA showed a significant main effect of alignment (F_{(1, 14)} = 50.140, P <0.001, partial η² = 0.782, BF₁₀ = 5,115.997), indicating that for the aligned face, was still holistic processing to drive the visible part of the face, helping the invisible part to break the suppression faster. Since the misaligned condition invalidated holistic processing comparing with the main effect (inversion: F_{(1, 14)} = 26.587, P <0.001, partial η² = 0.655, BF₁₀ = 38.452; component-suppression: F_{(1, 14)} = 17.581, P <0.001, partial η² = 0.557, BF₁₀ = 57,542.383), the interaction with alignment better explained the influence of inversion and component-suppression on the holistic processing. The effects of inversion × alignment (F_{(1, 14)} = 5.787, P = 0.031, partial η² = 0.292, BF₁₀ = 3.009) and component-suppression × alignment (F_{(1, 14)} = 21.169, P <0.001, partial η² = 0.602, BF₁₀ = 13.030) were both significant, suggesting that facilitation was influenced by inversion and component-suppression. After Bonferroni correction, only inverted upper-suppressed showed no difference (t₍₁₄₎ = 0.359, P = 1.000, Cohen’s d = 0.093, BF₁₀ = 0.278); the other three groups all showed significant difference (upright lower-suppressed: t₍₁₄₎ = 5.706, P <0.001, Cohen’s d = 1.473, BF₁₀ = 490.696; upright upper-suppressed: t₍₁₄₎ = 3.748, P = 0.008, Cohen’s d = 0.968, BF₁₀ = 20.056; inverted Lower-suppressed: t₍₁₄₎ = 3.527, P = 0.012, Cohen’s d = 0.911, BF₁₀ = 13.836). The holistic processing effect of lower-suppression groups was higher than that of upper-suppression by pair comparison, showed that for facilitation “eyes-to-mouth > “mouth-to-eyes”. The holistic processing effect of upright groups was higher than that of inverted groups by pair comparison and the inverted lower-suppressed still showed a significant difference, indicating that inversion reduces the holistic processing but it does not completely disappear.

In this experiment, we also included Chinese characters as stimuli (Fig. 6B). The data processing and analysis methods for the Chinese character conditions were the same as for the faces. The ANOVA showed a significant main effect of alignment (F_{(1, 14)} = 14.015, P = 0.002, partial η² = 0.500, BF₁₀ = 6.463), indicating that holistic processing still occurred for partly-suppressed words. The interaction between inversion and alignment was significant, indicating that for the holistic processing of words, the upper and lower components are also asymmetric. The interaction between component-suppression and alignment was not significant, showing that the influence of inversion on holistic processing did not reach a significant level. The Bonferroni-corrected t-test showed that only upright lower-suppressed showed a significant difference (t₍₁₄₎ = 5.422, P <0.001, Cohen’s d = 1.400, BF₁₀ = 314.628), inverted lower-suppressed showed a marginally significant difference (t₍₁₄₎ = 2.672, P = 0.072, Cohen’s d = 0.69, BF₁₀ = 3.415), and the other two groups showed no effect (upright upper-suppressed: t₍₁₄₎ = 0.850, P = 1.000, Cohen’s d = 0.219, BF₁₀ = 0.359; inverted upper-suppressed: t₍₁₄₎ = 0.360, P = 1.000, Cohen’s d = 0.093, BF₁₀ = 0.278). The same as for faces, the holistic processing effect of lower-suppression groups was higher than that of upper-suppression, showing that for facilitation “upper-to-lower > “lower-to-upper”. For inversion, based on our data, upright lower-suppressed reflected greater holistic ability than inverted lower-suppressed, indicating that the holistic processing of the inverted word was weakened, but the difference was not significant. The inverted word showed a trend of holistic processing.

Discussion

In this study, we investigated the potential integration of and interaction between information from visible and invisible components (i.e., across the conscious boundary) of faces and Chinese characters. The main experimental findings were that the invisible lower part of a face affected the perceived identity of the visible upper half-face, and that the visible part of a face (or Chinese character) facilitated the conscious access of the invisible part of the face (or Chinese character). These results constitute clear evidence of integrative interaction between the visible and invisible parts of a face, indicating that holistic processing of the faces does not require conscious awareness of all components. In addition, integrative interaction across the visibility boundary occurs for other types of visual stimuli beyond faces.

Importance of Visibility Control

In studies of information processing without awareness, it is of course important to ensure that the stimuli intended to be invisible indeed are not consciously perceived by observers. To this end, we used both subjective and objective measures of visibility in the main experiment (Experiment 1a). In the subjective test, participants reported the visibility of the masked components after each trial by selecting one of three options: saw nothing, not sure, and saw something. Participants also performed an objective visibility test to determine whether the suppressed components were intact or scrambled (no recognizable parts). If the participants’ perception of the masked stimuli did not help them to determine whether there were face parts in the image, then very little, if any, face information was consciously available to them, let alone information related to face identity and emotion. Together, these two visibility measures helped to ensure that the invisible components remained outside participants’ awareness. In Experiment 1b, the visibility test was performed immediately after the main test. Since the CFS effect typically weakens over time, if participants failed to determine the position of the stimulus under CFS suppression in the visibility test, indicating effective suppression, we would expect an even better suppression effect in the main test. We also examined the relationship between the visibility index and the effect size of holistic processing across participants. The fact that there was no correlation between the two provides further support for the interpretation that the holistic processing between visible and invisible components was not due to incomplete CFS suppression.

Evidence for Holistic Processing Without Awareness using More Sensitive Methods

The early efforts in addressing the question of whether holistic processing of faces can occur without awareness obtained null results [23]. We believe that a key reason for the current positive results supporting holistic processing is the selection of more sensitive test stimuli. In our experiments, we selected two faces with very different characteristics (gender and expression), and generated test faces with relatively small differences in the visible upper parts but very large differences in the invisible lower parts, to increase the potential effect size. Meanwhile, the slope of the psychometric function was adopted to index the potential integration effect; this was more sensitive than the mean response percentage under different conditions used in previous studies, as the slope estimate was more biased to the transitional faces which were more likely to be influenced than the original faces. For example, in Experiment 1a, the slope of the psychometric function was t₍₁₄₎ = 3.066, P = 0.008, Cohen’s d = 0.792, BF₁₀ = 6.447, while the mean response percentage compared to chance was t₍₁₄₎ = 1.812, P = 0.091, Cohen’s d = 0.468, BF₁₀ = 0.973. We also adopted a procedure that allowed participants to make judgments of a single test face relative to two memorized categorically different faces [22, 29], which avoided the direct comparison of two faces. The results from the exploratory control experiment indeed validated the point about the sensitivity of testing conditions. The holistic face effect was more salient for faces in the middle of the morphing transitions than the faces at the two ends.

Integration Across the Visibility Boundary

A key question in consciousness research is the capacity of unconscious processing [33, 34]. Is the information unconsciously obtained fragmented [35, 36], or is it processed as a whole [37]? What is the extent of holistic processing [38]? There are also questions about the robustness and interpretation of results that seem to support the holistic processing of high-level information without consciousness [39–41]. Our experiments provide new evidence for unconscious processing, and in particular, the integration of information across the consciousness boundary.

The Integrative Interaction Between Visible and Invisible Components is not Limited to Faces

Faces are the most representative category of visual stimuli that show holistic processing [17, 18]. However, most people also have extensive expertise in processing written words. In Experiment 2 we tested both faces and Chinese characters to investigate whether integrative interaction between visible and invisible components occurs beyond face stimuli, in this case, with words. The processing of words has been shown to exhibit certain aspects of holistic processing, such as the word superiority effect in English [42] and the composite effect of Chinese characters [24]. Our results extend the existing literature on the holistic processing of words by showing that the integration of word components occurs across the consciousness boundary. Since holistic processing is not an “all-or-none” property, our finding of unconscious global integration is likely generalizable to other types of visual object, but to different degrees.

The Inversion Effect and the Significance of Eyes

The results from Experiment 2 show a facilitation effect from visible to invisible components of faces and words. Interestingly, visible eyes facilitated the processing of an invisible mouth even when the faces were inverted. This is consistent with the results obtained previously showing that the face composite illusion remains stable at 180° for a visible face [43]. Some earlier studies have investigated the processing of higher-level information from invisible faces, such as attractiveness [44, 45], with results suggesting that inversion eliminates the attractiveness effect. In typical demonstrations of the holistic processing of faces, inverting the faces greatly reduces but does not completely eliminate the holistic effect. The current results provide new insights that some aspects of face integration survive the inversion, even when part of the face is invisible, provided that the the eyes are visible .

More generally, the results of Experiment 2 reflected the processing asymmetry between the upper and lower parts of both faces and Chinese characters. There was strong facilitation from the visible upper components to the invisible lower components, but there was little evidence for facilitation from the visible lower components to the invisible upper components. The upper-to-lower facilitation may reflect the typical order of attention distribution when viewing faces and Chinese characters. More specifically, the asymmetry effect on faces is likely related to the more prominent role of the eyes in face perception. For example, eye fixation density information shows that when we look at a face, more fixations occur around the eye region [46, 47]. In addition, the eyes likely provide more information about the individual face [48, 49]. The upper-lower asymmetry of Chinese characters might be related to the top-down sequence of how Chinese characters are written [50], which makes it more likely and easier for observers to generate predictions about the lower components given the visible upper components.

Conclusion

The results of the current study demonstrated a modified “composite face effect”. When only the top half of the face was visible, the invisible bottom half-face contributed to face recognition. In addition, the visible half-face facilitated the conscious accessing of the invisible half-face so that the invisible parts emerged into awareness more quickly. This effect was especially robust from visible eyes to invisible mouth, suggesting strong predictive processing from eyes to mouth. The facilitation effect from visible to invisible components was also found with Chinese characters, especially from visible upper to invisible lower parts, indicating a direction of prediction. Together, our results clearly show interactions between visible and invisible face parts as well as between word parts. We conclude that the integrative processing of faces and words occurs across the consciousness boundary.

Conflict of interest

The authors claim that there are no conflicts of interest.