Significance
Figure–ground segregation, segmenting images by the visual system into figures and background, is the first processing step on the way to visual perception and relies on three complementary processes: boundary detection, region filling, and background suppression. By combining psychophysics, functional magnetic resonance imaging, and effective connectivity analysis, we showed that awareness was important for both region filling and background suppression, but less so for boundary detection. More importantly, we revealed that the awareness-dependent region-filling and background-suppression processes in V1 were derived by feedback from FEF and DLPFC, respectively. Our study indicates an awareness-dependent figure–ground segregation and identifies the human prefrontal cortex as a source of this fundamental process.
Keywords: boundary detection, region filling, background suppression, awareness, prefrontal cortex
Abstract
Figure–ground modulation, i.e., the enhancement of neuronal responses evoked by the figure relative to the background, has three complementary components: edge modulation (boundary detection), center modulation (region filling), and background modulation (background suppression). However, the neuronal mechanisms mediating these three modulations and how they depend on awareness remain unclear. For each modulation, we compared both the cueing effect produced in a Posner paradigm and fMRI blood oxygen-level dependent (BOLD) signal in primary visual cortex (V1) evoked by visible relative to invisible orientation-defined figures. We found that edge modulation was independent of awareness, whereas both center and background modulations were strongly modulated by awareness, with greater modulations in the visible than the invisible condition. Effective-connectivity analysis further showed that the awareness-dependent region-filling and background-suppression processes in V1 were not derived through intracortical interactions within V1, but rather by feedback from the frontal eye field (FEF) and dorsolateral prefrontal cortex (DLPFC), respectively. These results indicate a source for an awareness-dependent figure–ground segregation in human prefrontal cortex.
Figure–ground segregation is a fundamental process by which the visual system segments images into figures and background (1, 2). Previous neurophysiological and brain imaging studies of figure–ground segregation have shown neuronal responses are enhanced in the region perceived to be the figure and suppressed in the region perceived to be the background, an effect known as figure–ground modulation (1, 3–9). Figure–ground modulation plays a key role in identifying and localizing visual objects (7) and capturing focused attention (10). Remarkably, evidence from numerous neurophysiological (5, 11–16), psychophysical (17, 18), and brain imaging (19, 20) studies, as well as computational models (2, 17, 21) have suggested that figure–ground modulation relies on three complementary processes: boundary detection (i.e., edge modulation), region filling (i.e., center modulation), and background suppression (i.e., background modulation). During figure–ground segregation, boundary detection is the process that detects feature discontinuities that signal boundaries between the figures and background, while region filling is the process that groups figural regions with the same (or similar) features together (17), and the background suppression is the process that inhibits homogeneous features in the background (5, 22, 23). However, little is known how these three processes depend on awareness.
A number of previous studies have supported an early feedforward processing phase for the boundary-detection and later feedback-processing phases for both region filling and background suppression in figure–ground modulation (2, 5, 21). These findings thus suggest that boundary detection is independent of awareness, whereas both region filling and background suppression are strongly modulated by awareness (12). Specifically, boundary detection is thought to be achieved through iso-feature inhibition (2, 21, 24, 25) within early visual areas, as early as the primary visual cortex (V1) (9, 14, 26–30), in which neurons preferring the same or similar features are more likely to suppress each other via lateral connections (31). The region-filling process, however, requires iso-feature excitation in which neurons representing the similar features enhance each other’s activity. In contrast to several studies suggesting the existence of the region-filling process within V1 (1, 9), most previous studies indicate that the region-filling process arises from feedback projections to V1 from a higher cortical area(s) (11, 12, 14, 15, 20, 32). Similarly, several neurophysiological studies have suggested that the background-suppression process may also be derived by feedback to V1 from a higher cortical area(s); it is the later processing phase in which neural activity elicited by the background is suppressed by the preceding segregated figures (5, 22, 23). However, it remains unknown which and how the top-down feedback from a higher cortical area(s) drive the region-filling and background-suppression processes in V1 that enhances the response of neurons tuned to the same feature and suppresses the neural activity elicited by the background, respectively. Also, it is unknown whether and how these feedback processes interact with awareness.
Furthermore, another unclear but related issue is how boundary detection and region filling in figure–ground segregation attract focused attention. In fact, it is well known that successful segregation of a figure, as defined by an orientation contrast (Fig. 1A) from the background, leads to pop-out, which automatically attracts bottom-up attention to this salient figure location (10, 21, 25). However, it is not known whether there are different neural mechanisms by which the boundary detection and region filling attract our focused attention and whether the attentional attraction triggered by these two processes interacts with awareness.
Fig. 1.
Stimuli, psychophysical protocol, and data. (A) Two sample orientation-defined figures presented in the upper visual field (Left: large figure; Right: small figure). The orientation contrasts between the figure bars and the background bars was 60° (the yellow dot indicates the fixation point). (B) Large (Left) and small (Right) grating probes, with the same diameter as the large and small figures, respectively. (C) Low- (Left) and high- (Right) luminance mask stimuli used in the visible and invisible conditions, respectively. (D) Psychophysical protocol. A figure–ground stimulus was presented for 50 ms, followed by a 100-ms mask and another 50-ms fixation interval. Then a large or small grating probe, with the same diameter as the large figure and small figure, respectively, was randomly presented for 50 ms with equal probability and presented randomly at either the figure location (valid cue condition) or its contralateral counterpart (invalid cue condition) with equal probability. The grating probe was orientated at 45° or 135° away from the vertical. Subjects were asked to press one of two buttons as rapidly and correctly as possible to indicate the orientation of the grating probe (45° or 135°). The psychophysical cueing effect for the large (E) and small (F) figures and the large and small gratings in both visible and invisible conditions. Each cueing effect was quantified as the difference between the reaction time of the probe task performance in the invalid cue condition and that in the valid cue condition. Error bars denote 1 SEM calculated across subjects and colored dots denote the data from each subject.
To address these questions, we used a modified version of the Posner paradigm (33, 34) to measure the spatial cueing effect induced by figure boundary (edge modulation) or figure center (center modulation) of the large figure (Fig. 1 A, Left), and by the whole small figure or its surround background (background modulation, Fig. 1 A, Right). Blood oxygen level-dependent (BOLD) signals evoked by the figure boundary and figure center of the large figure, as well as the whole small figure and its surround background were also measured. Using a backward masking paradigm (10) with low- or high-luminance masks to render the whole figure–ground stimulus visible or invisible (confirmed by a two-alternative forced choice [2AFC]) to subjects, respectively, we examined how figure–ground segregation interacts with awareness. We also performed interregional correlation and effective connectivity analyses to examine the neural mechanisms of this potential interaction.
Results
Psychophysical Experiments.
In psychophysical experiments, there were two possible figures: the first consisted of 10 × 10 bars (large figure, Fig. 1 A, Left) and the second consisted of 2 × 2 bars (small figure, Fig. 1 A, Right). For both of the two figures, there were two possible probes: a large grating or a small grating (Fig. 1B), which had the same diameter as the large figure and small figure, respectively. In both visible and invisible conditions (confirmed by the 2AFC test, SI Appendix, Fig. S1), using a modified version of the Posner paradigm (33, 34), we measured the spatial cueing effect induced by the figure boundary or figure center of the large figure, and by the whole small figure or its surround background, as shown in Fig. 1D. A valid cue condition was defined as a match of quadrant between the figure and probe (for example, both the figure and probe were presented in the Upper Left quadrant); an invalid cue condition was defined as a mismatch (Fig. 1D). Subjects were asked to press one of two buttons as rapidly and correctly as possible to indicate the orientation of the grating probe. There was no significant difference in false alarm rate, miss rate, or removal rate (i.e., correct reaction times shorter than 200 ms and beyond three SDs from the mean reaction time in each condition were removed) across conditions (all P > 0.05, partial eta-squared, ηp2 < 0.097, SI Appendix, Fig. S2). The cueing effect for each figure and each grating was quantified as the difference between the reaction time of the probe task performance in the invalid cue condition and that in the valid cue condition.
Behaviorally, in our study, for the large figure, the cueing effect induced by its figure boundary (edge modulation) or figure center (center modulation) was measured by the large grating and small grating, respectively. Fig. 1E shows the cueing effect for each condition, and all these cueing effects were significantly above zero [all t(26) > 5.026, P < 0.001, ηp2 > 0.493]. These results demonstrate that, despite the visibility (visible or invisible) of the figures, both the figure boundary and figure center could attract subjects’ attention to their locations, allowing them to perform more proficiently in the valid than the invalid cue condition of the probe task. To test our hypothesis that the region filling (center modulation) but not the boundary detection (edge modulation) is modulated by awareness, we further submitted the cueing effect to a repeated measures ANOVA with awareness (visible and invisible) and probes (large grating and small grating) as within-subjects factors. The main effect of the probe was not significant [F(1, 26) = 0.185, P = 0.671, ηp2 = 0.007], but the main effect of awareness [F(1, 26) = 14.658, P = 0.001, ηp2 = 0.361] and the interaction between these two factors [F(1, 26) = 7.541, P = 0.011, ηp2 = 0.225] were both significant. Post hoc paired t tests showed that the cueing effect on large grating was greater than that on small grating in the invisible condition [t(26) = 2.495, P = 0.019, ηp2 = 0.193], but not in the visible condition [t(26) = −1.750, P = 0.092, ηp2 = 0.105]; the cueing effect of the visible condition was greater than that of the invisible condition on small grating [t(26) = 4.771, P < 0.001, ηp2 = 0.467], but not on large grating [t(26) = 0.317, P = 0.754, ηp2 = 0.004]. These results are consistent with our hypothesis that awareness modulates the region filling, as measured by small grating, rather than the boundary detection, as measured by large grating.
For the small figure, the cueing effect induced by the whole figure or its surround background was measured by the small grating and large grating, respectively. Fig. 1F shows the cueing effect for each condition; all these cueing effects were significantly above zero [all t(26) > 6.563, P < 0.001, ηp2 > 0.624], except for the visible large grating condition [t(26) = 1.378, P = 0.180, ηp2 = 0.068]. A further repeated measures ANOVA with awareness (visible and invisible) and probes (large grating and small grating) as within-subjects factors showed that, the main effect of probe [F(1, 26) = 33.577, P < 0.001, ηp2 = 0.564], the main effect of awareness [F(1, 26) = 29.730, P < 0.001, ηp2 = 0.533], and the interaction between these two factors [F(1, 26) = 4.415, P = 0.045, ηp2 = 1.145] were all significant. Post hoc paired t tests showed that the cueing effect on small grating was greater than that on large grating in both the visible [t(26) = 6.568, P < 0.001, ηp2 = 0.624] and invisible [t(26) = 2.117, P = 0.044, ηp2 = 0.147] conditions, indicating a classical figure enhancement effect relative to the background. Intriguingly, the cueing effect in the visible condition was greater than that in the invisible condition on small grating [t(26) = 4.907, P < 0.001, ηp2 = 0.481], but was less than that in the invisible condition on large grating [t(26) = −3.464, P = 0.002, ηp2 = 0.316]. These results are consistent with our hypothesis that awareness can modulate the cueing effect induced by the figure’s surround background, as measured by the large grating.
Note that using a modified Posner paradigm, the spatial cue, such as the large and small figures in our study, could attract attention to the entire cued quadrant and induce a general Posner cueing effect. However, a number of previous studies have indicated that this attraction is not homogeneous in the cued quadrant. Instead, it depends on the spatial match between the cue and probe, i.e., the spatially specific cueing effect. Thus, to dissociate the cueing effect induced by the figure boundary, figure center, whole small figure, and its surround background, we manipulated the spatial match between the figure and grating probe (for example, for the large figure, the large and small grating probes, having a differently spatial match, were used to measure the cueing effect of figure boundary and figure center, respectively). Remarkably, our results showed the significant spatially specific cueing effect, which was further confirmed by a supplemental psychophysical experiment with the frame as the cue (SI Appendix, Fig. S4). In addition, it should be noted that, although we manipulated the spatial match between the figure and grating probe, we cannot deny the contribution from the general Posner cueing effect and thus what we focus on here are differences in the degree of cueing effect during differently spatial match conditions. Indeed, the general Posner cueing effect in our study was well demonstrated by a positive (rather than negative) cueing effect of small figure’s surround background (Fig. 1F). Compared to the invalid cue condition, although the large grating was presented away from the location of the small figure, they were still in the same cued quadrant in the valid cue condition and thus allowed subjects to perform more proficiently in the valid than the invalid cue condition of the large grating.
fMRI Experiments.
Using a block design, we measured BOLD signals evoked by the figure boundary and figure center of the large figure, as well as the whole small figure and its surround background. There were 10 different stimulus blocks, including 8 different figure blocks: 2 (figure: large/small) × 2 (visual field: left/right) × 2 (awareness: visible/invisible), and 2 mask-only blocks: low- and high-luminance masks. Each stimulus block was randomly presented once in each run, and consisted of the same five trials. On each trial in the figure and mask-only blocks, a figure–ground stimulus and the fixation were presented for 50 ms, respectively, followed by a 100-ms mask (low and high luminance for visible and invisible conditions, respectively) and 1,850-ms fixation (Fig. 2D). In the invisible condition, on each trial during both the figure and mask-only blocks, subjects were asked to press one of two buttons to indicate the location of the figure, which was left of fixation in one half of blocks and right of fixation in the other half at random (i.e., the 2AFC task). In the visible condition, on each trial during the figure block, subjects needed to perform the same 2AFC task of the figure, whereas during the mask-only block, subjects were asked to press one of two buttons randomly. Behavioral data showed that our awareness manipulation was effective for both visible and invisible conditions. In addition, there was no significant difference in subject performance between large figure and small figure (SI Appendix, Fig. S1C).
Fig. 2.
fMRI stimuli and protocol. (A) ROI definition. The checkered patch on the Left was used to define ROIs corresponding to the figure center of the large figure and the whole small figure; the checkered patch on the Right was used to define ROIs corresponding to the figure boundary of the large figure and the small figure’s background. (B) The transparent squares show the size and location of the checkered patches relative to the large figure (Top) and small figure (Bottom). The red squares (Left) indicate the figure center of the large figure and the whole small figure; the blue squares (Right) indicate the figure boundary of the large figure and the small figure’s background. (C) ROIs on an inflated cortical surface of a representative subject. The ROIs in V1 were defined as the cortical regions responding to the figure center (red region, V1 center) and figure boundary (blue region, V1 boundary) of the large figure. These cortical regions in V1 were also used as the ROIs corresponding to the whole small figure (red region, V1 figure) and its surround background (blue region, V1 background). Both V1 boundary of the large figure and V1 background of the small figure contained two separate regions with different eccentricities: the parafovea and periphery, indicated by the dashed and solid rings, respectively. The ROIs in LGN and V2 to V4 were defined as the regions responding to the whole figure since activated areas in these areas showed a great deal of overlap. The boundaries among V1 to V4, defined by retinotopic mapping, are indicated by the white lines. (D) Block design fMRI procedure. On each trial in the figure and mask-only blocks, a figure–ground stimulus and the fixation were presented for 50 ms, respectively, followed by a 100-ms mask (low- and high-luminance for visible and invisible conditions, respectively) and 1,850-ms fixation interval. In the Invisible condition, on each trial during both the figure and mask-only blocks, subjects were asked to press one of two buttons to indicate the location of the figure, which was Left of fixation in one half of blocks and Right of fixation in the other half at random (i.e., the 2AFC task). In the visible condition, on each trial during the figure block, subjects needed to perform the same 2AFC task of the figure, whereas during the mask-only block, subjects were asked to press one of two buttons randomly.
Regions of Interest Analyses in Striate Cortex.
Contralateral and ipsilateral regions of interest (ROIs) in V1 were defined as those responding to the figure center (i.e., V1 center) and figure boundary (i.e., V1 boundary) of the large figure (Fig. 2 B, Top) and their contralateral counterparts (those always corresponded to the background bars). Note that these cortical regions in V1 were also used as the ROIs corresponding to the whole small figure (i.e., V1 figure) and its surround background (i.e., V1 background), respectively (Fig. 2 B, Bottom). BOLD signals were extracted from these ROIs and then averaged according to the figure (large figure and small figure) and awareness (visible and invisible). For each block, the 2 s preceding the block served as a baseline, and the mean BOLD signal from 5 s to 10 s after stimulus onset was used as a measure of the response amplitude (34). For each figure and each ROI, the BOLD signal difference was quantified as the difference between the BOLD amplitude at the contralateral ROI and at the ipsilateral ROI. In both visible and invisible conditions, to examine whether there was a significant figure–ground modulation in V1, the BOLD signal differences of V1 boundary and V1 figure for the large figure and small figure, respectively, were compared with zero. Results showed that the BOLD signal differences of V1 boundary [visible condition: t(17) = 3.530, P = 0.003, ηp2 = 0.423; invisible condition: t(17) = 2.479, P = 0.024, ηp2 = 0.266] and V1 figure [visible condition: t(17) = 3.712, P = 0.002, ηp2 = 0.448; invisible condition: t(17) = 2.315, P = 0.033, ηp2 = 0.240] were significantly above zero, indicating a significant figure–ground modulation in V1 for both the large and small figures. In other words, neuronal responses evoked by both the large and small figures, despite awareness, were enhanced relative to the background (Fig. 3). Subsequently, for both large and small figures, these BOLD signal differences were submitted to a repeated-measures ANOVA with awareness (visible and invisible) and cortical area (V1 boundary and V1 center for the large figure; V1 background and V1 figure for the small figure) as within-subjects factors.
Fig. 3.
fMRI results. (A, Left) Blocked BOLD signals averaged across subjects in the ipsilateral and contralateral ROIs in V1 center, V1 boundary, LGN, and V2 to V4 for the large figure, during the visible and invisible conditions. Error bars denote 1 SEM calculated across subjects at each time point. (A, Right) BOLD amplitude differences between the blocked BOLD signals at the contralateral ROIs and those at the ipsilateral ROIs in V1 center, V1 boundary, LGN, and V2 to V4 during the visible and invisible conditions. Error bars denote 1 SEM calculated across subjects and colored dots denote the data from each subject. (B, Left) Blocked BOLD signals averaged across subjects in the ipsilateral and contralateral ROIs in V1 figure, V1 background, LGN, and V2 to V4 for the small figure, during the visible and invisible conditions. Error bars denote 1 SEM calculated across subjects at each time point. (B, Right) BOLD amplitude differences between the blocked BOLD signals at the contralateral ROIs and those at the ipsilateral ROIs in V1 figure, V1 background, LGN, and V2 to V4 during the visible and invisible conditions. Error bars denote 1 SEM calculated across subjects and colored dots denote the data from each subject.
For the large figure (Fig. 3A), the main effect of cortical area was not significant [F(1, 17) = 0.810, P = 0.381, ηp2 = 0.045], but the main effect of awareness [F(1, 17) = 6.049, P = 0.025, ηp2 = 0.262] and the interaction between these two factors [F(1, 17) = 6.255, P = 0.023, ηp2 = 0.269] were both significant. Post hoc paired t tests showed that the BOLD signal difference of V1 boundary was greater than that of V1 center in the invisible condition [t(17) = 2.269, P = 0.037, ηp2 = 0.232], but not in the visible condition [t(17) = −1.558, P = 0.138, ηp2 = 0.125]; the BOLD signal difference in the visible condition was greater than that in the invisible condition for V1 center [t(17) = 3.495, P = 0.003, ηp2 = 0.418], but not for V1 boundary [t(17) = 0.470, P = 0.644, ηp2 = 0.013]. These findings reveal that the figure center but not figure boundary is modulated by awareness, paralleling to the cueing effect in the psychophysical experiments (Fig. 1E). Furthermore, across the 18 subjects who participated in both the psychophysical and fMRI experiments, compared to the invisible condition, the enhanced V1-center response correlated significantly with the enhanced cueing effect of small grating in the visible condition (r = 0.530, P = 0.024, ηp2 = 0.281, SI Appendix, Fig. S5A). These results further indicate a close relationship between the cueing effect and V1 activity.
For the small figure (Fig. 3B), the main effect of awareness was not significant [F(1, 17) = 0.701, P = 0.414, ηp2 = 0.040], but the main effect of the cortical area [F(1, 17) = 29.162, P < 0.001, ηp2 = 0.632] and the interaction between these two factors [F(1, 17) = 7.5862, P = 0.014, ηp2 = 0.309] were both significant. Post hoc paired t tests showed that the BOLD signal difference of V1 figure was greater than that of V1 background in the visible condition [t(17) = 5.965, P < 0.001, ηp2 = 0.677], but not in the invisible condition [t(17) = 1.684, P = 0.110, ηp2 = 0.143]; the BOLD signal difference in the invisible condition was greater than that in the visible condition for V1 background [t(17) = 2.810, P = 0.012, ηp2 = 0.317], but not for V1 figure [t(17) = −0.716, P = 0.484, ηp2 = 0.029]. These results indicate that the background suppression in figure–ground segregation is modulated by awareness, like the cueing effect in the psychophysical experiments (Fig. 1F). A similar correlation analysis further indicates a close relationship between the cueing effect and V1 activity by showing that, compared to the invisible condition, the decreased V1-background response correlated significantly with the decreased cueing effect of large grating in the visible condition (r = 0.607, P = 0.008, ηp2 = 0.368, SI Appendix, Fig. S5B).
In addition, both V1 boundary of the large figure and V1 background of the small figure contained two separate regions with different eccentricities: the parafovea and periphery (Fig. 2C). It is unclear whether boundary-detection and background-suppression processes depend on different eccentricities. To examine this issue, for both V1 boundary and V1 background (SI Appendix, Figs. S6 and S7), we analyzed the BOLD amplitudes of their parafovea and periphery separately. The results provide the same qualitative conclusion, further confirming that, despite eccentricity, boundary detection was independent of awareness, whereas background suppression was strongly modulated by awareness.
ROI Analyses in Lateral Geniculate Nucleus and Extrastriate Cortex.
Contralateral and ipsilateral ROIs in lateral geniculate nucleus (LGN) and V2 to V4 were defined as those cortical areas that responded to the retinal inputs in the whole figure region and their contralateral counterparts, as the activated regions in these areas corresponding to the figure center and figure boundary of the large figure and the small figure and its surround background showed a great deal of overlap (Fig. 2C). For each figure and each ROI, the differences between the BOLD amplitudes at the contralateral ROIs and those at the ipsilateral ROIs are shown in Fig. 3. Similar to V1, in both visible and invisible conditions, there was a significant figure–ground modulation for both the large figure [visible condition: all t(17) > 3.748, P < 0.002, ηp2 > 0.452; invisible condition: all t(17) > 2.044, P < 0.057, ηp2 > 0.197] and small figure [visible condition: all t(17) > 3.216, P < 0.005, ηp2 > 0.378; invisible condition: all t(17) > 2.024, P < 0.059, ηp2 > 0.194] in V2 to V4, but not in LGN (all P > 0.05). A further repeated-measures ANOVA with awareness (visible and invisible) and brain area (LGN and V2 to V4) as within-subjects factors showed that, the main effect of brain area [large figure: F(4, 17) = 2.438, P = 0.078, ηp2 = 0.125; small figure: F(4, 17) = 1.552, P = 0.221, ηp2 = 0.084] and the interaction between these two factors [large figure: F(4, 68) = 0.089, P = 0.971, ηp2 = 0.005; small figure: F(4, 68) = 0.414, P = 0.708, ηp2 = 0.024] were not significant, but the main effect of awareness was significant [large figure: F(1, 17) = 19.933, P < 0.001, ηp2 = 0.540; small figure: F(1, 17) = 8.142, P = 0.011, ηp2 = 0.324], showing a greater BOLD signal difference in the visible condition than that in the invisible condition.
Whole-Brain and Correlation Analyses.
To examine a potential cortical or subcortical area(s) that showed a similar awareness-dependent figure–ground segregation in these retinotopically organized areas, we performed a group analysis and did a whole-brain search with a general linear model (GLM) procedure for a cortical and subcortical area(s) that showed a significantly greater response in the visible condition than that in the invisible condition, after subtracting their respective mask signals (data from the left and right visual fields were combined). Statistical maps were thresholded at P < 0.03 and corrected by false discovery rate (FDR) correction (35). The results showed that, for the large figure, the the right intraparietal sulcus (rIPS), bilateral insula, right frontal eye field (rFEF), right middle frontal gyrus (rMFG), and right dorsolateral prefrontal cortex (rDLPFC) demonstrated a greater response in the visible condition than that in the invisible condition (Fig. 4A). Importantly, the same cortical areas were found for the small figure (Fig. 4B).
Fig. 4.
Results of whole-brain and correlation analyses. Whole-brain search for DLPFC {large figure: [43, 9, 29], t(17) = 4.441, P < 0.001, ηp2 = 0.537; small figure: [39, 23, 23], t(17) = 5.713, P < 0.001, ηp2 = 0.658}, MFG {large figure: [34, 45, 10], t(17) = 5.742, P < 0.001, ηp2 = 0.660; small figure: [30, 43, 13], t(17) = 4.653, P < 0.001, ηp2 = 0.560}, FEF {large figure: [32, −6, 47], t(17) = 4.179, P = 0.001, ηp2 = 0.507; small figure: [40, 3, 29], t(17) = 4.885, P < 0.001, ηp2 = 0.584}, insula {large figure: [Left: −34, 17, 1] and [Right: 28, 18, 4], t(17) = 4.999, P < 0.001, ηp2 = 0.595; small figure: Left: −36, 14, 5] and [Right: 27, 17, 5], t(17) = 5.176, P < 0.001, ηp2 = 0.612}, and IPS {large figure: [26, −59, 41], t(17) = 4.470, P < 0.001, ηp2 = 0.540; small figure: [26, −62, 33], t(17) = 3.434, P = 0.003, ηp2 = 0.410}, with all showing a significantly greater response in the visible than the invisible condition for the large figure (A) and small figure (B). Error bars denote 1 SEM calculated across subjects and colored dots denote the data from each subject. (C) Correlations between DBOLD in V1 center of the large figure and that in V4 (Left) and FEF (Right) across individual subjects. (D) Correlation coefficients (r values) between DBOLD in V1 center of the large figure and that in other cortical/subcortical areas across individual subjects. (E) Correlations between DBOLD in V1 background of the small figure and that in IPS (Left) and DLPFC (Right) across individual subjects. (F) Correlation coefficients (r values) between DBOLD in V1 background of the small figure and that in other cortical/subcortical areas across individual subjects.
Together, our results showed that awareness could modulate the BOLD signal in V1 center of the large figure and in surround background of the small figure, as well as in LGN, extrastriate visual areas, and several frontoparietal areas. Thus, for each area, we computed a difference (DBOLD) to quantify how much the BOLD signal showed in the visible condition relative to that in the invisible condition. The difference was calculated as follows: DBOLD = VBOLD–IBOLD, where VBOLD and IBOLD are the BOLD signals in the visible and invisible conditions, respectively. Subsequently, for the large figure, to examine which area(s) modulated the awareness-dependent region-filling process in V1, we calculated the correlation coefficients between DBOLD in V1 center and that in other areas across individual subjects. The DBOLD in V1 center correlated significantly with the DBOLD in V4 (r = 0.548, P = 0.019, ηp2 = 0.300, Fig. 4 C, Left) and FEF (r = 0.744, P < 0.001, ηp2 = 0.554, Fig. 4 C, Right), but not in LGN, V1 boundary, V2, V3, V3A/B, IPS, insula, MFG, or DLPFC (Fig. 4D). For the small figure, to examine which area(s) modulated the awareness-dependent background-suppression process in V1, we calculated the correlation coefficients between DBOLD in V1 background and that in other areas across individual subjects. The DBOLD in V1 background correlated significantly with the DBOLD in IPS (r = −0.564, P = 0.015, ηp2 = 0.318, Fig. 4 E, Left) and DLPFC (r = −0.490, P = 0.039, ηp2 = 0.240, Fig. 4 E, Right), but not in LGN, V1 figure, V2, V3, V3A/B, V4, insula, FEF, or MFG (Fig. 4F). These results suggest that the awareness-dependent region-filling process in V1 may derive from feedback projections from V4 and/or FEF, whereas the awareness-dependent background-suppression process in V1 may derive from feedback projections from IPS and/or DLPFC.
Effective Connectivity Analyses.
To further examine which area is the source of the awareness-dependent region-filling and background-suppression processes in V1, we applied dynamic causal modeling (DCM) analysis (36) in SPM12 to examine functional changes in interregional connectivity related to the visible condition. For the region-filling process, effective connectivities among the FEF, V4, V1 boundary, and V1 center were analyzed (Fig. 5A). While for the background-suppression process, effective connectivities among the DLPFC, IPS, V1 figure, and V1 background were analyzed (Fig. 5D). For both of the two processes, we defined 11 different models for modeling the modulatory effect of the visible condition and fit each of their 11 models for each subject (SI Appendix).
Fig. 5.
DCM results for region-filling and background-suppression processes. (A) Eleven different models among FEF, V4, V1 center, and V1 boundary used to model the modulatory effect of the visible condition for region-filling process. The colored lines and arrows illustrate potential feedback modulations to V1 center. V1 center and V1 boudnary: ROI in V1 evoked by the figure center and figure boundary of the large figure, respectively. (D) Eleven different models among DLPFC, IPS, V1 figure, and V1 background used to model the modulatory effect of the visible condition for the background-suppression process. The colored lines and arrows illustrate potential feedback modulations to V1 background. V1 figure and V1 background: ROI in V1 evoked by the whole small figure and its surround background, respectively. Exceedance probabilities of the 11 models with the visible condition as the modulatory input for region-filling (B) and background-suppression (E) processes. The strength of the modulatory connections for the visible condition and its significance levels for region-filling (C) and background-suppression (F) processes (*P < 0.05).
Using a hierarchical Bayesian approach (37), we compared the 11 models by computing the exceedance probability of each model, i.e., the probability to which a given model is more likely than any other included model to have generated data from a randomly selected subject. The result showed that models 9 and 4 were the best ones to explain the modulatory effect by the visible condition for the region-filling (Fig. 5B) and background-suppression (Fig. 5E) processes, respectively. In the visible condition, the feedback connectivity was significantly increased from FEF to V1 center [t(17) = 2.738, P = 0.014, ηp2 = 0.306, Fig. 5C] for the region-filling process, but the feedback connectivity was decreased from DLPFC to V1 background [t(17) = −2.531, P = 0.022, ηp2 = 0.274, Fig. 5F] for the background-suppression process. No significant interregional feedback was found between other cortical areas in either model 9 or 4. Additionally, a supplemental DCM analysis with all frontoparietal areas modulated by awareness further confirms these results by showing that the awareness-dependent region-filling and background-suppression processes in V1 were derived by feedback from FEF and DLPFC, respectively (SI Appendix, Fig. S8).
Discussion
We examined how figure–ground segregation interacts with awareness and found the following psychophysical and neuroimaging results. First, we found support for previous neurophysiological (5, 11–16), psychophysical (17, 18), and brain imaging (19, 20) studies, as well as computational models (2, 17, 21) indicating the existence of three complementary processes in figure–ground segregation: boundary detection, region filling, and background suppression. Second, we found that both boundary detection and region filling attracted bottom-up attention in behavior. However, there was a critical distinction between the dependence on awareness of these two processes, with boundary detection independent of awareness and region filling strongly modulated by awareness. Third, we found that the segregated figure’s background could be suppressed and this suppression was also strongly modulated by awareness. Finally, the most parsimonious account of our results is that the awareness-dependent region-filling and background-suppression processes in V1 were not derived through intracortical interactions within V1, but rather by feedback from FEF and DLPFC, respectively. In addition, these results cannot be explained by the eye movement, head motion, or task difficulty, as in all these factors, there were no significant differences among our conditions (SI Appendix, Figs. S1, S3, and S5).
Awareness-Independent Boundary Detection.
Numerous neurophysiological (9, 14, 26–30) and brain imaging (10) studies, as well as computational models (2, 17, 21, 24, 25) have suggested that boundary detection is an early feedforward processing phase that is achieved through iso-feature inhibition within early visual areas, in which neurons preferring the same or similar features are more likely to suppress each other via lateral connections (31). For example, V1 neurons preferring the same or similar orientations are more likely to suppress each other. This iso-orientation suppression reduces V1 neural responses to a homogeneous background. At the same time, V1 neurons preferring, and thus responding to, boundaries escape this iso-orientation suppression. Our study confirms this early feedforward phase for boundary detection by showing that boundary detection was independent of awareness. The figure boundary, despite its visibility, can attract bottom-up attention in behavior. These findings not only replicate our previous studies (10) indicating an invisible bottom-up saliency map in human V1, but also support other studies reporting that boundary detection occurs quickly (28) and does not depend on whether the animal is awake (27) or under anesthesia (26, 30), and regardless of task relevance (38), the eye of origin (4, 39), or feedback from extrastriate visual areas (40).
We believe that the awareness-independent boundary detection in V1 does not reflect the assignment of boundary to the figure rather than the background, namely the border ownership (8). Previous studies have found that border ownership selectivity appears to be primarily in V2 (6, 8, 41) and is largely dependent on attention (41, 42), indicating that border ownership processing depends on feedback from higher-level cortical areas. Boundary detection, however, is an early feedforward processing as it is independent of attention (14) and awareness, as shown here. More convincingly, Hupé et al. (40) demonstrated that the V1 boundary signal also occurs even if V2 is inactivated by GABA injections.
Awareness-Dependent Region Filling in the FEF.
Compared to boundary detection, which is thought to be achieved through iso-feature inhibition, previous computational models (2, 17, 21) have proposed that the region-filling process requires iso-feature excitation, such that neurons representing the same (similar) features enhance each other’s activity, and these iso-feature excitations may arise from feedback projections from higher cortical areas back to V1 (11, 12, 14, 15, 20, 32). Our results support this feedback mechanism, as the region-filling process in V1 was strongly modulated by awareness (Figs. 1E and 3A), consistent with previous findings that the region-filling process induced neuronal responses primarily in superficial and deep cortical layers that receive feedback from higher visual areas (15), and strongly modulated by attention (4, 14, 43) and the size of the figure (44), and could be absent when a monkey is anesthetized (12) or failed to detect the figure (45). All of those studies indicate that the region-filling process in V1 may be controlled by feedback from higher areas. However, it is unknown which and how the top-down feedback from higher cortical area(s) drive the region-filling process in V1 that enhances the response of neurons tuned to the same feature. Remarkably, our current study provides strong and converging evidence that the awareness-dependent region filling in V1 was closely associated with feedback from the FEF. First, FEF responses were significantly modulated by awareness (Fig. 4A) and showed a pattern of activation consistent with that in V1 center (i.e., the ROI in V1 corresponding to the region-filling process, Fig. 3A). Second, V1-center’s responses were significantly predicted by FEF responses (Fig. 4C) rather than by responses in other frontoparietal cortical areas (Fig. 4D). Third, the DCM analysis indicated that the visible condition significantly increased feedback from the FEF to V1 center, and no significant interregional feedback was found in other cortical areas (Fig. 5C). Finally, a supplemental DCM analysis further confirmed a crucial involvement of FEF in the awareness-dependent region-filling process in V1 (SI Appendix, Fig. S8B).
Our study succeeded in linking awareness-dependent feedback from the FEF to V1 directly with the region-filling process in figure–ground segregation. Interestedly, previous neurophysiological (11, 14, 41), lesion (46, 47), and brain imaging (43, 48) studies have reported that the region-filling process in V1 is also associated with activations in extrastriate visual areas. Our study supported the involvement of extrastriate visual areas in the region-filling process in V1 by showing that V1 center’s responses were significantly predicted by V4 responses (Fig. 4C). Furthermore, our DCM analyses indicated that model 9 (FEF→V1 center and FEF→V4→V1 center) rather than model 1 (FEF→V1 center) was the best one to explain the region-filling process in V1, demonstrating that the feedback from V4 might also contribute to this process. However, more importantly, model 9 (FEF→V1 center and FEF→V4→V1 center) was better than model 4 (FEF→V1 center and V4→V1 center) to explain the region-filling process in V1 (Fig. 5B), indicating that the contribution of V4 in the region-filling process in V1 was also derived by feedback from FEF. Additionally, one should note that our study was unable to separate the figure center and figure boundary in V4 (Fig. 2C). Thus, further work is needed using neurophysiological techniques or ultra-high field fMRI to address whether the region filling and/or boundary detection in V4 is mediated by feedback from FEF, as well as to parse the relative contributions of these two processes in V4 to awareness-dependent modulation of region filling in V1.
Awareness-Dependent Background Suppression in the DLPFC.
In accordance with previous studies demonstrating a background-suppression process in figure–ground segregation (5, 22, 23), our study further revealed that this background suppression was largely modulated by awareness. Remarkably, our study also indicated that this awareness-dependent background suppression was best derived by feedback from DLPFC to V1, consistent with several neurophysiological studies showing that the background-suppression process is a later processing phase (22, 23), being strongest in superficial and deep layers and weaker in layer 4 (5).
One should note that the awareness-dependent modulation of the background-suppression process in our study was reflected by the decreased response to the background in the visible condition, relative to the invisible condition. The question is: how does awareness decrease V1 responses to the figure’s background? Previous neurophysiological (49, 50) and brain imaging (34, 51) studies have implicated prefrontal areas in the filtering of task-irrelevant distractors (52), and our findings are consistent with such an influence. Our findings suggest that the awareness-dependent background suppression in V1 is associated with feedback from DLPFC. First, DLPFC responses were significantly modulated by awareness (Fig. 4B) and showed a pattern of activation opposite to the pattern in V1 background (i.e., the ROI in V1 corresponding to the background-suppression process, Fig. 3B). Second, V1 background’s responses was significantly predicted by DLPFC responses (Fig. 4E). Third, the DCM analysis indicated that the visible condition significantly decreased feedback (i.e., increased suppression) from DLPFC to V1 background, and no significant interregional feedback was found in other frontoparietal cortical areas (Fig. 5F). Finally, a supplemental DCM analysis further confirmed a crucial involvement of DLPFC in the awareness-dependent background suppression in V1 (SI Appendix, Fig. S8D).
Although we emphasize the importance of DLPFC in awareness-dependent background suppression during figure–ground segregation, we cannot deny a potential contribution from parietal cortex, such as IPS. First, we found that V1 background’s responses were also predicted by IPS responses (Fig. 4E). Second, our DCM analyses indicated that, although the feedback modulation from IPS to V1 background was not significant (Fig. 5F), model 4 (DLPFC→V1 background and IPS→V1 background) rather than model 1 (DLPFC→V1 background) was the best one to explain the background-suppression process in V1, demonstrating that feedback modulation from IPS might also contribute to the background suppression (Fig. 5E). Further work is needed using neurophysiological techniques or ultra-high field fMRI to parse the relative contributions of prefrontal and parietal cortices to awareness-dependent modulation of background suppression.
Figure-Ground Segregation in the Prefrontal Cortex.
Our findings identified the human prefrontal cortex as a source of awareness-dependent figure–ground segregation. Note that this conclusion is based mainly on our DCM analyses, which depended on time-series models of fMRI data for an interpretation of causality (36). The interpretation of causality in our study finds support in previous lesion (53) and transcranial magnetic stimulation (54–56) studies showing a causal effect of prefrontal cortical disruption on sensory processing. Also, the prominent role of the prefrontal cortex in awareness-dependent figure–ground segregation evident here is consistent with recent neurophysiological findings that have revealed how prefrontal areas directly provide top-down modulation of sensory signals within posterior cortices (52, 57).
Most notably, however, our results provide neural evidence that two subregions of prefrontal cortex, namely the FEF and DLPFC, have distinct roles in figure–ground segregation. The FEF appears to directly send top-down biasing signals to enhance the response of neurons tuned to the same orientation (i.e., the iso-orientation excitation), resulting in the region-filling process; by contrast, the DLPFC appears to have a special role in suppressing the neural activity elicited by the background (i.e., background suppression) that may help boost the signal of a figure relative to its background (Fig. 6). This distinction, evident in our study, confirms previous studies that have implicated separate roles of these two prefrontal areas in top-down modulations of sensory cortex across various tasks (58), whereby the FEF has been shown to primarily enhance the neural response of task-relevant stimuli (52, 59) while the DLPFC has been shown to be involved in filtering out task-irrelevant distractors (50, 51).
Fig. 6.
Schematic illustration of the figure–ground segregation. First, features in the image, i.e., the local orientation of bars, are registered. Second, feature discontinuities that signal boundaries between the figure and background are detected (i.e., boundary-detection process) through local intracortical interactions within V1. Third, the neural response elicited by the center of the figure (i.e., the region-filling process) in V1 is enhanced through the increased feedback from FEF. The FEF directly sends top-down biasing signals to enhance the response of neurons tuned to the same orientation. Fourth, the neural response elicited by the preceding segregated figure’s background (i.e., the background-suppression process) in V1 is reduced through the decreased feedback (increased suppression) from DLPFC. The DLPFC directly filters out the task-irrelevant information. Finally, the neuronal response is enhanced in the region perceived to be the figure (dark gray region) and suppressed in the region perceived to be the background (light gray region), resulting that the visual system segments the image into the figure and background.
The involvement of prefrontal cortex in the awareness-dependent figure–ground segregation evident here also supports several theories of conscious awareness, including the neuronal global workspace (60), information integration (61, 62), and higher-order (63) theories, all of which have proposed that the neural activity in prefrontal (and parietal) cortex is essential for conscious awareness. Similar to our study, evidence from those theories typically used the report-based paradigm in which subjects overtly report their percept, such as speaking or pushing a button. The results of those studies showed that a broad frontoparietal network of areas could be activated during various tasks that contrast perceived stimuli with invisible stimuli (60). Several studies have argued that such report-based paradigms do not dissociate the brain regions required for pure conscious experience from those involved in conscious access and reportability (61, 64). Those studies found that posterior rather than prefrontal cortical areas were activated when using a no-report paradigm, such as recording eye movements and pupil dilation (65, 66). However, other studies have conversely reported that neurons in the prefrontal cortex selectively represent consciously perceived stimuli even when using no-report paradigm in which there is not any overt report (67). Furthermore, a recent study found that, in both rapid eye movement (REM) and non-REM sleep, the neural correlates of conscious experience appeared to be primarily in the posterior cortex; dreaming during REM sleep, however, was also associated with high-frequency activity in prefrontal cortex (68). In other words, the prefrontal cortex might also contribute to the conscious experience independent of the reportability. Further work is thus needed using both report-based and no-report paradigms to parse the crucial involvement of prefrontal cortical areas in awareness-dependent figure–ground segregation.
It is important to note that the attentional-engagement/expectation could account for the special role of prefrontal cortex in awareness-dependent figure–ground segregation as subjects may naturally pay more attention or expectation to stimuli with than without awareness, especially for our fMRI block design, in which the figure was always presented at the same quadrant on all five trials during a block (Fig. 2D). Given previous neurophysiological (5, 14) and brain imaging (4, 43) studies have demonstrated that figure–ground segregation can also be strongly modulated by attention, further work is needed to parse the relative contributions of attention/expectation and awareness to figure–ground segregation.
Conclusions
In sum, our study provides, to the best of our knowledge, converging evidence for awareness-dependent figure–ground segregation and for a critical role of feedback from the prefrontal cortex to V1 in this fundamental process (Fig. 6). Identifying the prefrontal cortex as a potential neural substrate of awareness-dependent figure–ground segregation gives insight into how the prefrontal cortex ignites networks supporting image understanding, perceptual organization and, more generally, consciousness in the human brain.
Materials and Methods
Subjects.
A total of 27 human subjects (9 male, 18 to 27 y old) were involved in the study. All of them participated in the psychophysical experiment. Twenty of them participated in the fMRI experiment. Two subjects in the fMRI experiment were excluded because of large head motion (>3 mm). They were naïve to the purpose of the study. They were right-handed, reported normal or corrected-to-normal vision, and had no known neurological or visual disorders. They gave written, informed consent, and our procedures and protocols were approved by the human subjects review committee of the School of Psychology at South China Normal University.
Stimuli.
Each figure–ground stimulus (Fig. 1A) had a regular Manhattan grid of 33 × 45 bars, presented in the upper visual field on a dark screen (0.203 cd/m2). Each bar was a rectangle of 0.0625° × 0.375° in visual angle. The center-to-center distance between the bars was 0.625°. All bars were identically oriented except for a square figure of bars with another orientation in either the Upper Left or the Upper Right quadrant. There were two possible figures: the first consisted of 10 × 10 bars (large figure, Fig. 1 A, Left) and the second consisted of 2 × 2 bars (small figure, Fig. 1 A, Right). Each figure was centered at 6.63° eccentricity. The orientation of the background bars was randomly chosen from 0° to 180° on each trial. The orientation contrasts between the figure bars and the background bars was 60°. Low- (0.529 cd/m2) and high- (42.446 cd/m2) luminance masks (Fig. 1C), which had the same grid as the figure–ground stimuli, rendered the whole figure–ground stimulus visible (experiment 1) or invisible (experiment 2) to subjects, respectively. Each element of the mask contained 12 intersecting bars oriented from 0° to 165° at every 15° interval. The bars in the masks had the same size and shape as those in the figure–ground stimuli. In psychophysical experiments, for each figure, there were two possible probes: a large grating (diameter: 6.25°; spatial frequency: 3.2 cycles/°; contrast: 0.70; phase: random) and a small grating (diameter: 1.25°; spatial frequency: 3.2 cycles/°; contrast: 0.70; phase: random), which had the same diameter as the large figure and small figure, respectively (Fig. 1B).
Psychophysical Experiments.
Visual stimuli were displayed on an IIYAMA color graphic monitor (model: HM204DT; refresh rate: 60 Hz; resolution: 1,280 × 1,024; size: 22 inches) at a viewing distance of 57 cm. Subjects’ head position was stabilized using a chin rest. A yellow fixation point was always present at the center of the monitor. Psychophysical experiments consisted of three experiments. Experiments 1 (visible) and 2 (invisible) investigated whether figure–ground segregation depended on the visibility of the figure–ground stimulus (confirmed by the 2AFC test, SI Appendix, Fig. S1). Subjects participated in experiments 1 and 2 on two different days, and the order of the two experiments was counterbalanced across subjects. In both experiments 1 and 2, each trial began with the fixation. A figure–ground stimulus was presented for 50 ms, followed by a 100-ms mask (low- and high-luminance in experiments 1 and 2, respectively) and another 50-ms fixation interval. The figure in the figure–ground stimulus served as a cue to attract spatial attention. Then a large or small grating probe was randomly presented for 50 ms with equal probability and presented randomly at either the figure location (valid cue condition) or its contralateral counterpart (invalid cue condition) with equal probability (Fig. 1D). The grating probe was orientated at 45° or 135° away from the vertical. Subjects were asked to press one of two buttons as rapidly and correctly as possible to indicate the orientation of the grating probe (45° or 135°). The experiment consisted of two sessions (two types of figure: the large figure and small figure), with the two sessions occurring on the same day; the order of the two sessions was counterbalanced across subjects. Each session consisted of four blocks. Each block had 72 trials, with randomly interleaving 18 trials from each type of trial (small grating-valid cue, small grating-invalid cue, large grating-valid cue, and large grating-invalid cue conditions). The cueing effect for each figure and each grating was quantified as the difference between the reaction time of the probe task performance in the invalid cue condition and that in the valid cue condition.
fMRI Experiments.
Using a block design, the experiment consisted of 11 to 14 functional runs. Each run consisted of 10 stimulus blocks of 10 s, interleaved with 10 blank intervals of 12 s. There were 10 different stimulus blocks, including 8 different figure blocks: 2 (figure: large/small) × 2 (visual field: left/right) × 2 (awareness: visible/invisible), and 2 mask-only blocks: low- and high-luminance masks. Each stimulus block was randomly presented once in each run and consisted of the same five trials. On each trial in the figure and mask-only blocks, a figure–ground stimulus and the fixation were presented for 50 ms, respectively, followed by a 100-ms mask (low- and high-luminance for visible and invisible conditions, respectively) and 1,850-ms fixation (Fig. 2D). In the invisible condition, on each trial during both the figure and mask-only blocks, subjects were asked to press one of two buttons to indicate the location of the figure, which was left of fixation in one half of blocks and right of fixation in the other half at random (i.e., the 2AFC task). Note that although the figure wasn’t presented in the mask-only blocks, subjects also indicated the location of the figure, since in the invisible condition they were unaware whether the figure was present or absent. In the visible condition, on each trial during the figure block, subjects needed to perform the same 2AFC task of the figure, whereas during the mask-only block, subjects were asked to press one of two buttons randomly, since in the visible condition, they were easy to perceive the absence of figures.
Retinotopic visual areas (LGN, V1, V2, V3, V3A/B, and V4) were defined by a standard phase-encoded method developed by Sereno et al. (69) and Engel et al. (70), in which subjects viewed rotating wedge and expanding ring stimuli that created traveling waves of neural activity in visual cortex. An independent block-design scan was used to localize the ROIs in V1 corresponding to the figure center (i.e., V1 center) and figure boundary (i.e., V1 boundary) of the large figure (Fig. 2 B, Top), and the whole small figure (i.e., V1 figure) and its surround background (i.e., V1 background) (Fig. 2 B, Bottom), as well as the ROIs in LGN and V2 to V4 corresponding to the whole figure since activated areas in these areas corresponding to the figure center and figure boundary of the large figure and the small figure and its surround background showed a great deal of overlap. The scan consisted of 12 12-s stimulus blocks, interleaved with 12 12-s blank intervals. In a stimulus block, subjects passively viewed 8-Hz flickering checkerboards. Each block type was repeated three times in the run, which lasted 288 s. The MRI data acquisition and analyses are presented in SI Appendix.
Whole-Brain Group Analyses.
In the whole-brain group analysis, a fixed-effects general linear model (FFX-GLM) was performed for each subject on the spatially nonsmoothed functional data in Talairach space. The first-level regressors were created by convolving the onset of each stimuli block with the default BrainVoyager QX’s two-gamma hemodynamic response function. Six additional parameters resulting from three-dimensional (3D) motion correction (x, y, and z rotation and translation) were included in the model. For each subject and each figure (the large figure and small figure), we first calculated fixed effects analyses for visible condition (visible-figure block vs. low-luminance mask-only block) and invisible condition (invisible-figure block vs. high-luminance mask-only block) separately (Fig. 2D). Next, a second-level group analysis (n = 18) was performed with a random-effects GLM to calculate the contrast between the visible and invisible conditions. Statistical maps were thresholded at P < 0.03 and corrected by FDR correction (35).
Effective Connectivity Analyses.
To directly confirm whether awareness modulated the region-filling process in V1 through the modulation of feedback from V4 and/or FEF, and the background-suppression process in V1 through the modulation of feedback from IPS and/or DLPFC, we applied DCM analysis (36) in SPM12 to our fMRI data. For the region-filling process, effective connectivities among the FEF, V4, V1 boundary, and V1 center were analyzed (Fig. 5A), while for the background-suppression process, effective connectivities among the DLPFC, IPS, V1 figure, and V1 background were analyzed (Fig. 5D). For both these two processes, we examined their 11 models for modeling the modulatory effect by the visible condition and fit each of their 11 models for each subject (SI Appendix). Using a hierarchical Bayesian approach (37), we compared the 11 models by computing the exceedance probability of each model, i.e., the probability to which a given model is more likely than any other included model to have generated data from a randomly selected subject. In the best model (models 9 and 4 for the region-filling and background-suppression processes, respectively), we examined the modulatory effect in the visible condition.
Supplementary Material
Acknowledgments
We thank Qinglin Chen for valuable comments. This work was supported by the National Natural Science Foundation of China (Projects 32022032 and 31871135), the Key Realm R&D Program of Guangzhou (202007030005), and the National Institute of Mental Health Intramural Research Program (NIH Clinical Study Protocol 93-M-0170, NCT00001360, ZIAMH002918-09).
Footnotes
The authors declare no competing interest.
This article is a PNAS Direct Submission.
This article contains supporting information online at https://www.pnas.org/lookup/suppl/doi:10.1073/pnas.1922832117/-/DCSupplemental.
Data Availability.
The anonymized datasets for this study are available at Open Science Framework, https://osf.io/75mnz/.
References
- 1.Lamme V. A., The neurophysiology of figure-ground segregation in primary visual cortex. J. Neurosci. 15, 1605–1615 (1995). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Roelfsema P. R., Lamme V. A., Spekreijse H., Bosch H., Figure-ground segregation in a recurrent network architecture. J. Cogn. Neurosci. 14, 525–537 (2002). [DOI] [PubMed] [Google Scholar]
- 3.Poltoratski S., Tong F., Resolving the spatial profile of figure enhancement in human V1 through population receptive field modeling. J. Neurosci. 40, 3292–3303 (2020). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Poltoratski S., Maier A., Newton A. T., Tong F., Figure-ground modulation in the human lateral geniculate nucleus is distinguishable from top-down attention. Curr. Biol. 29, 2051–2057.e3 (2019). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Poort J., Self M. W., van Vugt B., Malkki H., Roelfsema P. R., Texture segregation causes early figure enhancement and later ground suppression in areas V1 and V4 of visual cortex. Cereb. Cortex 26, 3964–3976 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Qiu F. T., von der Heydt R., Figure and ground in the visual cortex: v2 combines stereoscopic cues with gestalt rules. Neuron 47, 155–166 (2005). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Roelfsema P. R., Cortical algorithms for perceptual grouping. Annu. Rev. Neurosci. 29, 203–227 (2006). [DOI] [PubMed] [Google Scholar]
- 8.Zhou H., Friedman H. S., von der Heydt R., Coding of border ownership in monkey visual cortex. J. Neurosci. 20, 6594–6611 (2000). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Zipser K., Lamme V. A. F., Schiller P. H., Contextual modulation in primary visual cortex. J. Neurosci. 16, 7376–7389 (1996). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zhang X., Zhaoping L., Zhou T., Fang F., Neural activities in v1 create a bottom-up saliency map. Neuron 73, 183–192 (2012). [DOI] [PubMed] [Google Scholar]
- 11.Klink P. C., Dagnino B., Gariel-Mathis M. A., Roelfsema P. R., Distinct feedforward and feedback effects of microstimulation in visual cortex reveal neural mechanisms of texture segregation. Neuron 95, 209–220.e3 (2017). [DOI] [PubMed] [Google Scholar]
- 12.Lamme V. A., Zipser K., Spekreijse H., Figure-ground activity in primary visual cortex is suppressed by anesthesia. Proc. Natl. Acad. Sci. U.S.A. 95, 3263–3268 (1998). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Lamme V. A., Roelfsema P. R., The distinct modes of vision offered by feedforward and recurrent processing. Trends Neurosci. 23, 571–579 (2000). [DOI] [PubMed] [Google Scholar]
- 14.Poort J., et al. , The role of attention in figure-ground segregation in areas V1 and V4 of the visual cortex. Neuron 75, 143–156 (2012). [DOI] [PubMed] [Google Scholar]
- 15.Self M. W., van Kerkoerle T., Supèr H., Roelfsema P. R., Distinct roles of the cortical layers of area V1 in figure-ground segregation. Curr. Biol. 23, 2121–2129 (2013). [DOI] [PubMed] [Google Scholar]
- 16.Self M. W., et al. , The segmentation of proto-objects in the monkey primary visual cortex. Curr. Biol. 29, 1019–1029.e4 (2019). [DOI] [PubMed] [Google Scholar]
- 17.Mumford D., Kosslyn S. M., Hillger L. A., Herrnstein R. J., Discriminating figure from ground: The role of edge detection and region growing. Proc. Natl. Acad. Sci. U.S.A. 84, 7354–7358 (1987). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Wolfson S. S., Landy M. S., Examining edge- and region-based texture analysis mechanisms. Vision Res. 38, 439–446 (1998). [DOI] [PubMed] [Google Scholar]
- 19.Lamme V. A., Van Dijk B. W., Spekreijse H., Texture segregation is processed by primary visual cortex in man and monkey. Evidence from VEP experiments. Vision Res. 32, 797–807 (1992). [DOI] [PubMed] [Google Scholar]
- 20.Scholte H. S., Jolij J., Fahrenfort J. J., Lamme V. A., Feedforward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging. J. Cogn. Neurosci. 20, 2097–2109 (2008). [DOI] [PubMed] [Google Scholar]
- 21.Grossberg S., Mingolla E., Neural dynamics of form perception: Boundary completion, illusory figures, and neon color spreading. Psychol. Rev. 92, 173–211 (1985). [PubMed] [Google Scholar]
- 22.Chen M., et al. , Incremental integration of global contours through interplay between visual cortical areas. Neuron 82, 682–694 (2014). [DOI] [PubMed] [Google Scholar]
- 23.Landman R., Spekreijse H., Lamme V. A., Set size effects in the macaque striate cortex. J. Cogn. Neurosci. 15, 873–882 (2003). [DOI] [PubMed] [Google Scholar]
- 24.Itti L., Koch C., Computational modelling of visual attention. Nat. Rev. Neurosci. 2, 194–203 (2001). [DOI] [PubMed] [Google Scholar]
- 25.Li Z., A saliency map in primary visual cortex. Trends Cogn. Sci. (Regul. Ed.) 6, 9–16 (2002). [DOI] [PubMed] [Google Scholar]
- 26.Kastner S., Nothdurft H. C., Pigarev I. N., Neuronal correlates of pop-out in cat striate cortex. Vision Res. 37, 371–376 (1997). [DOI] [PubMed] [Google Scholar]
- 27.Knierim J. J., van Essen D. C., Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. J. Neurophysiol. 67, 961–980 (1992). [DOI] [PubMed] [Google Scholar]
- 28.Lamme V. A., Rodriguez-Rodriguez V., Spekreijse H., Separate processing dynamics for texture elements, boundaries and surfaces in primary visual cortex of the macaque monkey. Cereb. Cortex 9, 406–413 (1999). [DOI] [PubMed] [Google Scholar]
- 29.Lee T. S., Yang C. F., Romero R. D., Mumford D., Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency. Nat. Neurosci. 5, 589–597 (2002). [DOI] [PubMed] [Google Scholar]
- 30.Nothdurft H. C., Gallant J. L., Van Essen D. C., Response modulation by texture surround in primate area V1: Correlates of “popout” under anesthesia. Vis. Neurosci. 16, 15–34 (1999). [DOI] [PubMed] [Google Scholar]
- 31.Gilbert C. D., Wiesel T. N., Clustered intrinsic connections in cat visual cortex. J. Neurosci. 3, 1116–1133 (1983). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Hupé J. M., et al. , Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 394, 784–787 (1998). [DOI] [PubMed] [Google Scholar]
- 33.Posner M. I., Snyder C. R. R., Davidson B. J., Attention and the detection of signals. J. Exp. Psychol. 109, 160–174 (1980). [PubMed] [Google Scholar]
- 34.Zhang X., Japee S., Safiullah Z., Mlynaryk N., Ungerleider L. G., A normalization framework for emotional attention. PLoS Biol. 14, e1002578 (2016). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Genovese C. R., Lazar N. A., Nichols T., Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage 15, 870–878 (2002). [DOI] [PubMed] [Google Scholar]
- 36.Friston K. J., Harrison L., Penny W., Dynamic causal modelling. Neuroimage 19, 1273–1302 (2003). [DOI] [PubMed] [Google Scholar]
- 37.Penny W. D., Stephan K. E., Mechelli A., Friston K. J., Comparing dynamic causal models. Neuroimage 22, 1157–1172 (2004). [DOI] [PubMed] [Google Scholar]
- 38.Theeuwes J., Reimann B., Mortier M., Visual search for featural singletons: No top-down modulation, only bottom-up priming. Vis. Cogn. 14, 466–489 (2006). [Google Scholar]
- 39.Zhaoping L., Attention capture by eye of origin singletons even without awareness–a hallmark of a bottom-up saliency map in the primary visual cortex. J. Vis. 8, 1–18 (2008). [DOI] [PubMed] [Google Scholar]
- 40.Hupé J. M., James A. C., Girard P., Bullier J., Response modulations by static texture surround in area V1 of the macaque monkey do not depend on feedback connections from V2. J. Neurophysiol. 85, 146–163 (2001). [DOI] [PubMed] [Google Scholar]
- 41.Fang F., Boyaci H., Kersten D., Border ownership selectivity in human early visual cortex and its modulation by attention. J. Neurosci. 29, 460–465 (2009). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Qiu F. T., Sugihara T., von der Heydt R., Figure-ground mechanisms provide structure for selective attention. Nat. Neurosci. 10, 1492–1499 (2007). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Kastner S., De Weerd P., Ungerleider L. G., Texture segregation in the human visual cortex: A functional MRI study. J. Neurophysiol. 83, 2453–2457 (2000). [DOI] [PubMed] [Google Scholar]
- 44.Rossi A. F., Desimone R., Ungerleider L. G., Contextual modulation in primary visual cortex of macaques. J. Neurosci. 21, 1698–1709 (2001). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Supèr H., Spekreijse H., Lamme V. A., Two distinct modes of sensory processing observed in monkey primary visual cortex (V1). Nat. Neurosci. 4, 304–310 (2001). [DOI] [PubMed] [Google Scholar]
- 46.De Weerd P., Desimone R., Ungerleider L. G., Cue-dependent deficits in grating orientation discrimination after V4 lesions in macaques. Vis. Neurosci. 13, 529–538 (1996). [DOI] [PubMed] [Google Scholar]
- 47.Merigan W. H., Basic visual capacities and shape discrimination after lesions of extrastriate area V4 in macaques. Vis. Neurosci. 13, 51–60 (1996). [DOI] [PubMed] [Google Scholar]
- 48.Reppas J. B., Niyogi S., Dale A. M., Sereno M. I., Tootell R. B., Representation of motion boundaries in retinotopic human visual cortical areas. Nature 388, 175–179 (1997). [DOI] [PubMed] [Google Scholar]
- 49.Lennert T., Martinez-Trujillo J., Strength of response suppression to distracter stimuli determines attentional-filtering performance in primate prefrontal neurons. Neuron 70, 141–152 (2011). [DOI] [PubMed] [Google Scholar]
- 50.Suzuki M., Gottlieb J., Distinct neural mechanisms of distractor suppression in the frontal and parietal lobe. Nat. Neurosci. 16, 98–104 (2013). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.MacDonald A. W. 3rd, Cohen J. D., Stenger V. A., Carter C. S., Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 288, 1835–1838 (2000). [DOI] [PubMed] [Google Scholar]
- 52.Squire R. F., Noudoost B., Schafer R. J., Moore T., Prefrontal contributions to visual selective attention. Annu. Rev. Neurosci. 36, 451–466 (2013). [DOI] [PubMed] [Google Scholar]
- 53.Bichot N. P., Heard M. T., DeGennaro E. M., Desimone R., A source for feature-based attention in the prefrontal cortex. Neuron 88, 832–844 (2015). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Morishima Y., et al. , Task-specific signal transmission from prefrontal cortex in visual selective attention. Nat. Neurosci. 12, 85–91 (2009). [DOI] [PubMed] [Google Scholar]
- 55.Zanto T. P., Rubens M. T., Thangavel A., Gazzaley A., Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory. Nat. Neurosci. 14, 656–661 (2011). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Heinen K., Feredoes E., Weiskopf N., Ruff C. C., Driver J., Direct evidence for attention-dependent influences of the frontal eye-fields on feature-responsive visual cortex. Cereb. Cortex 24, 2815–2821 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Zhang X., Mlynaryk N., Ahmed S., Japee S., Ungerleider L. G., The role of inferior frontal junction in controlling the spatially global effect of feature-based attention in human visual areas. PLoS Biol. 16, e2005399 (2018). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Clark K., Squire R. F., Merrikhi Y., Noudoost B., Visual attention: Linking prefrontal sources to neuronal and behavioral correlates. Prog. Neurobiol. 132, 59–80 (2015). [DOI] [PubMed] [Google Scholar]
- 59.Armstrong K. M., Fitzgerald J. K., Moore T., Changes in visual receptive fields with microstimulation of frontal cortex. Neuron 50, 791–798 (2006). [DOI] [PubMed] [Google Scholar]
- 60.Dehaene S., Changeux J. P., Experimental and theoretical approaches to conscious processing. Neuron 70, 200–227 (2011). [DOI] [PubMed] [Google Scholar]
- 61.Koch C., Massimini M., Boly M., Tononi G., Neural correlates of consciousness: Progress and problems. Nat. Rev. Neurosci. 17, 307–321 (2016). [DOI] [PubMed] [Google Scholar]
- 62.Tononi G., Koch C., The neural correlates of consciousness: An update. Ann. N. Y. Acad. Sci. 1124, 239–261 (2008). [DOI] [PubMed] [Google Scholar]
- 63.Lau H., Rosenthal D., Empirical support for higher-order theories of conscious awareness. Trends Cogn. Sci. 15, 365–373 (2011). [DOI] [PubMed] [Google Scholar]
- 64.Tsuchiya N., Wilke M., Frässle S., Lamme V. A. F., No-report paradigms: Extracting the true neural correlates of consciousness. Trends Cogn. Sci. 19, 757–770 (2015). [DOI] [PubMed] [Google Scholar]
- 65.Aru J., Bachmann T., Singer W., Melloni L., Distilling the neural correlates of consciousness. Neurosci. Biobehav. Rev. 36, 737–746 (2012). [DOI] [PubMed] [Google Scholar]
- 66.Frässle S., Sommer J., Jansen A., Naber M., Einhäuser W., Binocular rivalry: Frontal activity relates to introspection and action but not to perception. J. Neurosci. 34, 1738–1747 (2014). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Panagiotaropoulos T. I., Deco G., Kapoor V., Logothetis N. K., Neuronal discharges and gamma oscillations explicitly reflect visual consciousness in the lateral prefrontal cortex. Neuron 74, 924–935 (2012). [DOI] [PubMed] [Google Scholar]
- 68.Siclari F., et al. , The neural correlates of dreaming. Nat. Neurosci. 20, 872–878 (2017). [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Sereno M. I., et al. , Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging. Science 268, 889–893 (1995). [DOI] [PubMed] [Google Scholar]
- 70.Engel S. A., Glover G. H., Wandell B. A., Retinotopic organization in human visual cortex and the spatial precision of functional MRI. Cereb. Cortex 7, 181–192 (1997). [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The anonymized datasets for this study are available at Open Science Framework, https://osf.io/75mnz/.






