Investigating category‐ and shape‐selective neural processing in ventral and dorsal visual stream under interocular suppression

Abstract

Recent behavioral and neuroimaging studies using continuous flash suppression (CFS) have suggested that action‐related processing in the dorsal visual stream might be independent of perceptual awareness, in line with the “vision‐for‐perception” versus “vision‐for‐action” distinction of the influential dual‐stream theory. It remains controversial if evidence suggesting exclusive dorsal stream processing of tool stimuli under CFS can be explained by their elongated shape alone or by action‐relevant category representations in dorsal visual cortex. To approach this question, we investigated category‐ and shape‐selective functional magnetic resonance imaging‐blood‐oxygen level‐dependent responses in both visual streams using images of faces and tools. Multivariate pattern analysis showed enhanced decoding of elongated relative to non‐elongated tools, both in the ventral and dorsal visual stream. The second aim of our study was to investigate whether the depth of interocular suppression might differentially affect processing in dorsal and ventral areas. However, parametric modulation of suppression depth by varying the CFS mask contrast did not yield any evidence for differential modulation of category‐selective activity. Together, our data provide evidence for shape‐selective processing under CFS in both dorsal and ventral stream areas and, therefore, do not support the notion that dorsal “vision‐for‐action” processing is exclusively preserved under interocular suppression. Hum Brain Mapp,36:137–149, 2015. © 2014 Wiley Periodicals, Inc.

INTRODUCTION

A widely accepted and influential theory in cognitive neuroscience concerns the functional specialization of the primate visual system into a dorsal “vision‐for‐action” and a ventral “vision‐for‐perception” stream [Milner and Goodale, 1995]. A core component of the dual‐stream theory is the hypothesis that ventral stream processes are closely associated with conscious perception, whereas processing in the dorsal stream can occur without giving rise to awareness [Milner, 2012]. To a large degree, the theory is based on neuropsychological data from patients with relatively circumscribed brain lesions in either visual stream, for example, patient DF who exhibits some intact visuomotor behaviors despite suffering from visual form agnosia due to ventral stream lesions [Goodale et al., 1991; James et al., 2003]. Attempts to test the dual‐stream theory in neurologically healthy observers have mainly focused on the differential effects of visual illusions on perception and action, but the results have remained controversial [Franz and Gegenfurtner, 2008; Westwood and Goodale, 2011].

Recently, a series of behavioral and functional magnetic resonance imaging (fMRI) studies using continuous flash suppression (CFS) in normal observers have provided new evidence suggesting that dorsal stream processing might in fact be independent of awareness. CFS is a powerful method of interocular suppression which can eliminate monocularly presented stimuli from conscious perception for up to seconds by presenting dynamic masks to the other eye [Tsuchiya and Koch, 2005; for a variant see Bahrami et al., 2007]. Visual priming experiments showed that the presentation of tools which were rendered invisible by CFS facilitated the recognition of visible tool images, while this effect was absent for other stimulus categories [Almeida et al., 2010; Almeida et al., 2008]. Interestingly, when backward masking was used to suppress stimuli from awareness, priming was not restricted to tools but was also found within other categories. In line with the dual‐stream theory, it was proposed that dorsal stream areas may form action‐relevant representations for man‐made manipulable objects (tools), even in the absence of awareness during CFS, while there are no such representations in ventral visual areas. More recent results, however, cast doubt on this scenario. In similar priming experiments, Sakuraba et al. [2012] observed tool priming whenever elongated stimuli including simple geometric lines were used as primes, but not when non‐elongated stimuli were used, suggesting that priming effects under CFS can be fully explained by stimulus shape alone and not by action‐relevance of stimuli.

In this study, therefore, we sought to investigate whether neural activity in the dorsal “vision‐for‐action” stream in response to images of tools is due to their close association with visually guided action or rather to their specific elongated shape. As a first step to answer this question, we compared blood‐oxygen‐level dependent (BOLD) responses to visible and invisible tools that were clearly manipulable, but not elongated, to activation to tools with an elongated shape, using both univariate and multivariate measures.

The second aim of our study was to contribute to the understanding of the CFS method per se and its neural underpinnings, respectively. Neuroimaging studies using CFS have already produced an extensive but heterogeneous body of data on the scope and limits of processing under interocular suppression [for recent reviews: Hesselmann, 2013; Sterzer et al., 2014]. Specifically, discrepant results were reported with respect to activity in both visual streams. Fang and He [2005] found that, in comparison to the ventral stream, activity in the dorsal stream was much less reduced in amplitude relative to visible stimuli. Hesselmann and Malach [2011], however, observed a stream‐invariant reduction of activity whenever stimuli were rendered invisible by CFS [also see Fogelson et al., 2014]. Here, we hypothesized that depth of interocular suppression would be a candidate factor susceptible to reconcile both previous findings. Based on psychophysical and fMRI data suggesting that suppression strength varies with CFS mask contrast [Dubois et al., 2008; Ledgeway et al., 2013], we expected to replicate Fang and He's findings with weak interocular suppression (low‐contrast CFS masks), and Hesselmann and Malach's results with strong suppression (high‐contrast CFS masks).

MATERIAL AND METHODS

Participants

All 26 participants had normal or corrected‐to‐normal vision, were naïve to the purpose of the study, and were paid for participation. Procedures conformed to local ethics guidelines and all observers gave informed written consent. The data of five participants had to be excluded (one due to excessive movement in the scanner, two had a 2AFC performance of >60% in the “invisible” condition, two because stimuli could not be rendered invisible despite low contrast). All remaining 21 participants (17 female) were right‐handed according to their own statement. Their mean age was 23 years (range 18 to 32). As tested with the hole‐in‐card test (a modified version of the original ABC test, Miles, 1930), 15 of the participants were right eye dominant.

Apparatus and Setup

During the psychophysical pretest the observers were seated in a dark environment, the only light coming from the experimental monitor and a second monitor, and viewed the dichoptic images on a 19″ CRT screen with a spatial resolution of 1280 × 960 pixels and a refresh rate of 60 Hz, via a mirror stereoscope that prevented all cross‐talk between the eyes. To stabilize head position, the participants placed their heads on a chinrest. The viewing distance from the eyes to the screen (including distances within the mirror system) was 66 cm, resulting in each pixel subtending approximately 0.024° of visual angle.

In the scanner, the stimuli were presented via a Sanyo LCD projector at 60 Hz that projected from the head‐end of the scanner onto an opal glass projection screen. The participant viewed the stimuli via a mirror fixed onto the head coil while wearing prism glasses. Together with these glasses, a cardboard divider that was installed between glass pane and mirror ensured dichoptic stimulation and prevented crosstalk [Schurger, 2009]. The viewing distance from the eyes to the screen (including distances within the mirror system) was 68 cm, resulting in each pixel subtending approximately 0.022° of visual angle.

Visual stimuli were generated with MATLAB 7.9.0 (MathWorks, Natick, MA) and the Psychophysics Toolbox 3 [Brainard, 1997; Pelli, 1997] and displayed via IBM‐compatible computers.

Stimuli

The stimuli consisted of gray scale images of tools (elongated and non‐elongated) and faces (Supporting Information Figure S1). They were adjusted so that the stimulus area (as compared to background area) was the same for all tools (9%), irrespective of elongation, and all faces (17%). They were low‐pass filtered with a 2D‐Gaussian filter with a standard deviation of 30 pixels. The low‐level image properties (luminance histograms and rotational average of the Fourier spectra) were matched across all exemplars (of all categories) using the SHINE toolbox [Willenbockel et al., 2010]. All elongated tool stimuli had a width/length ratio smaller than 0.4, while all non‐elongated tool stimuli had a width/length ratio larger than 0.4 (width orthogonal to the longest extension). Note that highly similar ratios can be found for the tool stimuli used in the studies by Sakuraba et al. [2012] and Almeida et al. [2013].

Interocular Masking

We used an interocular suppression paradigm [Tsuchiya and Koch, 2005] called CFS. This technique uses high‐contrast images flashed to one eye to suppress images presented to the other eye from awareness. The images consisted of rectangles of five gray levels (ranging from black to white) and sizes ranging from 4 to 18% of the size of the CFS area, which measured 5.1° (pretest) or 4.8° (scanner), respectively. The rectangles were positioned at random locations and at different random angles (0° and multiples of 7°) on the mask image. We used different angles of orientation for the CFS rectangles to ensure strong suppression for stimuli of various orientations, as suppression is strongest if mask and stimulus are alike in certain features [Yang and Blake, 2012]. Hundreds of these images were produced and flashed in random order at 10 Hz to the dominant eye. This rendered the stimulus presented to the non‐dominant eye largely invisible. The masks were presented on and off together with the stimulus presentation (see method section “Main experiment”). To promote stable binocular fusion during dichoptic presentation, a black‐and‐white square frame was presented around the stimuli. Its outer and inner dimensions in the pretest were 6.1° and 5.1° of visual angle, respectively, and in the scanner 5.7° and 4.8°, respectively. The stimulus images took up the same space. In the main experiment, we varied the contrast of the CFS masks; the Michelson contrast was either 0.49, 0.70, or 0.95, in the pretest the contrast was 0.70.

While aiming at different suppression levels that would be reflected in the neuronal data, it was important that the stimuli suppressed by the different masks would be judged similarly concerning their visibility as the trials judged as visible were discarded from the analysis. In a more sensitive control experiment (see method section “Control Experiment”), we further established the different suppression strengths of the CFS masks.

Procedure

Pretest

A few days before the main experiment, the participants completed a pretest in which the stimulus contrast for the main experiment was determined. After a stimulus presentation that conformed to that of the main experiment (see method section “Main experiment”), the participant had to press a key according to whether the stimulus (tool or face) had been visible or not. Based on this response, the stimulus contrast was decreased or increased following a logarithmic scale in the next trial (1‐up‐1‐down staircase). The stimuli were always masked with the medium CFS mask, and each participant completed two staircases of 20 trials for each of the stimulus categories (faces, non‐elongated and elongated tools). The stimulus contrast in the main experiment was set to the highest stimulus contrast that the participant always judged as invisible in the pretest. The contrast of faces and tools was adjusted separately. This individual adjustment of contrasts ensured maximal stimulus strength even under full suppression. The resulting average Michelson contrasts were 0.124 for faces and 0.263 for tools (for individual contrasts see Supporting Information Figure S2).

Main experiment

In each trial, all 10 images of one stimulus category were shown in random order to the non‐dominant eye while the CFS masks were shown to the dominant eye (at 10 Hz and only when a stimulus was shown). Each image was shown for 200 ms followed by a 300 ms blank. After this 5 s stimulation and a blank screen (1 s), the participants were presented with two questions that they had to answer via button press. First, they were asked what they thought had been presented (1: face, 2: tool) and had 2.5 s to answer during which the response options stayed on the screen. On button press, the selected option slightly changed its color so that the participants knew that their answer had been recorded. The second question concerned the subjective visibility of the stimulus. The observers had to choose one of the following options during another 2.5 s‐interval: 1 = “No experience,” 2 = “Weak glimpse,” 3 = “Almost clear,” and 4 = “Absolutely clear.” This was a short version of the 4‐point perceptual awareness scale (PAS) to which the participants were introduced during the instruction [Ramsoy and Overgaard, 2004]. The next trial started after a blank screen of 1–5 s (drawn from a uniform distribution). The paradigm is depicted in Figure 1.

Figure 1.

Continuous flash suppression (CFS) paradigm. On each trial, participants viewed 10 exemplars of a given stimulus category (tools or faces). Tools were either elongated or non‐elongated. Exemplars were presented to the non‐dominant eye for 200 ms followed by a 300‐ms blank. Dynamic masks were presented to the dominant eye to render stimuli invisible. After each trial, participants had to indicate whether tools or faces had been presented (2AFC) and report subjective stimulus visibility using the 4‐point perceptual awareness scale (PAS).

Each of the six runs contained 38 trials: in 16 of the trials faces were presented, in another 16 trials tools were presented (8 elongated, 8 non‐elongated), in the remaining six trials there was no stimulus. The stimuli were either suppressed (mask levels 0.95, 0.70, and 0.49, 10 trials each) or fully visible (mask level 0, 8 trials). There were no trials with mask level 0 and no stimulus. The total duration of the six fMRI runs was approximately 53 min.

Localizer experiment

In the localizer experiment, the stimuli were presented at maximum contrast (Michelson: 0.847) for 10 s, alternating between nine exemplars of one category: elongated and nonelongated tools, faces, and houses (not used for analysis). Within one miniblock, each exemplar was shown for 800 ms and the exemplars were shown in randomized order with interstimulus intervals of 200 ms. Between these miniblocks was always a pause of 6 s in which only the blank fusion frame was shown. There were eight miniblocks per category, the order of which was also randomized. To ensure participants' attention to the stimuli, one of the exemplars was repeated in two consecutive presentations to which the participants had to respond by pressing a button. The total duration of the localizer experiment was just under 9 min.

FMRI Acquisition

BOLD images were acquired by T2*‐weighted gradient‐echo echo‐planar imaging (field of view (FOV) 192 mm × 192 mm, 33 slices, repetition time (TR) 2000 ms, echo time (TE) 30 ms, flip angle 78°, voxel size 3 × 3 × 3 mm, interslice gap 10%) on a 3T MRI scanner (Tim Trio, Siemens, Erlangen). Two hundred and seventy three volumes were recorded for each of the six experimental runs, and 257 images for the localizer run. Anatomical images were acquired using a T1‐weighted MPRAGE sequence (FOV 256 mm × 256 mm × 192 mm, 160 slices, TR 1900 ms, TE 2.52 ms, flip angle 9°, voxel size 1 × 1 × 1 mm).

FMRI Data Analysis

Preprocessing

The images were preprocessed using statistical parametric mapping (SPM8, http://www.fil.ion.ucl.ac.uk/spm/software/spm8/) with the following steps: standard realignment, coregistration, normalization to MNI stereotactic space using unified segmentation [Ashburner and Friston, 2005], and spatial smoothing with 8 and 10 mm full‐width at half‐maximum isotropic Gaussian kernels for single‐subject and group analyses, respectively. The images for the multivariate pattern analysis underwent the same steps except for normalization and smoothing.

Exclusion criteria for participants and trials

Only participants with less than 60% performance in the 2AFC objective task (face versus tool) when rating the stimulus as completely invisible (PAS = 1) in the invisible condition were included in the analysis. For the remaining participants, we followed the following strategy. If their objective task performance was below 55% even when including PAS ratings of 2 (weak glimpse: “something was seen, but content unclear”), trials with PAS ratings of 2 were included. This strategy maximized the number of trials and was applied to 13 participants. If performance exceeded 55%, only trials with a PAS rating of 1 were included in the analyses of the invisible condition (N = 8).

General linear model

Using a general linear model (GLM) approach [Friston et al., 1994] with a mixed event‐related and block design, the 5 s intervals of stimulus presentation were modeled using a boxcar function convolved with the canonical hemodynamic responses function (HRF) implemented in SPM8.

In the main experiment, the first GLM for the analysis of elongated versus non‐elongated tools included nine regressors: “visible elongated tools”, “invisible elongated tools”, “visible non‐elongated tools”, “invisible non‐elongated tools”, “visible faces”, “invisible faces”, “CFS masks only”, “trials judged visible in “invisible” condition”, and “response screens”. The second GLM for the analysis of faces versus tools included seven regressors: “visible tools”, “invisible tools”, “visible faces”, “invisible faces”, “CFS masks only”, “trials judged visible in “invisible” condition”, and “response screens”. The third GLM for the analysis of the effects of suppression strength included 13 regressors: “visible tools”, “invisible tools” (×3: mask contrast 0.49, 0.70, and 0.95), “visible faces”, “invisible faces” (×3: mask contrast 0.49, 0.70, and 0.95), “CFS masks only” (×3: mask contrast 0.49, 0.70, and 0.95), “trials judged visible in “invisible” condition”, and “response screens”. The localizer experiment was analyzed using four regressors: “elongated tools”, “non‐elongated tools”, “faces”, and “houses”.

In both experiments, the design matrix included the first derivative of the canonical HRF and six rigid‐body realignment parameters as nuisance covariates. After high‐pass filtering at 1/128 Hz, we estimated single‐subject statistical parametric maps, then created contrast images, and entered these into one‐sample t‐tests at the group level.

The dorsal and ventral regions of interest (ROIs) were identified as follows: We mapped the group‐level contrasts “tools > faces” and “faces > tools” from the localizer experiment, respectively, at P < 0.001, uncorrected, and defined search spheres of 10 mm radius around local cluster peaks with the highest t‐values in left and right parietal cortex ([−27 −76 34] and [27 −79 37], t(20) = 5.54; t(20) = 6.92) and the right fusiform face area, FFA ([42 −52 −20], t(20) = 4.99). We refer to the parietal ROIs as left and right V3A/V7, following previous work [Hesselmann and Malach, 2011]. Note that these regions were not the only clusters in this map (Fig. 2) but were selected based on a priori knowledge from previous studies [Fang and He, 2005; Hesselmann and Malach, 2011].

Figure 2.

Tool‐ and face‐ selective areas defined by the localizer experiment. (A) Activation for faces > tools in the localizer run, P < 0.001 uncorrected. (B) Activation for tools > faces in the localizer run, P < 0.001 uncorrected. The lines show the position of the spherical ROIs (after rendering them in the same way as the depicted activations). [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Individually for each participant, we then created three ROIs for the univariate analysis by selecting a 2 mm radius sphere around the peak voxel within the corresponding group‐level search spheres for the same contrast at the single‐subject level. This procedure resulted in ROIs containing the peak voxel and directly neighboring voxels. A larger ROI containing all active voxels (same contrast at single subject level) within the group‐level search spheres was also created separately for each participant. We adjusted the threshold so that the ROI contained on average 48 voxels. However, as we found larger category selectivity in the peak voxel ROIs, we report the results from those ROIs. The peak voxel ROI coordinates are provided in Supporting Information Table S1. Parameter estimates were calculated using the MarsBaR Toolbox 0.42 (http://marsbar.sourceforge.net/), averaged across all ROI voxels.

Multivariate pattern analysis

The spherical ROIs defined at the group‐level for the univariate analysis were also used for the MVPA. For this purpose, they were retransformed into the native space of each participant. The number of voxels included in each of these ROIs was variable between participants due to this retransformation, as shown in Supporting Information Table S2. All voxels within these ROIs were selected for MVPA, which was conducted using a standard linear support vector machine as implemented by LIBSVM [Chang and Lin, 2011] with a cost parameter of c = 1 in the framework of “The Decoding Toolbox” [Görgen et al., 2012]. GLMs were recalculated on unnormalized and unsmoothed data. The prediction accuracies (face versus tool) were calculated for each participant and ROI using a leave‐one‐run‐out cross‐validation approach.

Dependent Variables and Data Analysis

Behavioral analysis

For the subjective visibility report, the frequency of each PAS rating was counted for each participant and then averaged, separately for each mask contrast. The result is displayed in Figure 3A. The objective 2AFC performances were compared to chance performance (50%) using a two‐sided t‐test. As a nonsignificant t‐test result is no evidence for the null hypothesis, we further conducted a Bayes analysis. This analysis provides the likelihood of the data given the null hypothesis and the likelihood of the data given the alternative hypothesis as well as their quotient—the Bayes factor (BF)—as an output. In contrast to null hypothesis testing, one needs to formulate the alternative hypothesis mathematically to be able to test it against the null hypothesis. In our case, the alternative hypothesis is that the participants had some impression of the stimulus and were thus above chance. Hence, we formulated the alternative hypothesis as a normal distribution with a mean of 63.51% (i.e., the average performance in the 2AFC task for trials in which the participants indicated degraded perception of the stimulus, PAS = 2) and a standard deviation of 23.8%. The BF was calculated using a freely available online Bayes calculator (http://www.lifesci.sussex.ac.uk/home/Zoltan_Dienes/inference/bayes_factor.swf), provided by Dienes [2011].

Figure 3.

Behavioral results. (A) PAS ratings dependent on the contrast of the CFS masks: Most stimuli were rated as absolutely or almost clearly visible in the condition without interocular masking. The three different mask contrasts led to similar subjective visibility ratings, with most trials being rated as eliciting no visual experience (PAS = 1). (B) Performance in the objective 2AFC task (tool/face discrimination) for all trials of the invisible condition which were included in the analyses. The dashed line represents chance level.

Univariate fMRI analysis

In all univariate analyses, the dependent variables were the ROI parameter estimates. In all univariate analyses of the invisible condition, we subtracted mask activation from the joint activation of mask and stimulus. To test for selective activation for faces and tools in the ventral and dorsal ROIs we used a two‐way repeated measures (rm‐) ANOVA with the factors category (face/tool) and ROI (ventral/dorsal). To test for preferential processing of elongated stimuli in the dorsal ROIs, a two‐way rm‐ANOVA with the factors shape (elongated/non‐elongated) and visibility (visible/invisible) was used. For completeness, the same analysis was carried out for the ventral ROI. To answer the question whether invisible tools were processed in the dorsal stream, a three‐way rm‐ANOVA with the factors category, ROI, and visibility was used. A three‐way rm‐ANOVA with the factors category, ROI, and mask contrast (0.49, 0.70, 0.95) tested for differential effects of mask contrast under invisibility.

Multivariate fMRI analyses

To test whether invisible tools were processed in the dorsal stream, we also used a multivariate approach. The dependent variables were the decoding accuracies and for statistical testing a permutation test was used, as t‐tests on decoding accuracies have been shown to be inappropriate [Schreiber and Krekelberg, 2013; Stelzer et al., 2013]. In a first step, the labels/classes were shuffled in all possible permutations, and then the decoding was carried out as for the original labels, which yielded 32 accuracies for each participant. For the group statistic, the permutations were combined with a boot‐strapping procedure: 100,000 times one of the 32 values was drawn for each participant and then averaged over participants. These values then constituted the test distribution under the null hypothesis. The number of boot‐strapped values that were higher than the actual accuracy divided by the number of all values yielded the P‐value (e.g., P = 400/100,000 = 0.004). An example of these test distributions for one of the tests is shown in Supporting Information Figure S5. To test for preferential processing of elongated stimuli, decoding accuracies were submitted to a two‐way rm‐ANOVA with the factors shape and visibility.

Control Experiment

The control experiment was conducted to establish different levels of suppression strength for the different mask contrasts. Thirty‐one (23 female) participants took part in this control experiment. All had normal or corrected‐to‐normal vision, were naïve to the purpose of the study, and were paid for participation. Procedures conformed to local ethics guidelines and all observers gave informed written consent. All participants were right‐handed according to their own statement. Their mean age was 22 years (range 18–29). As tested with the hole‐in‐card test (a modified version of the original ABC test; Miles, 1930), 23 of the participants were right eye dominant. Sixteen of the participants also participated in the main experiment and pretest.

The settings of the control experiment were as described for the main experiment and pretest. The experiment's procedure was adapted from Tsuchiya et al. [2006]. After initiating the trial by button press, a horizontal sinusoidal grating (Michelson contrast 0.10) was presented to the participant's non‐dominant eye while CFS masks were presented to the dominant eye. CFS masks were the same as in the main experiment and pretest. Subsequently, they confirmed by button press that they only saw the CFS masks, which initiated a contrast change of the upper or the lower half of the grating. The contrast increased for 250 ms and then decreased for 250 ms, initially to full contrast (Michelson: 0.95), following a Gaussian distribution with a standard deviation of 100 ms. After the contrast change, a beep was presented and the participants had to indicate if the contrast change had occurred in the upper or the lower half of the presentation area (2AFC). When the participants gave two correct responses in a row, the maximum of the contrast change was decreased by 20%, after an erroneous response it was increased by 20%. This step was reduced to ±10% after four reversals. After 12 reversals, the staircase finished and the geometric mean of the last 10 reversals yielded the contrast detection threshold (71% as measured by a 2 down‐1 up staircase procedure [Levitt, 1971]). A difference to the main experiment and pretest were the specifics of the fusion frame which were also adapted from Tsuchiya et al. [2006]. It consisted of one row of black and white squares surrounding the presented stimuli and masks.

RESULTS

Behavioral Results

Figure 3A shows that the different mask contrasts led to similar levels of subjective invisibility, with most trials being rated as eliciting no visual experience (PAS = 1) or a weak glimpse (PAS = 2), leading to our aspired result, namely that approximately the same number of trials for each mask contrast entered into our analyses. In the condition without interocular masking, most stimuli were rated as almost clearly (PAS = 3) or absolutely clearly visible (PAS = 4). Under the exclusion criteria described in the “Exclusion criteria for participants and trials” section, the participants had an average performance of 51.21% in the objective 2AFC task (Fig. 3B). This performance did not differ significantly from chance in the t‐test (t(20)= 1.63, P = 0.119). The Bayes analysis yielded a BF of 0.10, that is, substantial evidence for the null hypothesis of chance performance [Dienes, 2011].

Control Experiment

The control experiment revealed a significant influence of mask contrast on contrast detection thresholds (F(2,60) = 3.48, P = 0.037; Supporting Information Figure S3). Post hoc t‐tests showed that this was mainly due to a difference between the medium and the highest contrast levels (t(30) = −2.63, P = 0.013). The differences between the lowest and the medium mask contrast, and between the lowest and the highest mask contrast were not significant (t(30) = 1.24, P = 0.223; t(30) = −1.40, P = 0.173). Hence, interpretations about the effect of mask contrast on neural processing should focus on the difference between the conditions of medium and high contrast. For completeness, however, all results are plotted and discussed.

Univariate fMRI Results

We first verified the validity of the ventral and dorsal ROIs by testing for category‐selective responses to visible stimuli in the main experiment, that is, when stimuli were not masked by CFS. The results are shown in Figure 4. We found larger activations to faces in the FFA, and to tools in V3A/V7 (interaction category*ROI: F(1,20) = 41.20, P < 0.001). Of the main effects of the rm‐ANOVA, only the effect of category turned out to be significant (category: F(1,20) = 9.95, P < 0.005; ROI: F(1,20) = 2.78, P = 0.111). The FFA also responded to tool stimuli, albeit significantly weaker than to face stimuli (t(20) = 2.38, P = 0.027). This is in line with previous findings concerning the fusiform gyrus [Chao and Martin, 2000]. In V3A/V7, the response to tools was significantly stronger than to faces (t(20) = 6.07, P < 0.001). Visually, the same pattern emerged for stimuli that were visible because they broke through suppression (PAS ratings > 1). Here, the interaction did not reach significance (interaction category*ROI: F(1,20) = 1.35, P = 0.260; category: F(1,20) < 1, P = 0.943; ROI: F(1,20) = 9.82, P < 0.006).

Figure 4.

Category‐selective BOLD responses in FFA and V3A/V7 to visible face and tool stimuli in the main experiment. On the left, parameter estimates are shown for all visible trials (trials with a mask contrast of 0, i.e., no masks). On the right, parameter estimates are shown for trials in which the stimuli were subjectively visible (PAS rating > 1) despite masking by CFS. The activation to the masks alone was subtracted in this condition. Error bars denote ± 1 standard error of the mean (SEM).

Next, we addressed the question whether dorsal processing of tools depends merely on their (mostly elongated) shape attributes (i.e., whether elongated tools elicited higher activation than non‐elongated tools) or whether it reflects genuine activation to tools due to their connection to visually guided action, irrespective of shape (i.e., whether activation to elongated and non‐elongated tools was similar). For this purpose, we compared the activation in the dorsal ROIs to elongated tools with activation to non‐elongated but clearly manipulable tools, both in the visible and invisible condition. Figure 5 shows that there was no significant effect of shape (F(1,20) = 2.85, P = 0.107), supporting the notion of genuine tool selectivity in dorsal areas. In the remaining analyses, therefore, elongated and non‐elongated tools were analyzed together. There was, however, a significant effect of visibility: when the tools were rendered invisible with CFS, activation levels decreased, again irrespective of shape (effect visibility: F(1,20) = 9.28, P < 0.007; interaction shape*visibility: F(1,20) = 2.47, P = 0.132). In the ventral ROI, we found a significant interaction of shape and visibility, because in the visible condition non‐elongated tools yielded higher activation than elongated tools, while activation levels were similar in the invisible condition (F(1,20) = 4.60, P = 0.044; t(20) = 2.61, P = 0.017).

Figure 5.

BOLD responses to elongated and non‐elongated tools in V3A/V7, separately for the visible and invisible condition. In case of the invisible condition, the activation to the masks alone was subtracted. Error bars denote ± 1 SEM.

The latter finding of reduced BOLD responses under invisibility was confirmed in a further analysis in which we looked at a possible dissociation between the ventral and dorsal stream concerning the processing of stimuli that did not enter awareness (Supporting Information Figure S4). Both invisible tools and faces led to significantly reduced activation relative to visible stimuli in their respective ROI (effect visibility: F(1,20) = 6.36, P = 0.020), and this reduction turned out to be stream‐invariant (interaction ROI*visibility: F(1,20) < 1, P = 0.415; see Supporting Information Table S3 for a full summary of the effects). Faces, even when visible, did not show robust activation in dorsal ROIs, hence there was also no observable decrease due to invisibility. This is reflected in a significant three‐way interaction (category*ROI*visibility: F(1,20) = 47.46, P < 0.001). A highly similar reduction from the visible to the invisible condition was obtained with larger ROIs (see method section “General linear model”).

Finally, we tested the hypothesis that dorsal processing of invisible stimuli is spared when the contrast of the mask is low, thus yielding a lower suppression level relative to CFS masks of higher contrast. BOLD responses to invisible stimuli were analyzed with respect to stimulus category, ROI, and mask contrast. Note that for this analysis only 18 of 21 participants who had at least two trials for each of the mask contrasts in each category were included. The results are shown in Figure 6. As can be seen, there was no effect of mask contrast on BOLD responses to invisible stimuli (F(2,34) < 1, P = 0.977). None of the other main and interaction effects were significant (all P > 0.153). Supporting Information Figure S6 displays the processing of faces in the FFA and tools in V3A/V7 without the subtraction of the mask activation.

Figure 6.

Activation to faces and tools in FFA and V3A/V7. The activation to the masks alone was subtracted in the three invisible conditions. The ANOVA described in the main text was calculated only over the invisible conditions to be able to detect possible effects of mask contrast. The responses to invisible stimuli were low, irrespective of ROI or mask contrast and the differences between the three mask contrasts not significant. Error bars denote ± 1 SEM.

Multivariate fMRI Results

As the univariate analysis showed no sign of residual processing of the invisible stimuli, neither in the ventral nor in the dorsal ROI, we used MVPA to probe whether there was category selectivity in the ROIs even under invisibility [Haynes and Rees, 2006; Kamitani and Tong, 2005]. Figure 7A shows the average prediction accuracies from decoding tools from faces. Note that the values of left and right dorsal ROIs did not show significant differences and thus were averaged in Figure 7 for the sake of clarity. Visible stimuli were significantly decodable from each other above chance in all ROIs: 77.38% (P < 0.001) in the right FFA, 72.02% (P < 0.001) in V3A/V7. By contrast, in the invisible condition, the average prediction accuracies were not different from chance: 51.19% (P = 0.315) in the right FFA, 51.79% (P = 0.254) in V3A/V7. Next, we tested for effects of interocular suppression strength (Fig. 7B): If any decoding accuracy hinted at successful decoding of category, it was for the lowest mask contrast in the FFA (55.21%, P = 0.069; not corrected for multiple tests). For the higher mask contrasts, the decoding accuracies were 49.36% (P = 0.583) and 50.00% (P = 0.472) in the FFA. The decoding accuracies for V3A/V7 for the corresponding contrasts were 49.74% (P = 0.528), 50.32% (P = 0.475), and 47.28% (P = 0.799).

Figure 7.

(A) Decoding accuracies for faces versus tools in FFA and V3A/V7. Decoding accuracies for the visible condition (no masks) were significantly higher than chance as tested in a permutation test, while the stimulus category could not be decoded from the neural data in the invisible condition. (B) Decoding accuracies for faces versus tools in FFA and V3A/V7 plotted separately for visible (mask level 0) and invisible trials (mask levels 0.49, 0.70, and 0.95; only invisible trials were included). Note that the values of left and right dorsal ROI did not show significant differences and were averaged for the sake of clarity. Dashed line: approximate chance level (exact chance level is determined by the permutation method and is different for each test). Error bars denote ± 1 SEM. Asterisks: ***:P < 0.001, not corrected for multiple comparisons.

Finally, we tested if there was a decoding benefit for elongated tools compared to non‐elongated tools, the decoding being carried out versus faces in both cases (Fig. 8). For visible stimuli, both elongated and non‐elongated tools can be successfully decoded from faces (all P < 0.001). In case of invisible stimuli, the MVPA showed enhanced decoding of elongated relative to non‐elongated tools, in both the FFA (56.19%, P = 0.025), and right V3A/V7 (56.27%, P = 0.025), but not in left V3A/V7 (52.66%, P = 0.193). The decoding accuracies for non‐elongated tools in the FFA and V3A/V7 were virtually at chance level (∼50%, all P > 0.470; Supporting Information Figure S5 plots the results of the permutation tests underlying the P‐values). An ANOVA with the factor shape confirmed a tendency toward enhanced decoding for elongated compared to non‐elongated tools (F(1,20) = 4.24, P = 0.053; Supporting Information Table S4A provides a full summary of the effects). Pairwise comparisons revealed a tendency in right V3A/V7: t(20) = 1.91, P = 0.071 (FFA: t(20) = 1.42, P = 0.172; left V3A/V7: t(20) = 0.64, P = 0.532).

Figure 8.

Decoding accuracies faces versus elongated tools (left) and nonelongated tools (right) in FFA and left and right V3A/V7 for the visible and the invisible condition. Decoding was carried out separately for elongated tools versus faces and for non‐elongated tools versus faces. Dashed line: approximate chance level (exact chance level is determined by the permutation method and is different for each test). The MVPA showed enhanced decoding of elongated relative to non‐elongated tools, in both the FFA and right V3A/V7 but not in left V3A/V7. Error bars denote ± 1 SEM. Asterisks: *: P < 0.05; ***: P < 0.001, not corrected for multiple comparisons.

DISCUSSION

In this study, we investigated category‐ and shape‐selective fMRI‐BOLD responses in both visual streams when stimuli were either visible or rendered invisible by CFS. In line with behavioral priming experiments, we found that for the processing of invisible tool stimuli their shape plays a role. Specifically, the multivariate analysis showed enhanced decoding of elongated relative to nonelongated tools. To resolve conflicting earlier fMRI findings related to the differential link to awareness in dorsal and ventral stream [Fang and He, 2005; Hesselmann and Malach, 2011], we also looked at category‐selective BOLD activity in both visual streams, dependent on the depth of interocular suppression. For all levels of suppression, we observed a stream‐invariant reduction of responses compared to a visible condition. Each of these points will be discussed in turn.

Invisible Processing of Elongated Versus Non‐elongated Tool Stimuli

Our results show that visible tools elicit similar levels of activation in the dorsal visual stream, irrespective of their shape attributes. This finding is in agreement with research establishing the dorsal stream as an important area for the preparation of visually guided actions [Binkofski et al., 1998; Chao and Martin, 2000; Culham and Valyear, 2006], and it thus validates our dorsal ROIs.

One previous study reported largely preserved activity in the dorsal stream when images of tools were rendered invisible by CFS [Fang and He, 2005]. Building on this result, it has been proposed that behavioral priming effects observed for invisible tool primes under CFS are due to processing in the dorsal stream specific for stimuli with a relation to visually guided action [Almeida et al., 2010; Almeida et al., 2008]. Speaking against this scenario, however, Sakuraba et al. [2012] found priming effects for all types of elongated prime stimuli (e.g., lines and elongated vegetables) but not for non‐elongated primes.

Although we did not find a significant difference in the level of activation to invisible elongated and non‐elongated tools in a standard univariate analysis, the MVPA revealed enhanced decoding of invisible elongated tools compared to invisible non‐elongated tools (decoding for visible tools was similar for the two categories). We conclude that the activation that survives interocular suppression might in fact not reflect category‐specific processing but merely show that stimuli with certain shape attributes (e.g., elongated with a principal axis) are more likely to be processed under interocular suppression than others. This residual shape‐related activation could also explain why the presentation of elongated invisible tools primed the categorization of elongated tool stimuli [Almeida et al., 2010; Almeida et al., 2008]. Our findings support the notion of Sakuraba et al. [2012] that tool‐priming effects observed under CFS can be attributed to preferential processing of stimuli with an elongated shape. It is interesting to note that the tool stimuli used by Fang and He [2005] were exclusively elongated (personal communication). In a next step, therefore, one could investigate whether representations in the dorsal stream for non‐action‐relevant elongated images (such as geometric lines) are similar to those of elongated tools.

Despite their different accounts of the nature of the priming effects, both Almeida et al. [2008, 2010] and Sakuraba et al. [2012] assume that priming with tools and elongated stimuli originates from dorsal visual processes only [also see Hebart and Hesselmann, 2012]. Our results do not seem to support this notion as we found above chance decoding of elongated tools also in our ventral ROI, suggesting instead that both ventral and dorsal processes mediate the priming effects.

Based on a new series of priming experiments, Almeida et al. [2013] proposed that elongation, irrespective of the category of the elongated object, triggers grasp preparation and, therefore, represents a core feature of “manipulability”. According to Almeida et al., this action‐relevant cue can be extracted by the dorsal stream even under impoverished situations such as interocular suppression, whereas category cannot be processed. In this framework, our result of enhanced decoding of elongated tools despite interocular suppression could be interpreted as preserved processing of “manipulability” and not mere residual processing of elongated shape. But again, this scenario seems unlikely as we also observed enhanced decoding of elongated tools in the ventral visual stream. Importantly, Almeida et al. [2013] do not show whether their priming effects can be also found when non‐elongated tools are used as targets, thus not excluding the possibility of shape priming. It remains to be investigated whether the neural activity evoked by elongated stimuli reflects processing related to visually guided action or a mere preference in processing for elongated objects unrelated to action.

We found that successful decoding of category (face vs. elongated tool) was possible in right but not in left V3A/V7, although tool processing has been described as left‐lateralized [Chao and Martin, 2000; Johnson‐Frey et al., 2005]. Neither Fang and He [2005] nor Hesselmann and Malach [2011], however, reported a left‐lateralization of tool‐related BOLD responses in their experiments.

Stream‐Invariant Reduction of BOLD Responses Under CFS

In two previous fMRI studies, the link between BOLD responses in ventral and dorsal stream areas and visual awareness has been studied with similar experimental designs but conflicting outcomes [Fang and He, 2005; Hesselmann and Malach, 2011]. Both studies used CFS to suppress visual stimuli from awareness. While Fang and He [2005] showed preserved activity to invisible tool stimuli in dorsal areas, Hesselmann and Malach [2011] showed a reduction of activity across all ROI. We hypothesized that—under the assumption of different suppression levels in the two studies—we should be able to replicate Fang and He's findings with weak suppression (i.e., low‐contrast CFS masks) and Hesselmann and Malach's findings with strong suppression (i.e., high‐contrast CFS masks). Contrary to our expectation, we found a stream‐invariant reduction of BOLD responses to tool and face stimuli even for low suppression levels and were thus unable to replicate the findings of Fang and He [2005]. Importantly, we established the effectiveness of our modulation of suppression levels based on CFS mask contrasts in a control experiment. It should be noted, however, that measured suppression levels were significantly different only between medium and high mask contrasts.

Why have we not found a differential link of dorsal and ventral areas to visual awareness, even with varying suppression levels in the invisible condition? There are a number of possibilities how this came about. Possibly, the suppression in the invisible condition was still too strong in our study to find any neural activation to the suppressed stimuli. What speaks against this scenario is that we were able to decode the category of invisible stimuli in one multivariate analysis (elongated tool vs. face). Although this might be purely based on different low‐level stimulus properties (oval faces vs. elongated tools), it shows that at least some stimulus information reached higher visual areas despite suppression. Arguably, the suppression could still have been too strong to find univariate responses to invisible stimuli. Furthermore, as in previous studies [Hesselmann et al., 2011; Hesselmann and Malach, 2011; Ludwig et al., 2013], we tried to keep the suppression as low as possible but as high as necessary by individually adjusting the contrast of the stimuli. Individual contrast adjustment is one of three options; the other options would be to choose a very low contrast for all subjects, risking not to find any effect, or to choose a higher contrast and exclude all subjects with above chance performance due to visibility of the stimulus. In our case, adjustment of contrasts ensured that contrasts were sufficient to produce category‐selective BOLD responses whenever stimuli broke through suppression.

In this study, we used smaller ROIs than in the previous studies [Fang and He, 2005; Hesselmann and Malach, 2011]. The lack of response to invisible stimuli, however, cannot be attributed to this fact as a highly similar reduction from the visible to the invisible condition was also found with larger ROIs.

One of the central questions we addressed in this study is the difference in the depth of suppression possibly underlying different results in different CFS studies. Here, we managed the challenging task of manipulating the depth of interocular suppression while keeping the object stimuli equally invisible in all experimental conditions of the fMRI experiment. To achieve approximately the same number of trials for each condition (i.e., to avoid differential exclusions due to visibility), we had to make sure that the stimuli were largely invisible for all mask contrasts. In this range of mask contrasts yielding successful suppression, we further modulated suppression strength by means of the mask contrast. In the control experiment, we could show that these different contrasts indeed led to different contrast detection thresholds, albeit not in the parametrical fashion that we had anticipated. On a purely speculative note, we hypothesize that the mask of the lowest contrast might have been successful in suppressing the stimulus due to the higher similarity of stimulus and mask (in the contrast domain) as similarity of mask and stimulus (in terms of their spatial properties) has been shown to lead to high suppression [Yang and Blake, 2012].

To our knowledge, the BOLD responses reported by Fang and He represent the strongest nonconscious visual signal that has been observed under interocular suppression. For that reason alone, further investigation of these responses in dorsal visual stream seems worthwhile. Future studies should explore the effect of experimental parameters which have not been addressed by our study. For example, the impact of a primary detection task at fixation, which was present in Fang and He's experiment but absent in our experiment. Another relevant parameter might be the effective duration of intermittent CFS stimulation, which was 20 s in the case of Fang and He, and 5 s in our study. However, a potential challenge will be that longer stimulation generally results in more stimuli that escape suppression by CFS and thus become visible [Stein et al., 2011]. Finally, the spatial frequency of CFS masks might have had a major influence in producing the different results (Fang and He: “random noise”; Hesselmann and Malach: “Mondrians”) as Yang and Blake [2012] have shown that suppression strength depends on the respective spatial frequency of mask and stimulus.

CONCLUSION

To conclude, our data do not support the notion that opposed to ventral processes, dorsal stream activity is exclusively preserved under interocular suppression. We found a reduction in both visual streams for invisible as compared to visible stimuli. In the only condition in which we found successful stimulus decoding despite invisibility, both visual streams showed the same pattern of results. Whether this successful decoding under CFS is based on category information or on shape attributes remains a topic for future research.

Supporting information

Supplementary Information.