Dissociating the Effect of Noise on Sensory Processing and Overall Decision Difficulty

Abstract

It has been proposed that perceptual decision making involves a task-difficulty component, which detects perceptual uncertainty and guides allocation of attentional resources. It is thought to take place immediately after the early extraction of sensory information and is specifically reflected in a positive component of the event related potentials, peaking at ∼220 ms after stimulus onset. However, in the previous research, neural processes associated with the monitoring of overall task difficulty were confounded by those associated with the increased sensory processing demands as a result of adding noise to the stimuli. Here we dissociated the effect of phase noise on sensory processing and overall decision difficulty using a face gender categorization task. Task difficulty was manipulated either by adding noise to the stimuli or by adjusting the female/male characteristics of the face images. We found that it is the presence of noise and not the increased overall task difficulty that affects the electrophysiological responses in the first 300 ms following stimulus onset in humans. Furthermore, we also showed that processing of phase-randomized as compared to intact faces is associated with increased fMRI responses in the lateral occipital cortex. These results revealed that noise-induced modulation of the early electrophysiological responses reflects increased visual cortical processing demands and thus failed to provide support for a task-difficulty component taking place between the early sensory processing and the later sensory accumulation stages of perceptual decision making.

Introduction

Recently, it was proposed that there is a task-difficulty component of perceptual decision making, which is responsible for detection of perceptual uncertainty as well as allocation of attentional resources and takes place immediately after the early extraction of sensory information (Philiastides et al., 2006) (for review, see Heekeren et al., 2008). It was based on experimental findings (Philiastides et al., 2006) showing that increasing stimulus uncertainty by adding phase noise to the visual stimuli leads to increased amplitudes of a positive event-related potential (ERP) peak at ∼220 ms, corresponding to the P2 component of the ERP responses. Such modulation of P2 component was present only in those conditions, in which noise impaired task performance (i.e., increased task difficulty). It was also found that the neural processes underlying the noise effects on P2 component might involve a network of frontal cortical areas (Philiastides and Sajda, 2007). Based on these results, it was concluded that the modulation of P2 component by stimulus uncertainty might be considered a neural marker of a task-difficulty component of perceptual decision making (Philiastides et al., 2006; Philiastides and Sajda, 2007; Heekeren et al., 2008).

However, besides affecting the frontal cortical processes related to overall task difficulty, adding noise to the images will also lead to enhanced visual cortical processing demands, which in turn might modulate the early electrophysiological responses, including the P2 component. Object processing in the visual cortex involves a cascade of neural events, which are reflected in the P1-N1-P2 ERP complex (Murray et al., 2006; Rousselet et al., 2007; Smith et al., 2007; Barbeau et al., 2008; Okazaki et al., 2008). The P2 component, in particular, might reflect grouping processes (Schendan and Kutas, 2007; Schendan and Lucia, 2010) as well as high-level object processing (Halit et al., 2000; Itier and Taylor, 2002; Latinus and Taylor, 2005; Mercure et al., 2008) and might be modulated by the recurrent and feedback processes within the visual cortical hierarchy in the case of increased processing demands, such as during visual masking (Kotsoni et al., 2007). Therefore, at this point it is not clear to what extent the modulation of the P2 component in the study by Philiastides et al. (2006) reflected the effect of noise on visual processing or on the processes related to detection of overall task difficulty and allocation of attentional resources.

The goal of the present study was to dissociate the effect of noise on the visual cortical processing from its effects on the processes related to overall task difficulty. To this end, we measured ERP and fMRI responses to face stimuli with and without phase noise in a face gender categorization task. Task difficulty was manipulated either by adding noise to the stimuli or by adjusting the ability to categorize the face images by morphing along the gender axis. The combined results of our electrophysiological and fMRI experiments revealed that modulation of the early ERP responses, including the P2 component, reflects noise-related increase of visual cortical processing rather than the processes of detection of overall task difficulty and allocation of attentional resources.

Materials and Methods

Subjects

Altogether 19 (three left-handed, six females, mean ± SD age: 26 ± 3 years) subjects gave their informed and written consent to participate in the study, which was approved by the ethics committee of Semmelweis University. Separate EEG and fMRI experimental sessions were conducted, both of which were completed by 16 subjects with an overlap of 13 subjects. None of them had any history of neurological or ophthalmologic diseases, and all had normal or corrected-to-normal visual acuity.

Visual stimuli

In both sessions (EEG and fMRI), participants viewed images of human faces and performed a gender categorization task. Stimuli consisted of front-view grayscale photographs of four female and four male faces that were cropped and covered with a circular mask to eliminate the outer features. All images were equated for luminance and contrast. Subsequently, each face image was modified to create two distinct face sets, thereby increasing task difficulty in two different ways. In one set, the task was made more demanding by decreasing the gender difference between female and male stimuli (morphed set). This was achieved by a morphing algorithm (Winmorph 3.01) (Kovács et al., 2005, 2006, 2007) by pairing a female and a male photo, creating intermediate androgynous images. In the other set, noise was added to the images by decreasing their phase coherence (noisy set) using the weighted mean phase technique (Dakin et al., 2002), which resulted in the gradual elimination of the facial cues important for accurate gender judgment. We decided to use phase randomization in the current experiments primarily because task difficulty was modulated by adding phase noise in previous studies suggesting the existence of a task-difficulty component of perceptual decision making (Philiastides et al., 2006; Philiastides and Sajda, 2007). Observers were presented with three different conditions: images with a fixed morph level from the morphed set (noise-absent condition: N-a), images with a fixed phase coherence from the noisy set (noise-present condition: N-p), and the original unmanipulated images (control condition: C) (Fig. 1). The morph level and phase coherence was adjusted individually to achieve 75% accuracy and was based on pilot sensitivity measures. Stimuli were presented centrally on a uniform gray background and subtended 2 visual degrees, matching approximately the size of the fovea. The reason for using small foveal stimuli was the following. Phase randomization of the face stimuli leads to decreased saliency of the face features (such as eyes and eyebrows) that are diagnostic in the face gender categorization task used in the current study (Gosselin and Schyns, 2001). This in turn might result in enhanced spatial attentional demands in the case of noisy compared to intact faces. We thought that such a confounding effect of the difference in spatial attention selection between intact and noisy faces might be minimized by using small foveal stimuli.

Figure 1.

Stimuli. The panel on the left shows samples of the female and male face stimuli used in the control condition. By morphing these paired female and male face images, we created stimuli with ambiguous face gender attributes and used these intermediate androgynous face images in the N-a condition (top right). In the N-p condition (bottom right), we used noisy face stimuli, which were created by decreasing the phase coherence of the face images used in the control condition. The intermediate smaller face pairs in the middle of the figure are included to better illustrate the different image manipulations but were not presented during the experiment.

Stimulus presentation was controlled by MATLAB 7.1. (The MathWorks) using the Cogent 2000 toolbox (http://www.vislab.ucl.ac.uk/cogent.php) in both experiments. In the EEG experiment, visual stimuli were presented on a 22 inch Samsung CRT monitor at a refresh rate of 100 Hz and were viewed from 56 cm. In the case of the fMRI experiment, stimuli were projected onto a translucent screen located at the back of the scanner bore using a Panasonic PT-D3500E DLP projector (Matsushita Electric Industrial) at a refresh rate of 75 Hz. Stimuli were viewed through a mirror attached to the head coil at a viewing distance of 58 cm. Head motion was minimized using foam padding.

Procedure

Subjects performed a gender categorization task: they were required to judge the gender of the face images as accurately and fast as possible, indicating their choice with one of the mouse buttons. Button assignment was left for female and right for male for half of the subjects and was reversed for the other half. Before the experiment, each subject was given a practice session to get familiar with the task and to determine the exact morph and phase coherence level to be used to yield 75% accuracy.

The EEG and the fMRI experiment procedures were almost identical with only slight differences in timing. In the EEG paradigm, each trial started with an intertrial interval (ITI), which was randomized in the range of 1800–2200 ms and was followed by the presentation of the face for 250 ms. The response window was a maximum of 2 s but was terminated when subjects responded. The fixation point was present throughout the trial. Within a single run, the three conditions were intermixed and presented in random order. Each participant completed eight runs, yielding 192 trials altogether for each condition. In the fMRI paradigm, the faces were also presented for 250 ms, but the stimulus onset asynchrony (SOA) was set to either 5550, 7550, or 9550 ms, being varied in a pseudorandom fashion. The response window was also fixed in 2 s. Each subject performed seven functional runs, yielding 70 trials altogether for each condition. In addition, an extra 10% of all trials were blank, i.e., did not include a face stimulus and served as baseline. Similarly to the EEG, the three conditions were intermixed in a single run, but trial order was presented in an optimized pseudorandom order to maximize separability of the different conditions.

During the fMRI experimental session subjects performed a separate localizer scan to provide a map of face-selective regions for the ROI analysis. The scan was a block design of 16-s-long epochs of faces and nonsense objects (Kovács et al., 2008) interleaved with their Fourier phase-randomized versions, which served as baseline. Stimuli were presented with 0.5 Hz for 300 ms each. There were 8 face blocks, 8 object blocks, and 17 baseline blocks, making a total number of 33 blocks. Subjects performed a one-back task during the face and object blocks and reported the total number of one-back repetitions at the end of the run.

Behavioral data analysis

Responses and reaction times (RTs) were collected during both experiments. For further analysis, however, only RTs for correct response trials were used. Accuracy and mean correct RTs were analyzed with repeated-measure ANOVAs with condition (C vs N-a vs N-p) as within-subject factors. The Greenhouse–Geisser correction was applied to correct for possible violations of sphericity. Post hoc t tests were computed using Tukey honestly significant difference (HSD) tests.

Electrophysiological recording and analysis

EEG acquisition and processing.

EEG data were acquired using a BrainAmp MR (BrainProducts) amplifier from 60 Ag/AgCl scalp electrodes placed according to the extended 10-20 international electrode system and mounted on an EasyCap (Easycap) with four additional periocular electrodes placed at the outer canthi of the eyes and above and below the right eye for the purpose of recording the electrooculogram. All channels were referenced to joint earlobes online and later digitally transformed to an average reference; the ground was placed on the nasion. All input impedance was kept below 5 kΩ. Data were sampled at 1000 Hz with an analog bandpass of 0.016–250 Hz. Subsequently, a digital 0.1 Hz 12 dB/octave Butterworth zero phase high-pass filter was used to remove DC drifts, and a 50 Hz notch filter was applied to minimize line-noise artifacts. Finally, a 24 dB/octave low-pass filter with a cutoff frequency of 30 Hz was applied. Trials that contained voltage fluctuations exceeding ±100 μV, or electro-oculogram activity exceeding ±50 μV were rejected. Data processing was done using BrainVision Analyzer (BrainProducts).

ERP data analysis.

The trial-averaged EEG waveform—i.e., the event-related potential (ERP)—was computed as follows. Data were segmented into 800 ms epochs starting from 200 ms preceding the stimuli. Segments were baseline corrected over a 200 ms prestimulus window, artifact rejected, and averaged to obtain the ERP waveforms for each subject for each condition. Subject ERPs were averaged to compute the grand average ERP for visualization purposes. Statistical analysis was performed on early component peaks (P1, N170, and P2) of the grand average ERP waveform. Early peak amplitudes were computed as follows: peak latency was determined individually on the global field power (GFP) (see Fig. 3B) waveform (Bankó and Vidnyánszky, 2010). Since the GFP has a local maximum when the amplitude difference between electrodes is strongest, its maxima reflect components in the event-related potential (Lehmann and Skrandies, 1980; Hamburger and vd Burgt, 1991). Mean peak amplitudes were measured over electrode clusters in a 10 ms window centered on these GFP peak latencies. The clusters included electrodes where the respective ERP peak showed its maxima. P1 amplitude was measured over PO7, PO9, O1, and O9 and PO8, PO10, O2, and O10 electrodes, P2 amplitude over P5, PO3, PO7, and O1 and P6, PO4, PO8, and O2 electrodes, N170 amplitude over PO7, PO9, P7, and P9 and PO8, PO10, P8, and P10 electrodes, and P3b amplitude over CPz, P1, Pz, and P2 electrodes. Amplitude values were analyzed by three-way repeated-measure ANOVAs with condition (C vs N-a vs N-p), side (except for P3b), and electrode (4) as within-subject factors separately for each component. The Greenhouse–Geisser correction was applied to correct for possible violations of sphericity. Post hoc t tests were computed using Tukey HSD tests.

Figure 3.

Grand-average ERPs and peak amplitudes of the main experiment. A, B, ERPs evoked by the control, morphed, and noisy faces are shown in black, blue, and brown, respectively, alongside the GFP (B) waveforms averaged over subjects. C details the voltage topographies and amplitudes of the peaks P1, N170, P2, and P3b measured over the respective electrode clusters marked with black dots on the topographical maps. In the cases of P1, N1, and P2, the noise-present condition significantly differs from the rest, while in the case of P3b, there is only a nonsignificant trend. Maps are the average of the three conditions at the time point shown below. Note that cartoon heads are plotted with unrealistic head radius for better electrode visibility. Error bars indicate ±SEM (N = 16, **p < 0.01, ***p < 0.001; n.s., not significant).

Time-frequency and intertrial synchrony analysis.

Single-trial wavelet analysis was performed using the EEGLab toolbox (Delorme and Makeig, 2004) and custom-written Matlab software. Data were segmented into 3000 ms epochs starting from −1500 to 1500 ms relative to stimulus onset, baseline corrected over 1000 ms prestimulus window, and artifact rejected. Time-frequency spectrum was calculated using a sinusoidal wavelet (short-time DFT) transform computing the power spectrum over the frequency range of 4–30 Hz (with an interpolated frequency resolution of ∼0.5 Hz) in a sliding latency window (window size: 512 ms, step: 25 ms) and then averaging across data trials. The effective width of the sliding window decreased to provide a better time resolution at higher frequencies (Delorme and Makeig, 2004). Amplitude values, derived from the real C(f,t)_r and imaginary C(f,t)_i Fourier coefficients, were squared to create power measure, log transformed, and baseline corrected.

We computed the intertrial coherence (ITC) for assessing intertrial phase stability for a given time window and frequency bin as a measure of neural synchrony since the ERP is a result of intertrial phase-locked activity. The ITC is the length of the vector sum of the unitary phase vectors at a given time point and frequency for all single trials, which is then divided by the number of trials:

where n is the total number of trials, ϕ_i is the phase of the signal of the ith trial, | | denotes the norm operator, and j = $\sqrt{-1}$. The ITC value ranges from 0 to 1, with value 1 meaning constant phase across trials relative to stimulus onset (i.e., perfect intertrial synchronization) and value 0 meaning random phases (i.e., total absence of synchrony).

To visualize power and intertrial coherence changes across the frequency range, spectrograms were plotted. Moreover, for both measures time courses of the theta frequency band were obtained by averaging over the 4–7 Hz frequency bins. Spectral power was baseline corrected by subtracting the mean baseline value of −1000 to −300 ms relative to stimulus onset individually for each frequency bin. For visualization, a significance threshold value of p <0.05 was calculated as a function of the distribution of spectral power values during the baseline for each frequency with a bootstrap method (Melloni et al., 2007).

To detect significant phase ordering across trials, the Rayleigh test of uniformity of angle was used (Jervis et al., 1983; Tallon-Baudry et al., 1996). This statistical test involves computing the Rayleigh's R̄ parameter, which is mathematically identical to the ITC:

R̄ is then evaluated against a known distribution function and compared to the applicable critical value. Significant phase ordering (i.e., intertrial phase synchrony) was declared when the ITC exceeded the R̄ value corresponding to p_Bonf < 0.05. Bonferroni correction was used to account for the multiple comparisons across the ITC bins.

To compare conditions, further statistical tests were performed on the ITC time courses only. We reasoned that the ITC is a better correlate of the event-related responses since high ITC values in a given frequency band indicate stimulus-locked oscillatory patterns. Therefore, we evaluated the ITC averaged in the time window of 100–200 ms in the theta (4–7 Hz) frequency band. The window was chosen to encompass the peak region of theta ITC. ITC was measured on three different electrode clusters: a central cluster, a left visual, and a right visual cluster. Electrodes were chosen to match the peak topography of theta activity. The central cluster included FPz, C1, Cz, C2, and CPz, while the bilateral visual clusters included O9, PO7, PO9, P7, and P9 and O10, PO8, PO10, P8, and P10, closely matching the N170 electrode cluster. The obtained numbers were entered into a two-way repeated-measure ANOVA with condition (C vs N-a vs N-p) and electrode as within-subject factors for each cluster and frequency band. The Greenhouse–Geisser correction was applied to correct for possible violations of sphericity. Post hoc t tests were computed using Tukey HSD tests.

We conducted a Spearman correlation analysis to investigate the relationship between the ITC and the ERP, both of which reflect evoked activity. The P1, N170, and P2 amplitude values measured over bilateral visual clusters and averaged over a 10 ms window centered on the individual peaks (as described above) were correlated with the ITC values averaged in fixed time windows—100–140 ms, 150–200 ms, and 215–320 ms for P1, N170, and P2, respectively—which encompassed the range of individual variations of peak latency for the corresponding peak. To correct for multiple comparisons (c = 6), significance threshold was set to p_Bonf = 0.05 (p_unc = 0.008).

Analysis of eye-tracking data

Eye-gaze direction was assessed using a summary statistic approach. Mean eye position (x and y values) was calculated for each trial for the period when the face stimulus was present. Mean coordinates were collected separately for the three conditions, and from each of these eye-gaze direction datasets, spatial maps of eye-gaze density were constructed and then averaged to get a mean map for comparison. Subsequently, each of these maps was compared with the mean map collapsed across conditions and difference images were computed. The root mean squares of the density difference values for the difference maps were entered into a one-way ANOVA (Winston et al., 2004; Bankó et al., 2009).

fMRI imaging and analysis

Data acquisition.

Data were collected at the MR Research Center of Szentágothai Knowledge Center (Semmelweis University, Budapest, Hungary) on a 3.0 tesla Philips Achieva scanner equipped with an eight-channel SENSE head coil. High-resolution anatomical images were acquired for each subject using a T1-weighted 3D TFE sequence yielding images with a 1 × 1 × 1 mm resolution. Functional images were collected using 31 transversal slices (4 mm slice thickness with 3.5 mm × 3.5 mm in-plane resolution) with a non-interleaved acquisition order covering the whole brain with a BOLD-sensitive T2*-weighted echo-planar imaging sequence (TR = 2 s, TE = 30 ms, FA = 75°, FOV = 220 mm, 64 × 64 image matrix, 7 runs, duration of each run = 444 s, and a localizer run, duration = 528 s).

Data preprocessing and analysis.

Details of preprocessing and statistical analysis have been given previously (Kovács et al., 2008). Briefly, the functional images were corrected for acquisition delay and head motion and spatially smoothed with a Gaussian kernel of 8 mm FWHM (SPM8, Welcome Department of Imaging Neuroscience). In addition, gray and white matter in the anatomical images were coregistered with the mean functional T2* images followed by segmentation and normalization to the MNI-152 space using SPM's segmentation toolbox. The resulting gray matter mask was used to restrict statistical analysis on the functional files.

To define the regressors for the general linear model analysis of the data, a reference canonical hemodynamic response function was convolved with boxcar functions, representing the onsets of the experimental conditions. Low-frequency components were excluded from the model using a high-pass filter with 128 s cutoff. Variance that could be explained by the previous scans was excluded using an autoregressive AR(1) model, and movement-related variance was accounted for by the spatial parameters resulting from the motion correction procedure. The resulting regressors were fitted to the observed functional time series within the cortical areas defined by the gray matter mask. Individual statistical maps were then transformed to the MNI-152 space using the transformation matrices generated during the normalization of the anatomical images. Conjunction analyses were performed using the conjunction null hypothesis (Nichols et al., 2005). For visualization purposes, the conjunctions of contrasts (N-p > N-a) & (N-p > C) and (N-p > C) & (N-a > C) (see Fig. 6A) were superimposed with p_unc < 10⁻⁴ threshold onto the population average landmark and surface based (PALS-B12) standard brain (Van Essen, 2005) using Caret 5.62 (Van Essen et al., 2001) (http://brainvis.wustl.edu/wiki/index.php/). For other data visualization, the XjView 8. toolbox (http://www.alivelearn.net/xjview8/) was used. Stereotaxic coordinates are reported in MNI space, and regional labels were derived using the AAL atlas provided with XjView 8.

Figure 6.

Results of the whole-brain conjunctions and ROI analyses. A, depicts bilateral posterior LOC (DOT-LOS) as the only cortical areas showing higher activation to the noise-present condition relative to both the control and the noise-absent conditions (above) and left insula and right inferior frontal gyrus as the areas with stronger fMRI response to both noise-absent and -present relative to the control condition (below). Maps are displayed with p_unc < 10⁻⁴ on the PALS-B12 partially inflated brain (Van Essen, 2005) and on the averaged anatomical images of all 16 subjects. B, Based on an independent localizer, the ROI analysis in agreement with the whole-brain analysis shows higher bilateral DOT-LOS activation for noise-present condition relative to the other two, while FFA does not show any significant modulation (***p < 0.001; Ins, insula; iFG, inferior frontal gyrus).

Region of interest analysis.

Regions of interests (ROIs) were defined individually based on the localizer run. Areas matching our anatomical criteria and lying closest to the corresponding reference cluster (i.e., clusters from the random-effects group analysis of the localizer run, p_unc <10⁻⁴; t₍₁₅₎ > 4.88) were considered to be their appropriate equivalents on the single-subject level. The location of the fusiform face area (FFA) and the occipital face area (OFA) were determined as areas responding more strongly to faces than to objects or phase-randomized faces (p_unc <10⁻⁴; t₍₁₅₎ > 4.88). It was possible to define bilateral FFA in all 16 subjects (average MNI coordinates ± SD: 40 ± 3, −48 ± 3, −22 ± 2 and −38 ± 2, −49 ± 4, −21 ± 2 for right and left hemispheres, respectively), while bilateral OFA could only be defined in 14 subjects (41 ± 3, −72 ± 8, −13 ± 3 and −41 ± 3, −73 ± 7, −12 ± 3). For the remaining two subjects, the group-average OFA coordinates were taken from the random-effects group statistics of the localizer run (40, −68, −14 and −42, −76, −14). Object-selective areas were defined as the areas in the lateral occipitotemporal cortex and posterior fusiform gyrus that showed significantly stronger activation (p_unc <10⁻⁴; t₍₁₅₎ > 4.88) to objects than to faces. These included three distinct regions that closely corresponded to object-selective regions in the dorsal occipitotemporal cortex (DOT) and ventral occipitotemporal cortex (VOT) defined by Hasson et al. (2003); therefore, we decided to use their nomenclature. The regions were as follows: DOT-LOS on the lateral surface of the middle occipital cortex located in the lateral occipital sulcus, DOT-ITS on the lateral surface of inferior occipital cortex located close to the inferior temporal sulcus, and VOT-mpFs on the medial surface of the posterior fusiform gyrus (Hasson et al., 2003; Schendan and Stern, 2007). All the object-selective areas were part of the lateral occipital complex (LOC) (Malach et al., 1995) as conventionally defined by areas responding more strongly to intact objects than to their scrambled counterparts. DOT-ITS, which corresponds to the posterior dorsal portion of LOC (Grill-Spector et al., 1999, 2000), was identifiable in all 16 observers (48 ± 5, −66 ± 7, −10 ± 5 and −48 ± 4, −66 ± 4, −9 ± 5), while DOT-LOS could be defined in only 13 subjects (37 ± 4, −83 ± 4, 12 ± 4 and −37 ± 4, −84 ± 4, 12 ± 4). For the remaining three subjects, the DOT-LOS coordinates, similarly to the OFA, were taken from the group statistics of the localizer runs (40, −80, 16 and −38, −84, 14). Bilateral VOT-mpFs [also termed fusiform object area (Druzgal and D'Esposito, 2001)] was definable in 10 subjects (28 ± 4, −53 ± 10, −16 ± 1 and −27 ± 6, −51 ± 9, −16 ± 1), while the coordinates for the remaining 6 subjects were taken from the group statistics of the localizer runs (24, −46, −12 and −28, −58, −16).

A time series of the mean voxel value within a 4-mm-radius sphere around the local peak of the areas of interest was calculated and extracted from the event-related sessions (MarsBaR 0.38) (Brett et al., 2002), and the same GLM was applied to the data as used in the whole-brain analysis. The percentage signal change was estimated for each ROI for each observer to characterize the magnitude of the response change. We performed a two-way ANOVA for each area with hemisphere (L vs R) and condition (C vs N-a vs N-p) as factors for each area. We also performed planned contrasts to evaluate pairwise differences between conditions.

Control experiment

Subjects.

Altogether 14 (two left-handed, seven females, mean ± SD age: 26 ± 4 years) subjects gave their informed and written consent to participate in the control experiment, which was approved by the ethics committee of Semmelweis University. None of them had any history of neurological or ophthalmologic diseases, and all had normal or corrected-to-normal visual acuity. Six of the fourteen had also participated in the main experiment.

Visual stimuli and procedure.

Stimuli were similar to the stimuli used in the main experiment. In a gender categorization task, we used upright faces with three separate phase coherence values (60–50-40%), while in a tilt discrimination task, we used rotated faces with fixed low phase coherence (45%). Faces were tilted to the left and right with either 1 or 7 degrees, the former tilt resulting in a hard, the latter in an easy discrimination condition. Observers performed the face and tilt discrimination task in separate runs, the order of which was counterbalanced across subjects. Button assignment was also counterbalanced for the gender categorization task, but remained constant for tilt discrimination.

Stimuli were presented for 250 ms followed by the response window, which was a maximum of 1.5 s but was terminated when subjects responded. ITI was randomized in the range of 500–900 ms. Each participant completed three runs from each task, yielding 96 trials altogether for each condition. Data analysis for the gender categorization task was identical as described above, while for the tilt discrimination task paired Student's t tests were performed.

Electrophysiological recording and analysis.

All recording procedures and event-related analysis steps were identical with the main experiment. Here we focused only on the analysis of the P2 ERP component.

Results

Behavioral results

The behavioral results obtained during both the EEG and the fMRI sessions of the main experiment revealed no significant difference in the subjects' response accuracy and RTs between the noise-absent and noise-present conditions, indicating that the gender categorization task was equally difficult in these two conditions (Fig. 2A). On the other hand, response accuracy was higher and RTs were shorter in the case of the control condition as compared to the two difficult categorization conditions both during the EEG and the fMRI experiments (Fig. 2B). One-way repeated-measures ANOVAs showed significant main effects of condition with regard to both accuracy (F_(2,30) = 130.74, p < 0.0001 and F_(2,30) = 72.45, p < 0.0001 in the EEG and fMRI experiments, respectively) and RT (F_(2,30) = 35.95, p < 0.0001 and F_(2,30) = 76.61, p < 0.0001 in the EEG and fMRI experiments, respectively) in both experiments. Tukey HSD post hoc tests revealed that this was due to significantly higher accuracy and shorter response times in the control condition as compared to the two difficult conditions, which did not differ from each other significantly in any of these aspects (for post hoc p values, see supplemental Table 1, available at www.jneurosci.org as supplemental material). These results show that face gender categorization was much easier in the control condition than in the two difficult categorization conditions.

Figure 2.

Behavioral results. A, B, Both accuracy (A) and reaction times (B) were significantly impaired as a result of stimulus modulation both in the EEG and in the fMRI experiment. Error bars indicate ±SEM (N = 16 in both cases, ***p < 0.001).

Results of eye-tracking analysis

We tracked the eye position of 12 and 6 randomly selected subjects during the EEG and the fMRI sessions, respectively, to ensure that the observed differences were not due to differences in fixation patterns. The data revealed no significant differences in the deviation of fixation positions between the three conditions (no main effect of condition: F_(2,24) = 1.11, p = 0.37 and F_(2,24) = 1.52, p = 0.27 in the EEG and fMRI sessions, respectively). These indicate that there was no systematic bias in eye-gaze direction (i.e., fixation positions) induced by the different stimuli corresponding to the three conditions.

Results of EEG recordings

ERP results

Grand average ERP waveforms (Fig. 3A) in the noise-present condition differed from those in the noise-absent and control conditions during the first ∼300 ms following the stimulus onset, which time period is in the focus of the current study. These differences were most pronounced on the electrodes over the occipital and temporal lobes. Contrary to this result, the grand average ERPs in the noise-absent condition did not differ from those in the control condition in this early time window. These observations were confirmed by the statistical analysis performed on the amplitudes of the P1, N170, and P2 components of the ERP responses, by showing a significant difference between the noise-present condition and the other two conditions and no difference between the noise-absent and control conditions (Fig. 3C). The P1 and P2 amplitudes—peaking on the electrodes over the occipital cortex, in agreement with previous results (Curran et al., 1993; Tucker et al., 1994; Mercure et al., 2008)—were significantly larger in the noise-present condition than in the other two conditions (main effect of condition F_(2,30) = 45.96, p < 0.0001 and F_(2,30) = 11.20, p = 0.004; post hoc p values for N-p vs C and N-p vs N-a both p = 0.0001 and p < 0.001 in the cases of P1 and P2, respectively). On the other hand, amplitudes of the N170 component—which in agreement with previous findings peaked on the electrodes over the temporal cortex (for review, see Jacques and Rossion, 2007)—were significantly reduced in the noise-present compared with the other two conditions (main effect of condition F_(2,30) = 68.78, p < 0.0001; post hoc p values for N-p vs C and N-p vs N-a both p = 0.0001). In a control experiment (see supplemental Results, available at www.jneurosci.org as supplemental material), we also confirmed that in accordance with previous results (Philiastides et al., 2006; Rousselet et al., 2007, 2008), the magnitude of the noise effect on the N1 and P2 ERP components depended on the amount of the phase noise added to the face images: decreasing phase coherence resulted in gradual decrease and increase of the N170 and P2 components, respectively (Fig 4A). In addition, in our main experiment, we also found a hemispheric asymmetry in the noise-induced modulation of the ERP responses to the face stimuli: in the case of the N170 and the P2 components, the effect of noise was more pronounced over the right hemisphere (condition × side interaction: F_(2,30) = 20.70, p = 0.0001 and F_(2,30) = 4.48, p = 0.044 in the cases of N170 and P2, respectively). This appears to be in agreement with the large amount of previous results showing lateralization of neural processes associated with face processing to the right hemisphere (Puce et al., 1995; Kanwisher et al., 1997; Letourneau and Mitchell, 2008). Together these results imply that it is the presence of noise and not overall task difficulty that affects the early ERP responses, including the P2 component.

Figure 4.

Grand-average ERPs of the control experiment. In the case of the gender categorization task, there was a clear modulation of the P2 amplitude with changing the phase coherence of the images (A), while there was no modulation when changing the difficulty of the decision while keeping the phase coherence constant in the tilt discrimination task (B) (N = 14).

To provide further support for this conclusion, we performed a control experiment aimed at investigating the effect of task difficulty on the P2 component in the presence of noise. Even though the results of the main experiment provided direct evidence that the P2 component is not modulated by task difficulty in the absence of noise, it remained to be shown whether the same holds in the case of noisy images. In fact, the results of a previous study suggested otherwise when showing that the amplitude of the P2 component in response to low-phase-coherence images was reduced, similar to that evoked by the images with high phase coherence when subjects were required to attend to the color of the stimuli and perform an easy color categorization task as opposed to a difficult face versus car categorization with the same stimuli (Philiastides et al., 2006). Since in the case of phase-randomized face images, gender categorization performance can be modulated only in a very limited range, in our control experiment, we used a face tilt discrimination task. Subjects were required to make decisions about the tilt direction of rotated noisy faces (see supplemental Fig. S1, available at www.jneurosci.org as supplemental material). We manipulated difficulty by changing the degree of rotation but keeping the phase coherence of the images constant. Importantly, the results revealed that the P2 component was not affected by the difficulty of the task (Fig. 4B) (main effect of noise: F_(2,26) = 0.16, p = 0.69). At first, these findings appear to be at odds with the results of the study of Philiastides et al. (2006) showing task-dependent modulation of the P2 component. However, in this earlier study, difference in the P2 component was found between two conditions, which differed not only in task difficulty but also in the performed task: a difficult object categorization task versus an easy color categorization task. Therefore, it is not known whether the observed modulation of the P2 component in this experiment is due to the difference in overall task difficulty or due to the fact that in the color task, object information was irrelevant and thus not attended, leading to diminished noise processing in the visual cortical areas. In our control experiment, on the other hand, subjects' task was the same in the easy and difficult conditions. Moreover, to perform the face tilt discrimination task, shape-specific information (e.g., eye, mouth, and nose contours and their relative orientation) had to be extracted and processed in both conditions. Therefore, the lack of task-difficulty effects on the P2 component in our control experiment is in agreement with the results of the main experiment and suggests that modulation of the P2 in the Philiastides et al. (2006) study was due to the difference in the task conditions rather than overall task difficulty.

Results of intertrial synchrony analysis

To further investigate the effect of noise on the sensory processing of face stimuli, we performed an intertrial synchrony analysis on the EEG data obtained in the main experiment. The ITC provides a measure for the degree of phase locking for a specific oscillation to a stimulus, and it has been shown that phase-locked oscillations might modulate ERP components (Klimesch et al., 2004; Hsiao et al., 2006; Klimesch et al., 2007; Rousselet et al., 2007; Freunberger et al., 2009). Furthermore, improved temporal coordination within and across task-related brain regions may be an important mechanism underlying fast and efficient behavioral performance (Ghuman et al., 2008; Gilbert et al., 2010). Results of the intertrial synchrony analysis for a representative electrode (PO8) are shown in the cases of noise-absent and noise-present conditions as spectrograms in Figure 5A. In agreement with previous results, ITC measured between 100 and 200 ms after stimulus was high following stimulus presentation for theta and alpha oscillations, peaking in the theta band. Since the effect of noise on the ITC of alpha band oscillations was similar but weaker than that found in the case of theta band and previous research also suggests that event-related phase-locked components of power are often coordinated between theta and alpha (for review, see Klimesch et al., 2008), here we focus our analysis on the theta frequency band. The scalp distribution of the ITC values showed that ITC in the theta band peaked on the electrodes over the ventral occipital–temporal cortex (Fig. 5A). Thus the scalp distribution of the theta band ITC values was similar to the distribution of the N170 ERP component. The time course of ITC values for the theta band and the three clusters of electrodes are plotted in Figure 5B. The results show that adding noise to the stimuli has a strong effect on the theta band oscillations. Two-way ANOVAs conducted on ITC values averaged within the 100–200 ms poststimulus time window revealed reduced theta band ITC in the noise-present trials compared with noise-absent and control trials. This was significant over all three electrode clusters (main effect of condition: F_(2,30) = 40.44, p < 0.0001; F_(2,30) = 42.03, p < 0.0001; and F_(2,30) = 18.18, p = 0.0005; post hoc p values for N-p vs C and N-p vs N-a both p = 0.0001, p = 0.0001, and p < 0.0002 for frontal, right, and left visual electrode clusters, respectively). Contrary to this result, theta band ITC in the noise-absent condition did not differ from those in the control condition (post hoc p values for N-a vs C p = 0.83, p = 0.85, and p = 0.99 for frontal, right, and left visual electrode clusters, respectively). Thus, in agreement with the ERP results, we found that phase locking of theta oscillations was affected by the presence of noise but not overall task difficulty. Furthermore, the scalp distribution and direction of modulation to the introduction of noise was found to be similar in the case of the N170 component and theta oscillation, suggesting that they might reflect the same mechanisms of noise-induced modulation of face processing. To further investigate this possibility, we tested the relationship between the effect of noise on the ERP components and on the theta oscillations. Spearman correlations showed that the magnitude of noise-induced modulation—i.e., the difference in the noise-present relative to the noise-absent conditions—in the theta band ITC strongly correlated with that of the N170 component in the right hemisphere (r = −0.78, p = 0.0005) (Fig. 5C). Whereas no correlation was found between the effect of noise on the theta band ITC and the noise-induced modulation of P1 and P2 components [in the case of ITC measured over the temporal visual cortical cluster, r = 0.30, p = 0.26 and r = −0.06, p = 0.83 for P1 and P2, respectively (Fig. 5C); in the case of ITC measured over the occipital visual cortical cluster, r = 0.42, p = 0.11 and r = 0.22, p = 0.41 for P1 and P2, respectively]. Together these results showed that adding noise to the face stimuli also affected the evoked low-frequency oscillations, with the strongest modulation found in the theta band. Furthermore, our findings also suggest that the effect of noise on face processing might consist of different components: one is reflected in the reduced theta band ITC and N170 amplitudes, whereas another component might be indexed by an increase in the P1 and P2 amplitudes.

Figure 5.

Grand average ITC. A, Spectrogram of a typical electrode (PO8—position indicated by a white dot on cartoon heads) showing the N-a and N-p conditions above and below, respectively. Spectrogram of the C condition was very similar to that of the N-a condition and is not shown. Insets show the ITC topographies averaged over 100–200 ms and 4–7 Hz as indicated by the black boxes. Black dots show the three electrode clusters for which ITC time courses were calculated (B). Black, C; blue, N-a; brown, N-p. C, Correlation between the theta ITC difference between N-p and N-a conditions and the respective amplitude differences of P1, N170, and P2 peaks (N = 16). Noise-induced modulation of the power of the theta band oscillations was analogous to that found in the case of ITC (see supplemental Fig. S3, available at www.jneurosci.org as supplemental material).

Results of fMRI experiment

The scalp distribution of the noise-induced modulation of the ERP responses and evoked oscillations suggested that noise might affect primarily the visual cortical processing of facial information. However, localization of the generators of the EEG activities based on the scalp electrodes is very problematic. Therefore, we performed an fMRI experiment to localize the noise-related neural processes using the same task conditions. Whole-brain random-effects analysis of the fMRI data revealed a bilateral network of occipital and temporal cortical areas (for coordinates and labels, see Table 1) that were more active using a p_FDR<0.05 threshold in the noise-present than in the noise-absent condition, with the strongest activations found bilaterally in the visual cortex, a region commonly referred to as lateral occipital complex (LOC) (Malach et al., 1995; Grill-Spector et al., 1999, 2000; Hasson et al., 2003; Larsson and Heeger, 2006). Importantly, the presence of phase noise did not affect the fMRI responses in the primary visual cortex, which is in agreement with previous findings, showing that phase randomization modulates neural responses to the visual objects beyond the primary visual cortex (Olman et al., 2004; Tjan et al., 2006; Schendan and Stern, 2007; Rousselet et al., 2008). In addition, we also found stronger bilateral inferior parietal cortical and ventral temporal cortical activation in the noise-present than in the noise-absent condition. On the other hand, the reversed contrast showed no significant activations. Furthermore, when contrasted with the control condition, the noise-present condition displayed higher activity in bilateral LOC, right inferior frontal cortex, and left superior parietal cortex. However, the extent of visual cortical activations when the noise-present condition was contrasted with the control was reduced compared to that found in the noise-present versus noise-absent contrast, suggesting that some of the visual cortical regions that were found activated in the latter contrast were due to the weaker fMRI responses in these regions in the noise-absent than in the control condition (also see ROI analysis in supplemental Results, available at www.jneurosci.org as supplemental material). Thus, to determine the cortical areas showing higher fMRI response in noise-present conditions than in both the difficult noise-absent and the easy control conditions, we conducted a conjunction analysis of the N-p > N-a and N-p > C contrasts (Fig. 6A). The only region having higher activity in the noise-present relative to both the noise-absent and control conditions was the most dorsal and posterior among the LOC activation regions (Table 1). This cortical region appears to be in close correspondence with the retinotopically organized LO2 region introduced in the study by Larsson and Heeger (2006) as well as with the object-selective DOT-LOS region described by Hasson et al. (2003). Importantly, we found no significant differences in the fMRI responses between the noise-absent and control conditions. The lack of an effect of task difficulty on the fMRI responses in the visual cortex in the absence of noise is in agreement with a previous study (Xu et al., 2007), showing that task difficulty failed to modulate visual cortical responses when subjects were required to discriminate visual images of scenes without noise. However, in the study by Xu et al. (2007) the more difficult task led to stronger activations in the anterior cingulate cortex and in the left insula, although these activations were shown to be significant only without correcting for multiple comparison. In fact, in agreement with these results, at an uncorrected (p < 0.0001) threshold, in the current study we found stronger activations in the noise-absent versus control contrast in the supplemental motor area (t₍₁₅₎ = 7.98; x, y, z = −6, 4, 52) extending into middle cingulated cortex and in the left insula (t₍₁₅₎ = 5.53; x, y, z = −36, 20, 2), but not in the visual cortex even with threshold lowered to p_unc < 0.001 similarly to the Xu et al. (2007) study. Furthermore, we also tested whether there were overlapping brain regions that are more active in the two difficult conditions as compared to the control and thus might be involved in processes related to overall task difficulty. A conjunction analysis on the fMRI data revealed two activation clusters at an uncorrected (p < 0.0001) threshold (Fig. 6A), which were located in the right opercular inferior frontal cortex (t₍₃₀₎² = 5.10; x, y, z = 52, 16, 8) and in the left insula (t₍₃₀₎² = 4.60; x, y, z = −34, 24, 2), which is in agreement with the growing body of experimental results showing the involvement of these cortical regions in perceptual decision processes under high stimulus uncertainty (Grinband et al., 2006; Pleger et al., 2006; Philiastides and Sajda, 2007; Xu et al., 2007; Tosoni et al., 2008; Ho et al., 2009; Kovács et al., 2010). It is also in accordance with the results of previous studies suggesting that these regions form part of the executive attention network (Fan et al., 2005) and play a general role in the implementation of goal-directed task sets (Dosenbach et al., 2006).

Table 1.

MNI coordinates of significant fMRI activations

	MNI coords (x, y, z)	t(15) value	Cluster size	Area
N-p > N-a	32, −84, 8	9.00	1074	R Mid occipital (DOT-LOS)
	26, −88, −5	8.75		R Inf occipital
	−30, −90, 10	7.11	660	L Mid occipital (DOT-LOS)
	−26, −82, −10	6.23		L Inf occipital
	−28, −60, −10	4.80		L fusiform (VOT-mpFs)
	50, −56, −10	6.69	140	R Inf temporal
	32, −44, −16	5.24	33	R fusiform (VOT-mpFs)
	−54, −46, 38	5.14	23	L Inf parietal
	54, −50, 42	4.89	27	R Inf parietal
	−46, −48, 48	4.81	17	L Inf parietal
N-p > C	−34, −88, 2	10.45	137	L Mid occipital (DOT-LOS)
	52, 20, 10	7.20	28	R Inf frontal (Oper)
	34, −86, 10	7.03	60	R Mid occipital (DOT-LOS)
	−26, −60, 50	6.44	15	L Sup parietal
N-p>N-a & N-p>C	32, −86, 6	t(30)2 = 6.14	17	R Mid occipital (DOT-LOS)
	−34, −88, 4	t(30)2 = 5.55	10	L Mid occipital (DOT-LOS)

Significance level for all clusters listed is p_FDR < 0.05. coords, Coordinates; L, left; R, right; Inf, inferior; Mid, middle; Sup, superior; Oper, opercular.

Since previous research showed that there was a high individual variance in the location of the high-level, shape-specific visual cortical areas, we also performed an ROI-based analysis of the fMRI data in the object-selective (DOT-LOS, DOT-ITS, and VOT-mpFS) and face-selective (OFA and FFA) regions in the visual cortex (for ROI definition, see Materials and Methods). In agreement with the results of the conjunction analysis, the ROI analysis (Fig. 6B) revealed significantly higher activations only in the bilateral DOT-LOS in the case of the noise-present as compared to both the noise-absent and control conditions (main effect of condition: F_(2,30) = 17.09, p < 0.0001; p values for N-p vs C and N-p vs N-a both p < 0.0005), revealing that the higher activation of this most posterior and dorsal LOC subregion was indeed linked to the presence of phase noise in the stimuli and not to task difficulty per se. It was also found that the fMRI response was higher in the right than in the left DOT-LOS (main effect of hemisphere: F_(1,15) = 5.97, p = 0.027). Importantly, the fMRI response in bilateral FFA did not differ significantly among the three conditions (main effect of condition: F_(2,30) = 1.33, p = 0.28). For results of the OFA, DOT-ITS, and VOT-mpFs, see supplemental Results (available at www.jneurosci.org as supplemental material). The lack of noise effects on the fMRI responses in the face-specific visual areas of the inferior occipital and temporal cortex appears surprising in the light of the observed noise-induced decrease in the N170 amplitudes, since previous research showed that these areas are implicated in the generation of the face-specific N170 component (Linkenkaer-Hansen et al., 1998; Caldara et al., 2004; Deffke et al., 2007). A possible explanation for this observed discrepancy could be that our results showed that the N170 amplitude reduction at least in part is due to the reduced intertrial phase synchrony in the theta band oscillations in the presence of noise. Such modulation would not manifest itself in the fMRI responses, which lack the high time resolution of EEG. Together the fMRI results are consistent with the results of the EEG recordings by showing that the presence of noise, not overall task difficulty, is what affects visual cortical processing.

Discussion

The goal of the present study was to dissociate the effect of noise on sensory processing from that on the processes associated with overall task difficulty by recording EEG and measuring fMRI responses separately in the same task conditions. Our electrophysiological results revealed that noise-related neural processes affect the early P1, N170, and P2 components of the ERP responses as well as the intertrial synchrony of the low-frequency oscillations in the first 300 ms following stimulus onset. In particular, noisy stimuli led to enhanced amplitudes of the P1 and P2 components, which is in agreement with the results of earlier research (Tarkiainen et al., 2002; Philiastides et al., 2006; Rousselet et al., 2007, 2008; Schendan and Lucia, 2010). In addition to the modulation of P1 and P2 components, adding phase noise to the face stimuli decreased the amplitudes of the N170 ERP component as well as the phase synchrony in the theta band oscillations over the ventral occipitotemporal electrodes. These findings are again consistent with the previous research showing that N170 amplitudes are strongly reduced in the presence of both phase and white noise (Jemel et al., 2003; Tanskanen et al., 2005; Schneider et al., 2007; Rousselet et al., 2008) as well as with the earlier results suggesting that the N170 component is strongly associated with the evoked oscillations in the theta and alpha band (Hsiao et al., 2006; Rousselet et al., 2007; Smith et al., 2007; Hansen et al., 2010).

Furthermore, our fMRI results revealed stronger BOLD responses in the bilateral DOT-LOS, a posterior and dorsal portion of the LOC, when subjects categorized noisy faces as compared to the categorization of intact faces. This is in agreement with the putative role of LOC in object segmentation and grouping (Lerner et al., 2001; Kourtzi and Kanwisher, 2001) (for review, see Grill-Spector et al., 2001), perceptual closure (Sehatpour et al., 2006) and object recognition processes for illusory contour stimuli (Murray et al., 2002). Furthermore, based on its coordinates, this area corresponds to the shape-selective, retinotopically organized LO2 area introduced by Larsson and Heeger (2006), which represents shape information within a spatial coordinate system, enabling this region to play an important role in organization processes that rely on spatial relations in the visual image (e.g., segmentation, grouping, and region extraction). This is in line with the disruptive effects of phase noise, since the phases in an image carry location information, which in turn specifies object shape in terms of the spatial locations of features (Oppenheim and Lim, 1981; Bennett and Banks, 1987), and this feature location information is crucial for categorizing objects (Cave and Kosslyn, 1993). It is important to note, however, that in apparent contradiction to our increased LOC activation in the presence of phase noise, previous research showed that degrading the shape-specific information of images—e.g., by adding phase noise or white noise to the images—results in decreased fMRI responses in the LOC as compared to responses to the unmanipulated images (Malach et al., 1995; Rose et al., 2005; Bingel et al., 2007). The important difference, however, was that in these studies degraded images were task-irrelevant. Thus, reduced fMRI responses in the high-level object-selective areas in the presence of phase noise might primarily reflect decreased feedforward, automatic neural responses. However, when the task conditions require fine discrimination of a specific visual object feature—as was the case in the current study—adding noise to the stimuli will lead to increased processing demands in the regions specialized for the processing of the given feature, and this might mask the noise-induced automatic response reduction or even lead to increased activation, depending on the relative strength of the two components. Such an explanation appears to be in agreement with recent findings showing that in a motion direction discrimination task, fMRI responses in the motion direction-selective human MT+ are increased with decreasing the coherence of the global motion signal (i.e., increasing the noise) in the stimuli (Ho et al., 2009; Kayser et al., 2010). Since previous source localization results suggested that the neural generators of both P1 and P2 components might involve the lateral occipital cortex (for review, see Schendan and Lucia, 2010), an important task for future research is to uncover the relationship between the noise-related neural events reflected in the P1 and P2 and those leading to increased fMRI responses in the LOC. Another question that remains to be explored concerns the specific neural processes associated with the processing of different types of noise. For example, previous research suggests that white noise—in contrast to phase noise—might affect the processing of visual stimuli already in the early visual cortical areas, including the primary visual cortex (Tanskanen et al., 2005). Together, our electrophysiological and fMRI results show that increased sensory processing demand as a result of adding phase noise to the stimuli involves a cascade of visual cortical processes in the first ∼300 ms after the onset of the stimuli. The critical novel aspect of these results is that these noise effects are not confounded by the neural processes related to overall task difficulty.

Computational models (Smith and Ratcliff, 2004; Beck et al., 2008) and experimental studies in humans as well as in nonhuman primates (Heekeren et al., 2004) [for review, see Glimcher (2003), Gold and Shadlen (2007), and Heekeren et al. (2008)] suggest that the neural events underlying visual perceptual decision making consist of different processing modules. The first involves extraction and representation of sensory evidence in the visual cortical areas. In a later stage, sensory evidence is integrated and decision variables are computed in a network of frontal–parietal areas. In addition, recently it was proposed that there is also a task-difficulty component of decision making, which might involve a network of frontal cortical areas responsible for detection of perceptual uncertainty as well as allocation of attentional resources (Philiastides et al., 2006; Heekeren et al., 2008). This task-difficulty component is thought to take place immediately after the early extraction of sensory information and is specifically reflected in the P2 component of the event related potentials, peaking ∼220 ms after stimulus onset (Philiastides et al., 2006). However, in the previous research, neural processes associated with the monitoring of overall task difficulty were confounded by those associated with the increased sensory processing demands as a result of adding noise to the stimuli. Therefore, it was not known to what extent the modulation of the P2 component reflected one or both of these two processes, and as a result, it remained an open question whether noise-induced modulation of the P2 component might be considered a marker of a task-difficulty component of decision making between the early extraction and the later integration of visual information. In the current study, we provided the first evidence that the noise-induced modulation of both the early electrophysiological responses—including the P2 component—reflects increased sensory processing of the noisy stimuli and not the processes associated with overall decision difficulty. In particular, our results suggest that the noise-related processes associated with the increased amplitudes of the P2 component might involve enhanced neural processing in the LOC in the case of the noisy stimuli. Therefore, our results do not provide support for the proposed task-difficulty component of decision making that was thought to take place between the early sensory processing and the later accumulation stages, involve a network of frontal cortical areas, and be indexed by the P2 component of the ERP responses (Philiastides et al., 2006; Philiastides and Sajda, 2007; Heekeren et al., 2008). Instead, our findings are in agreement with the results of previous electrophysiological research on object processing (Halit et al., 2000; Itier and Taylor, 2002; Latinus and Taylor, 2005; Kotsoni et al., 2007; Schendan and Kutas, 2007; Mercure et al., 2008; Schendan and Lucia, 2010), showing that increasing sensory processing demands (e.g., by masking the task relevant stimuli) will trigger reentrant visual cortical processing mechanisms that follow the early sensory extraction of the visual information, which are reflected in the P2 component of the ERP responses. Furthermore, in agreement with previous findings, our fMRI results also show that the neural processes associated with overall task difficulty involve a network of frontal cortical areas, including the right inferior frontal cortex and the left insula (Grinband et al., 2006; Pleger et al., 2006; Philiastides and Sajda, 2007; Xu et al., 2007; Tosoni et al., 2008; Ho et al., 2009; Kovács et al., 2010). Although the related ERP results in the current study failed to reach the significance level, they suggest that the neural processes associated with overall task difficulty might take place in a late time period, starting ∼400 ms after stimulus onset.

By using both electrophysiological and fMRI methods, we were able to show that the noise-induced modulation of the P2 component of the ERP responses reflect increased processing of noisy stimuli in a network of visual cortical areas and thus cannot be considered evidence for the existence of a task-difficulty component taking place between the early sensory processing and the later sensory accumulation stages of perceptual decision making.