Perceptual experience modulates cortical circuits involved in visual awareness

Abstract

Successful interactions with the environment entail interpreting ambiguous sensory information. To address this challenge it has been suggested that the brain optimizes performance through experience. Here we used functional magnetic resonance imaging (fMRI) to investigate whether perceptual experience modulates the cortical circuits involved in visual awareness. Using ambiguous visual stimuli (binocular rivalry or ambiguous structure-from-motion) we were able to disentangle the cooccurring influences of stimulus repetition and perceptual repetition. For both types of ambiguous stimuli we observed that the mere repetition of the stimulus evoked an entirely different pattern of activity modulations than the repetition of a particular perceptual interpretation of the stimulus. Regarding stimulus repetition, decreased fMRI responses were evident during binocular rivalry but weaker during 3D-motion rivalry. Perceptual repetition, on the other hand, entailed increased activity in stimulus-specific visual brain regions: for binocular rivalry in the early visual regions and for ambiguous structure-from-motion in both early as well as higher visual regions. This indicates that the repeated activation of a visual network mediating a particular percept facilitated its later re-activation. Perceptual repetition was also associated with a response change in the parietal cortex that was similar for the two types of ambiguous stimuli, possibly relating to the temporal integration of perceptual information. We suggest that perceptual repetition is associated with a facilitation of neural activity within and between percept-specific visual networks and parietal networks involved in the temporal integration of perceptual information, thereby enhancing the stability of previously experienced percepts.

Introduction

Sensory and perceptual experiences influence the way we interpret new sensory input (Gilbert et al., 2001; Karmarkar & Dan, 2006). In most paradigms prior sensory stimulation and prior perceptual experience are difficult to disentangled. Here, we discern these influences using ambiguous visual stimulation. In particular, we ask whether prior perceptual experience alters the neural processing of ambiguous signals in the brain. The dissociation from sensory stimulation is permitted by occasional alternations in the perceptual interpretation of an ambiguous stimulus, while the sensory stimulation remains unchanged (Blake & Logothetis, 2002).

When short presentations of an ambiguous stimulus are interleaved with blank intervals, one of the possible interpretations of the stimulus tends to be perceived repeatedly on consecutive presentations (Orbach et al., 1963; Leopold et al., 2002). As shown by psychophysical investigations, this stability in perception cannot be explained as a resistance to change (Brascamp et al., 2008, 2009) or as repetition priming from one presentation to the next (Maier et al., 2003; Pearson & Brascamp, 2008). Rather, it reflects a form of perceptual memory that spans a timescale of at least several minutes and can be understood as a tendency to experience the perceptual interpretation that was most prevalent in the recent past (Pearson & Brascamp, 2008).

Repeated sensory stimulation is usually associated with a decrease in the amplitude of the neural response, which is ascribed to e.g. neural fatigue or more efficient encoding (Grill-Spector et al, 2006; Krekelberg et al., 2006; Kohn, 2007). Conversely, perceptual learning/memory can lead to an increased neural response, which is suggested to result from an increase in neural sensitivity (Miller et al., 1996; Dolan et al., 1997; Henson et al., 2000; Kourtzi et al., 2005; James and Gauthier, 2006; Turk-Browne et al., 2007). In line with this, we hypothesized that perceptual repetition during intermittent presentation of an ambiguous stimulus is associated with an increased neural response, while the mere repetition of the stimulus leads to a decreased neural response. These effects could be present in sensory regions specialized for the presented stimulus (Henson et al., 2000; Kourtzi et al., 2005) and/or in frontal and parietal regions involved in the attentional and mnemonic processing of sensory information (Miller et al., 1996; Corbetta et al., 2002; Rees et al., 2002; Pasternak & Greenlee, 2005).

To test our hypotheses, we used two different ambiguous stimuli: binocular rivalry and ambiguous structure-from-motion (SFM), also referred to as three-dimensional (3D-) motion rivalry (Figure 1). During binocular rivalry the proposed modulation of sensory neurons may be primarily observed in the early visual cortex (Visual area (V) 1-3; Gail et al., 2004; Haynes & Rees, 2005; Lee et al., 2005), while during 3D-motion rivalry these effects could be present in ventral visual regions implicated in 3D-shape processing (V4, LO (see Figure 2 for abbreviations); Kourtzi et al., 2003; Neri, 2005; Hinkle & Connor, 2005; Preston et al., 2008) or dorsal and parietal regions implicated in global motion in depth (hMT+, V3A, V7, POIPS (see Figure 2 for abbreviations); Paradis et al., 2000; Orban et al., 2006; Brouwer & van Ee, 2007; Brascamp et al., 2010; Figure 2).

Figure 1. Stimuli and paradigm.

A) We used either of two ambiguous stimuli. During binocular rivalry a perceptual conflict arises because the images presented to the left and right eye are incompatible. During 3D-motion rivalry (structure-from-motion without stereoscopic disparity) there is a conflicting depth-cue in the image. Via button presses the participants indicated which of the two possible percepts they perceived at any given time (leftward or rightward tilt during binocular rivalry; leftward or rightward rotation direction during 3D-motion rivalry). The changes in perception that occur when ambiguous visual input is presented enabled us to dissociate the neural effects of stimulus repetition from those of perceptual repetition (see D). B) Sequence of events during a block. A block consisted of an 18 seconds intermittent stimulation epoch (including 12 stimulus presentations) followed by a 16 seconds rest period. C) Sequence of events during an experimental run. One experimental run lasted 14 minutes and consisted of 24 blocks. Participants completed at least 4 runs per experiment. The ambiguous stimulus was the same throughout the run (left graph), but perceptual experience changed in an oscillatory fashion with periods of largely stable perception that lasted several minutes (middle graph). We used a ‘leaky integrator’ filter of the perceptual timecourses as a straightforward and validated tool to identify the early and late phases of these periods of stabilized perception (see methods). D) We hypothesized that the neural response, as measured with fMRI, is concurrently modified by both stimulus repetition and perceptual repetition. To investigate the influence of stimulus repetition we compared the first and second part of each experimental run (abbreviated ‘exp. run’). The influence of perceptual repetition was studied by comparing early and late stages of perceptually stable periods (abbreviated ‘stab. period’). Each block belonged to either one of the levels of stimulus repetition (early/late in exp. run, shown in open/filled grey circles) and either one of the levels of perceptual repetition (early/late in stab. period, shown in open/filled black circles). In this way, the effects of one measure were averaged out when the levels of the other measure were compared.

Figure 2. Regions-of-interest.

A) Schematic drawing of the locations of the ROIs. Colors indicate different cortical regions. V1, visual area 1; V2, visual area 2; V3V, ventral part of visual area 3; V3D, dorsal part of visual area 3; V4, ventral visual area 4; LO, lateral occipital area; hMT+, motion-selective medio-temporal area; V3A, visual area 3A (anterior to V3); V7, visual area 7; PIPS, posterior intraparietal sulcus; AIPS, anterior intraparietal sulcus; POIPS, parieto-occipital intraparietal sulcus (structure-from-motion sensitive parietal region); SMA/SEF, supplementary motor area / supplementary eye-fields; FEF, frontal eyefields; IFJ, inferior frontal junction; DLPFC, dorsolateral prefrontal cortex. B) An example (right) hemisphere showing the delineation of the ROIs. C) Table presenting Talairach coordinates averaged over participants.

Materials and Methods

Participants

Six observers (5 male, 1 female) participated in the ‘3D-motion rivalry’-experiment. A subset from these and one additional male observer participated in the ‘binocular rivalry’-experiment (total: 4 male, 1 female). All participants had normal or corrected to normal vision and gave written informed consent prior to participation. The study was approved by the ethics committee at the University of Birmingham, United Kingdom, and conforms to The Code of Ethics of the World Medical Association (Declaration of Helsinki).

Stimuli and Procedure

Stimuli were created using Mathematica (Wolfram Research) and Matlab (MathWorks Inc) and were presented in the center of a gray computer-screen (60 Hz, 1280×1040 pixels, gamma-linearized). For binocular rivalry we used two orthogonal sine wave black-and-white gratings, each presented to one of the eyes, on a mid-grey background (Figure 1A). The gratings subtended a circular patch of 2.9° in diameter and contained 1.38 cycles per degree (phase was chosen randomly at every presentation). The gratings were titled 45 degrees from vertical to either the left or right. Per experimental run the orientations were assigned randomly to one of the eyes. During binocular rivalry the perceptual conflict between the eyes results in the alternating dominance of either the left or right eye grating (Levelt, 1967).

For 3D-motion rivalry we used an ambiguous SFM stimulus that consisted of 175 leftward-moving and 175 rightward-moving dots (each 0.064° in diameter, depicted either in black or in white, on a mid-grey background) representing random points on the surface of a virtual globe (2.9° in diameter). The sinusoidal speed profile of the dots (fastest near the vertical meridian) created the percept of a globe revolving around its vertical axis with a period of 6.7 seconds. The three-dimensional interpretation of the stimulus, and thereby its direction of rotation, was ambiguous, because no depth cues differentiated the rightward moving surface from the leftward moving surface. Either of the two surfaces could thus be perceived in front of the other (Figure 1A).

At every stimulus presentation the participants reported their percept by pressing one of two corresponding buttons, and pressed no button when they could not differentiate the two percepts (for example, when they had a mixed percept or a transition between percepts within one presentation of the stimulus). Participants were instructed to maintain strict fixation on a centrally presented, static fixation dot subtending 0.19° in diameter (globe: green dot; gratings: red dot with grey circular surround of 0.38°). An experimental run consisted of 24 blocks. A block started with intermittent presentation of the stimulus (for 18 seconds) followed by a blank screen (for 16 seconds; Figure 1). The period with intermittent presentation contained 12 presentations (lasting 900 milliseconds each) interleaved with short blanks (lasting 600 milliseconds each). Each run started and ended with an additional rest period (blank screen lasting 16 seconds). During the rest periods the fixation dot remained visible. Each participant completed 4 runs per experiment (except 2 participants in the ‘binocular rivalry’-experiment who completed 5 runs). Blocks with less than 8 (out of 12) button responses were excluded from the analyses (this amounted to 3.3% of blocks).

Perceptual History and Design of Analysis

We investigated the influence of perceptual repetition as well as stimulus repetition on functional magnetic resonance imaging (fMRI) responses. Stimulus repetition was studied by comparing blocks early in each experimental run (the first half) with later blocks in that run (the second half), reasoning that effects of stimulus repetition may accumulate during a run. While the stimulus remained the same across a run, perception changed occasionally. Observers tended to experience the same percept for prolonged periods of time, a ‘perceptually stable period’, after which perception would switch and the other percept would be experienced during the next period. Perceptual repetition, i.e. repeatedly seeing the same perceptual interpretation, was studied by comparing blocks early in a perceptually stable period (after few repeats) with blocks late in a perceptually stable period (after many repeats). This approach was based on the idea that memory for the perceptual interpretation accumulates during such periods (Figure 1C and 1D), given previous findings that the tendency to experience the same percept across repetitions of an ambiguous stimulus grows as the same percept is seen over and over (Brascamp et al., 2008), i.e. it is a self-reinforcing tendency.

Whereas it was evidently easy to divide each experimental run into two halves to investigate the effects of stimulus repetition (first 12 and last 12 blocks), it was not possible to simply divide the perceptually stable periods into two halves, as these depended on the individual perceptual timecourses of the participants. The perception-based periods could differ in duration and did not always have an instantaneous beginning or ending (Figure 1C). We used a low-pass perceptual filter from Brascamp et al. (2009) as a tool to identify perceptually stable periods in an objective and validated way. The only assumption of this simple perceptual filter is that perceptual experience with a certain percept accumulates over time when the percept is seen, and slowly decays when the percept is not seen. The final measure of perceptual experience (memory) is the difference in experience between the dominant and the suppressed percept. Brascamp et al. (2009) have shown that this low-pass filter of perceptual timecourses accurately describes the long-term dynamics of perceptual stabilization during intermittent ambiguous perception. Its slow dynamics seem appropriate in relation to the sluggishness of the blood oxygen level-dependent (BOLD) response (note that the filter does not involve the fast dynamics of the model mentioned in Brascamp et al., 2009).

Specifically, the accumulated experience (E) of a given percept was calculated per block by ‘leaky’ integration of the proportion of presentations that percept was experienced (P) during that block: E = Eprev + P - (0.2*Eprev). Eprev refers to the E of the previous block. This measure of perceptual experience, E, was calculated separately for each of the two percepts (i.e. a given rotation-direction for the rotating globe; a given tilt-direction for the orthogonal gratings). This method takes into account recent percepts (P) as well as percepts longer ago (reflected in Eprev). Following Brascamp et al. (2009), we used the difference between the E value associated with the dominant percept and the E value associated with the suppressed percept as the final measure of perceptual experience E_final. Late in a perceptually stable period, when the current percept was also frequently seen in the past, E_final has a high value. Alternatively, early in a perceptually stable period, when the current percept was seen just a few times or when the opposite percept was seen frequently, E_final is small or negative.

We compared blocks early in a perceptually stable period with blocks late in a perceptually stable period (low vs. high values of E, overall median split; Figure 1C and 1D). We used a median-split, because this is a straightforward method that is free of assumptions regarding the shape of the relation between the BOLD signal change and the value of E and we had no a priori assumption regarding the detailed shape of this relation. Also, a median-split approach is not affected by possible serial correlations in the time series data and it is less sensitive to outliers than, for example, a linear regression analysis (a similar split method was used by Brascamp et al., 2009). When the values of E are distributed equally throughout the experimental run the differences related to stimulus repetition (i.e. the difference between the first and second half of an experimental run) are averaged out when the effects of perceptual repetition (i.e. low vs. high values of E) are investigated. Also, with an equal distribution the effects of perceptual repetition are averaged out when the effects of stimulus repetition are investigated. Given that several changes in perception occurred during an experimental run, the distribution of the blocks is likely to be fairly equal. To verify this we compared the number of blocks in which both tested variables (i.e. perceptual repetition and stimulus repetition) had the same value (i.e. both early or both late) with the number of blocks in which the tested variables had different values (early for one, late for the other). The resulting ‘relatedness index’ ((same - different) / (same + different)) is zero with equal distribution and positive or negative when the tested variables are positively or negatively related, respectively.

fMRI Data Acquisition and Analysis

A 3-Tesla Philips Achieva scanner at the Birmingham University Imaging Centre was used. T2*-weighted functional (2.5×2.5×3 mm resolution) and T1-weighted anatomical (1×1×1 mm resolution) data were collected with an eight-channel SENSE head coil. Echo planar imaging data (EPI, gradient echo-pulse sequences) with occipital, parietal and frontal coverage were acquired (repetition time was 2,000 milliseconds; echo time was 35 milliseconds). The number of slices was 32 for the binocular rivalry experiment and 32 (3 subjects) or 29 (3 subjects) for the 3D-motion rivalry experiment.

Preprocessing of functional data was performed using Brain Voyager QX (Brain Innovations BV) and included slice scan-time correction (cubic spline, ascending interleaved), head movement detection (trilinear) and correction (trilinear for 3D-motion rivalry experiment, trilinear/sinc for binocular rivalry experiment) and temporal high-pass filtering (2 cycles). No spatial smoothing was performed. For each participant, the functional imaging data between runs were co-aligned automatically and then manually aligned to anatomical data. All data were transformed to Talairach space and anatomical data were used for 3D cortical reconstruction, inflation and flattening. Matlab (MathWorks Inc) was used for further analysis of the averaged MRI timecourses per region-of-interest (ROI).

Regions-of-interest

We identified retinotopic visual areas V1, V2, ventral V3 (V3V), dorsal V3 (V3D), V3A (which is anterior to V3D), V7 and V4 using standard rotating-wedge mapping procedures and in accordance with known anatomical structures (Engel et al., 1994; Sereno et al., 1995; DeYoe et al., 1996; Figure 2). V7 was defined as a region anterior and dorsal to V3A (Tootell et al., 1998; Press et al., 2001; Tyler et al., 2005; Figure 2A). Three additional functional localizers were performed, all using a conventional block design. Motion-sensitive medial temporal area (hMT+, also known as V5) was defined as the set of voxels in the temporal cortex that responded significantly higher (p<10^-4) to a coherently moving array of dots than to a static array of dots (Zeki et al., 1991; Tootell et al., 1995). The lateral occipital area (LO) was defined as the set of voxels in lateral occipito-temporal cortex which responded more strongly (p<10^-4) to intact than to scrambled images of objects (Kourtzi and Kanwisher, 2000). Region ‘POIPS’, named according to its anatomical location in the parieto-occipital intraparietal sulcus (Figure 2A; Vanduffel et al., 2002; Orban et al., 2006), was defined as the set of voxels in the superior parietal lobule that responded more strongly (p<10^-4) to SFM than to motion without perceived depth. The stimuli used were moving random-dot patterns that either did or did not contain SFM cues. The stimuli had no stereoscopic depth and consisted of black and white dots on a mid-grey background. The participants completed two scanning runs, which lasted nearly 6 minutes each. During each run eight 18 s blocks with SFM, eight 18 s blocks without SFM and four 12 s blank fixation blocks were presented in pseudo-random order. The task of the participants was to report changes in the luminance of the fixation dot in the center of the stimulus, which occasionally changed from black to white or vice versa.

Based on previous studies (Kleinschmidt et al., 1998; Lumer et al. 1998; Constantinidis et al., 2001; Rees et al., 2002; Corbetta et al., 2002; Grefkes & Fink, 2005; Pasternak & Greenlee, 2005; Asplund et al., 2010; Sterzer & Kleinschmidt, 2010; Zanto et al., 2010) we may expect that memory for perceptual history also involves frontal and parietal regions involved in attentional and mnemonic processing of sensory information, particularly the anterior and posterior intraparietal sulcus (AIPS and PIPS), the supplementary eye-fields / supplementary motor area (SMA/SEF), the frontal eye-fields (FEF), the inferior frontal junction (IFJ), the dorso-lateral prefrontal cortex (DLPFC) and the insula (see Figure 2A). In accordance with the anatomical locations described in these previous studies, we defined each of these regions as the set of voxels, near the known anatomical location, that responded significantly more strongly to visual stimulation than to fixation (p< 0.05, Bonferroni corrected for total number of voxels; see table with Talairach coordinates in Figure 2C). In the binocular rivalry experiment we used the data from the 3D-motion rivalry experiment as a localizer of voxels with significantly stronger responses to visual stimuli than to fixation. Analogously, we used the data from the binocular rivalry experiment as a visual localizer for the 3D-motion rivalry experiment. By using independent datasets for localizing regions-of-interest (ROIs), we ensured statistical independence of the selection of the ROIs and the hypothesis testing per experiment. With the same method we also selected visually responsive voxels within each of the other ROIs (the retinotopic visual areas, hMT+, LO and POIPS) and included only those voxels in the data analysis.

There were two participants that only performed the 3D-motion rivalry experiment and not the binocular rivalry experiment. For these participants the data of the SFM functional mapping experiment served as an independent localizer of visually responsive voxels (stimuli with SFM and without SFM were collapsed; p<10^-4). This independent localizer was also utilized to define hMT+ in one of these participants and LO in both of them, because the corresponding localizers were not completed. In accordance with the average Talairach-coordinates of hMT+ and LO in the remaining subjects, the regions were defined as the set of voxels, near the target anatomical location, that responded significantly more strongly to stimuli than to fixation. One other participant performed only the binocular rivalry experiment and not the 3D-motion rivalry experiment. This participant completed an extra (5^th) run, which we utilized as an independent visual localizer (p< 0.05, Bonferroni corrected for total number of voxels) for hypothesis testing based on the remaining 4 runs.

BOLD Amplitude and Statistical Analysis

For each of the two levels (early and late) of perceptual repetition and stimulus repetition the event-related BOLD responses were calculated and referenced to the 2 MR-volumes preceding block-onset ((data–baseline) / baseline). We used this method, which is free of assumptions regarding the shape of the response, because we did not have a standard hemodynamic response to use as a model for the observed saddle-shape of the responses (see Figure 3B, 3D and 4B). Also, this method eliminates possible low-frequency fluctuations, because the activation is measured relative to the baseline activation just before block onset. The magnitude of the event-related responses was quantified as the integrated BOLD response (i.e. the area under the BOLD curve). The relative response change between the early and the late level ((late-early) / (late+early)) was statistically analysed for perceptual and stimulus repetition using PASW Statistics 18 (formerly SPSS Statistics). For statistical analysis the ROIs were grouped into 5 cortical regions: early visual (V1, V2, V3V, V3D), ventral occipital (V4, LO), dorsal occipital (V3A, V7, hMT+), parietal (POIPS, PIPS, AIPS) and frontal (SMA, FEF, IFJ, Insula, DLPFC) cortex. We performed a repeated-measures analysis of variance (ANOVA) over ROIs, with type of rivalry (binocular/3D-motion rivalry) and repetition modality (stimulus/perceptual repetition) as within factors and cortical region (early visual, ventral visual, dorsal visual, parietal and frontal) as a between factor (unless indicated otherwise). An α of 0.05 was adopted and a Greenhouse-Geisser correction was applied to all tests to correct for possible violations of sphericity.

Figure 3. Stimulus repetition suppression and perceptual repetition enhancement.

A) Modulations of the magnitude of the BOLD response related to stimulus repetition (black bars) and perceptual repetition (red bars) during binocular rivalry (left graph) and 3D-motion rivalry (right graph; ±SEM) in the early visual regions. Upward bars indicate repetition enhancement; downward bars indicate repetition suppression. In both experiments perceptual repetition was reflected in an enhancement of the BOLD response in early visual regions. * p< 0.05 (t-test across participants; depicted in for perceptual repetition and in grey for stimulus repetition). Abbreviations as in Figure 2A. B) Event-related BOLD responses for early visual region V1 during binocular (left graph) and 3D-motion rivalry (right graph), averaged over blocks in the early (= after a few repeats; dashed lines) and late (= after many repeats; solid lines) stage of a perceptually stable period (±SEM). In the early visual cortex there was perceptual repetition enhancement during binocular as well as 3D-motion rivalry. C) Repetition-related modulations of the magnitude of the BOLD response in the ventral visual regions (layout and colors as in A). In the ventral visual cortex perceptual repetition enhancement was only observed during 3D-motion rivalry and not during binocular rivalry. D) Event-related BOLD responses for ventral visual region V4, averaged over blocks in the early and late stage of a perceptually stable period (layout and colors as in B). Perceptual repetition enhancement was only observed during 3D-motion rivalry. E) Repetition-related modulations of the magnitude of the BOLD response in the dorsal visual, parietal and frontal regions (layout and colors as in A). While perceptual repetition was reflected in an enhancement of the BOLD response in stimulus-specific visual regions (see A-D), stimulus repetition resulted in an adaptation-like suppression of the BOLD response in higher-order regions, particularly during binocular rivalry.

Eye position recording and analysis

Eye positions were recorded for four participants during fMRI scanning of the 3D-motion rivalry experiment using a ASL 6000 Eye-tracker (Applied Science Laboratories) with a sample frequency of 60 Hz. Preprocessing was performed using the Eyenal software package (Applied Science Laboratories) and further analysis was performed using custom Matlab (Mathworks) software. We computed the number of blinks and saccades and the horizontal and vertical eye position during each block of intermittent stimulation. Because we wanted to compare the eye movement data to the event-related BOLD responses per block, we included blinks and saccades during the entire block, i.e. the short presentations within a block as well as the interleaved short blanks. Blinks were defined as periods in which no gaze position was recorded (no recognition of pupil or no coronal reflection) that lasted 100-400 milliseconds. Saccades were defined as periods with a rapid change of gaze position (velocities between 25-500 °/s) that lasted >2 sample points. The eye position during fixations was calculated over periods when the stimulus was present and no blink or saccade was detected. Eye positions were referenced per stimulus presentation to the mean over the 100-ms preceding stimulus onset (to remove drift).

Results

Behavioral results

Short presentations of ambiguous stimuli were interleaved with blank periods and participants were asked to indicate their percept at every presentation (Figure 1A and 1B). It is known that in such an intermittent paradigm perception tends to stabilize across repetitions of the stimulus. The participants indeed reported robust perceptual stabilization for prolonged periods of time, referred to as a ‘perceptually stable periods’, at the end of which perception switched and the other percept was experienced during the next period. The percentage of same percepts seen within a block was on average 81.2% (± 3.0% SEM) during the binocular rivalry experiment and 88.4% (± 2.9% SEM) during the 3D-motion rivalry experiment. Here, a ‘block’ refers to a sequence of 12 short presentations followed by a 16 seconds rest (Figure 1B). Perceptual stabilization across the rest periods separating the blocks, i.e. when the first percept of a given block is the same as the last percept of the previous block, was on average 69.0% (± 1.9% SEM) and 78.3% (± 5.8% SEM) for the binocular and 3D-motion rivalry experiments, respectively. The participants refrained from responding upon presentations where they could not differentiate the two percepts, for example because they experienced a mixed percept or a transition between percepts within one presentation of the stimulus. This occurred in 1.6% (± 0.7% SEM) and 3.0% (± 1.5% SEM) of the presentations for the binocular and 3D-motion rivalry experiments, respectively.

We had anticipated that the distinct and consecutive periods of perceptual stabilization for one or the other percept would result in a temporal de-correlation of stimulus repetition and perceptual repetition (Figure 1C). Indeed the number of a block, relative to the start of the run, was not correlated with the output value of the perceptual filter for that block (see next paragraph and Material and Methods; binocular rivalry: -0.032 ± 0.030 SEM, t₍₄₎= -1.1, p= 0.3; 3D-motion rivalry: 0.003 ± 0.034 SEM, t₍₅₎= 0.1, p= 0.9, slope of repeated-measures regression; Figure 5A). Stimulus repetition and perceptual repetition were thus not correlated in time, allowing separate investigations of these co-occurring phenomena.

Figure 5. Perceptual repetition and its psychophysical (behavioral) relation with stimulus repetition, within-block perceptual stabilization and eye movements.

A) Slopes (±SEM) of the regression between stimulus repetition, i.e. the number of a block relative to the start of the run, and perceptual repetition, i.e. the output value of the perceptual filter (see Figure 1C), for the individual participants (light grey bars) and the group of participants per experiment (dark grey bars, repeated measures regression). The results did not exhibit a consistent temporal relation between stimulus repetition and perceptual repetition on the group level, allowing separate investigations of these co-occurring phenomena (as explained in Figure 1D and performed in Figure 3). * p< 0.05 (vs. zero). B) The relation in time between stimulus repetition (Stim.), perceptual repetition (Percept.) and within-block perceptual stabilization, i.e. the percentage of same percepts within a block, for binocular rivalry (dark grey bars) and 3D-motion rivalry (light grey bars), as reflected in the relatedness index (see Materials and Methods; ±SEM). In both experiments perceptual repetition is positively related in time with the within-block perceptual stabilization (bars in the middle). Stimulus repetition was related with neither perceptual repetition (bars on the left; in line with A), nor the within-block perceptual stabilization (bars on the right). * p< 0.05 (vs. zero). C) Modulations of the magnitude of the BOLD response (±SEM) related to within-block perceptual stabilization per cortical region for binocular rivalry (dark grey bars) and 3D-motion rivalry (light grey bars). There was a modest suppression of the BOLD response in perceptually stable compared with unstable blocks, particularly in the frontal cortical region during binocular rivalry. Within-block perceptual stabilization was related in time with long-term perceptual repetition (see B), but the associated modulations of the BOLD response were markedly different (compare with Figure 3). * p< 0.05 (vs. zero). D) Eye movements (left graph) and fixations (right graph) in the early (dark grey bars) and late (light grey bars) stages of perceptually stable periods (±SEM). The left graph presents the average number of blinks and saccades per second during the intermittent presentation blocks (during stimulus presentations and short blanks). The right graph presents the horizontal and vertical eye position in degrees of visual angle (relative to baseline) during fixation on the stimulus. The early and late stages of a perceptually stable period did not differ in the frequency of eye movements or the eye position during fixation.

We reasoned that effects of stimulus repetition might accumulate during the experimental run. Therefore, we studied the effects of stimulus repetition by comparing blocks early and late in a run (first half vs. second half). Following a similar reasoning, we studied perceptual repetition by comparing the early and late stages of perceptually stable periods, as identified using a ‘leaky integrator’ perceptual filter (Figure 1C). The early and late stages of perceptually stable periods did not differ in the percentage of presentations where participants refrained from responding, indicating no difference in the occurrence of blended/mixed percepts (binocular rivalry: t₍₄₎= 0.2, p= 0.8; 3D-motion rivalry: t₍₅₎= 1.3, p= 0.2). Also, the number of blinks and saccades was similar (blinks: t₍₃₎= -1.1, p= 0.4; saccades: t₍₃₎= 0.7, p= 0.5), as was the eye position during fixation on the stimulus (horizontal: t₍₃₎= 1.1, p= 0.4; vertical: t₍₃₎= 1.0, p= 0.4; measured during the 3D-motion rivalry experiment; Figure 5D). The blocks in the early and late stages of perceptual repetition were equally distributed between the first and second halves of the runs, ensuring that effects of stimulus repetition were averaged out when the levels of perceptual repetition were compared, and the other way around (relatedness index: binocular rivalry: -0.08, t₍₄₎= -1.0, p= 0.4; 3D-motion rivalry: -0.06, t₍₅₎= -0.8, p= 0.4; see Material and Memethods for calculation of relatedness index; Figure 5B).

MRI results

We were interested in changes in the magnitude of the BOLD response under conditions of repeated perception of, and stimulation with, the same ambiguous stimulus. We hypothesized that the integrated BOLD response (reflecting the area under the BOLD curve) would be larger during the late than early stages of perceptual repetition (i.e. repetition enhancement). In contrast, for stimulus repetition we hypothesized an adaptation-like decrease of the integrated BOLD response in the late compared with the early part of the experimental runs (i.e. repetition suppression). On average the relative signal change ((late-early) / (late+early)) was indeed positive for perceptual repetition and negative for stimulus repetition (0.04 and -0.08, respectively; Figure 3), in line with these predictions. There were marked differences between binocular and 3D-motion rivalry, which will be described below.

Response changes during binocular rivalry

During binocular rivalry the influences of perceptual repetition and stimulus repetition clearly differed (F _{(1, 12)}= 212.4, p< 10^-8, effect of repetition modality) and the difference was similar in magnitude across the cortical regions (F _{(4, 12)}= 3.2, p= 0.06, no interaction between repetition modality and cortical region). However, the response changes in the early visual cortex were different from those in the other cortical regions (F _{(4, 12)}= 10.8, p< 0.001, effect of cortical region). Perceptual repetition enhancement was present only in the early visual cortex and not in any of the other cortical regions (early visual: t₍₃₎= 3.4, p< 0.05; other cortical regions: all p≥ 0.1), while stimulus repetition suppression was present in all but the early visual cortex (early visual: t₍₃₎= 1.5, p= 0.2; other cortical regions: all p< 0.05; t-tests over regions-of-interest per cortical region; Figure 3).

Response changes during 3D-motion rivalry

During 3D-motion rivalry the difference between the repetition modalities was also highly significant (F _{(1, 12)}= 60.4, p< 10^-5, effect of repetition modality) and there were no differences between the cortical regions (F _{(4, 12)}= 1.9, p= 0.2, no effect of cortical region; F _{(4, 12)}= 2.5, p= 0.1, no interaction between repetition modality and cortical region). Perceptual repetition enhancement was robust (t₍₄₎= 21.3, p< 10^-4), but the relative signal change related to stimulus repetition was not significant (t₍₄₎= = 1.4, p= 0.2, t-test over cortical regions; Figure 3). This indicates that during 3D-motion rivalry the magnitude of the perceptual repetition enhancement was similar across the cortical regions. This was in contrast to binocular rivalry, during which perceptual repetition enhancement was observed only in the early visual cortex (see above). As hypothesized, the perceptual repetition enhancement was thus more widespread during 3D-motion rivalry than during binocular rivalry (direct statistical comparison: F _{(1, 12)}= 38.6, p< 10^-4 effect of type of rivalry). Perceptual repetition enhancement was indeed reliable in the ventral visual regions V4 and LO as well as the early visual regions V1, V2 and V3V (all t₍₅₎≥ 2.7, all p< 0.05), while in the other regions-of-interest there was no real consistency across participants (all t₍₅₎≤ 2.1, all p≥ 0.09; t-tests over participants per region-of-interest; Figure 3). The perceptual repetition enhancement during 3D-motion rivalry was thus most robust in early visual and ventral visual cortex.

Response changes in the parietal cortex

In both experiments the event-related BOLD responses exhibited a characteristic ‘saddle-shape’, with a dip in activity around 15 seconds after stimulus onset (see BOLD curves in Figures 3 and 4). The amplitude of the response just before and just after this dip in activity could differ quite substantially, suggesting that activity levels changed not only across blocks, but also within blocks (see Figure 1B for definition of ‘block’). With regard to perceptual repetition the first and second part of the response (measured 6-14 and 16-24 seconds after block onset, respectively) had similar amplitudes in the early stages of perceptual stabilization, whereas in the late stages of perceptual stabilization the second part of the response was enhanced relative to the first part ((second - first) / (second + first); difference divided by sum to correct for differences in the overall magnitude of the response). This suggests that activity in the later stage of the block was enhanced (Figure 4B).

Figure 4. Perceptual repetition enhancement of the within-block response.

A) Within-block changes in the shape of the BOLD response (±SEM) as reflected in the difference in amplitude between the first and second part of the response. Upward bars indicate that the second part of the response was relatively enhanced in the late compared with the early stage of a perceptually stable period; downward bars indicate suppression of the second part of the response. Perceptual repetition enhancement of the within-block response change was found in the parietal cortex, suggesting additional neural processing when there is memory of stable perceptual history. * p< 0.05 (t-test across participants). Abbreviations as in Figure 2A. B) Event-related BOLD responses for the anterior intraparietal sulcus (region AIPS) during binocular and 3D-motion rivalry (graphs are labeled with an icon of the corresponding stimulus), averaged over blocks in the early (= after a few repeats; dashed lines) and late (= after many repeats; solid lines) stage of a perceptually stable period (±SEM). For both types of rivalry, the first and second part of the within-block response had similar amplitudes in the early phase of a perceptually stable period, whereas in the late stage of a perceptually stable period the second part was larger in amplitude than the first part.

Interestingly, during both experiments this effect was observed in the parietal cortex (both t₂) ≥ 8.7, both p< 0.05), particularly the anterior intraparietal sulcus (AIPS, both p< 0.05, t-tests over participants per experiment), rather than the sensory cortical regions (early, ventral and dorsal visual; all p≥ 0.06; t-tests over ROIs per cortical region per experiment; F _{(4, 12)}= 5.1, p< 0.05, effect of cortical region across both experiments). During both experiments similar responses were also observed in the frontal cortical region, but they appeared more variable between participants. Importantly, there were no differences between binocular and 3D-motion rivalry in the localization of the within-block response enhancement (direct statistical comparison: F _{(4, 12)}= 0.8, p= 0.5, no interaction between type of rivalry and cortical region; Figure 4A). Further, for stimulus repetition we did not observe any consistency in the within-block modulation of the response across participants (all p> 0.07, t-tests over participants per ROI per experiment; F _{(1, 12)}= 55.1, p< 10^-5, effect of repetition modality across both experiments).

The influence of within-block perceptual stabilization

We observed a modest repetition suppression in the parietal and frontal cortical regions in relation to within-block perceptual stabilization, i.e. the percentage of same percepts within a block (Figure 5C; binocular rivalry: F _(4,16)= 17.1, p< 10^-4; 3D-motion rivalry: F _{(4, 12)}= 5.4, p< 0.05, effect of cortical region). A median-split procedure was used in the same way as for the other analyses described above. Regarding the individual ROIs the effect of within-block perceptual stabilization was significant in AIPS, SMA, IFJ and Insula during binocular rivalry only. This pattern of results may remotely resemble the results for stimulus repetition (see Figure 3). However, there was no correlation in time between these variables, ensuring that the effects of within-block perceptual stabilization were averaged out when the effects of stimulus repetition were investigated (relatedness index: binocular rivalry: -0.01, t₍₄₎= -0.2, p= 0.8; 3D-motion rivalry: -0.05, t₍₅₎= -1.3, p= 0.3; see Materials and Methods for calculation of relatedness index; Figure 5B).

The percentage of same percepts within a block was generally smaller in the early than in the late stages of a perceptually stable period (relatedness index: binocular rivalry: 0.54, t₍₄₎= 7.7, p< 0.01; 3Dmotion rivalry: 0.51, t₍₅₎= 12.2, p< 10^-4; Figure 5B). Although long-term perceptual repetition was thus related in time with the within-block perceptual stabilization, the associated modulations of the BOLD response were markedly different (see Figure 3). Also, the within-block response enhancement observed in the anterior intraparietal sulcus in relation with perceptual repetition (Figure 4) was not observed in relation with within-block perceptual stabilization (binocular rivalry: -0.03, t₍₄₎= -1.0, p=0.4; 3D-motion rivalry: 0.09, t₍₅₎= 2.2, p= 0.08, t-test over participants).

Overview of results

Minutes-long periods of largely stabilized perception were reported during passive viewing of either a binocular rivalry or 3D-motion rivalry stimulus. While stimulus repetition per se was associated with a decreased BOLD response, these involuntarily/automatically occurring perceptually stable periods were associated with an increased BOLD response in visual brain regions specific for the type of rivalry. More specifically, perceptual repetition enhancement was observed in early visual cortex during binocular rivalry, whereas it was present in both the early visual and the ventral visual cortex during 3D-motion rivalry (Figure 3). Perceptually stable periods were also characterized by a within-block response enhancement in the parietal cortex. This parietal effect of perceptual experience was similar for the two types of rivalry and was not observed in any of the visual regions (Figure 4).

Discussion

We investigated how perceptual experience modifies neural processing in sensory and cognitive brain regions. It is inherently difficult to separate the influence of prior perceptual experience from that of prior sensory stimulation. In contrast to previous studies, we discerned these two influences using ambiguous stimuli. Passive viewing of intermittent presentations of an ambiguous stimulus is known to elicit distinct and consecutive periods of stabilized perception that can last several minutes (Figure 1; Orbach et al., 1963; Leopold et al., 2002). We repeatedly presented either a binocular rivalry or a 3Dmotion rivalry stimulus, interleaved with short blank intervals, and observed that the mere repetition of the stimulus evoked an entirely different pattern of activity modulations than the repetition of a particular perceptual interpretation of the stimulus. Perceptual repetition was associated with an enhanced response in stimulus-specific visual brain regions (Figure 3) as well as a response change in the parietal cortex that was similar for the two types of stimuli used and was not observed in any of the visual regions (Figure 4). Stimulus repetition, on the other hand, resulted in an attenuated response in higher-level regions, particularly during the binocular rivalry experiment (Figure 3). Below we will discuss these results in further detail.

Perceptual repetition enhancement in stimulus-specific visual brain regions

During a period of repeated experience of the same binocular rivalry percept we found an enhanced BOLD response in early visual regions. These low-level regions process basic stimulus features and modulate their activity in response to changes in binocular rivalry perception (Gail et al., 2004; Haynes & Rees, 2005; Lee et al., 2005). When the same 3D-motion rivalry percept was repeatedly experienced the enhanced BOLD response was present not only in early visual regions, but also in ventral visual regions (Figure 3). The additional involvement of ventral visual regions is consistent with their role in processing relative disparity (Hinkle & Connor, 2005; Neri, 2005; Preston et al., 2008) and 3D-shape (Kourtzi et al., 2003).

Previous studies have also reported experience-dependent increases of fMRI responses in early visual regions (Schwartz et al., 2002; Furmanski et al., 2004) and ventral visual regions (Dolan et al., 1997; Kourtzi et al., 2005; James & Gauthier, 2006; Turk-Browne et al., 2007). Since the BOLD signal is an indirect measure of the combined activity of many sensory neurons (Logothetis & Wandell, 2004), it does not provide information regarding the physiological mechanisms underlying these effects. The stimulus-specific localization of the observed perceptual repetition enhancement is an indication that neurons tuned to the features of the stimulus were involved (Kourtzi et al., 2005; Grill-Spector et al., 2006; Krekelberg et al., 2006).

It is unlikely that the observed increased response is confounded by eye movements or difficulty of recognition of the percepts (recognition hypothesis proposed by Henson et al., 2000), as there was no difference between the early and late stages of a perceptually stable period in the number of eye movements, the eye position during fixation or the frequency of blended/mixed percepts (Figure 5). For 3D-motion rivalry we had anticipated an additional role of motion-sensitive dorsal and parietal brain regions (Paradis et al., 2000; Orban et al., 2006; Brouwer & van Ee, 2007; Preston et al., 2009; Brascamp et al., 2010); however, this was found only in a subset of the participants. Possibly, the involvement of these regions was determined by the extent to which the task-strategy of the participant employed spatial attention (Shulman et al., 1999; Corbetta et al., 2002) and/or the processing of coarse depth-judgments and absolute disparity (as contrasted to the processing of detailed depth-perception and relative disparity in ventral regions; Neri, 2005; Preston et al., 2008; Anzai & DeAngelis, 2010).

Possible mechanisms underlying perceptual repetition enhancement

There are some tentative explanations for the observed perceptual repetition enhancement based on previous neurophysiological and neuroimaging findings and, most likely, multiple of these physiological mechanisms play a role (Karmarkar & Dan, 2006; Holtmaat & Svoboda, 2009). Firstly, the number of activated neurons may have increased (Gilbert et al., 2001). For example, an increase in the number of neurons responsive to the features of the dominant percept can occur when the tuning curves of individual neurons shifted toward these features (Kohn & Movshon, 2004; Ghisovan et al., 2009). Also, additional neural processing may have been recruited, such as activity specific to perceptual stability (Rees et al., 2002; Sterzer & Rees, 2008; Pitts & Britz, 2011) or the perception of ambiguous stimuli (Sterzer et al., 2009; see also degraded stimuli: Rainer et al., 2004; Kourtzi et al., 2005; James & Gauthier, 2006). An increase in activity - or disinhibition - of the neurons associated with the suppressed percept may also contribute (Hock et al., 1996; Klink et al., 2010; de Jong et al., 2012), similar to the phenomenon of motion opponency (Petersen et al., 1985; Krekelberg et al., 2006). or otherwise

It could also be that the sensitivity/excitability of the activated neurons increased during a perceptually stable period, rather than the number (Frenkel et al., 2006; Grill-Spector et al., 2006). An increased excitability of percept-specific neurons at the moment of the perceptual choice could bias the competition between the possible percepts in favor of the most experienced percept, thereby favoring re-occurrence of this percept (Noest et al., 2007; Wilson, 2007; Heekeren et al., 2008). During further processing of the stimulus the initial increase in excitability may ultimately lead to an increase in the amplitude of the BOLD response (in a fashion similar to the accumulation hypothesis proposed by James & Gauthier, 2006). There is a variety of physiological changes that can lead to an increase in neural excitability, such as experience-dependent connectivity changes, local recurrent excitation, a reduction in neural noise, an increase in sub-threshold activity or a change in local field potentials (Hock et al., 1996; Crist et al., 2001; Schwartz et al., 2002; Rainer et al., 2004; Noest et al., 2007; Holtmaat & Svoboda, 2009; Klink et al., 2012).

In future electrophysiology studies the above-proposed increase in excitability can presumably be measured upon activation of the involved neurons, soon after the onset of the stimulus (Heekeren et al., 2008). Yet, its presence might be measurable during the blank periods in between the stimulus presentations as well (Britz et al., 2011; Ehm et al., 2011; Hsieh et al., 2012), as is also seen during the delay period in working memory tasks (Miller et al., 1996; Naya et al., 2003). The idea of increased activity during the blank periods fits with the observed effects, as this would eventually contribute to the BOLD response measured for the entire block of intermittent stimulation. A previous fMRI study reported that percept-specific activations lingered on during the first few seconds after removal of an ambiguous stimulus (Sterzer & Rees, 2008). If the duration of this ‘lingering on’ of activity is modulated by perceptual repetition, increased activity during the blank periods can also result from an increase in the duration of the activations to each individual stimulus presentation.

In all, there are several possible mechanisms underlying the observed perceptual repetition enhancement, namely an increase in the number of activated neurons, an increase in the excitability of percept-specific neurons or an increase in activity during the blank periods. Future studies are needed to investigate these speculations, as they cannot be distinguished with the present fMRI data. Regardless of the underlying mechanism, the present data suggest that the repeated activation of the visual networks biased toward the dominant percept facilitated later re-activation of these networks.

Perceptual repetition reflected in parietal regions

In contrast to the stimulus-specific activations in the visual cortex, we observed a change in the ‘saddle shape’ of the hemodynamic response in the parietal cortex that was similar during binocular and 3Dmotion rivalry (see Figure 4B). Saddle-shaped responses have been observed previously with unambiguous sensory stimuli, particularly when the stimulation blocks were long-lasting (> 16 seconds; e.g. Boynton et al., 1996; Friston et al., 1998; Soltysik et al., 2004). In the present study we observed that, over the course of a perceptually stable period, the second part of the hemodynamic response became larger than the first part of the response (Figure 4A). A previous study also found an enhancement of the later part of the response when the first and second presentations of an unfamiliar visual stimulus were compared (Martens & Gruber, 2012).

The second peak in the response occurred too early to be an off-set effect. Also, the effect could not be attributed to within-block perceptual stabilization (Figure 5). Given that it was observed for both types of rivalry it probably relates to aspects of perception that are independent of specific percepts or stimuli. The early visual cortex is necessary for conscious perception, but not sufficient (Rees et al., 2002; Tong, 2003), and parietal regions could have an additional interpretive or evaluative involvement in perceptual experience (Dolan et al., 1997; Gilbert et al., 2001; Sterzer et al., 2009). The intraparietal sulcus in particular has been implicated in monitoring ambiguous perception over time in paradigms that investigated perceptual switches during continuous stimulus presentation (Rees et al., 2002; Kanai et al., 2011) or the repetition of the most recent percept across a single interruption of the stimulus (trial-to-trial perceptual stabilization; Sterzer & Rees, 2008; Britz et al., 2011).

The perceptual stabilization in the present study did not require conscious effort or active mnemonic processing (Pearson & Brascamp, 2008) and cannot be explained as repetition priming of the most recent percept (Long & Toppino, 2004; Pearson & Brascamp, 2008). Perhaps the repeated co-activation of parietal networks involved in the integration of perceptual information over time and visual networks dedicated to the dominant percept strengthened the connections between them. When the parietal networks are associated with the dominant rather than the suppressed percept, the stability of the dominant percept could be further enhanced. Also, parietal regions could provide stimulusspecific feedback to visual cortex, for example to counteract the above-proposed disinhibition of neurons associated with the suppressed percept (Kanai et al., 2011). However, these and other speculations need further investigation.

Stimulus repetition suppression

Experience with the stimulus per se evoked an entirely different pattern of results than the perceptual repetition enhancement described above. While the perceptual experience of the stimulus changed occasionally, the stimulus itself was the same throughout an experimental run. Over the course of an experimental run the repeated presentation of the binocular rivalry stimulus resulted in a widespread suppression of the BOLD response, possibly due to neural fatigue and/or more efficient or sparser encoding (Grill-Spector et al, 2006; Krekelberg et al., 2006; Kohn, 2007). In line with earlier reports that repetition suppression is often confined to higher visual regions this effect was not present in early visual regions (Krekelberg et al. 2006). The repeated presentation of 3D-motion rivalry was not associated with suppression of the BOLD response, perhaps because the small moving dots that constituted the 3D-motion stimulus resulted in less luminance adaptation than the stationary black and white bars that constituted the binocular gratings (Figure 1A).

Conclusions

Experience-driven modulation of neural processing in the adult brain is likely to be important for adapting our behavior to dynamic environments. The present results indicate that the repeated activation of visual networks mediating a particular percept enhanced later re-activation when compatible visual input is presented. Possible physiological mechanisms might be an increase in the number of activated neurons, an increase in the excitability of the percept-specific neurons, and an increase in activity during the blank periods, but these speculations need further investigation. In contrast, the parietal cortex contributed to perceptual stability in a manner that was similar for the two ambiguous stimuli tested. We speculate that perceptual experience is associated with a facilitated neural response within and between percept-specific visual networks and parietal networks involved in the temporal integration of perceptual information. The parietal regions may modulate percept-specific processing in visual areas. Future human and animal electrophysiology investigations into the temporal dynamics underlying these effects may advance our understanding of the experience-dependent activity in these cortical circuits. We conclude that the visual and parietal cortices play dissociable and complementary roles in the interpretation of ambiguous sensory information based on previous experience.

Abbreviations

3D-motion

three-dimensional motion

AIPS

anterior intraparietal sulcus

BOLD

blood oxygen level-dependent

DLPFC

dorsolateral prefrontal cortex

FEF

frontal eye-fields

fMRI

functional magnetic resonance imaging

hMT+

motion-selective mediotemporal area

IFJ

inferior frontal junction

LO

lateral occipital area

PIPS

posterior intraparietal sulcus

POIPS

parieto-occipital intraparietal sulcus (parietal region sensitive to SFM)

ROI

region of interest

SMA/SEF

supplementary motor area/supplementary eye-fields

SFM

structure-from-motion

V3A

visual area 3A

V3D

dorsal part of visual area 3

V3V

ventral part of visual area 3

V4

ventral visual area 4