Skip to main content
PMC home page
iScience logoLink to iScience
. 2021 Feb 26;24(3):102234. doi: 10.1016/j.isci.2021.102234

Bistable perception alternates between internal and external modes of sensory processing

Veith Weilnhammer 1,2,5,∗, Meera Chikermane 1, Philipp Sterzer 1,2,3,4
PMCID: PMC7967014  PMID: 33748716

Summary

Perceptual history can exert pronounced effects on the contents of conscious experience: when confronted with completely ambiguous stimuli, perception does not waver at random between diverging stimulus interpretations but sticks with recent percepts for prolonged intervals. Here, we investigated the relevance of perceptual history in situations more similar to everyday experience, where sensory stimuli are usually not completely ambiguous. Using partially ambiguous visual stimuli, we found that the balance between past and present is not stable over time but slowly fluctuates between two opposing modes. For time periods of up to several minutes, perception was either largely determined by perceptual history or driven predominantly by disambiguating sensory evidence. Computational modeling suggested that the construction of unambiguous conscious experiences is modulated by slow fluctuations between internally and externally oriented modes of sensory processing.

Subject areas: Biological Sciences, Neuroscience, Sensory Neuroscience

Graphical abstract

graphic file with name fx1.jpg

Open in a new tab

Highlights

  • •

    Perception fluctuates between external and internal modes of sensory processing

Biological Sciences ; Neuroscience ; Sensory Neuroscience

Introduction

Imagine walking down a dark and unfamiliar street. As you struggle to identify potential obstacles, you are confronted with an ongoing stream of sensory signals, each compatible with multiple interpretations. In such situations, your previous perceptual experiences may provide valuable clues about how to interpret the ambiguous sensory data. Yet, relying too heavily on the past is risky, as you may end up overlooking unexpected changes in the environment.

Experimentally, the influence of preceding experiences on perception is usually investigated in tasks that require participants to perform perceptual decisions in a sequence of consecutive trials (Bergen and Jehee, 2019; Fründ et al., 2014). Such experiments reveal that, even in the absence of any correlation between the stimuli that are presented on successive trials, perception is significantly biased toward preceding choices (Abrahamyan et al., 2016; Fischer and Whitney, 2014; Fritsche et al., 2017; Hsu and Wu, 2020; Liberman et al., 2014; Urai et al., 2017, Urai et al., 2019). Importantly, perceptual history effects increase when sensory information becomes unreliable (Bergen and Jehee, 2019; Fründ et al., 2014). This reflects the idea that, when making perceptual decisions in situations of uncertainty, the brain may rely more strongly on internal predictions (Friston, 2005, 2010) that reflect the continuity of the sensory environment.

Integrating the internal information provided by perceptual history with the available external stimulus information may thus benefit perception by preventing erratic responses to unreliable sensory signals (Friston, 2005, 2010). However, the effects of perceptual history may also become mal-adaptive: when relying too strongly on preceding experiences, observers may become prone to ignore conflicting stimuli, which may lead to hallucinatory perceptual states that diverge from the true cause of the sensory data (Horga and Abi-Dargham, 2019; Powers et al., 2017).

In this work, we studied how visual perception balances external with internal sources of information in situations where perceptual history has a particularly strong effect. To this end, we investigated how preceding experiences impact the perception of ambiguous stimuli, i.e., stimuli that are compatible with two mutually exclusive perceptual states and typically give rise to bistable perception (Leopold et al., 2002). During bistable perception, observers experience spontaneous transitions between the two perceptual states, whereas the sensory data remain constant (Logothetis et al., 1996). Importantly, when the ambiguous stimuli are presented in successive trials separated by blank intervals, perception tends to stabilize in one of the two interpretations (Maloney et al., 2005), indicating a pronounced effect of perceptual history (Pearson and Brascamp, 2008).

Here, we estimated the strength of perceptual history during bistable perception using a staircase procedure that dynamically adjusted the degree of perceptual ambiguity of structure-from-motion stimuli. By quantifying the effect of perceptual history relative to graded levels of sensory ambiguity, we investigated the computational mechanisms of integrating internal with external information during bistable perception.

Results

To study how perceptual history is balanced against external sensory information during bistable perception, we asked 20 participants to indicate whether they perceived partially ambiguous random-dot-kinematograms as rotating to the left or the right (Figure 1A and Video S1). At each trial, we attached a 3D signal to a subset of the stimulus dots. This enabled us to parametrically manipulate the stimulus' signal-to-ambiguity ratio (Weilnhammer et al., 2020) (SAR). Ranging between 0% and 100%, these varying levels of disambiguating sensory information enforced one of the two stimulus interpretations (i.e., the direction of disambiguation). Within each experimental run, both directions of disambiguation occurred in equal number and in random sequence.

Figure 1.

Figure 1

Open in a new tab

Psychophysical staircase

(A) Graded ambiguity. Participants viewed partially ambiguous structure-from-motion stimuli and indicated whether they perceived 3D rotation to the left or to the right. In runs R1-4, we dynamically adjusted the signal-to-ambiguity ratio (SAR) according to a staircase procedure that was based on the number of stimulus-congruent trials computed within blocks of 10 successive trials. During the final runs R5 and R6, we fixed the SAR to the average SAR obtained during runs R1-4.

(B) Stimulus-congruent percepts across runs. In runs R1-4 (depicted in red), the staircase procedure introduced dynamic adjustments in the SAR, reducing the frequency of stimulus-congruent percepts to approximately 75% (R1: 94.88 ± 1.1%; R2: 84.92 ± 1.55%; R3: 80 ± 1.33%; R4: 77.25 ± 1.09%). In runs R5-6 (depicted in blue), the SAR was fixed to the average SAR from the preceding runs R1-4 (60.25 ± 2.36%). Stimulus-congruent percepts amounted to 87.21 ± 3.23% in R5 and 82.5 ± 4.41% in R6.

(C) Stimulus-congruent percepts across levels of SAR. Stimulus-congruent percepts were more frequent at higher levels of disambiguating sensory information, ceiling at 100%. Pooled data are represented as mean ± SEM.

Video S1. Partially disambiguated Lissajous figure, related to figure 1

Please used red-blue filter glasses to view the stimulus.

Download video file (7MB, flv)

Perception integrates perceptual history with disambiguating sensory information

In the first four runs (R1-4, Figure 1B), we estimated individual threshold SARs necessary to induce balanced frequencies of stimulus-congruent and stimulus-incongruent percepts (i.e., trials perceived as congruent or incongruent with the disambiguating sensory information, respectively; Figure 2A). To this end, we dynamically adjusted the SAR based on the proportion of stimulus-congruent responses in consecutive 10-trial blocks. This psychophysical staircase decreased the SAR if less than 80% of trials were perceived as stimulus-congruent. Conversely, we increased the SAR if the proportion of stimulus-congruent trials fell below 80%. As expected, stimulus-congruent percepts were less frequent at lower SARs (F(1, 265.07) = 181.5, p = 7.25×10−32, BF10=5.22×1028, main effect of SAR, Figure 1C). In runs R5-6, stimuli were presented at the individual threshold SAR (i.e., the average SAR from runs R1-4), which yielded stimulus-congruent percepts in 84.85 ± 3.12% of trials (Figure 1B).

Figure 2.

Figure 2

Open in a new tab

External and internal modes

(A) Stimulus- and history-congruent perceptual states. To visualize the influence of disambiguating sensory information and perceptual history, the upper panel depicts the time course of presented stimuli (L/R: disambiguating stimulus information for leftward/rightward rotation; dashed line) and the associated time course of perception (solid line). Perception is stimulus-congruent when the presented stimulus matches the associated perceptual state (i.e., overlap between the dashed and the solid line). History-congruent perception occurs when the perceptual state at a given trial matches the perceptual state at the preceding trial. The lower panel depicts the dynamic probabilities of stimulus-congruent percepts (green) and history-congruent percepts (orange) computed in sliding windows of ±5 trials for a representative participant. Perceptual processing switched between prolonged intervals of internal mode (green line below orange line), external mode (green line above orange line), and intermediate mode (overlap between green an orange line).

(B) Average autocorrelation coefficients of stimulus- and history-congruence. Despite constant SAR at threshold, both stimulus and history congruent were highly autocorrelated. If the index trial was perceived as congruent with visual stimulation (left panel) or perceptual history (right panel), the observer was more likely to experience stimulus- or history-congruent perceptual states, respectively, for approximately 25 trials. After that, the observer was more likely to experience incongruent states. The opposite relation holds for incongruent perceptual states at the index trial. Group-level averages were fitted using local polynomial regression fitting. Purple dots indicate trials at which the autocorrelation coefficients differed significantly from chance level (p < 0.05, two-sided one-sample t tests).

(C) Stimulus-congruent percepts during internal and external mode for SAR at threshold. During external mode, stimulus-congruent percepts made up for almost 100% of trials (95.49 ± 1.28%) but, interestingly, did not differ significantly from chance level during internal mode (54.71 ± 3.39%).

(D) Stimulus-congruent percepts during internal and external mode across the full range of SAR. Linear mixed effects modeling indicated that the frequency of stimulus-congruent percepts increased with levels of SAR. Internal mode was associated with a strong reduction of stimulus-congruent percepts (main effect of mode), which was more pronounced at low levels of SAR (mode × SAR interaction). Please note that any main effect of mode was expected, because external and internal mode were defined based on the dynamic probability of stimulus congruence. Pooled data are represented as mean ± SEM.

Conversely, higher SARs reduced the impact of perceptual history (Figure S1). This resulted in a strong inverse relationship between stimulus- and history-congruent percepts (i.e., trials perceived in congruence with the immediately preceding percept), which were anti-correlated both within (average Pearson correlation coefficient ρ = −0.9 ± 0.02, T(19) = −49.25, p = 1.66×10−21, BF10=1.34×1018, one-sample t test; Figure S2A) and across participants (ρ = −0.77, p=7.2×10−5, BF10=203.27, Pearson correlation; Figure S2B).

We did not find any systematic bias toward one of the two perceptual interpretations (average probability of rightward rotation: 51.86 ± 3.04%; T(19) = 0.61, p = 0.55, BF10=0.27, one-sample t test). Absolute biases were small, amounting to 13.99 ± 1.57% across participants. Error responses were negligible, occurring in only 1.6 ± 1.57% of trials. Unclear percepts were not reported by the participants.

In logistic regression applied to each individual participant's behavioral data, trial-wise perceptual responses were best predicted based on both the current sensory information and the previous percept, as compared with reduced logistic regression models (Figure S2C) that used only stimulus information (T(19) = −9.39, p = 1.45×10−8, BF10=8.89×105, paired t test) or only perceptual history (T(19) = −16.46, p = 1.06×10−12, BF10=6.54×109) for prediction.

Two additional control analyses confirmed that both disambiguating sensory information and perceptual history significantly modulated the perception of partially disambiguated stimuli. Firstly, general linear mixed effects modeling with a binomial link function indicated a highly significant effect of both disambiguating sensory evidence (z = 45.55; p = 0) and perceptual history (z = 28.51; p = 8.62×10−179), while controlling for the within-participant correlations using random intercepts.

A second possibility for this group-level inference is provided by general estimating equations (Hanley, 2003), which offer a non-parametric way of accounting for within-participant correlation by estimating population average effects. Likewise, this approach revealed a highly significant effect of disambiguating sensory evidence (Wald = 38.6; p = 5.2×10−10) and perceptual history (Wald = 74.33; p = 0, correlation structure = “independence”).

These results indicate that the effect of perceptual history is not limited to fully ambiguous stimuli (Pearson and Brascamp, 2008) but modulates perception through a weighted integration with varying levels of disambiguating sensory information (Bergen and Jehee, 2019). This finding aligns with the well-known observation that perception is co-determined by both sensory data and past experiences (Chopin and Mamassian, 2012; Fischer and Whitney, 2014; Fritsche et al., 2017; Hsu and Wu, 2020; Liberman et al., 2014). Perceptual history may benefit perception as an internal representation (Friston, 2005, 2010; Körding and Wolpert, 2004; Teufel and Fletcher, 2020) that stabilizes conscious experience when external sensory information is incomplete or unreliable. On your night-time walks, previous experiences may thus help you to avoid responding to irrelevant fluctuations in the ongoing stream of ambiguous sensory signals.

Perception fluctuates between temporally extended modes that are biased toward either external or internal information

In a next step, we examined how the probabilities of stimulus- and history-congruent percepts evolved within individual runs of the experiment (Figure 2A). Intriguingly, we found that both stimulus- and history-congruence were significantly autocorrelated (Figure 2B), indicating that the integration of perceptual history with sensory information was highly variable over time. For partially ambiguous stimuli presented at constant SARs (R5-6), we observed marked switches between intervals in which perception was either strongly driven by disambiguating sensory information (external mode, 73.25 ± 6.17% of trials) or determined by perceptual history (internal mode; 23.94 ± 5.84%), in addition to shorter intermediate intervals (2.81 ± 0.77%; Figure 2A, lower panel). Switches between these modes occurred on average every 39.9 ± 7.31 trials (179.53 ± 32.91 s).

Our analyses therefore revealed prolonged intervals of alternating biases toward either internal or external information. This finding is incompatible with the view that perception is best explained by integrating uncertain sensory data with only the immediately preceding perceptual state. As indicated by simulation analyses (Figure S3), such a Markovian assumption did not reproduce the autocorrelation of stimulus and history congruence (Figure S3B) and predicted longer external (T(19) = 2.75, p = 0.01, BF10=4.17, paired t test; Figure S3C) as well as shorter internal modes (T(19) = −3.49, p = 2.44×10−3, BF10=16.92).

In sum, these results imply that a stable moment-by-moment integration of current sensory information with the immediately preceding percept is not sufficient to explain the perceptual dynamics during graded ambiguity. Rather, our findings suggest that participants transition between temporally extended perceptual modes (Honey et al., 2017) that are biased toward either external information (i.e., disambiguating sensory information) or internal information (i.e., perceptual history).

Importantly, switches between internal and external modes could not be attributed to small fluctuations in the participants' sensitivity to disambiguating sensory information. At threshold (R5-6), stimulus-congruent percepts were close to 100% during external mode but ranged at chance level during internal mode (T(12) = 1.39, p = 0.19, BF10=0.61, one-sample t test, Figure 2C). Please note that the overall difference in stimulus-congruency between modes is expected, because external and internal mode were defined based on the dynamic probability of stimulus-congruent perceptual states.

Moreover, internal mode suppressed the sensitivity to disambiguating sensory information not only at the threshold but across the full range of SAR (F(2, 484.41) = 35.26, p = 5.04×10−15, BF10=4.78×1066; main effect of mode; Figure 2D). During runs in which the SAR was adjusted dynamically (R1-R4), transitions from internal to mode were more likely to occur when the available sensory information was reduced (F(2, 472.71) = 5.25, p = 5.58×10−3, BF10=3.57, mode × SAR interaction). In sum, these control analyses argue against the view that between-mode transition may result exclusively from a threshold phenomenon.

As a second caveat, we asked whether the observed transitions between internal and external mode constitute a perceptual phenomenon or, alternatively, occur only due to cognitive processes that are situated downstream of perception (Brascamp et al., 2018). In this context, it may be argued that the participants' attention to the experimental task may have fluctuated over time (Rosenberg et al., 2013; Zalta et al., 2020), leading to intervals of stereotypic reporting behavior. We addressed this potential confound by analyzing response times (RTs, see Figure S4), which have been shown to link closely with on-task attention (Prado et al., 2011; Rosenberg et al., 2013).

In contrast to stimulus and history congruence, response times remained stable across the experimental runs (Figure S4A) and did not vary across levels of SAR (R1-R4; F(1, 261.5) = 0.05, p = 0.82, BF10=0.15; Figure S4B). At threshold, RTs did not differ between external and internal mode (T(12) = 0.74, p = 0.48, BF10=0.35, paired t test; Figure S4C). Moreover, when analyzing RTs according to the factors mode and SAR in runs R1-R4 (Figure S4D), we found that, during internal mode, RTs increased for escalating levels of SAR. Speculatively, this mode × SAR interaction (F(2, 476.5) = 10.73, p = 2.77×10−5, BF10=538.42) could reflect the increase in conflict between the history-congruent state and the available sensory information (Weilnhammer et al., 2020).

At the same time, both the absence of any mode effect on RTs at threshold as well as the sensitivity of internal-mode RTs to levels of SAR argue against the notion that internal mode is caused by the participants paying less attention to the experimental task. In line with this observation, we found no changes in the distribution of normalized RTs, as participants transitioned between internal and external mode (see Figure 4E for group RTs collapsed across participants and Figure 4F for individual distributions).

Figure 4.

Figure 4

Open in a new tab

Computational modeling: Results

(A) Model-level inference. Random-effects Bayesian model selection identified MLearning+/Oscillation+ as the clear winning model (group-level protected exceedance probability = 99.82%).

(B) Parameter-level inference. This model assumes that the external sensory signals is detected with a sensitivity parameter of α = 0.11 ± 0.01. The internal representation derived from perceptual history is updated as a function of the sequence of percepts according to learning rate ω = 0.05 ± 0.11. κ, the impact of accumulating perceptual history on perception, fluctuated according to a sine function with an amplitude of 1.27 ± 0.06, a frequency of 0.09 ± 3.41×10−3 (in nbtrials−1), and a phase of 2.37 ± 0.2. Pooled data are represented as mean ± SEM.

As a final control analysis, we checked whether internal mode was associated with an enhanced impact of the perceptual state experienced at the preceding trial (i.e., perceptual history), as opposed to the disambiguating sensory information presented at the preceding trial (i.e., stimulus history). As expected, history-congruent perceptual states dominated periods of internal mode processing (94.15 ± 1.03%), whereas stimulus history had no detectable influence on perception in these intervals (49.79 ± 1.03%; T(19) = −0.2, p = 0.84, BF10=0.24, one-sample t test).

During internal mode, perceptual history thus strongly determines conscious experience, overriding otherwise effective sensory information. As you interpret ambiguous sensory information on your walk through the dark, relying on an internal representation of your surroundings may dramatically increase the energy efficiency of perception. However, this is only adaptive in stable environments, i.e., when sensory events are highly auto-correlated. In volatile environments, internally biased sensory processing may cause perception to get stuck in the past, resulting in hallucinatory experiences that ignore relevant conflicts (Weilnhammer et al., 2020) with sensory information (Horga and Abi-Dargham, 2019).

Computational modeling indicates that between-mode transitions are best explained by a fluctuating impact of accumulating perceptual history

How can perception achieve an adaptive balance between external and internal mode? To address this question, we investigated the potential computational mechanisms that could lead to the observed oscillations between internally and externally biased modes of perceptual processing. To this end, we constructed a set of four generative behavioral models (Wilson and Collins, 2019) (Figure 3) that differed across two dimensions.

Figure 3.

Figure 3

Open in a new tab

Computational modeling: Modelspace

To investigate the computational mechanisms of between-mode transitions, we constructed a space of four behavioral models that differed along two dimensions. Each model's quantities are shown in three separate panels. Along a first dimension (horizontal arrow), we manipulated whether perceptual history effects were represented exclusively by the perceptual state at the preceding trial (left side) or, alternatively, dynamically accumulated according to a learning rate ω (right). Perceptual history and its updating are displayed in the upper panel of each model. The blue line represents μ2, i.e., the tendency to expect rightward (above zero) or leftward (below zero) rotation at the upcoming trial. The red line depicts dynamic precision-weighted prediction errors ε2 that update μ2 in response to the sequence of perceptual experiences. Along a second dimension (vertical arrow), we contrasted models that assumed a stable influence of perceptual history on perception (top) against models that assumed a systematic fluctuation in the impact of perceptual history. κ (green line; middle panel) represents the weight at which perceptual history impacts on perception. The lower panel shows the perceptual prediction μ2ˆ (blue line, provided by a sigmoid transform of μ∗κ), the disambiguating sensory information u (red) and the participants' response y (black).

On the first dimension, we asked whether biases toward internal mode arise from the sequence of previous experiences. We reasoned that, if perceptual history effects dynamically accumulate over time (Brascamp et al., 2008; Pearson and Brascamp, 2008), perception would be more strongly biased toward a perceptual state if the current trial was preceded by a long sequence of history-congruent trials. Accumulating perceptual history effects could eventually become strong enough to override otherwise effective sensory information, thereby creating intervals during which perception is strongly determined by internal information.

To this end, we adopted a Bayesian modeling approach that frames perception as an inferential process in which perceptual decisions are determined by posterior distributions (Friston, 2010). Following Bayes' rule, such posterior distributions are computed by integrating a likelihood distribution representing the sensory evidence (i.e., disambiguating sensory information for left- or rightward rotation at a given SAR) with the prior probability of perceptual states (i.e., perceptual history).

The null model MLearning−/Oscillation− (see transparent method section and Figure 3 for details) assumes that the effect of perceptual history (i.e., the estimated prior probability of perceptual states) depends only on the perceptual state at the immediately preceding trial. Its weight on perception is determined by the parameter κ. The impact of sensory information, in turn, depends on the sensitivity parameter α.

By contrast, in the alternative model MLearning+/Oscillation−, the estimated prior probability of perceptual states depends not only on the response at the preceding trial but dynamically accumulates over time according to a two-level Hierarchical Gaussian Filter (Mathys et al., 2014). Thus, the implicit belief in the probability of perceiving leftward rotation increases as a function of the number of preceding trials that have been experienced as rotating toward the left (and vice versa). The second-level accumulation of perceptual history is governed by the learning-rate parameter ω.

In this model, switches between modes can only be driven by experience. Once perceptual history effects have accumulated and caused the estimated probability of leftward rotation to increase significantly above chance level, switches to external mode are enabled by prediction errors that are caused by the experience of rightward rotation (and vice versa).

As an alternative explanation, we reasoned that switches between modes could additionally be facilitated by systematic fluctuations in κ, the parameter governing the impact of perceptual history on perception. When κ is low, perceptual states are more likely to be history incongruent, increasing the likelihood of prediction errors that enable the transition from internal to external mode. To test whether such fluctuations provide a plausible explanation of our behavioral data, we introduced a second dimension to our model space by constructing MLearning+/Oscillation+ and MLearning−/Oscillation+. Instead of estimating κ as a stable parameter, these models enable oscillations in κ that are governed by parameters for amplitude amp, frequency f (in nbtrials−1), and phase p.

We inverted all models based the trial-wise perceptual responses given by our participants and used random-effects Bayesian model family selection (Stephan et al., 2009) to determine whether the dynamic accumulation of perceptual history (dimension 1) and systematic fluctuations in its impact (dimension 2) were likely to represent a computational mechanism of mode switches.

On the first dimension, we found that models assuming a dynamic accumulation of perceptual history (Learning+) outperformed Learning− models at a protected exceedance probability of 100%. On the second dimension, Bayesian model selection indicated that our data were better explained by models that assumed a fluctuating impact of perceptual history (Oscillation+) as compared with Oscillation− models at a protected exceedance probability of 99.98%. MLearning+/Oscillation+ was therefore identified as the clear winning model (protected exceedance probability = 99.82%; see Figure 4A for model-level inference at the participant level and Figure 4B for posterior parameter estimates).

With this, our computational approach suggests that switches between internal and external mode are governed by two interlinked processes: In line with previous findings (Brascamp et al., 2008; Pearson and Brascamp, 2008), we found that perceptual history accumulates over time. Eventually, accumulating perceptual history may override disambiguating sensory information, causing a transition to from external to internal mode. In isolation, however, such a process falls short of explaining transitions in the opposite direction. Because perceptual history effects continue to accumulate during internal mode, they should eventually become impossible to overcome (Wexler et al., 2015). Crucially, our modeling results propose that fluctuations in the impact of perceptual history enable transition from internal to external mode by temporarily de-coupling the perceptual decision from implicit internal representations of the environment.

Discussion

In this work, we show that perceptual history modulates perception through a weighted integration (Bergen and Jehee, 2019) with varying levels of sensory information. Perceptual history therefore acts as an internal representation (Friston, 2005, 2010; Teufel and Fletcher, 2020) that stabilizes perception when sensory signals are ambiguous. Intriguingly, we found that the balance between perceptual history and disambiguating sensory information slowly alternates between internally and externally oriented modes of sensory processing. Computational modeling indicated that between-mode transitions were likely to be caused by fluctuations in how strongly perception was driven by the accumulating effects of perceptual history.

It may be argued that temporally extended biases toward internal information are not generic but specific to the class of structure-from-motion stimuli (Longuet-Higgins, 1986) investigated here. Indeed, structure-from-motion induces relatively long perceptual dominance durations (Weilnhammer et al, 2014, 2016, 2020). In addition, individual observers have been shown to exhibit stable idiosyncratic biases toward one of the two stimulus interpretations (Mamassian and Wallace, 2010; Weilnhammer et al., 2020), which can become strong enough to override disambiguating 3D cues (Wexler et al., 2015).

In this work, however, two factors speak against the view that transitions to internal mode were caused exclusively by strong perceptual biases. Firstly, we found relatively weak imbalances between the two possible states induced by our partially ambiguous structure-from-motion stimulus (see Results section). Secondly, we observed frequent transitions from internal to external mode while sensory information was held constant at threshold (see Figure 2), arguing against stable biases as the primary determinant of internally biased processing during graded ambiguity. Yet, to empirically assess this caveat, future work should investigate whether between-mode transitions occur also for ambiguous stimuli that induce shorter dominance durations, such as the Necker cube (Kornmeier and Bach, 2005). This would help understand whether fluctuations between internal and external mode depend on the type, strength, and temporal characteristics of bistable perception or, alternatively, occur independently of these factors and thus constitute a more general feature of perceptual processing.

As a second alternative explanation of our results, it may be proposed that fluctuating biases toward internal or external mode do not represent a perceptual phenomenon but, conversely, occur only due to processes that are situated downstream of perception, such as changes in reporting behavior (Brascamp et al., 2018) that are caused by periodic changes in how well participants attended to the experimental task (Zalta et al., 2020). Our analysis of response times (Figure S4), which are classically linked to fluctuating attention in paradigms such as the Continuous Performance Task (Rosenberg et al., 2013), did not yield any evidence for systematic differences in response behavior between internal and external mode. Yet, future experiments should apply no-report paradigms (Frässle et al., 2014), pupillometry (Lawson et al., 2020), or experimental manipulations of on-task attention (Alais et al., 2010) to dissociate post-perceptual processes from the perceptual phenomenon of mode-switching proposed in this work.

In a similar vein, it may be argued that slow fluctuations between externally and internally biased perception reflect epiphenomena that may arise from arbitrary constraints of neural processing (Honey et al., 2017). On the other hand, there may also be a specific computational benefit to slow transitions between external and internal model (Honey et al., 2017; Palva and Palva, 2011; VanRullen, 2016): In stable environments, internal mode may come with the benefit of a dramatic reduction in the energy demands of perception (Friston, 2010). Periodic switches to external mode may ensure that internal representations are updated in response to potential changes in the environment (Honey et al., 2017). In contrast to simultaneous processing, periodic mode switches may allow the brain to differentiate between internal and external sources of information (Honey et al., 2017). This may help perception to solve the credit-assignment problem, i.e., deciding whether to update internal representations of the environment or, alternatively, to modify beliefs about the reliability of sensory information (Weilnhammer et al., 2018). Thus, mode switching may represent a process that helps constructing stable representations of the environment despite ongoing sensory inputs (Bengio et al., 2015).

Indeed, fluctuations between externally and internally biased processing have been described in a variety of cognitive domains, including perception (Monto et al., 2008), episodic memory (Duncan et al., 2012), and waking state (McGinley et al., 2015). Switching between external and internal processing modes may thus represent a general computational mechanism that helps to adaptively integrate prior predictions with new information (Honey et al., 2017). Alterations in the temporal dynamics of mode switching may therefore represent the neurocomputational basis of psychotic experiences that often co-occur across cognitive domains, such as hallucinations, delusions, and altered sense of agency (Horga and Abi-Dargham, 2019; Sterzer et al., 2018).

To test the hypothesis that mode switches represent an adaptive mechanism that occurs across cognitive domains, future research should investigate whether transitions between external and internal mode can be induced experimentally. Based on the results of our computational modeling analysis, it may be hypothesized that participants should be more prone to transition from internal to external mode when repeatedly confronted with information that contradicts past experiences. Conversely, transitions from external to internal mode should occur more swiftly when participants receive information that is in line with prior predictions. Further down the line, it may be speculated that the overall frequency of mode switches could be altered by experimentally manipulating the volatility of the input data (Iglesias et al., 2013; Mathys et al., 2014).

Likewise, the existence of mode switches should be further substantiated by investigating whether external and internal modes can be determined based on markers that are independent of the perceptual response such as pupillary response or heart rate (Lawson et al., 2020). Together with an experimental manipulation of between-mode transitions, such markers could help to understand whether transitions between internal and external modes indeed represent an adaptive cognitive strategy that aids learning, or, alternatively, result from independent phenomena such as adaption (Chopin and Mamassian, 2012), attention (Alais et al., 2010), or response behavior (Frässle et al., 2014).

Limitations of the study

In this study, we have shown that bistable perception cycles through prolonged periods of enhanced and reduced sensitivity to disambiguating stimulus information. This finding suggests that conscious experience is characterized by slow fluctuations between internally and externally oriented modes of sensory processing. As a first limitation, our work investigated between-mode transitions only for a specific class of bistable stimuli (ambiguous structure-from-motion). Future work should test whether alternations between internally and externally oriented modes of processing also occur in other bistable stimuli, in particular in relation to paradigms that induce shorter dominance durations. As a second limitation, our work defines internal and external modes solely on the basis of behavior. Future studies should apply independent markers for internal and external modes (such as pupillometry) to probe between-mode transitions irrespective of behavioral reports. As a third limitation, our study does not provide an experimental control of pre- and post-perceptual processes such as attention or response behavior. Future experiments should use no-report paradigms or experimental manipulations of on-task attention to confirm that mode-switching represents a perceptual phenomenon rather than a process that occurs up- or downstream of perception.

Resource availability

Lead contact

The lead contact for this study is Veith Weilnhammer (veith-andreas.weilnhammer@charite.de).

Material availability

Materials have been deposited at OSF: https://doi.org/10.17605/OSF.IO/Y2CFM.

Data and code availability

Original data and code have been deposited at OSF: https://doi.org/10.17605/OSF.IO/Y2CFM.

Methods

All methods can be found in the accompanying transparent methods supplemental file.

Acknowledgments

Author VW is a fellow of the Clinician Scientist Program funded by the Charité – Universitätsmedizin Berlin and the Berlin Institute of Health. This program was initiated and led by Prof. Dr. Duska Dragun to enable physicians to pursue a parallel career in academic research. With great sadness we have received the news that Prof. Dragun passed away on December 28th of 2020. We dedicate this publication to her as a mentor, friend, role model, and stellar scientist.

PS is funded by the German Research Foundation (STE 1430/8-1) and the German Ministry for Research and Education (ERA-NET NEURON program 01EW2007A). We acknowledge support from the German Research Foundation (DFG) and the Open Access Publication Fund of Charité – Universitätsmedizin Berlin.

Author contributions

  • •

    VW and PS conceptualized the study.

  • •

    VW designed the experiment.

  • •

    VW and MC collected the data.

  • •

    VW and PS wrote the initial draft and edited the manuscript.

  • •

    VW, MC, and PS reviewed the manuscript.

Declaration of interests

The authors declare no competing interests.

Published: March 19, 2021

Footnotes

Supplemental information can be found online at https://doi.org/10.1016/j.isci.2021.102234.

Supporting Citations

The following references appear in the supplemental information: Gekas et al., 2019.

Supplemental information

Document S1. Transparent methods and Figures S1–S4
mmc1.pdf (476.5KB, pdf)

References

  1. Abrahamyan A., Silva L.L., Dakin S.C., Carandini M., Gardner J.L. Adaptable history biases in human perceptual decisions. Proc. Natl. Acad. Sci. U S A. 2016;113:E3548–E3557. doi: 10.1073/pnas.1518786113. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Alais D., Boxtel J.J.van, Parker A., Ee R.van. Attending to auditory signals slows visual alternations in binocular rivalry. Vis. Res. 2010;50:929–935. doi: 10.1016/j.visres.2010.03.010. [DOI] [PubMed] [Google Scholar]
  3. Bengio Y., Lee D.-H., Bornschein J., Mesnard T., Lin Z. Towards biologically plausible deep learning. arRxiv. 2015:1502.04156. [Google Scholar]
  4. Bergen R.S.van, Jehee J.F. Probabilistic representation in human visual cortex reflects uncertainty in serial decisions. J. Neurosci. 2019;39:8164–8176. doi: 10.1523/JNEUROSCI.3212-18.2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Brascamp J., Sterzer P., Blake R., Knapen T. Multistable perception and the role of the frontoparietal cortex in perceptual inference. Annu. Rev. Psychol. 2018;69:77–103. doi: 10.1146/annurev-psych-010417-085944. [DOI] [PubMed] [Google Scholar]
  6. Brascamp J.W., Knapen T.H.J., Kanai R., Noest A.J., van Ee R., van den Berg A.V. Multi-timescale perceptual history resolves visual ambiguity. PLoS One. 2008;3:e1497. doi: 10.1371/journal.pone.0001497. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Chopin A., Mamassian P. Predictive properties of visual adaptation. Curr. Biol. 2012;22:622–626. doi: 10.1016/j.cub.2012.02.021. [DOI] [PubMed] [Google Scholar]
  8. Duncan K., Sadanand A., Davachi L. Memory’s Penumbra: episodic memory decisions induce lingering mnemonic biases. Science. 2012;337:485–487. doi: 10.1126/science.1221936. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Fischer J., Whitney D. Serial dependence in visual perception. Nat. Neurosci. 2014;17:738–743. doi: 10.1038/nn.3689. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Frässle S., Sommer J., Jansen A., Naber M., Einhäuser W. Binocular rivalry: frontal activity relates to introspection and action but not to perception. J. Neurosci. 2014;34:1738–1747. doi: 10.1523/JNEUROSCI.4403-13.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Friston K. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 2010;11:127–138. doi: 10.1038/nrn2787. [DOI] [PubMed] [Google Scholar]
  12. Friston K. A theory of cortical responses. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 2005;360:815–836. doi: 10.1098/rstb.2005.1622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Fritsche M., Mostert P., Lange F.P.de. Opposite effects of recent history on perception and decision. Curr. Biol. 2017;27:590–595. doi: 10.1016/j.cub.2017.01.006. [DOI] [PubMed] [Google Scholar]
  14. Fründ I., Wichmann F.A., Macke J.H. Quantifying the effect of intertrial dependence on perceptual decisions. J. Vis. 2014;14:1–16. doi: 10.1167/14.7.9. [DOI] [PubMed] [Google Scholar]
  15. Gekas N., McDermott K.C., Mamassian P. Disambiguating serial effects of multiple timescales. J. Vis. 2019;19:1–14. doi: 10.1167/19.6.24. [DOI] [PubMed] [Google Scholar]
  16. Hanley J.A. Statistical analysis of correlated data using generalized estimating equations: an orientation. Am. J. Epidemiol. 2003;157:364–375. doi: 10.1093/aje/kwf215. [DOI] [PubMed] [Google Scholar]
  17. Honey C.J., Newman E.L., Schapiro A.C. Switching between internal and external modes: a multiscale learning principle. Netw. Neurosci. 2017;1:339–356. doi: 10.1162/NETN_a_00024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Horga G., Abi-Dargham A. An integrative framework for perceptual disturbances in psychosis. Nat. Rev. Neurosci. 2019;20:763–778. doi: 10.1038/s41583-019-0234-1. [DOI] [PubMed] [Google Scholar]
  19. Hsu S.M., Wu Z.R. The roles of preceding stimuli and preceding responses on assimilative and contrastive sequential effects during facial expression perception. Cogn. Emot. 2020;34:890–905. doi: 10.1080/02699931.2019.1696752. [DOI] [PubMed] [Google Scholar]
  20. Iglesias S., Mathys C., Brodersen K.H., Kasper L., Piccirelli M., Ouden H.E.den, Stephan K.E. Hierarchical prediction errors in midbrain and basal forebrain during sensory learning. Neuron. 2013;80:519–530. doi: 10.1016/j.neuron.2013.09.009. [DOI] [PubMed] [Google Scholar]
  21. Kornmeier J., Bach M. The Necker cube–an ambiguous figure disambiguated in early visual processing. Vis. Res. 2005;45:955–960. doi: 10.1016/j.visres.2004.10.006. [DOI] [PubMed] [Google Scholar]
  22. Körding K.P., Wolpert D.M. Bayesian integration in sensorimotor learning. Nature. 2004;427:244–247. doi: 10.1038/nature02169. [DOI] [PubMed] [Google Scholar]
  23. Lawson R.P., Bisby J., Nord C.L., Burgess N., Rees G. The computational, pharmacological, and physiological determinants of sensory learning under uncertainty. Curr. Biol. 2020;31:163–172.e4. doi: 10.1016/j.cub.2020.10.043. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Leopold D.A., Wilke M., Maier A., Logothetis N.K. Stable perception of visually ambiguous patterns. Nat. Neurosci. 2002;5:605–609. doi: 10.1038/nn0602-851. [DOI] [PubMed] [Google Scholar]
  25. Liberman A., Fischer J., Whitney D. Serial dependence in the perception of faces. Curr. Biol. 2014;24:2569–2574. doi: 10.1016/j.cub.2014.09.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Logothetis N.K., Leopold D.A., Sheinberg D.L. What is rivalling during binocular rivalry? Nature. 1996;380:621–624. doi: 10.1038/380621a0. [DOI] [PubMed] [Google Scholar]
  27. Longuet-Higgins H.C. Visual motion ambiguity. Vis. Res. 1986;26:181–183. doi: 10.1016/0042-6989(86)90079-9. [DOI] [PubMed] [Google Scholar]
  28. Maloney L.T., Dal Martello M.F., Sahm C., Spillmann L. Past trials influence perception of ambiguous motion quartets through pattern completion. Proc. Natl. Acad. Sci. U S A. 2005;102:3164–3169. doi: 10.1073/pnas.0407157102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mamassian P., Wallace J.M. Sustained directional biases in motion transparency. J. Vis. 2010;10:23. doi: 10.1167/10.13.23. [DOI] [PubMed] [Google Scholar]
  30. Mathys C.D., Lomakina E.I., Daunizeau J., Iglesias S., Brodersen K.H., Friston K.J., Stephan K.E. Uncertainty in perception and the hierarchical Gaussian filter. Front. Hum. Neurosci. 2014;8:825. doi: 10.3389/fnhum.2014.00825. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. McGinley M.J., Vinck M., Reimer J., Batista-Brito R., Zagha E., Cadwell C.R., Tolias A.S., Cardin J.A., McCormick D.A. Waking state: rapid variations modulate neural and behavioral responses. Neuron. 2015;87:1143–1161. doi: 10.1016/j.neuron.2015.09.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Monto S., Palva S., Voipio J., Palva J.M. Very slow EEG fluctuations predict the dynamics of stimulus detection and oscillation amplitudes in humans. J. Neurosci. 2008;28:8268–8272. doi: 10.1523/JNEUROSCI.1910-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Palva J.M., Palva S. Progress in Brain Research. Elsevier B.V.; 2011. Roles of multiscale brain activity fluctuations in shaping the variability and dynamics of psychophysical performance; pp. 335–350. [DOI] [PubMed] [Google Scholar]
  34. Pearson J., Brascamp J. Sensory memory for ambiguous vision. Trends Cogn. Sci. (Regul. Ed.) 2008;12:334–341. doi: 10.1016/j.tics.2008.05.006. [DOI] [PubMed] [Google Scholar]
  35. Powers A.R., Mathys C., Corlett P.R. Pavlovian conditioning–induced hallucinations result from overweighting of perceptual priors. Science. 2017;357:596–600. doi: 10.1126/science.aan3458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Prado J., Carp J., Weissman D.H. Variations of response time in a selective attention task are linked to variations of functional connectivity in the attentional network. NeuroImage. 2011;54:541–549. doi: 10.1016/j.neuroimage.2010.08.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rosenberg M., Noonan S., DeGutis J., Esterman M. Sustaining visual attention in the face of distraction: a novel gradual-onset continuous performance task. Atten. Percept. Psycho. 2013;75:426–439. doi: 10.3758/s13414-012-0413-x. [DOI] [PubMed] [Google Scholar]
  38. Stephan K.E., Penny W.D., Daunizeau J., Moran R.J., Friston K.J. Bayesian model selection for group studies. NeuroImage. 2009;46:1004–1017. doi: 10.1016/j.neuroimage.2009.03.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sterzer P., Adams R.A., Fletcher P., Frith C., Lawrie S.M., Muckli L., Petrovic P., Uhlhaas P., Voss M., Corlett P.R. The predictive coding account of psychosis. Biol. Psychiatr. 2018;84:634–643. doi: 10.1016/j.biopsych.2018.05.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Teufel C., Fletcher P.C. Forms of prediction in the nervous system. Nat. Rev. Neurosci. 2020;21:231–242. doi: 10.1038/s41583-020-0296-0. [DOI] [PubMed] [Google Scholar]
  41. Urai A.E., Braun A., Donner T.H. Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias. Nat. Commun. 2017;8:14637. doi: 10.1038/ncomms14637. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Urai A.E., De Gee J.W., Tsetsos K., Donner T.H. Choice history biases subsequent evidence accumulation. eLife. 2019;8:e46331. doi: 10.7554/eLife.46331. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. VanRullen R. Perceptual cycles. Trends Cogn. Sci. 2016;20:723–735. doi: 10.1016/j.tics.2016.07.006. [DOI] [PubMed] [Google Scholar]
  44. Weilnhammer V., Röd L., Eckert A.-L., Stuke H., Heinz A., Sterzer P. Psychotic experiences in schizophrenia and sensitivity to sensory evidence. Schizophrenia Bull. 2020;46:927–936. doi: 10.1093/schbul/sbaa003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Weilnhammer V.A., Ludwig K., Sterzer P., Hesselmann G. Revisiting the Lissajous figure as a tool to study bistable perception. Vis. Res. 2014;98:107–112. doi: 10.1016/j.visres.2014.03.013. [DOI] [PubMed] [Google Scholar]
  46. Weilnhammer V.A., Sterzer P., Hesselmann G. Perceptual stability of the lissajous figure is modulated by the speed of illusory rotation. PLoS one. 2016;11:e0160772. doi: 10.1371/journal.pone.0160772. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Weilnhammer V.A., Stuke H., Sterzer P., Schmack K. The neural correlates of hierarchical predictions for perceptual decisions. J. Neurosci. 2018;38:5008–5021. doi: 10.1523/JNEUROSCI.2901-17.2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Wexler M., Duyck M., Mamassian P. Persistent states in vision break universality and time invariance. Proc. Natl. Acad. Sci. U S A. 2015;112:14990–14995. doi: 10.1073/pnas.1508847112. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wilson R.C., Collins A.G. Ten simple rules for the computational modeling of behavioral data. eLife. 2019;8:e49547. doi: 10.7554/eLife.49547. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Zalta A., Petkoski S., Morillon B. Natural rhythms of periodic temporal attention. Nat. Commun. 2020;11:1–12. doi: 10.1038/s41467-020-14888-8. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Video S1. Partially disambiguated Lissajous figure, related to figure 1

Please used red-blue filter glasses to view the stimulus.

Download video file (7MB, flv)
Document S1. Transparent methods and Figures S1–S4
mmc1.pdf (476.5KB, pdf)

Data Availability Statement

Original data and code have been deposited at OSF: https://doi.org/10.17605/OSF.IO/Y2CFM.

Articles from iScience are provided here courtesy of Elsevier

RESOURCES