Subjective visual perception: from local processing to emergent phenomena of brain activity

Abstract

The combination of electrophysiological recordings with ambiguous visual stimulation made possible the detection of neurons that represent the content of subjective visual perception and perceptual suppression in multiple cortical and subcortical brain regions. These neuronal populations, commonly referred to as the neural correlates of consciousness, are more likely to be found in the temporal and prefrontal cortices as well as the pulvinar, indicating that the content of perceptual awareness is represented with higher fidelity in higher-order association areas of the cortical and thalamic hierarchy, reflecting the outcome of competitive interactions between conflicting sensory information resolved in earlier stages. However, despite the significant insights into conscious perception gained through monitoring the activities of single neurons and small, local populations, the immense functional complexity of the brain arising from correlations in the activity of its constituent parts suggests that local, microscopic activity could only partially reveal the mechanisms involved in perceptual awareness. Rather, the dynamics of functional connectivity patterns on a mesoscopic and macroscopic level could be critical for conscious perception. Understanding these emergent spatio-temporal patterns could be informative not only for the stability of subjective perception but also for spontaneous perceptual transitions suggested to depend either on the dynamics of antagonistic ensembles or on global intrinsic activity fluctuations that may act upon explicit neural representations of sensory stimuli and induce perceptual reorganization. Here, we review the most recent results from local activity recordings and discuss the potential role of effective, correlated interactions during perceptual awareness.

1. Introduction

In the—not so remote—past, arguments in favour of a scientific approach to the problem of consciousness were met with skepticism. Arguably, a major hindrance was the first-person perspective of conscious experience that prohibited a quantitative approach to a process that is inherently subjective. However, in the psychological tradition, it was gradually realized that ambiguous stimuli could penetrate into features of conscious experience that are intersubjective and repeatable [1]—such as periods of subjective perceptual dominance and suppression as well as spontaneous perceptual alternations—and therefore used to unravel the general mechanisms mediating the instantaneous content of conscious perception [2,3].

Almost simultaneously with the first experimental study that combined ambiguous visual stimulation with extracellular electrophysiological recordings in the non-human primate brain [4], Crick & Koch [5] suggested that narrowing down and simplifying the search for the mechanisms of consciousness by studying the neural correlates of subjective perception in a single sensory modality could provide valuable insights into the mechanisms underlying all aspects of conscious processing. This proposal led to an explosion in the number of studies using multistable visual stimuli such as the Necker cube (figure 1a), Rubin's face-vase illusion (figure 1b), binocular rivalry (BR) (figure 1c), structure-from-motion (SFM), etc., which, instead of having a unique interpretation, are characterized by multiple (at least two) perceptual solutions and therefore provide an excellent method for studying the neural mechanisms of subjective, conscious perception (figure 1) [3,6].

Figure 1.

Paradigms of ambiguous visual stimulation used to study subjective perception. (a) Necker cube. Two possible interpretations of the cube are randomly switching back and forth in perception. The surface of the cube that appears in front suddenly retrieves to the back and disappears from perception only to be replaced by the perception of the surface that was previously not visible, in the back of the cube. Within a given temporal window, only one surface is consciously perceived, a phenomenon reflecting the struggle of the visual system to conclude on a unique conscious interpretation of the visual stimulus. (b) The face-vase illusion. Similar to the Necker cube two faces or a vase are continuously alternating in perception. (c) Binocular rivalry (BR). When two disparate visual patterns (here a face and a checkerboard) are continuously presented through a stereoscope in corresponding parts of the two retinas, ambiguity drives visual perception to fluctuate between the two competing stimuli although these stimuli remain unchanged. Periods of stimulus dominance are followed by perceptual suppression in an unpredictable manner, characterized by stochastic temporal dynamics. (d) Binocular flash suppression (BFS). Here, perceptual suppression is externally induced by a stimulus flash. Specifically, a stimulus is presented in one of the eyes and after one second, a second stimulus is flashed on the contralateral eye resulting in the perceptual suppression of the originally presented pattern (upper panel). In electrophysiological recordings, by changing the order of stimulus presentation, it is possible to detect the pattern of neuronal activity during the perceptual dominance or suppression of a preferred stimulus (lower panel). (Online version in colour.)

In BR (figure 1c), probably the most extensively used paradigm, each retina is stimulated with a different visual pattern, both occupying corresponding receptive fields. These conditions of sensory ambiguity elicit visual competition and the content of perception starts to fluctuate between each stimulus in a spontaneous and stochastic manner revealing the dynamic nature of neural phenomena that mediate perceptual alternations [6]. Therefore, each of the competing stimuli enters periods of perceptual dominance and suppression, gaining access to perceptual awareness or being suppressed and therefore momentarily disappearing from conscious perception. These states of perceptual dominance and suppression can also be studied using other forms of externally induced suppression of stimulus visibility like binocular flash suppression (BFS) [7] (figure 1d), generalized flash suppression (GFS) [8] (figure 2a) or motion-induced blindness (MIB) [12] that exploit the saliency of an external stimulus flash to induce the temporary invisibility of a target stimulus. However, continuous BR is unique for deciphering the mechanisms underlying the endogenously generated transitions between the competing stimuli [6].

Figure 2.

(a) In the paradigm of generalized flash suppression (GFS) a salient surround pattern induces perceptual suppression of a target (red dot) stimulus. Monkeys released a lever whenever the target stimulus was invisible. MUA increasing (blue solid lines) or decreasing (red solid lines) in response to perceptual suppression were found in dorsal and ventral pulvinar but not in LGN. Responses to physical stimulus disappearance are depicted with dashed lines (adapted from [9]). (b) Mean SUA responses to monocular physical alternation (left) and BFS (right) in V1 for preferred and non-preferred orientations of grating stimuli. In the right plot, the modulation of responses during flash suppression (t = 1000–2000) represents only a tiny fraction of the monocular, purely sensory, stimulation response (left plot) (adapted from [10]). (c) Mean SUA responses to monocular physical alternation (left) and BFS (right) in the LPFC. The modulation during BFS (right plot) is close to the modulation observed during perception without visual competition (left plot) (adapted from [11]). (d) Suggested corticothalamic network (LPFC-STS/IT-pulvinar) where subjective coding of perceptual dominance and suppression resembles the responses observed without any underlying visual competition. The pulvinar is depicted on the surface of the cortex for illustration purposes.

The neural correlates of perceptual dominance and suppression during visual rivalry are currently conceptualized to involve competing neuronal ensembles, each one holding an explicit representation of a competing sensory stimulus. Specifically, the activity dominance or suppression of each ensemble is thought to mediate the respective dominance or suppression of the encoded stimulus during subjective perception. The detection of such modulations has been the focus of the quest for the neural correlates of consciousness which are believed to reflect the minimal set of such neural responses that are both necessary and sufficient for conscious perception. When extracellular electrophysiological recordings in the non-human primate brain were combined with visual rivalry, such perceptually modulated neurons were detected across a plethora of visual brain areas. However, robustly modulated neurons that followed reliably the dominance or suppression of their preferred stimulus were less likely to be found in early, sensory cortical and thalamic areas but more likely in higher-order, association cortical areas and thalamic nuclei. Specifically, competitive forces in early visual areas are strong enough to prohibit inferring the consciously perceived or suppressed stimulus from the observation of neural responses, since firing rates are high for both the dominant and the suppressed pattern. However, in association areas competition has been resolved and neural responses reflect reliably the dominance or suppression of a preferred stimulus. As a result, it is currently believed that the increased strength of competitive interactions between neurons in early, sensory visual areas is related to processes occurring before the resolution of visual rivalry has been achieved, while the outcome of this processing and thus the content of conscious perception is reflected in association areas [6].

The major focus of this review is on studies undertaken in the non-human primate brain. The non-human primate studies are relevant to human consciousness due to the similarity of psychophysical measures in paradigms of bistable perception between humans and macaques and the evolutionary closeness between the macaque and human visual system. In §2, we provide the most recent overview of the current state of knowledge on the electrophysiological correlates of perceptual awareness, which derives almost exclusively from monitoring local activity. However, scrutinizing local activity by employing single-unit, single-electrode recordings lacks sufficient spatial resolution to reveal emergent phenomena such as mesoscopic and macroscopic patterns of electrophysiological activity whose dynamics have been proposed to be critical for the emergence of conscious perception [13–21]. Based on findings from local activity recordings, in §3 we discuss how population coding and dynamic systems theory, that emphasize the value of tracking simultaneously the activity of large populations of neurons within and across different areas, could contribute to a deeper understanding and refining of the mechanisms mediating perceptual awareness.

2. Locally recorded activity during conscious perception

In intracortical, extracellular recordings, a single electrode picks up the mean extracellular field potential (mEFP) which is the aggregate electrical activity generated by a local neuronal population. In the high-frequency range of the mEFP signal (approx. 0.6–6 kHz) it is possible to detect action potentials that typically cross a voltage threshold of 20–30 μV and reflect the discharge activity of a local neuronal ensemble within a radius of 140–300 μm around the electrode tip [22–24]. The total number of action potentials (spikes) recorded from a single electrode is commonly referred to as the multi-unit spiking activity (MUA), while similar spike waveforms detected in this band-passed signal are assumed to originate from the same single unit (single unit activity—SUA). Since this locally recorded spiking activity reflects the output of neurons within a local site, tracking local spiking activity during paradigms of subjective visual perception reveals the output of a local population and therefore the degree of explicit neuronal coding during periods of perceptual dominance and suppression.

Such local recordings of neuronal discharge activity during ambiguous visual stimulation helped to address five important issues in perceptual awareness. These are (a) The debate regarding the nature of competition during BR (i.e. monocular versus binocular neurons), (b) the identification of a network of brain regions where neuronal activity is more likely to reflect the content of perceptual awareness, (c) the profile of neural activity during perceptual suppression of a preferred stimulus across the visual processing hierarchy, (d) the neurodynamical mechanisms of perceptual transitions and (e) the role of subthreshold signals (local field potentials). These topics are presented in detail in the following sections.

(a). Local discharge activities resolve interocular versus stimulus feature competition during rivalrous perception

Some indirect evidence from psychophysical experiments implied that competition during BR could be instantiated in the brain as a competition between the mean discharge activities of two pools of monocular neurons, suggesting a model of interocular competition that mediates perceptual awareness [25,26]. In this model, monocular pools of neurons are the rivaling components, and their competition for activity dominance gives rise to the respective alternations subjectively experienced as alternations between the two stimuli presented in each eye. The interocular competition model underlines the importance of perceptual suppression between sensory-driven inputs originating in different monocular channels and not between representations of stimuli that are independent of the eye of origin [6,26,27]. Therefore, the direct experimental validation of this model using electrophysiological recordings could shed light on an elementary question, which is whether rivalry has a purely sensory substrate, involving competition between monocular inputs and dominance or suppression of monocular information, or if it involves a competition between stimulus representations that are not necessarily bound to the eye of origin. Single electrode recordings have the necessary spatial resolution to detect ocular dominance columns and monocular neurons and they were used to address this fundamental question.¹

In the retinogeniculate visual pathway, the lateral geniculate nucleus (LGN), a thalamic nucleus that is the first relay structure where visual input is processed before reaching the primary visual cortex (V1), contains neurons that respond to sensory input presented to a particular eye. If a strictly interocular competition model was valid, inhibition during BR could already be manifested in the activity of monocularly driven single units in this subcortical area of the visual pathway. However, electrophysiological recordings failed to reveal any effect of BR on the firing rates of monocular neurons in the LGN [28]. The absence of LGN spiking activity modulation during BR indicates that perceptual suppression is still undetectable in this stage of visual processing. This conclusion was confirmed by more recent experiments showing that neuronal discharge activity in the LGN is totally unaffected by perceptual suppression, even when stimulus invisibility is induced through other forms of perceptual illusions seen by both eyes, such as the GFS paradigm [9].

Further evidence against the interocular competition model in BR came from recordings in V1 and V2 which showed that only 14% of MUA and 20–25% of SUA in these regions significantly increase their firing rate when a preferred stimulus is consciously perceived and decrease their firing rate when the same stimulus is perceptually suppressed [10,29,30]. None of these studies found a larger contribution of monocular neurons or monocularly driven MUA in the rather weak traces of perceptual modulation during rivalrous perception. In fact, the overall trend suggests that most of the neurons and MUA sites that followed phenomenal perception in these primary and secondary sensory areas were binocular since they were driven equally well by both eyes. The results of these studies indicate that monocular channel information in LGN or V1 does not reflect the outcome of competitive interactions during BR and therefore the content of conscious visual perception could be a higher-order cortical representation related to the activity of binocularly driven, feature-selective neurons.

(b). Local discharge activities reveal a global network of explicit coding during perceptual awareness

Apart from arguments against a dominant influence of monocular information, SUA/MUA recordings revealed the low probability of LGN and V1 neurons reflecting the content of conscious perception (figure 2a,b), as well as the small magnitude of perceptual modulation in the discharge rates in these areas compared with monocular stimulation (i.e. without visual competition). Supporting the findings of V1 extracellular recordings during BR, a study in the same cortical area using bistable SFM stimuli found that only 20% of the neurons follow the content of subjective perception [31], while V1 and V2 neurons were not responsive to perceptual suppression induced by GFS [32,33] or MIB [34]. Such dissociation between neuronal activity in primary visual cortex and consciousness is also supported by other results showing that disparity-selective neurons in V1 are able to differentiate between local depth cues even in the absence of conscious stereopsis perception [35].

Although recordings during ambiguous stimulation in the primary visual cortex did not show any obvious temporal differences (i.e. early versus late components) in the amplitude of perceptual modulation between perceptual dominance and suppression (e.g. [29]), a late onset (more than 100 ms) component of spiking activity in V1 was found to be suppressed specifically when figure–ground segregation failed to be consciously perceived [36]. This finding indicates a mixture of activity in V1 where the initial transient responses are more related to sensory, stimulus-driven, modulation while the delayed response component correlates more to consciously perceived effects of perceptual organization, reflected in the segmentation between figure and ground [37]. This divergence of neural responses in V1 has been suggested to underlie an initial feed-forward sweep of activity which does not discriminate between conscious and non-conscious processing, and a later feedback loop that is related to conscious processing.

These findings suggest that neuronal activity in V1 despite being necessary is not sufficient for perceptual awareness since it does not represent a reliable and explicit neural correlate of subjective perception or perceptual suppression [38–40].² On the contrary, hints from psychophysics (reviewed in [27]) or the potential extrastriate, feedback source of the delayed component in V1 activity [41] indicate that later areas in the visual hierarchy could provide a more explicit correlate of conscious perception.

Indeed, more reliable correlates of subjective visual perception were found in extrastriate and higher-order association cortical areas. Specifically, in hierarchically intermediate cortical areas V4 and middle temporal (MT) cortex where neurons are also tuned to stimulus features such as orientation and direction of motion, SUA and MUA were modulated in accordance with the perceptual dominance and suppression of a preferred stimulus substantially more than in V1. However, even in this processing stage competitive forces between the rivaling populations were particularly strong, resulting in roughly 25–60% of the recorded neurons following the phenomenal perception of a preferred visual stimulus in paradigms such as BR, BFS or SFM [4,30,31,42–44]. Most interestingly, in MT the strength of perceptual modulation appears to be influenced by factors like the context of stimulus competition which, when taken into account, could boost perceptual modulation to 70–90% of the recorded neurons [42].

One step further in the visual processing hierarchy, the superior temporal sulcus (STS) and the inferior temporal (IT) cortices are the target of afferent projections from intermediate areas V4 and MT, and the source of efferent, feedback projections to V1. Neurons in this large cortical expanse exhibit preference for higher-order stimuli such as faces and complex objects. Electrophysiological recordings during BR and BFS in the temporal cortex demonstrated that almost 90% of the recorded units in STS and IT cortex reflect the phenomenal perception of a preferred stimulus [45]. Although not directly comparable to the temporal cortex, very similar responses were collected from recordings in the human medial temporal lobe (MTL) where almost 70% of feature selective neurons fired more during the phenomenal perception of their preferred stimulus during BFS [46]. Most importantly, the magnitude of firing rate modulation during ambiguous stimulation in these temporal areas is very similar to the magnitude observed during stimulation without any underlying stimulus competition, in striking contrast to the miniscule perceptual modulation in V1 which represents only a tiny fraction of the unambiguous sensory responses.

These findings indicate that neurons in the temporal lobe reflect the outcome of processes mediating perceptual awareness, where ambiguities and competition in the sensory input have been resolved and neural activity represents explicitly the content of subjective perception [6,27,45]. However, these temporal areas are not the final processing stage in the ventral visual stream that is involved in object perception [47]. Temporal cortical regions are reciprocally connected through monosynaptic connections with visually modulated areas of the lateral prefrontal cortex (LPFC) [48–51] where single units respond to faces and complex visual objects similarly to cells recorded in the temporal cortex [52–54]. Therefore, an intriguing question is whether the content of conscious perception is represented with the same magnitude in the LPFC. In a recent study, we found that the large majority of single units (approx. 60–90%, depending on the original strength of sensory modulation) and local MUA (approx. 75–95%) in the LPFC reflect the perceptual dominance and suppression of a preferred stimulus during 1 s of ambiguous stimulation externally induced by BFS [11] (figure 2c). It is also probable that in the prefrontal cortex there is some functional specialization of the mechanisms involved in subjective perception since the feature selective modulated units during BFS were mostly found in the inferior convexity of the LPFC, while neuronal correlates of perceptual transitions unrelated to stimulus preference were identified in the discharge rate modulation of single units in the macaque frontal eye fields, which predicted perceptual alternations during the paradigm of MIB [55].

These studies support the ‘frontal lobe hypothesis’ [38,39], suggesting that the neural correlates of conscious perception should have access to brain areas related to planning and decision making, such as the prefrontal cortex, in critical position to affect motor behaviour. In particular, our BFS study [11] showed conscious perception-related activity in the LPFC during passive fixation—that is without any planning, memory, decision making or motor component contaminating neural activities. Therefore, the current evidence strongly suggests that LPFC spiking activity reflects a relatively reliable correlate of conscious perception and not a consequence as proposed recently by Aru et al. [56]. However, findings indicating that unconsciously triggered control is feasible [57] and involves the prefrontal cortex [58,59] indicate that conscious prefrontal processing is not a prerequisite for control processes (see also [60]). Therefore, the ‘frontal lobe hypothesis’ has to be re-evaluated.

In summary, electrophysiological recordings during ambiguous visual stimulation suggest that neurons in the temporal and prefrontal cortices reflect perceptual dominance and suppression much more robustly than the respective neurons in striate and extrastriate cortical areas. Interestingly, temporal and prefrontal areas are reciprocally connected, not only through corticocortical monosynaptic connections, but also indirectly through a subcortical pathway involving the higher-order, pulvinar thalamic nucleus [61–65]. It is thus not surprising that during GFS the magnitude of perceptual modulation of discharge activity in the pulvinar, which receives afferents from the frontal and temporal cortices, is not only detectable compared to LGN, but also close to the magnitude observed in the temporal and prefrontal cortices [9]. Specifically, 40% and 60% of the recorded sites in the ventral and dorsal pulvinar, respectively, reflect perceptual suppression. The somewhat lower perceptual modulation of spiking activity in the ventral pulvinar could be associated to the stronger connectivity of this part of the pulvinar with the primary visual cortex [65] where the correlates of awareness are weak. In contrast, dorsal areas of the pulvinar communicate mainly with association cortical areas.

These results suggest that prefrontal and temporal areas of the cortex as well as the pulvinar could form a cortico-thalamo-cortical network that represents reliably the content of subjective visual perception and reflects perceptual suppression (figure 2d). Most importantly, these findings demonstrate that explicit, conscious processing and perceptual suppression are not localized in a unique cortical area but they should rather be conceptualized as an emergent property of a global network involving at least two cortical areas (i.e. temporal and prefrontal cortices) and the pulvinar thalamic nucleus. It has been suggested that due to the absence of a well-defined parcellation of the cortical topography in the pulvinar, rival populations in the cortex could be in competition to recruit thalamic elements in order to outlast each other in activity [66]. In the context of subjective perception, it is possible that when neuronal populations in prefrontal and temporal visual areas are in a dominant state during BR, long-range binding through the pulvinar results in a beneficial state that outlasts the suppressed, rival cortical populations. In the future, simultaneous recordings in IT, prefrontal cortex and the pulvinar could reveal dynamic interareal interactions that mediate both perceptual dominance and suppression states as well as spontaneous perceptual alternations. The identification of this network of explicit encoding and perceptual suppression during subjective visual perception could help to elaborate and constrain models studying access to consciousness that emphasize the importance of recurrent corticothalamic networks forming a global functional workspace as a necessary condition for conscious perception [67–69]. Studies using electroencephalography (EEG) and magnetoencephalography (MEG) techniques have already started revealing such macroscopically observed, global functional networks that involve coherent activity between frontal, temporal, parietal and occipital areas predicting conscious perception or perceptual alternations during ambiguous stimulation [13,14,16,19,20,70]. Both findings from local, single electrode electrophysiological recordings and EEG/MEG could guide simultaneous intracortical recordings which would allow fine spatial resolution and analysis of coherent structures for stimulus-specific spiking activity which is not feasible in EEG/MEG measurements.

(c). Local discharge activities suggest two modes of non-conscious processing during perceptual suppression

Neural processing of a preferred stimulus could also continue during its subjective invisibility. The magnitude of SUA and MUA firing when a preferred stimulus is perceptually suppressed could therefore indicate the strength of non-conscious processing during perceptual suppression. In some cases, this processing is strong enough to induce a complete reversal of the monocularly induced discharge patterns, resulting in higher firing rate when the preferred stimulus is suppressed compared to the respective rate when the same stimulus is dominant. The spatial extent of this type of processing appears to be constrained in striate and extrastriate areas. On the other hand, weaker traces of ongoing but rather residual neural processing during perceptual suppression can be found in almost all areas of the cortical hierarchy as far as in the LPFC. Therefore, although weak, traces of continuous processing during perceptual suppression appear to be present in both sensory and association areas, indicating that explicit, conscious coding and ongoing non-conscious processing during perceptual suppression could coexist within the same cortical network.

More specifically, V1, V4 and V5 (MT) showed significant evidence for the first mode of non-conscious processing during perceptual suppression, since in these areas a fraction of the perceptually modulated selective neurons and MUA (approx. 20–50%) increase their firing rate when their preferred stimulus is perceptually suppressed [4]. In these areas, the particularly strong response profile during perceptual suppression could indicate that some neurons or local populations are more sensitive to non-conscious processing during perceptual suppression compared with processing during perceptual dominance. It has also been suggested that this mode of non-conscious processing during perceptual suppression reflects the perturbation of processes involved in perceptual grouping through feed-forward and feedback connections between different visual areas [27].

In striking contrast, such strong non-conscious stimulus processing during perceptual suppression is nearly absent in the spiking activity of both human MTL and macaque STS/IT cortex where none of the modulated cells consistently fired more during the perceptual suppression of a preferred stimulus [45,46]. Similarly, in the LPFC only a small percentage of SUA and MUA (approx. 5%) were modulated during the perceptual suppression of a preferred stimulus [11]. However, in both the prefrontal and temporal cortices the firing rate of the recorded populations during perceptual suppression of a preferred stimulus is slightly elevated compared to physical removal of the same stimulus (see, for example, the LPFC spiking response in figure 2c and the modulation during perceptual suppression in [45, fig. 5]). This firing pattern might reflect a second mode of non-conscious processing during the course of perceptual suppression with residual characteristics that can still be detected in association cortical areas, even after visual competition has been resolved.

These findings from BR and BFS are in accordance with other studies of conscious perception using paradigms of visual masking and show that non-conscious processing of visual signals can be detected all the way up to the prefrontal cortex (for a review, see [59,67]).

(d). Mean local discharge rates and neurodynamical mechanisms of perceptual transitions

Although the SUA and MUA electrophysiological studies revealed the pattern of neuronal discharge activity across different brain areas during stable states of perceptual dominance and suppression, it is still not clear what the mechanism is that underlies a spontaneous switch in the dominant activity and therefore a spontaneous perceptual alternation during BR. This is a key issue since the underlying mechanism could reflect the processes allowing access of a particular content to consciousness. Most of the work on this question has been performed in theoretical models using methods of computational neuroscience where different neurodynamical mechanisms have been proposed to mediate the competition dynamics between the rivaling neuronal pools, involving features such as adaptation, cross-inhibition and noise [26,71–86].

The vast majority of these models examine two possible mechanisms that could be responsible for an alternation in the dominance between antagonistic neuronal populations, as observed in BR. The first alternation mechanism is adaptation-driven with cross-inhibition between the two competing pools, and assumes that a slow adaptation process resulting in fatigue of the discharge activity in the dominant pool results in a subsequent decrease of cross-inhibition. The cross-inhibition decrease leads to a disinhibition of the suppressed population which can take over the competition, becoming active and inactivating the originally dominant population [26,71,73,76–79,82–86]. The potential importance of adaptation in BR is well supported by psychophysical studies showing a gradual decrease in the strength of a dominant stimulus and an increase in the probability of a perceptual switch as a function of increasing dominance duration [7,71]. This indirect psychophysical evidence for the significance of adaptation is confirmed by electrophysiological observations in the temporal and prefrontal cortices, which reveal a gradual adaptation of the mean population discharge response during the perceptual dominance of a preferred stimulus [11,45].

The second mechanism is known as noise-driven and assumes that the underlying neurodynamical system is bistable. Here, noise is the main source inducing a transition between the two states (minima), by causing a jump over the barrier (maximum) separating the two stable attractors of the system [71,73,75,78,81–83,85,87]. In these theoretical models when noise is absent, alternations are impossible since the system relaxes indefinitely in one of the two attractors that represent the two possible states of perceptual dominance. The most important evidence for the role of noise in perceptual transitions is the stochastic temporal dynamics of the perceptual transitions observed during BR.

Recent theoretical work suggests that both adaptation and noise operating in a fine balance induce the stochastic properties of perceptual transitions observed in BR [71,83–85,88]. In a recent study [88], we attempted to constrain a mean-field reduced model of rivalrous perception (figure 3a, left) with the mean population response observed in the LPFC during BFS by detecting in which space of a bifurcation separating noise from adaptation regimes (figure 3a, right) the neuronal and behavioural correlates of BFS and BR overlap. Our simulations showed that the mean population discharge pattern observed during externally induced perceptual alternations (BFS) in the LPFC can be obtained only in a narrow region around the bifurcation separating a noise-driven, attractor regime from an adaptation-driven, oscillatory regime (figure 3b). Therefore, it is probable that neither noise nor adaptation forces alone have a primary, crucial role in externally induced (BFS) perceptual switches. The population discharge pattern observed in the LPFC during BFS could be the result of either adaptation or noise alone. However, the fact that all BR and BFS experimental constraints (psychophysical and electrophysiological, respectively) overlap in a narrow region before the bifurcation (figure 3c) provides the first electrophysiologically constrained indication that a noise-driven attractor mechanism might be active during spontaneous perceptual transition in BR.

Figure 3.

(a) Left: Schematic of the unified noise and adaptation competition model. U_i denotes the population rate on each competing neuronal population i. I_i denotes the external visual input on each population. Mutual inhibitory connections β are represented by filled circles and recurrent excitatory connections α by arrows. The recurrent dashed arrows on each population symbolize a slow adaptation process (rate frequency adaptation), where the adaptation variable is denoted by H_i on each population i. Right: Bifurcation analysis of the noise-free dynamic system as a function of the strength of the adaptation process (φ_H). A bifurcation from a bistable (‘attractor’) regime to an oscillatory (‘oscillations’) regime occurs at 0.45. Left of the bifurcation, a double-branched bistable region (solid lines) emerges. The upper (green) and lower (orange) branches correspond to the high and low activity of the dominant and suppressed population, respectively. Right of the bifurcation, an oscillatory region emerges. In this region, the maximum and the minimum (green and orange circles) of the populations’ activity during rivalry periodic oscillations are shown. (b) Bifurcation analysis of BFS simulation. Upper panel: Flash suppression dynamic range of the competition neuronal model for the noise free case. Flash suppression is characterized by plotting the maximum value of the populations’ activity (U₁ in red and U₂ in blue) during the last second of the simulations. Flash suppression corresponds to the region where only the second flashed population U₂ shows high activity and the first stimulated population U₁ is suppressed. Lower panel: Temporal evolution of the populations’ firing rate activity U₁ (red) and U₂ (blue) for progressively stronger levels of adaptation (black arrows). Only the type of simulated neuronal behaviour observed in the region labelled ‘flash suppression’ is similar to the electrophysiological observations in the LPFC. (c) Flash suppression (FS), BR dominance time (Tdom) and BR coefficient of variation (CV) as a function of the adaptation level φ_H and with a level of noise σ optimized for each point such that the FS index (number of effective trials/100) is maximal and the CV is as near to 0.5 as possible. There is only one dynamic region where the model is consistent with experimental constraints; that is the narrow, noise-driven, region just before the bifurcation at 0.45. (d) Statistical features such as correlated variability could affect population coding. Here, ΔI_shuffled is a measure of the effect of spike count correlations on the amount of information in a population code. Ellipses represent 95% confidence intervals of the response distributions while the diagonal lines show the optimal decision boundary between stimulus 1 and stimulus 2. The larger the overlap of the ellipses the less information is contained in the population code. Correlations affect ΔI differentially depending on the similarity of the tuning between the two units. During ambiguous stimulation such statistical relationships in the firing of the competing populations could be relevant for perceptual dominance, suppression and perceptual transitions (adapted from [89]).

The mechanism that is validated by the mean discharge response during BFS, implies that adaptation progressively decreases the stability of the perceptually dominant state [83], but just until a critical point where the state becomes unstable. Therefore, adaptation is not enough to induce a transition without the dominant influence of noise which drives the alternation even before the dominant state becomes unstable.

The neuronal responses used to constrain this model were recorded from the macaque LPFC where they reflect reliably the content of conscious visual perception [11]. Therefore, the results and the interpretation of this modelling study are under the assumption that the dynamics of competition at this stage of cortical processing, acting between explicit representations of neuronal activity, is the critical factor for perceptual transitions. Indeed, the human prefrontal cortex is also involved in the temporal dynamics of perceptual alternations during subjective perception as evidenced in patients with prefrontal cortex lesions showing substantial ‘freezing’ of alternations [90–92]. However, a dominant role for the prefrontal cortex during perceptual transitions remains to be experimentally confirmed. In this context, a much needed study is to record electrophysiological activity in the prefrontal cortex during BR to exclude the possibility that distinct neural mechanisms support dominance and suppression in different paradigms.

(e). Local field potentials and perceptual awareness

In contrast to spiking activity, the local field potential (LFP) is a component of the extracellular field potential that is much slower (approx. 0.1–200 Hz) than spiking activity, and is believed to reflect excitatory and inhibitory postsynaptic potentials and therefore synaptic input and dendritic processing [93–95] within a radius between 500 μm up to several millimetres around the tip of the recording electrode. Thus, although recorded locally, LFP reflects the integration of signals across a larger area compared to MUA and represents more the input and local processing than the output of a local population. Therefore, studying local LFPs in combination with MUA could contribute to our understanding of the specific computations and input–output transformations performed in a given cortical or thalamic area during subjective perception, for example, revealing modulations that are detected in the input and local processing, but not in the output of a brain region.

Indeed, LFP recordings during perceptual suppression seem to resolve a dispute between electrophysiological studies and functional magnetic resonance imaging (fMRI) studies regarding the modulation of LGN and V1 during perceptual awareness. In particular, a number of fMRI studies show significant perceptual modulation in the blood-oxygen-level-dependent (BOLD) signal in LGN and V1 during BR [96–101]. High-frequency (gamma) oscillations in LGN and V1/V2 exhibit negligible perceptual modulation in BR and GFS, similar to spiking activity [9,10,29,32,33].³ This finding is not surprising since high-frequency (‘gamma’) LFP power is affected mostly by local discharge activity, while low frequencies reflect mainly neuromodulatory processes [104–106]. However, the power of low-frequency LFPs is more consistently modulated [9,10,29,32,33] and a study that combined fMRI with electrophysiological recordings during GFS found that suppression is indeed reflected in the BOLD and low-frequency LFPs (5–30 Hz), but not in spiking activity and high-frequency LFPs [33]. Furthermore, the modulation of LFPs in V1 appears to be laminar dependent, with supragranular layers being more modulated than infragranular grey matter [107]. This result may indicate that modulation of low-frequency LFPs and BOLD signal in V1 is either due to feedback input from extrastriate cortex where perceptual suppression modulates spiking activity, or due to modulatory signals of subcortical origin. However, the fact that low-frequency LFPs are more likely to reflect processes performed in a larger spatial scale compared to high-frequency LFPs, indicates that the modulation observed during perceptual suppression is most probably non-specific, targeting indiscriminately dominant and suppressed populations.

By contrast, in association cortical areas such as the LPFC the power of high-frequency gamma oscillations was more consistently modulated in sites where MUA also followed subjective stimulus visibility [11]. Therefore, the modulation of gamma LFPs in the LPFC could reflect fairly local input originating from modulated populations in the temporal cortex. On the other hand, beta (15–30 Hz) oscillations in the LPFC exhibited a different, distinctive pattern characterized by desynchronization followed by rebound of activity regardless of the local dominance or suppression of the local neuronal populations and therefore independent of conscious or unconscious processing. Beta oscillations have been suggested to reflect control processes in the frontal cortex and this dissociation between beta LFPs and local processing of conscious perception may underlie a similar dissociation between control functions and consciousness [60].

In a sense, a locally recorded LFP reflects an emergent signal arising from non-linear synaptic and dendritic operations corresponding to integration of inputs in a local site (with differences in the spatial resolution of the signal depending on the frequency examined). Therefore, locally recorded LFP is a signal that reflects the complexity and dynamics of local computations occurring within a few millimetres of the electrode tip and represents the only mesoscopic intracortical signal that has been so far examined with extracellular recordings during perceptual awareness. In §3, we discuss the potential involvement of more complex, emergent patterns of discharge and oscillatory activity of neural populations in conscious perception.

3. Correlated activity and self-organization processes

(a). Correlated neuronal activity and population coding in perceptual awareness

Apart from the LFPs, slow and correlated fluctuations in the activity of neuronal populations can also be detected in the coherence of discharges across neurons. These spike-to-spike coherence measures have been associated with the magnitude of correlated variability or noise across neuronal populations [108]. So far, in every single electrophysiological study of visual awareness, the spiking activity of single units and recorded sites has been studied in isolation and the difference in the magnitude of their averaged firing rate during perceptual dominance and suppression was the measure used to infer the relative contribution of a given cortical area in perceptual awareness. However, due to the probabilistic characteristics of neuronal coding, statistical features such as the magnitude and structure of noise could have important consequences for population coding [89,109–113]. Therefore, measuring and assessing the effects of correlated noise within and between the competing neuronal pools could impose significant constraints on stimulus coding during subjective visual perception.

Specifically, the total amount of noise in a neuronal ensemble can be described by the covariance matrix that includes both individual and interneuronal, correlated, variability [113]. Interneuronal variability is commonly referred to as ‘noise correlation’ and its impact on neuronal information processing has been studied extensively in a plethora of processes such as stimulus drive [114], neuronal adaptation [115,116], perceptual discrimination [117], attention [108,118–120], perceptual and associative learning [121,122] and behavioural context [123]. In these studies, correlations were shown to be detrimental, beneficial or not relevant to the efficiency of population codes, depending on their structure and magnitude. In general, for neuronal populations that have similar tuning (or ‘signal correlation’), correlations are believed to be detrimental since pooling neuronal responses is not able to average out common noise fluctuations. By contrast, correlations are thought to be beneficial for neuronal populations with opposite tuning (figure 3d).

It is therefore important for future electrophysiological studies investigating visual consciousness to record simultaneously from multiple neurons and compare the variability in neuronal discharges during visual rivalry to the respective variability during perception without any underlying visual competition in order to re-evaluate the efficiency of population signal averaging, that is currently thought to encode the content of perceptual awareness, particularly in higher-order cortical areas [38,39]. For example, these measurements could either reinforce the current belief that V1 is indifferent to subjective visibility or reveal properties of neuronal coding that are not reflected in the discharge rate but rather in more fine population properties such as the noise entropy [124] which could potentially represent dominance and suppression. In a similar fashion, for association cortical and thalamic areas, where firing rates reflect dominance and suppression, it is necessary to study how visual competition affects neuronal variability and therefore population coding during subjective perception in order to confirm their role in explicit coding during conscious perception.

Most importantly, it is probable that spontaneous fluctuations of correlated firing within and between competing ensembles (for example, spontaneous transitions from correlated to decorrelated states) could be related to spontaneous perceptual transitions. It would be particularly interesting to associate both theoretically and experimentally such intrinsic neurophysiological fluctuations of coherent population activity to the theoretical concept of noise used in computational models of spontaneous perceptual transitions.

Recently, a role for ongoing fluctuations of cortical activity in conscious perception was indeed detected in functional imaging studies that found a relationship between stimulus awareness and prestimulus ongoing fluctuations of the BOLD signal [125,126]. However, the spatial resolution of fMRI is not adequate to reveal fluctuations within each antagonistic ensemble as well as between them and uncover the role of such population-specific fluctuations in determining the perceptual outcome. On the contrary, due to limitations in the spatio-temporal resolution of fMRI, these BOLD fluctuations may be more related to ultra-low-frequency (less than 0.1 Hz) fluctuations that span neuronal populations regardless of their tuning and feature-specific properties.

(b). Mesoscopic spatio-temporal activity patterns as internal context and a potential role in perceptual reorganization

The endogenously generated perceptual transitions observed in paradigms of multistable perception reflect a, yet unknown, mechanism of state transitions in brain activity. Until now the focus of research has been on the dynamics of competition between rivaling pools of feature selective neurons. However, it has been suggested that the source of this perceptual reorganization process might be independent of the dynamics of antagonistic ensembles and rather reflect a general mechanism underlying perceptual selection that induces the transitions in feature-specific neuronal populations only as a secondary effect [127]. This selection mechanism has been described as a form of involuntary exploratory attention shifting [127] and could be related to ongoing fluctuations of neural activity [125,126] and association areas such as the prefrontal cortex where both BOLD and electrophysiological signals reflect perceptual transitions [55,128] and have a crucial role in representing internal contextual and motivational states assumed to result in perceptual reorganization [127].

Indeed, contextual influences on subjective perception have been well-documented psychophysically (for a review, see [129]) but there is a lack of electrophysiological studies examining how internal context is associated with perceptual reorganization. A candidate neural substrate of internal context could be provided by spatio-temporal patterns of electrophysiological activity whose intrinsic dynamics could offer a link between the state transitions observed in ongoing brain activity and spontaneous perceptual transitions [21,130,131]. The theoretical framework for such state transitions is well established [15,17], and pioneering work by Walter J. Freeman has revealed the dominant role of nonlinear mesoscopic brain dynamics in creating a context for external stimulation (for a review, see [131]). The availability of optical imaging and multielectrode extracellular recording methods (e.g. Utah arrays, tetrode arrays) has already demonstrated the existence of such complex spatio-temporal patterns (travelling waves, spiral waves, etc.) in the mammalian brain both in vitro and in vivo (figure 4) [132–134]. The spontaneous transitions between different mesoscopic patterns are believed to occur due to changes in the effective connectivity of the spatially extended coupled oscillators and could be of great computational significance [135–137]. It is an essential step to perform such invasive recordings of spatio-temporal activity during perceptual awareness tasks and examine how the dynamics of effective connectivity leading to self-organized activity patterns are related to the dynamics of perceptual dominance, perceptual suppression and perceptual reorganization.

Figure 4.

(a) Mesoscopic patterns observed in vitro using voltage sensitive dyes (VSDs) in the rat visual cortex. A planar wave in the upper pannel is replaced by a spiral wave starting a few frames after the frame marked with an arrow (adapted from [132]). (b) Spatio-temporal patterns can be frequency-specific as in the motor cortex where beta oscillations are organized as travelling waves of propagating activity (adapted from [133]). (c) In the rat hippocampus frequency-specific travelling waves can be observed in the theta LFP band [134]. (d) Phase differences resulting in propagating spatio-temporal patterns can be explained as delayed excitations resulting from a single oscillator, propagating pulses in an excitable network or stable, phase-locked differences between weakly coupled oscillators. Spontaneous changes in the effective interactions between oscillators could result in state transitions between different patterns that might be of great computational value and we hypothesize that they could also be involved in spontaneous transitions in perceptual awareness (adapted from [135]).

4. Conclusion

Although local recordings of electrophysiological activity advanced our understanding of perceptual awareness, they also confirmed that the underlying mechanisms involve multiple areas and populations across the brain. In particular, neurons in higher cortical (LPFC, IT/STS) and thalamic (pulvinar) association areas are more heavily involved in coding explicitly the contents of consciousness and could therefore form a global network of explicit coding, associated to concepts involving a global neuronal workspace in conscious perception. Future efforts should concentrate on understanding the role of functional interactions and spatio-temporal patterns formed by the activity of neuronal populations within and across these areas but also in areas where SUAs are seemingly not correlated to subjective perception. Such studies, along with the development of more elaborate paradigms that disentangle different stages in conscious perception [56] could further enhance our understanding of the mechanisms mediating the emergence of visual consciousness.

Funding statement

This research was supported by the Max Planck Society.