The interplay of attention and consciousness in visual search, attentional blink and working memory consolidation

Abstract

Despite the acknowledged relationship between consciousness and attention, theories of the two have mostly been developed separately. Moreover, these theories have independently attempted to explain phenomena in which both are likely to interact, such as the attentional blink (AB) and working memory (WM) consolidation. Here, we make an effort to bridge the gap between, on the one hand, a theory of consciousness based on the notion of global workspace (GW) and, on the other, a synthesis of theories of visual attention. We offer a theory of attention and consciousness (TAC) that provides a unified neurocognitive account of several phenomena associated with visual search, AB and WM consolidation. TAC assumes multiple processing stages between early visual representation and conscious access, and extends the dynamics of the global neuronal workspace model to a visual attentional workspace (VAW). The VAW is controlled by executive routers, higher-order representations of executive operations in the GW, without the need for explicit saliency or priority maps. TAC leads to newly proposed mechanisms for illusory conjunctions, AB, inattentional blindness and WM capacity, and suggests neural correlates of phenomenal consciousness. Finally, the theory reconciles the all-or-none and graded perspectives on conscious representation.

1. Introduction

Trying to understand consciousness and its neural basis is a major scientific challenge. While empirical findings from different experimental paradigms (see [1–6] for reviews) provide important contributions, these results need theories and models of consciousness and related functions for a coherent picture to emerge [5–9].

A pioneering theory of consciousness is Bernard Baars’ global workspace (GW) theory [5,10–12]. According to this theory, unconscious processes take place in local modules distributed across the brain, while conscious processing involves global access to these widespread modular representations. For conscious processing, this architecture requires a central processor, the GW, which is able to access information from widespread cortical and subcortical sources. Conversely, the GW can broadcast information on a global brain scale and thus provide a gateway to extensive perceptual and memory representations. The GW theory has been revised over the past few years to emphasize dynamic aspects and processing flexibility; in its most recent version the GW is considered a functional capacity for dynamically coordinating and propagating neural signals over different task-related networks [13].

Ned Block [6,14] endorsed the GW in his notion of access consciousness, which he distinguishes from phenomenal consciousness. The related notion of access to consciousness plays a central role in the influential global neuronal workspace model developed by Dehaene and collaborators [7,15–20]. The model constitutes one of the most developed approaches to the neural basis of human consciousness in cognitive neuroscience to date. We will follow its lead in our present theoretical effort.

Independently of the developments in theories and models of consciousness, concepts similar to the GW have been proposed in theories of attentional selection in visual perception. In some of these theories [21–23], attentional selection occurs late, at the level of encoding in (visual) working memory (WM) [24,25], after a stage of parallel object recognition. In the integrated object competition hypothesis [21,23], objects compete as integrated units across multiple visual and non-visual representation systems, akin to the GW. Also the locus of attention in the processing hierarchy is similar to that for conscious access in the GW. However, the integrated object competition hypothesis does not make a distinction between modular and integrated processing; before and after selection the units of representation are always integrated objects.

Early-selection theories, by contrast, assume stages similar to those in GW theories. For instance, in Anne Treisman's [26–29] influential feature integration theory, selection is needed for integration of featural information into object representations, whereas single features can be represented ‘preattentively’ at an earlier stage. This distinction mirrors that in GW theory: a first stage of parallel representation in different modules without capacity limitations, before one of the items, e.g. a visual object, is selected for access [16–18]. However, the locus of selection differs between these theories. Whereas the GW involves higher-level processes associated with prefronto-parietal areas [7,15–20], in feature integration theory [26–29] and the guided search model [30–32], attentional selection is situated at mid-level, with visual feature binding and object identification, i.e. before conscious access. These processes are typically associated with the visual cortex. Although both theories, therefore, are focused on processes such as selection and integration of information, they crucially differ about the levels of representation and processing, as well as the brain areas involved.

To our knowledge, the global neuronal workspace model does not specify how the GW relates to perceptual processes involving selective attention. We will address this issue here, together with how the GW relates to WM. We will sketch a theory of attention and consciousness (TAC) on the interplay of attentional and conscious processing. TAC aims ultimately to provide a complete description of stages and levels of representation, from early perceptual processing to encoding in WM, through multiple processing steps involving attentional selection and conscious access. TAC envisages being a synthesis of key aspects drawn from different theories of visual attention: parallel versus serial mechanisms, information integration, and levels of representation and selection.

In §§2–5, we review and discuss experimental findings, theories and models that are relevant to TAC. Section 2 considers notions from the global neuronal workspace model about processes, dynamics and levels of representation that are being extended in TAC to attentional processing. Section 3 focuses on the distinction between attentional selection and conscious access. Section 4 provides a review of findings and theories regarding attentional selection, in particular, visual search [22,27]. Section 5 considers the two ‘interface’ phenomena of the attentional blink (AB) and (visual) WM consolidation central for TAC. These phenomena have independently, or sometimes conflictingly, been accounted for by theories and models of conscious access and attentional selection. After having presented TAC in §6, in §7, we discuss implications of the proposed theory.

2. Insights from the global neuronal workspace model

(a). Global workspace processes and dynamics

The global neuronal workspace model assumes a first stage, in which feature information is represented in sensory maps. Information representing multiple items is processed in a parallel fashion, without involvement of attention. Representations reaching minimal thresholds of duration and intensity can be amplified by top-down attention, and enter a second stage of representation in the GW, via a winner-takes-all dynamics. The winning information is globally available for access and report [18]. The GW operates like a representational and processing ‘bottleneck’: only one large-scale reverberating neural assembly in the thalamo-cortical system, integrating information from multiple brain modules, remains active at any given moment in time [16,17]. Discrete transitions from one winner-takes-all assembly to the next take place in a serial manner [16].

Zylberberg et al. [33] recently proposed a framework for serial GW operations, involving both discrete processing steps and massively parallel interactions during each time-step (see also [34,35]). This framework involves continuous and parallel neuronal operations for evidence accumulation and discrete (transient and sustained) ones for response selection, decision, memorization and top-down cognitive control. TAC endorses this integrated approach.

In the global neuronal workspace model, subsets of GW neurons interact cooperatively for coherent encoding of mental objects, such as visual or memory items, and cause inhibition of neurons coding for competing ones [16,17]. This process is likely to be reflected in brain activity by states (modes) of enhanced and highly correlated (coherent or synchronous) neural activity within assemblies of distributed neurons coding for a given object. Entering such a state is characterized by amplification of local activity and subsequent ignition, i.e. a sudden rise in activation and coherence involving multiple, even distant areas in the thalamo-cortical system. Simulation studies have shown that these states can be entered rapidly [17].

Whereas these simulation studies lead to the prediction that conscious access and related ignition in the thalamo-cortical system of the brain occur rapidly, electrophysiological evidence suggests that ignition related to conscious access takes place rather slowly after stimulus onset, with a delay of about 300 ms [20]. This observation appears consistent with our current theoretical perspective, which suggests that there are multiple stages preceding conscious access. In particular, TAC proposes that a crucial stage prior to conscious access is attentional selection, taking place between early perceptual representations and later encoding in WM.

TAC proposes that attentional processes involve coherent cooperative and competitive neural interactions, amplification, ignition (i.e. rapid transition) of neural activity patterns, as well as evidence accumulation, just like for conscious access in the GW. The global neuronal workspace model has considered these processes within the GW [16,17]. In neurocomputational modelling studies, we showed that similar dynamics could also account for the limited capacity of visual working memory (VWM) and feature binding in visual object maintenance [36,37].

(b). Levels of representation and conscious access

Dehaene & Naccache ([15]; see also [18]) introduced a useful classification in which a subset of information encoded in the brain is inaccessible at a conscious level of representation (set I1), another set of information could be consciously amplified and accessed if it was attended to (set I2), with only a subset of the latter being mobilized into the GW (set I3) in a winner-takes-all process. Set I1 refers to unconscious representation, set I2 to preconscious representation and set I3, given by a selected assembly of widely distributed, activated and interacting neurons, to conscious representation. More recently, Ned Block [14] has reconsidered sets I2 and I3 in terms of broad (set I2) versus narrow (set I3) cognitive accessibility, related to his earlier distinction between phenomenal and access consciousness [6]. However, these distinctions are not explicitly related to stage-wise processing. In our proposal, while we maintain that set I1 is unconscious and I3 is conscious, we consider processing and representational steps that mediate between set I2 and set I3, i.e. for the transition from preconscious to conscious representation.

Dehaene and Naccache's view suggests that, with conscious access, there is a major discrete (all-or-none) transition from representations at the level of set I2 to set I3. This transition is believed to depend on the involvement of prefronto-parietal cortical areas for providing top-down attention (amplification). By contrast, the transfer from set I1 to set I2 would be a gradual process, depending on activation spreading in posterior cortex [18]. Yet, top-down attention may not only operate for the amplification of representations in set I2 for all-or-none access in set I3, but also can take the form of graded, continuous and attentional modulation of relatively early perceptual representations [38].

Our current perspective takes into consideration that representation and conscious accessibility of perceptual information are dynamic, i.e. subjected to change over time. Indeed, in some conditions consciously accessible information in I2 can return to an inaccessible level (I1). For example, in the AB phenomenon (see below), previously accessible information about the second target (in I2) can subsequently become inaccessible (in I1) due to a combination of masking [39–41] and selection failure [42]. Unconscious, preconscious and conscious neural representations appear to a remarkable extent dynamic and interactive, and to depend in different ways on top-down attentional processes.

Finally, besides levels of representation and accessibility (as expressed in the distinction of I1, I2 and I3), here we consider stages of representation and processing related to attention and consciousness (see also [43], for an emphasis on stages of conscious representation). Whereas levels of representation refer to the nature of mental representations in terms of cognitive accessibility (unconscious, preconscious and conscious), processing stages refer to the sequence of major steps or transitions during the time-course of perceptual processing. Although both are related to some extent, i.e. earlier stages of perceptual processing involve unconscious representation and conscious representation occurs at later processing stages, they are not necessarily connected. It is equally possible late in the time-course of processing that some stimulus (object) representations do not reach the conscious access level, for instance, because they lose out in competition with other object representations. Moreover, we should consider that stimulation may either reach an unconscious or preconscious level of representation, depending on the strength and duration of the stimulus in early stages of processing.

3. Conscious access and attentional selection

As argued in a recent review by Dehaene & Changeux, ‘In cluttered displays selection appears to be a prerequisite of conscious access: when faced with several competing stimuli, we need attentional selection in order to gain conscious access to just one of them … These findings indicate that selective attention and conscious access are related but dissociable concepts that should be carefully separated, attention frequently serving as a ‘gateway’ that regulates which information reaches conscious processing’. [1, p. 202]. Dissociation between selective attention and conscious access is amply documented (see also [1]): attentional selection can occur without conscious processing [44], and this can modulate the processing of unconscious stimuli [45,46]; attention can be directed to the location of a target stimulus that remains invisible [47,48]. A single target does not require attentional selection for conscious access [49].

The dissociation is reflected in distinct event-related potentials (ERPs) associated with these two processes (stages). Conscious access appears related to the late positivity (LP) component, which is predominantly localized to the parietal and central electrode sites. LP typically begins between 300 and 400 ms after stimulus presentation [50–52]. Selective spatial attention can be associated with the N2pc ERP component [53–55], observed at posterior sites (i.e. over visual cortex), approximately 200 ms after stimulus presentation. An ERP correlate of non-spatial attentional selection is the selection negativity (SN) component, at about 200 ms in the occipitotemporal sites [56]. Finally, earlier attentional modulation can be related to earlier ERP components such as P1 and N1 [57]. These observations lead us to postulate a bottleneck for selective attention within the visual system, separate from the GW.

4. Visual search and theories of attentional selection

Feature integration theory [26–29] originally proposed that sensory (visual) information is initially processed in parallel, preattentively, in multiple distributed feature maps. Thus, several visual features are registered early, automatically and rapidly, unconstrained by any processing capacity limitations. Immediately after this early, preattentive stage, feature binding leading to integrated object representations would take place by serial shifts of a spatial attentional spotlight operating on a ‘master map’ of object locations. Notably, this theory postulates a processing bottleneck related to the limited capacity of the attentional spotlight for feature binding: only one object at a time. Several versions of feature integration theory have been proposed over the years. The first version [27] did not assume a master map of locations, while a later revised version [28] postulated this mechanism along with a form of inhibitory tagging of distractor locations. The last aspect has been considered in depth in the guided search model developed around the same time [32].

Ever since Treisman's influential studies, visual-attentional processing has predominantly been studied in visual search tasks. These tasks require a speeded judgement about the presence or absence of a target object or feature among other distractor objects or features in a display. Visual search tasks involve attentional and conscious access processes for detecting and reporting the presence (or absence) of a target object among a set of distractor objects in the display.

As introduced by Treisman & Gelade [27], feature search, also known as disjunctive search, refers to the condition in which target and distractors are differentiated by a single property such as colour, shape, orientation or size. Feature search is generally very efficient, with short response times, resulting in a pop-out effect. Indeed, reaction times do not increase with the number of distractors around the target [22,27]. This is very different with conjunction search. This type of search takes place when target and distractors do not have a single distinctive feature, such that some may resemble the target in, for instance, colour and others in orientation, size or shape. Response times in conjunction search typically increase linearly with the number of distractors. In feature integration theory, this effect is accounted for by shifts of attention needed to bind features into integrated objects and to scan serially object locations searching for the target [26–29].

Feature integration theory has claimed to explain the experimentally observed phenomenon of illusory conjunctions, i.e. feature binding errors [58]. However, illusory conjunctions are also observed with long display exposure durations [59]. As we will see in §5, TAC advances an account of illusory conjunctions in terms of failure to enhance the proper feature-related activation among modular feature representations, which may occur with long as well as short display exposure durations.

Several investigations have called into question the sharp distinction between feature and conjunction search. For instance, visual search for targets defined by a conjunction of motion and form features [60] and for three-dimensional shapes [61] occurs more efficiently than feature integration theory would tell us to expect. Moreover, visual search for targets and distractors that differ in orientation can be serial for certain orientation combinations, even though neurons encode orientation in early visual areas [30]. Finally, visual search that was initially serial for certain stimuli can become parallel after some practice [62]. TAC reconciles such conflicting findings, by endorsing the two modes of visual search suggested by feature integration theory, i.e. with and without serial attentional shifts, but without the rigid distinction between feature and conjunction (object)-based processing.

According to the other influential approach to visual search emphasizing serial processing, the guided search model [30–32], information from top-down and bottom-up processing is used to create a ‘ranking’ of visual items in order of their attentional priority. In a visual search task, selective attention will then be directed to the visual item with the highest priority. If that item is ‘rejected’, then attention will serially move on to the next item and the next, and so forth. In the guided search model, an activation map provides a representation of spatial locations in which the level of activation at an encoded location reflects the likelihood that it contains a target. This likelihood is based on an early preattentive processing of basic features producing the activation map.

But why, in this account, do we need serial processing? Luck et al. [63] argued that when the visual system must process multiple objects simultaneously, as in the visual search paradigm, the neural coding of individual objects could become ambiguous. This representational ambiguity would arise due to the massive involvement of large receptive fields, coarse coding and distributed representations in the visual system. Their ambiguity resolution theory suggests that the primary role of visual selective attention within the ventral object recognition pathway of the visual system is to resolve object coding ambiguities [63,64]. A similar view was advanced in the ‘visual attention model’ (VAM) proposed by Schneider [65].

Related to these views, selective attention, neural network and neural coding theorists have argued that the massive distribution of feature representations in multiple cortical areas [66] entails the binding problem, i.e. achieving integrated and unambiguous representations of perceptual objects in cluttered scenes [67–70]. Whereas for feature integration theory [70] the binding problem is solved by serial attentional spotlight shifts, neural network theorists proposed that the binding problem in the visual system could perhaps be solved by neural synchrony [67–69], enabling a parallel representation of multiple objects. However, biologically constrained cortical network simulations have shown that, realistically, only a very few objects can be unambiguously represented in parallel through neural synchrony [37,71].

The problem of selection among parallel representations of multiple objects may be considered to require a winner-takes-all dynamics, similar to the one operating in the GW. Winner-takes-all dynamics between competing spatial locations, in combination with serial shifts of attention, characterize the saliency-based search mechanism proposed by Itti & Koch [72,73]. This model emphasizes bottom-up saliency of visual object locations, in terms of a saliency map. Walther & Koch [74] proposed a related computational model for identifying and attending to salient proto-objects in natural scenes. Walther and Koch's model integrates the saliency map model with a biologically plausible and computationally efficient mechanism of object recognition, proposed by Riesenhuber & Poggio [75]. The model performs attentional selection on real-world images of salient proto-objects, as well as subsequent recognition. It has to be noted, however, that this model does not assume ‘preattentive’ computation of proto-objects. Instead, proto-objects are identified and used ‘postattentively’. In contrast, in a later computational model [76] based on Claus Bundesen's ‘theory of visual attention’ (TVA) [25], proto-objects are computed preattentively and bias competition for VWM access via their attentional weights, in a process of ‘attentional filtering’. Proto-objects play a central role also in TAC. However, we do not require a saliency map to drive attentional shifts for object identification and subsequent processing stages.

Some theories of selective attention in visual perception [22] emphasize parallel competition for a late selective encoding in WM. The biased competition framework [77] proposes that attentional selection and related neuronal response enhancement take place in a parallel competition among all objects in the visual scene [78]. The biased competition framework suggests that bottom-up and top-down factors bias competition among object representations in visual processing, with an important role for cortical feedback mechanisms. These mechanisms enable certain features of an object or whole objects to be prioritized for attentional selection and further processing.

Within the biased competition framework, the integrated object competition hypothesis proposed by Duncan [21] and Duncan et al. [23] suggests that competition for limited perceptual processing capacity takes place between integrated object representations. Each of these integrated object representations is a collection of feature representations distributed in multiple cortical areas. Competition is thus distributed throughout multiple cortical areas while in parallel, cooperation occurs between neurons coding for features belonging to the same object. When attention is directed to one feature of an object, all of its features will tend to become dominant in their respective cortical modules. TAC assumes a similar mechanism in its visual attentional workspace (VAW).

5. Across attention and consciousness: the attentional blink and working memory consolidation

In this section, we consider two phenomena that have been explained based on either attentional selection or conscious access theories: the AB and WM consolidation.

(a). The attentional blink

With their limited processing capacity for conscious access and response selection, serial GW operations can account for experimental findings relating to the AB effect [16,33]. The AB phenomenon [39] is observed in rapid serial visual processing conditions in which stimuli are presented in a rapid sequence at the same location. Two targets are embedded in a sequence of distractors. If a first target (T1) is followed within 500 ms by a second one (T2), T2 is often not detected, thus giving rise to the AB [39]. Here, we focus on the AB and its related effects.

Neural network models based on GW processing principles, with serial all-or-none target representations in the GW [16,79,80] account reasonably well for the basic AB effect, that is by contrasting the processes occurring for short and long lags (shorter and longer than a few hundred milliseconds, respectively) between T1 and T2. These models are based on the winner-takes-all competition in GW layers, with reverberating activities suppressing for a few hundred milliseconds neurons coding for other targets in the GW.

These models, however, do not account for a number of effects related to the AB. For example, for the AB to occur, T2 has to be masked by a distractor [81]. Another important effect related to the AB that has not been accounted for by these models is that T2 is generally reported if it immediately follows T1 in the sequence (in the lag 1 sparing effect; see [82]), or if it is presented 500 or more milliseconds after T1. Moreover, the sparing effect can be temporally stretched, even for hundreds of milliseconds, when more than two targets are presented in succession [83,84]. This effect of spreading the sparing is, however, abolished with the interleaved presentation of a distractor between two targets [83,84].

These and other effects have motivated rivalling approaches to the AB that do not involve the GW. In particular, the ‘boost and bounce’ theory of temporal attention [85] has proposed that there is no central role for capacity limitations or bottlenecks. This theory provides a rapidly responding gating mechanism (or attentional filter) that seeks to enhance relevant and suppress irrelevant information. When items sufficiently match the target description, they elicit transient excitatory feedback activity (a ‘boost’ function), meant to provide access to WM. However, in the AB task, the distractor after the target is boosted accidentally, subsequently resulting in a strong inhibitory feedback response (a ‘bounce’), which has the effect of closing the gate to WM.

Recently, we have shown how a neurodynamic model called ViSA (standing for visual selection and awareness) provides a unifying computational account for a range of key AB effects, in combination with a set of VWM effects [40]. The sparing effects associated with the AB are accounted for by a mid-level representation of multiple targets. In contrast with the boost and bounce theory, in ViSA the GW plays a crucial role in the emergence of the AB.

In this article, we propose a new theory accounting for a range of AB effects by endorsing GW principles from earlier models. Following the lead of ViSA, we recognize the capacity of parallel target representation before conscious access in the GW, which is assumed to involve graded interactions between the GW and perceptual modules. Our present theory, however, from thereon suggests different GW mechanisms and an account of the AB different from that of ViSA and earlier GW models. Finally, our present scope is wider when compared to ViSA and earlier GW models. Here we also consider various perceptual representation and attentional processes, such as in visual search, in combination with conscious access.

(b). Consolidation in working memory

A phenomenon that can be related to conscious access in the GW, and to the AB in particular, as also modelled in ViSA [40], is WM consolidation. Consolidation is the process of transforming a fleeting perceptual representation into a durable WM representation, which is stable against new perceptual inputs [86–88]. Evidence with the AB paradigm suggests that without masking of T2 consolidation is delayed rather than prevented, and T2 can thus be reported [41]. There is also evidence of a reduced AB in a condition where T1 is followed by a blank [86], which would cause stronger activation of the T1 representation. In the ViSA model, this effect has been explained in terms of faster consolidation of T1 clearing the way for processing of T2 in the GW, even with the same lag as the one that in the masked condition leads to the AB [40].

Several studies have suggested that for consolidation in WM, and specifically in VWM, the component dealing with short-term maintenance and manipulation of task-relevant visual information [24,25,89] is severely limited in processing capacity [40,86–88,90]. The rate of consolidation could, it might seem, be gauged from the time-course of the AB. The fact that the AB lasts up to 500 ms has been taken to indicate that consolidation proceeds at the (very) slow rate of up to 500 ms for a single item stored in WM [86,88].

Other experimental evidence appears to contradict slow consolidation, however. Vogel et al. [91] (see also [92]) used a VWM change detection paradigm with two arrays of coloured squares separated by a retention interval of about 1 s. A pattern mask was presented at different time intervals. Their findings suggest that consolidation is actually a relatively rapid process that can form about three durable VWM representations within 200 ms of visual processing. In view of this evidence, VWM consolidation has been regarded as a relatively rapid process, the rate of which is comparable to that of attentional scanning in visual search [27,31].

Both estimates of the speed of consolidation for encoding in WM may appear to be in conflict, but could be reconciled, based on the observation that both attentional selection and conscious access are needed for WM consolidation. The estimate from the change detection paradigm may be tapping attentional selection only, while the estimates coming from the AB task may involve processes relating to conscious access (see also [93]).

These two different time-scales may relate to the two different bottlenecks in visual information processing. In fact, based on visual search findings, processing steps of attentional shifting can be estimated in the order of tenths of milliseconds (generally in the 30–60 ms range; see [31,94]), whereas GW processing steps can be gauged to be in the order of a few hundred milliseconds (about 200–400 ms; see [12,16,20,33]). The theory proposed in this article provides a unitary framework for these two different bottlenecks.

6. A theory of attention and consciousness

(a). Premises

In §§2–5, we have observed a gap between theories and models of attentional selection and visual search on one side, and theories and models of conscious access and related processing on the other. Bridging this gap is crucial for understanding the interactions between conscious and attentional processing, and related phenomena.

This section will present our effort to bridge the gap by offering a unified neurocognitive TAC. The theory assumes two processing bottlenecks, one at the level of visual attention and another at the level of the GW. We focus on the interplay between attentional selection at the perceptual processing bottleneck and conscious access in the GW, and consider the mediating representations and processes. We also address control operations in the GW in a new perspective. Based on these aspects, the theory would enable us to provide a new account of the AB and WM consolidation phenomena.

The theory assumes that although operating on different time-scales, conscious access and attentional selection involve similar mechanisms and processes. These include gradual evidence accumulation and all-or-none discrete transitions (ignitions) in distributed (multimodular) thalamo-cortical networks. These mechanisms, which enable efficient information representation, selection and transmission in the brain (see also [33,94]), might have evolved to serve different, yet related functions in cognition and adaptive behaviour.

We now clarify some of the terms used in our proposal. In order to avoid the ambiguous term ‘selective attention’, here ‘top-down attentional modulation’ refers to guidance or bias, resulting from signals from prefrontal and parietal cortex top-down to visual cortex before, during or after stimulus onset [38]. ‘Attentional filtering’ refers to a stage supporting a subsequent attentional selection of targets through rapid serial shifts of attention. ‘Attentional selection’ refers to selection of a given object representation after stimulus presentation, prior to conscious access. The term ‘amplification’ is used generically in reference to cooperative (recurrent) interactions in neural assemblies engaging in gradual mutual enhancement. Amplification may refer to both local and global cooperative (recurrent) interactions. ‘Top-down amplification’ denotes the gradual effect of top-down attentional modulation on perceptual representations, which may also be recurrently enhanced by bottom-up signals. The term ‘ignition’ denotes a sudden transition to a state of elevated activation and coherence in distributed representations [17,95], which again may happen either locally or globally.

Because of the high temporal resolution of ERPs and the ample evidence for their association with various processing stages, ERP components are used to characterize multiple processing stages in TAC, from feature to proto-object representation, and as related to various attentional processes.

(b). Two workspaces for attentional and conscious processing

In TAC, the two processing bottlenecks each involve separate and interacting workspaces: the GW and the VAW. We will first describe the two workspaces and their interactions, next the stages of representation and processing implicated in the theory, with related visual search effects, and then the theory explanations of the AB and WM consolidation.

(i). The executive routers: higher-order representations in the global workspace

TAC endorses the notion of GW from earlier theories and models [5,7,10–13,15–19]. In particular, in line with the global neuronal workspace model [7,15–19], we assume that the GW crucially involves areas in lateral (ventral and dorsal) prefrontal, posterior parietal and anterior cingulate cortices. TAC, however, originally postulates higher-order representations of executive operations besides object (target) representations in the GW. These include conscious access for motor response selection, access for consolidation in (visual) WM, top-down attentional modulation and amplification, and top-down executive inhibition. Each of these operations is represented and activated (ignited) by a control (or executive) router in the GW, i.e. an assembly of neurons which codes coherently for a given executive operation, and inhibits neurons encoding competing operations, in a higher-order winner-takes-all GW dynamics.

The neural assemblies for the control router function in the GW require long-distance afferent and efferent projections. The latter are also directed to neurons that inhibit competing routers. These GW routing assemblies would also need to aggregate subsets of distributed neurons with ramping response properties for evidence accumulation [33,96], all-or-none reverberating responses [16,36] for enabling the winner-takes-all dynamics, and properties to sustain top-down control and maintenance [97].

We posit that a winner-takes-all competition between executive operations in the GW is critical to enable coherent top-down amplification in the thalamo-cortical system, together with coherent amplification and ignition processes occurring at the level of target object representation [17]. We therefore propose that processes of amplification, ignition, evidence accumulation, all-or-none representation and winner-takes-all competition for target (object) representation in the GW [17,33] are also involved in higher-order selection of executive operations in the GW. For example, the same target, T1, can be subjected to two different executive operations in the GW. Thus, we may have attentional selection of T1, denoted with S(T1), and consolidation of T1 for encoding in VWM, denoted by C(T1). Attentional selection and conscious access of another subsequently presented target, T2, are denoted by S(T2) and C(T2). Only one of these four instantiations, i.e. S(T1), S(T2), C(T1) and C(T2), can take place at any given time in terms of GW activation and control. In particular, when C(T1) is taking place, S(T2) cannot take place simultaneously. This functional logic will be applied below to account for AB and transient attentional enhancement effects. According to TAC, it is the higher-order competition between selection and consolidation, rather than between T1 and T2, that will explain the AB.

(ii). The visual attentional workspace

Our notion of VAW refers to a dynamic thalamo-cortical network for visual processing characterized by a limited (processing) capacity. The VAW is based on cooperation and competition between spatially distributed neurons, evidence accumulation, attention-dependent amplification and ignition dynamics, just like in the GW [17,33]. In the VAW, amplification and ignition processes take place for visual attentional filtering, attentional selection, object identification, object localization and buffering before consolidation in VWM.

TAC supposes that thalamo-cortical areas involved in amplification and ignition for attentional processing in the VAW also engage in recurrent interactions within and across visual streams. In this respect, two areas in parietal and prefrontal cortex, and one in the thalamus, may constitute the core of the VAW. These are the lateral intraparietal area (LIP) in the intraparietal sulcus, the frontal eye fields (FEFs) in prefrontal cortex and the pulvinar in the thalamus. Given the interactivity in the VAW, we hypothesize that the LIP is pivotal for gradual evidence accumulation computations; the FEF is for all-or-none ignitions or discrete transitions (shifts); and the pulvinar is for distributed amplification during both graded atttentional modulation and all-or-none attentional filtering.

Several studies suggest that unlike most prefrontal areas, FEF and LIP project directly to many visual areas [98,99] (figure 1a,b). This connectivity enables the VAW to mediate between a large set of areas within the visual cortex. These cortical areas are placed at a high level within the visual processing hierarchy [66]. LIP has been implicated in higher-level evidence accumulation for decision making, based on the observation of ‘ramping’ neurons in this area [102]. Buschman & Miller [94] have recently proposed that LIP and FEF together play a crucial role in serial visual-attentional shifts. Also the pulvinar, a collection of higher-order associative thalamic nuclei, is characterized by extensive reciprocal connections with many visual areas, including LIP and FEF [103,104] (see figure 1a). Experimental findings show its important involvement in visual attention processes [105,106], and theoretical proposals suggest its saliency (priority) map function [107].

Figure 1.

The visual attentional workspace (VAW), and its connections with the global workspace (GW). (a) Schematic of connections between areas in the visual cortex, at different levels of representation, and areas FEF, BA45 and BA46 in lateral prefrontal cortex, and the pulvinar in the thalamus. Connections involving the FEF, LIP and the pulvinar, as well as GW areas in lateral prefrontal cortex, are depicted with thin lines. Note how area LIP is connected with areas in the two streams of visual cortex, and with prefrontal areas. The core VAW areas, LIP, FEF and the pulvinar, are labelled in black (red online). GW areas are labelled in black with dotted contours (yellow online). Intermediate level areas V4 and MT are labelled in light grey (green online) and higher level areas CIT, AIT and 7a in dark grey (purple online). Areas in between these, PIT and MST, are labelled in mid grey (blue online). Note the recurrent connectivity patterns between visual cortex areas at different levels with both FEF and the pulvinar, as well as the direct connections between higher-level visual areas and GW areas in prefrontal cortex. Note also the connections between FEF, LIP and the pulvinar with GW areas in prefrontal cortex. Finally, note the role of intermediate level areas, such as V4 and MT, in connecting to both pathways and to lower level areas such as V2 and V1. V4 and MT also connect with LIP, FEF and the pulvinar. Feedback connections to lower-level areas such as V2 and V1 are not shown in this panel (adapted from [100]). (b) Illustration of reciprocal connectivity patterns in lateral cerebral cortex involving areas LIP and FEF with area BA46 in dorsolateral prefrontal cortex and intermediate level areas, as well as with area IT for object recognition, in visual cortex. This panel also shows connections with lower level areas, in particular, with V1 (reproduced from [101]). (Online version in colour.)

In the VAW, therefore, LIP, FEF and the pulvinar may be seen to engage in dynamic (functional) coupling with areas at lower, intermediate and higher levels of representation in visual cortex. We regard as particularly important the VAW interactions between FEF, LIP and the pulvinar with ‘intermediate level’ areas in visual cortex such as V4 and the medial temporal lobe (MT) [103,104,108–110]. These processes would be required for attentional filtering; they enable spreading of the amplification and ignition processes across the visual pathways, plausibly extending from V4 and MT backwards to V3, V2 and V1 [66].

In contrast, interactions with areas in inferotemporal (TEO and TE) and parietal posterior cortex (MST and 7a) are implicated in attentional selection of objects, such as in area TE in the ventral pathway, and of locations such as in area 7a in the dorsal pathway [66,111]. Adjacent visual areas, such as TEO (or LOC in humans; see [112]) in between TE and V4, and MST in between 7a and MT [66], would also contribute to the ignition of selected representations.

FEF, LIP and the pulvinar are also connected with several areas in the intraparietal sulcus, parietal posterior and (e.g. dorsolateral) prefrontal cortex [103,104,113], which are all assumed to be involved in the GW [7,19]. FEF, LIP and the pulvinar would thus enable the VAW to mediate between the prefrontal and parietal areas of the GW. Thus, FEF, LIP and the pulvinar can be regarded as the main anatomical hubs in a dynamic thalamo-cortical visual processing network. These areas thus appear essential for integrating representations and processing in visual cortex with the GW, especially when attentional and conscious processing work together as, for instance, in the AB task.

(c). Stages of processing

Here, we will deal with stages of processing in TAC, from early representation of visual features to encoding in VWM, through multiple attentional processes and conscious access. These stages are schematized in figure 2. The figure shows the sequence of stages, with their related level of representation, the involved processing systems (workspaces), and the ERP components with which they are associated.

Figure 2.

Processing stages in TAC. The associated levels of representation (accessibility) and ERP components appear on the left. Neural computations and processing steps at different stages appear on the right. Also shown are discrete processing steps of different durations (e.g. for attentional filtering, selection and maintenance; light blue online), ‘ramping’ evidence accumulation operations of different durations (light red online), sensory coding (green online) and phasic responses related to ignition (and ‘decision’) (yellow online). Note the sequences of neural operations and computations within and across stages. On the right, note also the implication of the visual attentional workspace (VAW), global workspace (GW) and working memory (WM), at different stages. (Online version in colour.)

In what follows, we will sketch our account for a range of perceptual and cognitive phenomena, including visual search effects, such as the difference between feature-based and conjunction-based search, target-distractor similarity, distractor heterogeneity and illusory conjunctions, as well as for the effects relating to conscious access in the AB, including effects of lag 1 sparing, spreading the sparing, AB masking effects. We will also make an effort to explain why the AB occurs in some trials but not in others. TAC will enable us to resolve the controversy between fast and slow accounts of consolidation in VWM. We will touch on issues of processing in the visual cortex and its interactions with parietal and prefrontal cortices, and on the possibility of identifying stages of processing with ERP components related to attention and awareness.

(i). From visual features to proto-objects

Proposing a ‘middle’ way between on the one hand feature integration theory [26–29], which assumes a parallel representation of segregated features, and on the other hand the integrated object competition hypothesis [21,23], which suggests that objects compete for selective attention as integrated representations, we assume that the main level of representation in the visual system on which attentional selection operates is proto-objects (see also [3,74,76,114]). Proto-objects are temporally related to the sensory register or iconic memory [115,116], for which lateral interactions in the visual system enable a transient maintenance or persistence [8,43]. In TAC, these lateral interactions also bring about a certain degree of integration between features encoded in distributed modules of the visual system. Cooperative interactions between neurons encoding for the same proto-object distributed within and across visual areas and streams may result in a graded coherence in terms of neuronal firing rates and/or spike synchronization [37,71,117].

Competitive interactions between different proto-objects take place within modules. These are the product of inhibitory interactions between neurons coding features of different objects. To reduce binding failures, stronger competition may be expected within visual areas characterized by larger neuronal receptive fields, which face greater ambiguity with respect to binding of component features, in particular with cluttered visual scenes [66].

The theory nevertheless permits occasional binding failures, i.e. accidentally combining features of two objects into one spurious one, in order to account for illusory conjunctions [58,59]. TAC accounts for this effect as resulting from incongruent resolution of competition in visual modules between feature representations belonging to different proto-objects. Given cooperation between neurons coding for features of the same proto-object in different modules, in combination with competition between neurons coding for features of different proto-objects within modules (see also [37,118]), allows such incongruent resolutions. For example, neurons coding for a feature of a proto-object may ‘win’ in the colour module, while neurons coding for a feature of another proto-object win in the shape module. Attentional selection of these incongruent neural activations would give rise to illusory conjunctions. For example, with the presentation of a blue circle and a green square, the illusory conjunction of a green circle might be experienced. According to TAC, this would be due to local amplification of ‘green’ and ‘circle’ representations in different visual processing modules. A mechanism of local amplification is needed to enable pop-out and fast feature detection in feature-based search. Note that these effects do not involve the globally coherent amplification and ignition that would characterize the stage of attentional selection.

Rather, attentional filtering should prevent the spurious resolutions that give rise to illusory conjunctions. After an initial stage of persistence in the sensory register, top-down attentional modulation can bias the competition between proto-objects in the visual cortex. Indeed, by using a variation of the experimental paradigm proposed by Landman et al. [119] (see also its discussion in [8]), we recently found that after about 200 ms from display offset, iconic maintenance is biased towards (proto)objects with higher attentional priority [120]. This biased (whole-field) competition between proto-objects is plausibly related to an amplification of firing of the neurons coding for features of the proto-objects with higher (attentional) priority in a given task, and inhibition of competing proto-objects.

Notably, TAC does not postulate either ‘static’ saliency [72,73], or activation maps to drive attentional shifts for filtering [31,32]. It rather endorses dynamic and interactive principles of function, informed by the coexistence of functional segregation and integration within and between streams in the visual cortex, and in the interaction with the associative thalamus (pulvinar). We assume that the average level of activation and coherence of a distributed proto-object representation in the visual system is ‘measured’ in area LIP of the VAW. Together with FEF and extrastriate visual areas, LIP is thus supposed to play a crucial role for rapidly making and executing ‘decisions’ [31] or transitions in visual search, such as to select the next proto-object for shifting, to reject a probable distractor or select a (likely) target at each shift. The pulvinar might modulate the coherence of proto-object representations with higher priority, and transmission to the GW [106].

We now consider which ERP components may serve as indicators of stages in TAC before attentional selection. The earliest stage of visual representation in TAC can be related to C1, which is the earliest visual ERP component (peaking at 60–90 ms after stimulus onset). C1 is generated in the primary visual cortex and is not affected by attention [121–123], although the later portion of this component may reflect contributions from visual areas other than V1 [124]. The following P1 (80–130 ms) and N1 (150–200 ms) components are enlarged when attention is directed towards the location of the evoking stimuli (reviewed in [57]). These two components are often considered as one P1/N1 complex in which N1 has the largest amplitude among early ERP waves. This complex is a renowned indicator of early spatial attention. Proto-objects and their early attentional modulation, with involvement of extrastriate (associative) visual areas and feedback connections, may thus be related to the P1/N1 complex. Indeed, there is evidence that local cortical feedback signals influence lower-level areas rapidly [125,126]. In conditions leading to visual awareness, proto-objects may be related to the visual awareness negativity (VAN). Since VAN has been observed during passive viewing [127] and in response to distractors [128], it appears to occur independently from attentional factors, though it can be modulated by attention [129]. It has furthermore been proposed that recurrent interactions within the visual system might underlie VAN [129]. According to TAC, VAN might thus reflect the degree of integration and coherence of distributed neural representations of proto-objects, with and without top-down attentional modulation. These representations would correspond to the level of set I2 in Dehaene and Naccache's classification [15], and enable further stages of processing towards conscious access and report.

(ii). Attentional filtering

According to TAC, serial shifts are needed for attentional filtering of proto-objects in demanding visual search conditions, with cluttered scenes or with distractors that share features with the target. We assume that in such conditions, even with biased competition, ignition processes for attentional selection cannot take place due to strong and widespread competition. This affects conjunction based search conditions [27,31]. Serial attentional shifts are regarded as critical to overcome the completion and achieve amplification of the neural representations of distributed features belonging to coherent proto-objects. This process equates to attentional filtering in TAC. FEF has the capacity to influence responses of neurons that vary widely in what visual attribute they encode, be it colour, orientation, motion, form or object identity [130]. At each step, local all-or-none ignitions within FEF (see also [94]) driven by widely distributed signals in the VAW, would reciprocally lead in a top-down fashion to transient coherent amplification of distributed features of the currently focused proto-object.

Electrophysiological evidence based on source localisation with the N2pc component has suggested that shifts of attention initiate in parietal areas; attentional focusing ignites distributed extrastriate areas in the occipital and inferotemporal cortex [131]. Local amplification within modules, in combination with spreading of amplification to other modules amongst neurons coding for the same object, as suggested by the integrated object competition hypothesis, occurs at a relatively early stage. We therefore propose that the onset of N2pc in parietal areas might reflect relatively early top-down amplification from the FEF, induced by the convergence of synaptic inputs from widely distributed neurons coherently coding for proto-object features. LIP and the pulvinar might similarly act as a hub in modulating activation and coherence of proto-object representations, thereby igniting attentional shifts in FEF.

Concurrent with amplification, there is transient inhibition of competing proto-objects. Distractor rejection occurs each time after attention is newly focused. The rejection would take place via a rapid evidence accumulation at ‘rejection’ neurons (in LIP), causing the suppression of the currently active neurons in FEF. This would enable the activation of other neurons in the same area, and lead to a shift to another proto-object. If, however, ‘selection’ neurons in LIP accumulate evidence about a likely target during an attentional shift, the rejection neurons are inhibited. This enables the next stage of attentional selection rather than a next attentional shift (see below).

The balance of excitation and inhibition would provide guidance to attentional shifts towards the most likely target(s) representations [31,63,64]. To enable attentional shifting, spatial encoding must be available to LIP, FEF and the pulvinar [94,104]. This does not require, however, an explicit representation of spatial locations in an activation map. Rather, it may be based on distributed proto-object representations. FEF is also reciprocally connected with areas of the ventral (recognition) processing stream with large receptive fields and low spatial resolution, such as area TE [132] (figure 1).

The dynamic balance of activation, inhibition and coherence of proto-object representations thus plays a role throughout the two visual pathways for driving attentional shifts, i.e. ignitions in FEF and subsequent distributed top-down amplification. We furthermore propose that VAW interactions between FEF, LIP, the pulvinar and inferotemporal cortex enable the suppression of ‘chunks’ of similar distractors in visual search [22,31]. An attentional shift onto one of these distractors would lead to a transient enhancement of the neural representations of other distractors of the same category, through feedback, and their subsequent spreading of collective inhibition via LIP and FEF.

According to TAC, attentional filtering steps are characterized by variable durations. In particular, it would take a longer time at each attentional shift to reject a proto-object as a distractor if this shares features with the target. This would be a consequence of stronger local amplification of target than distractor feature representations. Evidence accumulation towards target detection would inhibit competing neurons from igniting a negative decision and a shift of focus to the next target. Such shift-related negative decisions may occur in LIP and cause suppression of the currently active neurons in FEF. This recurrent inhibition enables the ignition of the next most active and coherent proto-object assembly in the VAW. Thus, TAC can account for the effect of target-distractor similarity [22,31] in visual search by longer durations of attentional filtering steps, in combination with a higher likelihood for distractors similar to the targets to capture an attentional shift, leading to a delay in target detection.

TAC proposes that the mechanism of serial attentional shifting is activated by the ignition of an assembly of neurons acting as a router within FEF and other areas of lateral prefrontal cortex. This higher-order ignition is assumed to take place rapidly after stimulus presentation, and be caused by cluttered displays (scenes) with spatially distributed target features bringing neuronal coding ambiguities [63–65]. There is evidence that FEF (e.g. [133]) and other lateral prefrontal cortex areas [134] can be activated rapidly after stimulus onset. There is also evidence that these areas can encode for object (target) features [135]. In TAC, therefore, the level at which the mechanism involving serial attentional shifts is activated, is adaptively modifiable. Thus, it does not depend on rigid criteria for choosing between conjunction-based and feature-based search [27].

(iii). Attentional selection and intermediate buffer

In TAC, pop-out in visual search [27] is related to rapid ignition, in which a local bottom-up amplification leads to a first ignition in the module where the target feature is processed, and then to a further amplification leading to a spread of ignition to more anterior cortical areas. This process would enable localization of the target feature. Furthermore, while local amplification only involves target-relevant features, ignition during attentional selection may extend to multiple modules, and involve both relevant and irrelevant features [21,23]. Notably, ERP research has shown that pop-out in visual search is modulated by top-down information at early and late stages of processing, and is therefore not simply just a bottom-up phenomenon [53].

Attentional selection occurs, if it does, after serial attentional filtering. If a focused proto-object during attentional shifting is detected as a target, further amplification and ignition processes take place in the VAW. Ignition in the VAW only takes place after prior ignition of the GW router for top-down attention or amplification, which may, for example result from cueing. After ignition, this router rapidly propagates top-down modulatory signals for response amplification to neurons coding for target objects in the VAW. Reciprocally, when a target is detected, this brings about ignition in the VAW; further bottom-up activation is propagated to the GW router to sustain top-down attentional amplification. At the same time, however, when one or more targets are represented in the intermediate buffer after attentional selection, these representations jointly propagate activation to another competing router in the GW, priming the ignition of access and consolidation. The associated evidence accumulation, in terms of slowly ramping firing rates [33] would then cause an increased inhibition of the GW router neurons for top-down attentional amplification. Eventually, evidence accumulates at the GW router for access and consolidation. The accumulation leads to the ignition of this router, and to the concurrent inhibition of the previously active router for top-down attentional amplification.

The intermediate buffer for visual attention posited in TAC would also be crucially involved in the task introduced by Landman et al. [119]; the task requires participants to encode a cued object in VWM, with a variable interval between the offset of the memory array, and retro-cue presentation. According to TAC, first the GW router for top-down attentional amplification needs to be ignited to enable attentional filtering, selection and buffering of multiple objects in the display. This higher-order (control) ignition in the GW would control the ignition of the VAW by a rapid top-down amplification of neuronal firing and synchrony in LIP and FEF coupled with visual cortical areas for attentional processing and the pulvinar. The retro-cue, rapidly processed given the activation of the GW router for top-down attentional modulation, would lead to the ignition of the other competing GW router for consolidation (access) and encoding in VWM.

Finally, we hypothesize that during the AB, temporary decreases of firing rates and synchronization of attentional neurons in LIP, FEF and the pulvinar take place in the VAW. These effects are supported by the experimentally observed suppression (‘freezing’) of the N2pc during the AB period [136,137]. During this period, amplification and ignition in the visual system involving LIP, FEF and the pulvinar are disabled. Otherwise, the ignition and amplification produced during access in the GW [17], in combination with top-down attentional modulation, could spuriously enhance interfering feature and proto-object representations in the visual system.

(iv). Conscious access, consolidation and encoding in visual working memory

After attentional selection and buffering have taken place, the process of conscious access for consolidation and encoding in VWM can begin. Insofar as the duration of this process can be estimated based on the AB, this is a slow process, lasting up to several hundred milliseconds. In computational modelling studies [16,40,79], this process is ignited by recurrent neuronal excitation within a large assembly of neurons distributed in multiple areas. Areas in prefrontal cortex, such as the dorsolateral prefrontal cortex (e.g. BA46), might play a crucial role for the GW ignition process related to the AB [7,19,138].

According to TAC, the AB reflects the temporary disabling of top-down attention (amplification), due to the concurrent ignition of target (T1) consolidation (access) for encoding in WM. This process involves competing GW routers. When the GW router for top-down attention is disabled, amplification and ignition processes related to attention cannot take place in the VAW. Indeed, there is evidence of suppression (‘freezing’) of the N2pc component, related to (spatial) attentional selection in the visual cortex, during the AB [136,137].

Therefore, for short lags between T1 and T2, the probability is still low that the consolidation process for T1 has started, and the suppression has begun of top-down attention necessary for selecting (and then accessing and consolidating) T2. For large lags between T1 and T2, the consolidation of T1 has most likely terminated, making top-down attention available again for the amplification and ignition necessary to select (and then access and consolidate) T2.

The effects of lag 1 sparing and spreading the sparing reviewed above indicate that the ignition of the GW router for target consolidation is delayed by about 100 ms after detection of a target, and that this latency can be stretched with the subsequent detection of other targets. TAC straightforwardly accounts for this in terms of sustained bottom-up activation of the router for top-down attentional amplification. In this process, however, the increasing number of selected targets represented in the intermediate buffer leads to an increased ‘accumulation’ (ramping or temporal integration) of activation at the competing router for access and consolidation in the GW. Thus, the same process activating the router for top-down attentional amplification over time also causes the accumulation of activation that ignites the competing router.

Reduced AB effects with unmasked presentation of T1 or T2, as reviewed in §5, can readily be explained in terms of enhanced proto-object representations in the VAW. In case of unmasked T1, the ignition processes for attentional selection and conscious access (including WM consolidation), would accelerate for T1, enabling attentional selection and further access to the GW for T2. In case of unmasked presentation of T2, maintenance in the VAW as a proto-object would later enable its access to GW, once T1 had been consolidated [41].

Furthermore, the reduced AB with concurrent task-irrelevant mental activity [139], such as listening to music, can be explained in terms of cooperation between T2 and the other concurrent information to deactivate the router in GW that ignites the consolidation of T1. After the deactivation, T2 would be more likely to be attentionally selected and then buffered for consolidation.

Section 5 showed that the AB occurs in some trials but not in others, with the same lag. Thus, the mechanism of suppression between GW routers, and of the router for top-down attentional modulation (amplification), should not always be insurmountable. This would enable a higher flexibility of the GW control mechanism. We posit that if the distributed representation of the T2 proto-object in the VAW becomes enhanced, e.g. due to fluctuations in oscillatory synchrony [140,141], the activation of the GW routers can fluctuate as well. This would permit T2 to enter the intermediate buffer for consolidation along with T1. Here we refer to this phenomenon as sparing recovery. This effect may manifest itself in enhanced coupling of slow and fast oscillations at the (right) temporo-parietal junction, an area connected to the intraparietal sulcus, a key hub for proto-object and higher-level object representation in the VAW [142].

Finally, TAC might reconcile the controversy between slow and fast accounts of consolidation in VWM, reviewed in §5. Indeed, with the mechanisms proposed here, attentional selection of multiple targets would rapidly take place in the VAW, amplified by feedback from the GW. Attentional selection and the subsequent intermediate buffering would rapidly ‘save’ target object representations from lower-level interference or masking. Then, at a next stage of conscious access in the GW, these target representations can be slowly consolidated, while the representations of any interfering perceptual inputs are ‘depotentiated’ through the temporary disabling of top-down amplification and ignition in the VAW. These mechanisms are controlled online within the GW.

7. Theory implications

According to our proposed theory, considerable integration and selection of perceptual (visual) information take place before conscious access, i.e. before representation in the GW. TAC thus postulates multiple processing stages occurring between early visual representation and conscious access. The sequence of experimentally observed ERP components C1–P1/N1–N2pc (see §6b) supports the timeline proposed in TAC, from proto-object representation to attentional filtering and selection. The theory also suggests that dynamic signatures of conscious representation, such as amplification and ignition [17], can be found at earlier stages than GW access.

(a). Attentional blink and inattentional blindness

Unlike in earlier GW formulations [16,40,79], in which the AB is explained in terms of competition between target (object) representations, in TAC the AB is originally explained by the competition between executive routers, or higher-order representations of access and top-down attentional modulation (amplification) operations. Thus, as clarified by TAC, the AB can truly be regarded as a blink of attention, in both its top-down modulation and selection aspects. This blink arises from the interaction with consciousness: conscious access directed to T1 is taking the stage, transiently disabling attentional processes directed to any other incoming stimuli, such as T2. This logic appears more parsimonious than one based on inhibition between any neural assemblies encoding different target objects. Just the competition between two higher-order neural assemblies would prevent simultaneous activation of two targets presented with the AB lags in the GW, with remarkably less neural machinery and metabolic (synaptic) resources going into the explanation.

The same logic can also account for the inattentional blindness phenomenon, in which (normal) subjects engaged in intense mental activity fail to notice salient but task-irrelevant sensory stimuli [143]. Inattentional blindness has been modelled in the global neuronal workspace framework in terms of ignition dynamics driven by internal states [17]. Recent studies have, however, shown that the effect of inattentional blindness also takes place when the inattended visual stimuli are expected (e.g. [144,145]). TAC explains this evidence based on a transient decoupling between the GW, where the representation of task-relevant information is maintained, and the VAW. This decoupling prevents top-down amplification and ignition or target selection, necessary for subsequent conscious access. As for the AB, TAC suggests that conscious access temporarily disables processes necessary for attentional selection of and then further conscious access to other salient and even task-relevant visual information. In other words, according to the theory, both the AB and inattentional blindness arise due to the interplay of conscious access and attentional selection: the current consciousness of some information leads to the ‘blink of attention’ or inattention towards other salient and even prioritized information, which cannot become conscious, given that before conscious access attentional selection must take place.

A striking prediction of our theory is that a suppression (‘freezing’) of the N2pc component should be observed in inattentional blindness conditions, as already observed in the AB [136,137]. We also predict a suppression of the non-spatial SN attentional selection component in both AB and inattentional blink conditions.

(b). Interactions within and between workspaces

TAC assumes a VAW, in interaction with the GW, for attention-based perceptual processing. The VAW, which is characterized by dynamics and mechanisms similar to those in the GW [17], operates through the functional coupling between areas LIP, FEF and the pulvinar with GW areas controlling conscious access (such as dorsolateral prefrontal cortex) on one side, and areas in visual cortex (such as areas V4 and MT) on the other. The VAW dynamics supporting attentional filtering and selection are enabled and disabled by the GW. In particular, the AB is related to a temporary disabling of the VAW by executive routers in the GW. According to TAC, however, during conscious access GW areas can interact recurrently with visual cortex via ventral and dorsal streams, selectively for a currently accessed (target) object. For example, neurons in lateral prefrontal areas BA45 and BA46 can engage in recurrent interactions with neurons in higher-level visual areas in inferotemporal and parietal posterior cortex coding for a currently accessed target. These interact with neurons in intermediate level visual areas, such as V4 and MT, in turn interacting with lower-level areas V3, V2 and V1. Areas LIP, FEF and the pulvinar would not be needed for these selective global recurrent interactions sustained during conscious access to a given target. These would just reflect the operation of the GW.

TAC assumes that visual attentional processes take place without explicit control computations in a master [28], saliency (e.g. [73]), activation [31,32], or priority (e.g. [114]) map. Rather, TAC endorses dynamic principles from the global neuronal workspace model [17], and extends these to visual attentional processing. Attentional filtering and selection are thus based on amplification and ignition processes involving distributed thalamo-cortical representations and cooperative multi-modular interactions. The cortical areas LIP and FEF, and the higher-order associative thalamic pulvinar nuclei, which have been proposed separately or alternatively to mediate a saliency or priority map function in earlier models (e.g. [94]), are considered in TAC jointly to form a hub for the dynamic coordination of attentional modulation, filtering and selection processes in the VAW.

The key role of the VAW for perceptual and attentional processes indicates that important integration and selection steps in information representation and processing take place before conscious access in the GW. Also, the roles of amplification and ignition in the VAW, before conscious access, indicate that key dynamic signatures of conscious representation can be found at earlier stages than GW access. This suggests that the VAW can actually be the leading neural correlate of phenomenal consciousness [6,14] with respect to visual perception, as related to set I2 (proto-objects) and attentional processing stages in between I2 and I3, and different from access consciousness in set I3. Previous theoretical views and models had suggested that the correlates of phenomenal consciousness are the local recurrent cores in the visual system [8,43], or the cortical feedback onto sensory maps providing an explicit representation of object features [79]. The VAW can be regarded as a more complete characterization of both recurrent interactions and feedback signalling in the visual cortex. Furthermore, it involves neurodynamic patterns previously suggested as neural correlates of conscious access in the GW. Indeed, recurrent interactions, feedback signalling, amplification and ignition jointly take part in the VAW. Finally, the VAW formulation, also in terms of its relationships with the GW notion, sheds light on differential roles of attentional processes for phenomenal and access consciousness.

(c). Relationships with other theories and further hypotheses

The interactive aspect of conscious access in TAC is shared with the ViSA model [40], according to which conscious access, such as during WM consolidation, is an interactive process between GW and lower-level perceptual areas. However, unlike ViSA and earlier GW models [16,17], TAC postulates that dynamic control of visual attention is exerted by a higher-order representations in the GW. Moreover, unlike these models, TAC characterizes attentional processing in the visual cortex, and not only conscious access in the GW. Finally, TAC appears to reconcile hierarchical and interactive models of visual awareness (see [101] for a discussion), given that the theory emphasizes both the leading role of key areas (hubs) for ignition, and distributed interactions throughout the visual system, including feedback to lower-level visual areas, such as V1.

The recently proposed ‘task-driven theory of visual attention and working memory’ (TRAM) [114] shares important theoretical assumptions with TAC. In particular, both theories include proto-object representation and emphasize several stages (or phases) of processing as well as working (short-term) memory consolidation. However, TRAM differs substantially from TAC in the account of the AB proposed. TRAM assumes that the AB emerges due to locking of attentional resources during T1 consolidation, with a discontinuous episodic representation of the targets. Moreover, unlike TAC, which features fast serial attentional shifts guided by top-down attentional modulation (lasting less than 100 ms; see also [31]), TRAM's first two processing stages refer to Bundesen et al.'s ‘neural theory of visual attention’ (NTVA) ([107]; see also [25]), a parallel and capacity limited attentional model. Therefore, also visual search phenomena, such as the target-distractor dissimilarity effect, are explained in substantially different ways between the two theories: TAC accounts for target-distractor dissimilarity in terms of patterns of attentional shifts, but TRAM in terms of attentional weights and parallel biased competition. Finally, TAC, in contrast to TRAM, does not assume an explicit priority map for attentional filtering.

TAC assumes that the two bottlenecks involved in the discrete processing steps for, respectively, attentional selection and conscious access, have different time-scales. Interestingly, the combination of the these fast and slow processing cycles would imply four fast processing cycles nested within a slow cycle. These nested processing cycles might support multi-item representation and maintenance, such as in WM (and VWM in particular). Thus, the ‘magical’ number four, which is the capacity of WM [146]), and VWM in particular [25,36,89,107], might reflect the reiteration of four fast processing cycles within a slow processing cycle. TAC further suggests the interpretation of this representational and processing format in terms of rapid attentional shifts for discrete items within a conscious access step corresponding to a higher-order representation of a memory set, during WM maintenance. Thus, the VAW might be crucially involved in interplay with the GW for control of WM rehearsal (see also [12]) in VWM maintenance. This hypothesis is supported by evidence for the involvement in VWM maintenance of the intraparietal sulcus (in which LIP is located) [147,148] and FEF in VWM [149], possibly in interaction with lower-level visual areas [150].

Finally, TAC can also be regarded as a synthesis of all-or-none [16,151] and graded [152] views of conscious representation. Indeed, through the stages of representations and processing in TAC both all-or-none ignition processes, and gradual evidence accumulation, amplification and consolidation processes, take place. In both workspaces, nonlinear all-or-none ignition processes are combined with gradual processes of evidence accumulation, which may further give rise to another, higher-level ignition. Moreover, the all-or-none processes are not only involved in conscious access in the GW, but also in earlier stages of representation and processing.