A roadmap for the study of conscious audition and its neural basis

Abstract

How and which aspects of neural activity give rise to subjective perceptual experience—i.e. conscious perception—is a fundamental question of neuroscience. To date, the vast majority of work concerning this question has come from vision, raising the issue of generalizability of prominent resulting theories. However, recent work has begun to shed light on the neural processes subserving conscious perception in other modalities, particularly audition. Here, we outline a roadmap for the future study of conscious auditory perception and its neural basis, paying particular attention to how conscious perception emerges (and of which elements or groups of elements) in complex auditory scenes. We begin by discussing the functional role of the auditory system, particularly as it pertains to conscious perception. Next, we ask: what are the phenomena that need to be explained by a theory of conscious auditory perception? After surveying the available literature for candidate neural correlates, we end by considering the implications that such results have for a general theory of conscious perception as well as prominent outstanding questions and what approaches/techniques can best be used to address them.

This article is part of the themed issue ‘Auditory and visual scene analysis’.

1. Introduction

How subjective perceptual experience—i.e. conscious perception—emerges from patterns of brain activity is a fundamental question of modern neuroscience. Although relationships between neural activity and conscious perception have been pondered since the beginning of the mind–brain sciences [1–6], many have viewed the problem as scientifically intractable. However, employing increasingly powerful methods [7], recent years have seen a dramatic increase in studies pursuing the so-called neural correlates of consciousness, or NCC [8], defined as the ‘the minimal set of neural events jointly sufficient for a specific conscious experience’ [9] (e.g. the phenomenal experience of hearing a sound). This definition, in contrast with others that distinguish between the NCC and the NCCC (neural correlates of the contents of consciousness), subsumes both the ‘preconditions’ and content-specific processes associated with specific conscious percepts [10], acknowledging that all conscious experiences have content (though varying in richness [11,12], an idea also underscored by early auditory selective-attention studies [13]). However, it remains important for theories of consciousness to specify the ways in which neural activity must be structured in order to account for the specific content, or qualia, of a given conscious percept (e.g. hearing a sound with a specific pitch or timbre [14]).

Though the idea had long existed outside the specific domain of the NCC, a primary conceptual advance that paved the way for a neuroscientific study of conscious perception is the so-called contrastive analysis [15], which involves comparing brain activity between behavioural conditions that are equivalent in every respect except for their conscious contents (i.e. perceived versus not perceived, or perceived as X versus perceived as Y) (but see [16]). Examples include threshold psychophysics [17] (where a sensory stimulus with fixed, peri-threshold intensity is only sometimes perceived), multistable perception [18] (where the same physical stimulus can be perceived in different ways), and situations where salient stimuli fail to reach consciousness due to distraction, or ‘load’ (at either sensory or cognitive levels) [19], as in inattentional blindness [20] or informational masking (i.e. auditory masking not accounted for by physical interactions at the auditory periphery [21]).

Despite classic examples in both the somatosensory [22] and auditory [23] domains before the term NCC came into existence, the vast majority of both data and theory has come from vision (see [10,24] for reviews). However, in the light of potential differences in anatomy, coding, and functional roles between vision and audition, the extent to which such work applies to conscious perception (and its neural correlates) in the auditory domain is not readily apparent [14,25]. Studies conducted outside the context of vision are therefore vital for the field at large, lest it risk missing potentially crucial aspects of the NCC that do not reveal themselves in that modality. Furthermore, acoustic stimuli have particular advantages over their visual counterparts in that they can readily be presented to individuals in sleep, sedation, or disorders of consciousness. Although the brain activity that can be observed during these various states typically takes a different form than that found in wakefulness, it is nevertheless the case that much of the auditory system remains responsive, facilitating the study of which activity patterns are either necessary or sufficient for conscious auditory perception, and potentially providing clinical diagnostics and therapeutics [26].

Here, through a comprehensive review of conscious auditory phenomena and their putative neural correlates—the NCC_A, if you will—as well as a critical examination of the functional role it serves (if any), we attempt to provide a roadmap for the future study of the neural basis of conscious audition. We focus on the aspects of brain activity that relate directly to conscious perception of acoustic stimuli, and only briefly mention the ‘background conditions’ [10] that are necessary but not constitutive for conscious perception [27]. In particular, we focus on how conscious perception emerges, and of which elements or groups of elements, in complex auditory scenes, processes that are likely intimately related to auditory scene analysis [28], i.e. how the auditory system perceptually organizes complex acoustic signals arriving at the ears. Finally, we outline the sort of experiments that will be needed going forward in order to (i) isolate the ‘true’ NCC_A from their prerequisites or consequences (also termed pre- and post-perceptual processing) [16,29,30], (ii) relate conscious auditory perception to other cognitive processes (attention, for example) and (iii) begin to formulate a neurally based theory of conscious auditory perception, i.e. a theory that explains what gives rise to subjective auditory experience, and what functional role such experience might have [31–33].

2. Prominent models of the NCC_V

In order to determine how the putative NCC_A to be discussed later fit into the larger NCC context, we start with a brief overview of some of the most prominent, neurally based models of conscious perception [34]. Owing to the relative paucity of NCC studies conducted outside the context of vision, such models are necessarily based on conscious visual experience and its putative neural correlates [10,24]. On the other hand, it is rarely explicitly acknowledged that such models might not apply in other modalities, making the auditory system an ideal test bed with which to examine such models [14,25].

(a). Global-workspace theory

One prominent neurally based model of conscious perception is global-workspace theory, or GWT [35]. Based on an earlier, exclusively cognitive theory [15], GWT posits that in order for sensory information to enter consciousness, it must be made available to a host of disparate cognitive systems (e.g. working memory, report) via a global neuronal workspace constituted by recurrent activation between sensory areas and frontoparietal cortex. The model views consciousness as indissociable from the functions served by the global workspace, and makes strong predictions in several areas, namely (i) that only one piece of content, or object, can be dominant in consciousness at any one moment, (ii) that there can be no conscious perception without some amount of attention, and (iii) that conscious perception necessarily involves ‘ignition’ (large-scale nonlinear amplification) of the activity of frontoparietal workspace neurons.

(b). Coalitions of neurons

Another neurally based model of conscious perception posited a competition between coalitions of neurons, each representing a different object or piece of content, with the winner of the competition determining the contents of perception at that moment [36]. Originally agnostic regarding where such coalitions might be housed, recent iterations have focused on high-level sensory areas in ‘the back of the brain’, i.e. non-primary visual cortex [10]. Importantly, this theory predicts a one-to-one mapping between activity in certain neurons and the contents of conscious perception (an extension of anatomical location theories—for review see [37]), such that the stimulation of those neurons would evoke conscious perception of that content, and their loss—via inactivation or damage—would preclude it.

(c). Local recurrence

This model of conscious perception posits the importance of local, recurrent processing between early and higher-order sensory (i.e. visual) areas [38]. An important feature of the local-recurrence model is that because it arises exclusively from recurrent activity within sensory areas, conscious perception is posited to be completely dissociated from other cognitive mechanisms required for attention, memory and perceptual report. This, in turn, leads to the prediction of certain phenomenological—i.e. conscious—states that are not only not accessed, but not accessible and thus cannot be reported [38,39]; this is quite controversial [40,41]. Note that the distinction between feed-forward processing (which is not posited to be conscious) and recurrent processing (which is posited to be conscious) is also made by both the global-workspace and winning-coalition theories, where the recurrence in GWT is between sensory areas and frontoparietal cortex, and between different levels of the sensory (visual) hierarchy for the winning-coalition model, although this has not always been precisely specified; cortical recurrent processing clearly plays an important role for many contemporary theories of consciousness [42,43], as might recurrent processing between cortex and the thalamus [44].

(d). Reverse hierarchy theory

This theory posits that the default nature of what we experience is based on high-level representations located near the ‘top’ levels of sensory processing hierarchies [11], and that only with effort and behavioural relevance can we consciously perceive environmental detail of which we are not normally aware. Based on the existence of anatomical sensory hierarchies where increasingly abstract information is represented at anatomical sites increasingly distant from the sensory periphery, this theory implies that conscious perception is probably, or at least most often, driven by activity in sensory association areas (where the most abstract-level representations of sensory stimuli are found). While it also allows for conscious access to low-level details (which may only be explicitly represented at sites closer to the sensory periphery) under certain behavioural conditions, it remains agnostic about how and where such details become consciously represented.

(e). Information integration

This theory of consciousness [45] relies on a threshold level of informational complexity, i.e. the number of possible informational states possessed by a system, and, to account for the unified nature of conscious experiences (i.e. the binding problem [46]), the integration of that information. An important feature of this model is that it predicts non-zero levels of phi (the value used to quantify consciousness level) for virtually any system that has the capacity to process and integrate information, which leaves open the possibility of consciousness in non-biological substrates but, unlike panpsychism, not in aggregates of (biological) organisms who may be individually conscious (e.g. human social networks or ant colonies) [47]. Another important feature is that complexity, per se, is insufficient, allowing for the existence of highly complex systems that are nevertheless unconscious.

(f). Self-sustaining signal regeneration

Why might recurrent activity be crucial for sustaining awareness, either in global or local circuits? The brain can be regarded as a network of loops that provide signal paths for patterns of spiking activity. The patterns of activity themselves can interact to create dynamics of mutually reinforcing or competing coalitions of neural signals that are not unlike the autocatalytic sets and autopoietic systems that characterize living organization [48].

Whether a given neuronal signal builds up and becomes self-sustaining, or is superceded by newer incoming patterns, depends critically upon the loop-gains of competing circuits. At any one time, the loop-gains of different circuits can be increased by disinhibition or by pattern-matching. Such attentional mechanisms facilitate the neural signals propagating through those circuits. Likewise, loop-gains can be decreased by non-specific inhibition or pattern mismatches. In this theory, the primary requisite of conscious perception is the ability to actively regenerate and sustain neural activity patterns, with the sets of patterns in circulation at a given moment being crucial in determining the contents of consciousness [48,49].

(g). The intermediate-level theory

This theory, now known as attended intermediate-level representations, or AIR, states that conscious perception is the result of so-called intermediate-level representations that are both active and attended [50,51]. Which level qualifies as intermediate is somewhat open to interpretation, but might be that which lies between high-fidelity sensory representations, which are too low, and perspective-invariant object representations, which are too high. In this view, conscious perception emanates from mid-level representations of sensory objects that are perspective- and content-specific (e.g. a particular instance of an object as opposed to object category). However, according to the theory, for a representation to become conscious, it must also be attended, thus positing an inextricable link between consciousness and attention.

(h). Higher-order or ‘metacognitive’ theories

So-called ‘higher-order’ theories of conscious perception [52] posit that first-order representations of potentially conscious content are insufficient; in order for some piece of content to be consciously perceived, a secondary (i.e. higher-order) representation that is in some way about the first-order representation—i.e. ‘mental states’ that represent oneself as being in the relevant first-order mental states [53]—must also be present. While there are several variations of such theories, from a neural point of view, they are similar to GWT insofar as an important prediction is the necessity of activity in supramodal brain areas (e.g. frontoparietal cortex [53]). However, they differ from GWT in the relationship between conscious perception and other cognitive processes.

(i). Anchored loops

A related model (insofar as it connects sensory representations to self-representations in generating conscious perception) posits that neural activity must be linked to internal processes related to the self [54]. From evidence related to parietal lesions, neglect syndromes, and electrical stimulation, it has been argued that conscious awareness of a visual event requires not only local activation of primary visual cortex, but also intact functioning of parietal regions corresponding to the apparent locations of the visual targets. The parietal regions contain maps of extrapersonal space and body schemas, such that visual events become bound to internal maps of ‘minimal self’ (location, body space and ownership). In this model, local and global recurrent activity need to be linked to activation of relevant parietal maps in order to produce conscious visual awareness [55].

(j). The attention-schema theory

Although it does purport to be a higher-order theory in the same sense as most others, a related proposal, the so-called ‘attention-schema’ theory, posits that conscious perception emerges as a result of the brain constructing a perceptual model of attention [56]. However, the attention-schema theory diverges from other higher-order theories in centring on the right superior temporal sulcus and right temporoparietal junction, rather than frontoparietal areas, as the likely neural locus of the higher-order representation that gives rise to conscious perception. This is due to the fact that these areas are known to be highly engaged during social interaction—in the attention-schema theory, the same mechanisms that attribute awareness to other brains are also responsible for attributing it to one's self—and are also areas that, when damaged, lead to neglect [57,58], a neurological condition in which individuals fail to perceive objects and events in contralesional (left) space. An interesting feature of this theory is the relationship it posits between attention and consciousness. Because consciousness results from the brain's construction of a perceptual model, or schema, of the processes of attention, the two processes are strongly linked. However, it allows for dissociations via errors in schema construction.

(k). Model taxonomy

What is the relationship between the various models of conscious perception? How are they similar and how do they differ regarding their predictions for the NCC_A? One way to classify these models is according to what predictions they make regarding where in the brain the processing that gives rise to conscious perception takes place. Although certainly oversimplified, one class of models attributes conscious perception to activity in the relevant sensory cortices [10,38], which is directly opposed to another class of models which posit that conscious perception can arise only when certain supramodal brain areas become involved (e.g. frontoparietal cortex [24], the ventral attention system [59]), not that the latter class of theories do not also require the involvement of relevant sensory cortex. However, some models—information integration theory, for example—would not necessarily fall into either category according to this classification.

Another way to classify models of conscious perception is via the relationship they posit, if any, between consciousness and attention. While the two have long been thought to be inextricable (such that attention is necessary for conscious perception [36,40,51,52]), or even one and the same process [60], recent work has suggested that they may be distinct processes that, though strongly linked, are dissociable under certain circumstances [56,61]. The models that posit an inextricable link between attention and conscious perception would predict that the latter is not possible in the complete absence of the attention (it is unclear whether this is achievable in practice) or without engagement of attentional networks.

Yet a third way to classify the various models of conscious perception is in what function they ascribe to it, if any (cf. §3). Those models that do ascribe some function to consciousness (or, equivalently, its physical substrate) generally argue that it can have a downwards causal influence on the processes that gave rise to it, thus determining or constraining the system's current or future states [45,56,62–65]. By contrast, models that do not ascribe any function to conscious perception argue that while it may (or may not) be an emergent property of neural activity, it is epiphenomenal in the sense that it does not have causal influence on the system [66,67]. In this view, the sense that our conscious states control our behaviour is illusory, perhaps because conscious states, while not causal, can often predict subsequent conscious states (for discussions on the neuroscience of volition, see [68]). Critically, however, whether or not conscious perception indeed has causal influence on behaviour, it remains a natural phenomenon in need of scientific explanation. We return to what predictions would be made by each type of model in §§3 and 6.

3. What does (conscious) audition do?

In order to shed light on the NCC_A and how they relate to the NCC in other modalities (particularly vision), it is worth examining the organism-level function(s) that are performed by the auditory system, and in particular, the function(s) that may be served by having conscious auditory experience, if any. Note that we use the term ‘function’ in the utilitarian or operative sense rather than a teleological one. That is, as the title of this section suggests, we mean to ask the question of what consciousness does (or, specific to our case, what conscious audition does) as opposed to what it should do or what it was designed to do [31]. Here, it is also worth noting that whether consciousness ‘does’ anything (i.e. whether it has any causal effect at all) or is instead completely epiphenomenal (as a result of the organization of certain types of information-processing systems) is a topic of intense debate (even among the authors). However, given that the answer to this question is by no means settled, it is certainly worth considering the possibility.

(a). Environmental sensitivity

The main feature of audition that distinguishes it from the other senses is the extent of its sensitivity to diverse types of events in the environment; no other sense maintains responsiveness in as widely varying environmental contexts as does audition (at least for environments in which biological organisms exist). Like vision (but unlike touch), audition is a far sense in that it is sensitive to distant sound sources and events presuming they are sufficiently intense. By contrast, audition is similar to touch (but dissimilar to vision) in that it remains ‘open’ to external events during sleep and in total darkness (though brighter light sources can traverse the eyelids). In fact, we know from behavioural and electrophysiological studies (particularly of the k-complex) that acoustic stimuli are evaluated to a high level during sleep [69–72]. Furthermore, salient sound, particularly when meaningful or deviant, is quite effective at inducing transitions from sleep to wakefulness [69,73,74]. Finally, unlike either vision or touch, audition has a resting sensitivity to external events covering the whole of three-dimensional space, be they distal or proximal [75]. In humans and other primates, detection of events or objects via the visual system requires either that they be within a relatively limited field of view or that the organism actively orient itself such that they are (which is probably tightly coupled to attentional reorienting). Detection of events via the somatosensory system typically requires the event to occur in close proximity to the organism.

Thus, beyond its role in encoding acoustic information, it follows from these features of audition's environmental sensitivity that it might also serve as the primary alerting modality (i.e. the ‘early-warning system’) [75,76], providing critical information about the environment and the need to reorient attention, thus conferring great phylogenetic advantage. We know, for example, that salient sounds can both capture visual attention and activate human visual cortex even when they bear no relevance for completing a visual task [77] (see also [78]). Given the intimate relationship between attention and consciousness, differences in the relationship between attention and individual sensory systems are likely to bear heavily on the study of conscious perception in those systems. Therefore, when devising strategies and experiments to study conscious auditory perception, it might be prudent to remain open to the possibility that it differs (at least in certain respects) from that of conscious perception in other modalities.

(b). Detection, discrimination, recognition and scene analysis

Like other sensory systems, the auditory system supports the perception of events, objects and their attributes. Auditory events are transient changes in the acoustic environment whose temporality (onset, duration, offset) is a salient feature. By contrast, auditory objects are quasi-stationary, perceptual collections of grouped attributes whose temporality is relatively less prominent. The perceptual features of auditory objects include loudness, pitch, timbre, duration, timing relative to other events and auditory spatial attributes (apparent location, distance and size). These correspond, respectively, to acoustic dimensions of sound level; periodicity; spectral shape and fluxes in frequency, phase, and level; duration; relative timing; and differences related to sound fields and their presentation to the ears. Auditory cognitive attributes are related to accumulated experiences with objects, formation of similarity classes, external world models and anticipatory implications. Mechanisms for auditory separation and grouping of sounds (auditory scene analysis) are critical for tracking and anticipating the likely behaviour of other animals and objects in the immediate environment. They are also essential for acquisition of tokens in human speech (phonetic elements, syllables, words and phrases) and animal acoustic (calls) communication systems. (For further discussion on auditory objects and their neural basis, see [79].)

Accurate representations of acoustic events facilitate effective and appropriate action by providing real-time information related to environmental conditions and changes. Auditory representations enable different classes of environmental sound sources, such as predators, prey, offspring, mates and potential rivals, to be detected, discriminated, recognized and located.

Another major function performed by audition is scene analysis [80], that is, how the environmental stimuli that impinge on our sensory organs are perceptually grouped, organized or decomposed into representations which ideally constitute individual sources (though ambiguous and incorrect groupings are also possible). Auditory scene analysis, as it is known [28,81], provides such grouping information on the basis of the fundamental features of sound (including frequency, spectral structure, intensity, and interaural time and level differences) and their loose perceptual counterparts (pitch, timbre, loudness and perceived location, respectively), any combination of which can emanate from a single sound source.

Along with prior information (or schemas, as Bregman termed them), these features give the brain information about the sound sources present in the environment, and in complex settings, these grouping processes (collectively termed auditory scene analysis) allow a listener to selectively attend one source or another (see also [82,83]). It is here that conscious perception of sound might provide considerable benefit to the organism or system possessing it, as the deployment of volitional attention is possibly enhanced by consciousness [32,62,64] or (equivalently) the processes that give rise to it. Furthermore, because there is evidence to suggest that the grouping process itself might be affected by selective attention [84–86], the processes underlying conscious auditory perception might, in turn and via attention, impact the bottom-up perceptual organization of sound. Though this is not to say that all mechanisms of auditory scene analysis require or are even influenced by consciousness, it remains an open question whether there are some that do (or are) [11,51].

(c). Speech and music

Speech communication is one of the most important aspects of our human self-perception, and the cortical systems for speech are intimately linked to those of audition [87]. This does not mean that there is no language without audition: it is clear that non-auditory communication can be effectively used by the deaf. Still, a coevolution of speech, language and auditory systems has probably shaped the human brain. Typically, the information comprising human speech goes beyond its semantic content, and it may thus consist of information that is not primarily processed at a conscious level. At a more basic level, semantic comprehension of spoken language is thought to be a clearly sufficient marker of consciousness [88,89].

Related to prosody, music is a unique and ubiquitous human phenomenon that produces diverse cognitive, emotional, motivational, hedonic, aesthetic and social-psychological (e.g. bonding) effects on its listeners [90]. Although most music is experienced in the waking state, appropriately structured music can trigger changes in conscious state, e.g. from waking to sleep and vice versa, facilitate induction of alternative conscious states (e.g. hypnosis, trance) and may even have therapeutic potential [26].

4. Conscious auditory phenomena and their putative neural correlates

What are the perceptual and cognitive phenomena that need to be explained by a theory of conscious auditory perception [91], and what are their neural correlates? The purpose of this section is to provide a comprehensive overview of conscious auditory phenomenology as well as to survey the work that has attempted to identify the aspects of neural activity that are putatively associated with it.

(a). Sound detection

As in any other sensory modality, the most basic phenomenon that must be explained by a theory of conscious audition is the detection of the presence of sensory stimulation (here, sound). The most basic way in which to examine this is to present peri-threshold sounds (either in quiet or with some amount of steady-state background noise)—while simultaneously recording brain activity—and ask the listener to indicate on a trial-by-trial basis whether or not they heard the sound.

One of the earliest examples of an empirical study examining the neural correlates of consciousness, auditory or otherwise, was conducted by Hillyard et al. [23]. While simultaneously recording single-channel electroencephalography (EEG) from the vertex (referenced to the mastoid), they presented listeners with a tone-in-noise detection task and sorted trials into hits (true positives), misses (false negatives), false alarms (false positives) and correct rejections (true negatives). The critical comparison of hits versus misses for physically identical stimuli revealed both an early vertex negativity (peaking between 100 and 200 ms) as well as a later, broad positivity (peaking between 250 and 350 ms) associated with tone detection. While the early response is probably related to one of the well-described negative-going components arising from auditory cortex (AC) [92–94], the latter response is probably the P3 or its subcomponent the P3b, a distributed potential with multiple generators [95] associated with context updating and conscious recognition of task-relevant target stimuli [96,97]. Interestingly, particularly as it relates to important current debates regarding the P3b's candidacy as an NCC, passive presentation of clearly suprathreshold tones only elicited the first of the two responses; the P3(b) was absent.

Apart from a subsequent study from the same authors [98], there have not been many directly related follow-ups to this now-classic report, at least not in the auditory domain. What has been done in more recent years has focused on how pre-stimulus brain states can affect whether subsequent near-threshold acoustic events are consciously perceived [99–103]. Although the effects are subtle and multifaceted, the answer to this question seems to be ‘yes’. However, because these aspects of brain activity are measured before the arrival of an acoustic event in question, they are not likely to constitute true NCC_A but rather putative prerequisites [16,29] that can impact subsequent stimulus-driven activity. In the subset of studies that reported post-stimulus activity [101–103], it is clear that hits elicited larger activity in AC than did misses. Although exact source topographies and response polarities are difficult to determine, the post-stimulus response from at least one study [103] appears consistent with the awareness-associated vertex negativity reported by Hillyard et al. [23].

Using a modified version of the so-called multitone masker stimulus [104,105] whereby otherwise-suprathreshold target tones are rendered inaudible, another line of work has examined neural correlates of conscious auditory perception in the context of more complex acoustic scenes [106–111]. These studies all make use of a phenomenon known as informational masking. Although not adopted by everyone, the broadest definition of informational masking is perceptual masking that cannot be accounted for by physical interactions at the level of the auditory periphery [21] (distinguishing it from so-called energetic masking). The multitone masker paradigm achieves this by way of both target uncertainty (the pitch of the target-tone stream is not specified on a trial-by-trial basis) and target-masker perceptual similarity (i.e. the presence of a ‘cloud’ of masker tones placed randomly in frequency and time), while energetic masking is mitigated by the presence of a ‘protected frequency region’ around the target stream (figure 1). In searching for the NCC_A, this paradigm has a distinct advantage over those using near-threshold stimuli, namely that the phenomenological distinction between perceived and unperceived targets is large due to the fact that the target stimuli are well above sensory threshold. This is perhaps akin to crowding effects in inattentional blindness or search paradigms from vision in that if attention is directed towards the to-be-presented target stream, it is readily perceived (as the change often is in change-blindness or change-deafness paradigms). It is also important to note that a motor response is only required at the moment of initial detection of the target stream, leaving several subsequent perceived target-stream tones free of motor response and planning.

Figure 1.

Schematic illustrations of three popular stimulation paradigms in the study of auditory scene analysis and conscious auditory perception. (a) The ABA-streaming paradigm, in which the same physical stimulus can give rise to two different percepts (one stream or two). (b) The jittered multitone masker paradigm, in which the regularly repeating target tones (shown in blue) are only sometimes perceived. (c) The stochastic figure-ground paradigm, where the coherence of certain spectral elements in and across time can give rise to the perception of an auditory figure distinct from the background. Reproduced from [112].

A consistent finding from these studies is the presence of a broad, negative-going deflection arising from (non-primary) AC between approximately 100 and 250 ms—the so-called awareness-related negativity, or ARN—that is only elicited when listeners report being aware of the target tones [107–110,113] (figure 2). However, another study has cast doubt on whether the ARN indeed reflects perceptual awareness of individual masker-embedded targets [111]. That study presented listeners with only two target tones and found that only the second elicited an ARN (though this was not the case for another previous report, where even the first of four successive masker-embedded target tones elicited an ARN [109]). Thus, whether the ARN reflects individual target-tone awareness as opposed to something else (e.g. selective attention [85,114] or stream segregation [111]) remains a bit unclear. The only activity from [111] that was enhanced for both detected targets over their undetected counterparts was (i) an early response arising from AC interpreted as the P1 (a vertex-positive component peaking between about 50 and 80 ms) and (ii) a broad, later response interpreted as the P3b, the latter of which was also observed by a recent intracranial study [113]. What these multitone masking studies further indicate is the extent to which information can be represented in AC without giving rise to conscious auditory perception, which is not typically clear from studies employing simpler near-threshold detection paradigms. Specifically, most of these studies observed robust evoked activity in AC in response to undetected targets, namely early components including the P1 and steady-state response associated with even earlier components [107,108,110].

Figure 2.

Recent results showing characteristic source topographies and waveform morphologies of both the mismatch negativity (MMN) (panels a–c) and awareness-related negativity (ARN) (panels d–f). (a) Typical source topography of the MMN, with sources in anterolateral portions of human AC. (b) Source waveforms of responses to standard tones (black), deviant tones (grey) and their subtraction (deviants minus standards, lower panels), clearly revealing the MMN as a prominent negativity between 100 and 200 ms. (c) Associated amplitude quantifications. (d) Source waveforms for virtual targets, i.e. targets that were absent (grey), detected and undetected standard tones (solid and transparent blue, respectively), detected and undetected deviant tones (solid and transparent orange, respectively), and their respective difference waveforms (lower panels). Panel (d) shows both the ARN as a strong, negative-going deflection in response to detected targets as well as the fact that when embedded in the multitone masker cloud, the MMN was only elicited when the standard stream was detected prior to the occurrence of the deviant. (e) Corresponding source locations and amplitude quantifications (f). Reproduced from [110].

(b). Change detection

Another paradigm used to study conscious auditory perception is change detection. The most popular set-up is the oddball paradigm, where infrequent ‘deviant’ sounds are interspersed with more frequent standards [115]. In addition to the response evoked by standard sounds, such deviants also elicit a so-called mismatch negativity (MMN), a change-related component arising from AC and typically peaking between 130 and 150 ms [93].

The MMN has long been considered an automatic, pre-attentive component, which does not reflect conscious processing itself, but typically leads to perceptual awareness by capturing attention [116]. However, this view has been challenged by the observations that (i) the MMN is modulated by attention [117] and perceptual organization [118], and (ii) that generation of the MMN requires perceptual awareness of the standard stream [110]. The latter study embedded classical oddball sequences into multitone masker stimuli and had listeners indicate their perceptual awareness of the repetitive standard sounds; the MMN was evoked only when listeners were aware of the standard stream prior to the onset of the deviant, but not when the standard stream remained subliminal. While listeners in that study indicated perceptual awareness of the standards and not the change per se, previous studies had also demonstrated the MMN's proximity to behavioural deviance detection, suggesting that it, too, may be coupled to conscious change detection [119] (but see [120]).

More recently, change detection has been probed for more complex scenes, which led to the demonstration that listeners often remain unaware of changes that occur well above the sensory threshold, termed change deafness. A typical paradigm presents a short scene of auditory objects [121], or more basic sounds [122], and repeats the scene after a brief silent period. In the second presentation, objects that were present in the initial scene may be removed or replaced, or new objects added. While the detection rate for removed objects in such set-ups decreases rapidly with the number of objects in the scene [121,122], detection of added objects is much easier [122]. While it was initially thought that change deafness occurs outside attention (and probably also working memory) [121], later evidence suggested a role for more basic mechanisms, particularly for added objects, and that change detection may not require perceptual awareness of the object in the first scene [122] (though more recent data suggest that awareness of individual objects can have an effect [123,124]).

M/EEG studies of change deafness have consistently observed a vertex-negative response between approximately 100 and 200 ms for behaviourally detected changes that is smaller or absent for missed changes [125–127] (but see [123]), similar to findings for detection [23,107], with likely sources in AC. However, it remains unclear if this response is associated with implicit awareness of the change per se [122,128], or explicit awareness of the object in which the change occurred [121]. Implicit change detection might also be supported by earlier change-related responses that appear even on trials where the change was not reported [126,128]. While recent evidence from modelling of behavioural data generally challenges the concept of implicit perception [129], the papers reviewed above suggest that change detection in complex scenes may operate on representations at different levels, both before and after those associated with conscious perception. fMRI results show frontoparietal, insular and anterior cingulate activity that also dissociates between perceived and unperceived changes [130]. This activity is potentially related to the late P3a and P3b components which, along with the earlier negative-going change-related responses, are also often observed for perceived changes in change-detection paradigms [123,125,126]. However, as for simple detection, it remains unclear whether these components (and associated fMRI activity) reflect change perception per se, as opposed to task-related post-perceptual processes [16,29,30].

(c). Scene analysis and bistable perception

Perhaps, the converse question to how physically different stimuli can give rise to similar conscious percepts (cf. §4d) is how physically identical stimuli can be perceived differently. This is known as bistable (or multistable) perception [18], of which there are several classic examples from vision, with perhaps the best-known being the Necker cube (cf. fig. 4 of [9]). There are also examples of bistable perception in the auditory domain, with the most ubiquitous being auditory streaming [131–135]. This stimulus consists of a sequence of alternating tones or noises such that for different parameters, the sequence sounds as though it is emanating either from a single, alternating source (one stream), or from two concurrent, isochronous sources (i.e. two streams) (figure 1). Interestingly, there is a large acoustic parameter space for which the sequence can be perceived either as one or two streams, with both spontaneous switches and some degree of volitional control regarding which is perceived at a given moment.

This enables the study of conscious contents by comparing brain activity between epochs during which one stream is perceived with those during which two streams are perceived. Many studies have attempted to identify neural activity that is associated with the percept of one or two streams [136–139] or the perceptual transition from one to the other [140,141] (see also [142]). The general findings from studies examining differences between one- and two-stream percepts are enhanced activity in posterior AC [136,138,139,143] and the intraparietal sulcus (IPS) [143,144] during two- versus one-stream percepts. The fact that a human intracranial study did not find correlates of such bistable streaming percepts (despite the high spatio-temporal resolution and broad coverage associated with intracranial recordings) further suggests that they may be restricted to certain anatomical areas (posterior AC and the IPS) or encoded in such a way so as to require multivariate analysis methods to reveal them [137]. Switches between percepts during streaming have been associated with transient activity in a distributed cortical/subcortical network including AC, thalamus, supra-marginal gyrus, insular cortex and the inferior colliculus [140,141], which may reflect the brain's updated context about the acoustic scene [140]. Similar response profiles have also been observed for the so-called verbal transformation effect, in which repeated presentation of the same speech token can lead to changes in its perception (e.g. ‘stress’ versus ‘dress’) [145]. Whether such transient activity reflects a cause or consequence of switching between perceptual states remains unclear.

A related scene-analysis paradigm that may also turn out to have implications for neural theories of conscious auditory perception makes use of the so-called stochastic figure-ground (SFG) stimulus [146]. Like the multitone masker paradigm described above, this stimulus is also composed of multiple random pure tones, with a target (or ‘figure’) comprising a set of synchronous tones that repeat over successive presentations of otherwise random chords (figure 1). Listening to these stimuli appears to induce activity in both AC (with N1/N2-like scalp topographies [147]) and the IPS [146,148], with larger activity both for stimuli evoking more distinct perceptual figures and for active over passive listening. When taken together with similar findings from the aforementioned streaming [143,144] and multitone masker [111] studies, these results suggest a role for the IPS in either auditory scene analysis or conscious auditory perception. However, whether the IPS plays an active role in auditory scene-analysis processes [144,146,148] or rather reflects pre- or post-perceptual processes [111] (e.g. attentional state or decisions, respectively) remains an open question [112].

In any case, the figure-generated auditory-cortex response from the aforementioned magnetoencephalography (MEG) study [148] closely resembles the ARN from [107–111] in that it is a late, surface-negative response. Furthermore, like the ARN, it covaries with perceptual detection of the figures, at least for certain stimulus parameters [149]. However, an interesting difference to note is that while the ARN is elicited by individual tones which together comprise a foreground stream, the evoked response in the SFG paradigm is elicited by a figure and not the individual chords comprising the said figure. While this is partially due to the fast repetition rate of the chords (25 ms, for example), it is nevertheless important to consider how the SFG and multitone masker paradigms relate. For example, although the ARN is elicited by individual tones, both it and listeners' perception of the individual tones generating it may be greatly enhanced when the tones group as a stream and segregate from the random cloud in which they are embedded [111] (see also [109]). Thus, while the result from [111] that only the second of a two-tone figure elicited an ARN suggests that it may not reflect perceptual awareness of individual target tones, it leaves open the possibility that it reflects a phenomenological representation of the figure (similar to ideas from reverse hierarchy theory [11]) such that perception of the stream improves subsequent perception of its individual elements [150], perhaps even gating to awareness prior target tones that were initially subconscious [151,152].

Another scene-analysis paradigm that bears on the study of conscious auditory perception is the auditory continuity illusion, where a physically interrupted sound can be perceived as continuous in the presence of interrupting noise [153,154]. A general consensus from neural studies of the continuity illusion, irrespective of whether they measured unit firing [155], event-related potentials [156,157], spectral perturbations [157–159] or the fMRI BOLD signal [160–162], is that physically identical stimuli that are perceived to be discontinuous elicit more activity than those perceived to continue through the interrupting noise. This activity arises from AC and (in humans) probably in the N1 latency range, and has been interpreted to reflect conscious perception of the silent gap present in the stimulus of interest [106] or perceptual restoration of its missing portion [79]. Involvement of other brain areas in perceptual restoration remains to be identified (though see [162]) and likely depends on the nature of the stimuli used (e.g. pure tones versus vowel stimuli, where the latter might be expected to produce activity in speech-specific regions).

Finally, there exists an entire class of auditory illusions and bistability phenomena related to the fact that pitch perception has two dimensions—class and height—with the class dimension being circular [163] (cf. §4d for further discussion of pitch) (see also [164]). These phenomena, many of which have been well described by Deutsch [165], have yet to be fully exploited or understood from a neural perspective [166] (but see [167–170]), and include infinitely rising/falling pitches [163], the ‘tri-tone paradox’ (where successive harmonic complex tones separated in pitch by six semitones—i.e. a tri-tone—can be perceived as either rising or falling), and the octave and scale illusions [171] (see also [172]). Another pitch-related illusion discovered by Deutsch that has yet to be fully exploited (but see [173]) in the study of the NCC is the speech to song illusion, in which an isolated token of speech can be perceived as song when repeated indefinitely.

In the interpretation of the above studies and also for future studies of conscious perception, one thing should be reiterated: while bistable phenomena are important for studying perception because of the constant physical stimulus, this does not guarantee that they reveal ‘true’ NCC. It is important to consider whether the neural patterns that dissociate two percepts may instead reflect neural processes associated with automatic scene analysis, and that they may not yet be sufficient for conscious perception itself.

(d). Auditory invariances and context effects

Beyond the mere presence of a conscious auditory percept (i.e. detection of sound), there are several types of auditory phenomena that allow investigators to begin to approach the phenomenological qualities of conscious audition. One example is the perception of pitch [174], a quality of sound reflecting its ‘highness’ or ‘lowness’. Although a full treatise on pitch is outside the scope of the present article, it bears on theories of conscious audition insofar as many physically different acoustic stimuli can all give rise to the same pitch percept, an example of perceptual invariance (timbre operates similarly [175]). How such invariance is achieved, and what aspects of neural activity give rise to it, is a fundamental question of conscious audition. What we know concerning the neural basis of pitch is that some specificity for pitch has been observed in more lateral parts of AC, near the border between core and secondary fields [176]. However, exactly which area, and exactly how pitch is represented there, remain unclear.

Another example of perceptual invariance in audition are phonemes, which may be the fundamental elements of speech perception [177], and which are perceived categorically (i.e. large changes in acoustic features do not give rise to correspondingly large changes in phonetic/syllabic perception unless they cross a categorical boundary); this is reflected by neural activity in posterior AC [178–180].

Examples of context dependence include sine-wave speech, where spectrotemporally dynamic pure tones can be perceived as speech depending on the availability of prior information [181] and the McGurk effect [182], where the perception of a consonant-vowel pair can be altered by simultaneously presented visual information (e.g. an acoustic ‘ba’, when paired with a video of the utterance ‘ga’ leads to perception of ‘da’ in most listeners). Stronger activity for sine-wave speech (when perceived as speech) has been observed in the superior temporal gyrus [183] and superior temporal sulcus [184], consistent with these areas' involvement in speech processing [185–187]. For the McGurk effect, functional imaging studies indicate that it is also reflected by activity in AC [188–190]. For example, when ‘va’ was used as standard and ‘ba’ as a deviant, no MMN was evoked when ‘ba’ was perceived as ‘va’ because of the video [188]. As the McGurk effect can be reduced with transcranial magnetic stimulation over the superior temporal sulcus, the latter is a likely site for audio-visual integration and top-down control in AC on the superior temporal plane [191].

(e). Auditory imagery

Apart from stimulus-related processing, it is also possible to imagine sound, including complex scenes (e.g. polyphonic music). In most cases, these sounds are recalled from memory, though we are also able to imagine novel sounds (a melody, for example). While these contexts are quite distinct from conscious perception of actual sound, we may nevertheless expect considerable overlap of the neural systems involved. Functional imaging studies have consistently demonstrated activation of secondary AC during auditory imagery [192–194] as well as activation of tonotopically appropriate regions of primary AC corresponding to imagined pitch [195].

In a related set-up, Meyer et al. [196] played short, silent videos that implied the presence of sound and were able to decode which video was played based on BOLD activity in AC despite not having instructed participants to imagine sound. This strongly suggests that AC activity shows specificity for imagined content. Given the close relationship between short-term memory and selective attention [197], part of the specificity observed during auditory imagery [195] may be related to stimulus-specific attention [198]. In a direct comparison, however, similar activation patterns were observed for perception and short-term memory, whereas the representation of auditory imagery was distinct of the former and supposedly at a more abstract level [199]. Another important aspect of auditory imagery, particularly in musical contexts, is the potential association with silent vocalizations, which may play an important role [200]. At this point, the degree to which activation of frontal and parietal areas during auditory imagery [192,193,200] are related to cognitive control, memory retrieval or motor programmes requires further clarification.

(f). Auditory experience in the absence of corresponding sound sources

Drastically more vivid auditory experience in the absence of sound may occur automatically in a number of pathological states, with tinnitus being the most common. One model of tinnitus suggests that spontaneous firing rate in the subcortical pathway and in AC is elevated, probably as a consequence of peripheral deafferentation and maladaptive central inhibition [201]. Thus, the pathophysiology is likely in parts of the auditory pathway that are necessary but not sufficient for perceptual awareness, whereas higher-order cortical circuits—which likely form part of the constitutive areas for conscious auditory perception—simply pick up the pathological activity from earlier stations, which then gives rise to the subjective percept of tinnitus.

However, illusory auditory perception does not require subcortical coding, as is evident from epilepsy and cortical stimulation studies. In epilepsy, pathological synchronous activation emanates in cortical circuits and evokes motor activity or experience that depends on epileptogenic source and spread. For example, auditory sensation is a known symptom of lateral temporal lobe epilepsy, ranging from elementary tonal or ringing sensations up to music, complex scenes and human voices [202,203]. Briefer sensations can be evoked by cortical stimulation of temporal cortex [6], which has been used for intraoperative cortical mapping. While we do not know the neural code that is induced in auditory fields by either epileptic seizures or electrical stimulation, it is unlikely that it follows physiological routing from primary to secondary cortex and back. It appears more likely that a more isolated activation starting in (non-primary) auditory fields (and potentially projecting from there to primary AC and other areas) is sufficient to produce auditory experience in this context, although the evidence is vague at this point.

The third instance of pathological auditory perception is verbal hallucinations, which are a characteristic symptom of schizophrenia. Functional imaging during auditory hallucinations has provided variable results [204]: while one study showed primarily AC activation [205], other studies additionally show activation in the precentral gyrus and frontotemporal language networks [204,206]. Whether the AC is important for the external attribution of auditory verbal hallucinations [205], or if AC dysfunction could even be a source of auditory hallucinations [207], remains unclear at this point. In any case, coactivation of auditory and frontal fields during auditory hallucinations as well as auditory imagery is generally consistent with a role for these regions in conscious auditory perception.

(g). Lack of auditory awareness in patients with circumscribed cortical lesions

In contrast to auditory experience without sound, lesions of NCC-relevant structures can abolish conscious experience of sound that is present. However, some caution is warranted here. First, the NCC concept does not state that a given NCC network needs to be exclusive for, for example, auditory perception, just that its activation is sufficient (i.e. neural ‘degeneracy’ [208]). Thus, conscious experience may continue because a parallel system may also be sufficient to give rise to conscious auditory perception. Second, the lesion-related lack of conscious experience cannot be taken as proof that the lesion involves the NCC. For example, ascending auditory pathway is expected to be necessary but not sufficient for auditory experience; their destruction is expected to abolish conscious auditory experience but their activation, per se, is insufficient for it.

Keeping these limitations in mind, a relevant and much-debated syndrome is cortical deafness [209]. Analogous to vision [210,211], cortical deafness had been expected with bilateral lesions of the primary AC; however, convincing cases have been scarce [212]. Many patients with extensive bilateral AC lesions are profoundly deaf acutely, but typically regain basic sound sensitivity [213] and even basic (if impaired) discriminatory functions [214,215]. However, more complex functions such as speech and music perception and sound-source identification typically remain well below normal [216,217]. Similar observations have been made after bilateral removal of auditory core fields in monkeys [218], which cause initial unresponsiveness and subsequent threshold elevations, probably indicating elementary auditory processing deficits with regained auditory awareness.

Our interpretation of this literature is that persistent cortical deafness requires complete bilateral destruction of core, belt and parabelt areas of AC [212]; even small amounts of residual cortical tissue receiving projections from the auditory thalamus may permit awareness of sound presence. Even with complete lesion of core and belt AC, however, one patient was reported to have residual auditory awareness [219], enabling an almost normal audiogram. However, this required the patient to pay focused, undivided attention to the task. While no description of the patient's subjective hearing experience was reported, the authors' classification of the phenomenon as ‘deaf hearing’ [219], like blindsight [220,221], suggests that his detection of sound was implicit rather than phenomenological. This interpretation is supported by another case of a patient with completely destroyed auditory cortices who showed occasional reaction to sound but who denied any subjective experience of sound and was completely deaf in audiometric testing [222]. Based on functional brain mapping in the first case [219], those authors suggested that remaining auditory-related cortex in the middle temporal gyrus could be the source of auditory responsiveness (there was also activation of prefrontal areas). However, it remains unclear how auditory information was routed towards these structures, and alternative pathways, for example via unspecific thalamic nuclei, may need to be considered as source of the phenomenon.

The most disabling deficit for patients with extensive, bilateral AC lesions is typically impaired speech perception. While this may be attributed to elementary processing deficits in some patients [214,215,223], it can also occur with relatively intact basic hearing and core AC. This syndrome is known as pure word deafness [224], where ‘pure’ is meant to indicate that these patients do not suffer from aphasia. Among the variable lesion configurations that have been observed to cause pure word deafness [225], some patients have relatively intact core and belt cortex, but bilateral lesions of the superior temporal sulcus [226,227], a region that has been suggested to be involved in pre-lexical speech processing [185]. Most patients who suffer from pure word deafness also have some deficit in recognizing non-speech auditory objects, also referred to as auditory (object) agnosia [228]. Only a few reports exist that have tested both and found a dissociation of impaired speech perception and object agnosia [229,230], and a dissociation of lesion sites that favour one over the other has not been established. In summary, observations based on bilateral cortex lesions provide multiple lines of evidence for a role of AC in perceptual awareness of sound identity. Lastly, while AC is very likely part of constitutive areas giving rise to the awareness of sound source presence given overwhelming neuroimaging evidence and the fact that large-scale bilateral lesions to AC usually result in profound acute deafness, most patients with such lesions typically recover their sound sensitivity after some time. Thus, whether AC is absolutely necessary for basic conscious audition remains unclear.

In contrast with the visual system, unilateral lesions of the AC do not typically cause impaired perceptual awareness of contralateral sounds; only in the presence of synchronous stimulation in the ipsilesional ear is awareness of contralateral stimuli typically impaired [231–233]. Similar extinction phenomena are observed in patients with neglect [58,234,235]. Neglect is a multimodal disorder subsequent to right-hemisphere, peri-sylvian lesions, which typically involves reduced awareness of visual objects in the left hemifield [59]. As with unilateral AC lesions, lack of awareness of sound presence is rarely observed in the absence of competing sound sources. Instead, patients with severe neglect may mislocalize sound to the right, a phenomenon known as alloacusis [58].

One model suggests that a lesion to the right-lateralized ventral attention system is the source of neglect [59]. This system is thought to automatically redirect attention to new or salient stimuli. A lesion of the ventral attention system may indeed also be the cause of a deficit in responding to rare targets, which has been observed as a prominent auditory manifestation of neglect [236], and independent of sound lateralization [233]. Whether this target-detection deficit is directly linked to a lack of awareness of these sounds, or just to awareness of their task relevance needs further evaluation before it can potentially be considered a lesion model of impaired auditory awareness despite intact AC.

5. Auditory markers of states/levels of consciousness

Owing to the relative ease with which acoustic stimuli can be delivered in less-than-ideal settings (e.g. during sleep, under anaesthesia or in disorders of consciousness), responses to such stimulation are increasingly being evaluated as potential markers of conscious states/levels [89,237,238], particularly when other types of stimuli (visual, for example) are difficult to present. As mentioned in §3, the auditory system remains quite responsive during various stages of behavioural unconsciousness, including sleep, anaesthesia, minimally conscious state, persistent vegetative state (VS) and even coma. However, such responses are morphologically distinct from those during wakefulness [74,239]—except, perhaps, for very light sleep (stage 1, for example, which is not consistently interpreted as sleep [240])—and thus the question becomes which of these somewhat anomalous responses reflects conscious perception or, minimally, some level [241] of consciousness.

In sleep, the transition from conscious wakefulness to unconscious sleep is generally taken to occur between sleep stage 1 and deeper stages such as stage 2 or rapid-eye-movement sleep. Interestingly, despite the long-held notion that it is pre-attentive and even preconscious, and although there is little consensus, the neural response that tracks this transition most closely may in fact be the MMN [111,242–245]. This is particularly true if one sub-categorizes stage 1 into responsive and non-responsive portions [243]. The MMN is equally sensitive to transitions between wakefulness and sedation under anaesthesia in that it is completely abolished during or even prior to behavioural unconsciousness [242,246]; even under light anaesthesia its amplitude is drastically reduced [244,247]. The situation is a bit more complicated in the disorders-of-consciousness literature, where the MMN has been observed during states of behavioural unconsciousness, although the inter-individual variability typical of evoked responses may partially explain the discrepancy across studies. However, as has been noted [238,248–250], the MMN (and particularly its emergence across successive recording sessions) is highly predictive of imminent recovery of consciousness, suggesting that the MMN may be one of the earliest available indicators of partial awareness in such patients. Lastly, strong support for the MMN as a marker of consciousness comes from a recent study in which it was found generation of the MMN required awake, behaving listeners to be perceptually aware of the preceding context regularity [111].

Other candidate markers of consciousness include both early [251,252] and later [237,243,253,254] components. Of the earlier components, the N1 seems to be the most closely related to conscious state [74,239,240,242,246,247,253]. While some have identified the middle-latency and associated auditory steady-state responses as potential markers [251], we know from other studies that these responses are themselves insufficient to infer consciousness [108]. However, the absence of these responses is taken to be a strong indicator of unconsciousness [252]. For later components such as the P3b, the converse is generally true. That is, in inferring consciousness, the P3b is taken to be a sufficient but unnecessary marker, such that individuals in whom it is observed can generally be inferred to be conscious, while the converse inference cannot be made; individuals in whom the P3b is not observed may yet be conscious.

While evoked potentials have received the most attention in the search for candidate auditory markers of consciousness, more recently, some have used more expensive and powerful techniques such as MEG and fMRI. One of the studies that provided an even greater impetus for neural markers of consciousness in behaviourally unresponsive individuals is the command-following example of Owen et al. [255], where a putative VS patient was demonstrated to be conscious based on their ability to generate imagery-based activation patterns that were indistinguishable from healthy, conscious controls (for review of this and other methods see [256]). However, in this paradigm, the bar for inferring consciousness is set quite high, and probably has a considerable false-negative rate, similar to the inference of spoken language comprehension [88,89,257]. More recent methods that infer consciousness based on the brain's response to naturalistic stimuli (e.g. movies) may prove to be more sensitive [258].

6. Toward a comprehensive framework for conscious audition

Further specifications of the NCC_A, their relationship to attention and other cognitive variables, and the potential system-level effects of conscious perception are needed in determining the plausibility of the models summarized in §2. At present, there is little consensus concerning these questions, either empirically or in the models, which vary widely in their emphasis. This stems at least in part from (i) diverse definitions on what consciousness is or does, (ii) confounds related to the nature of paradigms used to examine it, and (iii) the fact that most models derive exclusively from vision. Given potentially fundamental differences between vision and audition outlined earlier, it may prove difficult—at least initially—to incorporate findings from the auditory domain into a generalized, parsimonious account of the neural basis of conscious perception, and it may be necessary to combine specific features of different models into a larger, overarching framework.

(a). Isolating the NCC_A

Recent years have seen a dramatic rise in studies attempting to identify neural correlates of conscious perception. While this is particularly true in vision, our review of the literature suggests a similar case in the auditory domain. Not surprisingly, much of the work that has taken place has focused on activity in AC (particularly secondary AC), which is almost certainly a central hub of conscious auditory perception and auditory cognition in general [259]. However, what other aspects of brain activity might constitute the NCC_A has not been examined to nearly the same extent as either their visual counterparts (but see [112,114,141,145,254,260]).

Furthermore, the role of extra-AC activity is called into question by a host of recent visual studies presenting strong evidence against such activity constituting part of the NCC_V [261–263] (for reviews, see [10,30]). Motivated at least in part by the potential confound of overt perceptual reports [16,29,30], the picture that emerges from these studies is that traditional NCC markers from global-workspace theory (frontoparietal activity or the P3b, as measured via non-invasive neuroimaging) may instead reflect ‘consequences’ of conscious perception associated with task relevance and post-perceptual processing. However, it should be noted that the core predictions of global-workspace theory (i.e. that ‘ignition’ of activity in global-workspace neurons in frontoparietal cortex is necessary for conscious perception), while damaged, remain NCC candidates despite the fact that certain markers may have been wrong. There is little doubt that auditory information has (perhaps automatic) access to frontal and parietal cortex [264,265], but whether this aspect of brain architecture supports basic phenomenal awareness as opposed to processes necessary for the perception–action cycle remains difficult to tackle [27,39]. In addition, there may still exist latent NCC-related frontoparietal activity that is difficult to measure without the use of more advanced methods [266,267] (see also [268]) or appropriate animal models [269].

Additionally, animal models (in addition to human intracranial recordings and high-field fMRI [270]) might help to determine whether the putatively awareness-related surface-negative responses in AC, which correspond well with similar negativities in vision [25], reflect recurrent processing (posited by several models as a critical processing mode for conscious perception [35,36,271]) either within auditory association cortex or between it and other brain areas. While this is suggested by their latency and by the fact that similar negativities in monkeys arise from superficial cortical layers [272], more work is needed, particularly in establishing NCC_A experiments in non-human primates.

In the auditory domain specifically, several studies have suggested the importance of extra-auditory brain areas [100,103,112,141,145] and late physiological markers [112,114,254,260] in conscious auditory perception. However, the same issues of task relevance and use of overt reports that exist in vision are also relevant here, and studies that have used passive (in addition to active) versions of NCC_A paradigms have found no evidence for such activity [108,137], and in some cases specific evidence against it [23]. Thus, further work along these lines is clearly needed in order to isolate the NCC_A from pre- and post-perceptual processes, particularly regarding the extent to which late, distributed responses (in M/EEG) and task-positive frontoparietal networks (in fMRI) correlate with conscious perception. This is particularly true when attempting to examine potential NCC_A in the context of complex auditory scenes, where several cognitive processes in addition to those giving rise to conscious perception are likely to be present (selective attention and context updating, for example).

Another way to approach this problem is via the use of specific physiological markers which, via traditional report-based paradigms, can be shown to correlate with conscious perception. One can then examine such markers in the absence of overt reports in order to sort data according to moment-to-moment conscious contents. One marker that may prove particularly useful is pupil dilation, which has been used to examine the NCC [261] and other cognitive variables, including selective attention [273], listening effort [274] and deviant detection [275,276]. It is therefore conceivable that phasic pupillary dilations can also be used as a proxy for conscious perception in studies of the NCC_A. Furthermore, probably by virtue of its coupling to norepinephrine systems and the locus coeruleus [277,278], pupil dilation is also a sensitive marker of arousal [275,279], which in turn correlates with activity in resting-state networks [280] and possibly AC [281], perhaps partially explaining the influence of pre-stimulus resting-state activity on peri-threshold sound detection [100].

Finally, another question that has seldom been posed (but see [11,45,51]) is at what level of abstraction from sensory features to stream/object representations does conscious (auditory) perception emerge. That is, what are we aware of, and how does this change with behavioural conditions or context. This is likely to be closely linked to both selective attention and the processes underlying auditory and visual scene analysis, and is also probably highly context dependent. We know, for example, that certain details of complex sensory stimuli as well as complex auditory/visual scenes are only consciously accessible when attention is directed towards them or when they become task relevant. Does this also change the neural signatures of conscious perception? How would this impact the goal of ‘isolating’ or ‘distilling’ neuronal activity associated with conscious perception? The results of such questions will likely bear on the question of how the brain represents conscious information.

(b). Relationship between conscious perception, selective attention and listening load

Both informational masking [21,282] and change deafness [122] depend on the complexity of the scene or the processing load. In accordance with the biased competition model of attention [283] and the load theory of attention [19], automatic detection of sound presence and change [117] could be reflected by the N1 and MMN in the absence of perceptual competition [107,108,111]. When under load, however, top-down attention (coordinated by the dorsal attention system [284]) probably strongly impacts which elements of the scene reach perceptual awareness. This is consistent with larger effects of selective attention in the presence of two interleaved auditory streams versus a single stream [285]. However, listeners probably remain basically aware of both streams under such intermediate amounts of load even if they are not typically aware of the unattended streams' content [13]; furthermore, though diminished, tones in both streams evoke an N1. Thus, another phenomenon, inattentional deafness [286], may potentially be explained along the same lines (but see [287] for an example that contradicts a major role of perceptual load).

In any case, the relationship between attention and conscious perception is a central question for the study of the NCC [61]. And again, knowledge concerning this relationship comes primarily from vision. However, the extent to which the auditory and visual modalities share attentional resources remains unclear [288]. Thus, it is perhaps imprudent to presume a priori that the relationship between attention and conscious perception will operate similarly in audition and other modalities, indicating the need for auditory stimuli/paradigms that allow for the independent manipulation of attention and awareness, and the findings of which should be incorporated in computational models (see [83]). However, as in vision, this is likely to prove quite challenging.

To the extent that auditory and visual attention rely on shared resources, as is suggested by the global-workspace model [35] and others, attention to a visual task may concurrently reduce the processing depth of sound; evidence for this comes in the form of smaller auditory-evoked responses with enhanced visual attentional load [289]. However, the degree of response suppression is not as great as that found with informational masking [107], which is a within-modality (i.e. auditory) phenomenon. A recent behavioural study has also shown strong within-modality (auditory) load effects [290], but to our knowledge, the influence of intra- versus cross-modal load on auditory perceptual awareness (but see [291]) and associated brain activity has not been directly compared. However, if conscious auditory perception turns out to depend strongly on activity in AC, one might expect stronger effects of auditory as opposed to visual or other cross-modal load. To reliably address this question, sources of informational masking in the ascending auditory pathway will need to be carefully controlled. By contrast, models which posit common NCC substrates [35] for various sensory modalities would predict similar effects of intra- and cross-modal load.

(c). Does conscious audition affect anything?

One of the questions that we posed earlier (§3) was whether conscious audition has the capacity to affect other brain processes. In general, this question (of potential downwards or circular causal influence of consciousness) has received far less attention than that of upwards causation (i.e. how conscious perception arises, admittedly the focus of the present article). Although this is intensely debated, many theories of conscious perception allow for such effects, and experiments designed to examine this hypothesis are certainly worth pursuing. Another way to ask this question is: are there any aspects of auditory function that depend (either positively or negatively) on the presence of subjective auditory experience? As we mentioned in §3, one candidate process that might depend on conscious audition is the control of selective auditory attention [32]. Although it is intuitively appealing that conscious perception helps in selecting relevant sensory information in complex environments, there is evidence that this and other processes that intuitively occur consciously can also occur in the absence of consciousness under certain circumstances [31,33]. The question going forward is where the boundary is between processes that depend on conscious perception and others that do not. Recent evidence in the visual domain suggests that conscious perception can have strong and specific effects on the control of attention, perhaps stabilizing it with respect to temporal fluctuations and distraction [292]. This remains to be explored in the context of audition, particularly in light of how each system (visual and auditory) interacts (potentially differently) with attentional networks and how this is affected by neglect.

Authors' contributions

A.R.D., P.A.C. and A.G. wrote the paper.

Competing Interests

The authors declare no competing interests.

Funding

This research was supported by the German Federal Ministry of Education and Research (BMBF) grant 01EV0712.