Neural signature of the conscious processing of auditory regularities

Abstract

Can conscious processing be inferred from neurophysiological measurements? Some models stipulate that the active maintenance of perceptual representations across time requires consciousness. Capitalizing on this assumption, we designed an auditory paradigm that evaluates cerebral responses to violations of temporal regularities that are either local in time or global across several seconds. Local violations led to an early response in auditory cortex, independent of attention or the presence of a concurrent visual task, whereas global violations led to a late and spatially distributed response that was only present when subjects were attentive and aware of the violations. We could detect the global effect in individual subjects using functional MRI and both scalp and intracerebral event-related potentials. Recordings from 8 noncommunicating patients with disorders of consciousness confirmed that only conscious individuals presented a global effect. Taken together these observations suggest that the presence of the global effect is a signature of conscious processing, although it can be absent in conscious subjects who are not aware of the global auditory regularities. This simple electrophysiological marker could thus serve as a useful clinical tool.

When we perceive a stimulus, our brain generates a complex pattern of neural activity, reflecting the summation of a large number of information-processing stages, some of which correspond to the conscious processing of perceived representations, whereas others reflect nonconscious processing. How could we disentangle the respective correlates of conscious and nonconscious perception within the same experimental paradigm using neurophysiological measures? From a clinical perspective, designing a simple neurophysiological test that could selectively monitor conscious-level processing and assess its integrity would be extremely useful for patients suffering from coma, persistent vegetative or minimally conscious states.

Neurophysiological monitoring of the perceptual categorization of a rare auditory deviant stimulus delivered within a serial flow of frequent standard stimuli offers a relevant step toward this goal. A rich literature demonstrates that the detection of novel auditory stimulus includes 2 distinct neural events, a mismatch negativity response (MMN) (1) followed by a later neural response labeled P300 (P3a and P3b) complex (2, 3). The respective properties of these 2 responses suggest that the MMN mostly reflects a preattentive, nonconscious response (4), whereas the late component of the P300 complex (P3b) has been theorized as an index of working memory updating (5) and is empirically associated with conscious access (6). The P3b component has been shown to be insensitive to interstimulus intervals (ISI) exceeding tens of seconds, and has even been observed for ISI as long as 10 min (7), thus implying an active maintenance of previous stimuli in conscious working memory. In sharp contrast, the MMN vanishes when the ISI exceeds a few seconds (8), suggesting a fast decay characteristic of nonconscious iconic memory (9, 10). In addition to this temporal distinction between MMN and P3b responses, MMN (and P3a) are largely resistant to top-down and attentional effects. They can even be observed during rapid eye-movement sleep (11) and anesthesia (12), and in unconscious comatose (13–15) or vegetative state patients (16, 17) or in response to visual subliminal stimuli (18, 19), whereas the P3b is highly dependent on attention and conscious awareness of the stimulus (6, 20).

Still, MMN and P3b events are close in time, sometimes difficult to differentiate, and the fine distinction between P3a and P3b makes it extremely difficult to identify with certainty a P3b component in individual subjects. To circumvent these limitations, we designed a paradigm in which 2 embedded levels of auditory regularity are defined, respectively at a local (within trial) and at a global (across trials) time scale. We predicted that violations of the local regularity should elicit measurable ERPs in both conscious and nonconscious conditions, but that violations of the global regularity should be detected only during conscious processing. In other words, the presence of an ERP signature of violation of the global regularity in an individual subject would be diagnostic of conscious processing. The auditory channel was chosen because we aimed at using this test with non communicating human patients, in whom auditory stimulation is easy to deliver and elicits robust activations without requiring active eye opening.

To validate our paradigm, we first analyzed its brain mechanisms with high temporal and spatial resolutions by combining high-density scalp ERP, intracerebral LFP, and fMRI measurements in conscious subjects submitted to distinct experimental manipulations of their consciousness of the stimuli. We then probed the scientific and clinical potential of our test by recording 8 noncommunicating patients either in the vegetative state (VS) or in the minimally conscious state (MCS). VS is a clinical condition lacking any behavioral sign of conscious processing despite a preserved waking state. MCS is a condition characterized by intermittent and discrete signs of conscious processing (21).

Results

The ERP Local-Global Paradigm.

On each trial, a series of 5 brief sounds was presented over a total of 650 ms (see Fig. 1A). The first 4 sounds were always identical, either high or low-pitched, whereas the fifth sound could be identical (locally standard trials) or different (locally deviant trials) to the preceding ones. In distinct experimental blocks, global regularity was defined according to the relative frequency of the 2 types of elementary trials. Globally standard trials were delivered pseudorandomly on 80% of trials, whereas the global deviant trials were presented with a frequency of 20%. Importantly, local and global regularities could be manipulated orthogonally, thus defining 4 types of trials: local standard or deviant, and global standard or deviant (see Fig. 1B). All subjects were presented with these 4 conditions. Each block started with 20–30 global standard trials to establish the global regularity before the occurrence of the first global deviant trial.

Fig. 1.

Experimental design. (A) On each trial 5 complex sounds of 50-ms-duration each were presented with a fixed stimulus onset asynchrony of 150 ms between sounds. Four different types of series of sounds were used, the first 2 were prepared using the same 5 sounds (AAAAA or BBBBB), and the second 2 series of sounds were either AAAAB or BBBBA. (B) Each block started with 20–30 frequent series of sounds to establish the global regularity before delivering the first infrequent global deviant stimulus.

In Experiment 1, 11 normal volunteers were instructed to actively count the number of global deviant trials while high-density recordings of scalp ERPs were collected. The local and global effects affected distinct time windows of the averaged ERPs. Violation of the local regularity (local deviant: local standard ERPs) was associated with 2 successive electrical events: first a vertex centered mismatch negativity appeared ≈130 ms after the onset of the fifth sound, followed by a central positivity with simultaneously bilateral occipito-temporal negativities ranging from 200 to 300 ms (see Fig. 2). Violation of the global regularity correlated with a central positivity, simultaneous to a frontal negativity, which appeared ≈260 ms after the onset of the fifth sound, and persisted until the end of the 700 ms epoch of interest. Interestingly, a late temporal window (320–700 ms) was affected only by the violation of the global regularity. Note that in this active counting task, no interaction was observed between local and global ERPs effects (see Figs. S1 and S2).

Fig. 2.

Local and global ERP effects in the active counting task. Averaged voltage scalpmaps of the local and global subtractions (deviant minus standard) are plotted (top) from 100 to 484 ms after the onset of the fifth sound. Corresponding thresholded t tests scalpmaps (red) are shown for each condition. ERPs of 3 representative electrodes are shown (bottom box) for the 4 elementary conditions (local/global X standard/deviant).

Global Effect Requires Awareness of Stimuli.

To evaluate the impact of awareness of the stimuli on these local and global effects, we used 2 manipulations of attention. In Experiment 2, 11 other normal volunteers were instructed to engage in mind-wandering while the sounds were played. The ERP local effect was essentially identical to the one observed in Experiment 1 (see right middle of Fig. 4). In sharp contrast, the global effect decreased dramatically, and no significant effect was observed beyond 400 ms. In Experiment 3, 10 other normal volunteers were asked to detect a visual target in a rapid stream of successive letters, and to neglect the auditory stimuli. Again, the ERP local effect was extremely similar to the 2 preceding experiments (see right lower of Fig. 4). Crucially, violation of the global regularity did not elicit any measurable ERP effect. The disappearance of the global effect coincided with the absence of subjective awareness of the global structure of the task: although each of subjects from the first group (active counting task) reported the presence of locally standard and globally deviant stimuli (AAAAA as rare stimuli), only 3 subjects noticed their presence in the passive group, and none in the visual interference group. In this interference group only 1 subject reported that 1 type of trial was more frequent than any other during each block. Only 1 other subject reported that all blocks began with a series of identical trials. Global deviant trials were never repeated, and reappeared randomly after 2 to 5 global standard trials. A single subject from the interference group could report the existence of this pseudorhythm, whereas it was reported by all subjects belonging to the counting group. A regularity awareness score (RAS) was calculated on these 4 items (see SI Appendix) and rated on average at 4/4 in Experiment 1, 2.3/4 in Experiment 2, and 0.4/4 in Experiment 3. Those 3 experiments thus suggest that an ERP signature of global violations is only observed when subjects are conscious of the global regularity structure and of its violations.

Fig. 4.

Local and global effects in individual subjects. (Left) Each horizontal line summarizes an individual subject t test statistics (left for the local effect; right for the global effect). Individual statistics are plotted for 10 subjects from each of the Counting (top), Mind-wandering (middle) and Visual interference (bottom) groups. (Right) For each group, we computed a mean individual statistical index by defining linear bins of individual t test statistical significance (0 for P > 0.05; 1 for P < 0.05; 2 for P < 0.01; 3 for P < 0.005; 4 for P < 0.001; 5 for P < 0.0005; and 6 for P < 0.0001).

Activation of a Global Workspace Network during Global Effect.

We then determined which brain areas are activated during the global and local effects (see Fig. 3). In Experiment 4, 9 other normal volunteers were engaged in the active counting task while brain activity was measured using fMRI. Local violation activated the bilateral superior temporal gyri (STG) including primary auditory cortices (see Fig. 3A), as observed in previous studies of the fMRI correlates of the auditory MMN (22, 23). In contrast with this anatomically localized pattern, fMRI revealed the activation of a brain-scale distributed network during global violation, including bilateral dorso-lateral prefrontal, anterior cingulate, parietal, temporal and even occipital areas (see Fig. 3A).

Fig. 3.

Brain dynamics of local and global effects. (A) Brain fMRI activations are shown for the local (top left) and the global (top right) effects in Talairach's space (horizontal and sagittal slices). For each anatomical view, 1 intracerebral electrode is displayed (yellow disk). (B) Averaged LFPs (top) and LFPs power (bottom) are plotted against time for local standard (green) and local deviant (red) trials (left pair), and for global standard (green) and global deviant (red) trials (two right pairs). (C) Source activity averaged across the Broadman area including the corresponding intracerebral electrode (BrainStorm software, Matlab) is plotted against time.

Intracerebral local field potentials (LFPs) were recorded in 2 epileptic patients implanted for the presurgical mapping of epileptogenic networks. These 2 patients were recorded during the active counting version of the task. Of special interest here, 1 patient had 24 recording sites mostly located in the lateral and mesial parts of the left temporal lobe, 15 of which (62.5%) showed a local effect peaking ≈100–220 ms (see Fig. 3B). All 15 electrodes were located in the lateral part of the temporal cortex and most of them within the STG [Broadman Area (BA) 22], in agreement with previous reports of MMN generators (4). Five recording sites (20.8%) showed a global effect in a later temporal window, as observed with scalp ERPs. The second patient had 29 recording sites, mostly implanted in the frontal lobe, 8 (28%) of which showed a significant global effect within a window ranging from 250–600 ms. In particular, LFPs revealed activations of anterior cingulate (BA 32) and dorso-lateral prefrontal (BA 44) cortices during the global effect (see Fig. 3B). Interestingly we also observed 3 frontal electrodes showing an early component of the local effect peaking ≈120 ms after sound onset, concurrent to the scalp recorded MMN. This result strengthens the debated proposal of a frontal generator of the MMN, in addition to temporal lobe generators (24, 25).

To assess the congruence of fMRI and LFP measurements, we normalized the patients' brain anatomy in Talairach's space. A strong convergence was observed between fMRI maps, intracerebral recordings and source reconstruction of scalp ERP effects (see Fig. 3 and SOM Table S1). The local effect essentially fitted with the MMN network in auditory cortex, whereas the global effect was subtended by the activity of a global workspace network, particularly involving prefrontal cortex, and previously associated with conscious processing (26).

Local and Global Effects Are Detectable in Individual Subjects.

We then ran individual analyses of scalp ERP data, to assess the power of our method to detect a global effect in single subjects. For each subject, a t test based statistics was computed for each time frame. A local effect was observed in 32/32 (100%) of subjects. In Fig. 4, each of 30 subjects from experiments 1–3 is represented as 2 horizontal lines summarizing respectively the time courses of the local and global effects. We compared the distributions of individual statistics of the local and global effects across the 3 conditions (active, passive, interference) by defining 6 linear bins of significance (from 0 for P > 0.05 to 6 for P < 0.0001). Bonferroni corrected F tests were performed for each time frame. No difference was observed for the local effect between the 3 groups (see mean values on Fig. 4, right). Concerning the global effect, it was detected in all subjects belonging to the active counting group, whereas 6/11 subjects showed an early global effect in the mind-wandering group (Experiment 2), and only 3/10 subjects showed weak effects in the visual interference group (Experiment 3).

Global Effect Can Be Observed in Some Minimally Conscious Patients.

Given that our method detects the presence of a global effect in 100% of subjects performing the active counting task, we used this active task to diagnose conscious processing in 8 noncommunicating patients suffering from disorders of consciousness. Based on the clinical criteria defined by the Aspen group (27), 4 patients were in a minimally conscious state (MCS) and 4 patients were in a vegetative state (VS) (see Table S2 for clinical details). Among the 4 VS patients group, 3 of them had a local effect, but none of them showed a global effect. By contrast, all MCS patients showed a significant local effect and 3 of them demonstrated a clear global effect (see Fig. 5). Interestingly, these 3 patients evolved to a fully conscious state during the following weeks, whereas the MCS patient without a global effect remained in an MCS state by the time of this publication. This last MCS patient without a global effect showed a late (>600 ms) ERP response to the local violations, as observed in 1 normal volunteer from the mind-wandering group and 3 volunteers from the visual interference group. This may suggest that this MCS patient processed consciously the local deviant trials, yet without being able to detect the existence of a global regularity. Fluctuating arousal may have prevented him from actively maintaining task instructions. Two additional MCS patients also showed such a late local effect, which may therefore index partial consciousness together with impaired cognitive abilities.

Fig. 5.

Local and global effects in noncommunicating patients. Individual statistics are plotted for the 8 noncommunicating patients VS or MCS patients. (Left) Each horizontal line summarizes an individual subject statistics. (Right) Averaged high-density ERPs of the local (top) and global (bottom) effects of patient MCS 1. Voltage scalp topographies are shown for the MMN ≈200 ms (top left), for the local effect vertex-positivity ≈300 ms (top right), and for the global effect (bottom).

Discussion

In this study, we designed an auditory paradigm in which 2 embedded levels of auditory regularity are defined, respectively at a local (within trial) and at a global (across trials) time scale.

Auditory Cortex Subtends the Automatic and Nonconscious “Local Effect.”

Violation of the local regularity elicited 2 major ERP effects within an early 130–300 ms temporal window: first a typical MMN, followed by a central positivity with simultaneously bilateral occipito-temporal negativities. These 2 effects are highly suggestive of an automatic, nonconscious, and encapsulated mode of processing. Indeed, these local effects remained largely unchanged whether subjects had to count the number of global violations (experiment 1), to mind-wander (experiment 2), or to engage their attention in a competing RSVP task (experiment 3). Moreover, combination of fMRI maps, ERP source estimations, and intracranial recordings demonstrated that these local effects were not only local in time, but also in space: they mostly originated from a restricted anatomical network, the epicenters of which are bilateral superior temporal auditory cortices, and at a smaller extent, probable frontal generators. Finally, our finding that these local effects could still be observed in some patients lacking behavioral evidence of conscious processing (VS patients) strengthens their automaticity. This subset of results strongly suggests that the existence of an early ERP local effect in a noncommunicating patient reflects the preserved nonconscious integration of auditory environment, as previously observed for the MMN in more basic paradigms (13, 15, 16, 27), but need not imply conscious perception. Interestingly however, 3 MCS patients, including the patient without significant ERP global effect, showed local effects with unusually late components. These late local effects, less frequently observed in the control subjects of our 3 experiments, may index the conscious processing of local regularities. Indeed, MCS patients are probably more prone to miss the global regularity of the auditory stimuli, because of fluctuations in their level of arousal and to difficulties in decoding and actively maintaining task instructions. As a consequence, they may be more attracted than controls to consciously process violations of the local regularities. If correct, this hypothesis would suggest that the late component (> 400 ms) of the ERP local effect could index conscious processing of local violations. One test of this hypothesis might consist in recording normal controls while they perform an active task focusing on violation of local regularities.

A Global Workspace Network Subtends the Conscious “Global Effect.”

Taken together, our results strongly suggest that the reaction of the brain to violations of global regularities can serve as a marker of conscious processing of the auditory environment. However, being merely awake and aware is not enough to elicit a global effect. In conscious controls, a global effect was present only when subjects were conscious of the global regularity violations. When we interfered with this specific conscious content by mind-wandering instructions or a visual interference task, the global effect vanished in parallel with conscious reportability of the global regularity. Anatomically, brain regions activated during the global effect spanned over a brain-scale cortical network including prefrontal, cingulate, parietal and temporal regions. This network fits nicely with our previous theoretical proposal that conscious processing is subtended by the coherent activity of a global workspace network (26, 28), and not by the transient and isolated activation of any cortical region, including prefrontal cortex, as recently observed (29–32). Additionally, the absence of a global effect in the VS patients indicates that our test is not a simple measure of vigilance, but a measure of subjective conscious contents.

A Potentially Specific Clinical Tool.

Our converging set of ERP, fMRI, and intracerebral recordings demonstrates that the local-global paradigm constitutes a promising clinical tool to diagnose conscious processing. Even a single vertex electrode, placed at bedside, followed by a few minutes of auditory testing, might suffice to identify conscious patients who are aware of the global auditory regularities. When it is observed, this global effect seems to be a specific diagnostic marker of consciousness. Note however, that the reverse is not true: the absence of a global effect does not exclude the possibility of conscious processing, given that it is absent in distracted or mind-wandering conscious controls who could not report the global rule. One should keep in mind that a patient may still be conscious but unable to understand the instructions, to actively maintain attention, or to deploy working memory processes necessary to perform the task. In such cases, we tentatively propose that a late local effect may still be suggestive of conscious processing of the local variations in a subject who could not extract the global auditory rule. When the global effect is absent, and only an early and transient local effect is observed, interpretation must be cautious. This pattern may signal nonconscious processing, as previously observed in comatose patients (13–15) and in vegetative state patients (16, 17), but also a patient who is aware but not attending. Indeed, all functional measures of conscious processing are also subject to variations of arousal, attention, and task performance (33). Furthermore, there is no clear consensus concerning the definition of consciousness, and other theoretical models may propose that a form of “phenomenal consciousness” (34), or of “unattended consciousness” (35, 36) still accompanies the early local effect.

Overall, our study confirms the relevance of using active tasks even in noncommunicating patients, to probe their cognitive abilities with neurophysiological methods (28–30). One may consider it urgent to integrate some of these measures into the clinical assessment of the conscious state in noncommunicating patients in whom a mere behavioral assessment of cognitive abilities is of limited power.

Materials and Methods

Subject and Patients.

Experiments were approved by the Ethical Committee of the Salpêtrière hospital. The 41 normal controls (mean age = 27.0 ± 3.0; sex-ratio = 0.9) tested (32 with scalp ERPs + 9 with fMRI), and the 2 epileptic patients gave written informed consent. Three scalp ERP subjects were not included in the analysis because of excessive movement artefacts. Concerning the 8 noncommunicating patients (see Table 2), scalp ERP recordings were done after families gave informed consent. In addition to clinical examination, we used the French version (established by Laureys in 2004) of the revised Coma Recovery Scale (CRS-R) by Kalmar and Giacino (37). Note also that the local effect had a direct clinical impact by probing the presence of a MMN. This bedside neurophysiological test is a routine exploration with both a functional diagnosis and an outcome prognosis values (15, 27).

Auditory Stimulations.

Series of 5 complex 50-ms-duration sounds were presented via headphones with an intensity of 70 dB and 150 ms SOA between sounds. Each sound was composed of 3 sinusoidal tones (either 350, 700, and 1400 Hz, hereafter sound A; or 500 Hz, 1000 Hz, and 2000 Hz, hereafter sound B). All tones were prepared with 7-ms rise and 7-ms fall times. Four different series of sounds were used, the first 2 using the same 5 sounds (AAAAA or BBBBB); and the second with the final sound swapped (either AAAAB or BBBBA). Series of sounds were separated by a variable interval of 1350 to 1650 ms (50-ms steps). All subjects heard 8 blocks (3–4 min duration), in randomized order for each subject (each of the 4 possible block types was presented twice). The blocks were designed to contain the sound series with a different sound in the end, either as an infrequent stimulus (block type a: 80% AAAAA/20% AAAAB; block type b: 80% BBBBB/20% BBBBA); or as a frequent stimulus (block type c: 80% AAAAB/20% AAAAA; block type d: 80% BBBBA/20% BBBBB). All block types presented a local regularity (the fifth sound could be different or identical to previous sounds) and a global regularity (one of the series of sounds was less frequent than the other). Each block started with 20–30 frequent series of sounds to establish the global regularity before the first infrequent stimulus arrival. In each block the number of infrequent trials varied between 22 and 30. For the ‘mind-wandering’ condition, subjects were instructed as follows: “During the next 30 min different sounds will be played by the headphones for successive periods of approximately 3 minutes. You don't need to pay attention to these sounds. Please close your eyes, and let yourself mind-wander.”

The interference group (experiment 3) performed a continuous letter detection task during each block of auditory stimuli. The visual task began 5–10 seconds before the onset of the series of sounds and finished 5–10 seconds later. The visual stimuli were letters appearing at the center of the screen. The visual stimuli were cyan, dark yellow, black, magenta, or red characters rendered on a light gray background. There were 12 different letters and, thus, 24 possible stimuli (majuscule or minuscule letters) in each block. The uppercase letters A, D, T, and X were used twice each as targets, for a total of 8 blocks. The letters subtended 1° of visual angle horizontally and vertically. Subjects had to press the spacebar to targets in each block as fast as they could. The maximum time for presentation and response was 1,000 ms. After each button press the response time appeared on screen as feedback. No causal relation prevailed between visual and auditory stimuli. All stimuli were presented using Eprime v1.1 (Psychology Software Tools Inc.).

For the fMRI experiments, the same experimental set-up (structure and timing) was used, with an additional silent baseline period (19.2 seconds) at the beginning and at the end of each of the 8 blocks.

High-Density Scalp ERPs.

ERPs were sampled at 250 Hz with a 128-electrode geodesic sensor net (EGI) referenced to the vertex (for experiment 3 and for the 8 patients, a 256-electrode geodesic sensor net was used). We rejected voltages exceeding ± 200 μV, transients exceeding ± 100 μV, or electro-oculogram activity exceeding ± 70 μV. Trials were then segmented from −200 ms to + 1300 ms relative to the onset of the first sound. Bad channels were interpolated. Trials with >25 bad channels were rejected. The remaining trials were averaged in synchrony with stimulus onset, digitally transformed to an average reference, band-pass filtered (0.5–20 Hz) and corrected for baseline over a 200-ms window before stimulus onset. For the 8 patients recorded in the noisy environment of the intensive care unit, we used a baseline correction over the 800-ms window before stimulus onset. All those processing stages were performed in the EGI Waveform Tools Package. Matlab 7.0 scripts were used to compute sample-by-sample paired t tests with a triple criterion of: P < 0.01 for at least 10 consecutive samples on 10 electrodes.

For individual subject statistics, unpaired t tests were calculated for each time sample on individual trials. An effect was considered significant if it satisfied a triple-threshold: t test P value <0.01 on a minimum of 5 consecutive samples, on a minimum of 10 electrodes (20 for the 256 channels net). To further assess the reliability of our test, we visualized for each time-sample the significance of the local and global effects using a 5-levels P value color scale (see Fig. 4), ranging from 0.01, 0.005, 0.001, 0.0005, and 0.0001.

For source estimation, cortical current source density mapping was obtained using a distributed model consisting in 10,000 current dipoles. Dipole locations and orientations were constrained to the cortical mantle of a generic brain model built from the standard brain of the Montreal Neurological Institute using the BrainSuite software package. This head model was then warped to the standard geometry of the sensor net. The warping procedure and all subsequent source analysis, and statistical estimation of the Z-scores relative to the baseline (−200 ms to + 600 ms) were processed with the BrainStorm software package (http://neurimage.usc.edu/brainstorm). EEG forward modeling was computed with an extension to EEG of the overlapping-spheres analytical model. Cortical current maps were computed from the EEG time series using a linear inverse estimator [weighted minimum-norm current estimate or WMNE, see (38) for review].

Local Field Potentials and fMRI Methods.

See SI Text.