Mismatch negativity (MMN) elicited by abstract regularity violations in two concurrent auditory streams

Abstract

The study investigated whether violations of abstract regularities in two parallel auditory stimulus streams can elicit the MMN (mismatch negativity) event-related potential. Tone pairs from a low (220–392 Hz) and a high (1319–2349 Hz) stream were delivered in an alternating order either at a fast or a slow pace. With the slow pace, the pairs were perceptually heard as a single stream obeying an alternating low pair-high pair pattern, whereas with the fast pace, an experience of two separate auditory streams, low and high, emerged. Both streams contained standard and deviant pairs. The standard pairs were either in both streams ascending in the direction of the within-pair pitch change or in the one stream ascending and in the other stream descending. The direction of the deviant pairs was opposite to that of the same-stream standard pairs. The participant's task was either to ignore the auditory stimuli or to detect the deviant pairs in the designated stream. The deviant pairs elicited an MMN both when the directions of the standard pairs in the two streams were the same or when they were opposite. The MMN was present irrespective of the pace of stimulation. The results indicate that the preattentive brain mechanisms, reflected by the MMN, can extract abstract regularities from two concurrent streams even when the regularities are opposite in the two streams, and independently of whether there perceptually exists only one stimulus stream or two segregated streams. These results demonstrate the brain's remarkable ability to model various regularities embedded in the auditory environment and update the models when the regularities are violated. The observed phenomena can be related to several aspects of auditory information processing, e.g., music and speech perception and different forms of attention.

1. Introduction

The human auditory system has a remarkable capability to segregate even simultaneous auditory sources to clearly separate auditory “objects” (Carlyon, 2004; Ciocca, 2008; Darwin, 1997), although the exact underlying neural mechanisms are still disputed (Shamma and Micheyl, 2010; Snyder and Alain, 2007; Winkler et al., 2012). For example, while walking on a crowded street, the incoming auditory stimuli are easily grouped to voices of different speakers and to various natural and non-natural sounds (e.g., dog barks, car horns). Moreover, if there occurs a change in some sound source which has remained constant for a while, our attention is often automatically diverted to these potentially significant changes: a car driver's discussion with the co-passenger is interrupted when a sudden strange sound appears among the even hum of the motor. The present study addresses the brain mechanisms of such phenomena.

Some well-known factors determining whether two concurrent auditory stimulus sources are segregated into separate streams or perceived as forming a single auditory object are the pitch difference between the sources and the timing of the stimuli (see, e.g., Darwin, 1997). In the simplest form, if two tones appear in an alternating order (ABABAB…) with rather long inter-stimulus intervals (ISIs) and the two tones are close in pitch, the tones are usually perceived as a single auditory stream, following an up-down-up-down… pattern. In contrast, if the ISI is short (e.g., <300 ms) and the pitch separation large, two separate auditory streams, high and low, are typically perceived.

The mismatch-negativity (MMN) component of the event-related brain potential (ERP) has been extensively used as an index of automatic auditory information processing (for reviews, see Näätänen et al., 2007; Winkler, 2007). MMN is elicited by violations in the regular aspects of the auditory stimulation even when the participant's attention is directed elsewhere from the auditory stimuli (e.g., on watching a video). In the basic “oddball paradigm”, the participant is presented at short intervals with physically constant “standard” stimuli, which are infrequently replaced by “deviant” stimuli (e.g., tones of a different pitch). The deviant stimuli elicit MMN, which is seen in the ERPs as a frontocentral negativity, generated at the auditory and frontal cortices, and peaking 150–200 ms after the onset of the deviance (Näätänen et al., 2007). According to Winkler's (2007) predictive model theory of MMN, the brain automatically extracts and encodes rules of the various regularities embedded in the auditory environment. On the basis of these regularity representations (models), the brain generates predictions of what kind of sounds are likely to be encountered in the near future. In case the prediction fails, e.g., a deviant stimulus is presented, the models are updated, reflected in the elicitation of MMN. It has also been proposed that one of the main functions of the MMN mechanism is to automatically monitor the auditory environment and to involuntarily switch the person's attention to potentially important changes in the auditory stimuli (Escera and Corral, 2007). MMN has also been used to study the processes related to stream segregation and regularity extraction, as it provides an index of rather early, preattentive analysis of auditory features in the brain (for a recent theoretical model, see Schröger et al., 2014). For example, it has been demonstrated that when alternating high and low tones, presented at a fast pace, are segregated into two streams, an MMN is elicited by the deviant events occurring both in the high-tone and low-tone streams (Sussman et al., 1998). However, the MMN can be elicited even with paces so slow that the perceptual segregation does not automatically occur. This has been demonstrated in conditions where the high and low series do not overlap in pitch range (Shinozaki et al., 2000) or when voluntary attention is used to enhance the perceptual segregation (Sussman et al., 1998).

The preattentive auditory analysis reflected by the MMN is not restricted only to basic physical stimulus features (e.g., pitch, intensity, spatial location). It includes also more complex, "abstract" regularities based, for example, on the relationships between various physical features (for a review, see Paavilainen, 2013). In a pioneering study, Saarinen et al. (1992) presented their participants with series of tone pairs (two 60-ms tones separated by a 40-ms silent gap; silent inter-pair interval 640 ms). The position of the tone pairs in the pitch dimension varied randomly. Thus, there was no physically identical, repetitive standard stimulus. Instead, the invariant feature of the standard pairs was the direction of the pitch change: the standard pairs were ascending (i.e., the second tone of each pair was higher in pitch than the first tone), whereas the deviant pairs were descending. An MMN was elicited by the deviant pairs in an ignore condition, suggesting that the auditory system preattentively derived an “abstract” invariant feature (“rise in pitch”) from a set of individual varying physical events.

The previous MMN studies related to stream segregation (e.g., Shinozaki et al., 2000; Sussman et al., 1998, 1999, 2001; Yabe et al., 2001) have mainly used simple physical deviants to elicit the MMN. However, in real life the auditory stimuli and the regularities embedded in them are often far more complex. The streams do not necessarily consist of physically identical stimuli. Instead, they may involve continuous variation over wide ranges in different features, but nevertheless the streams can be segregated, as in music or speech processing. In the present study, we wished to determine, by using Saarinen et al.'s (1992) paradigm, whether deviations in the directions of within-pair pitch changes can elicit MMN even when they occur in two parallel stimulus streams. To our knowledge, only one previous study has addressed this question. Paavilainen et al. (1995) applied Saarinen et al.'s (1992) paradigm under more demanding conditions with regard to information load. They presented tone pairs to left and right ears in random order, the standard pairs being in the left ear ascending and in the right ear descending in pitch. In order to make the two stimulus classes more discernible from each other, the left- and right-ear stimuli were also located on different pitch areas (left: 523–932 Hz, right: 1319–2349 Hz). In an ignore condition, the deviant pairs in both ears (reversed in the direction of the pitch change) elicited an MMN. The result was interpreted as indicating that the brain was able to extract abstract attributes from a rapid dichotic stimulation separately for the two ears, and even when the attributes were opposite between the ears.

However, an alternative interpretation for their results is possible: as the input from the left and right ear is transmitted predominantly to the contralateral auditory cortices (see, e.g., Hiscock and Kinsbourne, 2011), it could be argued that in the experiment two separate and opposite regularity representations were formed, i.e., one on the left cortex for the right-ear regularity and the other on the right cortex for the left-ear regularity. In order to rule out this explanation, Paavilainen et al.'s (1995) setup was modified in the present study so that there were no spatial location cues for the two stimulus classes: all tone pairs were delivered binaurally. By using pitch differences, we exploited the auditory stream segregation phenomenon to divide the tone pairs perceptually into two classes (“high-stream stimuli”, “low-stream stimuli”). The research questions were as follows:

• 1)Can the preattentive brain mechanisms, reflected by the MMN, extract abstract regularities from the high and low stimulus streams even when the regularities are opposite between the two streams, e.g., high-stream standards are ascending and low-stream standards descending?
• 2)Is the regularity extraction dependent on whether there perceptually exists two segregated or only one stimulus stream? Our intention was to manipulate the participants' perceptual experience by presenting the tone pairs either at a fast (two streams) or at a slow pace (single stream).
• 3)Does strong focusing of attention on either stream affect the MMN, i.e., how “automatic” the regularity extraction mechanisms are? Several studies have reported attention effects on the MMN when rapid binaural stimulation and strong focusing of attention on the basis of a spatial cue (i.e., attend right-ear stimuli vs. attend left ear stimuli) has been used (e.g., Woldorff et al., 1991; Näätänen et al., 1993; Paavilainen et al., 1995). In the present study, we wished to find out whether attention effects could be found when instead of a spatial cue only a pitch cue is separating the attended and unattended “channels” from each other.

2. Methods

2.1. Participants

Fourteen university students (7 males, 7 females; age range 19–52 yrs., mean 28 yrs.; four left-handed) participated in the experiment. All the participants reported having normal hearing and no diagnosed neurological disorders. Three were active players of some musical instrument. Informed consent was obtained from all individual participants included in the study. The experimental procedure was carried out in accordance with the Declaration of Helsinki and was accepted by the University of Helsinki Ethical Board for Behavioural and Human Sciences.

2.2. Stimuli and procedure

The stimuli were tone pairs consisting of two 50-ms sinusoidal tones with 5-ms rise and fall times, separated by a 30-ms silent gap. The pairs were located either on a low (220–392 Hz; “low stream”) or on a high frequency area (1319–2349 Hz; “high stream”). The intra-pair interval between the two tones was always one full step on a musical scale so that in both streams, there were five types of ascending and five types of descending pairs (see Table 1; the descending pairs were reversals of the ascending ones). Pairs from low and high stream were delivered in an alternating order either at a fast (interpair interval 70 ms) or slow pace (270 ms). With the slow pace, the pairs were supposed to be perceived as a single stream obeying an alternating pattern (low pair - high pair - low pair - high pair…), whereas with the fast pace, rather an experience of two separate auditory streams, low and high, emerged. These interpair intervals were chosen on the basis of pilot studies where the five experimenters were listening to the stimuli in order to find optimal parameters producing the single-stream and two-stream perceptions. The auditory stimuli were delivered binaurally via earphones at a comfortable loudness level.

Table 1.

The frequencies (in Hz) of the tones forming the pairs in the low and high streams. Depending on the condition, the order of the tones was varied so that the within-pair pitch change was either ascending or descending. The deviant pairs were always reversals of the corresponding standard pairs.

Low stream:	High stream:
220–247	1319–1480
247–277	1480–1661
277–311	1661–1865
311–349	1865–2093
349–392	2093–2349

The experiment consisted of four different conditions. It was randomized whether each stimulus block started with a low-stream or with a high-stream pair. The conditions were always presented in the following order (for a schematic illustration, see Fig. 1):

• 1)Same direction, fast pace (SF): the standard pairs in both streams were ascending and the deviant pairs descending. On each trial, one of the ten possible tone pairs (Table 1), alternatingly from the low and high stream, was randomly presented so that the probability of each standard pair was .18 and the probability of each deviant pair .02. Thus, the total probability of the standard pairs was .90 and that of the deviant pairs .10. There was always at least one standard pair between two deviant pairs. The interpair interval was 70 ms. The stimulus block consisted of 3000 tone pairs, lasting 10 minutes. During the stimulus presentation, the participant's task was to attend a muted video movie of his/hers own choice.
• 2)Different direction, fast pace (DF): Other parameters were the same as in SF except that now in the low stream, the standard pairs were ascending and the deviant pairs descending and in the high stream, the directions were vice versa.
• 3)Different direction, slow pace (DS): The parameters were the same as in DF except that now the interpair interval was 270 ms and the pairs were presented in two blocks of 1500 pairs. There was a short (1–2 min) pause between the blocks.
• 4)Different direction, fast pace, attend (DFA): The parameters were the same as in DF except that instead of watching the video, the participant's task was to attend either the low (half of the participants) or the high stream (the other half of the participants) and press a button whenever they detected a deviant pair in the attended stream. The pairs were presented in two blocks of 1500 pairs. There was a short (1–2 min) pause between the blocks. Before the condition, the participants were explained the general features of the auditory stimuli and told which kind of deviants their task was to detect. They were also allowed to shortly practice their task before the condition began.

Fig. 1.

A schematic illustration of the stimuli used in the study. Tone pairs (either ascending or descending in within-pair pitch change) were presented in an alternating order from the high and low streams (HP = high pair, LP = low pair). Within both streams, the position of the pairs varied randomly over a wide pitch range. A pitch area where no pairs were presented separated the high and low streams from each other (dashed rectangle). (A) In the Same-direction condition, the standard pairs (white) were ascending both in the high and low streams and the occasional deviant pairs (black) descending. (B) In the Different-direction conditions, the low-stream standard pairs were ascending and the deviant pairs descending whereas in the high stream the directions were reversed.

2.3. ERP recording and analysis

The EEG (sampling rate 250 Hz, bandpass 0.1–40 Hz) was recorded with Ag/AgCl-electrodes placed at Fpz, Fz, F3, F4, Cz, Pz and at the left (LM) and right (RM) mastoids. As the MMN is known to have a frontal scalp distribution (e.g., Näätänen et al., 2007), most electrodes were placed over the frontal areas and the frontal-line electrodes (F3, Fz, F4) were also selected to be used for the statistical analysis of MMN. The vertical eye movements were recorded with an electrode above the right eye and horizontal eye movements with an electrode at the outer canthus of the right eye. The reference electrode was attached on the tip of the nose and the grounding electrode on the forehead.

The EEG was cut to 400-ms epochs starting at the onset of the first tone of a pair and averaged separately for the standard and deviant pairs in each condition. The epochs containing EEG changes over ±100 μV were omitted from the averaging. The 50-ms period preceding the onset of the second tone of a pair (i.e., 30–80 ms) served as the baseline (as the onset of the second tone was the earliest moment to indicate whether the pair was a standard or a deviant). The grand-average ERPs were calculated by averaging together the corresponding ERPs from the 14 participants. As half of the participants in the DFA condition attended low stream and half the high stream, the attended low- and high-stream ERPs were averaged together in the DFA grand-averages. The same was done for the unattended low- and high-stream ERPs.

3. Results

Fig. 2 shows the grand-average ERPs at Fz. In SF and DF conditions (panels A and B), the MMN is seen in the deviant-pair ERPs both in the low and high streams as a late negative enhancement relative to the standard-pair ERPs. The corresponding deviant pair minus standard pair difference waves are presented in Fig. 3. The mean ERP amplitudes at F3, Fz, and F4 during 150–300 ms were measured and analysed with a four-way repeated-measures ANOVA (directions: electrode [F3, Fz, F4], condition [SF, DF], stream [high, low], stimulus type [standard, deviant]). A statistically significant effect of stimulus type was obtained (F(1,13) = 7.31, p = .018), confirming the presence of MMN. Condition X stimulus type and stream X stimulus type interactions were nonsignificant. Thus, the data revealed no statistically significant MMN amplitude differences between the SF and DF conditions or between the high and low streams (however, one must interpret these results with caution as they might also result from too low statistical power with 14 participants).

Fig. 2.

The grand-average ERPs at Fz in the different conditions for the standard (thin lines) and deviant (thick lines) pairs. A–C: The ERPs for the high-stream and low-stream stimuli in the ignore conditions. D: The ERPs from the attend condition for the attended-stream and unattended stream stimuli. The black rectangles indicate the timing of the stimulus pair (50-ms tone, 30-ms gap, 50-ms tone). The MMN is the late negative enhancement in the deviant-pair ERPs compared to the standard-pair ERPs.

Fig. 3.

The grand-average difference waves (deviant pair minus standard pair) at F3, Fz and F4 in the different conditions. The black rectangles indicate the timing of the stimulus pair.

Also in the DS condition an MMN is seen in the ERPs (Fig. 2C), and in a three-way repeated-measures ANOVA (mean amplitude 180–250 ms; directions: electrode [F3, Fz, F4], stream [high, low], stimulus type [standard, deviant]) the effect of stimulus type was significant (F (1,13) = 4.83, p = 0.047)1. Fig. 2D shows the attended-stream and unattended-stream ERPs. Again, a small MMN seems to be present in both cases, although here the effect of stimulus type did not reach significance (mean amplitude 180–240 ms; F(1,13) = 1.84, p = n.s.). However, it should be noted that in all the eight instances presented in Fig. 2, a similar late negative enhancement during 150–250 ms seen in the deviant-pair ERPs (relative to the standard-pair ERPs) as predicted on the basis of MMN theory (sign test, one-tailed p = 0.0039). This consistency across conditions and streams gives further support for the presence of MMN in the present data.

4. Discussion

The presence of the MMN in the SF condition was expected, as there are plenty of previous data demonstrating that deviations in the within-pair pitch change elicit the MMN (e.g., Paavilainen et al., 1998, 1999). However, it is interesting that the MMN was, on the basis of Fig. 2, elicited also in the DF condition: in this condition, the directions of the standard and deviant pairs were opposite in the high and low streams and of all the pairs, 50 % were descending and 50 % ascending. Consequently, there were no standard and deviant events on the global-probability level. In order to generate the observed MMNs, the brain obviously must have somehow managed to segregate the stimuli into two streams and extract the opposing regularities, the one present in the high and the other in the low stream. Contrary to Paavilainen et al. (1995) study, there were no spatial cues for the two streams as all the pairs were presented binaurally. This fact rules out the MMN explanation based on two separate regularity representations in the left and right auditory cortices, discussed in the Introduction. Hence, on the basis of the DF condition results, the stream segregation and abstract regularity extraction seem to operate already on the relatively early processing level, reflected by the MMN.

Interestingly, an MMN was elicited also in the DS condition where the tone pairs were presented at a pace so slow that at the perceptual level, the participants should have no more experienced two separate streams. Instead, rather a single stream obeying a pattern of alternating high and low pairs should have been heard. However, here one must assume, of course, that the participants perceived the tones similarly as the experimenters in the pilot studies (see Methods). In that case the result would suggest a dissociation between the brain mechanisms leading to conscious perception and the preattentive regularity extraction mechanisms, reflected by the MMN: the MMN mechanism extracts the regularities specific to high and low streams even in conditions where the streams are no more perceptually segregated. Similar findings have been reported previously with basic physical regularities (Shinozaki et al., 2000). Possibly, the higher brain mechanisms can in some circumstances interfere with the initial auditory scene analysis based on the early preattentive regularity representations, leading to different final perception.

However, it should be noted that in the present study (as in stream segregation studies in general, too), there may have been inter-individual differences in the way how the participants perceived the tones. In the previous MMN studies on stream segregation, physically constant standard stimuli have been used. The present study was the first one to exploit an abstract-feature MMN paradigm where the tone pairs in the high and low streams randomly varied over large pitch areas (see Fig. 1), and the feature separating the standard and deviant pairs was a higher-order one (i.e., the direction of the pair). It is possible that as a result of the random variation in the pitch of the pairs, the perception of stream segregation may have, at least in some participants, been weaker than when physically constant streams are used. The relatively small MMN amplitudes, observed in the present study, may also be related to the use of the abstract-feature paradigm.

The DFA condition was included to study the effects of selective attention on the MMN generator mechanism. The participants' attention was directed either to the high or low stream and their task was to detect the deviant pairs in that stream. Both in the attended and unattended streams a small MMN, quite equal in amplitude, appeared to be present (Fig. 2). Thus, directing attention to one of the stimulus streams did not, at least, appear to enhance the MMN to deviants in that stream or, alternatively, supress MMNs in the unattended stream, as has been previously reported (see, e.g., Woldorff et al., 1991; Näätänen et al., 1993; Paavilainen et al., 1995). However, as no statistically significant stimulus effect was obtained in the DFA condition, these results must be considered as tentative only.

The lack of clear attention effects in the DFA condition may also be related to the fact that all the participants reported after the DFA condition that they found the detection task very difficult, or even impossible to perform. This was obviously due to the very fast stimulation rate and continuous random variation in the tone pairs (inherent in the abstract-feature paradigm) which effectively prevented the deviant pairs from “popping out” amongst the standard pairs. Consequently, it could even be questioned whether the present setup was optimal for studying the effects of selective attention on the MMN: it may have been difficult for the participants to keep their attention continuously focused on the designated stream when there were no salient “targets” to be found in that stream. This may have led to a lack of proper attentional set and also contributed to the absence of clear attention effects in the DFA condition. One possible method to overcome this problem in future abstract-feature studies could be to use otherwise similar setup as in the present DFA condition but assign a small random subset of the standard pairs (ca. 10–20 %) in the designated stream as “targets” by presenting them, for example, on a clearly softer intensity than the other stimuli. The participants task would be to press button to these salient “rare soft standards”. This task might ensure that their attention is effectively focused on the designated stream while the real aim would be to compare the MMNs to the deviant-direction pairs between the attended and unattended streams. However, despite of the afore-discussed potential problems in interpreting the present DFA data, the finding that the same deviants that were (at least on the basis of the participants' reports) very difficult to detect even when attended, appeared to elicit an MMN when presented in the ignore (DF) condition, is in line with many previous results suggesting a dissociation between MMN elicitation and conscious deviance detection (e.g., Paavilainen, 2013; Paavilainen et al., 2001, 2007).

In conclusion, the present study provides further evidence for the view that the brain can segregate auditory stimuli to different streams already at the relatively early preattentive processing level, indicated by the MMN, and perform abstract feature analysis and regularity extraction separately for the streams. Moreover, these processes seemed to occur independently of whether the stimulus streams are perceptually segregated or not. The results are a further demonstration of the remarkable ability of the brain to model various complex regularities embedded in the auditory environment (for a review, see Paavilainen, 2013). Consequently, the present results can be interpreted in terms of Winkler's (2007) predictive model account of the MMN (see Introduction). The use of abstract-feature paradigm improves the ecological validity of the results, compared to earlier stream-segregation MMN studies where physically identical standard stimuli, rarely occurring in natural auditory environments, were used to form the streams. Consequently, the observed phenomena may be relevant for understanding the neural basis of several “real-world” functions of auditory information processing, e.g., music and speech perception and different forms of attention.

Declarations

Author contribution statement

Petri Paavilainen, Crista Kaukinen, Oskari Koskinen, Julia Kylmälä, Leila Rehn: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.