Encoding of illusory continuity in primary auditory cortex

SUMMARY

When interfering objects occlude a scene, the visual system restores the occluded information. Similarly, when a sound of interest (a ‘foreground’ sound) is interrupted (occluded) by loud noise, the auditory system restores the occluded information. This process, called auditory induction, can be exploited to create a continuity illusion. When a segment of a foreground sound is deleted, and loud noise fills the missing portion, listeners incorrectly report hearing the foreground continuing through the noise. Here we reveal the neurophysiological underpinnings of illusory continuity in single neuron responses from awake macaque monkeys’ primary auditory cortex (A1). A1 neurons represented the missing segment of occluded tonal foregrounds by responding to discontinuous foregrounds interrupted by intense noise as if they were responding to the complete foregrounds. By comparison, simulated peripheral responses represented only the noise and not the occluded foreground. The results reveal that many A1 single neuron responses closely follow the illusory percept.

INTRODUCTION

In natural environments, a sound of interest (a ‘foreground’ sound) is often obscured by brief interrupting sounds produced by other objects (background sounds). For example, when a monkey attempts to identify another monkey’s vocalization, background bird chirps might interrupt the monkey vocalization. When interrupting background sounds are loud enough to completely obliterate a short underlying foreground segment, the auditory system fills in the occluded segment through a process called auditory induction, so-called because the foreground preceding and following the background sound induces perceptual restoration of the missing foreground segment. If there were no inducing foreground segments, preceding and following the loud noise, the foreground would be imperceptible because of masking (Fig. 1E,F) by the background. Auditory induction is known by other names, such as amodal completion, fill-in or phonemic restoration: (Bregman, 1990; Miller et al., 2001; Petkov et al., 2003; Warren, 1970; Warren, 1972), and is an example of a general ability of the brain to perceptually organize sensory input to fill-in occluded information (Komatsu, 2006; Pessoa and De Weerd, 2003).

Figure 1.

Schematized spectrograms demonstrating stimuli and their relationship to illusory induction and masking. Spectrograms of (A) continuous and (B) discontinuous tones, and interrupting noise centered in (C) continuous and (D) discontinuous tones. High intensity interrupting noise causes perceptual restoration (induction) of the deleted tone segment with the tone being reported as continuous even when it is not. Here both a continuous foreground and noise are perceived. Spectrograms of surrounding noise temporally overlapping entire (E) continuous and (F) discontinuous tones. High intensity surrounding noise masks the tone and only noise is heard.

Auditory induction can be exploited to create an illusion, which was originally demonstrated with speech sounds. When segments were deleted from speech, the result was poor comprehension. However, when the removed segments were filled with loud noise, speech comprehension improved dramatically, providing compelling evidence that the brain restored the deleted information (Warren, 1970). Further studies (Bashford et al., 1988; Warren, 1972; Warren et al., 1988) demonstrated that illusory induction is not speech specific, but rather a general process that occurs with many foregrounds, including tones (Figure 1 A-D illustrates the stimulus configurations used in our study). This illusory induction has also been called the continuity illusion and is conceptually related to visual illusory contours (Day and Kasperczyk, 1983; Kanizsa, 1979), illusory motion (Assad and Maunsell, 1995), and induction (Rossi and Paradiso, 1996). An important requirement for auditory induction is that energy be present at induced frequencies. Thus, induction can be thought of as a process of perceptually organizing and assigning sound energy to various objects, selectively allocating ambiguous energy in to a coherent scene, rather the than the creation of an illusory percept in the absence of sensory stimulation.

Auditory induction has been studied behaviorally in humans (Kluender and Jenison, 1992; Warren, 1970; Warren et al., 1994; Warren, 1972; Wrightson and Warren, 1981), cats (Sugita, 1997), and monkeys (Miller et al., 2001; Petkov et al., 2003). However, the relationship of neuronal activity to induction remains a mystery. Psychophysical studies have provided two principles that guide the search for induction’s neural basis (Bregman, 1990; Bregman and Dannenbring, 1977; Houtgast, 1972). The first called the ‘sufficiency of evidence rule’ states that during the occluding noise some neural activity should be indistinguishable from activity that would have occurred if the tone actually continued through the noise (Bregman, 1990). The second, termed the ‘no discontinuity rule’ (Bregman, 1990), states that there should be no neural evidence of transitions in the foreground (i.e., no evidence of the onset or offset of the foreground sound). This rule is based on experiments showing that induction is reduced or eliminated by placing a discontinuity or transition just prior to occluding-noise onset. For example, no induction occurs if an amplitude ramp is inserted into the foreground just prior to the interrupting noise, even if the ramp is an amplitude increase, which actually strengthens the foreground signal (Bregman and Dannenbring, 1977).

A heuristic model derived from these two rules helps to reveal the required single neuron responses (Fig. 2). First, to support the ’sufficiency of evidence rule’ neurons with sustained firing to the foreground (Fig. 2A-C) should respond during induction as if the foreground were continuous (Fig. 2C). Many auditory neurons have sustained responses to at least one stimulus (Wang et al., 2005). Next to obey the ’no discontinuity rule‘ the model predicts that responses to tone transitions or discontinuities are eliminated by the occluding noise. Many auditory cortical neurons, even in awake preparations, are highly sensitive to amplitude transitions in sounds, commonly yielding phasic responses to tone onsets and offsets (Creutzfeldt et al., 1980; Erulker et al., 1956; Fishbach et al., 2001; Katsuki et al., 1959; Recanzone, 2000). Such phasic responses are well suited for detecting discontinuities in sounds and become more common as one ascends the auditory system (e.g., inferior colliculus, Walton et al., 1997; thalamus, Schreiner, 1980; auditory cortex, Steinschneider et al., 1995; Eggermont, 1999;). To obey the ’no discontinuity rule‘ phasic responders should fail to respond to tone transitions (Fig. 2F and I) during induction. This model leads to a neural representation of induction as follows. When loud noise fills the gap (Fig. 1D), three neural response components behave as if a continuous foreground were present, even though a part of it was deleted. Sustained responders should fire continuously (Fig. 2C) as if there were no pause in the tone. Offset responders must fail to detect the offset of the initial tone segment (Fig. 2F). Third, onset responders should fail to detect the re-introduction of the tone (Fig. 2I).

Figure 2.

Heuristic model of single-neuron response correlates of auditory induction. On the top of each column are schematic spectrograms of the following stimuli: (A) a continuous tone, (B) a discontinuous tone, and (C) a discontinuous tone interrupted by intense noise. The latter (C) causes induction, and to be consistent with perception of a continuous tone during induction, responses in the 3^rd column (C,F,I) should be like those to a continuous tone (A,D,G). Each row shows schematized peri-stimulus time histograms (PSTHs) for AI neurons with sustained (A-C), offset (D-F), and onset (G-I) responses to the three stimuli.

To correctly interpret the neural evidence it is necessary to recognize that induction comprises the percepts of both a continuous foreground and the interrupting noise. This is quite distinct from the percept created by loud masking noise surrounding the foreground in time (Fig. 1E-F), which causes subjects to hear only noise (Bregman, 1990; Kluender and Jenison, 1992; Petkov et al., 2003). Therefore, when loud, interrupting (inducing) noise (Fig. 1C-D) occludes a sound, the brain should respond as if a complete foreground and a noise were present. However, when loud surrounding (masking) noise (Fig. 1E-F) is presented with the foreground sound the brain should respond as if only noise were present. Here we contrast monkey primary auditory cortical responses to illusory induction and masking stimuli to evaluate whether single neurons encode the illusory induced sound features (induction) as opposed to following the physical stimulus attributes (masking). We used stimuli identical to those from a recent psychophysical study demonstrating auditory induction and masking in macaques (Petkov et al., 2003). Results are consistent with the model of Fig. 2 and the perception of both the illusory foreground segment and interrupting noise during induction.

RESULTS

Consistent with the hypothesized model of auditory induction, many A1 neurons represented the induced segment of occluded tones by responding to discontinuous tones occluded by intense noise (Fig. 1D) as if responding to the complete tone without noise (Fig. 1A). One such neuron responded to a continuous tone with an onset response followed by a pause and then sustained discharge (a sustained response, Fig. 3A). When a gap was introduced (Fig. 3B) during the period of sustained discharge to the continuous tone, a significant reduction in activity relative to the continuous tone response began ~35 ms after gap onset (compare Fig. 3A to 3B during red/dark bins, p < 0.001 bootstrap, see Methods). However, responses to both continuous (Fig. 3D) and discontinuous (Fig. 3E) tones interrupted by loud inducing noise were similar to the responses to isolated complete tones (compare Fig. 3D,E to 3A, no significant differences, bootstrap). The decrease in activity associated with gaps (Fig. 3B, red/dark) was no longer observed when loud noise filled the gap (compare Fig. 3B to 3E, red, p < 0.001). In other words, the neuron responded as if the tone were complete under conditions that have been shown in monkeys and humans to cause illusory completion of the deleted segment (Kluender and Jenison, 1992; Petkov et al., 2003).

Figure 3.

Sustained single neuron response consistent with induction. Above each Peri-stimulus time histogram (PSTH) are schematized stimulus spectrograms (see A for spectrogram axis labels). Vertical grey lines in PSTHs align stimulus events. (A,B) PSTH and raster plots to continuous and discontinuous tones. Red (darker if printed in black and white) bins highlight times when the response is most different between continuous and discontinuous tones. (C) Response to 63 dB isolated noise. (D,E) Responses to continuous and discontinuous tones interrupted by 63 dB noise. In (C) isolated noise is time aligned to the identical noise components in the combined tone-noise stimuli, so noise onset is actually at time = 172 ms. Also in (C), the bins prior to time = −28 ms are empty because only 200 ms pre-stimulus spontaneous activity was collected for all stimuli. This should not be considered a lack of a response; spontaneous activity for this stimulus (C) can only be observed from time = −28 to time = 172 ms.

This neuron was not excited by the 63 dB SPL noise presented in isolation (Fig. 3C), indicating that simple linear addition of the noise response (Fig. 3C) to the discontinuous tone response (Fig. 3B) cannot explain the response to the stimulus that is the linear combination of the two (Fig. 3E). The stimuli in this study were carefully chosen such that the discontinuous tone with noise stimulus (Fig. 3E in this example) was created by adding the discontinuous tone stimulus (Fig. 3B) to the noise stimulus (Fig. 3C). This allows application of a standard definition of linearity: the response to two stimuli added together is linear if it equals the sum of the response to the two stimuli presented in isolation. This definition will be used throughout the text. By this definition, while adjusting for spontaneous activity, linearity is violated because the response to the discontinuous tone with noise (e) should be less than the response to the discontinuous tone (b) because the noise is weakly inhibitory (c). The response to the discontinuous tone with noise is actually larger than the response to the discontinuous tone without noise, suggesting that there is an opposite effect of the noise on the response to the discontinuous tone (net excitatory) than when the noise was presented in isolation (net inhibitory). Therefore, the discontinuous tone plus noise response (e) is much greater than the linear addition of the response to its components (b and c).

Another neuron that behaved in a manner consistent with the perception of induction responded to tones with sustained inhibition followed by excitation to tone offset (a phasic offset response, Fig. 4A). When a silent interval was introduced into the tone, the neuron responded to the first tone segment’s offset with excitation (Fig. 4B, red/dark). When the high-intensity interrupting (inducing) noise was added to the tones, the responses (Fig. 4D, E) were similar to the response to a complete tone in isolation (Fig. 4A). This demonstrates that the neuron responded as if the tone were complete under conditions known to cause induction of the deleted segment (Fig. 4E). The response to the stimulus that causes illusory continuity (Fig. 4E), once again, cannot be predicted by simple linear summation of the responses to its components presented individually (Figs. 4 B,C). The first violation is the neuron’s inability to detect the gap in Fig. 4E (red bins) where linear summation of the responses to (Fig. 4B and 4C) predicts a larger response during the red bins; it should be noted the response at the same time to the non-illusory stimulus in Fig. 4D is roughly linear, i.e., equal to the response in Fig. 4C plus the response in Fig. 4A. The second violation of linearity is the elimination of the excitatory response to noise presented in isolation (see asterisk in Fig. 4C, at time ~300 ms) when the noise was presented in combination with tones (Fig. 4D or 4E). While there are several possible explanations for this response, inhibition by the tone is a likely contributor.

Figure 4.

A single neuron’s phasic (offset) response that is consistent with induction. Same format as Fig. 3. Note that the noise alone response, asterisk in (C) and the gap response (B) are missing during induction.

Onset-responding neurons also behaved in a manner consistent with induction. One exemplary neuron responded to tone onset (a phasic onset response, Fig. 5A). For discontinuous tones the neuron also responded to the tone re-onset after the gap (Fig. 5B), and was excited by noise (Fig. 5C). This neuron also non-linearly responded to the combined tone-noise stimuli. The gap related response was suppressed by the presence of the noise in Fig. 5E, even though the noise by itself was excitatory (Fig. 5C). The result was that the stimulus known to cause illusory induction (Fig. 5E) caused responses consistent with induction; that is, the response to a discontinuous tone interrupted with noise (Fig. 5E) was similar to the response to a continuous tone (Fig. 5A), and dissimilar to the noise (Fig. 5C) and gap (Fig. 5B) responses.

Figure 5.

Onset response consistent with induction. Same format as Fig. 3.

Another exemplary neuron responded with excitation to both tone onset and offset (Fig. 6A), and with corresponding excitation to gaps in tones (middle peak of activity in Fig. 6B, red/dark). Short, loud interrupting noise removed the gap-related response, making both complete and incomplete noise-interrupted tone responses (Fig, 6D,E) similar to isolated complete tone responses (Fig. 6A). Therefore, this neuron responded as if the tone were complete under conditions known to cause illusory completion of the deleted segment (Fig. 6E). The 4 examples (Figs. 3–6) demonstrate responses consistent with the induction model of Fig. 2.

Figure 6.

A Single neuron’s response is consistent with induction for interrupting noise, and masking for surrounding noise. Same format as Fig. 3 except the right column (F-H) is for intense masking surrounding noise that causes only noise (and no tone) to be heard. Note that the differences in responses to interrupting and surrounding noise onsets likely are accounted for by differences in the onset ramps (0 msec for interrupting noise and 25 msec for surrounding noise, see METHODS).

In contrast to the percept of induction, where occluded tone segments are heard continuing through brief interrupting noise (Fig. 1C,D), loud noise completely surrounding tones in time (Fig. 1E,F) creates a masking percept where only noise (and no tone) is heard (Bregman, 1990; Kluender and Jenison, 1992; Petkov et al., 2003). Accordingly, responses consistent with masking require only that responses to combined tone-noise stimuli resemble responses to noise presented in isolation, whereas neural responses consistent with induction require representations of both a continuous tone and the interrupting noise.

Neurons reflected the corresponding induction and masking perceptions with interrupting and surrounding noise respectively. The neuron demonstrating induction related responses in Fig. 6E for interrupting noise, responded as if being masked when presented with intense surrounding noise. For the long duration noise, a short and weak onset response was followed by sustained inhibition (Fig. 6F). This noise response differed markedly from tone responses, which had a stronger, longer excitatory component and no sustained inhibition (Fig. 6A). When loud noise surrounded continuous or discontinuous tones, the neuron responded as if only noise were presented (compare Figs. 6G & H to Fig. 6F): a response consistent with masking.

Given the two distinct percepts of masking and induction, we predicted that neurons would respond to discontinuous tones with intense surrounding noise as if to isolated noise: a neuronal correlate of masking (i.e., cells only detect noise). However for discontinuous foregrounds with intense interrupting (inducing) noise, a different result was predicted. Because during induction both the noise and the induced deleted foreground segment are perceived (Bregman, 1990; Kluender and Jenison, 1992; Warren, 1970; Warren, 1972), in order to be consistent with the percept of induction the neuronal population must represent both the induced tone segment and the occluding noise. Accordingly, for interrupting noise we expected the population to represent the continuous tone (illusion) as well as the noise.

To test these predictions, differences between masking and induction were quantified using a response index, a normalized tone-noise similarity index (TNSI, see METHODS: Data acquisition and Data analysis for details on the analysis and the neural sample). TNSI values ranged from −1 to +1 signifying that responses to discontinuous tones presented with the loud noise were similar to noise (−1, masking) or tone (+1, induction) responses (Fig. 7A). Responses to tones presented with intense surrounding noise were more like noise-only responses (median TNSI = −0.37; Fig. 7B), consistent with the masking percept of hearing only noise. In contrast, for loud interrupting noise the population of neurons representing the tone and noise were more positively distributed (median TNSI = −0.07; Fig. 7C). Differences in median TNSI values for surrounding and interrupting noise were statistically significant, indicating that, for interrupting noise, more neurons responded as if the missing tone segment were present than did for surrounding noise (Kolmogorov-Smirnov [K-S] test Z = 1.78, p = 0.004); Mann-Whitney: Z = 3.43, p = 0.001, see Methods). This effect was observed separately for onset, sustained, and offset response components, consistent with the model in Fig. 2 (sustained: interrupting noise n = 140 vs. surrounding noise n = 71, K-S test, Z = 1.60, p = 0.012; onset: interrupting noise n = 93 vs. surrounding noise n = 49, Z = 1.52, p = 0.02; offset: interrupting noise n = 124 vs. surrounding noise n = 67, Z = 1.40, p = 0.04).

Figure 7.

Population of single unit responses supports a neural representation of masking for surrounding noise and of induction for interrupting noise. (A) Tone Noise similarity (TNSI) varies from +1 (response to a discontinuous tone presented with intense noise equals the isolated continuous tone response) to −1 (response to discontinuous tone presented with intense noise equals the isolated noise response). (B and C) Single neuron TNSI distributions for surrounding and interrupting noise. (E) Neurons with positive TNSI values (see arrow in c) responded as if noise-interrupted discontinuous tones were continuous, and not is if they were discontinuous. (D-E) Tone Gap similarity index (TGSI) varies from +1 (response to discontinuous tone presented with intense interrupting noise equals the isolated continuous tone response) to −1 (response to discontinuous tone presented with intense noise equals the isolated discontinuous tone response). Arrowheads in (B-D) show the median, and asterisks (*) show the TNSI and TGSI value for the example cell in Figure 6.

For the neurons with positive TNSI values for interrupting noise -- those hypothesized to represent the tone, rather than noise during induction – we wanted to determine whether they represented the illusory tone segment because the TNSI does not rule out their representing the gap. If they responded as if a gap were present, this would suggest that the neurons neither supported masking nor induction. The tone-encoding neurons responded as if a continuous tone were presented indicating that they represented the induced tone segment. We quantified this using a tone-gap-similarity index (TGSI). TGSI values ranged from −1 to +1 signifying that responses to discontinuous tones presented with the loud noise were similar to responses to a discontinuous (−1) or continuous (+1) tone presented without noise. The results with the TGSI indicated that when loud interrupting noise was used, 74% (63/85) of the neurons’ responses were closer to a continuous than to a discontinuous tone response (Fig. 7E). The effect was significant (one sample t-test differed from 0, t = 6.3, p = 0.000; one sample K-S test of uniformity Z = 2.7, p = 0.000; median TGSI = 0.42). This indicates that most of the neurons representing the tone over the noise during the inducing stimuli are representing a continuous (rather than a discontinuous) tone, consistent with the induction percept.

The observations so far were based on recordings taken near each neuron’s best-frequency response (BF, see Methods). We also collected responses to 2 kHz tones --those used in the psychophysical studies of macaque induction (Petkov et al., 2003) --which were not always close to the neurons’ BFs. This provided data from a larger population. We saw similar, albeit expectedly weaker, relationships in this dataset (see Supplementary Notes and Supplementary Fig. 1). We also quantified several neuronal response characteristics that seemed to contribute toward induction (see the Supplementary Notes and Table).

Simulated peripheral neuron responses cannot support induction

It is not clear to what extent the periphery can support induction correlate we observe in A1. To compare A1 to the auditory nerve, we provided our stimuli as input to a cascaded peripheral processing model (see METHODS: Simulation of peripheral responses). We then applied the same analysis for the simulated responses as we did for our A1 data.

Figure 8 shows an exemplary simulated peripheral ‘eighth-nerve neuron's' response to the stimuli we used to evaluate induction for A1 neurons. When the continuous tone was used as a stimulus (in this case a 2 kHz tone centered at the 'BF' of this simulated neuron), the model shows a largely sustained 'response', with adaptation following stimulus onset (Fig. 8A). When a discontinuous tone was used there is a cessation in activity with a tone re-onset response following the gap in the tone (Fig. 8B). There was little variability in tone/gap responses for these simulated peripheral neurons. Importantly, this peripheral neuron responded strongly to the interrupting noise by itself (Fig. 8C) including when this noise was added to a continuous (Fig. 8D) or discontinuous (Fig, 8E) tone. The TNSI value for this example was negative (−0.62) showing that this example does not support induction.

Figure 8.

Simulated peripheral responses support only the physical properties of the stimuli. Shown is a modeled 'eighth nerve neuron' response to the stimuli used to assess induction in our A1 neurons. Here the tonal stimuli are 2 kHz tones centered at the ‘BF’ of the simulated neuron. For display purposes the spontaneous 'firing rate' is not shown. (A-E) is in the same stimulus/response format as for Fig. 3a-e. (F) is as in Figure 7a, which schematizes the range of TNSI response values that can be obtained including how to interpret positive and negative values (positive values show that the response to the stimulus known to elicit induction are like those to a continuous tone, negative values that the response to this stimulus is as if only noise were presented). (G) shows the modeled distribution of peripheral responses. Arrowhead show the median, and the asterisk (*) shows the TNSI value for the example shown in (A-E).

Our impression from this and other examples was that the simulated peripheral neurons strongly responded to the noise, causing highly negative TNSI values (which supports masking rather than induction). This was confirmed by modeling a distribution of peripheral responses for neurons whose tone frequency/best-frequency relationships (supplemental Figure 1) matched those of the neurons sampled in A1 for Figure 7C. Thus for stimuli containing interrupting noise we modelled an identical distribution (in number) of 'neurons' as collected for our sample of A1 neurons (see Supplementary Fig. 1, and METHODS). The distribution of interrupting noise TNSI resulting from the simulated auditory nerve neurons had a highly negative median (−0.77, Fig. 8G). The median TNSI using surrounding noise was similarly negative (median = − 0.71, n = 92).

DISCUSSION

Using stimuli that cause auditory induction the population of neurons we sampled represented both the missing tone segment and the occluding noise, both of which are perceived during induction. Neurons representing the missing tone segment responded to discontinuous tones occluded by intense noise as if responding to complete tones. Consistent with the model of Fig. 2, this neural code included both phasic responses that fail to detect discontinuities and sustained responses that continue through the occluded segment.

Population codes: perceiving a complete tone and noise

We observed many neurons representing the missing tone segment in our inducing stimuli, although these were a minority of the neurons (63/179). There are two steps that led to a reduction from the 179 total neurons to 63 signalling the illusory segment. The first step was separating tone-encoding (85/179) from noise-encoding (94/179) neurons. The second step was the observation that a minority of the tone-encoding neurons (22/85) responded as if a gap were present in the tone.

During induction both a continuous foreground (in our case, a tone) and an interrupting noise are perceived. Thus, to be consistent with the percept the population of neurons should represent both. A separate representation of tones and noise is consistent with the hypothesized parallel analysis of sound bandwidth for a multi-spectral wavelet like analysis (Schreiner et al., 2000; Schreiner and Sutter, 1992; Sutter, 2005). Using the Tone-Noise Similarity Index (TNSI), we estimated that 94 neurons represented the noise and 85 represented the tone. It is worth considering that there are many intermediate bandwidth neurons in A1 (Recanzone et al., 2000; Schreiner et al., 1992), and many of these can respond to both the tones and noise. Such neurons could have TNSI values near 0. Thus our approach of defining neurons with TNSI > 0 as representing the tone and those with TNSI < 0 as representing the noise likely results in some categorization errors.

Of the 85 putative tone-encoding neurons, 63 had positive Tone-Gap-Similarity Index (TGSI) values indicating they responded more as if the deleted segment were present (induction) than absent. Why then would we find 22/85 neurons with negative TGSI values? Some might result from the classification errors in TNSI noted above causing us to inadvertently sample noise encoding neurons. Furthermore some neurons that represented the tone and noise might have a noise response that caused a response similar to the gap. These provide examples of how negative TGSI values could result from either statistical variation in responses (measurement noise), or reproducible responses caused by some neurons that encode both foreground and background. In either case, this suggests that there is some ambiguity in the neural code in A1 that must be resolved. This type of ambiguity in a brain area representing both foreground and background is inescapable and suggests some intriguing possibilities. First, the transformation which creates induction might not be complete at the level of A1. Second, an unbiased observer might be able to decide based on the aggregate activity of the population of foreground encoding neurons. One decision function can result from equal weighting of all tone-encoding neurons’ responses, so a positive median or mean TGSI from the distribution would lead to a decision that a continuous tone was present. Another possibility is greater weighting being for neurons that more selectively represent a tone (i.e., neurons with more positive TNSI values). The aggregate of our data supports that A1 neurons represent the missing tone segment and the inducing noise, consistent with the entire induction percept.

Population codes: The importance of multiple neuronal response types

While in this study we have found neurons that encode missing tone segments, it seems unlikely that an isolated population of ‘induction’ neurons or a single response type could account for all induction phenomena for two reasons. First, induction has been found for every foreground sound tested. Second the ‘no discontinuity’ and ‘sufficiency of evidence’ rules demand different contributions from different neuronal response types.

The general model, that a neuronal population should respond as if the induced foreground and interrupting noise were both present, is powerful because it does not depend on any single response type or physiological mechanism and therefore can be applied to any foreground sound. For the specific case of tonal foregrounds, this leads to a simplified model (Fig. 2): onset and offset responses fail to detect the transitions in tones, and sustained response neurons continuously respond through the induced segment, which comply with both the ‘no discontinuity’ and ‘sufficiency of evidence’ rules. These conditions were both met by our results.

The psychophysical data supporting the two induction rules indicate that both sustained and phasic responses are important for induction. For an illusory inducing stimulus (Fig. 1D), if neurons responding to tone onset fired at the re-introduction of the tone (reporting discontinuity) and sustained responders continued to respond as if a continuous tone were present (reporting a continuous tone), the brain would have to resolve whether the tone actually continued through the noise. Evidence of tone onset/offset appears to weigh heavily in such resolution because psychophysical evidence of discontinuity in the tone disrupts induction (Bregman and Dannenbring, 1977). This makes sense because the noise contains energy at the tone frequency, so whether that energy comes from the tone or noise is ambiguous. If there is clear evidence of tone offset or onset at the gap this resolves the ambiguity and suggests that the noise energy does not belong to the tone. But if only phasic neurons were present in a population, their decreases in activity might be erroneously interpreted as signalling the absence of a stimulus. However, within a population of neurons this reduction in activity can carry important information (Newsome et al., 1989) as long as some neurons, such as sustained responders, indicate the continuing presence of the sound. In this case a decrease in activity can be just as informative as an increase because it supplies complementary information. If the brain were only to consider sustained responders, problems would arise also. A1 neurons must encode many sounds so even an increase in activity may ambiguously represent different sounds or sound features. Thus, the joint activity from different neural response types can disambiguate the different sounds or sound mixtures that either class individually might not.

Relationship to masking

To what degree are processes akin to perceptual masking responsible for the interrupting noise results? Although removal of responses to transients can be thought of as a form of masking, simple peripherally-mediated energetic masking (where a very loud continuous noise eliminates perception of a fainter foreground sound), cannot be responsible for the A1 results. Such energetic masking is commonly associated with auditory nerve responses, where large isolated excitatory noise responses dominate combined noise/tone responses (Rhode et al., 1978). The result is that the weak response produced by the low intensity tone cannot be extracted from the much larger response created by the loud noise. We suspected auditory nerve would only show a masking and not an induction correlate. The TNSI distributions obtained from our simulations of auditory nerve responses demonstrated highly negative TNSI values consistent with masking, i.e., responses more similar to the noise than the tone. In contrast, when recording in A1 using illusory inducing stimuli more positive TNSI values were observed. This argues that factors other than known energetic masking properties of the periphery are contributing to the cortical responses. While we can rule out this sample form of masking, it is not unreasonable to predict that other more complex forms of masking, such as backwards masking (Brosch et al., 1998; Pickett, 1959), contribute to induction.

Mechanisms of auditory induction

We have shown that the responses of A1 neurons are consistent with induction and the simulated auditory nerve responses are not. An outstanding question is how do these cortical response properties arise and where in the brain? Our experiments were designed to address whether A1 activity represented the illusory sound segment, and not necessarily to determine the mechanisms creating them. A1 was chosen as an initial area of study because it lies at the boundary between early and late processing of sound. From a cognitive neuroscience perspective, and because of the results of lesion studies suggesting the involvement of auditory cortex in conscious perception of sounds, e.g., (Graham et al., 1980; Michel et al., 1980), A1 might be thought of as an early processing stage for encoding a perceptual phenomenon like induction. However, from an auditory physiology and mechanistic perspective A1 might be thought of as a higher station in the auditory system that obtains many of its properties subcortically. By choosing A1 our results provide a crucial starting point for mechanistic studies studying in more detail how and where the response properties that we report in cortex are created, and for performing studies in higher cortical areas to look for changes in these representations.

We stated in the Introduction that the continuity illusion results from trying to perceptually organize a potentially ambiguous sound signal into auditory objects. To illuminate how different brain areas might contribute to grouping sounds we might look to another psychophysical example that involves perceptual grouping, comodulation masking release (CMR). CMR describes an increased ability to detect an unmodulated tone in the presence of a modulated noise when the noise is comodulated across bandwidth. As the envelope of the noise is confirmed across more frequency bands, it becomes easier to perceive the tone as a distinct object, and to detect it. There is evidence for important contributions to CMR at the level of the cochlear nucleus (Pressnitzer et al., 2001), with progressive refinement and improvement of CMR related properties with ascension up to A1 (Las et al., 2005). However, we cannot assume the same holds for induction. Although both induction and CMR are related to perceptual grouping, CMR and induction are very different perceptually and use very different stimuli. Therefore while the previous CMR studies potentially provide a framework from which to view our induction results, CMR cannot directly speak to the neural origins of the cortical responses we see. The question of where the response properties that we observe are created can only be addressed by recording from many auditory stations. Still it is an intriguing possibility that similar progressive refinement along the auditory neuroaxis might occur for induction, and that auditory cortex might play a pivotal role in generating the representation of auditory objects (Nelken et al., 2003).

In addition to where these response properties arise the question arises as to how they are created. Our findings highlight that induction requires coding multiple sound properties, through several neuronal response types, and suggest the involvement of multiple cellular mechanisms. Explanations of the results can be made by describing how neurons respond to the time varying stimulus spectrum and/or to the stimulus envelope. With respect to the time varying frequency spectrum, the examples that we report lead to several intriguing possibilities, including but not limited to different nonlinear inhibitory effects (Figs. 4 and 5) as well as disinhibition or facilitation (Fig. 3 and 6). However there are a plethora of possible alternative explanations, and at this point it would be premature to speculate. Additionally we observed a much wider variety of responses than the most prominent examples shown here, typical of the heterogeneity of A1. Despite the heterogeneity of responses, the population of sampled neurons responded in a manner consistent with the induction percept. An advantage of our approach is that, regardless of the exact mechanisms shaping the induction-related responses, our analyses make it possible to evaluate the relationship between the responses of the population of A1 neurons and induction. This was achieved by comparing responses to the stimuli that cause induction to responses to individual stimulus components that were either physically present in the inducing stimulus or those that were perceived by subjects listening to the inducing stimulus. Ultimately, to reveal the mechanisms responsible for the observed correlate of induction will require recording from multiple brain regions and performing intracellular recording experiments.

The diversity of the observed induction-related responses with tonal foregrounds suggest that the ecological pressure to maintain stable representations of interrupted sounds is important enough that it has been selected upon or acquired through multiple neuronal encoding mechanisms. As such, searching for the induction neuron, single brain region responsible for induction, or single cellular mechanism responsible for induction might turn out to be a futile endeavor.

Auditory induction and attention

Many studies show that certain forms of auditory induction appear to be a fairly automatic process that can occur outside the focus of attention (Bregman, 1990; Micheyl et al., 2003). Further, our behavioral work in monkeys (Petkov et al., 2003) supports the idea that induction cannot be entirely overridden by attention. There the animals were unable to overcome the illusion despite being rewarded for detecting a gap in the sound. These results argue that some aspects of induction might be due to processing ‘early’ in the auditory system where the influence of ‘top-down’ cognitive control is not as strong as in non-primary areas of human auditory cortex (Grady et al., 1997; Petkov et al., 2004; Pugh et al., 1996). Such a viewpoint is also consistent with EEG induction correlates found in humans not attending sounds (Micheyl et al., 2003), and with cortical based modelling of induction at the initial stages of auditory cortex (Husain et al., 2005).

On the other hand, there is evidence that some forms of induction can utilize feedback connections. For example induction with speech is though to also invoke feedback (Sivonen et al., 2006). Other forms of perceptual grouping are known to build up over time and can be influenced by the redirection of subjects’ attention (Carlyon et al., 2001). Such an attention effect has yet to be demonstrated with tonal foregrounds, however, and would likely be complementary to the automatic processes already reported. Because these monkeys were passively listening, we believe the present results provide a basis for understanding the ‘pre-attentive’ foundations of perceptual induction, without assuming induction arises in A1 or excluding the possibility of further top-down modulation in higher areas.

Summary and Conclusions

In summary, our results support that under conditions that produce induction, the illusory tone segment is represented in A1. Of the major types of neurons investigated in this study, all three responded as if the tone were present -- offset responders fail to encode gap initiation, sustained responders provide activity as if the tone continued, and onset responders fail to signal the gap’s termination. This result is consistent with two principles of induction (Bregman, 1990; Bregman and Dannenbring, 1977; Houtgast, 1972): (1) there should be no neural evidence of gap onset/offset, and (2) during the noise neural activity should be indistinguishable from activity that would have occurred had the tone actually continued. We conclude that A1 neurons demonstrate the brain’s ability to compensate for transient noise in the environment by ‘filling-in’ segments of sounds occluded by noise.

METHODS

Stimuli

Stimuli, presented from speakers (O'Connor et al., 2000; O'Connor et al., 2005; Petkov et al., 2003) placed 1.5 m from the animal, were identical to those previously reported for tonal foregrounds used in psychophysical experiments (Petkov et al., 2003) except that the frequency of the foreground could also be set to the best frequency of the recorded activity (see Supplementary Fig. 1). The foreground was a 45 dB SPL (unfiltered calibration, Brüel & Kjær 2231 sound level meter) 400 msec tone (cosine ramped, 8 msec rise/fall times), with a sample frequency of 50 kHz, or in the cases of the tone frequency being higher than the Nyquist frequency at 100 kHz. Transitions into and out of the silent gap -- temporally centered in the tone – had 3 msec rise/fall times. The gap duration --silent portion plus transitions -- was 56 msec. Noise (broadband, 25 kHz cut-off) was calibrated in RMS level (dB SPL, re 20 micro Pascals). Interrupting-noise was un-ramped and temporally centred in the foreground, corresponding to noise presentation from 172–228 msec after initial foreground onset. When a gap was present this corresponded to noise completely overlapping the gap (including ramps) but not the tone segments outside of the gap (Fig. 1D, 3E). Surrounding-noise (450 msec including 25 msec onset/offset ramps reaching their plateau when the foreground began, and beginning offset transition when the foreground was completed) temporally encompassed the entire foreground. In this paper we only report results for 63 dB SPL noise conditions. The noises within a given type and intensity were ‘frozen’ so that the only difference between continuous and discontinuous stimuli with noise was the presence or absence of a silent gap in the foreground. Additionally we presented the two noise types (interrupting and surrounding, e.g., Fig. 6C,F) in isolation and the continuous and discontinuous tones in isolation so that responses to combined tone/noise stimuli could be compared to responses to tones and noise in isolation. Interrupting noise presented in isolation (e.g., Figs.3C, 4C, 5C) was time-aligned to it’s occurrence in the combined stimuli (Figs. 3D-E, 4D-E, 5D-E), and in the figures is aligned to start at time = 172 ms (e.g., Fig. 3C).

Data acquisition

Standard extracellular recording techniques were used to record from the right hemispheres of two naïve adult macaque monkeys, conforming to the PHS policy on animal care. Subjects were on a restricted water access protocol approved by the UC Davis animal care and use committee. Extracellular recordings occurred with macaques awake, seated, head restrained in a primate chair, designed to be “acoustically transparent", within a double-walled, sound attenuated, and foam lined chamber (IAC: 2.9 x 3.2 x 2.0 m³, internal). For further recoding and single unit (spiking neuron) isolation details see (O'Connor et al., 2005).

We recorded from 304 single units with interrupting-noise, of which 153 were also recorded with surrounding-noise. We recorded with the foreground frequency close to BF and at 2 kHz (the frequency used in macaque psychophysics (Petkov et al., 2003)), resulting in a sample of 494 and 210 neurons recorded with interrupting and surrounding noise, respectively. Our analyses are either from neurons recorded with the tone frequency close to BF (see RESULTS, Fig. 7), from the entire sample (see Supplementary Table 1 and Supplementary Fig. 1), or from a subdivision of the sample based on the type of response of neurons to different sounds (Supplementary Fig. 3).

For localizing recordings from A1, we first stereotaxically guided the electrodes to A1’s relative anatomical position within the macaque auditory cortex (Paxinos et al., 2000). Then we identified A1 by its response latency, responsiveness to tones and its direction of tonotopic gradient for best frequency (BF) responses to tones (Merzenich and Brugge, 1973; Recanzone et al., 2000). The area extended by the tonotopic gradient in the antero-posterior direction (Hackett et al., 2001) and the medio-lateral extent of tone responsiveness(Rauschecker and Tian, 2004) supported that recordings were from field A1.

Data analysis

Determining a neuron's best-frequency response (BF)

BF was determined using an interpolation method so as not to rely solely on the response to a single tone frequency. This incorporated the tone frequency eliciting maximal response (S_b, sum of spikes response) and the responses to the two neighboring frequencies (S_a and S_c, response to the neighboring lower and upper tone frequencies, respectively). From these three responses we determined a weighting factor as follows:

$$w=(({\text{S}}_{\text{b}}-{\text{S}}_{\text{a}})/((2\ast {\text{S}}_{\text{b}})-{\text{S}}_{\text{a}}-{\text{S}}_{\text{c}}))$$

Then BF was calculated:

$$\text{BF}={\text{f}}_{\text{a}}\ast 2^(w\ast \text{OctRange})$$

where f_a is the tone frequency (Hz) eliciting the S_a response and OctRange is the range in octaves between f_a and f_c. In the case of two frequencies with maximal responses, the BF was half way (in octaves) between these two frequencies (e.g., w becomes 0.5).

We obtained a fairly even sample of BFs from our entire sample (on an octave scale) ranging from 150 to 40,025 Hz. The relationship of neuronal BF to the tone frequency used is shown in Supplementary Figure 1A.

Tone Noise Similarity Index (TNSI) and Tone Gap Similarity Index (TGSI)

The TNSI was used to quantify how similar each single-unit’s response to the discontinuous tone with intense noise (DTIN) was to the isolated tone response (T) or isolated noise response (N):

$$\text{TNSI}=(A-B)/(A+B),$$

where A = | N – DTIN | and B = | T – DTIN |. N, DTIN, and T for the analysis presented in the paper was the response in spikes counted over a time window as described below. When noise was used in isolation (e.g., Fig. 3C, 4C, 5C), the window for counting spikes was aligned with when it would occur in the combined tone/noise stimulus. The TGSI was similar to the TNSI except in the above equation the noise response (N) is replaced with the response to the tone with a gap (G), so that A = | G – DTIN |.

Procedures for comparing responses

Many A1 responses have multiple components (inhibitory and excitatory) with high temporal precision. To prevent these responses from opposing each other, four different procedures were used to compare responses each with their own advantages. Three involved choosing a time window over which to count spikes, and one involved comparing (correlating) entire peri-stimulus time histograms (PSTHs) without choice of a time window. These spike count and correlation measures were then used to derive TNSI and TGSI values. All statistical tests with TNSI and TGSI yielded the same results (for both significant and not significant effects), regardless of which of the four procedures were used. For Figures (3–7) and associated analyses in the text we strictly used the method based on a statistical criterion, where all neurons that contributed could be said to significantly encode gaps in tones. Elsewhere, we also report results using the other methods, including those from the entire sample, in Table 1 and the Supplementary Online Material.

For the methods in the paper, a statistical criterion was used to define the time window over which to count spikes. First, a difference PSTH (5 msec bins, 50 stimulus repetitions) was created by subtracting the discontinuous tone (without noise) PSTH from the continuous tone (also without noise) PSTH. Then a neuron was evaluated only if this difference PSTH, following the onset of the gap, had a maximum (of absolute value) that was significant. The statistical criterion was that two consecutive bins were above- 2 standard deviations (SD) or one bin above- 4SDs from the pre-stimulus “spontaneous” activity. If this criterion was met, the starting and ending points of the window were determined by finding, in both directions from the maximal bin, the third consecutive bin that was below two SD; the analysis window was identified as starting and ending on these bins. The distribution of these response windows showed a narrow median width of 75 ms with a median starting position of 241 ms following tone onset (69 msec after gap onset). We also counted spikes fired to the two types of noise within these windows and saw that more spikes were elicited by the short-duration interrupting noise (median 9.9 spikes) than the longer duration surrounding noise (median 5.3 spikes), differing at the p < 0.05 level (K-S test). This rules out that more negative TNSIs for surrounding noise were due to larger excitatory responses to the longer duration noise.

This method may be preferred for evaluating neurons involved in induction because it selects a narrow statistically based window of the response to the gap, without considering the noise responses. The method selects the neurons that show that they can discriminate continuous from discontinuous tones in a statistically significant manner and therefore those most likely to represent the studied foreground differences.

Classifying tone responses

We objectively classified tone responses as a prior classification of macaque A1 responses (Recanzone, 2000). We windowed the tone stimulus response into three intervals: early (0–200 ms), late (200–400 ms) and offset (425–625 ms). Significant responses (2 bins above 2-SD or 1 bin above 4-SD of the baseline activity) occurring within the early interval were identified as ‘Phasic-onset’ responses. ‘Phasic-offset’ responses were defined as significant responses in the offset interval. ‘Sustained’ (excitation) responses were identified as significant responses occurring in both the early and late periods. Sustained inhibition was difficult to detect using our standard statistical criterion (these responses were usually close to the mean spontaneous level), thus we assigned inhibitory responses to the 'Sustained' category if they were below the mean spontaneous level for more than 65 ms.

Statistical Analyses

To test whether interrupting vs. surrounding noise distributions differed (e.g., Fig. 7C vs. 7B), we used the non-parametric Kolmogorov-Smirnov (K-S) test which allows non-normally distributed data to be tested. A non-parametric test such as this was also important because the sample size for the surrounding noise distribution was smaller due to over-sampling of neuronal data for the stimuli containing interrupting noise. Results were statistically the same (for significant, at p < 0.05, or not significant effects) when also using the more common non-parametric Mann-Whitney test. Although the K-S test is sensitive to differences in shape as well as central tendency, it seemed our effects were largely based on differences in central tendency since subtracting the mean or median of the distributions between Interrupting and Surrounding noise (see Figs. 7B and C) removed the significance of the observed differences. For testing whether a single distribution differed from zero we used a one sample t-test (two-tailed prediction) and a one sample K-S test of uniformity in the distribution.

Simulation of peripheral responses

We used Malcolm Slaney's Auditory Toolbox (version 2) which is coded in Matlab and implements a number of models of peripheral processing (Slaney, 1998). Using the toolbox we implemented a cascaded model to obtain simulated responses of auditory (eighth) nerve fiber/neuron responses. The first component of the model is an auditory filter bank proposed by Patterson and colleagues (Patterson et al., 1995; Patterson, 1992; Slaney, 1993), which uses a gammatone bank of auditory filters with an equivalent rectangular bandwidth based on measurements of critical bands (ERB, (Glasberg and Moore, 1990). This models basilar membrane motion which will elicit potentials in the inner hair cells, the output of which can be considered a narrow-band auditory filter or channel. Here we used 75 channels, with upper and lower frequencies matching the range of BFs found in our A1 neuron samples. The second 'Meddis' component (Meddis, 1986; Meddis, 1990), was added at the output stage of the ERB filterbank (Slaney, 1998). This model simulates response properties such as adaptation following stimulus onset. Standard parameters for this model were used (Meddis, 1990; Slaney, 1998). The output of this model is the spike probability of an auditory nerve fiber, i.e., eighth nerve neuron.