Recalibration of the auditory continuity illusion: sensory and decisional effects

Abstract

An interrupted sound can be perceived as continuous when noise masks the interruption, creating an illusion of continuity. Recent findings have shown that adaptor sounds preceding an ambiguous target sound can influence listeners’ rating of target continuity. However, it remains unclear whether these aftereffects on perceived continuity influence sensory processes, decisional processes (i.e., criterion shifts), or both. The present study addressed this question. Results show that the target sound was more likely to be rated as ‘continuous’ when preceded by adaptors that were perceived as clearly discontinuous than when it was preceded by adaptors that were heard (illusorily or veridically) as continuous. Detection-theory analyses indicated that these contrastive aftereffects reflect a combination of sensory and decisional processes. The contrastive sensory aftereffect persisted even when adaptors and targets were presented to opposite ears, suggesting a neural origin in structures that receive binaural inputs. Finally, physically identical but perceptually ambiguous adaptors that were rated as ‘continuous’ induced more reports of target continuity than adaptors that were rated as ‘discontinuous’. This assimilative aftereffect was purely decisional. These findings confirm that judgments of auditory continuity can be influenced by preceding events, and reveal that these aftereffects have both sensory and decisional components.

1 Introduction

In natural environments, sounds of interest are often masked momentarily by extraneous sounds, such as noise. Nevertheless, under some circumstances, listeners can still hear a target sound that is briefly interrupted by noise as ‘continuing through’ the noise. Remarkably, this may occur even when the target is physically interrupted, as long as the noise effectively masks the interruption (Houtgast, 1972; Warren et al., 1972). Psychophysical studies have revealed that this auditory continuity illusion occurs with various kinds of sounds, including tones (e.g., Ciocca et al., 1987; Miller et al., 1950; Thurlow, 1957) and speech (e.g., Bashford et al., 1987; Verschuure et al., 1983; Warren, 1970) (for reviews, see Bregman, 1990; Warren, 1999). Behavioral animal studies have further indicated that the phenomenon is experienced also by birds (Braaten et al., 1999; Seeba et al., 2009), cats (Sugita, 1997), and monkeys (Miller et al., 2001; Petkov et al., 2003). Neurophysiological studies have investigated the neural basis of the phenomenon at the level of the thalamus (Schreiner, 1980) and of the auditory cortex (Heinrich et al., 2008; Micheyl et al., 2003; Petkov et al., 2007; Riecke et al., 2007; Riecke et al., 2009a; Shahin et al., 2009; Sivonen et al., 2006; Sugita, 1997).

Most previous studies of the continuity illusion have not considered the possible role of sequential effects of prior auditory stimuli on the target sound, although such ‘aftereffects’ are well described in the literature for other auditory phenomena. For example, psychophysical studies have shown that prior adaptor stimuli can induce changes in behavioral thresholds for detecting and/or discriminating auditory features such as intensity or loudness (e.g., Mapes-Riordan et al., 1999; Marks, 1993; Scharf et al., 2002), amplitude- or frequency-modulation (e.g., Green et al., 1974; Tansley et al., 1983; Wakefield et al., 1984), relative frequency (Schellenberg et al., 1994), location (e.g., Frissen et al., 2003; Frissen et al., 2005; Phillips et al., 2005), phonemic category (e.g., Eimas et al., 1973; Sawusch et al., 1981; Simon et al., 1978), voice gender (Schweinberger et al., 2008), and the number of perceived streams (e.g., Bregman, 1978; Snyder et al., 2009a; Snyder et al., 2008). Several of these psychophysical findings are paralleled by recent neurophysiological studies showing that prior acoustic input can induce rapid changes in the responses of neurons in the central auditory system (e.g., Bartlett et al., 2005; Brosch et al., 1997; Micheyl et al., 2005; Sekuler, 1994; Ulanovsky et al., 2003; Ulanovsky et al., 2004; Werner-Reiss et al., 2006). Remarkably, some of these changes in neural response may persist for a few seconds or more (e.g., Condon et al., 1991; Malone et al., 2002; Ulanovsky et al., 2003; Ulanovsky et al., 2004; Werner-Reiss et al., 2006).

Inspired by these previous studies, a recent study of the continuity illusion has investigated whether prior adaptor stimuli influence listeners’ ratings of perceived continuity (Riecke et al., 2009b). This study used as adaptor stimuli a tone that alternated with noise bursts. The adaptors were rated illusorily as continuous when the signal-to-noise ratio (SNR) was low (i.e., when the noise masked the gaps in the tone), and they were rated correctly as discontinuous when the SNR was high. The target stimulus was identical to the adaptor stimuli, except that the SNR was adjusted in such a way that the continuity of the target was ambiguous. The results revealed that listeners were more likely to judge the ambiguous target as continuous when it was presented after adaptors that were rated discontinuous than after adaptors that were rated continuous.

However, it is unknown whether this ‘contrastive’ aftereffect of the adaptors is sensory or decisional in nature: The aftereffect might alter the perception of continuity (sensory aftereffect) or the criteria applied to rate continuity (decisional aftereffect). The aftereffect might further be due to the sensation evoked by the adaptors (sensory factors) or listeners’ categorization of the adaptors as either ‘continuous’ or ‘discontinuous’ (decisional factors). In addition, it needs to be investigated in more detail at which processing stage(s) in the auditory system these aftereffects originate.

Here, we addressed these unresolved issues in three psychophysical experiments. We studied listeners’ continuity ratings of interrupted and uninterrupted targets (stimuli with an ambiguous SNR) as a function of different adaptors (stimuli with low, high, or ambiguous SNR). The ratings were analyzed in terms of sensitivity (d’) and decision criterion (C) (Green, 1966; Macmillan, 1991), which allowed us to distinguish between sensory and decisional aftereffects (for similar applications of signal detection theory [SDT] in studies of the continuity illusion, see Bennett et al., 1984; Kluender et al., 1992; Samuel, 1981). The aftereffects of physically identical ambiguous adaptors were further compared to the aftereffects of physically different unambiguous adaptors, which allowed us to distinguish between sensory and decisional factors. In addition, the adaptor and target stimuli were presented either to the same ear or to opposite ears, which allowed us to investigate at which auditory processing stages the aftereffects may originate.

2 Materials and methods

2.1 Participants

Thirty eight volunteers (26 females) between 19 and 57 years old (mean: 25) with no reported hearing problems participated in the study after giving informed consent. Three different groups of listeners (N = 14, 14, 17) participated in experiment 1, 2, and 3 (respectively), including three listeners who participated in both experiments 2 and 3, and two other listeners (authors M.V. and L.R.) who participated in all experiments. Except for the latter two, all participants were uninformed about the study background and received payment for their participation. As shown in supplemental Figures S1 and S2, excluding the data of the two non-naïve participants and the two oldest participants (aged 57 and 39) from the analyses did not change the overall results reported below.

2.2 Stimuli

The same stimuli were used as inRiecke et al. (2009b). These stimuli were comprised of a frequency sweep, which was either uninterrupted or periodically interrupted by noise (Figure 1A). The sweep was obtained by multiplying the instantaneous frequency of a tone by a logarithmic function. The frequency of the sweep spanned a range from 1 to 3 kHz over the course of the 5-s stimulus duration. The noise was obtained by band-pass filtering broadband Gaussian noise between 0.9 and 3.6 kHz (3-dB cutoff frequencies; finite impulse response filter). The interrupted sweep was created by modulating the amplitude of the tone with a square wave function (500-ms period; 50 % duty cycle), and filling the 250-ms gaps with the filtered noise. All amplitude onsets and offsets were linearly ramped with 25-ms rise–fall times in such a way that the midpoints of the tone off-ramps coincided with the midpoints of the noise on-ramps, and vice versa. The amplitude of the tone remained constant at 60 dB sound pressure level across all three experiments. The amplitude of the noise was adjusted relative to the amplitude of the tone (i.e., the SNR was adjusted) to produce a continuity illusion in some conditions, and not in other conditions (see Task and design).

Figure 1. Stimuli and experimental design.

A, Auditory stimuli consisted of an ascending tone in which portions were replaced by noise bursts, as illustrated by the sound spectrogram. B, Trials comprised an adaptor stimulus presented twice and a subsequent target stimulus. Listeners judged the continuity of the stimuli during visually cued response intervals. C, For the interrupted target stimuli, the relative amplitude of the tone and noise (SNR) was defined individually from thresholds for the continuity illusion of the interrupted tone (−6.8 dB on average). The uninterrupted target stimuli were physically identical to the interrupted target stimuli, except that the tone was physically uninterrupted. D, In experiment 1, the adaptors comprised relatively loud or soft noise, or no adaptors were presented. E, In experiment 2, the same stimuli as in experiment 1 were presented differently, i.e, the adaptors and target within each trial were presented either to the same ear or to opposite ears. F, For experiment 3, physically identical, perceptually ambiguous adaptor stimuli were used that were identical to the interrupted target stimuli. ‘Subjective’ adaptor conditions were created post hoc by sorting trials according to how listeners had judged the ambiguous adaptors (i.e., as either continuous [‘cont.’] or discontinuous [‘disc.’]).

2.3 Apparatus

Stimuli were sampled at 44.1 kHz with 16 bit resolution using Matlab 7.0.1 (MathWorks Inc., Natick, MA). They were presented diotically (experiments 1 and 3) or monotically (experiment 2) via headphones (HMD 25-1, Sennheiser electronic, Wedemark, Germany) in a sound-attenuated chamber using Presentation 9.30 software (Neurobehavioral Systems, Inc., Albany, CA, USA) and a Creative Sound Blaster Audigy 2ZS sound card (Creative Technology, Ltd., Singapore).

2.4 Task and design

Participants performed a modified yes-no task. They were given written instructions to attend to the tone, to ignore the noise, and to judge the overall continuity of the tone after each stimulus by pressing one of two buttons (labelled as ‘mostly continuous’ and ‘mostly discontinuous’). Judgments had to be given within a 200-ms response interval that was indicated by a visual cross turning green at stimulus offset (Figure 1B).

Before the main experiment, participants’ ability to perform the task and participants’ perception of the interrupted tone were assessed using approximately 25 training trials. Individual thresholds for the continuity illusion were estimated by adjusting the SNR using a method of limits (Fechner, 1960). The measured thresholds corresponded to the 50 %-point on the psychometric functions. In the main experiment, these thresholds were used to create perceptually ambiguous stimuli comprising the interrupted tone, i.e., interrupted target stimuli that were approximately equally likely to be judged as ‘continuous’ or ‘discontinuous’ (Figure 1C). In addition, stimuli comprising the uninterrupted tone were created; these uninterrupted target stimuli were physically identical to the interrupted target stimuli (same SNR), except that the tone was uninterrupted (Figure 1C). On average, the SNR-threshold for the continuity illusion that was used for the target stimuli in the three experiments was equal to −6.8 ± 3.4 dB (mean ± standard deviation [SD] across all listeners).

On each trial, the interrupted or uninterrupted target stimulus was presented (with equal probability), preceded by two presentations of one adaptor stimulus (defined in the next section; Figure 1B). All stimuli were separated by a response interval; responses that fell within this interval or slightly outlasted it were considered as valid, whereas trials comprising no response or a clearly delayed response were excluded from further analysis. Trials were counterbalanced and presented in individually randomized blocks. Five, four, and six blocks (block duration: approximately five, eight, and seven minutes) were presented during experiments 1, 2, and 3, respectively. Listeners were permitted to take breaks between blocks. The local ethics committee approved the study procedures.

2.4.1 Design of experiment 1

In experiment 1, two adaptor stimuli comprising either soft noise (soft adaptors, SNR = +8 dB) or loud noise (loud adaptors, SNR = −20 dB) were presented prior to the interrupted or uninterrupted target stimulus. Based on earlier findings (Riecke et al., 2009b), the soft adaptors were expected to be rated as clearly discontinuous and bias the rating of the subsequent ambiguous target towards continuity, and vice versa for the loud adaptors. The aim of experiment 1 was to replicate these previous findings, and to determine whether the aftereffects of the adaptors on listeners’ continuity ratings of the target reflect sensory or decisional aftereffects (see Introduction). A baseline no-adaptor condition in which the adaptors were replaced by silent intervals of the same duration as the adaptors was also tested. Each of the three adaptor conditions (Figure 1D) was presented 30 times to each listener.

2.4.2 Design of experiment 2

The aim of experiment 2 was to investigate whether the previously observed aftereffects ‘transfer’ across the two ears. The same stimuli were used as in experiment 1, but they were presented differently, i.e., adaptors and target within each trial were presented either to the same ear, or to opposite ears. This variation in the presentation mode allowed us to test whether the adaptors exerted aftereffects that were lateralized to the adapted ear or the non-adapted ear. Potentially confounding effects due to left- vs. right-ear differences were avoided by counterbalancing the same-ear and opposite-ear conditions across the left and right ear. Each of the four adaptor conditions (Figure 1E) was presented 32 times to each listener.

2.4.3 Design of experiment 3

The aim of experiment 3 was to determine the extent to which the previously observed aftereffects are induced by sensory or decisional factors of the adaptors (see Introduction). The design was similar to that of experiment 1, except that all adaptor stimuli were identical to the interrupted target stimuli; thus, these adaptor stimuli were physically identical and perceptually ambiguous. Listeners’ judgments of these perceptually ambiguous adaptors were used to separate trials post hoc into two groups which are referred to as subjective adaptor conditions: the ‘continuous’ adaptor condition included only trials on which listeners had reported the ambiguous adaptor stimulus as ‘continuous’, whereas the ‘discontinuous’ adaptor condition included only trials on which listeners had reported the same ambiguous adaptor stimulus as ‘discontinuous’ (Figure 1F; for details see Statistical analysis). To make the two subjective adaptor conditions about equally likely, trials were immediately preceded by a perceptually unambiguous stimulus comprising either low SNR or high SNR with equal probability (the same stimuli referred to as ‘adaptors’ in experiment 1) and the response interval. This pre-adaptor stimulus served to bias the perception of the subsequent adaptor stimuli towards either veridical discontinuity percepts or continuity illusions. Each of these two trial types was presented 60 times to each listener.

2.5 Statistical analysis

To estimate listeners’ sensitivity and decision criterion, d’ and C were computed as follows. Listeners’ reports of the interrupted target as ‘continuous’ were considered as false alarms (FAs), whereas reports of the uninterrupted target as ‘continuous’ were labelled as hits. Following SDT (Green, 1966; Macmillan, 1991), it was presumed that listeners’ sensory noise levels were randomly distributed across trials and that this noise contributed additively and independently to the sensory representations of the different target stimuli. Hit and FA rates were transformed into z scores using the inverse-cumulative standard-normal distribution. A constant of 0.5 was added to all response counts to enable z-transformation for ceiling cases, i.e., where rates of zero or one were measured (Brown et al., 2005). The resulting z scores were then used to compute d’ and C. This was done separately for each listener. In this model, d’ reflects the sensitivity for detecting the continuity illusion, whereas C reflects the decision criterion relative to that of an unbiased maximum-likelihood listener (a C-value of zero corresponds to the criterion value of an unbiased listener).

To assess the significance of the aftereffects of the adaptors on FA rates, hit rates, d’, and C, each of these measures was compared statistically across the adaptor conditions, using SPSS 15.0 (SPSS inc., Baltimore, MD, USA). Group data exploration using Kolmogorov-Smirnov tests revealed that in most conditions the distributions of these measures did not diverge significantly from normality. Therefore, all statistical analyses were performed using parametric tests as follows. For repeated measures, ANOVAs and two-tailed paired t-tests were used, whereas for independent measures (group comparisons across experiments), two-tailed independent samples t-tests were used. To avoid inflated type-I error probabilities caused by multiple comparisons, all probability values obtained from a given sample population were corrected using the false-discovery rate (FDR; Benjamini et al., 1995). Only such FDR-corrected values are reported (see Results section) and a significance criterion α = 0.05 was used.

In the statistical analyses of experiment 3, two different datasets were considered. The first dataset was specified by sorting trials according to listeners’ judgments of the second subjective adaptor (i.e., the ambiguous stimulus that immediately preceded the target stimulus). Data from four listeners were discarded due to an insufficient number of samples (less than 15 repetitions in a subjective adaptor condition). For each of the remaining 13 listeners, the subjective adaptor conditions were counterbalanced by rejecting random trials from the condition that was reported more frequently, resulting in 42 ± 15 repetitions (mean ± SD across listeners) of each condition. The second dataset was specified by considering only the trials on which listeners had given identical judgments of the first and second subjective adaptor (i.e., the two ambiguous stimuli that immediately preceded the target stimulus). Compared to the first dataset, this second dataset comprised trials with longer adaptation intervals that matched the length of the adaptation intervals in experiments 1 and 2. Furthermore, the second dataset comprised data from only ten listeners, each including 27 ± 12 repetitions (mean ± SD across listeners) of each condition.

3 Results

In all experiments, FA rates were significantly higher for the loud adaptor (86.6 % on average) than for the soft adaptor (3.1 % on average) (all t₁₃ > 10.46, P < 0.0000002; Figure 2). This indicates that the loud adaptor was judged as ‘continuous’ far more often than the soft adaptor, as expected based on previous results (Riecke et al., 2009b).

Figure 2. Ratings of adaptor stimuli in experiments 1–3.

The graphs show proportions of continuity reports of the adaptor stimuli in experiment 1–2 (A–B) and of the pre-adaptor stimuli in experiment 3 (C). The different stimuli are indicated by the upper schematic spectrograms. Listeners judged the loud adaptor (dark gray) mostly as ‘continuous’ (false alarm [FA]) and the soft adaptor (light gray) mostly as ‘discontinuous’. All graphs show means ± SE across listeners. *** P < 0.0005.

3.1 Experiment 1

Analysis for sensory aftereffects in experiment 1 revealed that the measure of sensitivity, d’, varied significantly across the adaptor conditions (F_2,12 = 7.19, P < 0.02; Figure 3A). Significantly smaller d’-values were observed following soft adaptors than following loud adaptors (t₁₃ = 2.73, P < 0.03) or following no adaptors (t₁₃ = 3.76, P < 0.01). No significant difference was observed following loud adaptors compared to following no adaptors (t₁₃ = 0.79, P = 0.4). Since d’ was derived from comparing ratings of sounds that evoke illusory continuity vs. ratings of sounds that evoke veridical continuity, the smaller values of d’ indicate that listeners perceived illusory continuity and veridical continuity as rather similar. More specifically, the smallest values of d’ in the soft-adaptor condition indicate that after these soft adaptors, listeners had most difficulty in identifying whether targets were truly interrupted or truly uninterrupted.

Figure 3. Ratings of target stimuli in experiment 1.

A, The bar graph shows listeners’ sensitivity (as measured by d’) following the different adaptor stimuli or following silence (indicated by the upper stimulus spectrograms). Following the loud adaptors (dark gray) or silence (white), listeners could easily identify whether the target stimuli were truly interrupted or truly uninterrupted, as shown by d’-values far above zero in these conditions. Following soft adaptors (light gray), however, listeners perceived the different target stimuli as more similar, as reflected by significantly smaller d’-values in this condition. B, Significant aftereffects on listeners decision criterion (as measured by C) were observed, indicating that following soft adaptors, listeners used a more liberal decision criterion (i.e., they were more inclined to report the target as ‘continuous’). C, Following the soft adaptors, listeners rated the interrupted target mostly as ‘continuous’, and vice versa following the loud adaptors. Without prior adaptors, the interrupted target was ambiguous (i.e. FA rates were around the chance level of 50 %). D, Similar but smaller contrastive aftereffects were found for hit rates (i.e., continuity reports of the uninterrupted target). All graphs show means ± SE across listeners. *, **, ***: P < 0.05, 0.005, 0.0005; NS: not significant.

Analysis for decisional aftereffects revealed that the measure of decision criterion, C, also varied significantly across the adaptor conditions (F_2,12 = 31.49, P < 0.00003; Figure 3B). Significantly smaller values of C were observed following soft adaptors than following loud adaptors (t₁₃ = 8.12, P < 0.000008) or following no adaptors (t₁₃ = 6.05, P < 0.00005). Only following the soft adaptors, the values of C differed significantly from zero (mean: −1.19; t₁₃ = 7.44, P < 0.00001), the value corresponding to an unbiased maximum-likelihood listener. This indicates that after soft adaptors, listeners used a more liberal decision criterion (i.e., they were more inclined to report the target as continuous).

Analysis of FA rates revealed that the interrupted target was judged as ‘continuous’ more often following the soft adaptors than following the loud adaptors (t₁₃ = 7.40, P < 0.00003; Figure 3C). This finding of a contrastive aftereffect replicates previous results (Riecke et al., 2009b). Similar results were obtained from analysis of hit rates, showing that also the uninterrupted target was judged as ‘continuous’ more often following the soft adaptors than following the loud adaptors (t₁₃ = 4.21, P < 0.002; Figure 3D). This aftereffect was smaller than the aftereffect observed for the interrupted target (t₁₃ = 3.34, P < 0.006). Moreover, the uninterrupted target was generally judged as ‘continuous’ more often than the interrupted target (t₁₃ = 5.32, P < 0.0003). Nonetheless, the aftereffects for interrupted and uninterrupted targets were qualitatively similar (i.e., of the same direction), being contrastive in both cases (Figures 3C, 3D). In sum, the adaptors induced a sensory aftereffect and a decisional aftereffect (see previous sections on d’ and C) which were reflected in contrastive changes in listeners’ continuity ratings.

Additional analyses revealed that following the adaptor stimuli, FA rates were significantly different from 50 % indicating that listeners performed beyond chance level when rating the interrupted target (all t₁₃ > 4.87, P < 0.0005). In contrast, the FA rates measured in the no-adaptor condition were not significantly different from 50 % (t₁₃ = 1.13, P = 0.3), indicating that after 10 seconds of no adaptors (i.e., silence), listeners were indecisive about the continuity of the interrupted target. Statistical results from comparisons across the adaptor conditions are summarized in Figure 3 (and in Figures 4 and 5 for experiments 2 and 3; see the next two sections).

Figure 4. Ratings of target stimuli in experiment 2.

Same layout as Figure 3, except that the no-adaptor condition is omitted, and both the loud adaptor condition and the soft adaptor condition are split according to their presentation mode. The latter term indicates whether the adaptors and the target within a trial were presented to the same ear or to opposite ears. The dichotic results from experiment 2 were overall similar to the diotic results from experiment 1 (Figure 3). The sensory aftereffect of the adaptors did not depend significantly on whether the adaptors and target were presented to the same ear or to opposite ears (A), suggesting that this aftereffect ‘transferred’ across the two ears. The presentation mode affected the decisional aftereffect of the adaptors (B), indicating a stronger impact on listeners’ decision criterion for adaptors that were presented to the opposite ear than the target.

Figure 5. Ratings of target stimuli in experiment 3.

Same layout as Figure 3, except that the no-adaptor condition is omitted and the adaptor conditions were defined according to how listeners had judged the physically identical ambiguous adaptor stimuli (i.e., as either continuous [‘cont.’] or discontinuous [‘disc.’]). A, The ambiguous adaptors had no significant aftereffect on listeners’ sensitivity, indicating that they did not affect listeners’ perception of continuity. B, A significant aftereffect of the ambiguous adaptors on listeners’ decision criterion was observed, indicating that listeners were more inclined to judge the target as ‘continuous’ when they had judged the preceding adaptor as ‘continuous’ (dark gray) than when they had judged the same adaptor as ‘discontinuous’ (light gray). C, This assimilative decisional effect was also reflected in listeners’ FA rates: listeners rated the interrupted target as ‘continuous’ more often following reports of the ambiguous adaptors as ‘continuous’ than following reports of the same adaptors as ‘discontinuous’. D, For ratings of the uninterrupted target, a consistent aftereffect was observed. Overall these aftereffects of the physically identical adaptors contrasted with those of the physically different adaptors observed in experiments 1 and 2, indicating that the former were due to decisional factors rather than sensory factors (for details, see main text).

3.2 Experiment 2

Data analyses for experiment 2 confirmed the overall results of experiment 1. As before, significantly smaller values of d’ were observed following soft adaptors than following loud adaptors (Figure 4A). This sensory aftereffect (i.e., the difference in d’-values across the adaptor conditions) was found when the adaptors and interrupted target were presented to opposite ears (t₁₃ = 2.76, P < 0.05) and a consistent trend was observed when they were presented to the same ear (t₁₃ = 2.16, P < 0.08). The aftereffect further did not differ significantly between these two presentation modes (t₁₃ = 0.65, P = 0.5).

As in experiment 1, values of C varied significantly across the adaptor conditions; this was the case in the same-ear condition (t₁₃ = 3.09, P < 0.02) and also in the opposite-ear condition (t₁₃ = 3.77, P < 0.008; Figure 4B). This decisional aftereffect (i.e., the difference in C-values across the adaptor conditions) further was significantly larger in the opposite-ear condition than in the same-ear condition (t₁₃ = 2.47, P < 0.03).

As in experiment 1, the interrupted target was judged as ‘continuous’ more often following the soft adaptors than following the loud adaptors (see FA rates in Figure 4C). This contrastive aftereffect was found for both the same-ear condition (t₁₃ = 2.97, P < 0.02), and the opposite-ear condition (t₁₃ = 4.03, P < 0.005). This aftereffect (i.e., the difference in FA rates across the adaptor conditions) further differed significantly between the two presentation modes (t₁₃ = 2.40, P < 0.04), which may be explained by the observed shift in the decision criterion (see section 3.1 on C). Qualitatively similar results were observed for the uninterrupted target (see hit rates in Figure 4D). However, for the uninterrupted target, the aftereffect reached statistical significance only for the opposite-ear condition (t₁₃ = 2.75, P < 0.05), not for the same-ear condition (t₁₃ = 1.55, P = 0.1). Furthermore, these trends did not differ significantly across the two presentation modes (t₁₃ = 1.89, P = 0.1).

In sum, the results from experiment 2 confirm those of experiment 1. In addition, they reveal that the contrastive sensory aftereffect did not depend crucially on whether the adaptors and target were presented to the same ear or to opposite ears.

3.3 Experiment 3

The aftereffects that were found in experiment 3 differed strongly from the aftereffects that were observed in experiments 1 and 2. In contrast to the aftereffects in experiments 1 and 2, the aftereffects in experiment 3 were not induced by sensory differences between the adaptor stimuli, since these stimuli were physically identical.

Analysis for sensory aftereffects revealed negative results: values of d’ did not vary significantly across the subjective adaptor conditions (t₁₂ = 0.42, P = 0.7; Figure 5A). However, analysis for decisional aftereffects revealed significantly more negative C-values following adaptors that were judged as ‘continuous’ than following adaptors that were judged as ‘discontinuous’ (t₁₂ = 3.35, P < 0.006; Figure 5B). This indicates that listeners adjusted their decision criterion depending on their previous judgment; they were more inclined to judge the target as ‘continuous’ when they had judged the preceding adaptor as ‘continuous’. This contrasts with experiments 1 and 2, where listeners were more inclined to judge the target as ‘continuous’ following (soft) adaptors that they had judged mostly as ‘discontinuous’.

The criterion shift was reflected in listeners’ continuity rating data, showing that the interrupted target was judged as ‘continuous’ more often following reports of the adaptor as also ‘continuous’ than following reports of the same adaptor as ‘discontinuous’ (t₁₂ = 2.76, P < 0.02; see FA rates in Figure 5C). This ‘assimilative’ aftereffect was observed also for the uninterrupted target (t₁₂ = 2.94, P < 0.02; see hit rates in Figure 5D).

The assimilative aftereffects observed in experiment 3 differed from the aftereffects observed in experiments 1 and 2, which were contrastive in nature. Statistical group comparisons of experiment 1 vs. experiment 3 confirmed that the physically different adaptors induced significantly stronger aftereffects (defined as differences in d’ and differences in C across the adaptor conditions) than the physically identical adaptors (differences in d’: t₂₅ = 2.10, P < 0.05; differences in C: t₂₅ = 8.42, P < 0.00000001). Thus, the stronger aftereffects observed in experiments 1 and 2 could be ascribed to physical rather than subjective differences between the adaptor conditions.

The results reported above were based on a dataset in which the adaptor conditions were defined by listeners’ judgments of the second adaptor that immediately preceded the target (see section 2.5). Additional analyses of another dataset comprising only trials on which listeners had given identical judgments for the first and the second adaptor provided slightly more significant statistical outcomes, despite being based on smaller numbers of trials.

4 Discussion

4.1 Summary of results

The results of this study demonstrate a contrastive aftereffect associated with the continuity illusion: Listeners were more likely to judge a physically discontinuous tone as continuing through noise when a preceding tone contained a clear gap, compared to when the preceding tone contained a gap that was masked by a loud noise. By analyzing the data within the framework of SDT, we could determine that this aftereffect reflects a genuine change in listeners’ perception of continuity, in addition to a change in listeners’ decision strategy (experiment 1). The results further indicate that the aftereffect ‘transfers’ across the two ears, i.e., it is observed even when the preceding tone and the present tone are presented to opposite ears (experiment 2). Finally, the results reveal that also physically identical auditory stimuli can induce an aftereffect, depending on whether these stimuli were judged as continuous or discontinuous (experiment 3). In contrast to the contrastive aftereffect observed in experiments 1 and 2 using physically different stimuli, this latter aftereffect is assimilative, and it affects listeners’ decision strategy, rather than listeners’ perception of continuity.

4.2 Factors that potentially contributed to the observed aftereffects

Two types of aftereffects have been identified in cognitive tasks, one based on sensory factors and one based on decisional factors (for review, see Jones et al., 2006). Sensory factors refer to the influence of a priorly experienced sensation on the perception or judgment of a current stimulus. A prime example of this is neural adaptation (or ‘fatigue’), wherein the sensation evoked by a current stimulus is reduced when this stimulus is preceded by a similar one (e.g., Bartlett et al., 2005; Brosch et al., 1997; Ulanovsky et al., 2003; Ulanovsky et al., 2004; Werner-Reiss et al., 2006). Decisional factors refer to the influence of priorly made decisions (or categorizations) on the perception or judgment of a current stimulus. For example, when listeners are presented twice with the same stimulus, they may decide to give the same response to the second stimulus as they gave to the first stimulus (e.g., Verplanck et al., 1952).

A previous study that aimed at separating sensory and decisional factors has shown that sensory factors can induce contrastive aftereffects, whereas decisional factors may induce assimilative or contrastive aftereffects (Jones et al., 2006). Such or similar patterns have been observed in a variety of tasks focusing on sensory features in different modalities, including loudness or pitch (Jesteadt et al., 1977; Marks, 1993; Mori et al., 1995; Petzold, 1981), geometric size (Jones et al., 2006; Stewart et al., 2002), or taste intensity (Rankin, 1991; Schifferstein et al., 1992), and they are consistent with our results on auditory continuity. The sensory factors have been interpreted as influencing sensory adaptation processes, or short-term memory traces of prior stimuli (Petzold, 1981), and they may operate prior to decisional processes (Marks, 1993). Decisional factors, on the other hand, have been interpreted as influencing relatively late cognitive processes (Marks, 1993) and they may operate on guessing strategies (Petzold, 1981; Ward, 1971) or representations of response categories (Treisman, 1984).

4.3 Contrastive aftereffect of prior gaps on the continuity illusion

The aftereffect on continuity illusions that was observed in experiments 1 and 2 could be induced by sensory factors or decisional factors. These two factors were correlated in experiments 1 and 2, so their effects cannot be easily disentangled in these experiments. However, the contribution of sensory factors in experiments 1 and 2 can be inferred, at least partially, by considering the contribution of these factors in experiment 3. In contrast to experiments 1 and 2, sensory factors in experiment 3 were absent, since the adaptor stimulus remained physically constant. Thus, sensory factors could contribute in experiments 1 and 2, but not in experiment 3, which may explain why a sensory aftereffect was observed in experiments 1 and 2, but not in experiment 3. Based on these considerations, the sensory aftereffect observed in experiments 1 and 2 can be ascribed to sensory factors. The sensory properties of the adaptor that varied across the conditions in experiments 1 and 2 include the salience of the gap in the tone and the loudness of the noise (relative to that of the tone). The lack of a significant difference in sensitivity (as measured by d’) following the no-adaptor (baseline) condition versus the loud-noise condition indicates that the loudness of the noise was not a factor. In contrast, a significant aftereffect on sensitivity was observed that depended on the salience of the gap in the tone.

4.4 Potential neural mechanisms for the contrastive aftereffect on the continuity illusion

The contrastive aftereffect of prior gaps on the continuity illusion could originate in neurons that are sensitive to temporal gaps. Such ‘gap detectors’ are abundant in the central auditory system, including in the primary auditory cortex (for reviews, see Phillips et al., 2002; Recanzone et al., 2008), and neural responses to sound onsets/offsets have been associated with the continuity illusion (Heinrich et al., 2008; Husain et al., 2005; Petkov et al., 2007; Riecke et al., 2007; Riecke et al., 2009a; Shahin et al., 2009). The relevance of neural gap detectors for the continuity illusion is consistent with psychophysical data indicating that the continuity illusion depends on listeners’ failure to perceive gaps in a target sound (Bregman et al., 1977). Our current finding that the presentation of interrupted tones in soft noise (i.e., sounds with clearly audible gaps) increases the likelihood of perceiving a subsequent interrupted tone as continuous could be due to the adaptation of such gap detectors in the auditory system. This explanation could be verified in the future, for example, by comparing the response of gap-sensitive auditory neurons to interrupted sounds that were presented after sounds containing either gaps or no gaps.

Although it is conceivable that the aftereffect observed in experiments 1 and 2 originates before the auditory cortex (e.g., at an earlier processing stage), our observation that it may transfer across the two ears (experiment 2) suggests that the aftereffect does not originate before the cochlear nuclei in the brainstem - the first stage along the ascending auditory pathway at which sensory input from the left and right ear is combined (for review, see Davis, 2005). .

An interesting question is whether the suggested mechanism can also account for visual filling-in phenomena (Pessoa et al., 2003). Illusorily filled surfaces have been shown to evoke afterimages (Shimojo et al., 2001) and to influence subsequent visual discrimination thresholds even in the opposite eye (Morgan et al., 2007). Such aftereffects are thought to originate after binocular integration and to result from the adaptation of border representations in early visual cortex (for review, see Komatsu, 2006). Our results show some superficial similarities to these previous findings. However, more research is needed in order to draw conclusions about potentially analogous mechanisms in the auditory and visual systems (for similar discussions, see King et al., 2009; Petkov et al., 2010).

4.5 Assimilative aftereffect of prior continuity reports on decision criteria

The aftereffect observed in experiment 3 differed fundamentally from that observed in experiments 1 and 2: it could mainly be ascribed to decisional factors (instead of sensory factors), it was assimilative (instead of contrastive), and it only affected listeners’ decision criterion (instead of listeners’ decision criterion and continuity perception). A possible explanation for this stimulus-independent assimilative aftereffect is that listeners relaxed their decision criterion (i.e., they used a bias towards reporting continuity) following a report of continuity in order to respond consistently across presentations of similar stimuli (Shepard, 1957). Listeners seemed to follow this strategy especially in experiment 3, probably because successive stimuli were most similar in that experiment. On the other hand, in experiments 1 and 2, listeners showed clear contrastive changes in their decision criterion, probably because successive stimuli were more dissimilar in these experiments. These different observations are consistent with previous reports that the strength of assimilative decisional aftereffects may increase with the similarity of the adaptor and target stimuli, and that, when the adaptor and target differ greatly, the aftereffects can become contrastive (Jones et al., 2003; Jones et al., 2006; Marks, 1993). Unlike the sensory aftereffect observed in experiments 1 and 2, the decisional aftereffects observed in all three experiments may originate at a post-perceptual stage (Samuel, 1981) that could involve non-auditory cortical regions (for review on the neural basis of decision making, see Gold et al., 2007).

4.6 Methodological considerations

Because masked thresholds for tones in flat-spectrum noise tend to increase with frequency, the initial portion of the tone sweep (near 1 kHz) would have been more audible during noise than the final portion (near 3 kHz). However, this small difference unlikely affected our results, since the task required listeners to rate the entire stimulus rather than particular stimulus portions.

The pre-adaptors in experiment 3 were physically different; however, their aftereffects likely decayed before the target was presented more than 10 seconds later. This is supported by the results of experiment 1, which show that listeners’ performance returned to chance level (i.e., to the non-adapted case) within 10 seconds of silence.

A different problem may arise when the listener’s performance is at ceiling (i.e., FA rates and/or hit rates are equal or close to zero or one) since measures of sensitivity may become inaccurate (Brown et al., 2005). To address this potential issue in our study, we conducted supplemental statistical analyses in which we identified ceiling cases (for details, see Supplemental information). Exclusion of these cases and re-analysis of the remaining data revealed outcomes (see supplemental Figure S3) that were slightly less significant, but generally consistent with the results of experiments 1 and 3, indicating that ceiling cases, although present, constituted no major confound in our results. For experiment 2, several aftereffects did not reach significance (see supplemental Figure S3), which may be explained by insufficient statistical power due to more ceiling cases, suggesting that the sensory aftereffects observed in experiment 2 should be interpreted with some caution.

The validities of sensitivity and criterion measures utilized here rely on the assumption that listeners’ internal noise contributed equally to the sensory representations of the different stimuli. This equal-variance Gaussian model has been shown to provide a good approximation to a large number of psychophysical data and is commonly used by many researchers (Wickens, 2002). It would be interesting to investigate in future studies whether this assumption also holds in the context of illusory phenomena such as the auditory continuity illusion. This would require detailed measurements of receiver operating characteristics using a rating paradigm and stimuli similar to those used here but with a larger number of SNRs.

4.7 Previous studies of aftereffects on non-illusory auditory phenomena

A series of adaptation studies on auditory streaming by Snyder and colleagues (2009a; 2008; 2009b) have shown that the frequency separation of two alternating sound streams may exert an aftereffect on subsequent streaming. More specifically, these results have demonstrated that prior streams with large frequency separation may decrease the likelihood that subsequent streams with ambiguous frequency separation are judged as segregated (Snyder et al., 2009a; Snyder et al., 2008). It was found that such contrastive aftereffects may persist for approximately 13 seconds. The observed aftereffect was relatively frequency-unspecific and auditory-specific, since it could be induced by prior auditory streams of different frequencies, but not by visual streams (Snyder et al., 2009b). It was further found that changes in listeners’ judgments of physically identical stimuli can induce an assimilative aftereffect on subsequent judgments of streaming.

Adaptation studies on speech have shown that the categorization of phonemes depends negatively on prior speech, including illusory speech (Samuel, 1997), and even nonspeech (Holt, 2005; Holt, 2006). Thus, these contrastive aftereffects may depend not only on linguistic or phonotactic probabilities of prior speech, but more generally on the spectral probabilities of prior auditory input (Holt, 2005). Consistently, it has been suggested that such an aftereffect may arise from adaptation at early sensory rather than lexical stages (Holt, 2006; Samuel, 1997). This idea is further supported by the observation that this aftereffect may arise even when listeners are not attending to the prior adaptor sounds (Samuel et al., 1998). However, it has been shown that the aftereffect of prior speech can be explained at least partially by changes in listeners’ decision strategy (Diehl et al., 1978; Elman, 1979). In addition, it is likely that speech adaptation also involves adaptive changes in higher-level representations, given that the aftereffect influences only phonemes that belong to a well-established perceptual category (Aravamudhan et al., 2008).

These aftereffects on auditory streaming and phonemic categorization are qualitatively similar to the aftereffects on the continuity illusion observed in the current study, suggesting that a general auditory mechanism may be involved (Snyder et al., 2008). However, as indicated earlier (see section 4.2), such a mechanism may operate also in non-auditory modalities. The current results extend most of the above findings by showing that the contrastive aftereffects influence listeners’ auditory perception (in this case, the perception of auditory continuity, as measured by changes in d’) and listeners’ decisions (as measured by changes in C), whereas the assimilative aftereffects specially influence listeners’ decisions. Presumably, the perceptual aftereffects arise from neural adaptation in sensory feature-specific circuits (in this case, auditory gap detectors), whereas the decisional aftereffects arise from strategy shifts at hierarchically higher, modality-unspecific processing stages.

4.8 Conclusion

The continuity illusion serves to reduce interference by extraneous sounds in natural scenes where multiple sounds often coincide. We propose that prior sounds recalibrate this illusion to different auditory scenes by adapting neural gap detectors in central neural circuits. Specifically, exposure to gaps may increase the likelihood that subsequent sounds appear more stable during interfering sounds. This recalibration of continuity hearing could serve to optimize perceptual stability according to the acoustic dynamics of the environment, which may be especially useful for situations in which interrupting sounds are unlikely to signal ecologically relevant events, such as in crowded scenes.