Temporal Prediction in lieu of Periodic Stimulation

Abstract

Predicting not only what will happen, but also when it will happen is extremely helpful for optimizing perception and action. Temporal predictions driven by periodic stimulation increase perceptual sensitivity and reduce response latencies. At the neurophysiological level, a single mechanism has been proposed to mediate this twofold behavioral improvement: the rhythmic entrainment of slow cortical oscillations to the stimulation rate. However, temporal regularities can occur in aperiodic contexts, suggesting that temporal predictions per se may be dissociable from entrainment to periodic sensory streams. We investigated this possibility in two behavioral experiments, asking human participants to detect near-threshold auditory tones embedded in streams whose temporal and spectral properties were manipulated. While our findings confirm that periodic stimulation reduces response latencies, in agreement with the hypothesis of a stimulus-driven entrainment of neural excitability, they further reveal that this motor facilitation can be dissociated from the enhancement of auditory sensitivity. Perceptual sensitivity improvement is unaffected by the nature of temporal regularities (periodic vs aperiodic), but contingent on the co-occurrence of a fulfilled spectral prediction. Altogether, the dissociation between predictability and periodicity demonstrates that distinct mechanisms flexibly and synergistically operate to facilitate perception and action.

SIGNIFICANCE STATEMENT Temporal predictions are increasingly recognized as fundamental instruments for optimizing performance, enabling mammals to exploit regularities in the world. However, the notion of temporal predictions is often confounded with the idea of entrainment to periodic sensory inputs. At the behavioral level, it is also unclear whether perceptual sensitivity and reaction time improvements benefit the same way from temporal predictions and periodic stimulation. In two behavioral experiments on human participants, we find that periodic stimulation facilitates response readiness, whereas temporal predictions improve the precision of auditory processing. This latter effect arises regardless of the nature of temporal regularities (periodic vs aperiodic), but depends on the co-occurrence of a fulfilled spectral prediction.

Introduction

Temporal predictions are believed to play a fundamental role in the way we sample sensory information, in particular in the auditory domain (Jones, 1976; Schroeder and Lakatos, 2009; Arnal and Giraud, 2012; Nobre et al., 2012). Temporal predictions can be derived from the quasiperiodic occurrence of many natural stimuli. Such exogenous temporal regularities can be exploited to optimize perceptual analysis in sensory systems when relevant events are likely to occur. Current theories and empirical findings suggest that this enhancement is achieved by the entrainment of neuronal oscillations to periodic stimulation, which temporally modulates the excitability of task-relevant neuronal populations (Large and Jones, 1999; Schroeder and Lakatos, 2009; Cravo et al., 2013).

However, exogenous temporal predictions can also be derived from nonperiodic inputs, such as gradual change in tempo (Cope et al., 2012). This suggests that temporal predictions can be dissociated from the idea of entrainment to periodic sensory inputs. In addition, the effects of temporal predictions have been proposed to interact with other, content-based predictions (Nobre et al., 2012), whereas the consequences of periodic stimulation might not. Previous studies interchangeably used perceptual sensitivity/accuracy (Jones et al., 2002; Lawrance et al., 2014) and reaction times (Lakatos et al., 2008; Lange, 2009; Stefanics et al., 2010) to measure the effects of temporal predictions, though it is not clear whether perceptual sensitivity and reaction speed improvements result from the same mechanism. To investigate these issues, we assessed the relative influences of exogenous temporal predictions and periodic stimulation on perceptual sensitivity (d′) and response readiness (mean correct reaction times).

In two psychoacoustic experiments, we measured the ability of human participants to detect near-threshold auditory signals embedded in streams of sounds. A first experiment (Exp. 1) aimed at testing the relative behavioral influences of temporal predictions inferred from either isochronous or heterochronous temporal regularities, thereby directly dissociating exogenous temporal predictions from periodic stimulation at the input level. In a second experiment (Exp. 2), we capitalized on recent findings revealing synergistic interactions between multiple types of predictions (e.g., temporal, spatial, spectral) during sensory processing (Doherty et al., 2005; Rimmele et al., 2011; Rohenkohl et al., 2014). We investigated the relative effects of “when” and “what” cues on human behavior by manipulating independently the temporal and spectral content of auditory stimuli. The underlying rationale is that spectral manipulations should specifically magnify the behavioral consequences of exogenous temporal predictions and dissociate them from other nonprediction-based effects of periodic stimulation. By contrasting the effects of these experimental variables on perceptual sensitivity and response times, we find that periodic stimulation facilitates response readiness, whereas temporal predictions improve the quality of auditory processing. This latter effect arises regardless of the nature of temporal regularities (periodic vs aperiodic; Exp. 1), but depends selectively on the co-occurrence of a fulfilled spectral prediction (Exp. 2).

Materials and Methods

Participants.

Forty individuals (Exp. 1: 19 participants; age range, 18–50 years; 9 females; Exp. 2: 21 participants; age range, 19–59 years; 11 females) with normal audition and no history of neurological or psychiatric disorder participated in the experiments after providing informed consent according to the local ethics committee guidelines.

Apparatus.

Auditory stimuli were created on Matlab (The MathWorks) and delivered binaurally through headphones at a 44.1 kHz sampling rate using the Psychophysics-3 Toolbox. In both experiments, participants were required to attend to streams of various 100-ms-long sounds (see below) presented simultaneously with visual cues. Visual cues were displayed on a gray background (spatial resolution of 1024 by 768 pixels and vertical refresh rate of 60 Hz) and informed participants whether the simultaneous sound was a reference (Figs. 1A, 2A, white cross) or a target stimulus (Figs. 1A, 2A, red circle). On targets, participants performed a two-choice task (see below) and responded using their left and right index fingers. For each participant, the d′ sensitivity index of signal detection theory and mean correct reaction times were calculated separately for each condition. Responses with a reaction time exceeding ±2 SDs were excluded from analyses.

Figure 1.

Experiment 1. A, Each trial consisted of a stream of simultaneous auditory and visual stimuli. Reference stimuli corresponded to a 440 Hz (red) pure tone co-occurring with a white cross. A red circle specified occasional target stimuli, on which participants had to discriminate between a standard (440 Hz, red) and deviant (880 Hz, blue) pure tone. B, We investigated three conditions whereby the presented auditory streams were : 1) periodic predictable (PP; at 1.3, 2.25, 2.9, 3.45, or 3.9 Hz), 2) aperiodic predictable (AP), or 3) aperiodic unpredictable (AU). Notice that similar SOAs were used across the three conditions. C, D, Average sensitivity (C, d′) and correct reaction times (D, Correct R.T., in seconds) for the three conditions. E, Pearson correlation across participants between the effect of temporal predictability (AP vs AU)—or stimulus periodicity (PP vs AP); or both (PP vs AU)—on d′ and on Correct R.T. Error bars indicate SEM. Stars indicate significant differences (n = 19; p < 0.05). n.s., Significant absence of differences (Bayes factor, <0.33).

Figure 2.

Experiment 2. A, Each trial consisted of a stream of simultaneous auditory and visual stimuli. Reference stimuli were composed of a (pink or blue) noise co-occurring with a white cross, while a red circle specified occasional target stimuli. On them, participants had to detect the possible occurrence (50%) of a pure tone embedded in the noise. To modulate temporal predictions, reference stimuli were either presented with a fixed or jittered SOA. To modulate spectral prediction, the (blue or pink) noises presented during reference and target stimuli were either similar or different. Notice that pre-target and post-target SOAs were kept constant across conditions. B, C, Average sensitivity (B, d′) and correct reaction times (C, Correct R.T., in seconds) as a function of temporal and spectral predictions. D, Pearson correlation across participants between the (main) effect of temporal (T⁺ vs T⁻)—or spectral (S⁺ vs S⁻)—predictability on d′ and on Correct R.T. Same conventions as Figure 1 (n = 20).

Experiment 1.

The first experiment (Exp. 1) investigated how temporal regularities (whether isochronous or heterochronous) influence participants' behavior during a standard/deviant discrimination task. Auditory sequences of tones were presented in three temporal contexts: Periodic Predictable (PP), Aperiodic Predictable (AP), and Aperiodic Unpredictable (AU; Fig. 1A,B). We combined stimulus-onset asynchrony (SOA) of five possible values (255, 290, 345, 445, or 770 ms) according to our experimental conditions: In the PP condition, tones were presented at constant intervals, resulting in isochronous (temporally predictable) sequences at 1.3, 2.25, 2.9, 3.45, or 3.9 Hz. The AP condition corresponded to the ordinal temporal arrangement of SOA, alternating between decreasing and increasing intervals (Fig. 1B). Here, participants could exploit heterochronous temporal regularities to infer the temporal occurrence of forthcoming tones. The AU condition corresponded to the pseudorandomized alternation of the five SOAs, so that no focused temporal predictions could be derived from this sequence. Importantly, each SOA was always repeated twice so that pre-target and post-target stimulus intervals were identical. Finally, to make conditions comparable, all things being equal, we ensured that across conditions the same amount of each SOA value was associated with both reference and target tones. As a consequence, we pooled all SOA values in subsequent analyses.

Within each stimulus sequence, 12 target tones (six standard 440 Hz tones and six deviant 880 Hz tones; indicated by a red circle) were inserted in a pseudorandomized order every 1.5 to 6 s among the stream of reference tones (80–220 440 Hz tones; indicated by a white cross). To maximize the putative impact of predictive mechanisms (Wyart et al., 2012), tones were presented at a low signal-to-noise ratio using a continuous, noninformative background white noise presented at ∼40 dB SPL. The loudness of embedded tones was individually titrated before the experiment (using a staircase calibration procedure) to reach an accuracy of 75% in the AU condition. The average signal-to-noise ratio across participants and conditions was −16.5 dB. A total of 180 target tones per condition were presented. The experiment lasted ∼1 h.

Experiment 2.

In a second experiment (Exp. 2), we investigated the relative influence of (and possible interaction between) temporal and spectral predictions of behavior. In a detection task, participants listened to sequences of colored (either blue or pink; see below) noises and were instructed to report the presence/absence of occasionally embedded pure tones presented at threshold loudness on target stimuli. Temporal predictability depended on whether the sequence was rhythmic or not, whereas spectral predictability was manipulated by varying the spectral context, i.e., by presenting the same colored noise or not during reference and target stimuli. While we manipulated the uncertainty of the prediction signal in the time domain (informative or not), we manipulated the prediction error signal in the frequency domain (Feldman and Friston, 2010): spectral information of noise stimuli was always highly predictable but could be violated or not, in an oddball fashion. This two-by-two factorial design resulted in four experimental conditions (Fig. 2A): predictable time, predictable spectral context (T⁺S⁺); unpredictable time, predictable spectral context (T⁻S⁺); predictable time, unpredictable spectral context (T⁺S⁻); and unpredictable time, unpredictable spectral context (T⁻S⁻).

In T⁺ conditions, noises were presented at constant intervals, resulting in rhythmic (temporally predictable) sequences at 2.5 Hz (SOA, 400 ms). During T⁻ sequences, the SOAs were drawn randomly from five possible values (200, 300, 400, 500, and 600 ms). This arrangement resulted in nonrhythmic sequences, preventing participants from deriving focused temporal predictions about the occurrence of upcoming stimuli. To avoid any possible confound related to foreperiod or masking effects (Cravo et al., 2013), pre-target and post-target SOAs were kept constant (400 ms) in all conditions. In S⁺ conditions, the noise was constant throughout the trial, i.e., similar for reference and target stimuli. This allowed participants to habituate to its spectral content, i.e., form a spectral prediction, which presumably facilitates the inhibition of its masking influence during the detection task performed on target stimuli (Pfafflin, 1968). During S⁻ sequences, the noise differed between reference and target stimuli. As a consequence, whereas participants habituated to the spectral content of reference stimuli (87.5% of the stimuli), target stimuli (12.5%) were associated with an unpredictable noise.

Each sequence consisted of a stream of 96 noises presented at ∼40 dB SPL and included 12 target stimuli (indicated by a red circle) pseudorandomly presented after 5–11 stimuli. In half of the cases, a target (2027 Hz pure tone) was inserted. Across conditions, the contingency between target (and reference) stimuli and either pink or blue noises was fully balanced. Importantly, the target tone frequency was set to 2027 Hz so that both noises had an equivalent masking influence on its detection: pink and blue noises have symmetrical 1/f power density spectra that respectively decrease and increase by 3 dB per octave (with a frequency endpoint at 21,880 Hz). The target tone frequency (2027 Hz) corresponds to the intersection of pink and blue noise spectra, ensuring in principle that both noises have the same masking influence on target detection. We validated this assumption by demonstrating that the color of the noise had no detectable influence on perceptual sensitivity and response speed in any of our analyses (blue or pink: F_(1,19) < 1.1, p > 0.2; Bayes factor, <0.50). The loudness of the target tone was individually titrated before the experiment (using a staircase calibration procedure) to reach an accuracy of 75% in the T⁻S⁺ condition. The average signal-to-noise ratio across participants and conditions was −20 dB. After the calibration session, participants performed a short training session composed of 64 trials to get used to the different conditions. A total of 192 target stimuli per condition was presented. The experiment lasted ∼1 h.

Statistical procedures.

All analyses were performed at the single-subject level before applying standard parametric two-sided statistical tests at the group level (e.g., paired t tests, repeated-measures ANOVAs, Pearson correlations). One subject was excluded from Experiment 2 due to unexploitable reaction times. When necessary, to provide an unbiased decision criterion with regards to the null hypothesis, we additionally used Bayesian statistics to derive a Bayes factor. We used a standard approach to compute the Bayes factors between “null” and “effect” hypotheses at the population level using the Akaike Information Criterion (Kass and Raftery, 1995). According to this symmetric hypothesis comparison measure, a Bayes factor of ≤0.33 provides significant evidence in favor of the null hypothesis. Bayes factors were also computed for correlation coefficients (Wetzels and Wagenmakers, 2012).

Results

Temporally predictable stimulation enhances auditory sensitivity

In Experiment 1, we first estimated and compared target discrimination sensitivity d′ in the three experimental conditions: periodic predictable (PP), aperiodic predictable (AP), and aperiodic unpredictable (AU). We found d′ to vary significantly as a function of the temporal context (repeated-measures ANOVA, F_(2,36) = 4.1, p < 0.05; Fig. 1C). Post hoc comparisons indicated that perceptual sensitivity is greater both in PP and AP conditions than in the AU condition (PP vs AU: t₍₁₈₎ = 2.2, p < 0.05; AP vs AU: t₍₁₈₎ = 2.4, p < 0.05), but not different between PP and AP conditions (PP vs AP: t₍₁₈₎ < 0.1, p > 0.5). To evaluate the likelihood that this null effect corresponds to a genuine absence of difference, we computed the corresponding Bayes factor (see Materials and Methods). We obtained a Bayes factor of 0.29 for this null effect, suggesting that the “null” hypothesis (no difference in d′ between PP and AP conditions) is more likely than the alternative (significant difference between conditions). This pattern of results indicates that temporal predictions can be generated from both periodic and aperiodic stimulation regularities to enhance sensory processing. This conclusion suggests, in turn, that the resulting sensitivity enhancement does not rely decisively on the rhythmic entrainment of cortical oscillations to the average stimulation frequency, but rather on participants' ability to infer when the stimulus will occur.

Periodic stimulation reduces reaction times

The analysis of response times yielded a significant, yet different, pattern of effects (F_(2,36) = 4.0, p < 0.05; Fig. 1D): faster correct responses in PP than aperiodic (AP and AU) conditions, regardless of the presence (or absence) of temporal regularities (PP vs AP: t₍₁₈₎ = 2.2, p < 0.05; PP vs AU: t₍₁₈₎ = 2.4, p < 0.05). Interestingly, we found response times to be delayed to the same extent in both aperiodic conditions (AP vs AU: t₍₁₈₎ = 0.1, p > 0.5; Bayes factor = 0.30). Therefore, and in striking contrast with the improvement of auditory sensitivity by temporal predictions during aperiodic stimulation, response times benefit from the periodicity of stimulation, not from its predictability alone.

We further investigated whether temporal predictability and periodicity truly produce independent effects on auditory sensitivity and response times, respectively. For this purpose, we estimated the degree of correlation between the two effects across tested participants (n = 19). For every pairwise difference between experimental conditions, we observed dissociable effects of temporal predictability and periodicity on auditory sensitivity and response times (PP vs AP: r = −0.18, p > 0.2; Bayes factor = 0.23; PP vs AU: r = 0.15, p > 0.5; Bayes factor = 0.21; AP vs AU: r = 0.20, p > 0.2; Bayes factor = 0.25). This statistical independence between sensitivity and response time improvements further supports a dual process whereby stimulation periodicity quickens motor response, whereas its predictability enhances perceptual sensitivity.

Interaction between temporal and spectral predictions

In Experiment 2, we assessed the respective influences of temporal and spectral predictions—and their possible interaction—on the speed and accuracy of participants' responses. An ANOVA conducted on target detection sensitivity d′ indicated significant effects of both temporal (F_(1,19) = 6.5, p < 0.05) and spectral predictions (F_(1,19) = 25.5, p < 0.001), together with an interaction between these two factors (F_(1,19) = 11.2, p < 0.01; Fig. 2B). Post hoc comparisons showed that violated spectral prediction impairs auditory sensitivity regardless of temporal predictability (T⁺S⁺ vs T⁺S⁻: t₍₁₉₎ = 5.6, p < 0.001; T⁻S⁺ vs T⁻S⁻: t₍₁₉₎ = 4.0, p < 0.001). By contrast, temporal predictability enhances perceptual sensitivity only when spectral prediction is fulfilled, not when it is violated (T⁺S⁺ vs T⁻S⁺: t₍₁₉₎ = 3.9, p < 0.001; T⁺S⁻ vs T⁻S⁻: t₍₁₉₎ = 0.5, p > 0.5; Bayes factor = 0.34). This pattern indicates that the impact of temporal predictions on perceptual sensitivity is contingent on the fulfillment of content-based (here, spectral) predictions.

Turning to response times, we replicated the observation from Experiment 1 that temporally predictable (periodic) stimulation reduces response latencies (F_(1,19) = 13.6, p < 0.01; Fig. 2C). By contrast, we found no effect of spectral prediction on response times (F_(1,19) = 1.7, p > 0.2; Bayes factor = 0.71). Although the Bayes factor suggests a lack of statistical sensitivity for this test, the value obtained for the interaction between temporal and spectral predictions on response times (F_(1,19) = 0.1, p > 0.5; Bayes factor = 0.32) indicates that periodic stimulation accelerates responses independently of the validity of the co-occurring spectral prediction.

As in Experiment 1, we tested whether temporal and spectral predictions produce independent effects on auditory sensitivity and response times. Again, we estimated the between-participant correlation coefficient (n = 20) between the effects of each type of prediction on sensitivity and response times (Fig. 2D). In both cases, Bayes factors provided significant evidence in favor of statistically independent effects of temporal and spectral predictions on the speed and accuracy of participants' responses (temporal: r = 0.12, p > 0.5; Bayes factor = 0.20; spectral: r = 0.04, p > 0.05; Bayes factor = 0.17).

Importantly, none of the observed differences between conditions in either experiment could be accounted for by a change in speed–accuracy trade-off (i.e., joint increases or decreases in sensitivity and reaction times). These findings instead indicate a double dissociation, with perceptual sensitivity and response speed relying respectively and exclusively on temporal predictions and periodic stimulation. Additionally, they reveal that the influence of temporal predictions on perceptual sensitivity depends on the validity of spectral priors.

Discussion

The present study aimed at identifying the respective influences of periodic stimulation and temporal predictions on the speed and accuracy of perceptual decisions. Our findings primarily confirm that both perceptual sensitivity and reaction speed are improved when stimuli are presented within rhythmic streams of events (Jaramillo and Zador, 2011; Rimmele et al., 2011; Cravo et al., 2013). This result is compatible with the notion of “oscillatory entrainment,” which stipulates that sensory processing is facilitated through the periodic alignment of neuronal excitability with incoming sensory events (Schroeder and Lakatos, 2009). Although this mechanistic interpretation has the grace of simplicity and accounts for a wealth of experimental data, it leaves unaddressed several important questions that we aimed to clarify here (but see also Rohenkohl and Nobre, 2011; Breska and Deouell, 2014).

First, we dissociate empirically the effects of the temporal structure of stimuli on the two main dimensions of choice behavior: perceptual sensitivity and reaction times. Our findings show that, compared with other forms of temporally predictive cues, periodic stimulation clearly facilitates behavior at the motor response level only (Exp. 1) and irrespective of the quality (fulfilled vs violated) of the co-occurring spectral prediction (Exp. 2). We suggest that this effect might be linked to the phase-correction mechanism that operates during sensorimotor synchronization, e.g., motor tracking of auditory sequences (Repp, 2005; Repp and Su, 2013). While the neuronal substrates of this periodicity effect are still debated, it could result from the entrainment of sensorimotor loops to the periodic stimulation, inducing a cyclic “energization” of response readiness, analogous to the passive process of physical resonance observed with weakly coupled oscillators (Pantaleone, 2002).

Perceptual sensitivity, on the other hand, is modulated regardless of the form of regularity of the stimulus (Exp. 1), and depends on the quality of the co-occurring spectral prediction (Exp. 2). This effect is rather suggestive of a distinct, predictive process. We propose that a single mechanism might support predictions based on any temporal regularity without requiring stimulus-based oscillatory entrainment. Such a mechanism provides a parsimonious explanation to jointly account for the seemingly disparate effects of temporal predictions inferred from isochronous (Lakatos et al., 2008) or heterochronous streams of events (Exp. 1; Cope et al., 2012), symbolic cues (Nobre, 2001), or hazard functions (Cui et al., 2009). One candidate is the attention-dependent “period-correction” mechanism observed during sensorimotor synchronization (Repp, 2005; Repp and Su, 2013), which might (transiently) align the phase of ongoing activity so that predicted events coincide with the optimal processing state (Schroeder and Lakatos, 2009).

We observed a main effect of spectral manipulation on perceptual sensitivity in Experiment 2 (Fig. 2B), which suggests that participants were forming a spectral prediction leading to neural adaptation to the background noise at the sensory level. This resulted in a weakened influence of the noise on target detection in the S⁺ (compared with S⁻) condition (Pfafflin, 1968). In addition to this, we observed that the influence of temporal predictions is cancelled when spectral prediction error occurs (Exp. 2). This suggests that spectral priors dominate over temporal ones in our paradigm (Prince et al., 2009), which supports the idea of a hierarchy of predictive filters in sensory cortices (Rohenkohl et al., 2014).

At the neurophysiological level, we suggest that predictive timing mechanisms might be instantiated by the motor system. The motor system is automatically recruited during temporal processing (Chen et al., 2008; Fujioka et al., 2012), a result compatible with the concept of active sensing in the auditory domain (Arnal, 2012; Morillon et al., 2015). According to this theory, efferent (motor) signals might be used to predictively improve sensory processing in time. These copies of real or simulated motor commands can take multiple forms commensurate with the repertoire of movements, thereby offering a flexible mechanism to encode various kinds of temporal predictions.