Periodicity versus Prediction in Sensory Perception

The importance of temporal expectation for sensory perception has been demonstrated across diverse paradigms and multiple modalities. Overall, the findings are consistent: temporal expectation results in greater encoding precision, higher perceptual sensitivity, and decreased response times during behavioral tasks. Temporal expectation itself can take several forms. “Controlled” expectation arises when a symbolic cue indicates that a stimulus will occur at a particular time in the future (Rohenkohl et al., 2012). Temporal expectations also arise through rhythmic sensory stimulation (McAuley and Jones, 2003). Finally, a form of “automatic” temporal expectation builds up over time if a sensory event is expected but precisely when it will occur is unknown (Nobre et al., 2007).

By designing a task that explores the space between symbolic and rhythmic expectations, Morillon et al. (2016) investigated whether the advantages conferred by temporal expectation, specifically the enhancement of perceptual sensitivity and facilitation of motor responses, arise as a result of temporal prediction in general, or are specific to periodic stimulation. In their first experiment, stimulus sequences consisted of 12 “target tones” (six 880 Hz deviants, six 440 Hz standards) embedded pseudorandomly every 1.5–6 s in a stream of 440 Hz “reference tones,” and listeners were tasked with identifying deviant target tones. Tone sequences were embedded in continuous white noise and presented in three different temporal contexts. In the periodic predictable (PP) context, tone onsets were equally spaced at one of five chosen values of stimulus-onset asynchrony (SOA; 255, 290, 345, 445, 770 ms). In the aperiodic predictable (AP) condition, the five SOAs were arranged ordinally such that they alternated between progressively increasing and decreasing intervals. Listeners could therefore exploit the pattern of time intervals to form temporal predictions about the onset of the next tone even though the stream of tones itself was not isochronous. Finally, in an aperiodic unpredictable (AU) condition, SOAs were chosen in a pseudorandom manner. Visual cues presented simultaneously on a gray background informed participants whether a given sound was a reference (white cross; ∼92% of the time) or a target (red circle; ∼8% of the time).

The authors found that performance, as measured by d′ (a criterion-free measure of perceptual sensitivity derived from signal detection theory), did not differ significantly between the two predictable conditions (PP and AP), but was worse in the unpredictable condition (AU). In contrast, reaction times were fastest in the periodic condition (PP), and were not significantly different between the two aperiodic conditions (AP and AU).

The expectations manipulated in this experiment were purely temporal, and the authors went one step further in a second experiment to examine the main effects and possible interactions between predictability of temporal and spectral, or frequency-based, features. In this paradigm, reference and target stimuli were bursts of colored (either blue or pink) noise, and listeners were tasked with detecting a pure tone (“target tone”) on half of the target stimuli (“target noise”). Colored noises had symmetrical 1/f power density spectra that either increased (blue) or decreased (pink) by 3 dB per octave and intersected at 2027 Hz, the frequency of the target tone. The authors manipulated temporal (T) and spectral (S) predictability in a balanced factorial paradigm (T⁺S⁺, T⁺S⁻, T⁻S⁺, T⁻S⁻, where + and − denote predictable and unpredictable conditions, respectively). In T⁺ conditions, SOA was fixed at 400 ms, and in T⁻ conditions, SOA was drawn pseudorandomly from one of five possible values (200, 300, 400, 500, 600 ms). Reference and target noises were identical in the S⁺ context, and of contrasting colors in S⁻. The same visual cues described previously indicated whether a given noise was a reference (87.5% of the time) or a target (12.5% of the time).

Consistent with their results from Experiment 1, the authors found that periodic stimulation (T⁺) yielded faster reaction times. In addition, they showed that this was the case regardless of whether target and reference noises were of the same color (S⁺ or S⁻). In contrast, auditory sensitivity was facilitated only by periodic stimulation (T⁺) if target noises were also spectrally predictable (S⁺). Spectral unpredictability (S⁻) impaired auditory sensitivity regardless of temporal predictability (T⁺ or T⁻).

The major contribution made by this study is the behavioral dissociation between response speed and perceptual acuity, the former resulting from periodic stimulation only, and the latter from temporal expectation more generally. The implications of these findings are potentially far-reaching and will be discussed below, but first it is worth noting a few methodological issues. First, the major finding of the double dissociation could have been even more compelling if Experiment 2 had included an AP condition because, as it stands, the T+ condition here is both predictable and periodic. This underscores a more general issue that the results from Experiments 1 and 2 are difficult to compare directly, their paradigms differing to the point that one could consider Experiment 1 a deviant “detection” task (detecting an 880 Hz deviant tone in a stream of 440 Hz standard tones) and Experiment 2 a “discrimination” task (discriminating a 2027 Hz target tone in target-colored noise). An alternative paradigm using a common stimulus and task in a 2 × 3 factorial design with two kinds of predictions (temporal and spectral) and three levels of predictability (PP, AP, and AU for both features) could have yielded more clearly interpretable main effects and interactions. Finally, the justification for using an audiovisual task with visual cues for the target is unclear, and somewhat muddles the interpretation of the results because the two modalities might contribute differentially to the formation of the temporal expectations probed in this study.

These methodological criticisms aside, the effects reported, though relatively small, are significant and imply that temporal expectation through periodic stimulation affects motor preparation in a way that other forms of temporal expectation do not. The role of motor regions in rhythmic timing is well established (Grahn and Brett, 2007). Specifically, functional imaging studies have shown that striato-thalamo-cortical areas are more active during temporally regular sequences, whereas olivocerebellar circuits are more strongly activated during irregular sequences (Teki et al., 2011, 2012). This is consistent with neurophysiological recordings that have identified beta-band oscillations in the striatum as a putative marker for temporal regularity (Bartolo et al., 2014). Striatal beta activity represents a general sequence-initiation signal but persists subsequently only for regular sequences and is suppressed for irregular ones (Bartolo and Merchant, 2015). An open question is whether the motor system is recruited only when sensory stimuli are strictly periodic. Though Morillon et al. (2016) show no motor facilitation in the aperiodic predictable condition, they chose a form of aperiodicity that is also arrhythmic. An interesting follow-up experiment would be to explore rhythmic (but aperiodic) patterns, such as those used by Nozaradan et al. (2011), which have been shown to entrain cortical activity, but whose interaction with the motor system is not fully understood.

In Experiment 2, a possible neural mechanism for the establishment of spectral predictions at the sensory level that was not particularly emphasized in the current study is stimulus-specific adaptation (SSA), which manifests as a suppression of neural responses to a repeated stimulus (Ulanovsky et al., 2003, 2004). SSA represents a potential single-neuron correlate of the mismatch negativity and refers to the observation that neural responses are typically stronger for “deviant” (ie, relatively rare or unexpected) acoustic events than for commonly occurring and therefore highly predictable “standard” events (for review, see Näätänen et al., 2007; Nelken and Ulanovsky, 2007). In Experiment 2, the target tone in the S⁺ context is a clear deviant, potentially evoking a strong mismatch signal. In the S⁻ context, however, the color of the noise potentially represents the most salient deviant, which could have masked the target tone and resulted in the lower d′ observed. Thus, SSA could possibly account for why spectral predictability enhanced perceptual sensitivity in the S⁺ context relative to S⁻. The formation of spectral predictions may therefore be explained by neural adaptation, a process that may or may not interact with the formation of temporal predictions through oscillatory entrainment or active sensing mechanisms discussed below.

Of special note is the authors' finding that perceptual sensitivity in the spectrally predictable context is further facilitated by periodic stimulation, despite the observed dissociation between perceptual acuity and response speed. The authors interpret this result as suggestive of a hierarchy of predictive filters in sensory cortices where predictions about “what” will occur supersede predictions about “when” something will occur, suggesting that “what” and “when” predictions interact to shape perception. Indeed the suppression of neural activity in response to repeated sounds is strongest when the sounds occur at regular intervals (Costa-Faidella et al., 2011). Furthermore, the activation of a periodicity detector model based on low-frequency modulation filters correlates strongly with perceptual acuity in temporally jittered sequences (Rajendran et al., 2016). These observations are consistent with findings that entrainment of low-frequency delta oscillations to the temporal structure of an attended stimulus can boost perceptual sensitivity (Lakatos et al., 2008; Schroeder and Lakatos, 2009), and furthermore suggest that such response gain might require that elements of the attended stimulus (“what”) be spectrally predictable.

The neural substrates underlying “what” and “when” predictions are not yet fully understood, but the present findings are consistent with the prevailing idea that the motor system interacts with sensory circuits, possibly via cross-frequency coupling between delta and beta oscillations, to modulate temporal predictions for adaptive behavior (Arnal et al., 2015). The authors suggest a model of active sensing, whereby overt or covert motor signals may be used to predictively improve sensory processing in time (Morillon et al., 2015). This model is similar to the Action Simulation for Auditory Prediction hypothesis (Patel and Iversen, 2014), which proposes that simulation of rhythmic movement in motor planning regions helps the auditory system predict the timing of an upcoming musical beat. The findings by Morillon et al. (2016) and the aforementioned mechanistic hypotheses may furthermore relate to studies probing how sensorimotor circuits encode temporal structure in speech and music through rhythmic oscillations in the beta-band (Arnal, 2012; Fujioka et al., 2012; Bartolo et al., 2014, 2015; Teki, 2014; Merchant et al., 2015).