The Psychophysics of Brain Rhythms

Abstract

It is becoming increasingly apparent that brain oscillations in various frequency bands play important roles in perceptual and attentional processes. Understandably, most of the associated experimental evidence comes from human or animal electrophysiological studies, allowing direct access to the oscillatory activities. However, such periodicities in perception and attention should, in theory, also be observable using the proper psychophysical tools. Here, we review a number of psychophysical techniques that have been used by us and other authors, in successful and sometimes unsuccessful attempts, to reveal the rhythmic nature of perceptual and attentional processes. We argue that the two existing and largely distinct debates about discrete vs. continuous perception and parallel vs. sequential attention should in fact be regarded as two facets of the same question: how do brain rhythms shape the psychological operations of perception and attention?

Introduction

Neurons convey information by means of electrical signals. Due to intrinsic properties of neuronal networks (e.g., conduction delays, balance between excitation and inhibition, membrane time constants), these electrical pulses give rise to large-scale periodic fluctuations of the background electric potential, which constitute the brain “rhythms” and oscillations (Buzsaki, 2006). Some oscillations – like the “alpha” rhythm at 8–13 Hz can be seen with the naked eye on an electro-encephalographic trace (Berger, 1929), while others require sophisticated analysis methods or recordings with a higher signal-to-noise ratio (using intra-cerebral probes in animals and, more rarely, in humans). There are many theories implicating brain oscillations in the performance of particular cognitive functions such as perception (Eckhorn et al., 1988; Gray et al., 1989; Engel et al., 1991; Singer and Gray, 1995; von der Malsburg, 1995), attention (Niebur et al., 1993; Fell et al., 2003; Womelsdorf and Fries, 2007), consciousness (Koch and Braun, 1996; Gold, 1999; Engel and Singer, 2001), and memory (Lisman and Idiart, 1995; Klimesch, 1999; Kahana et al., 2001). There are also flurries of experimental studies supporting (and sometimes, invalidating) these theories based on electrophysiological measurements of brain activity (Revonsuo et al., 1997; Tallon-Baudry et al., 1997, 1998; Fries et al., 2001; Jensen et al., 2002; Gail et al., 2004; Ray and Maunsell, 2010). It is somewhat less ordinary, on the other hand, to investigate the consequences of brain oscillations using psychophysical techniques. Yet one major prediction of the above-mentioned theories is directly amenable to psychophysical experimentation: indeed, an oscillatory implementation at the neuronal level should imply that the relevant cognitive function fluctuates periodically, and such fluctuations should be measurable with standard (or slightly more sophisticated) experimental psychology techniques.

The purpose of this article is to review some of the psychophysical techniques that have been applied recently to the study of brain oscillations. In so doing, we will touch upon two classical debates in perception research. First, scientists have long theorized that oscillations could divide the continuous sequence of inputs feeding into our perceptual systems into a series of discrete cycles or “snapshots” (Pitts and McCulloch, 1947; Stroud, 1956; Harter, 1967; Allport, 1968; Sanford, 1971), but this idea is far from mainstream nowadays; we refer to this debate as “discrete vs. continuous perception” (VanRullen and Koch, 2003). Second, there is another long-standing debate known as “parallel vs. sequential attention”: does attention concentrate its resources simultaneously or sequentially when there are multiple targets to focus on? Though this discussion is generally disconnected from the topic of brain oscillations, we will argue that it is in fact germane to the previous question. The sequential attention idea – which has traditionally been the dominant one – originated with the assumption that high-level vision cannot process more than one object at a time, and must therefore shift periodically between the stimuli (Eriksen and Spencer, 1969; Treisman, 1969; Kahneman, 1973), just like our gaze must shift around because our fovea cannot fixate on multiple objects simultaneously. Interestingly, sequential attention theories require a (possibly irregular) cyclic process for disengaging attention at the current target location and engaging it anew. It is easy to see – although little noticed in the literature – that discrete perception is merely an extension of this idea, obtained by assuming that the “periodic engine” keeps running, even when there is only one stimulus to process. This connection between the two theories will be a recurring thread in the present narrative.

Of course, the primary source of evidence about these topics is (and should remain) based on neurophysiological recordings, which provide direct measurements of the neuronal oscillations. For example, we have recently reviewed our past EEG work on the perceptual correlates of ongoing oscillations, with a focus on linking these oscillations to the notion of discrete perception (VanRullen et al., 2011). Another example is a recent study in macaque monkeys revealing that oscillatory neuronal activity in the frontal eye field (FEF, a region involved in attention and saccade programming) reflects the successive cycles of a clearly sequential attentional exploration process during visual search (Buschman and Miller, 2009). In this review we focus on purely psychophysical methods, not because they are better than direct neurophysiological measurements, but because they also inform us about psychological and perceptual consequences of the postulated periodicities. However, due to the inherent temporal limitations of most psychophysical methods, it should be kept in mind that in practice this approach is probably restricted to the lower end of the frequency spectrum, i.e., oscillations in the delta (0–4 Hz), theta (4–8 Hz), alpha (8–14 Hz), and possibly beta (14–30 Hz) bands. Higher-frequency oscillations (e.g., gamma: 30–80 Hz) may still play a role in sensory processing, but they are generally less amenable to direct psychophysical observation.

There exist many psychophysical paradigms designed to test the temporal limits of sensory systems but that do not directly implicate periodic perception or attention, because their results can also be explained by temporal “smoothing” or integration, in the context of a strictly continuous model of perception (Di Lollo and Wilson, 1978). These paradigms are nonetheless useful to the discrete argument because they constrain the range of plausible periodicities: for example, temporal numerosity judgments or simultaneity judgments indicate that the temporal limit for individuating visual events is only around 10 events per second (White et al., 1953; Lichtenstein, 1961; White, 1963; Holcombe, 2009), suggesting a potential oscillatory correlate in the alpha band (Harter, 1967). We will not develop these results further here, focusing instead on paradigms that unequivocally indicate periodic sampling of perception or attention. Similarly, although some psychophysical studies have demonstrated that perception and attention can be entrained to a low-frequency rhythmic structure in the stimulus sequence (Large and Jones, 1999; Mathewson et al., 2010), we will only concentrate here on studies implicating intrinsic perceptual and attentional rhythms (i.e., rhythms that are not present in the stimulus). We will see that progress can be made on the two debates of discrete vs. continuous perception and sequential vs. parallel attention by addressing them together rather than separately.

Rhythmic Sampling of a Single Stimulus: Discrete vs. Continuous Perception

Suppose that a new stimulus suddenly appears in your visual field, say a red light at the traffic intersection. For such a transient onset, a sequence of visual processing mechanisms from your retina to your high-level visual cortex will automatically come into play, allowing you after a more or less fixed latency to “perceive” this stimulus, i.e., experience it as part of the world in front of you. Hopefully you should then stop at the intersection. What happens next? For as long as the stimulus remains in the visual field, you will continue to experience it. But how do you know it is still there? You might argue that if it were gone, the same process as previously would now signal the transient offset (together with the onset of the green light), and you would then recognize that the red light is gone. But in-between those two moments, you did experience the red light as present – did you only fill in the mental contents of this intervening period after the green light appeared? This sounds unlikely, at least if your traffic lights last as long as they do around here. Maybe the different stages of your visual system were constantly processing their (unchanged) inputs and feeding their (unchanged) outputs to the next stage, just in case the stimulus might happen to change right then – a costly but plausible strategy. An intermediate alternative would consist in sampling the external world periodically to verify, and potentially update, its contents; the period could be chosen to minimize metabolic effort, while maximizing the chances of detecting any changes within an ecologically useful delay (e.g., to avoid honking from impatient drivers behind you when you take too long to notice the green light). These last two strategies are respectively known as continuous and discrete perception.

The specific logic of the above example may have urged you to favor discrete perception, but the scientific community traditionally sides with the continuous idea. It has not always been so, however. In particular, the first observations of EEG oscillations in the early twentieth century (Berger, 1929), together with the simultaneous popularization of the cinema, prompted many post-war scientists to propose that the role of brain oscillations could be to chunk sensory information into unitary events or “snapshots,” similar to what happens in the movies (Pitts and McCulloch, 1947; Stroud, 1956; Harter, 1967). Much experimental research ensued, which we have already reviewed elsewhere (VanRullen and Koch, 2003). The question was never fully decided, however, and the community's interest eventually faded. The experimental efforts that we describe in this section all result from an attempt to follow up on this past work and revive the scientific appeal of the discrete perception theory.

Periodicities in reaction time distributions

Some authors have reasoned that if the visual system samples the external world discretely, the time it would take an observer to react after the light turns green would depend on the precise moment at which this event occurred, relative to the ongoing samples: if the stimulus is not detected within one given sample then the response will be delayed at least until the next sampling period. This relation may be visible in histograms of reaction time (RT). Indeed, multiple peaks separated by a more or less constant period are often apparent in RT histograms: these multimodal distributions have been reported with a period of approximately 100 ms for verbal choice responses (Venables, 1960), 10–40 ms for auditory and visual discrimination responses (Dehaene, 1993), 10–15 ms for saccadic responses (Latour, 1967), 30 ms for smooth pursuit eye movement initiation responses (Poppel and Logothetis, 1986). It must be emphasized, however, that an oscillation can only be found in a histogram of post-stimulus RTs if each stimulus either evokes a novel oscillation, or resets an existing one. Otherwise (and assuming that the experiment is properly designed, i.e., with unpredictable stimulus onsets), the moment of periodic sampling will always occur at a random time with respect to the stimulus onset; thus, the peaks of response probability corresponding to the recurring sampling moments will average out, when the histogram is computed over many trials. In other words, even though these periodicities in RT distributions are intriguing, they do not unambiguously demonstrate that perception samples the world periodically – for example, it could just be that each stimulus onset triggers an oscillation in the motor system that will subsequently constrain the response generation process. In the following sections, we present other psychophysical methods that can reveal perceptual periodicities within ongoing brain activity, i.e., without assuming a post-stimulus phase reset.

Double-detection functions

As illustrated in the previous section, there is an inherent difficulty in studying the perceptual consequences of ongoing oscillations: even if the pre-stimulus oscillatory phase modulates the sensory processing of the stimulus, this pre-stimulus phase will be different on successive repetitions of the experimental trial, and the average performance over many trials will show no signs of the modulation. Obviously, this problem can be overcome if the phase on each trial can be precisely estimated, for example using EEG recordings (VanRullen et al., 2011). With purely psychophysical methods, however, the problem is a real challenge.

An elegant way to get around this challenge has been proposed by Latour (1967). With this method, he showed preliminary evidence that visual detection thresholds could fluctuate along with ongoing oscillations in the gamma range (30–80 Hz). The idea is to present two stimuli on each trial, with a variable delay between them, and measure the observer's performance for detecting (or discriminating, recognizing, etc.) both stimuli: even if each stimulus's absolute relation to an ongoing oscillatory phase cannot be estimated, the probability of double-detection should oscillate as a function of the inter-stimulus delay (Figure 1). In plain English, the logic is that when the inter-stimulus delay is a multiple of the oscillatory period, the observer will be very likely to detect both stimuli (if they both fall at the optimal phase of the oscillation) or to miss both stimuli altogether (if they both fall at the opposite phase); on the other hand, if the delay is chosen in-between two multiples of the oscillatory period, then the observer will be very likely to detect only one of the two stimuli (if the first stimulus occurs at the optimal phase, the other will fall at the opposite, and vice-versa).

Figure 1.

Double-detection functions can reveal periodicities even when the phase varies across trials. (A) Protocol. Let us assume that the probability of detecting a stimulus (i.e., the system's sensitivity) fluctuates periodically along with the phase of an ongoing oscillatory process. By definition, this process bears no relation with the timing of each trial, and thus the phase will differ on each trial. On successive trials, not one but two stimuli are presented, with a variable delay between them. (B) Expected results. Because the phase of the oscillatory process at the moment of stimulus presentation is fully unpredictable, the average probability of detecting each stimulus as a function of time (using an absolute reference, such as the trial onset) will be constant (left). The probability of detecting the second stimulus will also be independent of the time elapsed since the first one (middle). However, the probability of detecting both stimuli (albeit smaller) will oscillate as a function of the delay between them, and the period of this oscillation will be equal to the period of the original ongoing oscillatory process (adapted from Latour, 1967).

More formally, let us assume that the probability of measuring our psychological variable ψ (e.g., target detection, discrimination, recognition, etc.) depends periodically (with period 2π/ω) on the time of presentation of the stimulation s; to a first approximation this can be noted:

$$p(\psi =1|s=1)=p_0\cdot (1+a\cdot \sin (\omega \cdot t))$$ (1)

where p₀ is the average expected measurement probability, and a is the amplitude of the periodic modulation. Since the time t of stimulation (with respect to the ongoing oscillation) may change for different repetitions of the measurement, only p₀ can be measured with classical trial averaging methods (i.e., the “sine” term will average out to a mean value of zero). However, if two identical stimulations are presented, separated by an interval δt, the conditional probability of measuring our psychological variable twice can be shown to be (there is no room here, unfortunately, for the corresponding mathematical demonstration):

$$p(\psi =2|s=2)=p{0}^2\cdot (1+(((a^2)/2)\cdot \cos (\omega \cdot \delta t)))$$ (2)

The resulting probability only depends on the interval δt (chosen by the experimenter), and thus does not require knowledge of the exact oscillatory phase on every trial. This means that, using double stimulations and double-detection functions, one can derive psychophysically the rate ω of the periodic process, and its modulation amplitude a (Figure 1).

In practice, unfortunately, this method is not as easy to apply as it sounds. One important caveat was already mentioned by Latour: the inter-stimulus delay must be chosen to be long enough to avoid direct interactions between the two stimuli (e.g., masking, apparent motion, etc.). This is because the mathematical derivation of Eq. 2 assumes independence between the detection probabilities for the two stimuli. To ensure that this condition is satisfied, the stimuli should be separated by a few 100 ms (corresponding to the integration period for masking or apparent motion); on the other hand, this implies that several oscillatory cycles will occur between the two stimuli, and many external factors (e.g., phase slip, reset) can thus interfere and decrease the measured oscillation. This in turn suggests that the method may be more appropriate for revealing low-frequency oscillations than high-frequency ones (e.g., gamma). Another important limitation is that the magnitude of the measured oscillation in the double-detection function (2) is squared, compared to the magnitude of the original perceptual oscillation. Although this is not a problem if the perceptual oscillation is strong (i.e., the square of a number close to 1 is also close to 1), it can become troublesome when the original perceptual oscillation is already subtle (e.g., for a 20% modulation of the visual threshold, one can only expect a 4% modulation in the double-detection function). Altogether, these limitations may explain why Latour's results have, so far, not been replicated or extended.

Temporal aliasing: The wagon wheel illusion

Engineers know that any signal sampled by a discrete or periodic system is subject to potential “aliasing” artifacts (Figure 2): when the sampling resolution is lower than a critical limit (the “Nyquist rate”) the signal can be interpreted erroneously. This is true, for instance, when a signal is sampled in the temporal domain (Figure 2A). When this signal is a periodic visual pattern in motion, aliasing produces a phenomenon called the “wagon wheel illusion” (Figure 2B): the pattern appears to move in the wrong direction. This is often observed in movies or on television, due to the discrete sampling of video cameras (generally around 24 frames per second). Interestingly, a similar perceptual effect has also been reported under continuous conditions of illumination, e.g., daylight (Schouten, 1967; Purves et al., 1996; VanRullen et al., 2005b). In this case, however, because no artificial device is imposing a periodic sampling of the stimulus, the logical conclusion is that the illusion must be caused by aliasing within the visual system itself. Thus, this “continuous version of the wagon wheel illusion” (or “c-WWI”) has been interpreted as evidence that the visual system samples motion information periodically (Purves et al., 1996; Andrews et al., 2005; Simpson et al., 2005; VanRullen et al., 2005b).

Figure 2.

Temporal aliasing. (A) Concept. Sampling a temporal signal using too low a sampling rate leads to systematic errors about the signal, known as “aliasing errors.” Here, the original signal is periodic, but its frequency is too high compared with the system's sampling rate (i.e., it is above the system's “Nyquist” frequency, defined as half of its sampling rate). As a result, the successive samples skip ahead by almost one full period of the original oscillation: instead of normally going through the angular phases of zero, π/2, π, 3π/2, and back to zero, the successive samples describe the opposite pattern, i.e., zero, 3π/2, π, π/2, and so on. The aliasing is particularly clear in the complex domain (right), where the representations of the original and estimated signals describe circles in opposite directions. (B) The wagon wheel illusion. When the original signal is a periodically moving stimulus, temporal aliasing transpires as a reversal of the perceived direction. This wagon wheel illusion is typically observed in movies due to the discrete sampling of video cameras. The continuous version of this wagon wheel illusion (c-WWI) differs in that it occurs when directly observing the moving pattern in continuous illumination; in this case, it has been proposed that reversed motion indicates a form of discrete sampling occurring in the visual system itself.

There are many arguments in favor of this “discrete” interpretation of the c-WWI. First, the illusion occurs in a very specific range of stimulus temporal frequencies, compatible with a discrete sampling rate of approximately 13 Hz (Purves et al., 1996; Simpson et al., 2005; VanRullen et al., 2005b). As expected according to the discrete sampling idea, this critical frequency remains unchanged when manipulating the spatial frequency of the stimulus (Simpson et al., 2005; VanRullen et al., 2005b) or the type of motion employed, i.e., rotation vs. translation motion, or first-order vs. second-order motion (VanRullen et al., 2005b). EEG correlates of the perceived illusion confirm these psychophysical findings and point to an oscillation in the same frequency range around 13 Hz (VanRullen et al., 2006; Piantoni et al., 2010). Altogether, these data suggest that (at least part of) the motion perception system proceeds by sampling its inputs periodically, at a rate of 13 samples per second.

There are, of course, alternative accounts of the phenomenon. First, it is noteworthy that the illusion is not instantaneous, and does not last indefinitely, but it is instead a bistable phenomenon, which comes and goes with stochastic dynamics; such a process implies the existence of a competition between neural mechanisms supporting the veridical and the erroneous motion directions (Blake and Logothetis, 2002). Within this context, the debate centers around the source of the erroneous signals: some authors have argued that they arise not from periodic sampling and aliasing, but from spurious activation in low-level motion detectors (Kline et al., 2004; Holcombe et al., 2005) or from motion adaptation processes that would momentarily prevail over the steady input (Holcombe and Seizova-Cajic, 2008; Kline and Eagleman, 2008). We find these accounts unsatisfactory, because they do not seem compatible with the following experimental observations: (i) the illusion is always maximal around the same temporal frequency, whereas the temporal frequency tuning of low-level motion detectors differs widely between first and second-order motion (Hutchinson and Ledgeway, 2006); (ii) not only the magnitude of the illusion, but also its spatial extent and its optimal temporal frequency – which we take as a reflection of the system's periodic sampling rate – are all affected by attentional manipulations (VanRullen et al., 2005b; VanRullen, 2006; Macdonald et al., under review); in contrast, the amount of motion adaptation could be assumed to vary with attentional load (Chaudhuri, 1990; Rezec et al., 2004), but probably not the frequency tuning of low-level motion detectors; (iii) motion adaptation itself can be dissociated from the wagon wheel illusion using appropriate stimulus manipulations; for example, varying stimulus contrast or eccentricity can make the motion aftereffects (both static and dynamic versions) decrease while the c-WWI magnitude increases, and vice-versa (VanRullen, 2007); (iv) finally, the brain regions responsible for the c-WWI effect, repeatedly identified in the right parietal lobe (VanRullen et al., 2006, 2008; Reddy et al., 2011), point to a higher-level cause than the mere adaptation of low-level motion detectors.

Disentangling the neural mechanisms of the continuous wagon wheel illusion could be (and actually, is) the topic of an entirely separate review (VanRullen et al., 2010). To summarize, our current view is that the reversed motion signals most likely originate as a form of aliasing due to periodic temporal sampling by attention-based motion perception systems, at a rate of ∼13 Hz; the bistability of the illusion is due to the simultaneous encoding of the veridical motion direction by other (low-level, or “first-order”) motion perception systems. The debate, however, is as yet far from settled. At any rate, this phenomenon illustrates the potential value of temporal aliasing as a paradigm to probe the discrete nature of sensory perception.

Other forms of temporal aliasing

The sampling frequency evidenced with the c-WWI paradigm may be specific to attention-based motion perception mechanisms. It is only natural to try and extend the temporal aliasing methodology to perception of other types of motion, to perception of visual features other than motion or to perception in sensory modalities other than vision. If evidence for temporal aliasing could be found in these cases, the corresponding sampling frequencies may then be compared to one another and further inform our understanding of discrete perception. Is there a single rhythm, a central (attentional) clock that samples all sensory inputs? Or is information from any single channel of sensory information read out periodically at its own rate, independently from other channels? While the first proposition reflects the understanding that most have of the theory of discrete perception (Kline and Eagleman, 2008), the latter may be a much more faithful description of reality; additionally, the sampling rate for a given channel may vary depending on task demands and attentional state, further blurring intrinsic periodicities.

The simple generic paradigm which we advocate to probe the brain for temporal aliasing is as follows. Human observers are presented with a periodic time-varying input which physically evolves in an unambiguously defined direction; they are asked to make a two-alternative forced choice judgment on the direction of evolution of this input, whose frequency is systematically varied by the experimenter across trials. A consistent report of the wrong direction at a given input frequency may be taken as a behavioral correlate of temporal aliasing, and the frequencies at which this occurs inform the experimenter about the underlying sampling frequency of the brain for this input.

Two main hurdles may be encountered in applying this paradigm. The first one lies in what should be considered a “consistent” report of the wrong direction. Clearly, for an engineered sampling system, one can find input frequencies at which the system will always output the wrong direction. For a human observer, however, several factors could be expected to lower the tendency to report the wrong direction, even at frequencies that are subject to aliasing: measurement noise, the potential variability of the hypothetical sampling frequency over the duration of the experiment, and most importantly, the potential presence of alternate sources of information (as in the c-WWI example, where competition occurs between low-level and attention-based motion systems). In the end, even if aliasing occurs, it may not manifest as a clear and reliable percept of the erroneous direction, but rather as a subtle increase of the probability of reporting the wrong direction at certain frequencies. Recently, we proposed a method to evaluate the presence of aliasing in psychometric functions, based on model fitting (Dubois and VanRullen, 2009). (A write-up of this method and associated findings can be accessed at http://www.cerco.ups-tlse.fr/∼rufin/assc09/). Results of a 2-AFC motion discrimination experiment were well explained by considering two motion sensing systems, one that functions continuously and one that takes periodic samples of position to infer motion. These two systems each give rise to predictable psychometric functions with few parameters, whose respective contributions to performance can be inferred by model fitting. Evidence for a significant contribution of a discrete process sampling at 13 Hz was found – thus confirming our previous conclusions from the c-WWI phenomenon. Furthermore, the discrete process contributed more strongly to the perceptual outcome when motion was presented inter-ocularly, than binocularly; this is compatible with our postulate that discrete sampling in the c-WWI is a high-level effect, since inter-ocular motion perception depends on higher-level motion perception systems (Lu and Sperling, 2001).

The second pitfall is that the temporal resolution for discriminating the direction of the time-varying input under consideration should be at least as good as the hypothesized sampling frequency. If the psychometric function is already at chance at the frequency where aliasing is expected to take place, this aliasing will simply not be observed – whether the perceptual process relies on periodic sampling or not. Our lab learned this the hard way: many of the features that we experimented with so far, besides luminance and contrast-defined motion, can only be discriminated at low-temporal frequencies – they belong to Holcombe's “seeing slow” category (Holcombe, 2009). For example, we hypothesized that motion stimuli designed to be invisible to the first-order motion perception system, such as stereo-defined motion (Tseng et al., 2006), would yield maximal aliasing as there is no other motion perception system offering competing information. Unfortunately, these stimuli do not yield a clear percept at temporal frequencies beyond 3–4 Hz, meaning that any aliasing occurring at higher frequencies would have escaped our notice. The “motion standstill” phenomenon reported by Lu and colleagues (Lu et al., 1999; Tseng et al. 2006) with similar stimuli at frequencies around 5 Hz remains a potential manifestation of temporal aliasing, although we have not satisfactorily replicated it in our lab yet. We also hypothesized that binding of spatially distinct feature conjunctions, such as color and motion, could rely on sequential attentional sampling of the two features (Moutoussis and Zeki, 1997), and should thus be subject to aliasing. Again, we were disappointed to find that performance was at chance level at presentation rates higher than 3–4 Hz (Holcombe, 2009), precluding further analysis. We also attempted to adapt the wagon wheel phenomenon to the auditory modality. Here, perception of sound source motion (e.g., a sound rotating around the listener) also appeared limited to about 3 Hz (Feron et al., 2010). We then reasoned that frequency, rather than spatial position, was the primary feature for auditory perception, and designed periodic stimuli that moved in particular directions in the frequency domain – so-called Shepard or Risset sequences (Shepard, 1964). Again, we found that the direction of these periodic frequency sweeps could not be identified when the temporal frequency of presentation was increased beyond 3–4 Hz.

In sum, although temporal aliasing is, in principle, a choice paradigm to probe the rhythms of perception, our attempts so far at applying this technique to other perceptual domains than motion (the c-WWI) have been foiled by the strict temporal limits of the corresponding sensory systems. What we can safely conclude is that, if discrete sampling exists in any of these other perceptual domains, it will be at a sampling rate above 3–4 Hz. We have not exhausted all possible stimuli and encourage others to conduct their own experiments. There are two faces to the challenge: finding stimuli that the brain “sees fast” enough, and using an appropriate model to infer the contribution of periodic sampling to the psychometric performance (in case other sources of information and sizeable variability across trials should blur the influence of discrete processes).

Rhythmic Sampling of Multiple Stimuli: Sequential vs. Parallel Attention

Let us return to our previous hypothetical situation. Now you have passed the traffic lights and driven home, and you turned on the TV to find out today's lottery numbers. There are a handful of channels that can provide this information at this hour, so you go to “multi-channel” mode to monitor them simultaneously. The lottery results are not on, so you will wait until any channel shows them. How will you know which one? You try to process all channels at once, but their contents collide and confuse you. What if one of them shows the numbers but you notice it too late? By focusing on a single channel you would be sure not to miss the first numbers, but then what if you picked the wrong channel? In such a situation, it is likely that you will switch your attention rapidly between the different candidate channels until you see one that provides the required information. Your brain often faces the same problem when multiple objects are present in the visual field and their properties must be identified, monitored or compared.

A long-standing debate

Visual search

Whether your brain simultaneously and continuously shares its attentional resources (i.e., in “parallel”) between candidate target objects, or switches rapidly and sequentially between them, has been the subject of intense debate in the literature. We refer to this debate as “parallel vs. sequential attention.” Originally, attention was assumed to be a unitary, indivisible resource, and consequently the sequential model was favored, often implicitly (Eriksen and Spencer, 1969; Treisman, 1969; Kahneman, 1973). The first two decades of studies using the visual search paradigm were heavily biased toward this assumption (Treisman and Gelade, 1980; Wolfe, 1998): when a target had to be identified among a varying number of distractors and the observer's RT was found to increase steadily with the number of items, it was assumed that the additional time needed for each item reflected the duration of engaging, sampling and disengaging attention (hence the term “serial search slope”). It was only in the 1990s that this assumption was seriously challenged by proponents of an alternate model, according to which attention is always distributed in parallel among items, and the increase in RT with increasing item number simply reflects the increasing task difficulty or decreasing “signal-to-noise ratio” (Palmer, 1995; Carrasco and Yeshurun, 1998; Eckstein, 1998; McElree and Carrasco, 1999; Eckstein et al., 2000). Both models are still contemplated today – and indeed, they are extremely difficult to distinguish experimentally (Townsend, 1990).

Multi-object tracking

The same difficulty also plagues paradigms other than visual search. Multi-object tracking, for instance, corresponds to a situation in which several target objects are constantly and randomly moving around the visual field, often embedded among similarly moving distractors (Pylyshyn and Storm, 1988). Sometimes, the objects are moving in feature space (i.e., changing their color or their orientation) rather than in physical space (Blaser et al., 2000). The common finding that up to four – or sometimes more (Cavanagh and Alvarez, 2005) objects can be efficiently tracked at the same time has been taken as evidence that attention must be divided in parallel among the targets (Pylyshyn and Storm, 1988). However, in the limit where attention would be assumed to move at lightning speed, it is obvious that this simultaneous tracking capacity could be explained equally well by sequential shifts of a single attention spotlight, than by divided or parallel attention. Indeed, at least some of the existing data have been found compatible with a sequential process (Howard and Holcombe, 2008; Oksama and Hyona, 2008). Since there is no general agreement concerning the actual speed of attention (Duncan et al., 1994; Moore et al., 1996; Hogendoorn et al., 2007), the question remains open.

Simultaneous/sequential paradigm

Other paradigms have been designed with the explicit aim of teasing apart the parallel and serial attention models. In the simultaneous/sequential paradigm, the capacity of attention to process multiple items simultaneously is assessed by presenting the relevant information for a limited time in each display cycle. In one condition (simultaneous) this information is delivered at once for all items; in the other condition (sequential) each item's information is revealed independently, at different times. In both conditions the critical information is thus shown for the same total amount of time, such that a parallel model of attentional allocation would predict comparable performance; however, a serial attentional model would suffer more in the simultaneous condition, because attention would necessarily miss the relevant information in one stimulus while it is sampling the other(s), and vice-versa (Eriksen and Spencer, 1969; Shiffrin and Gardner, 1972). The paradigm was recently applied to the problem of multiple-object tracking (Howe et al., 2010), and the data were deemed incompatible with serial attention sampling. A major source of confounds in this paradigm, however, is that, depending on stimulus arrangement parameters, certain factors (e.g., grouping, crowding) can improve or decrease performance in the simultaneous condition independently of attention; similarly, other factors (e.g., apparent motion, masking) can improve or decrease performance in the sequential condition. It is unclear in the end how to tease apart the effects of attention from the potentially combined effects of all these extraneous factors.

Split spotlight studies

To finish, there is yet another class of experiments that were intended to address a distinct albeit related question: when attention is divided among multiple objects, does the focus simply expand its size to include all of the targets, or does it split into several individual spotlights? To test this, these paradigms generally measure an improvement of performance due to attention at two separate locations concurrently; the critical test is then whether a similar improvement can also be observed at an intervening spatial location: if yes, the spotlight may have been simply enlarged, if not it may have been broken down into independent spotlights. Psychophysical studies tend to support the multiple spotlights account (Bichot et al., 1999; Awh and Pashler, 2000). The same idea has also been applied to physiological measurements of the spotlight, demonstrating that EEG or fMRI brain activations can be enhanced by attention at two concurrent locations, without any enhancement at intervening locations (Muller et al., 2003; McMains and Somers, 2004). Now, how do these results on the spatial deployment of attention pertain to our original question about the temporal dynamics of attention? Inherent in the logic of this paradigm is the assumption that attentional resources are divided constantly over time; multiple spotlights are implicitly assumed to operate simultaneously, rather than as a single, rapidly shifting focus. To support this assumption, authors often use limited presentation times (so attention does not have time to shift between targets), and speculate on the speed of attention. As mentioned before, since this speed is largely unknown, a lot of the data remain open to interpretation. In fact, our recent results in a very similar paradigm (in which we varied the delay between stimulus onset and the subsequent measurement of attentional deployment) showed that multiple simultaneous spotlights can in fact be observed, but are short-lived; when several target locations need to be monitored for extended periods of time, the attentional system quickly settles into a single-spotlight mode (Dubois et al., 2009). In another related study we found that attention could not simultaneously access information from two locations, but instead relied on rapid sequential allocation (Hogendoorn et al., 2010).

The conclusion from studies that have used this kind of paradigm is also fairly representative of the current status of the entire “sequential vs. parallel attention” debate, which we have briefly reviewed here. As summarized in a recent (and more thorough) review by Jans et al. (2010), most of the so-called demonstrations of multiple parallel attention spotlights rely on strong – and often unsubstantiated – assumptions about the temporal dynamics of attention. In sum, parallel attention has by no means won the prize.

Temporal aliasing returns

Could aliasing (see Temporal Aliasing: the Wagon Wheel Illusion and Other Forms of Temporal Aliasing) provide a way of resolving the “sequential vs. parallel attention” debate? If attention focuses on each target sequentially rather than continuously, the target information will be sampled more or less periodically, and should thus be subject to aliasing artifacts; furthermore, the rate of sampling for each target should be inversely related to the number of targets to sample (i.e., the “set size”). On the other hand, parallel attention models have no reason to predict aliasing; and, even if aliasing were to occur, no reason to predict a change of aliasing frequency as a function of set size. We recently tested this idea using a variant of the continuous wagon wheel illusion phenomenon (Macdonald et al., under review). On each 40 s trial, we showed one, two, three or four wheels rotating in the same direction; the frequency of rotation was varied between trials. From time to time, a subset of the wheels briefly reversed their direction, and the subjects’ task was to count and report how many of these reversal events had occurred in each trial. We reasoned that any aliasing would be manifested as an overestimation of the number of reversals. As expected from our experiments with the c-WWI, we found significant aliasing in a specific range of rotation frequencies. Most importantly, the frequency of maximum aliasing significantly decreased as set size was increased, as predicted by the “sequential attention” idea. Although the magnitude of this decrease was lower than expected (the sampling frequency was approximately divided by 2, from ∼13 Hz down to ∼7 Hz, while the set size was multiplied by 4), this finding poses a very serious challenge to the “parallel attention” theory.

The blinking spotlight of attention

The crucial difficulty in distinguishing parallel and sequential accounts of attention is a theoretical one: any “set size effect” that can be explained by a division of attentional resources in time can, in principle, be explained equally well using a spatial division of the same resources (Townsend, 1990; Jans et al., 2010). There is a form of equivalence between the temporal and spatial domains, a sort of Heisenberg uncertainty principle, that precludes most attempts at jointly determining the spatial and temporal distributions of attention. In a recent experiment, however, we tried to break down this equivalence by measuring set size effects following a temporal manipulation of the stimuli – namely, after varying their effective duration on each trial (VanRullen et al., 2007). Thus we could tell, for example, how well three stimuli were processed when they were presented for 300 ms, and we could compare this to the performance obtained for one stimulus presented 100, or 300 ms (or any other combination of set size and duration). The interest of this procedure was that different models of attentional division (e.g., sequential and parallel models) would make different predictions about how the psychometric function for processing one stimulus as a function of its duration should translate into corresponding psychometric functions for larger set sizes. To simplify, the space–time equivalence was broken, because performance for sequentially sampling two (or three, or four) stimuli each for a fixed period could be predicted exactly, by knowing the corresponding performance for a single stimulus that lasted for a duration equivalent to the sampling period; a simple parallel model, of course, could be designed to explain the change in performance from one to two (or three, or four) stimuli, but if the model was wrong it would then do a poor job at explaining performance obtained at other set sizes.

We compared three distinct models of attention allocation, each with a single free parameter. In the “parallel” model, all stimuli were processed simultaneously, and only the efficiency of this processing varied as a function of attentional load (i.e., set size); the cost in efficiency was manipulated by the model's free parameter. The second model, coined “sample-when-divided,” corresponded to the classic idea of a switching spotlight: when more than one stimulus was present, the otherwise constant attentional resource was forced to sequentially sample the stimulus locations; the model's free parameter was its sampling period, which affected its ability to process multiple stimuli. Finally, we decided to consider a third model, termed “sample-always,” similar to the previous one except for the fact that it still collected and integrated successive attentional samples even when a single stimulus was present (see Figure 3 for an illustration); this model's behavior was also governed by its sampling period. Our strategy was, then, to compare the three models’ ability to emulate the actual psychometric functions of human observers.

Figure 3.

Relating discrete perception with sequential attention. (A) A sensory process that samples a single visual input periodically illustrates the concept of discrete perception. (B) A sensory process that serially samples three simultaneously presented visual stimuli demonstrates the notion of a sequential attention spotlight. Since many of our findings implicate attention in the periodic sampling processes displayed in (A), we propose that both types of periodic psychological operations (A,B) actually reflect a common oscillatory neuronal process. According to this view, the spotlight of attention is intrinsically rhythmic, which gives it both the ability to rapidly scan multiple objects, and to discretely sample a single source. (The yellow balls linked by red lines illustrate successive attentional samples.)

Compatible with existing findings (Palmer, 1995), we revealed that the parallel model could outperform the classic version of the sequential model – the “sample-when-divided” one, which considers that the attentional spotlight shifts around sequentially, but only when attention must be divided. However, the truly optimal model to explain human psychometric functions was the other variant of the sequential idea, a model in which attention always samples information periodically, regardless of set size. The rate of sampling was found to be ∼7 Hz. When attention is divided, successive samples naturally focus on different stimuli, but when it is concentrated on a single target, the samples continue to occur repeatedly every ∼150 ms, simply accumulating evidence for this one stimulus. In other words, our findings supported a “blinking spotlight” of attention (VanRullen et al., 2007) over the sequentially “switching spotlight” (and over the multiple “parallel spotlights”).

Conclusion

The notion of a blinking spotlight illustrates a fundamental point: that discrete sampling and sequential attention could be two facets of a single process (Figure 3). Proponents of the discrete sampling theory should ask themselves: what happens when there are more than one relevant stimuli in the visual field? Can they all be processed in a single “snapshot”? Advocates of sequential attention should ponder about the behavior of attention when it has only one target to monitor: is it useful – or even possible – for attention to pause its exploratory dynamics?

The simple theory that we propose is that periodic “covert” attentional sampling may have evolved from “overt” exploratory behavior (i.e., eye movements), as a means to quickly and effortlessly scan internal representations of the environment (VanRullen et al., 2005a; Uchida et al., 2006). Just as eye movements continue to occur even when there is only one object in the scene – lest the object quickly fade from awareness (Ditchburn and Ginsborg, 1952; Coppola and Purves, 1996; Martinez-Conde et al., 2006) – it is sensible to posit that attentional sampling takes place regardless of the number of objects to sample. Perception can then be said to be “discrete” or “periodic,” insofar as a very significant portion of its inputs (those depending on attentional mechanisms) are delivered periodically. For example, the ∼13 Hz discrete sampling responsible for the continuous wagon wheel illusion was found to be driven by attention (VanRullen et al., 2005b; VanRullen, 2006; Macdonald et al., under review). The frequency of this sampling progressively decreased to ∼7 Hz when two, three, and finally four “wagon wheel” stimuli had to be simultaneously monitored (Macdonald et al., under review). Interestingly, this ∼7 Hz periodicity was also the one indicated by our model of the “blinking spotlight” of attention (VanRullen et al., 2007). Altogether, our data raise the intriguing suggestion that attention creates discrete samples of the visual world with a periodicity of approximately one tenth of a second.

To conclude, we argue that it is constructive to unite the two separate psychophysical debates about discrete vs. continuous perception (see Rhythmic Sampling of a Single Stimulus: Discrete vs. Continuous Perception) and sequential vs. parallel attention (see Rhythmic Sampling of Multiple Stimuli: Sequential vs. Parallel Attention). Discrete perception and sequential attention may represent perceptual and psychological manifestations of a single class of periodic neuronal mechanisms. Therefore, psychophysical progress in solving those debates could, ultimately, contribute to uncovering the role of low-frequency brain rhythms in perception and attention.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.