Spikes, BOLD, Attention and Awareness: A comparison of electrophysiology and fMRI signals in V1

Abstract

Early fMRI studies comparing results from fMRI and electrophysiology experiments supports the notion that the blood oxygen level dependent (BOLD) signal reliably follows the spiking activity of an underlying neuronal population averaged across a small region in space and a brief period in time. However, more recent studies focusing on higher-level cognitive factors such as attention and visual awareness report striking discrepancies between the fMRI response in humans and electrophysiological signals in macaque early visual areas. Four hypotheses are discussed that can explain the discrepancies between the two methods: (1) the BOLD signal follows local field potential (LFP) signals closer than spikes, and the only the LFP is modulated by top-down factors, (2) the BOLD signal is reflecting electrophysiological signals that are occurring later in time due to feedback delay, (3) the BOLD signal is more sensitive than traditional electrophysiological methods due to massive pooling by the hemodynamic coupling process, and finally (4) there is no real discrepancy, and instead weak but reliable effects on firing rates may be obscured by to differences in experimental design and interpretation of results across methods.

Introduction

Suppose physicists were to hand over a new telescope to astronomers that provided a view of the stars with unprecedented clarity. But then suppose that the astronomers were told that nobody understood precisely how the device translated the incoming electromagnetic signal into the viewable image. Would it be valid to make scientific conclusions from such a telescope? Such is the story of functional MRI, and other vascular-dependent neuroimaging methods. Research over the past 20 years has yielded hundreds of thousands of publications using fMRI, but a detailed understanding of the neurovascular coupling process remains elusive. How is this justified? The main reason is that fMRI results generally make sense. To push the astronomy analogy further – suppose that when the new telescope was pointed toward a well-known object like the moon, the images matched well with previous observations with established telescopes. This calibration test would help justify using the new device on other, less well understood celestial objects.

For fMRI, a standard calibration set are results from the electrophysiological recording literature from the macaque visual cortex. Much is known about the response properties of neurons in the macaque primary visual cortex for stimulus properties such as contrast, receptive field location, orientation and spatial frequency. Established computational models of these responses allow for a quantitative prediction of an averaged population response (e.g. (Heeger, 1992, 1993l, see Carandini et al., 2005, for a discussion of these models). A quantitative prediction of the location, amplitude and time-course of the fMRI signal can then be made by assuming that the BOLD signal reflects this population response averaged over a local region in space and period in time (Boynton, Engel, Glover, & Heeger, 1996).

The first section of this review shows how there is good agreement between the predicted and measured BOLD signals for stimulus-driven responses in early retinotopic visual areas of the human visual cortex. Manipulations of stimulus location, contrast, adaptation, orientation, motion and color all produce fMRI responses that are consistent with what is expected from electrophysiological responses in the macaque visual cortex. Many of these stimulus-driven results were obtained early in the history of fMRI, providing confidence to the research community that this new device was measuring something meaningful.

While these early studies measured responses to sensory stimuli fMRI research has gradually shifted emphasis to cognitive manipulations such as attention and awareness (Illes, Kirschen, & Gabrieli, 2003). Advances in macaque electrophysiological recording techniques, including the awake-behaving preparation and multi-electrode penetrations provide a new set of measurements to calibrate with the fMRI response. Surprisingly, these more recent electrophysiological recordings associated with higher-level cognitive tasks make predictions that often do not match well with their corresponding fMRI measurements. The second section of this review discusses how fMRI signals in V1 seem more strongly affected by top-down factors such as attention and awareness than what is predicted from firing rates of neurons in the primary visual cortex of monkeys.

The third section of this review discusses four hypotheses for these apparent discrepancies between human fMRI and monkey electrophysiology. The first hypothesis is that the BOLD signal is primarily driven by synchronized inputs that are strongly affected by feedback. The second is that the sluggish BOLD signal may be hiding the fact that top-down influences are occurring later in time. The third hypothesis is that relatively small top-down influences are more easily detected with fMRI due to the large amount of pooling associated with the hemodynamic coupling process. The fourth hypothesis is that the discrepancies may be inflated due to species differences, differences in experimental design, and interpretation of results.

Stimulus Driven Results

Linearity

Ideally, the BOLD signal reflects the activity of a neuronal population averaged over a narrow region in cortical space and time. An averaging process like this results in a linear system which satisfies the properties of superposition and scaling. Superposition means that the response to two or more combined stimuli is the sum of the responses to each stimulus alone. Scaling means that multiplying the input by a factor leads to an equal scaling of the output. A system that satisfies these two properties can be completely described by the system's impulse response function, which is the response to a stimulus which, in the limit, is infinitely short in duration but has unit amplitude. Knowing the impulse response function completely describes the system because any stimulus can be described by a sequence of shifted and scaled impulses. The output to any stimulus can therefore be described by the corresponding sequence of shifted and scaled impulse response functions. This process of shifting, scaling and summing the impulse response function is called convolution.

Linearity in time

Linearity of the fMRI time-course is assumed in nearly all analysis methods for fMRI data (e.g. Cohen, 1997). Linearity is particularly important for event-related designs in which the stimulus events are presented in such a rapid succession that the associated slow BOLD response to each stimulus overlap in time (Buckner, 1998). Typically, an fMRI voxel's time-course is compared to a predicted time-course based on convolving the time-course of the stimulus or cognitive task with a hemodynamic impulse response function (HDR). Either a canonical HDR is assumed which through convolution predicts a response than is compared statistically to the measured fMRI signal, or the HDR for a given voxel is estimated by finding the HDR that when convolved with the input best predicts the fMRI time-course in a least-squares sense (a process called deconvolution, Dale & Buckner, 1997). In either case, the properties of superposition and scaling are assumed to be true.

There is no a priori reason that the hemodynamic coupling process should be linear. Not only does linearity predict that the fMRI signal will grow indefinitely in proportion to the strength of underlying neuronal response, it also predicts that the shape of the time-course of the fMRI response should not change with either the strength of the neural response, or with previous response history.

Fortunately, repeated tests show that the assumption of linearity holds true, at least to a first approximation. An early analysis of the BOLD response in human primary visual cortex showed that a single HDR could predict the time-course of the fMRI signal to a range of pulsed and periodically presented flickering checkerboard stimuli (Boynton, et al., 1996). Subsequent studies tested the property of superposition more directly by estimating the contribution of the fMRI response to successive stimuli by subtracting out the fMRI response to previous stimuli. Again, to a first approximation, the assumption of linearity holds up remarkably well (Dale & Buckner, 1997). Since these original studies, the assumption of linearity over time has been tested with reasonable success in other modalities including the auditory cortex (Robson, Dorosz, & Gore, 1998), motor cortex (Bandettini & Cox, 2000), and somatosensory cortex (Arthurs & Boniface, 2003).

The linear model is not perfect. The actual fMRI response to very brief stimuli is systematically larger than predicted from longer stimulus durations (Boynton, et al., 1996; Vazquez & Noll, 1998);(Robson, et al., 1998) (Bandettini & Cox, 2000). This nonlinearity is probably not due to neuronal transient or adaptation effects, since the time-course of the magnetoencephelography (MEG) signal does not show this relatively large response to short stimuli (Tuan, Birn, Bandettini, & Boynton, 2008).

Similarly, the estimated response to repeated stimuli is smaller than expected, particularly for interstimulus intervals shorter than two seconds (Huettel & McCarthy, 2000). This reduction in the fMRI signal with repeated presentation may be caused by neuronal adaptation and not a hemodynamic nonlinearity. This is supported by the fact that the fMRI response mostly recovers if the orientation of the stimulus is switched by 90 degrees after several seconds of stimulation (Fang, Murray, Kersten, & He, 2005). The time-course of these fMRI adaptation effects in V1 is consistent with those measured with single-units (Carandini, Movshon, & Ferster, 1998). This gives us confidence that the stimulus-specific adaptation effects exploited by with the fMRI-adaptation technique (Grill-Spector & Malach, 2001) are neuronal in origin (see Krekelberg, Boynton, & van Wezel, 2006, for a review).

Linearity in space

A second assumption commonly made in the analysis of fMRI data is linearity in space. One prediction is that the BOLD response pooled across spatially separate neuronal responses should be equal to the sum of the BOLD signal to responses in each region separately. Hansen, David, & Gallant (2004) tested this prediction by taking advantage of the retinotopic organization in V1 and presenting visual stimuli at discrete locations either sequentially or simultaneously. They found that the BOLD signal in V1 reflects the sum of neural signals across the cortex in a spatially linear fashion.

Linearity in space means that the spatial pattern of the BOLD signal across the cortex can be predicted by convolving the spatial pattern of the underlying neural response with an impulse response function in space, called a hemodynamic point spread function. This assumption essential to a recently developed method for measuring a given voxel's ‘population receptive field’ in which both temporal and spatial linearity are assumed for predicting a given voxel's time-course to a visual stimulus that is varying in both space and time (Dumoulin & Wandell, 2008).

Contrast response

A ubiquitous property of cells in the primary visual cortex is their monotonically increasing response to stimulus contrast (Geisler & Albrecht, 1997). Contrast-response functions of typical macaque V1 neurons increase for low contrasts and then level out, or saturate at high contrasts. The fMRI response in human V1, however, continues to increase up to 100% contrast. While this seems like a discrepancy, (Heeger, Huk, Geisler, & Albrecht, 2000) estimated the overall population response based on electrophysiology results from Geisler & Albrecht (1997) showed that since not all V1 neurons saturate with contrast, the population based contrast-response does not saturate either. Their electrophysiologically-based contrast response function matched up well with the contrast response functions measured with fMRI (Boynton, Demb, Glover, & Heeger, 1999). This is important because it shows that the BOLD signal is not just monotonic, but it grows in proportion to the mean of the underlying neural activity as predicted for a linear system.

Motion coherence

A similar comparison was made in motion-sensitive areas for the dimension of stimulus coherence (Rees, Friston, & Koch, 2000). Earlier, Shadlen, Newsome, & Movshon (1993) measured the effect of motion coherence on macaque MT neurons using random dot stimuli. Spike rates increased monotonically, on average, with motion coherence for motion in the preferred direction of the neuron, and decreased with motion in the anti-preferred direction. (Rees, et al., 2000) measured the fMRI response to stimuli in area MT+ (believed to be the human homologue of macaque MT) and found the BOLD signal increased with increasing motion coherence. They then estimated a population-based average from the electrophysiology results and found that the overall population of MT neurons should also increase with motion coherence. A direct quantitative comparison of the predicted and measured effects of motion coherence matched up well. This result is significant because the fMRI response could have gone up, down or remained flat with stimulus coherence, depending on how the fMRI response pools signals from the underlying electrophysiological response.

Motion opponency

A related study compared electrophysiological responses in macaque to human fMRI responses using moving vs. counterphase modulated gratings (Heeger, Boynton, Demb, Seidemann, & Newsome, 1999). A 100% contrast counterphase modulated grating is identical to the physical sum of two 50% contrast gratings moving in opposite directions. It may seem that in a direction-selective visual area like MT, the population response to a 100% counterphase grating should be greater than a single 50% contrast moving grating since the former should excite twice as many neurons as the latter. However, it is known that the typical MT neuronal response to a stimulus moving in the preferred direction is suppressed by a second stimulus moving in a non-preferred direction – a phenomenon known as motion opponency (Snowden, Treue, Erickson, & Andersen, 1991);(Bradley, Qian, & Andersen, 1995);(Simoncelli & Heeger, 1998). Although a counterphase modulated grating should excite two subpopulations of neurons tuned to opposing directions, each subpopulation response should be weaker than that for a single grating alone. Thus, the overall population response to a counterphase grating could either increase or decrease for a counterphase modulated grating, depending on the strength of motion opponency and the pooling mechanisms of the hemodynamics.

Heeger, et al. (1999) estimated the effect of motion opponency on the population response of macaque MT neurons using a series of full-field moving and counterphase gratings. Crucially, the same full-field gratings were used in a corresponding fMRI study in humans. In macaque area MT, the average response across the sample of MT neurons for 100% contrast counterphase gratings was actually lower than that for 50% contrast moving gratings. This population average matched the fMRI response in human area MT+ to nearly identical stimuli.

It should be noted that in V1 there was no difference between the response to the single moving grating and the counterphase grating, indicating that the population of V1 neurons showed something in between responding independently to the two components of the counterphase grating and motion opponency. This is consistent with the finding that motion-opponency effects appear weaker in macaque V1 than MT (Snowden, et al., 1991).

Color-opponency

A standard model of human color processing poses that a linear combination of the signals from the three cone classes (L, M and S) are combined to produce three opponent responses typically called red-green (L-M), blue-yellow ((L+M)-S) and luminance (L+M) mechanisms. Color-opponency is believed to be represented early in the visual processing stream and were originally found in macaque LGN (Derrington, Krauskopf, & Lennie, 1984)(Reid & Shapley, 1992). Early studies in macaque V1 showed evidence of color opponency, but the number of opponent neurons seemed small compared to what was expected from psychophysical measures (Johnson, Hawken, & Shapley, 2001; Lennie, Krauskopf, & Sclar, 1990; Thorell, De Valois, & Albrecht, 1984). To the contrary, a number of functional MRI studies comparing L-M to L+M contrast inputs suggest that are a relatively large number of underlying color-opponent neurons in human V1 (S. A. Engel & Furmanski, 2001; S. Engel, Zhang, & Wandell, 1997; Kleinschmidt, Lee, Requardt, & Frahm, 1996). It turns out, however, that a relatively small number of color opponent neurons in the V1 population can lead to large population-based opponent signals. Schluppeck & Engel (2002) showed this by using the results of the electrophysiological study in V1 by Johnson et al., (2001) to predict the response to the stimuli used in the neuroimaging study by Engel et al. (1997). A simple linear pooling rule with a threshold nonlinearity predicted population responses to various directions in chromatic contrast that are remarkably similar to the fMRI results reported in Engel et al. (1997).

Receptive field location

It is easy to take for granted the ease in which visual area boundaries can be delimitated using standard phase-encoded responses generated by sweeping rings and wedges (S. A. Engel et al., 1994; Sereno et al., 1995). But a precise retinotopic map measured with fMRI requires the local vasculature at a given location to pool from a region of grey matter that is not only restricted in space but is also unbiased in central location. It is easy to imagine a scenario where the BOLD response to a spatially localized stimulus behaves roughly linear over time, but is significantly mislocalized in space due to the nature of downstream vascular pooling. This may indeed be the case for human area V4 in the ventral visual cortex (Winawer, Horiguchi, Sayres, Amano, & Wandell, 2010), but vascular artifacts seem to be the exception. For example, in humans it has been demonstrated that the visual area boundaries between V1 and V2 measured with fMRI are consistent with structural imaging measures of the stria of Gennari in V1 (Bridge et al., 2005), and the fMRI-based retinotopic maps measured with fMRI in the macaque align well with local anatomical and physiological measurements (Brewer, Press, Logothetis, & Wandell, 2002).

More recently, a new ‘population receptive field’ or pRF method for retinotopic mapping has been developed which models an fMRI voxel's response as linear convolution of the stimulus over time restricted to a specific Gaussian kernel in space (Dumoulin & Wandell, 2008). Predictions from this space-time linear filter model are remarkably close to the actual fMRI response to full-field sweeping bar stimuli, providing more support for the linear model. In addition, across voxels estimates of the Gaussian kernels' location, size, and density is consistent to what is expected from electrophysiological studies in monkeys (Harvey & Dumoulin, 2011).

Orientation selectivity

A fundamental property of V1 neurons is orientation selectivity (Hubel & Wiesel, 1959). Orientation selective neurons are clumped together in V1 forming homogenously tuned orientation ‘columns’, each approximately 0.5 mm across. This spatial scale is too small to be imaged directly using traditional fMRI which use voxels that are around 2-3mm in width (but see Yacoub, Harel, & Ugurbil, 2008). However, two indirect methods, adaptation and multi-voxel-pattern-classification (MVPA) have been used to reveal evidence of orientation selectivity in subpopulations of neurons within voxels. After adapting by prolonged exposure to a stimulus of one orientation, the subsequent fMRI response in V1 to a briefly presented stimulus becomes orientation selective, with the weakest response at the adapting orientation (Fang, et al., 2005). Unlike the rapid adaptation effects seen in ventral visual areas (Grill-Spector & Malach, 2001), measurable adaptation effects in V1 do not occur with short adaptation periods (Boynton & Finney, 2003; Finney & Boynton, 2003). Thus, the rate of adaptation is consistent with time-constants found in the mammalian visual cortex (Albrecht, Farrar, & Hamilton, 1984), indicating that at least part of the source of the fMRI adaptation effect with fMRI is neuronally-based (see Krekelberg, et al., (2006) for a discussion).

Although the fMRI response in a given V1 voxel is nearly constant across stimulus orientations, there is sufficient reliability in the pattern of responses to different orientations across voxels to make inferences about orientation selectivity in the underlying neuronal population (Kamitani & Tong, 2005). This information can be extracted using “multi-voxel pattern analysis”, or MVPA techniques in which the pattern of fMRI responses across voxels for a given ‘test’ stimulus is compared to responses to a ‘training set’ of patterns induced by a range of orientations. Because the pattern of voxel responses in V1 and other early visual areas varies systematically with stimulus orientation, the orientation of the test stimulus can be accurately predicted well above chance. This came as a surprise to many fMRI researchers, especially considering that information about orientation, motion (Kamitani & Tong, 2006) and color selectivity (Brouwer & Heeger, 2009) was sitting on their computer file systems all along. This is a robust effect and works for a variety of classification algorithms. The physiological source of these reliable patterns is not well understood – recent evidence shows that it may be driven by a global signal such as radial bias and/or the oblique effect (Op de Beeck, 2009; Freeman, Brouwer, Heeger, & Merriam, 2011; Mannion, McDonald, & Clifford, 2009) rather than by a biased sampling of orientation columns within each voxel (see Boynton, 2005, and Kriegeskorte, 2009, for further discussion). While the evidence that human V1 contains orientation selective neurons is not surprising, the development of the MVPA technique opened the door for novel discoveries about orientation selectivity in the context of higher-order cognitive factors such as attention (Kamitani & Tong, 2005) and awareness (Haynes & Rees, 2005a) that will be discussed in the next section.

In summary, the studies reviewed above show that for stimulus-driven responses there is good agreement between the BOLD fMRI signal in humans and what is expected from single-unit measurements in Macaque primary visual cortex. But it will be shown below that manipulations of cognitive factors such as attention and awareness can break this correspondence. For some reason, top-down influences on visual responses may affect fMRI responses in early visual areas much more than what is predicted from electrophysiological recordings in the macaque.

Top-Down Modulation

Spatial Attention

In the late 1990's three articles were published around the same time showing that spatial attention modulates fMRI responses in the human primary visual cortex (Gandhi, Heeger, & Boynton, 1999; Martinez et al., 1999; Somers, Dale, Seiffert, & Tootell, 1999). These findings showed robust modulations of the fMRI response in V1 from voxels associated with attended peripheral stimuli compared to unattended stimuli placed in the opposite visual hemifield. These findings were surprising because electrophysiological recordings in macaque showed little or no modulation with spatial attention shifting in and out of the receptive field of a V1 neuron (Luck, Chelazzi, Hillyard, & Desimone, 1997; Motter, 1993).

Numerous studies have since replicated the V1 spatial attention effect with fMRI (e.g. Ciaramitaro, Buracas, & Boynton, 2007; Li, Lu, Tjan, Dosher, & Chu, 2008; Slotnick, Schwarzbach, & Yantis, 2003). fMRI responses in human V1 are now known to modulate with spatial attention even in the absence of a physical stimulus (Kastner, Pinsk, De Weerd, Desimone, & Ungerleider, 1999; Ress, Backus, & Heeger, 2000; Silver, Ress, & Heeger, 2007). These attentional effects can be just as strong as in the presence of a stimulus across a range of contrasts (Murray, 2008). This means that the effect of attention on the fMRI contrast-response function in V1 and other early visual areas is additive (Buracas & Boynton, 2007), and not multiplicative or divisive as expected from the electrophysiology literature in areas V4 and MT (Reynolds & Heeger, 2009; Reynolds, Pasternak, & Desimone, 2000; but see Li, et al., (2008) and a discussion by Boynton (2009).

Feature-based attention

Attention to a specific feature, such as a direction of motion (Martinez-Trujillo & Treue, 2004) or orientation (McAdams & Maunsell, 1999) enhances the response to visual neurons selective to that feature, and suppressed response to neurons tuned away. This feature-based effect has been shown to operate on neurons with receptive fields well outside the spatial focus of attention (Treue & Martinez Trujillo, 1999). Feature-based attention effects have been found in macaque areas MT and V4, but so far not in area V1.

However, fMRI responses in V1 have been shown to be strongly modulated by feature-based attention. In one study, the fMRI response to an unattended stimulus was shown to increase when attention was directed elsewhere to a stimulus sharing a matching feature compared to attention to an opposing feature (Saenz, Buracas, & Boynton, 2002). This result was found for both direction of motion (up vs. down) and color (red vs. green) in all reported visual areas, including V1.

The influence of feature-based attention on responses to attended stimuli has also been demonstrated using MVPA techniques. Kamitani & Tong (2005) showed that not only could stimulus-driven responses to orientation be successfully classified from fMRI responses in V1, but also that merely instructing subjects to attend to a single component of a plaid stimulus lead to successful decoding of the attended orientation in V1. A feature-based attentional effect was also found for motion using MVPA in area V1 and other early visual areas (Kamitani & Tong, 2006).

Surprisingly, successful pattern classification could also be obtained in V1 for the attended direction of motion in V1 corresponding to an unstimulated visual hemifield (Serences & Boynton, 2007). This implies that some sort of change in the baseline response analogous to the spatial attention effects is occurring without visual stimulation. To date, no robust effects of feature-based attention have been found on electrophysiological baseline response in MT or any other macaque visual area.

Saccadic suppression

A saccadic eye movements can reach speeds of hundreds of degrees per second, causing the retinal image to move rapidly in the direction opposite of the saccade. Despite this massive motion signal, no perception of motion is experienced during a saccade (Dodge, 1900). Typical theories of saccadic suppression involve an attenuation of the motion signal through an efference copy mechanism signaled by the command to initiate a saccade. Where in the brain this motion signal is suppressed can be measured either with fMRI or by electrophysiological methods by simply recording responses for physically non-moving stimuli during saccadic eye-movements.

Early neuroimaging studies demonstrated a decrease in responses in occipital cortex related to saccade frequency using PET (Paus, Marrett, Worsley, & Evans, 1995) and fMRI (Wenzel et al., 2000). More recent work has shown that these suppressive effects can be localized to V1 (Sylvester, Haynes, & Rees, 2005; Vallines & Greenlee, 2006). Sylvester et al. (2005) found robust reductions of the BOLD signal in V1 and the LGN during saccades when a visual stimulus is presented (interestingly, responses were increased during saccades with no stimulus). Vallines and Greenlee (2006) found a drop in the fMRI response in V1 for stimuli presented near the saccadic onset, consistent with behavioral measures of saccadic suppression.

Monkey electrophysiology studies show weaker and less consistent effects of saccades on firing rates of V1 neurons. If anything, there may actually be an increase in firing rate near the onset of a saccade (Super, van der Togt, Spekreijse, & Lamme, 2004). Kagan, Gur, & Snodderly (2008) found variability in the effects of saccades on V1 responses. In one third of their neurons they did find a brief suppression in the firing rate, but this was followed by a stronger and longer-lasting increase after onset of the saccade.

Size constancy

The ability to obtain reliable and stable retinotopic maps with fMRI has been essential to our understanding of not only the structural organization of the human visual system, but it has also provided a means to study functional organization by a allowing us to study the effects of experimental manipulations within specific area-by-area regions of interest. However, there is evidence that even the estimates of receptive field location based on the BOLD signal can be influenced by top-down factors.

Murray, Boyaci, & Kersten (2006) studied the effects of perceived depth of a stimulus on the size of the stimulus' representation in the primary visual cortex. The perceived depth of a foveally-placed disk of fixed visual angle was manipulated by placing it in a hallway drawn with 3-D perspective depth cues. The disk appeared larger when it was made to look farther away, demonstrating the well-known phenomenon of size constancy. Surprisingly, even though the retinal size of the disk remained constant, the spatial extent of the fMRI response elicited by the disk increased with perceived depth just as though its physical size had increased. In a subsequent study, this same group found that the effect of perceived size on the fMRI response was reduced when attention was directed away from the stimulus and to a demanding task at fixation (Fang, Boyaci, Kersten, & Murray, 2008). The authors argue that focusing attention at fixation reduced feedback activity from higher visual areas that process 3-D depth cues. This result is remarkable because it implies that there must be V1 neurons with receptive fields at the edge of the stimulus that may or may not be excited by the stimulus, depending on its perceived depth. This is equivalent to saying that the receptive fields of V1 neurons are shifting with 3-D depth cues. The attention manipulation implies that this shift is not stimulus-driven, but has something to do with a combination of excitation and suppression from top-down signals associated with 3-D depth cues.

This effect has not yet been studied in monkeys. Until recently, receptive field locations were considered to be an invariant property of neurons in early visual cortex. But recent electrophysiological studies have shown that attention can affect the shape of the receptive field of neurons in areas MT (Womelsdorf, Anton-Erxleben, & Treue, 2008) and V1 (Roberts, Delicato, Herrero, Gieselmann, & Thiele, 2007). Thus, it is certainly possible that 3-D depth cues may also affect receptive field properties.

Binocular rivalry

When two disparate images are presented to each eye, the percept tends to alternate between the two images over a period of seconds – a time course well within the limitation imposed by the sluggish hemodynamic response. This dissociation between stimuli and perception has been a useful tool for understanding the neural correlates of consciousness because fluctuations in the neuronal response that correlate in time with the percept must reflect the internal state of the observer, and not changes in then physical stimulus (Blake & Logothetis, 2002; Crick, 1996; Crick & Koch, 1995).

The methods for studying binocular rivalry with fMRI vary, but a straightforward way is to use two stimuli that differentially excite a brain area of interest. For V1, high and low contrast orthogonal grating stimuli can be used e.g. (Polonsky, Blake, Braun, & Heeger, 2000) since as discussed above, high contrasts produce a larger V1 response than a low contrasts. A voxel's response can be associated with the perceived stimulus by correlating the time-course of the fMRI response with the observer's report of the percept. Using this and similar methods, a number of fMRI studies have shown fMRI responses in V1 (Haynes & Rees, 2005b; Lee & Blake, 2002; Lee, Blake, & Heeger, 2005; Polonsky, et al., 2000) and even the LGN (Wunderlich, Schneider, & Kastner, 2005) that strongly follow time-course of perceptual rivalry. This modulation of the fMRI signal can be as strong as the modulation driven by a physical alternation of the stimulus V1 (Polonsky, et al., 2000).

On the other hand, the results from monkey electrophysiology experiments in early visual areas are weaker, despite similar methods. While spike rates for around 90% of the neurons recorded in the inferior and superior temporal sulci show a significantly stronger response during the percept of a preferred stimulus (Sheinberg & Logothetis, 1997), only about 20% of the neurons in earlier visual areas have responses that correlate with the percept (V1, V4 and MT) (Leopold & Logothetis, 1996; Logothetis & Schall, 1989).

Why the Discrepancy? Four Hypotheses

While The BOLD signal is seen to modulate strongly with attention, saccadic suppression, and binocular suppression in V1, the firing rate of macaque V1 neurons appear to be less strongly affected. Below is a discussion of four hypotheses that could explain the consistencies and discrepancies between spikes and BOLD described above.

1) The LFP hypothesis

A natural hypothesis for the discrepancies between BOLD and spikes is that the BOLD signal is not driven explicitly by spiking activity. Recent studies measuring simultaneous electrophysiological and BOLD signals in monkeys supports an ‘LFP hypothesis’ in which local field potentials (LFP's) are a significantly better predictor of the BOLD signal (Goense & Logothetis, 2008; Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001; Niessing et al., 2005; see Ekstrom (2010) for an extensive review).

If the BOLD signal is most strongly associated with LFP's, then a possible explanation for the consistencies and discrepancies between spikes and BOLD is that top-down modulatory signals influence LFP signals more than spikes. Without a strong top-down influence, spikes might correlate well with LFP's and therefore spikes should correlate well with the BOLD signal. However, factors such as attention, binocular suppression, and saccadic suppression may strongly affect LFP's (but not spikes) and therefore the BOLD signal as well (see Muckli, 2010, for a similar discussion).

Maier et al. (2008) found support for the LFP hypothesis by simultaneously measuring fMRI and electrophysiological signals in monkeys that were experiencing binocular rivalry. Using a ‘generalized flash suppression’ paradigm in which the perception of a monocular target dot is suppressed in the presence of binocular surrounding dots, they found that like previous reports, the BOLD response to the target in V1 decreased when it was perceptually suppressed. Also like previous reports, spiking activity to the target in V1 during perceptual suppression did not drop at all. However, a spectral analysis of the LFP signals revealed that unlike spiking activity, the LFP region of the power spectrum (5-30 Hz) did indeed drop during perceptual perception (but not in the higher region of 30-90Hz). So binocular suppression has a differential effect on LFPs and spikes, and the BOLD signal follows the LFP response.

It follows that the LFP signals in monkey V1 should also be enhanced by attention since attention strongly increases the fMRI signal in human V1. There is some evidence that attention affects LFP's in areas MT and V4 of the monkey. A recent study reported the effects of attention on spikes and LFP's on responses in direction-selective area MT, where attention is known to affect firing rates (Khayat, Niebergall, & Martinez-Trujillo, 2010). As expected, spatial and feature-based attention had a significant influence on firing rates of MT neurons. Attention also enhanced the LFP power in the low frequency (5-30Hz) range. The authors cautiously state that attention modulates the LFP signal more strongly than spiking activity. This makes sense: the effects of spatial attention on BOLD signal in human MT+ are large (e.g. Buracas & Boynton, 2007; Buracas, Fine, & Boynton, 2005; Gandhi, et al., 1999) compared to the more modest effects of spatial attention on firing rates in monkey MT (e.g. Seidemann & Newsome, 1999).

The effects of attention on LFP signals in V1 appear to be less consistent than in V4 or MT. One study in humans (Yoshor, Ghose, Bosking, Sun, & Maunsell, 2007) reported LFP measurements from clinical subdural electrodes over V1 and V2 in patients and found no effect of spatial attention on their LFP signals. This is unexpected under the LFP hypothesis. The lack of an attentional effect found in human LFP signals may be due to differences in the recording methods. LFP's in monkeys are acquired through penetrating electrodes, while the human LFP's were measured with surface-based electrodes. Signals from these different methods may be reflecting LFP's emanating from different cortical depths; recent work using an array of electrodes varying in cortical depth and a current source density model suggests that the LFP signals do vary across cortical layers (Maier, Aura, & Leopold, 2011).

A recent study in monkey V1 (Chalk et al., 2010) actually found a decrease in the LFP power in the gamma range (30-50Hz) with attention in V1. This is unlikely due to any differences in the experimental design because the same paper reported an increase in LFP power at the same frequency range with attention in area V4, consistent with previous reports (Bichot, Rossi, & Desimone, 2005; Fries, Reynolds, Rorie, & Desimone, 2001). This result is puzzling. If LFP signals are strongly correlated with the BOLD signal, then we should find a decrease in the BOLD signal with attention, which has never been seen. It is unclear why the effects of attention on LFP's should be different between V1 and higher visual areas. The authors make several suggestions, but favor the hypothesis that attention reduces the strength of inhibitory drive which is inherently synchronous.

2) Delayed Feedback

A second hypothesis for these discrepancies have to do with delayed feedback and the slow time-course of the fMRI response. Electrophysiological studies typically report mean firing rates from the initial response to a stimulus or behavioral condition. But modulations in early visual areas due to attention and other cognitive factors may occur later on as a result of delayed feedback. For example, Lamme, Rodriguez-Rodriguez, & Spekreijse (1999) found that while the orientation of textures is encoded in monkey V1 as early as 55msec, figure-ground effects show up later (80 -100 msec). Similarly, effects of attention in V1 have been found to appear well over 200 msec after stimulus onset (Roelfsema, Lamme, & Spekreijse, 1998; see Lamme & Roelfsema, 2000, for a review).

This argument can explain discrepancies between EEG signals and the fMRI response in V1. For example, the early component of the VEP (the C1) which is typically attributed to signals emanating from V1 is not always affected by spatial attention (Clark & Hillyard, 1996). One of the first groups to discover the attentional effect on the V1 BOLD signal replicated this null C1 EEG result and hypothesized that their fMRI results must be due to modulations occurring later in time (Martinez, et al., 1999).

The effect of attention on the C1 is controversial, however. Two recent studies using more advanced source localization techniques do find an effect of attention on the early C1 component (Kelly, Gomez-Ramirez, & Foxe, 2008; Poghosyan & Ioannides, 2008). Steady-state EEG measures localized to V1 also show a modulation by attention (Lauritzen, Ales, & Wade, 2010).

A similar story comes from neuroimaging investigations of the attentional blink (AB). The attentional blink is the phenomenon that during rapid serial visual presentation, observers often fail to detect the second of two targets if it appears within 500-700 msec after the first (Raymond, Shapiro, & Arnell, 1992). Using similar paradigms, two groups found a reduction in the BOLD signal to the second of two successive stimuli in V1, matching the reduction of behavioral accuracy in a target identification task (Stein, Vallines, & Schneider, 2008; Williams, Visser, Cunnington, & Mattingley, 2008). However, a recent EEG study in humans failed to find a physiological correlate of the attentional blink in the C1 component (Jacoby, Visser, Hart, Cunnington, & Mattingley, 2011). These investigators conclude that “… reduced neural activity in V1 during the AB is driven by re-entrant signals from extrastriate areas that regulate early cortical activity via feedback connections with V1.” These re-entrant signals are presumably occurring later in time, leaving the C1 component to behave in a stimulus-driven fashion.

3) Massive Pooling by the Hemodynamic Coupling Processes

A third explanation for the discrepancy between the BOLD signal and spikes may have to do with the relative sensitivity of the two measures. The noise in the fMRI signal is the result of two factors: noise caused by neuronal variability, and noise associated with hemodynamics and MR scanning physics.

Consider the ability for a neuroscientist to find a hypothetical small effect of attention in V1. Suppose that single V1 neurons have a mean firing rate of 20 spikes/sec to an unattended stimulus, but increase to 21 spikes/sec when attention is directed into their receptive fields. It is known that for firing rates of single neurons, the variance typically grows roughly in proportion to the mean (with a typical constant of proportionality of about 1.5 for a typical trial) (e.g. Geisler & Albrecht, 1997). The trial-to-trial variance to a 20-21 spike/sec mean response should therefore be around 30 spikes per second. This means that increase of 1 spike/sec for the mean with attention is much less than the standard deviation (an effect size of about 0.18). A power analysis shows that a neurophysiologist would need to measure about 450-500 independent trials or neurons to have an 80% chance of correctly detecting an effect of attention (using a standard independent measures t-test).

On the other hand, consider a typical 3×3×3 mm fMRI voxel that is presumably pooling responses across about a quarter million neurons (Braitenberg & Schuez, 1998). Even assuming a covariance across the firing rates of these neurons of 0.2 (Zohary, Shadlen, & Newsome, 1994), the standard error of the mean for these neurons in a given trial should be around .014 spikes/sec. This is miniscule compared to the 1 spike/sec increase with attention. We can therefore consider the trial-to-trial variability of the mean response across neurons within a voxel to be negligible. This is supported by fMRI results showing that unlike neuronal responses, the variability of the BOLD signal remains roughly constant across response magnitude (e.g. Boynton, et al., 1999; Boynton, et al., 1996).

Now, consider the fMRI response to the same attentional effect. Assuming linearity of the BOLD signal, Rees et al. (1997) calculated that a 1% increase in the BOLD signal corresponds to a 9 spike/sec increase in the neuronal response. Using a similar argument, but with different stimuli and data, Heeger, et al. (2000) computed a smaller value of 0.4 spikes/sec. Taking an intermediate value of 4 spikes/sec for a 1% increase in the BOLD signal, our hypothesized attentional effect of 1 spike/sec should produce an average increase of 0.25% signal change in the BOLD signal, which is consistent with published results (Buracas & Boynton, 2007). An increase of this magnitude can detected reliably in V1 using standard fMRI protocols, even from a single 6 minute scan within a single subject.

Though these calculations are rough estimates, they illustrate that it is plausible that electrophysiological methods may not have the power to detect signal changes that may easily be detected with fMRI. Fortunately, the sample size obtained using electrophysiological methods keeps increasing with advanced methods such as multiple electrode arrays. As a corollary, note that the LFP signal presumably involves pooling of neuronal responses, so that the LFP hypothesis mentioned above might also be a pooling issue as well.

This pooling argument has been used to explain the recent perplexing claim that the BOLD signal can be modulated without any associated changes in the neuronal response. Sirotin & Das (2009) measured electrophysiological responses and hemodynamic responses simultaneously in monkeys with a novel optical imaging technique and found predictable fluctuations in their hemodynamic signals (both blood volume and blood oxygenation) within V1 in time with a the anticipation of a perceptual task, even though the animals were sitting in virtually total darkness. This result itself is perhaps not surprising since, as discussed above, the BOLD signal is known to be affected by attention in the absence of visual stimulation. However, the corresponding electrophysiological signals showed no corresponding anticipatory effect. This result has inspired a great deal of speculation about the functional role of the hemodynamic response, including the idea that the vascular system is plumbed to flood specific cortical regions in the anticipation of upcoming metabolic demand due to likely neuronal responses (Vanzetta & Slovin, 2010). If true then the BOLD signal may be reflecting something that has very little to do with underlying neuronal activity, but is instead measuring something that is indeed interesting, but perhaps not what we were hoping for.

On the other hand, (Kleinschmidt & Muller, 2010) make the argument that perhaps there actually was a weak anticipatory neuronal response that was measurable in the hemodynamic response, but their electrophysiology methods were too insensitive to detect it.

A correlate of the pooling hypothesis is that the vascular system doesn't have to reflect signals from the exact location of the underlying cortex. Recall that the top-down effects described here are all expected in the spiking activity in higher visual areas. BOLD effects in V1 could be reflecting signals from some distance away, either via direct draining veins from higher visual areas or through a secondary plumbing effect in which changes in blood volume and flow in one region influences the flow to other regions in the tightly connected vascular system. A vascular artifacts have been used to explain the variability in the retinotopic maps in human V4 as measured with fMRI (Winawer, et al., 2010).

4) Differences between experimental design and analysis

We should not rule out the possibility that there are actually weak, but reliable effects of attention and awareness in the firing rates of neurons in human V1. A final hypothesis for the discrepancy between BOLD and spikes could be that these effects are obscured by differences in species, experimental design and data interpretation across the experimental methods. It is important to acknowledge that electrophysiology and fMRI studies are rarely conducted by the same research groups with the same stimuli and especially with the same subjects.

Species differences

Most of the discrepancies described above were between human fMRI studies and electrophysiology studies on monkeys. It is hardly debatable that for stimulus-driven responses, the monkey visual system has served as a valuable model for the human visual system. However, as vision research moves toward more cognitive manipulations, this species comparison could come into question. At some point the monkey model is going to break down as we push toward higher-level processes such as consciousness, learning and decision making. It is therefore possible that species differences may be a factor in manipulations of attention and awareness.

Still, there is probably more to the BOLD/spike discrepancy than species differences. Recall that the study by Maier et al. (2008) in which both electrophysiology and fMRI measures were obtained on the same monkeys still found the discrepancy between BOLD and spikes (but not between LFP and spikes). Also, the attention study in humans by Yoshor et al. (2007) failed to find significant effects of attention in their subdural electrode responses in V1, unlike the human fMRI studies.

Experimental design

Electrophysiologists typically tune their stimuli to match the receptive field properties of the cell being recorded in order to maximize firing rates. For V1, this means that stimuli are typically restricted in spatial extent, and by both spatial and temporal frequency. In contrast, fMRI experiments employ large stimuli with broad spatial and temporal frequency spectra (like flickering checkerboards), again in order to maximize responses. This makes a comparison between monkey electrophysiology and human fMRI difficult.

A direct comparison between neuronal activity and BOLD signals requires an estimate of the electrophysiological response across a population of neurons, not necessarily tuned to respond maximally to the stimulus or task. For example, feature based attention may increase or decrease the firing rate of a neuron depending on the relationship between the attended feature (e.g. orientation or direction of motion) and the preferred feature for the neuron (e.g. Martinez-Trujillo & Treue, 2004). The corresponding effect of feature-based attention on the fMRI response could increase or increase, depending on the distribution of attentional effects across the underlying neuronal population.

In fact, the few quantitative studies that have attempted to compare fMRI and BOLD, either by equating stimuli (e.g. Heeger et al., 1999) or by estimating the fMRI response based on population responses of neurons (Rees et. al, 2000;Heeger et al. 200), found little discrepancy between the measures. These happen to be stimulus-driven studies.

Cognitive factors probably differ across experiments even more than stimuli. Certainly instructions and training for subjects varies between humans and monkeys, so it is hard to tell how to compare cognitive strategies across species.

Data interpretation

Differences between BOLD and spikes may be a matter of data interpretation and conclusions made based on selected studies in the literature. For example, while it is widely cited that attention does not strongly affect V1 firing rates in monkeys, a close inspection of the literature shows that indeed there are studies that do show positive results. One of the earliest studies of attentional modulation found effects of spatial attention for an orientation discrimination task in V1 (Motter, 1993). In fact, effects of attention in V1 (and V2) were at least as large as in V4. Also, while Luck, et al. (1997) found little or no effect when attention was directed to a single stimulus within a V1 or V2 receptive field, attentional effects were large in V2 for multiple stimuli inside the receptive field. Unfortunately receptive fields were too small for a similar experiment in V1 so the authors were unable to conclude if attention did affect V1 responses for multiple stimuli. Other studies have also shown significant, but not necessarily large effects of attention (Haenny & Schiller, 1988; Herrero et al., 2008) and task difficulty (Chen et al., 2008) in V1 firing rates. Also, while Yoshor et al. (2007) showed no statistically significant effect of attention on their subdural electrophysiological signals, there was a great deal of variability in their data, and in fact 5 out of their 6 subjects did show a positive effect. As described above in the pooling hypothesis, a weak attentional effect in V1 could still easily be detectable in human V1 with fMRI.

Perhaps the most striking discrepancy between BOLD and spikes is on the baseline effects when attention is directed without physical stimulation. Again, however, while it is generally considered that baseline firing rates in V1 are not modulated by attention, a close look at the published results shows that indeed, there does appear to be a small but consistent effect across studies (Boynton, 2009). And again, a small effect in firing rates could still result in a reliable fMRI signal change.

As for binocular rivalry, it is true that electrophysiological responses from neurons in higher visual areas track the percept more closely than in V1, there are still a substantial proportion of V1 neurons that follow the percept (Leopold & Logothetis, 1996; Logothetis & Schall, 1989). Again, what may be seen as a small effect for an electrophysiologist may result in a large effect in the indiscriminate fMRI signal.

Discussion

The hope has always been that the BOLD signal is reflecting underlying spiking activity in a reasonable, perhaps linear fashion. Over the years this hope has turned almost into an assumption: the BOLD signal is often simply called ‘brain activity’, ignoring the complicated and poorly understood relationship between hemodynamic changes and the actual underlying neuronal response.

The discrepancies between BOLD and spikes might therefore be a reason to question the validity of fMRI studies. However, it should be noted that in the top-down cases described here the BOLD signal is showing positive effects of attention, awareness, and saccadic suppression whereas the spiking measures typically show null results. In fact, it is very difficult to find a study that fails to show a BOLD effect where it is expected from a monkey electrophysiology experiment. It may sound like heresy, but if fMRI had been invented before monkey electrophysiology, the inability of the spiking signal to detect the influence of these top-down factors probably would have been considered to be a limitation of the electrophysiological method.

According to the first hypothesis for the discrepancy, the BOLD signal is following LFP's more closely than spikes, and that the LFP signal is strongly affected by top-down signals. LFP's presumably reflect synchronized or correlated neuronal activity and are typically associated with incoming input and local processing (Logothetis, et al., 2001). LFP signals may therefore be conveying important information about how information is processed and transferred in the brain. It is an intriguing hypothesis that after all this effort to compare fMRI to spiking activity, the BOLD signal is actually measuring something more functionally relevant than spikes. On the other hand, if these synchronized, correlated signals aren't leaving V1 in the form of spikes, then their functional relevance is not obvious.

The second hypothesis for the discrepancy is that the slow dynamics of the BOLD signal are measuring top-down neurophysiological influences that are occurring later than typical windows used in electrophysiological recordings. This hypothesis is easily testable by simply lengthening the window in time that the electrophysiologists use in their measurements. Actually, no new experiments are needed: the results from say, an attentional study in macaque V1, must be sitting on someone's computer file system somewhere.

The third hypothesis is that the fMRI signal is more sensitive than the electrophysiological recording method due to massive pooling by the vascular system. This could explain why fMRI is able to measure what might be very weak changes in neuronal firing rates. This matches well with the fourth hypothesis that there may actually be weak but reliable effects of attention and awareness in the firing rates of V1 neurons.

Conclusions

20 years ago physicist provided neuroscientists with a device that works much as expected when aimed at a simple target, like a stimulus driven signal. But, when it is aimed at a distant target that is less well understood, such as top-down manipulations due to attention or awareness, the device can detect things that are not expected based on the results from other standard technology. It would be as if physicists provided astronomers with a new mysterious telescope that works with something other than light. When pointed at an obvious target like the moon, the reconstructed images make sense, but when pointed at the stars, unexpected information is revealed about these distant objects.

The first three hypotheses for these unexpected results all assert that there is something in the electrophysiological signal that's driving the BOLD signal, but it's not the traditional stimulus-locked average of immediate spiking activity. Instead, the BOLD signal might be reflecting the electrophysiological signal at either a different frequency or time, or may simply be more sensitive to amplitude. Note that the four hypotheses are not mutually exclusive. The real answer probably involves a combination of them all.

The discrepancies described above are concerning, but they could also provide a clue about how this ‘new’ device works. In vision science, illusions are exploited to study how the visual system works by studying vision under conditions that the system does not work as expected. Analogously, the best insights into the hemodynamic coupling process are likely to be made through comparisons of electrophysiology and BOLD signals specifically in top-down conditions. The four hypotheses described here are testable, and in fact data supporting or rejecting them may already have been acquired.