Spatial Homogeneity and Task-Synchrony of the Trial-Related Hemodynamic Signal

Abstract

There is growing evidence that functional brain images in alert task-engaged subjects contain task-related but stimulus-independent signals in addition to stimulus-evoked responses. It is important to separate these different components when analyzing the neuroimaging signal. Using intrinsic-signal optical imaging combined with electrophysiology we had earlier reported a particular ’trial-related signal‘ in the primary visual cortex (V1) of alert monkeys performing periodic fixation tasks. This signal periodically modulated V1 tissue blood volume, in time with anticipated trial onsets. Unlike visually evoked blood volume changes, however, this signal was present even in total darkness. Further, it could not be predicted by concurrently recorded spiking or local field potentials. Here we use our earlier recording techniques to analyze the spatial distribution of this trial-related signal over our imaged area (10 mm square, subdivided into a 16 × 16 grid, i.e. at 625 micron resolution). We show that the signal is spatially coherent and essentially homogeneous over the imaged region and fails to be predicted by concurrent electrode recordings even at the resolution of a single grid square at the electrode tip. As a corollary we show that the signal is critically linked to the animals’ engagement in a task. Not only does the trial-related signal entrain accurately and precisely to any task timing at which the animal was willing to perform; the signal also loses the entrained trial-locked pattern dramatically, within a single trial, when the animal stops performing correctly. Thus the signal is very unlikely to be an ongoing task-independent vascular oscillation. These findings will help categorize the likely distinct varieties of non-stimulus-related signals evoked during behavioral tasks, and lead to a further understanding of the elements comprising the net neuroimaging response.

Introduction

Functional neuroimaging – such as fMRI – is used primarily as a proxy for local neural activity to be then related to behavioral processes or sensory inputs. However, there is a growing appreciation of the complexity of the brain hemodynamics that form the basis of neuroimaging (Logothetis and Wandell, 2004;Logothetis, 2008;Attwell and Iadecola, 2002;Iadecola and Nedergaard, 2007;Raichle and Mintun, 2006). Thus while it was earlier assumed that hemodynamics directly reported local spiking (Rees et al., 2000;Heeger et al., 2000), current evidence shows that the signal likely reflects a complex mix of afferent synaptic inputs, feedback, and neuromodulatory control (Logothetis et al., 2001;Mathiesen et al., 1998;Hamel, 2004;Krimer et al., 1998). Moreover, the brain processes that underlie the neuroimaging signal, even in early sensory areas, are likely to comprise much more than just stimulus-driven input. Rather, the signal – particularly, when recorded from alert subjects performing behavioral tasks – contains a number of non-sensory components including ongoing activity (Fox et al., 2006), anticipation (Kastner et al., 1999;Ruff and Driver, 2006;Serences et al., 2004;Sylvester et al., 2008), attention (Ress et al., 2000;Silver et al., 2007;Moradi et al., 2007), and task structure (Jack et al., 2006;Donner et al., 2008;Sylvester et al., 2007). Thus the challenge in functional neuroimaging is to reliably separate these distinct signal components as a step towards studying their behavioral and neural correlates.

A recent study in our laboratory has allowed us to further inform the ongoing debates regarding the composition of the neuroimaging signal (Sirotin and Das, 2009). We recorded neural signals (using electrodes) from alert monkey primary visual cortex (V1) simultaneously with hemodynamics (using intrinsic-signal optical imaging (Bonhoeffer and Grinvald, 1996;Shtoyerman et al., 2000), a high-resolution optical analog of fMRI (Fukuda et al., 2006) that visualizes local changes in blood volume and oxygenation (Sirotin et al., 2009;Devor et al., 2003;Sheth et al., 2004;Nemoto et al., 2004). When the monkeys performed a periodic visual task we found a prominent hemodynamic signature of the trial structure independent of visual input. This signal – hereafter, the ‘trial-related signal’ – entrained to anticipated trial onsets even in essentially complete darkness and was on average about 40% as strong as the strongest visually evoked signal. Notably, the trial-related signal was dramatically different from visually evoked signals in its relationship with local neural responses. The visually evoked hemodynamics could be reliably predicted from local neural activity (multi-unit activity, MUA, or gamma-band local field potentials, LFP) using a standard hemodynamic response function (HRF). The trial-related hemodynamics, on the other hand, could not be so predicted by these local neural measures or any other locally recorded LFP signal down to 2 Hz (Sirotin and Das, 2009). A question raised by our finding was: how does the trial-related signal interact with visually evoked responses recorded from subjects engaged in structured visual tasks? And: how does the presence of the trial-related signal affect the interpretation of functional brain images?

In this paper we characterize the spatial distribution of the trial-related signal, and we show that this signal depends critically on the animal correctly performing a behavioral task.

In analyzing the spatial distribution we test two specific alternatives arising from two possible functional models of the signal as suggested by earlier literature. In the first alternative, the trial-related signal could reflect a form of attention relevant to the animal’s task. The animal needs to periodically maintain tight visual fixation on a small cue (a tiny pinhole, 1-2 arc min, i.e. ~ 1-2 cone diameters in extent), noting any change in its color (green = ‘fixate’, red = ‘relax); for some trials the animal also needs to disregard a bright visual stimulus at a few degrees of eccentricity (typically ~ 2-3 degrees). In human subjects, tasks of a similar nature reliably evoke a BOLD fMRI response that is likely a manifestation of attention. This signal is sharply peaked at the fovea, the focus of attention, with a ‘Mexican Hat’-like ring of suppression that is strongest at the eccentricity of distracters (in our case, the visual stimulus at an eccentricity of 2-3 degrees) (Smith et al., 2000;Heinemann et al., 2009;Muller and Kleinschmidt, 2004;Muller et al., 2005;Hopf et al., 2006a;Slotnick et al., 2002). As noted by some commentators, such an attention-like signal would have a sharp gradient and a zero crossing in the region of V1 that we were imaging, dropping down from positive values at the foveal end of the recording chamber to negative values at the opposite end (Kleinschmidt and Muller, 2010). The second alternative that we tested is that the trial-related signal may be similar to the ‘global response component’ or the ‘task structure activation’ seen in some recent human fMRI studies. These signals – found in retinotopic visual cortex – are associated with the subject’s act of making a response, and are distinct from both attention-like and stimulus-evoked responses. In spatial distribution, these signals were found to be largely uniform and non-specific, extending mainly over V1 (‘task structure activation’: stronger in peripheral parts of the visual field (Jack et al., 2006) or over much of lower retinotopic visual cortex, areas V1-V4 (‘global response component’: (Donner et al., 2008). In our imaging chamber such signals would be essentially flat and unstructured over the entire imaged area.

For this study we used our published technique of intrinsic-signal optical imaging combined with electrophysiology in alert macaque V1 (Sirotin and Das, 2009). We spatially segmented the imaged (10 × 10 mm) cortical region into a 16 × 16 grid (625 × 625 micron grid squares) and mapped the signal correlation and amplitude at the resolution of these grid squares. In addition, we used a recording electrode to sample neural activity from one of these grid squares. Based on this local measure of neural activity, we obtained the spatial distribution of the predictability of the hemodynamic response, i.e. we calculated how well the hemodynamic signal in each grid square could be predicted based on the neural signal at this one location. For comparison with the trial-related signal, we also performed the same analyses on a hemodynamic signal (the response to a small, localized visual stimulus presented during the periodic fixation trials (Sirotin et al., 2009) that did have a sharp gradient including a zero crossing in our imaged region, similar to the ‘Mexican Hat’-like signal expected for focused attention.

In brief, we found that the trial-related signal had homogeneously flat maps of signal correlation and amplitude over V1, and was uniformly poorly predicted by concurrent electrode recordings. This was completely unlike the ‘Mexican Hat’ signal simulated as the response to a local visual stimulus. This latter had strongly structured maps of signal correlation and amplitude and an equally structured pattern of match to electrode recordings: high positive correlation at the electrode tip, which became progressively poorer and even reversed sign with distance.

In addition we show that the trial-related signal is critically linked to the animals’ engagement in a task and is thus very unlikely to be a task-independent ongoing vascular oscillation. Unlike an ongoing oscillation, the trial-related signal entrains with remarkable accuracy and precision to the task in which the animal was engaged, over the full range of task periods at which the animal was willing to perform. As a corollary, the signal flattens out or otherwise changes shape rapidly – in as little as a single trial – as soon as the animal stops performing correct trials.

These results build at a number of levels on our earlier published findings regarding the trial-related signal. First, we more firmly establish the task-dependent nature of this signal. Next, by characterizing the spatial distribution of this signal we lay the groundwork for studying its interactions with visually evoked V1 responses. Finally, by demonstrating that it is flat over imaged V1, unlike retinotopically focused attention signals, our results suggest the presence of at least two distinct classes of non-sensory hemodynamic signals in task-engaged subjects: a class of spatially localized attention-like signals; and a distinct class of spatially flat global task-timing-related signals, including our trial-related signal. Thus these findings should help lead to a broader understanding of non-stimulus-related brain imaging signals manifested during behavioral tasks.

Methods

The methods are similar to those described in our earlier published work (Sirotin and Das, 2009;Sirotin et al., 2009). All results were obtained using a technique that we developed to, in-effect, simultaneously image blood volume and oxygenation along with concurrent electrode recordings from alert behaving macaques. The blood volume and oxygenation responses were deduced using intrinsic signal optical imaging, a technique based on measuring changes in the absorption of light reflected off the cortical surface (Bonhoeffer and Grinvald, 1996). Hemodynamic signals obtained with this technique are well matched to those obtained using fMRI (Fukuda et al., 2006). We imaged at two wavelengths, alternated rapidly (15 frames / sec overall, 7.5 frames / sec per wavelength) relative to hemodynamic signal changes. One wavelength, 530 nm, is equally absorbed by oxy- and deoxyhemoglobin (Prahl, 2007). Thus increased absorption of light at this ‘volumetric’ wavelength reports local increases in tissue hemoglobin concentration without distinguishing between oxy- and deoxyhemoglobin. The signal thus effectively measures changes in local cerebral blood volume, CBV. (Footnote: While this measure of blood volume is comparable to CBV in fMRI, the two measures are not entirely equivalent. The fMRI measure of CBV monitors the volume of blood plasma in tissue. The two measures of CBV will differ if the blood hematocrit changes). The second wavelength, 605 nm, is roughly 5-fold more strongly absorbed by deoxy- than by oxyhemoglobin. Thus changes in light absorption at this ‘oximetric’ wavelength carry information about changes in blood oxygenation independent of changes in total hemoglobin or CBV. (Footnote: For later experiments which did not require simultaneous measurements of oxy- and deoxyhemoglobin we only used the volumetric wavelength, at 15 frames / sec). It is important to note that we imaged continuously rather than synchronized to trial onsets. This ensured that we could compare recordings in all phases of each trial.

All experimental procedures were performed in accordance with the NIH Guide for the Care and Use of Laboratory Animals and were approved by the Institutional Animal Care and Use Committees (IACUC) of Columbia University and the New York State Psychiatric Institute.

Task

The animals’ task required only passive fixation. They were trained to fixate and relax, periodically, cued by the color of a fixation cue (green= ‘fixate’; red = ‘relax’; fixation window: 0.5^O radius; monitor distance: 133 cm; fix duration: 4 sec starting shortly after each trial onset, with a juice reward if the animal held fixation correctly; total trial durations ranging from ~7 – 20 sec). Eye fixation and pupil diameter were recorded with an IR eye tracker (Matsuda et al., 2000). To characterize the trial-related signal with minimal contamination by visual input, we had the animals perform the majority of fixation experiments reported here in a completely dark room, with a mask covering even the stimulus presentation monitor. The fixation cue was visible just through a pinhole in the mask (size 1-2 arc min.); the cue is very difficult to see except when looked at directly (see (Sirotin and Das, 2009). For these dark-room trials we dark adapted along with the animal in the dark room to ensure that nothing else was visible other than the fixation cue. For a subset of the experiments we presented the animal with a visual stimulus on a normal monitor background.

Optical imaging: surgery, recording chambers, artificial dura, imaging hardware

We used standard alert-monkey optical imaging procedures (Shtoyerman et al., 2000). After the monkeys were trained on the visual fixation task, craniotomies were performed over the animals’ V1. Glass-fronted stainless steel recording chambers were implanted, under surgical anesthesia, using standard sterile procedures. The exposed dura was resected and replaced with a clear silicone artificial dura, arranged to image a ~ 10-mm diameter area of V1 covering a visual eccentricity of ~1^O – 5^O. After the animals had recovered from the surgery, cortical activity from their V1 was optically imaged, routinely, while the animals performed their task. Recording chambers and artificial dura were fabricated in our lab using published methods (Arieli et al., 2002).

To image the cortex we used a Dalsa 1M30P camera (binned to 256 × 256 pixels, 15 frames / sec), with a frame grabber: Optical PCI Bus Digital (Coreco Imaging, Boston, MA). The software was developed by us, based on a system by V. Kalatsky (Kalatsky and Stryker, 2003). Illumination was provided by high intensity LEDs (Agilent Technologies, Purdy Technologies), with emission wavelengths centered at 530 nm and 605 nm, filtered through individual interference filters (Omega Optical). We used a home-built circuit to power the LEDs and switch them on and off alternately, in synchrony with each camera frame. The cortex was visualized using a ‘macroscope’ of back-to-back camera lenses focused on the cortical surface (Ratzlaff and Grinvald, 1991). Images were obtained continuously, simultaneously recording signals from the camera, trial timing and behavioral data (trial onset and offset, eye position, pupil size, time stamps of the animal acquiring or breaking fixation; trial outcome, i.e. whether the trial was successfully completed; visual stimulus onset and offset for the trials that employed a visual stimulus)

Optical imaging: image processing

All images were first corrected for residual brain movements by aligning spatially to the first frame of the experiment, assuming only solid body movement artifacts (shift + rotation) (Lucas and Kanade, 1981). The blood vessel pattern of the first frame was used as the template, and alignment was effected using a gradient descent algorithm based on minimizing the sum square difference with subsequent frames. In addition, a pulse artifact was removed using Runline (Chronux Toolbox for MATLAB) a LOWESS (locally weighted scatterplot smoothing) local regression technique (regression window: 1 sec, moved in 0.4 sec steps),. The raw time course of the mean hemodynamic signal was obtained as the average of the signal over the imaged area, frame by frame.

Next, movies of the cortical signal (7.5 frames / sec or 15 frames / sec) were obtained by aligning image sequences temporally to the trial onset and averaging, frame by frame, across all trials where the animal correctly held fixation. Each pixel of the movies was divided by the mean of the signal at that pixel, averaged over the trial duration. This normalization procedure expresses the signal as percent change relative to the mean over the trial. For some experiments, as a control, the signal was normalized not by the trial mean but rather, the mean of the frames immediately prior to trial onset, thus expressing the signal as a fractional change following trial onset (Supplementary figures).

Optical imaging: power spectrum analysis (Fig 5)

Figure 5.

Trial-related signal entrains precisely to trial timing, for all trial periods tested. a: Section of the raw hemodynamic signal, averaged over imaged area, for a sequence of correct trials. Trial period = 6.8 sec. Monkey fixated and relaxed on schedule indicated along the lower line: ‘Fixate’: 4-sec fixation; ‘Relax’: inter-trial interval. Black vertical lines: trial onsets. b: Same experiment, mean of all trials where animal performed at least 3 correct trials in a row (N=425 trials). c,d: Corresponding figures for a different experiment with trial period = 10.1 sec, N=54 trials. e,f: Same, for period = 15.4 sec, N=78 trials. g: Left: Power spectrum of hemodynamic signal in panels a,b: as a function of time in the experiment (moving 200-sec window in 20-sec steps). The black arrowhead on the left (1/6.8) shows the frequency (0.147 Hz) corresponding to the trial period of 6.8 sec. Right: Mean of power spectrum, averaged over the full experiment. The dashed vertical line at 0.146 Hz marks the midpoint of the frequencies bounding the spectral peak, and defined = the peak frequency for the spectral power. This value was then converted to the corresponding period (=6.85 sec) to compare with trial period. Vertical arrowhead marks the frequency (0.147 Hz) corresponding to 6.8 sec. h: Same, for all 3 experiments in panels a-f. i: Population scatter plot, showing periods obtained from the power spectrum peak (as in panel g, right hand side) plotted against trial period, for a representative set of 10 experiments with periods ranging from 6.8 sec to 19.4 sec. Black dots: data points, one per experiment. Red line: regression line (y=0.96x + 0.45) where x = trial period (in sec), y = period obtained from spectral peak (in sec).

The full timecourse of the signal –averaged over the full imaged area, frame by frame – was first high-pass filtered at 160 sec using Runline to remove very low frequency drifts and fluctuations (order of 0.01 Hz and lower). Sequences of correct trials were then extracted and concatenated. This typically generated long continuous sequences of correct trials with only a small number of discontinuities where correct sequences were joined to each other (~9% of error trial breaks, i.e. ~ 9 breaks in every sequence of 100 trials giving sequences of >~ 10 correct trials in a row). These concatenated correct trials were then spectrally analyzed as a function of time in the experiment using a multi-taper analysis routine (mtspecgramc. Chronux Toolbox for MATLAB), with a 200-sec moving window stepped in 20-sec steps.

Electrophysiology

(Fig 3,4): Extracellular electrode recordings were made simultaneously with the optical imaging, using a low-profile electrode holder that fit under the imaging optics, custom designed and fabricated at Columbia. Commercial, plastic-insulated or glass-coated etched tungsten electrodes (FHC, Alpha Omega; impedances 300-800 KΩ) were positioned using regular micromanipulators (Narishige: mechanical for coarse, hydraulic for fine movements). Over the course of the project, recording sites sampled cortical depths starting from the most superficial to about 1500 microns below the pial surface.

Figure 3.

Concurrently recorded spiking is uniformly poor at predicting the trial-related signal across the entire imaged V1, unlike the structured prediction seen with the ‘simulated Mexican Hat’. ‘Visually Stimulated’ a-c: Fit of spiking to hemodynamic responses for the ‘simulated Mexican Hat’, i.e. same data set as in Fig 2. N = 31 trials. a: Raw traces from a sequence of trials comparing spiking (black) with predicted (blue) and measured hemodynamics (green, red) from two sites marked as in Fig 2 (green dot = electrode tip). The inset shows the optimal HRF kernel obtained by fitting the spiking to the hemodynamic signal at grid square containing the electrode tip. The predictor = the convolution of this HRF kernel with the spike trace. Top: ‘Electrode site’: grid square containing the electrode. Note the very good match between the predicted and measured hemodynamics, particularly for the increases in blood volume following each burst of spiking (downward peaks: e.g. green arrowhead). The predictor is consistently unable to match the wiggles at trial boundaries (small black arrow) Middle: ‘Trough’: from location marked with red dot, same location as in Fig 2. The measured signal is roughly in antiphase with the predictor, giving negative values of r, and R². b: map of r; inset: Black trace (+/− 1 sd: red trace) shows mean r as a function of radial distance from the electrode, calculated by taking the means on annular rings encircling the electrode site. c: Map of R², inset: R² as a function of radial distance from electrode. ‘Dark-room’ d-f: As in a-c, but from a set of fixation trials in complete darkness, performed on the same day with the electrode in the same location. N = 29 trials. Note the marked similarity of the raw measured hemodynamic traces from the two sites with each other (panel d), and the consistently poor match with the predictor at all sites.

Figure 4.

Summary data for goodness of fit: (R²: coefficient of determination) for predictors derived from spiking and all LFP frequency bands. Top: matches of the different optimal predictors to hemodynamics for visually stimulated trials. All LFP bands carry some information about the spatial structure of the hemodynamic signal, though predictors from spiking and high gamma (66-130 Hz) give the best match. Same data set as in Fig 3a-c. N = 31 trials. Bottom: matches of optimal predictor to dark-room trials. Note the uniformly flat maps of R² all essentially at 0. Same data set as in Fig 3d-f; N = 29 trials.

The electrode signal was amplified and processed separately for spiking and LFP (Plexon amplifier and recording software; ‘spiking’ : 100Hz-8KHz bandpass; ‘LFP’: 0.7-170 Hz bandpass). ‘Multi Unit Activity’ (MUA) events were defined from the spiking signal as each negative-going crossing of a threshold, which was set at roughly four times the root mean square (RMS) of the baseline electrical activity recorded while the animal looked at a gray screen. The MUA was then binned into 133.33 msec bins (corresponding to the 7.5 Hz camera frame rate) and aligned temporally to the hemodynamic signal using simultaneously recorded synchronization events. The LFP data were spectrally decomposed using mtspecgramc (Chronux Toolbox for MATLAB; sliding window of 1 sec, step size of 250 ms, frequency range 2-130 Hz) and interpolated into a continuous power spectrum aligned to the hemodynamic traces. The LFP time courses were calculated separately for each frequency band: the ‘gamma band’ consisted of the signal from 66-130 Hz. All other LFP signals, from 2 Hz to 50 Hz, were separated into 6-hz bands (2-8 Hz, 8-14 Hz, 14-20 Hz, 20-26 Hz, 26-32 Hz, 32-38 Hz, 38-44 Hz, 44-50 Hz).

Fitting electrode signals to hemodynamics

To calculate the correlation between the electrophysiology and hemodynamics we first calculated, for each band of the electrode signal (MUA, each LFP band), the optimal HRF kernel that ‘best’ predicted the measured hemodynamics from the given electrophysiology. The prediction using this optimal kernel was then matched with the measured hemodynamics to give a measure of the goodness of fit.

The optimal HRF kernels were calculated as follows: we fitted the hemodynamic signal recorded from the particular grid square containing the electrode tip, to the concurrent electrode signal. This was done separately for each band of the electrophysiological signal (i.e. MUA and the different LFP bands). For each fit we assumed that the hemodynamics can be predicted from electrophysiology using a causal gamma-function kernel (i.e. non-zero only for times t > 0). This is a standard functional form that can be parametrically expressed in terms of the amplitude, time to peak and full width:

$$\mathit{HRF}(t,T,W,A)=A^{\ast }{\left(\frac{t}{T}\right)}^{\alpha }\ast \exp \left(\frac{t-T}{-\beta }\right)$$ (1)

where α = (T /W)² *8.0*log(2.0), β = W² /T /8.0 / log(2.0), and A is the amplitude, T is the time to peak, and W is the full width at 75% maximum.

For each fit we started with an initial predictor, obtained by convolving the electrophysiological signal with the initial kernel (starting parameter values: A < 0, T ~= 3 sec, W ~= 3 sec). The optimal kernel was then obtained by parametrically varying A, T and W to minimize the sum of square of the residual, i.e. of the difference between the predictor and the recorded hemodynamic signal. We used a downhill simplex algorithm: fminsearch, MATLAB. Note that we performed this fit using the full, raw sequences of hemodynamic and the relevant electrophysiological signals, concatenated together for all the trials where the animal held fixation correctly; we did not use periodic traces constructed from the mean signals since that can lead to errors of fitting (Das and Sirotin, 2011).

Results

The results are presented in four sections. The first three sections address the issue of spatial homogeneity while in the fourth section we show that the signal is critically dependent on the animal being engaged in a behavioral task.

1: The Trial-related Signal is Homogeneous over Imaged V1

These results were obtained from three rhesus macaques, monkeys ‘S’, ‘T’ and ‘V’. ‘S’ and ‘V’ had contributed to our published work while monkey ‘T’ is new.

The homogeneity of the trial-related signal over the 10-mm extent of the imaged region of V1 can be appreciated most clearly from the ‘blood volume’ image, i.e. the one obtained using the ‘volumetric’ green (530 nm) LED illumination (Fig 1 shows a representative example). Over the course of the trial cycle the image changes from a featureless uniform gray, to an overall increase in brightness over the entire image, down to an overall darkening of the whole image, and back to neutral gray (see the individual image frames, Fig 1a). Since increased absorption at this wavelength measures increased local tissue blood volume (Sirotin et al., 2009;Devor et al., 2003;Sheth et al., 2004;Nemoto et al., 2004), the cycle of brightening followed by darkening corresponds to a cyclic reduction followed by an increase of blood volume relative to the trial mean, over the imaged surface. As we noted in our earlier publication, the overall brightening (blood volume reduction) is temporally correlated with a further relative brightening (constriction) of pial arteries over the imaged surface, while the overall darkening is correlated with a further darkening (dilation) of the arteries ((Sirotin and Das, 2009); also, supplementary Fig S1 indicates how arteries and veins are identified). This additional brightening and darkening of the arterial network stands out over the mean signal change. But this pattern is largely uniform and synchronous over the visible arterial network.

Figure 1.

Trial-related signal is largely uniform over imaged V1 surface. Signal imaged at the ‘volumetric’ wavelength (green, 530 nm. NOTE: All figures in this paper show the volumetric signal alone, except where specifically mentioned, in Supplementary Fig S1). a: Top: Time course of signal. Green trace: mean signal over entire imaged surface (downward signal = increased blood volume) averaged over 51 trials. Gray: signal in each of 16 × 16 squares tiling the imaged surface. Bottom: images of signal at three selected time points, expressed as fractional change over the trial mean at each grid square. Grayscale key on the right. The edges of the 16 × 16 squares are indicated as the black and white line segments surrounding each image (to avoid clutter, the full 16 × 16 grid of squares is not shown; they are clearly visible in panels b,c). b: Image of correlation (Pearson’s r) of the signal in each individual tile (gray traces in panel a top) with the mean across all 255 grid squares excluding the current square. (Value of r color coded, key to the right). c: ‘Amplitude’: map of the signal amplitude (see text for definition). Note the high amplitudes on grid locations over the prominent artery signal evident in the individual images, panel a. ‘Mask arteries’: Amplitude map obtained after masking the arteries. Mask shown in inset; defined as pixels where the signal variance is higher than a selected threshold. ‘Fit plane’: plane fitted to ‘Mask arteries’ signal using 2-D linear regression. Red arrow indicates direction of gradient (magnitude = 0.012 / mm of cortex). Color coding common to all three images as well as panel d. d: 3-D plot of the fitted plane; note that it has a very shallow tilt, with value close to 1 over the entire imaged area. The base of the plot shows an image of the vasculature over the same region, along with lines of azimuth (x) and elevation (y) with respect to the fovea, obtained from earlier measurements of retinotopy. The black arrow indicates the approximate direction of the foveal representation while the red dashed arrow indicates the gradient of the fitted plane (approximate angular difference: 22 deg). e: Histogram of angular differences between gradient of fitted plane and the direction of foveal representation, over the population of images tested (N=10, 3 monkeys).

To analyze the spatial homogeneity of the signal more closely we subdivided the imaged surface into a 16 × 16 grid and measured the mean signal separately in each grid square (625 × 625 micron, i.e. 0.4 mm² grid squares). Though the amplitude of the mean signal varied somewhat for the different grid locations, the time courses of the signal in each grid square match each other closely (Fig 1a, individual thin gray traces). We quantified this match by correlating the signals with each other in the following manner: for each grid square we correlated the signal in that square with the mean of the signals from the 255 other squares. As a measure of correlation we calculated Pearson’s ‘r’ over the trial length (~50 – 150 time points, corresponding to ~7-sec to 20-sec trials sampled at 7.5 frames/sec). For the given example the mean r thus calculated was 0.97 (0.0007) [N = 256; all averages expressed as mean (sem)], implying that the signals at each grid square were highly correlated with each other, i.e. synchronously positive or negative at each point on the imaged region, over their time course. Such a high r value precludes any possibility of the signal crossing the baseline and reversing sign within the imaged region of cortex (see the next section for the correlation pattern expected when the signal has a zero crossing). Over the population of images, this mean r (256 grid squares for each image) equaled 0.94 (0.01), [N=10 images, 3 monkeys], implying a similarly highly correlated signal in each case.

It could be argued that showing spatial uniformity of high correlation is not enough to demonstrate uniformity of the signal over the cortical surface. High correlation implies similarity of time course independent of signal amplitude. Thus despite the uniformly high correlation there could still be a systematic variation of the signal amplitude over the imaged area of cortex. We tested for this possibility by plotting the signal amplitude over the gridded cortical surface. The choice of a measure of amplitude required some deliberation. A simple measure of signal strength, such as standard deviation, is inadequate: it is blind to spatial variations in the sign (polarity) of the signal; further, it does not distinguish between signal and noise (i.e. random variation in the signal at different spatial locations). Thus we decided to use, as our measure of amplitude, the normalized dot product of the signal S_i,j in each grid square with $\stackrel{‒}{S}$, the mean signal averaged over the full area. For each grid square (i, j) this normalized dot product A_i,j is defined thus:

$$A_{i,j}=\frac{S_{i,j}\cdot \stackrel{‒}{S}}{\mid \stackrel{‒}{S}{\mid }^2}$$ (2)

The numerator of this expression is the regular dot product between S_i,j and $\stackrel{‒}{S}$ defined over the (50 – 150-dimensional) vector space of trial length, while the denominator normalizes the dot product by the squared magnitude of the mean signal. If S_i,j were identical to $\stackrel{‒}{S}$ in both time course and amplitude the corresponding A_i,j would be equal to 1. Values other than one could arise due to differences in signal time course or amplitude. Here, since the signals at each grid square had essentially identical time courses (Fig 1a), variations in the value of A_i,j directly reflect variations in amplitude.

(Note that for mean-subtracted signals, as the ones used for this analysis, the amplitude thus defined is closely related to Pearson’s r. Pearson’s r between S_i,j and $\stackrel{‒}{S}$ can be expressed as: $r_{i,j}=\frac{S_{i,j}\cdot \stackrel{‒}{S}}{\mid S_{i,j}\mid \mid \stackrel{‒}{S}\mid }$. Unlike the A_i,j, Pearson’s r is normalized by the product of the magnitudes of $\stackrel{‒}{S}$ and S_i,j giving, in effect, the cosine of the ‘angle’ between S_i,j and $\stackrel{‒}{S}$ independent of signal amplitudes.)

The amplitude map thus generated (Fig 1c) did show a pattern of inhomogeneity with sharply defined regions of high signal amplitude. Comparison with the image of the arterial signal (Fig 1a: time points 8 sec, 13.3 sec) suggests, however, that this pattern simply corresponds to the high signal at the arteries, pixellated at the resolution of the grid squares. To get a better measure of the amplitude map over cortical tissue away from the arteries we recalculated the signal amplitude after masking the arteries, i.e. calculating the mean signal amplitudes only over those portions of each grid square that lay outside the artery mask (Fig 1c, ‘Mask arteries’). We then fitted a regression plane through these recalculated signal amplitudes (Fig 1c, ‘Fit plane’).

The regression plane thus obtained has a shallow gradient, 0.012 / mm of cortex, equaling 1.2%, i.e. a total change of just 12% in the height of the plane from its highest to its lowest point over the imaged 10 mm of cortex (Fig 1d). We had earlier mapped the retinotopy over this region of cortex by imaging V1 responses to lines and small bars at defined locations in visual space. This allowed us to locate the lines of azimuth (X) and elevation (Y) over V1 and identify the approximate direction towards the fovea which lay roughly 1 deg outside the imaged area. Comparing with the gradient of the fitted plane shows that the direction of highest upward slope in the plane lies, in this case, less than 25^O away from the direction towards the fovea; the angle could be smaller given the approximations involved in estimating the direction of the fovea. Thus it is possible that the signal amplitude has a small gradient sloping down from the fovea. But this slope is shallow and it rules out any zero crossing in the imaged region. This value of the slope is likely to be an upper limit. Inspection of the artery map shows a high concentration of pial arteries in the upper right quadrant of this particular region of V1, i.e. the general direction of the fovea. Thus even the measured slope may only be an artifact of the higher residual arterial signal in that direction (see Fig 1a, images and Fig 1c, vascular map of arteries).

A similar calculation for our population of images, i.e. fitting a regression plane through each image, suggests that any measured slope in the signal intensity over the imaged cortex is largely a reflection of signals in arteries that happen to lie to one or the other side of the imaged region. The slope of each fitted plane was modest (mean slope: 0.02 (0.004) / mm, N=10, 3 animals) and in each case it matched the direction of the larger arteries in the image. Over the population, this direction was unrelated to that of the fovea and even included directions essentially opposite to the fovea (Fig 1e). In particular, we saw no evidence of consistent peaks of the signal towards the fovea.

The hemodynamic signal imaged at the ‘oximetric’ wavelength, 605 nm, was similarly flat, on average, rising and falling synchronously over the imaged surface (Supplementary Fig S1). At this wavelength, veins carry a signal distinct from arteries. Thus the map of signal correlation over V1 is locally more heterogeneous (Supplementary Fig S1b,c) than the equivalent correlation map for blood volume (Fig 1b). However, while the signal time course is distinct in veins and arteries, it is homogenous over the imaged surface for the given blood vessel type. In particular, the oximetric signal, like the blood volume signal, shows no sign of a zero crossing in the imaged area.

We checked whether the observed flatness of the imaging signal could have been influenced by our quantifying the signal as the fractional change relative to the mean (see methods). Our logic for such a quantification is that the trials involved periodic fixations in darkness. Since there is no stimulus, it is not possible to express the signal as a fractional change from a pre-stimulus baseline, which is the typical quantification used. Moreover, since it is a continuous periodic signal, there is no one phase within each period that can be selected as a baseline. As a control, however, we divided the image movie over a trial by the mean for the four frames immediately preceding the trial, thus expressing the signal as the fractional change following trial onset. The results were essentially identical to those obtained with the signal expressed as fractional change relative to the mean, with an identical time course and equally flat maps of signal correlation and amplitude (Supplementary Fig S2, for the same imaging signal as in Fig 1).

2: A simulated Mexican Hat-like image centered near the fovea has correlation and amplitude maps very different from the trial-related signal

To provide a counterpoint for the results obtained with the trial-related signal we performed the same calculations – mapping correlation and amplitude – for a ‘simulated Mexican Hat’, a V1 response that does possess a retinotopic focus and a zero crossing similar to the Mexican Hat-like spatial distribution expected for foveally centered attention signals. The purpose is to show that our analysis methods are quite capable of detecting high spatial gradients and zero crossings when they exist, thus emphasizing that the lack of such features in the trial-related signal (Fig 1) imply a real spatial homogeneity in its distribution.

A hemodynamic signal with just such a Mexican Hat-like shape is evoked in V1 of alert behaving macaques when a brief, spatially localized visual stimulus is presented while the animal fixates periodically. This signal is sharply peaked at the retinotopic locus of the stimulus, dropping with a steep gradient to a periphery of the opposite sign (Fig 2; also, Supplementary Fig S3a). This spatially multiphasic center-surround pattern on cortex likely results from the combination of distinct signal components. We find that, for our stimulation conditions, the stimulus evokes only a monophasic ‘pointspread’ in anesthetized animals (Das and Gilbert, 1995) which can be revealed after blank-trial subtraction in alert animals (Sirotin and Das, 2009); also, Fig S3c and Discussion). The stimulus-independent signals, captured in the blank-trial response, are largely homogeneous over imaged V1, with trial periodicity but at a temporal phase roughly opposite that of the point spread (Fig S3b). Thus, without blank-trial subtraction, the net signal has a center-surround spatial structure and provides an adequate test to see if these features can be detected by our analysis procedure.

Figure 2.

Net hemodynamic signal for a ‘simulated Mexican Hat’ shows high spatial gradient and a zero crossing within imaged region. Conventions as in Fig 1. a: Trial-averaged traces (top: 31 trials) and images at three representative time points (bottom) for the net response to a small visual stimulus positioned near the edge of the imaged region (see text for details of stimulus). Traces in green, black and red identify traces from grid square locations marked by corresponding dots on the image (green = stimulus center). b: Correlation (Pearson’s r) of the signal at each grid square with the signal at the stimulus center. c: Amplitude map. d: Regression plane fitted through amplitude map. Gradient: 0.079 / mm of cortex.

Parenthetically, we must note that our ‘simulated Mexican Hat’ is not truly a Mexican Hat in shape; it is likely not modeled as a difference of Gaussians but rather, as a single Gaussian peak rising from a uniformly flat plane of opposite sign (i.e. temporal phase). Just for brevity, however, we will use the term ‘simulated Mexican Hat’.

As a visual stimulus, we presented a small grating patch (0.25 deg radius, spatial frequency 4 cycles / deg, drift velocity 4 deg / sec, 100% contrast) centered by the edge of our imaged region that was closest to the fovea (stimulus center: (−1.48, −1.06) degrees in visual angle from the fovea). The time course of the evoked V1 hemodynamic response was visualized as a movie, as usual, quantifying the signal in terms of percent change over trial mean at each imaged pixel. As before, the imaged surface was subdivided into a 16 × 16 grid for further quantitative analysis. In addition, we recorded neural activity with an electrode at the retinotopic location of the stimulus center (we mapped the receptive field and preferred orientation at the electrode tip by varying the orientation and location of a 0.25-deg radius grating patch visual stimulus. The optimal grating patch was then used as the visual stimulus for the subsequent imaging and neural recording). To compare visually evoked and trial-related signals we alternated blocks of trials with visual stimulation, as described, with blocks where the animal performed the same fixation task but in total darkness. Concurrent neural and hemodynamic signals were recorded and analyzed separately for the visually stimulated and dark-room blocks. Results from electrode recordings are presented in the next section, while this section describes the spatial distribution of the visually evoked signal.

Unlike the spatially homogeneous trial-related signal, the net hemodynamic response measured with stimulation had sharply varying signal time courses at different grid squares over the imaged surface. This is evident qualitatively from the structured pattern of the image, rising to a peak of increased darkness (Fig 2a, image at 5.3 sec) or increased brightness (Fig 2a, image at 9.3 sec) at the location of the electrode tip, which corresponds to the center of the stimulus (green dot on images). The signal at this location (green trace in Fig 2a) showed a prominent, stimulus-evoked signal reduction, i.e. darkening, that peaks roughly 4 sec after stimulus onset (green arrowhead, Fig 2a). This is due to the stimulus-evoked increase in local blood volume. However, signals at farther grid squares have progressively more distinct time courses, very different from just scaled versions of the time course at the stimulus center. For example, the red trace in Fig 2a, from the location identified by the red dot on the image (about 2 degrees away in retinotopic distance) has a time course that is roughly opposite to that at the stimulus location. This signal rises to peak brightness (reduced blood volume; red arrowhead, Fig 2a) when the signal at the stimulus center is darkening, and vice versa. The signal at an intermediate location (black dot on image) has, correspondingly, an intermediate time course (black trace, Fig 2a; also see Supplementary Fig S3).

We quantified the variation in signal time course over the imaged area by correlating the signal at each grid square with the signal at the stimulus center (Pearson’s r over the 10-sec trial, i.e. 150 time points at 15 frames / sec). Unlike the flat correlation map for the trial-related signal (Fig 1b), the correlation map here was highly structured, dropping from the value of 1.0 at the stimulus center to negative values (min: −0.38) at distal locations 2 – 4 degrees away in retinotopic distance (Fig 2b; for retinotopic scale, compare with the retinotopic map, Fig 1d). Thus, unlike the trial-related signal, the correlation map for this net hemodynamic signal with visual stimulation shows a sharp gradient and a sign reversal at about 2-4 degrees of eccentricity from its center.

As with the trial-related signal (Fig 1c) we also calculated the map of signal amplitude A_i,j, defined here as the normalized dot product of the signal S_i,j in any given grid square (i, j) with the signal S_PEAK at the stimulus center:

$$A_{i,j}=\frac{S_{i,j}\cdot S_{\mathit{PEAK}}}{\mid S_{PEAK}{\mid }^2}$$ (3)

where ∣S_PEAK∣ is the magnitude of S_PEAK. The amplitude map thus defined was also prominently structured, like the correlation map. Unlike the case with the trial-related signal where the amplitude was positive and homogeneous over the entire imaged area and regions of high amplitude just corresponded to the larger arteries, the signal amplitude here is high at the stimulus center and drops monotonically to negative values farther away (Fig 2c, compare with Fig 1c). A regression plane fitted to this amplitude map was sharply tilted, crossing the baseline into negative values well within the imaged area of cortex (Fig 2d: gradient = 0.079 / mm of cortex, i.e. 79% over the 10 mm imaged area). Our analysis techniques can thus provide clear evidence of spatial gradients and zero crossings when these features do exist in an image. Further, the lack of such features in the trial-related signal cannot be due to poorer signal to noise: the amplitude range (~ 5% peak to peak: Fig 1a) of the trial-related signal for the particular data set in Fig 1 is actually higher than the amplitude range for the signal in Fig 2 with visual stimulation (~ 3% peak-to-peak, Fig 2a).

3: Unlike the ‘simulated Mexican Hat’ signal, the trial-related signal is poorly predicted overall by concurrent electrode recordings

Next we wanted to test how well the trial-related signal at different locations in the image was predicted by the electrode signal recorded at one location. In particular, there was some concern that neuronal spiking may reliably predict the trial-related signal near the electrode tip but could be uncorrelated or anticorrelated with the trial-related signal farther away leading to an apparent – but incorrect - lack of correlation with the signal in the mean (Kleinschmidt and Muller, 2010).

We decided to test for this possibility by mapping the predictability of the hemodynamic signal from the concurrent electrode recording, for all grid squares at varying distances from the recording electrode. We did this analysis not just for spiking (MUA) but systematically for a full set of LFP bands: high gamma (66-130 Hz) and all LFP bands from 50 Hz to 2 Hz at 6-Hz intervals (LFP bands 44-50 Hz, 38-44 Hz, 32-38 Hz, 26-32 Hz, 20-26 Hz, 14-20 Hz, 8-14 Hz and 2-8 Hz; we skipped 50-66 Hz to avoid line pickup). We first fitted each electrophysiological signal to the hemodynamic signal from the specific grid square containing the electrode tip. We obtained, thereby, the optimal gamma-function HRF kernel that best predicted the local hemodynamic trace from the given MUA or LFP (see methods). We then quantified the match between the optimal predictor thus obtained – convolving the optimal HRF with the appropriate MUA or LPF – with the recorded hemodynamic signal not only at the grid square containing the electrode tip but at each of the 16 × 16 grid squares.

We used two different measures of goodness of fit. First, to relate signal predictability to our earlier maps of signal correlation, we calculated the Pearson’s r between predicted and measured hemodynamics. However, Pearson’s r is independent of signal amplitude and does not give a measure of the fraction of the signal strength (variance) that is explained by the predictor. To quantify the latter we also calculated, for each grid point (i, j), the coefficient of determination ($R_{i,j}^2$) defined thus:

$$R_{i,j}^2=1-\left(\frac{{\sigma }_{{\mathit{ERROR}}_{i,j}}^2}{{\sigma }_{i,j}^2}\right)$$ (4)

where ${\sigma }_{i,j}^2$ is the variance of the measured signal S_i,j, while ${\sigma }_{{\mathit{ERROR}}_{i,j}}^2$ is the variance of the ERROR_i,j = (S_i,j − predictor) i.e. the difference between the measured and predicted signals. This measure is used commonly in the literature; see, e.g., (Logothetis et al., 2001). As a counterexample to the trial-related signal, we carried out the identical calculation – i.e. obtaining the optimal fit and then measuring the goodness of fit over the image – for the ’simulated Mexican Hat’ signal described in the last section. For the latter case the optimal HRF kernel fitted the recorded electrophysiology to the hemodynamic signal at the stimulus center (which is also the grid square containing the electrode tip). The results of these analyses are shown in detail for the specific case of the fit to the spiking (MUA) signal in Fig 3 and in summary for all the electrophysiological signals in Fig 4 and Supplementary Fig S4.

For the visually evoked ‘simulated Mexican Hat’ the goodness of fit between the spiking-derived predictor and recorded signals varied sharply over the imaged region (Fig 3a-c) with a prominent gradient and a zero crossing, as in the correlation and amplitude maps for the same data set (Fig 2). Unsurprisingly both measures r and R² were highest for the grid square at the stimulus center, i.e. the location used to derive the optimal spike-derived HRF. At this location the time course of the predictor closely matched the raw hemodynamic signal, trial by trial (r = 0.79. See traces, Fig 3a, top, ‘Electrode site’) and the predictor accounted for a large fraction of the measured signal variance (R² = 0.60; this is comparable to our published values for the match to a full-field visually evoked response). Both measures r and R² get progressively worse farther away from the electrode tip, crossing over into negative values (at an eccentricity of 5 mm for r, of 2 mm for R². Fig 3b,c. 1 degree in visual space corresponds to about 2-4 mm on cortex over the imaged range of visual eccentricity). The negative r reflects the phase of the measured signal at distal locations, roughly opposite that of the predictor and of the signal at the center (e.g. Fig 3a bottom: ‘Trough’); this phase mismatch also means that the error at these distal locations is higher than the measured signal, leading to a negative R². (Note, again: this mismatch between predicted and measured hemodynamics distally is presumably because the net hemodynamic signal at progressively farther grid squares has a decreasing contribution from the visually evoked ‘point spread’ and is thus increasingly dominated by stimulus-independent components that happen to have the opposite temporal phase; Supplementary Fig S3.)

By contrast with the ’simulated Mexican Hat’ signals, the trial-related signals recorded during the dark-room trials were uniformly poorly predicted from concurrent spiking (Fig 3d-f). Note that the optimal HRF used to make these predictions was also obtained by fitting the spiking data – recorded in the dark-room trials – to the hemodynamic signal from the grid square at the electrode tip. Even so, the predictor obtained from spiking poorly matched the trial-related hemodynamic signal at all grid locations including the electrode tip (Fig 3e,f). The spatial profile for both r and R², as a function of radial distance from the electrode tip, is essentially flat and = 0 (Fig 3e,f insets). This poor match is not due to the trial-related signals being of low amplitude relative to the visually evoked signals: this is evident from visual inspection (compare hemodynamic traces in Fig 3a,d; note that the traces shown are the continuous, raw signals from the given grid squares, with no averaging) or quantitatively (the mean strength (standard deviation) of the trial-related signal = 0.7% (0.0007%), N=256 grid squares. This is comparable to the mean strength of the visually stimulated hemodynamic signal, 0.8% (0.0007%), and is about half the magnitude of the visually evoked signal at the stimulus center: 1.37%). It is particularly noteworthy, on the other hand, that the trial-related signal time courses are well matched to each other at all locations . E.g. the signal at the location of the red dot is very similar, in time course, to the signal at the electrode tip (green dot; signals: Fig 3d), reflecting the uniformly high correlation shown earlier (Fig 1b). This suggests, strongly, that the trial-related signal is being driven uniformly, in synchrony, over V1 but by some process independent of the measurable spiking activity.

We similarly analyzed the predictability of the recorded hemodynamics from electrode recordings for a full set of LFPs ranging down to 2 Hz. The summary results of goodness of fit are shown in Fig 4 (maps of R²) and Supplementary Fig S4 (maps of Pearson’s r). All LFP bands carried some information about the spatial structure of the ‘simulated Mexican Hat’ though as expected from our published results (Fig S13,(Sirotin and Das, 2009) the lower frequency LFP bands gave poorer predictions than high gamma (66-130 Hz) or spiking. By contrast, all recorded LFP signals and spiking were uniformly poor at predicting the trial-related signal whether at the electrode tip or away (Fig 4, supplementary Fig S4).

4: The trial-related signal in V1 is critically dependent on the animal being engaged in a visual task

The data presented so far, both in our published work (Sirotin and Das, 2009) and the earlier figures in this paper make the case that the trial-related hemodynamic signal in V1 is robust, spatially homogenous, and not predictable by electrode recordings even in the immediate vicinity of the electrode tip. There was some concern that the trial-related signal may be a form of ongoing vascular oscillation such as ‘vasomotion’ (Mayhew et al., 1996;Mayhew et al., 1999) with no obvious task-linked or behavioral significance. Two lines of evidence argue, however, that the trial-related signal is linked intimately to task structure and performance. First, the signal period entrains with remarkable accuracy and reliability to the task timing for all task periods that the animal was willing to undertake. Further, the signal is strong only when the animal is performing the task correctly; it can change shape and decline rapidly when the animal stops executing correct trials.

Finally, it is important to recall that the V1 trial-related signal described here is modality-specific, being present in V1 only for visual tasks; it is not a global arousal signal evoked by periodic tasks per se. We showed this in our earlier published work by testing the animal on a period auditory pitch-discrimination task, which failed to evoke the trial-related signal in V1 (Supplementary fig. S12, (Sirotin and Das, 2009).

The V1 signal entrains accurately and precisely to the visual task

Earlier we reported that when the animal was engaged in alternating blocks of short and long trials, the trial-related signal exhibited correspondingly short and long periods; further, the signal switched period within a single trial when the task period changed (Fig 3 in (Sirotin and Das, 2009). Here (Fig 5) we demonstrate the accuracy and precision with which the trial-related signal entrains to the task when the animal is performing a sequence of correct trials, i.e. holding fixation correctly when cued to do so trial by trial. Fig 5a shows a continuous sequence of the hemodynamic signal recorded when the animal was performing periodic fixation trials with a period of 6.8 sec. This is the raw hemodynamic signal, averaged spatially over the imaged area for each frame and filtered to remove a heart-beat artifact (see methods) but with no further processing or temporal averaging. Note the almost clockwork-like precision with which the raw signal follows the trial timing. The same clockwork-like precision is evident in the mean signal, where the average is taken only over sequences where the animal performed at least 3 correct trials in a row (Fig 5b). Figs 5c-d and 5e-f shows similar pairs of raw and mean signals for two other trial periods, 10.1 sec and 15.4 sec respectively. In each case the recorded signal is noteworthy for the regularity of entrainment to the trial period.

To quantify each signal periodicity further we calculated the power spectrum of the raw hemodynamic signal (Fig 5g-i). The spectrum was obtained as a function of time in the trial by using a 200-second window for the spectral analysis, moving in 20-sec steps over the sequence of correct trials (see methods). For the 6.8-sec-period task thus analyzed in Fig 5g, the power spectrum exhibits a sharply defined spectral band located around the frequency corresponding to 6.8 sec, i.e. 0.147 Hz (1/6.8). This spectral band is essentially constant over the entire course of the experiment implying that the signal held the same precise period over the full duration (Fig 5g, left panel; 425 correct trials, i.e 2890 sec = ~48 min). The mean power spectrum averaged over the experiment shows a prominent peak at the same frequency, extending more than an order of magnitude over the local baseline (Fig 5g, right panel). Similar analyses for the other two experiments illustrated in Fig 5 give mean spectral profiles with similarly prominent peaks at the frequency corresponding to the relevant trial period (Fig 5h. the spectra for the 10.1-sec and 15.4-sec experiments also contain secondary peaks at higher harmonics of the base trial frequency).

To quantitatively compare each spectral peak with the corresponding trial period we defined the location of the spectral peak in each case as its midpoint, i.e. the mean of the two frequencies corresponding to the vertical sides defining the peak along the frequency axis (see Fig 5g, right panel). This frequency was then converted into the corresponding time period (1/frequency). Fig 5i compares the time period thus obtained with the period of the task that the animal was engaged in, i.e. the period of the fixation cues presented to the animal. Results are shown here for a representative set of 10 experiments covering the range of task periods used by us through multiple experiments stretching over 4 years (task periods: 6.8 sec to about 20 sec). The periods derived from spectral peaks clearly correspond closely to their respective trial periods (regression y = 0.96x + 0.45 where y: period from spectrum in sec, x: trial period in sec.

The signal declines or changes shape rapidly when the animal stops performing correct trials

All of our published data so far, as well as the other figures in this paper, have focused on the signal measured during correct trials. Fig 6a shows an example of a trial sequence with two trials where the animal made an error, failing to fixate at the start of the trial (designated ‘No Fix’ errors, to distinguish them from errors where the animal starts fixating but breaks prematurely). The hemodynamic signal for the ‘No Fix’ error trials is largely flat and very different from that during correct trials. This shape change is rapid, within the trial: the correct trial immediately preceding the first ‘No Fix’ error had a shape and amplitude like that of other correct trials in the set. The signal also recovers rapidly, in just one trial after the animal resumes fixating correctly. It is important to emphasize that these trials are in close to total darkness, like the trials reported in our earlier publication (Sirotin and Das, 2009). Thus the difference in hemodynamic response between error and correct trials is not due to differences in visual stimulation since there is essentially no light falling on the retina in any of the trials.

Figure 6.

The trial-related signal collapses or changes shape as soon as the animal stops performing correct trials. a: Eye position (top) and hemodynamic signal (bottom) recorded from a continuous sequence of trials. For most trials, the animal fixates correctly when cued to do so (the eye position trace shows the location of the eye relative to the fixation point. In degrees, with zero at the bottom of the scale. On correct trials the animal correctly held his eye very close to fixation for the full duration of the cue). On two trials marked ‘Error (No Fix)’ the animal fails to fixate properly by the start of the trial. Compare the hemodynamic signals for the error vs. correct trials. b: Hemodynamic signal for all pairs of consecutive correct trials. Vertical dashed line: onset of second trial in pair. Gray: individual traces; green: mean (N=242). c: Same, for transition pairs where the animal made a ‘No Fix’ error after a correct trial (N=29). Dot products with mean of all correct trials: 0.09 (error), 0.87 (preceding correct). d: Population scatter plot of dot products of No Fix error vs. preceding correct, with the means of all corresponding correct trials, N=42 experiments.

To study the change in signal with ‘No Fix’ errors systematically we identified all the pairs of trials in this experiment where the animal made such an error immediately following a correct trial (N=29 such transition trials, out of a total of N=242 correct trials. The animal often made sequences of error trials; in such cases, only the first of the sequence met this criterion). Over this population of trial pairs we see that the mean hemodynamic signal declines and flattens sharply in the ‘No Fix’ error trial even though the signal looks quite normal for the correct trial immediately preceding (Fig 6c). By contrast, consecutive pairs of correct trials evoke the typical periodic repetitive signal (Fig 6b) just like the signals shown in Fig 5.

We wanted a quantitative measure to test how closely the mean signal for these error and correct trials matched the overall mean signal for all correct trials in the experiment. For this we used the normalized dot product, defined in a manner exactly as before:

$$A=\frac{S\cdot \stackrel{‒}{S}}{\mid \stackrel{‒}{S}{\mid }^2}$$ (5)

where $\stackrel{‒}{S}$ is the mean over all correct trials while S is the signal of interest, i.e the mean of either the ‘No Fix’ error trials or of the preceding correct trials. For this experiment the normalized dot product for the mean ‘No Fix’ trials was very small (0.09), consistent with the observation that the mean signal largely flattens out when the animal fails to fixate. By contrast, the preceding correct trial has a large dot product (0.87) reflecting its essential similarity to all the other correct trials. This rapid collapse of the mean trial-related signal for the error trial after a signal that is essentially normal in the previous correct trial implies that the signal is intimately related to the animal’s performance of the task and argues convincingly that it is not some ongoing behaviorally irrelevant vascular oscillation.

We performed a similar analysis on the population of 128 dark-room fixation experiments carried out in 3 animals over the last 4 years. Not all the experiments were usable for this analysis, either because there were not enough trials overall to get reasonable statistics, or because the animal made very few or no ‘No Fix’ errors. We therefore set a criterion, of selecting only those experiments with at least 70 trials in total and at least 5 transition trials where the animal made a No Fix error following a correct trial. In addition we set a lower limit on the magnitude of the mean signal for all correct trials to avoid dividing by excessively small numbers in equation 5 (we set the criterion that the signal magnitude (standard deviation) >= 0.002). Of the total of 128 experiments, 42 met this set of criteria. For each of these 42 we calculated the dot products of the mean ‘No Fix’ and preceding correct trials with the mean of all correct trials for the experiment. The results are shown in Fig 6d. There is some scatter in the dot products for the ‘No Fix’ error trials, with values ranging from close to −2 (an extreme change in shape, including a reversal in sign) up to 0.76; but the values are clustered around 0 (mean = −0.009 (0.07), N=42). By contrast, the dot products for the preceding correct trials are tightly clustered around 1 (mean = 1.04 (0.04), N=42). Over this full population of experiment we thus see that the hemodynamic signal changes shape or declines dramatically in ‘No Fix’ error trials despite maintaining a reliably trial-locked pattern right up to the correct trial preceding the error..

Discussion

In this paper we describe the spatial structure of the trial-related hemodynamic signal that we demonstrated, recently, in V1 of alert macaque monkeys performing periodic visual tasks. As a corollary, we show that this trial-related signal is critically dependent on the animal being engaged in – and correctly performing – a behavioral task.

For analyzing the spatial structure of the signal we started with two broad alternative hypotheses, both with antecedents in earlier literature. One hypothesis was that the trial-related signal could be structured like a ‘Mexcan Hat’ analogous to the fMRI maps of attentional signals (Smith et al., 2000;Heinemann et al., 2009;Muller and Kleinschmidt, 2004;Muller et al., 2005;Hopf et al., 2006b;Slotnick et al., 2002): peaked at the locus of attention (in our case, the fovea) and dropping sharply down to a ring of negative values at eccentricities beyond about 2 – 3 degrees. The alternative hypothesis was that the signal was spatially largely uniform over V1, like some other task-structure-related signals reported recently in human fMRI studies (Jack et al., 2006;Donner et al., 2008).

We found that the trial-related signal is remarkably synchronous and largely homogeneous over the entire imaged area of V1 (Fig 1). We used two measures to test for signal homogeneity: the similarity of the signal time course, and of signal amplitude, over a 16 × 16 grid covering the 10-mm imaged region. The time courses of the trial-related signal measured at different locations within each image were highly correlated with each other (Fig 1b: mean r=0.94 (0.01), N=10, 3 animals) with no apparent spatial structure to the correlation map. The maps of signal amplitude were more heterogenous: signals recorded near arteries were consistently higher in amplitude than signals from other V1 locations though they shared a very similar time course (Fig 1c-d). Residual gradients that remained after masking the vascular artifacts were weak and ruled out any possibility of a zero crossing in the imaged region.

To control for the possibility that the lack of any visible spatial structure in the signal was due to inadequacies in our analysis methods, we performed the same analyses on a ’simulated Mexican Hat’: the V1 response to a small, localized visual stimulus close to the fovea, presented while the animal was engaged in a periodic fixation task (Fig 2). In the alert, fixating animal this stimulus leads to a center-surround spatial pattern on V1, focused on the retinotopic location of the stimulus. It likely arises because the monophasic, sharply peaked ‘point spread’ pattern expected as the specific response to the visual stimulus (Das and Gilbert, 1995) would be accompanied, in the alert behaving animal, by stimulus-independent components. These include the periodic trial-related hemodynamic signal as well as periodic visual inputs other than the controlled stimulus (Note that trials with visual stimuli are no longer in a dark room; thus there are other objects visible in the room, for the animal to look at periodically when not fixating on the monitor). The negative surround may also relate to the ‘Negative BOLD’ signal evoked distally by a local stimulus (Shmuel et al., 2006;Devor et al., 2007;Boorman et al., 2010). Notwithstanding this complex origin, however, the net V1 hemodynamic response is sharply peaked at the location of the stimulus, and drops to negative values far from the stimulus location with a sharp gradient and a zero crossing in the imaged region. Thus it is a reasonable simulation of the ‘Mexican Hat’ expected for an attentional signal centered at the fovea.

Our analysis of this ‘simulated Mexican Hat’, showed clear spatial structure, unlike the flat profile of the trial-related signal. Both the correlation map of signal time course (Fig 2b), and the map of signal amplitude (Fig 2c-d) peaked sharply at the stimulus center, dropping to negative values beyond an eccentricity of 2- 3 degrees. Thus our analysis methods are clearly capable of identifying a gradient and zero crossing in signals that do possess it. Moreover, the lack of such a structure in the trial-related signal is not due to poor signal-to-noise in the latter. In the one direct comparison of trial-related to visually evoked signal strengths (Fig 3) the mean strength of the trial-related signal (0.7% (0.0007%), N=256 grid locations) was comparable to that of the signal in the presence of a visual stimulus (0.8% (0.0007%)) and approximately half the amplitude of the peak visually driven signal (1.37%). These amplitude ratios are consistent with our published results showing that over our population of recording sites, the amplitude of trial-related signals was, on average, 37% the amplitude of the maximal signals at the same locations during visually driven trials (N=28, 2 animals; (Sirotin and Das, 2009). Thus the homogenous spatial structure and globally correlated time course of the trial-related signal are as reliable as the gradient and zero crossing evident with visually driven responses.

With concurrent electrode recordings we also mapped the goodness of prediction of the trial-related hemodynamic signal from electrophysiological recordings, at the electrode tip and at varying distances from the electrode tip (Fig 3). For comparison, we made the same measurement for the ‘simulated Mexican Hat’. The ‘simulated Mexican Hat’ signal gave a prominently structured map of goodness of prediction from a spiking-based predictor, with high positive values at the electrode tip – centered on the stimulus – dropping monotonically to high negative values farther out (Fig 3a-c). By contrast, even though the trial-related signal was comparable in amplitude to visually evoked responses, the goodness of fit for the trial-related signal was uniformly very poor over the entire imaged area including the electrode tip (Fig 3d-f). We found the same, uniformly poor goodness of fit using all measured electrode signals (spiking, all LFP bands from high gamma (66-130 Hz) down to 2 Hz; Figs 4, S4). Moreover, the trial-related signal time course at the electrode tip was strikingly similar to that measured farther away, reflecting the uniformly high correlation measured over the imaged area (Fig 1b). Thus, unlike visually driven processes, the trial-related signal is likely driven in unison, over the entire V1, but by a process that is not predicted by locally measured electrode signals.

These results should allay concerns that our earlier findings – showing that the trial-related hemodynamic signal cannot be predicted from local electrode signals – were due to methodological problems with our measurements. One commentary, in particular, suggested that our electrode and hemodynamic measurements were sampling very different populations of neurons. In particular, it suggested that: 1: The trial-related signal is shaped like a Mexican Hat with a zero crossing in our imaged region. Thus the spatial average gives a number that is unrepresentative of the signal. 2: Our electrode recordings were from the zero crossing of the presumed Mexican Hat. Thus the recorded spiking inappropriately samples only a region of low signal that is also unrepresentative of the full hemodynamic signal (Kleinschmidt and Muller, 2010). Without belaboring our results any further, we believe that we have addressed both of these concerns.

This compelling evidence of spatially homogenous trial-related hemodynamic signals unpredicted by local electrode recordings raises the concern that the signal may be a form of ongoing vascular oscillation unrelated to behavior, analogous to ‘vasomotion’ (Mayhew et al., 1996;Mayhew et al., 1999). In our earlier publication we have already shown that the signal reflects task structure, switching from short to long periods in synchrony with task timing (Fig 3, (Sirotin and Das, 2009). Here we quantify the accuracy and precision with which the period of the trial-related signal matches the period of the task in which the animal is engaged. Since these are experiments conducted with alert behaving macaque monkeys, they require the cooperation of the experimental subject. Thus we were able to test a limited range of trial timings, from 6.8 sec (about the smallest trial period that can include a 4-sec fixation period) to about 20 sec (the animals’ performance became unreliable for longer trials, with a large number of incorrect fixations). Over this full timing range, however, the trial-related signal had almost clockwork-like trial-by-trial precision. Further, the signal period calculated from a power spectrum accurately matched the trial period, i.e. the timing with which fixation cues were presented to the animal (Fig 5). This rules out the possibility that the trial-related signal is an ongoing oscillation with some intrinsic period (e.g. a ‘vasomotion-like’ 10-sec) that accidentally entrains to trials of the same timing. Moreover, the signal depended critically on the animal actually being engaged in the task. The clockwork-like trial-related signal was observed only as long as the animal performed correct fixation trials. It changed shape or collapsed dramatically as soon as the animal stopped performing correct trials and failed to fixate (Fig 6). This provides further compelling evidence that the trial-related signal is critically linked to the animals’ behavior and correct vs. incorrect performance of a task.

In this context, it is important to recall – and emphasize – that the signal is not a global autonomic arousal due to periodic tasks, per se (Supplementary fig. S12, (Sirotin and Das, 2009). On initially encountering the trial-related signal in V1 we had noted that it is accompanied by trial-linked pupillary dilations and periodic fluctuations in the heart rate (e.g. Fig 1, Sirotin and Das, 2009). We were thus concerned that the signal reflected a global periodic arousal. To test for this possibility, we trained the animal on a periodic auditory pitch discrimination task. In a completely dark room – lacking even the fixation cue – we had the animal pull a lever, wait for an auditory tone to come on, and then wait further for the tone to change pitch. This was the cue for him to release the lever as quickly as possible, and pull the lever back again to start the next trial. Task timings were arranged to be comparable to the visual fixation task performed on the same day. This periodic auditory task evoked periodic, trial-linked fluctuations in the heart rate, and pupillary dilations, of even larger amplitude than the visual task. However, it did not evoke a trial-related fluctuation in the V1 blood volume signal (Supplementary fig. S12, (Sirotin and Das, 2009). Thus the V1 trial-related signal reflects a mechanism that is triggered specifically by visual tasks. In addition, it means that even though the V1 signal is accompanied by trial-linked fluctuations in the heart rate during visual tasks, it cannot be regressed away reliably using just the heart rate signal (Iacovella and Hasson, 2011) since there are even larger heart rate fluctuations accompanying the auditory control task.

These results should help establish, firmly, that the ‘trial-related signal’ is a behaviorally relevant hemodynamic signal in the brain, intimately tied to the correct performance of behavioral tasks, and yet distinct from visuospatial attention. The distinction from attention is evident in the spatial homogeneity of this signal. We found no reliable gradients in the amplitude map of the trial-related signal within our imaged region of V1, or any systematic evidence of peaks at the fovea. The measured gradients were weak (mean: 0.02 (0.004) / mm, i.e. 20% over 10 mm, N=10), likely due to uncorrected vascular artifacts and pointed in directions that were random relative to the direction of the fovea. It is true that our imaging window is limited in size (~ 1.3 cm in diameter) compared to the full extent of V1 (~ 2 cm × 5 cm, each hemisphere). Thus we cannot rule out the possibility of a sharp change in signal distribution or gradient outside the imaged area. However, any reasonable extrapolation of the gradient outside the imaged area suggests that the trial-related signal is a largely uniform signal present synchronously over the entire area of V1. Thus it is likely similar to ‘global’ or ‘task structure-related’ signals reported earlier (Jack et al., 2006;Donner et al., 2008;Sylvester et al., 2007), and distinct from retinotopically focused attention (Smith et al., 2000;Heinemann et al., 2009;Muller and Kleinschmidt, 2004;Muller et al., 2005;Hopf et al., 2006a;Slotnick et al., 2002). There is now growing evidence that the neuroimaging data recorded from the brains of alert, task-engaged subjects reflect a number of hemodynamic signals beyond the stimulus-evoked response. Our results should help further the study and categorization of such signals.

Both our earlier work and the current findings show that this trial-related signal has a relationship with local neural activity that is very different from that demonstrated by visually evoked hemodynamics. It is important to emphasize, however, what we are not stating – we are not claiming that the trial-related signal is ‘not neural’; just that it must be governed by underlying processes (neuromodulatory input from brain stem centers? feedback from higher cortical areas? direct neuromodulatory control of blood vessels?) that are very different from the feedforward input underlying the visually driven signal. Because it is so substantial, however – about 40% of the maximal visually driven signal in amplitude – the trial-related signal must be separated appropriately from stimulus-evoked responses when interpreting imaging signals as a measure of local neural activity. We hope these results will help lead to further targeted research aimed at understanding the neural and behavioral correlates of the complex neuroimaging signal.

Supplementary Material

01

Supplementary Figure S1: a: Identifying arteries vs. veins. V1 region imaged at the ‘oximetric’ 605 nm (Left, red) vs. ‘volumetric’ 530 nm (Right, green). Veins (‘V’) are darker than arteries (‘A’) in the oximetric image since this wavelength is preferentially absorbed by deoxyhemoglobin; unlike in the volumetric image where veins and arteries are equally dark. (Same region as the one illustrated in Fig 1. Both images are high pass filtered (2.5 mm, Gaussian σ) to flatten the overall higher illumination intensity near the center). b-e: Image analyzed to show mean traces, correlation, amplitude map and 3-d plot for ‘oximetric’ 605 nm image. Same experiment as in Fig 1 (dual-wavelength, with alternate frames imaged at 530 and 605 nm). Conventions as in Fig 1. Note the distinct time courses of the signal in veins vs. arteries in the images at selected time points, panel b bottom row.

02

Supplementary Fig S2: Control: dividing by pre-trial-onset baseline rather than by trial mean. The movie over the trial was divided, pixel by pixel, by the mean of the 4 image frames immediately pre trial onset (time 0). a: Timecourse of the mean signal thus expressed as fractional change after trial onset, compared with the timecourse of the signal expressed as fractional chage around trial mean. The two timecourses were essentially identical. b: Individual image frames at 3 time points for the ‘divided by mean’ signal, identical to the images in Fig 1a, except that the first frame is from time 0.0 sec rather than 1.33 sec. c: Corresponding image frames from the ‘divided by pre-trial-onset baseline’ movie. Essentially identical to the images in part b, except that the frame at time 0 is almost uniformly gray (by definition: since that is right after the defined baseline), unlike in part b where there is a faint vascular pattern. d: Timecourse of the signals from the 16 × 16 individual squares (gray) and the mean (green) for the signal normalized by pre-trial-onset baseline. The signals are essentially identical to those in Fig 1a, upper panel, except that here the individual gray lines all pass almost exactly through amplitude 0 at time 0 (by definition, since that is right after the baseline). e, f: Correlation and amplitude maps corresponding to similar maps in Fig 1. They are essentially identical to the ones in Fig 1.

03

Supplementary Fig S3: Relation between the ‘simulated Mexican Hat’ and its trial-related and visually-evoked components. ‘Stimulated trials (without subtracting blank trials)’ a-c: Net response to a small visual stimulus presented while the animal performed its periodic fixation task. Stimulus was an oriented bar subtending 0.25 deg of visual angle. NOTE: different experiment from the one in Figs 2-4. a: Top: traces from individual grid squares; green trace is the response from the stimulus center, marked by a green dot on the 2-D images. Note the distinct time courses of the various signals, essentially switching in polarity across cortex. The time line for one trial (12.7 sec duration) is indicated below the traces (‘Fixate’: green; ‘Stim’: red; ‘Relax’: gray; time t=0 aligned to stim onset). Averaged over N = 82 trials. Bottom: images at selected time points showing the polarity switches imaged on the cortical surface. b, c: maps of correlation and amplitude, as in Fig 2b,c.. ‘Blank trials’ d-f: Response to identical fixation trials but with no visual stimulus (trials interleaved with the Stimulated trials shown in panels a-c). N = 82 trials. Same conventions as in Fig 1. ‘Isolated point spread: Stimulated trials after subtracting (dividing by) blanks’ g-i: Stimulated-trial signal, divided pixel-by-pixel by the corresponding blank-trial signal. Essentially identical to subtracting the blank from the stimulated while also normalizing for inhomogeneities in illumination over the cortical surface. The divided signal – a measure of the visually evoked response alone, with minimal contamination by the trial-related signal – shows no significant polarity changes any more, just a graded decline in response amplitude from the peak (green trace) to a slightly noisy baseline (e.g. black trace, from location identified by black dot.). This monophasic response is reflected in the correlation map (panel h) and amplitude map (panel i); compare with panels b, c (note the different color scale for amplitude in panel c as vs. panels f and i).

04

Supplementary Figure S4: Summary data: fits of Pearon’s r for predictors derived from spiking and all LFP frequency bands. Same as Fig 4, but for Pearson’s r as the measure of goodness of fit.

Research Highlights.

• Images from task-engaged subjects contain distinct ‘task-related’ signals.

• Here we analyze a ‘trial-related’ signal seen in V1 of monkeys in a periodic task.

• We show that the signal has uniform time course and amplitude over imaged V1.

• The signal is also uniformly poorly predicted by electrode recordings.

• As a corollary we show that the signal cannot be explained as an ongoing vascular oscillation but is intimately tied to correct task performance.

• The signal entrains with great accuracy and precision to trials when the animal is performing a sequence of correct trials.

• The signal switches shape and amplitude rapidly, within a single trial, when the animal takes a break from correct fixation.

• It is likely distinct from attention but like earlier-reported ‘task-related’ signals.

• This suggests there are distinct categories of non-stimulus-driven imaging signals.