Spatial and temporal scales of neuronal correlation in visual area V4

Abstract

The spiking activity of nearby cortical neurons is correlated on both short and long time scales. Understanding this shared variability in firing patterns is critical for appreciating the representation of sensory stimuli in ensembles of neurons, the coincident influences of neurons on common targets, and the functional implications of microcircuitry. Our knowledge about neuronal correlations, however, derives largely from experiments that employed different recording methods, analysis techniques, and cortical regions. Here we studied the structure of neuronal correlation in area V4 of alert macaques using recording and analysis procedures designed to match those used previously in primary visual cortex (V1), the major input to V4. We found that the spatial and temporal properties of correlations in V4 were remarkably similar to those of V1, with two notable differences: correlated variability in V4 was roughly one third the magnitude of that in V1, and synchrony in V4 was less temporally precise than in V1. In both areas, spontaneous activity (measured during fixation while viewing a blank screen) was roughly twice as correlated as visual-evoked activity. The results provide a foundation for understanding how the structure of neuronal correlation differs among brain regions and stages in cortical processing, and suggest that it is likely governed by features of neuronal circuits that are shared across visual cortex.

INTRODUCTION

The spiking activity of pairs of nearby cortical neurons is correlated. The correlations occur over a range of time scales, from slow fluctuations in responsivity over periods of hundreds of milliseconds (correlated variability) to coordinated firing over a range of a few milliseconds (synchrony). Both forms of correlation have been implicated as important for population coding. Correlated variability is directly related to the amount of information present in a population of neurons (Zohary et al., 1994; Shadlen and Newsome, 1998; Abbott and Dayan, 1999; Averbeck et al., 2006), and the particular structure of the correlation – its dependence on tuning similarity, neuronal separation, and other factors – determines the magnitude of the information content (Averbeck et al., 2006). Synchrony has been associated with a number of visual and cognitive phenomenon, including the binding of visual objects, attention, and arousal (Gray, 1999; Shadlen and Movshon, 1999). Given the broad range of effects for which correlations may play a role, it is apparent that a more complete understanding of the structure of correlated activity is needed.

Correlated variability and synchrony have been measured by a number of studies of primary visual cortex, or V1 (see Cohen and Kohn, 2011 for review). This work was performed in a number of laboratories with different stimuli, recording methods and analysis procedures, leading to a fragmented picture of the structure of correlation. In work performed by an author of the present report, the structure of neuronal correlation in V1 was found to depend on the distance between neurons, their tuning similarity, and the temporal scale (Smith and Kohn, 2008). Outside V1, a small number of studies have measured correlated variability in areas such as V4 (Cohen and Maunsell, 2009; Mitchell et al., 2009) and MT (Zohary et al., 1994; Bair et al., 2001; Cohen and Newsome, 2009; Huang and Lisberger, 2009). These studies tended to focus on one time scale, and they used diverse recording techniques and visual stimuli.

Our goal was to evaluate how neuronal correlations compare between separate, but closely linked, visual cortical regions. We used 100-electrode arrays (Fig 1A) to record from populations of neurons in macaque area V4, which is proposed to play a role in a number of mid-level visual processes, including spatial and feature-based attention, figure-ground segregation, color and brightness perception, and shape representation (see Roe et al., 2012 for review). V4 receives its primary input from V1, and therefore we studied whether the prominent features of neuronal correlation that we observed in V1 (Smith and Kohn, 2008), namely the dependence of correlation on distance, tuning similarity, and time scale, were also present in V4. We used the same stimuli and analysis methods as we had in V1, in order to make as close a comparison as possible. We found that correlations among V4 neurons were quite similar to those found in V1, except for decreased magnitudes and a broader relationship between time scales and distance. We conclude that the structure of neuronal correlation is governed primarily by features of neuronal circuits that are shared across visual cortex.

Figure 1.

Experimental methods. A, A photograph of the array, consisting of a 10 × 10 grid of silicone microelectrodes with 400 μm spacing and 1.0 mm length. B, A lateral view of the macaque brain with major sulci indicated with thick black lines and minor vessels and sulci with thin black lines. The rough position of the inserted arrays, anterior to the lunate sulcus (LS), posterior to the superior temporal sulcus (STS) and medial/dorsal to the inferior occipital sulcus (IOS), is indicated with the red arrow. C, For a single pair of V4 neurons, the plot on the left shows the Z-scored spike counts of one neuron on all of the trials (of all 12 stimuli) plotted as a function of the same metric in a second neuron. The Pearson’s correlation of these data points was 0.14 (p<0.0001). For the same pair of neurons, a the jitter-corrected CCG is shown (with a jitter window of 50 ms). The vertical gray tick represents zero time lag and the horizontal gray tick represents 0 coinc./sp. Scale bars are shown below the CCG.

METHODS

Surgical preparation

Two adult male rhesus macaque monkeys (Macaca mulatta) were surgically prepared in aseptic conditions under isofluorothane anesthesia. In one animal (Monkey D), we implanted a scleral search coil for eye position monitoring (Judge et al., 1980) and a post for immobilizing the head during experiments (see Sommer and Wurtz, 2000), with an acrylic implant and titanium screws affixing the eye coil connector and headpost to the skull. In the other animal (Monkey B), we implanted only a titanium headpost held to the skull with titanium screws. All procedures were approved by the Institutional Animal Care and Use Committee of the University of Pittsburgh and were in compliance with the guidelines set forth in the United States Public Health Service Guide for the Care and Use of Laboratory Animals.

In addition, we implanted a 100-electrode “Utah” array (Blackrock Microsystems) in visual area V4 of each animal. The array consists of a 10x10 grid of silicon microelectrodes (each 1 mm in length) spaced 400 um apart (Fig 1A). The implantation was performed in the same surgery as the headpost and eye coil for Monkey D, but in a separate surgery for Monkey B. Area V4 was located with guidance from stereotaxic coordinates (Horsley-Clarke: the intersection of 5 mm posterior and 12-15 mm dorsal with the surface of the skull) and then identified directly by performing a craniotomy and durotomy and visualizing the lunate and superior temporal sulci. The arrays were implanted using a pneumatic insertion device (Rousche and Normann, 1992) on the prelunate gyrus just dorsal to the inferior occipital sulcus (Fig 1B). This resulted in neuronal receptive fields centered roughly 1.5° from the fovea for Monkey B and 3° for Monkey D.

Data collection

Signals from each microelectrode in the array were amplified and bandpass filtered (250 Hz to 7.5 kHz) by a Cerebus system (Blackrock Microsystems). For each electrode in the array, waveform segments that exceeded a threshold (periodically adjusted using a multiple of the rms noise on each channel) were digitized (30 kHz) and stored for offline analysis and sorting. Waveform segments were sorted with an automated algorithm that clustered similarly shaped waveforms using a competitive mixture decomposition method (Shoham et al., 2003). We manually refined the output of this algorithm with custom time-amplitude window discrimination software (written in MATLAB; MathWorks) for each electrode, taking into account the shape of the waveform and the distribution of interspike intervals.

Following the offline sorting procedure, we computed the signal to noise ratio (SNR) of each candidate unit as the ratio of the average waveform amplitude to the SD of the waveform noise (Nordhausen et al., 1996; Suner et al., 2005; Kelly et al., 2007). Candidates that fell below a SNR of 2.75 were discarded as probable multiunit recordings. We also eliminated neurons for which the best visual stimulus (gratings as described below) did not evoke a response of at least 2 spikes/s (33 units removed). The remaining candidate waveforms (113 units) were deemed to be single neurons of sufficient quality and visual responsiveness to warrant further analysis.

In Monkey D, we monitored eye position with a scleral search coil system (Riverbend Instruments; Birmingham, AL). In Monkey B, we employed an infrared tracking system (ISCAN, Inc; Woburn, MA). In each case, the animals were required to maintain fixation within 0.5° of a small spot. Eye movements were analyzed online by the behavioral control software to ensure fixation within the required window, but these data were not saved due to a firmware error.

Visual stimulation and behavioral task

Visual stimuli were indistinguishable for the two animals, except for their sizes and positions that were matched to the receptive fields studied in each monkey. For Monkey B, we generated the stimuli using the REX real-time system (Hays et al., 1982), while for Monkey D we used custom software written in Matlab using the Psychophysics Toolbox extensions (Brainard, 1997; Kleiner et al., 2007). All stimuli were displayed on a CRT monitor with a resolution of 1024 by 768 pixels and a temporal refresh of 100 Hz, viewed at a distance of 57 cm. We used look-up tables to correct for nonlinearities in the relation between input voltage and phosphor luminance in the monitor. The mean luminance of the display was approximately 40 cd/m². All stimuli were presented in a circular aperture surrounded by a gray field of average luminance. We mapped the spatial receptive fields of each channel by presenting small, drifting gratings at a range of spatial positions. We centered our stimuli on the aggregate receptive field of the recorded units.

An animal was required to fixate a 0.15° blue dot for 100 ms to begin the trial. A drifting sinusoidal grating stimulus was then presented for 1000 ms. After the last video frame containing the stimulus and fixation spot, the animal was required to make a saccade to a small dot randomly positioned along a circle 8° from fixation. If the animal left the central fixation window within 300 ms and made a saccade to the target window, it received a liquid reward. We presented full-contrast drifting sinusoidal gratings at 12 orientations spaced equally (30°). The spatial frequency (1.3 cpd) and temporal frequency (6.25 Hz) values were chosen to match the values we used in a previous experiment in V1 neurons (Smith and Kohn, 2008). The position and size (2.5° in Monkey B and 5° in Monkey D) of the grating were sufficient to cover the receptive fields of all the neurons. Stimulus orientation was block randomized, and blocks were repeated 61 times for Monkey B and 80 times for Monkey D.

We also collected a block of data in which the animal was required to perform the same task without the grating present. Thus, the data collected represent a 1000 ms period of spontaneous activity. In Monkey B, 301 spontaneous trials were collected after the grating data. In Monkey D, 200 spontaneous trials were collected prior to the grating data.

The data presented here represent all of the neurons recorded from a single behavioral session from each animal (the first day of recording for this paradigm). The same data collection procedures were employed over the course of several days in each animal, but we chose to be conservative and not present data from successive days or pool data across days. We considered this important because uncertainty in the identity of single units or multi-unit clusters recorded in successive days would almost certainly result in false overestimates of the number of uniquely recorded neuron pairs. We note, however, that analyses of the data in later days, and analyses of the data pooled across days, produced results that were equivalent to the results reported here, for all tests.

Correlation analyses

The methods used to characterize correlation were presented in detail in two previous publications (Kohn and Smith, 2005; Smith and Kohn, 2008). Here, we describe the measures in brief. The spike count correlation, or r_sc, quantifies the correlated variability as discussed in the Introduction. It is the Pearson correlation coefficient of the evoked spike counts of a pair of neurons in response to repeated presentations of the same stimulus. The r_sc captures the shared trial-to-trial variability of firing rates averaged over the full 1000 ms stimulus presentation. For all trials of each stimulus orientation, we normalized each neuron’s responses to a mean of zero and unit variance (Z-score), and calculated r_sc between neuronal pairs after combining responses to all stimulus orientations (Fig 1C, left).

As a measure of the relative spike timing or synchrony of two neurons at a millisecond time scale, we computed the spike train cross-correlogram (CCG; Perkel et al., 1967). We normalized the CCG by the geometric mean spike rate of the pair of neurons. This is the most commonly used normalization, the results of which are most easily compared to r_sc (see Kohn and Smith, 2005 for further discussion). A normalized CCG was computed for each of the 12 stimulus orientations and then smoothed with a 5 ms kernel ([0.05 0.25 0.40 0.25 0.05]). We then took the mean value at each time lag across the CCGs to produce an average CCG for each pair (Fig 1C, right).

We calculated CCGs with two types of correction. The standard shuffle-corrected CCG was created by subtracting the expected value of all possible CCGs produced using the same data set but with non-simultaneous trial pairings. This method removes correlations locked to the stimulus, but results in a CCG that reflects correlation on a range of time scales. We also used a jitter correction method (Amarasingham et al., 2012) to remove slow temporal correlations as well as stimulus locked correlations. This is created by subtracting the expected value of CCGs produced from a resampled version of the original data set with the spike times randomly perturbed or jittered within a small window of time (the jitter window). Details on this method have been published previously (Smith and Kohn, 2008). We used a jitter window of 50 ms for most of the analysis shown, with the exception of Figure 6 where the jitter window was varied as described.

We calculated the tuning similarity of each pair of neurons as the Pearson’s correlation between the mean responses of each cell to all 12 orientations (r_signal). For neurons with similar orientation tuning r_signal is near 1, for neurons with dissimilar tuning r_signal approaches −1.

As a measure of correlation on a time scale greater than the length of a single trial, we calculated the trial cross-covariance (TCC) following the procedure of Bair et al (2001). Simply put, this is the cross-correlation of the Z-scored spike counts for a pair of neurons across all trials of a given stimulus. For a trial lag of zero, the TCC is equal to r_sc. If the correlation between a pair of neurons arises only from time scales shorter than the duration of a trial, then the TCC will consist of an isolated peak at zero trial lag with all other values approaching zero.

Using the TCC, we estimated the long-term component of r_sc (long-term correlation, or r_lt). This was done by replacing the value of the TCC at zero with the average of each immediate neighbor. We then convolved the TCC with a Gaussian (SD=4 trials) and determined the new value at zero (r_lt).

While r_sc is typically computed over the full length of the trial, correlation can arise on various time scales. In order to assess the contribution of effects on these time scales, we also computed a metric known as r_ccg (introduced by Bair et al., 2001). This is the integral of the shuffle-corrected CCG divided by the geometric mean area of the autocorrelograms of the pair of neurons. When calculated over the full trial duration, r_ccg is equal to r_sc. In shorter windows of integration, r_ccg is useful for revealing the magnitude of correlated variability on that time scale.

All indications of variation in the graphs and text are standard errors of the mean (s.e.m.). The statistical significance of all results was evaluated with two-tailed t-tests, unless otherwise noted. All values of ‘r’ indicate Pearson’s correlation unless otherwise noted. Significance of Pearson’s correlation values was assessed after applying the Fisher Z-transform to the data.

RESULTS

We measured pairwise correlations among all possible pairs, excluding pairings on the same electrode, of 113 neurons recorded from arrays implanted in area V4 of two macaque monkeys (Fig 1). This resulted in 3240 total pairs (1818 from one animal and 1422 from the other).

Correlated variability

Pairs of neurons in visual cortex, including areas V1 and V4, exhibit shared trial-to-trial fluctuations in response strength (Kohn and Smith, 2005; Smith and Kohn, 2008; Cohen and Maunsell, 2009; Mitchell et al., 2009). Spike count correlation (r_sc) measures this correlated variability using the Pearson correlation of the evoked spike counts of two simultaneously recorded neurons over the entire 1000 ms stimulus period. Across all 3240 pairs of neurons recorded from both animals, the average r_sc was 0.058 ± 0.001 (0.063 in Monkey B and 0.053 in Monkey D), significantly greater than zero (p<0.0001) and similar to correlation levels found in two previous studies of V4 activity (Cohen and Maunsell, 2009; Mitchell et al., 2009).

We grouped pairs of neurons based on the distance of the electrodes from which they were recorded, and computed the mean and s.e.m. of r_sc within each distance bin (Fig 2A). For both animals, r_sc decreased with increasing separation between the electrodes (r=−0.120, p<0.0001; Fig. 2A shows binned data for illustration, but the correlation test was performed on the 3240 total pairs).

Figure 2.

Correlated variability. A, The average r_sc for pairs of neurons as a function of distance. Neuronal pairs were grouped in 0.5 mm distance groups starting at 0.25 mm. The redB line represents data from Monkey B, while the black line represents data from Monkey D., Average r_sc for pairs of neurons as a function of r_signal, or tuning similarity. C, The dependence of r_sc on both distance and tuning similarity for the pooled data from both monkeys. Bins containing less than 10 neurons are removed (indicated with white). The scale of the surface plot is indicated by the bar at the right of the panel. The data in this and all subsequent surface plots were smoothed with a two-dimensional Gaussian (SD of 1 bin) for display purposes only.

A number of previous studies have reported that pairs of neurons with similar visual tuning exhibit the strongest spike count correlation, while pairs with dissimilar or opposite tuning are less correlated (Zohary et al., 1994; Lee et al., 1998; Bair et al., 2001; Kohn and Smith, 2005; Smith and Kohn, 2008). This was also true in our V4 data. We measured the tuning similarity of a pair of neurons by taking the Pearson’s correlation of their tuning curves to the 12 grating stimuli (r_signal). There was a significant positive correlation between r_sc and r_signal (r=0.147, p<0.0001; n=3240 pairs), indicating that neurons with similar orientation tuning tended to have more strongly correlated trial-to-trial variability (Fig 2B).

In the two comparisons described above and in all other conditions we compared, the results were substantively similar between the animals (exceptions noted below). We therefore pooled the data from both animals for the remainder of the analyses. Figure 2C summarizes the relationship between r_sc, r_signal, and distance for all of the data from both animals. Pairs of neurons with the largest r_sc (indicated with red) tended to be close together and show similar orientation tuning (upper left of plot). Neurons with the smallest r_sc were physically distant and had dissimilar tuning (blue regions, lower right of plot).

Variations among neuronal responses based on the stimulus history could conceivably affect measures of correlation. To determine if such effects were present in our data, we ranked the stimuli from best to worst for each neuron. We then computed z-scored rates for each trial from each neuron, and then found the average z-scored rate after each stimulus, sorted by rank. The goal was to measure whether the presentation of a good or bad stimulus tended to have a measurable effect on the subsequent stimulus. Responses were not significantly different between the stimuli that followed the best stimulus compared to the stimuli that followed the worst stimulus (p=0.904), and there was no trend for the responses to vary as a function of preceding stimulus rank. We conclude that stimulus history does not play a significant role in influencing the correlation measures we report here.

In area V1, where there is a columnar structure for orientation, nearby neurons have similar orientation tuning. This kind of trend might manifest in our data as a negative correlation between r_signal and distance. In our previous study in V1 (Smith and Kohn, 2008), we found no such interaction in the full data set (r=0.000, p=0.947), although 1 of the 7 implants did show a significant negative correlation. Here, in V4, we found similarly inconsistent results between implants – there was a negative correlation between r_signal and distance in one animal (r=−0.125, p<0.0001) but not in the other (r=−0.034, p=0.193). We conclude that the presence or absence of such a trend in a particular array implant depends on the spacing of the electrodes in the array relative to the features of the cortical environment where the implant is performed, such as columnar structure, lateral connections, and inter-areal projection patterns.

Time scales of correlated variability

Correlated trial-to-trial variability can arise from network interactions on multiple time scales. Our measurement of r_sc was made over the full 1000 ms duration of the stimulus. To extend this analysis, we used another metric, r_ccg, calculated as the integral of the CCG normalized by the area under the autocorrelograms of the neurons in the pair. By changing the temporal window of integration, r_ccg is useful for assessing correlation arising from precise temporal coordination (when the window is small) all the way up to long time scale interactions (when the window is large). When the window of integration is set to the full stimulus duration, r_ccg is mathematically equivalent to r_sc (Bair et al., 2001).

Figure 3A shows the value of r_ccg computed over a range of temporal integration windows for neuronal pairs with a range of separations. Cross-sections of this surface (Fig 3C) in four distance groups (0-4.0 mm in 1.0 mm steps) reveal that r_ccg increases as a function of the size of the temporal integration window, reaching a peak at the full stimulus duration (1000 ms). The value of r_ccg at this point (the rightmost edge) is equivalent to r_sc. The time scale of correlation had a similar dependence on tuning similarity, as shown in Figure 3B and D.

Figure 3.

Effect of distance and tuning similarity on the time scale of correlation. A, Surface plot showing the value of r_sc as a function of the distance between electrodes for a range of integration windows (1 ms up to 1 s). All data in this figure are pooled from both monkeys. B, Analogous surface plot of the value of r_sc, this time as a function of the tuning similarity (r_signal). The blank white region at the bottom indicates that no neuronal pairs had a r_signal value between −0.9 and −1.0. The scale bar for panels A and B is shown at the right. C and D, Line plots of r_ccg as a function of integration window taken for several cuts from the surfaces in A and B.

Synchrony

Synchrony is a measure of pairwise correlations on a millisecond time scale, and is generally measured by a shuffle-corrected CCG. As described above, the value of r_sc is related to the area under the full shuffle-corrected CCG. In Figure 4A, we show this shuffle-corrected CCG (black lines) calculated for neuronal pairs grouped by electrode separation in 0.5 mm increments. The effect described above, a decrease in r_sc as a function of distance, is clearly visible in the decrease in the area under the CCG as the pairs of neurons were increasingly farther apart. Inspection of the overall shape of the CCG revealed that there were interactions on at least two time scales, a sharp peak on the order of tens of milliseconds riding on top of a broader hill spanning hundreds of milliseconds. This was similar to our previous findings in V1, which also demonstrated two dominant time scales of neuronal interactions (Smith and Kohn, 2008).

Figure 4.

Synchrony. A, Shuffle-corrected CCGs (black line) and jitter-corrected CCGs (gray lines, with a jitter window of 50 ms) for pairs of neurons in V4. Nearby neurons are shown at the top, and distant pairs at the bottom, in 0.5 mm bins. The vertical gray ticks represent zero time lag and the horizontal gray ticks represent 0 coinc./sp. Vertical and horizontal scale bars are shown at the bottom left. B, The same jitter-corrected CCGs as in A, but with an expanded vertical and horizontal scale (indicated with the scale bars at the bottom right).

We used a jitter-correction method to isolate neuronal interactions on fast time scales from correlated changes in trial-to-trial variability on a long time scale (see Methods and Smith and Kohn, 2008). In our prior study of V1 (Smith and Kohn, 2008), we found that a jitter window of 50 ms abolished correlation on a time scale longer than the window size and effectively isolated the synchronous interactions among neurons from the slower time scale interactions. Here we analyzed our V4 data in the same way, with jitter-corrected CCGs superimposed on the corresponding shuffle-corrected CCGs (Figure 4A, gray lines, and Figure 4B, showing same jitter-corrected data magnified). It is clear that the slow interactions are removed; the jitter-corrected CCG shows only a peak on brief time scales (about 5-10 ms half-width).

To quantify these pairwise synchrony effects, we calculated the area under the jitter-corrected CCG peak for each pair of neurons, integrating from −10 to +10 ms around the zero time lag bin, and averaged this value for sets of neurons within each bin of pair separation distances (Fig 5A). Similar to our result for correlated variability (r_sc), we found that synchrony was stronger among nearby neurons and decreased with distance (r=−0.147, p<0.0001). We also grouped neurons based on their tuning similarity (r_signal, Fig 5B). Neurons with similar tuning tended to fire more synchronously (r=0.051, p=0.004). Although highly significant, this effect was weaker than the distance dependence. Unlike the rest of the findings that we report, this effect was present in only one of the two monkeys (Monkey D, r=0.111, p<0.0001; Monkey B, r=0.013, p=0.63).

Figure 5.

Influence of distance and tuning similarity on synchrony. A, Synchrony, as measured by the average area under the CCG within 10 ms of the zero time lag bin, as a function of the distance between neurons. B, Synchrony as a function of tuning similarity (r_signal). C, The dependence of synchrony on both distance and tuning similarity shown as a surface plot. Bins containing less than 10 neurons are removed (indicated with white). The bar at the right of the panel indicates the scale.

We further analyzed pairs of neurons for synchronous interaction considering distance and tuning similarity simultaneously (Fig 5C). We found a general trend for nearby neurons with similar tuning preferences to have more synchrony, consistent with our findings in V1 (Smith and Kohn, 2008). However, in V1 we saw synchrony drop to values indistinguishable from zero for pairs of neurons 2 mm or more apart, whereas synchrony in V4 remained above zero at that distance. Thus, the falloff of synchrony with distance was steeper in V1 than in V4. We know, however, that the spatial scale of correlation can depend on the time scale of analysis (Smith and Kohn, 2008), so it seemed possible that this could account for the apparent lack of complete falloff in V4. To test this possibility, we performed an additional analysis designed to compare synchronous activity across different time scales. We computed jitter-corrected CCGs with a range of window sizes (10, 20, 50, 100 ms). In each case, we computed the synchrony as the area under the CCG peak within half the window size of the zero time lag bin (5, 10, 25 and 50 ms). We then normalized the average synchrony values to a peak value of one. Figure 6A shows normalized synchrony as a function of distance plotted for these various jitter windows (solid gray and black lines), for a shuffle-corrected CCG (dashed black line, with area calculated under the entire CCG), and for the V1 data from Smith and Kohn (2008) that used a 50 ms jitter window (red line with open symbols). Synchrony in V4 on a very fine timescale (10 ms window, i.e. pairwise spiking of two neurons with synchrony ≤ 5 ms, thick gray line) had a distance dependence that closely matched that in V1 at a much longer time scale (50 ms). For longer time scales in V4 (50-100 ms, thin black and dark gray lines), the distance dependence was similar to that for the shuffle-corrected CCG (and thus to r_sc). This implies that, in V4, only correlated spiking with a very high temporal precision (≤ 5 ms) has a tight spatial extent, while even moderately precise alignment (50 ms or more) has a relatively broad spatial distribution. This is not to say that >50 ms synchrony is unlimited in space across V4; its limit was simply beyond the spatial extent that our arrays could test.

Figure 6.

Measuring the time scale of synchrony. A, Normalized synchrony (adjusted to a peak of one) as a function of distance between neurons. The dashed line represents the shuffle-corrected CCGs, the solid gray lines represent jitter-corrected CCGs of various temporal windows, and the solid red line represents V1 data from Smith and Kohn (2008), using a jitter window of 50 ms. In the V4 lines, the area under the peak is measured for plus and minus one half of the jitter window on either side of the zero time lag bin. For the shuffle-corrected CCG, area was measured under the entire CCG (±1 s). B, Normalized synchrony, with the same conventions as in A, plotted as a function of tuning similarity (r_signal).

We also computed synchrony as a function of tuning similarity using similar analyses (Fig 6B). Here, the V1 data (red line) seems to fall within the range of the V4 data. For both V4 and V1, on long and short time scales, synchrony is only weakly correlated with r_signal.

Although the average CCG metrics in V4 agreed well with V1, we did notice that among individual pairs of V4 neurons, CCGs with an obvious peak near zero were substantially less frequent than we had previously observed in V1. Using the same metric we employed previously in V1 (Smith and Kohn, 2008), we determined for each pair whether the jitter-corrected CCG (50 ms window) had a significant peak. A peak was deemed significant if it had a bin within 10 ms of zero time lag that exceeded 5 SDs of the noise level (averaged values from −250 to −200 and 200-250 ms). For nearby pairs (< 0.75 mm), only 3.6% (23 of 645) fulfilled this criterion in V4, while in V1, 13% of pairs at this distance had significant peaks (Smith and Kohn, 2008). We did notice, as in V1, that in V4 the number of significant peaks decreased with distance (1.3% of peaks > 0.75 mm). To quantify this observation, we calculated the percentage of pairs that had significant peaks at each distance in 0.25 mm bins starting from the nearest neurons and proceeding to the most distant pairs we recorded. We found that the percentage of significant pairs decreased sharply with increasing separation (Spearman’s ρ=−0.87, p<0.0001). Thus, while we have noted some differences between synchrony in V4 and in V1, the overall trend was for similar characteristics in the two areas, such as dependence on distance and tuning similarity.

Correlation of spontaneous activity

Spontaneous cortical activity tends to be more correlated than evoked responses (Kohn and Smith, 2005; Smith and Kohn, 2008). This effect has been reported primarily in anesthetized animals, and has not been explored thoroughly in the awake state. Correlation decreases among V4 neurons when attention is directed at their receptive fields (Cohen and Maunsell, 2009; Mitchell et al., 2009) and among neurons in rat barrel cortex during active whisking (Poulet and Petersen, 2008). There is some reason to believe, therefore, that the difference between spontaneous and evoked activity is a general one, present in anesthetized and awake animals and across sensory domains. We tested this in our experiment by recording data while the animals fixated with a blank screen present. In the two monkeys, we recorded 1122 pairs of neurons in which both had a sufficient firing rate (2 sp/s) to compute r_sc. The mean r_sc in that group was r=0.110 ± 0.003, greater than the evoked r_sc value computed above (p < 0.0001). The value of r_sc in the spontaneous (dashed line) and evoked (solid line) conditions is shown in Figure 7A, binned by the distance between neurons. At all of the distances, the spontaneous r_sc is greater than the evoked r_sc by a factor of 1.9 on average. This ratio is very close to that observed in V1 (1.7; Smith and Kohn, 2008). The value of r_sc was also correlated between the spontaneous and evoked conditions (r=0.215, p<0.0001), but the r_sc during the spontaneous state was not predicted by the value of r_signal in the evoked data for the same pairs (r=0.035, p=0.289).

Figure 7.

Differences between spontaneous and evoked activity. A, Correlated variability r_sc) as a function of the distance between neurons for spontaneous (dashed line) and evoked (solid line) activity. B, The time scale of correlation is shown here for the same neurons, with a plot of r_ccg as a function of integration window.

We also examined whether this difference between spontaneous and evoked r_sc would hold at all time scales. We computed the r_ccg for both spontaneous (dashed line) and evoked (solid line) conditions in Figure 7B. We found that at all time scales, spontaneous activity was more correlated than evoked activity. This is consistent with our observations of r_sc on a long time scale and with a previous report in V1 (Kohn and Smith, 2005).

Interactions on long time scales

Correlation can also arise from slow processes, such as motivation or arousal, that act over a time scale of greater than one trial (on the order of several seconds). One way to look for such effects is to compute the trial cross-covariance (TCC) of the spike counts (Bair et al., 2001), essentially the r_sc computed with various trial offsets for each pair. At a trial lag of zero, the TCC is equal to the value of r_sc. For a given trial lag, a non-zero value indicates that the spike count from one neuron in the pair is correlated with the spike count of the other neuron several trials later. Because our trials consisted of 1 s of stimulus presentation, a one trial lag is at least 1 s plus the time required for the animal to make a saccade to the target and return to fixation. In practice, the median time between the start of one trial and the start of the next was 2.5 s.

The TCC values for our evoked (thin black line with standard error indicated by the shaded gray region around the line) and spontaneous (thin red line with light red shaded region) V4 data are shown in Figure 8. The value of the TCC was clearly greater than zero away from the central point, indicating that some long time-scale fluctuations were present in the data that affected r_sc. In order to assess these effects, we replaced the value at zero with the average of the adjacent values and smoothed the curves for evoked (thick black line in inset panel) and spontaneous (thick red line) activity (following the procedure of Bair et al., 2001). The value of r_sc can be considered to be approximately the sum of a short term (r_st) and long term (r_lt) component. The value of r_lt is read from the smoothed curves at zero, and the value of r_st can be approximated as the difference between r_sc and r_lt. This analysis complements the r_ccg metric above in that it provides a view of the constituents of r_sc, this time on long time scales.

Figure 8.

Long time scale correlation. Here the trial cross-correlogram (TCC) is shown for spontaneous (red line) and evoked (black line) activity. The standard error of the mean around each data point is shown as a shaded region (red or light gray) around the TCC. The TCC shows the amount of correlated variability as a function of the trial lag between the responses of the neurons. For example, at lag of one trial, the responses of one neuron are correlated with the responses of another neuron collected during the subsequent stimulus. The value at a lag of zero trials is the same as r_sc. The inset shows an enlarged view of the data near a trial lag of zero. The thick gray and red lines represent fits to the TCC data with the zero-lag point excluded (see Methods) for evoked and spontaneous data, respectively. The vertical scale bar represents. a TCC value of 0.02 and the horizontal scale bar represents a lag of two trial. The intersection of the two scale bars indicates the origin.

Across all pairs of neurons, the value of r_lt in evoked (0.016 ± 0.0005) and spontaneous activity (0.010 ± 0.0006) was significantly greater than zero (p < 0.0001) and was roughly 25% of the magnitude of r_sc at lag = 0 in the evoked case (and 8% in the spontaneous case), indicating the presence of some amount of correlation that fluctuates at a time scale of many seconds to minutes. The value of r_lt was larger (p<0.0001) in the evoked than spontaneous case, which indicates that part of this long term fluctuation is related to the presence of a visual stimulus. The tuning similarity of a pair of neurons was unrelated to r_lt (r=−0.003, p=0.87), but there was a small but significant correlation between r_lt and the distance between neurons (r=0.036, p=0.039). The value of r_lt was also correlated with r_sc in both the evoked (r=0.516, p<0.0001) and spontaneous (r=0.210, p<0.0001) conditions.

The value of the TCC decreased rapidly as the trial lag increased. Given that the length of time between trials is not constant because of fluctuations in behavioral performance, it is difficult to establish the true time course of the slow fluctuations underlying r_lt. In the evoked response, there appeared to be two processes giving rise to long time-scale correlation. The value of the TCC was highest within about 10 trials of zero lag, after which it dropped to roughly half the magnitude and remained above zero until trial lags of 50-100. In our two animals, the median time between successfully completed trials was 2.5 s. This would indicate that processes on the order of 25 s and 2-5 minutes, respectively, contributed to the slow fluctuations we measured in the evoked response.

Effects of rate on correlation

Firing rate is known to influence the magnitude of neuronal correlation – neuronal pairs that fire more are likely to have larger values for r_sc and synchrony (de la Rocha et al., 2007; Smith and Kohn, 2008). In our population of 113 neurons, the average firing rate was 5.5 ± 0.52 sp/s in response to the grating stimuli (averaged across all orientations). In all of the pairs recorded among these neurons, there was a significant correlation between the geometric mean firing rate of the pair and the r_sc (r=0.32, p<0.0001, n=3240). This was also true for synchrony as calculated for Figures 4 and 5 (r=0.32, p<0.0001, n=3240). There was, however, no relationship between firing rate and distance (r=−0.01, p=0.44), indicating that changes in firing rate alone cannot explain the results presented here.

The magnitude of spike count correlation we observed in V4 (0.058) was less than in our previous report in V1 (0.176; Smith and Kohn, 2008). Given the influence of rate on correlation, a difference in firing rates between V1 and V4 could explain this result. To determine if this was the case, we took the geometric mean firing rate for a pair of neurons to each orientation of the stimulus, and then averaged across the orientations to produce a single measure of the responsiveness of each pair. We then log-transformed this value to normalize the firing rate distributions and grouped pairs of neurons with similar mean rates. The value of r_sc is shown for V1 (open circles) and V4 (filled circles) as a function of mean rate in Figure 9. The average r_sc in V1 was significantly higher (p < 0.05, one-tailed t-test) than V4 for pairs of neurons in every group above 0.75 sp/s (only 5 pairs of V1 neurons were below this mean rate). Thus, the lower magnitude of r_sc in V4 compared to V1 cannot be explained by differences in firing rate.

Figure 9.

Correlation and rate. The spike count correlation (r_sc) is plotted here as a function of the mean firing rate of the neuronal pair for V1 (open circles) and V4 (filled circles). The V1 data are taken from Smith and Kohn (2008), recorded in anesthetized macaque monkey. Asterisks indicate a significant difference (p < 0.05, one-tailed t-test) between V4 and V1. For rate-matched groups of neurons, r_sc is larger in V1 than V4 across a range of firing rates from 0.75 sp/s to the highest rates we recorded. Note that there were only 5 total pairs of V1 neurons that comprised the data for the leftmost two open circles.

DISCUSSION

We found that correlations in V4 depend on both inter-neuronal distance and tuning similarity. Across time scales, correlations were greater for spontaneous than stimulus-evoked activity, roughly by a factor of two. The results were qualitatively similar to those we found in V1, although quantitatively more modest.

Correlated trial-trial variability

Few studies have attempted to thoroughly measure correlated variability across neurons. Two previous investigations in V4 (Cohen and Maunsell, 2009; Mitchell et al., 2009) used an attentional task. Cohen and Maunsell (2009) reported that correlated variability was higher for V4 pairs of neurons with similar orientation tuning, but was not dependent on distance. The former result agrees with our data, while the latter does not. Their study utilized a 6x8 Utah array, with a maximum electrode separation roughly 2/3 as large as our 10x10 array. Given the relatively slow decrease in correlated variability with distance, the decrease may have been less noticeable in their data. Alternatively, the falloff of cortical variability with distance may depend on factors that vary with eccentricity; their arrays were implanted more dorsally than ours, leading to more eccentric receptive fields. There is some evidence that central and peripheral V4 have different interareal connection patterns (Ungerleider et al., 2008). Cohen and Maunsell (2009) did not find significant correlated variability between neurons in opposite hemispheres, indicating that for very large distances, neurons in V4 are effectively independent.

Most studies of correlated variability report similar numbers across a wide variety of stimuli and recording preparations (see Cohen and Kohn, 2011 for review), but one recent study was an exception. Ecker et al (2010) reported a mean value of r_sc in primate V1 of 0.01, significantly greater than but very close to zero. They suggested that larger values of r_sc from previous studies were due to errors in spike sorting or uncontrolled cognitive factors. We did not include pairs of neurons recorded from the same electrode, and our results were robust to changes in criteria for included units (adjusting the SNR threshold), indicating that spike sorting errors were not the source of our larger r_sc. Our animals performed a basic fixation task during which attention might have influenced V4 activity; in principle, fluctuations in attention from trial to trial could lead to an increase in observed r_sc. However, it is unlikely that such fluctuations explain our r_sc magnitude, given its similarity to that found in other studies of V4 (Cohen and Maunsell, 2009; Mitchell et al., 2009) and V1 (Smith and Kohn, 2008). Finally, unless downstream neurons have access to the source of such fluctuations and can factor them out, they would appear as “noise” to those neurons.

Synchrony

In the visual system, synchrony has long been proposed as a mechanism for binding object representations (Gray, 1999) as well as for coordinating information transmission between cortical regions (Fries, 2009). More recently, synchrony has been proposed as a substrate for attention effects in V4. Most studies have focused on gamma oscillations (typically 30-80 Hz) as the measure of synchrony (Fries et al., 2001; Bichot et al., 2005; Taylor et al., 2005; Rotermund et al., 2009), with less investigation of synchronous spiking activity. Cohen and Maunsell (2009) found only chance levels using a shuffle-corrected CCG. However, their 200 ms analysis window may have made detection difficult due to small numbers of spikes. We also found only a small percentage of pairs of V4 neurons with significant CCG peaks, but by averaging the noisy signals present in individual CCGs, we were able to measure the structure of synchrony. Our study thus represents the first systematic investigation of spiking synchrony in area V4. We found that the presence of synchronous spiking activity, at very fine temporal scales (5 ms), was highly dependent on distance: it was present almost exclusively among neurons separated by less than 2 mm. Synchrony also tended to be stronger among neurons with similar tuning.

Comparison between V4 and V1

We concluded that V4 differs from V1 in two distinct ways: it had weaker correlated variability and less temporally precise synchrony. The spike count correlation we observed in V4 (0.06) was consistent with two recent studies in V4 (0.04 by Cohen and Maunsell, 2009; 0.05 by Mitchell et al., 2009) but markedly less than that found in V1 recordings (range 0.1-0.25: Kohn and Smith, 2005; Gutnisky and Dragoi, 2008; Smith and Kohn, 2008; Poort and Roelfsema, 2009; Samonds et al., 2009; Rasch et al., 2011). Our previous study in V1 (Smith and Kohn, 2008) used methods and analyses most closely matching those used here and thus provides the best direct comparison. The present study differed from that one in three notable ways. First, Smith and Kohn (2008) used opiate-anesthetized animals. Correlation values are similar, however, among V1 reports that used anesthetized (Kohn and Smith, 2005; Smith and Kohn, 2008) and awake (Gutnisky and Dragoi, 2008; Poort and Roelfsema, 2009; Samonds et al., 2009; Rasch et al., 2011) preparations, implying that effects of opiate anesthesia on r_sc are negligible. Second, our V4 recordings were slightly deeper than our V1 recordings (1.0 vs. 0.6 mm array insertion depths). Thus, we were likely closer to the middle layers in V4. In V1, correlations depend on cortical layer, with a minimum value found in the middle layers (Hansen et al., 2012; Smith et al., 2013). However, in V2, the magnitude of correlation is relatively constant across layers (Smith et al., 2013), suggesting that a strong dependence of correlation on cortical layer may be unique to V1 (and perhaps other primary sensory regions). Laminar trends alone are thus an unlikely explanation for decreased correlations found in V4 vs. V1. Finally, in our awake animals, it is possible that small eye movements during stimulus presentation introduced spurious correlations. For neurons with similar receptive fields, microsaccades could tend to bring the two receptive fields jointly into slightly better or worse alignment with the visual stimulus, leading to correlated modulation in the two neurons’ firing rates from trial to trial. Because of an error in eye position data collection, we could not apply offline analysis to rule out this scenario definitively. However, microsaccade-related effects were unlikely for three reasons: (1) We found that r_sc was higher in spontaneous activity than evoked. Microsaccades should have led to higher correlations in evoked activity, as receptive fields were jittered over a visual stimulus. (2) The relationships between correlation and other factors (e.g. distance and firing rate) in V4 were remarkably similar to the relationships found in V1, for which data were collected under anesthesia and paralysis such that eye movements were absent and could not have affected correlations. (3) Another study in area V4 (Mitchell et al., 2009) reported a similar level of neuronal correlation as we observed, and found that fixational eye movements had no appreciable effects on correlated activity. We conclude that the correlation differences we observed between V4 and V1 were not artifactual.

Synchrony in V4 neurons was somewhat less temporally precise than in V1, with a smaller proportion of V4 pairs having significant peaks, and yet the average CCG peak had a similar height. This may have in part been due to fewer trials in our V4 data and added noise from behavioral variability. In V4, synchrony fell off rapidly as the distance between neurons increased, but this was true only when isolating precise spiking (within 5 ms of the peak). In V1 the distinction between fast and slow time scales of correlation was made more easily. There are few data available on the precise time scales of correlation across visual cortex, and comparisons with other studies are difficult because analysis methods vary widely for computing synchrony. The data that are available show synchrony that is generally comparable with that observed here (Bair et al., 2001; Thiele and Stoner, 2003). Synchrony is also known to depend on stimulus features such as orientation (Gray and Viana Di Prisco, 1997; Friedman-Hill et al., 2000; Frien et al., 2000; Kohn and Smith, 2005) and spatial integrity (Zhou et al., 2008), leaving open the possibility that the differences we found between V4 and V1 might be due, in part, to the relative effectiveness of the two stimuli at driving the neurons.

The differences we observed between V4 and V1 are likely to reflect the local cortical architecture of the areas. Horizontal connections in V4 follow a similar profile to V1, but with somewhat larger patches with greater spacing (Yoshioka et al., 1992). V4 also appears to have a different pattern of connections centrally than peripherally (Ungerleider et al., 2008), and a functional organization for color and form (Tanigawa et al., 2010). All of these differences could affect neuronal correlation. Finally, correlations in various cortical circuits might also differ due to mechanisms such as gamma oscillations produced by recurrent networks or biophysical properties of each area’s neurons (Traub et al., 2001; Hasenstaub et al., 2005; Morita et al., 2008).

Differences between spontaneous and evoked activity

Studies have described extensive structure in spontaneous population activity, using methods as varied as intracellular recordings (Okun and Lampl, 2008; Poulet and Petersen, 2008), field potentials (Fukushima et al., 2012), optical imaging (Tsodyks et al., 1999; Kenet et al., 2003), and MRI (Nir et al., 2008). A comprehensive study of single-neuron variability across cortical areas (Churchland et al., 2010) showed a decrease shortly after stimulus onset, consistent with the idea that the pre-stimulus (or “spontaneous”) period activity has higher correlated variability. We found that correlation in spontaneous activity, when measured over any time scale, exceeded that of evoked activity. Over the length of the trial, the difference was approximately twofold, very close to the ratio reported in V1 in anesthetized macaques (Smith and Kohn, 2008). It is therefore reasonable to ask about the ecological relevance of spontaneous state measurements. The spontaneous state, in both anesthetized and alert animals, may be closely related to the “unattended” condition (Cohen and Maunsell, 2009; Mitchell et al., 2009). When a visual stimulus is presented, attention is exogenously drawn to that stimulus, reducing correlation. A cue that affects the allocation of attention can further modulate correlation, pushing it closer to the attended or unattended conditions.

Conserved structure of neuronal correlation

In two previous studies in V1 (Kohn and Smith, 2005; Smith and Kohn, 2008), we demonstrated that synchrony and correlated variability have different stimulus dependence and spatial extent, and suggested that at least two distinct mechanisms contribute to correlation. Here, we extend those findings to V4, a stage beyond V1. Despite the differences discussed above, the overall trend was for both correlated variability and synchrony to follow similar rules in the two areas. It is likely, therefore, that neuronal correlation is primarily driven by circuits that are highly conserved across different regions of visual cortex, such as recurrent feedforward/feedback and horizontal intracortical connections.