Natural Movies Evoke Spike Trains with Low Spike Time Variability in Cat Primary Visual Cortex

Abstract

Neuronal responses in primary visual cortex have been found to be highly variable. This has led to the widespread notion that neuronal responses have to be averaged over large numbers of neurons to obtain suitably invariant responses that can be used to reliably encode or represent external stimuli. However, it is possible that the high variability of neuronal responses may result from the use of simple, artificial stimuli and that the visual cortex may respond differently to dynamic, naturalistic images. To investigate this question, we recorded the responses of primary visual cortical neurons in the anesthetized cat under stimulation with time-varying natural movies. We found that cortical neurons on the whole exhibited a high degree of spike count variability, but a surprisingly low degree of spike time variability. The spike count variability was further reduced when all but the first spike in a burst were removed. We also found that responses exhibiting low spike time variability exhibited low spike count variability, suggesting that rate coding and temporal coding might be more compatible than previously thought. In addition, we found the spike time variability to be significantly lower when stimulated by natural movies as compared with stimulation using drifting gratings. Our results indicate that response variability in primary visual cortex is stimulus dependent and significantly lower than previous measurements have indicated.

Introduction

As an animal navigates through its environment, it relies on the responses of its sensory systems to identify objects and locations in its surroundings. This implies a reproducible relationship between specific features of the environment (such as colors, sounds, smells, etc.) and certain patterns of neuronal activity in the sensory system. In contrast to this notion, previous studies in visual cortex have found high variability in spike counts to repeated stimulus presentations (Schiller et al., 1976; Tolhurst et al., 1983; Vogels et al., 1989; Buracas et al., 1998) (but see Gur et al., 1997; Kara et al., 2000; DeWeese et al., 2003; Amarasingham et al., 2006; Gur and Snodderly, 2006; Nawrot et al., 2008; Benedetti et al., 2009). In fact, the Fano factor (FF), the ratio of the variance of the spike counts across repetitions to the mean spike count, was often found to be higher than that of a random Poisson process, which has a FF of 1. The variability in spike times has also been shown to be quite high (Tomko and Crapper, 1974; Shadlen and Newsome, 1998; McAdams and Maunsell, 1999; Tolhurst et al., 2009) (but see Richmond and Optican, 1990; McClurkin et al., 1991; Bair and Koch, 1996; Buracas et al., 1998; Nawrot et al., 2008; Benedetti et al., 2009; Maimon and Assad, 2009). As a result, neuronal responses have often been modeled using a non-uniform Poisson process, in which the probability of a spike or the firing rate would be constant for a short time window, and then change to a different rate as a function of the stimulus (Victor and Purpura, 1996; Berry and Meister, 1998; Oram et al., 1999). Using this model, the variance in the spike counts in each time window would be equal to the mean spike count, and the spike times would be randomly distributed.

However, these studies have most often used simple, artificial stimuli like drifting bars and gratings, and random dot patterns, which lack the spatial (Field, 1987) and temporal (Dong and Atick, 1995) characteristics of natural visual images and thus might not be predictive of how these neurons would respond under natural stimulation. Indeed, a number of recent studies have shown that, when given appropriate stimuli, neurons that previously showed high response variability can exhibit highly reproducible responses (Mainen and Sejnowski, 1995; Bair and Koch, 1996; de Ruyter van Steveninck et al., 1997; Buracas et al., 1998; DeWeese et al., 2003; Gur and Snodderly, 2006) (but see Kayser et al., 2003).

To study the responses of visual cortical neurons to more naturalistic stimulation, we recorded unit activity in the primary visual cortex of anesthetized cats, while presenting the animals with repeated presentations of movies of natural images (Yen et al., 2007). We computed the spike count and spike time variability and compared them with surrogates generated using a non-uniform Poisson process with a relative refractory period. We also performed these analyses on a subset of cells that were stimulated with both natural movies and drifting sinusoidal gratings. This enabled us to compare directly the variability elicited by complex and simple stimuli.

Materials and Methods

Surgery, electrophysiology, and visual stimulation

All experiments were conducted on anesthetized and paralyzed cats (3–4 kg) of both genders. The surgical, electrophysiological methods, as well as the details of the visual stimulation have been described in detail previously (Yen et al., 2007). We present a summary of the relevant details below.

Recordings were made using either sharp tungsten electrodes (four cells) or 16-contact, multitetrode silicon probes (84 cells) provided by NeuroNexus. Monochromatic natural movie sequences were presented at a distance of 57 cm using a 20 inch color monitor (ViewSonic P810) at a resolution of 800 × 600 pixels. During the course of this study, we used three different graphics cards, with increasing refresh rates of 85, 120, and 150 Hz, and refresh intervals of 11.8, 8.33, and 6.67 ms.

Movie frames were digitized from a variety of DVD movies covering a wide range of natural and man-made scenes. These included Hollywood movies like The Big Lebowski, The Matrix, Everest, as well as two documentaries called The Ultimate Guide—Big Cats and House Cats and Animals in the Wild. Individual movie frames had dimensions of 640 × 480 pixels and were converted to grayscale images. The movie was positioned at the center of the monitor screen, occupying 32 × 24° of visual angle (0.05°/pixel), and was presented at rates of three, four, or six refreshes per frame (corresponding to the 85, 120, and 150 Hz graphics cards), making the frame rate of the movies 28.3, 30, and 25 Hz, respectively, with corresponding frame durations of 35, 33, and 40 ms. For each recording site, we selected a short movie sequence, ranging in duration from 15 to 30 s, and presented that clip 60–200 times to the dominant eye.

We also presented stimuli consisting of drifting, oriented sinusoidal gratings, with different temporal and spatial frequencies. Each block consisted of pseudorandom permutations of combinations of 24 orientations, 5 temporal and 5 spatial frequencies, for a total of 600 different stimulus conditions. We presented each condition for 2 s, and at the end of each block, we collected a 5 s period of spontaneous activity. One block thus lasted for 1205 s, and we presented a total of five blocks at each recording site. All grating stimuli were presented using a 150 Hz graphics card.

The protocol used in the experiments was approved by the Institutional Animal Care and Use Committee at Montana State University and conformed to the guidelines recommended in Preparation and Maintenance of Higher Mammals during Neuroscience Experiments (NIH publication no. 91-3207; National Institutes of Heath, Bethesda, MD; 1991).

Data analysis

We recorded 88 well isolated neurons from all layers of area 17 in eight anesthetized cats. All 88 cells exhibited responses that were significantly different from random spike trains [see details in the study by Yen et al. (2007)]. We also recorded from 10 cells in an additional two animals in which we were able to obtain responses to both natural movies and drifting sinusoidal gratings. We focus first on the 88 cells for which we only recorded responses to movie stimuli and carry out the comparison between movie and grating responses in the last section.

To examine the distributions of intervals between spikes, we computed the interspike interval (ISI) histograms for each cell using bin sizes of 0.2 ms. For 28 of the cells, we found <50 intervals that were <50 ms. This made it difficult for us to compute the spike recovery function (see below), which was required for our analysis. In addition, we were concerned that such highly sparse activity could make the analysis of temporal variability difficult. Indeed, we found that these cells exhibited significantly lower variability than the rest of the cells, and we therefore chose to exclude these cells from further analysis. Thus, in the subsequent analysis, we analyzed the remaining 60 cells.

Spike latency analysis.

To examine spike times relative to the onset of movie frames, we computed the peristimulus time histogram (PSTH) relative to the start of each movie frame. In other words, we computed a histogram (at 1 ms resolution) of the latency between each spike time and the start time of the movie frame in which the spike occurred. This allowed us to view the distribution of spike times in relation to the frames of the movie and the screen refreshes on the monitor. To quantify refresh-following in the PSTHs, we zero-padded the histograms to a length of 64 points and computed the fast Fourier transform (FFT) on the entire window. We then computed the ratio of the magnitude of the frequency component closest to the refresh rate of the monitor, to the magnitude of the DC component. After visual inspection, cells with magnitude ratios >0.13 (n = 9) were classified as having refresh-following and removed from further analysis. We also performed a multitaper analysis but found the simpler FFT approach to give better results.

Surrogate generation.

To determine the spike count variability and temporal reliability of our data, we compared it to surrogate data computed using methods from Berry and Meister (1998). Briefly, spike trains were modeled as time-varying Poisson processes with an explicit recovery function to model the absolute and relative refractory period of a neuron. This was done by modulating the stimulus induced firing rate with a function that suppresses firing completely during the absolute refractory period of a cell, followed by an increase to simulate the relative refractory period.

To compute the spike recovery function for each cell, the ISI histogram was first computed with bin sizes of 0.2 ms. Next, the counts were converted to probabilities, p(t), by dividing by the total number of interspike intervals. The recovery function, w(t), was computed up to the interval with the highest probability using Equation 1. The recovery function was then normalized so that the values ranged from 0 to 1 as follows:

The spike recovery function was then used to compute the stimulus-induced firing rate, which has also been called the free firing rate (Berry and Meister, 1998). Essentially, this computes the probability of firing if the cell did not have a refractory period and reflects the response conditioned only on the stimulus. This is computed first by placing the recovery function whenever there was a spike and averaging the resulting probability function across repetitions to obtain the probability of free firing, W(t). The free firing rate, q(t), was then obtained by dividing the observed firing rate, r(t), by W(t).

The free firing rate, q(t), and the recovery function, w(t), were then used to generate the surrogate spike trains. First, a set of random numbers, α_i, uniformly distributed between 0 and 1 were generated. Following a spike a time t_i, the next spike, at time t_i+1 was determined by numerically integrating Equation 2, given by the following:

In this study, we tested the hypothesis that the neuronal responses can be modeled using a time-varying Poisson process changing at the frame rate of the movies (i.e., 25 frames per second, or 40 ms duration per frame), so we computed the recovery function and the free firing rate at a resolution of 0.2 ms, and then averaged 200 bins (equivalent to 40 ms duration) of the free firing rate together to create surrogate spike trains that were generated with underlying free firing rate dynamics that were similar to the frame rate of the movie.

Burst-removed data and surrogates.

Since our data contained numerous cells that responded with bursts of spikes (see Fig. 2), we studied the effects of burst firing on response variability. To accomplish this, we created a second data set in which only the first spike in a burst was retained. We defined a burst as a series of spikes with interspike intervals <5 ms (Cattaneo et al., 1981; Bair et al., 1994; DeBusk et al., 1997; Maldonado et al., 2008). As it would be inappropriate to compare the burst-removed data with the original surrogates, since the spike counts will be greatly reduced, we compared the burst-removed data to a second set of surrogates, generated using the same procedure as for the original data, but with the recovery function and the free firing rate calculated from the burst-free data. Figure 6 shows an example of the effect of removing bursts.

Figure 2.

The response of a cell to a natural movie stimulus. A, Raster plot showing the spikes elicited in the cell to repeated movie presentations. Each row is a single repetition of a movie, and a total of 50 repetitions are shown for this cell. B, PSTH of the cell. C, Raster plot of the response segments highlighted by the gray rectangles in B. The panel on the left shows frames with low temporal variability, while the panel on the right shows frames with higher temporal variability. The vertical red lines denote frame boundaries; only the top three frames with the lowest entropy are shown to avoid clutter.

Figure 6.

Variability and surrogates. Raster plots of the original data (A) and the equivalent burst-removed data (B) are shown in the top row. The surrogates generated are shown in the bottom two rows. The activity in the windows marked with the letter “a” is well modeled by the surrogates, while the window marked with the letter “b” contains features that are not well modeled by the surrogates.

Spike count variability.

We characterized spike count variability by first computing the spike count in each movie frame and then computing the mean and variance of the spike counts across repetitions. The variability is then expressed as the FF, which is the ratio of the variance to the mean. The FF in each window was compared with those computed from the same window in the surrogates to obtain an FF score (FFS) that indicated the percentage of surrogates that exhibited higher FFs compared with the data. Windows with FFS ≥ 95 were marked significant. To look at how much lower the variability in the data was compared with the surrogates, we also determined a FF Z-score (FFZ) for the spike count variability in each window by subtracting the mean FF value for the surrogates from the FF of the data and dividing by the SD of the surrogate FF values. If the FFZ was more negative than −2, for example, that would mean that the spike count variability in the data was <2 times the SD of the surrogates.

Spike time variability.

To investigate the variability of spike times across stimulus repetitions, we developed a method to quantify spike time entropy, referred to here as temporal reliability entropy (TR-entropy). Figure 1A illustrates how temporal variability was computed. First, we subdivided the spike trains in each movie frame into n non-overlapping bins. Each bin inside a window was given a rank according to how many spikes it contained, such that the bin with the smallest number of spikes was ranked first. If more than one bin had the same number of spikes, they were each assigned the mean rank (e.g., if the second, third, and fourth ranked bins had the same number of spikes, they were all assigned the rank 3), following the procedure used in the Spearman rank order correlation measure to correct for equal ranks. The rank of a bin could therefore be any number in the range 1, 1.5, 2, 2.5, … n, which meant there were 2n − 1 possible ranks in each bin. For each window, we grouped all the repetitions together to calculate the probability distribution of the rank for each bin to create an n × (2n − 1) map. If the spike times within a window were identical across repetitions, the probability distribution map will be zero everywhere except in n boxes. However, if the spike times were unreliable, the maps will be quite flat (the rank of a bin in each repetition would be arbitrary). The bottom panel in Figure 1A shows the rank probability maps for the three frames shown in the top panel with different spike time variability.

Figure 1.

Illustration of methods. A, The top panel shows sample rasters from three different frames (all with mean spike counts ≥ 1), arranged from high to low entropy. The left-most frame returned a TR-entropy value of 0.098, which was near the bottom of the distribution for this cell, the middle frame returned a TR-entropy value of 0.081, which was near the median, while the right-most frame was among the frames with the lowest TR-entropy for this cell, with a value of 0.069. The bottom panel shows the resulting rank probability map (see the text for more details). B, TR-entropy and spike time variability. Each curve shows the behavior of the TR-entropy measure in windows with mean spike counts between 0 and 1 (red), 1 and 2 (green), and 2 and 3 (blue) as a function of jitter (see Materials and Methods for more details). In 80% of the windows, TR-entropy was highly positively correlated with jitter (Spearman's R > 0.5; p < 0.05).

To quantify the distributions in these maps, and thus the spike time reliability, we computed the entropy for each map using the following:

where P(r_i,j) is the probability of getting a rank i in bin j. It is important to point out that it was quite convenient to compute the entropy on the ranks instead of the actual numbers of spikes because this reduced the number of possible values that could be reached in each bin. Here, the universe of possibilities was 2n − 1, which was quite reasonable for 100 repetitions. By using ranks instead of raw spike counts, we were also able to control for changes in overall spike counts due to state changes in the animals. In all our calculations, we used n = 10, which meant that our bin sizes were ∼3.5–4 ms depending on the frame duration of the movies.

To verify that the rank-based entropy measures spike precision, we computed its value on simulated data in which we controlled the precision of spikes explicitly. This was done by randomly selecting the response from a single stimulus presentation for one of our cells. We then replicated these responses 100 times, to create simulated responses with zero temporal variability across stimulus repetitions. We then degraded the precision of these responses by adding to each spike time a random jitter drawn from a zero mean Gaussian process with increasing SD. This gave us several response sets with decreasing temporal precision. We then computed the entropy of the ranked bins as explained above for each of these response sets. In Figure 1B, we plot this entropy as a function of the amount of jitter applied to generate the surrogate data. The three different lines represent the TR-entropy of windows with mean spike counts (MSCs) between 0 and 1 (red), between 1 and 2 (green), and between 2 and 3 (blue). As Figure 1B shows, there was a high degree of correlation between TR-entropy and jitter. We found that 83.0% of the windows used in the simulation in Figure 1B had Spearman's R > 0.5 (p < 0.05).

Note that, in the entropy calculation, windows with low mean spike counts will naturally have lower entropy values compared with windows with high mean spike counts since only a few bins will have nonzero counts. To quantify how the entropy values behaved with increasing mean spike counts, we created random spike count arrays that contained 100 rows and 10 columns corresponding to the 100 repetitions and n = 10 bins used in our analysis. To simulate a window with a mean spike count of 1, we randomly incremented the spike count in 100 points in the array. To simulate a window with a mean spike count of 2, we randomly incremented the spike count in 200 points in the array, and so on. We did not simulate a refractory period, so some points in the array ended up with spike counts >1. We created 1000 such arrays for each mean spike count and computed the entropy values for each of these arrays. We then found the range of entropy values at each mean spike count that 95% of the random arrays occupied (i.e., we found the 2.5 and 97.5% percentiles of the entropy values) and then used a cubic spline to interpolate between the entropy values calculated at discrete mean spike counts to obtain a range of entropy values that we would expect from randomly placed spikes for each mean spike count (see Fig. 9).

Figure 9.

Spike time variability for 51 cells. A, The spike time entropy values for all the windows in the 51 cells in our data set are plotted against the mean spike count. Windows with MSC < 1 are plotted in gray, while the rest are plotted in black. Windows with TRES ≥ 95 are plotted in blue (for MSC < 1) and red (for MSC ≥ 1). The two green lines denote the 2.5 and 97.5 percentile of the entropy values obtained from 1000 random spike trains (see Materials and Methods). B, The distribution of TREZ for all the windows is shown in logarithmic scale on the vertical axis. The distribution for the subset of windows with MSC ≥ 1 is shown in dark gray, while the distribution for all windows is shown in light gray. C, Event interquartile range plotted against event MSC, using the same color code as for the TR-entropy shown in A. D, The distribution of event interquartile range Z-scores. The color code is the same as in B. E–H, The equivalent plots for the burst-removed data.

The entropy in each window was also compared with the entropies in the surrogates to obtain the temporal reliability entropy score (TRES). This score measured the percentage of entropy values obtained from the surrogate data that exceeded the entropy value calculated from the experimental data. Since larger entropy meant greater spike time variability, the TRES measured the percentage of the surrogate windows that exhibited higher spike time variability than the experimental data. Windows with a TRES ≥95 (i.e., windows in which the entropy of the data was lower than 95% of the surrogates) were marked significant. A normalized entropy measure, the temporal reliability entropy Z-score (TREZ) was also obtained by subtracting the mean entropy value for the surrogates from the entropy value of the data and dividing by the SD of the surrogate entropy values. Negative TREZ values indicated that the spike time variability in the data was lower than the variability in the surrogate data.

Since the TR-entropy represents a novel way of computing spike time variability, we compared our results to spike time variability computed using a standard approach based on identifying response events in the PSTH of a cell (Keat et al., 2001; Butts et al., 2007; Kumbhani et al., 2007; Desbordes et al., 2010). First, we identified event boundaries by identifying points at which the firing rate exceeded the 95th percentile. Second, events were required to have at least one spike in 50% of the trials, to prevent response epochs with intertrial nonstationarities from being included in the analysis. We then computed the spike time precision of an event by computing the interquartile range of the spike times contained within the event (Kumbhani et al., 2007). To facilitate comparison to the TR-entropy measure, we also computed the event-based spike variability of the same surrogate datasets used to compute TRES and TREZ. This was done by using the event boundaries identified from the data, and computing the spike time interquartile range of the corresponding time period of the surrogates. This allowed us to form an interquartile range score (IQRS) and interquartile range Z-score (IQRZ), which like the TRES and TREZ, measured the percentage of surrogates that was more variable than the data, and the z-scores for the interquartile range of the data compared with the surrogates, respectively.

Relationship between variability and stimulus.

To investigate whether the variability in the neural spike trains could be related to the properties of the stimulus, we performed a series of analyses on the stimulus frames. First, we investigated whether low variability in the responses was simply due to large luminance changes in the movie due to scene transitions. We used a measure to identify scene transitions previously described by Yen et al. (2007). Briefly, we computed the differences in gray level of every pixel between consecutive movie frames, and defined scene transitions as frames in which 60% of the pixels changed by 10 gray levels. We confirmed these scene changes by visual inspection of the movie frames.

A more detailed analysis sought to investigate whether stimulus motion in the frames preceding a response window influenced the spike time variability of that window. We computed the motion across five consecutive movie frames, following the techniques in the study by Ma et al. (2002). Briefly, temporal gradients were constructed across adjacent frames, and the entropy of the distribution of these gradients was estimated across five frames. This gave us a motion value for each pixel of the movie frames. We then used the spatial outline of the receptive field of each cell (obtained using hand-mapping during the experiment) to select the pixels in the motion maps that overlapped with the receptive field. By taking the average motion value for these pixels, we ended up with an average motion value for each frame, and the distribution of these frame-by-frame motion values was then compared with the distribution of TREZ values.

Results

Interspike interval distributions

Our goal in this study was to quantify the spike count and spike time variability in the neuronal responses to repeated presentations of natural movies. However, it has been shown that spike count variability can be affected by the relative refractory period (Berry et al., 1997; Kara et al., 2000; Keat et al., 2001). To account for the effect of the relative refractory period on the response variability for each cell, we first computed ISI distributions for each unit in our database. Figure 2 shows the responses of one of the cells in our database to natural movie stimuli. The raster plot in Figure 2A shows the responses to 50 stimulus repetitions, while Figure 2B shows the corresponding PSTH. In Figure 2C, we plot the rasters for the two 2 s segments highlighted in Figure 2B. Both the raster and PSTH show clear epochs of vigorous responses, with quieter periods in-between. As the close-ups in Figure 2C show, the variability across repetitions for different parts of the response is quite different. In the segment on the left, spikes appeared with seemingly high precision across repetitions, while in the segment on the right, spikes were missing in a substantial fraction of the repetitions. These differences were also reflected in the spike time variability measure that we developed (TREZ; see Materials and Methods) for the two segments. The three frames with the lowest TREZ in each segment are highlighted in-between the vertical red lines. For the first segment, these three frames had TREZ values of −9.9, −6.3, and −4.9, while the three frames in the second segment had TREZ values of −2.4, −2.2, and −1.6. This shows that the TREZ measure was able to quantify correctly that the spikes in the first segment had significantly lower spike time variability than the spikes in the second segment.

Before embarking on the variability analysis, we investigated whether some of our cells were locked to the refresh rate of the presentation monitor (Wollman and Palmer, 1995; Williams et al., 2004). We excluded cells exhibiting such refresh-following from any variability analyses as they might show artificially high levels of temporal precision close to the refresh intervals of the monitor, typically 6.67, 8.33, or 11.7 ms (see Materials and Methods). As such, we subtracted each spike time by the onset time of the movie frame in which the spike occurred and then computed a histogram of the spike times relative to the start of the movie frame. This is shown in Figure 3. The cells were sorted by the refresh rates used in the experiments, with the cells recorded using a monitor with a 150 Hz refresh rate (i.e., a movie frame consisting of six monitor refreshes) appearing first, followed by the cells recorded with monitors with 120 and 85 Hz refresh rates, corresponding to four and three monitor refreshes per movie frame, respectively.

Figure 3.

Spike latencies. Histograms of spike times relative to the start of the movie frame in which the spikes occurred. Bin width is 1 ms. The vertical lines in each histogram indicate the times of the monitor refreshes. The movie frames were presented for six monitor refreshes at a rate of 150 Hz (pink bars), four refreshes at 120 Hz (blue bars), and three refreshes at 85 Hz (gray bars). The horizontal axis limits correspond to the length of a movie frame and were 0–40 ms for cells 1–21 and 0–35 ms for the remaining cells.

Cells 52–60, recorded using a monitor with a 85 Hz refresh rate, showed clear refresh entrainment (see Materials and Methods for the quantitative measure we used), which resulted in these cells exhibiting significantly lower spike time variability than the remaining cells [two-sample Kolmogorov–Smirnov (KS) test, p < 0.05]; therefore, we excluded these cells and retained the remaining 51 cells for the subsequent analyses.

Figure 3 also revealed some interesting aspects of the spike latencies. In some cells, the spike times were uniformly distributed with respect to the onset of a movie frame, but in other cells, there appeared to be a clear bias in terms of the latency of the spikes (e.g., cells 8, 9, 10, 15, 16, 19, 20, and 25). Although there appears to be an increase in the number of spikes occurring at the start and end of each movie frame, the increase at the start of the movie frame in all likelihood is the continuation of the increase at the end of the preceding movie frame. This perhaps reflects the fact that these cells have response latencies between 33 and 40 ms, which is consistent with other studies of response latencies in the primary visual cortex of the anesthetized cat (Samonds and Bonds, 2005).

The ISI histograms for each of the 60 cells in our database are shown in Figure 4. The histograms were computed by taking the base 10 logarithm of the ISIs and binning this distribution using 100 equally sized bins to emphasize any multimodality within the histograms. A large fraction of the cells exhibited a predominance of short ISIs. The maximum ISI value in the histogram was <4 ms for 41 of the 60 cells. The remaining 19 cells are marked with asterisks in Figure 4. Most of the cells (47 of 60) exhibited multimodal peaks within the ISI histograms. The occurrence of two distinct modes; one centered at short ISIs and one centered at slightly longer intervals, indicated that these cells exhibited a significant degree of bursting firing activity. Cells exhibiting endogenous bursting behavior can be further grouped into two main classes as follows: intrinsically bursting (IB) and chattering (CH) (Gray and McCormick, 1996; Wang, 1999; Nowak et al., 2003; Traub et al., 2003; Cunningham et al., 2004; Takekawa et al., 2007). Using the criteria reported by Nowak et al. (2003) (i.e., IB cells have a large mode at intermediate ISIs and a narrower peak at short ISIs, while CH cells have a higher peak at short ISIs and a smaller and broader peak at the interburst interval), we categorized cells 2, 27–31, 54, and 55 as chattering (Fig. 4, blue backgrounds), and cells 1, 22, 26, 52, and 53 as intrinsically bursting (Fig. 4, pink backgrounds).

Figure 4.

ISI histograms. Plots of the log ISI histograms for all 60 cells. The histograms were computed by taking the base 10 logarithm of the ISIs and binning this distribution using 100 equally sized bins. The peak of the ISI histogram occurred <4 ms for all but 19 of the cells (marked with asterisks). Eight cells were labeled as CH (blue background), while 5 cells were labeled as IB (pink background). The ordering of the cells is the same as in Figure 3.

To investigate whether the predominance of short ISIs shown in Figure 4 was due to spike doublets or bursts consisting of three or more spikes, we computed the joint interval histogram for each cell (i.e., each spike interval vs its subsequent interval) (Rodieck et al., 1962; Doiron et al., 2003), and the results are shown in Figure 5. The first interval is plotted on the vertical axis and the subsequent interval is plotted on the horizontal axis. If spike doublets occurred with greater frequency (Traub et al., 1996; Higgs and Spain, 2009), we should find short intervals preceding longer intervals, as well as longer intervals preceding short intervals. A high occurrence of doublets would appear as vertical and horizontal bands on the 2D histogram (Fig. 5, cell 4). However, if bursts were more likely, we should see a majority of short intervals preceded and followed by other short intervals. This would appear as spherical regions in the bottom left corner of the 2D histograms (Fig. 5, cell 51). We found multispike bursting to be the most common property among our cell population, with most (40 of 60) of the cells showing a greater density of points in the bottom left corner of the histograms. In addition, a number of the cells appeared to have vertically elongated regions in the histograms (e.g., cell 25). This suggests the occurrence of larger first intervals followed by smaller subsequent intervals, which is typical of burst firing (Kepecs and Lisman, 2003).

Figure 5.

Joint interval histograms. Each graph plots interspike interval duration (vertical axis) versus the subsequent interval duration (horizontal axis) for intervals between 0 and 5 ms. Histogram bins were computed at 0.2 ms resolution. Consecutive short intervals, which are reflected in these plots as spherical blobs in the bottom left corner of the histogram, appeared to be most common, suggesting that many of the cells responded with bursts of firing. The percentage of burst firing in each cell is shown in parentheses after the cell number in the title of each plot.

We also quantified the percentage of burst firing in each cell by dividing the number of spikes contained in bursts in which the interspike intervals were <5 ms (Cattaneo et al., 1981; Bair et al., 1994; DeBusk et al., 1997; Maldonado et al., 2008), by the total number of spikes. This number is shown in parentheses for each cell in Figure 5. Surprisingly, the cells did not fall into easily segregated categories. Instead, we found that burst firing made up <30% of the total number of spikes in the majority of the cells, although the proportion was as high as 84% in one cell (cell 51).

Having analyzed the ISI distributions of the cells, we found evidence of two types of cells, exhibiting varying degree of bursting behavior. This prompted us to investigate the effect of this bursting on spike count and spike time variability. We were also able to estimate the refractory period via Equation 1 and generate Poisson surrogate spike trains that exhibited the same refractory effects. This was an essential step in modeling the cellular responses.

Variability and surrogates

We turned next to the issue of spike count and spike time variability in the neuronal responses to natural movies. Spike count variability has been shown to be significantly affected by the refractory period of individual neurons (Berry et al., 1997; Kara et al., 2000; Keat et al., 2001), and previous studies have shown that retinal spike trains may be modeled using time-varying Poisson processes with appropriate refractory periods (Berry and Meister, 1998). Therefore, we compared the variability of individual neurons in our database with surrogate spike trains created using a similar model to investigate how the variability in cortical spike trains compared with those found in the retina.

Using the ISI histograms in Figure 4, we computed the spike recovery function for each neuron to generate surrogate spike trains that could be compared with the experimental data. Examples of the surrogate spike trains are shown in Figure 6A. The data are shown in the top raster plot, while the surrogate spike trains generated with firing rates changing at the frame rate are shown in the bottom raster plot. The surrogate spike trains replicated most of the features of the data with the exception that they generally exhibited more uniformity in their response across trials. This was due to the fact that each repetition of the surrogates was generated using the identical free-firing rate, which in turn was computed by averaging across repetitions (see Materials and Methods). In the response epoch shown, the surrogate spike trains were able to replicate the data in most of the time windows (an example window is labeled with the letter “a”). This means that the precision of the responses was relatively coarse as surrogates generated with firing rates changing at the frame rate were sufficient to approximate the data. However, in some of the windows (e.g., in the window labeled with the letter “b”), these surrogates were not able to replicate the finer features of the data.

As shown in Figures 4 and 5, as well as in the raster plot for the data in Figure 6A, most of the cells in this study exhibited some degree of burst firing. These bursts could significantly affect the variability of the data compared with the surrogates, which are largely free of burst firing since we did not attempt to model bursting in the surrogates. To study the effect of burst firing on response variability, we generated new spike trains from the original data by retaining only the first spike in a burst and removing the remaining spikes (for details, see Materials and Methods). We then regenerated the surrogates, using the same procedure as for the original data, but with the recovery function and the free firing rate calculated from the burst-free data. Figure 6B illustrates the resulting rasters after removing spikes within bursts. The top panels show raster plots of the data, while the bottom panel shows the surrogates generated at the frame rate. These surrogates were generated based on the burst-removed data and were thus quite different from the surrogates shown in Figure 6A, which were more similar to the data.

The spike count variability (Fano factor) and spike time variability (TR-entropy) in the data, compared with those of the surrogates, is shown in Figure 7. A 1 s segment of the data is displayed in the top raster plot. The FF for the data and surrogates are shown in the middle plot. The mean and SD of the FF for the surrogates are shown in green. As expected, since the surrogates were generated using a Poisson model, the FFs were fairly close to 1, with occasional values <1, presumably due to the effect of the relative refractory period. We found that the FF for the data was unstable when the MSC was <1 spike per window, so we separated the windows into those with MSC ≥ 1 spike per window (black circles), and those with MSC < 1 (gray circles). In addition, windows in which 95% of the surrogates exhibited higher FF than the data (i.e., FFS ≥ 95) are plotted in red (for windows with MSC ≥ 1) and blue (for windows with MSC < 1, not observed in this plot). The plot in the bottom panel uses the same color scheme to illustrate the comparisons of the spike time variability measure (TR-entropy) between the data and the surrogates.

Figure 7.

Spike count and spike time variability of the data compared with the surrogates. Windows with MSCs < 1 are plotted in gray, while the remaining windows are plotted in black. Windows with FFS ≥ 95 or TRES ≥ 95 are highlighted in blue (for MSC < 1) and red (for MSC ≥ 1). The mean and SD of the surrogates are plotted in green in each of the plots.

Spike count variability

The Fano factors for all windows and all 51 cells in our study are shown in Figure 8A. Windows with MSC ≤ 1 are shown in gray, while windows with higher mean spike counts are shown in black. Windows with FFS ≥ 95 are plotted in blue (for MSC < 1) and red (for MSC ≥ 1). The green dotted line in the top plot at FF = 1 denotes the FF for a Poisson process, where the variance is equal to the mean. In agreement with previous observations in mammalian visual cortex (Schiller et al., 1976; Tolhurst et al., 1983; Vogels et al., 1989), the Fano factors for a large majority of windows were >1, indicating that the overall spike count variability was high. However, certain portions of the movie generated responses that were much less variable. We found FF < 1 in 17.1% (6599 of 38,600) of the windows with nonzero mean spike counts in our database, and 15.8% (255 of 1615) of the windows with MSC ≥ 1. In total, the percentage of cells with at least one response window in which FF < 1 while the MSC ≥ 1, was 56.9% (29 of 51). The FF and the mean spike count for all windows with MSC ≥ 1 were negatively correlated (r = −0.29; p < 0.001). This is consistent with previous reports that the FF decreases at high spike counts due to the limits imposed by the refractory period of spiking neurons.

Figure 8.

Spike count variability for 51 cells. A, The Fano factor for all the windows in the 51 cells in our data set are plotted against the mean spike count. Windows with MSC < 1 are plotted in gray, while the rest are plotted in black. Windows with FFS ≥ 95 when compared with the surrogates are plotted in blue (for MSC < 1) and red (for MSC ≥ 1). The green dashed line in the top panel indicates a Fano factor of 1, which describes a Poisson process. B, The distribution of Fano factor z-scores for all the windows are shown using a logarithmic scale on the vertical axis. The distribution for the subset of windows with MSC ≥ 1 is shown in dark gray, while the distribution for all windows is shown in light gray. C, Fano factor computed for events plotted against event MSC, using the same color code as for the frame-based FF shown in A. D, Distribution of FFZ computed based on events. The color code is the same as in B. E–H, The equivalent plots for the burst-removed data.

The effect of the refractory period was also demonstrated in our comparison of the data with the surrogates. Even though the surrogates were generated using a Poisson process, they exhibited very similar FF to the data, including values that were <1 (data not shown). When comparing the data to the surrogates, we found that only 1.18% (19 of 1615) of the windows with MSC ≥ 1 exhibited FF < 95% of the surrogates (FFS ≥ 95). This is smaller than the number of windows expected to have FF smaller than the surrogates if those windows occurred with a 5% chance. Therefore, the FF of the data could not be said to be significantly different from that of the surrogates. A possible concern arises from the observation that windows with FF < 1 also tended to have high activity. These are exactly the windows that an unmodified Poisson process is unable to model, and if our model fails for these windows too, we cannot claim to have captured spike count variability. However, when analyzing all the windows conditioned on the mean spike count, we found that the number of windows with FFS ≥ 95 did not exceed that expected from chance for any of the MSC-bins tested. In other words, even the high-activity windows were well modeled by the modified Poisson process. Additionally, the distribution of FFZs was weighted toward positive values (shown in Fig. 8B), again indicating that very few windows exhibited variability that was significantly lower than the surrogates. The windows with FFS ≥ 95 occurred in 23.5% (12 of 51) of the cells, 4 of which were previously classified as chattering, and 1 which was intrinsically bursting. These results indicate that spike count variability in the responses to natural movies appear to be well modeled by a Poisson process with a relative refractory period, with a time-varying rate close to that of the movie frame rate.

The results from the burst-removed data (shown in Fig. 8E–H) were significantly different. In this case, we found 62.8% (353 of 562) of the windows with MSC ≥ 1 exhibited FFS ≥ 95. The distribution of FFZ now included a significant number of windows with variability that was significantly lower than the surrogates (i.e., FFZ < −3). These windows occurred in 52.9% (27 of 51) of the cells, 5 of which were chattering, and 1 that was intrinsically bursting. This indicated that the spike count variability we observed in the original data was largely due to burst firing. Once the subsequent spikes in a burst were removed, the spike count variability appeared to be much lower than that expected from a Poisson process with a relative refractory period. It also appeared that the spike count variability was not correlated to the simple cell type (i.e., chattering or intrinsically bursting) classification measure used.

Figure 8, C and G, show the distribution of Fano factors for the events identified in our data. Using the criteria that an event constituted an elevation of the firing rate above its 95th percentile, and that at least 50% of the trials had to contain at least one spike within an event (see Materials and Methods), we identified 442 events in the original data and 415 events in the burst-removed data. The distribution of Fano factors for events was qualitatively similar to the frame-based Fano factors, with the percentage of events with MSC ≥ 1 and FFS ≥ 95 increasing from 5.7% (25 of 442) to 26% (106 of 415) when burst spikes were removed. However, this increase was less dramatic than the increase from 1.18 to 62.8% for the frame-based Fano factors, perhaps due to the smaller number of events.

We also plotted the distribution of Fano factor Z-scores obtained from events in Figure 8, D (original data) and H (burst-removed data). These distributions exhibited the same effect as the frame-based Fano factor Z-scores (i.e., a highly significant change after burst spikes were removed).

Spike time variability

The spike time variability, quantified by the TR-entropy, for the cells in our database is shown in Figure 9A. Similar to Figure 8A, windows with MSC < 1 are plotted in gray, while those with MSC ≥ 1 are plotted in black. Windows in which the entropy of the data was <95% of the surrogates (i.e., TRES ≥ 95) are plotted in red if MSC ≥ 1, and in blue if MSC < 1. To put the entropy values for the data in context, we also computed the entropy in 1000 windows with randomly placed spikes and obtained the range containing 95% of the values (see Materials and Methods). The lower and upper boundaries of the 95% interval for these values are illustrated by the pair of green lines within Figure 9, A and E. In total, when compared with the surrogates, 12.1% (4660 of 38,600) of all windows exhibited TRES ≥ 95 and for windows with MSC ≥ 1, this fraction increased to 49.3% (796 of 1615), and the effect was present in 68.7% (35 of 51) of the cells. Of these 35 cells, 4 were previously classified as chattering, while 1 was intrinsically bursting. The distribution of the entropy z-scores (TREZ) shows large numbers of windows with entropy values that were >3 SDs smaller than the mean of the surrogates (Fig. 9B). The TREZ and the mean spike count for all windows with MSC ≥ 1 were weakly, but significantly, negatively correlated (r = −0.13; p < 0.001). This means that higher firing rates led to lower TREZ values (i.e., lower spike time variability).

As noted in Materials and Methods, we excluded 28 extremely sparse firing cells and 9 cells with refresh artifacts from our database. We note here that these cells did indeed exhibit significantly lower TR-entropy than the 51 cells included (one-sided two-sample KS test, p < 0.0001 for both groups of cells), which justified excluding them from the main analysis.

When bursts of spikes were removed from the data, we found relatively similar results (Fig. 9E–H), with 44.1% (248 of 562) of the windows with MSC ≥ 1 exhibiting TRES ≥ 95. These windows were found in 45.1% (23 of 51) of the cells, of which 5 were chattering cells.

The distribution of interquartile ranges for spike times computed within events are plotted in Figure 9C for the original data, and in Figure 9G for the data with burst spikes removed. The median and quartiles of these distributions were 11.8, 7.06, and 24.2 ms for the original data, and 11.3, 7.31, and 22.6 ms for the data with burst spikes removed (shown in Fig. 9D,H). Both distributions changed relatively little as a result of removing burst spikes, showing that both TR-entropy and event interquartile range captured the effect that burst spikes were no more variable than nonburst spikes.

Our results suggest that spike time variability was much lower than that expected from a Poisson process and that the time of the first spike in a burst exhibited as little variability as the time of the subsequent spikes in a burst. Contrary to what we found for spike count variability, the distribution of TREZ did vary across cell type in windows with MSC ≥ 1, with response windows (n = 24) from cells labeled as intrinsically bursting exhibiting higher spike time variability than response windows (n = 797) from cells labeled as chattering (one-sided two-sample KS test, p < 0.05). This is shown in Figure 10, in which we created box plots of the distribution of TREZ (blue boxes) for response windows (n = 794) from unclassified cells, intrinsically bursting cells, and chattering cells. The decrease in variability for chattering cells was interesting, as previous studies have found intrinsically bursting cells to be mostly located in the supragranular and infragranular layers, while chattering cells are found more frequently in layers 2/3 and 4 (Nowak et al., 2003). Thus, although not conclusive, our results seemed to suggest that cells located in the input layer (layer 4) are less variable than cells located in the processing layers (layer 2/3, and layers 5 and 6), which has also been shown in the study by Kara et al. (2000). Notably, we did not observe this effect when we computed spike time variability using events (i.e., IQRZ; shown in the green boxes in Fig. 10). In this case, events from chattering cells (n = 107) did not exhibit spike time variability significantly different from events from intrinsically bursting cells (n = 8) (one-sided KS test, p = 0.3). However, events from chattering cells did exhibit higher spike time variability than events from unclassified cells (n = 254) (one-sided KS test, p = 0.01).

Figure 10.

Distribution of variability Z-scores for different cell classes. TREZ is shown in blue, while IQRZ is shown in green, and we have excluded windows/event with MSC < 1. Chattering cells exhibited significantly lower TREZ than the intrinsically bursting cells, as well as the unclassified cells, but the effect could not be observed in the IQRZ distributions.

Spike count and spike time relationship

In our data, 43.9% (112 of 255) of the windows with MSC ≥ 1 and FF < 1 also exhibited TRES ≥ 95 (we used FF < 1 instead of FFS ≥ 95 as the former gave us 255 windows, while the latter only gave us 19 windows). These windows occurred in 45% (12 of 51) of the cells. The FF and the TREZ were highly correlated (Pearson's r = 0.29, Spearman's r = 0.33, p < 0.001 for both measures; Fig. 11A), indicating that the windows with low spike count variability were also the windows with low spike time variability. This relationship can be seen even more clearly with the burst-removed data (Pearson's r = 0.87, Spearman's r = 0.95, p < 0.001 for both measures; Fig. 11B). It turns out that it can be shown (see below, Mathematical expression of the Fano factor) that the variability of spike counts in a window is lower when the rate function is non-uniform within the window. This suggests that neuronal encoding schemes based on rates and temporal variations may be more compatible than previously thought.

Figure 11.

Relationship between spike count and spike time variability. The TREZ values for the windows in which FF < 1 and MSC ≥ 1 are plotted against Fano factor for the original data (A) and the burst-removed data (B). In both cases, we found a positive correlation, but the effect was substantially larger when bursts were removed. This indicated that windows with low spike time variability also exhibited low spike count variability. C, D, The equivalent data are shown for events. Contrary to what we found for TREZ, the relationship between IQRZ and Fano did not change substantially after bursts were removed (Spearman's r = 0.27, p = 0.004 for original data; r = 0.19, p = 0.02 for burst-removed data).

Figure 11, C and D, shows the equivalent relationship for the event-based method. In this case, IQRZ and Fano were significantly positively correlated for both original and burst-removed data, but the relationship did not change substantially after burst spikes were removed (Spearman's r = 0.27, p = 0.004 for original data; r = 0.19, p = 0.02 for burst-removed data). In fact, there was a slight trend toward IQRZ being less correlated with Fano after bursts were removed, indicating that the event-based variability measure did not exhibit the same sensitivity to changes in spike count variability as did the TR-entropy-based method.

Relationship to stimulus properties

We performed a simple event-triggered analysis to see whether the windows with low variability in spike count (FF < 1) and spike time (TRES ≥ 95) were simply due to large luminance changes as a result of scene changes. In the windows with MSC ≥ 1 and FF < 1 in the original data, we found that only 5.9% (15 of 255) of the windows occurred within two movie frames of a scene change (a duration of ∼66–80 ms). Similarly, we found that 3.1% (25 of 796) of the windows with MSC ≥ 1 and TRES ≥ 95 occurred within two movie frames of a scene change. In the burst-removed data, we found the corresponding numbers to be 2.0% (9 of 454) and 2.0% (5 of 248). We therefore concluded that the effect of scene changes in our data was minimal. In all cases, only a very small fraction of the windows were preceded by scene changes. Instead, responses with low variability are likely to be related to specific features within the movie.

In an effort to gain a better understanding of the effect of stimulus properties on spike count and spike time variability, we used an approach from the study by Ma et al. (2002) to quantify the stimulus motion present in the movie frames. We found that, across the whole population of cells, the neural responses in movie frames that were preceded by high motion tended to have lower spike time variability than frames preceded by lower motion (i.e., the distribution of motion values for frames with TRES ≥ 95 was higher than the distribution of motion values for frames with TRES < 95, two-sample KS test, p < 0.0001). However, when we examined this relationship on a cell-by-cell basis, we found only a weak correlation between spike time variability (i.e., TREZ) and frame motion (i.e., the median Pearson's R across 16 cells with significant correlations of p < 0.05, was 0.09). One explanation for this lack of sensitivity to frame motion could be that the different cells were tuned to different directions of motions, with cells exhibiting different degrees of variability for preferred and nonpreferred directions. Since our simple motion measure did not include directional information, we were unable to conclude whether direction tuning was present in our cells. Some evidence for this could be seen when we compared the distribution of TREZ conditioned on motion over a range of motion bins for each cell. For 21 of 51 cells, a nonparametric ANOVA test (i.e., Kruskal–Wallis) on these distributions revealed that TREZ was indeed significantly different for some motion bins (p < 0.05), which indicated that motion tuning could be present in these cells.

Comparison with grating responses

We also recorded from an additional 10 cells that were presented with both movie and grating stimuli. The movie stimuli were the same as for the other 88 cells (with total stimulus durations of 4200–4500 s), while the grating responses consisted of drifting, sinusoidal gratings with different combinations of temporal and spatial frequencies (with a total stimulus duration of 6025 s). The ISI distributions for each of the 10 cells are plotted in Figure 12A, with grating responses plotted in red and movie responses in blue. Except for differences in the absolute counts, perhaps due to differences in the total stimulus duration in the two stimulus conditions, the two distributions look remarkably similar.

Figure 12.

Comparison between movie and grating responses. A, ISI distributions for the 10 cells presented with both movie (blue trace) and grating (red trace) stimuli. B–D, The median of the mean spike counts (B), Fano factor (C), and TR-entropy values (D) for each of the 10 cells (each cell is plotted in a different color). The error bars are the 25th and 75th percentile of the mean spike counts and the TREZ values for the movie (vertical axis) and grating (horizontal axis) responses for each cell. If the values from the movie and grating responses were identical, they would lie on the black diagonal line. The grating responses exhibited higher mean spike counts than the movie responses, which is usually thought to be associated with lower response variability. So it was perhaps not surprising that the grating responses exhibited lower Fano factors (i.e., lower spike count variability). However, what was surprising was that, despite their higher spike counts, the grating responses also exhibited higher TR-entropy values (i.e., higher spike time variability) than the movie responses.

We repeated the same variability analysis for these 10 cells, using a window size of 40 ms for both the movie and grating responses. In the movie responses, we found that 19% (49 of 258) of the windows with MSC ≥ 1 exhibited low spike time variability (TRES ≥ 95), while only 5.2% (1182 of 22,592) of the windows in the grating responses exhibited low spike time variability. For spike count variability, these proportions were 2.3% (6 of 258) for movies and 0.89% (201 of 22,592) for gratings. Although the total number of windows in the two cases were quite different, this suggested that spike count and spike time variability were higher in the grating responses than in the movie responses.

To investigate these differences further, we plotted the median MSC values for the movie responses against those for the grating responses for each of the 10 cells in Figure 12B. Similarly, we also plotted the median Fano factor and TR-entropy values for the movie responses against the grating responses in Figure 12, C and D, respectively. In all cases, windows with MSC = 0 were excluded. The error bars in Figure 12B–D indicate the 25th and 75th percentile of the distributions for each cell. We found the distribution of mean spike counts for the grating responses to be significantly higher in each of the 10 cells (one-tailed two-sample Kolmogorov–Smirnov test, p < 0.001). However, the distributions of both Fano factors and TR-entropy values exhibited a more nuanced behavior. Looking at the last two panels in Figure 12C, D, it appears that the Fano factors tended to be higher for movie than for grating responses, while the opposite trend seemed to hold for the TR-entropy values. This is interesting as we found a weak but significant negative correlation between the mean spike count and the TREZ earlier in the movie responses, implying that more vigorous responses elicited less spike time variability. Here, even though the grating responses exhibited significantly higher mean spike counts, they also exhibited higher TR-entropy values (i.e., higher spike time variability). This again suggests that, for each of the cells, the spike time variability in the movie responses were lower than the variability in the grating responses.

As the mean spike counts were in general quite low (the median MSC for all cells were <1 in Fig. 12B), we wanted to exclude the possibility that the difference in spike time variability was only found in windows with weak evoked responses. As a result, we compared the distribution of TREZ values for windows with MSC ≥ 1 in the movie responses to a similar distribution in the grating responses. Of the eight cells that contained response windows with MSC ≥ 1, the distribution of TREZ for the movie responses was significantly lower than that for the grating responses for five of them (one-sided, two-sample Kolmogorov–Smirnov test, p < 0.05). This showed that the spike time variability in response windows with vigorous responses was also lower in the movie responses compared with the grating responses.

Since we observed much higher mean spike counts for grating responses than for movie responses (Fig. 12B), we attempted to rule out the effect of mean spike counts on spike time variability by directly comparing the spike time variability in the grating responses to the movie responses for response windows with the same mean spike count. We divided the mean spike counts into six equal-sized bins ranging from 0 to 3. We plot the distribution of TR-entropy for movie and grating responses, conditioned on these MSC bins, collapsed across the 10 cells in Figure 13. However, in comparing the TR-entropy values between the movie and grating responses, the fact that we only had five repetitions for the grating responses could bias the entropy toward lower values, thus resulting in artificially low entropy values compared with the movie responses. In an effort to account for this negative bias in computing the TR-entropy for the grating responses, we subsampled the movie responses to create 1000 response sets per cell, with each set consisting of five randomly selected repetitions from the full response set. Thus, the two distributions had the same systematic bias toward low entropy values. We then compared each of the 1000 sets of TR-entropy distributions for the movie responses to the distributions of TR-entropy for the grating responses in each MSC bin using a one-sided, two-sample Kolmogorov–Smirnov test. To correct for multiple comparisons, we counted the number of p values <0.05 and compared that with the 95th percentile of a binomial distribution with N = 1000 and p = 0.05. Using this approach, we found the movie responses to have significantly lower TR-entropy than the grating responses for MSC between 0 and 2.5. For MSC > 2.5, there were not enough windows to make a reliable comparison.

Figure 13.

Movie and grating TR-entropy (TRE) distributions conditioned on mean spike count. The title of each of the plots indicates the range of the mean spike count. Each panel shows the distribution of TR-entropy values for the grating responses collapsed across all 10 cells, conditioned on the MSC. For movie responses, we resampled the distribution of TR-entropy values of each cell to create 1000 distributions with the same number of repetitions as the grating responses, and each panel concatenates all of these distributions, again conditioned on the MSC. The red horizontal lines correspond to the medians, the boxes represent the lower and upper quartiles, while the whiskers span 1.5 times the interquartile range. The red crosses represent outliers. The notches indicate the 95% confidence interval of the median. The distributions of TRE values for the movie responses were much tighter around the median than the grating responses, especially for low mean spike counts, which explain the qualitative differences in the boxplots between the two conditions.

The difference in the number of repetitions, and the possible bias this could introduce in the analysis, could also be accounted for by considering TREZ, which was normalized to the surrogates. We thus repeated the above MSC-conditioned analysis on the distribution of TREZ values for movie and grating responses. For the first three MSC bins (i.e., for windows with 0 < MSC ≤ 1.5), the distribution of TREZ values for the movie responses was significantly more negative (i.e., spike time variability was lower) than the distribution for the grating responses (one-sided Kolmogorov–Smirnov test, p < 0.05). We saw the same trend for windows with MSC between 2 and 2.5 (p = 0.035). For 1.5 < MSC ≤ 2.0, we observed the opposite effect [i.e., TREZ for movie was larger than TREZ for grating (p < 0.001)], while for the last bin (i.e., for 2.5 < MSC ≤ 3), there was no difference between the two distributions (p = 0.69).

These two analyses of both TR-entropy and TREZ suggested that movie responses truly did exhibit lower spike time variability than grating responses, even when the firing rates were equal. We repeated this analysis using different numbers of subsampled response sets (n = 250 and 500, as well as matching n to the number of windows in the grating responses), but they did not exhibit any significant differences in the result.

Mathematical expression of the Fano factor

To understand how precise spike times and substimulus rate modulations can influence spike count reliability, we used a very simple model of the response of a neuron. This model was based on the assumption that the presence of a spike in a neuron at a given time, t, depends solely on a firing probability, p(t), which is constant across repetitions. This assumption is the basis of the well known time-varying Poisson process. This assumption is obviously oversimplified since many factors can affect p(t) across trials, such as changes in the depth of the anesthesia (Hartveit and Heggelund, 1994; Kisley and Gerstein, 1999), or ongoing brain activity (Arieli et al., 1996; Azouz and Gray, 1999). This is probably the case in part of our data set and may explain why we found a majority of Fano factors >1.

We added to this assumption the observation that neurons have an absolute refractory period that prevents them from firing more than one spike in a 1 ms interval (in our database of 60 cells, the minimum ISI was 1.1 ms). Thus, we modeled the response of the cell in a window of n ms, as a succession of n independent Bernoulli trials, each having a firing probability p(n). To check for independence between time bins in our data, we used 35 ms windows (corresponding to a movie frame) with FF < 1 and mean spike counts of at least 1 (n = 134). For each window, we computed the product between spike counts, s_i(t), in bins of 1 ms (spike counts were either 0 or 1), and averaged them across repetitions as follows:

with N = 60. We computed the product for t = 1 − 35 ms and t = 1 − 30 ms for a total of 1050 values for each 35 ms window. To obtain the expected values for independent spikes trains with the same firing probability, we used a Monte-Carlo trial shuffling procedure as follows:

where x(i) is shuffled and in general different from i. We repeated the trial-shuffled procedure 100 times to obtain 100 sets of surrogate values. We then determined whether the values obtained from the data were higher than the surrogates at p < 0.01. Across the 134 windows, only 0.34% (479 of a possible 140,700) of the products were significantly higher than the surrogates. That was well below chance level and showed that the time bins were indeed independent.

In each trial, the presence or absence of a spike in bin n was determined by whether a number, drawn randomly from a uniform distribution from 0 to 1, was smaller than p(n). This model, which we will refer to as a time-varying Bernoulli process, is comparable with a Poisson process with a very precise rate modulation and a realistic absolute refractory period, which was found to model neuronal responses correctly in the visual cortex of alert primates (Buracas et al., 1998). To test the fit between the Fano factor computed for our data and for the time-varying Bernoulli process, we selected 35 ms windows in every cell in which the FF was <1, and the mean spike count was >1 spike per window. For each window, we computed the firing probability, p(n), across 60 repetitions in 1 ms steps. We then generated 100 sets of 60 spike trains using p(n) and computed the FF for each set of 60 surrogate spike trains. Of the 134 windows selected, only 6 windows had a FF that deviated from the mean of the surrogate spike trains by >3 SDs, which is slightly higher than would be expected by chance. Apart from these bins, the time-varying Bernoulli process produced FF values that were very similar to those obtained from our data.

We modeled the neuron response to n repetitions of the same stimulus, with a duration of m ms, by n binary time series of m samples. Each sample value, s_ij (sample i in trial j), could either be 1 (spike) or 0 (no spike). The probability for a spike in s_ij is determined solely by the time-varying firing probability p(i), which is identical across trials.

The number of spikes in each time series is then the following:

Since it is the sum of m independent Bernoulli processes, the mean and variance of N_j can easily be computed as follows:

Mean

Variance

It follows that the Fano factor would be as follows:

which expands out to:

FF can also be written as follows:

where M(p) is the mean value of p, and C(p) is a direct measure of the correlation between the probability function p(i), and a flat probability distribution, q(i) = 1/$\sqrt{m}$, as follows:

Intuitively, for an imposed value of the mean firing rate, low Fano factors are reached for high values of the norm as follows:

This defines a simple geometrical problem that provides an intuitive idea of which distributions of p generate the lowest FF. Let us first define P to be an m-dimensional vector of the values p_i, and E to be the m-dimensional vector in which all values are 1. Then FF(p) can also be written as FF(p) = 1 − ‖P‖²/(P · E). That is, for a given mean firing rate (P · E, which defines a hyperplane of dimension m − 1, orthogonal to E), FF(p) is minimal for vectors P with the highest norm, within that hyperplane.

While the Fano factor of a constant or time-varying Poisson process is 1 over any time interval, the FF of a time-varying Bernoulli process, as shown in Equation 11, can be <1. This is also true of a time-varying Poisson process with an absolute refractory period (Berry and Meister, 1998). The FF is thus expressed in terms of the mean value, M(p), of p(n) in a response window, and C(p), a measure of the correlation between p(n), and a flat probability distribution, q(n) = 1/√m, with m being the number of 1 ms Bernoulli trials in the response window. C(p) is therefore a measure of the uniformity of p(n); the less uniform the firing probability, the lower the FF. Thus, for a given mean probability of firing in a window, M(p), FF(p) will be at its maximum when p(n) is uniform [i.e., p(n) = p₀]. In this case, FF(p) = 1 − M(p) = 1 − p₀, which will be close to 1, except when the probability of firing is near the saturation regime (where p₀ is close to 1), where FF(p) will be close to 0. FF(p) will be at its minimum when C(p) is at its minimum, that is when p(n) is zero everywhere except for a single bin, where it is mM(p). Then, FF(p) = 1 − mM(p), which will be 0.

In our simulations with some non-uniform rate functions, we found significant negative Spearman's rank order correlations (p < 0.001) between the analytical Fano factor obtained from Equation 11 and the TRES/TREZ values. This means that the Bernoulli model also shows low spike count variability when the temporal precision is high. We also found significant positive Spearman's rank order correlations (p < 0.001) between the analytical Fano factor and the measured FF, which further validates the use of the Bernoulli model.

Discussion

When presented with natural movies, neuronal responses in striate visual cortex largely exhibited spike count variability that was comparable or higher than the variability exhibited by a Poisson process. Occasionally, we did observe response epochs exhibiting low spike count variability (i.e., FF < 1), but these were not statistically significant in either number or value. Burst firing, however, appeared to make a significant contribution to the increase in spike count variability, suggesting that the number of spikes in a burst was highly variable. We also found that spike time variability was much lower than that exhibited by a Poisson process with a relative refractory period, and that burst firing, however, did not significantly affect spike time variability. This suggested that, even though the number of spikes in a burst was highly variable, the time of the spikes was conserved across repetitions. In addition, we found that response windows exhibiting low spike time variability also showed low spike count variability, a result that could be due to a nonstationary free firing rate.

Our results show that a modified, inhomogeneous Poisson process is unable to reproduce the temporal precision of striate cortical spike trains. A major shortcoming of inhomogeneous Poisson processes in reproducing temporal reliability is that they lack a refractory period. The fact that neurons are unable to generate spikes at arbitrarily high frequencies makes the original spike trains more regular than those produced by a random process without such limitations. We attempted to correct for this difference by explicitly modulating the free firing rate such that a refractory period was used. Despite this addition to the spike generation model, we were not able to match the precision of the experimental data. A pertinent question is whether this failure is the result of our choice of the spiking model. In a study by Barbieri et al. (2001), the authors found that an inverse Gaussian distribution exhibited a better fit to the spiking activity of place cells in rat hippocampus. Notably, this distribution did a much better job modeling short ISIs. We did not test this model on our data, and it is possible that we could have obtained better matches to the precision of the responses in regions of high firing if we had. However, since the inhomogeneous Poisson process is still the most widely used model for neural spike trains, it provides a recognizable standard to compare neural variability.

When comparing our spike time variability measure with that using PSTH events, we found largely similar results. However, the TR-entropy measure can be computed for all response periods, and as such is able to express spike time variability as a continuous temporal function, unlike the event-based measure, which only quantifies temporal variability in a small portion of disjointed windows. The event method is also quite dependent on the ability to detect events accurately. Figure 14 illustrates the difficulty in identifying events in our data, in which single events often encompass multiple “subevents.” We also observed a smaller change in the spike count variability computed from events after removing burst spikes than what we did for variability computed using TR-entropy. This was probably caused by the fact that events were harder to compare between the two conditions, as the removal of spikes sometimes significantly affected the event boundaries, causing some events to disappear, while others resulted in large changes in the interquartile range. In addition, using the TR-entropy measure, we were able to observe laminar differences in variability that were largely consistent with previous studies, while the event-based method gave us rather inconclusive results. A relationship between spike count and spike time variability that was clarified after removing burst spikes was also seen with the TR-entropy measure. This relationship was less clear for the event method, and notably did not improve after burst spikes were removed. Based on these observations, we believe that our TR-entropy measure complements the events-based measure and has the ability to analyze spike time variability for signals in which event identification is problematic.

Figure 14.

Illustration of the difficulties encountered in trying to identity events. Several events were extracted from a single cell. The thick, horizontal red line indicates the 95th percentile of the firing rate, which was the threshold we used to extract events, while the vertical, thin red lines indicate the identified event boundaries. The blue line indicates the firing rate, while the back dots are individual spikes. As can be seen, some of the events clearly included multiple “subevents,” which increased the width of event, and therefore also the interquartile range of the distribution of spikes encompassed by the event.

Our results show that, while a time-varying Poisson process was a poor model for the temporal variability of cortical neurons, the variability in spike counts was more consistent with such a model. A number of studies have uncovered sub-Poissonian spike count variability in diverse cortical areas (DeWeese et al., 2003; Uzzell and Chichilnisky, 2004; Gur and Snodderly, 2006; Benedetti et al., 2009; Maimon and Assad, 2009). However, Poisson-type variability was shown by Beck et al. (2008) to enable the brain to make optimal Bayesian decisions. Our results are consistent with this notion and support the idea that spike count variability tends to increase as one moves to higher cortical areas (Kara et al., 2000) (but see Maimon and Assad, 2009).

Our results also support the notion that bursting increases spike count variability (Kepecs and Lisman, 2003). We observed a larger fraction of windows with low spike count variability after removing the bursts, clearly indicating that the first spike in a burst tended to be more reliable than the subsequent spikes. This increase in variability caused by burst spikes might be offset by the ability of these spikes to increase the probability that a downstream cell will fire a spike in response to input, thereby increasing the overall reliability of the circuit (Gray and McCormick, 1996; Kara and Reid, 2003; Arganda et al., 2007).

Previous studies have found reliable responses in neurons sensitive to visual motion using drifting artificial stimuli that moved with non-uniform rates of drifts and directions (Bair and Koch, 1996; de Ruyter van Steveninck et al., 1997; Buracas et al., 1998). Although neurons in the primary visual cortex are also known to be sensitive to visual motion, they respond to a much broader range of stimulus features than just motion. This was also confirmed in our data, in which we could only find evidence of a weak relationship between frame motion and spike time variability. Although a single-cell analysis revealed that individual cells exhibited different spike time variability in response to frames with different motion content, the motion measure we used was relatively crude, notably lacking information about motion direction. An alternative motion measurement with higher sensitivity will be required to determine how spike time variability of cortical cells is modulated by motion in the stimulus.

We showed that the responses of striate cortical cells to natural scenes exhibited lower spike time variability than responses to simple grating stimuli (Tolhurst et al., 2009), and this finding was consistent after matching mean spike counts between the two types of responses. The fact that spike time variability was lower for movie responses than for grating responses in windows with the same mean spike count, is consistent with the current evidence that natural stimuli are sparsely encoded by cortical cells (Vinje and Gallant, 2000, 2002; Kayser et al., 2004; Yen et al., 2007). Indeed, low temporal variability allows for more information to be encoded in the neural responses, and therefore our results suggest that information may be encoded with fewer spikes in the movie responses compared with the grating responses.

High reliability in neural responses to natural movie stimuli was recently reported by Haider et al. (2010). Interestingly, reliability, as well as sparseness, increased when the nonclassical receptive field was stimulated, compared with when only the classical receptive field received stimulation. This result is consistent with our present study, as the movies we used spanned a large portion of the visual field. Thus, we believe that the fact that we found lower temporal variability in the movie responses compared with the grating responses can at least in part be explained by the same network effects observed by Haider et al. (2010); rich, full-field stimuli engage the cortical network, whereas these effects are much less pronounced when simplistic stimuli are presented. Indeed, previous studies have found that combinations of simple stimuli, such as bars of different orientations, engage the visual cortex differently than if those features were presented in isolation. As a result, spike time variability decreased, which allowed more information about the stimulus to be extracted (Kayser et al., 2004).

Sources of response variability within sensory cortices have been extensively reviewed previously (Fontanini and Katz, 2008; Tiesinga et al., 2008). Striate neurons live within a heavily influential network (Arieli et al., 1996; Tsodyks et al., 1999), in which the overall “state” augments the receptive field properties (Wörgötter et al., 1998; Wörgötter and Eysel, 2000), and are not only influenced by anesthesia type and depth (Villeneuve and Casanova, 2003), but can also be modulated directly through cholinergic inputs (Herculano-Houzel et al., 1999; Rodriguez et al., 2004). These factors likely influence spiking behavior (Azouz and Gray, 1999, 2000) and should be incorporated into more sophisticated models (Kelly et al., 2010) that address response variability. Our finding that cells can exhibit high spike count variability, but low spike time variability, supports the notion that network modulation is a major cause for neuronal variability. A possible explanation for this observation is that spikes may be actively inhibited at the dendrites, thus failing to participate in the activation of downstream cells. The failure of a cell to produce a spike is thus not necessarily a result of variability in the firing mechanism, which could be highly precise, but may rather be caused by the influence of surrounding cells.

An interesting question is whether neuronal variability varies for cells located in different cortical layers. Some evidence for this exist in the literature [e.g., low spike count variability in layer 4 (Gur et al., 1997; Kara et al., 2000; Lee et al., 2010) and high spike count variability in layer 2/3 (Gershon et al., 1998; Lee et al., 2010)], suggesting that cells located in the input layer (layer 4) tend to be less variable than cells in other layers. Although the distribution of spike count variability did not exhibit any difference among different cell types, we did find that chattering cells, which are mainly located in the lower layer 2/3 and layer 4 (Nowak et al., 2003), exhibited less spike time variability than the other cells. This suggests that cells in the input layer tended to be less temporally variable than cells in other layers, consistent with what has been found for spike count variability (Gur and Snodderly, 2006).