Unsupervised learning of temporal regularities in visual cortical populations

Abstract

The brain’s ability to extract temporal information from dynamic stimuli in the environment is essential for everyday behavior. To extract temporal statistical regularities, neural circuits must possess the ability to measure, produce, and anticipate sensory events. Here we report that when neural populations in macaque primary visual cortex are triggered to exhibit a periodic response to a repetitive sequence of optogenetic laser flashes, they learn to accurately reproduce the temporal sequence even when light stimulation is turned off. Despite the fact that individual cells had a poor capacity to extract temporal information, the population of neurons reproduced the periodic sequence in a temporally precise manner. The same neural population could learn different frequencies of external stimulation, and the ability to extract temporal information was found in all cortical layers. These results demonstrate a remarkable ability of sensory cortical populations to extract and reproduce complex temporal structure from unsupervised external stimulation even when stimuli are perceptually irrelevant.

How the brain extracts and reproduces temporal regularities from incoming sensory information is poorly understood. Here, the authors discover the ability of neural populations in the primary visual cortex of behaving monkeys to extract precise temporal structure from unsupervised repetitive external stimulation.

Introduction

When we hear a tune a few times, it will often play by itself in our head. Similarly, once accustomed to a temporal sequence, we will easily notice an anomaly when the sequence is altered¹. Learning such temporal statistical regularities is involved in many behaviors^2,3 and is therefore a general task that the brain needs to address. However, how neurons extract and reproduce temporal regularities from incoming sensory information is unknown. This issue has been previously addressed in the context of learning repeated sequences of events^4–6, characterized by the order and duration of each event. Experimental evidence has revealed that the learning of temporal sequences may occur in the hippocampus, as captured by reactivation, or replay phenomena^7–11. Reactivation of stimulus-induced neuronal activity has also been shown in rat prefrontal cortex^12–14, birdsong circuits during sleep¹⁵, monkey visual¹⁶, motor, and somatosensory cortex during quiescent awake states¹⁷, and human temporal lobe¹⁸. However, replay typically occurs on a highly compressed timescale, and while the order of events is retained, their absolute, or even relative durations, are not. Certainly, retaining duration information is important, for instance in relation to predicting the timing of an event or for planning motor actions.

A large body of knowledge of how the brain recognizes and generates temporal sequences originates from human studies on sequence learning. However, the techniques involved were mainly noninvasive, e.g., psychophysics and imaging^19–21, which provided only limited mechanistic insight. Macaque primary visual cortex (V1) is a well-studied brain region that has long been used to examine experience-dependent plasticity^22–25, but not temporal sequences. Here, we combined in vivo electrophysiology in behaving monkeys with an optogenetic protocol to examine the ability of V1 populations to extract and reproduce temporal information after repeated, direct, stimulation of multiple neurons cued by a visual stimulus^26–28. We used optogenetic light pulses, rather than luminance-based stimuli, to directly activate single cells because visual stimuli would have activated many interacting brain areas involved in visual processing to render our main hypothesis difficult to test. We discovered a remarkable ability of V1 neural populations to extract the precise temporal structure from unsupervised external stimulation.

Results

Optogenetic assay for embedding temporal patterns in V1 populations

We used optogenetic stimulation to examine the ability of V1 neural populations to extract temporal information from successive light pulses. Channelrhodopsin-2 (ChR2), a blue light-sensitive cation channel under the control of an α-CaMKII promoter, was expressed in excitatory cells in area V1 using the VSV-pseudotyped lentivirus (Fig. 1a, see “Methods”). This allowed us to simultaneously place a linear electrode array and fiber optic in V1 to record neural activity at multiple sites. Light pulses emitted from the tip of the optical fiber activated transfected neurons, which exhibited increased activity in response to light stimulation (Supplementary Fig. 1). Light-induced responses spread across multiple contacts on the laminar electrodes and decayed to zero at greater distances (>800 μm) from the channel with the greatest activation (Fig. 1b).

Fig. 1. Experimental design.

a Virus injections (VSV-pseudotyped lentivirus carrying the ChR2-GFP gene) were made in a columnar fashion (1.0 µl per site) at 5 cortical depths in V1. Electrophysiological recordings were performed using 16-channel laminar electrodes tightly coupled to a fiber optic for light delivery. b Distribution of optically-induced activity across electrode contacts for one example session. Inter-contact spacing is 100 µm (most superficial channel is labeled ‘1’). c Behavioral task design. After fixation is achieved, a visual cue was presented in one corner of the screen. Optogenetic stimulation started simultaneously with the cue and was present in 50% of trials. Half of the trials also contained a visual stimulus (oriented gratings presented at 4 different contrasts; these trials were used only in control experiments). All conditions were randomly interleaved. d Cartoon illustrating trial structure and visual cue occurrence. e Average behavioral performance for the contrast detection task (all orientations) in laser (blue) and control (red) conditions (P = 0.658, n = 40 sessions, two-sided Wilcoxon signed rank test). The line shows a fourth order polynomial fit. Error bars represent s.e.m. f False alarm rates in the absence of the visual stimulus are not significantly different between laser and blank trials (P = 0.761, n = 40 sessions, two-sided Wilcoxon signed rank test).

Trials began when monkeys (n = 2) started to fixate on the center of a monitor while a visual cue (1-deg white square) was briefly presented in one corner of the screen for 34 ms (>8 deg outside the receptive fields of the recorded cells). In 50% of trials (laser trials), optogenetic stimulation (10 laser pulses lasting 10 ms each delivered at 35 Hz in 35/40 sessions, ~300 ms total duration), started concomitantly with the appearance of the visual cue (Fig. 1c). In the other 50% of trials there was no laser stimulation (Fig. 1d). Half of all trials contained a visual stimulus (oriented grating presented at 4 different contrasts ranging from 2–10%), and animals were trained to report the presence of the stimulus (all conditions were randomly interleaved). Only trials with no visual stimulus were used for the main analysis, while trials containing a visual stimulus were included only in control analyses. Optogenetic stimulation did not induce phosphenes when presented in isolation (Fig. 1e, P = 0.761, n = 40 sessions, Wilcoxon signed rank test), and did not influence detection performance at any stimulus contrast when presented in conjunction with visual stimuli (Fig. 1e, P = 0.069, P = 0.613, P = 0.410, P = 0.784, n = 40 sessions, Wilcoxon signed rank test; stimulus orientation was not aligned with the cells’ preferred orientation, as in ref. ²⁷). In the ‘blank’ condition, there was no significant difference in false alarm rates between control and laser trials (Fig. 1f, P = 0.761, Wilcoxon signed rank test).

For each session, we analyzed the responses of neurons showing a statistically significant response to laser stimulation (N = 394 cells). We first separated laser and no-laser trials and examined a spiking interval equal to the duration of laser stimulation (300 ms) starting at the onset of laser stimulation (corresponding to the offset of visual cue presentation). Next, we calculated a moving average of the spike trains over trials (10 trial-average with 2 trials lag) for both laser and no-laser trials, followed by z-scoring neurons’ responses and averaging them over all laser responsive neurons in each session. This allowed us to obtain the temporal profile of the population response across trials in each condition (Fig. 2a, b, bottom panels).

Fig. 2. V1 populations spontaneously extract temporal information.

a Neural activity in early trials for one session. (top panels) Spiking activity for multiple laser-responsive neurons (each horizontal band) for first 20 trials with (left) and without (right) laser stimulation. (bottom panels) Average population firing rates for the first 20 trials. Light blue stripes represent the actual duration of the laser pulses (left), or the times at which the laser pulses would appear, aligned with the cue onset (right). Red rectangles represent statistically significant firing rate peaks during the blank condition temporally aligned with the time of the laser pulses (P < 0.05, Wilcoxon signed rank test). (lower panel) Zoom-in of a relevant time period in blank trials equivalent to the duration of the first 4 laser pulses. b Neural activity in late (last 20) trials for the same session. All panels are the same as in (a). The number of significant peaks in the blank late trials increases relative to early trials. c Mean population firing rate (n = 35 sessions) as a function of time since cue onset for laser (blue) and blank (red) late trials and for the shuffled condition (orange; spike times were shuffled in each trial). d Description of laser-blank correlation coefficient calculation for the early and late trials in the same session. Light blue stripes are the same as in (a). Only the time duration of the first 4 laser pulses is shown. e Correlation coefficient between the average firing rates during laser and blank trials (groups of 10 trials) for two sessions. The lines show the second order polynomial fit. f Average correlation coefficient between the mean firing rates during laser and blank trials over all sessions (blue) compared with the same measure but with shuffled spikes (orange) in blank trials (n = 40). Error envelopes represent s.e.m. g Correlation coefficient for early trials (magenta), late trials (blue), and shuffled spike trains late trials (orange). Each circle represents one session, solid rectangles are averages over sessions (n = 40; P = 3.85e-08, P = 7.05e-08, two-sided Wilcoxon signed rank test).

Neural populations spontaneously extract and reproduce temporal information

We examined whether the population of neurons directly stimulated by light can extract the precise temporal information embedded in the sequence of laser pulses. Light stimulation evoked reliable responses in the population of neurons across trials. Indeed, in light stimulation trials, in the absence of visual stimulation, neurons exhibited fluctuating activity tracking the laser pulses; the peaks of elevated activity time-locked to laser pulses are clearly visible in both ‘early’ and ‘late’ trials (Fig. 2a, b, left panels for a sample session). These light-induced On-Off responses were highly reliable: (i) the mean response of the population of cells directly stimulated by light did not vary significantly across trials (P > 0.05, Wilcoxon signed rank test), and (ii) the temporal profile of population activity induced by laser pulses was strongly correlated across trials (correlation coefficient ranged between 0.113-0.889, P = 0.02-7.86e-117, Pearson correlation).

Notably, in the no-laser (blank) trials, despite the absence of external stimulation, the population of neurons gradually ‘picked up’ the stimulation frequency to adapt their firing pattern such as to spontaneously fluctuate at the frequency of light pulses in the late trials. This occurred as trials progressed to indicate an adaptive process whereby neurons learned and reproduced the temporal dynamics of light stimulation (Fig. 2b, right panel). The restructuring of the temporal profile of population activity was prominent for the late (Fig. 2b) but not early trials (Fig. 2a). Indeed, in late trials (typically after trial 30 in most sessions), the neural population exhibited spontaneous On-Off responses, in the absence of laser stimulation, that were synchronized with the population response in laser trials when neurons were repeatedly driven by light (Fig. 2c, n = 35 sessions using the same laser frequency, i.e., 35 Hz). This effect disappeared when we shuffled the timing of spikes in the 300-ms interval following the appearance of the visual cue at the beginning of each no-laser (blank) trial (Fig. 2c).

We quantified the extent to which the population of neurons reproduced the temporal structure embedded in the sequence of light pulses by calculating the Pearson correlation coefficient between the temporal profile of the mean population response in laser and blank trials throughout the session (across blocks of trials, Fig. 2d). The mean correlation coefficient increased as trials progressed (Fig. 2e, f). The correlation coefficient in late trials was significantly higher compared to both early and shuffled trials (Fig. 2g; P = 3.85e-08, P = 7.05e-08, Wilcoxon signed rank test; ‘early’ and ‘late’ were defined based on the dynamics of the correlation coefficient, see Fig. 2f and “Methods”; see also Supplementary Fig. 2 showing that V1 populations can extract temporal information from repetitive sequences of visual stimuli). Using waveform analysis to separate our cell population in putative excitatory and inhibitory neurons, we found that both cell types were able to extract and reproduce temporal information although the effect was more pronounced in excitatory cells (Supplementary Fig. 3; P = 8.09e-07, excitatory; P = 0.0158, inhibitory; Wilcoxon signed rank test compared with shuffled spikes trials). Gradually dropping cells from the neural population significantly reduced the correlation coefficient (Supplementary Fig. 4). The ability of the neural population to extract the temporal structure and fluctuate at the stimulation frequency was further confirmed using the Fast Fourier Transform (FFT) analysis of both laser and blank, no-laser, population responses, which showed significant peaks in power at the stimulated frequency (Supplementary Fig. 5). The increase in power of activity was accompanied by a phase entrainment at those frequencies. Thus, we measured the degree of phase synchronization between the laser and blank spike trains by calculating the phase locking value (PLV) and find an increase in PLV from early to late trials both for session by session (Supplementary Fig. 6; PLV = 0.148 to 0.196, P = 0.021, Wilcoxon signed rank test) and population averaged signals (n = 35 sessions at 35 Hz; PLV = 0.183 to 0.493). Notably, the ability to learn and reproduce the temporal dynamics embedded in the light sequence was not accompanied by changes in the firing rates of neurons between early and late trials (Fig. 3a, P = 0.265, Wilcoxon signed rank test), which indicates that action potentials were temporally realigned with the repeated presentation of periodic laser stimulation across trials while preserving mean spike counts.

Fig. 3. Quantitative analysis of the ability of neural populations to extract temporal information.

a Average firing rates during early (magenta) and late (blue) blank trials. Each circle represents one session, solid rectangles are averages over sessions (n = 40, P = 0.265, two-sided Wilcoxon signed rank test). b (left) Cartoon illustrating the shifting of the blank spike train interval with respect to the laser spike train. Correlation with laser trials was computed by using blank firing rates starting at the same time as the cue (zero time lag) or shifted at various times before/after cue. (right) Mean correlation coefficient at different time lags between laser-blank trials for various groups of trials (moving averages) from early to late trials (n = 40). See also Supplementary Fig. 9. c Late trials correlation coefficient for the population responses (blue, n = 40), compared with the mean correlation coefficient of single cells responses (green, N = 394) and shuffled single cells responses (orange, N = 394). The correlation coefficient for individual cells is significantly different relative to the entire population (P = 4.82e-14, two-sided Wilcoxon rank-sum test) and shuffled condition (P = 0.0005, two-sided Wilcoxon signed rank test). See also Supplementary Fig. 10. d Laser-blank correlation coefficient at different time lags (blank trials shifted at various times before/after cue – see panel b) for the population responses (blue, n = 40) and single cells (average, green, N = 394). e (top) Cartoon illustrating trial structure for sessions with a delayed visual cue with respect to the onset of the laser pulses. (bottom) No increase in the correlation coefficient if the visual cue is delayed by 700–1000 ms with respect to the onset of the laser pulses (n = 28 sessions). Error bars or envelopes in all panels represent s.e.m.

The periodic population response occurring in the late ‘blank’ trials was only associated with the time interval when light pulses are ‘expected’ in the laser trials and did not persist after the sequence of light pulses has ended. Indeed, using the 300-ms spike trains starting immediately at the end of laser-stimulation interval did not yield a trial-by-trial increase in the correlation coefficient between light-evoked and blank population responses (Supplementary Fig. 7). This demonstrates that the temporal effects revealed here do not constitute reverberatory activity post stimulation, as reported in a previous imaging study in rat visual cortex⁵. Finally, there was a significant drop in the ability of neural populations to extract temporal information after laser stimulation was extinguished (Supplementary Fig. 8).

Population activity encodes precise temporal information

The synchronization between the population responses in the blank and laser trials was temporally precise. Shifting the ‘blank’ spike train by >4 ms relative to the ‘laser’ spike train (Fig. 3b, left panel) temporally desynchronized the two population responses to cause a sharp decline in correlation coefficient, which was conspicuous for the late trials (Fig. 3b, right panel, see also Supplementary Fig. 9). While single neurons were associated with a small, but statistically significant, laser-blank correlation coefficient, and hence had a poor ability to exhibit temporal fluctuations (R_neurons = 0.011, P = 0.0005, Wilcoxon signed rank test, compared with shuffled condition), when pooled together, the neural population was able to reliably replicate the periodic neural response²⁹ at the frequency used in the laser trials (Fig. 3c, correlation coefficients: R_population = 0.072, P = 4.82e-14, Wilcoxon rank-sum test; see also Supplementary Fig. 10). Thus, the population averaged activity was more reliable than the combined effect of individual neurons. Moreover, using a subset of individual neurons that had the strongest response produced a smaller scale effect than the population averaged activity. Furthermore, examining the 300-ms spiking responses of both individual neurons and neural population at various shifting delays with respect to the visual cue (Fig. 3b) reveals a decline in laser-blank correlation coefficients for the late trials (Fig. 3d).

The extraction of temporal information by V1 populations is associated with the onset of the visual cue, present in both the laser and blank trials (the visual cue was presented simultaneously with light onset in laser trials). When we randomly delayed the visual cue by 700–1000 ms in each trial with respect to the onset of laser pulses (Fig. 3e, n = 28 separate sessions, N = 238 laser-responsive cells), we found a lack of trial correlation between the laser and blank population responses (P = 0.585, Wilcoxon signed rank test, comparing correlation coefficient between early and late trials). Thus, altering the precise timing between the visual cue and light stimulation onset rendered neural populations unable to extract temporal information, thus abolishing the temporal effects revealed here. We further asked whether the fixation point (FP) itself, displayed prior to the appearance of the visual cue (Fig. 1c and “Methods”), might have contributed to the temporally precise population response seen in the blank condition. However, given that the actual time when fixation was achieved in a single trial was highly irregular, the FP-cue time delay was randomly distributed across trials (in the 450–1000 ms range; Supplementary Fig. 11). Thus, given the randomness of FP occurrence, this event is unlikely to explain the precise temporal effects described here.

Temporal effects found over a range of light frequencies

The ability of neural populations to extract and reproduce temporal events was preserved over a wide range of laser stimulation frequencies (Fig. 4a; 35 Hz, n = 35 sessions; 10 Hz, n = 4; 15 Hz, n = 1; 20 Hz, n = 4; 50 Hz, n = 2), and the laser-blank correlation coefficients were statistically indistinguishable (P = 0.104, Kruskal–Wallis test, df = 4). Furthermore, when different frequencies were used across consecutive sessions performed on the same day (Fig. 4b upper panel, n = 5), the same neuronal population was able to adapt its firing pattern such as to successively extract temporal information across sessions and light frequencies. Thus, there was no significant difference between the mean laser-blank correlation coefficients of population activity in the late trials associated with the 2 frequencies (Fig. 4b, P = 1, Wilcoxon signed rank test; both were statistically significant when compared with the shuffled spike trains, P = 6.66e-04, Wilcoxon rank-sum test). That is, the neural population was equally proficient in extracting temporal information regardless of the frequency of laser stimulation (see also Supplementary Fig. 12).

Fig. 4. Characterization of main temporal effects.

a Correlation coefficient for sessions with different laser frequencies. Each circle represents one session, thick horizontal lines represent the means across sessions. Each color show sessions with the same laser frequency, mean of all sessions, and mean for the shuffled condition. b (top) Schematic illustrating the structure of successive sessions with different laser frequencies. (bottom) No statistically significant difference between the mean correlation coefficients (late trials) for successive sessions with different laser frequencies (n = 5, P = 1, two-sided Wilcoxon signed rank test). Both groups of sessions (blue – first, red – second) exhibited higher correlation coefficients compared with shuffled condition (orange, n = 10, P = 6.66e-04 for both comparisons, two-sided Wilcoxon rank-sum test). See also Supplementary Fig. 12. c Example current source density (CSD) showing the position of the laminar probe with respect to infragranular (IG), granular (G) and supragranular (SG) layers. d The late-early increase of the laser-blank correlation coefficient is statistically significant in different cortical layers (blue, SG: n = 30, P = 0.030; green, G: n = 34, P = 0.0023; red, IG: n = 27, P = 0.0015; two-sided Wilcoxon signed rank test, comparing correlation coefficient between early and late trials). e Mean population response for the laser-unresponsive neurons as a function of time in the trial (only late trials) for laser (blue) and blank (red) conditions. f Mean laser-blank correlation coefficient for laser and blank trials across trials for laser-unresponsive neurons is not significantly different from 0 throughout the session (P = 0.870, n = 35, two-sided Wilcoxon signed rank test comparing correlation coefficient between early and late trials). g Influence of visual stimuli on laser-blank correlation coefficient for late trials. Blue: mean laser-blank correlation coefficient in the blank (no stim) condition; magenta: mean laser-blank correlation coefficient when visual stimuli are presented (with stim condition). Each circle represents one session, thick horizontal lines represent the means across sessions (n = 40, P = 2.83e-07, two-sided Wilcoxon signed rank test). See also Supplementary Fig. 13. Error bars or envelopes in all panels represent s.e.m.

Importantly, we asked whether the ability of neural populations to extract temporal information depends on cortical layer. Therefore, we computed the current source density estimate (CSD^30,31,) to assign each recording contact to a cortical layer (supragranular, granular, or infragranular; Fig. 4c), and then analyzed the laminar distribution of the strength of temporal effects described here. However, we found that the increase in laser-blank correlation coefficient from early to late trials was present in all cortical layers with no significant differences depending on laminar location (Fig. 4d, P = 0.758, Kruskal–Wallis test, df = 2).

It could be argued that the observed temporal cyclic pattern in the population response in blank trials may represent endogenous fluctuations in neural activity rather than being induced by the repeated light stimulation in the laser trials. We thus repeated our temporal analysis exclusively using the laser-unresponsive cells in the population. However, we found that these cells are unable to extract and reproduce temporal information (n = 35 sessions, N = 215 neurons; Fig. 4e, f; correlation coefficient in late trials was not significantly different from early trials, P = 0.870, Wilcoxon signed rank test). Although there were fewer light-unresponsive neurons compared with light-responsive ones, the average firing rate during the blank trials was not significantly different between the two cell populations (P = 0.815, Wilcoxon rank-sum test), Furthermore, for a smaller number of laser responsive neurons, equivalent to the number of unresponsive neurons, we found that neural population was still able to extract the temporal information (Supplementary Fig. 4). We further reasoned that the laser-responsive cell population will lose the ability to extract temporal information from the repetitive sequence of light stimulation when the temporal information conveyed by the local cortical inputs is perturbed. We thus examined the 50% of trials containing a low-contrast visual stimulus (see “Methods”, Supplementary Figs. 13a and 13b), and tested whether the presence of a visual stimulus is sufficient to alter the spike timing structure of the neural population in laser trials, and hence diminish the temporal effects examined here. Confirming this prediction, presenting visual stimuli significantly reduced the laser-blank correlation coefficient (Fig. 4g, P = 2.83e-07, compared with non-visual stimulation condition, Wilcoxon signed rank test) to values that were not statistically significant from 0 (P = 0.638, Wilcoxon signed rank test). Furthermore, shifting the no-laser spike trains with respect to the offset of the visual cue (see Fig. 3b) rendered the correlation coefficient indistinguishable between early and late trials (P = 0.443, Wilcoxon signed rank test, Supplementary Fig. 13c).

Strongly coupled neurons extract more precise temporal information

We hypothesized that the cells that are more coupled to each other will exhibit a stronger ability to extract precise temporal information from repetitive exposure to light pulses. To test this hypothesis, we first performed a cross-correlation analysis for the pairs of light-responsive cells in each session. This allowed us to divide the cells into highly and weakly coupled neurons based on whether the number of significant cross-correlogram (CCG) peaks for a given cell was greater or smaller than the median number of significant CCG peaks between any individual neuron and the rest of the cells in the population (Fig. 5a, based on the laser trials CCGs). For instance, neurons with no statistically significant peaks in CCG with any of the cells in the population were labeled as ‘weakly coupled’ neurons, whereas the cells with for which all of the associated CCG peaks were statistically significant were labeled as ‘strongly coupled’ cells.

Fig. 5. Neuronal coupling impacts the ability to extract temporal information.

a Schematic of neuron classification into strongly coupled and weakly coupled cell classes. Top represents a strongly coupled neuron (dark blue triangle) characterized by a high number of significant CCG peaks with other neurons. Bottom represents a weakly coupled neuron (light blue triangle) characterized by a low number of significant CCG peaks with other neurons in the population. Examples of CCGs for each class of cells are shown on the right. b Laser-blank correlation coefficient as a function of trials for different classes of cells based on their coupling status. Highly coupled neurons (dark blue, n = 37 sessions) exhibited a more pronounced increase in correlation coefficient compared with weakly coupled neurons (light blue, n = 34 sessions; ** P = 5.91e-04, two-sided Wilcoxon rank-sum test). The thin lines show a second order polynomial fit. Error envelopes represent s.e.m.

After dividing the population of cells into weakly and strongly coupled neurons, we repeated the correlation analysis between the laser-blank population responses, as previously described (see “Methods”). Confirming our hypothesis, across sessions, the subpopulation of neurons that were more strongly coupled was more likely to exhibit a greater increase in the laser-blank correlation coefficient in the late trials compared to the subpopulation of weakly coupled cells (Fig. 5b; P = 5.91e-04, Wilcoxon rank-sum test). This suggests that the precise timing between incoming sensory events may be preferentially extracted by a specialized cell ensemble³² of tightly coupled cells. Notably, the coupling of neurons was independent of their firing rates as there was no difference in firing rates between strongly and weakly coupled cells (P = 0.107, Wilcoxon rank-sum test). Furthermore, we found a lack of significant correlation between the entrainment index of each cell (laser-blank correlation coefficient of each cell relative to the average population value) and its average CCG peak height (Supplementary Fig. 14, P = 0.760, Pearson correlation). We have also measured the time lag of the CCG peaks and found that 41% of the significant CCG had peak lags at τ = 0. Thus, the neuron pairs used in our sample belonged to both synchronous and asynchronous groups, corresponding to pairs of neurons with input from a common source and pairs that have direct synaptic connection.

Control analyses

Since monkeys were engaged in a task in which they signaled the presence of visual stimuli in each trial, we further examined whether behavioral performance influenced the ability of the neural population to extract temporal information from the sequence of laser pulses. First, we found no dependence between the laser-blank correlation coefficient in late trials and the number of correct trials across sessions (laser, R = 0.163, P = 0.315; blank, R = 0.091, P = 0.578, Pearson correlation). In addition, there was no significant difference in the correlation coefficients when using either all trials or correct trials only (Supplementary Fig. 15a, P = 0.345, Wilcoxon rank-sum test). Second, the change in laser-blank correlation coefficient from early to late trials was unrelated to task performance during the session as there was no difference in the percentage of correct responses between early and late trials in either laser or blank condition (P = 0.321, P = 0.967, Wilcoxon signed rank test, Supplementary Fig. 15b). Third, we ruled out potential oculomotor confounds. There was no dependence between the correlation coefficient in late trials and the number of microsaccades, fixation breaks, or bar releases across sessions (P = 0.764, P = 0.401, P = 0.288, Wilcoxon signed rank test). Additionally, we calculated the correlation coefficient between the spike counts and pupil size and that between spike counts and eye velocity across trials, but found no statistically significant effects (Supplementary Fig. 16). Furthermore, there was no significant difference in the number of microsaccades, eye velocity, and pupil size between early and late trials (P = 0.606, P = 0,121, P = 0.504, Wilcoxon rank-sum test). Additional control experiments ensured that the effects reported here are not due to phosphene induction or laser-induced local heating (see “Methods” and ref. ²⁷).

Discussion

It has long been hypothesized that neural populations in specific brain areas may be endowed with the ability to extract and reproduce complex temporal sequences from incoming stimuli. However, previous experimental studies have either focused on the neural representation of a single temporal interval^33–37 or on the representation of the order, but not duration, of multiple events in a sequence^7–18. Because of technical limitations, whether neurons can extract temporal information and reproduce multiple time intervals has been difficult to test at single-cell resolution in awake animals. We combined optogenetic stimulation and multiple-electrode recordings to reveal that neural populations in visual cortex extract and reproduce complex temporal information in an unsupervised manner after receiving repeated, direct, light stimulation. Notably, we found that populations of neurons exhibit remarkable temporal ability despite the fact that individual cells are not temporally precise.

The fact that neurons in early visual cortex, which are highly specialized to encoding visual features^23,38, gradually adapt their spiking pattern such as to reproduce repetitive temporal sequences in the absence of external stimulation, is surprising. Although the precise neural mechanisms underlying to the ability to extract complex temporal information have not yet been explored, our findings argue for a higher functional complexity of local intracortical connections than previously revealed, which must undergo plastic changes over the time scale of minutes upon multiple repetitions of light pulses in order to extract temporal information. Indeed, this is supported by our analysis revealing that correlations between laser-responsive neurons increase over trials as neurons acquire timing-like behavior (Supplementary Fig. 17; P = 1.35e-06, n = 192 significant pairs, Wilcoxon signed rank test, blank trials).

Previous work has shown that individual neurons in visual cortex are able to generate spontaneous responses when one stimulus from a repetitive stimulus sequence is omitted^35,39. Indeed, a subset of V1 neurons exhibited responses that were predictive of the upcoming stimulus, and the omission of an expected stimulus triggered strong neuronal responses. Although these results have been initially formulated in the context of predictive coding proposing that individual neurons encode sensory prediction errors, the spiking responses to expected and omitted stimuli were nearly identical⁴⁰. However, the major difference between our results and the previous research on predictive coding in visual cortex lies in the temporal specificity and scale of the effects. Whereas previous work has shown that the absence of one stimulus in a sequence can be compensated by the underlying circuitry, our results demonstrate remarkable timing abilities of a neural population to self-generate repetitive responses, triggered by an external cue, temporally similar to the original stimulus sequence. These temporally precise responses occurred for the entire duration of the stimulus sequence, not only for one omitted stimulus in the sequence.

The ability of visual cortical populations to extract temporal information from repetitive sequences was examined using optogenetic light stimulation rather than visual stimuli. However, this is advantageous as it allows us to identify putative mechanisms, such as the fact that neural populations in V1 gradually adapt their responses to extract temporal regularities from external stimulation without influences from the rest of the brain, including the effects transmitted by top-down higher cortical and thalamic structures. Indeed, we have already demonstrated the ‘locality’ of optogenetic stimulation during wakefulness⁴¹, i.e., the fact that optogenetically stimulating V1 neurons at similar light intensity as that used here activates local neural populations without propagating the electrical signals to neighboring neural populations in the same cortical area²⁷. This rules out top-down expectation effects that could have involved multiple cortical areas as a possible mechanism for the temporal effects shown here. Indeed, using visual stimuli instead of optogenetic stimulation (e.g., Supplementary Fig. 2) would elicit a perceptual response to the stimulus sequence, absent when V1 neurons are stimulated optogenetically, activating large cortical pathways, including feedforward and feedback projections, and involving stimulus expectation. Thus, it is remarkable that such an important property, the extraction of temporal regularities from external stimuli, is shown at such an early stage of visual processing. Furthermore, given that the type and specificity of neuronal connections in visual cortex are similar to those in other sensory cortical areas^42–46, our results suggest that temporal effects similar to those reported here may be demonstrated in other sensory areas to possible constitute a general property of neocortical circuits when they are directly stimulated with repetitive stimulus sequences. This could provide clues to guide future research focused on how our brains are able to extract and reproduce temporal regularities from a wide range of sensory inputs.

Methods

Ethics statement

All experiments were performed in accordance with protocols approved by the Animal Welfare Committee (AWC) and the Institutional Animal Care and Use Committee (IACUC) for the University of Texas Health Science Center at Houston (UTHealth).

Animals and surgery

Two male rhesus monkeys (Macaca mulatta; M1, 8-years old, 15 kg; M2, 12 years old, 13 kg) were used in the experiments. Monkeys were previously trained in fixation/detection tasks and were surgically implanted with a titanium headpost device and a 19-mm recording chamber (Crist Instruments) in area V1.

Viral vector injections

ChR2 was expressed specifically in V1 excitatory cells using the same lentiviral vector as used in the past in monkeys by us and other investigators^26,27. High titer (>10⁹ IU ml⁻¹) purified VSV-pseudotyped lentivirus carrying the ChR2-GFP gene was obtained from the University of North Carolina Gene Therapy Center Vector Core. The virus was injected in a columnar fashion (Fig. 1a) at multiple neighboring sites (“clusters”) within V1 through a very fine needle (29 gauge) connected via mineral oil filled tubing to a Hamilton syringe mounted on a perfusion pump (KD Scientific). The needle was advanced by a precision, computer controlled micro-manipulator (NAN instruments) through a custom grid mounted on top of the standard recording chamber to the lowest depth at which unit activity was found in preliminary experiments. After a 15 min of waiting (to allow for stabilization), 1 μl of virus suspension was delivered over a 10 min period. The needle was then retracted at a slow rate (0.1 mm min⁻¹) in 200–300 μm steps at 3–4 additional depths where 1 μl of virus suspension was delivered respectively. Five-minute wait periods were interleaved before and after each virus delivery and retraction steps. Each 1 μl has been shown to diffuse into about 1 mm³ of surrounding tissue²⁷. Multiple injections were performed in each V1 chamber (8 for M1, 11 for M2) closely grouped together and forming a rectangular pattern across the cortical surface.

Optogenetic stimulation and electrophysiology

A time period of 4–6 weeks was allowed for virus expression. Optical stimulation was provided using a 100 mW, TTL controlled, DPSS blue (473 nm) laser (RGBLase) coupled to a 200 μm optical fiber. The end of the fiber was inserted into a 356 μm beveled stainless-steel cannula and mounted on the micro-manipulator. The light intensity at the tip of the fiber was within the range of 6.7–14.5 mW per mm² (integrating sphere sensor, S124C Thor Labs). Light intensity was held constant across experiments. Prior to each experiment, both the optical fiber and the electrodes were advanced into the brain simultaneously to the expected depth. Using our custom grid, the spacing between the devices at target depth was ~0–300 μm, with most sessions having a spacing of ~ 100–200 μm²⁷. Optical stimulation of the neurons was achieved by delivering 5–15 bursts of 5–15 ms light pulses at 10–50 Hz. The laser output was regulated via TTL pulses driven by a waveform generator (Model 3220 A, Agilent Technologies), controlled by the Experiment Control Module (ECM, FHC Inc.). The laminar electrodes (U-probe, Plexon Inc) consisted of a linear array of 16 equally spaced contacts (100 μm inter-contact spacing). Each electrode contact was 25 μm in diameter and platinum iridium coated. The impedance at each contact was 0.3–1.0 MΩ. Real-time extracellular neuronal signals (simultaneous 40 kHz A/D conversion on each channel) were analyzed using the Multichannel Acquisition Processor system (MAP system, 64 channel, Plexon Inc). Light induced artifacts were sometimes present in the local field potentials, but not in the high-pass filtered spike data. This was confirmed with periodic recordings in saline. To ensure that the effects reported here were solely due to optical stimulation (rather than laser-induced local heating, or some other unexpected variable), additional control sessions in untransfected cortex were performed utilizing the same procedures described above, except that the optic fiber was positioned 1–3 mm from the nearest injection site. As reported in our previous studies, no light evoked neural activity, or behavioral modulation was observed²⁷.

Behavioral task

Before each recording session, monkeys performed fixation tasks to map receptive field locations, identify light-responsive units, and identify cortical layers. Monkeys sat in a conventional primate chair, head-restrained, while stimuli generated using Matlab with Psychophysics Toolbox⁴⁷ were presented binocularly on a dark background computer screen 90 cm away. Eye position was continuously monitored using an infrared, mirror-based eye tracking system operating at 1 KHz (EyeLink II, SR Research Ltd.) and synchronized with neuronal data using the ECM programmable device. Monkeys maintained fixation on a 0.2-deg fixation point in order to obtain juice reward. Receptive field locations were mapped using 0.33-deg reverse correlation stimuli (full contrast, sinusoidal gratings, four equally spaced orientations) presented at random, nearby, screen locations. To identify light-responsive units, monkeys performed a fixation task while laser stimulation pulses (24 Hz, 10 cycles, 10 ms width) were delivered to the transfected cortical area.

In the main experiment, soon after animals achieved fixation (450 – 1000 ms after the appearance of the fixation point), a visual cue (small 1-deg white rectangle) was flashed for 34 ms in the top left corner of the screen. Optogenetic stimulation was delivered through a fiber optic in 50% of trials (laser trials) and light delivery started simultaneously with the onset of the visual cue (Fig. 1c, d). Light stimulation consisted of 10 laser pulses lasting 10 ms each, delivered at 35 Hz in most sessions (n = 35). Different laser frequencies were also tested in other sessions (10 Hz, n = 4; 15 Hz, n = 1; 20 Hz, n = 4; 50 Hz, n = 2; Fig. 4a, b). The other 50% of trials (no laser trials), had no optogenetic stimulation (Fig. 1d). Half of all trials contained a visual stimulus (oriented sinusoidal gratings presented at 4 different contrasts; peak luminance 0.107 to 0.280 cd m⁻², while the no stimulus condition had a luminance of 0.08 cd m⁻²; Tektronix, J17; with a diameter of 2–3 deg that covered the multiple receptive fields of the cells recorded and displayed for 800–1300 ms at 2–4 deg eccentricity). Animals reported the stimulus presence by releasing a lever in order to obtain juice reward. Trials with visual stimulus were used as a control condition; only the trials with no visual stimulus were used in the main analysis. All conditions were randomly interleaved. A separate set of control experiments (n = 28 sessions) was performed to assess the role of the visual cue as the trigger of the temporal information extraction, i.e., the visual cue was delayed with respect to the onset of laser stimulation (which lasted 300 ms on average) by 700–1000 ms (constant within the session, see Fig. 3e). All other parameters were identical to those in the main experiment.

In a separate set of experiments (n = 9), we employed a similar protocol as described above, except that visual stimuli were used instead of optogenetic light pulses. The stimulus sequence consisted of successive presentations (n = 5) of identical full-contrast sinusoidal gratings (2.2 cpd) presented at neurons’ preferred orientation. The gratings had a diameter of 2–3 deg and covered the multiple receptive fields of the cells recorded, and each stimulus was displayed for 100 ms at 2–4 deg eccentricity. Two stimulation frequencies were used (constant within the session), 4 Hz and 6 Hz (5 and 4 sessions, respectively) with a total stimulation duration of 1200 and 900 ms, respectively. Stimuli were present in 50% of trials (randomly interleaved) and no behavioral response (fixation only) was required at the end of the trial. As in our main experiment described above, a visual cue (small 1-deg white rectangle) was flashed for 34 ms in the top left corner of the screen simultaneously with the onset of the first grating in stimulus trials or at time the grating was expected to appear (1000–1200 ms after the fixation point start) in blank trials.

Phosphene controls

Non-cell-type-specific optogenetic stimulation has been linked to phosphene induction⁴⁸, artificial percepts typically generated by electrical stimulation⁴⁹. We examined different measures reflective of phosphene induction and we found no significant difference between laser and blank trials in false alarm rates (when animals incorrectly reported the presence of a stimulus; Fig. 1f), number of aborted trials in each session due to fixation breaks, number of premature response bar releases and microsaccade count²⁷. We concluded that optogenetic stimulation under our experimental conditions is unlikely to induce phosphenes.

Cell classification

Spike sorting was performed offline using waveform-based principal component analysis software (Offline sorter, Plexon Inc). Subsequent analysis was performed using Matlab (Mathworks Inc). We identified cells based on their functional responses to the light stimulation and to the visual stimuli. Light sensitive cells were identified by comparing the firing rates on trials during laser-on period (first 300 ms) with the equivalent period in blank trials (in the absence of any visual stimulus; statistical criterion was P < 0.05, Wilcoxon rank sum test). Low responsive cells, i.e., average firing rate < 1 spk s⁻¹, as well as unstable neurons were removed from the analysis. Light sensitive cells which did not have a sustained response during the non-stimulation trials were also eliminated as they did not allow us to perform the required data analysis. For the main experiment, from a total of 652 cells, we selected 394 laser responsive cells from two monkeys across 40 recording sessions (M1: 21 sessions; M2: 19 sessions). For each session, the same neurons were used in all the analyses associated with the laser and no laser trials. There were also 215 neurons that were unresponsive to the laser pulses but responded to the visual stimuli presented on screen. In the control experiments where the visual cue was delayed with respect to the laser onset (n = 28 sessions) we found N = 238 laser responsive cells. The 9 additional sessions used in the laser frequency analysis had a number of 109 laser responsive neurons. The visual stimulation sessions (n = 9) used a set of 213 stimulus responsive neurons.

Laser-blank correlation coefficient

We separated laser and no-laser trials and selected a spiking interval equal to the duration of laser stimulation (300 ms) starting at the offset of the visual cue presentation. We then computed a moving average of the spike trains over trials (10 trial-average with 2 trials lag) for both laser and no-laser trials. We subsequently z-scored these responses and averaged them over all the responsive neurons in each session. We thus obtained a population response (PSTH) as a function of time across the session for either laser or non-laser trials. In order to assess the significance of the results we repeated this procedure for no-laser trials after shuffling the timing of spikes within the 300 ms interval (shuffled blank). In order to compute the temporal relationship between laser-evoked population responses and the population response in the blank, no-laser trials we calculated the Pearson correlation coefficient between the population responses in the laser and blank (no-laser) trials. The Pearson correlation R(x,y) of two time series x(n), y(n), n = 1, 2,…, N is given by:

$$R\left(x,y\right)=\frac{{\sum }_{n=1}^N\left[x\left(n\right)-\overline{x}\right]\left[y\left(n\right)-ȳ\right]}{{\sigma }_x{\sigma }_y}$$ 1

where $\overline{x}$ and $ȳ$ are the means of x and y, respectively, and σ_x and σ_y are the standard deviations of x and y, respectively⁵⁰. MATLAB function corrcoef was used to compute the Pearson correlation coefficient. The correlation coefficient was calculated for various groups of trials (moving averages) from early to late trials (with a 2-trial lag). We then computed the averages of the first and last 6 groups of trials (see boxes in Fig. 2f) and labeled them as ‘early’ and ‘late’ trials (Fig. 2g). The late and early intervals were selected based on the overall evolution of the correlation coefficient in time (see Fig. 2f) by performing a one-way analysis of variance (Kruskal–Wallis test) and selecting the period at which the correlation coefficient became statistically different as well as a corresponding stable baseline (<10% change) period after the trial start. For our main analysis, we used only the trials without a visual stimulus (laser or blank trials) and the light-responsive cells (see above). The same methods were employed in control tests, using either the visual stimulation trials (Fig. 4g) or the laser unresponsive cells (Fig. 4e, f). To examine the effects exhibited by single neurons (Fig. 3c, d), we repeated the above protocol, but skipped the average over cells after the z-score, and computed the correlation coefficient for each single cell instead of the entire population. In the control test used to study the temporal persistence of the effects (Supplementary Fig. 7), the laser-evoked population response during the stimulation period were compared with the population response in blank trials starting at a later time, after cue onset. The relationship with the visual cue was examined by shifting the blank spike train interval with respect to the laser spike train in either direction in 2 ms increments (Fig. 3b, left panel). The correlation with laser trials was then computed by using blank firing rates starting at the same time as the cue (zero-time lag) or at various times before/after cue.

Cell dropping procedure

For each session, we sequentially removed neurons from the analysis and then recomputed the correlation coefficient using all combinations of the remaining neurons. We started by removing one neuron and then recomputed the correlation coefficient for the remaining cells. We repeated this procedure for all combinations of neurons and then averaged the results across cells. We next removed two neurons from the total pool and then used the same procedure. These steps were repeated until we removed all but one neuron from the total population. The final results were averaged across sessions and plotted based on the number of neurons removed from the population (Supplementary Fig. 4).

Current Source Density Analysis

The laminar position of the electrode contacts was confirmed in a separate passive fixation task using a reverse correlation stimulus consisting of full contrast circular sinusoidal oriented gratings spanning 0–180° (eight gratings, 22.5° orientation step) presented at the receptive field location. Local field potentials (LFP) responses were processed to obtain evoked potential (ERP) traces for each contact. The current source density (CSD) was computed by using the second spatial derivative of the LFP timeseries across the equally spaced laminar contacts. The laminar CSD profile was then inspected to confirm a primary sink in the granular layer and assess the granular layer reference at the contact closest to the sink centroid (Fig. 4c). All contacts above and below the reference were then assigned to one of three possible layers: supragranular, granular, and infragranular.

Cross-correlograms

CCGs were computed by sliding the spike trains (300 ms) of each cell pair and counting coincident spikes within 1 ms time bins for each pair of neurons, normalized by the geometric mean spike rate, and corrected for stimulus induced correlations by subtracting an all-way shuffle predictor^51,52. CCGs were smoothed with a 2 ms Gaussian kernel and fitted to a Gaussian function. The significance of the CCG peaks was assessed for peaks (−50 to +50 ms interval) that exceeded three standard deviations of the noise level (tail; time lag from −400 to −250 ms and from 250 to 400 ms). Mean CCGs were obtained by averaging CCGs for all the pairs. Two measures of the CCGs were computed, one using only early and late trials (using blank trials, Supplementary Fig. 17) and one for the whole session (laser trials, used to assess the coupling of each cell; see Fig. 5b).

Eye movement analysis

Eye positions for a 350 ms interval, starting 50 ms before laser onset, were convolved with a low-pass linear finite impulse response (FIR) filter with a 50 Hz cutoff frequency. Microsaccades were identified as deflections of the eye position during which eye velocity exceeded 10 deg s⁻¹ for at least 10 consecutive ms, and eye acceleration exceeded 1000 deg s⁻¹ during a 40-ms interval centered at the maximum of the eye velocity. Successive microsaccades separated by <30 ms were considered a single eye movement. Eye velocity during microsaccades was calculated as the vector sum of x and y derivates independently for laser and blank trials. Pupil area was recorded by the eye tracker system (EyeLink II, SR Research Ltd.), and was represented in arbitrary units separately for the laser and blank trials.

Statistical analysis

Statistical significance was assessed using two-tailed nonparametric tests. Specifically, for comparing the laser and no laser conditions, early and late trials, Wilcoxon signed rank test was used. We compared either the variables computed for each session (e.g., correlation coefficient) or the values for the whole neuronal populations recorded across sessions (e.g., CCG height of each pair). In case of unmatched observations with unequal sample size, Wilcoxon rank-sum test was used. We chose these tests rather than parametric tests, such as the t-test, for their greater statistical power (lower type I and type II errors) when data are not normally distributed. For populations involving two or more independent variables, a one-way analysis of variance Kruskal–Wallis test was used.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Author contributions

Conceptualization: V.D.; data analysis: S.P. and A.A.; resources: V.D.; data curation: S.P. and A.A.; writing and editing: S.P., A.A. and V.D.; visualization: S.P.; supervision: V.D.; project management: V.D.; funding acquisition: V.D.

Peer review

Peer review information

Nature Communications thanks Tatiana Engel, Ji Xia and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Data availability

Source data is available at (https://zenodo.org/records/14729094). Additional raw data will be available from the corresponding author upon request. Source data are provided with this paper.

Code availability

Data analysis was performed using MATLAB (Mathworks), using readily available functions. The code upon which this study was based is available from the corresponding author upon request.

Competing interests

All the authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41467-025-60731-3.