Neural representation of time in cortico-basal ganglia circuits

Abstract

Encoding time is universally required for learning and structuring motor and cognitive actions, but how the brain keeps track of time is still not understood. We searched for time representations in cortico-basal ganglia circuits by recording from thousands of neurons in the prefrontal cortex and striatum of macaque monkeys performing a routine visuomotor task. We found that a subset of neurons exhibited time-stamp encoding strikingly similar to that required by models of reinforcement-based learning: They responded with spike activity peaks that were distributed at different time delays after single task events. Moreover, the temporal evolution of the population activity allowed robust decoding of task time by perceptron models. We suggest that time information can emerge as a byproduct of event coding in cortico-basal ganglia circuits and can serve as a critical infrastructure for behavioral learning and performance.

Timing of movements on short time-scales, on the order of hundreds of milliseconds, is essential for everyday behavior such as walking up stairs and, famously, for the highly skilled movement control required by behaviors such as playing the piano. Distributed sets of brain regions, especially including cortico-basal ganglia circuits, have been implicated in temporal representation across intervals of time (1–5). How such representations are achieved is not known. Influential models have suggested schemes using time-stamp codes in which individual neurons having single peaked responses distributed across multiple delays to specific events (6) or schemes using neuronal populations codes (3–5, 7–11). These theories naturally link timing to learning, now recognized as a major function of cortico-basal ganglia circuits (12–14). Keeping track of time is critical for solving the “credit assignment problem” in reinforcement-based learning, because the time delay between an event and the reward that it leads to must be encoded (15–17). Time-stamp coding of events has been explicitly incorporated in temporal difference models of reinforcement learning in basal ganglia circuits (15–16). However, evidence of time-stamp coding has not been found in neural recordings (3, 18), and evidence for population coding is also still largely restricted to responses to particular trained intervals (19–22).

We reasoned that if there is a cortico-basal ganglia timing system that builds temporal representations, it should be possible to decode time from the activity of neurons recorded in the neocortex and striatum of animals performing a simple sensorimotor task. Moreover, time-stamp encoding might be more evident with tasks not involving interval training, because interval training could force population activity toward the trained intervals rather than broad coverage of short time (21). We therefore trained macaque monkeys in a visually guided sequential saccade task that had temporal structure but did not explicitly require precise timing of specific intervals (23–24) (Fig. 1). We searched for neural time representations of the task by recording from large populations of neurons with multiple electrodes implanted simultaneously in the dorsolateral prefrontal cortex (DLPFC) and the caudate nucleus (CN).

Fig. 1.

Sequential saccade RSQ4 task (A) and approximately Gaussian saccade latencies (287 ± 36 msec; 30,697 trials) of saccades made to first Go signal (B).

Results

Two macaque monkeys were trained to make saccades in response to visual targets presented sequentially on a computer screen in front of them (Fig. 1). In each trial of the standard task, the monkey had to fixate for 1 sec a central red fixation spot and then was required to make saccades to four sequentially presented red target spots, selected randomly to appear in up, down, right, or left directions at adjacent points on a grid of gray potential targets (the RSQ4 task, SI Text). Each target remained illuminated for 400 msec and then was extinguished when the next target turned red. In a few trials, the task was modified to have different intervals (600 or 800 msec) between the targets, variable numbers of targets, or fixed sequences (the non-RSQ4 tasks, SI Text).

We recorded from 5,699 well-isolated single units in the DLPFC (n = 2,496) and the CN (n = 3,203) in two monkeys for 252 days over a period of 3 years, using implants of 4–48 simultaneously implanted electrodes in chronic 1- to 3-month sessions. Among these, 1,613 units in the DLPFC and 2,035 units in the CN were selected for detailed analysis based on strict criteria (Materials and Methods and Table S1). We constructed the response profiles of the units by aligning spikes of multiple trials with the events and computing and smoothing the peri-event time histograms (PETHs) (Materials and Methods and Fig. S1).

To characterize the large number of neural response patterns obtained by aligning spikes to the fixation onset in the RSQ4 task, we developed a clustering algorithm that groups units with similar response profiles (Materials and Methods). Units with unique profiles were left unclassified. We classified 1,004 DLPFC neurons into 66 clusters, leaving 609 unclassified. For the CN, 1,070 units were classified into 35 clusters, with 965 left unclustered. The averaged profiles of all of the prefrontal and striatal clusters are shown in Fig. S2; representative units are shown in Fig. 2. The profiles exhibited great variety (Fig. 2). There were dominant peak responses after the first Go signal or for each Go, phasic responses riding on ramping activity or suppressed firing, and prominent late responses.

Fig. 2.

Averaged response profiles of selected neuronal clusters in the dorsolateral prefrontal cortex (DLPFC) (A) and the caudate nucleus (CN) (B) and examples of corresponding spike raster plots. Cluster IDs and units per cluster shown above profiles. Traces: First black, fixation period; alternating red and blue, successive 400 msec Go periods; green, “extra-peak” period (23); last black, reward period and intertrial interval.

Time-Stamp Representations of Events.

The clustering results showed that responses related to particular task events occurred over a strikingly large range of times, suggesting that neurons in the DLPFC and the CN might time-stamp task events. To test this idea, we examined in detail the activity of neurons with predominant phasic responses after the task-start signal, the first Go signal, and the last target-off.

Our largest samples were for the first Go units (Fig. 3). At first Go, the monkeys had been fixating for 1 sec and simply had to respond with a saccade, a natural response in primates. To distinguish between “sensory” responses to the first Go signal and “motor/preparatory” responses related to the first saccade, we took advantage of the fact that the first saccades were distributed in time (287 ± 36 msec, Fig. 1B), whereas the Go signal occurred at a fixed time. Responses significantly time-locked to saccade onset were classified as primarily “motor;” otherwise, we used the term “visual” as a descriptor for the other phasic responses that occurred after the first Go signal and before a response to the second Go signal was possible (Materials and Methods and Figs. S3 and S4). Of the 70 DLPFC neurons whose main activity was a phasic response after the first Go signal, we classified 55 as visual and 15 as motor. In the smaller sample of striatal first Go-responsive units, we classified 10 as visual and 6 as motor.

Fig. 3.

Evidence for time-stamp responses. Single DLPFC (A) and CN (C) units having single dominant peak responses to first Go (red dots), task start (green), or target off (blue) during the Go, fixation, or extra-peak periods, respectively. Gray vertical lines indicate event times. Profiles are normalized to maximum firing rates; peak rates and latencies are shown above. Black curves represent RSQ4 trials; gray curves represent non-RSQ4 trials. (B and D) Peak half-widths and distributions of peak times for DLPFC (B) and CN (D) units.

Remarkably, the peak responses of the visual units covered most of the time between the first and the second Go periods (Fig. 3, representative units). The peak latencies ranged from 145 to 435 msec for the DLPFC units (Fig. 3 A and B) and from 165 to 485 msec for the CN units (Fig. 3 C and D). Manipulating inter-Go intervals or the number or sequence of the saccades (the non-RSQ4 tasks) did not alter these peaks (Fig. 3). Peak half-widths tended to increase with increasing latency (Fig. 3 B and D), but the peak responses nevertheless provided a fine coverage of time after the event, far beyond that demanded by the saccadic response. The peak responses were absent during other Go periods, the fixation period and after last target-off, suggesting that they are not simple visual responses but time-stamp the first Go signal.

We found similarly dispersed (145 to 355 msec) peaked visual responses following task start, after which the monkey had simply to hold fixation for 1 sec. These occurred mainly in the DLPFC (Fig. 3; n = 37 for DLPFC, n = 3 for CN). Dispersed responses also occurred after the last target-off during the 400-msec “extra-peak” period (23) (Fig. 3; n = 33 in DLPFC, n = 14 in CN). The monkeys rarely moved their eyes during this period of waiting for the variably timed reward delivery (23). The peak responses were absent in other periods of the task. Thus, for all three periods sampled, the phasic visual responses time-stamped the first Go, task start or last target off, and had remarkably dispersed onset times.

The dominant activity peaks of the motor neurons time-locked to the fixation onset during the fixation period (n = 47 in DLPFC; n = 9 in CN) or to the first saccade during the Go period (n = 17 in DLPFC; n = 6 in CN) were mainly postsaccadic (24), with latencies distributed ≈100 msec and dispersed from 15 to 295 msec (Fig. S5).

These results indicated that even though the monkeys simply had to be attentive and reactive in the task, making saccades to each newly lit visual stimulus without pressure to time their visuomotor responses, prefrontal and striatal neurons formed a time-stamp representation of the sensory and motor events in the task. Given the small sample of these dominant-peak neurons, the time-coverage of the representation is impressive.

Population Coding of Time.

The diversity of neural responses shown by the clustering results further suggested that the ensemble prefrontal and striatal activity might be used to distinguish different time points during the task, as proposed in population models of time representation (3–5). We tested this idea by using the profiles of the prefrontal or striatal neurons as inputs to a perceptron model of a decoding neuron (25). The perceptron was driven by the weighted sum of the firing rates of the input neurons at each time point, with firing rates taken from the profiles, and it fired if the sum exceeded a threshold (Materials and Methods). We included all accepted DLPFC or CN units with rate variations (maxima − minima) > twice the rate fluctuations and with maxima >5 Hz for DLPFC units (n = 506) and >3 Hz for CN units (n = 429). The population thus included not only the time-stamp units, but also units with multiple peaks or with gradual changes in their firing rates during the task. We determined whether suitable weights and thresholds could be found so that the perceptron given prefrontal or striatal activity would fire only at one time point during the task, thus decoding that specific time. We then tested whether perceptrons could be constructed for all time points during the task.

To measure the robustness of the temporal discrimination of the perceptrons, we computed for each perceptron the separation margin, which indicates how much the weighed sum differs when the perceptron fires compared with when it does not (Materials and Methods). We calculated the set of weights and the threshold that maximized this margin and thus gave the most robust decoder [i.e., a support vector machine (26)]. For time points throughout the task, we plotted the maximum margins and the corresponding noise levels, here defined as the maximum margins obtained by using randomized input profiles (Materials and Methods).

The results were clear cut. Perceptrons given the activity of either the prefrontal units or the striatal units recorded could decode every time point in the task, at levels well above the noise levels, with a resolution of 50 msec during the Go periods, the first 500 msec of the fixation period, and the extra-peak period (Fig. 4 A and E). Remarkably, the maximum margin plots emphasized the beginning and end of the saccade sequences, a pattern similar to that previously found to hold for the spike activity of a subset of DLPFC neurons (23). Individual perceptrons showed sharp time-stamp decoding (Fig. 4 B and F). Profiles with large, sharp changes in firing rate at different times in the task enhanced the robustness and precision of the time decoders (Fig. 4 D and H; SI Text). Changing the resolution and the number of input neurons did not alter the results (Fig. S6; SI Text).

Fig. 4.

Perceptrons driven by DLPFC inputs (A–D) or by CN inputs (E–H) can decode time during the task. (A and E) Maximum margins for perceptrons with prefrontal (A) and striatal (E) inputs discriminating single time points from all others at least 50 msec apart. Colors are as in Fig. 2. Gray trace indicates the noise level. (B and F) Examples of weighted sums of inputs for three perceptrons. Red dots indicate times decoded. (C and G) Ten trial-by-trial spike responses of online versions of decoders in B and F receiving raw spike as inputs. (D and H) Selected profiles of most influential prefrontal (D) and striatal (H) units for the decoders during first Go period.

We next asked whether we could decode time trial-by-trial by using the raw spikes—required if the decoders are representative of real neurons—instead of by using the activity profiles, which were based on spikes from multiple trials. For this analysis, we reasoned that if each neuron belongs to a group of neurons that all have the same (or closely similar) profiles, the group-averaged spike rates in a single trial will approximate well the individual neuron's profile, especially when the group size is large. Our clustering results (Fig. 2) suggested that such groups exist. We therefore generated for each prefrontal and striatal activity profile a group of 500 artificial neurons whose spikes were sampled from the profile (Materials and Methods). The trial-by-trial spike responses of the decoders were sharp, as illustrated in Fig. 4 C and G, for the decoders (Fig. 4 B and F) given raw spikes from the corresponding groups of artificial neurons: in the 10 trials shown, each decoder spiked within 10–30 msec of the decoded time. It was further possible to build online decoders with the raw spikes of the recorded neurons only, using the averaged cluster profiles shown in Fig. 2 and Fig. S2. The numbers and sizes of the clusters were small, and consequently, the maximum margins were small, and noise at each trial was high. Nevertheless, reliable prefrontal online first Go decoders could be built (Fig. S7).

Discussion

Our findings suggest that prefrontal and striatal neurons carry time-stamp representations of short time, and that the population activity patterns of prefrontal and striatal neurons encode time information that can be read out by a simple linear threshold model. Critically, our findings indicate that time is encoded by these neurons even when the animals perform a simple task that does not have precise-timing requirements. Much evidence suggests a distributed neural encoding of time. Imaging, pharmacological, and lesion studies demonstrate that both neocortical and subcortical brain regions are involved in timing short intervals (4–5, 7–8, 19–20, 27–36). A range of neuronal firing patterns has been proposed to underlie the computation of time, including oscillatory activity (5, 7, 10), ramping spike activity (19–20), spectral responses (6), and network-produced ensemble patterns (8, 11).

Our findings suggest that time may be encoded as part of the infrastructure of neural representations of events and actions.

Time-Stamp Coding.

The responses that we identified as providing time-stamp representations had dominant peaks at latencies distributed from ≈150 to 500 msec after single visual events signaling task start, first Go, or last target-off. They occurred after only one of these events. Thus they provided signals tied to a single event and the passage of time since that event, with separate sets of neurons time-stamping different events.

These firing patterns are strikingly similar to the response profiles posited in the spectral timing theory, which postulates that time-encoding neurons respond to a transient event with varying time delays (6). Such encoding of time after events, also called a “complete compound stimulus” (37), is used in reinforcement learning to solve the temporal credit assignment problem (15–17). The presence of time-stamp coding that we found here in both the prefrontal cortex and the striatum thus supports reinforcement learning models positing cortico-basal ganglia loops with the capacity to solve the credit-assignment problem (16). This kind of time-stamped representation of short time also makes the learning of timed actions easy, as simple associations made from subgroups of such neurons and other neurons in motor control regions would suffice. Thus the time-stamp coding that we found could underpin flexible neural computations in short time.

Our ability to observe this time-stamp activity may have been due to the fact that we sampled the activity of thousands of neurons and that we did not impose explicit training on particular intervals, features that distinguish our experiments from most previous timing studies.

Population Encoding of Time.

Besides these time-stamp responses, we found that populations of cortical and striatal neurons have a variety of response profiles during the task. The firing rates of these neurons waxed and waned in various epochs of the task, so that as a population, the neurons could have encoded time information at a fine resolution. We demonstrated this by constructing perceptrons that were driven by the neuronal populations throughout the task but responded only at single time points with 50-msec resolution. The responses of the perceptrons were similar to those of the time-stamp neurons. This result supports theories suggesting that population activity is used to build temporal representations in cortico-basal ganglia circuits (4–5, 10).

The firing pattern of a given neuron recorded in our task could have been influenced by a multitude of external events, such as the Go signals, and also by events generated by the subjects, such as the saccade onsets. These signals carried only crude time information: the central light fixation point signaled the onset of the fixation period; the Go signals and the saccades indicated the target-saccade events during the movement period; and the last target light offset signaled task end, which was followed by a variable delay and then reward. The neural encoding of time that we observed was at a much finer scale. The neurons responded with different delays to the same events, and they had different response onsets even when their response profiles were similar.

We found that neurons with sharp variations in their firing rates contributed the most to the accuracy and robustness of the time decoders (Fig. 4 D and H). This relationship is consistent with the general result that narrow tuning curves are most accurate in reconstruction of one-dimensional physical variables (38), in our case the time. Notably, times at the action boundaries of the entire task were more robustly encoded than times within the task. This could reflect the control demands of initiating and then ending a series of related actions forming an entire sequence (23). Several technical issues related to detecting temporal code in our data are discussed in SI Text.

Timing in Cortico-basal Ganglia Loops.

How are the time representations in the DLPFC and the CN constructed? One possibility is that the latencies of the neural responses to visual inputs are distributed (39) because visual signals reach the DLPFC and the CN by multiple routes with varied latencies. Indeed, the dispersion of visual responses in the DLPFC is ≈273 msec (40), which approximately matches the range of the peak response latencies after the visual cues that characterized the time-stamp neurons in our recordings. A wide distribution of latencies could be a precondition for fine-scale time encoding with population activity. In this view, time encoding would be unlikely in regions with limited response ranges, such as the primary visual cortex (V1; 63 msec) and the frontal eye field (FEF; 50 msec) (41), but quite possible in other regions with large response range, such as the human hippocampus (>500 msec) (42). However, additional mechanisms are needed to transform the dispersed visual responses into a variety of neural response profiles, including the time-stamp responses, that could encode unique time points in tasks with durations far beyond the visual response range. We suggest that the time representations we observed reflect an intrinsic tendency of the brain to form the basis of temporal computations, perhaps through a reward-independent self-organizing process, similar to the formation of cortical feature maps driven by natural visual stimuli (43–45), even in situations in which precise timing is not critical for performance.

Although we recorded more phasically responding neurons in the DLPFC than in the CN, and found the neurons in the CN to have noisier firing patterns, the time encoding in these two regions was quite similar. This result suggests that the prefrontal cortex and the striatum might be intimately interrelated moment-to-moment through prefrontal cortico-basal ganglia loops (24). This suggestion is supported by the fact that the firing patterns in two brain regions were similar despite the emphasis of phasic responses in DLPFC and smooth firing in CN. We did not have local field potential recordings in sufficient number to look for evidence of a relation between the spike activity patterns that we observed and oscillatory activity in the prefrontal cortex or striatum, and thus could not address the striatal beat frequency model proposed by Matell and Meck (5, 10). Our data do suggest that the spiking patterns would be compatible with such a model.

Our results raise the possibility that the representation of time may reflect an inherent tendency for the brain to represent time as part of ongoing task-specific information processing. If so, neural circuits might build time representations as an infrastructure to use when needed. Such encoding would have major advantages for neural processing related to learning how to control actions, because all of the elements needed to form on demand new associations between events and precisely timed actions would be available. The great variety of neural responses that we observed, the fine-scale coverage of time in the perceptron decoding, and the differential coding robustness at salient events suggest that cortico-basal ganglia circuits may actually encode all events and the passage of time since the events, at least at short time scales. Decoding algorithms based on our findings should also be valuable for brain-machine interface strategies to improve timing deficits encountered clinically, as in Parkinson's disease.

Materials and Methods

For details, see SI Text.

Behavior, Electrophysiology, and Single-Unit Analysis.

Two monkeys were fitted with recording chambers and eye coils according to National Institutes of Health and Massachusetts Institute of Technology guidelines for animal experimentation (1). The monkeys were trained to make saccades in response to visual targets presented sequentially on a 9 × 9 grid on a computer screen. Multiple tungsten electrodes were implanted chronically in the prefrontal cortex and caudate nucleus bilaterally and used for up to 1–3 months. Spike activity was sorted into clusters by Autocut under manual control and was analyzed with custom software (3–4). Single units with low firing rates or unstable firing patterns across the session or no task-related activities or assessed as the same as previously recorded units on the same electrodes were excluded from the analysis. The PETH of a single unit was constructed by aligning spikes from the RSQ4 trials with fixation onset and counting spikes in 10-msec bins spanning from −100 msec to 5,000 msec. PETHs were also constructed relative to other task events. A smoothed PETH (sPETH) was obtained by using the least-square fitting to the PETH with B-form cubic splines. The distance between two sPETHs is defined as one minus the Pearson's product-moment correlation coefficient with the means subtracted. A “core point clustering algorithm,” which constructs the neighborhood structure of the points representing the sPETHs and groups points close to each other, was used to cluster the response profiles of the single units.

Distinguishing Sensory and Motor Responses.

Peaks in sPETHs were detected by least-square fitting the curves with asymmetric Gaussian functions with linear base. The peak latency and the height were taken as the position and value of the maximum of the fitted curve, respectively. The robustness of the peak was assessed by using the bootstrap resampling technique. A peak was assessed as “motor” response if its height was significantly reduced when the saccade timings were randomized across trials. It is assigned as “visual” otherwise.

Population Coding of Time.

The duration of the task (−100 to 5,000 msec relative to the fixation onset) was divided into 510 bins of 10 msec. At each time bin T, a perceptron was constructed. Its response at task time t is determined by the weighted linear sum of the firing rates of selected single units in the DLPFC and in the CN taken from the normalized sPETHs aligned at the fixation onset. If the sum exceeded a threshold, the perceptron fired. The weights and threshold were adjusted such that the perceptron fired only when t = T, and that the margin, defined as the minimum of the differences between the sum at T and those at other times, was maximized (maximum margin). The maximum margin depends on the resolution of the perceptron, as well as the number and tuning widths of the input neurons (shown with a theoretical model discussed in SI Text; Fig. S8). To demonstrate that it is possible for the perceptron to decode time by using spikes at single trials instead of the response profiles constructed from spikes over multiple trials, the inputs to the perceptron were replaced by the spikes of artificial neurons at a single trial, which were created by using a sampling process based on the response profiles of the single units. There were 500 artificial neurons for each single unit. Online perceptrons were also constructed by using the raw spikes of the single units.