Memory Takes Time

Summary

Memory is an adaptation to particular temporal properties of past events, such as the frequency of occurrence of a stimulus or the coincidence of multiple stimuli. In neurons, this adaptation can be understood in terms of a hierarchical system of molecular and cellular time windows, which collectively retain information from the past. We propose that this system makes various time scales of past experience simultaneously available for future adjustment of behavior. More generally, we propose that the ability to detect and respond to temporally structured information underlies the nervous system’s capacity to encode and store a memory at molecular, cellular, synaptic and circuit levels.

Introduction

Synaptic plasticity is generally accepted as the principal implementation of information storage in neural systems (Kandel et al., 2014). It involves relative and/or absolute changes in synaptic strength, established by a wide range of mechanisms and persisting for various periods of time, ranging from seconds and minutes to days, weeks and in the limit, a lifetime.

While “synaptic plasticity” is a relatively unambiguous term applicable to a specific cellular phenomenon, the basic concept of memory is broader, more abstract and thus more open to interpretation. For the purposes of this review, we consider memory as any perturbation in a system, caused by external stimulation, which persists past the cessation of the initial stimulation and alters the system’s responsiveness to subsequent stimulation. Thus memory, in the broadest of terms, is an adaptation to the past. We should point out that this definition of memory can also apply to many perturbations in non-biological systems, including viscoelastic deformation, anomalous diffusion, capacitor voltage changes, and stock market fluctuations (Sun et al., 2011; Westerlund, 1991). However, in biological systems, memory underlies the ability to adapt behavior to experience and thus allows an organism to increase its fitness ontogenetically. As Schacter and colleagues put it, one “remembers the past to imagine the future” (Schacter et al., 2007). Brains can thus be understood as prospective devices.

Why is synaptic plasticity a highly suitable mechanism for the implementation of memory? The primary reason is that changes in synaptic strength are not only stochastic or genetically preprogrammed, but dependent on the previous history of the synapse – in other words, the current state of a synapse is a function of its past states. A pattern of synaptic stimulation (whether by transmitters, modulators, cytokines, or changes in membrane potential) produces a cellular response in either or both the pre- and postsynaptic neurons. This response in turn alters the way subsequent stimulation produces subsequent synaptic responses, i.e. causes a change in synaptic strength. This feedback is what defines memory as an adaptive response.

Various forms and classifications of synaptic plasticity have been characterized. The distinction between short-term, intermediate-term and long-term plasticity is based on the mechanistic requirements for their formation (Goelet et al., 1986). Short-term plasticity, lasting seconds, depends on post-translational modifications of pre-existing proteins. New protein synthesis is required for intermediate-term plasticity lasting minutes to hours (Sutton et al., 2002), and new gene expression for long-term plasticity, which typically lasts for hours or longer. Additionally, plasticity can be expressed either as an increase (potentiation) or decrease (depression) of synaptic strength. Finally, plasticity can be expressed either presynaptically, postsynaptically, or both (Kandel et al., 2014).

While synaptic plasticity has been analyzed in neuroscience literature for decades, its direct empirical connection to memory is often more in the realm of metaphor rather than mechanism, largely because the term “memory” can have many meanings in a wide range of contexts. Yet a deep conceptual understanding of memory is critical for the general field of neuroscience. Ideas such as memory trace, memory acquisition, memory consolidation, or memory retrieval, to name a few, are routinely used to formulate scientific hypotheses. While certainly open to various definitions, they linguistically presuppose the idea of “a memory”, which may be seen as separate from the “memorizer”, who acquires it by “memorizing”. Synaptic plasticity in this paradigm is implied to aid the memorizer in the memorizing procedure. Various mechanisms of synaptic plasticity, such as phosphorylation of ion channels or expression of regulatory genes, are typically said to “contribute”, “underlie”, or “be required for” memory formation, which is seen as a distinct, if elusive, end to the mechanistic process of memory acquisition.

This general approach may be justified if memory is defined solely through its behavioral expression. From a behavioral standpoint, memory can indeed be easily parsed into stages of acquisition, retention, and expression or retrieval. This disassociation of “memory input” and “memory output” in turn necessitates a concept of a memory store, an idea that historically has been central to memory research (Fig. 1). Multistore models of memory (Fig. 1A, a) explain the transition between short- and long-term memory by the transfer of information between physically distinct stores. Depth of processing models (Fig. 1A, b) presume that distinct temporal domains of memory are localized to the same store, but are processed differently (Craik, 1972). Yet seen from a modern cellular and molecular biological perspective, behavioral categorization of memory stages imposes an arbitrary and perhaps unnecessarily restrictive “scale constraint” on what amounts to behavior and therefore memory in the first place. For example, a movement of a limb is typically considered behavior, whereas a movement of an ion across the cell membrane is not, and thus can be part of a “memory mechanism”, but not the “memory” itself if the definition of memory requires the additional constraint of behavioral expression.

Fig. 1. Frameworks for understanding memory.

A. Theoretical models explaining the transition between short-term, intermediate-term and long-term memory. (a) In multistore models learned information is progressively passed on between multiple stores utilized for short- or long-term retention. (b) In depth of processing models the same store transitions from short- to long-term retention of information (Craik, 1972). (c) In the temporal hierarchy model advocated in this review, many levels of short- and long-term information simultaneously contribute to ongoing experience at any given time. No particular store can be isolated within the complete biological system that retains information. B. Experience can be seen as a series of temporally limited deviations from homeostasis. The temporal structure of experience, illustrated by coloration, defines the temporal structure of memory shown in A. For example, repeated-trial learning simultaneously retains information from the most recent trial as well as from all combinations of all preceding trials.

In this review, we agree with and extend a broader view that memory cannot be viewed separately from the system that encodes it (Milner et al., 1998), i.e. the “memorizer”. No single object, process, or state represents “a memory”. No “store” can be isolated as a particular substructure within the complete biological system that retains external information. Rather, the structure of memory lies in the temporal domain. Experience can be seen as a series of temporally limited deviations from homeostasis, the timing of which determines the structure of resulting memory (Fig. 1B). Memory is inherently and fundamentally multi-leveled, with each level represented as a distributed combination of distinct physical entities forming a hierarchy of cause and effect (Fig. 1A, c). Instead of treating memory as an end to memorization, we view biological memory, and specifically synaptic plasticity, as functions of a holistic, dynamic, hierarchically structured system that represents the timing of past events. It consists of multiple nested levels of molecular, cellular and higher-order homeostatic perturbations, each with particular temporal properties. Levels of this hierarchical system are linked to each other through both emergence (coincidence detection) and recursion (feedback).

A key feature of this system is its ability to represent temporal particularities of past experience which span milliseconds to years (Fields et al., 2005; Fuster and Bressler, 2012; Loewenstein and Sompolinsky, 2003; Markram et al., 1997; Mons et al., 1999; Tsien, 2000). These physical representations of the past range from single atoms to neuronal populations and states of entire organ systems, which allows the organism to use many past timescales simultaneously to modify ongoing behavior (Fig. 1). Long-term episodic memory is physically represented in countless variables of cortical and hippocampal state and structure and is expressed as autobiographical recall of polymodal experiences. Short-term facilitation (as revealed for example by paired-pulse facilitation) is physically represented in post-translational states of membrane ion channels, expressed as briefly altered membrane conductance, and constitutes memory of the fact that the synapse was used milliseconds ago (Buonomano and Maass, 2009; Zucker and Regehr, 2002). What is in common between these extremes of scale and complexity is the representation of temporal information retained from past events by the organism’s own devices.

We thus place time and temporal patterning at the center of the concept of memory and argue that the nervous system’s extraordinary ability to represent time at multiple time scales is a prerequisite for its unmatched capacity for information storage and, by extension, for its adaptive value.

It is important to stress at the outset that some of the themes that we will discuss in this review have been previously considered in a variety of contexts. This is especially true of several discussions addressing two major intersecting themes: (i) learning and memory (Kandel, 2012; Kandel et al., 2014; Korte and Schmitz, 2016), and (ii) temporal patterning (Buonomano and Maass, 2009; Feldman, 2012; Lynch et al., 2013; McGaugh, 1966; Philips et al., 2013; Smolen et al., 2016). So what is new in the current review? What we hope to accomplish here is to consider memory processing from a novel perspective. Specifically, we will develop the argument that identifying and elucidating specific and unique temporal domains in memory processing falls short of capturing the extremely wide range of nested temporal domains that are simultaneously operating in the service of encoding and storing long term memories. We will propose that this broader consideration of a wide range of scaled and integrated temporal domains can afford unique insights that can both elucidate the mechanistic complexity of memory storage, and suggest novel experimental approaches that have the capacity to significantly advance our understanding of the mechanisms of long-term memory storage in the nervous system.

This review is composed of four primary sections. We first discuss theoretically how the terms “coincidence”, “hierarchy” and “pattern extraction” apply to time representation in neurons, and consider the phrase “time window” as a reference to various homeostatic disturbances limited in time. In the following section, we address the hierarchy of these time windows, which we propose form a nested system of temporal pattern detection (Table 1). We discuss its molecular and cellular mechanics and their connection to synaptic plasticity. Finally, in the next section we discuss synergistic interactions between time windows, both within and across levels of the hierarchy. We conclude in the last section by summarizing the discussion and considering “time” more broadly as the key variable in the functioning of a nervous system.

Table 1. Classification of neuronal time windows.

Descriptions and examples of regulatory inputs related to temporally restricted homeostatic disturbances in the neuron. Similar classification can be applied to signaling in many non-neuronal cells.

Nature of time window	ON inputs		OFF inputs
	Description	Examples	Description	Examples
First messenger availability	Release of the messenger by upstream effector cells	Firing of presynaptic cell; constitutive or regulated secretion of neurotrophins	Removal of messenger by diffusion or active transport	Reuptake of 5-HT by presynaptic termini
	Transport or diffusion of the messenger to the receptor	Diffusion of neurotransmitter across synaptic cleft
Receptor state	Activation of receptor (binding to first messenger)	Dimerization and cross-phosphorylation of RTKs	Deactivation of receptor	Desensitization of GPCRs; glycine binding by NMDAR
	Activation of receptor-coupled effectors	Dissociation of heterotrimeric G-proteins	Deactivation of receptor-coupled effectors	Guanine exchange in G-proteins facilitated by RGS proteins
Second messenger availability	Production of second messenger	Lipid phosphorylation and cleavage, influx or release of Ca2+, cyclization of ATP or GTP, synthesis of NO, depolarization	Removal of second messenger	Dephosphorylation of lipids, removal of Ca2+ to internal stores, recruitment of DAG to anabolic pathways, hydrolysis of cyclic nucleotides, shunting of membrane potential
	Stabilization of second messenger	Inhibition of phosphodiesterases, phosphatases, sustained changes in membrane potential
	Diffusion of second messenger to the site of action	Diffusion of Ca2+ to the nucleus; spread of the membrane potential to the trigger zone	Diffusion of second messenger away from the site of action	Diffusion of cGMP away from cationic channels in rod photoreceptor cells; attenuation of membrane potential
Target molecule state	Binding to second messengers or cofactors	Binding of calmodulin (CaM) to Ca2+, binding of AMPK to AMP	Dissociation of second messengers or cofactors	Dissociation of CaM and Ca2+ upon decrease in Ca2+ concentration
	Covalent modifications	Mediated by ligases, e.g. kinases or ubiquitin ligases. Phosphorylation of ion channels in response to stimulation	Removal of covalent modifications	Mediated by hydrolases, e.g. phosphatases or DNA demethylases. Resetting of the epigenetic code
	Intermolecular interactions	Regulated assembly of ribosomes, proteasomes; binding of RISC to miRNA-mRNA complexes	Disassembly of complexes	Depolymerization of cytoskeletal filaments
Target molecule availability	Transport or diffusion of the target molecule to the site of action	Translocation of kinesin-bound GluR2 to dendrites; diffusion of Ca2+/CaM to the nucleus; capture of CaMKII RNA at the synapse	Transport or diffusion of the target molecule away from the site of action	Removal of AMPARs from the postsynaptic density
	Synthesis of target molecule	Availability and activation of transcription factors in the nucleus or translation factors in the cytosol; favorable epigenetic landscape	Degradation of target molecule	Proteolytic degradation of the PKA regulatory subunit

Abbreviations: 5-HT, 5-hydroxytryptamine; RTKs, receptor tyrosine kinases; GPCRs, G-protein-coupled receptors; AMPAR, α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid receptor; NMDAR, N-methyl-D-aspartate receptor; AMPK, 5′-adenosine monophosphate-activated protein kinase; RGS, regulator of G-protein signaling; DAG, diacylglycerol; CaM, calmodulin, CaMKII, Ca²⁺/calmodulin-dependent protein kinase II, PKA, protein kinase A.

It is beyond the scope of this review to consider all possible temporal interactions observed in neurons. Rather, by considering specific examples, we aim to provide a conceptual framework through which any new pathway contributing to synaptic plasticity can be understood. We posit that memory as a biological process is defined by a hierarchy of homeostatic perturbations that occur on multiple time scales and simultaneously represent the timing of past events during ongoing experience. Although representation of temporal relationships at the microcircuit and network levels is beyond the scope of this review, we propose that it may extend and parallel a temporal hierarchy entrenched in molecular and cellular biology of individual neurons, potentially representing a result of a unified evolutionary drive to adapt behavior to temporal regularities of the external world.

Temporal interactions and feedback in synaptic plasticity: theoretical considerations

Coincidence and continuity: an example drawn from NMDA receptors

In order to drive plasticity, events inside and outside of the neuron must co-occur within a defined temporal range. To illustrate this principle of coincidence, we will consider a well-characterized example of the N-methyl-D-aspartate receptor (NMDAR), an important regulator of synaptic plasticity. A key feature of NMDARs is their permeability to Ca²⁺ ions, which, upon entry into the postsynaptic cell, trigger a multitude of downstream biochemical cascades. However, to permit calcium entry, NMDA receptors must be simultaneously exposed to the neurotransmitter glutamate and membrane depolarization (Markram et al., 1997; Spruston et al., 1995; Yuste et al., 1999). Neither exposure can be seen as a discrete event. Both transmitter availability and membrane depolarization are alterations in states persisting within a specific and limited temporal window. Depolarization that activates NMDARs can arise from several sources. For example, backpropagating action potentials (bAPs) (Blanke and VanDongen, 2009; Spruston et al., 1995; Stuart et al., 1997), which are longer in duration than axonal spikes and allow for a relatively persistent window of depolarization, can be sufficient to cause dissociation of an inhibitory Mg²⁺ ion from NMDAR (Blanke and VanDongen, 2009). Depolarization can also arise from dendritic stimulation alone, either independently of bAPs or synergistically with them (Blanke and VanDongen, 2009; Enoki et al., 2004; Gordon et al., 2006; Schiller et al., 2000; Schiller and Schiller, 2001). When pairs or trains of EPSPs occur, NMDARs amplify the later EPSPs (Clark and Collingridge, 1996; Collingridge et al., 1988; Nicoll et al., 1992; Thomson, 1997; Thomson et al., 1988), suggesting that ongoing local glutamatergic stimulation of dendrites can continuously recruit NDMAR channels independently of bAPs or dendritic spikes. Backpropagating and/or persistent states of dendritic depolarization act to greatly increase the number of NDMAR channels active and to prolong their activity. Thus, all sources of depolarization converge to open a single temporally defined window during which glutamatergic input can engage NMDAR activation.

This example illustrates that “coincidence” is not, in fact, an all-or-none state when applied to a neuron, a synapse or even a population of molecules. Depending on the temporal properties of depolarization (e.g. whether it arises from bAPs or ongoing local stimulation) and transmitter availability (which in turn depends on the timing of action potentials arriving at the presynaptic neuron as well as the diffusion and uptake of the neurotransmitter), either or both of the criteria for calcium influx through a given NMDAR may be satisfied for various periods of time. For a population of NMDARs, coincidence, and by extension its downstream effect – aggregate Ca²⁺ influx – is more appropriately described a temporally graded continuum rather than an immediate and discrete state (Gerstner and Kistler, 2002).

Similarly, the biochemical regulation of complex systems such as molecular pathways, gene networks, or cellular ensembles typically proceeds in a temporally graded, continuous manner. To emphasize this continuity, we find it helpful to re-evaluate the traditional molecular biological terminology of “activators”, “inhibitors” or “targets”. “Single-molecule language” reinforces the impression of discrete-state mechanics, whereby enzymes or genes can be either active, or inactive, depending on the upstream signals. In this review, we utilize the terms “ON and OFF pathways” to refer to temporally graded inputs contributing to a given response. Each ON or OFF pathway is an abstraction of many individual activation/inhibition events.

Hierarchy of timing: an example drawn from cAMP signaling

There is no single event, or even single type of event, that causes synaptic plasticity. Instead, the cell possesses a vast repertoire of temporal windows defined by the state and availability of individual molecules or systems of molecules (Buonomano and Maass, 2009). These windows range from the transiently activated state of NDMARs in a depolarized membrane to long-term changes in transcriptional or even epigenetic state of the cell. Any such window is characterized by ON and OFF mechanisms which together determine its timing. These temporal windows are organized into a hierarchical system, which can be effectively illustrated by considering another relatively well understood example, cAMP signaling.

cAMP is produced from ATP by the action of adenylyl cyclases. Canonically, an adenylyl cyclase is activated by a G-protein, which provides the ON mechanism for a time window of elevated cAMP during signaling via G protein-coupled receptors (GPCRs). This window is closed by phosphodiesterases, enzymes that hydrolyze cAMP. Needless to say, the two reactions do not proceed in isolation or sequentially. It is more informative to think of a steady intracellular cAMP level in a state of equilibrium between ON and OFF mechanisms. When adenylyl cyclase is activated, the equilibrium shifts towards a higher level of cAMP. This level persists until activation of adenylyl cyclase subsides, e.g. when the extracellular signal causing its activation is terminated. OFF mechanisms then take over by simple Michaelis-Menten kinetics, and cAMP levels return to the steady state. The transient state of cAMP increase produces a transient state of PKA activation, which increasingly phosphorylates downstream proteins. Thus, a temporal window of elevated cAMP serves as an ON mechanism for a different temporal window, defined by the increased quantity of PKA-phosphorylated proteins. The OFF mechanism for this novel window is provided by dephosphorylation or protein turnover, ultimately returning the levels of phosphorylation to the baseline. A temporal window conditional on a particular pattern of events can therefore itself serve as a condition for novel, emergent temporal windows, establishing a “hierarchy of timing”. As we will discuss below, this hierarchy is not fixed in stone since temporal windows of different levels interact to produce emergent responses.

“Time windows” as temporally restricted homeostatic disturbances

The term “time window” is commonly used to refer to a period of time with a distinct beginning and end, particularly when the state of a system is somehow altered within these given temporal limits (Fulton et al., 2005; Montarolo et al., 1986; Seidenbecher et al., 1997; Tsodyks and Markram, 1997). It is important to underscore that for the purposes of this review we define a “time window” differently from “time period”. Biological molecules do not generally “respond” to time per se (as could be argued for radioactive nuclei). They respond to chemical or physical covariates of time in a dynamic system, such as disturbances in levels of second messengers or effector proteins. In order to be meaningful (i.e., to carry information), these covariates must have self-limiting dynamics. For example, in order for transient increases of intracellular cAMP to serve a signaling purpose, a delayed mechanism to return cAMP to an equilibrium level must be present. It is these temporally self-limiting disturbances in cellular homeostasis we will refer to as “time windows”. Absolute time is therefore only one of the variables in “molecular time”.

This definition of a “time window” also includes higher-order temporal features of homeostatic disturbances, such as amplitude and rates of onset or decay. This is important since first- or second-order kinetics can carry temporal information themselves. For example, a rapid, high and short rise in postsynaptic calcium is required for induction of LTP, whereas a slow, low and long-lasting rise is needed for LTD (Evans and Blackwell, 2015). It is difficult to disambiguate individual roles for the variables of (i) time, (ii) rate of increase and (iii) peak value of calcium, all of which contribute to the instantaneous intracellular concentration of calcium, the ultimate factor responsible for the plasticity-driving function of calcium influx. By artificially buffering calcium concentration at a constant level, the effect of timing can be disambiguated from the effect of peak value and rates of onset/decay. However, effector molecules do not discriminate between calcium concentrations produced by slow or fast kinetics – the only variable each individual molecule “takes into account” is instantaneous calcium concentration. Expression of LTP or LTD is therefore an emergent response to a particular pattern of calcium fluctuation, a product of integrating coincidence of activity of many effector molecules over the time-course of calcium changes. Higher-order temporal variables such as rates and amplitudes can be ultimately reduced to the timing of upstream events.

To summarize, “molecular time” is measured in relative temporal windows, which are themselves a function of the state of their respective ON and OFF pathways (Fig. 2). Importantly, “ON” and “OFF” do not denote binary switches. We will use these terms simply to indicate pathways that contribute to either the onset or decay of the state underlying the time window.

Fig. 2. Hierarchy of time windows.

A “time window” is a temporally self-limiting event that constitutes a deviation from homeostasis. Specific patterns of time windows at level i−1 (e.g. overlap between time windows of first messenger availability) contribute ON or OFF influences towards a time window at level i (e.g. a time window of second messenger availability), converting transient stimuli into a more persistent cellular response which can influence the cell’s response to subsequent stimulation. This novel time window represents a form of memory, an adaptive response to the past. This timed response, typically in combination with other inputs/time windows, can in turn contribute ON or OFF influences towards a time window at level i+1 (e.g. a time window of increased protein phosphorylation). Thus, a memory at level i is at the same time a stimulus contributing to a more persistent memory at level i+1.

Pattern extraction as temporal emergence

A final term that requires consideration is “pattern extraction”. In abstract terms, any process is characterized by a particular temporal profile of its occurrence, and therefore mere responsiveness of a system to such timed processes is trivial. Pattern extraction, on the other hand, is defined here as an emergent response to multiple time windows that by themselves do not have the capacity to create the response. To use an everyday example, a single tap on a touchscreen device produces a response to an event, whereas a double-tap (that is, two taps separated by a specific, limited time period) produces a response to a pattern, which the device extracts from the operator’s actions. This response is emergent, since it is derived from the responses to individual taps but not simply a sum of the two.

As discussed above, a transient intracellular calcium elevation is an emergent response to a temporal overlap of membrane depolarization and transmitter binding by NMDAR. As we will show below, such emergent responses themselves constitute time windows that are used for pattern extraction at higher hierarchical levels, which typically correspond to longer time periods and more stable changes in synaptic strength.

Emergent responses produced within the hierarchy typically feed back onto the previous levels of detection and alter their subsequent responsiveness. For example, a transient NMDAR- dependent calcium elevation may cause more lasting changes in the composition of the membrane receptors, resulting in stronger depolarization following the same level of neurotransmitter. Since depolarization contributes one of the ON inputs for NMDAR activation, it alters the time window of subsequent calcium entry upon repetition of stimulation, increasing its probability of interaction with other time windows in the cell, thus producing even more lasting responses. Since we have defined memory as a persistent perturbation caused by past stimulation and altering the effect of future stimulation, it follows that biological memory is fundamentally hierarchical in nature.

Neuronal time windows: all good things must come to an end

First messenger availability and receptor state

First messengers include any extracellular signals that evoke a specific response in a target cell. These may include neurotransmitters, cytokines, growth factors, hormones and even extracellular ions such as calcium (Hofer and Brown, 2003). These molecules typically transduce their effects in the cell by binding to extracellular domains of membrane-spanning receptors, although some membrane-permeant molecules such as nitric oxide, arachidonic acid, and glucocorticoids, all of which have been implicated in synaptic plasticity, exert their effects intracellularly (O’Dell et al., 1991; Pavlides et al., 1996). Environmental factors such as heat (Caterina et al., 1997), light (Kuhn and Dreyer, 1972), or stretch (Vandorpe et al., 1994) can also be viewed as first messengers if they elicit a specific (i.e., receptor-driven) response.

Second messenger availability

The term “second messenger” is usually applied to a small molecule whose main function is to communicate functional states between intracellular biomolecules. For example, cyclic nucleotides allow transmission of dopamine receptor activation to PKA activation (Otmakhova and Lisman, 1998; Stoof and Kebabian, 1981). Depolarization or hyperpolarization of the membrane can be thought of as special cases of second messenger signaling. Fluctuations in instantaneous polarization are generally triggered by first messengers (neurotransmitters) opening ligand-gated ion channels, but additionally by voltage-gated channels that respond to changes in membrane polarization. Thus, the extent of membrane depolarization in a given spatial domain can be viewed as conceptually equivalent to a given concentration of a second messenger. The advantage of “voltage signaling” over conventional chemical signaling is its unparalleled speed, which is particularly useful for communicating signals rapidly between distant parts of a cell.

Most signaling pathways in eukaryotic cells show at least some degree of compartmentalization. Thus for example, cAMP availability does not provide a single time window, but a family of time windows defined by distinct subcellular locations, corresponding ON/OFF pathways, and downstream effects. In both cultured Aplysia sensory neurons and neurons of the intact lobster stomatogastric ganglion, neuromodulatory stimulation with 5-HT produces a rapid and transient elevation of cAMP primarily in fine neurites. After prolonged stimulation however, cAMP diffuses throughout the dendritic tree and eventually to the cell body (Bacskai et al., 1993; Hempel et al., 1996). This diffusion is significant since cAMP and PKA can have different functions in distinct subcellular compartments (Liu et al., 2004). Hypothetically, the “escape” of cAMP from fine neurites can be explained by synergy of ongoing cAMP production with the saturation of relevant sub-membrane OFF pathways, such as phosphodiesterase activity. Indeed, various PDE isoforms can sculpt cAMP gradients through highly compartmentalized hydrolysis (Houslay, 2010). This example illustrates that “molecular time” defined by second messenger availability and employed for information storage is a product of many competing non-linear influences with their own “temporal agenda”.

State of the target molecule

Many intracellular protein ensembles exist in a state of equilibrium, or stable non-equilibrium (steady state) (Henzler-Wildman and Kern, 2007; Nooren and Thornton, 2003). At the structural level, cytoskeletal filaments, ribosomes, proteasomes and many other quaternary and supramolecular complexes are continuously assembled and disassembled. Metabolic pathways are similarly controlled within a homeostatic system that has been likened to a cellular economy of supply and demand (Hofmeyr, 2008; Hofmeyr and Cornish-Bowden, 2000). Post-translational modifications must be reversible in order to serve a regulatory function. Correspondingly, opposing enzymatic pathways exist for modifications with phosphate, ubiquitin and ubiquitin-like modifiers, methyl and acetyl groups, ADP-ribose, O-GlcNAc and other regulatory modifications occurring in the cytosol and nucleoplasm. Structural and enzymatic interactions are tightly linked and often depend on each other. Together, they represent a vast variety of molecular states that can be used to generate time windows. Many of these states are controlled by the time windows of binding to upstream cofactors and second messengers.

Because post-translational modifications are not encoded in the genome, and because proteins (as opposed to first and second messengers, as well as RNA and DNA) are the primary mediators of biological function, post-translational regulation is ideally suited for implementing experience-dependent changes of cellular function. It is therefore not surprising that most persistent states that underlie intracellular temporal computations are largely defined by the post-translational modifications of proteins and protein ensembles.

Availability of the target molecule

It is well established that various forms of long-term memory and long-term potentiation require RNA and/or protein synthesis for induction and consolidation (Davis and Squire, 1984; Flexner et al., 1963; Martin et al., 1997; Nader et al., 2000). In both cases, the requirement is typically temporally restricted, setting up critical temporal windows during which gene expression or protein synthesis must take place in order for lasting changes in the synaptic state to persist (Bourtchouladze et al., 1998; Montarolo et al., 1986; Pearce et al., 2017).

These translational and transcriptional temporal windows are not easily separated for three primary reasons. First, they are often, although not always, causally and temporally coupled. The onset of transcription typically determines the onset of translation, although additional factors (e.g. transport of RNA, or recruitment of translation machinery) synergistically contribute regulation. Second, both transcription and translation can be regulated at the protein level. For example, degradation by the ubiquitin-proteasome system can control both the availability of proteins and, indirectly, the availability of RNA by targeting transcription factors in the nucleus (Muratani and Tansey, 2003). Finally, RNA and protein synthesis in eukaryotic cells proceeds in distinct subcellular locations and requires highly regulated bidirectional trafficking between the nucleus, soma, and dendrites to ensure functional consequences of both RNA and protein synthesis in neurons (Poon and Jans, 2005).

For these several reasons, it may be more informative to refer to temporal windows of RNA and protein availability at a given subcellular location, taking into account all relevant temporally defined constraints. By definition, availability of a protein is dependent on, but not fully accounted by, the availability of its RNA.

Regulation of availability at a given location can also be achieved by redistribution of pre-existing proteins from other locations. Translation-dependent and translation-independent regulation of local protein availability can converge. For example, local availability of Ca²⁺/calmodulin-dependent protein kinase II (CaMKII) at the postsynaptic density (PSD) increases following high-frequency stimulation (HFS) (Shen and Meyer, 1999). This is achieved immediately by redistribution of existing kinase but is subsequently supported by local protein synthesis (Wu et al., 1998) and subunit exchange (Stratton et al., 2014). Induction of LTP is associated with rapid increases in the spine content of αCaMKII mRNA (Havik et al., 2003). In cultured mouse hippocampal neurons, this was observed within 10 min after HFS (Kao et al., 2010). Additionally, HFS causes redistribution of polyribosomes from the dendritic shafts into spines, observed 2 h after stimulation (Bourne et al., 2007; Ostroff et al., 2002). Thus, both translation-independent and translation-dependent mechanisms of regulating CaMKII availability converge at the PSD with characteristic temporal profiles.

Indeed, the interaction of initial, rapid-onset, protein synthesis-independent time windows of local protein availability with delayed, slow-onset, translationally and transcriptionally controlled temporal windows of global protein and RNA availability is a core mechanism for nuclear-to-cytoplasmic signaling underlying “tag and capture” models for synaptic plasticity (Barco et al., 2002; Bramham and Messaoudi, 2005; Redondo and Morris, 2011). For example, NMDAR calcium transients promote actin polymerization, which recruits microtubule entry into dendritic spines. This has been proposed to allow entry of molecular cargo distributed from the cell body into potentiated spines (Merriam et al., 2013), either constitutively, or as a consequence of previous stimulation. The essence of “tag and capture” is coincidence of a slow global alteration in homeostasis, caused by previous stimulation, with a fast local disturbance, triggered by ongoing activity.

More broadly, any form of neuronal plasticity involves conversion of a transient response (narrow time window) into a more persistent response (wide time window) by means of synergy with other responses, both transient and persistent (Fig. 2). Such conversion is not restricted to a single level, but instead occurs hierarchically, allowing for the progressive generation of more and more persistent responses to recurrent patterns of stimulation. A time window at a given level of the hierarchy can be seen simultaneously as “memory” of levels below and “stimulus” for levels above (Fig. 2). In the next section, we discuss several examples of such hierarchical conversion of time windows.

Synergy of time windows: transient-to-persistent conversion

Memory traces in the nervous system are widely believed to be stored as relative strengths of synaptic connections (Martin et al., 2000). Although debate still exists as to whether synaptic plasticity is sufficient to explain memory encoding and storage (Ryan et al., 2015), most studies support the view that a broader statement is warranted: storage of information in the nervous system is a function of neuronal states. Synaptic plasticity is one example of functionally significant alterations in the states of neurons, whereby changes in two synaptically coupled neurons result in net change in the probability of signal transmission between them. There are however other ways to alter the state of a neuron (with potential effects on information storage and transmission), which are not necessarily confined to specific synapses. For example, many forms of learning produce cell-wide changes in membrane components such as ion pumps and channels, which collectively alter cellular excitability and can profoundly influence the behavior of a neuronal network (Mozzachiodi and Byrne, 2010; Yizhar et al., 2011).

What determines the state of a neuron? At each given moment its state is a product of both currently active outside factors and pre-existing cellular conditions. These in turn are a product of previous genetic, maternal, developmental, epigenetic and/or environmental factors that had influenced the fate of the neuron from embryogenesis to synaptic plasticity in maturity. Aside from genetic and maternal factors, which are shared by all neurons in a mature organism, any other variable in the neuronal state is determined by forces acting from outside of the cell. This includes stimulation by any developmental factors; stimulation by other neurons, including synaptic connectivity; organism-wide signaling by the endocrine system, and many other external signals that elicit a response in a neuron or its precursor. The continuum of development and synaptic plasticity is highlighted by the fact that many mechanisms (e.g., the requirement for growth factor signaling) are shared among these processes (Bonhoeffer, 1996; Patterson and Nawa, 1993; Schinder and Poo, 2000).

Ontogenetic changes in neuronal states and hence synaptic plasticity can therefore be ultimately traced back to extracellular signals that neurons receive. These signals are typically transient, whereas changes that they elicit in the target cell can persist over a wide range of temporal domains. The key feature of temporal analysis by the neuron is therefore its ability to convert transient signals into persistent responses. This is achieved by nested synergistic interactions between time windows, starting from the those set by first messenger availability, followed by intracellular time windows defined by second messengers and target molecules. To better illustrate how such nesting of transient time windows contributes to the generation of persistent responses, we will primarily focus on the relatively well-characterized signaling pathway centered around 5-HT, cAMP, PKA, CREB and C/EBP.

First messenger synergy

We have considered a “first messenger” as any biologically active substance or physical force eliciting a controlled change in the state of a cell. How do first messenger time windows interact with one another? In some cases, the first messenger that triggers a memory is the same first messenger whose responsiveness is altered. Such is the case of typical hippocampal LTP induced by Schaffer collateral tetanization (Dudek and Bear, 1993). In this case, multiple identical rapidly repeated bouts of glutamate elevation in the vicinity of CA1 dendrites alter the response to subsequent repetitions of the same bouts of glutamate. In other cases, one first messenger can influence the responsiveness to other first messengers. For example, during sensitization of the Aplysia sensorimotor synapse, repeated bouts of 5-HT cause long-term facilitation by increasing neurotransmitter release from the presynaptic sensory neuron (Brunelli et al., 1976), a phenomenon called heterosynaptic plasticity. In this case, 5-HT alters the cellular response to mechanical stimulation of the skin, which for a sensory neuron can be seen as equivalent to a distinct first messenger.

These plasticity-inducing effects do not accumulate in a linear additive fashion. Instead, the neuron responds to particular temporal patterns of stimulation. Given sufficient temporal spacing, repeated stimuli are treated as single stimuli since the cell returns to homeostasis before the onset of the repetition. On the other hand, with no spacing at all the stimuli simply combine additively into a single stimulus of increased amplitude. “Massed” stimuli, however, do not generally produce the same effects as spaced stimuli, a difference that is consistently observed on multiple time scales of behavioral learning and its cellular analogs (Kornmeier and Sosic-Vasic, 2012; Smolen et al., 2016). Thus, neuronal learning depends on the specific temporal relationships between the time windows of first messenger availability. In other words, neurons learn by extracting temporal patterns from the history of their stimulation. The pattern can be as simple as high-frequency stimulation. In other cases, more complex temporal relationships between stimuli are extracted. One example is spike time-dependent plasticity (STDP), where the relative timing of a bAP and a synaptic input determines whether and to what extent the synapse will weaken or strengthen (Song et al., 2000). Functionally, STDP “rewards” connections that are “causal” to postsynaptic firing, and “punishes” the ones that are not (Abbott and Nelson, 2000). Since bAPs are themselves ultimately a result of external stimulation, STDP can still be viewed as an interaction between the timing of first messenger time windows.

Interactions between first messenger time windows are the ultimate source of information for a neuron. Yet first messengers themselves do not interact with one another directly. They elicit responses in the cell that occur with a novel temporal profile, typically persisting after the decay of the first messenger. These novel cellular time windows interact with responses elicited by other first messengers, both ongoing and carried over from previous stimulation. The simplest example is non-linear summation of depolarization caused by coincident synaptic inputs (Canepari et al., 2007; Costanzo et al., 1999; Wolf et al., 1998). Neurotransmitter binding throughout the dendritic arbor produces EPSPs and IPSPs which interact in a distinct temporal manner depending on the biophysical properties of the dendrite such as resistance, capacitance and the presence of voltage-gated currents (Curtis and Eccles, 1960; Davies and Collingridge, 1996; Ferster and Jagadeesh, 1992). Neurotransmitter timing windows therefore converge to produce emergent time windows of membrane depolarization at a given location. A similar example is provided by the interaction between depolarization and NMDAR activation/calcium influx, described above. To generalize, the most basic first messenger integration occurs at the level of second messengers, including depolarization as a special case.

Second messenger synergy

Second messengers represent a fundamental point of convergence for signaling by many distinct extracellular ligands. A striking example is olfaction. Mammalian genomes contain ~500–1000 olfactory receptor genes, subject to extensive genetic recombination (Buck, 1996; Hasin-Brumshtein et al., 2009). In any given organism this genetic diversity results in a unique repertoire of olfactory GPCRs. All of them however converge on the same second messenger: cAMP.

Second messengers therefore allow neurons to extract the most basic, rapid patterns of external stimulation and “generalize” the actions of multiple first messengers. Ions like calcium and small molecules like cyclic nucleotides provide the cell with a “temporal toolkit” of homeostatically controlled, extensively regulated, highly convergent and highly divergent regulatory nodes. Time windows of second messengers, produced in response to patterns of first messenger availability, elicit changes in the functional state of downstream proteins, also with characteristic temporal profiles often distinct from the second messenger time windows. These novel protein state windows interact with responses elicited by both first and second messengers. For example, Ca²⁺-dependent adenylyl cyclases display distinct temporal requirements for activation (Shobe et al., 2009; Yovell and Abrams, 1992). These enzymes are activated by the coincidence of a calcium time window with a neurotransmitter time window. In Aplysia, pairing of 5-HT and Ca²⁺ causes potentiation of cyclase activity if the Ca²⁺ influx immediately precedes 5-HT binding (Fig. 3b), but not if it follows the 5HT signal (Fig. 3a). FFiring of different presynaptic cells (1) determines the timing of 5-HT and glutamate release (2). The time window of 5-HT availability determines the onset of G-protein activation, which in turn triggers the activation of adenylyl cyclases (4) and thus the time window of increased production of cAMP (5). Glutamate bouts trigger time windows of depolarization (6, see also Section IA), which upon coincidence with ongoing glutamate bouts cause Ca²⁺ influx via NMDA receptors (7). This contributes to the broadening of EPSPs (widening of the depolarization time window) (8) and to the onset of a “receptive state” of Ca²⁺-dependent adenylyl cyclases (9). In order to contribute to cAMP production, this state must coincide with subsequent 5-HT-mediated activation (10). Thus temporal interactions between bouts of first messenger (glutamate) lead to a time window of a second messenger (Ca²⁺), which leads to a time window of a “receptive state” of an enzyme, which then interacts with a time window produced by another first messenger (5-HT) to produce distinct time windows of another second messenger (cAMP). Such “nesting” of time windows allows for progressive extraction and integration of patterns of increasing scale. This specificity of downstream response to a particular order of upstream events resembles the aforementioned “causality extraction” observed in STDP (Abrams et al., 1991; Yovell and Abrams, 1992).

Fig. 3. Nesting of time windows in the regulation of Ca²⁺-dependent adenylyl cyclases.

The timing of presynaptic firing (1) determines the onset of transmitter availability (2). For 5-HT, this determines the timing of G-protein activation (3) and consequently, the basal activation of ACs (4), which generate a time window of elevated cAMP (5). Multiple successive glutamate bouts produce relatively sustained depolarization (6), which, by synergizing with ongoing glutamate availability, results in Ca²⁺ influx via NMDARs (7). This leads to increased postsynaptic depolarization in response to glutamate availability (8) and to enhanced activity of Ca²⁺-dependent ACs (9). (a) When 5-HT precedes glutamate stimulation, enhanced activity of ACs does not coincide with G-protein activation. (b) When Ca²⁺ influx precedes 5-HT availability, the activation of ACs by G-proteins and enhancement by Ca²⁺ coincide (10), resulting in an extended time window of cAMP availability.

Synergy of target molecules: enzymatic interactions

Ultimately, patterns of first and second messenger fluctuations translate into fluctuations of protein states throughout the neuron. Typically, these higher-order fluctuations, “protein state time windows”, persist for a longer time, that is, beyond the time windows of first and second messengers. Biochemically, this persistence can be achieved by two means: (1) feedback loops leading to sustained production of the first/second messenger, or (2) autonomous states, whereby a protein becomes independent of the activating factor. We will discuss each in turn.

First-/second-messenger feedback loops

Persistent neuronal responses can be created by feedback loops involving first messengers. Such responses necessarily involve secretion of the first messenger into the extracellular space, and transmembrane signaling back into the cell. Such autocrine signaling by a secreted messenger can be linked to the production and secretion of that messenger, creating a form of positive or negative feedback. For example, in embryonic hippocampal neurons BDNF serves as a self-amplifying autocrine factor (Cheng et al., 2011). BDNF elevates cAMP and PKA activity, which leads to increased secretion of BDNF and membrane insertion of its receptor TrkB. Exogenous BDNF has also shown to induce dendritic accumulation of BDNF and TrkB mRNAs (Righi et al., 2000), suggesting the existence of a positive feedback loop acting through sustained release of a self-acting first messenger.

Alternatively, persistent feedback loops can be generated by sustained production of second, rather than first messengers. For example, network modeling suggests that PKC is initially activated by Ca²⁺ and DAG. PKC then indirectly stimulates MAPK, which activates cPLA2, which causes production of arachidonic acid, which activates PKC at the basal level of DAG (Bhalla and Iyengar, 1999). Multiple modes of stimulating PKC are therefore essential for the feedback loop, but ultimately the extended time window of PKC activity is achieved through sustained production of a second messenger.

Autonomy or kinases or kinase ensembles

cAMP/PKA

As discussed in section II, signaling through cAMP and protein kinase A provides a well-characterized example of temporal hierarchy employed in synaptic plasticity (Kandel, 2012). We will now consider this pathway in more detail to illustrate consolidation of transient cellular responses into persistent changes.

PKA consists of two catalytic subunits (C) and two inhibitory regulatory subunits (R) that dissociate upon binding of cAMP (Fig. 4, 2). cAMP (1) is produced by adenylyl cyclase, for example following stimulation of neurons with 5-HT or other neuromodulators such as BDNF in the examples above. This generates a short-term (minutes) time window for PKA activity (Hempel et al., 1996), which is closed by phosphodiesterases that hydrolyze cAMP and allow re-binding of R subunits to the C subunits (3). It is well-established that upon prolonged stimulation, catalytic subunits of PKA translocate to the nucleus, where they phosphorylate CREB1 (cAMP response element-binding protein 1) (4), a transcription factor triggering a vast array of nuclear responses to stimulation (see below). The PKA holoenzyme and the R subunit of PKA are both excluded from the nucleus (Bacskai et al., 1993).

Fig. 4. Persistent PKA activation by positive feedback.

The time window of cAMP availability (1) determines the time window of PKA in a dissociated state (2) (both R and C subunits available in a free form). PKA is deactivated by reassociation of the subunits (3). Availability of free C determines the timing of phosphorylation of its substrates, including CREB-1 (4). This results in enhanced proteasomal activity, either by transcriptional induction of 26S activators such as ApUch (7, 8), or directly by PKA-mediated phosphorylation of the proteasome (9). These time windows of increased proteasomal activity shorten the time of R availability (5) and consequently extend the time window of C availability (6).

Separately, upon prolonged stimulation of sensory neurons with 5-HT, the R subunits are degraded via the ubiquitin-proteasome system (Bingol and Sheng, 2011; Chain et al., 1999; Hegde et al., 1993) (5), which results in prolonged, cAMP-independent activation of PKA (Chain et al., 1999) (6). Both binding to cAMP and dissociation of the holoenzyme are necessary but not sufficient for R subunit degradation. According to a long-standing model, initial dissociation of the holoenzyme requiring the presence of perinuclear cAMP causes CREB1-dependent transcription of many genes, including, in Aplysia neurons, ApUch (7), a deubiquitinating enzyme (DUB) required for proteasomal function which may increase its ability to process ubiquitinated substrates (Hegde et al., 1997) (8). This increases cAMP-dependent degradation of the R subunit and prolongs the activation of PKA in the absence of cAMP (Chain et al., 1999). Although the effect of proteasome-bound DUBs on their overall activity is controversial (Lee et al., 2011; Peth et al., 2009; Peth et al., 2013), other mechanisms may exist that link PKA activity with increased proteasomal function, including the newly discovered route for direct activation of 26S by PKA phosphorylation (Lokireddy et al., 2015) (9).

In either case, the regulation of downstream signaling by PKA involves multiple levels of synergy, whereby a pattern of transient perturbations produces a more lasting change. First, cAMP itself, and specifically perinuclear cAMP acts as a temporal integrator of receptor stimulation, with the ON mechanisms involving the rates of production and the kinetics of diffusion towards the nucleus, and the OFF mechanisms mediated by phosphodiesterases. The resultant time window of increased perinuclear cAMP acts as an ON signal for several other time windows. PKA activation/dissociation/susceptibility to C nuclear translocation and R degradation depend on the cAMP window most directly – that is, both the onset and decay of an altered PKA state are temporally controlled by local cAMP concentrations. Indeed, the extent of eventual translocation is proportional to the peak kinase activity in the vicinity of the nucleus (Bacskai et al., 1993). CREB1 activation as a time window integrates multiple ON inputs and independently controlled OFF inputs (such as dephosphorylation) and thus is temporally dissociable from cAMP elevation (Liu and Graybiel, 1996). Similarly, the dynamics of elevated proteasome activity are linked to but not exclusively controlled by cAMP levels (Upadhya et al., 2006).

The prolonged activation of PKA must therefore integrate multiple partially independent time windows. The increased activity of the proteasome resulting from a previous increase in cAMP synergizes with the current increase in cAMP needed to dissociate the PKA holoenzyme and make the R-subunit available for ubiquitination and proteolysis. This results in a novel, emergent time window – that of a locally decreased R:C ratio, with the direct effect of autonomous PKA activity. The window is probably closed by synthesis of new R subunits.

CaMKII

Many other kinases or kinase ensembles demonstrate conceptually similar mechanisms of establishing persistent states based on transient signals. Perhaps the richest example of second-messenger-induced autonomy is provided by CaMKII, a critical factor in long-term plasticity. This kinase is part of a broader CaMKII/calcineurin system, a powerful temporal integrator intricately tuned to the patterns of transient Ca²⁺ influx into the cell. Both CaMKII and calcineurin are activated by Ca²⁺/calmodulin (CaM), but while CaMKII tends to positively regulate synaptic strength (Malinow et al., 1989; Pi et al., 2010), calcineurin typically opposes LTP and is involved in LTD (Mulkey et al., 1994; Winder et al., 1998). While calciumindependent mechanisms for CaMKII activation exist (Erickson et al., 2008; Jalan-Sakrikar et al., 2012), canonically, the timing of incoming calcium spikes is critical in determining the cumulative state of the enzyme. Through self-activation and self-inactivation dynamics discussed in detail elsewhere (Coultrap and Bayer, 2012) (Lisman and Zhabotinsky, 2001) (Li et al., 2012), CaMKII exhibits related, but partially independent time windows of altered total activity, altered autonomy, and altered responsiveness to Ca²⁺/CaM. At each given moment, each of these time windows interacts with the current concentration of intracellular calcium and “updates” itself. For example, coincidence of ongoing Ca²⁺ input with already present enhanced responsiveness of CaMKII to Ca²⁺/CaM leads synergistically to the prolongation of the enzyme’s enhanced activity and autonomy via autophosphorylation at T286 and translocation of the activated enzyme to the PSD (Coultrap and Bayer, 2012). Thus during high-frequency stimulation, total activity and total autonomy of CaMKII gradually rise and extend in time, because CaMKII’s responsiveness to Ca²⁺/CaM remains high. Consequently, high-frequency stimulation leads to LTP. On the contrary, during low-frequency stimulation, responsiveness of CaMKII to Ca²⁺/CaM is reduced via inhibitory autophosphorylation at T305/306, which blocks further increases in autonomy and activity (Coultrap and Bayer, 2012). At the same time, calcineurin’s responsiveness to Ca²⁺/CaM remains unaltered, which leads to a increase in the corresponding OFF input relative to the ON pathways for CaMKII. As a result, total activity of the kinase is reduced in response to further stimulation by Ca²⁺, which promotes LTD (Li et al., 2012).

PKC/PKM

A distinct mechanism for persistent activation has been demonstrated for atypical (DAG and Ca²⁺-independent) PKC. In Aplysia, persistent activation of PKC is achieved through calpain-mediated cleavage of a regulatory domain (Bougie et al., 2009; Sutton et al., 2004). This was observed during intermediate-term activity-dependent facilitation of sensorimotor synapses, which involves synaptic potentiation in neurons that receive neuromodulatory input (e.g., 5-HT) coincident with their own activation. In cells that receive 5-HT stimulation alone, only transient activation of PKC occurred through the activity of phospholipase C-coupled receptors. However, neuronal activation in combination with the 5-HT stimulation additionally promoted calcium influx, which activated calpains that cleaved the transiently activated PKC to yield constitutively active product termed PKM. Thus, in Aplysia, the prolonged availability of PKM is achieved through limited proteolysis, which is an emergent result of coincidence between transient events: 5-HT input and neuronal activation.

In vertebrates, some atypical PKMs can be synthesized locally at the synapse from separate transcripts without the regulatory domain which normally inhibits basal PKC activity (Bougie et al., 2009; Sacktor, 2012). In this case, instead of regulating autonomy of an enzyme, cells regulate production of an autonomous enzyme. However, temporal synergy of upstream signals remains an essential feature of this evolutionarily novel form of autonomy. Initial induction of PKMζ, one of the isoforms of PKM expressed in neurons, involves at least two signals that must be integrated: (i) local inactivation of Pin1, an inhibitor of translation (Hernandez et al., 2003; Kelly et al., 2007a; Kelly et al., 2007b), and (ii) ongoing phosphoinositide signaling, which stimulates PDK1 required to activate newly synthesized PKMζ (Kelly et al., 2007a; Yao et al., 2013). Such coincidence creates an emergent time window of PKMζ activity. Critically, this time window can persist beyond the lifetime of individual PKMζ molecules, potentially lasting for months. This is explained at least in part by the fact that both the synthesis and phosphorylation of nascent PKMζ are stimulated by active PKMζ itself, ensuring positive feedback (Westmark et al., 2010). This potential for extended autonomy, as well as localized nature of PKMζ induction, has been used to explain its role in LTP maintenance (Jalil et al., 2015). Inhibition of PKMζ, an isoform of PKM displaying such translational regulation, is hypothesized to lead to erasure of long-term memory traces (Ling et al., 2002; Shema et al., 2007), although a related subtype of PKM, PKMi/A, may contribute to these effects (Volk et al., 2013).

It is particularly striking that while the mechanism for generating a persistently active PKM from atypical PKC has shifted in the course of evolution from regulated proteolysis to a multi-protein translational and/or autophosphorylating feedback loop, the role of the extended PKM time window in memory maintenance remained essentially the same. Functionally, PKMs promote synaptic facilitation by recruiting AMPARs to the PSD (Migues et al., 2010) and increasing AMPAR-mediated currents (Ling et al., 2002), as do many other regulatory factors (Anggono and Huganir, 2012). Ancestral PKC may have been initially specialized for establishing and maintaining apical compartments in polarized cells (Jalil et al., 2015). This initial function in distributing membrane proteins to specific locations at the surface of the cell may have been adopted in metazoans for synapse development and glutamate receptor redistribution. This is supported by experimental evidence for PKMζ involvement in synapse maturation (Liu et al., 2009; Yoshii et al., 2011). Further selective pressure may have favored various activity-dependent mechanisms capable of producing sustained activation of PKMζ. The different mechanisms for PKM production observed in invertebrates and vertebrates may therefore both be a consequence of the same evolutionary drive favoring a regulated temporal link between transient extracellular signaling and sustained cellular response, and acting on a protein initially specialized for maintenance of spatial specificity, a definitive feature of signal processing by neurons.

Sustained kinase activation resistant to protein turnover is not limited to PKMs. It is a relatively common feature of signaling networks involving a catalytic cycle. As discussed above, autonomy of CaMKII after its initial induction is sustained by increased local protein synthesis (Wu et al., 1998) and subunit exchange (Stratton et al., 2014), paralleling translational feedback observed for PKMs. Similarly, computational simulations of the MAPK pathway, which is required for LTP (English and Sweatt, 1997), demonstrate a capacity for persistent activity and, to a degree, resistance to external dephosphorylation and turnover (Bhalla and Iyengar, 1999; Smolen et al., 2008). These simulations were carried out by isolating the MAPK cascade from its numerous partners within the interactive signaling network (Kopec and Carew, 2013), many of which can clearly contribute to the onset and decay of sustained catalytic cycles, as in the case of self-sustaining cross-talk between PKC and MAPK (Bhalla and Iyengar, 1999).

To sum up, while protein half-lives typically range from minutes to days, protein states can persist for much longer. This is realized using dynamic molecular systems incorporating stable interactions, post-translational modifications, and/or synthesis of new proteins that assume the state of proteins undergoing turnover. Such self-stabilizing feed-forward loops are typical in synaptic plasticity. On the one hand, they are often inducible by fast and transient stimuli and can therefore act to integrate upstream events into a more persistent state that remains after the decay of its trigger. On the other hand, these persistent states are rarely truly stable, i.e. irreversible. Even if the modification itself is irreversible, its “inactivation” can generally be achieved by turnover, and even if a dynamic system is resistant to turnover, it can be returned to homeostasis by an increase in respective OFF forces such as rates of dephosphorylation or protein degradation. Thus, “persistent states” are in fact extended temporal windows with distinct dynamics of activation and inactivation. Epigenetic remodeling is an extreme example of a temporal window that can persist autonomously (i.e. in the absence of the original stimulus) for a lifetime and even into the subsequent generations.

Synergy of target molecules: structural interactions and transport

The examples described so far amount to lasting alterations in “biochemical state” of pre-existing proteins. Aside from “biochemical” changes (e.g. post-translational modifications, cofactor binding, limited proteolysis etc.) changes in the protein state can involve structural rearrangements (e.g. recruitment into supramolecular assemblies, polymerization, conformation change). The distinction is not a strict one. “Structural states” can occur concurrently with “biochemical states” and serve a similar regulatory function. Biochemical states can also trigger structural states, and vice versa. In both cases, a transient change can lead to the onset or decay of a more lasting change.

Supramolecular complexes

A striking example of transient signals triggering a lasting structural change is provided by the prion-like properties of CPEB (cytoplasmic polyadenylation element-binding protein), which contributes to the maintenance of long-term facilitation (Si et al., 2003a; Si et al., 2003b). CPEB acts as a regulator of local protein synthesis at axon terminals. In the basal state CPEB exists in a largely inactive conformation as a soluble monomer. Signaling downstream of 5-HT and PI3 kinase converts some copies of CPEB to an active conformation that forms aggregates. These aggregates act as prions by recruiting additional monomers and converting them to the activated state. Activated aggregates recruit the poly(A) polymerase machinery, allowing activation of previously dormant local mRNAs including the mRNA for CPEB itself. Newly synthesized CPEB proteins continue to sustain translation by assuming the active conformation long after the initial inducing stimulus decays. A “structural cycle” arises, bearing striking resemblance to the “biochemical” cycle of PKMζ or CaMKII autonomy. In both cases, an initial stimulus produces a translation-dependent, lasting, self-perpetuating state that can transcend degradation of individual components.

A special case of lasting “structural time windows” is cytoskeleton rearrangements. Cytoskeleton dynamics underlie morphological plasticity (e.g., changes in the size, shape, or quantity of synapses or neuronal processes) (Schubert and Dotti, 2007), transport of plasticity-related molecules between different compartments of the cell (Bramham and Wells, 2007; Kapitein and Hoogenraad, 2011), and regulate many essential synaptic factors such as CaMKII (Okamoto et al., 2007) or PKMζ (Kelly et al., 2007b).

Cytoskeleton assembly is typically dynamically regulated. Continuous assembly and disassembly of actin and tubulin polymers constitute the ON and OFF mechanisms for a particular state of cytoskeletal filaments at a particular cellular location. Plasticity-inducing stimuli can contribute to these mechanisms by directly modifying the dynamics of cytoskeleton assembly, as in the case of NMDAR-dependent polymerization of actin at dendritic spines (Saneyoshi and Hayashi, 2012). Similarly, microtubule assembly and invasion into potentiated spines can be triggered by signaling from the synapse, as discussed below (Merriam et al., 2013)

Association of proteins with already present cytoskeletal filaments is also subject to dynamic regulation. Binding to or disassociating from actin or tubulin polymers can regulate the effector protein’s downstream activity, as in the case of CaMKII, which is released from actin filaments to be redistributed into the PSD during LTP induction (Okamoto et al., 2007).

Subcellular transport

A critical feature of synaptic plasticity is its specificity to individual synapses. Local changes in synaptic state are not simply a generalized cellular response, but a result of coordination between the synapse, which retains positional information, and the rest of the cell, which provides the synapse with so-called plasticity-related products (PRPs) (Redondo and Morris, 2011). More generally, an essential feature of neuronal plasticity is coordinated transport of cargo between different compartments of the cell.

Such transport can occur in the absence of cytoskeletal binding, i.e. by diffusion, as in the cases of nuclear translocation of second messengers, regulatory subunit of PKA, or, to some extent, CaM (Deisseroth et al., 1998; Mermelstein et al., 2001). Most forms of transport, however, appear to be facilitated by the cytoskeleton and specific motor proteins such as dynein, kinesin or myosin. In either case, translocation of molecules to a given location can be seen within our conceptual framework as a time window of their availability at that location. This time window, in turn, is a result of upstream temporal integration.

Cytoskeleton-mediated transport

Subcellular transport typically involves coincidence of multiple temporally limited states at the “sending” and “receiving” ends of the transport route. In fact, the very notion of “transport” may be confusing since it evokes the notion of a single goal-directed action, whereas in reality it is a synergistic result of multiple, independently controlled processes. In principle, cytoskeleton-dependent transport of cargo can be regulated at multiple levels: availability and activation of appropriate motors (Goldstein and Yang, 2000), availability and activation of appropriately directed filaments (Goldstein and Yang, 2000; Hammond et al., 2010; Konishi and Setou, 2009; Sirajuddin et al., 2014; Yu et al., 2000), motor-cargo interactions (Morfini et al., 2002; Sato-Yoshitake et al., 1992; Wang et al., 2008) or motor-filament interactions (Guillaud et al., 2008; Yin et al., 2012) (Wang and Schwarz, 2009). In reality, many constitutive and inducible pathways probably cooperate to ensure directionally, temporally and cargo-specific transport of RNA and proteins throughout the neuron.

Temporal pattern extraction is embedded in the functioning of this form of transport since it requires favorable coincidence of multiple time windows of availability and state independently of the mode of regulation. These processes collectively alter the availability of organelles, vesicles, proteins or RNA molecules at given subcellular locations. For example, synaptic tag and capture is a form of PRP transport from soma to dendrite, which involves at least two independent processes: an increase in availability of PRPs in the dendritic shaft, and an increase in their “unloading” at the “tagged” synapse. The former is further controlled by the availability of PRPs at the soma, and by the states of locally available motor proteins and cytoskeleton filaments. All of these phenomena are regulated by the timing of neuronal stimulation. Therefore, synaptic tag and capture can be seen as a lasting cell-wide response to a distinct temporal pattern of transient events occurring at various subcellular locations.

Diffusion

In the case of diffusion, rates of diffusion to and from particular cellular locations determine the time windows for protein availability, which can integrate with the timing of ongoing stimulation to produce synergistic responses. In cultured hippocampal neurons, CaM is rapidly (~15 s) translocated to the nucleus following a brief depolarizing stimulus (Mermelstein et al., 2001). However, the decay of nuclear CaM occurs at a much slower time scale (tens of minutes to hours), probably owing to the kinetics of CaM transport in and out of the nucleus. Accordingly, an initial weak stimulus that did not produce significant CREB1 phosphorylation but caused CaM nuclear translocation, markedly increased the phosphorylation of CREB1 15 s after a second weak stimulus delivered 45 s after the first. This increased phosphorylation was not attributable to augumented Ca²⁺ influx and did not require further CaM translocation. Given that downstream nuclear signaling by CaMKK and CaMKIV steeply depends on CaM, which is likely to be limiting, such fast “priming” was probably a result of an overlap between a time window of increased nuclear CaM retained from the initial stimulus, and a time window of Ca²⁺ influx from the second stimulus. Disruption of microtubules or actin filaments did not prevent nuclear translocation of CaM, suggesting simple diffusion as the transport mechanism. However, it probably occurs in complex with other proteins (Thorogate and Torok, 2004), since CaM does not contain nuclear localization motifs and would be expected to be strongly buffered in the cytosol.

Higher-order synergy and cellular states

In the previous sections, we have addressed the conversion by neurons of transient external stimuli into persistent states of pre-existing cellular proteins, achieved through biochemical and structural means including subcellular redistribution. A hallmark of long-term memory, however, is its reliance on new protein and RNA synthesis. It is therefore dependent on translational, transcriptional, and epigenetic time windows, which we and others collectively term a “cellular state” (Burrill and Silver, 2010; Levenson and Sweatt, 2006; Marshall and Bredy, 2016). Cellular states are reducible to the history of individual molecule states in the same way that the states of individual molecules are reducible to the history of external stimulation. Cellular time windows, or time-restricted fluctuations of cellular state, are initiated by relatively transient signaling, e.g. post-translational modifications of transcription factors or histones, translocation of effectors into the nucleus, or assembly of essential functional complexes such as ribosomes. Production of new RNA and proteins is therefore a persistent result of a particular configuration of transient upstream protein states.

Synergy of transcriptional cellular states

Typically, regulation at the transcriptional level is highly convergent and highly divergent. For example, CREB1, a central transcriptional regulator of neuronal plasticity, differentiation and survival, integrates a wide range of intracellular signals and produces an equally varied multitude of cellular responses (Lonze and Ginty, 2002). CREB1 can be phosphorylated at a critical Ser-133 residue by a variety of kinases including PKA, Akt, PKC, PKG, ERK, and CaMKII. These activating pathways can interact additively or synergistically (Johannessen et al., 2004). An additional mechanism for transcriptional synergy is provided by transducer of regulated CREB activity 1 (TORC1) which is activated by coincidence of Ca²⁺- and cAMP-dependent signals but not by either of the stimuli alone, translocates to the nucleus and activates CREB1 in a Ser-133-independent manner (Kovacs et al., 2007). Therefore, activation of CREB1 integrates multiple upstream, transcription-independent intracellular time windows, and transcription of CRE-containing genes is an emergent response to such integration.

Genes whose transcription is induced by pre-existing transcription factors in response to external stimulation are termed immediate-early genes. The products of these genes are typically induced rapidly and transiently, which would seem to challenge their role in establishing a lasting response to fast stimuli. However, many of the immediate-early gene products are themselves transcription factors, which upon their synthesis become available for further temporal integration. For example, CCAAT enhancer-binding protein (C/EBP) (Fig. 5) is transcription factor encoded by an immediate-early gene and controlled by CREB1 (Alberini et al., 1994). In hippocampal neurons, DNA-binding activities of C/EBP isoforms β and δ are enhanced by increased cAMP or Ca²⁺ signals (Yukawa et al., 1998) (Fig. 5, 1). Interestingly, CaMKIV activated by the Ca²⁺ signal induces expression of C/EBP members, but also directly enhances C/EBP-dependent gene transcription of late-response genes (Yukawa et al., 1998) (2). Therefore, C/EBP represents a nested system for temporal integration (Fig. 5). The time window of C/EBP availability is driven at the ON end by CREB1 activation (3), which itself integrates many upstream signaling pathways (1). However, the time window of C/EBP product availability is driven by the synergistic interaction between the timing of C/EBP availability and ongoing stimulation by second messengers and kinases.

Fig. 5. Integration of multiple temporal scales within a time window.

C/EBP is a immediate- early transcription factor whose expression is stimulated by CREB-1 (3), which integrates many signaling pathways including those promoted by cAMP and Ca²⁺ (1). CaMKIV, a nuclear kinase stimulated by Ca²⁺ signaling cascades, can directly activate C/EBP (2), promoting late-response gene expression. Thus, C/EBP action is dependent on Ca²⁺ signaling at multiple temporal domains.

The dichotomy of immediate-early and late-response genes likely represents the tip of the iceberg in the hierarchical “nesting” of transcriptional control. As illustrated by C/EBP, nested transcriptional time windows, in addition to cascading onto each other, can additionally share regulatory mechanisms, like cAMP or Ca²⁺ signals. This suggests a flexible mode of regulation, whereby entire cascades can be simultaneously altered at multiple levels. “Transcriptional nesting” is further convoluted by positive feedback, as in the case of Aplysia CREB1, which binds to the promoter of its own gene sustaining its transcription (Liu et al., 2008). In this case, the time window of increased CREB expression serves as its own ON input.

An interesting elaboration of this nested system is employed in self-sustaining transcriptional loops such as those underlying circadian rhythmicity. For example, in mammalian tissues the transcription factors CLOCK-BMAL1 positively regulate the expression of the so-called clock- controlled genes (CCGs) (Gallego and Virshup, 2007). These include genes encoding the proteins CRY and PER, which are produced during each circadian cycle and act to repress CLOCK-BMAL1-mediated transcription including their own. Because of that, daily CRY and PER accumulation in the cell is self-limiting. A delayed mechanism involving phosphorylation, ubiquitination and degradation then reduces CRY and PER protein levels (Eide et al., 2005; Yoo et al., 2013), re-enabling their transcription and a new circadian cycle. Thus, a time window of decreased CRY and PER proteins (time window A) contributes to the ON pathway of increased CCG expression (time window B), which contributes the OFF pathway for time window A by resynthesizing CRY and PER. Instead of a clear hierarchy of upstream and downstream time windows, in this case the interactions between levels of temporal analysis nest into one another.

Negative feedback in general is typical for many transcriptional networks. The distinct and functionally critical feature of the CLOCK-BMAL1-CRY-PER system is the delay between time windows B and A, i.e. the delay in CRY/PER degradation (Gallego and Virshup, 2007). If there were no delay, the reciprocal ON and OFF pathways would equilibrate, and no oscillations would be observed in an unperturbed system, as is the case with typical transcriptional networks regulated by negative feedback. However, the delay in the degradation of CRY/PER (that is, the opening of time window A) leads to a delay in the expression of CCGs (the opening of time window B), which leads to a further delay in resynthesizing CRY and PER (the closing of time window A) and consequently, a delay in inhibition of CCG transcription (the closing of time window B). Equilibrium is thus never reached, resulting in bidirectional circadian “swings”. This core circuit is self-sustaining, however many external inputs can contribute to the onset or decay of the corresponding time windows and thus influence circadian rhythmicity. For example, melatonin, by inhibiting the proteasome, can interfere with CRY/PER degradation (Vriend and Reiter, 2014, 2015), thus prolonging the period of the cycle.

In summary, transcriptional states are not merely a single level, but rather a class of levels in the hierarchy of time windows defined by ongoing gene expression. Levels within this class can interact through nesting, whereby downstream transcriptional time windows of increasing persistence are supported by upstream, typically more transient transcriptional time windows. The specific hierarchy of downstream and upstream time windows, however, can be difficult to define, as illustrated by the circadian rhythm example.

Synergy of epigenetic cellular states

An extreme example of protein state persistence leading to an emergent cellular state is histone modification. Even though acetylation and methylation of individual histones are relatively transient, and histones themselves are turned over within days, the epigenetic landscape as a whole can persist for years and in some cases be inherited (Barth and Imhof, 2010; VerMilyea et al., 2009). The contribution of these lasting, potentially lifelong epigenetic changes to memory formation and maintenance has been first recognized about a decade ago (Chwang et al., 2006; Levenson et al., 2004). The specifics of experience-dependent epigenetic remodeling, e.g. the exact ON and OFF pathways that contribute to epigenetic time windows, remain largely unknown. Globally, however, histone modification in postmitotic neurons has been shown to be triggered by neuronal activity and required for certain forms of learning and memory across tasks and brain regions (Zovkic et al., 2013). For example, distinct H3K9me2 and H3K4me3 patterns were induced in the hippocampus and entorhinal cortex following contextual fear conditioning, and the inhibition of H3K9me2 in the entorhinal cortex, but not in the hippocampus, enhanced memory formation (Gupta-Agarwal et al., 2012). DNA methylation has also been implicated in the regulation of transcription of critical memory-related genes. For example, methyl-CpG binding protein 2, which in the hypothalamus influences transcription of thousands of genes, was shown to associate with CREB1 at the activated, but not at repressed targets of this transcription factor (Chahrour et al., 2008).

Even before the discovery of activity-dependent histone modifications, epigenetic changes have been proposed as an attractive mechanism for memory storage (Crick, 1984; Holliday, 1999). Within the repertoire of molecular time windows available to neurons, epigenetic states are probably the longest lasting. However, as in the case with other time windows, epigenetic time windows depend on the balance between ON and OFF pathways which can be independently regulated (Zovkic et al., 2013). Multiple signaling cascades, most notably MAPK/ERK (Chwang et al., 2006), have been shown to influence histone modification during memory formation (Zovkic et al., 2013). ERK involvement in epigenetic remodeling is perhaps not surprising given the central role of this signaling node in relatively slow, long-lasting, low-sensitivity integration of many signaling cascades including signaling by growth factors and other developmental and organism-wide transducers (Wu et al., 2001). ERK activation typically requires stronger or longer-lasting stimulation than the rapid activation of kinases such as PKA or PKC, but once engaged, the ERK system is well-suited for generating self-sustaining responses (Smolen et al., 2008).

As in the case of other time windows we have described, epigenetic states feed back onto lower levels of temporal analysis, i.e. influence the ways in which the upstream, more transient time windows interact. For instance, fear conditioning leads to increased methylation and decreased expression of the PP1 gene in the hippocampus 1 h after training (Miller and Sweatt, 2007), and increased methylation and decreased expression of the cortical calcineurin gene 30 days after training (Miller and Sweatt, 2007). Since both phosphatases are widely utilized as OFF inputs for LTP-induced time windows (such as CaMKII phosphorylation described above), epigenetic time windows can contribute long-term OFF influences to the processing of incoming stimulation at multiple levels. ON inputs can be similarly regulated, as evidenced for example by the epigenetic suppression of CREB2 in response to 5-HT in the Aplysia nervous system. CREB2 is a memory suppressor which inhibits CREBI-regulated, immediate-early memory enhancer genes. Thus, the time window of CREB2 promotor methylation, which peaks between 12 and 24 h of 5-HT application, contributes an ON input to the time window of CREB1 target expression.

In summary, although epigenetic influences on information processing are only beginning to be understood, available evidence firmly places epigenetic states within the nested hierarchy of neuronal time windows with its characteristic feedback onto lower levels of temporal analysis.

Conclusions

The biological utility of memory stems from its ability to modify future behavior based on past experience. This requires storage of information. In this review, we have advanced a model that views information processing and storage in neurons as a series of temporal analyses within a dynamic, hierarchical system of cellular perturbations, which we conceptually consider as “time windows”.

The timing of physiological phenomena, especially those occurring at the micro- and nanoscales, can be notoriously difficult to address experimentally. In recent decades, a great deal of progress has been made in the development of real-time techniques such as live-cell imaging and single-molecule tracking, but overall our knowledge stems from experimental evidence with extremely poor temporal resolution. Such commonly employed techniques as inhibitor treatments, overexpression of recombinant enzymes, or fixed-cell imaging all “collapse” many biological effects occurring on various time scales into a single state which is then assayed in a realistic time frame. A notable exception is electrophysiology, with its relatively easy access to the timing of millisecond-scale events. Indeed the high resolution of electrical measurements has provided neuroscience with a temporal toolkit unique in biology. Yet changes in cell membrane potential – the source of all electrophysiological data – are merely one of many perturbations in homeostasis, many of them equally fast, that continuously occur in the cell and in the organism as a whole. The timing of each of them may be just as critical as frequencies of spikes.

Storage of information is storage of relationships between objects or events. Real-world experience is an infinite-parameter space containing too much potential information to realistically process and store. In biological systems, the particularities of experience – that is, interaction with the outside world – must be first abstracted into a manageable number of variables that could be physically represented as an object or state inside the body. For example, the auditory system receives as input the aggregate vibration of air, but must perform a form of Fourier transform before sound information can be processed and stored. The aggregate vibration is split into bins of individual frequencies to which various populations of cells respond specifically. These individual frequencies are not in fact physically observable in the vibrating air. They are an approximation of a real-world phenomenon as a sum of simple, manageable stimuli with various temporal properties. The same could be said about experience as a whole. The organism, by interacting with the environment, “recodes” the features of this interaction as a combination of many homeostatic disturbances operating at various rates. Hormones and growth factors are typically produced in response to slow, global alterations in organismal state such as chronic stress or changes in dietary patterns. Neurotransmitters are released by neurons and absorbed at an extremely fast rate, which allows to encode fast homeostatic perturbations as rates of neuronal firing.

When these signals are received by a given target cell, they trigger cascades of perturbations which follow the corresponding time scales: hormones and growth factors induce slow neuronal changes, while neurotransmitters cause fast disturbances. Thus, the “holistic”, infinite-parameter experience is parsed by the cell, and cumulatively the organism, into a range of manageable variables and time scales, which collectively provide a reasonably useful approximation of the experience. This model of the past is represented by the organism’s internal devices - molecules, cells, and their states. Its only variable physically derived from the external world is the variable of time. The utility of the model therefore depends on the accuracy of generating specific responses to specific temporal properties of external stimuli. Neurons, as the main bearers of information about experience, must be able to represent time.

This does not imply that absolute time is faithfully preserved by some kind of neuronal “clocks”. Instead, cells, and especially neurons, are adapted to generating sustained but temporally restricted responses to different temporal patterns of stimulation. Stimulation is never a discrete event but rather a perturbation in the level of locally available first messenger, occurring with a characteristic temporal profile – a time window. First messenger time windows are integrated by the neuron into responses that change the responsiveness to subsequent first messenger time windows. Neurons therefore “measure” relative timing of time windows, and “recode it” into new time windows. A hierarchy of time windows is established, with homeostatic perturbations at each level typically influencing the effects of perturbations at other levels both above and below.

An experience that lasted a year cannot be useful in the future if it requires a year for behavioral retrieval. Its utility is based on its sustained ability to influence ongoing events occurring on shorter time scales. This consideration highlights the utility of the neuron’s hierarchical system of time windows. Rather than representing and utilizing time scales independently, neurons “nest” them into a single system that holistically determines the cell’s responsiveness to stimulation at each given moment. Nesting of time scales allows cells to immediately take into account both slow and fast perturbations in their state.

A key feature of this nested system is its dynamic nature – a memory can never said to be “final” or set in stone. In effect, ongoing stimulation by first messengers continuously “updates” the prior receptive state of the cell to produce a posterior receptive state. An important consequence of viewing cellular memory in such Bayesian terms is the blurring of the traditional distinction between memory encoding, storage, and retrieval. Encoding of memory amounts to a specific change in the state of a neuron in response to a temporal pattern of first messenger stimulation. Storage of memory amounts to the alteration in responsiveness to subsequent stimulation as a result of this altered state, and retrieval – the altered response itself.

In fact, a key point advanced in this review is that the behavioral concept of memory has no privileged position within the broader, abstract framework of memory, but is simply one of the higher-order levels at which memory can be considered. Indeed, the dichotomies of experience and behavior, stimulus and response, memory induction and memory retrieval all depend on the chosen frame of reference. Any memory within a given system can be reduced to internal changes in response to external events, but what is external and what is internal to this system is a matter of perspective. In this review, we have employed a clear and convenient boundary, defined by the neuronal cell membrane, between stimuli (events originating outside cells) and responses (events originating inside cells). A behavioral approach to memory similarly considers a system with a clear boundary: an individual organism. In both systems, transient external events induce more lasting internal changes, i.e., both systems have memory. Between the cell and the organism, there are many subsystems that can be similarly said to possess memory, but their boundaries may be less clearly defined. Consider the example of the hippocampal CA3 autoassociative network (Rebola et al., 2017). If seen separately from the rest of the brain, each mossy fiber or perforant path input can be seen as a separate external source of information for the network. Thus, the network as a whole can be said to integrate transient inputs from the dentate gyrus (DG) and entorhinal cortex (ECx) and convert them into more lasting, stable states, which would represent memories of these upstream signals and be reducible to their relative timing (Brandalise et al., 2016; Brandalise and Gerber, 2014; Mishra et al., 2016; Rebola et al., 2017). If the CA3 autoassociative network is viewed as part of the larger hippocampal formation together with the DG, then the DG–CA3 projections must be seen as internal to the system. From this perspective, ECx provides all the primary inputs, the timing of which is integrated into the holistic state of the hippocampus (Diana et al., 2007; Eichenbaum et al., 2007; Yeckel and Berger, 1990), including among other variables the strength of DG-CA3 synapses and the configuration of the CA3 autoassociative network.(Diana et al., 2007; Eichenbaum et al., 2007; Yeckel and Berger, 1990)

A collision of metaphor and mechanism occurs when multiple perspectives are considered at the same time. Hubener and Bonhoeffer express a view similar to ours: that “real” (behavioral) memory, defined by the phases of encoding, storage and retrieval, is part of a broader “continuum” of sensory-driven, experience-dependent changes in the brain (Hubener and Bonhoeffer, 2010). In response, Takeuchi et al. point out that in many cases, there is no isomorphism between lower-level plasticity and higher-level memory: “the function(s) that activity-dependent synaptic plasticity serves will, in our view, depend critically on the neural circuit in which that plasticity is embedded in a non-monotonic manner” (Takeuchi et al., 2014).

In other words, memory is defined not only by what is being memorized, but also by what is memorizing it. As an example, Takeuchi and colleagues (Takeuchi et al., 2014)provide hippocampal encoding of context in episodic memories, which they argue is a function not simply of ongoing experience, but also of the pre-existing representations of context. To resolve this disagreement, we find it useful again to refer to the boundaries of the systems in question. Hubener and Bonhoeffer’s perspective considers the brain holistically, taking into account all its ontogenetic experience-dependent changes, including the initial establishment of context representations, which may have occurred years prior to the experience but still necessarily involved signals from the external world. Takeuchi et al., on the other hand, consider various brain regions independently of one another (i.e., the context representation is seen as an independent input into the hippocampal network), yet still refer to “function” from a behavioral standpoint – that is, role of a given subsystem in the organism’s behavioral output. Viewing experience-dependent changes as a hierarchy, rather than a continuum, may ease this inherent tension between the organismal and cellular perspectives on the nervous system.

In this review, we have restricted our discussion of temporal processing to single neurons. There is no reason to assume though that temporal analysis by the organism or the nervous system is restricted to intracellular computations. The principle of brain computation as a whole has been described as hierarchical abstraction (Ballard, 2015). Information available at lower levels of the hierarchy – states of sensory systems, for example – is hierarchically recoded into representations of increasing generality, which are then used to interpret the incoming lower-order data. We contend that in any such case of “recoding”, the actual information transmitted between levels of analysis is reducible to temporal patterns. “Spiking” behavior of neurons is sometimes likened to a binary code – a code utilizing two alternative values to represent information. More appropriately, spiking should be seen as a code employing a single “value” plus relative time.

To conclude, time is a fundamental variable in neuronal computations. Neurons – and probably nervous systems as a whole – possess a fundamentally multi-leveled system for extraction of temporal information from past experience. The timing of past events is converted into a hierarchy of homeostatic perturbations that influence the effect of future events, thereby altering behavior. The adaptive, temporally regulated functioning of this system is the essence of what is commonly called memory.