The role of hierarchical dynamical functions in coding for episodic memory and cognition

Abstract

Behavioral research in human verbal memory function led to the initial definition of episodic memory and semantic memory. A complete model of the neural mechanisms of episodic memory must include the capacity to encode and mentally reconstruct everything that humans can recall from their experience. This paper proposes new model features necessary to address the complexity of episodic memory encoding and recall in the context of broader cognition, and the functional properties of neurons that could contribute to this broader scope of memory. Many episodic memory models represent individual snapshots of the world with a sequence of vectors, but a full model must represent complex functions encoding and retrieving the relations between multiple stimulus features across space and time on multiple hierarchical scales. Episodic memory involves not only the space and time of an agent experiencing events within an episode, but also features shown in neurophysiological data such as coding of speed, direction, boundaries and objects. Episodic memory includes a spatiotemporal trajectory not only of a single agent, but also segments of spatiotemporal trajectories for other agents and objects encountered in the environment consistent with data on encoding the position and angle of sensory features of objects and boundaries. We will discuss potential interactions of episodic memory circuits in the hippocampus and entorhinal cortex with distributed neocortical circuits that must represent all features of human cognition.

Summary

This paper reviews a theory of episodic memory and cognitive function in which spatiotemporal trajectories are coded on multiple scales of place and time. As proposed previously (Hasselmo et al., 2010), this coding can address detailed scales of the spatiotemporal trajectories of individual features of objects (a finger, or an eyelid) as well as other dimensions such as the color or size of individual features. The previous model was extended by the proposal of dynamical matrices that can code trajectories and boundaries (Hasselmo, 2018) allowing interpolation and extrapolation of not only predicted place and time, but also velocity (direction and speed) and acceleration. In a hierarchical manner, this framework can also represent the spatiotemporal trajectory of overall objects, barriers and agents observed by an agent within an event (in egocentric or allocentric coordinates), as well as the spatiotemporal trajectory and features such as egocentric direction of the individual observing the episode.

Introduction: Challenges for episodic memory models

The challenge for episodic memory models is understanding how neural circuits could encode and retrieve the complex range of features that a human can recall from episodic memory of individual experiences. A complete model of episodic memory will require understanding not only of the neural mechanisms for encoding and retrieval in regions directly implicated in episodic memory such as the hippocampus and entorhinal cortex but also the interaction of regions coding episodic memory with higher cognitive function throughout distributed regions of the neocortex (Bhandari and Badre, 2018; Hasselmo and Stern, 2018; Lundqvist et al., 2018)

Episodic memory was first defined by Tulving (Tulving, 1972) to focus on the difference between the encoding and recollection of a specific sensory experience at a specific place and time, versus the encoding and retrieval of facts in general knowledge which was addressed by the term semantic memory (Tulving, 1984; Baddeley, 2001; Tulving, 2001). The term semantic memory had been developed by Quillian (Collins and Quillian, 1969) as a description of memory for facts and general knowledge about the world. Episodic memory was later contrasted with other forms of memory such as knowing how to do things, referred to as procedural or implicit memory (Cohen and Squire, 1980; Cohen and Eichenbaum, 1993), and with the active, conscious maintenance of recent information, known as working memory (Baddeley, 2001). Despite being contrasted with semantic memory, a model of episodic memory must ultimately include its interaction with elements of semantic memory. When we remember throwing a ball to a dog, we code many features of the episodic memory in terms of general knowledge of the rules governing the interaction with both objects and agents (Badre and Frank, 2012; Buschman and Miller, 2014). In fact, human episodic memory does not accurately retrieve most details of episodes (Misra et al., 2018), but instead we reconstruct much of the world from general semantic knowledge.

Here we address some of the challenges for a complete model of episodic memory within the framework of an earlier model (Hasselmo, 2009; Hasselmo et al., 2010; Hasselmo, 2012). This earlier model addressed how neural mechanisms for coding of grid cells in entorhinal cortex could provide the coding of the spatial location and time of events within an episode, linking individual events and items with neural representations of spatiotemporal trajectories (see Figure 1). This framework describes the neural encoding of all features of episodic memory in terms of multiple spatiotemporal functions corresponding not only to the movement of an agent within a scene, but also the regular features of objects and barriers in the environment, as well as the movement of other agents and objects (Hasselmo et al., 2010). A recent extension of this framework proposes that dynamical gating matrices represented and regulated by neural activity (Hasselmo, 2018) can mediate encoding and retrieval of the spatiotemporal functions with the capacity for interpolation or extrapolation of complex functions. This could involve creation of dynamical matrices within structures such as the hippocampus, entorhinal cortex and retrosplenial cortex.

Figure 1.

A. Example of a model of an individual episodic memory encoded and retrieved as a spatiotemporal trajectory (dotted white line) of a person moving through the Boston University campus, including encounters with people at different positions. B. The encoded trajectory is shown in gray, with people encountered at different positions (asterisk, circle, square). C. The retrieval of the spatiotemporal trajectory by the model is shown in black, with shapes indicating the accurate retrieval of individual people encountered at their proper positions.

As an example, Figure 2 shows how sampling of specific points on a trajectory (asterisks) allows computational creation of a dynamical matrix and starting state vector representing properties such as location, velocity and acceleration. The dynamical matrix and initial state vector can be represented by patterns of neural activity, and Figure 2 shows how a dynamical matrix and state vector can encode and retrieve the spatiotemporal trajectory of an object (Hasselmo, 2018). Retrieval of the full trajectory can be generated by sequential multiplication of the initial state vector by the dynamical matrix (Hasselmo, 2018). More complex trajectories can be encoded by higher-level dynamical matrices resetting the component elements of lower level dynamical matrices, to allow transitions between different trajectory segments. This direct influence of higher matrices on the elements of lower level matrices is an important part of the framework. The encoding of spatiotemporal trajectories by dynamical matrices can facilitate the subsequent retrieval of the memories through reactivation of these functions. The coding of multiple spatiotemporal functions moves beyond the coding used in many models in which episodic memories are represented as arrays of feature vectors (McNaughton and Morris, 1987; Treves and Rolls, 1994; Hasselmo and Wyble, 1997; Norman and O’Reilly, 2003).

Figure 2.

Interpolation and extrapolation of a trajectory. The parabolic trajectory of a thrown ball could be encoded as a series of six snapshots of the trajectory, with coding of the location and velocity of the ball at each position (shown with asterisks). However, this snapshot coding does not contain information about the dynamics of the ball. In contrast, the same number of elements of encoded information can be used to encode and retrieve the starting position and velocity and the dynamical matrix that describes the full trajectory, allowing accurate recall of the trajectory (gray dots) thaat interpolates between positions along the trajectory (including between snapshots) and also extrapolation of the trajectory beyond the last snapshot. This is a much more efficient representation than the snapthos

This framework addresses the problem that a complete theory of episodic memory function must account for the encoding and retrieval of all aspects of mental experience, including not only the overall spatial location and time of an event, but also the detailed sensory features of individual objects and agents within that event, including their location and time of appearance as described previously (Hasselmo et al., 2010), as well as their direction of movement, their speed of movement, the configuration of component features (Hasselmo et al., 2010), and semantic knowledge about these objects and agents. Episodic memory also includes encoding and retrieval of memory for internal thoughts, motivations and intentions of an individual agent and their conjectures about the thoughts and intentions of other agents. Thus, a complete neural model of episodic memory requires a general neural code for representing all aspects of human cognition that ultimately includes all dimensions of cognition and rule learning that contribute to the experiences that can be retrieved from episodic memory.

Tulving defined a query for episodic memory as: “What did you do at time T in place P?” (Tulving, 1984) and also described retrieval as the mental voyage from the current time to the past episode being remembered, but he did not focus on time within the past episode, or independent coding of the evolution of different features. Tulving (Tulving, 1984) states that his model “describes a ‘snapshot view’ of episodic memory [that] produces many snapshots whose orderly succession can create the mnemonic illusion of the flow of past time.”

The framework presented here seeks to use neural encoding and retrieval of spatiotemporal trajectories to move beyond the focus on snapshots in Tulving’s model. This focus on single snapshots is also characteristic of many neural network models that code the full viewpoint of the world as a static vector at one point in time. In contrast, the framework of episodic memory reviewed here uses models (Hasselmo, 2009; Hasselmo et al., 2010; Hasselmo, 2012) that include encoding and retrieval of features such as speed and direction of movement on multiple different time scales. Rather than representing discrete snapshots of time, this framework represents the velocity vectors of individual agents within an episode, so that their position could be computed at different spatial and temporal scales (rather than a single frame rate). The use of velocity vectors allows continuous motion to be interpolated or extrapolated beyond individual snapshots (Figure 2), and can include encoding and retrieval of factors such as head direction and the speed and direction of movements. Consistent with this, other researchers have shown that episodic memories involve encoding and retrieval of a perspective, either consisting of the point of view of the person retrieving the memory (a “field” perspective) or a third-person observer perspective (Nigro and Neisser, 1983; Robinson and Swanson, 1993; Conway, 2009). The description below will show how this property of episodic memory could arise from the functional role of neurons that respond on the basis of current head direction and code the position of barriers or spatial viewpoints.

A complete neural theory of episodic memory must address circuit mechanisms that could encode and retrieve the location and time of events and features on multiple scales (Howard and Kahana, 2002; Howard et al., 2004; Howard et al., 2014b). Episodic memories can include not only the encoding and retrieval of the place and time of an event (when John gave Mary a book), but also the location and movement of agents and objects within that event (standing by the bookshelf, John gave Mary a book that she set on the table), and the location and movement of features within individual agents and objects (the book was open in John’s right hand, and Mary closed it with the index finger of her left hand). This framework contrasts with many existing neural network models of brain function that summarize views of the world as single snapshots that are coded in large scale vectors and processed in a feedforward manner (LeCun et al., 2015). The development of LSTM circuits allows incorporation of memory dynamics that could function on multiple time scales (Hochreiter and Schmidhuber, 1997; Bhalla, 2017a). A complete model will require representing, interpolating and extrapolating complex relationships between stimulus features in multiple different scales of space and time.

Episodic memory as a spatiotemporal trajectory

As shown in Figure 1, many episodic memories involve encoding and retrieval of segments of continuous spatiotemporal trajectories. An example spatiotemporal trajectory is shown with dotted lines in the figure. The trajectory includes the agent’s sense of relative location, but also contains a point of view from a specific direction at each moment, and aspects of the timing of the trajectory, such as the speed of walking, and the relative timing of events at one location. Different positions along this spatiotemporal trajectory are associated with individual events involving interacting with specific people or objects.

As noted in a previous article (Hasselmo et al., 2010), this same framework could be used to encode and retrieve information on different scales for episodic memory. For example, that article notes that the appearance of each individual person or object in the episode would be associated with separate encoding of the locations and spatiotemporal trajectory of the people or objects for possible subsequent retrieval. This encoding can take place on multiple scales both for trajectory encoding and object encoding. Multiple scales of grid cell coding have been shown to be effective for encoding and retrieval of trajectories in episodic memory (Hasselmo, 2009; Hasselmo et al., 2010; Hasselmo, 2012), and the planning of future trajectories (Erdem and Hasselmo, 2012; Erdem and Hasselmo, 2014). Encoding can also address different objects scales. Instead of encoding and retrieving only the spatiotemporal trajectory of one person, episodic memory could encode and retrieve the position of limbs on different humans or animals, and the movement of these limbs to different locations. On an even smaller scale, this framework could be used to encode and retrieve the position and movement of fingers within a hand, or the position and movement of eyelids and lips within a face. The accurate encoding of spatiotemporal trajectories could potentially provide a general framework for encoding and retrieval of both episodic and semantic memory on multiple different scales.

The coding of spatiotemporal trajectories was previously described in terms of coding of speed and direction at each position along a trajectory (Hasselmo, 2009; Hasselmo et al., 2010; Hasselmo, 2012). But extension of this work models how trajectories can also be coded in terms of complex functions such as linear dynamical systems that include both velocity and acceleration (Hasselmo, 2018) or representations such as Bezier functions which would allow a potentially more concise representation of significant segments of a trajectory (Hasselmo, 2018). For example, as shown in Figure 2, the trajectory of a projectile such as a ball thrown through the air can be encoded and retrieved as a small dynamical matrix and initial state vector representing the full trajectory of the dynamical system rather than coding individual snapshots of the position of the ball along the trajectory.

As described previously (Hasselmo, 2018), the dynamical matrix could consist of patterns of activity in another population that interacts in a multiplicative manner on the dendritic tree. This dynamical system provides a highly accurate method for retrieval (gray dots in Figure 2) to interpolate between remembered positions, or extrapolate beyond the last remembered position by replicating the application of the dynamical relationship to compute different points along the trajectory. This framework for coding of relations by complex functions could be used to code the complex relations necessary for performance of complex cognitive tasks such as Raven’s progressive matrices (Hasselmo, 2018) which represents a test of general intelligence (Carpenter et al., 1990; Rasmussen and Eliasmith, 2011; Raudies and Hasselmo, 2017; Barrett et al., 2018).

The same technique for coding the transformation between adjacent points on a trajectory could be used for coding the uniform transformation of many points in space, for example due to rotation of an object or an agent. If this transformation is coded as a pattern of gating activity in the network (Hasselmo, 2018), then the activity pattern replicating the transformation could then be replicated to generalize the transformation across different features. In this framework, relatively small transformation matrices could be used to compute the viewpoint of the world at any position along a trajectory. This would provide a much more efficient mechanism for encoding and retrieval of episodic memory (Hasselmo, 2018). Rather than storing a snapshot of every spatial view of the world for an agent moving around a room for several hours, instead the system can store the allocentric location and direction of the features of the room, and encode and retrieve the allocentric location and direction of a person’s trajectory, and the egocentric angle of different visual features can be extracted and retrieved from the transformation of location and direction anywhere along the trajectory, as shown in Figure 3. This framework is similar to the index theory of hippocampal function presented previously (Teyler and DiScenna, 1986; Teyler and Rudy, 2007), but provides an explicit mathematical representation of how episodic memory can link to a concise semantic memory representation of functions that allows interpolation and extrapolation of more complex details of a memory in neocortical circuits.

Figure 3.

Episodic memory of spatial views can involve encoding and retrieval of associations of individual spatial locations with the viewpoint from different spatial locations. In this example, the spatiotemporal trajectory is encoded and retreived by the interaction of speed coding and direction coding that drives the firing of grid cells. The grid cells can update the firing of place cells that are associated with individual spatial views. Alternately, the individual spatial views can update the firing of place cells that drive grid cells. Hebbian modification (bidirectional arrow) allows encoding (top) and retrieval (bottom) of associations of each place cell with a distinct spatial view, which can be represented by a single relational code of features in the environment so that neurons responding to the individual position and direction on a trajectory can encode associations that allow retrieval of the visual feature angle of specific objects in the visual environment.

Lesion and imaging data on anatomical circuits for episodic memory

Neuropsychological and fMRI studies demonstrate an important role for the hippocampus and associated structures in episodic memory. Bilateral removal of anterior hippocampus, entorhinal cortex and parahippocampal gyrus in patient HM caused striking deficits for episodic memory tests (Scoville and Milner, 1957; Corkin, 1984), but sparing of working memory (Scoville and Milner, 1957). Neurophysiological data described below demonstrate the patterns of neuronal activity in the hippocampus and entorhinal cortex are associated with encoding and retrieval of episodic memories in human subjects (Kreiman et al., 2000a; Kreiman, 2007; Rutishauser et al., 2008), and with retrieval or planning of spatiotemporal trajectories during behavioural tasks in rodents (Johnson and Redish, 2007; Pfeiffer and Foster, 2013).

Episodic memory impairments can also be caused by damage to subcortical regions including the anterior thalamus and mammillary bodies (Vann and Aggleton, 2004) and by damage to the medial septum and medial prefrontal cortex caused by aneurysms of the anterior communicating artery (DeLuca and Cicerone, 1991; DeLuca, 1993). These subcortical structures contain neurons coding aspects of behaviour such as head direction in anterior thalamus (Taube, 1995; Stackman and Taube, 1998) and movement direction in the medial septum (Welday et al., 2011). The medial septum and anterior thalamus also influence theta rhythm dynamics in cortical circuits that could be essential to a temporal code for episodic memory (Buzsaki et al., 1983; Stewart and Fox, 1990; Bland and Colom, 1993; Buzsáki, 2002).

Studies in animals demonstrate the role of the hippocampus and associated cortical regions and connections via the fornix with the medial septum and anterior thalamus. Non-human primates show permanent impairments of memory for trial unique objects in delayed non-match to sample tasks after hippocampal lesions (Zola-Morgan and Squire, 1986), and combined hippocampal, perirhinal and parahippocampal lesions (Meunier et al., 1993; Zola-Morgan et al., 1993), and transient impairments after entorhinal lesions (Gaffan and Murray, 1992; Leonard et al., 1995). Lesions of the fornix, which disrupt medial septum input and anterior thalamic output for the hippocampus, impair the construction of a snapshot memory for spatial location of visual features (Gaffan and Harrison, 1989) or associations with responses (Gaffan et al., 1984).

Similarly, studies in rodents showed impairments in delayed alternation caused by hippocampal lesions (Ainge et al., 2007) or lesions of the fornix or the septum (Stanton et al., 1984; Freeman and Stanton, 1991; Aggleton et al., 1995; Ennaceur et al., 1996). An increase in the number of repeat visits to arms in the 8-arm radial maze occurs with hippocampal lesions (Becker et al., 1980) and fornix lesions (Olton et al., 1979; Hudon et al., 2002) and by lesions (Mitchell et al., 1982) or inactivation of the medial septum (Chrobak et al., 1989).

Hippocampal lesions impair finding of a single fixed platform location in the Morris water maze (Morris et al., 1986) including in a variant relevant to episodic memory requiring the rat to retrieve a new platform location on each day (Steele and Morris, 1999). Impairments in the Morris water maze also appear with lesions of the dorsal entorhinal cortex (Steffenach et al., 2005), dorsal presubiculum (postsubiculum) (Taube et al., 1992) and fornix (Eichenbaum et al., 1990), and also with pharmacological inactivation (Brioni et al., 1990) or lesions of the medial septum (Marston et al., 1993) that will reduce theta rhythm. Performance in the Morris water maze is also impaired after lesions of the retrosplenial cortex (Czajkowski et al., 2014) and parietal lesions (Hoh et al., 2003).

The involvement of medial temporal lobe structures in encoding and retrieval of episodic memories is consistent with fMRI data from human subjects. Human subjects show substantial fMRI activation associated with the encoding of new information into episodic memory (Stern et al., 1996; Wagner et al., 1998; Kirchhoff et al., 2000). Consistent with coding of a spatiotemporal trajectory, fMRI data demonstrated hippocampal and parahippocampal activity at the start of a trajectory when a subject retrieves a previously learned overlapping trajectory (Brown et al., 2010; Brown and Stern, 2014). Subsequent work showed that retrieval of a planned trajectory involves reactivation of intermediate locations on the trajectory (Brown et al., 2016). Studies show coding of navigationally relevant information in fMRI activity, with hippocampus coding path length and entorhinal cortex coding the Euclidean distance to the goal (Howard et al., 2014a; Chrastil et al., 2015). Evidence also indicates that the coding of the trajectory and features along the trajectory involve an interaction of regions coding spatial location and direction (Chrastil et al., 2016) with regions coding visual cues such as optic flow and feature angle (Chrastil et al., 2015; Sherrill et al., 2015). These fMRI data support the role of large scale neural activity in these regions for the encoding and retrieval of spatiotemporal trajectories.

Neurophysiological data on coding relevant to episodic memory

As noted above, research has not converged on a model for the neural coding of the broad range of events in an episodic memory. The nature of coding depends upon the details of neurophysiological interactions within cortical circuits. Here we will review evidence for functional coding by neurons in the structures implicated in episodic memory, with a focus on the open questions about coding of episodes. Electrophysiological recordings in the human hippocampus, amygdala and entorhinal cortex demonstrate firing properties consistent with their role in the encoding and retrieval of episodic memories. Unit recordings in these structures show neural responses to highly specific categories of visual stimuli such as faces or animals (Kreiman et al., 2000b; Rutishauser et al., 2011), and reveal neurons that fire during encoding of stimuli as well as during retrieval by mental imagery (Kreiman et al., 2000a). Neurons in these structures show highly detailed coding of individual identities of famous actors or famous landmarks (Quiroga et al., 2005), and highly selective firing during retrieval as tested by self-report of mental imagery (Ison et al., 2015) and during recognition or recall of stimuli (Rutishauser et al., 2008). Thus, extensive data show clear responses to non-spatial stimuli in humans. In rodents, neurons show a variety of responses to the spatial and temporal dimensions of behavioral tasks , but also show responses to non-spatial features such as objects or odors in the environment (Wood et al., 1999). Neurons also show responses suggestive of the retrieval of prior experience (Ferbinteanu and Shapiro, 2003), or the planning of future experience (Johnson and Redish, 2007). The following sections provide more detail on these properties.

Theta rhythm and coding of episodic memory

The effects of fornix lesions and medial septum lesions described above highlight the potential role of theta rhythm in coding of episodic memory. These same lesions of medial septum or fornix reduce theta power in the hippocampus (Rawlins et al., 1979) and entorhinal cortex (Mitchell et al., 1982). Theta rhythm is a prominent band of rhythmic activity in the range of 6–10 Hz that appears in structures including the hippocampus, entorhinal cortex and retrosplenial cortex during movement in a range of different species (Buzsaki et al., 1983; Buzsáki, 2002), and shows important correlations with running speed. Theta rhythm is also associated with gamma frequency oscillations (Bragin et al., 1995; Csicsvari et al., 1999; Colgin et al., 2009).

Evidence suggests that the theta rhythm could play an essential role in a temporal code for episodic and semantic memory. There have been general discussions about whether the coding of sensory input involves a code based on mean firing rate versus a temporal code based on the time of individual spikes. The phenomenon of spiking relative to theta phase provides some of the strongest evidence for a temporal code. The field will benefit from more explicit testing of detailed codes that can address the full complexity of cognitive representations retrieved during episodic memory function.

Lesions of the medial septum cause impairments of memory encoding performance that correlate with the change in theta rhythm (Winson, 1978; Givens and Olton, 1990; Givens and Olton, 1994), and the successful learning of memory traces is accompanied by increases in theta oscillatory power (Berry and Thompson, 1978; Seager et al., 2002; Griffin et al., 2004). Activation of both GABAergic and cholinergic subpopulations in the medial septum enhances temporal precision of hippocampal spiking activity (Dannenberg et al., 2015) and combined lesions of both GABAergic and cholinergic subpopulations in the medial septum severely impair memory performance (Pang et al., 2001). Activation of glutamatergic neurons in medial septum enhance theta rhythmicity in hippocampus via activation of cholinergic and GABergic neurons in medial septum (Robinson et al., 2016). Data from EEG studies indicate that theta oscillations recorded in the medial temporal lobe of human epileptic patients have been shown to increase during memory encoding and during navigation through real world and virtual environments (Kahana et al., 1999; Ekstrom et al., 2005; Aghajan et al., 2017; Bohbot et al., 2017). Some cells in human hippocampus do not show strong theta rhythmicity in their autocorrelograms (Viskontas et al., 2007), but human hippocampal neurons do show spiking that is specific to the phase of theta rhythm oscillations, and this phase specificity during stimuli predicts the subsequent ability to successfully retrieve those stimuli (Rutishauser et al., 2010).

Oscillatory dynamics at theta or gamma frequencies could be essential to the neural coding of episodic information. For example, as described further below, individual neurons in the hippocampus code sequences of positions along a trajectory based on the phase of firing relative to theta rhythm oscillations (O’Keefe and Recce, 1993). This phase code for position has also been demonstrated in the entorhinal cortex (Hafting et al., 2008) and the lateral septum, a hippocampal output structure where neurons do not exhibit spatial receptive fields but instead code position solely via phase (Tingley and Buzsaki, 2018). In addition, phase coding has also been shown for the duration of time within an interval (Pastalkova et al., 2008) and might also exist for coding of the position of an object in a sequence, or other dimensions of experience. Thus, theta rhythm might provide a substrate for a phase code of different dimensions of memory and behavior.

Phase coding might also exist for other frequencies. For example, different populations of neurons show spiking at specific phases relative to gamma frequency oscillations (Senior et al., 2008). The duration of each gamma cycle may be too short for coding of spiking on different phases within a gamma cycle, but a binary code could be maintained by different neurons firing at either the peak or the trough of individual gamma cycles. For example, firing on the peak could represent a “1” in a binary code, whereas firing on the trough could represent a “0” in a binary code. The updating of this binary code would require relatively complex patterns of synaptic connectivity to ensure that the binary code for individual neuron phase would be updated based on the update of other neuron binary codes.

Alternately, theta rhythm might play a role for the functional dynamics of cortical circuits, setting differences in dynamics for the encoding of new information on one phase of the oscillation, versus the retrieval of information at a different phase of the oscillation (Hasselmo et al., 2002). This is supported by empirical data on the timing of synaptic modification at different phases of theta rhythm (Huerta and Lisman, 1995; Hyman et al., 2003), data on the effect of phase specific manipulations that influence encoding or retrieval (Douchamps et al., 2013; Siegle and Wilson, 2014), and data on the coordination of activity in different brain regions relative to hippocampal theta (Hyman et al., 2002; Jones and Wilson, 2005; Colgin et al., 2009).

Other frequencies could be involved in similar regulation of functional gating. Neocortical regions show bursts of activity at beta and gamma frequencies associated with gating of information into working memory or top-down regulation of the flow of information into working memory (Buschman et al., 2012; Lundqvist et al., 2016; Lundqvist et al., 2018). Simulations have shown how bursts of gamma frequency activity could be involved in maintaining information in working memory (Fransén and Lansner, 1998; Lundqvist et al., 2010; Lundqvist et al., 2011), and how the network resonance of cortical regions at gamma and beta frequencies could be used to gate the flow of information between different cortical regions (Sherfey et al., 2018). Simulations need to explore the broad range of functional episodic memory mechanisms that could involve network oscillatory dynamics.

Trajectory retrieval by place cells

By Tulving’s initial definition, episodic memory requires a code for the spatial location of a memory. As noted above, this could include the overall location of an agent during an event, but could also include the location of individual agents and objects within an event, or even the location of parts of objects such as a person’s hand (Hasselmo et al., 2010). Encoding and retrieval of the spatiotemporal trajectory of an agent could involve neurons in the hippocampus termed place cells, that show responses dependent on the current, spatial location of a rodent (O’Keefe and Dostrovsky, 1971; O’Keefe, 1976). Virtual navigation studies show selective responses to virtual location by hippocampal neurons in humans (Ekstrom et al., 2003).

Neuronal activity of place cells does not only represent the coding of current position, but also demonstrates retrieval or planning of segments of spatiotemporal trajectories. Early studies of place cells focused on a rate code for spatial location (O’Keefe and Dostrovsky, 1971; O’Keefe, 1976), but later studies revealed a striking temporal code in which the finer-grained position of an animal relative to the position along a trajectory through a specific firing field is reflected in a shift from late to early phase of firing of place cells relative to hippocampal theta rhythm oscillations (O’Keefe and Recce, 1993; Skaggs et al., 1996). On a population level, it appears that place cells fire in sequential order within a theta rhythm based on their relative coding of position (Foster and Wilson, 2007; Feng et al., 2015). In addition, when animals are making a decision about future movement based on memory of the task, experimental data show place cell sequences that appear to be associated with the retrieval of prior trajectories or planning of future trajectories by the animal (Johnson and Redish, 2007; Pfeiffer and Foster, 2013; Olafsdottir et al., 2015). Sequential firing of place cells within a theta cycle have been found to reflect future trajectories to a goal location providing one possible neural mechanism for prospective planning (Wikenheiser and Redish, 2015).

These studies provide the evidence of specific retrieval activity that could be linked to episodic memory. Related to this, neurons have also been shown to fire selectively for specific left or right trajectories in continuous spatial alternation tasks (Frank et al., 2000; Wood et al., 2000) and in a delayed non-match to position task (Griffin et al., 2007), suggesting that they may depend upon the retrieval of prior trajectories or planning of future trajectories (Hasselmo and Eichenbaum, 2005). Recordings on a plus-shaped maze with different future or past arms shows that some neurons selectively respond based on the prior segment of the trajectory, whereas others show selectivity for the future segment of a trajectory (Ferbinteanu and Shapiro, 2003). Surprisingly, neurons did not show trajectory specificity in the delayed version of the spatial alternation task possibly due to difficulty in determining the time of retrieval (Ainge et al., 2007).

Thus, evidence supports the encoding and retrieval of trajectories through of a series of spatial locations by the firing rate and the firing phase of place cells. However, the mechanism by which the coding of place arises is still uncertain. Neurons show changes in response to the running speed and direction (McNaughton et al., 1983; Muller and Kubie, 1987; Gothard et al., 1996; Terrazas et al., 2005), and models have proposed that place cells could arise from path integration in other regions that drive the hippocampal place representation (Touretzky and Redish, 1996; Redish and Touretzky, 1997; Samsonovich and McNaughton, 1997; Redish and Touretzky, 1998; McNaughton et al., 2006). But path integration suffers from accumulation of error. As an alternative, other models propose that place cells could be driven by the learned response to visual features in particular configurations (Hetherington and Shapiro, 1993; Byrne et al., 2007; Bicanski and Burgess, 2018), based on a transformation of egocentric feature angle to allocentric position based on the current heading direction of the animal (Byrne et al., 2007).

Spatial coding by grid cells

The coding of spatial location along a spatiotemporal trajectory has been proposed to involve an interaction of both place cells and grid cells (Hasselmo, 2009; Hasselmo et al., 2010; Hasselmo, 2012). Place cells in the hippocampus primarily code individual locations in the environment, whereas grid cells in the entorhinal cortex have been shown to fire when a foraging animal visits an array of spatial locations in the environment (Hafting et al., 2005). The firing fields occur in a regular array of locations that can be described as falling on the vertices of tessellated equilateral triangles, or in a hexagonal pattern. Different grid cells show different spatial scales (Hafting et al., 2005; Sargolini et al., 2006; Barry et al., 2007; Stensola et al., 2012), allowing a population of grid cells to code individual spatial locations (Gorchetchnikov and Grossberg, 2007; Fiete et al., 2008; Sreenivasan and Fiete, 2011; Mathis et al., 2015). Models also indicate how grid cell firing can be used to explore and find possible relational trajectories between locations (Erdem and Hasselmo, 2012; Erdem and Hasselmo, 2014), or to directly compute the relationship between different spatial locations (Bush et al., 2015). Many grid cells conjunctively code both location and the current head direction of the animal (Sargolini et al., 2006).

Models of grid cell location implicitly focus on different coding principles. For example, a firing rate code for grid cells is the emphasis of models using attractor dynamics to generate grid cell firing patterns within a population of neurons with feedback inhibition (Conklin and Eliasmith, 2005; Fuhs and Touretzky, 2006; McNaughton et al., 2006; Burak and Fiete, 2009; Giocomo et al., 2011; Yoon et al., 2013; Widloski and Fiete, 2014). These models are supported by the distinct grid cell modules that share properties of spacing and orientation (Barry et al., 2007; Stensola et al., 2012), and the change in correlation with different anatomical distances between grid cells (Heys et al., 2014). These models use path integration that could accumulate errors, but could be reset by sensory cues (Touretzky and Redish, 1996; Bush and Burgess, 2014; Hardcastle et al., 2015).

Alternately, a temporal code model of grid cells codes location based on firing updated by shifts in frequency of input theta rhythm oscillations induced by velocity (Burgess et al., 2007; Burgess, 2008; Hasselmo, 2008; Giocomo et al., 2011; Bush and Burgess, 2014; Hasselmo, 2014). These models directly replicate data on the theta phase precession firing of grid cells relative to theta rhythm oscillations (Hafting et al., 2008). Consistent with this model based on theta rhythm, experiments show a loss of grid cell spatial selectivity during loss of theta rhythm caused by inactivation of medial septum (Brandon et al., 2011; Koenig et al., 2011) which spares the coding of head direction (Brandon et al., 2011) and place cell firing in hippocampus (Brandon et al., 2014). The loss of grid cell selectivity appears to specifically depend on inactivation of GABAergic neurons in the medial septum (Robinson and Brandon, 2018). Inactivation of cholinergic neurons does not appear to alter theta rhythm or speed coding in a manner that would alter grid cells (Dannenberg et al., 2019).

The role of oscillatory dynamics is further supported by changes in intrinsic rhythmicity of entorhinal neurons with spatial scale (Jeewajee et al., 2008) and shifts in running speed (Jeewajee et al., 2008; Hinman et al., 2016). The influence of internal noise can be reduced by redundant coding in a population of neurons (Zilli and Hasselmo, 2010) or combining interference with attractor dynamics (Bush and Burgess, 2014; Hasselmo and Shay, 2014). The influence of external noise can be reduced by resetting location based on the angle of sensory cues (Bush and Burgess, 2014) consistent with more accurate coding near environmental boundaries than at a distance from boundaries (Hardcastle et al., 2015). The role of sensory input angle is supported the loss spatial coding during inactivation of regions providing head direction input (Winter et al., 2015).

Another model of grid cells involves the self-organization of input from place cells (Kropff and Treves, 2008; Si et al., 2012) with spike-frequency accomodation of entorhinal neurons resulting in a time-varying response to input from place cells that self-organizes into the pattern of grid cell responses. These models (Krupic et al., 2014; Dordek et al., 2016) are supported by evidence showing that development of place cells appears earlier than grid cells (Wills et al., 2010; Wills et al., 2012), and that place cell inactivation reduces grid cell firing (Bonnevie et al., 2013), whereas grid cell inactivation does not prevent place cell firing (Brandon et al., 2014), though changes in grid cell scale do influence place cell scale (Mallory et al., 2018)

Time coding for spatiotemporal trajectories

As noted above, the coding of episodic memory also requires the coding of the specific time of events on multiple different scales, ranging from the time of one episode versus another episode, to the timing of different events within an episode, to the timing of specific movements during a particular event. The neural coding of time could involve the patterns of activity shown by populations of neurons referred to as time cells. This refers to neurons that respond at specific time intervals during behavioral tasks. These cells have been described as episode cells (Pastalkova et al., 2008) or time cells (MacDonald et al., 2011). They have been shown to fire during the delay period between the encounter of a cue and a delayed response (MacDonald et al., 2011; MacDonald et al., 2013). These types of responses have also been shown in spatial alternation tasks, in which neurons will fire at specific intervals during running on a running wheel (Pastalkova et al., 2008) or during running on a treadmill (Kraus et al., 2013; Kraus et al., 2015).

The coding of time in episodic memory has been proposed to occur on multiple different temporal scales (Howard and Kahana, 2002; Howard et al., 2014b). Consistent with this, calcium imaging data on multiple time cells in the hippocampus show that these neurons not only respond at specific time intervals within individual trials (Mau et al., 2018), but that these neurons also show changes in their response across trials within a day, and the neurons also change their population response across days (Mau et al., 2018). This provides strong empirical evidence for spatiotemporal coding across multiple scales. It is important to note that the neurons that code specific time intervals also often code specific spatial locations as place cells or grid cells (Kraus et al., 2013; Kraus et al., 2015). Thus, the same populations of neurons can simultaneously code space and time along a spatiotemporal trajectory. The models of grid cell firing in entorhinal cortex can account for the fact that many grid cells and place cells will also respond at specific time intervals during running (Burgess et al., 2007; Hasselmo, 2008). Consistent with the fact described below that grid cells are dependent on oscillatory input, the inactivation of medial septum also prevents the temporal specificity of firing by time cells (Wang et al., 2015). Alternately, both abstract models (Howard and Kahana, 2002; Howard et al., 2014b) and biological detailed models (Tiganj et al., 2015; Liu et al., 2018) show how the coding of time can arise from exponential decay of spiking rate, consistent with data from slice preparations (Tahvildari et al., 2007) and in vivo recording (Tsao et al., 2018).

Direction coding

The memory of a spatiotemporal trajectory of an agent requires not only its location over time in the environment, but it also includes encoding and retrieval of its heading within the environment. In fact, the spatiotemporal trajectory includes not only the direction of overall body movement, but also the moment-to-moment direction of the animal’s head (Raudies et al., 2015). Consistent with this, many neurons coding the head direction of rodents were discovered, initially in the dorsal presubiculum by Ranck and Taube (Taube et al., 1990) and later in a range of structures including the anterior thalamus (Taube, 1995), the lateral mammillary nucleus (Stackman and Taube, 1998), the entorhinal cortex (Sargolini et al., 2006; Brandon et al., 2013; Giocomo et al., 2014), and the retrosplenial cortex (Chen et al., 1994; Cho and Sharp, 2001). Head direction cells respond selectively based on the current allocentric direction of the animal’s head, regardless of the location of the animal in the environment and independent of the relative position of individual landmarks or features. During sleep, head direction cells appear to replay the sequence of positions encountered during behavior (Peyrache et al., 2015). An angular velocity signal based on self-motion appears to be integrated to drive head direction neurons in structures such as the anterior thalamus, lateral mammillary nucleus and dorsal tegmental nucleus (Taube and Bassett, 2003; Clark and Taube, 2012). As noted above, episodic memory includes specific viewpoints of an event, and head direction cells could provide an important means of remembering specific viewpoints. In addition, evidence indicates that the allocentric spatial code could arise from a transformation from egocentric viewpoints, which could use an interaction of head direction cells with egocentric viewpoint cells to create an allocentric coordinate frame (Byrne et al., 2007; Bicanski and Burgess, 2018).

The coding of direction is standard in robotics which use coding of the “pose” of the agent, which includes both location and heading direction (Milford et al., 2010; Milford and Wyeth, 2010). Vision-based robotics includes coding of barriers and objects, and the transformation from the coordinates relative to the heading of the agent (egocentric coordinates) into coordinates of the absolute position of the agent relative to the overall environment (allocentric coordinates) (Byrne et al., 2007; Milford et al., 2010; Milford and Wyeth, 2010).

The head direction signal is not equivalent to a movement direction signal, and analysis of periods where head direction does not match movement direction shows that these cells primarily code head direction (Raudies et al., 2015), though they can code angular head motion (Clark and Taube, 2011). This supports the notion that place codes are driven more strongly by the angle of visual features dependent on head direction, rather than path integration.

Boundary coding

As noted above and in a previous paper (Hasselmo et al., 2010), the coding of episodic memory requires not only coding of the spatiotemporal trajectories of a single person, but would require the coding of the relative spatiotemporal trajectory of objects and barriers within the environment. This could include tracking of the position of environment boundaries relative to the agent. For example, your memory of walking through an apartment would include spatial viewpoints of the position of different walls of the rooms and hallways as you follow a trajectory through the environment, as shown in Fig. 3.

A range of data has demonstrated the influence of environmental boundaries on coding by populations of neurons. Early data showed that place cell firing depends on the relative position of the walls of the environment (O’Keefe and Burgess, 1996), leading to the novel theoretical proposal that a specific class of neurons might code the position of an animal relative to boundaries (Burgess et al., 2000; Savelli et al., 2008; Hartley et al., 2014). This modeling prediction was later supported by data showing the existence of boundary cells that respond at a specific distance and angle from boundaries in allocentric coordinates (Savelli et al., 2008; Solstad et al., 2008; Lever et al., 2009). Modeling proposed that egocentric visual input about boundaries combined with head direction cells could be transformed to code allocentric spatial location (Byrne et al., 2007; Bicanski and Burgess, 2018). The egocentric coding of boundary position has been demonstrated in recent data which shows neurons with selectivity for the distance and angle of boundaries in egocentric coordinates (Hinman et al., 2017; Alexander et al., 2018; Hinman et al., 2019), suggesting that coding of boundaries in egocentric coordinate frames could contribute to the coding of these boundaries in allocentric coordinate frames.

Object coding

As mentioned above, the coding of an episodic memory can involve non-spatial features, such as the identity of specific objects (Kreiman et al., 2000a). A number of studies have focused on the potential representation of objects in the hippocampus and entorhinal cortex. Recordings in the hippocampus and related cortical structures in humans show neurons with many types of non-spatial object selectivity (Kreiman et al., 2000b; Quiroga et al., 2005; Kreiman, 2007; Rutishauser et al., 2011). Similarly, many neurons in rodent hippocampus also show selective responses to non-spatial features such as specific odors or rewards (Wiener et al., 1989; Otto and Eichenbaum, 1992; Wood et al., 1999). Recordings in the rodent lateral entorhinal cortex show that some neurons respond to specific objects (Deshmukh and Knierim, 2011; Keene et al., 2016) or odors (Young et al., 1997), though these neurons appear less frequently than spatially coding neurons in medial entorhinal cortex. Recordings also show object responses in the rodent perirhinal cortex (Deshmukh et al., 2012). Responses to specific objects have been shown in perirhinal cortex and entorhinal cortex of monkeys (Wilson et al., 1990; Riches et al., 1991; Suzuki et al., 1997), along with evidence that these responses decrease or increase with repeated presentations of objects (Riches et al., 1991; Suzuki et al., 1997), consistent with models of familiarity recognition in which the responses to familiar objects increase or decrease (Sohal and Hasselmo, 2000; Bogacz and Brown, 2003).

A full episodic memory also requires encoding and retrieval of the location and movement of objects in addition to the interaction of agents in an episode. Neurons in the hippocampus respond to a particular orientation and distance from objects (Deshmukh and Knierim, 2013), and entorhinal neurons respond to landmark position as well as path integration (Campbell et al., 2018). Coding of objects also occurs in egocentric coordinates (Wang et al., 2018), and coding of the previous location of an object as trace cells in hippocampus (O’Keefe and Nadel, 1978) and entorhinal cortex (Tsao et al., 2013). Of particular interest with regard to shared mechanisms for coding agents and objects, there is new evidence that neurons that respond to the current location of a rat also respond to the location of another rat in the same environment (Danjo et al., 2018), which includes showing theta phase coding of the position of the other rat within a firing field. These data provide important evidence of the relational coding in entorhinal cortex of object location in egocentric coordinates relative to the agent. These mechanisms for relational coding could relate to the general coding of objects and features in a broad range of neocortical structures. There might be a general mechanism for the coding of cognitive representations within cortical regions that could be applied across sensory modalities, as supported by evidence that cortical regions can take over the function of other damaged cortical regions (Roe et al., 1990; Roe et al., 1992; Abel et al., 2015).

Speed coding

In contrast to a snapshot view of episodic memory, the spatiotemporal trajectory view of episodic memory would code functions to allow a smooth continuous reconstruction of the elements of a trajectory. This requires coding not only of position at different time points, but also the velocity and possibly even the acceleration along the trajectory. This would allow more accurate interpolation or extrapolation of the trajectory on any desired scale (Hasselmo, 2018). Consistent with this framework, neurons respond not only to the position and time during events, but also to dynamical variables such as running speed. The coding of running speed has been shown in a number of different functional classes of neurons in the entorhinal cortex (Wills et al., 2012; Kropff et al., 2015; Hinman et al., 2016; Hinman et al., 2018) and hippocampus (McNaughton et al., 1983).

Cells throughout the hippocampal formation spike theta rhythmically and, in addition to the firing rate speed signal in MEC, the coding of speed by the frequency of rhythmic firing has been shown experimentally in single cells in MEC (Jeewajee et al., 2008; Hinman et al., 2016). The rhythmic frequency of single cell firing usually increases as a rat runs faster, but there are also cases where the rhythmic frequency decreases as the rat runs faster (Dannenberg et al., 2019). Given that both speed signals have been identified in MEC, we investigated whether the same cells express both speed signals similarly and found that the two signals are actually independently expressed in single cells (Hinman et al., 2016). This suggests that both rate coding and temporal coding might occur for running speed. The change in coding of running speed over different time scales also suggests that neurons might be coding acceleration (Dannenberg et al., 2019). Acceleration has been used to enhance the encoding and retrieval of trajectories using dynamical matrices (Hasselmo, 2018). Another potential implication is that temporal coding of running speed may not only involve rhythmic frequency, but might actually involve a phase code for running speed in terms of the timing of spikes relative to theta phase or even gamma phase for different running speeds.

Context coding

The coding of spatiotemporal trajectories requires that different trajectories overlapping in the same location in the environment can be disambiguated. This is consistent with neurophysiological data showing that the coding of trajectories is highly context dependent. Early data showed that place cells in one environment will often turn off in a different environment and be replaced by other place cells that were inactive in the first environment (Muller and Kubie, 1987; Lever et al., 2002). Even within an environment, neurons will show strongly context-dependent activity. For example, when a rat runs on the stem of spatial alternation task, individual neurons will fire selectively based on the past or future turning response in hippocampus (Wood et al., 2000; Ferbinteanu and Shapiro, 2003) and entorhinal cortex (Frank et al., 2000). Neurons also code other features of the environment such as the identity of odors or the presence of reward (Eichenbaum et al., 1987; Wood et al., 1999), and can show highly selective firing based on the influence of a spatial context on the reward valence of individual objects (Komorowski et al., 2009; McKenzie et al., 2016). In some cases, the change in coding may reflect a rotation of firing fields relative to the environment (Kinsky et al., 2018). Additionally, recent work has revealed that spatially-specific but non-grid cells of the MEC globally remapped in response to contextual changes to an environment (Diehl et al., 2017). Grid cells, on the other hand, have been found to exhibit either nodal rate alterations or translational shifts of the full map in response to contextual alterations to the environment (Marozzi et al., 2015; Diehl et al., 2017). There is some evidence that contextually sensitive or non-sensitive MEC neurons actually form molecularly and anatomically distinct sub-populations within the region (Kitamura et al., 2014; Ray et al., 2014; Kitamura et al., 2015). These data are consistent with episodic coding of spatiotemporal trajectories, rather than coding only in a static spatial map.

As described here episodic coding of an event within an episode, and the more general coding of objects for cognitive processing can and should involve multiple different spatiotemporal trajectories in different coordinate systems. This is consistent with evidence from the retrosplenial cortex, which provides an important interface between sensory systems such as the visual and somatosensory system with memory-related structures such as the entorhinal cortex and hippocampus. In particular, electrophysiological data from the retrosplenial cortex demonstrates the coding of multiple different dimensions that include allocentric coding of location in space, egocentric coding of turning directions on a maze, and pathway-centric coding of position relative to the start and end of a particular path (Alexander and Nitz, 2015, 2017). This coding of multiple different coordinate systems is important for the robust and flexible coding of the environment both for coordinate transformations necessary when planning different behaviors and for the robust coding of episodic memory of a trajectory.

In order to account for the multiple complex representations in episodic memory and the transformations between different coordinate systems, we need more sophisticated neural representations of these types of competing production systems that can neurally represent higher order relations within sensory input as complex functions based on dynamical matrices which could also act as transformation matrices between different coordinate frames. The coding of complex relations can be useful both for modeling of cognitive processing (thought) and for coding the full richness of episodic memory and cognition.

Need for more exploration of model space

While we have reviewed much exciting data on the functional coding relevant to episodic memory in the hippocampus, entorhinal cortex and related cortical structures, future progress in this field requires more extensive exploration of the full space of possible neural models. The space of possible neural models of cognition is enormous, and only a small portion of this space has been explored in research. There are many dimensions of neural function that may be relevant to cognition. Dimensions that have already been explored extensively in current models include the modifiable strength (or weight) of excitatory and inhibitory synaptic connections between neurons (Bliss and Collingridge, 1993), the magnitude of tonic depolarization of populations of neurons, and the threshold for firing of individual neurons. Synaptic dimensions of neural function that have not been explored as extensively as synaptic weight and bias include the nonlinear interactions of synapses on the dendritic tree. This could include multiplicative interactions of synapses due to NMDA receptors (Mel, 1993; Poirazi et al., 2003) or internal chemical waves (Bhalla, 2017b, a), which have been used in sigma-Pi networks (Durbin and Rumelhart, 1989). These multiplicative interactions could play a role in gating the flow of information between populations of cortical neurons (Hasselmo and Stern, 2018). Gating of information flow can be supplemented by temporally coordinated firing of inhibitory interneurons providing cell-type and cell-compartment specific inhibition. Two examples of such microcircuits are axoaxonic inhibitory interneurons (Klausberger and Somogyi, 2008) targeting the axon initial segment of pyramidal neurons, and the interneuron-specific VIP interneurons inhibiting PV and SOM neurons to cause disinhibition of excitatory principal neurons (Pi et al., 2013; Fu et al., 2014).

Another area that has not been explored are the intrinsic dynamics of neurons that include the spike frequency accomodation (or adaptation) caused by calcium-activated potassium currents (Connors et al., 1982; Barkai and Hasselmo, 1994), the resonance and rebound spiking caused by hyperpolarization activated cation currents (Klink and Alonso, 1993; Ferrante et al., 2016), and the persistent spiking caused by calcium-sensitive non-specific cation currents (Egorov et al., 2002; Fransén et al., 2003; Fransén et al., 2006). These dimensions of function have been effectively simulated in reduced form in single neuron models (Izhikevich, 2003), and biophysical models have explored the memory function of channels such as adapation currents (Barkai et al., 1994), and persistent spiking currents (Fransén et al., 2002). But implications for network dynamical function have not been fully explored.

An important step in the exploration of model space would be to systematize what we do and do not know. For example, it could be productive to characterize the functional attributes of known neural circuit models. The systemization of the properties of atomic elements provided an important framework for chemistry (Mendeleev, 1869) and mathematical systematization guided research in particle physics (Gell-Mann, 1964). As potential examples, the functional dynamics of second-order differential equations have been characterized along the dimensions of the trace and determinant of the Jacobian matrix (see page 96 of (Hirsch and Smale, 1974)). There have been efforts to provide a systematic categorization of existing set of neural network architectures and their functional properties (Van Veen, 2016), but the field might benefit from a more systematic approach to understanding the large space of neural models that have not yet been explored.