Hippocampal-cortical interaction in decision making

Abstract

When making a decision it is often necessary to consider the available alternatives in order to choose the most appropriate option. This deliberative process, where the pros and cons of each option are considered, relies on memories of past actions and outcomes. The hippocampus and prefrontal cortex are required for memory encoding, memory retrieval and decision making, but it is unclear how these areas support deliberation. Here we examine the potential neural substrates of these processes in the rat. The rat is a powerful model to investigate the network mechanisms underlying deliberation in the mammalian brain given the anatomical and functional conservation of its hippocampus and prefrontal cortex to other mammalian systems. Importantly, it is amenable to large scale neural recording while performing laboratory tasks that exploit its natural decisionmaking behavior. Focusing on findings in the rat, we discuss how hippocampal-cortical interactions could provide a neural substrate for deliberative decision making.

Deliberation in the rat

Behavioral observations in naturalistic settings demonstrated that the brown rat, Rattus norvegicus, has remarkable spatial memory (Calhoun 1963; Telle 1966). Individuals can remember locations of food sources, paths, predators and competitors in an environment and use this information for efficient foraging. Moreover, natural environments are complex and change over time. The rat needs to adapt to any changes it encounters, such as the presence of predators and obstacles on its foraging routes. As a result, relying solely on reflexive or habitual routines is insufficient for successful foraging in the long term. Memory, in particular the ability to learn spatial relationships and to modify stored representations when the world changes, is therefore critical for a foraging rodent to obtain food while avoiding danger.

Making navigational decisions based on stored memories requires planning and evaluation of potential trajectories based on information about past foraging experiences. This representation of the external world stored in memory has been referred to as the “cognitive map” (Tolman et al. 1946; Tolman 1948). Navigational decision making can therefore be seen as an internal deliberation process whereby different routes through the animal's cognitive map are evaluated. In this type of decision making, the animal considers the action and consequences of multiple options, weighing up the pros and cons in order to decide on the most suitable option.

We expect that this type of deliberation is most important during the learning of new routes and following changes to the environment. In these situations, no single option stands out as being correct and therefore many alternatives need to be considered. Many aspects of natural foraging behavior are captured in laboratory behavior paradigms that requires a rat to find food reward by navigating along defined paths or in an arena. Thus, these paradigms are well suited to investigate deliberation in a spatial context that is ethologically relevant. Indeed, a rat performing a laboratory task that requires a choice between different arms of a maze pauses briefly at maze choice points and turn its head back and forth to scan the possible routes before choosing one, a behavior termed vicarious trial and error (Muenzinger and Gentry 1931; Tolman 1938; Johnson and Redish 2007). This behavior is most frequent in the learning phase of the task when the rat is still unsure of the rules.

Neural signatures of deliberation

The processes of deliberation include the initial formation of memory, its subsequent recall to provide the options for deliberation and the comparison and evaluation of these options. These processes depend on a number of brain regions, including the hippocampus and the prefrontal cortex (PFC). While patterns of activity important for memory formation have been identified in the hippocampus, the neural substrates and patterns of activity that support the recall and evaluation of choice options based on memory remain somewhat unclear. We will review findings in the fields of memory and decision making and propose candidate neural mechanisms in the hippocampus and PFC to support these steps of deliberation.

To find the pattern of neural activity and mechanisms mediating different steps of deliberation, we need to identify the prerequisites necessary to support deliberation. Since deliberation is more important during learning, we expect the pattern of activity to be more prominent during this period. The pattern of neural activity also needs to occur at locations where deliberation is important. These locations are where a rat needs to plan its next trajectory, which include the starting point and subsequent choice points. The content of the pattern of activity should also contain information useful for the decision. In the case of navigational decisions, the pattern of activity should contain spatial representations not limited to the current location but instead representations of extended trajectories or remote locations of interest. Lastly, the pattern of activity must be required for decisions that require deliberation. Having these requirements in mind, we will explore various patterns of neural activity in the hippocampus and the PFC and determine if they are likely candidates for deliberation.

Hippocampal contribution to deliberation

If experience is to be used in the future for making decisions, it must be encoded and stored. Strong neural correlates of memory encoding processes can be found in the hippocampus across species (Squire 1992). In the rat, the hippocampus forms representations of experience that are expressed through the sequential firing of unique ensembles of pyramidal neurons as the animal explores an environment. The pyramidal neurons are often active at specific spatial locations in the environment, giving rise to their name: “place cells” (O'Keefe and Dostrovsky 1971). The location in which a place cell fires is called its “place field”. However, place cells represent much more than place. Numerous studies have shown that these neurons can respond to a complex combination of spatial and non-spatial information including external sensory stimuli and aspects of task relevant behaviors (Wible et al. 1986; Eichenbaum et al. 1987; Wiener et al. 1989; Sakurai 1996; Frank et al. 2000; Wood et al. 2000; Ferbinteanu et al. 2003; Pastalkova et al. 2008; MacDonald et al. 2011). Taking account of these findings, the ensemble activity of “place cells” in the hippocampus is perhaps best described as encoding a moment by moment representation of different dimensions of experience (Eichenbaum et al. 1999).

Theta oscillations

Physiologically, hippocampal information processing can be characterized as a continuum of states that appears to reflect the relative dominance of processed sensory input and stored representations (Kemere et al. 2013). During periods of active exploration, processed sensory input drives coordinated neuronal activity in the hippocampus, and is associated with synchronous oscillation in the 7-10Hz range, termed theta (Buzsaki 2002; Hasselmo and Stern 2013). Theta oscillations are most prominent during locomotion and less so during pauses in activity.

During periods of theta oscillation, each place cell fires at a preferred phase of each period. As the animal traverses the place field of a place cell, the phase at which spikes fire gradually shifts earlier with each theta cycle, a phenomenon called theta precession(O'Keefe and Recce 1993; Skaggs and McNaughton 1996). Thus, activity of place cells during theta oscillation seems well suited to encode information about the rat's current location and a local trajectory on a rapid timescale. Having a representation of the current location is necessary for the animal to determine if it is in a place where a decision needs to be made. This information could also be used by other brain areas to monitor if the current course of action matches the choice made in the original decision.

With respect to deliberation, we expect to observe representations of potential trajectories beyond the current location of the rat. Representations of locations ahead of the rat during theta have been reported at trajectory choice points and correspond with vicarious trial and error behavior (Johnson and Redish 2007; Gupta et al. 2012). This study suggests theta oscillation during pauses at a choice point is a period when place cells momentarily encode non-local representations and could reflect the exploration of future trajectories. At the same time, these sequences have been seen in the context of tasks where a familiar environment is altered to contain a different series of choice points, and further work will be required to determine the characteristics of these sequences in other circumstances.

Sharp-wave ripples

At low movement speeds and when a rat is not actively exploring, another network pattern of hippocampal activity known as sharp-wave ripples (SWRs) becomes prominent. SWRs are brief high frequency oscillations, between 150-250Hz (Buzsaki 1986) originating from the CA3 region and propagating to the CA1 region of the hippocampus (Buzsaki 1986; Csicsvari et al. 2000). During this period, ensembles of pyramidal neurons become briefly reactivated (Wilson and McNaughton 1994; Skaggs and McNaughton 1996; Kudrimoti et al. 1999; Foster and Wilson 2006; Jackson et al. 2006; Diba and Buzsaki 2007; Davidson et al. 2009). Cells whose activity would span many seconds as the animal explores, during theta oscillation, are reactivated within a few hundred milliseconds. Importantly, ensemble firing on this fast time scale often preserves the original temporal relationship between cells. Relating the sequence of firing of individual cells and their place fields in an environment reveals representations of actual trajectories traversed by the animal (Lee and Wilson 2002; Foster and Wilson 2006; Diba and Buzsaki 2007). These observations provide strong evidence that during SWRs the hippocampus reactivates a time-compressed version of past experiences.

SWR replay events have been linked to the offline strengthening of cortical representations of memory through repeated activation of cortical ensembles during sleep or inactivity (Buzsaki 1996; Frankland and Bontempi 2005; Diekelmann and Born 2010; O'Neill et al. 2010; Girardeau and Zugaro 2011). Since the first reports of hippocampal replay during sleep, replay has subsequently been observed to occur frequently during active behavior (Kudrimoti et al. 1999; Foster and Wilson 2006; Jackson et al. 2006; Diba and Buzsaki 2007; Davidson et al. 2009; Karlsson and Frank 2009; Dupret et al. 2010). On a network level, the reactivation of memory sequences during SWRs could provide a convenient mechanism to quickly recall memories during the awake state (Carr et al. 2011).

The dynamics of SWRs and the behavioral correlate of deliberation, vicarious trial and error, are similar: both are initially more frequent and then decrease with learning (Muenzinger 1938; Jadhav et al. 2012). When a rat is first exposed to a novel task, SWRs are prevalent. As the animal learns and becomes familiar with the task, the number of SWRs gradually decreases (Cheng and Frank 2008; O'Neill et al. 2008; Singer and Frank 2009). When a change in the task is introduced, such as exposing the rat to a second novel environment, an increase in the number of SWRs is again observed which then decays with familiarization (Karlsson and Frank 2008). This is consistent with deliberation playing a more important role during early learning, before schemas and habits form. Further, SWRs are more prevalent at low speeds, consistent with deliberation, although they can be seen during higher speed locomotion as well (O'Neill et al. 2006; Cheng and Frank 2008), raising the possibility that in some cases SWR activity could contribute to deliberation during movement.

It is unknown what environmental or behavioral cues trigger SWRs but the locations where SWRs tend to occur may offer some clues. Many SWRs occur at path choice points (Karlsson and Frank 2009; Jadhav et al. 2012), which are also locations where vicarious trial and error is reported (Muenzinger 1938). Other locations where SWRs occur are the destination of a trajectory, usually the reward location, and during brief pauses in locomotion (Kudrimoti et al. 1999; Foster and Wilson 2006; Jackson et al. 2006; Diba and Buzsaki 2007; Davidson et al. 2009; Singer and Frank 2009; Dupret et al. 2010). These are all locations where decisions are made about where to go next in order to find reward. We suggest that activity at these locations corresponds to a decision making state that biases the hippocampus to initiate SWRs.

If SWRs are critical for learning and deliberation, the reactivated memory should encode trajectories or include locations relevant for active navigation. One important location, useful for the decision at hand, is the current location of the rat. This is in fact the case, as decoding of place cell reactivation during SWRs shows replayed trajectories are biased to include the current location of the animal (Diba and Buzsaki 2007; Davidson et al. 2009; Karlsson and Frank 2009; Pfeiffer and Foster 2013). Mechanistically, the place cell coding for the animal's current location is active and could seed the activation of a chain of interconnected place cells, giving rise to trajectories starting from or pointing to the current location. This network property provides a convenient means to initiate deliberation with relevant choice options from the current location. The lengths of replayed trajectories have been reported to span several meters, which represent realistic trajectories in the environment (Davidson et al. 2009; Pfeiffer and Foster 2013). Long trajectories can also be represented across multiple SWRs, with each SWR reactivating a ion of the entire trajectory (Davidson et al. 2009). This suggests each SWR encodes a discrete trajectory, similar to an episode of an experience.

The goal location is another important feature when planning trajectories since a desirable trajectory should end at the goal, normally where a reward is found. Evidence for goal location representation comes from tasks where a rat is allowed to explore an open field to find a hidden reward, without having to follow predefined tracks (Dupret et al. 2010; Pfeiffer and Foster 2013). Since these reward locations change over time, the rat needs to actively form and use memory in order to efficiently obtain reward. Awake replay events that occur during these tasks reactivate place cells that represent reward locations (Dupret et al. 2010) and contain trajectories that can be biased to terminate at the goal (Pfeiffer and Foster 2013). This suggests replay events encode trajectories that re useful for making navigational decisions towards a goal.

We have investigated the content of replay events with respect to deliberation in our W-track alternation task. In this task the rat must learn to visit each of the side arms from the center arm of the maze in an alternating pattern (Figure 1). To perform the task correctly, the rat needs to learn and apply two rules, each appropriate for specific locations. When the rat is on the center arm, it has to make an “outbound” decision. This involves choosing the side arm that was not visited in the previous trial. To apply the rule correctly, the rat needs to recall from memory the arm visited in the past trial to choose the correct arm for the next trial. When the rat is on either side arm, it has to make an “inbound” decision, which involves returning to the center. The “inbound” decision requires the rat to always return to the center arm when it is on either of the side arms. The “inbound” decision is more similar to an association between place and action since it does not depend on the history of previous visits.

Figure 1.

The W-track alternation task. To perform the task and continuously receive reward, the rat must visit the arms in an alternating pattern. An inbound trial starts on the outside arm and ends on the center arm. An outbound trial starts in the center arm and ends on the outside arm.

Deliberation implies choosing between different options. This is also evident in the range of replayed trajectories decoded during SWRs. Replayed trajectories are not only limited to those eventually taken by the rat but also those not taken. For “outbound” decisions on the W-track, replay of trajectories for both the correct and incorrect arm occur equally often (Singer et al. 2013). In the open field foraging task, the replayed trajectories are not replicas of the actual future trajectory but are enriched for ones that point in the direction of the reward (Pfeiffer and Foster 2013). Both findings suggest replay events provide potential trajectory options, not exclusively the one that is taken.

Even more intriguing are findings that replayed trajectories could reflect trajectories from another similar environment (Karlsson and Frank 2009) or in rare instances, novel routes in the local environment that have not been experienced by the animal (Gupta et al. 2010). In humans, the hippocampus is implicated in the retrieval of memories that are considered remote, such as autobiographical memory (Nadel et al. 2003; Addis et al. 2007). These observations suggest the hippocampus could allow exploration of potential choices far beyond what is local or purely based on recent experience. This mechanism could be important especially in novel situations when the immediate course of action is not apparent. This process shares hallmarks of imagination, the mental function where existing memories are arranged to form simulations of novel situations. The ability of the hippocampus to reactivate or rearrange existing experiences to generate novel sequences has been suggested to support imagination and creativity (Dudai and Carruthers 2005; Schacter and Addis 2007; Schacter et al. 2007; Buckner 2010). By accessing memories of remote experiences that share even minimal similarities with the current experience, or even combining memory to form novel situations, the hippocampus could provide options for the brain to try out. Thus, the memory functions of the hippocampus may be repurposed in to provide a flexible system that would support rapid behavioral adaptation and access to a wide range of possible choices.

Finally, results from our laboratory provide a direct link between SWRs and memory-guided decision making. In our study we interrupted SWRs while a rat learned the W-track alternation task (Jadhav et al. 2012). When SWRs are interrupted while the rat is learning the task, performance on the outbound trials is impaired. In the context of deliberation, disruption of SWRs may have prevented the recall of choice options through replay (Figure 2). This is especially important during the learning stage as the rat has not acquired the alternation rule and is unsure of whether to choose the left or right arm. Normally, SWRs may reactivate place cell sequences that correspond to trajectories to each of the side arms, both potential future alternatives trajectories. When SWRs are disrupted, memory for the alternatives would not be available to inform the decision. Interestingly, unlike the impairment in “outbound” performance, SWR interruption does not impair performance on the “inbound” trials. This result may be explained by the “inbound” rule not needing information from memory about the previous trial. Deliberation is not required since a place-response rule of always return to center arm if on the outside arm is sufficient.

Figure 2.

SWRs could provide choice options for deciding which outside arm to visit when making an outbound decision. Potential trajectories are reactivated during SWRs during the decision-making period. When SWRs are disrupted, the choice options may not be available at the time of deliberation, impairing the decision-making process.

The results from studies described above, involving different tasks with a wide variety of spatial topologies, suggest memory replay during SWRs fulfills the requirements for a neural signature that supports deliberation. We suggest that replay events are a manifestation of the recall of episodes of past experience that form the alternatives for the exploration of potential future choices. Our findings that, during learning, replay is not consistently biased towards the correct versus the incorrect future option (Singer et al. 2013) imply that some other brain areas must be required to evaluate these possibilities to determine which one represents the best choice. Regions of the PFC are likely to be critical for this evaluation.

Hippocampal-cortical interaction and deliberation

The PFC consists of a set of anatomically interconnected regions with similar patterns of connectivity across species (Uylings and van Eden 1990; Uylings et al. 2003; Seamans et al. 2008). These regions have been shown to be important in adaptive behavior and decision making (Chudasama 2011; Kesner and Churchwell 2011; Euston et al. 2012) and rats with inactivated medial PFC and hippocampus are unable to make correct choices between rewarded or unrewarded locations based on memory (Kesner 1989; Floresco et al. 1997; Ragozzino et al. 1998; Seamans et al. 2008).

The areas of the rodent PFC likely to be important for deliberation include the anterior cingulate, medial and ventral medial PFC; also referred to as the pre- and infralimbic cortices (Kesner and Churchwell 2011). Anatomically, the hippocampus sends projections from CA1 and subiculum directly to the prelimbic cortex (Swanson and Cowan 1977; Jay et al. 1989; Van Groen and Wyss 1990; Verwer et al. 1997; Degenetais et al. 2003; Tierney et al. 2004) and also indirect connections from CA1 via the anterior thalamic nuclei (Cenquizca and Swanson 2007; Prasad and Chudasama 2013; Varela et al. 2013). The direct pathway consists of projections from a population of hippocampal pyramidal cells (Klausberger and Somogyi 2008) that synapse onto both pyramidal (Carr and Sesack 1996) and local inhibitory interneurons (Gabbott et al. 2002) in the PFC. Consistent with the diversity of targets, anaesthetized experiments have shown PFC network responses to hippocampal stimulation is complex (Laroche et al. 1990; Jay et al. 1992; Takita et al. 2013).

Hippocampal - PFC interactions during theta oscillations

The hippocampus and PFC interact at theta frequencies during exploration. Some PFC units show phase locking to hippocampal theta oscillation and moreover fire after hippocampal units (Siapas and Wilson 1998; Hyman et al. 2005). Theta phase precession, similar to that seen in hippocampal place cells, has also been observed in PFC units (Jones and Wilson 2005a). This suggests a potential mechanism where information is exchanged through oscillatory synchrony between hippocampus and PFC during behavior. The importance of theta modulated firing in the PFC is supported by observations that coherence in the theta frequency range of the local field potential (LFP) between the two regions increases at the choice point of a maze, where the animal has to choose between arms (Jones and Wilson 2005b; Benchenane et al. 2010; Remondes and Wilson 2013).

Importantly, this increase in coherence is correlated with performance and is significantly higher after the animal had learned the rule. Once behavioral performance plateaus, the increase in coherence is higher preceding the choice point for a correct choice (Jones and Wilson 2005b). Based on these findings, we suggest that theta coherence reflects transmission of information about current location and local trajectory from the hippocampus to the PFC, perhaps to ensure a learned rule is appropriately executed given the current behavioral state and physical location of the animal.

The only instance where activity during theta is thought to be consistent with deliberation is that place cells with place fields a head of the animal's position become active at choice points during vicarious trial and error in the absence of SWRs (Johnson and Redish 2007). What effect these non-local activity in the hippocampus has on the PFC remains to be determined. This could be an intriguing mechanism for hippocampal-cortical interaction during theta that supports the exploration of future trajectories but further characterization is needed to fully understand the function of these representations.

Hippocampal - PFC Interaction during Sharp-wave ripples

We suggest that theta-related coherence may be better suited to support behavior in well learned tasks when deliberation is no longer necessary. We therefore consider activity in the PFC time locked to hippocampal SWRs as a potential candidate for deliberative decision making. At present, however, coordinated activity between PFC and hippocampus at the time of SWRs has been described in detail only during sleep. The PFC shows time locked LFP changes to SWRs depending on sleep state (Siapas and Wilson 1998; Sirota et al. 2003; Battaglia et al. 2004; Molle et al. 2006; Wierzynski et al. 2009). Similar to the hippocampus, both the presence of cell assemblies representing progression through a task (Fujisawa et al. 2008) and the reactivation of these cell ensemble sequences have been reported in the PFC (Euston et al. 2007). Importantly, specific representations in the PFC can be observed at the time of hippocampal SWRs (Peyrache et al. 2011). These observations suggest that SWRs can drive PFC activity during sleep, and suggest that this could also occur during the awake state. That possibility is consistent with a recent study that reported an increase in BOLD activity of primate PFC following SWRs and in one animal this was observed in the awake state (Logothetis et al. 2012).

The strong activation of PFC following SWRs during sleep and the data from the primate fMRI study suggests that interaction between the two regions at the time of SWRs could be a mechanism where, during waking behavior, reactivated representations from the hippocampus are used for deliberation. To understand how this might work, it is important to consider that one function of the PFC that is especially relevant for deliberation is its capacity to temporarily maintain representations, referred to as working memory (Goldman-Rakic 1995). In tasks during which a subject is required to memorize a cue over a delay, neurons in the PFC become active during the delay (Fuster and Alexander 1971) and the pattern of activity is contingent on the context of the task (Kubota and Niki 1971; Romo et al. 1999).

This mechanism could provide the solution to temporarily storing representations, which are then used for decision making (Curtis and Lee 2010). Using the W-track alternation task as an example, the rat has to choose the opposite arm to the previously visited one. In order to deliberate, we hypothesize the brain needs to maintain at least two types of representations: the previously visited arm and also the potential choice options for the upcoming decision. In this context, the PFC would become a buffer for active memory traces that must remain accessible at the time of decision making. Representations reflecting previously chosen options have already been observed in the PFC (Sul et al. 2010). This suggests that a memory trace of the just visited arm, in the form of a unique ensemble of PFC neurons, could also be maintained in the PFC during deliberation for the next choice. Representation of additional possible future choices would come from the hippocampus during SWRs. For example, as the rat is about to make the next outbound decision, the hippocampus could reactivate a sequence of place cells representing a trajectory of one of the possible arms (Figure 3). A subset of these reactivated pyramidal cells will have projections to the PFC and act as the conduit of patterned information. In this case the integrity of patterned information from the hippocampus in the form of cell ensembles representing precise trajectories, would have to be preserved in the PFC. On the W-track, even though the two possible outbound trajectories share a common middle arm, the hippocampal replay for each of the arms will consist of overlapping cells but also non-overlapping cells that represent the outer arms. Thus the overall hippocampal representation for each of the potential trajectories is different and could be distinct enough to modulate different subsets of PFC neurons to form a distinct PFC representation for each potential trajectory. We note, however, that exactly how the sequential activation of hippocampal cells associated with memory replay would engage PFC cells remains unclear; it could be that each trajectory is represented by a distributed but non-sequential PFC pattern, for example.

Figure 3.

A proposed mechanism for deliberation in the W-track alternation task. The hippocampus recalls alternative choice trajectories from memory and provides this serially to the PFC. The PFC compares the option against a rule for the task and a representation of the previous trajectory stored in working memory. If the comparison conforms to the rule, a decision is made to accept that option. In case it does not, this process is reiterated.

Whatever the nature of the representation, we would expect that PFC computations related to rule learning (Euston et al. 2012) would then be employed to evaluate the specific option from the hippocampus. The rule of the W-track alternation task is to choose the arm not visited in the last trial when making an outbound decision. Learning is required to formulate this rule over time but any stage during learning, an imperfect rule based on previous experiences could be sufficient to evaluate the choices provided by the hippocampus. The PFC could apply the rule when comparing the reactivated trajectory to the previous trajectory in working memory and decide whether the reactivated trajectory is the correct choice.

More generally, evaluation of choice options is a critical part of deliberation. In the W-track alternation task, evaluation of each trajectory alternative means whether it conforms to the alternation rule. However, deliberation may involve alternatives that are not strictly correct or incorrect, but differ in terms of desirability. Evaluation of these alternatives implies comparing each one in order to find the most appropriate choice. The parameters involved in determining the best alternative include expected reward, perceived cost and uncertainty; all of which are influenced by the internal behavioral state, such as motivation, at the time of decision-making. Activity correlated with these decision parameters has been found in the frontal cortices, including the PFC (Sugrue et al. 2005; Padoa-Schioppa and Assad 2006; Mainen and Kepecs 2009; Sul et al. 2010; Rushworth et al. 2011; Wallis 2011). Additionally, the PFC is reciprocally connected to the striatum (Groenewegen et al. 1997; Vertes 2004), which is involved in forming action-outcome associations (Penner and Mizumori 2012). The ventral striatum in particular is thought to direct the evaluation of actions in instrumental learning. Further, SWRs trigger both changes in activity in various regions in the frontal cortices (Logothetis et al. 2012) and reactivation of reward associated firing in the ventral striatum (Lansink et al. 2009). We therefore suggest a potential evaluative mechanism: a reactivated trajectory in the hippocampus could reinstate associated representations across these cortical and striatal regions. This global reactivation could occur during deliberation, akin to simulating the result of choosing that option, thus generating an overall evaluation signal that is used to make the decision. Each alternative is evaluated until one reaches a decision threshold.

We note that the hypotheses above posit that choices are evaluated in series, rather than in parallel. This one experience per SWR model predicts that a series of SWRs serve to recall multiple choices that can be compared. That possibility is based in part on the available evidence, which suggests that replayed sequences resemble individual trajectories rather than a mixture (Karlsson and Frank 2009). This model would require the brain network evaluating these options to maintain concurrent representations of all the options at the time of the evaluation process in order to select the most appropriate one. Maintaining multiple concurrent representations in the same network without interference is also non-trivial. However, a recent primate study hints at such a mechanism. Multiple sensory input could be simultaneously represented by the PFC network while the decision could be selectively made on a particular subset of the representation (Mante et al. 2013). The possibility that multiple options are evaluated in parallel would have the benefit of promoting faster decision-making.

Finally, while we have focused on the potential mechanisms for hippocampal input to influence PFC activity, feedback from the PFC may also modulate the hippocampus during deliberation. Previous work has demonstrated a PFC - hippocampus interaction, where lesioning and pharmacological inactivation of PFC activity affects hippocampal place fields (Kyd and Bilkey 2003; Kyd and Bilkey 2005; Hok et al. 2013). While it is unclear if and how PFC feedback maybe involved specifically in deliberation, two pathways have been identified that may relay information from the PFC to the hippocampus. The PFC projects directly to the entorhinal cortex (Vertes 2004) and also indirectly via the nucleus reuniens, situated in the midline of the thalamus (Hoover and Vertes 2012; Cassel et al. 2013; Varela et al. 2013). These may provide the anatomical substrates for feedback from the PFC, but further work will be required to understand the role of these projections in deliberative behaviors.

Conclusion

Decisions often involve the selection of one option from many alternatives. In this process memory of outcomes of past experience is critical in guiding future choices, particularly in situations, such as learning, where deliberation is beneficial. Based on the criteria for patterns of neural activity expected for a deliberative process, we suggest that hippocampal replay during SWRs is a neurophysiological mechanism underlying internal exploration and evaluation of choices during deliberation. It is entirely possible alternative deliberation mechanisms involving the hippocampus and PFC exist, particularly during periods of active locomotion when SWRs occur less frequently. Through its interaction with regions such the PFC, hippocampal replay could engage other networks in evaluating potential choice options on a timescale appropriate for making decisions.