Selective Theta-Synchronization of Choice-Relevant Information Subserves Goal-Directed Behavior

Abstract

Theta activity reflects a state of rhythmic modulation of excitability at the level of single neuron membranes, within local neuronal groups and between distant nodes of a neuronal network. A wealth of evidence has shown that during theta states distant neuronal groups synchronize, forming networks of spatially confined neuronal clusters at specific time periods during task performance. Here, we show that a functional commonality of networks engaging in theta rhythmic states is that they emerge around decision points, reflecting rhythmic synchronization of choice-relevant information. Decision points characterize a point in time shortly before a subject chooses to select one action over another, i.e., when automatic behavior is terminated and the organism reactivates multiple sources of information to evaluate the evidence for available choices. As such, decision processes require the coordinated retrieval of choice-relevant information including (i) the retrieval of stimulus evaluations (stimulus–reward associations) and reward expectancies about future outcomes, (ii) the retrieval of past and prospective memories (e.g., stimulus–stimulus associations), (iii) the reactivation of contextual task rule representations (e.g., stimulus–response mappings), along with (iv) an ongoing assessment of sensory evidence. An increasing number of studies reveal that retrieval of these multiple types of information proceeds within few theta cycles through synchronized spiking activity across limbic, striatal, and cortical processing nodes. The outlined evidence suggests that evolving spatially and temporally specific theta synchronization could serve as the critical correlate underlying the selection of a choice during goal-directed behavior.

Introduction

There is increasing knowledge about the neuronal representation of subcomponent processes underlying goal-directed behavior (Wang, 2008). However, it has remained unclear how component processes for goal-directed behavior, including the retrieval of stimulus evaluations and reward expectancies, the retrieval of long-term and prospective memories, the representation of contextual task rules, and the assessment of sensory evidence are integrated to select and pursue one goal among alternatives (Johnson et al., 2007). One empirical barrier to unravel the mechanisms for integrating multiple subcomponent processes lies in the distributed organization underlying many neuronal functions: The different domains of reward evaluations, memory and cognitive control are represented by neuronal clusters within widely separate brain circuits including the hippocampus, striatum, frontal, and parietal cortex, among many others (Pennartz et al., 2009; Haber and Knutson, 2010). Each cluster of neurons is thought to serve as a functionally specific computationally unit (Douglas and Martin, 2007), whose local operations requires information from other clusters. Thus, local microcircuit operations depend on distant computational outputs, which calls upon a temporal reference to coordinate and structure information exchange through spiking activity.

Here, we survey evidence that structuring neuronal information processing is subserved by a common state of rhythmic theta activity: Theta rhythmic states fulfill all major criteria to serve as a general integrative mechanisms (Buzsaki, 2006; Wang, 2010): (i) Theta activity is pervasive in neuronal systems, emerging not only in almost all areas of the brain from subthalamic nuclei, to thalamic relays, the limbic system (including the hippocampus), striatum, and neocortex; (ii) Theta rhythmicity is generated and sustained at the level of cellular membranes in many neuron classes; (iii) The theta cycle structures the spike timing of many neurons as is evident in theta phase locked spike output (e.g., Klausberger and Somogyi, 2008); (iv) Theta phase locked spiking output conveys specific information beyond the average firing rate (Jensen and Lisman, 2000; Huxter et al., 2008; Kayser et al., 2009; Hyman et al., 2010; Panzeri et al., 2010). Moreover, as shown below, (v) local theta activities synchronize between distant areas indicating that excitatory long-range projections functionally couple local and distant computations; (vi) Theta synchronization can occur specific in space and time, with theta coherent activity being capable to filter and prune out-of-phase firing neuronal groups and to trigger high-frequency coherent gamma-band activity.

These six physiological characteristics suggest that theta rhythmic activity serves as a central neuronal mechanism supporting cognitive functions. It has been suggested that theta oscillations could achieve such a task because it “can group and segregate neuronal assemblies and […] assign various computational tasks to them” (Buzsaki, 2002). Here, we survey recent literature, which convincingly link theta rhythmic activity to function. There are several recent reports that particularly suggest a behavior predictive role of theta activity, beyond that of phase locking of spikes to the theta rhythm. These recent insightful reports show that functionally predictive theta phase coherence is (i) internally generated without external triggers, (ii) is occurring with high temporal specificity around “decision points,” (iii) allows identification of functional assemblies of neurons that would have been undetected without a temporal theta reference, and (iv) that theta coherent network states have direct impact on single neuron effectiveness within local neuronal clusters.

In the following we begin by surveying these key findings and outlining the varying spatial scales at which theta activity is observed and measured in neurophysiological recordings. We would like to note that the discussion of functional correlates of theta rhythmic states will be restricted to retrieval and reactivation processes: It is beyond the scope of this article to either touch on the functional involvement of theta activity in exploratory behavior (Fries, 2009), in sequencing action plans during instrumental learning (Kimchi et al., 2009), in learning and encoding processes (Singer, 2009), or to lay out in any detail the increasingly concise evidence for theta phase specific encoding operations (Hasselmo, 2005; Lisman and Buzsaki, 2008; Zilli and Hasselmo, 2008).

Theta Rhythmic States at Single Neuron Membranes and Large Scale Networks

Theta rhythmic activity is typically measured in the local field potential (LFP) and hence reflects the summed transmembrane current flow among a larger group of neurons. However, the mechanisms underlying the generation and maintenance of theta rhythmic states are complex, and best understood for theta oscillations arising in the hippocampal formation (Wang, 2010). The theta rhythm in the hippocampus in vivo consists of a rhythmic series of somatic inhibitory postsynaptic potentials and dendritic depolarizing potentials (Leung and Yim, 1986; Kamondi et al., 1998; Buzsaki, 2002). Various mechanisms, including extrinsic drive from the medial septum, local hippocampal interneuronal circuitry, and intrinsic properties of the neurons may all underlie theta synchronized single neuron firing (Klausberger et al., 2003; Klausberger and Somogyi, 2008; Senior et al., 2008; Harvey et al., 2009). With regard to external origins of hippocampal theta, it has long been known that theta rhythmic firing persists in the medial septum even after the theta field rhythm is abolished in the hippocampus in vivo (Petsche et al., 1962). Inactivation or lesion of the medial septum abolishes the theta rhythm, giving rise to the initial concept that the medial septum is the pacemaker of the hippocampal theta activity (Stewart and Fox, 1990). However, independent theta rhythmicity has now been proposed for the majority of processing nodes in which it has been investigated including thalamic and deeper nuclei (McNaughton et al., 1995; Kirk et al., 1996; Woodnorth et al., 2003), colliculus (Pedemonte et al., 1996), rhinal cortex (Bilkey and Heinemann, 1999; Collins et al., 1999; Mizuseki et al., 2009), amygdala (Seidenbecher et al., 2003; Popescu et al., 2009; Rutishauser et al., 2010), cingulate cortex (Feenstra and Holsheimer, 1979; Borst et al., 1987; Talk et al., 2004; Hyman et al., 2005; Tsujimoto et al., 2006, 2010; Young and McNaughton, 2009; Womelsdorf et al., 2010), dorsal striatum, nucleus accumbens, and ventral tegmental area (Tabuchi et al., 2000; Kocsis et al., 2001; DeCoteau et al., 2007a,b; Lansink et al., 2009), and is found in all four lobes in the neocortex (see also below) (Cantero et al., 2003; Sirota et al., 2008).

However, it remains unclear how theta rhythmicity in the different brain areas is generated, and how they may be synchronized. In vitro studies have suggested several mechanisms at the level of the membrane of a single neuron and/or at the level of local neural circuits (Wang, 2010). At the level of single neuron membranes, depolarization of a single neuron induces membrane potential oscillations (MPOs) that can be subthreshold or resulting in action potentials (Alonso and Llinas, 1989; Leung, 1998; Chapman and Lacaille, 1999a,b). When low level of spiking occurs, spikes fire near the peaks of the MPOs. An intrinsic membrane potential oscillator may consist of a depolarizing current (Na⁺ or Ca²⁺) and a hyperpolarizing K⁺ current. Some neurons may not be oscillators, but may have membrane properties that increase their responsiveness (resonance) during theta rhythmic states (Gutfreund et al., 1995; Leung and Yu, 1998; Hutcheon and Yarom, 2000; Hu et al., 2002; Wang, 2010). Local circuits participate in the generation of theta field oscillations. Hippocampal principal cell firing synchronizes a network of inhibitory neurons that pace MPOs at theta frequency (Cobb et al., 1995). These local-circuit rhythms give rise to the robust and spontaneous theta field oscillations in the hippocampal CA1 subfield (Goutagny et al., 2009), and are abolished by blocking recurrent inhibition with an ionotropic glutamatergic antagonist or a GABA-A receptor antagonist. Additionally, the cholinergic agonist carbachol has long been known to activate theta-frequency field oscillations in hippocampal slices in vitro (Bland and Colom, 1993; Leung, 1998). The carbachol-induced rhythm is abolished by an ionotropic glutamatergic antagonist but not by a GABAergic antagonist (Traub et al., 1992).

Parallel to the ongoing clarification of the multiple mechanisms generating theta activity at local spatial scales, evidence in recent years document convincingly inter-areal theta coherent networks composed of multiple distant areas, including orbitofrontal–prefrontal–cingulate cortex (Tsujimoto et al., 2006, 2010), frontal to midline areas (Young and McNaughton, 2009), striatum to neocortex (Sharott et al., 2009) and with multiple studies showing theta coherence between subfields of the hippocampus with entorhinal cortex (Mormann et al., 2005, 2008b; Mizuseki et al., 2009), medial and lateral prefrontal cortex (Siapas et al., 2005; Wierzynski et al., 2009; Adhikari et al., 2010), parietal cortex (Sirota et al., 2008), ventral striatum (DeCoteau et al., 2007b; Berke, 2009; Lansink et al., 2009), cingulate cortex (Young and McNaughton, 2009), amygdala (Seidenbecher et al., 2003), and nucleus accumbens (Tabuchi et al., 2000).

The abundant existence of theta states, inter-areal theta networks, and single neuron membrane oscillations suggests that theta activity indexes a critical and possibly fundamental function for neuronal information processing. The following will survey the respective evidence, beginning with a recent study demonstrating functional consequences of theta activity at all three spatial scales of processing.

Theta Synchronization Underlying Selective Retrieval of Choice-Relevant Information

Component processes underlying goal-directed behavior and behavioral choices

During awake and active states, neuronal activation patterns are prepared to evaluate input to link it to motor action. This prepared state is frequently governed by a behavioral goal of the organism, which per definition biases neuronal circuits to process and evaluate sensory inputs according to their values to achieve the goal (Miller and Cohen, 2001; Glimcher et al., 2009). Evaluating the value of information in relation to a behavioral goal involves a multitude of processes. These processes are reflected in neuronal activation patterns and are supposed to take place around the time when the organism is forming the decision about which behavioral route to take in the face of multiple alternatives. Among the multitude of processes involved, at least four major component processes have been experimentally well isolated and linked to rhythmic theta synchronization within and between neuronal circuits: (i) The evaluation of stimulus–reward associations, (ii) the retrieval of working memory representations, (iii) the retrieval of long-term memories relevant to inform the expected value of possible choices, and (iv) the representation of the task rules, which govern how incoming sensory input map onto possible responses. In the following, we show that these component processes are associated with and possibly realized by rhythmically synchronizing neuronal groups in distant brain areas at theta frequency with high temporal and spatial specificity. In particular, the outlined evidence suggests that multiple types of choice-relevant information is structured by the theta cycle as sketched in Figure 1. The following will document the accumulated evidence supporting the hypothesis that theta synchronization across multiple areas could be a neuronal prerequisite which underlies the actual formation of a choice.

Figure 1.

Illustration of temporally coordinating the retrieval of choice-relevant information through theta rhythmic activity. The blue line refers to the theta activity not in one particular “decision” area, but to local theta rhythms in all those areas whose concerted spike coordination influence which choice is made. The gray shaded area indicates the time window of the theta cycle at which spike output is generated and received from participating neurons. The theta rhythmic retrieval hypothesis suggests that a choice is formed when all relevant sources of evidence (arrows) functionally activate around a similar phase of the theta cycle (solid arrows at the time of choice), which takes three cycles in the sketched outline. Empirical evidence for theta synchronized information of context, reward, memory, and sensory evidence is surveyed in the text.

Selective inter-areal theta synchronization at decision points

A decision point describes a point in time and space shortly before subjects choose to select one alternative action over another. Approaching a decision point is therefore an information rich period during which choice-relevant information is gathered and integrated. The outcome of this integration can be tracked in time and is often conceptualized in race models of decision making as the temporal accumulation of evidence (Gold and Shadlen, 2007). In the rodent literature, decision points can be identified in spatial tasks around the time when a rodent approaches a junction within a maze. Shortly before reaching the junction, a decision has to be formed whether a rightward, or leftward turn will lead to reward. Intriguingly, around the decision point several recent studies have reported selective inter-areal theta rhythmic synchronization predictive of the future choice and varying in close correspondence with learning the choice (Jones and Wilson, 2005a; DeCoteau et al., 2007b; Benchenane et al., 2010). In one study, Benchenane et al. (2010) recorded neuronal activity in the hippocampus and prefrontal cortex of the rat during the learning of location–reward associations in a Y-maze. Neuronal groups in hippocampus and prefrontal cortex phase synchronized their theta oscillations when the rat traversed the decision point (Figure 2A), similar to previous reports (Hyman et al., 2005, 2010; Jones and Wilson, 2005a,b; Siapas et al., 2005). Thereby, neuronal spike output of prefrontal neurons became more strongly phase locked to hippocampal theta oscillations while the rats learned the task, with the strength of theta coherence at the decision point closely correlating with the accuracy of the choice (Figure 2B).

Figure 2.

Hippocampus–prefrontal theta coherence at decision points in the rodent during a spatial–reward association task. (A) Inter-areal theta coherence emerges before the junction of the Y-maze. (B) The strength of theta coherence at the decision points goes along with accuracy to choose the correct direction to receive reward. (C) Proportion of theta modulated cell pairs with significant spiketrain correlations (y-axis) before and after learning the task. Red/Blue bars denote correlations for high/low theta coherence periods. (D) Cross-correlation of inhibitory and excitatory neurons show more effective inhibition (at short time lags, x-axis) during periods of high versus low theta coherence (bottom/top panels). (E) During theta rhythmic coherence (top panel), spike timing of pyramidal cells (bottom panel) phase aligned. Adapted with permission from Benchenane et al. (2010).

Intriguingly, the degree of inter-areal theta coherence did not only predict choice behavior, but also allowed to identify subsets of theta modulated neurons in the prefrontal cortex, whose spike trains became strongly correlated at a much faster time scale and particularly after the rats had learned the correct spatial choice (Figure 2C). This finding reveals that the occurrence of inter-areal theta coherence has highly selective consequences for the functional coupling among neurons within the target area: A small group of neurons formed a correlated assembly only during periods of enhanced theta coherence, but was otherwise not distinguishable from other functionally uncorrelated neurons. In other words, the small subset of functionally contributing neurons would have been undetected without the temporal theta reference. A similar conclusion has been arrived at with a recent error analysis of working memory performance in rats (Hyman et al., 2010). Correct and error trials could not be distinguished by firing rate changes in prefrontal cortex, but were distinguished by the lack of phase locking to hippocampal theta oscillations in a subset of theta modulated cells (Hyman et al., 2010). The implications of these findings are pervasive, suggesting that the identification of task selective neurons among the multitude of task-modulated and task-unmodulated neurons critically relies on a temporal frame of reference measured in the LFP.

Theta coherence, however, did not only reveal functionally selective subsets of neurons. It also had two more immediate consequences for individual neurons: First, theta coherence gated the effectiveness of inhibition within the local network. As illustrated by cross-correlation analysis, inhibitory interneurons transiently suppressed the firing of neighboring pyramidal cells particularly during high coherent theta states, which was evident even when controlling for the average firing rate between high and low theta states (Figure 2D). Thus, theta rhythmicity controlled local inhibition (Cardin et al., 2009). Secondly, periods of enhanced theta coherence shifted the phase of firing of pyramidal neurons to a narrow phase range close to the theta trough and about 180° away from the preferred firing phase of interneurons. In other words, theta rhythmic states between hippocampus and prefrontal cortex essentially synchronized pyramidal spike output in prefrontal cortex (Figure 2E). Such a phase alignment influences local computations as evidenced by the finding that subsets of theta modulated pyramidal cells formed a correlated assembly. As a consequence, this functionally defined assembly will be highly effective in influencing other target areas to which the theta synchronized pyramidal neurons connect. The consequence is that theta rhythmic information conveyed by local computations is made available to all those connected brain areas to which pyramidal cells project – being capable in principle to entrain and link to a larger theta coherent network conveying task selective information.

The described findings raise the critical question about the nature of local computations supporting the choice during decision point periods. For the discussed study (Benchenane et al., 2010) the basic component processes was a learned reward–location association that was retrieved at the decision point. Thus, the experiment isolated a spatial memory component and a reward component, which needed to be retrieved and activated in working memory to inform the correct choice. For these processes, the hippocampus and the prefrontal cortex are contributing mutually dependent computations, but likewise contribute separable components: Among the separable components is most likely the highly selective spatial and navigational code within the hippocampus (McNaughton et al., 2006a; Huxter et al., 2008; Itskov et al., 2008). Similarly, prospective activity may be crucially dependent on the hippocampus. In a recent study, discharges of hippocampal neurons recorded while rodents were pondering at the decision point before a T-junction of a maze were found to pre-play sequential action potentials as if the animal would have already chosen one arm of the maze to collect reward (Johnson and Redish, 2007). This internalized, prospective code vanished with learning the correct choice, suggesting that pre-play activity is an intriguing signature of initial learning through actively retrieving (imagining) future outcomes relevant for the choice to be made at the decision point. Interestingly, imagining future visuo-spatially rich scenes is dependent on the hippocampus also in humans (Hassabis et al., 2007). However, a similar analysis has probably not been applied to prefrontal cortical neurons, preventing to assign a unique and separable function to either area. Similarly, both areas are activated in spatial working memory tasks and have been shown to host internally generated, sustained (delay) activity indicative of short-term memory (Jones and Wilson, 2005a; Pastalkova et al., 2008; Hyman et al., 2010). Moreover, both areas convey reward-predictive signals. Indeed, reward-predictive signals in frontal cortices have been documented to vanish following disconnection from the hippocampus (Ramus et al., 2007), documenting the integrative nature of the involved processes.

This brief discussion pinpoints the requirement for temporal coordination of mutually interacting local computations from distant brain areas. In the example above, the selective theta synchronized computation reflected the retrieval of learned spatial–reward associations. However, in the following sections we survey further examples of theta synchronized information indicating that choice-relevant information is retrieved just before a choice is made. These subcomponent process involve the representation of stimulus–response (SR) mapping rules in preparation of task performance, the prediction of future rewards, the robust representation of sensory working memory content, and the internally triggered retrieval of longer-term memories.

Theta mediated reactivation of task rules in anterior cingulate cortex

In the previously discussed studies, the rodents were faced with a decision point at the Y-junction of a maze forcing them to select either a rightward or leftward turn conditional on learned spatial–reward association. However, choices often involve more abstract conditional rules, or task rules, on how sensory stimuli have to be mapped onto possible responses in order to receive reward. Approaching a behaviorally relevant stimulus (for the rats: the junction of the maze) is thereby triggering the retrieval of internal representations of the appropriate SR mapping rules under which a choice can then be reasonably formed. Thus, activating the relevant SR mapping rule in a particular situation is a prerequisite for coordinating the retrieval of choice-relevant information. Previous research has shown that the activation and control of SR mapping rules is achieved by neuronal groups in prefrontal and anterior cingulate cortex (ACC), among other areas pertaining to nodes of a larger cognitive control network (Dehaene and Changeux, 2000; Miller, 2000; Johnston et al., 2007).

Importantly, the neuronal representation of relevant SR mapping is structured in time by theta rhythmic synchronization in ACC (Womelsdorf et al., 2010). In this study, macaque monkeys performed blocks of two different tasks requiring them to either saccade toward or away from a peripheral stimulus (pro-saccade and anti-saccade tasks). Both tasks differed only in how a peripheral stimulus (randomly presented in the left or right hemifield) mapped onto a response. The SR mapping stayed constant over 30 consecutive trials before it was switched, forcing the monkeys to retain an internal representation of the currently relevant SR mapping rule and retrieving this representation prior to processing the sensory stimulus for efficient task performance, i.e., during a time akin to a decision point. Notably, at this decision point selective changes in theta activity in ACC predicted which SR mapping the monkey was about to apply upon peripheral stimulus onset (Figure 3A). In spatially separate neuronal groups, theta activity and synchronized spiking to LFP theta emerged for one versus the other SR mapping rule at 0.4 s before peripheral stimulus onset. Interestingly, a previous study reported that task selectivity becomes evident also in average firing rates from 0.4 s before stimulus onset, but the latency of task selective firing changed over trials on the same task in anterior cingulate and prefrontal cortex (Johnston et al., 2007): In ACC, firing rates were signaling the task rule at 0.4 s only in the immediate trials after task rules were switched, but its latency declined with trials on the same rule. Neurons in prefrontal cortex showed the reverse pattern: Task selectivity became apparent at 0.4 s before stimulus onset only after some trials on the same rule (Johnston et al., 2007). This pattern of results shows that LFP theta activity within ACC translated into higher spike counts only when the task demands had recently changed, i.e., after a task rule switch, and when prefrontal neurons did not yet fire selectively for one task versus the other. During the higher task demands to establish a new task rule, LFP theta activity and firing patterns were the earliest task selective signal recorded, likely reflecting a genuine local contribution of ACC circuitry for cognitive control. However, when task demands were established after few trials on the same task rule, the spike output contribution of the ACC declined, suggesting that task selective LFP theta activity reflected partly the input from prefrontal projections or other nodes of a larger cognitive control network.

Figure 3.

Task rule selective theta activity in macaque anterior cingulate cortex just before a “decision point.” (A) Time-frequency distribution of statistical reliable task selectivity of LFP power. Time zero marks the onset of a peripheral stimulus on either right/left side and requiring either a pro-, or an anti-saccade response. (B) Task selective theta activity emerged early in those correct trials, where the task representation had to be re-established following an error response (bottom left, and red line), compared to correct trials following other correct trials (top left and blue line). Adapted with permission from Womelsdorf et al. (2010).

According to this interpretation, theta activity reflects a common mode for flexible communication of local computations within a larger network. The degree to which individual nodes of the network will functionally contribute to the theta network is thereby depending on task demands. When task demands require the local computations from one node of the network, this node is expected to output their local operations by theta synchronized activity. Consistent with this suggestion, significant task selective theta activity in the ACC emerged particularly early in time in those trials that followed error trials, i.e., in trials with an enhanced demand to re-establish the correct task rule (Figure 3B). In contrast, when task demands do not call upon neuronal circuitry in one area, theta rhythmic activity would still be available by somato-dendritic activation of receiving neurons in the area, but local excitation is not sufficient to link spike output to the external mediated rhythmic modulation of excitability.

This scenario entails that the particular function of a cortical area can be inferred from the degree to which neurons synchronize their output to rhythmic theta activity. Accordingly, the ACC is expected to host neuronal circuitry that tracks task demands during performance in order to initiate selective control signals when task rules require re-adjustment (Womelsdorf et al., 2010). This suggestion is consistent with a wealth of previous functional correlations of ACC activation: ACC lesions impair optimizing behavioral choices when reward contingencies changes (Kennerley et al., 2006); ACC neurons are modulated by task difficulty or conflict (as e.g., in incongruent mapping conditions), and when SR rules change following altered reward contingencies, or when previous trials were performed incorrectly (Shima and Tanji, 1998; Kerns et al., 2004; Williams et al., 2004; Mansouri et al., 2009). The discussed results suggest that ACC neurons communicate selective error-correcting signals by synchronized theta output. Interestingly, these signals may well be utilized during working memory maintenance, suggesting a possible link to human scalp recordings implicating the ACC as the source for prominent frontal midline theta activity emerging during working memory processes (Gevins et al., 1997; Asada et al., 1999; Ishii et al., 1999; Jensen and Tesche, 2002), as well as time locked to erroneous responses (Luu et al., 2004; Debener et al., 2005).

Importantly, neuronal signals within ACC, which are selectively recruited for adjusting behavior critically requires that the ACC has immediate access to information about which behavior is adequate during task performance. Rodent studies have revealed that an intact ACC and prefrontal cortex are crucially required to memorize contextual information of previously experienced SR mapping rules (Bontempi et al., 1999; Frankland et al., 2004). These contextual associations are likely mediated by interactions with the hippocampal system and with reward circuitry in the orbitofrontal cortex and striatum. The following paragraphs will illustrate that neuronal clusters in these systems communicate by means of the same theta rhythmicity as shown in the above examples.

Reward anticipation is signaled by theta synchronized neuronal firing

Anticipation of future rewards requires the retrieval of how reward is associated with particular actions. Signatures of reward evaluations are found in the dorsolateral prefrontal cortex and anterior cingulate (Watanabe, 2007; Rushworth and Behrens, 2008), dorsomedial (caudate) and ventral striatum (Schultz et al., 1992; O'Doherty et al., 2004; Johnson et al., 2007), and the orbitofrontal cortex (Padoa-Schioppa and Assad, 2006; Wallis, 2007). In order for reward anticipation signals to modulate decisions, the respective reward-predictive signals need to be activated and made available to the system around the time of the decision. Intriguingly, several recent studies have begun to show that reward anticipation can be selectively decoded from transiently emerging theta rhythmic synchronization between the ventral striatum and the hippocampus and within orbitofrontal cortex (DeCoteau et al., 2007b; van Wingerden et al., 2010; Vinck et al., 2010).

Within rodent orbitofrontal cortex, a recent study has shown that part of the reward predicting signal is the synchronized firing of neurons to an underlying theta activity, evolving in close correspondence to the degree to which stimulus–reward associations are learned and become predictable for the brain (van Wingerden et al., 2010). In this study rodents had to discriminate one of two odors during an odor sampling period in order to decide to either approach or withhold approaching to a food well. When approaching the food well, rodents had to wait for 1 s before the trial outcome was disclosed with either a positive outcome (sucrose reward) upon correctly approaching, or a negative feedback (quinine) when approaching was incorrect. Figure 4A illustrates that during the correct anticipation of reward, theta activity emerged with spike output locked to the positive flanks of the theta oscillation. In contrast, during trials when reward anticipation was not maximal, there was no spike-LFP locking signature in the waiting period Figure 4B. Notably this pattern of enhance spike-LFP phase locking was evident for more than half of the recorded sites within OFC, was unaffected by overt movement confounds, and emerged with high temporal specificity during anticipation (Figure 4C). Interestingly, it was most strongly expressed, both, during the anticipation and consumption of a reward, in a selective group of anticipation cells. Notably, the latency of predictive theta activity varied in close correspondence with the predictability of the reward: When the specific odor associated with the preferred reward was reversed to indicate a non-preferred reward, theta activity decreased and vanished in few trials (Figure 4D). During the same time, trials with the second odor now predicting the preferred reward resulted in progressively increased anticipatory theta activity, evolving initially late during anticipation, and progressively shifting to early periods over trials (Figure 4D). These findings therefore reveal that reward anticipation is available to other systems in theta-synchronized spiking responses. Critically, theta synchronized output from the OFC about the anticipated reward in this described study was indicative of the type of outcome expected, rather than indicating a general state of un-selective reward expectation. Such reward selectivity is required to learn the stimulus–reward association in the first place, but also to inform other neuronal groups about the expected value of possible actions prior to choosing to engage in a particular action.

Figure 4.

Reward anticipation in orbitofrontal cortex conveyed through theta synchronized spike output. (A,B) LFP traces and accompanying action potentials during the anticipation period in example trials when a rodent awaited a sucrose (A), or quinine (B) outcome. Colored/gray lines show theta band filtered and raw LFP traces. (C) Time-frequency distribution of spike-LFP phase locking during the anticipation period (before reward delivery) shows selective theta band synchronization. (D) Timing and strength of theta band activity reverses with reversal of trial outcome associations. y-Axis denote trials relative to reversal; The upper two panels show transition from preferred (sucrose) reward to aversive outcome, while the bottom two panel shows the reversal back to the preferred outcome. Adapted with permission from van Wingerden et al. (2010).

Theta synchronized reward anticipation signals triggered by external cues

Intriguingly, the involvement of theta synchronization for the proposed link of reward prediction and the final choice can be inferred from other areas of the network supporting goal-directed behavior (DeCoteau et al., 2007b; van der Meer and Redish, 2009). In particular, DeCoteau et al. (2007b) recorded from the striatum (medial and lateral caudoputamen) and the hippocampus (CA1 subfield) in rodents learning to use a tone cue while running on a linear track toward a T-junction to choose either a leftward or rightward turn at the junction to receive reward (Figure 5A). The time period between tone cue and turn is reflecting a decision point in which sensory information and reward predictions need to be combined to inform the behavioral choice. At this decision point local theta activity within hippocampus and striatum evolved and synchronized between areas in those rats that learned the correct choice (Figure 5B), but not in rats, which did not learn to use the cue to inform the choice.

Figure 5.

Theta coherence of hippocampal and dorsomedial striatum. (A) Outline of a conditional spatial discrimination task, requiring rats to learn using a tone for choosing a right-/leftward turn in a T-maze. (B) Coherence of LFP activity between striatum and hippocampus during task performance. Inter-areal LFP coherence in the theta band (8–10 Hz) emerges around the time of the tone cue, i.e., at the decision point of the task. The red outlined box indicated the time-frequency region of interest. Note that coherence could not be explained by motor aspects of the task, and that the coherence was only observed in subjects learning the task as compared to non-learners. Adapted with permission from DeCoteau et al. (2007b).

This finding suggests that rising theta activity during cue presentations is the functional signature required to learn and apply the correct choice upon cue presentation. Consistent with such a functional interpretation, the authors report that during learning and low performance periods the theta phase relation between striatal and hippocampal LFP oscillations was highly variable within trials, while after learning and during high performance levels, the inter-areal phase relation remained similar and more reliable throughout the trials (DeCoteau et al., 2007b). This latter finding would be expected if inter-areal theta coherence signifies a functional correlate for establishing an exchange of information between areas. In this regard, it is important to note that at the decision point up to 17% of neurons within the striatum showed significant theta specific spike-LFP coherence (DeCoteau et al., 2007a), showing that local theta influences spike timing and consequently is structuring local computations similar to the findings by Benchenane et al. (2010).

As suggested above, the rising theta synchronization in orbitofrontal cortex and between hippocampus and striatum could be interpreted to reflect an ongoing reward anticipation signal, which conveys selective information about the value for possible choices. This interpretation is supported by further studies in rodent ventral striatum. For example, van der Meer and Redish (2009) has shown that ventral striatal neurons not only encoded reward at actual reward delivery points, but likewise represented reward around the time of decision points and before error correction. A different study has shown that the reward information during learning space–reward associations is the temporally compressed reactivated in ventral striatal neurons, while the spatial component is reactivated during learning particularly in spatially tuned cells in the hippocampus (Lansink et al., 2009). This latter finding suggests that reward associations with sensory features like locations are memorized with high specificity in the ventral striatum.

Utilizing these learned and selective reward signals often proceeds during tasks involving working memory for particular spatial locations or visual stimulus features. Importantly, sustaining reward and sensory selective signals in working memory has been shown to likewise involve theta rhythmic synchronization, which we will briefly outline next.

Theta synchronized working memory content about space and features

One of the primary roles of working memory is to keep memorized activity in a short-term memory buffer before it is used to inform a choice (Wang, 2008). Experimentally working memory is often studied in variants of delayed (non-)match-to-sample tasks. Persistent sample-specific activity during the delay emerges early on in the prefrontal cortex during the encoding stage, and can be maintained over extended time periods even in the presence of distracting sensory inputs (Miller, 2000). Despite the well documented relevance of prefrontal cortex for a robust short-term memory buffer, only few studies have implicated rhythmic theta synchronization within prefrontal cortex to maintain working memory representations, but have begun to implicate lower (delta) and higher (beta and gamma) frequency oscillations to support recurrent retrieval of memories (Buschman and Miller, 2009; Siegel et al., 2009). However, neuronal spike output of prefrontal cortex neurons can be strongly synchronized to theta activity originating in distant areas like the hippocampus, even without prominent local theta activity (Siapas et al., 2005). Importantly, as suggested above, only a small fraction of prefrontal neurons may be relevant for coding specific working memory content. Small, functionally relevant clusters of neurons have been documented to be theta modulated (Benchenane et al., 2010). Moreover, in a delayed non-match to sample task, only a subset of neurons were theta entrained in rodent prefrontal cortex, and only the lack of theta phase locking (but not their overall firing) allowed to predict an erroneous performance (Hyman et al., 2010). Thus, the absence of a prominent population theta rhythm in prefrontal cortex does not preclude that theta modulated spike timing contributes critically to task performance.

In addition to prefrontal cortex, the hippocampal formation has been implicated in working memory, particularly when multiple items and relational information have to be retained in working memory (Aggleton et al., 1992; Hannula et al., 2006; Olson et al., 2006; Axmacher et al., 2010). Hippocampal neurons are known to internally generate working memory representations, which are structured by theta activity in rodents (Lisman and Idiart, 1995; Jensen and Lisman, 1998; Lisman and Buzsaki, 2008), as well as in humans (Mormann et al., 2008a; Axmacher et al., 2010). In a recent study using a delayed spatial alternation task, in which rodents were kept in a different environment during a several seconds long delay, single neurons locked in a fixed temporal order to sequential cycles of the theta cycle during the delay (Pastalkova et al., 2008). Of particular importance is the finding that the spatio-temporal sequence of theta modulated firing was not triggered by external cues, and hence reflected the internal working memory of the previous chosen location the rodents needed to know in order to choose the alternative location in the trial after the delay. Importantly, theta patterned firing during the delay allowed to predict their performance after the delay, suggesting that sequential activation of an ensemble of hippocampal neurons is the critical functional correlate underlying successful task performance.

In addition to spatial working memory in the hippocampus, working memory for visual stimulus features and objects in visual cortex is likewise conveyed by theta modulated spiking output (Lee et al., 2005; Cashdollar et al., 2009). In visual cortical areas like area V4, persistent activity during delay periods is only sparse. However, selective delay period signals have been shown to emerge gradually toward the end of the delay. Lee et al. (2005) have shown that when the delay approaches the test stimulus onset, spike timing not only becomes gradually phase locked to a theta rhythm, but theta synchronized spiking to the LFP conveys visual information about the sample stimulus. Conceptually, the observed theta modulated spiking activity is a pure reflection of top-down retrieved sensory evidence, because it was absent at the beginning of the delay and hence could not be reactivated by sensory bottom-up inputs. And secondly, because it emerged as the temporal delay approaches the “decision point,” i.e., just before the test stimulus onset, which by definition is a top-down mediated matching choice.

It is important to note that theta modulated spiking occurred spatially selective among neurons separated by about 1 mm, becoming phase shifted and weaker beyond about 3 mm of neuron separation (Lee et al., 2005). The phase shift away from stimulus encoding neurons is particularly interesting: As the theta cycle signifies modulated excitability, top-down input to area V4 will be received at the most excitable theta phase only within a spatially narrow cluster of neurons. The same top-down input will have less influence on neuronal groups further away from target encoding cluster of neurons. Accordingly, selective theta synchronization may reflect the selective communication of top-down and bottom-up information (Womelsdorf et al., 2007; Womelsdorf and Fries, 2009).

The described animal studies about theta synchronized working memory content in hippocampus, prefrontal cortex, and visual cortex are complemented by a wealth of human studies utilizing intracranial recordings, scalp EEG recordings and MEG, and documenting long-range theta band coherence during various classical working memory tasks (n-back, Sternberg task, delayed matching tasks). These studies have documented that long-range theta synchronization selectively evolves during working memory demands among other areas between prefrontal cortex and temporal and parietal cortex (Sarnthein et al., 1998; von Stein and Sarnthein, 2000; Rizzuto et al., 2003; Mizuhara et al., 2004; Meltzer et al., 2008; Cashdollar et al., 2009; Payne and Kounios, 2009), and between hippocampus, frontal cortices, cingulate cortex and temporal cortex (Sammer et al., 2007; Cashdollar et al., 2009). Notably, theta rhythmic modulation during working memory has been shown to be regionally highly specific (Raghavachari et al., 2006). Increasing memory load can reduce theta activity in earlier visual cortex, while at the same time theta rhythmicity in temporal cortex and from frontal midline regions gradually increases with increasing working memory load in a Sternberg task (Michels et al., 2008; see also Krause et al., 2000; McEvoy et al., 2001; Raghavachari et al., 2001; Jensen and Tesche, 2002; Onton et al., 2005; Sauseng et al., 2009a,b). This finding resonates well with the spatial specificity of theta activity observed in area V4 (Lee et al., 2005).

Theta mediated retrieval of memory templates

In contrast to working memory paradigms, longer-term memory studies separate the encoding of relevant sensory information from retrieval periods by a longer time span. The involvement of rhythmic theta synchronization during the encoding (or learning) stage to predict successful memorization has been well documented and involve (i) enhanced theta activity and spike-LFP phase locking at the time of encoding, (ii) theta-phase dependent encoding success, (iii) replay of theta synchronized memories within the hippocampus and its projection areas during the delay (e.g., during REM sleep), and (iv) theta phase dependent synaptic plasticity (Fell et al., 2001, 2003; Sederberg et al., 2003; Hasselmo, 2005; Kahana, 2006; McNaughton et al., 2006b; Paz et al., 2008; Guderian et al., 2009; Lansink et al., 2009; Wyart and Tallon-Baudry, 2009; Duzel et al., 2010; Rutishauser et al., 2010). Discussing these aspects is beyond the scope of this article, which focuses on the theta signatures of internally cued retrieval processes.

Recent evidence suggests that the theta cycle allows to separate these internally triggered “retrieval” processes related to reactivating memory templates from “encoding” processes which are triggered by external input (Colgin et al., 2009). Within the hippocampus input primarily arrives from entorhinal cortex to the CA1 subfield, with the CA1 subfield forming an internal network with neurons in the CA3 subfield. Colgin et al. (2009) have shown that neurons from this internal hippocampal network (CA1–CA3) synchronize at a 40-Hz gamma rhythm at the descending phase of the theta cycle. During the brief time window within the theta cycle half of all CA3 neurons were locked to the 40-Hz gamma cycle. Importantly, this signature of an internal CA1–CA3 theta coherent assembly of neurons could be separated from an “external” theta coherent network, which synchronized a largely independent subset of CA1 neurons to external, entorhinal input at a different phase near the trough of the theta cycle (and proceeding at a higher gamma frequency) (Colgin et al., 2009).

These findings reveal the co-existence of two separable subregional clusters of neurons locking to different phases of the theta cycle: One cluster, which is functionally related to local computations and is internally generated and sustained. Single neurons pertaining to this functional group were identified as those CA1 and CA3 cells which synchronized predominantly at a low (40 Hz) gamma band and being theta phase locked to the descending (earlier) theta phase. Intriguingly, this cluster can be conceived of as the functional correlate of internally retrieved memory templates. In contrast to this first cluster, a second cluster of neurons reflects externally mediated computations by synchronizing the CA1 subfields to theta-rhythmic projections from the entorhinal cortex (see also Mizuseki et al., 2009). This second theta coherent network most likely reflects the functional coupling of a larger network of areas: The entorhinal and perirhinal cortex serve as the primary synaptic relays between the hippocampus and neo-cortical areas and the striatum (Burwell and Witter, 2002). Consequently, previous studies have shown that it is the CA1 subfield, which conveys a theta rhythm that influences frontal and parietal cortical activity (Jones and Wilson, 2005a; Siapas et al., 2005; Sirota et al., 2008), as well as the coupling to the ventral striatum (DeCoteau et al., 2007b; Lansink et al., 2009). Thus, the previously discussed evidence for inter-areal theta synchronized information about reward, reward–location associations, and working memory content proceeds on CA1 phase locked theta oscillation. Importantly, the findings by Colgin et al. (2009) show that relevant memories relying on internal hippocampal interactions are likely to influence the “external” theta network by synchronizing CA3 output to the theta cycle in CA1 at a slightly earlier phase. Such a phase dependent interaction of separable theta phases could serve as a fundamental mechanism to integrate information across multiple, seemingly independent theta coherent networks (see Mizuhara and Yamaguchi, 2007; Mizuseki et al., 2009; Adhikari et al., 2010).

The inferred internal retrieval process from within the hippocampus provides a powerful starting point for future studies to isolate the type of memories supported exclusively by an internal theta-synchronized hippocampal CA1–CA3 pathway. However, the functional evidence so far has more generally implicated that neuronal groups within CA1 are recruited for retrieving stimulus–stimulus and stimulus–reward associations as discussed in the preceding sections. In addition, functional EEG and MEG studies in humans have reported enhanced theta activity when subjects successfully recollect episodic and autobiographical information and information defined by contextual and relational information between items and events (Guderian and Duzel, 2005; Summerfield and Mangels, 2005; Steinvorth et al., 2010). Similarly, recollecting detailed information in contrast to a mere sense of familiarity (or low confidence) has been shown to correlate with enhanced theta activity in hippocampus as well as in prefrontal and temporal cortex (Jacobs et al., 2006). All this evidence is generally consistent with the assertion that hippocampal circuitry represents a convergence zone recruited not only for encoding and consolidating memories, but likewise for the retrieval and integration of diverse contextual information from long-term memories (Duzel et al., 2010). Reactivating these memories are essential when a decision point is approached and requires the evaluation of possible choices in a given context.

However, it should be noted that the functional impact of hippocampal activation during retrieval process is critically changing over time, becoming marginal after the first 5–25 days after encoding information (Bontempi et al., 1999; Frankland et al., 2004). During this time memories are thought to be consolidated by being transferred to target areas. Among these areas are the prefrontal cortex, the anterior cingulate, the ventral striatum, and the amygdala (Bontempi et al., 1999; Seidenbecher et al., 2003; Frankland et al., 2004; Paz et al., 2008; Lansink et al., 2009; Adhikari et al., 2010). Therefore, all these areas are implicated to host learned memory associations for particular information content. As shown in the previous sections many of these associations are dynamically reactivated in the form of selective synchronization to an underlying theta rhythm.

Conclusion

The previous sections surveyed recently accumulated evidence for selectively synchronized information to an underlying theta rhythm. Of particular importance is the abundant documentation of functional correlates of theta synchronized spiking activity within and between neuronal groups in brain regions conveying choice-relevant information. The capability to predict behavioral choices and learning success by the degree to which neuronal spike output is structured to an underlying rhythm strongly suggests that theta range activity indexes a fundamental mechanism to coordinate the integration of multiple types of information and subserves a critical cognitive function.

We speculate that the particular cognitive function supported by selective theta synchronization is the structured retrieval of choice-relevant information around decision points. The requirement to coordinate multiple sources of information to inform a choice is particularly virulent when automatic behavior breaks down and the organism is forced to simulate and evaluate possible trajectories of actions before continuing in an automatic fashion. These evaluation processes not only recruit diverse, spatially separated brain areas, but they also take time, suggesting that the duration of the theta cycle, in contrast to faster rhythms such as gamma-band synchronization, is optimal to allow many areas to be activated in a concerted, recurrent fashion to accumulate evidence across few successive theta cycles.

One intriguing implication of this perspective is the prediction that the ultimate selection of a decision relies on a sufficiently coherent network of selectively synchronized neurons conveying choice-relevant information. This perspective allows to formulate specific hypotheses about many unanswered empirical questions regarding the temporal coordination during the formation of a decision (see also Gold and Shadlen, 2007; Wang, 2008): First, does choice-relevant information arise in more than two areas “combined” by theta rhythmic spiking activity? It may well be that inter-areal integration is subserved by critical “hub” like nodes, which combine otherwise unconnected networks, which themselves may proceed on theta mediated, or theta independent mechanisms. Second, is the evidence (and confidence) in favor of one choice versus another conveyed by progressively “more” phase locked spikes across cycles of theta activity? Or is it the spiking phase in the theta cycle, which conveys critical information beyond the accumulation of the number of spikes? Answering these questions will certainly progress our understanding of the neuronal circuit mechanisms underlying flexible, goal-directed behavior.

Conflict of Interest Statement:

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.