Between the primate and ‘reptilian’ brain: rodent models demonstrate the role of corticostriatal circuits in decision making

Abstract

Decision making can be defined as the flexible integration and transformation of information from the external world into action. Recently, the development of novel genetic tools and new behavioral paradigms has made it attractive to study behavior of all kinds in rodents. By some perspectives, rodents are not an acceptable model for the study of decision making due to their simpler behavior often attributed to their less extensive cortical development when compared to non-human primates. We argue that decision making can be approached with a common framework across species. We review insights from comparative anatomy that suggest the expansion of cortical-striatal connectivity is a key development in evolutionary increases in behavioral flexibility. We briefly review studies that establish a role for corticostriatal circuits in integrative decision making. Finally, we provide an overview of a few recent, highly complementary rodent decision making studies using genetic tools, revealing with new cellular and temporal resolution how, when and where information can be integrated and compared in striatal circuits to influence choice.

Decision making is an information integration and comparison problem in which diverse sources of information from sensory, reward and memory systems must be brought together in order to evaluate choices. Formal accounts of decision making in diverse fields such as economics, psychology, and computer science model it as a two-step process (Rangel et al., 2008). In the first step, values are assigned to particular actions through a process of learning. In the second step, the relative values of available actions are compared to determine the probability of executing a particular motor response (Sugrue et al., 2005, Kable and Glimcher, 2009, Lee et al., 2012). Decision making in this way can be thought of as a process of dimensionality reduction, wherein multiple streams of information are mapped onto a single axis of value (Sugrue et al., 2005, Kable and Glimcher, 2009, Lee et al., 2012).

Much of what we know about the neural substrates of decision making derives from studies using non-human primates as a model system. These studies build on our extensive knowledge of sensory and motor systems of primates (Wurtz, 1968, Wurtz and Goldberg, 1972, Newsome et al., 1989, Salzman et al., 1990, Shadlen and Newsome, 1996), and make use of sophisticated quantitative methods for relating neural activity to behavior.

Recently, the rodent has emerged as a useful experimental model system for understanding the neural basis of decision making. Part of the appeal of using rodents is the growing availability of sophisticated molecular and genetic tools for monitoring and manipulating neural activity in identified cell types and subcircuits (Luo et al., 2008, Scanziani and Hausser, 2009, Kramer et al., 2013, Deisseroth, 2014). Other advantages include higher throughput, lower cost, and ethical arguments. By applying the quantitative methods and conceptual tools historically associated with primate studies of decision making, the rodent preparation has the potential to offer the best of both worlds.

Although much of the work on decision making in primates has focused on the role of the neocortex, there is growing evidence for the importance of the striatum. The striatum, sometimes inappropriately referred to as the “reptilian brain” is a more ancient structure in the timeline of evolution (MacLean, 1990), making it seem an unlikely candidate for understanding higher forms of cognition. Here we review arguments that suggest this view is misinformed. The striatum receives convergent input from the neocortex and other structures, positioning it ideally to act as a central arbiter for comparing the value of different choices. The role of the striatum in decision making appears to predate the evolution of the neocortex. Below we discuss how, in the evolution from amphibians to reptiles the elaboration of pallial-striatal connectivity may have enhanced behavioral flexibility. This elaboration of cortical-striatal connectivity continued in mammals, along with increased routing of sensory information through the cortex to the striatum.

Our review is organized in six subsections. The first describes how studies of orienting behavior provide a common experimental framework for study of decision making across primate and rodent. Second, we review literature suggesting the anatomical convergence of inputs into the striatum may enable evaluation of choices, and highlight how these cortical-striatal afferents have become elaborated during evolution. Third, we briefly review studies that establish that value and choice signals can be observed in striatal activity in both primates and rodents. Fourth, we highlight how the ability to independently study and manipulate the direct and indirect pathway in the rodent using genetic tools has permitted advances in understanding how these pathways regulate goal directed orienting. Fifth, we show how changing activity in cortical-striatal synapses from a primary sensory region is sufficient to alter action selection. And sixth, we lay out future directions for research.

Comparable circuits for orienting in primates and rodents

In animal studies of decision making, subjects must be trained to report their choices non-verbally. In primate studies, subjects can be trained to report their choices using a saccadic eye movement (Wurtz and Mohler, 1974, Sugrue et al., 2005, Gold and Shadlen, 2007, Kable and Glimcher, 2009, Lee et al., 2012). In rodent studies, subjects can report their choices by selecting the left or right port of a 3-port behavior box (Uchida and Mainen, 2003, Kepecs et al., 2008, Otazu et al., 2009, Erlich et al., 2011, Huberman and Niell, 2011, Meier et al., 2011, Carandini and Churchland, 2013); (see Fig. 1).

Figure 1. Tasks testing the roles of distinct striatal cell types and corticostriatal neurons in decision making.

(A) Sequence of events in the value-based probablistic switching task. Only correct responses are shown. The animal initiates a trial by entering its snout into the center port and then indicates choice by orienting its snout into a left or right peripheral port to earn a water reward. Animals can be trained to perform hundreds or even thousands of trials per session, allowing patterns to emerge even in the face of neuronal and behavioral variability. (B) Schematic of the effect of optogenetic stimulation in the dorsomedial striatum (DMS) of D1-Cre or D2-Cre mice. Stimulation of the DMS in D1-Cre mice induced a fixed shift in contralateral bias that scaled monotonically with stimulation strength. Stimulation of the DMS in D2-Cre mice induced an ipsilateral bias that also scaled with stimulation strength. (C) Sequence of events in the auditory “cloud-of-tones” discrimination task. (D) Schematic of the effect of optogenetic stimulation or inactivation of primary auditory cortex (A1) neurons that send projections to the striatum. Stimulation of A1 corticostriatal neurons induced a behavioral bias that was predicted by the preferred frequency of the stimulated neurons while inactivation caused an “anti-bias” as predicted by the preferred frequency of the inhibited region.

Interestingly, primate saccadic eye movements and rodent choice port selection seem to recruit readily comparable neural circuits. These circuits, which originally evolved to control whole head and body orienting movements, have been conserved throughout vertebrate evolution, and provide a unified framework for understanding how decisions are mapped onto motor responses across species (Grillner et al., 2008). Rodent head and body orienting behaviors, and primate saccadic eye movements, can both be induced by stimulating the superior colliculus (Wurtz and Goldberg, 1972, Stryker and Schiller, 1975, Dean et al., 1989). Upstream structures, including the cortical frontal eye fields (FEF) and lateral intraparietal area (LIP), which control eye movements in non-human primates, have rodent homologues with apparently similar function (Erlich et al., 2011). Similar parallels have been identified in areas such as the orbitofrontal (Feierstein et al., 2006), anterior cingulate (Kvitsiani et al., 2013), and medial frontal cortex (Sul et al., 2010, Rodgers and DeWeese, 2014).

The evolving role of the striatum in decision making

A complex network of brain areas are involved in decision making (Sugrue et al., 2005, Gold and Shadlen, 2007, Kable and Glimcher, 2009, Lee et al., 2012). In mammals, key areas include the prefrontal and motor cortex. However, there is growing evidence that the purely corticocentric approach is incomplete, and that other structures, including the basal ganglia, also play a central role.

The basal ganglia are set of subcortical nuclei present throughout the vertebrate phylogeny. Recent comparative anatomical studies have demonstrated that the organization of the basal ganglia has remained largely unchanged from the lamprey to reptiles and primates, a degree of conservation in the vertebrate line that spans 560 million years of evolution (Grillner et al., 2013, Robertson et al., 2014). This collection of subcortical nuclei control basic motor programs for fundamental behaviors such as orienting within the superior colliculus/tectum that are present in all vertebrates. By contrast, many vertebrate species lack a well-developed, six-layered neocortex.

The basal ganglia are made up of a collection of connected brain regions including the striatum, pallidum, subthalamic nucleus and substantia nigra as well as dopaminergic modulation from the midbrain (Albin et al., 1989a, Kreitzer and Malenka, 2008, Gerfen and Surmeier, 2011). The basic components of the basal ganglia are found across vertebrate species, from lamprey to primates (Grillner et al., 2008, Stephenson-Jones et al., 2012). The primary input structure of the basal ganglia is the striatum. The striatum can be sub-divided into the dorsal and ventral portions, which project to dorsal and ventral pallidal structures, respectively, as well as the substantia nigra. The main cells of the striatum are the medium spiny neurons (MSNs), which fall into two classes that differ in their pattern of anatomical projections and gene expression. “Direct pathway” MSNs project to substantia nigra and globus pallidus interna, and express the D1 subtype of dopamine receptor. “Indirect pathway” MSNs project to the globus pallidus externa and express the D2 subtype of dopamine receptor. Indirect pathway MSNs derive their name from the fact that they can influence activity in the substantia nigra only indirectly, via projections to the subthalamic nucleus. The differential expression of D1 and D2 dopamine receptors has been exploited using transgenic mice expressing cre recombinase under control of dopamine receptor promoters to target direct and indirect pathway MSNs (Gerfen and Surmeier, Gong et al., 2007).

In the now classic model of basal ganglia function (Albin et al., 1989b, DeLong, 1990, Gerfen, 1992), direct pathway MSNs disinhibit motor programs through basal ganglia outputs and thereby promote motor responses. The indirect pathway serves as a brake on movement by disinhibiting basal ganglia outputs and suppressing downstream brainstem motor programs for locomotor and orienting movements as well as on the motor thalamus (Kravitz et al., 2010). Although this scheme is incomplete (Calabresi et al., 2014), it has proven to be a valuable framework for understanding of number of clinically important disorders and for guiding experiments.

In the mammalian brain the dorsal striatum receives dense input from the entire cortical mantle, limbic system, and thalamus. Comparative anatomical studies show that pallial-striatal connections, homologous to mammalian corticostriatal connections, were radically elaborated in the anamniote-amniote transition (from amphibians to reptiles) (Reiner et al., 1998, Smeets et al., 2000). Reiner and colleagues (1998) propose that “this elaboration may have enabled amniotes to learn and/or execute a more sophisticated repertoire of behaviors and movements, and this ability may have been an important element of the successful adaptation of amniotes to a fully terrestrial habitat.” In the transition from the reptilian to mammalian lineage, elaboration of corticostriatal connectivity continued. This elaboration may have been driven by evolutionary rerouting of sensory information for additional processing in the telencephalic pallium and mammalian cortex. In amphibians, information from the sensory regions of the thalamus reaches the striatum by direct projection without involvement of the pallium (cortical homologue). In mammals, by contrast, sensory regions of thalamus project heavily to cortical regions which then heavily innervate the striatum (Smeets et al., 2000). Smeets et al. (2000) note “a major evolutionary trend is the progressive involvement of the cortex in processing of the thalamic sensory information relayed to the basal ganglia of tetrapods.” The sheer density of corticostriatal connectivity in the mammalian brain and its elaboration with evolutionary history thus underscores the likely importance of its function and its potential importance in species gains in behavioral and cognitive flexibility (Reiner et al., 1998, Smeets et al., 2000, Krauzlis et al., 2014).

The massive convergence of inputs onto the striatum may allow it to serve as a ‘ballot box’ in which various sensory modalities, motivation networks, and cognitive systems are able to ‘vote’ for a limited set of behavioral responses (Redgrave et al., 1999, McHaffie et al., 2005). Activity generated by striatal inputs from cortex, thalamus, hippocampus, and amygdala may serve as predictive representations of ‘states of the world,’ present and past, that can be integrated and compared in the striatum to guide behavior (Redish, 2004, Wall NR, 2013). These ‘states of the world’ can represent information of the immediate external world routed from sensory systems as well as memories, abstract rules and stimulus sets represented perhaps by association cortex and associative regions such as the hippocampus and amygdala. Integration of activity (or ‘votes’) in striatal neurons is a mechanism by which a common value scale may emerge and be used to guide action selection (Redish, 2004). Potentially, it could also support learning through reinforcement to guide future behavior where striatal afferents are specifically strengthened or weakened to bolster the power of specific associations to later drive behavioral responses with more powerful ‘votes.’

The striatum encodes value and choice

There is growing consensus that the striatum participates in reward related decision making and action selection across species (Balleine et al., 2007). In both awake primates (Lauwereyns et al., 2002, Samejima et al., 2005, Lau and Glimcher, 2008) and rodents (Kim et al., 2009, Sul et al., 2010, Sul et al., 2011), some striatal neurons encode the values associated with actions/choices, as well as other task-relevant decision variables. These value signals change flexibly as a function of the rewards associated with available choices. The stability of these value signals may differ in different subregions of the dorsal striatum (Kim and Hikosaka, 2013).

Lesion and inactivation studies also suggest that the basal ganglia are involved in the selection of responses in the context of task (Castane et al., 2010, Kim and Hikosaka, 2013). Lesions in the dorsal anterior-medial striatum of rodents disrupt the flexible, goal-directed selection of responses following changes in contingency, whereas lesions of the lateral striatum can disrupt habitual responses that do not change with devaluation of the outcome (Yin et al., 2004, 2006). The basal ganglia may also provide an evaluation of ongoing performance, which can facilitate learning (Brainard and Doupe, 2000, Pasupathy and Miller, 2005) or modulate the vigor or speed with which an action is performed (Desmurget and Turner, 2010, Turner and Desmurget, 2010, Wang et al., 2013).

The role of the direct and indirect pathway in action selection

The greater availability of genetic and optical tools in rodents makes it possible to assess the role of specific cell-types and synapses in decision making (Gerfen and Surmeier, 2011). This enables new experiments which can differentiate when the striatal direct and indirect pathways activated during decision making, and how their role in decision making differs.

According to the classical model, the direct pathway acts as a “go” signal to initiate movements, whereas the indirect pathway acts as a “stop” signal to terminate movements. This predicts that the neurons in the two pathways would be active at different times. Alternatively, in a left/right orienting paradigm, the direct and indirect pathways could be co-activated, and the decision to go left or right would emerge following competition between the pathways and the two hemispheres. Returning to the ballot box metaphor, activity in the direct pathway would be ‘votes for’ contralateral orientation, while activity in the indirect pathway would be ‘votes against’ contralateral orientation (or ‘votes for’ ipsilateral orientation). The votes ‘for’ and ‘against’ orientation in each pathway and each hemisphere would be counted just before taking action, and the orientation direction with the most ‘votes’ would dominate behavioral output. Presumably, structures downstream of the direct and indirect pathway would tally the results of such a vote to select a single motor response.

In support of this second model, a recent study shows that there is indeed concurrent activation of the direct and indirect pathways at the initiation of orienting movements (Cui et al., 2013). In this study, the experimenters expressed a genetically encoded calcium indicator in the MSNs of the direct or indirect pathway of the dorsal striatum using a cre recombinase dependent strategy in transgenic mice. They then used fiber optics and time-correlated single photon counting to observe the activity in each pathway while the mice initiated a task at a central port before orienting to the left or right to press a lever. Optical signals showed that both pathways were activated in the hemisphere contralateral to the direction of the movement and activation predicted the occurrence of the movement within 500ms.

These observations are consistent with another study in which optogenetic stimulation was used to positively identify neural populations (PINP technique, Lima et al., 2009) as members of either the direct and indirect pathway in mice learning a rapid motor sequence (Jin et al., 2014). Using PINP, Jin and colleagues found that a similar proportion of direct and indirect pathway neurons were active during the initiation and the termination of a movement. Differences in the duration of activation between the two pathways are also worth noting. While direct pathway neurons maintained sustained activity during the motor sequence, the activity of indirect pathway neurons reduced during a movement (Jin et al., 2014). As predicted by the anatomy, activity in the SNr reflected the activity of the direct pathway MSNs driving them, whereas activity in the GPe reflected activity of the indirect pathway neurons (Jin et al., 2014), consistent with other studies involving optogenetic stimulation of each pathway (Freeze et al., 2013). A recent follow up study by Tecuapetla and colleagues (2014) suggests that even in absence of any defined task, both striatal pathways in the dorsolateral striatum are activated during contralateral movements in an open-field. Optogenetic inhibition of either or both pathways in one hemisphere can also enhance ipsiversive movement (Tecuapetla et al., 2014), suggesting balanced activity of the two hemispheres is important. Together, these data gathered from the direct and indirect pathway of rodents using the access enabled by genetic tools, argue against a simple model in which all direct pathway neurons serve as a general ‘Go’ signal to initiate movements and indirect pathway neurons serving as a global ‘Stop’ signal. Instead, these data suggest the balance of coactivation of the two pathways in both hemispheres are likely important in action selection, particularly at the initiation of orienting movements.

Optogenetics have also been used to alter activity in the context of a decision making task in which animals must integrate specific choice and reward history in time. Here optogenetics have revealed a causal role for activity in the direct and indirect pathway activity in the probability of orienting based choices. In a recent study, Tai and colleagues (2012) (including authors L.-H.T. A.M.L. and L.W.) used Channelrhodopsin-2 (ChR2) to activate either direct (D1R-expressing) or indirect (D2-R expressing) pathway MSNs in mice performing a two-alternative choice decision task (Tai et al., 2012). Animals initiated each trial with a nose poke into a central port, and then poked the left or right port to obtain reward (Fig. 1A). In this task, the behavior was guided not by sensory cues, but rather by the recent probabilistic history of rewards. The rewarded port delivered a water reward for 75% of correct responses, and the side that was rewarded was periodically switched to ensure that animals continuously updated the relative value of the two alternate choices.

Consistent with a role for the dorsal striatum in orienting decisions, stimulation of direct pathway MSNs biased orientation toward the contralateral port, whereas stimulation of indirect pathway MSNs introduced an ipsilateral bias (Fig. 1B). Importantly, striatal stimulation did not induce a left or right choice directly, but instead introduced a bias that depended on recent reward history. This suggests that the activity elicited by stimulation must have been integrated into existing activity (in terms of the ballot box metaphor these would both count in a vote tally), and acted on downstream targets to bias behavior. Since projections from the striatum are largely ipsilateral, the contralateral bias probably arises from downstream efferent structures, such as the superior colliculus, which control contralateral movement.

Tai and colleagues next developed a computational model to estimate the value of each action on a trial-by-trial basis using the individual running history of recent choices and rewards. This estimate could then be used to predict the distribution of left versus rightward choices using the data set collected during the course of the task. As expected from reinforcement theory, stimulation mimicked an additive or subtractive change in the action value, suggesting this is computed by comparing the relative activity of direct and indirect pathway MSNs. The ability of indirect pathway MSNs to promote ipsilateral choices is consistent with a vote tally in a “winner-take-all” competitive framework (Kable and Glimcher, 2009, Lee et al., 2012).

In addition to biasing the rodent's choice, optogenetic striatal stimulation also altered the vigor of responses, speeding up or slowing down the initiation of movements in a manner that depended upon the relative value of the choices (Tai et al., 2012).The effect of stimulation on choice also decayed after a150 msec delay, indicating a brief decision window within the task. These data are interesting in that they imply that the striatal activity may have dramatically distinct and evolving effects on behavior based upon the timing within task and the need to coordinate activity with other brain structures. Both the time window and lateralized nature of the motor response is consistent with the results of Cui, et al. (2013) and Jin et al. (2014).

Together, these data support a model in which the balance of activity between the direct and indirect pathway dictates the direction of an orienting motor response, suggesting that corticostriatal, thalamo-striatal or other excitatory inputs to the dorsal striatum could similarly modulate behavior. Plasticity of any inputs converging on the striatum that differentiates between the direct and indirect pathway could presumably mediate reinforcement learning to bias future action selection. Recent evidence shows differential plasticity of inputs onto these two pathways: Shan and colleagues (2014) showed that training in a goal directed task alters the AMPA to NMDA ratio of glutamatergic synapses onto the direct and indirect pathway MSNs in opposite directions in mice (Shan et al., 2014). Plastic changes onto striatal direct and indirect pathway neurons may thus serve a vital function dynamically linking ‘states of the world’ to motor responses that bring the highest value rewards.

Corticostriatal projections from sensory cortex drive decisions

Information about the world is transduced through specialized sense organs—retina, cochlea, etc—after which is it subjected to further processing before passing through the thalamus to reach the sensory cortex. During a perceptual decision making task, this information must eventually be converted into a series of motor neuron impulses that drive an appropriate action. Tools now readily available for use in rodent models can also help answer, How does information propagate beyond the primary sensory cortex to guide decisions?

Sensory cortex makes many long-range connections to other cortical and subcortical areas. For example, neurons in the primary auditory cortex project to targets including secondary auditory areas, contralateral auditory cortex, posterior parietal cortex, motor and association cortex, amygdala, auditory thalamus, inferior colliculus, and auditory striatum. In principle, any of these could carry key information needed to make the decision.

Znamenskiy and Zador (2013) used an optogenetic approach to test the role of one of these projections, from the auditory cortex to auditory striatum, in auditory perceptual decisions (Znamenskiy and Zador, 2013). They first trained rats on a novel “cloud-of-tones” auditory discrimination task, inspired by “random dot motion” task used to study the perception of motion in area MT of primates (Salzman et al., 1990). The cloud-of-tones stimulus consisted of a series of short overlapping tone pips arrayed over three octaves. Subjects were required to judge whether there were more high- or low- frequency pips, and reported their decision by approaching the appropriate reward port (Fig. 1C). Stimulus difficulty was manipulated by varying the ratio of low to high tones. Subjects performed with 100% accuracy for easy stimuli and at chance for ambiguous stimuli, and varied smoothly between these extremes (Fig. 1D).

The projection from cortex to striatum originates in cortical layer 5 (Shepherd, 2014). To activate these neurons selectively, Znamenskiy and Zador used two approaches. In the first approach, they used an intersectional strategy to limit the expression of ChR2 to striatal-projecting neurons in the auditory cortex. They engineered a replication-incompetent strain of herpes simplex 1 (HSV-1) to express Cre recombinase. This strain undergoes efficient retrograde axonal transport, so that injection into the striatum induced expression of Cre only in striatal-projecting cortical neurons. They then injected a recombinant virus, adeno-associated virus (AAV-FLEX-ChR2), which expressed ChR2 in a cre-dependent fashion. Thus, only neurons in the auditory cortex that project to the striatum expressed ChR2. Optical stimulation of these specifically labeled corticostriatal neurons, mostly in layer 5, biased decisions in the cloud-of-tones task in a manner that was predicted by the frequency tuning of the neurons near the site of stimulation. The strength of the bias was quantified as the shift in the psychometric curve (Fig. 1D).

Znamenskiy and Zador also used a second approach to activate cortical inputs to striatum. They injected an AAV into auditory cortex that expressed ChR2 ubiquitously, but then delivered light to the striatum, thereby activating only those axons that project to the striatum. This approach also yielded a behavioral bias predicted by the frequency tuning of the neurons near the site of stimulation.

These results demonstrated that activation of the corticostriatal pathway is sufficient to bias choices. However, because of the artificial nature of ChR2 stimulation, they do not establish whether corticostriatal activity contributes to the formation of decisions under normal conditions. If corticostriatal activity normally contributes to decisions, then suppressing this activity during the task should lead to an ‘anti-bias’—that is a bias in the direction opposite to that induced by ChR2 activation. To test this, Znamenskiy and Zador targeted the expression of the light-activated proton pump Arch3 to corticostriatal neurons using the HSV-1-based approach described above. As predicted, inactivation of corticostriatal neurons biased subjects’ choices away from the reward port associated with the frequency band of the inactivation site. Taken together, the activation and inactivation data imply that corticostriatal projections play a vital role in generating motor responses in responses to discriminative sensory stimuli.

Figure 1 highlights the close parallel between these results, obtained by stimulation posteriolateral dorsal striatum, and the results of Tai and colleagues obtained by stimulating the dorsomedial striatum. This similarity suggests that the striatum can integrate relevant information from a wide range of sources, including primary sensory areas along with associative areas supporting value and memory systems. These results support the view that at the cortical-striatal transition, diverse forms of sensory evidence or choice and reward-related information may be converted into a common value axis (tallied votes in terms of the ballot box metaphor) for comparison and selection of the best action.

Future Directions: Following iterative processes of integration and evaluation through cortico-basal ganglia systems

Based upon these results, it is interesting to speculate that sensory discrimination tasks in other modalities could also be solved by the comparative tally of inputs from specific sensory modules into a common striatal “ballot box”. We anticipate that future studies are likely to elucidate whether the type of corticostriatal plasticity observed by Znamenskiy and Zador (2013) in their auditory “cloud-of-tones” tasks used to couple specific representations to behavioral outputs generalizes to other sensory modalaties, such as olfaction and vision. The olfactory tuberacle of the ventral striatum recieves direct input from the olfactory bulb. Recent anatomical studies suggest direct projections from V1 in mice to the anterior dorsomedial striatum with several previous studies (Faull et al., 1986, Khibnik et al., 2014), demonstrating the existence of visually driven units in rodents, cats, and monkeys (Hikosaka et al., 1989). Other studies have suggested that many of these neurons may also encode integrative multi-sensory responses (Reig and Silberberg, 2014). Given that the dorsolateral striatum's role in habit formation (Graybiel, 2008), it is arguable whether or not the behavioral biases identified in Znamenskiy and Zador (2013) are related to stimulus-response mappings or more flexible behavioral circuits that support choices that are goal-directed and sensitive to changing conditions. Questions like these may be addressable using devaluation experiments (Balleine and O'Doherty, 2010).

It is interesting to speculate that a “vote tallying” system also extends to complex sensory features, abstract rules, and representations of goals encoded by higher order associative areas such as the prefrontal cortex (Miller and Cohen, 2001). Recent evidence suggest that chemical genetic inhibition of orbitofrontal inputs to striatum can disrupt while optogenetic stimulation can enhance goal-directed behavior (Gremel and Costa, 2013). As mentioned above, the type of behavioral biases tied to these representations may similarly be tied to their relative balance of inputs onto direct and indirect pathway MSNs also acquired through a process of learning (Shan et al., 2014). The predicted enhancement of synaptic strength could potentially be studied in vivo using novel methods to couple electrophysiological (Znamenskiy and Zador, 2013) or optical measurements of specific striatal inputs (Gunaydin et al., 2014) while simultaneously utilizing methods similar to Cui, et al 2014 and Jin, et al 2014 to record the activity in downstream basal ganglia subcircuits and motor outputs. Ex vivo channelrhodopsin-assisted circuit mapping (MacAskill et al., 2012, Kress et al., 2013) and rabies tracing methods (Wall NR, 2013) could also verify the specificity of such altered synaptic input onto each pathway.

Together, the data covered in depth here strongly support a role for the dorsal striatum in action selection, in which a response with the highest value is selected based upon differential activity in the direct and indirect pathway. Formulations of reinforcement learning label this process the ‘actor’ and propose the existence of another separate but complementary ‘critic,’ which evaluates outcomes to support learning that guides future actions (Sutton RS, 1998, Joel et al., 2002, Niv and Schoenbaum, 2008). The data reviewed here are consistent with the hypothesis that the dorsal striatum plays the role of ‘actor.’ One possible identity for the ‘critic’ is the ventral striatum and its association with limbic, midbrain dopamine neurons, and prefrontal cortex (Britt et al., Stuber et al., Ambroggi et al., 2008, Stuber et al., 2008) This proposition is consistent with traditional views that the dorsal striatum is believed to support a role in motor control while the ventral striatum supports motivation and reinforcement (Montague et al., 2004, O'Doherty et al., 2004). During iterative trial and error learning, these dorsal and ventral striatal value systems may interact, with the ‘critic’ providing instruction to the ‘actor’ via spiraling connections between the striatum and dopaminergic regions of the midbrain (Alexander and Crutcher, 1990, Haber et al., 2000, Everitt and Robbins, 2005, Niv and Schoenbaum, 2008, Balleine and O'Doherty, 2010, Luscher and Malenka, 2011). This spiral organization spanning affective to motor components of the basal ganglia circuit, may enable a process by which motivation and incentive related computation influence goal-directed, and eventually, habitual action through reinforcement learning (Everitt and Robbins, 2005). Future efforts to investigate ventral striatal function using optogenetics will help address these issues (Lee et al., 2013).

In conclusion, the use of new optical and genetic tools in rodent models has enlightened our understanding of the role of the striatum and corticostriatal synapses in action selection in complex tasks that involve the integration of information. These data are consistent with hypotheses derived from comparative anatomy that suggest the elaboration of corticostriatal connectivity in the evolution of the brain may underlie increases in behavioral flexibility. Rodent models, thus, provide a link between the study of the integration of information and higher cognition in primate neocortex and mechanisms enabling flexibility since the evolution of ‘lower’ vertebrates. We predict the tractability of rodent models will greatly enable further study that will benefit our understanding of human decision making, cognition, and disease.

Highlights.

Rodent models of decision-making

Evolutionary elaboration of corticostriatal connectivity

Striatum “ballot box” as a solution to the problem of action selection

The role of the striatum and corticostriatal synapses in action selection