The Striatum’s Role in Executing Rational and Irrational Economic Behaviors

Abstract

The striatum is a critical component of the brain that controls motor, reward, and executive function. This ancient and phylogenetically-conserved structure forms a central hub where rapid instinctive, reflexive movements and behaviors in response to sensory stimulation or the retrieval of emotional memory intersect with slower planned motor movements and rational behaviors. This review emphasizes two distinct pathways that begin in the thalamus and converge in the striatum to differentially affect movements, behaviors and decision making. The convergence of excitatory glutamatergic activity from the thalamus and cortex, along with dopamine release in response to novel stimulation, provide the basis for motor learning, reward seeking, and habit formation. We outline how the rules derived through research on neural pathways may enhance the predictability of reflexive actions and rational responses studied in behavioral economics.

Introduction

The striatum is a cluster of interconnected nuclei that form a part of the basal ganglia. It is involved in decision making functions, such as motor control, emotion, habit formation, and reward. The term striatum describes the striped appearance of grey and white matter-containing nuclei that form distinct subcortical structures. The striatal nuclei were introduced in Neurographia universalis by the seventeenth century French anatomist Raymond Vieussens (1635–1715). Later work showed that the striatum is conserved across species from lamprey to humans (Grillner, Robertson and Stephenson-Jones 2013).

Early works in birds, including that by Otto Kalischer (1869–1942) suggested that connections between the cortex and striatum had localized effects on volitional movement, whereas the more primitive connections between the thalamus and striatum encoded their instinctive and automatic actions, including courting, feeding and fighting (Pepperberg 2002). The dichotomy of these two distinct but overlapping pathways was noted in patients with striatal degeneration or Parkinson’s disease, where movements become spastic such that the automatic activities of daily living are disrupted, while the higher-order but much slower volitional movements are relatively spared. This suggests that the primitive sub-cortical connections linking the thalamus, amygdala, and striatum tonically inhibit movements, which could be relieved in response to sensory input or the retrieval of emotional memory to promote swift instinctive and reflexive movements. In comparison, the cortex through its numerous corticostriatal projections promotes slower planned motor movements and thoughtful, rational behaviors. These two time-dependent and distinct pathways intersect at the striatum where they may compete for dominance or work synergistically to emphasize a response.

Here, we review the major neural pathways involved in reflexive and rational behaviors. We highlight how new experiences can modify this circuitry and show how reduced or recurrent increases in the release of the neurotransmitter dopamine can alter corticostriatal function. Much evidence suggests that these two pathways differentially affect reward seeking and decision making in humans. Reward seeking, and action-outcome learning require the corticostriatal pathway but over time appear to be encoded as habits within the more primitive thalamostriatal circuitry. Rational decision making appears to require the corticostriatal pathway, where action-outcome learning is motivated by the perceived value of a reward or a consumed good. Decision making is crucial for rational behaviors, where the rationality axiom in economics stipulates that a rational man maximizes his reward. Both reward seeking and the perceived value of the reward are likely determined by the release of dopamine. The neuroscientific and neuroeconomic models of decision making broadly follow similar steps (Table 1). Therefore, the knowledge derived from both physiological observations in the laboratory and economic observations in the developing neuroeconomics discipline are expected to provide overlapping models that will provide a better understanding of the rules derived through neural pathways and will enhance the predictability of reflexive actions and rational behaviors for individuals and populations.

Table 1:

The Economic and Physiological Steps of Decision Making

#	Economic Steps	Physiological Steps
1	Based on what you know about the product, predict the expected utility derived from consuming it	Gauge the expected reward from dopamine release based on the relative strength of corticostriatal inputs
2	Consider the opportunity cost of the product—Is there a better use for money or time?	Gauge whether other actions with a similar cost would yield a higher expected dopamine reward based on the relative strength of corticostriatal inputs
3	Purchase the object	Use the output mechanism of the striatum to execute the actions that will lead to the purchase of the object
4	Consume the object and get the utility associated with it	Consume the object and experience dopamine release
5	Reevaluate the expected reward associated with that good depending on the actual utility derived	Change the relative strength of the synapses of corticostriatal inputs in response to the actual dopamine release

Organization of the Striatum

The spiny projection neurons

Selection, execution, and suppression of appropriate motor movements and decisions require the striatum to collect, modulate and integrate cortical information. Electron microscopy studies indicate that the typical arrangement of the striatum is a three-element circuit, where axons from midbrain dopamine-producing cells converge with glutamate-releasing projections from the cortex and thalamus on the dendritic spines and shafts of spiny projection neurons (SPNs; Fig. 2) (Nirenberg, Vaughan, Uhl, Kuhar and Pickel 1996). All three elements of this synapse have at least one receptor for the other two, and in addition, may present AMPA, metabotropic glutamate, GABA, adenosine, cannabinoid and cholinergic receptors that provide signal modulation (Sulzer 2011; Wang and others 2012; Wang and others 2013b).

Figure 2.

The corticostriatal network and the effect of dopamine on corticostriatal terminals. The simplified cartoon shows the tripartite configuration of a glutamatergic corticostriatal input, a thalamostriatal input, and a dopaminergic (DA) midbrain input as they synapse on an SPN. Glutamatergic presynaptic elements generally display an asymmetric synaptic density with docked synaptic vesicles. The corticostriatal density contains VGLUT1 and docks near the spine head, while the thalamostriatal element contains VGLUT2 and docks on the spine shaft. A dopamine terminal is generally associated with the spine neck and shaft and displays a relatively small symmetric synaptic density (Totterdell and Smith 1989). Virtually all striatal synapses are within 2 μm of an apparent dopamine axonal varicosity, and thus are thought to receive dopamine input. SPNs express AMPA, NMDA and GABA-A receptors, in addition to D1 or D2 receptors (Wang and others 2012). Presynaptic afferents from the cortex express acetylcholine nicotinic and muscarinic receptors, adenosine and cannabinoid CB1 receptors and GABA-B receptors (Wang and others 2013a; Wang and others 2012; Wang and others 2013b). In the dorsal and ventral striatum, dopamine D2 receptors are found on presynaptic afferents to D2-SPNs (Bamford and others 2004b; Wang and others 2012). In the ventral striatum, D1-SPNs are expressed on presynaptic afferents to both D1- and D2-SPNs (Wang and others 2012). Presynaptic receptors on excitatory glutamatergic afferents from the thalamus have not been well characterized.

Modulatory dopamine inputs

The striatum is broadly divided into sectors whose function complements that of the overlying cortex. In mice, the dorsal motor striatum lies beneath the motor cortex and contains the caudate and putamen. SPNs in the dorsal striatum receive axons from midbrain dopamine-producing neurons in the substantia nigra pars compacta (SNpc) which acts to program motor movements and encode cognitive behaviors. The ventral striatum resides beneath the prefrontal cortex (PFC) and contains the nucleus accumbens (NAc) (Fig. 3). Another set of dopamine-producing cells in the ventral tegmental area (VTA) extends to the NAc and the infralimbic cortex of the medial PFC, and acts to program behaviors (Patton, Bizup and Grace 2013) and reward reinforcement (Han and others 2017). Dopamine neurons fire spontaneously at low rates with occasional bursts, and their proper function is essential for normal striatal activity (Sulzer 2011). In diseases like Parkinson’s with reduced dopamine availability movements and behaviors are slow and difficult to initiate, while in diseases with increased dopamine availability, movements and behaviors become erratic, suggesting that dopamine relieves the tonic inhibition that is provided by the basal ganglia.

Figure 3.

Simplified circuits that mediate habitual, rational, and emotional actions and behaviors. (A) This simplified schematic describes the connections between brain regions involved in executing movements. The red arrows are pathways proposed to mediate primitive or learned habitual movements, while blue arrows represent pathways involve in learned rational and goal-directed movements. [1] Signaling by dopamine from the SNpc that is coincident with either [2] excitatory VGLUT1 corticostriatal projections or [3] excitatory VGLUT2 thalamostriatal projections is relayed through [4] striatal SPNs. [5] Information is processed within the striatal output nuclei, which includes the globus pallidus, SNpr and STN, before being sent on to the thalamus. The thalamus then directs information back to the cortex or the striatum. (B) The drawing shows the output nuclei that receive inhibitory GABAergic signals from the SPNs. These nuclei allow signal inversion so that signals from D1R-expressing SPNs excite the thalamus, while signals from D2R-expressing SPNs are inhibitory. (C) The schematic describes the putative connections involved in irrational habits (red arrows) and reward-based, rational behaviors (blue arrows). (D) The simplified schematic shows the interconnections proposed to encode emotional behaviors. The amygdala receives inputs from the thalamus and from the sensory cortex. Emotion-based behaviors (red arrows) extend from the amygdala to the VTA, the mPFC, and the NAc. The mPFC (blue arrows) processes emotional responses using extinction, which allows goal-directed behaviors instead of emotion-directed behaviors to be executed if they conflict.

Excitatory glutamatergic inputs

Excitation by glutamate signaling from the cortex and thalamus is required to activate SPNs, which typically fire at low rates (~1 Hz) (Wilson and Groves 1981). Cortical projection neurons to the striatum primarily synapse on spines of SPNs and use the vesicular glutamate transporter 1 (VGLUT1) for packaging glutamate in synaptic vesicles (Fremeau and others 2001). Excitatory neurons in the thalamic intralaminar nucleus, including the centromedian and parafascicular (CM/Pf) nuclei, project to the striatum where they synapse on acetylcholine-releasing interneurons and on the dendritic shafts of SPNs (Lapper and Bolam 1992; Sidibe and Smith 1996; Sidibe and Smith 1999). The synaptic vesicles of thalamostriatal projections express VGLUT2 at their presynaptic sites and feature a higher probability of release compared to VGLUT1 (Fremeau and others 2001; Herzog and others 2001). The expression of distinct VGLUT isoforms by complementary populations suggests that they define distinct modes of neurotransmission, perhaps related to their phylogeny and function. Due to the low excitability of SPNs, temporally-correlated excitatory inputs from many convergent pathways (Wilson 1992) along with coincident dopamine input (Bamford, Wightman and Sulzer 2018) is required to depolarize the SPN. In other words, for SPNs to activate, confluent excitatory signals from numerous cortical regions—which may control functions such as motor control, sensation and emotion—generally need to arrive at a time when dopamine is also released.

The striatal output

Striatal SPNs are primarily divided into two main output types differentiated in both form and function. Approximately half the SPNs express D1-type dopamine receptors (D1-SPN), while the other half contain D2-type dopamine receptors (D2-SPN; Fig. 3A). D1-type receptors include D₁ and D₅ receptor subtypes, while D2-type receptors include D₂, D₃, and D₄ subtypes. A small number of SPNs in the NAc express both D1 and D2 receptors (Smith, Lobo, Spencer and Kalivas 2013). The axons of D1-SPNs form the “direct pathway” that provides positive feedback to the cortex and thalamus, while axons of D2-SPNs form the “indirect pathway,” which delivers negative feedback to these regions (Fig. 3B). Feedback from the SPNs to the thalamus and cortex rely upon the striatal output nuclei, which include the globus pallidus (GP) interna (GPi) and externa (GPe) and the subthalamic nucleus (STN).

Autonomously firing cells in the GP tonically inhibit cells in the ventral anterior, ventral lateral, and CM thalamus, which in turn excite the striatum and cortical layers III and V (Kuramoto and others 2009). Therefore, in the absence of dopamine or D1-SPN activity, the thalamostriatal pathway is tonically inhibited. D1-SPNs synapse on the GPi and impede this tonic inhibition to excite the thalamus and cortex. Signals from D2-SPNs connect with cells in the GPe and STN. These additional connections work to invert the signal which is then passed on to the GPi, thereby strengthening the tonic inhibition. Neurons in the GPi also synapse on the substantia nigra pars reticulata (SNpr) that inverts the signal before sending afferents to the thalamus, thereby promoting a balance between thalamic excitation and inhibition.

Motor and habit execution are determined when these circuit elements work together to select particular collections of striatal synapses which then excite the thalamus and cortex to produce appropriate corticofugal activation of the midbrain and spinal cord (Lei, Jia, Del Mar and Reiner 2004). The striatum’s role as a hub of signaling is evident in its composition: 95% of striatal cells are SPNs that link different parts of the brain. SPNs receive signals on their many dendrites, but project with only one axon (with collaterals), indicating a condensation of information derived from multiple inputs into one binary choice: to fire or not to fire. Therefore, the striatum enables and integrates planned motor movements and rational behaviors that enter along the corticostriatal pathway with primitive sub-cortical signals that pass along the thalamostriatal pathway. The confluence of dopamine release with glutamate signaling along these two time-dissociated pathways determines whether rational and thoughtful behaviors or swift, instinctive, and reflexive movements are processed. The behaviors may compete for dominance, but as processed signals from the cortex likely arrive after those sent directly from the thalamus, the actions may occur sequentially to emphasize or cancel a response already in progress.

The Corticostriatal Pathway

The corticostriatal pathway initiates contralateral planned movements and behaviors

The corticostriatal pathway constitutes a major excitatory output from the cortex and intersects with the excitatory thalamostriatal projections at SPNs. Corticostriatal projections originate from a diverse range of cortical regions and terminate in their corresponding striatal area. Corticostriatal projections are composed of axons from two types of cells that differ morphologically and functionally: pyramidal tract (PT)-projecting neurons and intratelencephalically projecting (IT) neurons (Fig. 4A) (Reiner, Hart, Lei and Deng 2010). Axons from PT-type neurons originate mainly in the lower cortical layer V, and send their main axon projection to the ipsilateral brainstem, where they cross at the medullary decussation and descend in the contralateral spinal cord (Cowan and Wilson 1994; Levesque, Charara, Gagnon, Parent and Deschenes 1996; Reiner, Jiao, Del Mar, Laverghetta and Lei 2003). These neurons typically carry to the striatum an efferent copy of motor commands sent to the spine (Reiner 2010) and innervate D2-SPNs that form the indirect pathway (Lei, Jia, Del Mar and Reiner 2004), likely to suppress conflicting movements or terminate a specific movement (Graybiel 2005). IT-type neurons originate in both layer III and upper layer V and preferentially carry sensory and motor planning signals to D1-SPNs of the direct pathway (Lei, Jia, Del Mar and Reiner 2004; Wilson 1987). IT-type corticostriatal axons project bilaterally to both the ipsilateral striatum and to the contralateral striatum via the corpus callosum. The IT-type neurons have a slower conduction velocity so their signals reach targets a few milliseconds after the PT signals arrive (Cowan and Wilson 1994; Wilson 1987), suggesting that IT-type projections may initiate planning for the next sequential motor movement.

Figure 4.

Proposed steps that lead to movements and decision making via the corticostriatal pathway. (A) The schematic shows the origin and destination of IT-type and PT-type projections. [1] Actions generated by the giant Betz cells of cortical layer V are [2] relayed to the midbrain and contralateral spinal cord by PT-type projections. [3] Their axon collaterals to ipsilateral D2-SPNs terminate the movement. [4] IT-type projections from layer III cortico-cortical neurons and from upper layer V neurons promote planning for the next action, [5] which after a processing delay is relayed by ipsilateral and contralateral D1-SPNs to the cortex via the thalamus (ipsilateral circuit is not shown). (B) Examples of optical FM1–43 destaining within the striatum in response to cortical stimulation. Corticostriatal terminals in the NAc were loaded with FM1–43 during a pre-measurement interval, and then the destaining rate during a subsequent stimulation period at individual nerve terminals was monitored as t_1/2 values (the time required for florescence to decay to half of it original value). The individual halftimes (t_1/2) of FM1–43 release from corticostriatal terminals in an acute slice preparation are shown in false color. Note that the preparation exposed to the dopamine releaser amphetamine (right; n = 112) has slower destaining puncta (higher t_1/2) than the control (left; n = 99). Scale bar, 10 μm. (C) The normal probability plot demonstrates presynaptic filtering of glutamatergic terminals in the NAc of mouse brain slices. Comparison of individual half-times of FM1–43 release shows that the dopamine releaser amphetamine filters cortical activity by reducing exocytosis of FM1–43 from terminals with lowest probability of release (i.e. those with the highest t_1/2). At 20 Hz, t_1/2 = 207 s; n = 330. At 20 Hz with amphetamine, t_1/2 = 264 s; n = 193; ***p < 0.001, Mann-Whitney. (D) This plot demonstrates that activation of D2 receptors inhibits glutamate release at higher frequencies. Distribution of mean t_1/2 of release for slices containing the NAc as a function of frequency. The release of FM1–43 was reduced at higher frequencies (20 Hz) when the slice was exposed the D2R agonist quinpirole (n = 80 – 96 puncta; ***p < 0.001, Mann-Whitney). (E) This graph demonstrates that activation of D1 receptors excites glutamate release at lower frequencies (1 Hz), while adenosine inhibits glutamate release at higher frequencies (20 Hz). The D1R agonist SKF38393 increased release at low frequencies (1 Hz). At higher frequencies of cortical stimulation (20 Hz), the D1R agonist reduced release by promoting presynaptic inhibition by adenosine (n = 97 – 260 puncta; ***p < 0.001 Mann-Whitney). (F) The schematic shows glutamatergic and dopaminergic inputs to D1- and D2-expressing SPNs in the NAc, including representations of presynaptic filtering. [1] Tonic dopamine selectively inhibits low-release probability synapses of the indirect pathway D2-SPNs through presynaptic D2 receptors (D2Rs). [2] Higher levels of dopamine modulate corticoaccumbal activity through D1Rs that strengthen glutamate release from presynaptic terminals innervating both D1-SPNs and D2R-SPNs. [3] Higher cortical frequencies coincident with dopamine release inhibits presynaptic activity of both direct pathway D1-SPNs and indirect pathway D2-SPNs via stimulation of A₁A receptors by adenosine, putatively produced in D1-SPNs following AMPA receptor (AMPAR) and NMDA receptor (NMDAR) activation. [4] Filtering by adenosine is followed by the selective presynaptic inhibition of D2R-expressing SPNs via stimulation of CB₁ receptors (CB₁R) via endocannabinoids putatively produced by D2-SPNs following activation of their D2 and group 1 metabotropic glutamate (mGluR₅) receptors. Data are shown as mean ± SEM. Panel B is reproduced with permission from Bamford et.al. Neuron. 42:653–663 (2004). Panels C - F are reproduced with permission from Wang et.al., J Physiol. 590:3743–69 (2012).

Striatal function depends on a balance between D1-SPNs and D2-SPNs

Much work indicates that normal function requires a balance between D1-SPNs and D2-SPNs so that transient shifts that favor either the direct or indirect pathway produce a specific movement or behavior (Bamford and Joyce 2005; Bamford, Wightman and Sulzer 2018; Beutler and others 2011; Darvas and Palmiter 2015; Wong and others 2015). Long-lasting changes in the balance between D1- and D2-SPNs can occur in diseases that affect the availability of dopamine, which can rapidly change the sensitivity of dopamine receptors that are expressed on SPNs (Bamford and others 2004a) and striatal interneurons. D1-type receptors are expressed on corticostriatal terminals in the NAc (but not in the dorsal striatum (Bamford and others 2004b)) where they enhance glutamate release (Wang and others 2012). D2-type receptors are expressed on corticostriatal and thalamostriatal glutamatergic projections and dopaminergic synapses, where they reduce the probability of neurotransmitter release (Bamford and others 2004a; Sulzer 2011).

Classically, dopamine was thought to facilitate appropriate movements by directing selected cortical signals into the direct “GO” pathway while inhibiting inappropriate movements via the indirect “NO GO” pathway (Gerfen and others 1990). In the case of Parkinson’s disease, the loss of dopamine makes the direct pathway less active by diminishing D1-SPN activity, while increasing output along the D2-SPN indirect pathway, resulting in the inhibition of movement (Mallet, Ballion, Le Moine and Gonon 2006). While this classical model may generally be accurate, its mechanisms appear oversimplified.

To gain an understanding of how dopamine controls cortical information entering the direct and indirect pathways within the NAcore, we combined presynaptic optical approaches with postsynaptic electrophysiology in CB1 receptor (CB1R)-null mice and in hemizygotic bacterial artificial chromosome transgenic mice expressing the reporter enhanced green fluorescent protein under the control of the D1 and D2 dopamine receptor promoters (Bamford and others 2004b; Wang and others 2012). These experiments demonstrated that dopamine release promotes a neurochemical cascade that channels cortical information through the direct pathway, while simultaneously filtering cortical inputs with a low probability of release (Fig. 4). We found that dopamine filters a subset of ipsilateral glutamatergic corticostriatal signals by selectively inhibiting high frequency and slow-releasing cortical inputs through endocannabinoids and D2Rs, while lower frequency signals with a high probability of release are passed. Changes in this frequency-dependent filtering of corticostriatal activity by minute-to-minute variations in dopamine cause a transient imbalance between the direct and indirect pathways that promote learning (Bamford, Wightman and Sulzer 2018) as well as behaviors associated with disease (Beutler and others 2011).

Striatal interneurons provide gating for reflexive and rational actions

Dopamine receptors are also expressed on striatal interneurons, which include tonically-active acetylcholine-releasing interneurons (Bennett and Wilson 1998) and at least 3 subtypes of gamma-amino butyric acid (GABA)-releasing interneurons (Tepper, Tecuapetla, Koos and Ibanez-Sandoval 2010). Dopamine modulates the activity of both acetylcholine and GABA interneurons and in turn, the interneurons moderate the excitability of SPNs via a host of pre-and postsynaptic nicotinic, muscarinic, and GABAergic receptors (Wang and others 2013a). Importantly, cholinergic interneurons are primarily excited by thalamic inputs from the CM/Pf (Sidibe and Smith 1999), whereas GABA interneurons are primarily excited by the cortex, with rather little thalamic innervation (Tepper, Tecuapetla, Koos and Ibanez-Sandoval 2010). Therefore, while there is crosstalk between these cells, cholinergic interneurons may be important for gating reflexive actions and learned behaviors, whereas GABA interneurons may function as spike timing for cortically-driven rational and goal-directed actions (Wang and others 2012). As such, the activity of corticostriatal and thalamostriatal pathways can be modified by minute, periodic, or long-term changes in dopamine availability.

Rational reward-based learning requires corticostriatal projections

The current model of the striatum’s reward mechanism is that dopamine reinforcement enables action-outcome learning, where rewarding actions are more likely to be selected in the future. Reward seeking is an important factor in motivation. Action-outcome learning is goal-directed when an organism discovers that a specific action such as rodent lever pressing will yield a reward (Cacciapaglia, Wightman and Carelli 2011).

Actions are chosen based on the predicted rewards that they will elicit. Learning occurs after a rewarding behavior has been completed, and the mechanism changes to accommodate the previous trial in a new reward prediction (Schultz 2006). Experience can change the predicted reward value, so that the organism learns to predict the true value of a reward more accurately. Since motivation is partly driven by predicted reward, a decrease in predicted reward leads to a lower chance of performing that activity while an increase in predicted reward leads to a higher chance of choosing that activity. If actions bear similar results over time, an organism will likely be better able to predict the true reward of an action, and the chance of selecting that action over other alternative behaviors changes to reflect the corresponding value of the reward.

Mechanistically, learning associated with reward can arise from either a change in presynaptic neurotransmitter release or a change in postsynaptic responsiveness (Bamford and others 2004a; Wong and others 2015). While dopamine release is required for establishing cue-dependent and reward-based learning, these behaviors appear to require coincident changes in corticostriatal activity that modify the balance between the activity states of D1- and D2-SPNs. The synaptic mechanisms underlying these behaviors have been examined by using mice exposed to repeated psychostimulants and by intracranial self-stimulation (ICSS), when an electrical impulse of the medial forebrain bundle stimulates dopamine release in the basal ganglia and cortex (Olds and Olds 1958).

The Wightman and Carelli labs have used ICSS with voltammetric techniques and rewards in rodents to measure dopamine transients that occur in response to a cue and a subsequent lever press that drives the delivery of cocaine or an electrical stimulation (Owesson-White and others 2009). These studies showed that dopamine release occurs immediately after the cue and then again following the lever press, with its associated stimulus delivery. When the stimulus is turned off (extinction protocol), dopamine release after the lever press does not occur and the cue responsive dopamine transient rapidly disappears. More sophisticated studies have combined in vivo recordings of single SPNs with ICSS recordings to measure the sub-second time course of dopamine transients around a behavioral event (Owesson-White and others 2016). A multimodal sensor allowed simultaneous fast scan cyclic voltammetry, single-unit recordings, and microiontophoretic ejection of different drugs that enable dopamine receptor identification. Results showed that dopamine transients were associated with three populations of SPNs: those that increased in firing at the cue, those that increased in firing around lever-press, and others that decreased in firing around lever press. SPNs that increased activity with the cue were identified as D2-SPNs, while most cells that showed excitation immediately before the lever extension were classified as D1-SPNs (Fig. 5A). When results of these microiontophoretic recordings were combined with optical and electrophysiological experiments performed in the Bamford and Sulzer labs (Bamford and others 2004b; Wang and others 2012; Wong and others 2015), the striatal microcircuit appears to react when both high cortical input and dopamine appear simultaneously. SPNs involved in mediating coordinated responses during rewarding behaviors occur only in those striatal regions with both local dopamine release and activated SPNs. The sensory cue produces an imbalance in striatal output, favoring the D2-SPN and motor arrest, while the lever extension enhances D1-SPN and motor activation producing the lever press (Fig. 5B). These shifts in the balance of SPN activity are augmented by synaptic filtering that reduces less effective excitatory inputs (Fig. 5C).

Figure 5.

Time course of D1-SPN and D2-SPN responses in the rat NAc during an ICSS trial consisting of a cue presentation followed two seconds later by lever extension and then a subsequent lever press (LP), which delivers an electrical stimulation to dopamine cell bodies in the VTA. (A) Time course of D2-SPNs that are cue excitatory and D1-SPNs that are LP-excitatory. The hatched areas indicate the time variation of either cue or LP. (B) Changes in the balance between D2- and D1-SPNs during the cue and the LP. The cue temporarily heightens D2R-SPN activity to create an imbalance in striatal output. The imbalance may alert the animal, perhaps by promoting an urge to move. The lever extension increases D1R-SPN activity, culminating in the lever press (LP). (C) Temporal sequence of chemical activity at SPN synapses. The resting activity of SPNs is low because mature SPNs have prominent potassium currents (Wilson and Kawaguchi 1996) and cortical inputs to D2-SPNs are tonically inhibited by endocannabinoids (Wang and others 2012). [1] The sensory cue triggers excitatory inputs to D2-SPNs, which [2] activate further via presynaptic D1Rs in response to [3] dopamine release. [4] Balance is restored following the release of endocannabinoids from D2-SPNs. [5] In animals that have learned to associate sensory cue to the lever press, another set of glutamatergic inputs subsequently activate with [6] convergent dopamine release to stimulate D1-SPNs. Coactivation of post-synaptic D1Rs, NMDA and AMPA receptors on D1-SPNs release [7] adenosine to inhibit presynaptic activity to both D1- and D2-SPNs and rebalances the circuits.

A neuroeconomic model of rational addiction

Economists Gary Becker and Kevin Murphy outlined a model of addiction in 1988, which indicated that becoming addicted to a drug is a rational choice (Becker and Murphy 1988) because consumers weigh the lifetime costs of addiction including health problems, decreased income, and the long-term price of drugs against the pleasure benefits that drugs provide. Our neuroscientific model of rationality supports their claim because intentional choice to take drugs can be rational: Drug addiction begins as a goal-directed behavior driven by a desire to seek the reward that they produce. Scientific addiction models demonstrate that rewards are the primary initial drive of addiction (Sjoerds, Luigjes, van den Brink, Denys and Yucel 2014). However, as drug abuse continues, drugs begin to lose their value to the user prompting a decrease in goal-directed behavior and a transition to irrational, habitual behavior. Nevertheless, since the addicts’ original choice to take drugs was dependent on the reward mechanism, his/her addiction was the result of a rational choice.

The Thalamostriatal Pathway

The thalamostriatal pathway mediates reflexive responses to sensory input and dopamine-dependent habit formation

Thalamostriatal afferents are largely composed of projections from the CM/Pf complex, but the rostral intralaminar, midline and specific relay nuclei also provide significant inputs (Lacey, Bolam and Magill 2007). The GPi and SNpr are two main inputs to the CM/Pf which, in turn, preferentially activates the D1-SPNs (Sidibe and Smith 1996). Thalamostriatal projections therefore close a positive feedback loop that includes the striatum, GPi, SNpr, and thalamus (Fig. 3).

Functionally, CM/Pf neurons generate reflexive, time-dependent responses to sensory stimuli, cues, and attention demanding tasks (Kinomura, Larsson, Gulyas and Roland 1996; Minamimoto and Kimura 2002). Behavioral events transmitted along the thalamostriatal pathway likely provide a strong basis for action selection in coordination with dopaminergic motivation that is essential for habit execution (Kimura, Minamimoto, Matsumoto and Hori 2004).

Habit formation requires thalamostriatal projections

The synaptic mechanisms underlying the reward-motivated behaviors, described above, appear to rely on the ventral striatum (Kalivas and Volkow 2005), where the NAc is the primary site that mediates the expression of learned behaviors in response to stimuli predicting motivationally relevant events (Kelley 2004). On the other hand, plasticity in the dorsal striatum relies upon dopamine input from the substantia nigra and is thus more closely tied to the motor function (Bamford and others 2008; Ferguson and others 2011). Sensitized locomotor responses to repeated dopamine release via psychostimulant use have long been used to assess parallel synaptic changes within the dorsal striatum. The use of the psychostimulants amphetamine and methamphetamine in mice are known to cause a short-term increase in locomotor activity and their repeated use promotes a progressive increase in this locomotor response. Experiments performed in the Bamford and Sulzer labs have shown that dopamine release by a psychostimulant reduces corticostriatal activity by depressing excitatory terminals with a lower probability of release (Bamford and others 2004b). In this way, dopamine can filter extraneous information (Dani and Zhou 2004). The repeated use of a dopamine-releasing psychostimulant promotes a chronic presynaptic depression (CPD) (Bamford and others 2008). CPD reduces corticostriatal activity during drug withdrawal and lasts >140 days in mice, perhaps the longest-lasting form of presynaptic plasticity yet characterized (Day and Carelli 2008). Drug reinstatement in withdrawal provokes a paradoxical presynaptic potentiation (PPP) of glutamate release that abnormally boosts corticostriatal activity. Behaviors consistent with CPD and PPP have been found following psychostimulant treatment, where a reduction in motor activity in withdrawal precedes an enhanced response to an AMPH challenge (Robinson and Camp 1987). The activity of the dorsal striatum in response to dopamine differs from the NAc as presynaptic corticostriatal projections in the dorsal striatum lack presynaptic D1Rs (Bamford and others 2004b) and the baseline equilibrium in response to dopamine shifts in favor of the direct pathway. The presence of dopamine and acetylcholine are both necessary and sufficient to support these long-term behavioral and synaptic adaptations following repeated psychostimulant administration. Additional experiments in the Bamford and Darvas labs using floxed mice to reduce choline acetyltransferase, the key enzyme in acetylcholine production, showed that PPP is at least in part, dependent on cholinergic interneuron function, implying that locomotor sensitization is at least partially reliant on thalamostriatal activity.

Action-outcome behaviors can become engrained over time, leading to the formation of habitual behaviors that are less susceptible to reward devaluation (Dickinson, Selleck, McMahon and Bronner-Fraser 1995). Habitual behaviors are based directly on a stimulus or cue. In habitual behaviors, the motivational functions of the corticostriatal pathway appear to become less important. Using viral manipulations in floxed VGLUT2 mice, the Bamford and Darvas labs have shown instead, that they heavily depend on the thalamostriatal pathway (Fig. 6) (Melief and others 2018). Mice without the thalamostriatal pathway have little difficulty in cognitive-dependent processing, such as learning new cue-based behaviors or remembering environmental cues. However, they demonstrate a slowness of thought (bradyphrenia) and movement (bradykinesia) that is characteristic of patients with Parkinson’s disease or those treated with drugs that block dopamine receptors. Without the thalamostriatal pathway, actions take longer to execute because they lose the automatic, habitual movements and are forced instead to depend rational, cortically-derived goal-directed behaviors.

Figure 6.

The effect of the thalamostriatal input on motor function was tested by using CAV2 viral manipulations in floxed (CAV2^Cre-Slc17a6^lox/lox) mice and CAV2^Cre-Slc17a6^+/+ controls. These VGLUT2 knockout mice specifically reduced excitatory signaling from the thalamus to the striatum. (A) VGLUT2 knockout mice (n = 8) showed no significant difference in spontaneous novelty-induced locomotion compared to control mice (n = 9). (B) VGLUT2 knockout mice (n = 9) had significantly more slips on the balance beam task compared with controls (n = 5). (C) VGLUT2 knockout mice (n = 17) performed significantly worse over time on the rotarod apparatus as compared with controls (n = 11), indicating an impairment in learned motor behaviors and innate balance. Data are shown as mean ± SEM. *p < 0.05, **p < 0.01, ANOVA. Statistical significance of genotype effects (repeated-measures two-way ANOVA) are shown in the headings of panels A and C (not significant, n.s.; **p < 0.01). Reproduced with permission from Melief et.al., npj Parkinson’s Disease. 4:23 (2018). Minor changes were made to the figure legends. http://creativecommons.org/licenses/by/4.0/

Habitual behavior is irrational, yet physiologically necessary

Habitual, reflexive behaviors have applications in economics and are most closely associated with heuristics, mental shortcuts based on prior experience that allow humans to circumvent the reward circuit, making the resulting decisions irrationally devised. Such a heuristic would include the tendency to buy stocks when their values are rising and to sell when their values are falling out of a learned expectation that such a trend tends to continue. This heuristic dominates Wall Street such that financiers have developed automated computer trading algorithms that tend to eliminate rational decisions (Kim 2007). Habits and heuristics are essential to human function because they limit the amount of critical thinking that a human must do. Many learned functions that have become reflexive, such as walking, driving, showering and dressing, do not require critical thinking—it is easier and faster to continue the learned behavior so long as it produces a reasonable and predictable result. If the outcome changes for the worse, the habit may be overridden by rational input from the cortex or by the potentially stronger emotional input from the amygdala, which conceivably can act to reprogram the behavior.

The Amydalostriatal Pathway

Emotional responses can be reflexive

Emotions are known to mediate goal-directed behaviors through the amygdala (Weiskrantz 1956), which receives sensory inputs from both the thalamus and the cortex (Fig. 3D) (Phelps and LeDoux 2005). The sensory inputs from the cortex receive the benefits of refined cortical processing, which are especially necessary if the stimuli are complex. In cases where cortical processing of sensory stimulation has not yet occurred, a more immediate reflexive response is driven by the cruder, yet faster thalamic input (Phelps and LeDoux 2005). The resulting response (fight, flight or freeze) is subsequently amplified or inhibited by the cortical input that has placed context on the nature of the stimuli (friend or foe).

Emotional responses are learned and evaluate risk

Emotional inputs rely on memory of past experiences which produced harmful results. Fear conditioning experiments have expanded the role of the amygdala in its role of developing, storing and executing these emotional responses (Phelps and LeDoux 2005). Emotional memories are logged following a negative stimulus through the interaction between the mPFC and the amygdala (Cardinal, Parkinson, Hall and Everitt 2002). The connections between the amygdala and the mPFC have been implicated in both representing incentive values and responding to conditioned punishment (Cardinal, Parkinson, Hall and Everitt 2002), suggesting that this amygdala-cortical pathway is involved in evaluating risk, the chance that a behavior will produce a harmful result. The mPFC projects to the NAc (Ferry, Ongur and Price 2000), providing the NAc with a cortically-processed emotional response. The amygdala’s projections to the VTA (Maeda and Mogenson 1981) also provide an indirect pathway where emotional responses may trigger dopamine release in the NAc (Mogenson, Jones and Yim 1980). The limbic and cortical systems combine in the NAc to motivate complex behavior (Mogenson, Jones and Yim 1980). The reward pathway and the emotional pathway oftentimes act conjointly, but, at times, they can conflict, especially when the cortex proves unable to extinguish powerful emotions like fear (O’Donnell 2010).

The economics of emotional behavior and risk

Emotions are primal instincts engrained in humans that provide protection from their reward-seeking tendencies. Rational decision making is the result of when rewards are fairly compared to risks resulting from a behavior. Conflicting emotional and reward mechanisms can result in decision making that is irrational. Prospect theory, first pioneered by Daniel Kahneman and Amos Tversky (Kahneman and Tversky 1979), outlines how humans respond to different probabilities of risk versus reward. They found that humans tend to overweight the probability of risk over the probability of reward. This differential weighing of risk versus reward correlates with a phenomenon of loss aversion wherein a loss hurts more than a gain of a similar magnitude (Kahneman and Tversky 1992). As humans preferentially weigh a loss of reward (as evaluated by risk) as more significant than the equivalent gain of reward, they will choose not to take risks that will, on average, yield a net positive reward. As such, the strength of emotions can lead to behaviors which must be classified as irrational since they do not maximize reward.

Because emotional responses had their evolutionary root in trying to keep the organism alive, it follows that fear of a possibly life-threatening situation can override reward seeking. Therefore, emotions can contradict the reward mechanism by preventing behavior that has previously resulted in negative reward feedback. The cortex has a role in mediating this conflict between reward and emotion, but the apparent relative strength of the emotional pathway suggests that emotional irrational behaviors oftentimes prevail. Since risk has been associated with the amygdala and the prefrontal cortex (Jung, Lee, Lerman and Kable 2018) a neurophysiological explanation for prospect theory and loss aversion could be that the emotional pathway is a more powerful contributor in motivation than is reward.

Is Irrational Behavior Bad?

Humans would likely be unable to function as well as we do without the irrational, reflexive behaviors that are determined by habits and emotions. The inability to use a habitual shortcut would make humans slower at executing well-learned behaviors (Melief and others 2018), and as a result, people might have less time to think critically and execute goal-directed behaviors. While habitual behaviors can have downsides including being unable to adjust appropriately to a change in circumstance or being unable to break an established addiction, they are essential for humans to retain learned skills and acquire new ones, and to shorten response times to deal with immediate dangers.

Emotional behaviors are also essential to human survival. They oftentimes reinforce the reward mechanism through dissuading actions that could come with harmful effects. Emotions may keep humans away from dangerous behaviors like drug addiction, even when pursuing a danger may be a perfectly rational thing to do. However, emotions do not always support a person’s best interest and in such a case, the cortex is responsible for controlling the emotions that can potentially do more harm than good.

Conclusions and Future Directions

The three decision making mechanisms of rewards, habits, and emotions can be broadly separated into cortically-processed, rational behaviors and subcortical, irrational, reflexive behaviors (Fig. 7). The cortex and the associated corticostriatal pathway have essential roles not just in making human rationality possible, but also in learning to encode the habitual and emotional programs that execute behaviors once certain environmental cues are observed. These habitual and emotional programs generalize situations based on the presence of those key cues, resulting in irrational behavior that does not fully evaluate reward potential and does not change based on reward devaluation. The neurophysiological evidence that humans frequently employ irrational behaviors suggests that economists need to reevaluate their models that were based on the classical assumption that humans make rational decisions. A closer collaboration between economists and neuroscientists will help create a better physiological and economic model of decision making.

Figure 7.

Hypothesized circuits that mediate reflexive and rational movements and behaviors. [1] Reflexive motor, behavioral and emotional responses or que-based habits are mediated through tonically-inhibited striatal-thalamic-striatal circuits, which are activated by coincident dopamine and glutamate release in response to unexpected sensory input via the [2] spinal cord and thalamus. [3] Motor movements, [4] reward seeking, and [5] action-outcome learning are mediated through the motor, prefrontal and medial prefrontal cortex, respectively. [6] Dopamine release provides motivation to obtain a reward and encodes value.

This review greatly simplifies the brain’s vast interconnectivity that mediates reflexive and rational movements and behaviors in humans. Whether the excitatory inputs involved in cue-dependent and reward-based learning originate entirely from the cortex or in part from the thalamus remains unclear. Similarly, different cortical regions likely participate in the production of a set of overlapping movements and behaviors that are co-jointly produced to emphasize a response.

Since reward comes after the action takes place, decisions must be made about the action based on predicted reward. A neuroscientific examination of decision making should evaluate these predicted reward values for actions that an animal has never performed and shed new light on developing ways to reduce first-time drug use. Two possible methodologies of reducing tendencies to take drugs is to decrease the expected reward value or to bolster the emotion-encoded risks associated with drugs. Further studies must be completed to determine which strategy is both most effective and cost efficient. Such a study would have wide-ranging implications in public health, economics, psychology, and neuroscience.

A neuroeconomic approach to reward studies can quantify reward in terms of dopamine release such that actual reward values can be biologically determined, leading to a more precise dictum of relative rewards. Furthermore, this evaluation can be carried over to the prices that consumers are willing to spend for a certain good, establishing a dollar value for a certain level of dopamine release. Such a study could personalize this relationship between biologically-determined reward to price, developing a means to predict the behaviors of individuals in contrast to larger populations, indicating a possible advantage of a neuroeconomic approach over a psychology-based behavioral economic approach. This study would likely attract significant interest from marketers who seek to target certain consumers.

Economists have established the concepts of prospect theory and risk aversion based on observation of human behavior. Scientists should now attempt to explain the underlying mechanism as to why risk is more influential in decision making than reward.

Figure 1.

Drawings demonstrate the major components of the striatal network in mice and the proposed circuits involved in reflexive and rational actions and behaviors. (A) The midline sagittal section shows the dorsal and ventral striatum caudal to the motor and prefrontal cortex. The thalamus, which includes the centromedian and parafascicular nuclei (CM/Pf), is posterior to the striatum and dorsal to the substantia nigra pars compacta (SNpc) and pars reticulata (SNpr). Movements are encoded in the dorsal striatum, while (B) decisions and behaviors are programmed in the nucleus accumbens (NAc) of the ventral striatum. Rational and goal-directed action and behaviors rely on corticostriatal-thalamocortical loops (blue arrows), while reflexive movements and behaviors depend on communication between the thalamus and the striatum (red arrows). These two pathways are modified by dopamine projections from the SNpc to the motor striatum and by dopamine afferents from the VTA to the NAc. (C1-C3) Coronal sections moving from rostral to caudal, as indicated in panel A, show the medial placement of the globus pallidus (palaeostriatum) and the ventrolateral placement of the amygdala (arachistriatum). The caudate and putamen that comprise the neostriatum are not separate structures in mice as they are in humans.