Abstract
Coupling stimuli and actions with positive or negative outcomes facilitates the selection of appropriate actions. Several brain regions are involved in the development of goal-directed behaviors and habit formation during incentive-based learning. This Review focuses on higher cognitive control of decision making and the cortical and subcortical structures and connections that attribute value to stimuli, associate that value with choices, and select an action plan. Delineating the connectivity between these areas is fundamental for understanding how brain regions work together to evaluate stimuli, develop actions plans, and modify behavior, as well as for elucidating the pathophysiology of psychiatric diseases.
I. Introduction
Incentive-based learning, the development of goal-directed behaviors and habit formation, is pervasive throughout life. Coupling stimuli and actions with positive (reward) or negative outcomes facilitates the selection of appropriate actions. When outcomes deviate from expectations, these links change to control future behavior. Following extended exposure, these goal-directed, outcome-guided responses can transition into the habits that allow us to operate efficiently in our environments but that can be hijacked in disease. Many brain regions are involved at different levels of incentive-based learning, from those that regulate basic survival functions to those mediating higher cognitive control of decision making. This Review focuses on the latter, cortical and subcortical structures and connections involved in attributing value to stimuli, associating that value with choices, and selecting an action plan to obtain a preferred outcome. These structures include the orbitofrontal cortex (OFC), the anterior cingulate cortex (ACC), the striatum, and midbrain dopamine (DA) neurons. In addition, parts of the dorsal prefrontal cortex (dPFC), amygdala, hippocampus, ventral pallidum (VP), and lateral habenula (LHb) are important regulators of the system. These areas form a complex neural network and delineating the connectivity between these regions will help us understand how they cooperate to evaluate environmental stimuli, transform that information into actions, and adapt future actions based on learned associations. It is also essential for elucidating the pathophysiology of psychiatric diseases associated with these cortical regions, including obsessive-compulsive disorder, depression, and addiction.
Anatomical and behavioral experiments in animals form the backbone for understanding this system. These studies, coupled with imaging studies, focus on the functional and structural connectivity of human brain regions involved in incentive-based learning and allow us to gain great insight into what comprises the network and how it changes with different contingencies. A key challenge is to translate what we know about the circuitry from the anatomical studies in animals to imaging (fMRI and diffusion-weighted MRI [dMRI]) in the human brain. The two main obstacles are determining homologies between species (especially cortex) and the lack of comparable spatial resolution that is only possible in animal tracing experiments but not in human imaging studies. Nonetheless, detailed anatomical comparisons show that the OFC and ACC are relatively homologous between nonhuman primates (NHPs) and humans (Ongür and Price, 2000; Petrides and Pandya, 1994) (discussed further below). This, along with advances in neuroimaging techniques that have increased spatial and temporal resolution, have put us in a good position to use NHP studies to gain a better understanding of human circuits that underlie incentive-based learning.
New techniques and behavioral paradigms have resulted in a dramatic increase in studies that focus on reward and decision making. However, given the different behavioral paradigms and technologies employed, the literature is complex and often difficult to synthesize. Our goal here is not to exhaustively review the literature but rather to focus on the NHP circuit anatomy and examine how this connectivity has implications for regional brain function. We first outline the anatomical circuitry, highlighting the functional implications. Then, we review the network and pathways that link these areas based on anatomical and imaging data. Finally, we discuss the association between disruptions in these circuits and disease.
II. Historical Perspective and Overview of the Basic Circuit
The classic studies of Olds and Milner revealed an internal system of specific structures that underlie motivation (Olds and Milner, 1954). Here, rats would work for electrical stimulation, with the most effective sites along the medial forebrain bundle. Pharmacological manipulation of those sites, in particular with drugs of abuse, supported the existence of linked structures comprising a motivational or reward circuit (Carlezon and Wise, 1996; Phillips and Fibiger, 1978). While several brain regions are part of this circuit, based on the most effective areas of self-stimulation, and on pharmacological, physiological, and behavioral studies, the nucleus accumbens (or the ventral striatum [VS]) and the ventral tegmental area (VTA) DA neurons are at its center (Kelley and Berridge, 2002; Schultz, 2000; Wise, 2002). Importantly, this basic, internally driven system of brain regions does not receive direct sensory input. Thus, information about the environment, value associated with stimuli, and cognitive regulation that controls the innate reactions lie elsewhere.
The OFC and ACC are the key cortical regions long associated with motivational control and adaptive behavior. The idea that the OFC is central for regulating decision making was demonstrated initially in 1848 (Harlow, 1848) and subsequently with lesions showing the inability to alter behavior when reinforcement contingencies change. Importantly, the OFC lies at the crossroads between sensory systems, the limbic system and cognitive association prefrontal cortical areas, with direct connections with all three. The ACC has been linked to emotional processing since Papez’s description of a circuit that mediates emotional expression (Papez, 1995). Shortly thereafter, cingulotomy, a replacement for the more extensive frontal lobotomies, was demonstrated as an effective treatment for major depressive disorder (MDD) and later treatment-resistant obsessive-compulsive disorders (OCDs) (Feldman et al., 2001; Greenberg et al., 2010). Like the OFC, the ACC lies at a crossroads of function. However, it is at the juncture of emotional, cognitive (attentional), and action outcome processing. The OFC and ACC provide the main cortical afferent projection to the VS. These inputs converge with those from the amygdala, hippocampus, and the massive DAergic input from the midbrain. While DA has long been associated with motor control, the use of DA antagonists (neuroleptics) demonstrated that motivation was diminished before compromising motor control. Detailed animal experiments followed, clearly showing the central role of DA for reinforcement learning (reviewed in Wise, 2002). The main midbrain outputs are to the striatum and cortex, and, most relevant here, to the VS, OFC, and ACC. The VS projects to the ventral pallidum (VP) and to the VTA/substantia nigra (SN), which, in turn, project back to the prefrontal cortex, via the thalamus. Thus, taken together, the brain reward/incentive circuit is now considered embedded within the corticobasal ganglia (BG) system.
Historically, the BG were best known for their relevance to motor functions. We now know that BG are responsible for a more complex set of functions that mediate the full range of goal-directed behaviors, including emotions, motivation, and cognition. This idea developed in the late 1970s with the concept of the VS and VP (Heimer, 1978) and the demonstration of a limbic cortico-BG circuit passing through the MD thalamus to nonmotor cortex (Figure 1A). This notion was later expanded in primates to include several separate circuits through the BG (Alexander et al., 1990). Importantly, the idea of a motivation-to-movement interface, rather than separate loops through BG circuits was developed soon after the discovery of the limbic component to the BG (Mogenson et al., 1980). Indeed, it is through the integration between functional circuits that incentive-based learning occurs, which can lead to the development of habits (Belin and Everitt, 2008; Draganski et al., 2008; Haber et al., 2000, 2006; Percheron and Filion, 1991).
Figure 1. Schematic Illustrating Key Structures and Pathways of the Reward Circuit.
(A) The place of the reward circuit in the basal ganglia system. Red arrows, the reward circuit; black arrows, cognitive and motor circuits. AC, anterior commissure; Amyg, amygdala; Cd, caudate nuclei; dACC, dorsal anterior cingulate; dPFC, dorsal prefrontal cortex; GP, globus pallidus; Hipp, hippocampus; LHb, lateral habenula; OFC, orbital frontal cortex; Pu, putamen; RMTg, rostromedial tegmental nucleus; SN, substantia nigra, pars compacta; STN, subthalamic n.; VA/VL/MD, ventral anterior/ventral lateral/medial dorsal nuclei of the thalamus; vmPFC, ventral medial prefrontal cortex; VS, ventral striatum; VP, ventral pallidum; VTA, ventral tegmental area.
(B) Overview of cytoarchitectonic divisions of lateral, medial, and orbital cortex in human (top) and monkey (bottom) adapted from Petrides and Pandya (1994). dACC, dorsal anterior cingulate; dlPFC, dorsal lateral prefrontal cortex; dPFC, dorsal prefrontal cortex; vlPFC, ventral lateral prefrontal cortex; vmPFC, ventral medial prefrontal cortex; OFC, orbital frontal cortex.
Understanding the Circuit through Its Connections
Two major advances that set the stage for modern neuroanatomy and the idea that connections between brain regions constituted functional networks were the development of the Golgi stain (1873) and the Marchi degeneration stain (1886). Improvements in this approach over the next eight decades not only demonstrated some basic connections of the corticocortical and cortico-BG system, but also stimulated much controversy regarding whether actual connections existed and were direct (not unlike issues raised today with dMRI). These early studies provided the basis for more detailed exploration using modern tracers. For example, the idea that the VS and that part of the substantia innominate should be reclassified as the VS and VP, respectively, and therefore constitute the limbic component of the BG was based on a modified degeneration method (Heimer, 1978). Around the same time, developments in chemical neuroanatomy gave insight into the localization of neurotransmitters and receptors within structures and specific pathways. Thus, the VTA-VS projection was shown to be DAergic, first with fluorescent methods, then with antibodies. The link between function and anatomy in the human brain exploded with the development of imaging methods in the mid-1980s. Soon after, with the development of fMRI, activation of the VS and cortical areas associated with reward-related tasks was visualized in the human brain.
While the basic anatomy and connectivity of the structures of the reward circuit are now well established, the pathways that link them are not. Indeed, other than sensory and motor systems, much of what we know about how axons from specific brain regions reach their targets relies largely on the older studies using lesions and degeneration stains to follow fiber trajectories. In fact, with a few notable exceptions, most NHP anatomy studies document the end points of connections, that is, whether A is connected to B, but most often without a thorough examination of the route fibers take between those points or their organization within the large white matter (WM) bundles. However, the development of dMRI, its use for determining pathways, and changes in those pathways associated with disease, has emphasized the importance of this knowledge.
III. Connections and Functions
The OFC and ACC
The orbital and cingulate cortex are complex and heterogeneous regions, each of which are further divided, based on architectonics and connections, into specific cortical areas: the ACC includes areas 24, 25, and 32; the OFC is divided into areas 11, 13, 14, and 12/4. Although determining the homologies between NHP and human prefrontal cortical areas is a complex and difficult task, there is reasonable overall agreement based primarily on cytoarchitectonics (Figure 1B) (Ongür and Price, 2000; Petrides and Pandya, 1994). However, there are a few regions, notably parts of area 32 and 10 that probably do not exist in the NHP. In addition, while in Brodmann’s map, area 11 lies along the midline (without a designated area 14), more recent maps show anatomical correspondence of area 14 along the midline across primates. The anatomical studies in animals provide a very detailed understanding of connections of these areas, revealing highly complex associations that vary in strength. In contrast, imaging studies cannot distinguish between cytoarchitectonic divisions directly and different areas have thus been functionally grouped. Taken together, rather than review all possible anatomical connections, we broadly define the areas and highlight the strongest links associated with each region.
The OFC can be generally parceled into somewhat functionally different regions based on a medial-lateral and caudal-rostral axis. The medial OFC, including gyrus rectus and roughly bounded by the medial OFC sulcus, contains primarily area 14, central OFC, areas 13 (caudally), and 11 (rostrally), is roughly bounded by medial and lateral OFC sulci. Area 12/47 is located most laterally (Chiavaras and Petrides, 2000) (Figure 1B). The caudal OFC region (in which we include parts of the insula cortex) includes areas 13, caudal 14, and caudal 12/47. This region has stronger inputs from primary sensory systems. In contrast, the more rostral region (areas 11, rostral 14 and 12/47), receives a combination of highly processed sensory inputs and has tighter links to rostral prefrontal cortex (PFC) areas (Barbas, 2007; Carmichael and Price, 1995b). Thus, more caudal areas are closely associated with primary sensory information, while rostral regions are linked to higher sensory processing. Lateral OFC areas are generally connected to ventral lateral PFC (vlPFC) and medial OFC areas to parts of the ACC and area 10, forming two somewhat separate streams of information processing, from primary reward to more abstract cognitive decision making (Wallis, 2010).
The ACC lies on the medial surface, extending from the level of premotor cortex, curving rostrally with the genu of the corpus callosum and then caudally, ventral to the callosum. In the human brain, areas 24 and 32 extend throughout this territory, with area 32 dorsal to area 24 caudally and extending rostral and ventral to it as it tucks beneath the genu. In the NHP brain, area 32 is more limited and primarily occupies a rostral position. Overall, the ACC can also be divided functionally along its dorsal-ventral and rostral-caudal axes. The dorsal ACC (dACC), primarily area 24 in the NHP, is tightly linked with many PFC areas, including lateral regions associated with cognitive control, and more caudally, with motor control areas. In contrast, the ventral and caudal ACC (areas 25 and ventral parts of 24 and 32) is more closely associated with visceral and emotional functions. These ventral areas have strong connections to the hypothalamus, amygdala, and hippocampus. From a functional perspective, imaging and lesion studies have identified an area referred to as the vmPFC. Depending on the specific study, this region may include different combinations of these regions but overall involves areas 25, parts of 32, medial OFC (area 14 and 11), and ventral-medial area 10. However, in some studies, experimenters focus on more caudal regions (areas 25, 32, 14 and caudal 10), while in others this region does not include area 25 but includes rostral areas 14 and 10.
OFC and Insula
Connectional Considerations
The OFC is highly intraconnected. Overall, lateral areas are tightly linked to vlPFC, while medial regions are linked to the vmPFC and dACC. In addition, patches throughout the OFC are connected to various cingulate, PFC, amygdala, and temporal lobe regions. For example, entorhinal cortex and the subiculum inputs are scattered throughout the rostral and caudal OFC, with the subiculum terminating in the medial region. From a functional perspective, there is a general caudal and rostral connectional distinction. Direct inputs from primary olfaction and gustatory information cortices along with more highly processed visual, auditory, and somatosensory systems terminate in the insula. The insula is divided into three cytoarchitectonic areas, arranged in a radial manner around the piriform olfactory cortex, that are associated with different sensory functions: agranular insula (Ia)—olfactory and autonomic functions; dysgranular insula (Id)—gustatory and some visual and somatosensory functions; and granular insula (Ig)—somatosensory, auditory, and visual functions (Carmichael and Price, 1995b; Mesulam and Mufson, 1993). Like the insula, the caudal OFC receives input from all of the sensory modalities and is considered important for integrating input from multisensory regions (Barbas, 2007; Carmichael and Price, 1995b; Morecraft et al., 1992). In addition to the sensory input, perirhinal cortex, an area important for object recognition, also projects primarily to the caudal OFC (Carmichael and Price, 1995a). Thus, the caudal OFC is central for processing sensory inputs related to object recognition and primary reward (food) and emotional valence. Caudal and rostral OFC are tightly connected. However, in contrast, to the caudal areas, the rostral OFC receives not only highly processed sensory information, it is also connected to cognitive areas of the frontal lobe, including the frontal pole and rostral areas 9 and 24 and lateral areas 45 and 46. Interestingly, the parahippocampal gyrus, which is linked to higher cognitive processes, also projects primarily to the rostral OFC (Figure 2A) (Ongür and Price, 2000).
Figure 2. Overview of OFC Connections and Pathways.
(A) General flow of OFC connections. Note not all are illustrated; for details, see Carmichael and Price (1995a, 1995b) and Barbas (2007). Numbers refer to cortical areas.
(B and C) Schematic pathways from two OFC injection sites (medial OFC = yellow [B]; lateral OFC = blue). Adapted from Lehman et al. (2011). AF, ventral amygdalofugal pathway; CB, cingulum bundle; CC, corpus callosum; dACC, dorsal anterior cingulate cortex; dPFC, dorsal prefrontal cortex; EmC, extreme capsule; IC/EC, internal and external capsules; ILF, inferior longitudinal fasciculus; MLF, medial longitudinal fasciculus; PFC, prefrontal cortex; SLF, superior longitudinal fasciculus; UF, uncinate fasciculus; vlPFC, ventral lateral prefrontal cortex; vmPFC, ventral medial prefrontal cortex.
To reach their targets, all OFC axons enter the uncinate fasciculus (UF) immediately adjacent to their cortical region and divide into three bundles, medial, dorsal, and lateral (Figure 2B). The medial bundle remains within the UF to innervate other OFC regions and the vmPFC. Fibers travel laterally within the UF to reach the extreme capsule or to enter the inferior and middle longitudinal fasciculus to terminate in the entorhinal cortex, parahippocampal cortex, superior temporal gyrus, etc. The dorsal bundle cuts directly through the UF, arches medially within the frontal WM to reach the corpus callosum to terminate in the contralateral hemisphere, the cingulum bundle (CB) to terminate in the ACC, and superior longitudinal fasciculus to terminate in dorsal and lateral PFC. The lateral bundle divides into fibers that travel in external and internal capsules. OFC axons are organized topographically in the internal capsule, with fibers from medial regions traveling ventral to those from lateral OFC areas. This positioning within the capsule impacts on the connections involved in lesions from stroke or invasive surgery for treatment of specific psychiatric disorders. For example, deep brain stimulation and anterior capsulotomy are two surgical treatments for OCD and depression that target the ventral part of the anterior limb of the internal capsule (Greenberg et al., 2003, 2010). Using the data mentioned above, we can predict that the more ventral electrodes or lesions will probably involve more medial vPFC connections, and more dorsally placed electrodes or lesions will involve more lateral parts of the vPFC. Fibers traveling to the amygdala and hypothalamus form small bundles that enter the amygdalofugal pathway or travel within the UF. These axons are difficult to follow using MRI tractography and therefore this connection in human imaging is not easily distinguished from other projections. Finally, a group of axons continues in the external capsule to the claustrum (Figure 2B).
Functional Associations
OFC’s unique access to both primary and highly processed sensory information, coupled with connections to the amygdala and cingulate, explain many of the functional properties of the region. Historically, the two cardinal tests of OFC function have been reward devaluation paradigms and stimulus-outcome reversal learning (McEnaney and Butter, 1969; Fellows and Farah, 2003; Izquierdo et al., 2004), both of which have been demonstrated with OFC lesions across species. In the reward devaluation paradigms, animals learn two stimuli that predict two different reward types before one of these outcomes is paired with illness (or devalued). Control animals will immediately select the stimulus whose outcome is not associated with illness and signals in human OFC have demonstrated appropriate revaluation of the stimuli (Kringelbach and Rolls, 2003; Gottfried et al., 2003; O’Doherty et al., 2000). Animals with OFC lesions are not able to make the correct choice and continue to select the devalued stimulus. Importantly, the outcome-predicting stimuli have not been paired with illness, only the outcomes themselves. However, on the decision trials, the subject is confronted with the two stimuli in the absence of the outcomes. In order to select appropriately, the animal must use each stimulus to predict the respective outcome, realize that one outcome is no longer valuable, and select the other stimulus. If the subject has instead learned that the stimuli are valuable per se, due to their historical relationship with reward, they will continue to select the inappropriate stimulus. Thus, the critical feature that is impaired by an OFC lesion is the capacity to use the stimuli to elicit sensory representations of the outcomes, which can then be evaluated (Burke et al., 2008). Similar findings can be observed in imaging studies of human OFC during devaluation studies. Electrophysiological studies demonstrate cellular activity that supports this function (Figure 3A). The activity of single cells predicts not only the value of a particular outcome, but also the type of reward that is to be expected (Padoa-Schioppa and Assad, 2008). Moreover, activity in the human OFC is suppressed after repetition of the same reward type only if it is predicted by the same stimulus on each occasion (Klein-Flügge et al., 2013) (Figure 3B). Insofar as repetition suppression indexes cellular representations, this suggests that the representations of valuable outcomes in the OFC are tied to the sensory stimuli that predicted them. Thus, linking between sensory representations of stimuli and outcomes is a key function of the OFC, consistent with its connectional proximity to sensory and reward-related regions.
Figure 3. OFC.
(A) OFC cells respond differentially to stimuli that predict different types of juice reward. The cell on the left codes only the value of juice C. The cell on the right fires whenever juice B is chosen. Adapted from Padoa-Schioppa and Assad (2008).
(B) Both medial and lateral OFC regions show fMRI adaptation (suppression) to stimuli that predict the same reward. However medial OFC shows suppression between two different stimuli that predict the same reward (green), but lateral OFC only shows suppression if a reward is predicted by the same stimulus (red). Coding in lateral OFC binds a stimulus to a reward. Adapted from Klein-Flügge et al. (2013).
(C and D) Double dissociation between lesions to OFC and ACC in reward-guided learning. OFC lesions cause deficits when macaques (C) or humans (D) learn about stimuli. ACC lesions cause deficits when it is actions that are being learned. Adapted from Rudebeck et al. (2008) and Camille et al. (2011b).
(E) A rostrocaudal gradient for reward in OFC. More caudal regions code for primary reward, whereas more rostral regions code for secondary reward. Adapted from Sescousse et al. (2010).
In reversal learning studies, one stimulus is paired with reward and another is not. Once learning has been acquired, the rewarding stimulus switches, so that reward is now delivered after the previously unrewarded stimulus. OFC lesions (across species) impair subjects in making this switch and they continue to choose the unrewarded stimulus for many trials (Fellows and Farah, 2003; Iversen and Mishkin, 1970; Izquierdo et al., 2004; McEnaney and Butter, 1969). A striking demonstration of the sensory nature of this deficit is that the deficit in learning these reversals disappears if actions rather than sensory stimuli are paired with reward (Camille et al., 2011b; Rudebeck et al., 2008) (Figures 3C and 3D). Using more sophisticated tasks, it is now clear that even initial learning (before reversal) is reliant on OFC if the contingent relationship between stimulus and outcome is complex. Detailed analysis suggests that when OFC-lesioned animals are rewarded, they are no longer able to pair this reward with the correct contingent stimulus but instead spread it among recent choices (Takahashi et al., 2011; Walton et al., 2010).
Intriguingly, reversal-learning deficits that are prevalent after aspiration lesions to OFC are not evident after excitotoxic lesions that spare the passing fibers (Rudebeck et al., 2013). As indicated below, the fibers passing through this area include not only OFC axons, but also reciprocal connections between the vmPFC and temporal lobe traveling through the UF, vmPFC fibers entering the medial forebrain bundle to terminate in the hypothalamus, amygdala, brainstem, and dACC connections with the OFC with some passing to subcortical areas. While, the OFC clearly plays a central role in reversal learning, the disruption of its association with a wider network maybe responsible for these deficits or prevent strategies to ameliorate them.
Consistent with connectional differences between caudal and rostral OFC, there is an apparent gradient between primary reward representations in more caudal OFC/insular cortex and representations of secondary reward such as money in more rostral OFC regions (Figure 3E) (Sescousse et al., 2010). Such dissociations may rely on the differential inputs from early versus higher sensory representations to caudal versus rostral OFC, respectively, or the preponderance of amygdala connections to caudal OFC but more executive regions such as dorsolateral prefrontal cortex (dlPFC) and frontal pole to rostral OFC. An intriguing possibility is that this gradient in connections related to both stimulus and outcome representations might allow OFC learn relationships between increasingly complex concepts and goals.
vmPFC
Connectional Considerations
The caudal vmPFC (primarily sACC–areas 25 and 32, and caudal area 14) is distinguished from the OFC and the dACC by its strong links to visceral and emotional processing stations, in particular, the hypothalamus, amygdala and the shell of the VS (Freedman et al., 2000; Ongür et al., 1998; Rempel-Clower and Barbas, 1998). Inputs from the hypothalamus, hippocampus, and amygdala terminate most densely in these caudal parts of the vmPFC, although these inputs are also found in specific regions of lateral OFC and parts of the dACC. The caudal vmPFC projects most densely to both the hypothalamus and amygdala. Moreover, there is also a tight link with parts of the insula. Thus, these strong reciprocal connections, its input from the hippocampus, along with a projection to the VS (discussed below) are special features of caudal vmPFC. The caudal vmPFC is also connected to more rostral vmPFC areas, and to some extent areas 10, 9, and dACC. In contrast, rostral vmPFC is not tightly linked to the amygdala and hypothalamus. Rather, these areas have closer connections to areas 10, 9, and 45 (Freedman et al., 2000; Ongür and Price, 2000). Thus, as seen with the OFC, more caudal vmPFC is associated with “primary” functional areas, while rostral regions are more tightly linked to cognitive-related areas (Figure 4A).
Figure 4. Overview of vmPFC Connections and Pathways.
(A) General flow of vmPFC connections. Note not all are illustrated; for details, see Carmichael and Price (1995a, 1995b).
(B) Schematic pathways from an injection site illustrating how different bundles separate from the injection site as they enter the white matter. Adapted from Lehman et al. (2011). AF, ventral amygdalofugal pathway; CB, cingulum bundle; CC, corpus callosum; dACC, dorsal anterior cingulate cortex; dPFC, dorsal prefrontal cortex; IC/EC/EmC, internal, external, and extreme capsules; MFB, medial forebrain bundle; NAccS, nucleus accumbens shell; SLF, superior longitudinal fasciculus; ST, stria terminalis; UF, uncinate fasciculus; vmPFC, ventral medial prefrontal cortex.
Pathways from the vmPFC to cortical and subcortical areas follow similar routes as those from the OFC with some notable exceptions (Lehman et al., 2011) (Figure 4B). Fibers enter the adjacent UF and divide into three bundles. A medial bundle travels dorsally to enter the subgenual CB and corpus callosum. A central bundle splits into three bundles that enter the internal, external, and extreme capsules and the striatum. A lateral bundle travels laterally in the UF to innervate OFC and the temporal lobe. However, unlike the OFC fibers, axons travel in the medial forebrain bundle to enter both the ventral amygdalofugal pathway and stria terminalis to reach the amygdala. Furthermore, they travel in small fascicules within the VS and below the anterior commissure, which are a ventral extension of the internal capsule. Thus fibers from the vmPFC will be interrupted in lesions of the VS but not with damage to the more conventionally defined internal capsule. Moreover, because these axons travel through the small bundles within the gray matter, they are difficult to follow through the internal capsule using dMRI tractography (Jbabdi et al., 2013).
Functional Associations
NHP studies of vmPFC are rare, but where data exist, they are broadly consistent with connectional anatomy (Rudebeck and Murray, 2011). For example, vmPFC’s strong, direct projections to ancestral regions such as hypothalamus and parts of the insula that maintain representations of internal states, such as hunger and thirst, may account for the fact that vmPFC cells are more likely to track values in the context of internally generated states such as satiety (Bouret and Richmond, 2010). In contrast, those in OFC are more likely to track values learned from external sensory stimuli (Bouret and Richmond, 2010). Such internal states are essential components underlying decisions. Together with dACC, the vmPFC is in an important position in the transition from valuation to choice. Indeed, when selective lesions are made to vmPFC and lateral orbitofrontal cortices, there is a double dissociation between learning about stimulus-outcome relations (lOFC) and using these relations to guide choice (vmPFC) (Noonan et al., 2010). For example, animals with vmPFC lesions make irrational choices that break fundamental economic axioms (Figures 5A and 5B). Although human lesions cannot be targeted with such anatomical precision, lesions that include the entire vmPFC (as well as many orbital regions), have a similar effect. For example, such patients are more likely to make errors of transitivity in simple value-guided choices (Camille et al., 2011a). In human imaging studies, vmPFC provides the most robust and reproducible valuation signal in the brain (Bartra et al., 2013; Clithero and Rangel, 2013) (Figure 5A), but the temporal dynamics of this signal (Hunt et al., 2012), and its dependence on local Glutamate and GABA concentrations (Jocham et al., 2012), suggest a role, not only in valuation of stimuli, but also selecting between these values. However, perhaps the most remarkable feature of the vmPFC valuation signal is its flexibility. While many other brain regions rely on experience to estimate values, vmPFC can encode values that must be computed on the fly. These computations may, for example, rely on an understanding of the structure of the environment (Hampton et al., 2006); on the generalization of concepts learnt in different situations (Kumaran et al., 2009) (connections to the hippocampus and temporal lobe); or on the integration of disparate sources of information (connections to dACC and OFC) (Behrens et al., 2008). Strikingly, if subjects are asked to ignore their own preferences, and instead guess what a very different individual would choose, vmPFC immediately reflects the values of this new individual (Janowski et al., 2013; Nicolle et al., 2012). If, however, the problem at hand is best solved by considering values learned from direct experience, the vmPFC could seamlessly revert to these more basic value computations (Wunderlich et al., 2011). This flexibility is likely to be linked to the strong rostral-caudal vmPFC connections, where the caudal part is closely linked to the amygdala, hypothalamus, and related regions, while the rostral part is more closely tied to the rostral pole, areas 10 and 9, regions implicated in “theory of mind.” Thus, information to caudal regions is driven by connections to the amygdala, hypothalamus, caudal OFC, and dACC. While more rostral areas are linked to area 10 and rostral area 9. In this respect, the vmPFC is a core node in the “default-mode network” (Figure 5C) (Fox et al., 2005; Raichle et al., 2001). This network, studied in the context of spatial navigation (Doeller et al., 2010), episodic memory, and future thinking (Bonnici et al., 2012; Schacter et al., 2012), has also recently been shown to be involved in reward-guided behavior. One intriguing suggestion is that explicit representations built in this network may be useful when deciding in novel untrained circumstances allowing experiences built in one context to generalize to others (Barron et al., 2013) (Figure 5D).
Figure 5. vmPFC.
(A) VMPFC value signals are robust and integrate many forms of reward. Similar to OFC, there is evidence of a caudorostral gradient between primary and secondary reward. Adapted from Clithero and Rangel (2013).
(B) Macaques with vmPFC lesions break economic axioms of rational choice. In a three choice task between stimuli X,Y,Z with different values, the ratio of X to Y choices (y axis) is dependent on the difference in value between X and Y (x axis) but should be independent of the value of Z. This is true before, but not after, medial OFC lesions (encompassing vmPFC) Noonan et al. (2010).
(C) vmFPC is a core node in the default-mode network, a set of brain regions that exhibit common activation profiles and are regularly found to be functionally coupled (Vincent et al., 2006).
(D) Connections to this default mode network may explain coactivation between vmPFC and hippocampus when subjects have to imagine the value of novel goods that have never been experienced. Adapted from Barron et al. (2013).
Dorsal ACC
Connectional Considerations
Areas 24 and 32 of the dACC exhibit a gradient in connectivity from the most rostral and ventral areas (32 and rostral 24) to the caudal and dorsal areas (central and caudal 24). The rostral dACC is connected primarily to the vmPFC, OFC, medial area 9, and the rostral temporal cortex. Amygdala connections are also concentrated in this rostral dACC region (Morecraft et al., 2012; Ongür and Price, 2000). Unlike more caudal dACC regions, it has minimal connections to posterior cingulate areas. These connections support its association with emotional processing and, from a functional perspective, the rostral dACC is closely aligned with the sACC (Figure 6A). However, it is also linked to dPFC areas, thus in a pivotal position between emotion and cognitive control. Central dACC is also connected with dPFC, but these connections are more widely spread and include lateral PFC regions. Moreover, this central dACC connects more extensively to the caudal cingulate. Thus, this region is more tightly linked to cognitive control regions compared to the rostral dACC. Finally, caudal dACC has the strongest connections with motor control regions and limited ones with OFC and PFC areas 10 and 9. However, interestingly, amygdala projections continue to terminate in patches throughout the dACC, including some in the caudal parts (Carmichael and Price, 1995a). It is important to bear in mind that there are not abrupt divisions based on anatomical connectivity, rather there is a continuum of connections. In other words, animal tracing experiments show that as one moves through the rostral-caudal cingulate axis, labeled fibers and cells spread caudally through the PFC, into motor control areas (Morecraft et al., 2012). They also progress through more caudal areas of the cingulate. Thus, the dACC is in a pivotal position within the frontal cortex to link between emotion, cognition, and motor control areas of frontal cortex.
Figure 6. Overview of dACC Connections and Pathways.
(A) General flow of dACC connections. Note that not all are illustrated; for details, see (Carmichael and Price 1995a, 1995b) and Morecraft et al. (1992).
(B) Schematic pathways from an injection site illustrating how different bundles separate from the injection site as they enter the white matter. AF, ventral amygdalofugal pathway; CB, cingulum bundle; CC, corpus callosum; dACC, dorsal anterior cingulate cortex; dPFC, dorsal prefrontal cortex; IC/EC/EmC, internal, external, and extreme capsules; OFC, orbital frontal cortex; PCC, posterior cingulate cortex; sACC, subgenual cingulate bundle; SLF, superior longitudinal fasciculus; UF, uncinate fasciculus; vmPFC, ventral medial prefrontal cortex.
To reach their targets, all cingulate fibers enter the CB (Figure 6B). Some fibers simply cut through it to reach frontal WM before splitting into pathways that are directed toward the corpus callosum, internal, external, and extreme capsules, or the striatum. Fibers in the internal capsule occupy a position dorsal to those from the OFC and vmPFC, preserving a dorsal-ventral topography within the capsule. In contrast to the subcortical axons, fibers traveling to PFC regions do not form a single bundle. These axons continue to cross through the CB as a continuous stream of fibers and course along its edge arching around the gyrus to reach dorsal medial and lateral frontal areas. Thus, fibers emerging from a given dACC region traveling to lateral and contralateral cortical areas, the striatum, thalamus, and brainstem, do not remain within the CB. However, those dACC fibers that do join the dorsal CB travel rostrally and caudally for significant distances to terminate throughout the cingulate cortex, including caudal cingulate cortex. Fibers also travel ventrally within the subgenual CB to reach the sACC. Some of these axons also join the UF to terminate in the OFC. Others join the amygdalofugal pathway and medial forebrain bundle to terminate in the amygdala and hypothalamus. As with the previously discussed topography of pathways, dACC fiber organization has implications when considering lesions and differences in WM integrity in patient populations compared with healthy control subjects.
Functional Associations
In contrast to the OFC and vmPFC, the dACC has a relative absence of sensory connections but is tightly connected to motor control areas. Thus, unlike OFC lesions, dACC lesions have little effect on learning reward reversals based on sensory cues, but they do if the reward are tied to two different actions (such as “turn” or “push”). In these tasks, animals with dACC lesions (Rudebeck et al., 2008) or humans with dorsomedial prefrontal lesions that include dACC (Camille et al., 2011b) cannot acquire the reversals. Human imaging studies have found ACC activation in a wide variety of tasks, many of which can be explained in the context of its role in selecting between actions (Alexander and Brown, 2011). dACC’s role in selecting between actions is exemplified in studies of simple value-guided choice. In circumstances in which vmPFC signals appear most consistent with a process of valuation of the stimuli or goals, dACC activity exhibits functional coupling with whichever motor cortex will execute the consequent action (Hare et al., 2010).
The dACC sits at the connectional intersection of the brain’s reward and action networks (Figures 7A and 7C). Consequently, dACC cells may allow computations in units of integrated action values. For example, when NHPs are asked to make decisions that span various factors (size or available reward, the probability and cost of receiving the reward), cells in OFC code for all these variables separately across the population, by either increasing or decreasing their firing (Kennerley and Wallis, 2009). In dACC, by comparison, there exists a population of cells with very particular properties (Kennerly et al., 2011). These cells code all decision variables in an integrated signal that does not distinguish different sources of value. They code this integrated value with a positive valence, such that more valuable choices are associated with higher activity and they code a prediction error signal that reflects the difference between experienced and expected reward.
Figure 7. ACC.
(A) A meta-analysis of human functional neuroimaging studies shows a rostrocaudal gradient in dACC between motor tasks (top) and reward tasks (bottom). Adapted from Beckmann et al. (2009).
(B) Average responses of 280 ACC cells to an error (INC), an exploratory reward (COR1), and an exploitative reward (COR) (Quilodran et al., 2008).
(C) Behavioral (black bars) and computational modeling (black dots) data show that people learn more and faster in volatile compared to stable environments because every new piece of information carries more weight on future actions. Decision making induces increased fMRI activity in more posterior parts of the ACC (orange), whereas monitoring the outcome of the decisions activates more anterior parts of the ACC (blue). At the boundary between these two areas, there is a region in the ACC (green), which is more active when people monitor outcomes that have a great influence on future choices, hence when they have a higher learning rate. Adapted from Behrens et al. (2007).
(D) Dorsal ACC signals predict monkey choices in a patch foraging experiment. While the animal is within a patch, dorsal ACC neurons ramp up in activity until they reach a threshold. That moment coincides with the animal leaving the current patch and starting the exploration of a new one. The faster the neurons ramp up, the less time the monkey spends with that patch. Adapted from Hayden et al. (2011b).
Activity in dACC and neighboring dorsomedial frontal cortex is of particular importance for both the control (Frith et al., 1991; Passingham et al., 2010) and evaluation (Walton et al., 2004) of self-generated actions. Self-generated actions are beneficial if they increase the subject’s long-term reward (Cohen et al., 2007), and evidence for dACC’s role in such exploration and consequent learning is prevalent across species and techniques. In NHPs, ACC cells are more active during exploratory periods when the best actions are unknown, than in the immediate subsequent periods when the animal must repeat or “exploit” this best action (Figure 7B) (Quilodran et al., 2008). This is true both when the actions are generated, and when the result of each action is observed. In humans, the dACC is also more active when subjects explore options not expected to generate immediate reward (Boorman et al., 2009; Daw et al., 2006) or move away from their long-term default preference (Boorman et al., 2013). Such exploratory behavior only has long-term value if it causes appropriate changes in future behavior and, again, across species dACC activity plays a role in ensuring this is the case. In NHP experiments, controls can integrate across many past experiences appropriately to guide the next choice (Kennerley et al., 2006; Sugrue et al., 2004); lesions to the dACC sulcus impair this ability, such that the animals are guided solely by the last outcome (Kennerley et al., 2006). Similarly, human subjects can adjust their learning appropriately in different contexts to ensure optimal behavioral control, and dACC activity tracks the extent to which new information is useful for changing future behavior (Behrens et al., 2007) (Figure 7C).
Recent accounts of dACC function have attempted to link arguments about learning and exploration, providing a more unified view of ACC’s role in reward-guided behavior. One argument that has been put forth is that key properties of dACC activity and its differences from ventral prefrontal activity are properties that are essential for foraging choice—a type of choice that is an important determinant of evolutionary success but not often studied experimentally (Hayden et al., 2011a; Kolling et al., 2012). At decision points, dACC activity is able to code for integrated values and weigh them against costs, allowing it to assess the value of the current foraging environment. At outcome times, it can signal the shift between exploiting a current resource and exploring for new resources elsewhere and can learn future behaviors from these foraging events. When NHPs perform a traditional patch foraging experiment, in which they must decide how much time to allocate to each patch, dACC cells slowly ramp up in activity, as the animal stays longer in a patch. When the dACC population reaches a threshold firing rate, the animal leaves the current patch and forages for a new one (Figure 7D) (Hayden et al., 2011b). Furthermore, when foraging-style choices, in which values must be compared against the global average, are directly compared against neuro-economic choices in which goods are compared against each other, human blood-oxygen-level-dependent (BOLD) activity reflects foraging choices better in ACC and neuroeconomic valuations in vmPFC (Kolling et al., 2012).
Integration and Competition between Cortical Systems
In order to understand the complex functions of cortical regions, it is important to understand how they differ from their neighbors. To this end, many studies of reward-guided behavior have focused on uncovering differences in processing between different cortical regions. Partly for these reasons, and also partly because of the difficulty of making simultaneous recordings or interventions across multiple areas, there is a relative scarcity of data about how these regions interact and how activity across regions is combined into a single behavioral output.
One possibility that makes direct analogy to perceptual decision processes is that processing takes a serial route through sequential cortical regions from stimulus through valuation to action (Kable and Glimcher, 2009; Rangel and Hare, 2010; Shadlen et al., 2008). Decision processes in cingulate and motor cortices are argued to rely on outcome valuations that occur in vmPFC and orbital cortices (Kable and Glimcher, 2009; Rangel and Hare, 2010). An extension of such a theory, consistent with anatomical connectivity, is that the processes that transform stimuli to outcome values can also be thought of serially, with processed sensory stimuli arriving in lateral orbital cortex through its inferotemporal connections. Computations in orbital cortices may associate stimuli with likely outcomes (Burke et al., 2008; Klein-Flügge et al., 2013) and these outcomes may be valued in the context of the current state (such as thirst or hunger) in vmPFC (Bouret and Richmond, 2010).
Such a serial view, however, must be nuanced when faced with data that suggest that signals in different brain regions appear to reflect decision processes in different contexts. For example, when monkeys are choosing between juices associated with different stimuli, signals in lateral orbital cortex do not only reflect stimulus values but also the decisions themselves and their likely outcomes (Padoa-Schioppa and Assad, 2006). In the context of selecting over actions, however, similar competition and decision signals can be seen in motor and pre-motor cortices (Cisek, 2007), and prefrontal mechanisms are argued to bias these competitions. This dissociation is underscored by lesion data discussed above that, across species, reveal double dissociations between orbital and cingulate lesions with respect to choosing over stimuli or actions (Camille et al., 2011b; Rudebeck et al., 2008). Such considerations imply a degree of separation between decision processes in the brain, arguing that different controllers may act to some extent in parallel (Cisek, 2012; Rushworth et al., 2012). Indeed, imaging data suggest that, as the context changes, different cortical regions may exert different levels of influence over behavior on a trial-by-trial basis (Hunt et al., 2013; Kolling et al., 2012; Lee et al., 2014).
This view raises questions about how such parallel processes may compete or be integrated to control actions. We can only speculate, given current data. It is possible that specialist computations are required to best arbitrate between different decision processes and that these arbitrations may occur in separate prefrontal regions that can evaluate the performance of each decision process (Lee et al., 2014). Alternatively, separate decision processes may compete directly, either through interregional inhibition or local competition for actions (Cisek, 2012).
Ventral Striatal and Basal Ganglia Connections
Connectional Consideration
The traditionally associated reward region of the striatum has expanded from the nucleus accumbens (NAcc) to the larger VS concept based on dense inputs from the OFC, dACC, and vmPFC. This extends the reward striatal territory to include not only the NAcc, but also the medial wall of the rostral caudate nucleus and the ventral, rostral putamen (Figures 8A-8C). Taken together, the VS occupies over 20% of the striatum in NHPs and about the same in humans (Haber et al., 2006; Tziortzi et al., 2014). Importantly, however, as with the NAcc, neither cytoarchitectonic nor histochemistry distinguishes clear boundaries between the VS and the dorsal striatum. This poses a problem for defining precise locations of activation in imaging studies. Nonetheless, broad boundaries are possible based on connectivity, particularly rostral to the anterior commissure. In the discussion below, we still refer to the NAcc, as it remains an important landmark for territories within the larger VS. Embedded within the NAcc is a small ventromedial sector, the shell (NAccS).
Figure 8. ACC.
(A) Schematic illustrating the ventral striatal connections. Fuchsia, vmPFC; red, inputs from OCF; orange, inputs from dACC; yellow, inputs from dPFC; green, inputs from premotor areas. White arrows, other inputs; gray arrows, outputs.
(B) 3D rendering demonstrating convergence of cortical projections from different reward-related regions. Note the interwoven pattern between several projection fields. Fuchsia, vmPFC; red, inputs from OCF; orange, inputs from dACC; yellow, inputs from dPFC.
(C) Dorsal prefrontal inputs superimposed on those from the vmPFC, OFC, and dACC. Note that the shell is located just caudal to the dACC input, as indicated by the arrow. Amy, amygdala; BNST, bed n. stria terminalis; dACC, dorsal anterior cingulate cortex; dPFC, dorsal lateral prefrontal cortex; Hipp, hippocampus; hypo, hypothalamus; NB, nucleus basalis; OFC, orbital frontal cortex; SNc, substantia nigra, pars compacta; VP, ventral pallidum; VS, ventral striatum; VTA, ventral tegmental area; vmPFC, ventral medial prefrontal cortex.
(D–F) Striatum. (D) Tractography reveals gradients of corticostriatal connectivity in human consistent with known NHP topographies. Left: the color of the striatal voxels reveals the strongest connection among the target regions considered (center). Importantly, the in vivo nature of tractography allows investigations of the functional consequences of connections (right). For example, the strength of striatal connections to hippocampus and amygdala correlated across subjects with novelty seeking scores on a personality-trait questionnaire (Cohen et al., 2009). (E and F) Ventral striatum BOLD reward prediction error signal depends on expected value and nigrostriatal tract connectivity strength (Chowdhury et al., 2013). (E) Schematic of the BOLD reward prediction error (RPE) effect size in the nucleus accumbens broken down into the components of the response related to reward and expected value. In older adults, the negative effect of expected value is diminished, leading to a smaller overall RPE signal. When given L-DOPA, the RPE signal in older adults is restored to that seen in healthy young adults, driven by the restoration of an adequate negative effect of expected value. (F) Older individuals with stronger nigrostriatal tract connectivity (measured with DTI) were shown to have a more negative effect of expected value on BOLD RPE signal in the nucleus accumbens.
Overall, the VS is similar to the dorsal striatum in its general connections and transmitter systems. Afferent projections to the VS, like those to the dorsal striatum, are derived from the same three major sources, a massive, generally topographic glutamatergic input from cerebral cortex; a large glutamatergic input from the thalamus; and a smaller but critical input from the brainstem, primarily from the midbrain DAergic cells. However, there are unique features to the VS. It alone also receives a dense projection from the amygdala and limited projection from the hippocampus (Fudge et al., 2002; Russchen et al., 1985). While serotonin, like DA, is distributed throughout the striatum, unlike DA, its terminals are densest in the VS. The NAccS subterritory has additional unique connections (indicated below) and transmitter and receptor distribution patterns that distinguish it from the rest of the VS (Meredith et al., 1996). The NAccS is in a particularly important position in the circuitry that underlies goal-directed behaviors, behavioral sensitization, and changes in affective states as demonstrated in rodent studies (Carlezon and Wise, 1996; Ito et al., 2004).
Projections to the striatum are organized in a general functional topographic manner (Figure 8A). The caudal vmPFC-striatal projections are concentrated within the NAcc, including the NAccS. This region also receives the densest input from Ia, the hippocampus, and amygdala (Chikama et al., 1997; Fudge et al., 2002; Russchen et al., 1985). The NAccS is also set apart from the rest of the VS by connections from the central nucleus (CeM), the medial nuclei of the amygdala, and the most limited thalamic input, primarily from the midline nuclei and the medial parafascicular n. Importantly, the DA input to the NAccS is also most confined to those from the VTA. Finally, the histochemical signature of the NAccS is also somewhat unique, in that this area is calbindin negative but has a high concentration of serotonin terminals. Connections from amygdala, vmPFC, VTA, and midline thalamic inputs also extend outside the NAccS, laterally into the NAcc and dorsally along the medial wall of the caudate adjacent to the ventricle. However, it is the basal nucleus and the magnocellular division of the accessory basal nucleus of the amygdala that provides the amygdala input (Fudge et al., 2002; Russchen et al., 1985). Neither the central amygdala nor the hippocampus projections extend here. Thus, the NAcc receives convergent input primarily from the olfactory and visceral-associated insula, from the vmPFC and amygdala. The NAccS receives the most restricted inputs, including afferent projections from the hippocampus and central amygdala.
The OFC-striatal terminals are positioned somewhat laterally to the vmPFC, extending into the rostral, ventral putamen and also along the medial part of the caudate. Topography is preserved such that more medial OFC inputs terminate medial to those derived from more lateral areas and rostral areas terminate rostral to caudal regions. However, these inputs overlap extensively with each other and also with those from the basal amygdala, but not from the hippocampus. This striatal area also receives axons from insula (Id) that conveys somatosensory information and is associated with feeding behaviors. Thus, the VS receives dual sensory inputs from both the OFC and insula. That is, insula (Ia and Id) project directly to the NAccS and laterally into the VS, respectively. Thalamic inputs are derived not only from midline and intralaminar nuclei, but also from the cor-tico-BG output nuclei, the ventral anterior nucleus, and medial dorsal thalamic nucleus (MD) (Giménez-Amaya et al., 1995). Finally, both VTA and medial SN, pars compacta DA cells project to this striatal region.
The dACC-striatal terminals terminate somewhat lateral to those from the OFC, outside of the NAcc. This territory does not receive a strong amygdala input, although some amygdala terminals extend into this area. In contrast to projections from the OFC and vmPFC, the dACC projections terminate primarily in the head of the caudate, extending ventrally into the rostral and central part of the putamen. The thalamic inputs to this region are primarily from the parafascicular nucleus and the MD, with few from the midline nuclei. Taken together, the vmPFC, OFC, and dACC terminals are concentrated in the rostral striatum, with the vmPFC projecting most medially (to the NAcc), and the dACC most laterally, and the OFC positioned between. In contrast, the dPFC projects throughout the rostrocaudal extent of the striatum. These terminals are generally located more laterally in the head of the caudate and in part of the rostral putamen (Haber et al., 2006; Selemon and Goldman-Rakic, 1985).
As indicated above, frontostriatal projections are generally topographically organized, which has been the basis for the concept of parallel processing of information through cortico-BG pathways (Alexander et al., 1990). However, there is a great deal of overlap between projections from different cortical areas. In particular, projections from the OFC, vmPFC, and dACC converge extensively, primarily at rostral levels. Moreover, the dACC and OFC also converge with inputs from dPFC regions (Figures 8B and 8C) (Haber et al., 2006). While neighboring cortical areas show that 80% of their terminal fields overlap, importantly, even cortical regions separated by 3 cm overlap by about 15%. While one might expect that terminals from adjacent cortical regions would overlap within the striatum, the idea that terminals from distant cortical regions would also overlap is more surprising. Hence, cortical connections from distant regions, and not necessarily functionally similar areas, converge within the striatum (Averbeck et al., 2014). These overlaps, referred to as critical nodes, are likely to be important for integrating information across diverse functional domains. Overall, the pattern of corticostriatal connectivity shows a similar overall organization in the human brain, providing a strong correlation between NHP anatomical tracing studies and human dMRI studies. Moreover, of particular importance, convergence between projections from different functional regions has also been demonstrated in the human striatum (Draganski et al., 2008). Taken together, a coordinated activation of different PFC inputs to the striatum that terminate in specific subregions could produce a unique combination to channel reward-based incentive drive in selecting between different valued options.
Functional Considerations
While functional differences are not routinely observed within the NAcc in human imaging studies, broad connectional differences elucidated in the NHP above appear conserved, even between putative shell and core regions of the NAcc. For example, amygdalar and hippocampal projections to NAcc appear to lie medial to OFC connections to the same structure when studied with dMRI tractography (Baliki et al., 2013). While such fine-grained distinctions are close to the limit of the current resolution for human imaging studies, the ability to measure connections in vivo and alongside function provides the ability to ask questions about the functional consequences of these different projections. For example, the relative size of the cortical input to NAcc compared against the size of hippocampal/amygdala NAcc projections predicts subjects’ relative propensity for reward-seeking versus novelty-seeking behavior (Cohen et al., 2009) (Figure 8D). The VS BOLD signal is regularly found to code for a reward prediction error (O’Doherty et al., 2004) (Rutledge et al., 2010): the reward minus the expectation of that reward. It has been suggested that this signal depends on DAergic input, as DA cells are known for a similar pattern of reward coding (Schultz, 2002), but other VS-projecting regions discussed above also code for reward. It is therefore likely that the VS signal contains several combined reward representations. Such an idea is supported by data in which connectional and functional data were acquired in the same subjects. The extent to which the VS signal looks like a reward prediction error, rather than a simple coding of reward, depends on the strength of connection (measured by dMRI tractography) between VS and the DAergic midbrain (Chowdhury et al., 2013). Notably, the relative representation of the prediction error can also be increased pharmacologically, by delivery of L-DOPA. Hence, the VS signal appears more similar to the DAergic cellular response both in subjects treated with a DA agonist, and in subjects who have a larger DA-VS projection (Figures 8E and 8F).
Overall, the striatum is now considered to play a central role in the learning process and the development of appropriate goal directed behaviors. This entails complex interactions between determining value versus risk and predictability based on previous experience. These calculations rely on integration between different aspects of reward processing and cognition to develop and execute appropriate action plans. While the concept of parallel, and segregated loops has dominated the field, the recent anatomic evidence reviewed above that demonstrates convergence of terminals from functionally diverse cortical areas, is consistent with the idea that cross talk between circuits during learning and adaptation is critical. Indeed, reward-responsiveness is not restricted to the VS, but found throughout the striatum. Likewise cells responding in working memory tasks are often found also in the VS, in addition to those in the dorsal striatum, (Apicella et al., 1991; Cromwell and Schultz, 2003; Delgado et al., 2005; Hassani et al., 2001; Levy et al., 1997; Takikawa et al., 2002; Tanaka et al., 2004; Watanabe et al., 2003). The critical nodes, described above, may be particularly sensitive to synchronizing information across functional areas to impact on long-term strategic planning and habit formation (Kasanetz et al., 2008; Pasupathy and Miller, 2005; Porrino et al., 2004). Functional imaging studies have previously not focused attention on these critical nodes, but rather segment the striatum along the conventional boundaries, limbic, associative, and motor regions. Nonetheless, the fact that these areas exist may help explain complex activation patterns following different reward-related paradigms throughout the striatum and help guide new ways to segment the striatum.
VS Route Back to Cortex via the BG
The VS, like the dorsal striatum, projects primarily to the pallidum and midbrain (Haber et al., 1990) (see Figure 1A). Based on its topographic input from the VS, the VP encompasses not only the subcommissural regions, but also the rostral pole of the external segment and the medial rostral internal segment of the globus pallidus. The more central and caudal portions of the globus pallidus do not receive this input. Fibers from the VS projecting to the midbrain are not as topographically organized. Rather, while the densest terminals are in the VTA and medial SN, they also extend laterally to innervate a large, dorsal subcomponent of the midbrain DAergic neurons (see the next section for a more detailed discussion on the SN). The VS also projects in non-BG regions, including the pedunculopontine nucleus, lateral hypothalamus (from the NAccS), periaqueductal gray, and the bed nucleus of the stria terminalis (Haber et al., 1990). Finally, axons from ventral regions of the VS terminate in the nucleus basalis, the main source of cholinergic fibers to the cerebral cortex and the amygdala (Haber, 1987; Záborszky and Cullinan, 1992), indicating VS route to the cortex and amygdala without going through the pallidal and thalamic circuit.
The VP, like the dorsal pallidum has indirect (via the medial STN and adjacent hypothalamus) and direct projections to the MD thalamus, thus completing the loop back to cortex (Haber et al., 1993; Ray and Price, 1993). The STN connection with the VP, along with a hyperdirect pathway from the vmPFC, OFC, and ACC to the medial STN, highlights the role of the STN in the reward pathway (Haber et al., 1993; Haynes and Haber, 2013). Finally the pallidal cells, ranging from the VP and extending dorsally and caudally, along the hypothalamus and medial dorsal pallidum send direct input to the LHb (Haber et al., 1993; Parent et al., 1981) and to the striatum (Spooren et al., 1996). The LHb, in turn, projects indirectly to the DA cells. Thus, the VP, STN, and LHb are important components of the reward circuit.
VP cells respond specifically during the learning and performance of reward-incentive behaviors. The complexity of the VP circuitry coupled with its central position in the reward circuit indicates that this structure is likely to be activated during imaging studies. Neuroimaging studies that document ventral striatal activation often document overlapping ventral pallidal activation. However, the lack of sufficient spatial resolution makes it difficult to distinguish the VP from the VS. LHb cells are inhibited by a reward-predicting stimulus but fire following a nonreward signal. This stimulation of the LHb directly or by following a nonreward signal inhibits DA cells (Ji and Shepard, 2007; Matsumoto and Hikosaka, 2007). Interestingly, few fibers from the LHb directly reach the SNc in the primates; rather, these cells project primarily to the RMTg, which, in turns projects to DA cells. An event-related fMRI study featuring adequate spatial and temporal resolution to visualize habenular activity indicated that negative, but not positive, feedback activates the habenular complex, consistent with findings from NHP electrophysiology (Ullsperger and von Cramon, 2003).
Midbrain DA Neurons
Arguably the most studied structure in the reward circuit is the midbrain system (Schultz, 2002; Wise, 2002). While traditionally DA cells have been divided into the mesolimbic (VTA), mesocortical, and nigrostriatal pathways, there are no clear boundaries between these cells groups, and much of the entire system has been associated with reward (Schultz, 2002). The main inputs to DA cells are GABAergic inputs from the BG (primarily from the striatum but also the external segment of the globus pallidus and VP), those from the brainstem and glutamatergic and cholinergic input from the pedunculopontine nucleus, and a serotonergic innervation from the dorsal raphe nucleus (Lavoie and Parent, 1994; Mori et al., 1987). In addition, there are limited projections from PFC, superior colliculus, and extended amygdala nucleus primarily to the VTA and dorsal DA cells (Frankle et al., 2006; Fudge and Haber, 2001; May et al., 2009).
The main outputs from the midbrain DA neurons are to the striatum and cortex (Lynd-Balta and Haber, 1994; Williams and Goldman-Rakic, 1998). Importantly, these two projection systems are quite different. Those to the striatum are derived from the entire midbrain DA system and have a general medial to lateral and inverse dorsal to ventral topographic organization. Thus, dorsal and medial regions project to the VS, while ventral and lateral cells project to the dorsal, lateral striatum. In contrast, DA-cortical projections are primarily derived from the VTA and retrorubral area and are not topographically organized. That is, cells projecting to functionally diverse cortical regions are intermingled. Moreover, individual neurons often send collateral axons to different cortical regions. Interestingly, in primate cortex, while DA fibers are found in the deep layers in specific cortical areas, they are most prominent in superficial layers, including a prominent projection throughout layer I, placing these axons in a direct position to modulate overall cortical function (Goldman-Rakic et al., 1999; Lewis, 1992). Thus, the nigrocortical projection is a more diffuse system compared to the more topographically organized nigrostriatal system. Finally, projections from the VTA and retrorubral area project to the amygdala, hippocampus, hypothalamus, periaqueductal gray, and the bed nucleus of the stria terminalis.
The main forebrain influence on DA cells is from the striatum. In NHP, projections from the striatum to the midbrain, and from the midbrain to the striatum, each create a loose topographic organization in which there is a medial to lateral and an inverse dorsal to ventral topography. Thus, dorsal striatonigral inputs are concentrated in the ventral midbrain and the ventral striatal projects to the dorsal midbrain. Importantly, each functional region differs in their proportional projections. The VS receives a limited midbrain input but projects to a larger broad area. In contrast, the dorsolateral striatum receives its input from a relatively large ventral region but projects to a relatively limited part. In other words, the VS influences a wide range of DA neurons but is itself influenced by a relatively limited group of DA cells, and the dorsolateral striatum influences a more limited midbrain region but is affected by a relatively large midbrain region (Haber et al., 2000). Therefore, the VS terminates in an area of DA cells that, in turn project to more dorsal striatal regions, particularly those that receive input from associative cortical regions (dPFC). This part of the ventral tier is reciprocally connected to the central (or associative) striatum but also projects to a more ventral regions and, thus, is in a position to interface with cells projecting to the dorsolateral (or motor) striatum. Taken together, the interface between different striatal regions via the midbrain DA cells is organized in an ascending spiral interconnecting different functional regions of the striatum and creating a feed-forward organization, from reward-related regions of the striatum, to cognitive and motor areas. Although the short latency burst-firing activity of DA that signals immediate reinforcement is likely to be triggered from brainstem nuclei, the cortico-striato-midbrain pathway is in the position to influence DA cells to distinguish reward and modify responses to incoming salient stimuli over time. This pathway is further reinforced via the nigrostriatal pathway, placing the striato-nigro-striatal pathway in a pivotal position for transferring information from the VS to the dorsal striatum during learning and habit formation (Figure 9A). One can hypothesize that the coordinated signal sent by cells in the critical nodes (in which there is convergence between the reward and associative circuits) to dopamine cells is further reinforced through the burst firing activity of the nigrostriatal pathway. This would not only impact those nodes, but also affect more dorsal striatal areas. Thus, through the striato-nigro-striatal system, information can be linked and transferred to other functional regions, during learning and habit formation (Belin and Everitt, 2008; Everitt and Robbins, 2005; Porrino et al., 2007; Volkow et al., 2006).
Figure 9. The Midbrain Dopamine System in the Reward Circuit.
(A) Schematic illustrating the complex connections between the striatum and substantia nigra. The arrows indicate how the ventral striatum can influence the dorsal striatum through the midbrain dopamine cells. Colors indicate functional regions of the striatum, based on cortical inputs. Midbrain projections from the shell target both the VTA and ventromedial SNc. Projections from the VTA to the shell form a “closed,” reciprocal loop but also project more laterally to impact on dopamine cells projecting to the rest of the ventral striatum, forming the first part of a feedforward loop (or spiral). The spiral continues through the striatonigro-striatal projections through which the ventral striatum impacts on cognitive and motor striatal areas via the midbrain dopamine cells. Red, inputs from the vmPFC; orange, inputs from the OFC and dACC; yellow, inputs from the dPFC; green and blue, inputs from motor control areas.
(B and C) Axial and sagittal views of the VTA BOLD response encoding positive reward prediction errors (RPEs) to the delivery of primary reward. (B) The VTA reward response is modulated by expectation. (C) VTA BOLD reward prediction error variation with reward expectation (D’Ardenne et al., 2008).
While, for technical reasons, reward prediction error signals are most commonly observed in the VS in humans, it is possible to record similar signals in the human midbrain (Figures 9B and 9C). Similar to cell recordings in NHPs (Schultz, 2002), BOLD signal in human midbrain increases in response to reward-predicting cues (Adcock et al., 2006; Wittmann et al., 2005). At outcome time, VTA BOLD activity codes for reward, and this coding is suppressed when these reward are expected (Figure 8E) (D’Ardenne et al., 2008; Klein-Flügge et al., 2011). Furthermore, consistent with electrophysiology (Hollerman and Schultz, 1998), VTA BOLD responses are sensitive not only to the value of reward that is expected, but also to the time that the reward is expected to be delivered. Rewards that are unexpected in amount or in time cause increases in VTA BOLD response (Klein-Flügge et al., 2013).
The graded connectivity between the midbrain and the striatum can also be measured with dMRI in humans, such that the VTA and medial SN project to the VS, and the lateral SN projects to dorsolateral striatum (Chowdhury et al., 2013). The interaction between the ventral and dorsal striatum via the DA cells (as described above) helps explains anatomical underpinnings of how reward learning results in reward-related actions. Thus, anticipatory BOLD responses in both lateral SN and dorsolateral striatum are tied to the intended action for the reward. Stimuli that should be approached to gain reward cause greater activity than those for which the rewarding action is to retreat (Guitart-Masip et al., 2011, 2012). Such interactions between intended actions and outcomes may provide a neural basis for Pavlovian influences on behavior (Dayan et al., 2006).
IV. Disruptions in Circuits Associated with Disease
Links between the Incentive Learning Circuits in Disease
There are several outstanding reviews that focus on the relationship between diseases and circuitry (Del Casale et al., 2011; Everitt et al., 2008; Graybiel and Rauch, 2000; Koob and Volkow, 2010; Price and Drevets, 2010). The dACC/OFC/vmPFC-BG system is most frequently associated with specific psychiatric disorders, specifically MDD, OCD, and addiction. Other psychiatric illnesses, such as schizophrenia and autism, also involve these areas; however, they appear to be less specific to this system. The literature on MDD, OCD, and addiction is massive but points to the vmPFC, OFC, dACC, striatum, and amygdala as key brain areas that repeatedly demonstrate differences between healthy control subjects and patients. The midbrain DA system, while central to incentive learning, is less frequently studied due to its size and the limitations of imaging. While each disease highlights a different combination of these structures, taken together, this complex incentive processing circuit appears to be central to them all, with additional components of parts of the dPFC and caudate nucleus. An important issue for imaging studies is that investigators often define the anatomical boundaries somewhat differently. In addition, given the resolution of imaging, the precision of activated areas or changes in connectivity in patient populations can be difficult to link to precise anatomical areas. For example, the medial or lateral parts of OFC may be grouped with the vmPFC or vlPFC, respectively. In studies with an emphasis on the OFC, changes may be reported for OFC that include parts of the sACC or vlPFC. The vmPFC may refer to caudal regions that include areas 25 and 32, or rostral regions, that involve more of area 10. Likewise, BG boundaries are often difficult to segment in imaging studies. For example, reported activation of the VS striatum may be actually centered more rostrally in the caudate nucleus or caudally in the globus pallidus. Despite these issues, the collective studies of different diseases consistently demonstrate patterns of activation and changes that involve various parts of the incentive-processing circuit. Additionally, the dMRI literature also highlights the OFC, dACC, and vmPFC by demonstrating differences in fractional anisotropy (FA) values and diffusivity between healthy control subjects and patients populations in the main cortical and subcortical WM bundles that connect these regions. Thus, the literature highlights differences in the ventral and dorsal “limbic” WM bundles, the UF and CB, and the main cortical-subcortical pathway, the internal capsule. Importantly, these WM bundles are targets for invasive surgeries for treatment of MDD and OCD, including deep brain stimulation (DBS).
Addiction and OCD
Consistent with the role of DA in reward and reinforcement learning, the initial stages for the reinforcing drug effects act primarily on the VTA-VS pathway. This initial stage of drug seeking is a goal-directed behavior, in that devaluation of the goal can be mediated by changes in the contingency between the behavior and its outcome. This stage is also associated with impulsivity, the tendency to act without foresight or cognitive control, often despite the awareness of negative consequences. In this respect, top-down, corticostriatal cognitive control circuits are thought to be involved. Recall that the vmPFC area plays a key role not only in valuation of stimuli, but in selecting between values, with lesions resulting in irrational choices. The vmPFC projects primarily to the shell part of VS. Over time, there is a shift from goal-directed learning to habit formation and compulsive drug taking, which persists despite negative consequences. It is at this stage that the rest of the VS and the dorsal striatum are recruited. The transition between ventral and dorsal striatum (initial drug seeking to a habit) occurs, in part, through the striato-nigro-striatal spiral described above (Belin and Everitt, 2008; Haber et al., 2000). Compulsive drug use is thought to involve the loss of cognitive control over incentive-based learned habits. This occurs, in part, through a decrease in prefrontal (dACC and dlPFC) inhibitory control over the dorsal striatum. While most studies focus on changes in the corticostriatal interface during the development of addictions, alterations in corticocortical connections also play a role. Thus, not only have changes in functional connectivity been shown between different PFC regions and striatum through the stages of addiction, but also between PFC areas (Goldstein and Volkow, 2002).
Interestingly, there are several features that addiction and OCD have in common. In both diseases, external cues are tightly linked to a particular action, such that the urge to carry out that action is difficult to suppress despite cognitive awareness of the consequences. These cues create a state of anxiety until the behaviors (often referred to as habits) are completed. Thus, compulsions, defined as an irresistible urge to act, to prevent or reduce anxiety or stress or dreaded event, despite a negative outcome, are common to both addiction and OCD (Leckman et al., 1997; Volkow and Fowler, 2000).
Abnormalities in the OFC are found in both diseases, including activation and volume differences between healthy control subjects and patients (Togao et al., 2010). Overall, the OFC appears to be hyperactive in OCD at rest, in symptom provocation, and during some tasks (Rauch et al., 1994). Activation of the OFC is also linked to craving in addiction; craving increases activation, while controlling craving decreases activation. As described in above, the OFC is particularly involved in devaluation of stimuli and stimulus-outcome reversal learning. Indeed, in both OCD and addiction, compulsive or addictive behaviors persist even though the probability of aversive outcomes is evident. In other words, patients continue to respond despite devaluation of these stimuli-outcome associations. Since a key distinction between a goal-directed behavior and a habit is reinforcement devaluation (Belin et al., 2009), the inability to devalue previous cue-induced responses related to disease may be a key factor in patients’ ability to modify behaviors appropriately (Gillan et al., 2014; Schoenbaum et al., 2006). Imaging studies also consistently show differences in the striatum between normal control subjects and patients, including altered connectivity between the OFC and VS, suggesting possible underlying pathology in the cortico-BG circuit. As indicated previously, the dACC plays a key role in selecting and switching behaviors when appropriate. It is in a transitional position between the reward and action networks, thus critical for selecting between possible choices of action. Thus, it is not surprising that changes in activation are also associated with both addiction and OCD, given that in both diseases, poor choices of action are chosen despite known negative outcomes (Fitzgerald et al., 2005; Schlösser et al., 2010; Volkow et al., 2010).
MDD
As mentioned above, there is a large imaging literature that examines task, nontask, structural, and WM-related differences between healthy control subjects and patients with MDD. Each of these types of studies reveals different aspects of the disease and, thus, various structures that show differences between healthy controls and patients. Nonetheless, the cingulate cortex (both dorsal and subgenual) and the amygdala are most often associated with MDD. Overall, the sACC activation is related to transient sadness (Mayberg et al., 1999), while the amygdala is generally overactive during negative emotion regulation (Groenewold et al., 2013; Siegle et al., 2002; Surguladze et al., 2005). These two regions, which are tightly linked anatomically, are coactivated during emotional processing in healthy control subjects. However, this regulation is disrupted in MDD, with a tighter link between the amygdala and sACC, and decreased coactivation of the amygdala with the dACC (Mathews et al., 2008), thus suggesting the role of the dACC as a regulator of emotion. The importance of cognitive control (or lack of) in MDD is emphasized by changes in activation of the dlPFC and parts of the dACC in MDD. For example, the dlPFC in MDD patients fail to show normal activation during emotional regulation (Erk et al., 2010; Hamilton et al., 2012). Other parts of the circuit, the OFC, and parts of the dACC, are equally linked to deficits in MDD. However, they show a variety of changes and are more difficult to put in a broader context. As mentioned above, while often studies refer to the sACC, it is often difficult to determine whether this is a distinction from the broader vmPFC area that includes not only the sACC but parts of the medial OFC and area 10. Finally, while the striatum is not as often linked to MDD in imaging studies, a striking exception is that MDD patients show reduced reward-related activation of the striatum (Pizzagalli et al., 2009). This is consistent with the role of the striatum in reward and its tight connections to the vmPFC, OFC, and amygdala.
Taken together, the incentive-based cortico-BG network that involves primarily the vmPFC, OFC, and dACC and their connections to the striatum shows the most disruption in diseases related to transitions between motivation and action. Importantly, as one might expect, the CB, UF, and parts of the internal capsule and corpus callosum, the major WM bundles that axons from these areas travel in, all show changes in FA values and diffusivity compared to healthy individuals in the diseases described above (Keedwell et al., 2012; Lochner et al., 2012; Romero et al., 2010). Furthermore, the main invasive surgical targets (DBS and lesions) for highly treatment-resistant MDD and OCD are located within these pathways: DBS in sACC, which includes the subgenual CB and parts of the UF (Mayberg et al., 2005), DBS or capsulotomy in the anterior limb of the IC, and cingulotomy in the dorsal CB (Greenberg et al., 2010; Shields et al., 2008). Moreover, experimental approaches for addiction target the VS and the internal capsule fibers located in this area.
Conclusion
The incentive-based learning circuit is a complex network comprised of several cortical and subcortical regions. We have outlined the circuitry underlying the intertwined and highly connected networks that provide the substrate for these integrations. In cortex, orbital networks link stimuli with outcomes; connections to ventral medial regions provide critical motivational input; and cingulate and dorsal prefrontal connections integrate such value signals with action representations. Within these regions, caudorostral gradients of connectivity mediate functional gradients from primary associations to higher cognitive function. The striatum, once thought to be organized in separate, segregated loops, also contains regions with converging inputs from multiple PFC areas. However, here, convergent cortical information differs from corticocortical interfaces, as this network exploits the basal ganglia for additional processing mediated directly by dopamine. Linking anatomical and physiological experiments in animals with human imaging studies is a powerful way to gain insight into the brain regions and mechanisms that control our decisions. By integrating information across species, we can build theories that can be tested in clinical populations, to form the foundation for understanding the neurocircuitry pertinent to psychiatric diseases.
Acknowledgments
We would like to acknowledge funding from the NIH (MH086400, MH73111, and MH086404) and Wellcome Trust WT088313AIa. We would like to thank the James S. McDonnell foundation and the Brain and Behavior Research Foundation (BBRF).
References
- Adcock RA, Thangavel A, Whitfield-Gabrieli S, Knutson B, Gabrieli JDE. Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron. 2006;50:507–517. doi: 10.1016/j.neuron.2006.03.036. [DOI] [PubMed] [Google Scholar]
- Alexander WH, Brown JW. Medial prefrontal cortex as an action-outcome predictor. Nat Neurosci. 2011;14:1338–1344. doi: 10.1038/nn.2921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Alexander GE, Crutcher MD, DeLong MR. Basal ganglia-thalamocortical circuits: parallel substrates for motor, oculomotor, “prefrontal” and “limbic” functions. Prog Brain Res. 1990;85:119–146. [PubMed] [Google Scholar]
- Apicella P, Scarnati E, Schultz W. Tonically discharging neurons of monkey striatum respond to preparatory and rewarding stimuli. Exp Brain Res. 1991;84:672–675. doi: 10.1007/BF00230981. [DOI] [PubMed] [Google Scholar]
- Averbeck BB, Lehman J, Jacobson M, Haber SN. Estimates of projection overlap and zones of convergence within Frontal-Striatal Circuits. J Neurosci. 2014;34:9497–9505. doi: 10.1523/JNEUROSCI.5806-12.2014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Baliki MN, Mansour A, Baria AT, Huang L, Berger SE, Fields HL, Apkarian AV. Parceling human accumbens into putative core and shell dissociates encoding of values for reward and pain. J Neurosci. 2013;33:16383–16393. doi: 10.1523/JNEUROSCI.1731-13.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barbas H. Specialized elements of orbitofrontal cortex in primates. Ann N Y Acad Sci. 2007;1121:10–32. doi: 10.1196/annals.1401.015. [DOI] [PubMed] [Google Scholar]
- Barron HC, Dolan RJ, Behrens TE. Online evaluation of novel choices by simultaneous representation of multiple memories. Nat Neurosci. 2013;16:1492–1498. doi: 10.1038/nn.3515. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bartra O, McGuire JT, Kable JW. The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage. 2013;76:412–427. doi: 10.1016/j.neuroimage.2013.02.063. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Beckmann M, Johansen-Berg H, Rushworth MF. Connectivity-based parcellation of human cingulate cortex and its relation to functional specialization. J Neurosci. 2009;29:1175–1190. doi: 10.1523/JNEUROSCI.3328-08.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Behrens TE, Woolrich MW, Walton ME, Rushworth MF. Learning the value of information in an uncertain world. Nat Neurosci. 2007;10:1214–1221. doi: 10.1038/nn1954. [DOI] [PubMed] [Google Scholar]
- Behrens TE, Hunt LT, Woolrich MW, Rushworth MF. Associative learning of social value. Nature. 2008;456:245–249. doi: 10.1038/nature07538. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Belin D, Everitt BJ. Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron. 2008;57:432–441. doi: 10.1016/j.neuron.2007.12.019. [DOI] [PubMed] [Google Scholar]
- Belin D, Jonkman S, Dickinson A, Robbins TW, Everitt BJ. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav Brain Res. 2009;199:89–102. doi: 10.1016/j.bbr.2008.09.027. [DOI] [PubMed] [Google Scholar]
- Bonnici HM, Chadwick MJ, Lutti A, Hassabis D, Weiskopf N, Maguire EA. Detecting representations of recent and remote autobiographical memories in vmPFC and hippocampus. J Neurosci. 2012;32:16982–16991. doi: 10.1523/JNEUROSCI.2475-12.2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boorman ED, Behrens TE, Woolrich MW, Rushworth MF. How green is the grass on the other side? Frontopolar cortex and the evidence in favor of alternative courses of action. Neuron. 2009;62:733–743. doi: 10.1016/j.neuron.2009.05.014. [DOI] [PubMed] [Google Scholar]
- Boorman ED, Rushworth MF, Behrens TE. Ventromedial pre-frontal and anterior cingulate cortex adopt choice and default reference frames during sequential multi-alternative choice. J Neurosci. 2013;33:2242–2253. doi: 10.1523/JNEUROSCI.3022-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouret S, Richmond BJ. Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. J Neurosci. 2010;30:8591–8601. doi: 10.1523/JNEUROSCI.0049-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burke KA, Franz TM, Miller DN, Schoenbaum G. The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature. 2008;454:340–344. doi: 10.1038/nature06993. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Camille N, Griffiths CA, Vo K, Fellows LK, Kable JW. Ventromedial frontal lobe damage disrupts value maximization in humans. J Neurosci. 2011a;31:7527–7532. doi: 10.1523/JNEUROSCI.6527-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Camille N, Tsuchida A, Fellows LK. Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J Neurosci. 2011b;31:15048–15052. doi: 10.1523/JNEUROSCI.3164-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carlezon WA, Jr, Wise RA. Rewarding actions of phencyclidine and related drugs in nucleus accumbens shell and frontal cortex. J Neurosci. 1996;16:3112–3122. doi: 10.1523/JNEUROSCI.16-09-03112.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Carmichael ST, Price JL. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J Comp Neurol. 1995a;363:615–641. doi: 10.1002/cne.903630408. [DOI] [PubMed] [Google Scholar]
- Carmichael ST, Price JL. Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys. J Comp Neurol. 1995b;363:642–664. doi: 10.1002/cne.903630409. [DOI] [PubMed] [Google Scholar]
- Chiavaras MM, Petrides M. Orbitofrontal sulci of the human and macaque monkey brain. J Comp Neurol. 2000;422:35–54. [PubMed] [Google Scholar]
- Chikama M, McFarland NR, Amaral DG, Haber SN. Insular cortical projections to functional regions of the striatum correlate with cortical cytoarchitectonic organization in the primate. J Neurosci. 1997;17:9686–9705. doi: 10.1523/JNEUROSCI.17-24-09686.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chowdhury R, Guitart-Masip M, Lambert C, Dayan P, Huys Q, Düzel E, Dolan RJ. Dopamine restores reward prediction errors in old age. Nat Neurosci. 2013;16:648–653. doi: 10.1038/nn.3364. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cisek P. Cortical mechanisms of action selection: the affordance competition hypothesis. Philos Trans R Soc Lond B Biol Sci. 2007;362:1585–1599. doi: 10.1098/rstb.2007.2054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cisek P. Making decisions through a distributed consensus. Curr Opin Neurobiol. 2012;22:927–936. doi: 10.1016/j.conb.2012.05.007. [DOI] [PubMed] [Google Scholar]
- Clithero JA, Rangel A. Informatic parcellation of the network involved in the computation of subjective value. Soc Cogn Affect Neurosci. 2013 doi: 10.1093/scan/nst106. Published online August 21, 2013. http://dx.doi.org/10.1093/scan/nst106. [DOI] [PMC free article] [PubMed]
- Cohen JD, McClure SM, Yu AJ. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philos Trans R Soc Lond B Biol Sci. 2007;362:933–942. doi: 10.1098/rstb.2007.2098. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cohen MX, Schoene-Bake JC, Elger CE, Weber B. Connectivity-based segregation of the human striatum predicts personality characteristics. Nat Neurosci. 2009;12:32–34. doi: 10.1038/nn.2228. [DOI] [PubMed] [Google Scholar]
- Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol. 2003;89:2823–2838. doi: 10.1152/jn.01014.2002. [DOI] [PubMed] [Google Scholar]
- D’Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science. 2008;319:1264–1267. doi: 10.1126/science.1150605. [DOI] [PubMed] [Google Scholar]
- Daw ND, O’Doherty JP, Dayan P, Seymour B, Dolan RJ. Cortical substrates for exploratory decisions in humans. Nature. 2006;441:876–879. doi: 10.1038/nature04766. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dayan P, Niv Y, Seymour B, Daw ND. The misbehavior of value and the discipline of the will. Neural Netw. 2006;19:1153–1160. doi: 10.1016/j.neunet.2006.03.002. [DOI] [PubMed] [Google Scholar]
- Del Casale A, Kotzalidis GD, Rapinesi C, Serata D, Ambrosi E, Simonetti A, Pompili M, Ferracuti S, Tatarelli R, Girardi P. Functional neuroimaging in obsessive-compulsive disorder. Neuropsychobiology. 2011;64:61–85. doi: 10.1159/000325223. [DOI] [PubMed] [Google Scholar]
- Delgado MR, Miller MM, Inati S, Phelps EA. An fMRI study of reward-related probability learning. Neuroimage. 2005;24:862–873. doi: 10.1016/j.neuroimage.2004.10.002. [DOI] [PubMed] [Google Scholar]
- Doeller CF, Barry C, Burgess N. Evidence for grid cells in a human memory network. Nature. 2010;463:657–661. doi: 10.1038/nature08704. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Draganski B, Kherif F, Klöppel S, Cook PA, Alexander DC, Parker GJ, Deichmann R, Ashburner J, Frackowiak RS. Evidence for segregated and integrative connectivity patterns in the human Basal Ganglia. J Neurosci. 2008;28:7143–7152. doi: 10.1523/JNEUROSCI.1486-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Erk S, Mikschl A, Stier S, Ciaramidaro A, Gapp V, Weber B, Walter H. Acute and sustained effects of cognitive emotion regulation in major depression. J Neurosci. 2010;30:15726–15734. doi: 10.1523/JNEUROSCI.1856-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Everitt BJ, Robbins TW. Neural systems of reinforcement for drug addiction: from actions to habits to compulsion. Nat Neurosci. 2005;8:1481–1489. doi: 10.1038/nn1579. [DOI] [PubMed] [Google Scholar]
- Everitt BJ, Belin D, Economidou D, Pelloux Y, Dalley JW, Robbins TW. Review. Neural mechanisms underlying the vulnerability to develop compulsive drug-seeking habits and addiction. Philos Trans R Soc Lond B Biol Sci. 2008;363:3125–3135. doi: 10.1098/rstb.2008.0089. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Feldman RP, Alterman RL, Goodrich JT. Contemporary psychosurgery and a look to the future. J Neurosurg. 2001;95:944–956. doi: 10.3171/jns.2001.95.6.0944. [DOI] [PubMed] [Google Scholar]
- Fellows LK, Farah MJ. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain. 2003;126:1830–1837. doi: 10.1093/brain/awg180. [DOI] [PubMed] [Google Scholar]
- Fitzgerald KD, Welsh RC, Gehring WJ, Abelson JL, Himle JA, Liberzon I, Taylor SF. Error-related hyperactivity of the anterior cingulate cortex in obsessive-compulsive disorder. Biol Psychiatry. 2005;57:287–294. doi: 10.1016/j.biopsych.2004.10.038. [DOI] [PubMed] [Google Scholar]
- Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci USA. 2005;102:9673–9678. doi: 10.1073/pnas.0504136102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frankle WG, Laruelle M, Haber SN. Prefrontal cortical projections to the midbrain in primates: evidence for a sparse connection. Neuropsychopharmacology. 2006;31:1627–1636. doi: 10.1038/sj.npp.1300990. [DOI] [PubMed] [Google Scholar]
- Freedman LJ, Insel TR, Smith Y. Subcortical projections of area 25 (subgenual cortex) of the macaque monkey. J Comp Neurol. 2000;421:172–188. [PubMed] [Google Scholar]
- Frith CD, Friston K, Liddle PF, Frackowiak RS. Willed action and the prefrontal cortex in man: a study with PET. Proc Biol Sci. 1991;244:241–246. doi: 10.1098/rspb.1991.0077. [DOI] [PubMed] [Google Scholar]
- Fudge JL, Haber SN. Bed nucleus of the stria terminalis and extended amygdala inputs to dopamine subpopulations in primates. Neuroscience. 2001;104:807–827. doi: 10.1016/s0306-4522(01)00112-9. [DOI] [PubMed] [Google Scholar]
- Fudge JL, Kunishio K, Walsh P, Richard C, Haber SN. Amygdaloid projections to ventromedial striatal subterritories in the primate. Neuroscience. 2002;110:257–275. doi: 10.1016/s0306-4522(01)00546-2. [DOI] [PubMed] [Google Scholar]
- Gillan CM, Morein-Zamir S, Urcelay GP, Sule A, Voon V, Apergis-Schoute AM, Fineberg NA, Sahakian BJ, Robbins TW. Enhanced avoidance habits in obsessive-compulsive disorder. Biol Psychiatry. 2014;75:631–638. doi: 10.1016/j.biopsych.2013.02.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Giménez-Amaya JM, McFarland NR, de las Heras S, Haber SN. Organization of thalamic projections to the ventral striatum in the primate. J Comp Neurol. 1995;354:127–149. doi: 10.1002/cne.903540109. [DOI] [PubMed] [Google Scholar]
- Goldman-Rakic PS, Bergson C, Krimer LS, Lidow MS, Williams SM, Williams GV. The primate mesocortical dopamine system. In: Bloom FE, Bjorklund A, Hokfelt T, editors. Handbook of Chemical Neuroanatomy. Chapter V. Amsterdam: Elsevier Science; 1999. pp. 403–428. [Google Scholar]
- Goldstein RZ, Volkow ND. Drug addiction and its underlying neurobiological basis: neuroimaging evidence for the involvement of the frontal cortex. Am J Psychiatry. 2002;159:1642–1652. doi: 10.1176/appi.ajp.159.10.1642. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gottfried JA, O’Doherty J, Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 2003;301:1104–1107. doi: 10.1126/science.1087919. [DOI] [PubMed] [Google Scholar]
- Graybiel AM, Rauch SL. Toward a neurobiology of obsessive-compulsive disorder. Neuron. 2000;28:343–347. doi: 10.1016/s0896-6273(00)00113-6. [DOI] [PubMed] [Google Scholar]
- Greenberg BD, Price LH, Rauch SL, Friehs G, Noren G, Malone D, Carpenter LL, Rezai AR, Rasmussen SA. Neurosurgery for intractable obsessive-compulsive disorder and depression: critical issues. Neurosurg Clin N Am. 2003;14:199–212. doi: 10.1016/s1042-3680(03)00005-6. [DOI] [PubMed] [Google Scholar]
- Greenberg BD, Rauch SL, Haber SN. Invasive circuitry-based neurotherapeutics: stereotactic ablation and deep brain stimulation for OCD. Neuropsychopharmacology. 2010;35:317–336. doi: 10.1038/npp.2009.128. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Groenewold NA, Opmeer EM, de Jonge P, Aleman A, Costafreda SG. Emotional valence modulates brain functional abnormalities in depression: evidence from a meta-analysis of fMRI studies. Neurosci Biobehav Rev. 2013;37:152–163. doi: 10.1016/j.neubiorev.2012.11.015. [DOI] [PubMed] [Google Scholar]
- Guitart-Masip M, Fuentemilla L, Bach DR, Huys QJ, Dayan P, Dolan RJ, Duzel E. Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J Neurosci. 2011;31:7867–7875. doi: 10.1523/JNEUROSCI.6376-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Guitart-Masip M, Chowdhury R, Sharot T, Dayan P, Duzel E, Dolan RJ. Action controls dopaminergic enhancement of reward representations. Proc Natl Acad Sci USA. 2012;109:7511–7516. doi: 10.1073/pnas.1202229109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haber SN. Anatomical relationship between the basal ganglia and the basal nucleus of Meynert in human and monkey forebrain. Proc Natl Acad Sci USA. 1987;84:1408–1412. doi: 10.1073/pnas.84.5.1408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haber SN, Lynd E, Klein C, Groenewegen HJ. Topographic organization of the ventral striatal efferent projections in the rhesus monkey: an anterograde tracing study. J Comp Neurol. 1990;293:282–298. doi: 10.1002/cne.902930210. [DOI] [PubMed] [Google Scholar]
- Haber SN, Lynd-Balta E, Mitchell SJ. The organization of the descending ventral pallidal projections in the monkey. J Comp Neurol. 1993;329:111–128. doi: 10.1002/cne.903290108. [DOI] [PubMed] [Google Scholar]
- Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci. 2000;20:2369–2382. doi: 10.1523/JNEUROSCI.20-06-02369.2000. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haber SN, Kim KS, Mailly P, Calzavara R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci. 2006;26:8368–8376. doi: 10.1523/JNEUROSCI.0271-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hamilton JP, Etkin A, Furman DJ, Lemus MG, Johnson RF, Gotlib IH. Functional neuroimaging of major depressive disorder: a meta-analysis and new integration of base line activation and neural response data. Am J Psychiatry. 2012;169:693–703. doi: 10.1176/appi.ajp.2012.11071105. [DOI] [PubMed] [Google Scholar]
- Hampton AN, Bossaerts P, O’Doherty JP. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci. 2006;26:8360–8367. doi: 10.1523/JNEUROSCI.1010-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hare TA, Camerer CF, Knoepfle DT, Rangel A. Value computations in ventral medial prefrontal cortex during charitable decision making incorporate input from regions involved in social cognition. J Neurosci. 2010;30:583–590. doi: 10.1523/JNEUROSCI.4089-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Harlow JM. Passage of an iron rod through the head. Boston Med Surg J. 1848;39:389–393. [PMC free article] [PubMed] [Google Scholar]
- Hassani OK, Cromwell HC, Schultz W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol. 2001;85:2477–2489. doi: 10.1152/jn.2001.85.6.2477. [DOI] [PubMed] [Google Scholar]
- Hayden BY, Heilbronner SR, Pearson JM, Platt ML. Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J Neurosci. 2011a;31:4178–4187. doi: 10.1523/JNEUROSCI.4652-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hayden BY, Pearson JM, Platt ML. Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci. 2011b;14:933–939. doi: 10.1038/nn.2856. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haynes WI, Haber SN. The organization of prefrontal-subthalamic inputs in primates provides an anatomical substrate for both functional specificity and integration: implications for Basal Ganglia models and deep brain stimulation. J Neurosci. 2013;33:4804–4814. doi: 10.1523/JNEUROSCI.4674-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heimer L. The olfactory cortex and the ventral striatum. In: Livingston KE, Hornykiewicz O, editors. Limbic Mechanisms. New York: Plenum Press; 1978. pp. 95–187. [Google Scholar]
- Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci. 1998;1:304–309. doi: 10.1038/1124. [DOI] [PubMed] [Google Scholar]
- Hunt LT, Kolling N, Soltani A, Woolrich MW, Rushworth MF, Behrens TE. Mechanisms underlying cortical activity during value-guided choice. Nat Neurosci. 2012;15:470–476. S1–S3. doi: 10.1038/nn.3017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hunt LT, Woolrich MW, Rushworth MF, Behrens TE. Trial-type dependent frames of reference for value comparison. PLoS Comput Biol. 2013;9:e1003225. doi: 10.1371/journal.pcbi.1003225. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ito R, Robbins TW, Everitt BJ. Differential control over cocaine-seeking behavior by nucleus accumbens core and shell. Nat Neurosci. 2004;7:389–397. doi: 10.1038/nn1217. [DOI] [PubMed] [Google Scholar]
- Iversen SD, Mishkin M. Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity. Exp Brain Res. 1970;11:376–386. doi: 10.1007/BF00237911. [DOI] [PubMed] [Google Scholar]
- Izquierdo A, Suda RK, Murray EA. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J Neurosci. 2004;24:7540–7548. doi: 10.1523/JNEUROSCI.1921-04.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Janowski V, Camerer C, Rangel A. Empathic choice involves vmPFC value signals that are modulated by social processing implemented in IPL. Soc Cogn Affect Neurosci. 2013;8:201–208. doi: 10.1093/scan/nsr086. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jbabdi S, Lehman JF, Haber SN, Behrens TE. Human and monkey ventral prefrontal fibers use the same organizational principles to reach their targets: tracing versus tractography. J Neurosci. 2013;33:3190–3201. doi: 10.1523/JNEUROSCI.2457-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ji H, Shepard PD. Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABA(A) receptor-mediated mechanism. J Neurosci. 2007;27:6923–6930. doi: 10.1523/JNEUROSCI.0958-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jocham G, Hunt LT, Near J, Behrens TE. A mechanism for value-guided choice based on the excitation-inhibition balance in prefrontal cortex. Nat Neurosci. 2012;15:960–961. doi: 10.1038/nn.3140. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kable JW, Glimcher PW. The neurobiology of decision: consensus and controversy. Neuron. 2009;63:733–745. doi: 10.1016/j.neuron.2009.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kasanetz F, Riquelme LA, Della-Maggiore V, O’Donnell P, Murer MG. Functional integration across a gradient of corticostriatal channels controls UP state transitions in the dorsal striatum. Proc Natl Acad Sci USA. 2008;105:8124–8129. doi: 10.1073/pnas.0711113105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Keedwell PA, Chapman R, Christiansen K, Richardson H, Evans J, Jones DK. Cingulum white matter in young women at risk of depression: the effect of family history and anhedonia. Biol Psychiatry. 2012;72:296–302. doi: 10.1016/j.biopsych.2012.01.022. [DOI] [PubMed] [Google Scholar]
- Kelley AE, Berridge KC. The neuroscience of natural rewards: relevance to addictive drugs. J Neurosci. 2002;22:3306–3311. doi: 10.1523/JNEUROSCI.22-09-03306.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kennerley SW, Wallis JD. Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus. J Neurophysiol. 2009;102:3352–3364. doi: 10.1152/jn.00273.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF. Optimal decision making and the anterior cingulate cortex. Nat Neurosci. 2006;9:940–947. doi: 10.1038/nn1724. [DOI] [PubMed] [Google Scholar]
- Kennerly D, Richter KM, Good V, Compton J, Ballard DJ. Journey to no preventable risk: the Baylor Health Care System patient safety experience. Am J Med Qual. 2011;26:43–52. doi: 10.1177/1062860610374645. [DOI] [PubMed] [Google Scholar]
- Klein-Flügge MC, Hunt LT, Bach DR, Dolan RJ, Behrens TE. Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron. 2011;72:654–664. doi: 10.1016/j.neuron.2011.08.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klein-Flügge MC, Barron HC, Brodersen KH, Dolan RJ, Behrens TE. Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J Neurosci. 2013;33:3202–3211. doi: 10.1523/JNEUROSCI.2532-12.2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kolling N, Behrens TE, Mars RB, Rushworth MF. Neural mechanisms of foraging. Science. 2012;336:95–98. doi: 10.1126/science.1216930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Koob GF, Volkow ND. Neurocircuitry of addiction. Neuropsychopharmacology. 2010;35:217–238. doi: 10.1038/npp.2009.110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kringelbach ML, Rolls ET. Neural correlates of rapid reversal learning in a simple model of human social interaction. Neuroimage. 2003;20:1371–1383. doi: 10.1016/S1053-8119(03)00393-8. [DOI] [PubMed] [Google Scholar]
- Kumaran D, Summerfield JJ, Hassabis D, Maguire EA. Tracking the emergence of conceptual knowledge during human decision making. Neuron. 2009;63:889–901. doi: 10.1016/j.neuron.2009.07.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lavoie B, Parent A. Pedunculopontine nucleus in the squirrel monkey: cholinergic and glutamatergic projections to the substantia nigra. J Comp Neurol. 1994;344:232–241. doi: 10.1002/cne.903440205. [DOI] [PubMed] [Google Scholar]
- Leckman JF, Grice DE, Boardman J, Zhang H, Vitale A, Bondi C, Alsobrook J, Peterson BS, Cohen DJ, Rasmussen SA, et al. Symptoms of obsessive-compulsive disorder. Am J Psychiatry. 1997;154:911–917. doi: 10.1176/ajp.154.7.911. [DOI] [PubMed] [Google Scholar]
- Lee SW, Shimojo S, O’Doherty JP. Neural computations underlying arbitration between model-based and model-free learning. Neuron. 2014;81:687–699. doi: 10.1016/j.neuron.2013.11.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lehman JF, Greenberg BD, McIntyre CC, Rasmussen SA, Haber SN. Rules ventral prefrontal cortical axons use to reach their targets: implications for diffusion tensor imaging tractography and deep brain stimulation for psychiatric illness. J Neurosci. 2011;31:10392–10402. doi: 10.1523/JNEUROSCI.0595-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Levy R, Friedman HR, Davachi L, Goldman-Rakic PS. Differential activation of the caudate nucleus in primates performing spatial and nonspatial working memory tasks. J Neurosci. 1997;17:3870–3882. doi: 10.1523/JNEUROSCI.17-10-03870.1997. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lewis DA. The catecholaminergic innervation of primate prefrontal cortex. J Neural Transm Suppl. 1992;36:179–200. doi: 10.1007/978-3-7091-9211-5_9. [DOI] [PubMed] [Google Scholar]
- Lochner C, Fouché JP, du Plessis S, Spottiswoode B, Seedat S, Fineberg N, Chamberlain SR, Stein DJ. Evidence for fractional anisotropy and mean diffusivity white matter abnormalities in the internal capsule and cingulum in patients with obsessive-compulsive disorder. J Psychiatry Neurosci. 2012;37:193–199. doi: 10.1503/jpn.110059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lynd-Balta E, Haber SN. The organization of midbrain projections to the striatum in the primate: sensorimotor-related striatum versus ventral striatum. Neuroscience. 1994;59:625–640. doi: 10.1016/0306-4522(94)90182-1. [DOI] [PubMed] [Google Scholar]
- Matsumoto M, Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature. 2007;447:1111–1115. doi: 10.1038/nature05860. [DOI] [PubMed] [Google Scholar]
- May P, McHaffie JG, Stanford TR, Jiang H, Costello MG, Coizet V, Hayes LM, Haber SN, Redgrave P. Tectonigral projections in the primate: a pathway for pre-attentive sensory input to midbrain dopaminergic neurons. Eur J Neurosci. 2009;29:575–587. doi: 10.1111/j.1460-9568.2008.06596.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mayberg HS, Liotti M, Brannan SK, McGinnis S, Mahurin RK, Jerabek PA, Silva JA, Tekell JL, Martin CC, Lancaster JL, Fox PT. Reciprocal limbic-cortical function and negative mood: converging PET findings in depression and normal sadness. Am J Psychiatry. 1999;156:675–682. doi: 10.1176/ajp.156.5.675. [DOI] [PubMed] [Google Scholar]
- Mayberg HS, Lozano AM, Voon V, McNeely HE, Seminowicz D, Hamani C, Schwalb JM, Kennedy SH. Deep brain stimulation for treatment-resistant depression. Neuron. 2005;45:651–660. doi: 10.1016/j.neuron.2005.02.014. [DOI] [PubMed] [Google Scholar]
- McEnaney KW, Butter CM. Perseveration of responding and nonresponding in monkeys with orbital frontal ablations. J Comp Physiol Psychol. 1969;68:558–561. doi: 10.1037/h0027639. [DOI] [PubMed] [Google Scholar]
- Meredith GE, Pattiselanno A, Groenewegen HJ, Haber SN. Shell and core in monkey and human nucleus accumbens identified with antibodies to calbindin-D28k. J Comp Neurol. 1996;365:628–639. doi: 10.1002/(SICI)1096-9861(19960219)365:4<628::AID-CNE9>3.0.CO;2-6. [DOI] [PubMed] [Google Scholar]
- Mesulam MM, Mufson EJ. The insula of reil in man and monkey. In: Peters A, Jones EG, editors. Cerebral Cortex. New York: Plenum Press; 1993. pp. 179–225. [Google Scholar]
- Mogenson GJ, Jones DL, Yim CY. From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol. 1980;14:69–97. doi: 10.1016/0301-0082(80)90018-0. [DOI] [PubMed] [Google Scholar]
- Morecraft RJ, Geula C, Mesulam M-M. Cytoarchitecture and neural afferents of orbitofrontal cortex in the brain of the monkey. J Comp Neurol. 1992;323:341–358. doi: 10.1002/cne.903230304. [DOI] [PubMed] [Google Scholar]
- Morecraft RJ, Stilwell-Morecraft KS, Cipolloni PB, Ge J, McNeal DW, Pandya DN. Cytoarchitecture and cortical connections of the anterior cingulate and adjacent somatomotor fields in the rhesus monkey. Brain Res Bull. 2012;87:457–497. doi: 10.1016/j.brainresbull.2011.12.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mori S, Matsuura T, Takino T, Sano Y. Light and electron microscopic immunohistochemical studies of serotonin nerve fibers in the substantia nigra of the rat, cat and monkey. Anat Embryol (Berl) 1987;176:13–18. doi: 10.1007/BF00309747. [DOI] [PubMed] [Google Scholar]
- Nicolle A, Klein-Flügge MC, Hunt LT, Vlaev I, Dolan RJ, Behrens TE. An agent independent axis for executed and modeled choice in medial prefrontal cortex. Neuron. 2012;75:1114–1121. doi: 10.1016/j.neuron.2012.07.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Noonan MP, Walton ME, Behrens TE, Sallet J, Buckley MJ, Rushworth MF. Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc Natl Acad Sci USA. 2010;107:20547–20552. doi: 10.1073/pnas.1012246107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- O’Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F, Kobal G, Renner B, Ahne G. Sensory-specific satiety-related olfactory activation of the human orbitofrontal cortex. Neuroreport. 2000;11:893–897. doi: 10.1097/00001756-200003200-00046. [DOI] [PubMed] [Google Scholar]
- O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 2004;304:452–454. doi: 10.1126/science.1094285. [DOI] [PubMed] [Google Scholar]
- Olds J, Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol. 1954;47:419–427. doi: 10.1037/h0058775. [DOI] [PubMed] [Google Scholar]
- Ongür D, Price JL. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb Cortex. 2000;10:206–219. doi: 10.1093/cercor/10.3.206. [DOI] [PubMed] [Google Scholar]
- Ongür D, An X, Price JL. Prefrontal cortical projections to the hypothalamus in macaque monkeys. J Comp Neurol. 1998;401:480–505. [PubMed] [Google Scholar]
- Padoa-Schioppa C, Assad JA. Neurons in the orbitofrontal cortex encode economic value. Nature. 2006;441:223–226. doi: 10.1038/nature04676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Padoa-Schioppa C, Assad JA. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat Neurosci. 2008;11:95–102. doi: 10.1038/nn2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Papez JW. A proposed mechnaism of emotion, 1937. J Neuropsychiatry Clin Neurosci. 1995;7:103–112. doi: 10.1176/jnp.7.1.103. [DOI] [PubMed] [Google Scholar]
- Parent A, Gravel S, Boucher R. The origin of forebrain afferents to the habenula in rat, cat and monkey. Brain Res Bull. 1981;6:23–38. doi: 10.1016/s0361-9230(81)80066-4. [DOI] [PubMed] [Google Scholar]
- Passingham RE, Bengtsson SL, Lau HC. Medial frontal cortex: from self-generated action to reflection on one’s own performance. Trends Cogn Sci. 2010;14:16–21. doi: 10.1016/j.tics.2009.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pasupathy A, Miller EK. Different time courses of learning-related activity in the prefrontal cortex and striatum. Nature. 2005;433:873–876. doi: 10.1038/nature03287. [DOI] [PubMed] [Google Scholar]
- Percheron G, Filion M. Parallel processing in the basal ganglia: up to a point. Trends Neurosci. 1991;14:55–59. doi: 10.1016/0166-2236(91)90020-u. [DOI] [PubMed] [Google Scholar]
- Petrides M, Pandya DN. Comparative architectonic analysis of the human and macaque frontal cortex. In: Boller F, Grafman J, editors. Handbook of Neuropsychology. Amsterdam: Elsevier; 1994. pp. 17–58. [Google Scholar]
- Phillips AG, Fibiger HC. The role of dopamine in maintaining intracranial self-stimulation in the ventral tegmentum, nucleus accumbens, and medial prefrontal cortex. Can J Psychol. 1978;32:58–66. doi: 10.1037/h0081676. [DOI] [PubMed] [Google Scholar]
- Pizzagalli DA, Holmes AJ, Dillon DG, Goetz EL, Birk JL, Bogdan R, Dougherty DD, Iosifescu DV, Rauch SL, Fava M. Reduced caudate and nucleus accumbens response to rewards in unmedicated individuals with major depressive disorder. Am J Psychiatry. 2009;166:702–710. doi: 10.1176/appi.ajp.2008.08081201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Porrino LJ, Lyons D, Smith HR, Daunais JB, Nader MA. Cocaine self-administration produces a progressive involvement of limbic, association, and sensorimotor striatal domains. J Neurosci. 2004;24:3554–3562. doi: 10.1523/JNEUROSCI.5578-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Porrino LJ, Smith HR, Nader MA, Beveridge TJ. The effects of cocaine: a shifting target over the course of addiction. Prog Neuropsychopharmacol Biol Psychiatry. 2007;31:1593–1600. doi: 10.1016/j.pnpbp.2007.08.040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Price JL, Drevets WC. Neurocircuitry of mood disorders. Neuropsychopharmacology. 2010;35:192–216. doi: 10.1038/npp.2009.104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Quilodran R, Rothé M, Procyk E. Behavioral shifts and action valuation in the anterior cingulate cortex. Neuron. 2008;57:314–325. doi: 10.1016/j.neuron.2007.11.031. [DOI] [PubMed] [Google Scholar]
- Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL. A default mode of brain function. Proc Natl Acad Sci USA. 2001;98:676–682. doi: 10.1073/pnas.98.2.676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rangel A, Hare T. Neural computations associated with goal-directed choice. Curr Opin Neurobiol. 2010;20:262–270. doi: 10.1016/j.conb.2010.03.001. [DOI] [PubMed] [Google Scholar]
- Rauch SL, Jenike MA, Alpert NM, Baer L, Breiter HC, Savage CR, Fischman AJ. Regional cerebral blood flow measured during symptom provocation in obsessive-compulsive disorder using oxygen 15-labeled carbon dioxide and positron emission tomography. Arch Gen Psychiatry. 1994;51:62–70. doi: 10.1001/archpsyc.1994.03950010062008. [DOI] [PubMed] [Google Scholar]
- Ray JP, Price JL. The organization of projections from the mediodorsal nucleus of the thalamus to orbital and medial prefrontal cortex in macaque monkeys. J Comp Neurol. 1993;337:1–31. doi: 10.1002/cne.903370102. [DOI] [PubMed] [Google Scholar]
- Rempel-Clower NL, Barbas H. Topographic organization of connections between the hypothalamus and prefrontal cortex in the rhesus monkey. J Comp Neurol. 1998;398:393–419. doi: 10.1002/(sici)1096-9861(19980831)398:3<393::aid-cne7>3.0.co;2-v. [DOI] [PubMed] [Google Scholar]
- Romero MJ, Asensio S, Palau C, Sanchez A, Romero FJ. Cocaine addiction: diffusion tensor imaging study of the inferior frontal and anterior cingulate white matter. Psychiatry Res. 2010;181:57–63. doi: 10.1016/j.pscychresns.2009.07.004. [DOI] [PubMed] [Google Scholar]
- Rudebeck PH, Murray EA. Balkanizing the primate orbitofrontal cortex: distinct subregions for comparing and contrasting values. Ann N Y Acad Sci. 2011;1239:1–13. doi: 10.1111/j.1749-6632.2011.06267.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rudebeck PH, Behrens TE, Kennerley SW, Baxter MG, Buckley MJ, Walton ME, Rushworth MF. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J Neurosci. 2008;28:13775–13785. doi: 10.1523/JNEUROSCI.3541-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rudebeck PH, Saunders RC, Prescott AT, Chau LS, Murray EA. Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat Neurosci. 2013;16:1140–1145. doi: 10.1038/nn.3440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rushworth MF, Kolling N, Sallet J, Mars RB. Valuation and decision-making in frontal cortex: one or many serial or parallel systems? Curr Opin Neurobiol. 2012;22:946–955. doi: 10.1016/j.conb.2012.04.011. [DOI] [PubMed] [Google Scholar]
- Russchen FT, Bakst I, Amaral DG, Price JL. The amygdalostriatal projections in the monkey. An anterograde tracing study Brain Res. 1985;329:241–257. doi: 10.1016/0006-8993(85)90530-x. [DOI] [PubMed] [Google Scholar]
- Rutledge RB, Dean M, Caplin A, Glimcher PW. Testing the reward prediction error hypothesis with an axiomatic model. J Neurosci. 2010;30:13525–13536. doi: 10.1523/JNEUROSCI.1747-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schacter DL, Addis DR, Hassabis D, Martin VC, Spreng RN, Szpunar KK. The future of memory: remembering, imagining, and the brain. Neuron. 2012;76:677–694. doi: 10.1016/j.neuron.2012.11.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schlösser RG, Wagner G, Schachtzabel C, Peikert G, Koch K, Reichenbach JR, Sauer H. Fronto-cingulate effective connectivity in obsessive compulsive disorder: a study with fMRI and dynamic causal modeling. Hum Brain Mapp. 2010;31:1834–1850. doi: 10.1002/hbm.20980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenbaum G, Roesch MR, Stalnaker TA. Orbitofrontal cortex, decision-making and drug addiction. Trends Neurosci. 2006;29:116–124. doi: 10.1016/j.tins.2005.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schultz W. Multiple reward signals in the brain. Nat Rev Neurosci. 2000;1:199–207. doi: 10.1038/35044563. [DOI] [PubMed] [Google Scholar]
- Schultz W. Getting formal with dopamine and reward. Neuron. 2002;36:241–263. doi: 10.1016/s0896-6273(02)00967-4. [DOI] [PubMed] [Google Scholar]
- Selemon LD, Goldman-Rakic PS. Longitudinal topography and interdigitation of corticostriatal projections in the rhesus monkey. J Neurosci. 1985;5:776–794. doi: 10.1523/JNEUROSCI.05-03-00776.1985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sescousse G, Redouté J, Dreher JC. The architecture of reward value coding in the human orbitofrontal cortex. J Neurosci. 2010;30:13095–13104. doi: 10.1523/JNEUROSCI.3501-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shadlen M, Kiani R, Hanks T, Churchland A. Neurobiology of Decision Making: An Intentional Framework. In: Singer CEW, editor. Better Than Conscious? Decision Making, the Human Mind, and Implications For Institutions. Cambridge: MIT Press; 2008. [Google Scholar]
- Shields DC, Asaad W, Eskandar EN, Jain FA, Cosgrove GR, Flaherty AW, Cassem EH, Price BH, Rauch SL, Dougherty DD. Prospective assessment of stereotactic ablative surgery for intractable major depression. Biol Psychiatry. 2008;64:449–454. doi: 10.1016/j.biopsych.2008.04.009. [DOI] [PubMed] [Google Scholar]
- Siegle GJ, Steinhauer SR, Thase ME, Stenger VA, Carter CS. Can’t shake that feeling: event-related fMRI assessment of sustained amygdala activity in response to emotional information in depressed individuals. Biol Psychiatry. 2002;51:693–707. doi: 10.1016/s0006-3223(02)01314-8. [DOI] [PubMed] [Google Scholar]
- Spooren WPJM, Lynd-Balta E, Mitchell S, Haber SN. Ventral pallidostriatal pathway in the monkey: evidence for modulation of basal ganglia circuits. J Comp Neurol. 1996;370:295–312. doi: 10.1002/(SICI)1096-9861(19960701)370:3<295::AID-CNE2>3.0.CO;2-#. [DOI] [PubMed] [Google Scholar]
- Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science. 2004;304:1782–1787. doi: 10.1126/science.1094765. [DOI] [PubMed] [Google Scholar]
- Surguladze S, Brammer MJ, Keedwell P, Giampietro V, Young AW, Travis MJ, Williams SC, Phillips ML. A differential pattern of neural response toward sad versus happy facial expressions in major depressive disorder. Biol Psychiatry. 2005;57:201–209. doi: 10.1016/j.biopsych.2004.10.028. [DOI] [PubMed] [Google Scholar]
- Takahashi YK, Roesch MR, Wilson RC, Toreson K, O’Donnell P, Niv Y, Schoenbaum G. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat Neurosci. 2011;14:1590–1597. doi: 10.1038/nn.2957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Takikawa Y, Kawagoe R, Hikosaka O. Reward-dependent spatial selectivity of anticipatory activity in monkey caudate neurons. J Neurophysiol. 2002;87:508–515. doi: 10.1152/jn.00288.2001. [DOI] [PubMed] [Google Scholar]
- Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci. 2004;7:887–893. doi: 10.1038/nn1279. [DOI] [PubMed] [Google Scholar]
- Togao O, Yoshiura T, Nakao T, Nabeyama M, Sanematsu H, Nakagawa A, Noguchi T, Hiwatashi A, Yamashita K, Nagao E, et al. Regional gray and white matter volume abnormalities in obsessive-compulsive disorder: a voxel-based morphometry study. Psychiatry Res. 2010;184:29–37. doi: 10.1016/j.pscychresns.2010.06.011. [DOI] [PubMed] [Google Scholar]
- Tziortzi AC, Haber SN, Searle GE, Tsoumpas C, Long CJ, Shotbolt P, Douaud G, Jbabdi S, Behrens TE, Rabiner EA, et al. Connectivity-based functional analysis of dopamine release in the striatum using diffusion-weighted MRI and positron emission tomography. Cereb Cortex. 2014;24:1165–1177. doi: 10.1093/cercor/bhs397. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ullsperger M, von Cramon DY. Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J Neurosci. 2003;23:4308–4314. doi: 10.1523/JNEUROSCI.23-10-04308.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vincent JL, Snyder AZ, Fox MD, Shannon BJ, Andrews JR, Raichle ME, Buckner RL. Coherent spontaneous activity identifies a hippocampal-parietal memory network. J Neurophysiol. 2006;96:3517–3531. doi: 10.1152/jn.00048.2006. [DOI] [PubMed] [Google Scholar]
- Volkow ND, Fowler JS. Addiction, a disease of compulsion and drive: involvement of the orbitofrontal cortex. Cereb Cortex. 2000;10:318–325. doi: 10.1093/cercor/10.3.318. [DOI] [PubMed] [Google Scholar]
- Volkow ND, Wang GJ, Telang F, Fowler JS, Logan J, Childress AR, Jayne M, Ma Y, Wong C. Cocaine cues and dopamine in dorsal striatum: mechanism of craving in cocaine addiction. J Neurosci. 2006;26:6583–6588. doi: 10.1523/JNEUROSCI.1544-06.2006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Volkow ND, Fowler JS, Wang GJ, Telang F, Logan J, Jayne M, Ma Y, Pradhan K, Wong C, Swanson JM. Cognitive control of drug craving inhibits brain reward regions in cocaine abusers. Neuroimage. 2010;49:2536–2543. doi: 10.1016/j.neuroimage.2009.10.088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wallis JD. Polar exploration. Nat Neurosci. 2010;13:7–8. doi: 10.1038/nn0110-7. [DOI] [PubMed] [Google Scholar]
- Walton ME, Devlin JT, Rushworth MF. Interactions between decision making and performance monitoring within prefrontal cortex. Nat Neurosci. 2004;7:1259–1265. doi: 10.1038/nn1339. [DOI] [PubMed] [Google Scholar]
- Walton ME, Behrens TE, Buckley MJ, Rudebeck PH, Rushworth MF. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron. 2010;65:927–939. doi: 10.1016/j.neuron.2010.02.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Watanabe K, Lauwereyns J, Hikosaka O. Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. J Neurosci. 2003;23:10052–10057. doi: 10.1523/JNEUROSCI.23-31-10052.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Williams SM, Goldman-Rakic PS. Widespread origin of the primate mesofrontal dopamine system. Cereb Cortex. 1998;8:321–345. doi: 10.1093/cercor/8.4.321. [DOI] [PubMed] [Google Scholar]
- Wise RA. Brain reward circuitry: insights from unsensed incentives. Neuron. 2002;36:229–240. doi: 10.1016/s0896-6273(02)00965-0. [DOI] [PubMed] [Google Scholar]
- Wittmann BC, Schott BH, Guderian S, Frey JU, Heinze HJ, Düzel E. Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron. 2005;45:459–467. doi: 10.1016/j.neuron.2005.01.010. [DOI] [PubMed] [Google Scholar]
- Wunderlich K, Beierholm UR, Bossaerts P, O’Doherty JP. The human prefrontal cortex mediates integration of potential causes behind observed outcomes. J Neurophysiol. 2011;106:1558–1569. doi: 10.1152/jn.01051.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Záborszky L, Cullinan WE. Projections from the nucleus accumbens to cholinergic neurons of the ventral pallidum: a correlated light and electron microscopic double-immunolabeling study in rat. Brain Res. 1992;570:92–101. doi: 10.1016/0006-8993(92)90568-t. [DOI] [PubMed] [Google Scholar]