Abstract
Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders.
Keywords: decision making, computational psychiatry, addiction, nucleus accumbens, amygdala
Experimental studies in animals and humans have long established the orbitofrontal cortex (OFC) as a critical brain region involved in adaptive decision-making processes. Lesions of the OFC disrupt the ability of subjects to adjust behavior in response to changes in the environment (Fellows & Farah, 2003; Hornak et al., 2004; McEnaney & Butter, 1969; Rudebeck, Saunders, Lundgren, & Murray, 2017; Schoenbaum, Setlow, Nugent, Saddoris, & Gallagher, 2003; Walton, Behrens, Buckley, Rudebeck, & Rushworth, 2010) and other studies have focused on evidence that the OFC was important for exerting inhibitory control over behavior (Dias, Robbins, & Roberts, 1996, 1997). Work over the last several years, however, has highlighted the distinct ways in which the OFC is involved in multiple aspects of decision-making processes (Saez et al., 2018) that extend beyond inhibitory control mechanisms. Electrophysiological and in vivo imaging studies have suggested that the OFC encodes value (Padoa-Schioppa & Assad, 2006; Plassmann, O’Doherty, & Rangel, 2007; Setogawa et al., 2019), identity (Howard, Gottfried, Tobler, & Kahnt, 2015; Klein-Flügge, Barron, Brodersen, Dolan, & Behrens, 2013; Stalnaker et al., 2014), choice (Hare, O’Doherty, Camerer, Schultz, & Rangel, 2008), outcomes (Tremblay & Schultz, 1999), prediction errors (Boorman, Rajendran, O’Reilly, & Behrens, 2016; Sul, Kim, Huh, Lee, & Jung, 2010; Tremblay & Schultz, 1999), action-outcome history (Massi, Donahue, & Lee, 2018; Sul et al., 2010), and even predictions of future actions and outcomes (Schoenbaum, Setlow, Saddoris, & Gallagher, 2003; Stalnaker, Liu, Takahashi, & Schoenbaum, 2018). These results suggest that the OFC is likely to contribute to multiple aspects of reinforcement learning that underlie decision making. Several appealing theories have emerged recently that have incorporated these factors associated with OFC function in order to ascribe a unitary function to the OFC, including hypotheses that the OFC is critical for linking actions to outcomes (e.g., credit assignment) or generating a cognitive map of task space (McDannald, Lucantonio, Burke, Niv, & Schoenbaum, 2011; Walton et al., 2010; Wilson, Takahashi, Schoenbaum, & Niv, 2014). We suggest that if the OFC is involved in these higher-order processes, then multiple reinforcement-learning signals must converge within the OFC (Rich & Wallis, 2016) to become integrated for decision-making functions. We, along with others, have suggested that individual computational steps may be encoded in separate OFC circuits (Frank & Badre, 2012; Frank & Claus, 2006; Groman, Keistler, et al., 2019) and propose here that unlocking these reinforcement-learning circuits can provide quantitative evidence and critical insights into the neural mechanisms of decision-making processes that are affected in neuropsychiatric conditions.
Decision-making abnormalities that are observed in clinical populations are often attributed to dysfunction of the OFC. Individuals diagnosed with substance dependence, depression, schizophrenia or obsessive–compulsive disorder (OCD) have difficulties adjusting choices following a change in contingencies (Fillmore & Rush, 2006; Remijnse et al., 2006; Robinson, Cools, Carlisi, Sahakian, & Drevets, 2012; Waltz & Gold, 2007). This impairment is similar to deficits observed in animals and humans following OFC lesions (Fellows & Farah, 2003; Hornak et al., 2004; Rudebeck et al., 2017; Walton et al., 2010). Moreover, neuroimaging studies have observed altered event-related BOLD activation and lower OFC gray matter volume in individuals with these disorders (Remijnse et al., 2006; Rotge et al., 2009), which has led to the hypothesis that the decision-making problems observed in these disorders are the behavioral consequence of OFC dysfunction. Decision making is a multifaceted process, however, that involves several computational steps, and disruptions in any one of these processes can result in impairments that appear qualitatively similar (Lee, 2013). It is possible, therefore, that the decision-making deficits in mental disorders are mediated by distinct computational processes that are controlled by select OFC circuits.
The recent advancements in computational models and sophisticated behavioral paradigms have begun to elucidate the reinforcement-learning mechanisms underlying decision-making deficits in normal and abnormal states. Combining these analytic approaches with emerging neuroscience techniques (e.g., multi-shell diffusion tensor imaging, fiber photometry, and single cell RNA sequencing) has the potential to improve our understanding of the neural mechanisms mediating decision-making computations, as well as aid in the identification of disease-specific biobehavioral mechanisms for the development of novel therapeutics. Here, we describe our recent work that has used a reinforcement-learning framework to interrogate the role of decision-making processes and lateral OFC circuits in addiction pathology in rats (Groman, Keistler, et al., 2019, Groman et al., 2020a, 2020b). The OFC consists of both medial (mOFC) and lateral (lOFC) subregions and there is evidence that the mOFC and lOFC may control distinct aspects in decision making (Gourley, Lee, Howell, Pittenger, & Taylor, 2010; Mar, Walker, Theobald, Eagle, & Robbins, 2011; Noonan et al., 2010, 2017; Rudebeck & Murray, 2011). Our recent work, however, has been focused on the lOFC as previous studies have consistently demonstrated a role of the lOFC in adaptive decision making (Hampshire, Chaudhry, Owen, & Roberts, 2012; Hervig et al., 2020; Schoenbaum, Setlow, Nugent, et al., 2003) but we hope to extend this work to the mOFC in future studies. Our goal, here, is to demonstrate the utility of computational approaches for providing translational insights into the neural circuits of decision making and to improve our understanding of OFC function in normal and abnormal states.
Neurocomputational Mechanisms of Addiction-Relevant Behaviors
Substance-dependent individuals have difficulties making adaptive decisions in dynamic environments (Ersche, Roiser, Robbins, & Sahakian, 2008; Fillmore & Rush, 2006; Ghahremani et al., 2011) and similar deficits have been observed in animals following exposure to drugs of abuse (Groman, Rich, Smith, Lee, & Taylor, 2018; Jentsch, Olausson, De La Garza, & Taylor, 2002; Schoenbaum, Saddoris, Ramus, Shaham, & Setlow, 2004; Zhukovsky et al., 2019) suggesting that the decision-making deficits observed in addicted individuals are, in part, a consequence of chronic exposure to drugs of abuse (Jentsch & Taylor, 1999; Lucantonio, Stalnaker, Shaham, Niv, & Schoenbaum, 2012; Schoenbaum, Roesch, & Stalnaker, 2006; Schoenbaum & Shaham, 2008). Recent studies in humans and animals have suggested that these deficits may be linked to abnormalities in the processing of negative outcomes. Substance dependent individuals and rats with a history of drug self-administration appear insensitive to negative outcomes (Ersche et al., 2016; Groman et al., 2018; Zhukovsky et al., 2019), which may be related to the disruptions in negative prediction error signaling that have been observed in substance-dependent individuals (Parvaz et al., 2015; Tanabe et al., 2013).
There is also evidence to suggest that decision-making problems that are present prior to drug exposure might enhance addiction risk (Dalley et al., 2007; Perry, Larson, German, Madden, & Carroll, 2005, 2008). For example, recombinant inbred mouse strains that have difficulties in reversal-learning paradigms self-administer greater amounts of cocaine than mouse strains that are better in these paradigms (Cervantes, Laughlin, & Jentsch, 2013). Similar relationships have been observed in longitudinal human studies, where decision-making problems in early adolescence were found to be predictive of problematic drug use two years after the initial assessment (Blair et al., 2018). These findings suggest that the decision-making impairments observed in addicted individuals may, in part, have been present prior to any drug exposure. It was not known, however, if the computational processes that predict drug use are the same or different from those that are affected by drug exposure.
Our work over the last several years has been examining the reinforcement-learning mechanisms mediating addiction-relevant behaviors in rats in order to identify the neurobiological mechanisms that mediate addiction pathology (see Figure 1). Decision making was assessed in rats using a three-armed bandit probabilistic reversal learning (PRL; Figure 1A) task before and after rats were trained to self-administer cocaine in long-access, drug-taking sessions for 21 days. In the PRL task, rats are first required to learn which one of three spatial locations is associated with the highest probability of reward delivery; this is referred to as the acquisition phase. After completing 120 trials, the reinforcement probabilities assigned to each of the spatial locations are changed and rats are required to adjust their choice behavior in order to maximize the rewards they earn; this is referred to as the reversal phase. We hypothesized, based on previous data — indicating that genetic lines of mice known to have poor reversal performance self-administer greater amounts of cocaine (Cervantes et al., 2013) – that the performance of rats in the reversal phase would be predictive of cocaine-taking behaviors and disrupted following cocaine self-administration. We found that rats who performed more poorly in the reversal phase of the PRL task prior to any drug exposure self-administered more cocaine than rats who performed better in the PRL task (Figure 1B; Groman et al., 2020b). Moreover, when we examined the performance of rats in the PRL task following cocaine self-administration, we found that cocaine self-administration resulted in a reversal-selective deficit that was quantitatively and qualitatively similar to that observed in previous studies (Figure 1C; Groman et al., 2020a).
Figure 1.
Computational framework for understanding the role of OFC circuits in addiction pathology. (A) Adaptive choice behavior is assessed in three-choice, spatial discrimination problems using stochastic reward schedules. The reinforcement probabilities assigned to each noseport are pseudorandomly assigned at the start of each sessions and remain stable for 120 trials (e.g., Acquisition phase). Once rats complete 120 trials, the reinforcement probabilities assigned to each noseport change (e.g., Reversal phase). Rats must adjust their choice behavior following this change to maximize the number of rewards they earn. (B) Rats that have greater difficulty in the reversal phase (e.g., poor reversal performance—orange lines) prior to any drug exposure self-administer more cocaine compared to rats that perform better in the reversal phase (e.g., good reversal performance—gray lines). (C) Cocaine self-administration in 6 h, long-access sessions for 21 days impairs the performance of rats in the reversal phase. (D) Computational analyses of choice behavior collected prior to cocaine self-administration revealed that individual differences in value updating following a positive outcome (i.e., positive value updating) were predictive of the rate of escalation in cocaine self-administration. Viral ablation studies demonstrated that ablation of amygdala (Amy) neurons projecting to the OFC reduces value updating following a positive outcome, suggesting that functional differences in amygdala projections to the OFC may mediate susceptibility to drug taking. (E) A computational analysis of choice behavior before and after cocaine self-administration revealed that value updating following a negative outcome (i.e., negative value updating) was disrupted: rats were more likely to persist with an unrewarded choice following cocaine self-administration. Viral ablation studies demonstrated that ablation of OFC neurons projecting to the nucleus accumbens (NAc) impairs value updating following a negative outcome, suggesting that disruptions in negative value updating observed following cocaine self-administration may be the result of drug-induced disruptions in OFC neurons projecting to the nucleus accumbens. * p < .05. ** p < .01. ** p < .001. This figure is adapted from work described in Groman, Keistler, et al., 2019, and Groman et al., 2020a, 2020b.
To determine if the decision-making processes predictive of cocaine-taking behaviors were the same or different from those affected by cocaine self-administration, the choice behavior of rats in the PRL task before and after self-administration were fit with a differential forgetting reinforcement-learning model (Groman, Keistler, et al., 2019; Ito & Doya, 2009). This model quantified four factors: 1) the degree to which value for the chosen option is maintained across trials, 2) the degree to which value for the unchosen options is maintained across trials, 3) the value updating in response to positive outcomes (e.g., the magnitude of change in the value of the chosen option following a rewarded action), and 4) the value updating in response to negative outcomes (e.g., the magnitude of change in the value of the chosen option following an unrewarded action). We first examined the relationship between the four parameters obtained from the choice behavior in the PRL task prior to cocaine self-administration and subsequent cocaine-taking behaviors. We found that individual differences in the magnitude of change in the value of the chosen option following a rewarded action, but not any of the other parameters, predicted cocaine-taking behaviors. Specifically, rats with smaller changes in their strategies following a positive outcome were less likely to persist with a previously rewarded action and subsequently self-administered more cocaine than rats with greater updating following a positive outcome (Figure 1D; Groman et al., 2020b). This suggests that rats with a lower likelihood of using positive outcomes to guide their subsequent choices are more susceptible to higher drug-taking behaviors. We then examined how these parameters changed following cocaine self-administration. The magnitude of change in the value of the chosen option following an unrewarded action, but not any of the other parameters, was significantly impaired following cocaine exposure: the increase in value updating in response to negative outcomes means that rats were more likely to repeat an unrewarded action after extended cocaine self-administration (Figure 1E; Groman et al., 2020a), which is correlated with the degree of perseverative responding in the PRL task. Drug-induced impairments in negative value updating might explain why drug use persists despite the negative consequences that occur with continued drug use (e.g., loss of a job, social support, and risk of imprisonment). These data suggest that the decision-making processes that mediate addiction susceptibility differ from those that are disrupted by drug use, and importantly demonstrate how disruptions in distinct computational mechanisms can masquerade as similar decision-making phenotypes.
There is substantial interest in the use of computational analyses for understanding the pathology of mental illness in the past several years. An outstanding question, however, is whether these computational approaches can provide meaningful insights into the neural mechanisms mediating disease states. Neuroimaging studies have observed structural and functional abnormalities in the OFC of cocaine-dependent individuals (Tanabe et al., 2009; Thompson et al., 2004; Volkow et al., 1991) and some of these differences have been linked to decision-making problems observed in these individuals (Bolla et al., 2003). The OFC, however, is a highly interconnected region, both sending and receiving projections from multiple sensory, subcortical and cortical regions (Haber, Kunishio, Mizobuchi, & Lynd-Balta, 1995; Heilbronner, Rodriguez-Romaguera, Quirk, Groenewegen, & Haber, 2016), and individual circuits may implement different computational steps involved in decision making. The structural and functional OFC abnormalities observed in substance-dependent individuals may, therefore, reflect disruptions in select OFC circuits. Moreover, to ascribe an integrative function for the OFC in decision-making processes, a circuit based experimental approach is warranted.
To investigate the role of lOFC circuits in decision making, we assessed decision making processes in rats in the PRL task before and after ablation of directionally selective lOFC circuits (Groman, Keistler, et al., 2019). We used a dual viral approach to restrict the expression of diphtheria toxin receptors to anatomically defined lOFC circuits. Rodents, unlike primates, do not normally express diphtheria toxin receptors and, therefore, are immune to the diphtheria toxin. Systemic administration of diphtheria toxin can then be used to ablate those neurons expressing diphtheria toxin receptors. Using this approach, we examined how specific lOFC circuits are involved in reinforcement-learning mechanisms of decision making. We found that ablation of either amygdala neurons projecting to the lOFC (Amy→lOFC) or lOFC neurons projecting to the nucleus accumbens (lOFC→NAc) impaired the performance of rats in the PRL task. When the choice behavior of rats was analyzed using the reinforcement-learning model described above, however, we found that the computational processes that led to these decision-making impairments differed. Ablation of the Amy→lOFC circuit disrupted value updating following positive outcomes, whereas ablation of the lOFC→NAc circuit disrupted the value updating following negative outcomes. These anatomically distinct lOFC circuits, therefore, contribute to unique aspects of reinforcement learning that are integrated and used to guide decision-making processes.
These circuit-based results suggest that the lOFC circuit that is involved in addiction susceptibility is different from that which is disrupted by drug use. Specifically, we hypothesize that changes in the formation of the Amy→lOFC circuit — likely through genetic and/or neurodevelopmental mechanisms — may impact risk for future drug-taking behaviors (Figure 1D, right). Indeed, there is evidence that amygdala-lOFC connectivity predicts alcohol use in adolescence (Peters, Peper, Van Duijvenvoorde, Braams, & Crone, 2017) and our recent work has demonstrated that value updating for positive outcomes improves during adolescence in the rat (Moin Afshar, Keip, Taylor, Lee, & Groman, 2020), a critical developmental period in the formation of neural circuits (Casey, Galván, & Somerville, 2016). In contrast, chronic exposure to drugs of abuse may selectively alter the lOFC→NAc circuit and, consequently, impair value updating for negative outcomes (Figure 1E, right). Drug use that persists in light of negative consequences might be the result of disruptions in negative value updating and an area of work we are actively investigating (Keip, Taylor, & Groman, 2019). For example, our preliminary data suggest that drug-induced changes in negative value updating are related to punishment-mediated suppression (e.g., footshock) of drug use: rats that have a greater drug-induced change in the value updating after negative outcomes are less likely to reduce their drug-taking behaviors in response to punishment when compared to rats that have a more attenuated drug-induced change in negative value updating (Keip et al., 2019). We hypothesize, therefore, that disruptions in negative value updating may be the mechanism by which compulsive patterns of drug use emerge in substance-dependent individuals (Ersche et al., 2016). Other addiction-relevant behaviors, such as cue-induced reinstatement and extinction, may be controlled by different prefrontal circuits. Indeed, our previous work has demonstrated that ablation of medial prefrontal cortical neurons projecting to the NAc prevents cue-induced reinstatement whereas ablation of amygdala neurons projecting to the NAc enhances responding during extinction in alcohol-exposed rats (Keistler et al., 2017). In substance dependent individuals, the amalgamation of preexisting deficits (e.g., low positive value updating) and drug-induced disruptions (e.g., impaired negative value updating) is likely to result in the persistent decision-making deficits and impairments in higher-order processes that have been observed in these individuals (Patzelt, Kurth-Nelson, Lim, & Mac-Donald, 2014; Voon et al., 2015) and in rats following exposure to drugs of abuse (Groman, Massi, Mathias, Lee, & Taylor, 2019).
Together, these translational studies in rats demonstrate how computational approaches can be used to generate a mechanistic bridge linking complex behavioral processes with specific lOFC circuits in order to obtain results that can inform findings in humans. Moreover, combining this computational approach with circuit-selective manipulations could determine how different OFC subregions (e.g., mOFC vs. lOFC) are involved in decision making and addiction-relevant behaviors.
Computational Disruptions in Neuropsychiatric Disorders
Our data demonstrate that lOFC circuits control multiple reinforcement-learning processes known to be disrupted in individuals with mental illness and provide the first steps in explaining the prevalence of decision-making deficits and lOFC dysfunction in mental illness. For example, the decision-making deficits observed in individuals with OCD may be due to disruptions in one OFC circuit, whereas those observed in individuals with depression may be due to disruptions in a different OFC circuit. These differences might not be observable in gross assessments of behavior or using traditional neuroimaging approaches (Maia & Frank, 2011; Montague, Dolan, Friston, & Dayan, 2012). In the following section, we describe recent work that has used a computational approach to probe the decision-making problems in individuals with OCD, depression, or schizophrenia, and discuss how these reinforcement-learning computations may be linked to abnormalities in specific OFC circuits.
Individuals with OCD have difficulties in reversal-learning tasks and neuroimaging studies have reported reductions in task-related activity within the OFC (Remijnse et al., 2006, 2009), as well as lower OFC gray matter volume (Rotge et al., 2009). Impairments in adaptive decision-making may arise because individuals with OCD are less likely to persist with the same choice, or stimulus, regardless of outcome. The tendency to repeat choices to a recently chosen stimuli, regardless of the outcome, is known as ‘stimulus stickiness’ (Kanen, Ersche, Fineberg, Robbins, & Cardinal, 2019). Kanen et al. (2019) suggested that reductions in stimulus stickiness might lead to the pathological levels of checking behavior that are observed in individuals with OCD and reflect a compulsive need to verify that alternative choices yield the predicted outcomes (Hauser et al., 2017). OFC lesions decrease stimulus stickiness in nonhuman primates and rodents (Verharen, den Ouden, Adan, & Vanderschuren, 2020; Walton et al., 2010) and there is evidence that this may be mediated by loss of OFC input into the dorsal striatum (Jung et al., 2010; Schilman, Klavir, Winter, Sohr, & Joel, 2010; Seymour, Daw, Roiser, Dayan, & Dolan, 2012; Wood & Ahmari, 2015). Consistent with this hypothesis is evidence that optogenetic manipulations of OFC-to-striatal circuits in mice can impact compulsive-like grooming behaviors in both normal mice and genetic models of OCD (Ahmari et al., 2013; Burguière et al., 2013). Future studies that manipulate OFC-to-striatal circuits and examine the impact on decision making could provide insight into the role of this circuit in choice stickiness and OCD-like behaviors.
The reversal-learning problems that are observed in patients diagnosed with depression appear similar to those observed in patients with OCD. However, evidence suggests that these reversal-learning deficits in individuals with depression are driven by disruptions in outcome-mediated learning (Chen, Takahashi, Nakagawa, Inoue, & Kusumi, 2015): individuals with depression are impaired in reward-based decisions, but decision-making under punishment appears to be comparable to control subjects (Robinson et al., 2012). Based on our circuitry studies in rats and previous work in nonhuman primates, reward-selective impairments in de-pressed individuals may be due to disruptions in amygdala projections to the OFC (Costa, Dal Monte, Lucas, Murray, & Averbeck, 2016; Groman, Keistler, et al., 2019; Rudebeck, Mitz, Chacko, & Murray, 2013). Consistent with this hypothesis is evidence that individuals with depression have reduced amygdala activation in affective labeling tasks (Ferri et al., 2017) and lower resting-state functional connectivity between the OFC and amygdala (Cheng et al., 2018). Studies that characterize the molecular profile of amygdala projections to the OFC could identify circuit specific pharmacological targets for treating reward-mediated deficits in individuals with depression (Mccullough et al., 2016).
Computational analyses of decision-making behaviors in individuals with schizophrenia (Waltz & Gold, 2007), however, suggests that many reinforcement-learning mechanisms, and likely multiple OFC circuits, are disrupted in these individuals. Specifically, individuals diagnosed with schizophrenia have deficits in outcome-mediated learning (Culbreth, Gold, Cools, & Barch, 2016; Dowd, Frank, Collins, Gold, & Barch, 2016; Murray et al., 2008; Weiler, Bellebaum, Brüne, Juckel, & Daum, 2009), exploratory behaviors (Strauss et al., 2011), learning rates (Hernaus et al., 2018), and choice stickiness (Schlagenhauf et al., 2014) when compared to unaffected controls. It is not surprising, therefore, that widespread abnormalities in OFC connectivity have been observed in individuals with schizophrenia (Chen et al., 2017), including lower functional connectivity between the OFC and the amygdala (Anticevic et al., 2014), dorsal striatum (Fornito et al., 2013) and other cortical regions such as cingulate cortex (Xu et al., 2017). Disruptions in these reinforcement-learning processes may be further exacerbated by the working memory deficits that are present in individuals with schizophrenia (Collins, Brown, Gold, Waltz, & Frank, 2014) that collectively may lead to disruptions in model-based learning (Culbreth, Westbrook, Daw, Botvinick, & Barch, 2016), representations of task structure (Adams, Napier, Roiser, Mathys, & Gilleen, 2018; Schlagenhauf et al., 2014), or belief updating (Feeney, Groman, Taylor, & Corlett, 2017; Reed et al., 2020).
Our computational framework may also help explain the heterogeneity in symptomatology within neuropsychiatric disorders, as well as the phenotypic overlap across diagnostic categories (Groman & Jentsch, 2012). For example, paranoia is the excessive concern that individuals are trying to harm you and is present throughout the population (Freeman, 2006), as well as being heightened in individuals with neuropsychiatric disorders (Leamon et al., 2010; Reich & Braginsky, 1994). We have recently reported that paranoia, regardless of diagnostic category, is related to unexpected uncertainty-driven belief updating, altered sensitivity to negative-feedback, and high expectations of task volatility (Reed et al., 2020). These computational processes were also disrupted in methamphetamine-exposed rats—an animal model of psychosis (Machiyama, 1992) – demonstrating the potential of this approach for developing translational links across species and diagnostic categories.
Summary
Here, we discuss how the integration of computational approaches with sophisticated neuroscience techniques in animals can provide clinically meaningful insights into the function of OFC circuitry in decision-making processes and in mental illness. We argue that a unitary theory of OFC function must embrace an empirical, circuit-based and computational approach. The reinforcement-learning framework described here represents a powerful translational, analytic and quantitative approach for interrogating the molecular and genomic mechanisms of disease states to identify novel therapeutic targets for early detection, prevention and treatment of these disorders.
Acknowledgments
This work was supported by Public Health Service grants from the National Institute on Drug Abuse (R01 DA041480, R01 DA043443, K01 DA051598), the National Institute on Mental Health (R21 MH120615), the National Institute on Alcohol Abuse and Alcoholism (P50 AA0012870), and the State of Connecticut, Department of Mental Health and Addiction Services through its support of the Ribicoff Research Facilities. This publication does not express the view of the Department of Mental Health and Addiction Services or the State of Connecticut. The views and opinions expressed are those of the authors.
Footnotes
Daeyeol Lee is a cofounder of Neurogazer Inc. Stephanie M. Groman and Jane R. Taylor report no biomedical financial interests or potential conflicts of interest.
Contributor Information
Stephanie M. Groman, Yale University.
Daeyeol Lee, Johns Hopkins University.
Jane R. Taylor, Yale University
References
- Adams RA, Napier G, Roiser JP, Mathys C, & Gilleen J (2018). Attractor-like dynamics in belief updating in schizophrenia. The Journal of Neuroscience, 38, 9471–9485. 10.1523/JNEURO-SCI.3163-17.2018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ahmari SE, Spellman T, Douglass NL, Kheirbek MA, Simpson HB, Deisseroth K, … Hen R (2013). Repeated cortico-striatal stimulation generates persistent OCD-like behavior. Science, 340, 1234–1239. 10.1126/science.1234733 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Anticevic A, Tang Y, Cho YT, Repovs G, Cole MW, Savic A, … Xu K (2014). Amygdala connectivity differs among chronic, early course, and individuals at risk for developing schizophrenia. Schizophrenia Bulletin, 40, 1105–1116. 10.1093/schbul/sbt165 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Blair MA, Stewart JL, May AC, Reske M, Tapert SF, & Paulus MP (2018). Blunted frontostriatal blood oxygen level-dependent signals predict stimulant and marijuana use. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 3, 947–958. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bolla KI, Eldreth DA, London ED, Kiehl KA, Mouratidis M, Contoreggi C, … Ernst M (2003). Orbitofrontal cortex dysfunction in abstinent cocaine abusers performing a decision-making task. NeuroImage, 19, 1085–1094. 10.1016/S1053-8119(03)00113-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Boorman ED, Rajendran VG, O’Reilly JX, & Behrens TE (2016). Two anatomically and computationally distinct learning signals predict changes to stimulus-outcome associations in hippocampus. Neuron, 89, 1343–1354. 10.1016/j.neuron.2016.02.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Burguière E, Monteiro P, Feng G, & Graybiel AM (2013). Optogenetic stimulation of lateral orbitofronto-striatal pathway suppresses compulsive behaviors. Science, 340, 1243–1246. 10.1126/science.1232380 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Casey BJ, Galván A, & Somerville LH (2016). Beyond simple models of adolescence to an integrated circuit-based account: A commentary. Developmental Cognitive Neuroscience, 17, 128–130. 10.1016/j.dcn.2015.12.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cervantes MC, Laughlin RE, & Jentsch JD (2013). Cocaine self-administration behavior in inbred mouse lines segregating different capacities for inhibitory control. Psychopharmacology, 229, 515–525. 10.1007/s00213-013-3135-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chen C, Takahashi T, Nakagawa S, Inoue T, & Kusumi I (2015). Reinforcement learning in depression: A review of computational research. Neuroscience and Biobehavioral Reviews, 55, 247–267. 10.1016/j.neubiorev.2015.05.005 [DOI] [PubMed] [Google Scholar]
- Chen X, Liu C, He H, Chang X, Jiang Y, Li Y, … Yao D (2017). Transdiagnostic differences in the resting-state functional connectivity of the prefrontal cortex in depression and schizophrenia. Journal of Affective Disorders, 217, 118–124. 10.1016/j.jad.2017.04.001 [DOI] [PubMed] [Google Scholar]
- Cheng W, Rolls ET, Qiu J, Xie X, Lyu W, Li Y, … Feng J (2018). Functional connectivity of the human amygdala in health and in depression. Social Cognitive and Affective Neuroscience, 13, 557–568. 10.1093/scan/nsy032 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Collins AGE, Brown JK, Gold JM, Waltz JA, & Frank MJ (2014). Working memory contributions to reinforcement learning impairments in schizophrenia. The Journal of Neuroscience, 34, 13747–13756. 10.1523/JNEUROSCI.0989-14.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Costa VD, Dal Monte O, Lucas DR, Murray EA, & Averbeck BB (2016). Amygdala and ventral striatum make distinct contributions to reinforcement learning. Neuron, 92, 505–517. 10.1016/j.neuron.2016.09.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Culbreth AJ, Gold JM, Cools R, & Barch DM (2016). Impaired activation in cognitive control regions predicts reversal learning in schizophrenia. Schizophrenia Bulletin, 42, 484–493. 10.1093/schbul/sbv075 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Culbreth AJ, Westbrook A, Daw ND, Botvinick M, & Barch DM (2016). Reduced model-based decision-making in schizophrenia. Journal of Abnormal Psychology, 125, 777–787. 10.1037/abn0000164 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dalley JW, Fryer TD, Brichard L, Robinson ES, Theobald DE, Lääne K, … Robbins TW (2007). Nucleus accumbens D2/3 receptors predict trait impulsivity and cocaine reinforcement. Science, 315, 1267–1270. 10.1126/science.1137073 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dias R, Robbins TW, & Roberts AC (1996). Dissociation in prefrontal cortex of affective and attentional shifts. Nature, 380, 69–72. 10.1038/380069a0 [DOI] [PubMed] [Google Scholar]
- Dias R, Robbins TW, & Roberts AC (1997). Dissociable forms of inhibitory control within prefrontal cortex with an analog of the Wisconsin Card Sort Test: Restriction to novel situations and independence from “on-line” processing. The Journal of Neuroscience, 17, 9285–9297. 10.1523/JNEUROSCI.17-23-09285.1997 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dowd EC, Frank MJ, Collins A, Gold JM, & Barch DM (2016). Probabilistic reinforcement learning in patients with schizophrenia: Relationships to anhedonia and avolition. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1, 460–473. 10.1016/j.bpsc.2016.05.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ersche KD, Gillan CM, Jones PS, Williams GB, Ward LH, Luijten M, … Robbins TW (2016). Carrots and sticks fail to change behavior in cocaine addiction. Science, 352, 1468–1471. 10.1126/science.aaf3700 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ersche KD, Roiser JP, Robbins TW, & Sahakian BJ (2008). Chronic cocaine but not chronic amphetamine use is associated with perseverative responding in humans. Psychopharmacology, 197, 421–431. 10.1007/s00213-007-1051-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Feeney EJ, Groman SM, Taylor JR, & Corlett PR (2017). Explaining delusions: Reducing uncertainty through basic and computational neuroscience. Schizophrenia Bulletin, 43, 263–272. 10.1093/schbul/sbw194 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fellows LK, & Farah MJ (2003). Ventromedial frontal cortex mediates affective shifting in humans: Evidence from a reversal learning paradigm. Brain: A Journal of Neurology, 126, 1830–1837. 10.1093/brain/awg180 [DOI] [PubMed] [Google Scholar]
- Ferri J, Eisendrath SJ, Fryer SL, Gillung E, Roach BJ, & Mathalon DH (2017). Blunted amygdala activity is associated with depression severity in treatment-resistant depression. Cognitive, Affective & Behavioral Neuroscience, 17, 1221–1231. 10.3758/s13415-017-0544-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fillmore MT, & Rush CR (2006). Polydrug abusers display impaired discrimination-reversal learning in a model of behavioural control. Journal of Psychopharmacology, 20, 24–32. 10.1177/0269881105057000 [DOI] [PubMed] [Google Scholar]
- Fornito A, Harrison BJ, Goodby E, Dean A, Ooi C, Nathan PJ, … Bullmore ET (2013). Functional dysconnectivity of corticostriatal circuitry as a risk phenotype for psychosis. Journal of the American Medical Association Psychiatry, 70, 1143–1151. 10.1001/jamapsychiatry.2013.1976 [DOI] [PubMed] [Google Scholar]
- Frank MJ, & Badre D (2012, March). Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: Computational analysis. Cerebral Cortex, 22, 509–526. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Frank MJ, & Claus ED (2006). Anatomy of a decision: Striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychological Review, 113, 300–326. 10.1037/0033-295X.113.2.300 [DOI] [PubMed] [Google Scholar]
- Freeman D (2006). Delusions in the nonclinical population. Current Psychiatry Reports, 8, 191–204. 10.1007/s11920-006-0023-1 [DOI] [PubMed] [Google Scholar]
- Ghahremani DG, Tabibnia G, Monterosso J, Hellemann G, Poldrack RA, & London ED (2011). Effect of modafinil on learning and task-related brain activity in methamphetamine-dependent and healthy individuals. Neuropsychopharmacology, 36, 950–959. 10.1038/npp.2010.233 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gourley SL, Lee AS, Howell JL, Pittenger C, & Taylor JR (2010). Dissociable regulation of instrumental action within mouse prefrontal cortex. European Journal of Neuroscience. 32, 1726–1734. 10.1111/j.1460-9568.2010.07438.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Groman SM, Hillmer AT, Liu H, Fowles K, Holden D, Morris ED, … Taylor JR (2020a). Dysregulation of decision-making related to mGlu5, but not midbrain D3, receptor availability following cocaine self-administration in rats. Biological Psychiatry. Advance online publication. 10.1016/j.biopsych.2020.06.020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Groman SM, Hillmer AT, Liu H, Fowles K, Holden D, Morris ED, … Taylor JR (2020b). Midbrain D3 receptor availability predicts escalation in cocaine self-administration. Biological Psychiatry. Advance online publication. 10.1016/j.biopsych.2020.02.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Groman SM, & Jentsch JD (2012). Cognitive control and the dopamine D2-like receptor: A dimensional understanding of addiction. Depression and Anxiety, 29, 295–306. 10.1002/da.20897 [DOI] [PubMed] [Google Scholar]
- Groman SM, Keistler C, Keip AJ, Hammarlund E, DiLeone RJ, Pittenger C, … Taylor JR (2019). Orbitofrontal circuits control multiple reinforcement-learning processes. Neuron, 103, 734–746. e3. 10.1016/j.neuron.2019.05.042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Groman SM, Massi B, Mathias SR, Lee D, & Taylor JR (2019). Model-free and model-based influences in addiction-related behaviors. Biological Psychiatry, 85, 936–945. 10.1016/j.biopsych.2018.12.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Groman SM, Rich KM, Smith NJ, Lee D, & Taylor JR (2018). Chronic exposure to methamphetamine disrupts reinforcement-based decision making in rats. Neuropsychopharmacology, 43, 770–780. 10.1038/npp.2017.159 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haber SN, Kunishio K, Mizobuchi M, & Lynd-Balta E (1995). The orbital and medial prefrontal circuit through the primate basal ganglia. The Journal of Neuroscience, 15, 4851–4867. 10.1523/JNEUROSCI.15-07-04851.1995 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hampshire A, Chaudhry AM, Owen AM, & Roberts AC (2012). Dissociable roles for lateral orbitofrontal cortex and lateral prefrontal cortex during preference driven reversal learning. NeuroImage, 59, 4102–4112. 10.1016/j.neuroimage.2011.10.072 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hare TA, O’Doherty J, Camerer CF, Schultz W, & Rangel A (2008). Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. The Journal of Neuroscience, 28, 5623–5630. 10.1523/JNEUROSCI.1309-08.2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hauser TU, Iannaccone R, Dolan RJ, Ball J, Hättenschwiler J, Drechsler R, … Brem S (2017). Increased fronto-striatal reward prediction errors moderate decision making in obsessive-compulsive disorder. Psychological Medicine, 47, 1246–1258. 10.1017/S0033291716003305 [DOI] [PubMed] [Google Scholar]
- Heilbronner SR, Rodriguez-Romaguera J, Quirk GJ, Groenewegen HJ, & Haber SN (2016). Circuit-based corticostriatal homologies between rat and primate. Biological Psychiatry, 80, 509–521. 10.1016/j.biopsych.2016.05.012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hernaus D, Xu Z, Brown EC, Ruiz R, Frank MJ, Gold JM, & Waltz JA (2018). Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals. Cognitive, Affective & Behavioral Neuroscience, 18, 1338–1351. 10.3758/s13415-018-0643-z [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hervig ME, Fiddian L, Piilgaard L, Božič T, Blanco-Pozo M, Knudsen C, … Robbins TW (2020). Dissociable and paradoxical roles of rat medial and lateral orbitofrontal cortex in visual serial reversal learning. Cerebral Cortex, 30, 1016–1029. 10.1093/cercor/bhz144 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hornak J, O’Doherty J, Bramham J, Rolls ET, Morris RG, Bullock PR, & Polkey CE (2004). Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. Journal of Cognitive Neuroscience, 16, 463–478. 10.1162/089892904322926791 [DOI] [PubMed] [Google Scholar]
- Howard JD, Gottfried JA, Tobler PN, & Kahnt T (2015). Identity-specific coding of future rewards in the human orbitofrontal cortex. PNAS Proceedings of the National Academy of Sciences of the United States of America, 112, 5195–5200. 10.1073/pnas.1503550112 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Ito M, & Doya K (2009). Validation of decision-making models and analysis of decision variables in the rat basal ganglia. The Journal of Neuroscience, 29, 9861–9874. 10.1523/JNEUROSCI.6157-08.2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jentsch JD, Olausson P, De La Garza R II, & Taylor JR (2002). Impairments of reversal learning and response perseveration after repeated, intermittent cocaine administrations to monkeys. Neuropsychopharmacology, 26, 183–190. 10.1016/S0893-133X(01)00355-4 [DOI] [PubMed] [Google Scholar]
- Jentsch JD, & Taylor JR (1999). Impulsivity resulting from frontostriatal dysfunction in drug abuse: Implications for the control of behavior by reward-related stimuli. Psychopharmacology, 146, 373–390. 10.1007/PL00005483 [DOI] [PubMed] [Google Scholar]
- Jung Y-C, Ku J, Namkoong K, Lee W, Kim SI, & Kim J-J (2010). Human orbitofrontal-striatum functional connectivity modulates behavioral persistence. Neuroreport, 21, 502–506. 10.1097/WNR.0b013e3283383482 [DOI] [PubMed] [Google Scholar]
- Kanen JW, Ersche KD, Fineberg NA, Robbins TW, & Cardinal RN (2019). Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: Remediating effects of dopaminergic D2/3 receptor agents. Psychopharmacology, 236, 2337–2358. 10.1007/s00213-019-05325-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- Keip AJ, Taylor JR, & Groman SM (2019). Unidirectional ablation of orbitofrontal-nucleus accumbens projections decreases sensitivity to negative outcomes in methamphetamine self-administering rats. Society of Neuroscience Annual Meeting, Chicago, IL. [Google Scholar]
- Keistler CR, Hammarlund E, Barker JM, Bond CW, DiLeone RJ, Pittenger C, & Taylor JR (2017). Regulation of alcohol extinction and cue-induced reinstatement by specific projections among medial prefrontal cortex, nucleus accumbens, and basolateral amygdala. The Journal of Neuroscience, 37, 4462–4471. 10.1523/JNEUROSCI.3383-16.2017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klein-Flügge MC, Barron HC, Brodersen KH, Dolan RJ, & Behrens TE (2013). Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. The Journal of Neuroscience, 33, 3202–3211. 10.1523/JNEURO-SCI.2532-12.2013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Leamon MH, Flower K, Salo RE, Nordahl TE, Kranzler HR, & Galloway GP (2010). Methamphetamine and paranoia: The methamphetamine experience questionnaire. The American Journal on Addictions, 19, 155–168. 10.1111/j.1521-0391.2009.00014.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee D (2013). Decision making: From neuroscience to psychiatry. Neuron, 78, 233–248. 10.1016/j.neuron.2013.04.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lucantonio F, Stalnaker TA, Shaham Y, Niv Y, & Schoenbaum G (2012). The impact of orbitofrontal dysfunction on cocaine addiction. Nature Neuroscience, 15, 358–366. 10.1038/nn.3014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Machiyama Y (1992). Chronic methamphetamine intoxication model of schizophrenia in animals. Schizophrenia Bulletin, 18, 107–113. 10.1093/schbul/18.1.107 [DOI] [PubMed] [Google Scholar]
- Maia TV, & Frank MJ (2011). From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience, 14, 154–162. 10.1038/nn.2723 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mar AC, Walker ALJ, Theobald DE, Eagle DM, & Robbins TW (2011). Dissociable effects of lesions to orbitofrontal cortex subregions on impulsive choice in the rat. The Journal of Neuroscience, 31, 6398–6404. 10.1523/JNEUROSCI.6620-10.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Massi B, Donahue CH, & Lee D (2018). Volatility Facilitates Value Updating in the Prefrontal Cortex. Neuron, 99, 598–608. e4. 10.1016/j.neuron.2018.06.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McCullough KM, Morrison FG, & Ressler KJ (2016). Bridging the Gap: Towards a cell-type specific understanding of neural circuits underlying fear behaviors. Neurobiology of Learning and Memory, 135, 27–39. 10.1016/j.nlm.2016.07.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McDannald MA, Lucantonio F, Burke KA, Niv Y, & Schoenbaum G (2011). Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. The Journal of Neuroscience, 31, 2700–2705. 10.1523/JNEUROSCI.5499-10.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McEnaney KW, & Butter CM (1969). Perseveration of responding and nonresponding in monkeys with orbital frontal ablations. Journal of Comparative and Physiological Psychology, 68, 558–561. 10.1037/h0027639 [DOI] [PubMed] [Google Scholar]
- Moin Afshar N, Keip AJ, Taylor JR, Lee D, & Groman SM (2020). Reinforcement learning during adolescence in rats. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 40, 5857–5870. 10.1523/JNEUROSCI.0910-20.2020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Montague PR, Dolan RJ, Friston KJ, & Dayan P (2012). Computational psychiatry. Trends in Cognitive Sciences, 16, 72–80. 10.1016/j.tics.2011.11.018 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Murray GK, Corlett PR, Clark L, Pessiglione M, Blackwell AD, Honey G, … Fletcher PC (2008). Substantia nigra/ventral tegmental reward prediction error disruption in psychosis. Molecular Psychiatry, 13, 267–276. 10.1038/sj.mp.4002157 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Noonan MP, Chau BKH, Rushworth MFS, & Fellows LK (2017). Contrasting effects of medial and lateral orbitofrontal cortex lesions on credit assignment and decision-making in humans. The Journal of Neuroscience, 37, 7023–7035. 10.1523/JNEUROSCI.0692-17.2017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Noonan MP, Walton ME, Behrens TEJ, Sallet J, Buckley MJ, & Rushworth MFS (2010). Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. PNAS Proceedings of the National Academy of Sciences of the United States of America, 107, 20547–20552. 10.1073/pnas.1012246107 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Padoa-Schioppa C, & Assad JA (2006). Neurons in the orbitofrontal cortex encode economic value. Nature, 441, 223–226. 10.1038/nature04676 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Parvaz MA, Konova AB, Proudfit GH, Dunning JP, Malaker P, Moeller SJ, … Goldstein RZ (2015). Impaired neural response to negative prediction errors in cocaine addiction. The Journal of Neuroscience, 35, 1872–1879. 10.1523/JNEUROSCI.2777-14.2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Patzelt EH, Kurth-Nelson Z, Lim KO, & MacDonald AW III. (2014). Excessive state switching underlies reversal learning deficits in cocaine users. Drug and Alcohol Dependence, 134, 211–217. 10.1016/j.drugalcdep.2013.09.029 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perry JL, Larson EB, German JP, Madden GJ, & Carroll ME (2005). Impulsivity (delay discounting) as a predictor of acquisition of IV cocaine self-administration in female rats. Psychopharmacology, 178, 193–201. 10.1007/s00213-004-1994-4 [DOI] [PubMed] [Google Scholar]
- Perry JL, Nelson SE, & Carroll ME (2008). Impulsive choice as a predictor of acquisition of IV cocaine self-administration and reinstatement of cocaine-seeking behavior in male and female rats. Experimental and Clinical Psychopharmacology, 16, 165–177. 10.1037/1064-1297.16.2.165 [DOI] [PubMed] [Google Scholar]
- Peters S, Peper JS, Van Duijvenvoorde ACK, Braams BR, & Crone EA (2017). Amygdala-orbitofrontal connectivity predicts alcohol use two years later: A longitudinal neuroimaging study on alcohol use in adolescence. Developmental Science, 20, e12448. 10.1111/desc.12448 [DOI] [PubMed] [Google Scholar]
- Plassmann H, O’Doherty J, & Rangel A (2007). Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. The Journal of Neuroscience, 27, 9984–9988. 10.1523/JNEUROSCI.2131-07.2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reed EJ, Uddenberg S, Suthaharan P, Mathys CD, Taylor JR, Groman SM, & Corlett PR (2020). Paranoia as a deficit in non-social belief updating. eLife, 9, e56345. 10.7554/eLife.56345 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Reich J, & Braginsky Y (1994). Paranoid personality traits in a panic disorder population: A pilot study. Comprehensive Psychiatry, 35, 260–264. 10.1016/0010-440X(94)90017-5 [DOI] [PubMed] [Google Scholar]
- Remijnse PL, Nielen MMA, van Balkom AJLM, Cath DC, van Oppen P, Uylings HBM, & Veltman DJ (2006). Reduced orbitofrontal-striatal activity on a reversal learning task in obsessive-compulsive disorder. Archives of General Psychiatry, 63, 1225–1236. 10.1001/archpsyc.63.11.1225 [DOI] [PubMed] [Google Scholar]
- Remijnse PL, Nielen MMA, van Balkom AJLM, Hendriks GJ, Hoogendijk WJ, Uylings HBM, & Veltman DJ (2009). Differential frontal-striatal and paralimbic activity during reversal learning in major depressive disorder and obsessive-compulsive disorder. Psychological Medicine, 39, 1503–1518. 10.1017/S0033291708005072 [DOI] [PubMed] [Google Scholar]
- Rich EL, & Wallis JD (2016). Decoding subjective decisions from orbitofrontal cortex. Nature Neuroscience, 19, 973–980. 10.1038/nn.4320 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Robinson OJ, Cools R, Carlisi CO, Sahakian BJ, & Drevets WC (2012). Ventral striatum response during reward and punishment reversal learning in unmedicated major depressive disorder. The American Journal of Psychiatry, 169, 152–159. 10.1176/appi.ajp.2011.11010137 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rotge JY, Guehl D, Dilharreguy B, Tignol J, Bioulac B, Allard M, … Aouizerate B (2009). Meta-analysis of brain volume changes in obsessive-compulsive disorder. Biological Psychiatry, 65, 75–83. 10.1016/j.biopsych.2008.06.019 [DOI] [PubMed] [Google Scholar]
- Rudebeck PH, Mitz AR, Chacko RV, & Murray EA (2013). Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex. Neuron, 80, 1519–1531. 10.1016/j.neuron.2013.09.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rudebeck PH, & Murray EA (2011). Balkanizing the primate orbitofrontal cortex: Distinct subregions for comparing and contrasting values. Annals of the New York Academy of Sciences, 1239, 1–13. 10.1111/j.1749-6632.2011.06267.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rudebeck PH, Saunders RC, Lundgren DA, & Murray EA (2017). Specialized Representations of Value in the Orbital and Ventro-lateral Prefrontal Cortex: Desirability versus Availability of Outcomes. Neuron, 95, 1208–1220. e5. 10.1016/j.neuron.2017.07.042 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Saez I, Lin J, Stolk A, Chang E, Parvizi J, Schalk G, … Hsu M (2018). Encoding of Multiple Reward-Related Computations in Transient and Sustained High-Frequency Activity in Human OFC. Current Biology, 28, 2889–2899. e4. 10.1016/j.cub.2018.07.045 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schilman EA, Klavir O, Winter C, Sohr R, & Joel D (2010). The role of the striatum in compulsive behavior in intact and orbitofrontal-cortex-lesioned rats: Possible involvement of the serotonergic system. Neuropsychopharmacology, 35, 1026–1039. 10.1038/npp.2009.208 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schlagenhauf F, Huys QJM, Deserno L, Rapp MA, Beck A, Heinze HJ, … Heinz A (2014). Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. NeuroImage, 89, 171–180. 10.1016/j.neuroimage.2013.11.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenbaum G, Roesch MR, & Stalnaker TA (2006). Orbitofrontal cortex, decision-making and drug addiction. Trends in Neurosciences, 29, 116–124. 10.1016/j.tins.2005.12.006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenbaum G, Saddoris MP, Ramus SJ, Shaham Y, & Setlow B (2004). Cocaine-experienced rats exhibit learning deficits in a task sensitive to orbitofrontal cortex lesions. European Journal of Neuroscience, 19, 1997–2002. 10.1111/j.1460-9568.2004.03274.x [DOI] [PubMed] [Google Scholar]
- Schoenbaum G, Setlow B, Nugent SL, Saddoris MP, & Gallagher M (2003). Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learning & Memory, 10, 129–140. 10.1101/lm.55203 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Schoenbaum G, Setlow B, Saddoris MP, & Gallagher M (2003). Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron, 39, 855–867. 10.1016/S0896-6273(03)00474-4 [DOI] [PubMed] [Google Scholar]
- Schoenbaum G, & Shaham Y (2008). The role of orbitofrontal cortex in drug addiction: A review of preclinical studies. Biological Psychiatry, 63, 256–262. 10.1016/j.biopsych.2007.06.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Setogawa T, Mizuhiki T, Matsumoto N, Akizawa F, Kuboki R, Richmond BJ, & Shidara M (2019). Neurons in the monkey orbitofrontal cortex mediate reward value computation and decision-making. Communications Biology, 2, 126. 10.1038/s42003-019-0363-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Seymour B, Daw ND, Roiser JP, Dayan P, & Dolan R (2012). Serotonin selectively modulates reward value in human decision-making. The Journal of Neuroscience, 32, 5833–5842. 10.1523/JNEUROSCI.0053-12.2012 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stalnaker TA, Cooch NK, McDannald MA, Liu T-L, Wied H, & Schoenbaum G (2014). Orbitofrontal neurons infer the value and identity of predicted outcomes. Nature Communications, 5, 3926. 10.1038/ncomms4926 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stalnaker TA, Liu T-L, Takahashi YK, & Schoenbaum G (2018). Orbitofrontal neurons signal reward predictions, not reward prediction errors. Neurobiology of Learning and Memory, 153, 137–143. 10.1016/j.nlm.2018.01.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Strauss GP, Frank MJ, Waltz JA, Kasanova Z, Herbener ES, & Gold JM (2011). Deficits in positive reinforcement learning and uncertainty-driven exploration are associated with distinct aspects of negative symptoms in schizophrenia. Biological Psychiatry, 69, 424–431. 10.1016/j.biopsych.2010.10.015 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sul JH, Kim H, Huh N, Lee D, & Jung MW (2010). Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron, 66, 449–460. 10.1016/j.neuron.2010.03.033 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tanabe J, Reynolds J, Krmpotich T, Claus E, Thompson LL, Du YP, & Banich MT (2013). Reduced neural tracking of prediction error in substance-dependent individuals. The American Journal of Psychiatry, 170, 1356–1363. 10.1176/appi.ajp.2013.12091257 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tanabe J, Tregellas JR, Dalwani M, Thompson L, Owens E, Crowley T, & Banich M (2009). Medial orbitofrontal cortex gray matter is reduced in abstinent substance-dependent individuals. Biological Psychiatry, 65, 160–164. 10.1016/j.biopsych.2008.07.030 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Thompson PM, Hayashi KM, Simon SL, Geaga JA, Hong MS, Sui Y, … London ED (2004). Structural abnormalities in the brains of human subjects who use methamphetamine. The Journal of Neuroscience, 24, 6028–6036. 10.1523/JNEUROSCI.0713-04.2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Tremblay L, & Schultz W (1999). Relative reward preference in primate orbitofrontal cortex. Nature, 398, 704–708. 10.1038/19525 [DOI] [PubMed] [Google Scholar]
- Verharen JPH, den Ouden HEM, Adan RAH, & Vanderschuren LJMJ (2020). Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex. Psychopharmacology, 237, 1267–1280. 10.1007/s00213-020-05454-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Volkow ND, Fowler JS, Wolf AP, Hitzemann R, Dewey S, Bendriem B, … Hoff A (1991). Changes in brain glucose metabolism in cocaine dependence and withdrawal. The American Journal of Psychiatry, 148, 621–626. 10.1176/ajp.148.5.621 [DOI] [PubMed] [Google Scholar]
- Voon V, Derbyshire K, Rück C, Irvine MA, Worbe Y, Enander J, … Bullmore ET (2015). Disorders of compulsivity: A common bias towards learning habits. Molecular Psychiatry, 20, 345–352. 10.1038/mp.2014.44 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Walton ME, Behrens TE, Buckley MJ, Rudebeck PH, & Rushworth MF (2010). Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron, 65, 927–939. 10.1016/j.neuron.2010.02.027 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Waltz JA, & Gold JM (2007). Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction. Schizophrenia Research, 93, 296–303. 10.1016/j.schres.2007.03.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Weiler JA, Bellebaum C, Brüne M, Juckel G, & Daum I (2009). Impairment of probabilistic reward-based learning in schizophrenia. Neuropsychology, 23, 571–580. 10.1037/a0016166 [DOI] [PubMed] [Google Scholar]
- Wilson RC, Takahashi YK, Schoenbaum G, & Niv Y (2014). Orbitofrontal cortex as a cognitive map of task space. Neuron, 81, 267–279. 10.1016/j.neuron.2013.11.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wood J, & Ahmari SE (2015). A framework for understanding the emerging role of corticolimbic-ventral striatal networks in OCD-associated repetitive behaviors. Frontiers in Systems Neuroscience, 9, 171. 10.3389/fnsys.2015.00171 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xu Y, Qin W, Zhuo C, Xu L, Zhu J, Liu X, & Yu C (2017). Selective functional disconnection of the orbitofrontal subregions in schizophrenia. Psychological Medicine, 47, 1637–1646. 10.1017/S0033291717000101 [DOI] [PubMed] [Google Scholar]
- Zhukovsky P, Puaud M, Jupp B, Sala-Bayo J, Alsiö J, Xia J, … Dalley JW (2019). Withdrawal from escalated cocaine self-administration impairs reversal learning by disrupting the effects of negative feedback on reward exploitation: A behavioral and computational analysis. Neuropsychopharmacology, 44, 2163–2173. 10.1038/s41386-019-0381-0 [DOI] [PMC free article] [PubMed] [Google Scholar]