Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2025 Mar 1.
Published in final edited form as: Addict Neurosci. 2024 Jan 15;10:100143. doi: 10.1016/j.addicn.2024.100143

The utility of a latent-cause framework for understanding addiction phenomena

Sashank Pisupati 1,4, Angela Langdon 2, Anna B Konova 3, Yael Niv 4
PMCID: PMC10959497  NIHMSID: NIHMS1960417  PMID: 38524664

Abstract

Computational models of addiction often rely on a model-free reinforcement learning (RL) formulation, owing to the close associations between model-free RL, habitual behavior and the dopaminergic system. However, such formulations typically do not capture key recurrent features of addiction phenomena such as craving and relapse. Moreover, they cannot account for goal-directed aspects of addiction that necessitate contrasting, model-based formulations. Here we synthesize a growing body of evidence and propose that a latent-cause framework can help unify our understanding of several recurrent phenomena in addiction, by viewing them as the inferred return of previous, persistent “latent causes”. We demonstrate that applying this framework to Pavlovian and instrumental settings can help account for defining features of craving and relapse such as outcome-specificity, generalization, and cyclical dynamics. Finally, we argue that this framework can bridge model-free and model-based formulations, and account for individual variability in phenomenology by accommodating the memories, beliefs, and goals of those living with addiction, motivating a centering of the individual, subjective experience of addiction and recovery.

Keywords: latent-cause inference, addiction, craving, relapse

Introduction: addiction and model-free reinforcement learning

Addiction is often defined as a chronic, relapsing condition1 characterized by compulsive behaviors that continue despite adverse consequences. As a result, many prominent computational models conceptualize addiction as an aberrant2 regime of the brain’s learning and decision-making systems (Gutkin & Ahmed, 2011; Mollick & Kober, 2020). In particular, early computational accounts of addiction focused on the effects of addictive substances on trial-and-error learning, or so-called model-free reinforcement learning (RL; Box 1), and the resulting hijacking of habitual behavior. There is a growing appreciation of alternative accounts that focus on deliberative planning/decision-making and model-based RL (see Gueguen et al., 2021; Mollick and Kober, 2020; Smith et al., 2021 for in-depth discussions of this literature), however we focus here on model-free RL as the most well-characterized account.

Box 1: Basic terminology.

Reinforcement learning (RL):

A computational theory for learning from feedback (rewarding or punishing outcomes O) about the best actions A to take in an environment.

State:

A representation of sensory observations S, including internal or external cues, that acts as a substrate for learning. The goal of reinforcement learning is to specify what action to make in what state.

Action policy:

The probability of taking each action A in a given state S.

Value:

A typically scalar representation of the cumulative future reward expected from a given state V(S) or from a particular action taken in that state Q(S,A)

Model-based RL:

A form of RL that uses a learned internal model of the transitions between states in the world, and the outcome in each state, to simulate the consequences of actions.

Model-free RL:

A form of RL that uses past experience to learn values and/or policies through trial-and-error, without estimating a model of the world.

Instrumental learning:

A learning setting in which outcomes O are contingent on performing certain actions A in a given state S.

Pavlovian learning:

A learning setting in which outcomes O are predicted by the presence of a certain state S alone, without any action contingencies.

Latent cause:

A grouping of sensory observations according to a hypothesized shared hidden variable "generating" the sensory observations. The structure and presence of latent (hidden) causes must be inferred.

One popular model-free RL account suggests that the unnaturally reinforcing properties of addictive substances give rise to features of addiction such as an inflated valuation of cues signaling drug rewards above and beyond those signaling non-drug rewards, and compulsive (i.e., hard to resist) habitual behaviors resulting from such overvaluation (Huys et al., 2016; Keramati et al., 2012; Kurth-Nelson & Redish, 2012). This view offers principled ways to model the effects of addiction observed in value-based decisions or economic choices (Ahn et al., 2016; Lim & Ersche, 2023; Smith et al., 2021; Vassileva et al., 2023). It also explains connections between addiction and the neuromodulator dopamine, which is a major molecular target of many addictive substances, and is thought to be a critical element of model-free RL algorithms such as temporal-difference (TD) learning in the brain – namely, a reward prediction error signal that is crucial for learning.

For instance, addictive drugs have been hypothesized to give rise to a persistent, non-compensable version of the prediction error in TD learning algorithms that learn state values, leading to the overvaluation of drug-predictive cues (Redish, 2004). In variants of TD learning that learn state-action values, extended drug exposure has been proposed to alter the basal reward level relative to which prediction errors are calculated, to account for the development of compulsive drug-taking actions and increased impulsivity (Dezfouli et al., 2009). Finally, several extensions of these models of addiction rely on actor-critic formulations, which use TD prediction errors to simultaneously update a “critic” that learns state values and an “actor” that separately learns action policies using signals from the critic (Box 1). Proposals for the influence of addictive drugs in these models range from aberrant learning that originates in the critic and consequently affects the actor (Piray et al., 2010; Takahashi et al., 2008) to aberrant prediction-error signalling that affects both the critic and actor simultaneously (Dayan, 2009; Takahashi et al., 2008), yielding compulsive habits.

Challenges to model-free RL formulations of addiction

Despite these successes in accounting for overvaluation and compulsion, purely model-free RL formulations fail to capture many crucial features of addiction phenomena such as outcome-specific craving (e.g. Konova et al., 2018), the dynamics of spiralling or relapse (e.g. Bornstein and Pickard, 2020), and alterations in processes such as planning and goal-directed choice (e.g. Hogarth, 2020; Simon and Daw, 2012). This is due to the fact that most model-free RL formulations rely on learning a scalar value for any given state or action that abstracts away information about specific forthcoming outcomes and uses cached past experiences rather than future goals for fast evaluation. Importantly, these models treat learning and unlearning as symmetric processes. Hence, although model-free RL formulations (and the habitual and/or dopaminergic systems they are associated with) are prominent in models of addiction, several authors have suggested that there is a need for additional theory development, and a complete account must involve model-based RL systems (Box 1), which rely on a predictive model of the world, hence retaining outcome sensitivity and being capable of flexible goal-directed planning (Epstein, 2020; Redish et al., 2008; Smith et al., 2021).

In this vein, a separate line of investigation has emerged that focuses on modeling the process of Bayesian inference of variables such as outcomes or actions under a predictive world model, and alterations of this process in addiction (Mollick & Kober, 2020; Smith et al., 2021). For instance, reduced behavioral and neural sensitivity to Bayesian prediction errors in a stop-signal task was shown to predict relapse in individuals with methamphetamine dependence (Harlé et al., 2019). Similarly, active-inference formulations (i.e., using a predictive model of preferred outcomes to infer the most likely actions) have demonstrated reduced sensitivity to unpleasant outcomes in substance use disorders (Smith et al., 2021). Prominent models of craving also rely on model-based planning (Redish & Johnson, 2007) or Bayesian inference accounts (Gu, 2018; Gu & Filbey, 2017).

In an attempt to reconcile the effects of addictive substances on both model-free and model-based systems, some “dual systems” accounts of addiction invoke interactions between the two systems, such as partial model-based evaluation terminating in model-free values (Simon & Daw, 2012), changes in the competition between the two systems (Fraser & Janak, 2019; Sebold et al., 2014; Smith et al., 2021; Vassileva et al., 2023), or changes in substrates common to both such as biased samples from memory during evaluation (Bornstein & Pickard, 2020) or replay (Simon & Daw, 2012) to account for craving, relapse and other seemingly goal-directed phenomena in addiction. Here we present a unified framework that offers explanations for several phenomena that challenge model-free formulations of addiction, and potentially bridges model-free and model-based explanations by reformulating the problem of reinforcement learning into one involving the discovery and inference of “latent causes”.

The latent-cause framework

A growing body of evidence has demonstrated the shortcomings of traditional model-free formulations of addiction, and has attempted to remedy them in multiple, seemingly distinct ways. However, one principle that emerges from these attempts and unifies them is the “contextual” or “latent-cause framework”. According to the latent-cause framework, individuals engaging in real world reinforcement learning are not simply learning unitary associations between reinforcers and their antecedent cues or actions (such as values or policies in traditional model-free RL), but are simultaneously discovering the hidden “contexts” or “latent causes” underlying both cues and reinforcing outcomes in the environment (Gershman et al., 2010; Gershman et al., 2015). This means that individuals are constantly segmenting their experiences into (an unknown number of) distinct latent causes, such that experiences only create or modify associations bound to the latent cause they are assigned to. Depending on the level of abstraction at which they are inferred, latent causes may correspond to different psychological constructs. These may be external to the individual - ranging from “categories” e.g. a desirable object to an entire “context” such as a stressful situation - or internal, from discrete phenomenological “states” such as craving to an entire self-concept e.g. the addicted version of me.

Formally, such a process of latent cause discovery can be captured using Bayesian nonparametric models (Gershman et al., 2010) with “infinite-capacity” priors that allow agents to flexibly create new latent causes in the face of unfamiliar or changing experiences, store old latent causes in memory, and infer their recurring presence when familiar experiences are encountered. Importantly, latent causes are unobserved, learnt in an unsupervised fashion, and beliefs about their inferred presence are heavily shaped by one’s priors and past experiences such that two individuals with differing histories may arrive at different latent cause structures underlying the same set of experienced cues and reinforcing outcomes (Pisupati et al., 2021). This flexibility in learnt latent structure sets these models apart from standard Bayesian inference accounts that typically assume a single, known latent structure, and allows these models to additionally capture the process of building a world model in the first place. Beyond addiction, such a framework has successfully accounted for a wide variety of learning phenomena in humans and animals (Gershman et al., 2015), and its neural underpinnings are beginning to be dissected in the prefrontal cortex and hippocampus (Boorman et al., 2021; Heald et al., 2023; Vaidya & Badre, 2022).

The latent-cause framework deviates from standard model-free RL in a number of key ways: First, it puts cues and outcomes on the same level, and learns latent causes associated with both (Figure 1a), hence retaining information about specific outcomes and enabling both cues and outcomes to trigger memories or expectancies of each other, and of old latent causes, any of which could act as potential substrates of outcome-specific craving (Konova et al., 2018; Redish & Johnson, 2007; Tiffany, 1999). Second, it allows for the segregation of associations, values or policies into multiple separate representations (one for each latent cause), enabling the learning of distinct (and even conflicting) associations for different contexts (Figure 1b). Hence, initial drug exposure could create learning of one set of associations, while later experiences such as spiraling into addiction, adverse outcomes, development of tolerance or abstinence from the drug could create a separate set of associations rather than modify the original association. The resultant protection of the original association leaves the individual vulnerable to relapse if and when the original latent cause is inferred to recur (Bornstein & Pickard, 2020; Redish et al., 2007). Similar ideas have been proposed and tested in the domain of fear learning and extinction (Dunsmoor et al., 2015) with applications to fear-related psychopathologies (Norbury et al., 2022). Finally, early protected latent causes could be sampled from memory in the future during decision evaluation or replay, leading to goal-directed behavior such as “chasing the first high” (Bornstein & Pickard, 2020) and fully deliberative, model-based control that is nevertheless insensitive to devaluation (Garrett et al., 2022). In the following two sections, we introduce and briefly examine the consequences of the latent-cause framework for Pavlovian and instrumental learning, and its potential to explain core addiction phenomena that is not well captured by pure model-free RL accounts.

Figure 1. The latent-cause framework and its utility in understanding craving and relapse.

Figure 1.

Open in a new tab

a-b. Differences between model-free RL (left column) and the latent-cause framework (right column). a. Outcome specificity of associations and craving states: Two outcome identities, O1 and O2, are paired with stimuli S1 and S2, respectively (middle). Left: Model-free RL abstracts away the outcome identities in favor of scalar values V (grey bars). Right: latent causes labelled C1 and C2 also represent abstractions, however, these are of both stimuli (cues) and outcomes, and retain outcome-specific expectations (colored bars) despite being “defocused” and generalizable, much like craving states. b. Asymmetry of learning and unlearning and vulnerability to relapse: In extinction, O1 that was initially paired with S1, no longer appears (middle). Left: In model-free RL, learning and unlearning are symmetric, such that the value of S1 first increases (when paired with O1) and then decreases in extinction (the first trial of extinction is marked by a dotted line). Right: In the latent-cause framework, the large change between acquisition and extinction can entail the formation of a new latent cause (pink) that is associated only with S1 and not with O1, leaving previously learnt knowledge (blue) intact, and therefore prone to relapse. c. Dynamics of latent-cause inference, craving and relapse: Simulation of the timecourse of latent-cause inference in reversal learning, showing potential connections to addiction phenomena. Early experience with an addictive substance leads to robust learning of drug associations and a strong policy of choosing the consumption action A1 (square), bound to a latent cause with strong expectations of reinforcing outcomes (blue). Following reversal (first dashed line), the act of consumption no longer gives rise to the same level of reinforcement, either due to tolerance (reduction in the hedonic value of the drug outcome) or other negative consequences of drug use, leading to abstinence and a drop in outcome expectations. This sudden change leads to the creation of a new latent cause (pink) that is not as strongly bound to the action policy or its outcome, and this new cause persists for some time. As time passes, exhaustion of this persistence or exposure to drug-related cues or contextual triggers increase the probability of inferring the return of the original drug-associated cause (blue). This leads to an increase in outcome expectations, resembling incubation of craving. Eventually, outcome expectations and/or craving may grow so large as to trigger a relapse event (second dashed line), resetting the cycle.

Pavlovian latent causes and craving

When applied to Pavlovian learning settings, the latent-cause framework has the potential to explain aspects of craving that model-free RL cannot. Since in this framework, individuals associate latent causes with both cues and reinforcing outcomes (rather than learn scalar values for cues in model-free RL), both cues and outcomes have the capacity to trigger inference of the associated latent cause, and consequently, expectations of the other stimulus. Importantly, since outcome information is retained (rather than abstracted away), these expectations are outcome-specific (Figure 1a). Latent states formed by drug-related cues and outcomes could potentially capture many properties of the subjective state of craving. The inferred, transient, presence of these latent states can be triggered by drug-associated or other contextual cues. In turn, the inferred states can give rise to outcome expectations and inflated values that are specific to both the drug outcome and the temporally restricted state of craving (Konova et al., 2018). Moreover, since latent causes represent groupings of cues, outcomes and any surrounding context, they are expected to generalize to stimuli that resemble those cues and outcomes. Such restricted similarity-based generalization of drug-related cues and outcomes has been observed in states of craving (Biernacki et al., 2022; Konova et al., 2018), and is particularly evident in polydrug users whose cravings for specific drugs can be triggered by historically drug-associated moods (Epstein et al., 2009). This is also akin to the “defocusing” of outcome representations proposed by Dayan and Berridge, 2014, which can enable the substitution of one addictive outcome with a similar one.

The dynamics of learning and inference about latent causes may help shed further light on temporal features of craving – for instance, “incubation of craving” (Grimm et al., 2001; Gu, 2018; Pickens et al., 2011), i.e. the phenomenon that craving reduces immediately after consumption but may increase steadily after a period of abstinence, with a characteristic timescale. Over the longer term, craving may continue to recur frequently (and with it the risk of relapse) or become more intermittent and even decrease, with longer adherence to abstinence. Such dynamics could potentially be captured by the distinct but linked timescales of inference and learning in the latent-cause framework (Figure 1c). After abrupt change points, one might infer that transient latent causes have become inactive. The anticipation of their short-term return may increase gradually with the probability of the current transient latent cause terminating (e.g., after a temporary period of abstinence) or increase abruptly in the presence of a drug-related cue, outcome or antecedent context (e.g. a stressful event, Gu, 2018). However, the long-term recurrence of these latent causes may be determined by learnt priors over latent causes, which in turn may depend on their “popularity” – a latent cause that has been sampled from memory more (or less) often in the past may be expected to return more (or less) often in the future (Bornstein & Pickard, 2020), acquiring a higher (or lower) learnt prior probability. Understanding individual differences in such priors and other components of learning and inference could potentially help disambiguate situations in which craving continues to recur and accelerate relapse from those in which it eventually reduces and leads to recovery (Bornstein & Pickard, 2020; Gu, 2018). It is worth noting here that while we are attempting to model the overall dynamics of craving in terms of an unobservable belief state similar to Gu, 2018, empirical measures of craving differ significantly between human (self-report) and rodent (responding in extinction) studies, and appropriate care will be needed in order to map our predictions about latent craving states onto these vastly different observable measures.

Instrumental latent causes and relapse

When applied to instrumental learning scenarios, the latent-cause framework can account for the asymmetric dynamics of learning and unlearning of drug-related associations, and the phenomenon of relapse (Bornstein & Pickard, 2020; Redish et al., 2007). Akin to the situation-recognition module proposed in Redish et al., 2007, the process of latent-cause inference has the potential to segregate learning during initial acquisition of a behavioral policy (Box 1) (e.g. initial drug consumption with large experienced payoffs) from learning during subsequent extinction or reversal of outcomes (e.g. abstinence, worsening payoffs due to tolerance or increasing costs), and can explain why both drug consumption and rehabilitation may be less likely to generalize outside the context they were learnt in (Robins et al., 1975). Segregation of different experiences to separate latent causes makes old associations resistant to unlearning or counter-evidence, and prone to relapse (Figure 1b). This can also explain behavior of “chasing the first high” despite the lack of recent experience of positive outcomes of drug consumption (Bornstein & Pickard, 2020), and accounts not only for the return of old policies (i.e. early drug consumption) but also their associated contextual memories (i.e. the first high). As in the Pavlovian case, such relapses (i.e., inference of resurgence of an old latent cause) can be triggered by drug-associated cues, passively re-experiencing drug outcomes (i.e. reinstatement, Song et al., 2022), or antecedent contexts (e.g. a stressful state that prompted initial consumption), and may be accompanied by outdated action-outcome expectations. Finally, latent causes associated with old policies may or may not be the same latent causes associated with outcomes, allowing for the possibility of craving and relapse to exist without each other (Redish, 2009), while still making one likely in the presence of the other – another potential axis of individual variability.

Latent causes as a bridge between model-free and model-based formulations

So far, we have considered the effect of latent-cause representations underlying values and policies that are still learnt through direct experience, similar to model-free associations. Going one step further, latent-causes could serve as building blocks of a predictive model of the world, acting as higher-order, partially observable state representations upon which model-based evaluation could take place. For instance, a drug-related cue could cause inference of a latent cause that effectively puts an individual into a state of craving or relapse, triggering either an old model-free habit, an outdated model-based evaluation (Garrett et al., 2022) or a goal to alleviate craving and a model-based search in pursuit of this goal (Redish & Johnson, 2007). In this way, the latent-cause framework acts as a bridge between model-free and model-based formulations, offering a common substrate for both habitual and goal-directed aspects of addiction (Redish, 2009; Redish et al., 2008; Simon & Daw, 2012). By bridging these model classes, the latent cause framework could potentially provide new links between results couched in terms of model-free RL and the growing body of work on the bidirectional influences of substance use on model-based planning and goal-directed decision making (Gueguen et al., 2021; Mollick & Kober, 2020; Smith et al., 2021).

Looking ahead: centering subjective experience

Beyond its ability to account for a number of features of addiction phenomena, the latent-cause framework shifts the emphasis away from experience-derived model-free learning that “takes control” over behavior and thus seemingly removes agency from the individual, instead putting the focus on subjective aspects of that experience and how it may interact with the past to influence decision-making in context (Gu et al., 2019; Kao et al., 2022; Pickard, 2018). Under this view, exteroceptive and interoceptive experiences are filtered through an individual’s model of latent causes in the world, which have been shaped by their past experiences, prior beliefs (both innate and learned) and goals. For instance, a person who expects – as a prior on how the world works – that latent causes are deterministic (also called a “black or white” cognitive bias, where events can be good or bad, safe or dangerous), would be more likely to infer segregated latent causes across conflicting experiences, which in turn can yield update-resistant beliefs or policies that may later resurface (Pisupati & Niv, 2022). Such prior expectations can be due to both innate tendencies and life experiences, and may comprise a self-reinforcing cycle as inference in light of prior beliefs can update prior expectations for the future. This account therefore leaves room for – and can help explain – individual differences in phenomenology and responding to the same set of drug-related or therapeutic experiences, due to differences in individual histories and innate tendencies. Moreover, it emphasizes the importance and therapeutic potential of samples from past memories as well as alternate imagined futures, in particular predicting that strengthening alternate drug-unrelated memories and futures is likely to be more successful at supporting recovery than attempting to erase or curb drug-related ones (Bornstein & Pickard, 2020).

Finally, the latent-cause framework can be applied to inference at many different levels of abstraction—all the way up to the individual’s own sense of self and agency, by treating the self-concept as a latent cause that groups together the characteristics, values and action policies of an individual to distinguish themselves from the environment. Individual differences in inferring self v.s. environmental latent causes could yield different (and even conflicting) narratives of identity in addiction, from individuals conceptualizing their self as a persistent whole that goes through addiction and possibly recovery, to split senses of self (an addicted and a non-addicted identity with conflicting values and policies), to addiction as being a cause that is entirely extraneous to identity (McIntosh & McKeganey, 2000; Pickard, 2021). The expressive power of this framework offers the potential for future work to attempt modeling complex, personalized phenomena in addiction within a single framework, all the while recognizing the insufficiency and limitations of unified narratives (Hammer et al., 2012; Pickard, 2018).

Acknowledgements:

We would like to thank Sama Ahmed, Nathaniel Daw, Gary Aston Jones, Diksha Gupta, Kirsten Smith, and David Epstein for helpful discussions and feedback on this work. This research was supported by NIDA grants R01DA054201 and R01DA042065, NIMH grant R01MH119511, a grant from the Templeton Foundation and the Intramural Research Program of the NIMH.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1

Note: We have deliberately eschewed the use of the word “disorder” here, in an effort to use non-pathologizing language around addiction, while acknowledging its disabling nature and recognizing that different individuals relate differently to their addiction

2

Such models often implicitly assume that this “aberration” lies within the brain, but many authors have questioned this view, arguing for agnosticism and/or careful consideration of social determinants e.g. Pickard, 2018. In our reformulation (similar to Bornstein and Pickard, 2020), we attempt to take a biopsychosocial view of a system operating as usual, but under unusual circumstances.

Declaration of interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References

  1. Ahn WY, Dai J, Vassileva J, Busemeyer JR, & Stout JC (2016). Computational modeling for addiction medicine: From cognitive models to clinical applications. Progress in brain research, 224, 53–65. [DOI] [PubMed] [Google Scholar]
  2. Biernacki K, Lopez-Guzman S, Messinger JC, Banavar NV, Rotrosen J, Glimcher PW, & Konova AB (2022). A neuroeconomic signature of opioid craving: How fluctuations in craving bias drug-related and nondrug-related value. Neuropsychopharmacology, 47 (8), 1440–1448. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Boorman ED, Sweigart SC, & Park SA (2021). Cognitive maps and novel inferences: A flexibility hierarchy. Current Opinion in Behavioral Sciences, 38, 141–149. [Google Scholar]
  4. Bornstein AM, & Pickard H (2020). “chasing the first high”: Memory sampling in drug choice. Neuropsychopharmacology, 45 (6), 907–915. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Dayan P. (2009). Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry, 42 (S 01), S56–S65. [DOI] [PubMed] [Google Scholar]
  6. Dayan P, & Berridge KC (2014). Model-based and model-free pavlovian reward learning: Revaluation, revision, and revelation. Cognitive, Affective, & Behavioral Neuroscience, 14 (2), 473–492. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Dezfouli A, Piray P, Keramati MM, Ekhtiari H, Lucas C, & Mokri A (2009). A neurocomputational model for cocaine addiction. Neural computation, 21 (10), 2869–2893. [DOI] [PubMed] [Google Scholar]
  8. Dunsmoor JE, Niv Y, Daw N, & Phelps EA (2015). Rethinking extinction. Neuron, 88 (1), 47–63. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Epstein DH (2020). Let’s agree to agree: A comment on hogarth (2020), with a plea for not-so-competing theories of addiction. Neuropsychopharmacology, 45 (5), 715–716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Epstein DH, Willner-Reid J, Vahabzadeh M, Mezghanni M, Lin J-L, & Preston KL (2009). Real-time electronic diary reports of cue exposure and mood in the hours before cocaine and heroin craving and use. Archives of general psychiatry, 66 (1), 88–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Fraser KM, & Janak PH (2019). How does drug use shift the balance between model-based and model-free control of decision making? Biological psychiatry, 85 (11), 886–888. [DOI] [PubMed] [Google Scholar]
  12. Garrett N, Allan S, & Daw ND (2022). Model based control can give rise to devaluation insensitive choice. bioRxiv. [Google Scholar]
  13. Gershman SJ, Blei DM, & Niv Y (2010). Context, learning, and extinction. Psychological review, 117 (1), 197. [DOI] [PubMed] [Google Scholar]
  14. Gershman SJ, Norman KA, & Niv Y (2015). Discovering latent causes in reinforcement learning. Current Opinion in Behavioral Sciences, 5, 43–50. [Google Scholar]
  15. Grimm JW, Hope BT, Wise RA, & Shaham Y (2001). Incubation of cocaine craving after withdrawal. Nature, 412 (6843), 141–142. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Gu X. (2018). Incubation of craving: A bayesian account. Neuropsychopharmacology, 43 (12), 2337–2339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gu X, & Filbey F (2017). A bayesian observer model of drug craving. JAMA psychiatry, 74 (4), 419–420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Gu X, FitzGerald TH, & Friston KJ (2019). Modeling subjective belief states in computational psychiatry: Interoceptive inference as a candidate framework. Psychopharmacology, 236 (8), 2405–2412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gueguen MC, Schweitzer EM, & Konova AB (2021). Computational theory-driven studies of reinforcement learning and decision-making in addiction: What have we learned? Current opinion in behavioral sciences, 38, 40–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gutkin B, & Ahmed SH (2011). Computational neuroscience of drug addiction (Vol. 10). Springer Science & Business Media. [Google Scholar]
  21. Hammer RR, Dingel MJ, Ostergren JE, Nowakowski KE, & Koenig BA (2012). The experience of addiction as told by the addicted: Incorporating biological understandings into self-story. Culture, Medicine, and Psychiatry, 36 (4), 712–734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Harlé KM, Angela JY, & Paulus MP (2019). Bayesian computational markers of relapse in methamphetamine dependence. NeuroImage: Clinical, 22, 101794. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Heald JB, Wolpert DM, & Lengyel M (2023). The computational and neural bases of context-dependent learning. Annual Review of Neuroscience, 46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hogarth L. (2020). Addiction is driven by excessive goal-directed drug choice under negative affect: Translational critique of habit and compulsion theory. Neuropsychopharmacology, 45 (5), 720–735. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Huys QJ, Deserno L, Obermayer K, Schlagenhauf F, & Heinz A (2016). Model-free temporal-difference learning and dopamine in alcohol dependence: Examining concepts from theory and animals in human imaging. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 1 (5), 401–410. [DOI] [PubMed] [Google Scholar]
  26. Kao C-H, Feng G, Hur JK, Jarvis H, & Rutledge R (2022). Computational models of subjective feelings in psychiatry. 10.31234/osf.io/kq8vf. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Keramati M, Dezfouli A, & Piray P (2012). Understanding addiction as a pathological state of multiple decision making processes: A neurocomputational perspective. In Computational neuroscience of drug addiction (pp. 205–233). Springer. [Google Scholar]
  28. Konova AB, Louie K, & Glimcher PW (2018). The computational form of craving is a selective multiplication of economic value. Proceedings of the National Academy of Sciences, 115 (16), 4122–4127. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kurth-Nelson Z, & Redish AD (2012). Modeling decision-making systems in addiction. In Computational neuroscience of drug addiction (pp. 163–187). Springer. [Google Scholar]
  30. Lim TV, & Ersche KD (2023). Theory-driven computational models of drug addiction in humans: Fruitful or futile? Addiction Neuroscience, 5, 100066. [Google Scholar]
  31. McIntosh J, & McKeganey N (2000). Addicts’ narratives of recovery from drug use:Constructing a non-addict identity. Social science & medicine, 50 (10), 1501–1510. [DOI] [PubMed] [Google Scholar]
  32. Mollick JA, & Kober H (2020). Computational models of drug use and addiction: A review. Journal of abnormal psychology, 129 (6), 544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Norbury A, Brinkman H, Kowalchyk M, Monti E, Pietrzak RH, Schiller D, & Feder A (2022). Latent cause inference during extinction learning in trauma-exposed individuals with and without ptsd. Psychological Medicine, 52 (16), 3834–3845. [DOI] [PubMed] [Google Scholar]
  34. Pickard H. (2018). The puzzle of addiction. In The routledge handbook of philosophy and science of addiction (pp. 9–22). Routledge. [Google Scholar]
  35. Pickard H. (2021). Addiction and the self. Noûs, 55 (4), 737–761. [Google Scholar]
  36. Pickens CL, Airavaara M, Theberge F, Fanous S, Hope BT, & Shaham Y (2011). Neurobiology of the incubation of drug craving. Trends in neurosciences, 34 (8), 411–420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Piray P, Keramati MM, Dezfouli A, Lucas C, & Mokri A (2010). Individual differences in nucleus accumbens dopamine receptors predict development of addiction-like behavior: A computational approach. Neural computation, 22 (9), 2334–2368. [DOI] [PubMed] [Google Scholar]
  38. Pisupati S, Langdon A, & Niv Y (2021). Two factors underlying maladaptive inference of causal structure can drive resistance to extinction in anxiety. Biological Psychiatry, 89 (9), S283. [Google Scholar]
  39. Pisupati S, & Niv Y (2022). Why do some beliefs and action policies resist updating? The 5th Multidisciplinary Conference on Reinforcement Learning and Decision Making, P495. https://rldm.org/wp-content/uploads/2022/06/RLDM2022_cameraready.pdf. [Google Scholar]
  40. Redish AD (2004). Addiction as a computational process gone awry. Science, 306 (5703), 1944–1947. [DOI] [PubMed] [Google Scholar]
  41. Redish AD (2009). Implications of the multiple-vulnerabilities theory of addiction for craving and relapse. Addiction, 104 (11), 1940–1941. [DOI] [PubMed] [Google Scholar]
  42. Redish AD, Jensen S, & Johnson A (2008). Addiction as vulnerabilities in the decision process. Behavioral and brain sciences, 31 (4), 461–487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Redish AD, Jensen S, Johnson A, & Kurth-Nelson Z (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling. Psychological review, 114 (3), 784. [DOI] [PubMed] [Google Scholar]
  44. Redish AD, & Johnson A (2007). A computational model of craving and obsession. [DOI] [PubMed] [Google Scholar]
  45. Robins LN, Helzer JE, & Davis DH (1975). Narcotic use in southeast asia and afterward: An interview study of 898 vietnam returnees. Archives of General Psychiatry, 32 (8), 955–961. [DOI] [PubMed] [Google Scholar]
  46. Sebold M, Deserno L, Nebe S, Schad DJ, Garbusow M, Hägele C, Keller J, Jünger E, Kathmann N, Smolka M, et al. (2014). Model-based and model-free decisions in alcohol dependence. Neuropsychobiology, 70 (2), 122–131. [DOI] [PubMed] [Google Scholar]
  47. Simon DA, & Daw ND (2012). Dual-system learning models and drugs of abuse. In Computational neuroscience of drug addiction (pp. 145–161). Springer. [Google Scholar]
  48. Smith R, Taylor S, & Bilek E (2021). Computational mechanisms of addiction: Recent evidence and its relevance to addiction medicine. Current Addiction Reports, 1–11. [Google Scholar]
  49. Song M, Jones CE, Monfils M-H, & Niv Y (2022). Explaining the effectiveness of fear extinction through latent-cause inference. arXiv preprint arXiv:2205.04670. [Google Scholar]
  50. Takahashi Y, Schoenbaum G, & Niv Y (2008). Silencing the critics: Understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model. Frontiers in neuroscience, 14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Tiffany ST (1999). Cognitive concepts of craving. Alcohol Research & Health, 23 (3), 215. [PMC free article] [PubMed] [Google Scholar]
  52. Vaidya AR, & Badre D (2022). Abstract task representations for inference and control. Trends in Cognitive Sciences. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Vassileva J, Lee J-H, Psederska E, & Ahn W-Y (2023). Utility of computational approaches for precision psychiatry: Applications to substance use disorders. In Computational neuroscience (pp. 211–231). Springer. [Google Scholar]

RESOURCES