A Bayesian Observer Model of Drug Craving

The reinforcement learning theory of drug addiction has been influential for decades.¹ Under this framework, drugs trigger dopamine (DA) release and exert their reinforcing effects through the mesolimbic pathway.¹ This model can account for compulsive drug-seeking behaviors in addiction. However, many empirical findings remain unaddressed. In particular, accumulating evidence suggests that DA, delivered by either addictive drugs² or by pharmacological treatments, such as the nicotine replacement therapy,³ is not sufficient to reduce craving. Craving persists even after compulsive drug-taking behavior stops,³ suggesting that craving and drug-seeking behavior are 2 distinct processes, despite the fact that they are often homogenized in laboratory settings.

Drug craving is one of the strongest predictors of return to use and is much more resistant to treatment compared with physical dependence. Distinct from drug-taking behavior, the subjectivity of craving makes it difficult to measure. Although initial attempts have been made,⁴ these results are largely correlational, and the controversy over the scientific investigation of craving remains. Specifically, these controversies center on the discrepancy between the subjectivity of craving and objective measures used in laboratories such as the cue exposure paradigm.^5–7 Several studies demonstrate that cue reactivity assessed by experimental craving paradigms does not correlate with subjective craving in smokers,⁵ marijuana users,⁶ or heavy-drinking individuals.⁷ This calls for the urgent need to develop a formal model of craving.

Bayesian models become a natural candidate to account for drug craving for several reasons. First, by taking into account the probabilistic and uncertain nature of the world, the Bayesian approach provides great explanatory power for a wide spectrum of cognitive and neural phenomena that cannot be explained otherwise (eg, reinforcement learning models)⁸: posterior probability = (likelihood × prior probability)/(marginal likelihood).

Converging evidence suggests that the human brain is able to perform Bayesian optimization of its model of the world and actively predicts and explains its sensations.⁸ In this context, the prior is our belief based on past experience, the likelihood is the probability of sensory data, and the posterior is our updated belief given the observed sensory evidence. Second, because our belief can be cast in terms of posterior probability, given our prior belief and sensory evidence, Bayesian models can well account for false beliefs,² a key deficit in many psychiatric disorders such as drug addiction and schizophrenia. Furthermore, the Bayesian approach can link disrupted neurochemistry to beliefs. Dopamine, the key neurotransmitter involved in addiction and many other psychiatric disorders, has been suggested to encode the uncertainty or precision of our beliefs and sensory data.⁸ Dopamine neurons increase firing in response to unexpected stimuli but not to expected stimuli even when they are rewarding.⁸ In other words, DA activity does not encode reward per se, but rather its variance.⁸

Here, we propose that subjective craving can be formally described in Bayesian terms as the posterior belief of the physiological state of the body, and persistent drug craving can be considered as a failure in Bayesian updating of bodily states owing to low uncertainty or high precision caused by high levels of DA (Figure, A). This model assumes that (1) individuals have prior knowledge of their own bodily states and update this representation continuously, and (2) there are 2 separate mechanisms underlying the representation of the body’s physiological states and its evaluation. Neurobiologically, DA modulates the synaptic efficacy and neural gain of the postsynaptic cell. Chronic drug use can therefore elevate the precision or reduce uncertainty of beliefs and sensory data and lead to aberrant updating of beliefs via increased synaptic DA levels and neural gain.⁸ This model provides experimentally testable predictions such as the following:

• Compared with nonaddicted individuals, addicted individuals have greater confidence in their estimation of bodily states (eg, sharper posterior as shown in the red line in Figure A).

• When deprived of drugs short term, addicted individuals will have a prior belief that is shifted toward more discomfort and more craving, resulting in the posterior belief of increased craving.

• Craving can be reduced to the lowest level if the addicted individual receives the desired drug and also anticipated the drug initially (Figure C, lower right panel).

• Heterogeneity in prior beliefs and expectations, subserved by individual differences in either neurobiology (eg, DA receptor availability) or sociopsychological factors (eg, peer influence), could explain why not all drug users become addicted.

• Therapies that correct maladaptive beliefs (eg, cognitive behavioral therapy) should be effective in reducing craving.

Figure. A Bayesian Observer Model of Drug Craving.

A, Compared with the nonaddicted state, addiction is marked by abnormally low uncertainty/high precision owing to increased dopamine (DA), resulting in failure in updating the posterior estimate of the physiological state of the body. B, Empirical data suggest that nicotine itself is not sufficient to reduce craving in smokers. Craving is reduced only when smokers expect to receive nicotine and when nicotine is delivered. Adapted from Figure 2 in Gu et al.² C, Bayesian simulation of the expectancy effect shown in B. Posterior craving is reduced to the lowest level when a drug is administered and when the individual expects the drug (lower right panel) compared with the other 3 conditions.

Some of these predictions have been verified by empirical findings. For instance, data suggest that cigarette craving was reduced the lowest level when smokers had a cigarette with nicotine and believed it contained nicotine, but not otherwise (Figure B),² validating prediction 3. This finding cannot be explained by reinforcement learning theories of addiction such as the incentive salience hypothesis,¹ which suggests that craving is caused by the oversensitivity of incentive salience attributed to drugs and should only depend on the availability of the desired drugs.

The idea of a Bayesian observer is also closely related to the concepts of metacognition and insight, which describe the ability to monitor one’s own psychophysiological states. The metacognitive framework separates an individual into 2 entities: one responsible for processing self-relevant physiological information (“object” level) and the other one for evaluating the object level (“meta” level).⁹ The Bayesian observer is directly related to the meta level of processing. Neuroimaging studies suggest that metacognitive processes are mostly implemented in the lateral prefrontal cortex.⁹ Coincidentally, evidence suggests that impaired insight, accompanied by prefrontal abnormalities, could be a hallmark of drug addiction.¹⁰ In this sense, the proposed Bayesian model also provides a mechanistic account for deficits in meta-cognition and insights in drug addiction.¹⁰

In summary, by proposing a Bayesian model of craving, we sketch a potential way to disentangle craving and choice behavior in addiction research. The implication of this model is far-reaching because this framework can be used to account for a wide range of subjective states in other psychiatric disorders such as food craving in binge eating disorder.