Investigating the computational underpinnings of addiction

Addiction is associated with inflexible, rigid decision making that may, in part, underlie the compulsive patterns of drug use that are characteristic of the disorder. One of the most widely used laboratory tasks for quantifying flexible decision making is the reversal-learning task, whereby subjects are required to adaptively adjust their choices in response to changes in reinforcement contingencies. Substance-dependent individuals and laboratory animals exposed to drugs of abuse have difficulties adjusting their choices in reversal-learning tasks [1, 2], but the precise mechanism contributing to these drug-induced, decision-making deficits is unknown. Inflexible decision making may be due to deficits in distinct components of learning, such as outcome encoding or value updating that are not readily observable in gross measures of performance, and identifying the precise behavioral system that is altered by long-term drug exposure may provide critical insights into the pathology of addiction.

Reinforcement-learning algorithms have been used to quantify the degree that individual reinforcement-learning computations influence choice, and, over the past 20 years, have been frequently applied to studies of mental illness in several psychiatric populations. The integration of these approaches in addiction research and, importantly, in several recent laboratory animal studies of addiction provide a key translational link for addiction research. Utilization of computational approaches in animal models of addiction could provide the mechanistic bridge between biology and maladaptive behaviors in addiction that is critically needed for developing new and effective treatments and/or preventions for addiction.

In their recent Neuropsychopharmacology article, Zhukovsky and colleagues [3] used a computational approach to interrogate the reinforcement-learning mechanisms that are altered following cocaine self-administration in rats. Zhukovsky et al. first trained rats on a two-choice, spatial discrimination task whereby responses in one of the two locations was reinforced with delivery of food pellet. Once the rats were reliably choosing the port associated with reward, the ability to adjust their choices in response to changes in the reinforcement contingencies (i.e., reversal learning) was assessed the following day. Zhukovsky et al. also assessed anxiety-like behaviors in these same rats using an open-field arena, as they have previously reported that differences in anxiety-like behaviors predicts subsequent drug-taking behaviors [4]. Rats were then trained to make a lever response to earn an intravenous cocaine infusion in short (1 h) and long-access (6 h) sessions across 13 days to model the escalation in drug use that is characteristic of addiction. Performance in the reversal-learning task was reassessed after a brief period of forced abstinence and ex vivo gene expression of dopamine and serotonin receptors quantified.

The main finding of Zhukovsky et al. [3] is that higher rates of escalation of cocaine-taking behaviors were associated in the reversal-learning task with an increase in the degree of exploration (e.g., β–inverse temperature parameter), and an increase in the probability of repeating an unrewarded choice following cocaine self-administration. In contrast, the probability that the rats would repeat a rewarded choice was not affected, indicating that higher rates of escalation in cocaine use was associated with a selective deficit in their ability to integrate negative outcomes into their choice. This finding adds to a growing body of work indicating that drug use is associated with maladaptive choices in response to negative outcomes observed in humans and non-humans exposed to drugs of abuse [5–7]. Deficits in negative feedback utilization may be a fundamental reinforcement-learning mechanism underlying the maladaptive choices made by substance-dependent individuals: characteristic of the persistent use of drugs regardless of the negative consequences experienced and/or the failure of punishment-mediated treatment of addiction. The mechanisms that contribute to drug-induced disruptions in negative feedback utilization, however, are unknown. Zhukovsky et al. provide some evidence implicating the 5-HT2C receptor. This is a particularly interesting target as the 5-HT2C agonist, lorcaserin, has been approved by the FDA for the treatment of obesity and proposed as a potential therapeutic for addiction [8]. Furthermore, these findings argue against the hypothesis that addiction is solely a reward deficiency syndrome, but rather integrate a computational aspect of addiction whereby decisions are no longer modulated by negative reinforcement.

An additional important finding of the Zhukovsky et al. study is that differences in the rate of escalation in cocaine-taking behaviors were predicted by pre-existing differences in anxiety-like behaviors: rats with pre-existing high levels of anxiety in the open arena test had higher escalation rates compared to rats with low levels of anxiety. This predictive relationship was selective to anxiety-like measures, as pre-existing differences in reversal learning performance did not predict escalation rates. Given that higher rates of escalation during adolescence are associated with an increased risk for developing alcohol and drug problems later in life [9], these findings suggest that measures of anxiety could serve as predictive biomarkers of addiction vulnerability in drug-naïve individuals, and may be particularly useful for identifying individuals who are at high risk for developing addiction. Many prevention-based strategies for addiction (e.g., family or community-based programs) are initiated after an individual begins using drugs of abuse, when substantial neural and behavioral adaptations may have occurred. The results of Zhukovsky et al. raise the possibility of implementing prevention-based strategies in high-risk individuals prior to any drug use when these strategies may be most effective. Moreover, given that impulsivity and anxiety have both been shown to predict cocaine self-administration, determining how these phenotypes interact within the same individual to influence drug-taking behaviors, and subsequent addiction-relevant behaviors (e.g., motivation and relapse to cocaine use) could aid in the development of personalized prevention strategies for addiction.

In conclusion, computational approaches in animal models of addiction, such as those used by Zhukovsky et al., can provide substantial insights into the behavioral consequences of addiction that can be directly translated to findings observed in substance-dependent humans. As these computational strategies are expanded and refined in future rodent studies – where individual reinforcement-learning processes can be causally linked to specific molecular and/or circuit-level systems – we may be able to identify more precise therapeutic targets for the prevention and treatment of addiction.

Funding and disclosure

SMG is funded by NIH/NIDA grants (R01 DA043443 and R01 DA041480) and a Young Investigator Award through the Brain and Behavior Research Foundation. The author declares no competing interests.