Computational mechanisms of addiction and anxiety: a developmental perspective

Abstract

A central goal of computational psychiatry is to identify systematic relations between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. The vast majority of psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Introduction

During adolescence, individuals undergo pronounced social, sexual, and intellectual changes, with the adolescent brain exhibiting corresponding structural and functional changes to adapt to these new demands (1). Adolescence is also a time of increased vulnerability to a wide array of mental disorders including anxiety, substance use disorders, mood disorders, psychosis, eating disorders, and personality disorders (2). Indeed, twenty percent of adolescents develop a mental disorder that persists into adulthood (3), with early emergence associated with greater clinical severity of mental illness (4,5). Despite widespread recognition that most forms of psychopathology can be conceptualized as developmental disorders (2), a mechanistic account of adolescents’ psychiatric vulnerability has proven elusive. Recent theoretical proposals argue that focusing on learning – the process through which experiences modify subsequent behavior – may be critical for achieving a more mechanistic understanding of the emergence of psychopathologies (6).

The field of computational psychiatry leverages computational methods in order to better understand and treat psychiatric symptomatology (7). A primary goal of the field of computational psychiatry is to identify latent learning processes that underpin clinically relevant dimensions of behavior (8,9). A central premise of this approach is that psychiatric disorders can be conceptualized as constellations of transdiagnostic behavioral phenotypes that reflect latent neurocognitive computations (10). A growing literature in adults has identified systematic relations between specific psychiatric symptom dimensions and latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices (8,11). The extension of this approach to study the development of psychopathology is only in its nascent stages, with initial theoretical efforts delineating putative computational mechanisms underlying the development of autism (12) and obsessive-compulsive disorder (OCD) (13), and a few studies successfully linking learning phenotypes to clinical symptoms in developmental populations (14,15). Despite the sparsity of empirical findings in developmental computational psychiatry, there may be value in integrating findings in adults that link specific learning computations to psychiatric symptom dimensions, with the growing body of research that has begun to characterize the normative development of these learning phenotypes using computational approaches (16). Here, we lay out a roadmap for future research leveraging such computational approaches to study the development of addiction and anxiety – two classes of disorders that have a clear peri-adolescent developmental trajectory (17), significant comorbidity, overlapping transdiagnostic symptomatology (18), and are proposed to involve aberrations in learning computations (19,20).

Addiction is a chronic, relapsing condition that typically begins during adolescence, characterized by cycles of craving, intoxication, binging, and withdrawal (21). Over forty percent of teens experiment with drugs (22) and while the majority reach adulthood without developing substance-use disorders, for some, experimentation rapidly progresses into problem use (23). Anxiety is “a future-oriented mood state associated with preparation for possible, upcoming negative events” (distinct from fear, an alarm response to present or imminent danger) (24). Anxiety disorders can be subdivided into conditions that involve heightened threat response, such as panic disorder, phobias, or posttraumatic stress disorder (PTSD), and conditions that involve increased worry and rumination, such as generalized anxiety (24). Anxiety has a lifetime prevalence of over thirty percent (25), with typical emergence between early school age and adolescence, and the majority of diagnoses made prior to young adulthood (17). Addiction and anxiety exhibit symptomatic overlap (see Figure 1.A) which may stem, in part, from common underlying learning mechanisms (see Figure 1.B). This review focuses on two learning computations that are implicated in both disorders, and exhibit developmental changes across adolescence: model-based control of action and Pavlovian learning processes (additional learning computations are discussed in Box 1). For each learning process, we detail its normative developmental trajectory and review studies in adults highlighting its roles in addiction and anxiety. We then discuss potential relations between developmental changes in computational phenotypes and symptom expression (see Figure 1.C for graphical illustration, and Table 1 for a summary of key empirical findings and theory-based hypotheses).

Figure 1. Development of computational phenotypes and psychiatric symptom expression.

A. Visual representation of the relation between transdiagnostic symptoms and psychiatric disorders. Each individual is represented as a rectangular mosaic, reflecting a specific constellation of symptoms and a corresponding categorical psychiatric diagnosis. B. Hypothesized relations between transdiagnostic symptom expression and computational phenotypes. Each point represents an individual’s position within a three-dimensional space defined by the three computational phenotypes of interest. Symptom expression may relate to location in the phenotypic space. For example, compulsivity is associated with reduced model-based control (e.g., (54,55) and rumination with increased model-based control (68). Increased avoidant behavior and compulsivity are both associated with greater Pavlovian-instrumental transfer (89-93,98). Finally, persistent threat response is associated with a tendency to infer multiple latent states, whereas overgeneralization is associated with a tendency to infer fewer states (114). Note that the proposed relations between regions of the multidimensional phenotypic space and transdiagnostic symptom expression are speculative. C. Potential relations between the development of computational phenotypes and expression of psychiatric symptoms. I. A computational phenotype may exhibit age-related changes (e.g., increases in model-based control with age) and the strength of that phenotype may relate to transdiagnostic symptom expression (e.g., decreased model-based control is associated with a heightened propensity to engage in compulsive behavior). II. A computational phenotype and the probability of an environmental exposure both change with age, and may interact to create a “high-risk zone” (bottom right quadrant of the graph) where individuals have a higher probability to develop symptoms. For example, both reliance on model-based control and the probability of drug exposure increase with age. Individuals with reduced model-based control and greater exposure to drugs may be at greater risk for developing compulsive drug consumption. Here, we assume a linear increase in environmental exposure with age, but other non-linear changes are possible as well (for example, exposure to alcohol may increase rapidly at the legal drinking age). III. Exposure to environmental conditions may alter the normative development of a computational phenotype. For example, individuals with heightened drug consumption might show a reduced developmental increase in model-based control, making them more vulnerable to the emergence of compulsive drug use. Here, we assume an age-invariant effect of environmental exposure on the developmental change in the computational phenotype, but age-specific windows of environmental influence (i.e., sensitive periods) are also possible. IV. Age-related changes in a constellation of specific computational phenotypes may yield windows of vulnerability to increased symptomatic expression. For example, increases in model-based control during adolescence could interact with adolescent-specific increases in negative valence bias to promote greater anxious rumination. Note that these examples are speculative illustrations of multiple potential forms of developmental vulnerabilities that could be examined in future computational developmental psychiatry research.

Box 1: Additional computational phenotypes of interest.

Valence asymmetries

The relative weight individuals place on positive versus negative outcomes may confer risk or resilience to psychopathology. Whereas healthy adults tend to overweight recent positive experiences relative to negative experiences (143,144), anxious individuals show the opposite tendency (145). Recent computational work assessing developmental changes in valence asymmetries without confounding task demands (16,146) has observed that adolescents overweight negative, relative to positive, outcomes in learning and in memory, more than children or adults (147-150). Such a tendency might contribute to internalizing disorders through increased encoding of negative events (151).

Adaptation of learning rates

Adapting one’s learning to dynamic environmental statistics allows for the optimization of behavior (152). Anxious individuals are less able to dynamically adjust learning rates in volatile environments (138,139,153,154), which may contribute to their difficulties coping with conditions of high uncertainty. Few studies have examined developmental change in the optimal adjustment of learning rates, though recent work suggests that the ability to adapt valenced learning rates may improve with age (155).

Metacognition

Systematic distortions in metacognition – the ability to reflect on and evaluate one’s behavior (156) – may be a transdiagnostic factor associated with poor mental health (157). A tendency to be overconfident in one’s performance (irrespective of the actual performance) is associated with heightened compulsivity, while a tendency toward underconfidence is associated with anxiety-depression symptoms (158). Metacognitive abilities improve from childhood through adolescence (159,160). Future studies should examine the developmental emergence of these metacognitive distortions, and how they may relate to the progression of anxiety symptomatology and addictive behaviors (161,162).

Table 1:

Empirical findings and theory-based hypotheses linking the development of specific computational phenotypes to the symptomatology of addiction and anxiety. 𝔼A-Empirical research conducted in animal models. 𝔼H-Empirical research conducted with human subjects. 𝕋- Theory or a review paper.

Computational phenotype	Relation to addiction	Relation to anxiety	Developmental findings	Hypothesized mechanisms of developmental vulnerability to psychopathology
Model-based control	Drug exposure decreases model-based control. 𝔼A(41-44) 𝕋(38) Model-based control is reduced in drug-dependent individuals. 𝔼H(46-50) 𝕋(38) Reduced model-based control is associated with greater propensity toward compulsive behavior. 𝔼H (54,55) Reduced model-based control predicts the emergence of compulsive drug consumption. 𝔼A(45) 𝔼H(58)	Worry and rumination may depend in part on model-based simulation processes. 𝔼H(68) 𝕋(65)	Reliance on model-based control increases with age. 𝔼H(15,29,32-36) 𝕋(28)	Vulnerability to developing addiction may be greater at younger ages due to reduced model-based control. Drug exposure during development might attenuate the normative development of model-based control. Age-related improvement in mental simulation abilities might lay the cognitive foundation for heightened worry and rumination (in interaction with negative valence biases in information processing).
Pavlovian processes
Pavlovian-instrumental transfer	Pavlovian-instrumental transfer is greater in drug-dependent individuals and high-risk drinkers. 𝔼H(89-93) Drug exposure increases Pavlovian-instrumental transfer. 𝔼A(84-88)	Anxious individuals exhibit greater Pavlovian-instrumental transfer.𝔼H(98,99)	Pavlovian-instrumental transfer decreases from childhood into adolescence 𝔼H(100) and may stabilize from adolescence into adulthood 𝔼H(100,101).	Developmental changes in Pavlovian-instrumental transfer may modulate the influence of valenced environmental exposures on symptom expression (e.g. cue-induced craving/drug seeking).
Latent state inference		Alteration in latent state inference relates to anxious symptomatology.𝔼H(117,118) Propensity to infer multiple latent states is associated with extinction-resistance of threat associations 𝔼H(114).	The tendency to infer distinct latent states based on shifts in environmental statistics may increase with age 𝕋(113).	Anxiety in younger individuals might reflect difficulty discriminating between threat and safety states due to a tendency to infer fewer latent states. At later ages, anxiety might reflect extinction-resistance of threat associations due to a tendency to infer multiple latent states.

Model-based control and its development

Two classes of algorithms can guide instrumental action selection: A “model-based” system evaluates actions by simulating their potential outcomes using a cognitive model of environmental states, which specifies how one transitions between states and the outcomes associated with each state; in contrast, a “model-free” system instead estimates and stores the mean values of actions based on previously experienced rewards and punishments (26). These two evaluative processes have different advantages and shortcomings. Model-free value computation, a process proposed to give rise to habitual actions (27), enables rapid, reflexive repetition of previously rewarded actions, but is not immediately sensitive to changes in current goals or action-outcome contingencies. In contrast, model-based evaluation is a slower but more flexible deliberative process, which confers greater sensitivity to environmental changes and supports counterfactual reasoning.

Convergent cross-species findings in diverse tasks suggest that reliance on model-based control increases with age (28). A common assay of model-based control is outcome devaluation. In this paradigm, individuals first learn to perform a rewarded action, which then gets devalued. For example, by allowing the individual to consume the reward until sated. An individual who learns and updates action-outcome relations will reduce performance of an action that now leads to undesirable reward, reflecting goal-directed consideration of the expected outcome (e.g., open the fridge only if I currently want food), rather than a habitual response based on a cached action value (e.g., open the fridge because it is a good thing to do). In a study examining outcome devaluation in one- to four-year-old children, sensitivity to devaluation of video clips increased with age (29). A study in adolescent and adult rodents found both age groups exhibited sensitivity to devaluation of food reward outcomes, but only adult rodents exhibited sensitivity to degradation of the action-outcome contingency, a manipulation that probes the sensitivity of actions to the strength of causal relation between action and outcome (30). These results suggest that these components of model-based cognition exhibit unique developmental trajectories. Model-based behavior can also be assessed in sequential decision-making tasks that index the degree to which an individual uses a mental model of the task structure to maximize reward, rather than simply repeating previously rewarded actions (e.g. the “two-step task” (31)). Several studies using such tasks in human participants have found that model-based control of action increases with age from middle childhood through young adulthood (32-36), an effect also evident within individuals longitudinally across adolescence (15). Moreover, while adults increase their recruitment of model-based control when it is incentivized (37), this capacity for “meta-control” is absent in children (32) and may increase gradually across adolescence into adulthood (33). Collectively, these findings suggest that model-based control increases across development, and that the age at which model-based control is recruited may depend upon the complexity of learning, updating, and planning with a mental model in a particular environment.

Reduced model-based control and the development of addiction

Reduced model-based control and increased dominance of model-free evaluation has been hypothesized to play a mechanistic role in the emergence of addiction. The progression of addiction typically involves a narrowing of goals to focus on drug seeking and consumption, even despite adverse consequences. Drug consumption is proposed to reflect, in part, a shift from goal-directed action, guided by positive hedonic consequences, to habitual, compulsive behavior that is insensitive to negative outcomes (38) (although there are compelling alternative accounts (39,40)).

Cross-species evidence corroborates theoretical proposals that habitual drug use is associated with decreased model-based control. Self-administration of alcohol in rodents that is initially sensitive to outcome devaluation becomes increasingly insensitive with prolonged consumption (41). Animals exposed to cocaine or methamphetamine similarly exhibit decreasing sensitivity to devaluation (42-44), suggesting drug consumption decreases model-based control of action. Notably, the degree to which drug exposure reduces model-based control may predict subsequent escalation of drug seeking and use (45). Relative to control participants, both alcohol- (46) and cocaine-dependent (47) patients exhibit decreased sensitivity to devaluation of actions associated with monetary rewards. Human studies using sequential decision-making tasks have similarly observed decreased model-based control in alcohol misuse and dependence (48,49) and various substance use disorders (50), as well as in gambling addiction (51). These convergent findings (but see (52,53)) suggest that reduced model-based control is associated with an increased propensity toward compulsive behavior (54,55), a transdiagnostic behavioral dimension characteristic of diverse psychiatric disorders.

Decreased engagement of model-based control at younger ages may confer a heightened propensity toward compulsivity that increases vulnerability to addiction (for a graphical depiction of this hypothesis see Figure 1.C.I). Indeed, earlier initiation of substance use predicts a greater likelihood of dependence (5), a more rapid progression to addiction (56), as well as greater addiction severity (57). Heightened compulsivity may increase the propensity for escalation of substance use during developmental periods in which experimentation is common. Indeed, model-based control amongst adolescents and young adults is associated with heightened subclinical compulsivity (15), and decreased model-based control at age 18 predicts greater increases in binge drinking behavior over a three-year period (58). Notably, while low model-based control at a given age may heighten vulnerability to addiction following drug experimentation, greater model-based control may confer resilience (Figure 1.C.II). Conversely, drug experimentation may also influence the longitudinal developmental trajectory of model-based control. Future studies should examine whether there are sensitive developmental periods during which drug exposure is particularly consequential (see Figure 1.C.III).

Heightened compulsivity in children is typically viewed as a developmentally normative behavioral characteristic. Parents typically restrict access to rewards for which children tend to have difficulty regulating their consumption (e.g., candy, video games). Such parental oversight makes it unclear whether youth itself represents a risk factor for the development of behavioral addictions. Online gaming provides one testbed for such hypotheses. A substantial proportion of children and teens engage in video-game play (59), and during the COVID pandemic, relaxation of parental restrictions on playtime were widespread (60). One recent study in Japanese middle-school students found that earlier age of regular video-game play was indeed associated with more problematic gaming (61). This preliminary evidence should be corroborated, and whether variation in model-based control in youth is associated with the development of problematic gaming should be directly examined.

Importantly, a heightened propensity toward reward-driven habit formation may also be harnessed to develop healthy routines that facilitate resilience. Youths highly engaged in school activities, academics, and sports are at lower risk for problematic substance use, potentially because these rewarding activities prevent exploration of less healthy reinforcers (62). A mechanistic understanding of the emergence of addictive behaviors will require characterizing how normative developmental trajectories of learning computations interact with specific aspects of individuals’ daily experiences (63).

A potential role for model-based control in the development of anxious symptomatology

Recent speculative theoretical proposals suggest that model-based simulation processes may underpin ruminative symptoms characteristic of several forms of anxiety and depressive disorders (64,65). Model-based control enables simulation of potential states of the world including counterfactual alternatives to past actions or potential future actions and the outcomes they might yield (66). This computational capacity supports adaptive planning but may become overly active and biased toward negative, low-probability events in anxious individuals (67), yielding excessive worry and rumination (65). Thus, anxious rumination likely reflects an interaction between heightened model-based control and negatively biased simulation (see Box 1 and Figure 1.C.IV).

Currently, there is little empirical evidence for the role of model-based simulation in anxiety. A recent study found that severity of social anxiety symptoms was associated with increased deliberative evaluation, indexed by a computational counterfactual updating parameter (68). However, a large-scale study observing no relation between anxious symptomatology and model-based control (54) suggesting that any potential relation with anxiety may be more complex.

Both model-based control and the tendency to engage in rumination increase with age from pre-adolescence into adolescence (69) possibly reflecting a common underlying developmental improvement in mental simulation ability (70). Developmental increases in model-based simulation may be largely adaptive, as deliberative anticipatory or retrospective processing of events can facilitate future planning. However, alterations in both the content and regulation of these deliberative processes may give rise to rumination and worry: anxious children overestimate the probability of rare negative events (67)and report being unable to terminate worry until perceived threats are removed (71).

Heightened rumination during adolescence prospectively predicts increases in anxiety following stressful life events (72), and accounts for comorbidity between anxiety and depression (73). Given the clear clinical significance of rumination, an important goal for future developmental studies is to understand how the development of model-based simulation abilities interact with environmental factors to modulate maladaptive deliberation.

Pavlovian learning processes

Through Pavlovian learning, cues that predict the presence of motivationally significant positive and negative events come to elicit reflexive and valence-dependent consummatory or defensive behaviors that can promote survival (74). Reward-associated cues typically drive approach responses and invigoration of ongoing behavior, whereas threat-associated cues typically drive withdrawal and behavioral inhibition. Pavlovian learning processes are centrally implicated in the etiology of both addiction and anxiety (75,76). Through Pavlovian-instrumental transfer, reward value assigned to drug-related cues and contexts can invigorate the instrumental behaviors involved in drug seeking and consumption. In addition, recent computational work suggests that anxiety may be associated with alteration in the process of inferring the latent states to which negative value is assigned during Pavlovian learning, rendering anxious individuals’ Pavlovian threat associations highly robust, resistant to change, and readily generalized to related stimuli and contexts. Below, we review findings illustrating how these two aspects of Pavlovian learning processes — Pavlovian-instrumental transfer and latent-state inference — are implicated in addiction and anxiety, and discuss how knowledge of the developmental trajectories of these processes may contribute to a mechanistic understanding of the emergence of these disorders.

The development of Pavlovian-instrumental transfer and its role in addiction

Interactions between Pavlovian and instrumental learning play a critical role in the cycle of addiction. Drug-predictive cues (e.g., sights, smells, contexts) acquire Pavlovian reward value through hedonic drug consumption experiences. Such cues can then elicit behavioral invigoration, fostering approach, attentional capture, and physiological arousal, which collectively facilitate craving and drug-seeking behavior (77). Through Pavlovian-instrumental transfer, positive value assigned to a reward-predictive stimulus can reflexively invigorate performance of instrumental actions previously learned to yield either that specific reinforcer or rewards more generally (78).

Drug-predictive cues tend to increase performance of reward-related instrumental actions (79-83). Moreover, drug exposure strengthens the influence of non-drug Pavlovian reward cues on instrumental action (84-88). Human studies s point to a key role of Pavlovian-instrumental transfer in addiction. Relative to healthy controls, alcohol-dependent individuals(89-91) and high-risk young drinkers (92,93) exhibit greater difficulty learning actions that conflict with the valence-dependent behavioral responses elicited by Pavlovian learning processes. The strength of this effect predicts long-term relapse (94-96), potentially by facilitating drug-related approach behaviors (97). These suggest that t Pavlovian biases may be a risk factor for the escalation of drug use and is further exacerbated by drug consumption. Notably, Pavlovian-instrumental transfer may also play a role in the heightened behavioral inhibition characteristic of anxiety ((98,99) see Box 1). However, to date, few studies have examined these relations computationally.

Two recent studies examined the developmental trajectory of Pavlovian-instrumental transfer using computational approaches. In a small cohort of participants, adolescents exhibited the smallest degree of Pavlovian bias on instrumental learning, relative to children and young adults (100). However, a larger study that tested teenagers and young adults, but not children, observed no age-related change in the degree of transfer (101), suggesting that Pavlovian-instrumental transfer may decrease from childhood into adolescence andthen stabilize into young adulthood. Given this sparse evidence base, future studies spanning a broad age range and employing more diverse computational assays of Pavlovian influence (e.g., Pavlovian pruning of decision trees (102), Pavlovian shaping of goal-aligned behaviors (103)) will be important for clarifying this developmental trajectory.

Studies of food consumption suggest that high levels of Pavlovian-instrumental transfer in childhood drive consummatory behaviors in response to reward-related cues. Children who attended more to advertisements for calorie-dense snacks during play consumed more of the snack afterwards (104). Moreover, food-cue-evoked salivation predicted consumption in overweight, but not normal-weight children (105), suggesting that such transfer effects may drive unhealthy consumption. Adolescent drinkers who exhibited heightened reactive responses to alcohol cues in the laboratory also reported greater craving and consumption in real-world contexts associated with use (106). Pavlovian-instrumental transfer may be a mechanism contributing to such escalation of reward-seeking and consumption. Individuals who initiate drug experimentation during developmental periods when Pavlovian-instrumental transfer is typically robust (or who exhibit atypically high levels of transfer) may be more prone to develop compulsive use. This underscores the importance of examining how this computational process is shaped by environmental factors and modulates the emergence of addictive behaviors (Figure 1.C.II). Moreover, diverse forms of Pavlovian learning reflecting model-free versus model-based underlying computations (i.e., sign-tracking versus goaltracking) (107) are differentially associated with addiction risk (108), highlighting the importance of investigating these learning dimensions in future developmental studies (109).

Latent state inference in aversive Pavlovian learning and the development of anxiety

Aversive Pavlovian learning has been proposed as a model for the learning processes through which threat expectations are acquired in anxiety (110). Aversive Pavlovian learning is commonly studied using paradigms in which a neutral stimulus (the conditioned stimulus, or CS) is repeatedly paired with an aversive unconditioned stimulus (US; e.g., mild electrical shock), after which CS presentation elicits reflexive defensive conditioned responses (e.g., freezing). The capacity to alter such associations is commonly studied using extinction paradigms, in which the CS is no longer paired with the US, and expression of conditioned responses typically decreases. However, extinguished responses can return under a number of circumstances, including a change in context (renewal), re-exposure to a US (reinstatement), or the mere passage of time (spontaneous recovery) (111). Through generalization, stimuli that share some degree of perceptual or conceptual similarity to the CS, but were never directly associated with the US, can come to evoke a conditioned response. Generalization typically decays as the dissimilarity between the CS and other stimuli increases (112).

Changes in the process of inferring the latent environmental states (or contexts) to which negative or positive value is assigned may contribute to variation in the acquisition, extinction, and generalization of Pavlovian learning. Under this computational account, learners segment their experience into latent states that capture regularities in the configuration of observed stimuli (e.g., CS and US) (113). Mismatch between the current configuration and the learned prototypical configuration of a state (akin to a prediction error) provides evidence of a new latent state. Sensitivity to the deviation between the stimulus configurations observed during acquisition and extinction may cause trials from these phases to be assigned to distinct latent states (i.e., separate threat and safety memories), whereas lower sensitivity allows reinforced acquisition trials and unreinforced extinction trials to be assigned the same latent state, effectively “overwriting” the original memory and eliminating the possibility that the conditioned response could later reemerge. Consistent with this account, computational analysis of participants’ conditioned responses during aversive learning and extinction (114) demonstrated that only individuals who appeared to infer two latent states showed spontaneous recovery following extinction training. Variation in latent-state inference can also account for stimulus generalization. If a reinforced CS and another similar stimulus are clustered within the same latent state, they will both acquire valenced expectations, as they will share an association with the US.

Relative to control participants, anxious individuals acquire more robust conditioned responses, exhibit attenuated extinction learning and stronger reinstatement, and show increased generalization of conditioned responding to unreinforced stimuli (115,116). Two recent studies suggest that these alterations in Pavlovian learning may relate to differences in latent-state inference. In an unselected sample, shock expectation ratings of high-trait-anxiety individuals were better explained by a model inferring multiple latent states, conferring the ability to reinstate previously learned threat associations, while ratings of low-trait-anxiety individuals were better explained by a single-state learning model (117). A second study found that PTSD patients who exhibited greater generalization of loss expectations and higher levels of avoidance symptoms were more likely to assign stimulus observations to a single underlying cause (118). These seemingly contradictory findings may identify two modes of latent-state inference that correspond to distinct dimensions of anxious symptomatology (persistent threat response versus overgeneralization) that differ in prevalence across these samples. However, these findings should be replicated and extended in order to better reconcile these apparent discrepancies.

Although aversive Pavlovian expectations are similarly acquired and expressed across development, both the extinction and generalization of these associations change markedly (119). In juvenile rodents, conditioned responding decreases during extinction, but unlike in adult animals, these responses do not tend to re-emerge (120-122). Extinction learning in juveniles also does not engage neural processes associated with threat-response inhibition (123), but instead appears to modify the original threat association (122,124), suggesting that extinction at this developmental stage may be more akin to unlearning. In contrast, adolescent rodents and humans exhibit greater difficulty extinguishing threat associations and heightened reemergence of conditioned responses following extinction, relative to pre-adolescents and adults (125,126). Aversive generalization also exhibits systematic changes across development. Relative to adolescents and adults, younger children show broader generalization of threat responses to novel stimuli. With age, the ability to discriminate between a CS associated with threat and unreinforced neutral stimuli gradually improves (127-130).

We do not know of studies that have used formal approaches to test whether latent-state inference exhibits systematic developmental changes. However, the changes in extinction and generalization described above are consistent with a proposal that the tendency to infer new latent states may be lower at younger ages, and increase with development. Moreover, a greater tendency to cluster conceptually or perceptually similar stimuli into the same latent state may contribute to the heightened generalization observed in younger individuals (127-131). Consistent with broader developmental improvements in memory specificity across adolescence (132), the tendency to infer distinct latent threat and safety states based on shifts in environmental statistics may increase with age, enabling extinguished responses to reemerge. Future studies using computational modeling approaches should test this proposal, with a particular focus on identifying the specific mechanisms that might underpin attenuated extinction and heightened reemergence of threat associations during adolescence (133).

Systematic developmental changes in latent-state inference might underpin corresponding age differences in anxiety-related symptomatology, and suggest avenues for tailoring treatment. A heightened tendency to cluster together threat-associated cues or contexts with similar, safe states may be a key driver of symptoms in anxious children (134,135), making therapies that facilitate discrimination ability particularly efficacious. As differentiation of threat and safety states improves with age, extinction-based exposure therapies might be modified to increase generalization between extinguished threat associations and earlier experiences, in order to prevent fear reemergence. For example, exposure protocols modeled after gradual extinction training, in which the frequency of CS-US pairing slowly decreases, may diminish the mismatch signals that drive new state inference and reduce spontaneous recovery and reinstatement effects (136,137). More generally, such cognitive heterogeneity across individuals highlights the importance of tailoring behavioral therapies to target specific types of maladaptive inferences.

Conclusion

Here, we synthesized knowledge about computational processes implicated in the etiology of addiction and anxiety with findings about the developmental trajectories of these computations. We highlighted areas where we have strong knowledge of both the clinical significance and the normative developmental trajectory of a cognitive process (e.g., the development of model-based control and its relevance to compulsivity), and those for which both developmental and clinical data are lacking (e.g., latent-state inference).

Going forward, computational psychiatry approaches may prove particularly valuable for characterizing developmental mechanisms of psychopathology. Latent cognitive computations that underpin clinically relevant behaviors likely contribute to the emergence, maintenance, or escalation of symptoms over developmental time. The objective behavioral metrics that this approach provides (e.g., model parameter estimates) can enable researchers to chart normative developmental changes in these computations, study their underlying neural mechanisms, and characterize the dynamic relation between computational phenotypes and clinical symptoms longitudinally. Importantly, normative trajectories of clinically relevant objective phenotypes may provide critical tools for diagnosis and intervention, circumventing challenges posed by reliance on clinical interviews and self-report measures. This may be especially true for children and adolescents, whose verbal abilities and metacognitive awareness may be less robust than adults’ and whose expression of symptoms can often be either atypical or masked by caregiving environments. Further, these measures may eventually be able to serve as actionable early indications of aberrant developmental trajectories that can indicate whether — and which — interventions should be considered in order to prevent the emergence or worsening of symptoms.

Several topics remain beyond the scope of this review but may be critical for progress in the field. In this paper, we have treated computational phenotypes as stable, trait-like properties that change on a developmental timescale. However, computational phenotypes can also change dynamically in response to shifts in environmental conditions (138,139), stress (see Box 2), changes in affective state (140), or other factors. Characterizing both the factors that elicit such dynamic changes in computational phenotypes and the temporal dynamics of these changes across both local and developmental time scales (7), are important avenues for future research. Understanding periods of vulnerability to psychopathology will also require more thorough investigation of the relation between neuroplasticity and learning. For example, a study detailing the brain mechanisms underlying the opening and closure of a developmental critical period for social reward learning (141) suggested a potential biological mechanism of efficacy of a drug that is used to treat PTSD (142). Such convergent insights illustrate how neurobiological research into the mechanisms of learning development can inform psychiatric treatment. Characterizing developmental periods during which the tuning of additional learning processes, and their underlying neural circuits, are most sensitive to environmental inputs may be critical for identifying targets for psychiatric intervention.

Box 2: Stress effects on transdiagnostic learning computations.

Threats, either real or perceived, can induce stress. Stress responsivity is increased during childhood and adolescence, with stress exposure yielding heightened physiological responses, lasting effects on developing cortical-subcortical circuitry, and impairments in stress coping in adulthood (163-166). Early-life stress is a well-established risk factor for the development of both anxiety and addiction (167,168), and mental illness more generally (169).

Across species, stress reduces model-based control, fostering increased habitual behavior (170-175). Stress does not appear to facilitate appetitive Pavlovian-instrumental transfer (176-178). However, it does promote the reemergence of previously extinguished reward associations (179,180), which may allow drug cues to regain their influence over behavior following periods of abstinence, consistent with stress-induced reinstatement of drug-seeking behavior (181).

Stress exerts marked effects on aversive Pavlovian learning that exacerbate anxious symptomatology. Stress enhances the consolidation of aversive associations and impairs extinction learning and retention (182). Acute stress increases Pavlovian-instrumental transfer (183), especially in anxious individuals (98), by fostering greater behavior inhibition in the presence of threat, compromising the flexibility of instrumental behavior. Moreover, traumatic stress exposure during mid-to-late adolescence can yield persistent increases in such passive avoidance behavior (99).

These demonstrations that learning computations are profoundly altered by stress exposure suggest that the environmental conditions in which learning occurs are a key determinant of psychiatric outcomes. This may be particularly true at earlier stages of development when sensitivity to stress is heightened.