Resource-rational psychopathology

Abstract

Psychopathology is vast and diverse. Across distinct disease states, individuals exhibit symptoms that appear counter to the standard view of rationality (expected utility maximization). We argue that some aspects of psychopathology can be described as resource-rational, reflecting a rational trade-off between reward and cognitive resources. We review work on two theories of this kind: rational inattention, where a capacity limit applies to perceptual channels, and policy compression, where the capacity limit applies to action channels. We show how these theories can parsimoniously explain many forms of psychopathology, including affective, primary psychotic, and neurodevelopmental disorders, as well as many effects of psychoactive medications on these disorders. While there are important disorder-specific differences, and the theories are by no means universal, we argue that resource rationality offers a useful new perspective on psychopathology. By emphasizing the role of cognitive resource constraints, this approach offers a more inclusive picture of rationality. Some aspects of psychopathology may reflect rational trade-offs rather than the breakdown of rationality.

Introduction

We lack a basic understanding and language to explain psychopathology, and without this understanding, we are limited in our ability to diagnose, treat, and prognosticate, among others. Here, we argue that nascent work at the intersection of cognitive science, economics, and information theory has the potential to provide the necessary explanatory framework. We begin with the premise that biological agents are inherently resource limited. Resource rationality formalizes the notion that people are doing the best they can, subject to natural information-processing constraints. This resource-rational perspective was developed to explain how people can perform optimally in some domains, and deviate from optimality in other domains (Lewis et al., 2014; Gershman et al., 2015; Griffiths et al., 2015; Lieder and Griffiths, 2020; Bhui et al., 2021; Gershman, 2021). We seek to extend this perspective to gain insight into symptoms that may be shared across states of psychopathology.

The nervous system evolved in the face of myriad constraints, including computational costs (Bossaerts et al., 2019), interference costs (Musslick et al., 2016), metabolic costs (Gailliot and Baumeister, 2007), and others (Shenhav et al., 2017). We focus here on channel capacity, an upper bound on how much information can be transmitted across brain regions (Attneave, 1954; Miller, 1956). We expound two theories of capacity constraints, one applied to perception and the other to action. We will focus primarily on dopamine, as this is the neurotransmitter system with the greatest support for our theories. Psychopathology is far more complex than a single neurotransmitter system and we leave out other relevant systems and brain structures not because they are unimportant, but because the link between them and our theories is more tenuous. We consider a deeper focus on neurobiology outside the scope of this perspective, which we aim to keep at a more theoretical level.

Before getting into details, it is worth stepping back to appreciate the larger conceptual pivot that resource rationality invites us to make. The concept of psychopathology was traditionally based on a division into “pathological” and “non-pathological” minds, but this division has been under strain from both empirical and sociological directions. Empirically, it has become increasingly recognized that many—perhaps all—mental disorders are points on a continuum; there is often no clean dividing line between pathological and non-pathological. Sociologically, the continuum view has led to a “neurodiversity” movement which aims to reframe pathological states as differences rather than deficits. As we will explain below, the resource rationality framework suggests a formalization of the continuum view, where individual differences in cognitive capacity lead to different optimal solutions. All of these solutions are optimal, yet they may lead to highly divergent phenotypes. The population may cluster around certain solutions, but these solutions do not necessarily reflect a normatively privileged status. Accepting this proposition opens the door to a computationally informed destigmatization of psychopathology.

Resource rationality does not abandon the notion that some states occupy extremes that require medical treatment. By analogy, a person with missing limbs may be doing the best they can with their available physical resources, but this does not mean that they couldn’t do better if supplied with prosthetic limbs. Similarly, resource rationality does not guarantee any particular absolute performance level; it only guarantees that an individual will attain a performance level that is superior to the set made available by their supply of cognitive resources. Psychiatric treatment may enable an individual to attain higher absolute performance levels. Resource rationality thus reconciles the continuous nature of psychopathology with the ostensible benefits of treatment.

Rational inattention: capacity limit applied to perception

Given the limitations of biological sensors as well as the statistical challenges of perception, the brain relies on prior, contextual information to constrain what it perceives. Broadly speaking, rational inattention asserts that agents rationally allocate their limited attentional resources (Sims, 2003; Woodford, 2009; Maćkowiak and Wiederholt, 2009; Mackowiak et al., 2018). An equivalent interpretation, as we will see, is that agents can pay a cognitive cost to obtain a more veridical representation of the world, so long as it does not exceed channel capacity.

Consider an agent inferring a latent variable, such as the time interval between two events. Because sensory signals are imprecise (e.g., time-keeping is noisy), the agent cannot be certain about the underlying latent variable. Bayes’ rule states that the agent should combine its sensory evidence with its prior beliefs (e.g., the typical distribution of time intervals) to compute a posterior probability distribution over the values of the latent variable. This is the standard setup in Bayesian models of perception. Importantly, sensory precision is traditionally taken to be outside the control of the agent—an exogenous factor. Rational inattention models generalize this setup to endogenize sensory precision, treating it as a function of attentional control. In other words, sensory precision is modeled as a kind of “cognitive action” that the agent can take, subject to a cognitive cost.

To formalize this idea, we need to first be more precise about what we mean by attention. Following prior work, we conceptualize attention in terms of mutual information (Itti and Baldi, 2009; Feldman and Friston, 2010). Mutual information expresses how much our uncertainty about the latent variable is reduced (on average) after observing data. Intuitively, attending to a signal means extracting information from it—i.e., reducing uncertainty. This information extraction process can be viewed as a kind of communication channel mapping inputs (signals) to outputs (percepts). Like all physical channels it is subject to a capacity limit (an upper bound on mutual information).

Agents will earn more reward on average when their sensory precision is higher. We will refer to the relationship between sensory precision and reward for a given task as the attentional incentive. Thus, an agent should increase sensory precision when the attentional incentive is higher. A second factor determining precision is the attentional cost incurred by increasing precision, which implicitly depends on the capacity limit. Evidence for these predictions, along with a more technical exposition, is covered further in Gershman and Burke (2022). Figure 1A summarizes the predictions.

Figure 1: Perception and action as communication channels.

A: Rational inattention describes perception as a communication channel, subject to a capacity limit, or upper bound on amount of information that can be transmitted across sensory channels. In this example, we highlight a stimulus being encoded by the brain either under low capacity conditions or high capacity conditions. The brain is able to increase the capacity of encoding by devoting greater attention - the cognitive process of reducing uncertainty about a stimulus. Under rational attention, the factors that increase attention are increased reward (in our framework, the attention incentive) or decreased cost of information. The consequences of encoding at differing capacities is highlighted to the right. Photo obtained from Smithsonian’s National Zoo and Conservation Biology Institute open access images.

B: Policy compression describes action selection as a communication channel, subject to a capacity limit. In this example, an agent is tasked with purchasing groceries under two possible states: low budget, or high budget. With a low budget, the optimal policy is to purchase generic items. With a high budget, the optimal policy is to purchase premium items. Under conditions of low capacity, the agent is state-insensitive, and purchases generic items regardless of the state. Under conditions of high capacity, the agent is highly state-sensitive, and exhibits the optimal policy. The consequences of these capacity limits is highlighted to the right.

Building a bridge to neurobiological mechanisms, Mikhael et al. (2021) developed a rational inattention account of tonic dopamine. Under this account, tonic dopamine subsumes the average reward theory of tonic dopamine, where it encodes the context (state)-specific average reward rate (Niv et al., 2007; Beierholm et al., 2013; Hamid et al., 2016). Specifically, rational inattention does not propose a different role for how dopamine encodes reward than what has been posited previously. Tonic dopamine, by reporting average reward, is hypothesized to set the baseline for learning the value of specific actions within a given state, and can give rise to phenomena like asymmetric learning and exploration/exploitation, ideas which we will elaborate below. Because the rational inattention framework couples average reward to sensory precision, it predicts that changes in tonic dopamine levels should control the allocation of attention, consistent with many pharmacological and physiological findings. The reward-attention coupling also provides an integration of the average reward theory with the active inference theory developed by Friston et al. (2012), according to which tonic dopamine controls the precision (“salience”) of external and internal cues (see also Shi et al., 2013, for a related theory applied to time perception).

Policy compression: capacity limit applied to action

All actions, from the mundane to the significant, require memory. These memories are stored in the brain as policies, or mappings from states to actions, where states are defined as the representation of information needed to predict reward (Sutton and Barto, 2018). As an example, imagine being tasked with purchasing groceries for the family. The state representation includes the items available for purchase as well as the individual preferences of family members, and the actions include either purchasing or not purchasing an item. Intuitively, you can satisfy each individual’s preferences, but at the cost of a mentally-demanding trip to the store. If you choose instead to ignore certain preferences, you can reduce cognitive demand, by reducing the number of items that must be remembered, at the cost of reducing the overall satisfaction of the family.

Policy compression formalizes this intuition, by conceptualizing the mapping from states to actions as a communication channel—just like we posited for perception—and postulating that this communication channel has a limited capacity (Parush et al., 2011; Gershman, 2020; Lai and Gershman, 2021). Under policy compression, agents must optimize the trade-off between reward and policy complexity, which we define as the mutual information between states and actions. Because policy complexity is a lower bound on the number of bits needed to store a policy in memory, more complex policies necessitate more bits. If the policy complexity exceeds capacity, then agents must “compress” the policy in order to transmit it across brain regions. Policies with high complexity require greater memory and can lead to greater reward. In contrast, policies with low complexity require less memory to implement, but are generally suboptimal. At the extreme, if the policy is the same in every state, then the policy complexity is minimized (mutual information is 0).

The optimal capacity-limited policy has a number of interesting features. First, it takes the form of the ubiquitous softmax function, in which an “inverse temperature” parameter governs the stochasticity in the policy. When capacity is high, policies become more deterministic (via a larger inverse temperature parameter) and concentrate on the action with maximal value. When capacity is low, policies become less state-dependent (via a smaller inverse temperature parameter). More specifically, the inverse temperature is lower (i.e., choices become more random) when varying the policy complexity has a greater effect on reward, which occurs at low values of policy complexity. Second, the optimal policy includes a perseveration term. When capacity is large, the inverse temperature term is large and actions are largely driven by the values of the underlying states. When the capacity is small, the inverse temperature term decreases, and the perseveration term can dominate the policy. Third, more complex policies result in slower response times, because the brain must inspect more bits to find the coded state (Hick, 1952; Lai and Gershman, 2021; Bari and Gershman, 2023). These regularities are summarized in Figure 1B.

Psychiatric phenomena

Mania

Rational inattention provides a rich language for describing numerous symptoms of mania, which we propose is best understood as an individual’s belief that their precision has increased, without an increase in true precision. In other words, mania may be the result of precision miscalibration, where precision is overestimated (Mikhael et al., 2021).

Clinically, mania can be a distinctly euphoric state (Cassidy et al., 1998b), one that patients are often unwilling to request or accept treatment for (Baldessarini et al., 2008). It is not uncommon for patients with bipolar disorder, a disorder characterized by oscillations between mania and depression, to self-discontinue medications, either because it makes them feel ‘depressed’ (e.g., relative to prior mania/hypomania) or in the hopes that they may experience a manic state (Devulapalli et al., 2010; Crowe et al., 2011). Anecdotally, some patients who accept treatment are doing so not for the mania itself to be treated, but because personal experience has taught them that their mania can develop into a mania with psychosis. Rational inattention provides a clue for the intoxicating effects of mania: a subjective increase in precision is associated with an increase in average reward. If mania causes patients to experience the world as highly rewarding, it’s understandable why they would desire to remain in that state.

This increase in the estimate of average reward manifests as an asymmetry in how agents estimate value functions. In mania, because agents come to expect reward (i.e., the prior over rewards is shifted higher), they exhibit a persistent ‘optimism’ in which their value functions are shifted higher, following either positive or negative feedback. In other words, relative to an agent with a veridical estimate of average reward, optimistic agents come to expect reward even when they objectively should not. In mania, this is consistent with clinical intuition and the empirical literature (Alloy et al., 2016; Kwan et al., 2020). Note that rational inattention does not predict faster learning (e.g., the trial-to-trial change in expectation) from positive feedback under an optimistic prior, only that the value function is initialized optimistically. Learning is otherwise consistent with the predictions of Bayesian inference, in which more surprising observations - those farther from the prior - are learned faster.

High estimated precision implies less dependence on a stored internal prior. Patients in a manic state display the expected hallmarks—they are highly attentive, ever-present, keenly aware of their environments, and outwardly directed. In the context of interval timing, high estimated precision also predicts a faster internal clock (see Mikhael et al., 2021). In mania, patients exhibit myriad symptoms consistent with a faster internal clock. They are are classically psychomotor agitated: they appear to be moving in fast motion, restless and always on the move, with rapid speech that can be difficult to interrupt (Cassidy et al., 1998a). A faster clock also results in faster thought, consistent with subjective experience and occasionally at a pace so rapid as to be aversive. The subjective sense of time is sped up (Bschor et al., 2004) and can become so grossly miscalibrated that patients will sense that tens of minutes have elapsed after only a minute or two.

In the context of reward learning, overestimated precision induces heightened sensitivity to noise, which may appear as a form of “exploration” (choice randomness). Clinically, this manifests in what the Diagnostic and Statistical Manual of Mental Disorders refers to as an “increase in goal-directed activity” (American Psychiatric Association, 2022). Patients in a manic state are famous for starting (but not necessarily completing) dozens of new projects, hobbies, books, television shows, and so on (Dailey and Saadabadi, 2018). In general, the new activities are consistent with what the patient feels is worth pursuing, and not just activity for its own sake. This is generally consistent with the notion of value-based random exploration: activities tend to be those of higher value.

Distractibility, a key diagnostic criterion for mania, is another facet of heightened noise sensitivity due to precision miscalibration. On this account, distraction arises when task-irrelevant distractors are misinterpreted as task-relevant cues.

One aspect worth emphasizing is that all of the above phenomena, with the exception of distractibility, are consistent with a true increase in precision and therefore a true increase in capacity. How can we discern a precision overestimation account of mania from a true increase in precision? The former predicts degraded perception while the latter predicts improvement. Consistent with precision overestimation, the literature supports the idea that perception is degraded in mania (Kohler et al., 2011; O’Bryan et al., 2014). As a separate prediction, the precision overestimation hypothesis predicts an increase in random exploration, while true precision increase predicts a decrease. There is some evidence to suggest increased exploratory behavior in mania (Ryu et al., 2017), although there is more work to be done.

Depression

Under rational inattention, if mania can be described by an increase in precision (either real or perceived), then depression in many ways can be viewed as its opposite. With a decrease in precision comes an increase in reliance on an internal prior. Patients with depression frequently speak of a subjective ‘grey’-ness of experience, with a sensation that they cannot perceive or experience the world as they did when they were well. If, in depression, the sensory precision is reduced, then the posterior will be dominated by the prior, and subjective experience will necessarily be less rich and less modulated by perception of the outside world. As a result, patients appear inattentive to the outside world (Keller et al., 2019), with their focus directed inward.

Just as increased sensory precision speeds up the internal clock, decreased precision slows it down (Bschor et al., 2004). This too explains the general slowness observed in depression, with overt psychomotor slowing manifesting as sluggish gait and movement, turning mundane tasks such as dressing and showering into time-consuming chores. Speech itself slows down (Koops et al., 2023) and patients report a sense of slowed thinking.

A reduction in precision signals to an optimal agent to expect reduced reward. This may explain the subjective intolerability of depressive states. It comes as no surprise that depression is a risk factor for suicidal thinking (Franklin et al., 2017), though there is some circularity here as suicidal thoughts are a diagnostic criteria for depression. A reduced expectation of reward also shifts the balance of learning towards pessimism. This may lead to the sense of hopelessness that is pervasive in depression (Abramson et al., 1989; Cusin et al., 2010). Reduced precision also renders behavior less responsive to feedback, consistent with what has been observed in depression (Steele et al., 2007). From the perspective of reinforcement learning theory, this is consistent with a reduced learning rate, which has been observed in depression (Brown et al., 2021), though other results have been equivocal (Chen et al., 2015).

Patients may be convinced that they lack agency to meaningfully affect their lives. Here we predict that depression may also reduce the attentional incentive, even in circumstances where patients have clear agency. In other words, patients will perceive a lack of controllability, even if this is at odds with reality (Miller and Seligman, 1975). There is a long and rich literature on learned helplessness in depression (Maier and Seligman, 1976, 2016), and rational inattention provides another perspective: if the attentional incentive is decreased or erased, rational agents should not allocate attention to the task at hand, as attention is only worth the cost if outcomes can be improved. This in turn can manifest as reduced motivation—after all, why engage if the result won’t change? Further, clinical experience suggests that to the extent that patients are motivated to interact with the world, they need to frequently be reminded of activities they find highly rewarding (i.e., encouraging patients to exploit). This is consistent with the idea that exploitation should be reduced under conditions of low capacity, consistent with what has been observed with reinforcement learning modeling in depression (Blanco et al., 2013).

Patients with dense depression are known to have cognitive deficits in numerous domains (Hack et al., 2023). Under policy compression, reduced capacity for actions is associated with a decrease in working memory and an increase in perseverative behavior. These deficits manifest clinically, as patients can have a difficult time retaining basic information and can be perseverative in their thoughts and their behaviors (Trick et al., 2016). These deficits have been identified in the lab, as depressed patient show pronounced perseverative errors in set shifting tasks (Martin et al., 1991; Channon, 1996; Ilonen et al., 2000; McGirr et al., 2012) and have working memory deficits (Channon et al., 1993; Burt et al., 1995; Christopher and MacDonald, 2005; Rose and Ebmeier, 2006).

Unlike mania, we make no strong claims about whether depression is due to a true decrease in precision vs. a decrease in estimated precision, as both of these mechanisms will give rise to all the symptoms we discussed. One conceptual difference is that a true decrease in precision results in a decrease in capacity, whereas precision underestimation only decreases perceived capacity, rather than actual capacity. In the latter case, cognitive abilities should be intact, in some sense, with deficits rendered by patients perceptions of their own abilities. In line with this thought, there is some evidence that the cognitive deficits in depression are mediated by variables such as effort (Moritz et al., 2017).

Stimulants

Prescribed stimulants fall into two major classes, amphetamines and methylphenidate. Amphetamines function, in part, by inducing the release of dopamine (Schiffer et al., 2006) whereas methylphenidate functions as a stronger inhibitor of the dopamine transporter (John and Jones, 2007), a protein which reuptakes dopamine into presynaptic terminals. Despite differences in mechanism (with some overlap), these drugs effectively increase synaptic dopamine concentration (Kuczenski and Segal, 1997). Prior work has shown that stimulants decrease the energy the brain uses to perform cognitively demanding tasks, analogous to a reduction in the attentional cost parameter under rational inattention (Volkow et al., 2008). Individuals take these medications to feel more attentive and focused. Time perception is sped up (Lake and Meck, 2013), in line with predictions, and with a faster clock comes faster movements, faster speech, and faster thinking. The mental states induced by stimulants are distinctly pleasurable and contribute in no small part to potential for misuse. Overall, there is remarkable similarity between the effects of stimulants and mania, but they are by no means identical.

Antipsychotics

Under rational inattention, if increasing dopamine can recapitulate the effects of increased capacity, then decreasing dopamine should do the opposite. Clinically, antipsychotics serve this function, a subset of which functions, in part, by blocking the D2 receptor (McCutcheon et al., 2023). Further, recent work has argued that clinically-efficacious antipsychotics, include those with minimal direct dopaminergic effects, function by modulating D1 receptor-expressing neurons in the striatum (Yun et al., 2023). Rational inattention on its own does not explain how antipsychotics reduce the hallucinations, delusions, and disorganized thought for which they are indicated, but we believe it does help explain the intolerability of this class of medications (Valenstein et al., 2004; Ascher-Svanum et al., 2010). In fact, the largest trial of antipsychotics to date chose ‘discontinuation of treatment for any cause’ as a primary outcome (Lieberman et al., 2005), which substantiates their intolerability. Patients who have tried numerous antipsychotics describe a sense of feeling subjectively slowed, cognitively dulled, with limited attention, and a sense that they are perceiving the world through a fog. All of these effects are consistent with a reduced capacity. Antipsychotics must be dosed carefully in psychotic illnesses, as they can exacerbate the cognitive symptoms that limit the ability of patients to function (Kelley et al., 1999; Kasper and Resinger, 2003), an entity once known as neuroleptic-induced deficit syndrome (Lader, 1993; Schooler, 1994)

Attention-deficit/hyperactivity disorder

Inattention is a cardinal feature of attention-deficit/hyperactivity disorder (ADHD), and may naturally be explained under rational inattention as reduced sensory precision. This explains the failure to ‘give close attention,’ ‘difficulty sustaining attention,’ and ‘often easily distracted by extraneous stimuli’ that are diagnostic symptoms. The increased reliance on the internal prior can contribute to the sense that their minds seem elsewhere. Distractibility and careless mistakes can be thought of as an increase in random exploration (Hauser et al., 2014) which, for reasons stated previously, is increased under more stringent capacity limits. The dislike of mentally effortful tasks is also well-explained: if one cannot provide the attention necessary to complete a task, due to reduced capacity, then it is rational to avoid those tasks. If stimulants work as proposed above, then the rationale for their use in ADHD is clear.

Schizophrenia

Schizophrenia and other psychotic illnesses are primarily characterized by their positive symptoms: hallucinations, delusions, and disorganized thought. In contrast, the negative symptoms more frequently restrict the ability of patients to fulfill typical societal roles and responsibilities (e.g., maintaining friendships, managing household tasks), in part due to our inability to adequately treat them (Aleman et al., 2017). These symptoms include amotivation, asociality, blunted affect, and general cognitive impairments (Correll and Schooler, 2020). From the perspective of policy compression, a number of these symptoms can be explained by a reduction in channel capacity. Indeed, patients with chronic schizophrenia exhibit reduced capacity (Gershman and Lai, 2021). This can be linked to blunted affect (decreased expressivity of emotions) and alogia (the reduction in quantity of words spoken). Reduced capacity can also explain working memory deficits, which have been robustly demonstrated in schizophrenia (Forbes et al., 2009; Collins et al., 2014). We note that reduced capacity in chronic schizophrenia is confounded by chronic antipsychotic use, which may contribute independently to changes in capacity.

Parkinson’s disease

Parkinson’s disease is characterized by widespread degeneration of the dopaminergic system (as well as other neuromodulatory systems). From the perspective of policy compression, individuals with Parkinson’s disease have reduced capacity compared to age-matched controls (Bari and Gershman, 2023). This reduced capacity can explain a number of cognitive symptoms seen in Parkinson’s disease, including: memory problems, language difficulties (word-finding difficulty, naming/misnaming, comprehending complex sentence structure, dysarthria), and general problem solving and executive functioning difficulties (Dubois and Pillon, 1996; Verbaan et al., 2007). Providing patients with dopaminergic agents increases capacity for actions, allowing subjects with Parkinson’s disease to entertain more complex policies (Bari and Gershman, 2023). This is consistent with clinical observation that dopaminergic therapy can relieve a number of cognitive symptoms (although not to the same extent as motor symptoms; Dubois and Pillon, 1996; Robbins and Cools, 2014). Counterintuitively, in our analysis, dopaminergic therapy slows participants down, as measured by response rates, which is opposite of the general effect of these treatments in relieving bradykinesia. This is consistent with policy compression, which predicts that more complex policies require greater time to decode (i.e., map from the the compressed representation to overt actions; Hick, 1952; Lai and Gershman, 2021).

Rational inattention provides a complementary perspective. Reduced capacity explains reduced attention and slower speed of thinking. It also explains the observation that, in subjects with Parkinson’s disease, dopaminergic therapy restores sensitivity to feedback (Frank et al., 2004; Rutledge et al., 2009). Another manifestation is the stronger central tendency effect in Parkinson’s disease. In the context of interval timing, this effect describes a tendency of subjects to overreproduce short intervals and underreproduce long intervals in timing reproduction tasks (Malapani et al., 1998, 2002; Shi et al., 2013). Under rational inattention, this is consistent with a strong migration towards the prior induced by conditions of low attention / low tonic dopamine (Mikhael et al., 2021; Mikhael and Gershman, 2022). Consistent with this account, providing subjects with dopaminergic medication reduces the magnitude of the central tendency effect.

Neurodevelopmental disorders: Specific learning disorders

Several specific learning disorders manifest as difficulties in processing specific sources of information. Examples include difficulties in processing language, written information, numerical / mathematical information, and social information. One consequence is a decrease in reliance on these sources, which over time atrophies the brain’s ability to use them. Under rational inattention, if these processing difficulties arise from aberrant precision, then it is rational to decrease reliance on them and focus on higher precision sources of information. This has the deleterious consequence of diminishing the brain’s ability to use this information, which can create difficulties with functioning later in life. This highlights the need to design curricula that force the brain out of the rational but deleterious underreliance on this aberrant information. Instead, if individuals can be trained to use this low-precision information, precision may increase with experience, especially during valuable critical periods early in life. We turn the interested reader to Jones et al. (2023) for a thoughtful perspective.

Perseveration

Under policy compression, perseveration emerges as the optimal policy under low capacity. If low capacity is common to numerous psychiatric conditions, then we would expect perseveration to arise as a transdiagnostic symptom. Indeed, perseveration is observed in numerous psychiatric conditions, some of which we have detailed above (Serpell et al., 2009). As an extreme, in delirium, it is not uncommon for patients to repeat answers to the first question asked, even if the answer is nonsensical. Patients can act out more complex policies, like those seen in addiction (Lane et al., 2007; Woicik et al., 2011), which are nevertheless resistant to change. It is seen in conditions ranging from schizophrenia (Crider, 1997), to depression (Martin et al., 1991), to frontal lobe pathology and other neurodegenerative conditions (Joseph, 1999; Oosterloo et al., 2019), to name a few.

Discussion and limitations

Our resource-rational framework has a remarkable degree of overlap with prior work in learning and decision making. First, much of the neurobiology of reward-based decision making is motivated by reinforcement learning theory, which has furnished the field with error-driven learning models (Sutton and Barto, 2018). In these models, reward prediction errors (the difference between actual and expected reward) drive sequential learning. As explained in the Appendix, one form of rational inattention uses this familiar error-driven update rule to estimate the posterior over the parameter of interest; if the parameter is reward, then this rule is the familiar reward prediction error. In deriving policy compression, we assumed the value function was known, an assumption which cannot hold for agents learning in novel environments. We have previously developed process models to iteratively estimate the value function using familiar error-based update rules (Gershman and Lai, 2021). We withhold a deeper discussion of process models as this remains an active area of research.

Second, policy compression provides insights into habits, repetitive behaviors that are famously insensitive to outcome devaluation or contingency degradation, and which can interfere with goal-directed behavior (Dickinson, 1985; Wood and Rünger, 2016; Miller et al., 2019). Similarly, low complexity policies are perseverative since they are dominated by the marginal action distribution, which is not dependent on rewards (Gershman, 2020). Policy compression does not, however, explain the shift from “goal-directed” behavior to habitual behavior that occurs with training (Balleine and O’Doherty, 2010). One hypothesis is that the shift to habitual behavior (i.e., policies of low complexity) may free up finite capacity, which we assume is fixed, so it is not all allocated to one task. Third, rational inattention subsumes an influential account relating tonic dopamine to average reward availability in a given context and response vigor (Niv et al., 2007). In other words, our derivation of rational inattention makes the same predictions as the average reward theory of dopamine, and extends it to precision.

One conceptual leap we have made is to propose that the affective symptoms in mania and depression arise not from deficits in reward processing, but from aberrations in attention. We made this leap based on parsimony, as aberrations in attention allow us to explain not just affective symptoms, but numerous symptoms related to psychomotor state, learning, and decision making. An influential account of anhedonia, a common symptom in depression characterized by the inability to experience pleasure, holds that it may arise as a consequence of impaired reward sensitivity (reduction in the perception of reward magnitude; Huys et al., 2013). The authors note that reward insensitivity, under certain assumptions, is equivalent to over-exploration. This latter view is close to what we propose with rational inattention, which suggests that over-exploration, as a consequence of precision underestimation, may masquerade as reward insensitivity.

We have argued that distractibility arises in mania, due to precision overestimation, and ADHD, due to reduced precision. How does rational inattention predict distractibility in both of these circumstances? At first pass, it would appear that only agents with reduced precision should be affected, since they do not have the sensory precision to focus on the task at hand. It is important to recognize that although distractibility arises in both mania and ADHD, they are observably distinct phenomena on clinical evaluation. What they share in common is an inability to follow a task through to completion. In mania, there is a sense that patients are intensely interested in their environments, focused not just on the task at hand but also task-irrelevant information. In ADHD, there is a sense of disinterest or ambivalence in the task at hand. In mania, therefore, distractibility arises from amplification of signal and noise, and patients with mania assign undue importance to inappropriate samples that impinge on their senses. In ADHD, distractibility arises from reduced attention to the signal. Phrased this way, rational inattention provides transdiagnostic insight into distractibility, and makes it clear that it arises from distinct computational aberrations. Note that rational inattention does not predict distractibility if precision is increased but not miscalibrated (true and estimated precision are equal). It is only in the setting of precision overestimation that we predict distractibility.

Although rational inattention and policy compression have broad implications for psychiatry, we do not want to leave readers with the impression that capacity limits offer complete explanations of psychopathology. How could they describe such a wide range of phenomena? In brief, they don’t, at least not in isolation. Our general belief is that the psychiatric phenomena we describe should not be seen as lying along a single dimension (capacity), but that disease states will require several dimensions to sufficiently define the relevant symptoms. For example, we propose that both depression and ADHD result from low capacity (under rational inattention), yet mood symptoms are nowhere to be found in the ADHD diagnostic criteria. We view this as an opportunity for computational psychiatry to identify the relevant symptom dimensions.

In mania, for example, we provide some insight into elation, but mood effects also include irritability or anger¹ (Cassidy et al., 1998b). Patients with depression may not exhibit any symptoms of slowing, nor any measurable cognitive deficits (Hack et al., 2023). Even more extreme, symptoms of mania may coexist with symptoms of depression, as in mixed states, in which racing thoughts coexist with psychomotor slowing, or more generally any combination of mood, speed of thought, and psychomotor state (Kraepelin, 1921; Marneros, 2001). The predictions of rational inattention hold for optimal agents, in which reward, precision, and capacity are linked. Perhaps, in mixed states and other disease states, this assumption of optimality is violated. For example, an agent with an optimistic prior and reduced sensory precision may be characterized by what Emil Kraepelin called ‘manic stupor,’ characterized by elevated mood and psychomotor slowing.

The notion that reward, precision, and capacity are linked for optimal agents has consequences for the relevant causal deficit. We highlighted particular deficits (e.g., precision miscalibration in mania) given our intuition for the relevant causal deficit. However, since the theory links multiple variables together, an aberration in any one could lead to the same symptoms. Neuroscience will play a valuable role in determining the relevant causal deficit. Rational inattention has recently gained traction in neuroscience (Mikhael et al., 2021; Grujic et al., 2022; Wu et al., 2022), and we are hopeful for deeper insights in the future.

Both mania and depression can become severe enough that symptoms of psychosis emerge. Since, under rational inattention, mania and depression exist on opposite sides of the spectrum, this observation suggests that psychotic symptoms may emerge via an independent process. Likewise, mania or depression can be complicated by comorbid anxiety. In ADHD, our theories do not yet provide insight into the hyperactivity symptoms that are the more frequently observed consequences of the disorder. In addition, our theory predicts slowed time perception in ADHD, which is at odds with empirical data demonstrating time perception is more rapid and normalizes with treatment (Smith et al., 2002; Ptacek et al., 2019). This highlights either a limitation of our theory, a limitation of our understanding of ADHD, or - most likely - both.

In schizophrenia, policy compression explains several negative symptoms, but it is does not exhaustively explain all negative symptoms, including asociality and amotivation. It also fails to describe anhedonia, although this may not be a failure of the theory, as there is a body of work advocating for intact hedonic drive in schizophrenia (Burbridge and Barch, 2007; Kring and Moran, 2008; Cohen and Minor, 2010; Dowd and Barch, 2010; Yee et al., 2010; Llerena et al., 2012). Neither policy compression nor rational inattention provides insight into the positive symptoms of psychosis, nor how antipsychotics function to reduce these symptoms.

Amotivation is an interesting case study. Amotivation is also a feature of Parkinson’s disease, and is relieved by dopaminergic agonists, at least early in the disease (Pagonabarraga et al., 2015), which we argued increases capacity for more complex policies. Similarly, patients with chronic schizophrenia, who have reduced capacity, also typically suffer from amotivation. Thus, there is suggestive, but incomplete, evidence for a link between capacity and amotivation.

Our discussion of stimulants and antipsychotics is likewise incomplete. Although stimulants and mania have some overlap—stimulant intoxication can manifest as mania—clinical experience makes it obvious that antipsychotics do not phenocopy depression. Although dopamine blockade via antipsychotics is a mainstay of treatment for mania, stimulants are not generally recommended for depression. This is in large part because the neurobiology is far more complicated than we have laid out here, with psychopathology sculpted by the relevant brain structures, neuromodulatory systems, cell types, receptor subtypes and densities, and myriad other details. We have couched capacity limits in the language of dopamine, but we recognize the complexity of disease. Depression, for example, is more commonly thought to involve dysfunction of the serotonergic system (Coppen, 1967; Meltzer, 1990; Owens and Nemeroff, 1994; Harmer et al., 2017), with recent work highlighting a role for extra-neuronal mechanisms (e.g., inflammation; Raison et al., 2006). Stimulants, as another example, involve the release of multiple neuromodulators, not just dopamine (our focus here). In short, a more complete picture will require a theory that encompasses multiple neuromodulatory systems.

Conclusion

Information-theoretic resource rationality provides a rich transdiagnostic language for describing psychopathology. We summarize our perspective in Table 1. Rationality, on this view, does not provide a single solution in phenotype space, but rather a Pareto frontier of optimal solutions. A wide range of psychopathology may be thought of not as suboptimal simply because it results in poorer task performance, but rather optimal performance under an illness-induced capacity limit. Our hope is that these frameworks provide rich ground for development of new theories, behavioral tasks, and for uncovering the neurobiological loci of mental illness.

Table 1:

Summary of psychiatric phenomena explained by rational inattention and policy complexity.

Phenomenon	Mechanistic Hypothesis	Symptoms Explained	Symptoms Not Explained
Mania	Rational inattention Precision miscalibration (overestimation)	Euphoria Optimistic value estimates Increased attention/focus on outside world Psychomotor agitation Rapid speech Rapid passage of time Distractibility	Dysphoria/irritability Mixed states Emergence of psychosis
Depression	Rational inattention Precision miscalibration (underestimation) or precision decrease	Decreased mood Inattentive to outside world/inwardly drawn Subjective “greyness” of experience Psychomotor slowing Slowed speech Slower passage of time Pessimistic value estimates Blunted response to feedback	Psychomotor agitation Irritability Mixed states Emergence of psychosis Depression without cognitive changes
	Rational inattention Decreased attentional incentive	Decreased perception of control (learned helplessness)
	Policy compression Reduced capacity	Working memory deficits Perseveration
Stimulants	Rational inattention Reduction in attentional cost	Euphoria Increased attention/focus on outside world Psychomotor agitation Rapid speech Rapid passage of time	Psychosis
Antipsychotics	Rational inattention Reduced capacity	Cognitive dulling / slowing Limited attention Difficulty perceiving world Exacerbation of negative symptoms	Antipsychotic effects
ADHD	Rational inattention Precision decrease	Failure to give close attention Difficulty sustaining attention Mind seems elsewhere Careless mistakes (exploration) Dislike of mentally effortful tasks Distractibility	Hyperactivity symptoms Faster perception of time
Schizophrenia	Policy compression Reduced capacity	Blunted affect Alogia Working memory deficits	Positive symptoms Amotivation Asociality
Parkinson’s disease	Rational inattention Reduced capacity	Central tendency effect Reduced attention Slower speed of thinking	Motor symptoms Sleep disorder Psychosis
	Policy compression Reduced capacity	Memory problems Language difficulties Word-finding difficulty Naming errors Difficulty with complex sentence structure Dysarthria General problem solving and executive functioning difficulties Reduced response time w/ dopamine therapy
Specific learning disorders	Rational inattention Imprecise sensory input	Underreliance on imprecise sensory input and overreliance on other sensory inputs	Practice improving precision of imprecise signals Reduced motivation to engage with imprecise signals (Matthew effect)
Perseveration	Policy compression Reduced capacity	Tendency to repeat actions across diagnostic entities	Hierarchical nature of perseveration (e.g., repetition of actions in some disorders, repetition of action sets in others)

Appendix: Technical details

This appendix summarizes theoretical results from past papers (Lai and Gershman, 2021; Mikhael et al., 2021; Gershman and Burke, 2022). We refer readers to those papers for further details.

Rational inattention

Suppose an agent is inferring a parameter $\mu$ about the world (this can be expected reward, a temporal interval, an object category, etc.) and it observes a sample $x$. Bayes’ rule prescribes a normative solution, which states that agents can combine what they observe, $P(x\mid \mu )$, with prior information, $P\left(\mu \right)$, to generate a posterior estimate of the parameter, $P(\mu \mid x)$:

$$P(\mu \mid x)\propto P(x\mid \mu )P(\mu ).$$

For analytical tractability, we will assume the sample distribution is Gaussian, $x\sim 𝒩\left(\mu ,{\lambda }^{-1}\right)$, with mean $\mu$ and precision (inverse variance) $\lambda$. If we assume the prior is also Gaussian, $\mu \sim 𝒩\left({\mu }_0,{\lambda }_0^{-1}\right)$, then the posterior is also Gaussian with mean $\hat{\mu }$:

$$\widehat{\mu }={\mu }_0+\frac{\lambda }{\lambda +{\lambda }_0}\left(x-{\mu }_0\right)$$

This equation takes the form of an error-driven update rule, where $x-{\mu }_0$ is the error and $\frac{\lambda }{\lambda +{\lambda }_0}$ is the learning rate, determined by the relative precision between the likelihood and prior. Note that if the agent is inferring reward, then $x-{\mu }_0$ corresponds to the reward prediction error in reinforcement learning theory. Greater relative precision of the prior shifts the posterior estimate closer to the prior (what the agent assumed), whereas greater relative precision of the likelihood shifts the posterior estimate closer to the likelihood (what the agent perceived). Under rational inattention, agents control the likelihood precision $\lambda$. When, then, does it make sense to modulate precision?

First, we must formalize what we mean by attention and how it increases information. The information transmission rate across sensory channels is

$$I(\mu ;x)=H(\mu )-H(\mu \mid x),$$

where $I(\mu ;x)$ is the mutual information between parameter $\mu$ and signal $x,H\left(\mu \right)$ is the entropy of the prior, and $H(\mu \mid x)$ is the entropy of the posterior. Intuitively, high mutual information means that observing the sample reduces uncertainty about the parameter estimate. Low mutual information means the prior and posterior distributions are similar and therefore observing the sample contributes little to uncertainty reduction. Mutual information formalizes what we mean by attention. For our Gaussian generative model, the mutual information is given by:

$$I(\mu ;x)=\frac{1}{2}\text{log}\left(1+\frac{\lambda }{{\lambda }_0}\right).$$

Shannon’s noisy-channel coding theorem states that the minimum number of bits needed to communicate $\mu$ without error across a noisy channel is on average equal to $I(\mu ;x)$. A corollary, therefore, is that errorless communication is impossible if the agent’s capacity is less than $I(\mu ;x)$, in which case there is a trade-off between the cost of attention and the cost of error. This trade-off is analyzed by rate distortion theory (Berger, 1971), which is equivalent to rational inattention (Denti et al., 2020).

Let us assume agents receive reward inversely proportional to the squared error $ϵ=(\mu -\hat{\mu }{)}^2$, which should motivate agents to reduce their error. Intuitively, if $\lambda$ is small, $\mu$ will migrate to the prior and cause a large error between the latent source and its estimate. As $\lambda$ grows large, this error is reduced. For simplicity, let us assume that the reward agents receive, $u\left(ϵ\right)$, is a monotonically decreasing and differentiable function of this error. Expanding the first-order Taylor series approximation around $ϵ=0$ yields $u\left(ϵ\right)\approx u\left(0\right)-\theta ϵ$, where $\theta >0$ is the negative slope of $u\left(ϵ\right)$ at $ϵ=0$. We can interpret $\theta$ as an attentional incentive parameter to capture the idea that agents should be motivated to pay attention when reward is contingent on error. The expected reward is therefore $U=\mathbb{E}\left[u\right(ϵ\left)\right]\approx u\left(0\right)-\theta \mathbb{E}\left[ϵ\right]$.

We can now write down the constrained optimization problem faced by agents

$${\lambda }^{*}=\underset{\lambda }{\text{argmax}}U-\kappa I(\mu ;m)$$

where $\kappa$ is the Lagrange multiplier. $\kappa$ can be interpreted as the attentional cost, to formalize the notion that attention is effortful. $\kappa$ can be thought of as an “exchange rate:” one unit of reward can be “bought” for $\kappa$ units of information. Written this way, $\kappa$ implicitly represents the capacity limit, with a large $\kappa$ representing low capacity (many bits for one reward) and small $\kappa$ representing high capacity (few bits for one reward).

This equation formalizes the tradeoff between reward $\left(U\right)$ and information: agents can increase $\lambda$ to increase reward $U$ by reducing their error $ϵ$, but doing so increases the information rate $I(\mu ;m)$, which the agent must keep at or below capacity. Solving this constrained optimization problem yields:

$${\lambda }^{*}=\text{max}\left(0,\frac{2\theta }{\kappa }-{\lambda }_0\right).$$

The optimal precision increases as 1) the attentional incentive, $\theta$, increases, 2) the attentional cost, $\kappa$ decreases, and 3) the prior precision, ${\lambda }_0$, decreases.

For the Gaussian generative model we described above, the optimal expected reward is:

$$U^{*}\approx u(0)-\frac{\theta }{{\lambda }^{*}+{\lambda }_0}=u(0)-\frac{\kappa }{2}.$$

Mikhael et al. (2021) used this equivalence to posit a rational inattention account of tonic dopamine. The authors propose a model in which tonic dopamine encodes average reward, $U$, and by the equivalence demonstrated above, also encodes posterior precision, ${\lambda }^{\mathrm{*}}+{\lambda }_0$ and the information-reward exchange rate, $\kappa$ (and implicitly, the capacity).

Policy compression

We model an agent that visits states, $s$, and takes actions, $a$, to earn reward. States are defined as the representation of information needed for reward prediction. Each state is visited with probability $P\left(s\right)$ and an action is chosen according to a policy $\pi (a\mid s)$, a probabilistic mapping from states to actions. We conceptualize the policy as a communication channel mapping states to actions. The minimum number of bits to achieve error-free communication of the state identity is given by the mutual information between states and actions:

$$I(S;A)=\sum _{s}P(s)\sum _a\pi (a\mid s)\text{log}\frac{\pi (a\mid s)}{P(a)},$$

where $P\left(a\right)={\sum }_{s}P\left(s\right)\pi (a\mid s)$ is the marginal action distribution. We use the term policy complexity to refer to $I(S;A)$. Intuitively, a policy with high complexity is highly state-dependent (e.g., each state maps uniquely to an action), whereas low complexity policies are more state independent. Similar to our derivation of rational inattention, we assume our agent is capacity limited, which induces a trade-off between policy complexity and reward. Agents must therefore compress the optimal policy if they lack the channel capacity to achieve error-free communication.

We can therefore define a joint optimization problem where the agent seeks to maximize reward subject to a capacity constraint. We define the optimal policy², π*, as:

$${\pi }^{*}=\underset{\pi }{\text{argmax}}\beta V^{\pi }-I^{\pi }(S;A),$$

where $V^{\pi }$ is the expected reward under policy $\pi$:

$$V^{\pi }=\sum _{s}P(s)\sum _a\pi (a\mid s)Q^{\pi }(s,a).$$

For analytic tractability, we assume that an agent either learns or has direct access to the action-value function $Q(s,a)$ which defines the expected reward after taking action $a$ in state $s$. Note that we place the Lagrange multiplier, $\beta$, on $V^{\pi }$ instead of $I^{\pi }(S;A)$ since it permits a more straightforward connection to reinforcement learning process models, as we will see.

Solving this equation yields the optimal policy, ${\pi }^{\mathrm{*}}$:

$${\pi }^{*}(a\mid s)\propto \text{exp}\left[\beta Q(s,a)+\text{log}{P}^{*}(a)\right],$$

where $P^{\mathrm{*}}\left(a\right)={\sum }_s{\pi }^{\mathrm{*}}(a\mid s)$ is the optimal marginal action distribution. The optimal policy is the ubiquitous softmax function, used widely in the reinforcement learning literature; the Lagrange multiplier, $\beta$, plays the role of the inverse temperature parameter, governing the exploration-exploitation trade-off. Note that our derivation of the optimal policy made no appeal to exploration/exploitation, which instead arose as a natural consequence of resource constraints. The precise value of $\beta$ is a function of the policy complexity:

$${\beta }^{-1}=\frac{d{V}^{\pi }}{d{I}^{\pi }(S;A)}.$$

At low policy complexity, where $\frac{d{V}^{\pi }}{d{I}^{\pi }(S;A)}$ is steep, the optimal $\beta$ is close to 0. In this regime, $Q$-values have minimal impact on the optimal policy and the marginal action distribution, $P^{\mathrm{*}}\left(a\right)$ dominates. In other words, at low policy complexity, state-independent actions dominate, an insight we have previously used to explain perseveration (Gershman, 2020).