Modeling subjective belief states in computational psychiatry: Interoceptive inference as a candidate framework

Abstract

The nascent field of computational psychiatry has undergone exponential growth since its inception in the early 2010s. To date, much of the published work has focused on choice behaviors, which are primarily modeled within a reinforcement learning framework. While this initial normative effort represents a milestone in psychiatry research, the reality is that many psychiatric disorders are defined by disturbances in subjective states (e.g. depression, anxiety, etc.) and associated beliefs (e.g., dysmorphophobia, paranoid ideation, etc.), which are not considered in normative models. In this paper, we present interoceptive inference as a candidate framework for modeling subjective – and associated belief – states in computational psychiatry. We first introduce the notion and significance of modeling subjective states in computational psychiatry. Next, we present the interoceptive inference framework, and in particular focus on the relationship between interoceptive inference (i.e. belief updating) and emotions. Lastly, we will use drug craving as an example of subjective states to demonstrate the feasibility of using interoceptive inference to model the psychopathology of subjective states.

Part I. Modeling subjective belief states in computational psychiatry research: significance and challenges

Computational psychiatry is a new and interdisciplinary field that seeks to understand the mechanisms underlying mental function and dysfunction using computational approaches (Friston et al. 2014; Maia and Frank 2011; Montague et al. 2012). This nascent field has enjoyed much success since its inception, providing powerful explanations and predictions for a wide range of disorders; including schizophrenia and psychosis (Adams et al. 2013b; Braver et al. 1999; Powers et al. 2017), depression (Rutledge et al. 2017), bipolar disorder (Mason et al. 2017), anxiety (Browning et al. 2015), autism (Lawson et al. 2017; Lawson et al. 2014), ADHD (Hauser et al. 2016; Hauser et al. 2014; Hauser et al. 2017b), OCD (Gillan et al. 2016; Hauser et al. 2017a), and addiction (Fiore et al. 2018; Gu 2018; Gu and Filbey 2017; Redish and Johnson 2007). The majority of these studies and theories have focused on overt behaviors, and in particular choice behaviors, using computational models of learning and decision-making. A handful of studies constitute first efforts to model subjective feelings using a decision theoretic approach [i.e. consider subjective states as corollaries of choice behaviors and decision processes; see (Rutledge et al. 2017) for an example and (Bach and Dayan 2017) for a review]. However, there is a substantial body of computational work that models subjective data directly in other domains of cognitive neuroscience - such as metacognition (Allen et al. 2016; Fleming and Daw 2017) and pain research (Jepma et al. 2018; Wiech 2016) - all of which could be crucial for modeling subjective states in computational psychiatry.

As Helen Keller famously said, “The best and most beautiful things in the world cannot be seen nor even touched, but just felt with the heart.” (Keller 1881). The subjective nature of internal feelings and beliefs presents several major challenges to the scientific investigation of the mental processes that underwrite them - and the definitions of subjective states are often loose and inconsistent. For instance, the definition of emotion differs drastically between different theories, disciplines, and studies (Bach and Dayan 2017; Gu et al. 2013; James 1884; Lange and James 1922; Lazarus 1991; LeDoux 2000; Papez 1937; Pessoa and Adolphs 2010; Phelps 2006; Picard et al. 2001; Schachter and Singer 1962). Here, we define subjective states as an agent’s beliefs about the world - either external or internal (see Part II for more detailed discussions). Second, subjective states are difficult to measure. Self-reports are probably the more direct and closest to one’s true internal feelings; yet these reports could sometimes be partially obscured by the individual’s ability to introspect and access their internal world (e.g. awareness or metacognition). On the other hand, objective measures, such as functional magnetic resonance imaging (fMRI) or electroencephalography (EEG), often provide signals that are not specific to distinct subjective states. For example, there is a considerable overlap between brain activations related to different emotions such as fear, disgust, and anger [see (Fusar-Poli et al. 2009; Murphy et al. 2003; Phan et al. 2002) for meta-analyses]. Lastly, existing analyses of subjective states are largely correlational and lack mechanistic or computational explanations. One might argue that these challenges must be overcome if affective neuroscience and computational psychiatry are to move forward.

Aberrant subjective states are prevalent across almost all disorders. Such examples include persistent feelings of sadness in mood disorders (APA 2013) and sticky beliefs in psychosis (Friston 2005b; Powers et al. 2017; Sterzer et al. 2018). While some subjective states can affect overt behaviors and related brain responses - and can thus be assessed indirectly through these choice behavior-related measures (Chiu and Deldin 2007; Huys et al. 2015) - many might not directly affect nor be inferred using choice behaviors (Chung et al. 2017; Rutledge et al. 2017). Take depression as an example. The literature suggests that depression affects many overt behaviors (and related neural circuits) such as error detection (Chiu and Deldin 2007; Pizzagalli et al. 2006), processing stimuli related to the self (Lemogne et al. 2009), accuracy in a reward reversal task (Robinson et al. 2012), and attention (Tian et al. 2016). However, two recent studies - using computational modeling and monetary decision-making tasks - demonstrate that depression did not affect reward sensitivity (Chung et al. 2017) or reward prediction error encoding (Rutledge et al. 2017). These studies represent some of the first efforts to formally quantify depression-related behaviors with computational modeling (i.e., reinforcement learning models). Nevertheless, the null findings from both studies could indicate that a decision-theoretic neuroscience approach alone may not help us disclose the complex nature of subjective feelings, and that direct modeling of subjective data and related neural and physiological concomitants might be necessary.

Part II. Interoceptive inference as a candidate framework

Here, we introduce a framework of interoceptive inference that has recently emerged from the computational neuroscience literature to explicitly account for subjective states. As it is typically described (Seth and Friston 2016), interoceptive inference makes two related but dissociable claims. The first claim is that approximate Bayesian inference about physiological states of the body underlies feeling states [see (Barrett and Simmons 2015; Gu and FitzGerald 2014; Gu et al. 2013; Pezzulo 2014; Pezzulo et al. 2015; Seth 2013; Seth 2014; Seth and Friston 2016) for a sample; also see a special issue in Phil. Trans. R. Soc. B (Tsakiris and Critchley 2016)]. This entails an important claim that ‘feelings’ are a certain kind of belief states that are updated during inference to provide the best explanation for interoceptive sensations (e.g., “I am anxious” is the best explanation current bodily sensations). The second claim is that physiological states, generating interoceptive sensations, are controlled using active inference (Seth and Friston 2016). In other words, autonomic reflexes work to align internal states with descending predictions from an agent generative model of herself. This is precisely analogous to active inference as a tool for controlling the motor system (Adams et al. 2013a). Taken together, interoceptive inference, with these two major claims, reconciles the conflict between the James-Lange and the Canon-Bard theories in the sense that they are both right - that subjective feeling or belief states are both cause and consequence of autonomic states.

Understanding mental states as Bayesian inference has a number of compelling aspects. In particular, it prescribes ‘first principle’ strategies for integrating existing beliefs with new information and for dealing with uncertainty; problems that are of key importance for enabling complex, adaptive behavior. It also naturally leads to mechanistic hypotheses about how such processes might be implemented (Aitchison and Lengyel 2016; Friston 2005a; Friston et al. 2017; Ma et al. 2006). This approach has been successful in accounting for a wide range of cognitive and neural phenomena; such as perception, learning, and memory [see (Doya 2007; Friston 2010; Knill and Pouget 2004; Moutoussis et al. 2014) for a sample of papers]. With the rise of the new field of computational psychiatry, Bayesian approaches have also proved to be successful in explaining aberrant perception and cognition in many psychiatric disorders such as schizophrenia (Moutoussis et al. 2011), autism (Pellicano and Burr 2012), and addiction (Gu and Filbey 2017; Schwartenbeck et al. 2015). In contrast to the fruitful results of Bayesian formulations in other areas of neuroscience and psychology, however, its application to subjective (feeling) states is rare.

In this article, we aim to describe how emotions - either basic (e.g. joy, fear) or social (e.g. shame, guilt) - can be formulated in the setting of interoceptive inference. Specifically, we consider emotions as Bayesian beliefs about interoceptive states. The mismatch between one’s prediction and the actual interoceptive signal contributes to an “interoceptive prediction error” signal. Agents seek to minimize this prediction error and can do so in one of two ways. First, they can alter their beliefs about internal states (perception). Second, they can alter their internal states to fit their beliefs (action, under active inference). This corresponds to the two aspects of interoceptive inference described above. If true, this model resolves a longstanding debate about the relationship between bodily signals and emotions, the center of debate between James-Lange and Cannon-Bard theories.

Specifically, William James (1842–1910) and Carl Lange (1834–1900) proposed that bodily responses, including physiological responses of the muscles, skin, and viscera, precede conscious emotional feelings (e.g. “my heart races therefore I perceive fear”) (James 1884; Lange and James 1922). In other words, emotions are the result of changing bodily states. On the contrary, Walter Cannon (1871–1977) and his student Philip Bard (1898–1977) proposed that physiological responses of the body and subjective experiences of emotion can occur independently from each other (Bard 1928; Cannon 1927). This view was developed based on experimental findings contradicting the James-Lange theory. For example, physiological responses of the body could manifest more slowly than human subjects’ reports of changed feelings (Bard 1928; Cannon 1927). While these early theories tried to determine a unidirectional relationship between bodily responses and subjective experience, under interoceptive inference, both processes are in play concurrently, suggesting that the brain-body relationship is a two-way street (Barrett and Simmons 2015; Gu and FitzGerald 2014; Ondobaka et al. 2017; Seth and Friston 2016). This explains the prima facie plausibility, but also incompleteness, of these two classic approaches to understanding emotion.

Cognitive labeling theories of emotion, first developed in the 1960s and 1970s, offered a more nuanced view of emotions and bodily states (Dutton and Aron 1974; Schachter 1964; Schachter and Singer 1962; Slochower 1976; Valins 1966). Despite some minor differences, these theories, in essence, all suggest that emotions do not solely depend on changes in bodily states; instead, cognitive processes are involved such that the same bodily responses can be interpreted in different ways depending on context. For example, increased heart rate could be felt as excitement in the context of a positive event (e.g., job promotion) but fear in a threatening situation (e.g., robbery). In the Capilano Suspension Bridge experiment - a notably study that examined sexual arousal and romantic relationships (Dutton and Aron 1974) - heterosexual male participants were approached by an attractive female on either a fear-arousing suspension bridge or a non-fear-arousing bridge and were asked to complete questionnaires related to the Thematic Apperception Test pictures (that lacked obvious sexual content). The sexual content of reports and attempts to contact the female experimenter post-experiment with greater in the suspension bridge condition. These effects were not observed if the (male) participants were approached by a male. Findings from this study were interpreted as evidence for cognitive labeling theory - that bodily arousal was labeled by people in the context sensitive fashion. Here, we propose that contextual cues and autonomic signals are assimilated as sensory evidence during interoceptive inference - to produce a posterior belief that best explains both; namely, a cognitive label or explanation. In this sense, interoceptive inference can be viewed as a formal (Bayesian) description of cognitive labeling. On this reading, interoceptive inference may provide an account of the mental and neuronal belief updating that underwrites cognitive labeling.

From the perspective of computational psychiatry, interoceptive inference is especially prescient as it provides a formal, quantitative approach (at least in principle) to understanding subjective feeling states. Existing computational work looking at affective disorders typically appeals to the framework of reinforcement learning and accompanying claims about dopaminergic function (Huys et al. 2015; Nestler and Carlezon 2006). However, while there is likely to be a close relationship between reward and certain aspects of affective states (Huys et al. 2015; Rutledge et al. 2017; Whitton et al. 2015), affective and motivational states (or ‘liking’ and ‘wanting’) are known to be dissociated at both behavioral and neurobiological levels (Berridge 2012; Robinson and Berridge 1993). Crucially, subjective (feeling) states have attributes that cannot be captured by a simple scalar reward. For these reasons, there is a natural need to develop and test formal accounts of interoceptive inference - and how it may relate to emotion in both normal and pathological cognition.

Interoceptive inference is a young field, and has undergone substantial theoretical developments over the past few years (Barrett and Simmons 2015; Gu and FitzGerald 2014; Ondobaka et al. 2017; Owens et al. 2018; Petzschner et al. 2017; Pezzulo et al. 2015; Seth 2013). These advances include the delineation of computational blocks involved in interoception (Petzschner et al. 2017), its relationship to self-perception (Seth 2013), emotion (Barrett and Simmons 2015; Gu et al. 2013), and embodied cognition and decision-making (Allen and Friston 2018; Gu and FitzGerald 2014), and importantly, its relevance to psychosomatic and psychiatric disorders (Owens et al. 2018; Petzschner et al. 2017). As such, empirical evidence that directly supports the predictions of this account is still limited (see (Allen et al. 2016; Gentsch et al. 2019; Jepma et al. 2018) for a few examples), but one might anticipate empirical results that endorse or refute its utility, over the next few years. One clear prediction is that emotional impairments should exist in people with deficits in the processing of bodily signals (i.e., due to imprecise sensory data) and emotional function could improve in individuals with better interoceptive accuracy; i.e., due to more precise sensory data; as reported in (Barrett et al. 2004). Relatedly, this model also predicts that strategies - that can help individuals recognize or infer explicitly (i.e., mentalize) the causes of bodily signals correctly - should also improve affective processing and responses. For example, psychotherapy that can re-interpret the emotional content and meaning of increased heart rate may help individuals with panic attacks.

There has been one fMRI study that directly applied Bayesian modeling to quantify subjective feelings (Xiang et al. 2013). Participants were asked to rate how happy they felt about a monetary offer they received while being scanned. Using an ideal Bayesian observer model, the authors found that feeling prediction errors were encoded in the nucleus accumbens, vmPFC, and posterior cingulate, whereas feeling variance prediction errors were computed in bilateral anterior insula. To date, this remains still the first and only fMRI study to use the Bayesian framework to model subjective feelings directly.

A recent study using EEG also provided direct evidence supporting the interoceptive inference hypothesis (Gentsch et al. 2019). In this study, participants viewed either neutral or angry faces followed by either the same (repetition; e.g. neutral-neutral, angry-angry) or different (alternation; e.g. neutral-angry, angry-neutral) emotional faces. Interoceptive responses were measured with heartbeat-evoked potential (HEP), which is the EEG response time-locked to the electrocardiogram (ECG) R-peak of the heartbeat. The authors found that angry faces, preceded by angry faces, elicited attenuated HEPs compared to those proceeded by neutral faces. In other words, the affective prediction of upcoming stimuli modulated interoceptive responses to those stimuli. Such differences between predicted and unpredicted emotional stimuli were not found for neutral stimuli; suggesting that the top-down predictions of interoception are based on bottom-up interoceptive responses. Taken together, these results shed light onto the intricate interactions between interoceptive predictions and inputs and provide initial support for the interoceptive inference framework. More empirical studies are needed to echo such efforts.

Part III. Drug craving as an example

In this last section, we consider a recent computational theory of drug craving, as an example of how interoceptive inference can be used to account for subjective states in particular disorders. Craving refers to an intense, urgent, or abnormal desire or longing, according to the Merriam-Webster dictionary. Naturally, this definition suggests that craving is a subjective state and is much more difficult to measure objectively and quantitatively. Clinically, craving directly relates to the subjective well-being and life quality of the individual, and can be dissociated from drug-related choice behaviors (Tiffany et al. 2012; Tiffany and Wray 2012). In humans laboratory studies, craving has been extensively studied using cue-exposure paradigms [see (Chase et al. 2011; Engelmann et al. 2012; Jasinska et al. 2014; Tang et al. 2012; Yalachkov et al. 2012) for reviews and meta-analyses]. However, it remains controversial as to what psychological processes are actually elicited by these paradigms and how they relate to real-life craving (Shiffman et al. 2015). For instance, drug cues are inherently valuable to addicted individuals and could thus induce reward processing, along with craving. Cue-elicited response studies typically contrast brain activities elicited by drug cues with those induced by non-drug cues (e.g. cigarette vs. pencil) and have reported widespread activations in dopaminergic and limbic regions including the midbrain (ventral tegmental area, VTA), ventral striatum, insula, anterior cingulate cortex (ACC), ventromedial prefrontal cortex (vmPFC), amygdala etc. (Chase et al. 2011; Engelmann et al. 2012; Jasinska et al. 2014; Tang et al. 2012; Yalachkov et al. 2012). Many of these regions are involved in value encoding (Rangel et al. 2008; Rushworth and Behrens 2008). It is thus difficult to tease apart the neural mechanisms underlying subjective craving vs. those supporting value encoding in cue-exposure paradigms.

We recently proposed the first Bayesian model of craving (Gu and Filbey 2017), building upon the interoceptive framework (Barrett and Simmons 2015; Gu and FitzGerald 2014; Gu et al. 2013; Seth and Friston 2016). Specifically, we consider craving as a posterior belief about bodily states associated with the availability of addictive substances (Gu and Filbey 2017). This inference model of drug craving has proven effective in accounting for several important experimental findings that are not explained by purely pharmacological models of drug addiction. In the human addiction literature, for example, it has been shown that craving depends on not only the availability of the addictive substance in the body, but also people’s beliefs about the presence of that substance (Gu et al. 2015; Juliano et al. 2011; Kelemen and Kaighobadi 2007; McBride et al. 2006). In addition to the well-known placebo effects where people respond to ‘sugar pills’ without active substances (McCusker and Brown 1990; Robinson and Berridge 1993), recent studies have demonstrated that - in nicotine addiction - craving was reduced (Gu et al. 2016) and ventral striatal responded to value (Gu et al. 2015) only when smokers had a nicotine cigarette and also believed there was nicotine in the cigarette, but not when they thought otherwise. These findings are complementary to previous studies reporting placebo effects; in the sense that both sets of results point to the importance of considering both prior beliefs (i.e., expect drug or no drug) and sensory evidence (drug delivered or not). These results are also interesting, in the sense that placebo effects primarily describe how beliefs can ‘up’ responses in the absence of active substances, while the Gu et al studies demonstrate how prior expectations can also ‘down-regulate’ behavioral and neuronal responses in the presence of active substances, such as nicotine. Using an interoceptive inference framework, we were able to simulate these findings (Gu et al. 2016) by systematically manipulating prior beliefs (e.g., whether the smoker expected to receive a cigarette with nicotine or placebo cigarette) and the likelihood of drug administration (e.g. whether the cigarette has nicotine or not). This work (Gu and Filbey 2017) suggests that the posterior beliefs - that undergird craving - are ameliorated when a drug is administered and when the individual expects the drug, compared to the other three alternatives (i.e. expect drug & drug absent, expect no drug & drug present, expect no drug & drug absent), corroborating empirical findings (Gu et al. 2015; Juliano et al. 2011; Kelemen and Kaighobadi 2007; McBride et al. 2006).

Incubation of craving, which refers to the effect that craving increases rather than decreases during early abstinence, is another important finding that remained unexplained by any previous computational framework. In one recent paper, we used the same model to further simulate previous experimental findings (Bedi et al. 2011; Conrad et al. 2008; Grimm et al. 2001; Lu et al. 2005; Parvaz et al. 2016) that craving could increase over time during early abstinence (Gu 2018). Taken together, this Bayesian framework has been proven powerful in accounting for craving in addiction, and provides a more general illustration of the potential for interoceptive inference models in building novel and formal hypotheses about the processes underlying psychiatric disorders.

Conclusion

In this short paper, we offer a viewpoint that interoceptive inference can serve as a candidate framework for modeling subjective states in computational psychiatry research. While we emphasize the importance of modeling subjective data directly [e.g. following similar practice in other areas of cognitive neuroscience (Allen et al. 2016; Fleming and Daw 2017; Jepma et al. 2018; Wiech 2016)], this view does not diminish the significance of modelling overt behaviors in terms of their subjective causes. Instead, we suggest that subjective experience can - and should be - modelled directly, and the interoceptive inference framework is a natural fit for this purpose. For example, in depression research, models of sad feelings and models of aberrant learning and decision-making are both necessary in order to achieve a comprehensive view of depression. More empirical studies will be needed to directly examine the validity of the interoceptive inference framework in both health and disease.