Representation of Anticipated Rewards and Punishments in the Human Brain

Abstract

Subjective value is a core concept in neuroeconomics, serving as the basis for decision-making. Despite extensive literature on the neural encoding of subjective value of reward in humans, the neural representation of punishment value remains relatively understudied. This review summarizes the current understanding of the neural representation of reward value, including the methodologies, involved brain regions, and the “common currency” representation of different types of rewards in decision-making and learning. We then survey existing research on punishment value, highlighting challenges in human studies of punishments and insights from animal research. We also connect individual differences in reward and punishment processing to stress-related mental disorders. The review advocates for the integration of both rewards and punishments within decision-making and learning frameworks, leveraging insights from cross-species studies, and utilizing ecological gamified paradigms to reflect real-life scenarios.

1. INTRODUCTION

“Nature has placed mankind under the governance of two sovereign masters, pain and pleasure. It is for them alone to point out what we ought to do, as well as to determine what we shall do.”

(Bentham, 1789)

Improving well-being is the primary goal of human behavior. It involves learning about potential rewards and punishments in the environment, making beneficial choices that maximize benefit and minimize harm, and updating the learning when circumstances change. At the heart of these processes is the concept of subjective value. Decision-making models suggest that to make choices individuals estimate and compare the subjective values of available options either to each other or to a predefined threshold (Kable & Glimcher, 2009).

Substantial evidence supports this description of the choice process, documenting correlations between properties of neural activation and measures of value (we refer to these signals as “value-related”). Moreover, much of the evidence is consistent with the existence of a unified value system, in which values of options across different categories and conditions are standardized into one common scale (Bartra et al., 2013; Clithero & Rangel, 2014; D. Levy & Glimcher, 2012). Such a “common currency” representation of value is intuitive and appealing, as it facilitates direct comparisons across different rewards, such as choosing between a restaurant and a movie for a night out. Most of this evidence, however, comes from positive domains, such as food (D. Levy & Glimcher, 2011), money (Kable & Glimcher, 2007), and nonfood consumables (Chib et al., 2009); similar evidence in negative domains is scarce (I. Levy & Schiller, 2021).

Historically, it was believed that positive and negative affective valences were processed by distinct neural systems: the ventral striatum (VS) for rewards and the amygdala for punishments. Subsequent research, however, proposed a unified neural system evaluating both (Tom et al., 2007). Recent findings present a more nuanced picture, where certain neurons/regions/circuits specialize in processing both rewards and punishments, while others are dedicated solely to one type of valence (Berridge, 2019; Tye, 2018). This review seeks to synthesize existing evidence and identify critical gaps in understanding how the brain encodes the value of anticipated punishments, an underexplored area within the vast literature on the encoding of reward value. Recent associations between psychopathologies and alterations in value-based learning and decision-making, with rewards or punishments, underscore the importance of understanding the neural basis of value to improve mental health diagnosis, prevention, and treatment.

We begin with a brief survey of how economists and neuroscientists have historically studied anticipation of rewards and punishments. Next, we examine the current understanding of neural representations of anticipated rewards, including different methodologies, key findings, and areas that remain debated or unknown. In contrast to the extensive research on rewards, the study of anticipated punishments remains limited. We discuss the sparse research, inconsistent findings, insights from animal studies, and open questions. Lastly, we explore the significant role of reward and punishment processing in various mental disorders and suggest directions for future research.

2. THE HISTORY OF VALUE

2.1. Subjective value

To discuss the neural representation of value, we first need to define “value”, a term that varies in meaning across contexts (O Doherty, 2014). Everything we encounter holds inherent or acquired value, manifesting as rewards (e.g., gaining $10 or consuming a delicious treat) or punishments (e.g., losing $10 or incurring a painful stimulus). A key question is how the brain recognizes these values upon encountering them. One proposal is that the brain handles these inputs through complementary “motifs” where, depending on the inherent or acquired value, information is integrated, diverged, or kept separate to balance efficiency and flexibility (Tye, 2018). The concept of value extends to “chosen value” (Padoa-Schioppa & Assad, 2006), which is the anticipated outcome of a selected option before it is received (e.g., the expected enjoyment of a chocolate cake before eating it). For comprehensive reviews on different types of value, see (Enel et al., 2021; Peters & Büchel, 2010; Rangel et al., 2008).

Here, we focus on the value of available options before a decision is made, which can be used as a basis for choice – often referred to as “offer value” (Padoa-Schioppa & Assad, 2006) or “decision value” (Peters & Büchel, 2010). We adopt the term anticipated value (Kahnt et al., 2010) as it is suitable even in scenarios where no decision is involved, such as passive viewing (I. Levy et al., 2011) or associative learning (Sescousse et al., 2015). Anticipated value integrates across multiple dimensions of each available option, such as the taste and health of food, its cost or the effort required to obtain it, and the likelihood and timing of receiving it. It also considers external context (Tymula & Plassmann, 2016) and internal states, such as hunger (D. Levy et al., 2013), thirst (C. Becker et al., 2015), fatigue (S. Harris & Bray, 2021) and mood (Rutledge et al., 2014). Importantly, anticipated values are subjective - specific to the situation and the individual (Falk & Scholz, 2018; Hayden & Niv, 2021; Kable & Glimcher, 2009; I. Levy & Schiller, 2021; Rangel & Clithero, 2012). For instance, a chocolate cake may hold a lower subjective value for an individual following a calorie-restrictive diet compared to another without such restrictions, even if they share an equal appreciation for its taste. Furthermore, both individuals will likely devalue the cake after consuming a similar one. Studying the neural representation of anticipated outcomes thus requires accurate estimates of these subjective values, based on behavioral or physiological measures (Figure 1). Research on subjective value in economics and neuroscience addressed this concept from different perspectives, which culminated in the field of neuroeconomics (Glimcher & Fehr, 2013).

Figure 1. Experimental paradigms for studying the neural representation of reward value.

Three typical measures estimate the subjective value of a reward: rating, where participants explicitly report how much they like the stimulus on a certain scale, from very little to very much (e.g., from 1 to 5); willingness to pay, where participants state the amount of money they are prepared to spend for this reward, either hypothetically or with an endowment (e.g., between $1 and $5); and choices, where participants do not explicitly report their subjective values but decide between different rewards (e.g., apple versus pear, apple versus grapes), through which their preferences and value estimates can be revealed. In addition to choices between different rewards, individuals can be asked to choose between an immediate small reward and a delayed larger reward (delay), between a small certain reward and a larger uncertain reward (uncertainty), or between a small reward with minimal effort and a larger reward that requires high effort (effort). In reinforcement learning studies, values are learned during the experiment, either passively (Pavlovian learning) or through making choices (instrumental learning). In the depicted examples, a blue triangle is repeatedly paired with a small monetary reward (one coin), whereas a yellow square is paired with a large monetary reward (bag of coins). Reward values are estimated in the behavioral paradigms based on either the behavior itself or different physiological measures. Behavioral measures include explicit self-reports or willingness to pay, model-free measures of choice behavior (e.g., the proportion of trials in which an option is chosen), and model-based estimates of latent variables. Physiological measures include pupil dilation, skin conductance responses, and heart rate variability. These estimates of value are then used in the neural analysis to search for correlations between value estimation and neural measures (e.g., activation magnitude, activation patterns, connectivity patterns). Figure created by Evelyn Pence.

2.2. Value in neuroscience and economics

The notion that choices are based on subjective estimates of value (or “utilities”) dates back to Bernoulli, who recognized the diminishing marginal utility of increasing monetary amounts (Bernoulli, 1738; Carlitz, 1954). In the 20^th century, Paul Samuelson and others showed that consistent decision-makers behaved as if they attempted to maximize utility on a single common scale (Samuelson, 1947). Based on a few simple axioms, Von Neumann and Morgenstern (1944) developed the expected utility theory, a model for choice under uncertainty, later refined by Kahneman and Tversky into the powerful prospect theory (1979). These decision models aimed to predict choices accurately, and were quite successful at that; whether decision-makers truly attempted to maximize subjective values was deemed irrelevant. Neuroscientific studies, on the other hand, aim to address the choice mechanism, by searching for neural correlates of subjective value. Early studies, using electrophysiology recordings in animals (Platt & Glimcher, 1999) or neuroimaging in humans (Delgado et al., 2000), manipulated objective aspects of value (e.g., outcome magnitude and probability) and observed corresponding signal changes in the brain. More recent studies borrow techniques from economics and psychology to infer subjective values and identify their neural correlates (Dorris & Glimcher, 2004; Hare et al., 2009; Kable & Glimcher, 2007; Lau & Glimcher, 2008; Plassmann et al., 2007; Sugrue et al., 2004). As neuroscientists integrated behavioral economics methods into their research, economists turned to neuroscience to obtain biological constraints on their models, leading to the birth of “neuroeconomics.”

In the upcoming sections, we examine our current understanding of how the brain encodes the subjective value of anticipated outcomes. Given that similar brain areas are implicated in both received and anticipated outcomes (Bartra et al., 2013), we also incorporate relevant findings on received outcomes, to the extent that they can inform our main questions of interest. Additionally, we draw insights from animal studies to complement human functional MRI (fMRI) research.

3. THE NEURAL REPRESENTATION OF SUBJECTIVE VALUE OF REWARDS

3.1. Common experimental paradigms

Human neuroimaging studies often rely on behavioral paradigms to estimate subjective values. The most straightforward method is asking participants to explicitly rate available options (Figure 1). This method, however, may be limited by participants ability or willingness to provide accurate reports. To overcome this limitation, economists devised experimental paradigms ensuring “incentive compatibility”, where choices made in the lab had real-life consequences. For example, participants may be asked to state their “willingness to pay” for an item, and then pay that amount from an endowed sum (G. Becker et al., 1964; Plassmann et al., 2007). The most common paradigm is that of revealed preference, where decision-makers disclose their preferences through choices rather than directly reporting them (Samuelson, 1947). An advantage of this method is that it can also be used in animal studies, for example, when monkeys choose between different quantities of two different juices (Padoa-Schioppa & Assad, 2006). While each monkey had its preferred juice, more juice was generally favored over less. By manipulating the quantities of the offered juices, Schioppa and Assad (2006) estimated the additional amount needed from the less preferred juice to render the monkey indifferent between the options (Padoa-Schioppa & Assad, 2006). This “indifference point” serves as an estimate of cardinal value, providing a quantitative basis for neuroeconomic models of choice.

In humans, many choice experiments require participants to choose between a variable reward option and a fixed reference (Figure 1). By keeping one option constant, changes in brain activity could be attributed to the varying option. More complex designs, can distinguish between value representations of single options, the sum of all options (Shenhav & Karmarkar, 2019; Williams et al., 2021), or the difference between values of available options (FitzGerald et al., 2009; Hunt et al., 2012). To enhance participant motivation, one choice is typically randomly rewarded with a real outcome at the end of the study. Such designs manipulate reward identity (Hare et al., 2009), time delay for receipt of reward (Kable & Glimcher, 2007), the likelihood of receiving it (i.e., uncertainty) (I. Levy et al., 2010), or the effort required for obtaining the reward (Lopez-Gamundi et al., 2021). As in the monkeys study (Padoa-Schioppa & Assad, 2006), parametric manipulations of choice characteristics are used to identify participants subjective values for different options.

But where do these values come from? studies of value learning, in which participants learn to associate cues or actions with outcomes (Figure 1), can also shed light on the origins and neural representation of value. Pavlovian learning tasks typically involve the presentation of cues and rewards without any required choice (e.g., passive viewing) (Van Leijenhorst et al., 2010; Zhang et al., 2015). Instrumental learning tasks require active choice between different options, where values are learned by trial and error (J. O Doherty et al., 2004; Ostlund & Balleine, 2007). Unlike the choice paradigms described above, in instrumental learning, participants discover the consequences of their choices and update the value of chosen options accordingly. Reinforcement learning models, used to describe both Pavlovian and instrumental learning, suggest that values are updated based on the discrepancy between the anticipated and obtained outcomes, termed prediction error. While instrumental learning tasks allow value estimation from choice behavior, Pavlovian learning tasks require additional measures, either explicit (e.g., ratings) or implicit (e.g., pupil dilation, skin conductance response, heart rate variability) (J. Li et al., 2011; Tzovara et al., 2018).

Across decision-making and learning tasks, behavioral and physiological estimates of subjective value are used in the analysis of various types of neuroimaging data analyses (Bartra et al., 2013; Polanía et al., 2014) (Figure 1). While some value estimates are directly observed (e.g., willingness to pay, pupil diameter), others are latent variables inferred from computational models. These models integrate objective parameters (e.g., monetary amount) with subject-specific ones. Models of temporal discounting, for example, include a parameter that accounts for the discount rate of the value of future rewards, whereas models of risky decision-making account for the level of risk aversion. These individual parameters are then used to compute subjective values, and much of the neural analysis focuses on the correlates of those values.

3.2. Value-related brain areas

Using the paradigms outlined in the previous section, human neuroimaging research on learning and decision-making linked a set of brain areas to subjective value (Clithero & Rangel, 2014; D. Levy & Glimcher, 2012; Newton-Fenner et al., 2023; Tobler et al., 2009). In this section, we briefly survey the anatomical properties of these value-related areas (Figure 2), before turning to their function and surveying evidence supporting the idea of a “common currency” representation of reward values.

Figure 2. Value-related areas in the human brain.

Using the experimental paradigms described in Figure 1, human neuroimaging research on learning and decision making identified a broad network of cortical (green) and subcortical (yellow) regions implicated in reward valuation. The figure presents medial (a) and lateral (b) views. Abbreviations: AI, anterior insula; dACC, dorsal anterior cingulate cortex; dlPFC, dorsolateral prefrontal cortex; dmPFC, dorsomedial prefrontal cortex; lOFC, lateral orbitofrontal cortex; mOFC, medial orbitofrontal cortex; PCC, posterior cingulate cortex; PPC, posterior parietal cortex; sgACC, subgenual anterior cingulate cortex; SN, substantia nigra; vmPFC, ventromedial prefrontal cortex; VTA, ventral tegmental area. Figure created by Evelyn Pence

The striatum, encompassing the caudate, putamen and nucleus accumbens (NAcc), is integral to value-based decision-making (Knutson et al., 2009; Kwak & Jung, 2019; Montague & Berns, 2002). Although traditionally divided into ventral and dorsal parts, neuroanatomical techniques show that these parts share the same basic structure without distinct boundaries (Voorn et al., 2004). Inputs from nearly every cortical region converge in the striatum, including the prefrontal cortex (PFC), orbitofrontal cortex (OFC) and anterior cingulate cortex (ACC) (Wilson, 2009). Additionally, it receives inputs from various subcortical structures, including the thalamus, ventral tegmental area (VTA), substantia nigra (SN), amygdala and hippocampus. The striatum projects to the output nuclei of the basal ganglia (e.g., ventral pallidum), then to the medial dorsal nucleus of the thalamus, and subsequently to the PFC (Haber & Knutson, 2010). It also sends signals to the SN, VTA, and hypothalamus. As we will see, the striatum is a central node in the valuation network, across decision-making and learning.

Other subcortical regions in this network include the amygdala, hypothalamus, anterior insula (AI) and habenula (marked in yellow, Figure 2). Deep within the temporal lobe, the amygdala receives sensory information from the thalamus, hippocampus and PFC, influencing emotional and autonomic responses through its projections to the hypothalamus, PFC and brainstem (Murray & Fellows, 2022; Pessoa, 2017). The hypothalamus, located at the brain s base, integrates neural inputs from the amygdala, hippocampus and brainstem to regulate hormonal release and autonomic functions via its control over the pituitary gland (Card & Rinaman, 2002; Stuber & Wise, 2016). The insula, situated deep within the lateral sulcus, receives information from the thalamus and amygdala, and projects to the PFC and back to the amygdala. While it is mainly known for integrating bodily sensations with emotional states (Uddin et al., 2017), its anterior part (AI) is associated with value-based decision-making (Loued-Khenissi et al., 2020). Near the thalamus, the habenula receives inputs related to motivational and emotional states from limbic structures, and modulates dopamine and serotonin release through its connections to the VTA and raphe nuclei, thus influencing mood and reward processing (Hikosaka, 2010; Hu et al., 2020).

The main cortical regions implicated in value processing are the ventral and dorsal parts of the medial PFC (vmPFC and dmPFC), medial and lateral parts of the OFC (mOFC and lOFC) and dorsolateral PFC (dlPFC) (marked in green, Figure 2). The mOFC/vmPFC region, in particular, represents value across various reward types and integrates multiple reward dimensions (Peters & Büchel, 2010). The cingulate cortex also contributes to value-based learning and decision-making, especially its posterior (PCC), dorsal-anterior (dACC) and subgenual-anterior (sgACC) parts (Figure 2). For detailed information on the structure and function of the PFC, we refer the reader to several comprehensive reviews (Menon & D Esposito, 2022; O Reilly, 2010; J. R. Reynolds et al., 2012). Finally, the posterior parietal cortex (PPC) is another cortical area implicated in adaptive value encoding (Kahnt et al., 2014; Zhang et al., 2017).

Given that animal research will inform our review, mentioning a common disparity between human and animal findings in value-based decision-making is important. While the OFC is often identified in animal research, human neuroimaging studies typically report an adjacent and partially overlapping region, the vmPFC. This discrepancy may arise from differences in brain anatomy, experimental paradigms and methods used to assess brain function (Wallis, 2012). Indeed, a recent intracranial EEG study in humans reported positive associations between subjective value and activity in both vmPFC and lOFC (Lopez-Persem et al., 2020).

3.3. Neural “common currency” representations of reward value in human decision-making

Proposals of a “common currency” value system, capable of encoding diverse values under varying conditions, have been around for more than 20 years (Chib et al., 2009; D. Levy & Glimcher, 2012; Montague & Berns, 2002; Sescousse et al., 2015). One of the first studies that strongly supported the existence of a common currency valuation system focused on intertemporal choice, or the effect of delay on subjective value (Kable & Glimcher, 2007). Early work suggested distinct neural systems for immediate and delayed rewards, with increased activity for immediate rewards in VS, mPFC and PCC, and for delayed rewards in lateral prefrontal and parietal areas (McClure et al., 2004). Using choice behavior, however, Kable and Glimcher (2007, 2010) showed that activity in VS, mPFC, and PCC tracks the subjectively discounted value of future rewards, challenging the notion of separate neural systems (Kable & Glimcher, 2007, 2010).

Subsequent studies used similar strategies to explore other factors that influence value. Risk and ambiguity, two forms of uncertainty where outcome probabilities are fully known or not precisely known, strongly affect choice behavior, but the effects vary substantially across individuals. Interestingly, while choice behavior under these conditions was largely uncorrelated across participants, common neural activity in the striatum and mPFC represented value under both risk and ambiguity (I. Levy et al., 2010). Effort-based subjective value is also integrated in the mPFC, including vmPFC (Hogan et al., 2019; Salamone & Correa, 2024; Westbrook et al., 2019) and dmPFC (Chong et al., 2017; Klein-Flügge et al., 2016; Lopez-Gamundi et al., 2021) regions. Moreover, some studies have directly compared different factors, reporting overlapping representations of subjective value related to probability and time delay (Peters & Büchel, 2009; Piva et al., 2019), probability and effort (Yao et al., 2023), and across all three domains (Seaman et al., 2018) in regions of mPFC.

Further research has extended the exploration of value encoding beyond monetary contexts, revealing overlapping representations across diverse reward categories (D. Levy & Glimcher, 2012). Activity in vmPFC correlated with participants willingness to pay for food items (Plassmann et al., 2007), different wines (Plassmann et al., 2008), and non-food consumables (Chib et al., 2009). In choice tasks, vmPFC and striatal activity were associated with the subjective value of probabilistic rewards for both monetary and food options (D. Levy & Glimcher, 2011), and with the difference in subjective value between incommensurable goods (FitzGerald et al., 2009). Some evidence even suggests encoding of social rewards, such as the value of good reputation, in the mPFC (Izuma et al., 2008).

While traditionally linked to fear conditioning and negative emotions (Calder et al., 2001; Medina et al., 2002; Öhman, 2005), the amygdala has also been consistently implicated in reward processing (Baxter & Murray, 2002; Hampton et al., 2007; Murray, 2007). For instance, amygdala activation was found to reflect reward value under risk and ambiguity (Hsu et al., 2005; I. Levy et al., 2010), and may also integrate reward value with spatial location (Ousdal et al., 2014). Single neurons in the human amygdala encode reward values when making choices between pictures of food items (Jenison et al., 2011), or independently of the task being performed (Mormann et al., 2019).

The findings we have surveyed so far are consistent with the common-currency hypothesis, but may also result from averaging of non-overlapping value representations across participants (J. P. O Doherty et al., 2017). Stronger support for the common-currency theory comes from studies employing multivariate analysis, which detected activation patterns in the vmPFC that predicted subjective values across different reward categories (Gross et al., 2014; McNamee et al., 2013).

Beyond representation of value across domains, the vmPFC is central to the integration of diverse inputs from perceptual and memory areas, that contribute to value computation (A. Harris et al., 2011; Kahnt et al., 2011; Philiastides et al., 2010). This includes taste and health considerations for snack foods (Hare et al., 2009), visual and semantic stimulus attributes of a stimulus (Lim et al., 2013), nutritive attributes contributing to overall food value (Suzuki et al., 2017), and emotional and cognitive appraisals of a prospect (Hutcherson et al., 2015).

Efficient value-based decision-making also requires considering the individual s internal physiological and psychological factors (e.g., hunger, thirst, mood) (Fooken & Schaffner, 2016; D. Levy et al., 2013; Senftleben et al., 2019; Symmonds et al., 2010; Yamada et al., 2013), as well as environmental and social factors (Fooken, 2017; Gluth et al., 2014). Some evidence suggests that such factors are considered in neural computations of value. For example, VS activity reflects value relative to a dynamic reference point, set by prior experiences and expectations (De Martino et al., 2009). The adjustment to a reference point may reflect the divisive normalization of neuron firing across inputs (Louie et al., 2013; Louie & Glimcher, 2012), similar to the visual system (Heeger, 1992), as well as adaptation based on the recent history of received rewards (Kobayashi et al., 2010; Padoa-Schioppa, 2009; Rangel & Clithero, 2012; Yamada et al., 2013).

3.4. Neural representations of reward value in reinforcement learning

Value signals in the brain are constantly acquired and updated, primarily through reinforcement learning, where value representations are updated based on prediction errors (PEs), reflecting the difference between expected and actual rewards (Schultz, 2016, 2017; Watabe-Uchida et al., 2017). Neural representations of value are present both in instrumental and Pavlovian learning, which operates without active decision-making (Sescousse et al., 2015). The neural encoding of PE is thus closely related to representations of anticipated value and can inform our understanding of those representations.

The seminal work of Schultz and colleagues highlighted the role of dopaminergic (DA) neurons in the VTA and SN in encoding reward PEs (Schultz et al., 1997). These neurons exhibit increased activity for unexpected rewards (positive PE), and reduced activity when expected rewards are omitted (negative PE). Over time, these dopamine responses transiently transfer from primary rewards to cues that reliably predict them (Schultz, 1998). Later studies showed that a broader brain network is involved in this process of reward learning, including the striatum, midbrain and other subcortical and cortical regions (Haber & Knutson, 2010).

In humans, reward PEs are observed in the striatum (Lohrenz et al., 2007; J. O Doherty et al., 2004), as well as the midbrain, ACC and mPFC (Garrison et al., 2013). Consistent with the concept of neural “common currency” in decision-making, the human striatum and vmPFC compute reward value on a unified scale also in reinforcement learning, irrespective of the reward type (H. Kim et al., 2011; Lin et al., 2012; Valentin & O Doherty, 2009).

The amygdala is also involved in value-based learning (Hampton et al., 2007; Hassall & Williams, 2017; J. Li et al., 2011), although primarily supported by evidence from animal studies (Chang et al., 2015; Costa et al., 2016; Grabenhorst et al., 2019). It might be particularly important for observational learning, where individuals learn indirectly through observing others actions and outcomes (Aquino et al., 2020; Cooper et al., 2012; Dunne et al., 2016).

3.5. Identity-specific value representations

A “common currency” representation of value facilitates comparisons across diverse objects, yet preserving specific identity information remains crucial for value updating. There is some evidence for category-specific value representations in distinct brain regions beyond the core value areas. For example, hypothalamic activity encoded the subjective value of food, but not money, consistent with its role in homeostasis (D. Levy & Glimcher, 2011), whereas PCC represented the value of money, but not food. Interestingly, PCC activity correlates with subjective value of monetary rewards in most studies (Kable & Glimcher, 2007; I. Levy et al., 2010; McClure et al., 2004), but not all (McCoy & Platt, 2005). Despite its involvement in decision-making, memory, learning, social and emotional processing, the PCC s role in representing subjective value is not fully understood (Foster et al., 2023), warranting further investigation to clarify its function.

More evidence for identity-specific value representations comes from multivoxel pattern analysis that revealed distinct activation patterns for different categories within common-currency brain areas (Gross et al., 2014; McNamee et al., 2013). Identity-specific and identity-general encoding of reward value have been observed in the OFC and vmPFC, respectively (Howard et al., 2015; Klein-Flügge et al., 2016). Additionally, widespread representation of category identity was found in regions signaling value and saliency, such as the vmPFC and striatum (Zhang et al., 2017). Finally, recent work (Williams et al., 2021) suggests that different brain regions may accommodate distinct value computations.

4. THE NEURAL REPRESENTATION OF SUBJECTIVE VALUE OF PUNISHMENTS

In recent decades, significant progress has been made in understanding the computational and neural mechanisms of reward-based learning and decision-making, while punishment-based learning and decision-making remain more elusive (Dayan, 2012; Palminteri et al., 2015). Theoretical considerations suggest that subjective value should be represented on the same scale for both positive and negative outcomes, to facilitate consideration of both positive and negative potential outcomes in the choice process. That is, the “common currency” representation of value across different categories (e.g., money, food), should further extend to the domain of punishments. More broadly, we can consider several plausible mechanisms of value representation, that are not mutually exclusive (Figure 3). These serve as a simplified framework for interpreting empirical results, rather than detailed neural mechanisms.

Figure 3. Potential value representations of rewards and punishments at the neuronal level.

Using the experimental Neurons that represent reward value may also represent punishment value (same neuronal population). Such representation can be (a) monotonic with activity either rising (left) or decreasing (right) as a function of value. They can also be (b) U-shaped, encoding the magnitude or saliency of the stimuli. In this case, activity will be high (or low) for both large rewards and large punishments (left or right, respectively). (c) Another possibility is that different neuronal populations encode the value of rewards (left) and punishment (right), with either increased or decreased activity as a function of value, in each population separately. (d) Finally, the principles of value representations themselves might differ between individuals, reflecting genetic predispositions or consequences of environmental experiences throughout life. Figure created by Evelyn Pence.

For a pure “common currency” representation, the same neuronal populations should encode positive and negative subjective values on a continuous monotonic scale, with neuron firing rates tracking value across the spectrum, increasing or decreasing as value shifts from “very bad” to “very good” (Figure 3A). This model allows for consistent representation of an option s anticipated value, even as it shifts between reward and punishment contexts (e.g., chocolate cake after much of it has been consumed). Such value representation, however, could be constrained by the dynamic range of neuronal firing rates, at the single-neuron or population levels, particularly in DA neurons with low baseline rates (Grace & Bunney, 1984).

Another possible representation involves the same neuronal population representing the magnitude (absolute value) of both expected rewards and punishments, depicted as a U-shaped curve indicating the saliency (rather than the value) of anticipated outcomes (Figure 3B). In this scenario, neural responses would be elevated for larger anticipated outcomes (rewards and punishments) and lower for smaller ones, or vice versa, signaling their significance without specifying their valence (positive or negative).

A third possibility is the existence of specialized neurons, distinct from those involved in reward processing, dedicated to encoding the expected value of potential punishments (Figure 3C). These neurons might coexist within the same brain regions (top) or be anatomically segregated (bottom). They could exhibit either increasing or decreasing firing rates as the value of the outcome increases (larger rewards or punishments). Finally, an intriguing yet understudied possibility is that the principles of value representation themselves (Figure 3A–C) differ between individuals (Figure 3D), for example, due to diverse life experiences such as trauma exposure. In the subsequent sections, we survey research in humans and animals in the context of these different frameworks.

4.1. Evidence from human neuroimaging studies

By replacing potential rewards with punishments, researchers can explore the neural encoding of anticipated punishments (Figure 4). Ideally, rewards and punishments should be included in the same experimental design, to facilitate direct comparisons. One of the first fMRI studies to do so, provided evidence in support of the “common currency” hypothesis (Tom et al., 2007). Sixteen participants made a series of choices about mixed lotteries where gains and losses were manipulated independently. Results revealed a broad set of cortical and subcortical regions, with increasing activity as potential gains increased, and decreasing activity as potential losses increased. These regions - including the striatum, vmPFC, ventral ACC, and mOFC – were all previously implicated in encoding the anticipated value of rewards (Tom et al., 2007). Representation of subjective value for both appetitive and aversive food was also reported in the mOFC (Plassmann et al., 2010). A more recent study, however, using the same monetary paradigm developed by Tom and colleagues, in a much larger sample (N=108), did not replicate the overlap in activation patterns for gains and losses (Botvinik-Nezer et al., 2020). A meta-analysis of 37 studies with losses and punishments also failed to observe such a monotonic representation of loss value (Bartra et al., 2013), although it did identify a representation of saliency (as discussed below).

Figure 4. Experimental paradigms for studying the neural representation of punishment value.

As in tasks measuring reward value, three different measures are usually used for estimating punishment subjective value: rating, where participants explicitly report how much they like (or dislike) the punishment or aversive stimuli on a certain scale, and although most individuals will report negative values (i.e., between 5 and 0), a minority may report positive values (i.e., between 0 to 5); willingness to pay, where participants state the amount of money they are willing to pay to avoid a specific punishment (e.g., a rotten apple with a worm inside), either hypothetically or with an endowment (e.g., between $1 and $5); and choices, where participants are asked to choose between different aversive outcomes (e.g., mild electric shock versus losing money, air puff to the eye versus viewing an aversive image). Furthermore, individuals can be asked to choose between an immediate small punishment and a delayed larger punishment (delay), between a small certain punishment and a larger uncertain punishment (uncertainty), or between a large punishment with minimal effort and a smaller punishment that requires high effort (effort). In reinforcement learning studies on punishments or other aversive stimuli, values are learned over time either passively (Pavlovian learning) or through trial and error (instrumental learning). In the illustrated examples, a blue triangle is repeatedly paired with an aversive outcome (e.g., mild electric shock), whereas a yellow square is paired with a safe outcome (e.g., no shock). Values of punishments are estimated from behavioral (model-free and model-based) and physiological measures, which are then used in the neural analysis (see Figure 1, bottom). Figure created by Evelyn Pence.

Learning studies provide some evidence for overlapping value representations of cues predicting gains or losses. In a Pavlovian conditioning task, activation patterns in the OFC and PPC encoding value of gain-predicting cues, also conveyed information about the value of loss-predicting cues (Kahnt et al., 2014). Similarly, in an instrumental learning paradigm, activity in the mOFC and lOFC increased with expected rewards and decreased with expected punishments (H. Kim et al., 2006). A later study using a similar task also reported overlapping activation for anticipated gains and losses in the striatum lOFC (S. H. Kim et al., 2015). However, caution is warranted in interpreting these results as reflecting common processing of gains and losses, as cues associated with losses may predict successful avoidance rather than the experience of loss itself, and this avoidance can be rewarding in the experimental context. Indeed, the mOFC responded positively to both gains and successful avoidance of losses (H. Kim et al., 2006). A separate analysis of late and early trials suggests that vmPFC activation aligns with a monotonic encoding of anticipated gains and losses (Palminteri et al., 2015), showing positive activity in late reward trials (when cues reliably predicted gains) and negative activity in early punishment trials (before avoidance learning was acquired).

Several studies have demonstrated neural representations of both value (monotonic) and saliency (u-shaped) in the monetary domain (Kahnt et al., 2014), food domain (Litt et al., 2011) and across domains (Zhang et al., 2017). Activity in the vmPFC and lOFC encoded value, whereas the rostral ACC and AI showed saliency representations. The VS contained multiplexed representations of both value and saliency (Litt et al., 2011; Zhang et al., 2017). Interestingly, stress-related psychopathology was associated with a shift towards predominant saliency representations in VS (Jia et al., 2023), echoing similar findings in the lateral habenula (LHb) of mice (Shabel et al., 2019). These studies hint at potential individual differences in principles of value representations, which may relate to symptoms of psychopathology (discussed in section 5).

Studies investigating the value of aversive outcomes (or aversive PEs) in the context of learning may offer additional insights. Early findings suggested monotonic representations of value, with both gains and losses positively correlated with mOFC activity and negatively with lOFC (Elliott et al., 2010; J. O Doherty et al., 2001). Other work identified such monotonic value representations in the dmPFC, proposing separate areas of the ACC/mPFC for gain-specific and loss-specific value representations (Fujiwara et al., 2009). A recent meta-analysis of appetitive and aversive PEs highlighted both conjunctions (in the striatum, amygdala, insula and subgenual cingulate) and distinct representations, with positive representations predominantly in the vmPFC and PCC, and negative ones mainly in the ventrolateral PFC (Corlett et al., 2022). Overall, human studies provide inconclusive evidence for whether anticipated punishments are part of the “common currency” value representation (as observed in the reward domain). A related question is whether different types of anticipated negative outcomes are themselves represented on a unified common scale. Loss of money (or points) is a convenient punishment to study: value can be carefully manipulated, and greater losses correspond to greater negative values. Many ubiquitous punishments, however, such as aversive food (Plassmann et al., 2010) or fearful faces (Morris et al., 2002), differ in nature.

Research on fear conditioning is particularly relevant, where aversive outcomes like electric shocks to the wrist or air puffs to the eye (Figure 4) are used to drive learning in Pavlovian and instrumental paradigms. Traditionally known as the brain s “fear center”, the amygdala is essential for fear conditioning (Duvarci & Pare, 2014; LeDoux, 2003). While it is consistently implicated in human fear-learning studies (Fullana et al., 2016; Phelps et al., 2004; Schiller et al., 2008), it also has a broader role in emotional processing, as well as detection and anticipation of rewards (Baxter & Murray, 2002; Hampton et al., 2007), making it a potential candidate for representation of both rewards and punishments (Manssuer et al., 2022). Another potential candidate is the striatum, showing overlapping representations of aversive PEs across both primary (e.g., shock) and secondary (e.g., monetary loss) reinforcers (Delgado et al., 2008; Garrison et al., 2013).

Expanding to additional aversive domains, a recent study examined general stimulus-type-specific neural representations of negative affect (Čeko et al., 2022). Across different types of aversive stimuli (mechanical pain, thermal pain, aversive sounds, and aversive images), participants aversion rating predicted activity in cortical and subcortical regions - including the OFC, mPFC, AI, VS, amygdala - suggesting a common-currency representation of the value of anticipated punishments. While the same areas have been implicated in representing anticipated rewards (see section 3), models trained on aversive stimuli could not predict responses to rewarding stimuli, and vice versa, suggesting separate neural circuits.

Overall, human findings are mixed, with some evidence for overlapping value representations of anticipated rewards and punishments, alongside indications of (at least partially) separate networks. Next, we turn to key findings from animal research to address limitations in spatial and temporal resolutions of non-invasive neuroscience research in humans.

4.2. Insights from animal research

The literature on encoding rewards and punishments in animals is vast, utilizing multiple techniques (lesion studies, electrophysiology, microdialysis, optogenetics) across species, from flies to non-human primates. While we do not intend to cover it comprehensively here, we aim to highlight the complexity at the single-neuron and circuit levels, complementing human neuroimaging studies with more precise localization, better temporal dynamics, and causal manipulations (impossible in humans). Differences in experimental paradigms, however, and the absence of verbal instructions, necessitate careful interpretation of findings from animal research, which naturally focus on learning processes.

Bearing these differences in mind, numerous findings from animal research correspond directly to human findings, offering a platform to examine similarities and differences in subjective-value representations of rewards and punishments. As we saw in section 3, the dopamine system is central to encoding the anticipated reward value (Schultz, 2007). Within the reward domain, dopamine activity is consistent with the “common currency” hypothesis in that it integrates subjective value across multiple dimensions, including amount, risk and reward type (Lak et al., 2014).

Is the dopamine system also involved in representing the value of anticipated punishments? While early studies suggested that dopamine is preferentially activated for rewards (Mirenowicz & Schultz, 1996), later work implicates it in punishment representation (Bromberg-Martin et al., 2010). In Pavlovian learning with rewards (juice) and punishments (air puff), DA neurons near the SN and VTA were excited by reward-predicting stimuli and inhibited by punishment cues (Matsumoto & Hikosaka, 2009b), consistent with a pure common-currency representation within the same neuron population (Figure 3A). Similarly, the same dopamine neurons were excited by reward (juice) and suppressed by reward omission (Fiorillo, 2013), further supporting monotonic value representation. These neurons, however, were insensitive to receipt or omission of punishment (air puffs or bitter juice), suggesting two distinct neural populations encoding reward and punishment values (Figure 3C). In yet another study, some DA neurons were excited by aversive outcomes and some were inhibited (Lammel et al., 2014), although excitation might have been due to the rewarding termination of the aversive stimulus. Finally, recent evidence points to decreased DA neuron activity in the VTA in response to pain (H. Yang et al., 2021). Schultz suggested that DA neurons may initially respond to novelty (i.e., unselective response), followed by a later response signaling reward PE (Schultz, 2016, 2019). Overall, it is clear that DA neurons in the VTA are not homogeneous, but rather encompass diverse functionally distinct populations (Lammel et al., 2014; Verharen et al., 2020). In addition, methodological variations, including differences in how DA neurons are identified, further contribute to the complexity of interpreting findings (Tye, 2018).

Variability exists not only among dopamine neurons but also in their projection targets, such as the medium spiny neurons in the striatum (Juarez & Zweifel, 2022). Traditionally, distinct striatal dopamine receptors were believed to mediate rewards and punishments (D1 and D2, respectively) (Kravitz et al., 2012; Surmeier et al., 2007; Verharen et al., 2019), but recent evidence reveals a more complex picture, where subtypes of both receptors are implicated in both domains (Liu et al., 2022; Soares-Cunha et al., 2020). Subpopulations of DA neurons, modulated by rewarding or aversive experiences, project to different regions in the striatum (de Jong et al., 2019; Lammel et al., 2011), with some projections consistent with saliency representation (Kutlu et al., 2021; van Elzelingen et al., 2022). In the vmPFC, another target area for midbrain DA neurons, distinct subpopulations are anatomically segregated based on their responses to reward-predicting cues, and aversive-predicting ones (Monosov & Hikosaka, 2012). Notably, recent research in mice suggests threat-specific information is decoded in a more dorsal PFC region, the dmPFC (Martin-Fernandez et al., 2023).

Different VTA dopamine neurons also receive diverse inputs from different sources (Lammel et al., 2012). For example, neurons in the LHb respond to cues predicting punishment (or absence of reward) and are excited by punishment receipt (while inhibited by reward receipt) (Hikosaka, 2010). These neurons show linear encoding of punishment value (Matsumoto & Hikosaka, 2009a), consistent with a unique role in encoding expected punishment (Figure 3C). However, LHb neurons may also have a broader role in predicting the values of future stimuli (Mondoloni et al., 2022), encoding both positive and negative events in innate and learned behaviors (Trusel et al., 2019; Wang et al., 2017), consistent with saliency representation of value (Figure 3B).

While initially associated with negative valence, substantial evidence indicates the involvement of the amygdala in processing both rewards and punishments across species (Janak & Tye, 2015; Murray, 2007; Wassum, 2022). In Pavlovian learning, monkey amygdala neurons encode the predictive value of visual stimuli associated with both appetitive and aversive outcomes (Paton et al., 2006). Using similar learning paradigms in rats, amygdala neurons were found to be involved in encoding sensory-specific reward memories, which are critical for adaptive decision-making (Sias et al., 2021). Although the amygdala integrates information about both anticipated rewards and punishments (Burgos-Robles et al., 2017), this likely involves distinct circuits for appetitive and aversive expectations (Beyeler et al., 2016; J. Kim et al., 2016).

In conclusion, data across species suggest both overlapping and distinct mechanisms for the representation of anticipated values of rewards and punishments. In addition to conceptual and methodological challenges, animal research underscores further challenges for human studies of punishment value, as detailed below.

4.3. Challenges in studying the neural encoding of punishment value

Our survey of existing literature highlights the inherent challenges in this line of research. First, while the anticipation of rewards, such as money or food, can be genuinely rewarding, creating authentic, ecologically valid expectations of punishment is harder (Mobbs et al., 2019). One challenge arises from the diverse nature of punishments and the difficulty in replicating them in controlled laboratory settings. For example, ethical constraints prevent participants from losing their own money directly, requiring them to be endowed with funds that they can then lose, or alternatively lose hypothetical money or points. Moreover, while monetary gains and losses serve as a convenient model, the nature of monetary loss differs significantly from other punishments, such as physical pain, suggesting that rewards and punishments are not opposite sides of a single continuous “value” dimension (Fiorillo, 2013). Including both rewards and punishments in the same experiment facilitates direct comparisons under the same reference, but poses empirical challenges due to the need for brief time-limited experiences within each trial. A further complication is that in real life many outcomes combine both positive and negative aspects, such as a pleasurable consumer good with a high cost or a potential large gain coupled with a risk of significant loss. Finally, external and internal contexts can alter an outcome s rewarding or punishing nature (e.g., the value of a cake when you are hungry versus feeling sick). This complexity is further compounded given the value normalization of neural activity by the choice set and history of outcomes (Bavard & Palminteri, 2023). A promising direction for overcoming these challenges is the use of ecologically valid gamified experimental designs, such as immersive virtual reality (VR) environments, to elicit more authentic sensations and emotions, leading to realistic behaviors (Allen et al., 2024; M. Li et al., 2022). Such designs encourage deeper task engagement, and allow for natural dynamic changes of valence.

Another challenge arises from the limited spatial resolution – and in the case of fMRI also temporal resolution - compared to animal research. As reviewed above, animal studies often reveal intermingled populations of neurons with different response properties. This complexity extends even within the reward domain, where neurons encoding different properties reside in the same location, with no anatomical segregation (Padoa-Schioppa & Assad, 2006).

Consequently, human neuroimaging results likely underestimate the true complexity of brain activity, potentially limiting conclusions regarding individual differences. This challenge underscores the necessity of cross-species studies which can overcome the conceptual and methodological limitations inherent to human-only or animal-only studies (Polley & Schiller, 2022). Another promising direction is intracranial recordings in humans (Lopez-Persem et al., 2020; Manssuer et al., 2022), which offer high spatial and temporal resolution, and serve as a direct measure of neural activity without signal distortion (e.g., by the skull or air-filled chambers within the skull).

5. INDIVIDUAL DIFFERENCES AND MENTAL HEALTH

The concept of distinct valence systems for positive and negative stimuli was originated in psychology over a century ago, and recently incorporated into clinical neuroscience (Tye, 2018). The positive and negative valence systems are also recognized as two fundamental dimensions of human behavior within the Research Domain Criteria framework (Cuthbert & Insel, 2013; Insel et al., 2010). Real-life valence estimation is more complicated, as the same stimuli often evoke mixed and even conflicting emotions, resulting in different decisions and behaviors. Similarly, the same brain regions (e.g., amygdala, striatum) respond to both positive and negative outcomes (Baxter & Murray, 2002; Eckstrand et al., 2021; Murray, 2007; Robinson et al., 2010), further complicating the picture. Furthermore, neural representations of rewards and punishments may differ between individuals (Figure 3D), potentially associated with various mental health disorders. Exploring the neurobehavioral representations of both rewards and punishments within individuals, may thus assist in the diagnosis, prediction and treatment of such disorders.

Animal research suggests that stress might impair accurate valence estimations. For example, stressful environments increase activity in areas within the NAcc, which produces fear in rats (S. M. Reynolds & Berridge, 2008). Similarly, stress transformed LHb reward responses in mice into punishment signals, interpreting “good as bad and bad as worse” (Shabel et al., 2019). In humans, while short-term stress might be beneficial for survival (e.g., increased vigilance at the expense of cognitive resources (I. Levy & Schiller, 2021; Shields et al., 2016), a transition into reward-driven behavior over time is crucial for facilitating stress recovery and resilience (Ben-Zion et al., 2018; Speer & Mauricio, 2017; Tugade & Fredrickson, 2004). In this section, we will explore how studies combining rewards and punishments may shed light on stress-related psychopathologies, such as anxiety, depression and post-traumatic stress disorder (PTSD).

Anxiety disorders often involve a heightened sensitivity to negative outcomes, reflected in the increased response of the salience network (e.g., amygdala, AI, dACC) to aversive or threatening stimuli (Graham & Milad, 2011; Hayes, Hayes, et al., 2012; Hayes, VanElzakker, et al., 2012). Yet, recent evidence also suggests deficits in positive valence functioning (Hinojosa et al., 2024; Nawijn et al., 2015). For example, PTSD patients exhibit impaired reward anticipation, reduced approach (reward-seeking) behavior, and diminished hedonic responses to rewards (Elman et al., 2009; Felmingham et al., 2010; Sailer et al., 2017). Indeed, biased neural responsivity to rewards versus punishments shortly after trauma predicted long-term PTSD symptoms (Ben-Zion, Shany, et al., 2022). This aligns with the theory that anxiety disorders arise from an imbalance between the approach and avoidance systems (Stein & Paulus, 2009). Notably, PTSD patients tend to prioritize avoiding punishments at the expense of obtaining potential rewards (Weaver et al., 2020). These findings highlight the importance of studying reward and punishment processing together, as novel therapeutic strategies for anxiety and PTSD may benefit from targeting both valence systems.

Recent neuroeconomics and neuroimaging studies in anxiety disorders revealed initial insights into alternations in reward and punishment processing. For example, PTSD patients exhibited enhanced saliency representations of rewards and punishments in the VS, and their vmPFC activity during the valuation of uncertain outcomes correlated with their PTSD symptoms (Jia et al., 2023). Moreover, decreased value tracking in the amygdala and smaller amygdala volume independently corresponded to more severe symptoms (Homan et al., 2019). However, larger amygdala volume shortly after trauma was associated with non-recovery from initial symptoms (Ben-Zion, Korem, et al., 2022). During a passive avoidance task, individuals with generalized anxiety disorder displayed a reduced correlation between PE and activity within vmPFC and VS, and specifically between punishment PEs (but not reward PEs) and activity within the lentiform nucleus/putamen (White et al., 2017). These findings are consistent with previous work showing disrupted aversive PE signaling in anxiety disorders (Paulus & Stein, 2006; Paulus & Yu, 2012), possibly reflecting a failure to appropriately adjust expectancies when predicted negative events do not occur, leading to excessive worry about everyday problems.

Chronic stress may also contribute to the development of major depressive disorder (MDD) (Fuchs & Flügge, 2011), often characterized by abnormal responses to both reward and punishment (Chen et al., 2015; Eshel & Roiser, 2010), although the precise neurobehavioral mechanisms remain unclear. In probabilistic reversal-learning tasks, MDD patients show reduced learning rates and lower value sensitivity for both rewards and punishments (Mukherjee et al., 2020). At the neural level, blunted value-related responses in the vmPFC during a prospective decision-making task correlated with more severe motivational and hedonic symptoms in MDD patients (Souther et al., 2022). However, studies in MDD have primarily focused on rewards rather than punishments, highlighting the need for future neuroeconomic investigations of both valence systems to uncover the underlying mechanisms of depression (Mukherjee et al., 2023). MDD may also involve deficits in effort-based valuation and decision-making, affecting the ability to balance energy requirements with potential benefits (Park et al., 2017; Treadway et al., 2012; X. Yang et al., 2014). Depressed patients exhibit steeper reward discounting by effort, and are less inclined to exert cognitive effort to obtain rewards (Ang et al., 2023). Furthermore, different types of effort may correlate with specific motivational deficits, with physical effort associated with anhedonia severity and cognitive effort associated with life functioning (Tran et al., 2021).

In conclusion, alternations in value computation during learning and decision-making may have implications for various mental illnesses (Mukherjee & Kable, 2014), as seen in anxiety, depression and PTSD. Notably, even within a single disorder, different symptoms may stem from deficits in either the negative or positive valence systems. For example, intrusion and hyperarousal symptoms in PTSD may be linked to increased threat responsivity, while avoidance and anhedonia symptoms may be associated with reduced reward functioning. Future research should investigate value representations of both reward and punishment across different psychopathologies, including transdiagnostic symptoms observed in multiple mental disorders (e.g., anhedonia, avoidance).

6. SUMMARY AND OPEN QUESTIONS

Decisions often involve weighing potential benefits against costs, such as deciding whether to invest in a new business venture despite the financial risks involved, or determining whether the personal growth opportunities of moving to a new city outweigh the emotional toll of leaving familiar surroundings behind. Representing these outcomes on a common scale enables dynamic value assessments and adaptability to changing contexts. Existing research suggests both unified and separate processing mechanisms for positive and negative valence, yet discrepancies exist.

A broader question, beyond the scope of this review, is whether values are even encoded in the brain (Hayden & Niv, 2021)(see sidebar). While signals in multiple brain areas correlate with value, there is little evidence for a causal role for these signals in the choice process, and choice implementation remains an open question. One possibility is that choices arise organically from iterative value computations (Hunt et al., 2012; Strait et al., 2014); another option is that values are first computed and then compared downstream of the valuation circuitry (Koscik et al., 2020). Furthermore, to make sound choices, the brain likely performs multiple value computations (such as goal value, decision value, outcome evaluation and prediction error)(Hare et al., 2008). The specific contributions of the multiple value-related brain regions to the various value signals are still not fully understood.

Methodological limitations, particularly ethical constraints, challenge the study of punishments. While real-life consequences are essential for revealing true preferences (Verschoor et al., 2016; Winkler & Murphy, 1973), the severity of aversive outcomes in studies is inherently limited. These constraints highlight the need for more ecologically valid, gamified experimental designs capable of simulating more authentic sensations and emotions, thus eliciting more realistic behaviors (Alsawaier, 2018; Sailer et al., 2017). Importantly, such designs should incorporate both anticipated rewards and punishments, throughout both learning and decision-making phases.

Cross-species research provides invaluable insights, overcoming the conceptual and methodological limitations inherent to human-only or animal-only studies (Polley & Schiller, 2022). Recent comparative analyses have begun to unravel the complex neural mechanisms of reward and punishment processing across different species (Bromberg-Martin et al., 2024; Rudebeck & Izquierdo, 2022; Wallis, 2012; Woo et al., 2023).

Last but not least, a deeper understanding of the shared and distinct neural circuits involved in prospective rewards and punishments, particularly in decision-making and learning processes, holds significant relevance for mental health research. Over a decade ago, Hasler (2012) advocated for applying economic concepts to psychiatric research to improve diagnostic classification, prevention and treatment options (Hasler, 2012). During this time, the field of computational psychiatry (K. J. Friston et al., 2014; Montague et al., 2012) helped to incorporate decision neuroscience into contemporary models of psychiatric disorders, specifically using formal models of brain functioning to understand the underlying mechanisms of psychopathology (Gillan et al., 2016; Huys et al., 2016; Robson et al., 2020; Zald & Treadway, 2017). Despite tremendous growth and initial excitement, computational psychiatry has yet to impact routine clinical practice (K. Friston, 2023; Hitchcock et al., 2022; Saez & Gu, 2023). Future endeavors should examine specific neural computations (e.g., value, PE) within individuals across different processes (e.g., learning, decision-making) and valence domains (e.g., rewards, punishments). Initial findings suggest potential links between psychiatric disorders (e.g., anxiety, depression, PTSD) and alterations in value representations, across rewards and/or punishments (Jia et al., 2023; Mukherjee et al., 2020). Future investigations should further identify robust associations between neural representations of anticipated rewards and punishments and mental health trajectories, paving the way for a new generation of computational psychiatry with tangible clinical implications.

SIDEBAR - IS VALUE REALLY ENCODED IN THE BRAIN?

While extensive research demonstrated widespread neural activation patterns correlated with subjective value, this does not directly imply that the brain utilizes these values for decision making. Indeed, valuation is not a necessary step for either decision making (Miller et al. 2019) or learning (Sutton & Barto 2018). Decision making potentially relies on direct learning of action policies rather than a comparison of stored values (Hayden & Niv 2021). Furthermore, the OFC, one of the main putative value-related areas, may play a wider role in cognition by representing cognitive maps of state or task space (Behrens et al. 2018, Schuck et al. 2016,Wilson et al. 2014). This perspective suggests that the effects of OFC and vmPFC lesions on decision making (Bechara et al. 1994, Fellows & Farah 2007, Yu et al. 2022) might stem from broader cognitive impairments rather than a specific problem in value encoding. Albeit a fascinating topic of ongoing research, some evidence exists for value signals in the absence of choice (Levy et al. 2011), even when attention is diverted away from value (Lebreton et al. 2009, Tusche et al. 2010), although vmPFC activity is more pronounced when choice is required (Grueschow et al. 2015). Knudsen & Wallis (2022) offer a reconciliatory perspective on OFC s function, suggesting it calculates values of states using a state-graph transition computed by the hippocampus. This ongoing debate underscores a vibrant area of research exploring the depths of how our brains navigate the complex landscape of value-based decision making.

TERMS AND DEFINITIONS LIST

Subjective Value

The utility of an option to the decision-maker, encompassing all its properties, the context, and the individual s preferences.

Common currency

A theory suggesting a unified scale in the brain for comparing diverse options and outcomes, facilitating decision-making across different contexts.

Neuroeconomics

An interdisciplinary field combining neuroscience, psychology, and economics to study decision-making.

Expected utility theory

A decision-making framework predicting rational choices by evaluating the expected outcomes weighted by their probabilities and subjective value.

Prospect theory

A behavioral economics framework describing how people decide between probabilistic alternatives, emphasizing the unequal weighting of potential gains versus losses.

Willingness to Pay

The maximum amount of money an individual is prepared to spend to obtain a reward or avoid a punishment

Revealed preferences

An economic theory by Paul Samuelson, asserting that preferences should be inferred from observed choice behavior.

Indifference point

Where two alternatives have equal subjective value to the decisionmaker

Uncertainty

When consequences of possible outcomes are not certain. Outcome probabilities can be fully known (risk) or not precisely known (ambiguity).

Pavlovian learning

Classical/associative learning where a neutral stimulus becomes associated with aversive or appetitive outcomes, without making decisions (passive viewing).

Instrumental learning

Operant learning based on the consequences of actions, where behaviors are strengthened or weakened by rewards or punishments.

Prediction error

The difference between the obtained and expected outcomes, used to drive learning of rewards and punishments.

Latent variables

Unobservable factors inferred from models, representing internal states that influence decision-making and learning in neuroeconomics.

Temporal discounting

The cognitive phenomenon by which future rewards are valued less than immediate rewards (i.e., delay discounting), assessed through the intertemporal choice task.

Normalization

The brain s mechanism for adjusting neural responses to stimuli based on context.Top of Form

Research Domain Criteria

A framework for studying mental disorders based on biological, cognitive, and behavioral measures across the full range of functioning.

Stress

A physiological and psychological response to perceived challenges, demands, or threats, impairing accurate representation of rewards and punishments.