The roles of valuation and reward processing in cognitive function and psychiatric disorders

Abstract

In neuroeconomics, valuation refers to the process of assigning values to states and actions based on the animal’s current representation of the environment while reward processing corresponds to processing the feedback received from the environment to update the values of states and actions. In this article, we review the brain circuits associated with valuation and reward processing and argue that these are fundamental processes critical in many cognitive functions. Specifically, we focus on the role of valuation and reward processing in attention, memory, decision–making, and learning. Next, the extant neuroimaging literature on a number of psychiatric disorders is reviewed (i.e., addiction, pathological gambling, schizophrenia, and mood disorders), and an argument is made that associated deficits in cognitive functions can be explained in terms of abnormal valuation and reward processing. The review concludes with the impact of this framework in clinical settings and prescriptions for future research, in particular with regards to the conversions of qualitatively different valuation systems into a system of common currency.

Valuation is an important step in any cognitive process. Specifically, valuation refers to the process of assigning values to states and actions based on the animal’s (human or non–human) current representation of the environment.¹ For example, hunting for prey has a different value depending on whether one is hungry or not. One of the roles of valuation is to produce a subjective common currency that can be used to compare the desirability of different states and actions. Importantly, valuation occurs before an action is selected. In contrast, reward processing occurs after an action has been performed and the animal received feedback from the environment.² Feedback from the environment includes, e.g., the outcome of the selected action (e.g., a prey was captured) and the new state that the animal finds itself in (e.g., no longer hungry, did not get sick after eating). The feedback received is used to update the value functions for states and actions, which in turn affects the valuation process for the next iteration.

One useful analogy to understand the distinction between valuation and reward processing is to look at the field of reinforcement learning (RL).³ An overview of the RL framework is shown in Figure 1. In RL, the agent receives information from the environment (called the state) and selects an action based on the state and value functions. The action is sent back to the environment, and in turn the environment returns a reward and the new state to the agent.^a Both states and actions have a value function, and computing values using these functions is what we call the valuation process. For example, realizing that the brakes on the car we are driving no longer work is not a valuable state. But noticing that there is no traffic on the freeway when trying to catch a plane is a valuable state. Likewise, stopping at a red light is a valuable action when driving, but stopping at a green light not so much (assuming that others are also lawfully following the signals). As explained earlier, the value of actions is computed in the context of the state (i.e., the same action can have a low or high value depending on the state). In the previous example, one can probably think of a scenario where driving through a red light can be valuable. The other focus of this review article, reward processing, would correspond to using the reward received from the environment to update the value functions of states and actions.^b

Figure 1.

A high–level representation of RL.

The main thesis of this review article is that valuation and reward processing are core cognitive functions on which many other cognitive functions rely. As a result, we argue that research on valuation and reward processing should not be limited to the field of decision–making, and should include other cognitive functions as well. One of the consequences of this thesis is that researchers studying other cognitive functions need to be careful in interpreting their results since a subset of the results obtained while studying other cognitive functions will be related to the core functions of valuation and reward processing (for a similar argument, see Ref. 6). In addition, cognitive deficits observed in psychiatric disorders might be related to atypical valuation functions.

What makes a state or action valuable?

The presentation so far relied on the intuitive notion that states and actions have values. This begs the question of what makes a state or action valuable? In general, states and actions are valuable insofar as they lead to reward.³ Specifically, states and actions that facilitate obtaining immediate and/or large rewards are most valuable, whereas states and actions that delay or reduce future rewards are not. One important aspect of the valuation process is discounting. Discounting refers to any factor that negatively impacts the value of states and actions.⁷ The most studied discounting factors are temporal delays and uncertainty.^1,8–12 Because valuation is based on the availability of rewards, these processes are deeply coupled, and it is difficult to study one topic without the other. For example, a well–adjusted (optimal) agent needs an accurate reward processing system in order to correctly estimate the values of states and actions. Likewise, valuation needs to be done properly in order to maximize the total amount of reward obtained from the environment.

Brain circuits supporting valuation and reward processing

Theories and models of decision–making in different disciplines (psychology, computer science, economy, etc.) typically assume that when options of different kinds are available, values in different units are converted into a single common currency.¹ There have been many studies that tried to find the neural equivalent of common currency, and specifically where and how it is computed. This section reviews some recent research addressing these topics.

The more convincing evidence in support of neural coding of common currency comes from (1) within–subject studies with different tasks, (2) within–task studies with different reward types, and (3) studies where participants build an exchange system for different types of rewards.¹³ Functional magnetic resonance imaging (fMRI) studies correlating blood–oxygen–level dependent (BOLD) signal and reward size (or related concepts like expected reward size) converge on a set of brain regions that include the ventromedial prefrontal cortex (vmPFC), orbitofrontal cortex (OFC), ventral striatum (VS), posterior cingulate cortex (PCC) and, to a lesser degree, the amygdala, insula, and posterior parietal cortex (PPC).^14,15 Below we review example studies for each type of evidence (for a more complete review, see Ref. 13).

1. Within–subject study with different tasks: Participants performed two interleaved tasks, one in which they had to choose between two visual stimuli (not associated with a specific motor response), and another in which they had to choose between two different motor responses (with no visual cues).¹⁶ In both tasks, the participants received probabilistic monetary rewards at the end of the trial. The results showed a correlation between activity in the medial portion of prefrontal cortex (PFC) and expected future reward. Other studies have used delayed probabilistic monetary rewards,¹⁷ and risky (known probabilities) and ambiguous (unknown probabilities) monetary lotteries.¹⁸ In both cases, the OFC and VS were found to be involved in valuation and reward processing.

2. Within–task study with different types of rewards: Participants had to choose between receiving (or losing) monetary rewards and consumer goods.¹⁹ The results showed that activity in vmPFC and PCC was positively correlated with the difference between the subjective value of the available options, while activity in the insula was negatively correlated with it. Other studies with different types of rewards when one of the choices was probabilistic,²⁰ when penalties were included,²¹ and when monetary and social rewards were compared,²² resulted in similar results, suggesting a role for vmPFC, striatum, and possibly insula in the representation of common currency.

3. Participant–built exchange system: In the first task, male participants passively viewed pictures of female faces with different levels of attractiveness followed by pictures of US currency ranging from –$5 to $5 (experience value).²³ In the second task, participants had to decide how much money they were willing to spend to see a female face at a particular level of attractiveness (monetary value). This allowed for establishing an exchange rate between money and level of attractiveness. The results show that experience value and monetary value are coded in different parts of vmPFC, and that the posterior part of vmPFC coded for the common currency (see also Ref. 24).

While the studies reviewed above sketch a qualitative portrait of the main findings, Bartra and colleagues²⁵ ran a coordinate–based meta–analysis (CBMA) to formally test which of the brain regions found in previous experiments is more likely to be the neural representation of common currency. The meta–analysis highlights a substantial overlap between brain regions that show positive and negative effects related to subjective value (e.g., anterior insula and parts of dorsal and caudal striatum). In addition, many brain regions showed significant BOLD signal in response to different reward types (monetary and primary incentive) at both the decision stage (valuation) and outcome receipt (reward processing). The results of the meta–analysis suggest that vmPFC and VS might be involved in the representation of common currency (see Figure 2), consistent with the individual studies reviewed in this section^13,15 (for other example tasks, see Ref. 26).

Figure 2.

Brain regions associated with the representation of common currency in Ref. 25.

Blue = vmPFC; Red = VS. The brain mask was adapted from the supplementary material of Ref. 25.

Functional connectivity

The previous section lists a number of brain areas related to valuation and reward processing, but what makes these areas “circuits” is their connectivity. Specifically, any brain area supporting valuation using a common currency needs to integrate information from multiple brain areas.¹³ For example, functional connectivity has shown causal connectivity from temporal cortex (a high–level visual processing area) to vmPFC when choosing between pictures of food items.²⁷ Likewise, temporal cortex, dorsal PFC, and the anterior cingulate cortex (ACC) show increased functional connectivity with vmPFC during evaluation periods, suggesting that decision values are also guided by attention.²⁸ In addition, the brain areas associated with valuation are likely to have multiple functions, and their pattern of connectivity with other brain areas may help pinpoint the exact role of each brain area.²⁹ For example, the ACC shows a difference in money–dependent connectivity for low vs. high pain conditions with the amygdala in a choice task where participants choose among pain–money combinations, suggesting that the amygdala plays an important role in modulating hedonic experience.⁷ Other cases where functional connectivity helped provide hints for the function of brain areas related to valuation used individual differences. For example, individual differences in the ability to relate confidence to decision performance is related to increased functional connectivity between vmPFC and rostrolateral PFC, a region related to metacognition.³⁰ Likewise, individual differences in valuation of social rewards are related to the strength of functional connectivity of the medial PFC, PCC, temporal cortex, and the temporal–parietal junction (all brain regions previously associated with social information) with a posterior segment of the vmPFC.²⁹

Cognitive functions

Valuation and reward processing are essential to cognitive function because most cognitive functions require expending some level of effort.³¹ For example, selectively attending to a stimulus,³² keeping items in working memory (WM),³³ and hypothesis–testing are effortful.³⁴ According to the adaptive gain theory,³⁵ task engagement relies on a cost/benefit analysis and effort is expended based on utility. Consequently, animals will (1) attend to,³⁶ (2) memorize,³⁷ (3) choose,⁷ and (4) learn³⁸ things that are valuable to them. Below we review some evidence showing the effects of valuation and reward processing in these four fundamental cognitive functions.

Attention

Like all brain functions, attention is adaptive and facilitates achieving goals that increase fitness.³⁹ Attention is usually separated into bottom–up and top–down processes.⁴⁰ Bottom–up attention is stimulus–driven and mostly “automatic” while top–down attention is goal–driven and “effortful”. In the attention literature, resources are devoted to salient locations and objects. We argue that saliency is a function of value magnitude and, as a result, that both types of attention depend on some combination of valuation and reward processing. Specifically, animals will find salient (and pay attention to) states and actions that have the highest (to approach) or lowest (to avoid) values.

The effect of valuation and reward on top–down attentional processing is more intuitive as top–down processing is often volitional and can benefit from task instructions and learning. In the former case, the task instructions explicitly inform the participant of what states and actions are valuable. In the latter case, the values of states and actions are learned using reward processing. As a result, the participant is often aware of top–down attentional strategies and can report what states and actions are valuable.^c This interpretation is supported by non–human primate work on the lateral intraparietal cortex (LIP). LIP is located in PPC and has been shown to code a subjective⁴² value–related signal^43,44 in a series of decision–making experiments. In attention tasks, single–unit recordings in the LIP show a sustained bias towards the location of stimuli, even when the response requires a saccade to a different location.^40,45 Together, these results strongly suggest that monkeys learn the values of locations and build a priority map (based on values) to direct top–down attention.

The relationship between bottom–up attention and valuation is more difficult to appreciate, and the reason might be that it is often decoupled from reward processing. Specifically, we argue that bottom–up processing relies mostly on values that have been learned and calculated through the evolutionary process. These values bias perception toward locations, stimuli, and cues that have survival value,³⁹ and are often hard–wired and not much affected by short–term reward processing. Values used in bottom–up processing could be represented in the superior colliculus, an important area for eye–movement that is strongly coupled with LIP.⁴⁰ For example, sudden movements in the visual periphery may signal threats. Likewise, singletons (i.e., objects that differ from their surroundings) may indicate novelty and the need for further processing. However, some bottom–up attention is also learned by the reward processing system. One interesting example was provided by Seitz and colleagues,⁴⁶ who had food– and water–deprived human participants passively look at sinusoidal gratings of various orientations. Some of the orientations were paired (without the participants’ knowledge) with drops of water rewards. After extensive exposure (there was no task), orientation sensitivity tests showed that participants had become more sensitive to the specific orientations that were paired with water during exposure. This bottom–up effect is not hard–wired and was learned by the reward processing system.

Note that the proposed framework is also compatible with the alternative account of attention of Awh and colleagues.⁴⁷ In their work, Awh et al. propose doing away with the top–down/bottom–up dichotomy and replace it with priority maps that integrate current goals, selection history, and physical salience. The framework that we are proposing emphasizes how top–down and bottom–up attention are affected by value (which is a similar concept to selection history), but the elimination of the dichotomy in favor of priority maps would not invalidate the proposed framework. Value and reward are important components of priority maps.⁴⁷

Memory

Memory is typically separated into a least two systems, namely WM and long–term memory (LTM). WM is the ability to maintain and manipulate information for a short amount of time.³³ WM is tightly connected with attention in that: (1) attention is often considered to be the gateway to WM^33,48 and (2) WM can often help in directing top–down attention.⁴⁰ WM relies on a distributed network that includes the lateral PFC, thalamus, striatum, and ACC.³³ It has been argued that the brain learns what is important, and what needs to be maintained in WM, using dopamine (DA)–mediated RL.^49,50 Specifically, phasic DA produces dips when an expected reward fails to appear, peaks when an unexpected reward appears, and reward processing updates the value of the WM items (which can be states or actions) using neural plasticity. As the value of WM items is learned, more valuable items are maintained while less valuable items fade away. As a result, valuation in WM is tightly coupled with reward processing.⁵¹

The second type of memory, LTM, is used to store and retrieve information for long periods of time so that it can be accessed when needed. Both the storage and the retrieval of memory traces have a cost, which can be calculated at different levels of analysis, from search time (cognitive) to metabolite expenditure (biological).⁵² When retrieving a memory, there is a gain if the memory is helpful in achieving a goal (i.e., obtaining a reward). This results in a cost/benefit analysis stating that the possible gain associated with encoding and retrieving a memory needs to be sufficient to offset the costs associated with the encoding and retrieval of the memory. One can make the simplifying assumption that memory cost is a constant, so the key variables determining the value of a memory are: (1) the magnitude of the possible gain associated with retrieving the memory when needed and (2) the probability of needing the memory. Anderson and Schooler⁵² used Bayesian analysis to evaluate the likelihood of needing a memory and found that it was related to the probability of retrieval. According to their analysis, the value of LTM items is related to the history of the item (e.g., how long ago was it last used, how often has it been used in the past), context (e.g., am I in a state similar to the state when the item was encoded), and content (e.g., is the item important). Encoding in LTM has been shown to rely on the hippocampus, dorsolateral PFC, and PPC while LTM retrieval relies on the dorsolateral PFC and OFC (for a review, see Ref. 53). Some of these areas overlap with valuation areas, namely the PPC and OFC. As a result, valuation is an important, and rational, determinant of what is encoded and retrieved from LTM.

Decision–making

Most of the work on valuation has been done in the context of decision–making, and there is little to no debate that decision–making relies on valuation (for example reviews, see Refs. ¹,¹²). In a typical decision–making task, the participant is given a number of options to choose from in order to maximize reward and minimize punishment. The rewards and punishments are often delivered in the forms of points or monetary compensation. Because the reward for all options often uses a single form of currency (e.g., money, points), decision–making experiments will typically be used to evaluate the relative value of loses and gains,¹² and the effect of various discounting factors.¹ It has been observed since the early 1970s that the subjective value of points and money is not related to their objective value using a simple linear function. As a result, the subjective value of the different choices is often described in term of utility. The most well–known theory using utility functions is prospect theory,^10,11 which has been successfully used in psychology and economics for almost 40 years. Because decision–making is often thought as the most “canonical” way of studying valuation, it has been shown to rely in large part on the brain areas related to valuation, i.e., medial PFC, VS, PCC, amygdala, insula, and PPC.^14,15 The role of reward processing in decision–making is to learn the subjective value (or utility) of each choice, which is covered in the next subsection.

Learning

Learning is a fundamental ability in every animal that allows for better adaptation to a changing environment. One important goal of learning is to correctly estimate the value of states and actions in order to maximize the future acquisition of rewards.³ From a behavioral perspective, animals learn to select states and actions that lead to reward (instrumental conditioning). From a neurobiological perspective, this usually amounts to modifying synapses in the brain to encode or update associations.⁵⁴

There are many forms of neural plasticity that operate over a wide range of time scales. The plasticity–related phenomena forming the neural basis of learning are widely–believed to be long–term potentiation (LTP) and long–term depression (LTD).⁵⁵ LTP and LTD refer to long–lasting increases and decreases (respectively) in synaptic efficacy, which results from simultaneously stimulating the pre– and post–synaptic neurons. LTP and LTD have been closely studied in many different brain regions and in many different cell types. While synaptic plasticity is possible without reward (e.g., Hebbian learning), plasticity in the basal ganglia (which includes the striatum) is thought to rely heavily on DA–mediated feedback.^56,57 As mentioned earlier, phasic DA produces dips when an expected reward is smaller than expected, and peaks when the reward is larger than expected. As a result, DA is widely believed to encode a reward signal for RL.^58–60 RL allows for learning the values of states and actions using trial–and–error, thus allowing for the striatum to play an important role in valuation and reward processing. Valuation and reward processing are strongly coupled and are jointly essential for learning since reward processing is used to learn values, and valuation is used to make choices leading to reward and provide the required feedback for RL.

Psychiatric disorders

Many psychiatric disorders are defined in part by abnormal reward–related behaviors, ranging from insensitivity to appetitive stimuli (e.g., anhedonia, a common symptom of major depression), to extreme thrill–seeking (e.g., reckless driving during an episode of mania), to compulsive reward–seeking despite catastrophic consequences (e.g., addiction). Psychiatric disorders are also linked to many cognitive deficits. While deficits in valuation and reward processing have most commonly been studied independently from cognitive deficits, we argue that many cognitive deficits in psychiatric disorders can be accounted for by abnormal reward–related behavior.

Addiction

Addiction is characterized by marked dysfunction in reward–seeking behavior: compulsive and continued use of drugs or alcohol despite significant physical, psychological, and social consequences. Substance abuse acts directly on the brain’s mesocorticolimbic circuit (i.e., the circuit innervating the striatum, PFC, and ACC with DA), and the transition from casual usage to addiction is thought to be caused by substance–induced sensitization of the brain to reward–associated stimuli.^61,62

Valuation

Neuroimaging studies have consistently linked addiction to abnormal valuation, as evidenced by altered neural reactivity to reward–predicting cues. For example, fMRI studies show that reward cues elicit increased activity throughout the mesocorticolimbic circuit, including the striatum, OFC, and ACC among healthy controls.⁶³ This activity is blunted in individuals with alcohol dependence,⁶⁴ nicotine dependence,⁶⁵ cocaine abuse,⁶⁶ and chronic cannabis usage,⁶⁷ indicating reduced valuation of monetary rewards across these substances.

Complementing these fMRI studies, event–related potential (ERP) studies have examined the modulation of the P3 to reward–predicting cues. The P3 reflects the allocation of attention to task–relevant, motivationally salient, or unexpected stimuli,⁶⁸ and in healthy individuals the P3 is increased to reward–predicting cues.^69,70 This effect is blunted among cocaine users,⁷¹ although this deficit may be state–dependent: in a follow–up study contrasting abstinent with current users, the P3 to reward cues was blunted only within the abstinent group, suggesting that cocaine usage (versus withdrawal) may temporarily normalize valuation of monetary rewards.⁷²

Other neuroimaging studies have examined how addiction alters motivated attention to drug–related stimuli. For example, in one study alcohol images elicited increased VS activity compared to neutral images only among individuals with alcohol dependence and not in healthy controls.⁶⁴ Interestingly, this study also used an incentive delay task, finding reduced VS activity to reward cues among users; on both tasks, abnormal valuation (i.e., reduced valuation of monetary reward, increased valuation of alcohol cues) was correlated with self–reported craving. Similarly, ERP studies have also found increased valuation of drug–related cues among users. Of interest in these studies was the late positive potential (LPP), an ERP that reflects the sustained allocation of attention to motivationally salient stimuli.^73,74 In two studies of addiction, controls exhibited the typical modulation of LPP amplitude, with an increased LPP to emotional versus neutral images.^75,76 Critically, drug–related images modulated the LPP only among users and not among controls, indicating that these stimuli are not inherently salient but have acquired salience among individuals with addiction. As with the aberrant processing of monetary cues, this abnormality may also be state–dependent: in one study, the increased LPP to drug images was largest among abstinent users, indicating that current usage may temporarily blunt the neural response to drug cues.⁷⁵ Overall, addiction is associated with an imbalance in valuation, which systematically alters the salience of environmental stimuli. Among individuals with addiction, normally salient stimuli (i.e., monetary cues) appear to be less salient, whereas drug–related stimuli are more salient.

Reward processing

In addition to these findings of abnormal valuation in addiction, other studies have focused on reward processing. ERP studies generally distinguish between two stages of reward processing: (a) the reward positivity (RewP) (also termed the feedback negativity/FN or feedback–related negativity/FRN) occurs 250–300 ms following feedback and tracks the early, binary evaluation of outcomes as either good or bad;⁷⁷ (b) and the P3, which tracks outcome magnitude, irrespective of valence.⁷⁸ In one study of individuals with alcohol dependence, the ERP waveform was blunted overall in users, but the effect of condition was intact: the RewP was increased for wins versus losses, and the P3 was increased for large versus small outcomes.⁷⁹ Importantly, a simple guessing task was used in this study, in which reward delivery was random and was not based on successful performance or learning.

A different pattern has emerged from studies in which the likelihood of reward varies across contexts. For example, one study used a guessing task in which reward probability varied across trials and was signaled by a cue.⁸⁰ Among controls, the RewP was modulated by both positive and negative reward prediction errors (i.e., increased for unpredicted versus predicted outcomes). Among current cocaine users, the RewP was modulated by unpredicted wins but not unpredicted losses, and among abstinent cocaine users the RewP was modulated by neither unpredicted wins nor unpredicted losses. Similarly, another recent ERP study used a modified incentive delay task in which performance feedback (i.e., successful versus unsuccessful) was presented separately from reward feedback (i.e., win versus loss).⁸¹ The P3 to performance feedback, as well as the RewP and P3 to reward feedback, were all blunted among users. Both of these studies indicate an impaired ability to update reward predictions based on shifting contingencies across trials, which likely relates to impaired reward learning.

Complementary data comes from two recent fMRI studies that specifically examined the acquisition of action–outcome contingencies. In one study of abstinent cocaine users, a spatial learning task was used in which participants navigated a virtual reality maze.⁸² Users exhibited blunted activity within the medial temporal lobe and several prefrontal regions during spatial navigation, but increased striatal activity to unexpected reward delivery. This pattern suggests a disruption between reward processing, spatial learning, and episodic memory (a form of LTM) among users. In a second study, stimulant users were examined during an implicit reward learning task.⁸³ Both users and controls effiectively learned the reward contingencies, but users exhibited increased activity within the insula, inferior frontal gyrus and dorsal striatum. Group differences were specific to the late trials in each block (i.e., after learning had occurred) and not the early trials (i.e., during learning). This suggests inefficient learning and greater resource allocation during decision–making among users.

Effect on cognitive function

Overall, these data indicate that addiction has an impact on both valuation and reward processing. The balance between drug and non–drug rewards is altered, such that drug–predicting environmental cues are more salient while non–drug reward cues are less salient (an attention effect). With regard to reward processing, neural reactivity to reward delivery (particularly unexpected rewards) appears to be unaffected. The modulation of reward processing based on the immediate context and the integration of reward processing with other cognitive functions, however, appears to be impaired (a learning effect). Of possible relevance to these findings on both valuation and reward processing, a recent study using resting–state fMRI observed reduced functional connectivity between the VS and multiple areas of the frontal cortex, and this connectivity deficit was related to poor behavioral performance on a sustained attention task;⁸⁴ no connectivity deficits were observed for the caudate or putamen. Thus, one possibility is that abnormal activation on reward tasks may be driven by impaired interactions between regions of the mesocorticolimbic circuit in addiction.

Pathological gambling

Pathological gambling is characterized by persistent gambling behavior despite overt negative consequences. Pathological gambling often interferes with interpersonal relationships, and profoundly influences financial and socioeconomic status. It shares phenotypic features with substance use disorders, as well as some common deficits in reward functioning.⁸⁵

Valuation

The pattern of abnormal valuation in pathological gambling appears to be context–specific. In one study using an incentive delay paradigm, the patient group exhibited reduced activity to monetary cues throughout the mesocorticolimbic circuit, including the medial PFC, ACC, VS, and inferior frontal gyrus.⁸⁶ While this study suggests diminished valuation in pathological gambling, other studies using more ecologically valid gambling paradigms show a different pattern. For example, patients showed riskier decision–making in a simulated Blackjack task.⁸⁷ Relatedly, patients showed increased activity in the striatum, amygdala, and OFC while making high– versus low–risk decisions in the Iowa gambling task (IGT).⁸⁸

Reward processing

Similar to valuation, the pattern of abnormal processing of reward outcomes appears to be context–specific. Pathological gambling has been linked to reduced activity in the medial PFC and vmPFC on an incentive delay task⁸⁶ and a simple guessing task.⁸⁹ On the other hand, ERP studies using more ecologically valid gambling paradigms have found evidence of hypersensitivity to reward outcomes. For example, the RewP is increased in patients on the IGT, and it is positively correlated with gambling symptom severity.⁹⁰ Similarly, RewP amplitude is increased among patients on a simulated Blackjack paradigm, particularly for high–risk reward outcomes.⁸⁷ Conversely, the RewP is relatively insensitive to losses among patients (i.e., it does not distinguish between full losses and near–misses).⁹¹ This hypersensitivity to positive rewards does not necessarily translate to effective reward learning: patients performed worse on the IGT in two studies, showing a behavioral insensitivity to implicit reward contingencies (a learning effect).^88,90

Effect on cognitive function

Together, these studies suggest that pathological gambling is associated with a riskier response style, increased valuation, and increased reward sensitivity in situations that closely approximate real–world gambling behavior. In situations where risk is low or held constant, however, pathological gambling is associated instead with reduced valuation and reward sensitivity. Both these phenomena negatively affect decision–making and learning.

Schizophrenia

Schizophrenia is a severe psychiatric disorder characterized by a large number of heterogeneous symptoms. Primary features of schizophrenia include positive symptoms (hallucinations, delusions), disorganized symptoms (thought disturbance), and negative symptoms (anhedonia, avolition).

Valuation

Consistent with motivational deficits that are commonly observed among patients with schizophrenia, neuroimaging studies have found evidence of blunted valuation. In one study using a monetary incentive delay task, unmedicated patients exhibited reduced activity to reward–predicting cues in the left VS, which was associated with negative symptom severity.⁹² This deficit has also been observed in patients taking typical antipsychotic medications⁹³ but not atypical antipsychotics.^94,95 In one study, valuation–related striatal activity was normalized when patients were switched from typical to atypical antispychotics.⁹⁶ One limitation of this research is that reward attainment requires successful response selection and execution, and these task processes are also commonly impaired in schizophrenia. To isolate valuation from motor–related activity, one study used a passive incentive delay task and found that patients exhibited intact striatal activation to reward–predicting cues.⁹⁷ Interestingly, anhedonia severity related to diminished left VS activity in both patients and controls.

Reward processing

Whereas schizophrenia is associated with impaired valuation in at least some contexts, these same studies generally show intact VS activation to reward delivery in patients compared to healthy controls.^94,95,97 Complementary data comes from an ERP study in which patients exhibited normal RewP amplitude in a simple guessing task.⁹⁸ Interestingly, while valuation–related activity has been related to negative schizophrenia symptoms, including apathy and anhedonia, in one study reduced striatal activity to reward outcome was related to comorbid symptoms of depression.⁹⁴

These studies suggest that the binary evaluation of outcomes (reward versus non–reward) may be unaffected in schizophrenia, but deficits in other aspects of reward processing have been reported. For example, in one study patients exhibited a blunted effect of outcome salience (large versus small outcome) in the ventrolateral PFC.⁹⁵ Other studies have observed deficits in the encoding of prediction error signals and expected reward values.^99,100 Broadly similar to the deficits described above in Addiction, schizophrenia appears to be characterized by context insensitivity and deficient updating of reward prediction based on shifting contingencies.

Of relevance to these neuroimaging findings is a neuropsychology study in which impaired reward processing was related to cognitive functioning and negative symptom severity.¹⁰¹ Participants performed significantly worse than controls in the IGT,¹⁰² indicating poor reward learning and impaired decision–making.¹⁰¹ This deficit in reward learning was associated with performance on several other cognitive tests, including overall intelligence, memory, and executive function, as well as negative symptom severity. Together with the neuroimaging findings described above, these data suggest a point of intersection between impaired valuation, disrupted integration of reward processing with other cognitive functions, and negative symptomatology in schizophrenia.¹⁰³

Effect on cognitive function

Schizophrenia is characterized by severe cognitive impairment across multiple domains, including executive function, memory, attention, decision–making, learning, and processing speed.¹⁰⁴ Neuropsychological studies have shown that cognitive impairment is specifically associated with negative rather than positive or disorganized symptoms,¹⁰⁵ indicating that motivational and hedonic deficits are closely linked with cognitive deficits. This is consistent with the proposal that cognitive symptoms are driven by disturbances within the brain’s mesocorticolimbic network, particularly dopaminergic signaling to the PFC.^106,107

Mood disorders

The most prominent mood disorders are major depressive disorder and the bipolar disorders, each of which are characterized by core deficits in reward–seeking behavior. As in schizophrenia, a primary symptom of depression is anhedonia, defined as diminished interest in, and pleasure derived from, normally enjoyable activities. The bipolar disorders, meanwhile, are characterized by oscillations of depressive and manic symptoms, with the latter characterized by impulsive and often risky reward–seeking (e.g., drug use, spending sprees). Bipolar I refers to patients that experience full episodes of mania, whereas bipolar II refers to subthreshold symptoms of mania (i.e., hypomania). All of these disorders are characterized by abnormal valuation and reward processing, and there is growing interest in characterizing shared and unique deficits across the mood disorders.¹⁰⁸

Valuation

Among the bipolar disorders there is evidence of increased valuation in both overt reward preference, as well as neural reactivity to reward–predicting cues. Due to the difficulty of examining patients while in a current manic state, studies typically examine patients who have a history of manic or hypomanic symptoms but who are currently in a neutral mood state (i.e., euthymic). Studies using temporal discounting tasks have observed stronger preferences for immediate over delayed rewards (which affects decision–making), in both bipolar I disorder¹⁰⁹ and in relation to hypomanic traits.¹¹⁰ In two fMRI studies using guessing tasks, elevated VS activity during the anticipation of reward outcomes was observed in euthymic bipolar I¹¹¹ and bipolar II patients.¹¹²

With regard to depression, however, evidence of abnormal valuation is more mixed. In two studies using a monetary incentive delay task, patients with unipolar depression exhibited normal striatal activation during reward anticipation.^113,114 While these studies suggest that valuation may be relatively unaffected in depression, a third study examined anticipatory striatal activation as a function of the previous trial’s outcome (e.g., anticipation after winning on the previous round).¹¹⁵ They found that the depressed group exhibited blunted anticipatory striatal activation only on trials following wins, suggesting that depression may disrupt the updating of valuation based on previous reward receipt and perhaps a difficulty in transferring previous success to influence expectations of future success.

While the studies described above assessed individuals experiencing current depressive symptoms, a different question is whether unipolar and bipolar depression exhibit distinct abnormalities in valuation when in the currently depressed state. One fMRI study used a guessing task to compare three groups: currently depressed bipolar I, current unipolar depression, and healthy controls.¹¹⁶ Both the unipolar and bipolar depressed groups were characterized by reduced ACC activity during reward anticipation. Only the bipolar depressed group, however, was characterized by increased ventrolateral prefrontal activation, perhaps reflecting increased valuation in the latter group, albeit in a different manner from the euthymic bipolar studies described above.

Reward

In contrast to the mixed evidence for valuation described above, there is consistent evidence linking depression to diminished neural activity involved in reward processing. Notably, in the two fMRI studies mentioned above showing intact striatal activity during reward anticipation, both studies observed blunted striatal activity to reward outcomes in depression,^113,114 and this same pattern has also been observed in depressed bipolar patients.^117,118

These fMRI data are complemented by several ERP studies showing a blunted RewP to reward in individuals with current depressive symptoms^74,119 and a diagnosis of major depressive disorder.¹²⁰ In one multimodal neuroimaging study, a blunted RewP in the depressed group was associated with blunted striatal reactivity to monetary rewards, suggesting a common neural deficit.¹²¹ Together, these data suggest that anhedonia, a cardinal symptom of depression, may be primarily characterized by reduced neural sensitivity to reward outcomes. This stands in contrast to one study reporting increased reward sensitivity among hypomanic–prone individuals.¹¹⁰ On a speeded response task in which rewards could be either immediate or delayed by up to a week, hypomanic–prone individuals exhibited an increased visual N1 and an increased RewP to immediate versus delayed rewards, indicating facilitating attentional processing of these outcomes.

Other research has examined how diminished reward sensitivity in depression may affect other cognitive functions, particularly reward learning. On reversal learning tasks, depression is associated with poor behavioral accuracy to unexpected rewards as well as reduced striatal reactivity.¹²² On probablistic tasks, individuals with depression exhibit reduced sensitivity to implicit reward contingencies, which is associated specifically with anhedonia severity¹²³ and predicts a poor treatment response.¹²⁴ Interestingly, a similar behavioral deficit in implicit reward learning has been reported in bipolar disorder,¹²⁵ which contrasts with the evidence of increased valuation and reward sensitivity described above, suggesting a disruption between these processes and reward learning per se.

Effect on cognitive function

Overall, the bipolar disorders appear to be most consistently associated with increased valuation (affecting decision–making), particularly in euthymic states, whereas both unipolar and bipolar depression are associated with diminished reward processing (affecting learning). Depression is also associated with poor reward learning, which may stem from diminished salience of reward outcomes. While the evidence to date is not entirely consistent, there appear to be both shared and distinct deficits across these disorders, highlighting the importance of considering both current mood state (i.e., manic, euthymic, depressed) as well as illness history (i.e., history of bipolar versus unipolar depression) when characterizing abnormalities in valuation and reward processing.

Conclusion

This article reviewed evidence that many cognitive functions are related to valuation and reward processing, and that many cognitive deficits observed in psychiatric disorders can be accounted for by deficits in valuation and reward processing. Specifically, attention deficits in addiction and schizophrenia have been related to valuation, memory deficits in schizophrenia have been related to reward processing, decision–making deficits in pathological gambling, schizophrenia, and mood disorders have been related to valuation, and learning deficits in addiction, pathological gambling, schizophrenia, and mood disorders have been related to both valuation and reward processing.

This framework has many possible implications. For example, if valuation and reward processing are core processes used by most other cognitive functions, then special care needs to be taken when studying any cognitive function to ensure that the results obtained are really specific to the cognitive function of interest and not pertaining to the core processes of valuation or reward processing.⁶ In addition, if valuation and reward processing are responsible for some cognitive deficits observed in psychiatric disorders, then these core processes might hint at a target for treatment.

Clinical implications

Research on abnormal valuation and reward processing in psychiatric disorders can be used to directly improve clinical care in multiple ways, including: (1) Identify mechanisms of illness risk. Abnormal reward processing prospectively predicts first–episode depression onset,¹²⁶ and abnormal valuation is present in unaffected first–degree relatives of schizophrenia patients.¹²⁷ These deficits may represent mechanisms of risk and could potentially be used to guide preventative efforts. (2) Predict course of illness. Poor reward learning and abnormalities in valuation and reward processing are apparent in remitted depression, which may help to explain subsequent depression recurrence.^128,129 (3) Predict treatment outcome. Abnormal valuation predicts treatment outcome in cocaine dependence, and could potentially be used to guide treatment selection.¹³⁰ (4) Provide novel targets for intervention. Direct stimulation of brain regions involved in valuation and reward processing is effective in treatment refractory mood disorders.^131–133

Dimensional approaches

Research on valuation and reward processing to date has largely been organized around categorical definitions of disorders (e.g., contrasting patients with a diagnosis of depression versus never-depressed controls). This is at odds with the fact that deficits in valuation, reward processing, and other cognitive functions typically cut across disorders. Thus, there is growing emphasis on dimensional approaches to psychiatric research, in particular the Research Domain Criteria Project (RDoC).¹³⁴ RDoC is an alternative classification system organized around core biobehavioral dimensions of functioning rather than existing diagnostic categories. In doing so, the ultimate aims are to identify new targets for treatment, detect biologically distinct subgroups of patients, and facilitate the dissemination of research findings to clinical decision–making.¹³⁵ It is believed that researching more narrowly defined, neurobiologcally–based symptoms (e.g., habit learning) will have greater success than focusing on more complex, heterogeneous illness categories as dined in the current nosology¹³⁶

Of relevance to the current review, RDoC includes the general domain of Positive Valence Systems, which itself is comprised of more specific constructs relevant to valuation (e.g., Reward Valuation, Effort Valuation, Expectancy) and reward processing (Initial Responsiveness to Reward Attainment, Longer-Term Responsiveness to Reward Attainment). Insofar as deficits in these constructs are related to each of the disorders reviewed here, it would be valuable for future research to consider how valuation and reward processing map onto complex psychiatric illness (i.e., combinations of mood, psychotic, and addiction symptoms). This dimensional approach can also be used to develop novel and broadly-applicable interventions. For example, one recent study outlined the development of a new behavioral treatment that specifically targets impairment in valuation.¹³⁷ Based on the pattern of evidence reviewed above, such an intervention could potentially be beneficially to patients with symptoms of addiction, depression, pathological gambling, and schizophrenia.

Limitations and future work

While this review article covered a wide array of topics in relation with valuation and reward processing, it is incomplete and many other relevant topics have been left out. For example, the topics of habit learning and automaticity have not been covered directly. In cognitive psychology, automaticity is typically considered in the context of attention (because automatic behavior require little attention, e.g., driving), and in neuropsychology habit learning is typically linked to addiction (because both addictions and habitual behaviors are performed in the absence of rewards).¹³⁸ In the context of the present framework, we argue that valuation and reward processing play an important role in learning these behaviors, but that repetition eventually makes behavior production rewarding (as hinted by the reviewed literature on addiction).¹ The reviewed research on addiction suggests that this might be related to abnormal functional connectivity in the brain circuit supporting valuation and reward processing,⁸⁴ which in turn suggests that more attention should be paid to functional connectivity, and also specifically to the default mode network (because of its overlap with the valuation/reward processing circuit and the importance of cognitive effort).¹³⁹ Future work needs to be devoted to these topics.

Another research area that has been under–studied is the more ecologically valid field of decision–making when different currency are used for each choice option. For example, taking a nap has some value, and eating a meal also has a value. However, these values are not directly commensurable, so one has to convert these values into a common currency in order to make a rational choice. Converting qualitatively different types of values into a common currency is one of the most intriguing functions of valuation (see Refs. 23,24). This function is also highly relevant to clinical research, since there is often an implicit assumption that disorders will be characterized by a global increase or decrease in valuation and/or reward processing, when instead the pattern appears to vary substantially depending on the nature of the reward and the situation. The most striking examples are for addiction and pathological gambling, where hypersensitivity to reward is only observed for more ecologically valid tasks. Future work should be devoted to studying choices where qualitatively different types of values needs to be converted into a common currency to see how the cognitive and neurobiological results would match the existing literature, and how these finding could translate into useful diagnostic tools for psychiatric disorders.