Reward Processing, Neuroeconomics, and Psychopathology

Abstract

Abnormal reward processing is a prominent transdiagnostic feature of psychopathology. The present review provides a framework for considering the different aspects of reward processing and their assessment and highlight recent insights from the field of neuroeconomics that may aid in understanding these processes. Although altered reward processing in psychopathology has often been treated as a general hypo- or hyper-responsivity to reward, increasing data indicate that a comprehensive understanding of reward dysfunction requires characterization within more specific reward processing domains, including subjective valuation, discounting, hedonics, reward anticipation and facilitation, and reinforcement learning. As such, more nuanced models of the nature of these abnormalities are needed. We describe several processing abnormalities capable of producing the types of selective alterations in reward related behavior observed in different forms of psychopathology, including (mal)adaptive scaling and anchoring, dysfunctional weighting of reward and cost variables, completion between valuation systems, and positive prediction error signaling.

INTRODUCTION

Alterations in reward processing are a feature of multiple forms of psychopathology. Indeed, reward-processing symptoms are explicitly instantiated as diagnostic criteria for multiple disorders in the DSM-V (American Psychiatric Association 2013) including criteria for all affective disorders, urges, cravings and abnormal valuation in addiction and impulse control disorders, the anhedonic symptoms of schizophrenia, and abnormally low valuation of rewarding social experiences in schizoid personality disorder and autism. Based on the prevalence of these disorders, and the centrality of reward to the expression of these conditions, reward alterations are arguably among the most common symptoms of psychopathology in humans, occurring at a level that is arguably only rivaled by negative emotionality as a broad of psychological symptomatology.

Given the prevalence of reward processing features in psychopathology, reward related symptoms can be seen as prototypically transdiagnostic in nature. Such symptoms may contribute to comorbidity of psychiatric conditions both because the symptoms appear in the formal diagnostic criteria for multiple disorders, and more theoretically interestingly, because the same or related reward processing abnormalities are a core component of the development and expression of multiple forms of psychopathology. Indeed, the National Institute of Mental Health’s Research Domain Criteria (RDoC), which attempts to characterize psychopathology based on functional domains, defines a group of reward related processes (labeled Positive Valence Systems), as one of five cross-cutting substrates for psychopathology (Insel et al 2010).

While recognizing the breadth of reward abnormalities in psychopathology is important, it would be a mistake to consider them homogenous across or even within different disorders. Indeed, within the RDoC framework, multiple distinct constructs make up the Positive Valence Systems domain, and differences in DSM-V criteria at least implicitly, if not always explicitly, appear to capture different reward processes.

In the present article, we aim to outline the current state of knowledge regarding reward processing in psychopathology. Our goal is not to provide a comprehensive review of the literature for each disorder, but rather to articulate a broad framework for conceptualizing the nature of reward abnormalities. We particularly highlight ideas derived from the burgeoning fields of behavioral and neuroeconomics, which in recent years have provided novel insights regarding processes related to valuation and decision-making. Although we draw significantly from this literature, especially with regards to mesolimbic dopamine (DA) functions, we note that a comprehensive review of neuroeconomics is also beyond the scope of the paper. Rather, our goal is to demonstrate the potential of these concepts as a source for hypotheses about the patterns of reward processing alterations that characterize psychopathology.

A TAXONOMY OF REWARD PROCESSES

We begin with a brief taxonomy of reward processes in order to characterize some of the key constructs and approaches that have guided the literature to date, and to facilitate precision in characterizing the specific reward processes that are altered in psychopathology. Different disciplines have characterized reward processes in distinct ways, often varying in terms of their emphasis on different features and functions such as subjective experience, learning, action facilitation and decision making.

Subjective Experience

Perhaps the most intuitive means of defining a rewarding stimulus or event is to measure the hedonic (pleasurable) experience of receiving it (O’Doherty 2014). When defined in relation to an event or object, the subjective experience is closely tied to the evaluation of the stimulus (how likeable it is). However, we can also define the subjective experience in terms of the affective or emotional experience itself (e.g., joy, pleasure, positive affect). In characterizing subjective hedonic experiences within affective space, the dimension of valence characterizes the intrinsic attractiveness of stimuli and the subjective experiences they evoke, with positive valence being attractive and negative valence aversive. This decades-old conceptualization has been a useful descriptor of animal behavior (Ferster & Skinner, 1957), affective and physiological states (Russell & Barrett 1999; Tellegen, 1988), and even economic choice (Li et al., 2011).

Reward Anticipation and Facilitation

Within the reward literature, a classic division is drawn between the hedonic impact of reward attainment and its anticipation. This distinction finds support across multiple levels of analysis, including neurophysiology, behavior, and subjective experience where it is often described in terms of ‘liking’ vs. ‘wanting’ (Berridge & Robinson 2003). However, one can also distinguish several related, but distinct aspects of reward anticipation. At the subjective level, reward anticipation can be characterized both as wanting (e.g., urges and craving), but also as excitement or tension. Behaviorally, it is principally displayed as approach behavior directed at acquisition or goal attainment. Additionally, it is reflected in what we term reward facilitation, which refers to the multiple perceptual, attentional, cognitive and motoric processes that are facilitated when rewards are at stake (Knutson et al 2001, Maunsell 2004). We note that the term reward anticipation is often used by researchers to describe this type of facilitation rather than an explicit anticipation of the reward. For instance, in tasks like the monetary incentive delay (MID) task (Knutson et al 2001), the term is used to refer to preparation to make a response in order to potentially gain a reward, rather than the expectation that the reward is about to be obtained.

Concepts from Behavioral Economics

As a discipline, behavioral economics has traditionally been concerned with processes of decision making. The field intersects with affective science in that subjective evaluative processes permeate decision making and both emotional evaluations and decision making rely on the valuation of potential and attained rewards and losses (Loewenstein 2000). Indeed, the subjective emotional experiences described above may be viewed as an emergent property of valuation. We therefore turn our attention to some key concepts from this literature.

Subjective Value

A central challenge to studying mechanisms for “reward processing” and its dysfunction is that each of us conceives reward differently. Economists have described this individualized valuation as “subjective value” (Kable & Glimcher 2007) or “utility”. While the most seemingly straightforward approach to the assessment of subjective value would be to simply ask people how rewarding they find something, precise estimates of subjective value can be difficult to achieve. People are often inconsistent about what value they place on various options and their answers can be heavily influenced by their prior responses or how questions are framed and ordered (Ariely & Norton 2008, Kahneman et al 2006). However, with enough data it is possible to generate individual utility functions that rank-order different options in some monotonic arrangement of preference. The magnitude of subjective value differences is also reflected in the stability of preference choices, with large differences leading to consistent choice, and lesser differences producing more variable choices.

Costs and Discounting

A critical component of any economic transaction is the cost necessary to obtain the potential reward. Indeed, one approach to determining the subjective reward value of something is to simply find the maximum price someone is willing to pay for the good or service (Becker et al 1964). In the neuroeconomics literature, this concept has been broadened to include the willingness to bear any type of response cost in order to acquire something. For example, researchers have used effort expenditure such as lever pressing during progressive ratio schedules or the vigor of responses to determine how much an individual is willing to “pay” to achieve a given reward (Niv et al 2007; Salamone et al 2016). Similarly, one can also examine how long an animal is willing to wait (temporal costs), lose opportunities to obtain another reward (opportunity costs), or is willing to risk not receiving a reward (Floresco et al 2008, Niv et al 2007; Schultz 2015, Wade et al 2000).

In an economic exchange, incorporation of response costs causes a discounting of the utility of obtaining the reward. For example, the utility of a reward decreases with the amount of time you have to wait to receive the reward. Given a choice between $10 now or $11 in a week, most people will choose $10 now, despite its lower absolute value. Across multiple choices with varying reward magnitudes and delays, we can quantify the individual’s level of temporal discounting as well as the shape of their discounting function (usually approximated by a hyperbolic discounting curve) (Odum 2011).

As noted above, in animal studies the willingness to expend effort can be used to gauge subjective value, and it can be similarly quantified in terms of discounting functions. This domain is highly salient in human choice behavior, where the amount of energy expended in pursuit of goals can vary enormously, and are magnified with repetition (for instance in terms of willingness to practice in order to develop skilled performance, exercise for health, or study to get good grades).

Traditionally, the behavioral economics literature has assumed that the brain calculates a general utility signal that integrates all the relevant features of various reward options, such as how long you will have to wait or work to gain a certain reward, and the probability of getting the reward. Some support for this assumption has emerged from recent studies of the firing of DA neurons, which appear to differentially fire based on the expected utility of different lottery options (Schultz et al 2015) and are sensitive to effort (Varazzani et al 2015), temporal (Kobayashi & Schultz 2008) and probability discounting (Fiorillo et al 2003), and exhibit the type of adaptive scaling necessary to represent a wide range of reward values under different contexts (Tobler et al 2005). While this remains an attractive theory, growing evidence challenges the hypothesis that a unitary neural signal for subjective utility exists. Lesion and imaging studies suggest, for example, the costs related to effort versus delay different valuation systems (Prevost et al 2010, Rudebeck et al 2006) and produce distinct (and often uncorrelated) discounting behavior (Klein-Flügge et al 2015). Moreover, recent work has found that contrary to predictions based on a utility model of dopaminergic activity, DA linked reward signaling in the striatum is heavily influenced by whether action is necessary to get a reward (Collins & Frank 2016, Syed et al 2016). Consequently, the identification of neural signals that appear to track a pure utility signal in one type of experimental design (e.g., when rewards of different magnitudes all require some action to acquire), may fail to generalize to other paradigms. Thus far, the key dimensions that drive the processing of response costs (e.g., with/without action) in combination with, or distinct from reward, are still being elucidated.

The importance of their being multiple valuation systems has implications beyond the calculation of utility. If the subjective hedonic experience is an emergent property of valuation processes and there are multiple valuation systems, then subjective hedonic and reward anticipation experiences may be similarly multi-determined. We return to the potential importance of multiple valuation and discounting systems later in this review as it has significant implications for characterizing reward processing abnormalities in psychopathology.

Reinforcement Learning

A final area that is frequently incorporated into taxonomies of reward processing is reinforcement learning. Although not necessarily a process related to reward per se, the majority of studies in this area have relied on the use of positively-valenced reinforcers as a means of studying the behavioral and neural mechanisms that underlie various forms of associative learning. Reinforcement learning has been especially useful as a means of elucidating neural signals that appear to track predictions from formal models of error-driven learning (e.g. reward prediction errors “RPE”) (Rutledge et al 2010, Schultz 2015, Schultz et al 1997), and as such provide a means of probing the extent to which brain areas in clinical populations are more or less sensitive to reward-relevant information (Frank et al 2004).

REWARD PROCESSING ASSESSMENT

Apathy, anhedonia, avolition, anergia, negative symptoms and fatigue on one end of the spectrum, and excessive goal related activity, positive urgency, and impulsivity on the other end of the spectrum are among just some of the many labels for reward-related symptoms as diagnosed in different disorders and described by clinicians from various nosologic backgrounds. In some cases, these names obscure important differences in symptom phenomenology and underlying neural mechanisms–such as the distinction between motivational and consummatory aspects of anhedonia in depression (Treadway & Zald 2013); in others, they may reflect differences in training and orientation, such as the tendency to label a reduction in motivation as “fatigue” or “weakness” in oncology and neurology, while the same presentation would likely be referred to as anhedonia or anergia in clinical psychology.

The potential impact of seemingly harmless differences in nomenclature have become increasingly apparent as the field has focused on identifying common pathophysiological mechanisms for clinical symptoms. Diagnosis of major depression includes multiple reward-related symptoms, including anhedonia, diminished sexual drive, low energy, and psychomotor slowing, all of which have long been conceptualized as distinct depressive symptoms, (Feighner et al 1972), yet the single “anhedonia” criterion has been defined so broadly that it can be met through demonstrated “loss of pleasure or interest” in previously enjoyed activities. Yet, pleasure and interest/motivation echo the distinction between ‘wanting’ and ‘liking’ aspects of reward behavior as instantiated within various neural circuits, and their lumping together provides a clear example of how current diagnostic criteria may be out of step with both phenomenological and neurobiological reality.

In seeking to refine the assessment of reward related abnormalities in psychiatric disorders, it is useful to consider the extent to which existing measures tap specific features within the taxonomy of reward processes. Below, we summarize some of the most prominent approaches to date.

Self Report

A number of self-report measures have been used to assess the extent to which patients and healthy individuals experience appetetive or consummatory subjective responses for typical or disorder-specific rewards (see Table 1 for representative examples). In the personality domain, several of these measures specifically attempt to tap aspects of a theorized behavioral activation system. This work builds on Gray’s (1970) reinforcement sensitivity theory in which individuals are posited to critically difer in their sensitivity to conditioned and unconditioned reward cues, which is manifested in approach motivation and impulsivity. Trait assessment of reward relevant processes are additionally embedded in a number of broad personality measures (McCrae & Costa 1987; Tellegen et al 1992). Although demonstrating significant utility, a limitation of a number of these measures is their tendency to “lump together” as equivalent a wide variety of positive emotional experiences, which restricts their interpretational precision (Barch & Dowd 2010, Gold et al 2008, Treadway & Zald 2013).

TABLE 1.

Representative self-report measures of reward processes and symptoms

Measure	Sub-Scales	Reference	Description
Snaith-Hamilton Pleasure Scale (SHAPS)		(Snaith et al 1995)	State-measure of enjoyment of everyday pleasurable activities.
Temporal Experience of Pleasure Scale (TEPS)	Anticipatory and Consummatory	(Gard 2006)	Trait-measure focused on dissociating anticipatory pleasure from consummatory enjoyment.
Chapman Anhedonia Scales	Physical and Social	(Chapman et al 1976)	Trait measure focused on enjoyment of various physical and social rewards. Developed to assess anhedonia in schizophrenia.
Fawcett-Clark Pleasure Scale (FCPS)		(Fawcett et al 1983)	Trait measure of enjoyment of work, time with familty, monetary rewards and physical sensations. Developed to assess anhedonic depression.
Mood and Anxiety Symptom Questionaire (MASQ)	Anhedonic-depression	(Watson et al 1995)	Low interest and pleasure, low positive affect. Developed to distinguish depressive symptoms vs. general distress and anxiety.
Specific Loss of Interest and Pleasure Scale (SLIPS)		(Winer et al 2014)	Assessment of recent change in enjoyment and interest.
Penn Alcohol Craving Scale		(Flannery et al 1999)	Brief scale assesses craving and ability to resist urges for alcohol.
Cocaine Craving Questionnaire		(Tiffany et al 1993)	Assesses desire, anticipation of positive outcome, anticipation of relief, and lack of control for cocaine. Administered as either a state (Now) or general craving measures.
Behavioral Activation Scale	Fun Seeking, Drive, Reward Responsiveness	(Carver & White 1994)	Developed with the Behavioral Inhibition Scale to operationalize Gray’s Reinforcement Sensitivity Theory.
Sensitivity to Reward Questionnaire		(Torrubia et al 2001)	Operationalization of Gray’s Reinforcement Sensitivity Theory with an emphasis on responses to specific reward cues.
Appetitive Motivation Scale		(Jackson & Smillie 2004)	Operationalization of Gray’s Reinforcement Sensitivity Theory with an emphasis on motivation to approach ideas and physical stimuli, and appraisal of obtaining rewards.

An additional concern related to self-reports is their frequent reliance on retrospective “mental averaging” of their daily experience over some period of time. A substantial amount of evidence from ecological momentary assessment (EMA) studies suggests that retrospective measures correlate only moderately with average experience when assessed using EMA (Solhan et al 2009, Trull & Ebner-Priemer 2013). These reporting biases are also likely to impact the types of neural correlates and biomarkers that show associations (Treadway & Leonard 2016). For example, a substantial amount of evidence now supports the presence of significant discrepancies among patients with schizophrenia regarding their believed and experienced negative symptoms; patients report significantly less expected enjoyment to laboratory stimuli as compared to their actual enjoyment (Gold et al 2008, Strauss & Gold 2012), are found to have difficulty reporting consistently about their preferences (Brown et al 2013, Strauss 2013, Strauss et al 2011), and appear unable to translate reported anticipation of pleasure into goal directed behavior (Gard et al 2014). Such inconsistencies between retrospective and ‘in-the-moment’ reports can limit the validity of such measures. Further, the extent to which retrospective reports may be more or less accurate is likely to depend on the individual, the symptom and the disorder. Conversely, there may be other symptom domains for which isolated assessment of beliefs about self experience and their associated biomarkers are more relevant. For example, repeated studies have shown the presence of a persistent negative bias in disorders such as depression (Joormann & Gotlib 2006, Korn et al 2014), leading to affective forecasting predictions that are often worse than experienced mood (Strunk et al 2006).

An alternative approach to examining hedonic processing emphasizes the evaluative aspect of reward by having individuals rate their affective responses to positively-valenced stimuli in a controlled laboratory setting (for reviews (Bylsma et al 2008, Gold et al 2008)). Early examples in humans focused primarily on self-report, but numerous studies have also utilized physiological responses (such as the post-auricular reflect), which avoid some of the inherent limitations of self-report. Similar approaches have proven useful in animal research, where self-report is infeasible). For instance, measuring lip smacking following sweet tastes has proven critical for isolating the neural circuitry for hedonic impact (Berridge & Kringelbach 2008).

Economic Exchange Measures

Within the last decade, economic exchange paradigms have increasingly been used to elucidate how psychopathology may involve alterations in the appraisal of costs and benefits as well as the heuristics that may guide decision-processes. This work has increasingly turned to the fields of behavioral neuroscience, economics, and computer science for inspiration, employing translational paradigms based on animal models economic discounting and “willingness to pay” tasks and models of reinforcement learning. For example, inter-temporal choice tasks have been widely used in studies of personality and externalizing psychopathology to index impulsive preferences (Bickel & Marsch 2001), while tasks such as the Effort Expenditure for Rewards Task (EEfRT), which assesses willingness to expend effort and sensitivity to probability and reward magnitude in decisions has been applied to conditions such as depression, schizophrenia, and autism (Damiano et al 2012, Reddy et al 2015, Treadway et al 2012b, Treadway et al 2015).

Several benefits of these tasks are immediately apparent; first they lend themselves to formalization of optimal and sub-optimal responses, which can help to better isolate true “deficits” in patient populations as opposed to mere “differences” between patients and controls. Additionally, these tasks often reflect many of the types of choices that individuals encounter in everyday life and that are known to be impacted by psychopathology (e.g., the cost-benefit or discounted value of using a substance, engaging in a risky behavior, performing a socially isolating activity), and can thus be thought to possess good external validity.

A natural extension of laboratory-based behavioral economic measures involves the use of formal trial-by-trial models to analyze behavior. This work, increasingly referred to as “computational psychiatry” (Montague et al 2012) attempts to simulate cognitive processes though the instantiation of formal models that can accurately predict a subject’s task behavior. Such models usually involve one or more free-parameters that are scaled to improve the model’s fit to a given subject’s data, and these parameters can become variables of interest in their own right. Importantly, the application of model-based approaches is the ability to examine behaviorally “unobservable” variables that may nevertheless have clear neural correlates and implications for behavior.

One widely studied example is reward prediction error (RPE) signals during reinforcement learning. RPEs are typically inferred from a computational model that attempts to estimate a subject’s expectations based on their behavior, which can then be used to assess the extent to which a subsequent outcome was predicted or not (e.g., if option A has rewarded me consistently in the past, and I keep choosing A, it is reasonable to assume that I expect A to be rewarded, and will be disappointed if it is not). Although they can only be indirectly inferred from behavior, modelled RPE signals have been shown to predict striatal responses during reinforcement tasks (Pessiglione et al 2006), learning (Schönberg et al 2007) as well as affective responses to reward receipt (Rutledge et al 2014), reflecting its broad associations with multiple aspects of reward processing.

Despite the advantages of computational approaches, there are limitations to this work in its current state that should be addressed in future studies. For one, they primarily (though not exclusively) rely on monetary incentives. Given well-known interactions between socioeconomic status and incidence of psychopathology (Kessler et al 1994), the general assumption that money represents a true “common currency” that will be equivalently valued across participants of differing backgrounds and mental health may not be justified. Additionally, these measures have rarely been normed in terms of their psychometric properties or demographic influences on performance (age, sex, IQ, SES, etc.). Creation of administration standards and normative performance metrics is clearly necessary if these measures are to become clinical tools.

Physiologiccal and Neuroimaging Measures

A final area of measurement for rewards has been the use of functional neuroimaging measures that are associated with reward anticipation, expected value, response costs, or hedonic impact. The most widely used paradigms in this literature include tasks that present positively valenced affective stimuli, (Keedwell et al 2005), require responses to obtain rewards, often with an attempt to dissociate anticipation and receipt of rewards (e.g., the MID task (Knutson et al 2001), guessing paradigms (Hajcak et al 2006), and gambling tasks (Delgado et al 2000)). These studies have in many cases shown excellent convergence with preclinical studies in animals, identifying for instance the ventral striatum (including nucleus accumbens and neighboring regions) as a key site for multiple features of reward processing and reinforcement learning. The results have allowed the development of objective markers of reward relevant processes which have been used to identify altered patterns of neural responses between clinical and healthy populations (discussed more below).

One must caution, however, against the temptation towards “greedy reductionism” in the interpretation of such differences. Perhaps the biggest concern reflects the problem of reverse inference in interpreting neuroimaging results. Specifically, just because an area activates during a specific process (say reward anticipation), it does not necessarily follow that the individual in more or less engaged in or responsive to that process based on the level of activation in the region. Neuroimaging signals are extremely sensitive to the specific parameters, design and experimental context of each study, and amplitude differences may not represent a deficit or dysfunction. Moreover, the differences that do emerge may be related to a psychological sub-process that differentiates the groups rather than the process of interest. For example, the increased psychological distress experienced by a group of patients relative to controls may manifest as a reduced response to a reward-predicting cue not because of a reduced anticipation for reward per se, but because of the presence of concurrent psychological pain that is part of the sequelae of the disorder. When it comes to the use of computation models, neuroimaging can also present challenges. Simulation studies have found that neuroimaging responses to RPE signals are fairly insensitive to individual differences in model parameters, such as learning rates (Wilson & Niv 2015). Consequently, while neuroimaging can reliably identify where in the brain RPE signals occur, differences in the amplitude of neural RPE signals in patients versus controls may prove relatively difficult to detect.

ABNORMALITIES OF REWARD PROCESSING

Evidence of Deficiencies in Aspects of Reward Processing

One of the most commonly tested hypotheses in psychopathology research is reduced reward processing. Indeed, a lack of responsiveness to life’s basic incentives has long been held as a core source of behavioral dysfunction for multiple disorders, particularly depression and schizophrenia and substance use (Blum et al 1996, Klein 1974, Meehl 1975). The operationalization of this hypothesis has evolved in different ways across disorders over the last several decades. In the case of anhedonic symptoms of depression and negative symptoms of schizophrenia, early self-report assessments and experimental studies often focused on affective ratings to pleasurable stimuli, such as pleasant images or sweet tastes. In both populations, self-report questionnaires have found robust group differences such that patients are far less likely to endorse enjoyment of various experiences as compared to healthy controls (Gold et al 2008, Watson & Naragon-Gainey 2009). For lab-based studies, however, many paradigms fail to find consistent alterations in reported pleasure (for reviews and meta-analyses, see (Bylsma et al 2008, Gold et al 2008, Treadway & Zald 2011), which may suggest important differences in the exact constructs assessed across these methods.

Where behavioral studies have been reasonably successful in detecting alteration in reward processing in psychopathology is in the areas of reinforcement learning, delay and effort discounting, and preference transitivity. In general, these studies have revealed that the behavior of clinical populations is less sensitive to manipulations of reward values. For example, Pizzagalli and colleagues have used a signal-detection approach with reinforcement learning to reliably discriminate between depressed and non-depressed individuals, particularly with anhedonic symptoms (Huys et al 2013, Pizzagalli et al 2008). In the case of effort discounting, patients with unipolar depression have been found to demonstrate reduced willingness to expend physical effort in exchange for monetary rewards (Clery-Melin et al 2011, Hershenberg et al 2016, Treadway et al 2012b, Yang et al 2014), suggesting either deficits in motivation or accentuated effort discounting. Importantly, however, this apparent consistency is belied by variable relationships with reported anhedonic symptoms. While some studies have identified inverse correlations between reward motivation and anhedonic severity (Hershenberg et al 2016, Treadway et al 2012b, Yang et al 2014), others found no relationship (Clery-Melin et al 2011). Interestingly, one recent study found that symptoms of self-criticism in depression may lead to greater effortful performance (Hershenberg et al 2016), thereby possibly masking the association between effort and anhedonia.

A similar pattern has emerged for schizophrenia, where a number of studies have found evidence for deficits in effort allocation rather than absolute effort expenditure (Barch et al 2014, Gold et al 2013, Reddy et al 2015). The associations between performance on effort-related measures and measures of negative symptoms in schizophrenia have been mixed, and have occasionally suggested that greater effort performance was associated with more severe negative symptoms (McCarthy et al 2016). One possibility is that schizophrenia patients are often limited in their ability to accurately report on and forecast their motivational states (Strauss & Gold 2012). Evidence for this has been found using tasks of preference transitivity (i.e., if you report liking A more than B and B more than C, you should also report liking A more than C), for which schizophrenia patients display marked inconsistencies (Strauss et al 2011). Additionally, recent EMA studies have found that schizophrenia patients performed fewer effortful daily activities, despite reporting greater anticipation of enjoying activities (Gard et al 2014). It is also important to note that measures of reward processing in schizophrenia may be at least partially confounded by the impact of antipsychotic medications (particularly first generation antipsychotics given their strong DA D2 receptor (DRD2) antagonistic properties), which could potentially produce abnormalities in reward processing that are misattributed as being caused by the disorder itself. If there are indeed negative effects of antipsychotics on reward processing, differences in medications across studies could contribute to variability in the expression of reward processing abnormalities (for additional discussion, see Gold, Waltz and Frank, 2015).

In neuroimaging studies, these behavioral reductions in sensitivity to reward information and manipulations are frequently (though not universally) accompanied by lower amplitude effects in areas known to show activation in response to rewards and reward predicting cues, such as the striatum, particularly the ventral striatum. For example, multiple studies in depression and schizophrenia have shown reduced striatal activity during preparation to make a speeded response for a reward or feedback about probabilistic reward outcomes (Greenberg et al 2015, Juckel et al 2006, Kumar et al 2008, Morris et al 2012, Pizzagalli et al 2009). However, evidence suggests that these reductions in striatal signals may occur for different reasons across disorders. In the case of schizophrenia, it has been demonstrated that presynaptic stores of DA are elevated (Fusar-Poli & Meyer-Lindenberg 2012), and may contribute to altered striatal signals through abnormal patterns of DA release that fail to differentiate between rewarded and unrewarded conditions (Winton-Brown et al 2014). In contrast, studies in depression suggest that altered striatal signals could arise from either hypodopaminergic states (Capuron et al 2012) or altered connectivity between striatum and medial prefrontal regions (Ferenczi et al 2016, Heller et al 2009).

In the case of substance use disorders, a prominent hypothesis has been termed the reward deficiency syndrome (RDS) (Blum et al 1996), which proposes that an absence of rewarding subjective experiences or lowered “hedonic tone” causes individuals to seek out and consume strong rewards (such as drugs of abuse). The theory links the problem to reduced DA function, specifically citing DRD2 genetic findings and neuroimaging results demonstrating lowered striatal DRD2 density in substance use disorder populations (Blum et al. 1996; Comings & Blum 2000) as evidence of lowered dopaminergic tone. Further clinical evidence of some aspect of lowered dopaminergic tone has been reported in studies by Volkow et al. (1997) who show lowered psychostimulant induced DA release in individuals with substance use disorders.

There are some key preclinical pieces of data that fit nicely with this model. Monkeys with lowered striatal DRD2 levels at baseline develop increased drug self-administration (Nader et al 2006), and rodents with impulsive premature responding on the 5-choice serial reaction time test, a phenotype that is vulnerable to developing drug self-administration, show lowered DRD2 expression in the striatum (Dalley et al 2008). Intriguingly, insertion of a virus that upregulates DRD2 expression decreases levels of self-administration in already drug self-administrating rodents (although it is unclear if this reflects a change in desire for the drug or a more rapid satiation due to the rodents’ needing less drug to achieve the same effect)(Thanos et al 2008). As with depression and schizophrenia, a number of studies have also observed decreased striatal activations during monetary reward anticipation in addiction samples, although some studies find contrary results (see (Leyton & Vezina 2013) for review).

However, there are a number of elements of this model that are difficult to integrate with existing data, especially if the RDS is treated as a global reward deficiency. First, we need to consider whether the RDS deficit reflects anticipatory reward, consummatory reward, or a homeostatic affective state of “hedonic tone”. It seems difficult to conceptualize addiction as a disorder of globally low anticipatory reward or wanting given the extreme states of desire experienced by the addict. Indeed, the DSM-III-V definitions of both substance use disorders and behavioral addictions emphasize the willingness to spend excessive amount of time, money and energy acquiring the desired reinforcing experience. Can such individuals really be considered to have a deficiency in anticipatory reward? An alternative possibility would be that their deficiency is in the consummatory phase. However, this seems unlikely to drive substantial reward seeking. If we devalue a food stimulus (such as by satiation), the individual will work less for it, not more for it. The third possibility is that the individual experiences lowered homeostatic level of satisfaction. Alterations in either reward wanting or liking in this case are secondary to a lowered affective state. This psychological conceptualization is at the heart of the RDS theory. However, direct support for this idea is limited. Indeed, data on addiction urges emphasize the greater importance of heightened negative affective states as a precipitating mood factor, and some data even suggest a stronger impact of heightened positive affective states rather than lowered positive affect in driving urges for consumption (Baker et al 1986, Brandon et al 1996).

The linkage of the RDS to DA functions also is difficult to fully incorporate with the mounting evidence regarding the distinction between anticipatory and consummatory reward, which demonstrates DA’s critical involvement in motivated behavior more than consummatory experience. The most relevant question here is what occurs as a consequence of lowered D2 receptors in the striatum in addicted individuals? This issue takes on particular importance as the RDS model argues for a psychological explanation (lowered reward processing) on the basis of a specific interpretation of a receptor measure. Given the work of Berridge and colleagues (Berridge & Robinson 1998), lowered DRD2 expression seem unlikely to cause an inability to experience consummatory reward. Moreover, DRD2 PET studies of patients with putatively reduced consummatory pleasure and negative symptoms have repeatedly failed to identify clear reductions in DRD2 expression (Howes & Kapur 2009, Treadway & Pizzagalli 2014). Acknowledging this issue, Blum and colleagues (2012) suggest the deficit may be more related to anticipatory reward. But if DRD2 is a marker of anticipatory reward sensitivity, these individuals with lower DRD2 levels should have lowered desire or wanting rather than craving. To try to resolve this seeming contradiction, Blum speculates that the remaining DRD2 receptors in these individuals are in a hypersensitive state, but data in support of this idea are lacking, and if it were true, it would seem difficult to characterize this as a primary reward deficiency. Finally, we may consider the possibility that DRD2 or other DA measures are related to a homeostatic affective tone, but at present direct evidence relevant to this hypothesis is lacking.

Another way to look at the DRD2 deficits is to consider them within the context of aging research. Age is among the strongest predictors of DRD2 receptor levels, with a decline of approximately 5–8% percent per decade of life (Antonini & Leenders 1993). The RDS hypothesis would seem to predict that we should see increasing rates of de novo addiction ir relapse in the elderly, but this is not seen (Blazer & Wu 2009).

Because it views a lowered dopaminergic tone as playing a causal role in addiction, one of the strongest predictions of the RDS hypothesis is what happens when DA transmission is lowered pharmacologically. Strikingly, as reviewed by Leyton and Vezina (2013), rather than causing drug seeking behavior or use, decreasing DA transmission diminishes cocaine cue-induced craving and the willingness to work for drug reward. These findings parallel data from Parkinson’s disorder, in that despite their deficient DA production and transmission, there is no evidence of increased addictive behavior off medication. Indeed, administration of DRD2 receptor agonists can cause the de novo development of addictive behaviors in this population (Dagher & Robbins 2009). These observations appear to run directly counter to the RDS hypothesis (although the sensitivity to the DRD2 agonists may reflect changes in the affinity or expression of DRD2 induced by sustained deficits in DA production).

Beyond addictive behavior, variants on an RDS-like model have also been prominent in theorizing about attention deficit hyperactivity disorder (ADHD). Several types of data support an RDS-like view of ADHD (Haenlein & Caul 1987). Individuals with ADHD have been observed to need greater incentives to modify their behavior (Kollins et al 1997). Neuropharmacological data also provide links to reduced DA functions (Volkow et al 2011). Finally, multiple studies have shown hyporesponsiveness of the ventral striatum during reward anticipation, with an effect size of Cohen’s d = 0.48–0.58 (Plichta & Scheres 2014). Interestingly, this reduced response may be associated with one of the most consistent reward processing abnormalities in ADHD, which involves a heightened temporal discounting of rewards (Barkley et al 2001). At least in adolescents, lowered ventromedial caudate responses during reward anticipation are associated with steeper rates of temporal discounting behavior (i.e., more impulsive choice behavior)(Benningfield et al 2014). Yet, a global RDS-like model of ADHD struggles to explain the robust effects of reward on task performance in ADHD, which can in some cases be stronger than in typically developing children (Luman et al 2005). Indeed, several theories of ADHD explicitly consider there to be enhanced reward sensitivity in the disorder (Douglas 1989, Sergeant et al 1999).

In raising these issues, we do not intend to question that there are substantial behavioral consequences of lowered DRD2 levels in the striatum, nor do we question that an individual’s level of satisfaction in life or level of rewarding experiences impact the readiness to engage in addictive behaviors, but we believe that a more nuanced interpretation of the psychological and pharmacological data is necessary to account for the reward processing abnormalities that characterize addiction and related disorders.

Evidence of Excessive Reward Processes

There are several mental health domains where one or more aspects of reward processing appear hyper-responsive. Among the most consistent findings arise in bipolar disorder, with increasing efforts aimed at clarifying specific components of reward processing alterations (see (Alloy et al 2015, Johnson et al 2012) for review). Whereas reward liking and learning appears relatively normal, pursuit of goals and the willingness to work for rewards appear heightened even in remission. Critically, reward anticipation linked neural responses (including ventral striatal and orbitofrontal responses) show elevations (Nusslock et al 2012). Increasing data also point to the importance of temporal features following rewards, with greater sustaining of positive affective responses, reduced satiety after reward attainment, and weaker responses to negative prediction error when reward contingencies change (Johnson et al 2012).

Excessive pursuit of specific reinforcers despite their substantial costs or associated risks is of course a hallmark of addiction, and not surprisingly incentive-motivational circuits are strongly activated by cues for drugs and other addiction-related stimuli (Leyton & Vezina 2013). The more difficult and contentious question is whether there is a pattern of hypersensitivity to rewards premorbid to the development of addiction that increases the likelihood of developing an addiction. Support for such a view can be found in several domains. At the level of self-report, measures of reward sensitivity, such as fun seeking and drive subscales of the Behavioral Approach Scale are strong predictors of both current and future substance use and addiction risk (Dawe et al 2004). However, once an addiction has developed, only a minority of studies suggest hyper-responsiveness to non-addictive rewards such as money (Leyton & Vezina 2013).

Features of some form of high reward responsiveness characterize multiple other externalizing disorders and behaviors, both in terms of correlates of personality measures, and in terms of neural responses. For instance, we have reported that impulsive-antisocial traits are positively associated with the level of ventral striatal responses during the reward anticipation phase of the MID task (Buckholtz et al 2010). Similar heightened ventral striatal responses appear in association with externalizing traits in adolescents (Bjork et al 2010). Thus, within the realm of addiction and externalizing disorders we are left with a quandary of how to integrate examples of hypo- and hyper- reward processes in any cohesive manner.

REFINING MODELS OF REWARD ABNORMALITIES IN PSYCHOPATHOLOGY

Given the multiple cases where a simple global hypo- or hyper-reward processing model appears insufficient to explain the combination of reward processing characteristics that arise in mental disorders, it seems likely that for the field to progress, more refined or nuanced models will be necessary that Toward that end we turn our attention to models and hypotheses that could explain the combined characteristics that could lead to the sort of combinations of altered reward processes that characterize multiple disorders.

A Maladaptive Scaling Hypothesis

Reward valuation is highly dependent upon the context of available rewards. Winning $50 could be delightful, but much less so when there was a possibility to win $500. Multiple brain regions show responses where firing occurs relative to predicted rewards, or the availability of other more preferred or less preferred rewards. For instance, DA neuronal firing scales with currently possible values rather than absolute vale of potential rewards (Tobler et al 2005), and cells in the orbitofrontal cortex differentially respond to the same food item depending upon whether it is the higher or lower valued of two options at the given moment (Tremblay & Schultz 1999). Such relative scaling appears highly sensitive to anchors, that is value representations against which the other values are compared. For instance, human fMRI activations are substantially altered by the best or worst possible outcomes in a given situation (Nieuwenhuis et al 2005). Reviewing behavioral and neural decision-making data, Seymour and McClure (2008) argue that the brain’s use of relative valuation and anchoring is a consequence of the need for integration of neural responses across a wide range of potential values.

An abnormal scaling hypothesis of reward abnormalities has a number of conceptual advantages over global hypo- or hyper- reward sensitivity models of psychopathology. The most obvious of these arises in the addiction domain, where it can explain the ability of particularly strong reinforcers (e.g., drugs, gambling) to act as an anchor that causes a downscaling of other natural rewards. Because of the devaluation of alternative rewards in the face of this anchoring, individuals may appear to have deficient valuation or desire for multiple rewards, leading to the appearance of a deficiency in response to or desire for other rewards, while at the same time demonstrating extremely strong desire for specific salient rewards.

One can imagine two premorbid situations that might make an individual particularly vulnerable to the establishment of a high anchor that causes a downscaling of other rewards: 1) the individual is relatively deprived of strong rewarding experiences in their environment (whether due to a poverty of environment or a weak sensitivity to potential rewards), 2) they have strong reward sensitivity/reinforcement learning, such that when exposed to a high value reinforce, it produces strong reinforcement learning. In both cases, there is a high differentiation between the reinforcing event resulting in a robust anchoring. An interesting feature of the second possibility is that it can potentially explain why some features of reward responsivity are higher than normal prior to exposure to the anchoring reward experience (representing a vulnerability to addiction), while simultaneously explaining why once addicted the individual appears hyposensitive to other potential rewards.

In their review of scaling and anchoring phenomena, Seymour and McClure (2008) emphasize the adaptive nature of such processes, but salient anchors may exert lasting maladaptive effects on the valuation of other reinforcers even when the anchoring reinforcer is not immediately available. For instance, in a recent optogenetic study, selective stimulation of the central nucleus of the amygdala during exposure to one food reinforcer, led not only to the amplification of the value of the reinforcer (reflected in both choice behavior and willingness to work for the paired reinforcer), it also caused a narrowing of motivation such that there was a reduction in the willingness to work for an alternate (nonpaired) reinforcer that was originally of equal value (Robinson et al 2014).

A different set of problem can arise if the scaling is too flat such that there is a lack of differentiation between different reward options. If scaling is flat, individuals may find it difficult to select among options, resulting in the sort of common decision problems that characterize major depression. Although studies have examined whether there are basic lowering of responses to rewards, fewer tests have examined the possibility of alterations in scaling in affective disorders. Using the EEfRT, we have observed that patients with major depression, show a reduced impact of reward magnitude in decisions to expend effort, suggesting that either the subjects showed lower differentiation in their coding of these reward differences, and/or they had difficulty integrating this information with the other parameters (such as effort costs) in optimizing their choice behavior (Treadway et al 2012a). Similar results were obtained using a different effort-manipulation (Clery-Melin et al 2011) in depression. More recently, an fMRI study demonstrated that while signals in the ventral striatum appropriately adapted to the range of available rewards in healthy controls, there was no such adaption in schizophrenia.(Kirschner et al 2016).

A critical feature of behavioral economics models is often the shape of functions related to valuation. For instance, temporal discounting tends to follow a hyperbolic curve, such that the decline in subjective value of a reward is much greater in the near future than when contrasting the same amount of time substantially in the future (Odum 2011). Unfortunately, much of the work on valuation in mental disorders (outside of temporal discounting) has yet to test for these sorts of functions (as opposed to differences in absolute ratings). Arguably the establishment of a high new anchor could not only produce a downward shift in the valuation of two other reinforcers, but to also cause them to be less discriminable in value (causing them to appear on a flatter slope of the curve).

The Dopamine Transfer Deficit Model

ADHD provides a useful condition for considering models of reward processing abnormalities because of the recent emergence of theories that attempt to explain the specific processes in which there is enhanced or blunted responsiveness to rewards (Luman et al 2010). The DA Transfer Deficit Model developed by Tripp and Wikens (2008) posits a core deficit in phasic DA firing to cues that predict reinforcement. Normally, DA cells fire when there is a positive prediction error (a reward that was unexpected, underpredicted, or better than expected), but as the reward becomes better predicted, the firing transfers to cues that predict the occurrence of the reward instead of the reward itself. A reduced transfer of firing is argued to lead to problems with the prediction and anticipation of future rewards and poorer control of behavior. Rather than a global problem with reward, the model predicts that responses to actual rewards are normal. Indeed, with weakened sensitivity to cues for future rewards, responses to immediate rewards will be heightened relative to rewards that are in the future, consistent with the classic impulsive bias seen in ADHD under conditions of intertemporal choice. The model further predicts that the impact of the reduced transfer is particularly salient under conditions of partial or discontinuous reinforcement where the level of prediction is weaker, whereas learning from continuous reinforcement will be normal.

A Competing Valuation Systems Hypothesis

In recent years, a growing number of human and animal researchers have found compelling evidence for the presence of multiple value systems that offer varying costs and benefits in terms of their speed, attentional demands, and flexibility. This is an important insight in that it suggests that rather than a singular valuation system, there are multiple processes through which valuation is calculated and used to prioritize actions. While no definitive taxonomy exists as of yet, there is emerging consensus around the presence of at least three behaviorally and neurobiological distinct systems: Pavlovian, habit and goal-directed value systems that each direct actions towards rewards or goals based on their coding of value (Rangel, Camerer, Montague, 2008; O’Doherty 2015). The Pavlovian system learns basic stimulus-response pairings, with the responses coming from a limited number of species-typical behavior, such as reaching towards an available piece of food. The habit systems learns mostly ‘automatic’ responses that allow long-term optimization of actions in context with significant repetition. While the Pavlovian and Habit systems have not always been considered in terms of explicit valuation systems in the context of economic decision making, they meet the basic characteristics of valuation systems in the extent to which they prioritize actions based on factors such as associative strength and past reinforcement history. Finally, a goal-directed system allows prioritization of actions that lead to short or long-term goals, allowing adaptive behavior that can over-ride the influence of Pavlovian and habit systems, and drive actions in novel situations in which there is not an adequate history for optimal automatic responses.

An interesting question therefore is how dysfunctional interactions among these multiple systems may contribute to so called “reward process” dysfunction in psychological disorders. For example, psychological stress–a potent, non-specific risk factor for pychopathology writ large–has long been hypothesized to produce “reduced reward processing” and “stress-induced anhedonia” (Willner et al 1992). While this has been observed in some studies, (Bogdan et al 2011, Pizzagalli et al 2007), the opposite pattern (Cavanagh et al 2011, Lighthall et al 2012) has also been seen. More interesting in the context of a competing systems model, several studies have found that stress impairs goal-directed control over habitual response patterns in both humans and animals, as evidenced by disruption in normal reinforcer devaluation (Dias-Ferreira et al 2009, Lemmens et al , Schwabe & Wolf 2009) or a bias towards Pavlovian learning over “model-based” goal-directed learning (Otto et al 2013). These studies, which allow tests of the interactions between different action-value systems, illustrate an important lesson: depending on the nature of the task, the effects of stress may appear to potentiate or attenuate so-called “reward-systems”. If, for example, depressed patients suffer from a dysfunctional goal-directed system, they may nevertheless appear to have intact–or possibly even elevated–reward responses in a task that can be adequately performed by a Pavlovian system.

Recognition of the importance of Pavlovian and habit based systems is of course not novel to understanding psychopathology. Such systems have been at the center of many behaviorally oriented theories of psychopathology, such as the importance of habits for obsessive-compulsive disorder, and Pavlovian processes in the ability of cues for reinforcers to influence behaviors in both anxiety disorders and addiction. Indeed, the inability of explicit goals to overcome the outcome of Pavlovian and habit based systems seems central to a wide range of psychopathology. However, treating these systems as each reflecting valuation processes may lead to novel approaches in characterizing psychopathology and its treatment. It is not simply that the individual with obsessive-compulsive disorder or an addiction needs to exert stronger top down control of their urges. Rather the value of the goal of abstinence must exceed the value coded by the Pavlovian and habit systems. It follows therefor that interventions should aim not only to increase the ability to resist urges, but to alter the relative valuation of the different systems (for instance bolstering that of the goal directed system and lowering the valuation of the other systems).

A Dysfunctional Weighting of Reward Parameters Hypothesis

A broad hypothesis for why there can be seemingly paradoxical evidence for hypo- and hyper- reward sensitivity in the same disorder (or the same individual), is the presence of extreme biases in the weighting of different reward and cost parameters. For instance, an accentuated weighting of temporal discounting could lead to both a hyper-responsivity to immediately available rewards and a hypo-responsivity to delayed rewards, consistent with the patterns seen in externalizing disorders.

In some cases, valuation and cost estimations may appear to be only minimally integrated. As noted above, in tasks such as the EEfRT, examples arise in which individuals with psychopathology, such as schizophrenia are sensitive to different parameters such as reward magnitude and probability, but fail to integrate these parameters in an optimal way. Precise characterization of what happens in these cases is lacking, but it is possible to speculate on ways in which such integration may fail. For instance, while value and costs may be calculated, the most salient feature may be the only feature given any weight during a choice, leading to a sensitivity to extremes, but minimal utilization of the gradations of the other parameters.

REWARD PROCESSING AS A TREATMENT TARGET

More nuanced models of reward processing in psychopathology also has relevance for understanding the mechanisms of action for treatment. A number of psychotherapy techniques focus heavily on modulation of reward-related phenomena, including behavioral activation therapy (BAT) (Dimidjian et al 2006) and future directed therapy (Vilhauer et al 2012) for depression, and motivational interviewing for addiction (Miller & Rollnick 2012).

In the case of BAT, the focus is to help patients re-engage in various activities (work, social, hobby etc.) that have been curtailed as a consequence of their depression. The premise of this focus is that the patients will find such activities more enjoyable and less effortful than they anticipate. Consequently, BAT provides patients with a series of positive prediction errors that can–over time–re-calibrate the patient’s expectations of the costs and benefits associated with engagement. Assuming this model of BAT’s effects is correct, it is interesting to consider how one might use measures of reinforcement learning and RPE signals to predict treatment response. We would predict that patients who, despite being depressed, show relatively normal RPE signals are the best candidates for BAT. Such a hypothesis assumes that there is some degree of dissociation between different reward processes. A model that treats reward processes as homogenous would be unlikely to make such a prediction as it would assume that RPE signals track with other reward processes.

Treatments may also be considered in terms of their ability to alter the subjective costs and rewards of behavioral change. This is particularly true in motivational interviewing with its direct emphasis on the client’s expression of the desire, ability, reasons, and need for change (Miller & Rollnick 2012). It can also be seen as an attempt to alter the relative scaling of goals and rewards, such that the subjective utility of more adaptive rewards can compete with and exceed the subjective utility of the maladaptive behaviors. The critical question for these types of interventions is whether they are sufficient to overcome the substantial past reinforcement learning and habit-related valuations.

CONCLUSIONS

In the present review we have attempted to provide a framework for developing a more precisely-defined view of reward processing abnormalities in psychopathology, with an emphasis on recent insights arising from the neuroeconomics literature. This work was stimulated by what we see as a lack of precision that often arises in the extant literature on reward processing in psychopathology. Consequently, we suggest that the field should eschew broad terms such as “reward sensitivity” for more precise descriptions when possible. We recognize that much remains unknown at both the behavioral and neurobiological level, and general terms may limit premature overspecificaiton in the absence of experimental data. But this benefit comes with the cost of reifying the existence of a putative “reward system” and fails to push future research paradigms forward in terms of the hierarchical, overlapping circuits that are involved in different aspects of valuation and hedonic processes.

We have argued that many of the alterations in reward processing in psychopathology are inconsistent with a unitary up or down regulation of all aspects of reward processing. As such certain conceptualizations, such as the reward deficiency syndrome concept have likely outlived their usefulness. That said, we fully recognize that more nuanced perspectives are likely to gain traction only if they can be tested and shown to outperform older theories in predicting the specific patterns of preserved and abnormal reward features in psychopathology. In this respect, the hypotheses put forth here require testing, and computational formalization. However, we hope that the presentation of these ideas stimulates such testing and generation of other hypotheses that can explain the complex nature of reward mechanisms in psychopathology.

Figure 1.

The figure displays four hypothetical scaling functions for subjective value assignment. Function A and A’ represent the same function that has been adaptively scaled based on a different available anchor. Function B (dashed line) shows a function with a pathologically reduced slope, while Function C is anchored by a reward that is so highly valued that most other rewards receive minimal subjective value. Individuals with functions B and C will not be able to differentiate between or be motivated by rewards that are in the left (lower absolute) half of scaled space.

Summary Points.

• Reward processing includes multiple distinct components related to valuation, discounting and learning, and involve multiple neural substrates

• Economic decision-making, physiological, neuroimaging, and ecological momentary assessment measures provide useful complements to more traditional self-report approaches

• Multiple components of reward processing show abnormalities in psychopathology, but they are not adequately explained by homogenous conceptualizations of excessive or deficient reward responses

• Specific abnormalities in reward processing can arise in the utilization and integration of different reward parameters

• Emerging hypotheses propose that more nuanced abnormalities in reward processing occur due to alterations in the scaling, weighting, transfer and competition of reward relevant parameters and processes.

• Neuroeconomic insights can help frame and refine psychological models of psychopathology and its treatment

Unresolved Issues.

• Formal tests of behavioral- and neuro-economics inspired hypotheses are generally lacking in patient populations, which leaves their relative merit for understanding and defining psychopathology unclear

• Scaling, weighting, and competing valuation models of reward abnormalities will require computational formalization if they are to be adequately applied to specific disorders such as substance use disorders, schizophrenia or major depression

• Behavioral economics focuses on conscious decisions, but it remains uncertain the extent to which characterizing decisions can adequately capture many psychopathological behaviors.

• The extent to which there are common versus parallel valuation systems and the manner in which such systems are integrated remain open to debate, which has substantial implications for the applications of these ideas to mental health domains.

• The extent to which it is possible to alter problems of scaling, weighting, transfer, and competition in reward processing is largely unexplored, which leaves their utility as treatment targets unknown.

Glossary

Abbreviations

ADHD

Attention Deficit Hyperactivity Disorder

DA

Dopamine

DRD2

D2 Dopamine Receptor

EEfRT

Effort Expenditure for Rewards Task

EMA

Ecological Momentary Assessment

MID

Monetary Incentive Delay task

PET

Positron Emission Tomography

RDS

Reward Deficiency Syndrom

RDoC

Research Domain Criteria

RPE

Reward Prediction Error

Key Terms

Adaptive Scaling

A transformation algorithm that can be optimized (shifted, compressed or expanded) depending upon the range of input values

Affective Forecasting

The prediction of one’s affect (emotional state) in the future (also called hedonic forecasting)

Ecological Momentary Assessment

Assessment in which participants repeatedly report on affect or behavior close in time to the experience in their natural environment

Mesolimbic Dopamine System

DA projections arising from the midbrain ventral tegmental area and projecting to the ventral striatum and limbic regions

Phasic Dopamine Firing

Brief burst of DA neuron firing potentials that carries an RPE signal, and is distinguished from steady, pacemaker-like, tonic activity.

Reinforcement Sensitivity Theory

Model initially proposed by Gray that posits that individuals critically differ in their responsiveness to cues for rewards and punishments.

Reward Prediction Error (RPE)

A learning signal corresponding to the discrepancy between the currently experienced and predicted reward

Subjective Value (utility)

The worth that a person places on a good (as opposed to an inherent value).

Temporal Discounting

Reduction in the subjective value of a reward that is to be acquired or consumed in the future.

Ventral Striatum

The inferior-medial part of the basal ganglia that includes the nucleus accumbens.